JODI-Oil Assessments

JODI-Oil Participation Assessment

The JODI-Oil Participation Assessment measures the degree of participation in the Initiative by each country submitting data to the Initiative. The bi-annual assessment considers three measures: timeliness, completeness, and sustainability. 

The JODI-Oil Participation Assessment Definitions: Sustainability, Timeliness, and Completeness

Sustainability

Sustainability measures the number of JODI-Oil questionnaires received from each participating country or economy over the assessment period (six months) and within the agreed submission deadlines. A Good rating (smiley face), is earned when a country or economy submits JODI questionnaire responses for all six months in the period.

JODI-Oil Participation Assessment - Good sustainability "good" if the 6 questionnaires have been submitted

JODI-Oil Participation Assessment - Fair sustainability "fair" if 4 or 5 questionnaires have been submitted

JODI-Oil Participation Assessment - sustainability less reliable "less reliable" when less than 4 questionnaires have been submitted

Timeliness

Timeliness evaluates whether or not data has been received within two months of the end of each reference month in accordance with the data submission deadlines established by the JODI Partner Organisations. A smiley face is awarded if all six submissions within a given assessment period were received by the agreed deadlines. 

JODI-Oil Participation Assessment - Good timeliness "good" when all 6 submissions during the assessment period have been received within two months of the end of the reference month relevant to each submission

JODI-Oil Participation Assessment - Fair timeliness "fair" when 4 or 5 submissions have been received on time

JODI-Oil Participation Assessment - Timeliness less reliable "less reliable" when less than 4 submissions have been received on time

Completeness

Completeness tracks the number of data points submitted out of the maximum of 42 in the JODI Questionnaire. To earn a smiley face, each country or economy must submit more than 90% of its data related to production, demand and stock changes.

JODI-Oil Participation Assessment - Good Completeness "good" when more than 90% of the data are given for production, stock change/closing and demand

JODI-Oil Participation Assessment - Fair Completeness "fair" when between 60% and 90% of the data are given

JODI-Oil Participation Assessment - Completeness less reliable "less reliable" when less than 60% of the data are given

Submission Deadline

Each JODI Partner Organisation establishes its own submission deadline with their member countries and economies. The common factor is that in all cases submissions must be made no later than two months from the end of the reference month.


Data Confidence Level Assessment

The assessment was carried out on different levels:

  • Comparability of the JODI data with other sources: monthly data from national and secondary sources has been assessed.
  • JODI data have also been compared with annual data (when available) in order to check whether the levels and trend over the years could be confirmed.
  • When no other sources were available for comparison with the JODI data, internal consistency and balance check have been carried out.

Examples of internal consistency checks: the sum of all the reported products with reported figure for Total Products is compared. When both, closing and stock changes data have been submitted, the consistency of the reported changes with the calculated ones is compared.

Example of balance check: the JODI Questionnaire does not collect full balance information, however some basic checks for reasonableness can be carried out e.g. supply + import - export + stock change should have a relation with demand.

Remark: for IEA /OECD countries, data in the JODI database are the MOS data for all months except data shown for M-1. Comparability for the last month has been derived from comparison with MOS data. This methodology is applied using a rolling 12 month period.

Colour Coding

  • Blue: A blue background indicates that results of the assessment show reasonable levels of comparability
  • Yellow: A yellow background indicates that the metadata should be consulted
  • White: A white background indicates that data has not been assessed
  • Purple: data under verification

More detailed notes on respective assessment approach by each JODI Organisation can be found from the link below.