1 / 21

An application to the Italian IIP

Revision analysis to detect possible weakness in the estimation procedures. An application to the Italian IIP. A. Ciammola, T. Gambuti and A. Mancini ISTAT European Conference on Quality in Official Statistics. Outline. Introduction Why revision analysis? A case study Next steps.

marika
Download Presentation

An application to the Italian IIP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Revision analysis to detect possible weakness in the estimation procedures An application to the Italian IIP A. Ciammola, T. Gambuti and A. Mancini ISTAT European Conference on Quality in Official Statistics

  2. Outline • Introduction • Why revision analysis? • A case study • Next steps

  3. Quality and some of its dimensions Accuracy Closeness of the estimate to the true (but unknown) value of the variable to be measured U Reliability Closeness between preliminary estimate and subsequent estimates Revision Reliability measure Timeliness Span between the reference period and the publication period

  4. Revision analysis • Real-time databases • collection of vintages • computation of revisions • Revisions • Rt = Lt – Pt • Rt = (Lt – Pt) / Lt • Revision measures • Size (MAR, RMAR, …) • Bias (MR, T-test) • Efficiency (News or Noise?, MSR, …)

  5. Useful references OECD / Eurostat Guidelines on Revisions Policy and Analysis http://www.oecd.org Themes related to revision policy and analysis • Recommendations for revisions policy and analysis • Guidelines for establishing a real-time database • Recommended statistical measures • Pre-programmed software for performing revisions analysis • A framework for revisions policy of key economic indicators • Comprehensive framework of reasons for revisions and their timing • Guidelines on how to decompose total revision into different reasons for revisions • Guidelines on how to use the results from revision analyses to improve compilation methods • Case studies on the relationship between timeliness of release and size of revisions

  6. Why do we measure revisions? For users Objective Availability of all the relevant information for using appropriately the estimates of short-term indicators at different stages of the revision process provision of information about past revisions schedule future revisions (statistical and definitional) real-time databases gathering all the vintages analysis of size, bias and efficiency of revisions

  7. Why do we measure revisions? For producers Underlying issues Targets Bias in the revision process Inefficiency in compilation of preliminary estimates Reduction of (the size of) “avoidable” revisions Detection of the source for bias / inefficiency

  8. A case study Italian Index of Industrial Production (IIP) • Sources and timing of revisions • Revision analysis • Identification of specific sources for bias • Some evidences

  9. 1. Sources and timing of revisions

  10. 2. Revision analysis IIP - Revisions on year-on-year growth rates (raw indices) Legend * a = 5% h=1 – after one monthh=12 – after 12 months MAR –Mean Absolute RevisionRMAR –Relative MAR MR –Mean RevisionSD –Standard Deviation

  11. 2. Revision analysis IIP - Revisions after one month on year-on-year growth rates (raw indices)

  12. 2. Revision analysis • Why this systematic component? • Late respondents? • Correction of errors? • Productivity coefficients? (In revisions after 1 month, only July 2004, January 2005, January 2006 and January 2007 are affected) • Which sectors? • All sectors? • Some specific sector? • How to proceed? • Simulation exercise • Top-down approach • Quality indicators

  13. 3. Identification of specific sources for bias Simulation exercise Removal of the effect of the productivity coefficients • Isolate sources of revisions external to the survey • Fulfil the condition necessary to compute the average contribution of each components to the IIP revisions PS: revisions computed on y-o-y growth rates

  14. 3. Identification of specific sources for bias Diagram describing the top-down approach

  15. 3. Identification of specific sources for bias Quality indicators • Revision measures • Contribution of each component to the mean revision of the higher component • Response rates

  16. 4. Some evidences MIGS - Revisions after one month on raw Y-o-Y growth rates LegendCND – Consumer non durables CDU – Consumer durablesCAP –Capital goods INT –Intermediate GoodsENE –Energy °Period Jan-04 / Dec-07 *a = 5%

  17. 4. Some evidences Revisions after one month on raw Y-o-Y growth rates

  18. 4. Some evidences Average weighted response rates

  19. 4. Some evidences Revisions after one month on raw Y-o-Y growth rates Legend S–Selected subset of INT (19 classes) SC,INT –Complement of S in INT(S U SC,INT= INT) SC,IIP –Complement of S in IIP(S U SC,IIP= IIP)

  20. Next steps • Checking the stability of results over time • Experimenting possible countermeasures to biased revisions • Treatment of late respondents with different estimators • Assessment of their effects on revisions • ISTAT web page on revisions • Real-time database for several short-term indicators • Revision analysis

  21. THANK YOU!

More Related