Loading in 2 Seconds...
Loading in 2 Seconds...
Using Performance Statistics to Design the 2006 Manufacturing Energy Consumption Survey. April 6, 2006 Robert Adler (202-586-1134) Robert.Adler@eia.doe.gov Tom Lorenz (202-586-3442) Thomas.Lorenz@eia.doe.gov. Acknowledgements. Census Bureau staff: Richard Hough Vicki Haitot
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Using Performance Statistics to Design the 2006 Manufacturing Energy Consumption Survey April 6, 2006 Robert Adler (202-586-1134) Robert.Adler@eia.doe.gov Tom Lorenz (202-586-3442) Thomas.Lorenz@eia.doe.gov
Acknowledgements Census Bureau staff: Richard Hough Vicki Haitot Stacey Cole
Objective • To improve data quality in the Manufacturing Energy Consumption Survey by examining performance statistics on: • Data status flags • Differential response rates
Discussion Topics • Boosting participation in Internet electronic form • Other performance statistics that would be useful to track • Risks in targeting non-response follow-up to larger establishments
Organization • Background on MECS • Examination of data status flags to evaluate effectiveness of questionnaire • Comparison of status flag data: respondents who completed an Excel version vs a written version • Examination of differential response rate to suggest other changes to improve data quality in the MECS
Background MECS is a quadrennial survey of manufacturing industries based on the North American Industry Classification System (NAICS) • 3 survey forms • “A”: Mailed to most manufacturers • “B”: Petroleum refineries only • “C”: Sent to Chemical, Paper, Iron and Steel, and other NAICS industries that are energy intensive or have complicated energy flows • Self-administered written questionnaire
Background - continued • Designed and sponsored by EIA and conducted by the U.S. Census Bureau because data are confidential by Title 13 of the U.S. Code • Mandatory by federal law • Statistical sample from list frame (15,500 sample cases in 2002)
Data Status Flags R:Reported data; either through questionnaire or follow-up telephone or e-mail A:Analyst correction of reporting error I:Imputed data; missing data imputed by price or other information (rare) S: Source data: An entirely different set of data was used instead of MECS for the item E: System edit; data changed due to systematic edit because of common mistake in reporting
3 2 1 Figure 1: Status Flags of Selected Data Items, Manufacturing Energy Consumption Survey, 2002
3 2 1 Table 1: Type of Status Flag as Percentage of Total, Manufacturing Energy Consumption Survey, 2002
Results for Selected Items • Natural gas used as a fuel one of the most reported data items in the MECS • Important to minimize costly analyst intervention • The 22 percent “A” flags represent mostly conversions of the reporting units after failure of a price range check • Most of those analyst corrections could have been avoided if the respondents were warned of an edit failure while at the establishment site
Results for Selected Items - LPG Non-fuel • “S” flag for LPG nonfuel represents the substitution of Economic Census—Manufacturing (ECM) data for MECS • Data editors discovered that Chemical respondents did not interpret the MECS reporting requirements correctly • “Non-fuel” or “feedstock” use was not seen as a “material input,” the ECM term. Thus, as part of the 2006 MECS, we intend to use that term along side the other traditional MECS terms
Creation of Excel Version • EIA developed an Excel workbook version of the 2002 MECS C-Form • Excel version looked like the written questionnaire • Added capabilities to Excel version include: • price and value range checks of major energy sources • automatically calculates derived data items (e.g., total consumption of electricity) • automatically copies reported and derived data to later sections
Limitations Time and other constraints prevented it from optimal development • Not all desired onsite edits or screeners could be included because respondents had to save data to a CD and/or diskette • Data from the Excel version was not fully integrated with Census data capture routines • CD and/or diskette had to be mailed to the same Census address as the paper questionnaires • Responses were keyed the same as with the normal paper-and-pencil mode
Participation Rate • 323 responses from the Excel version (8.9% of the total number of C-form responders) • Excel responders accounted for 12.4% of the total consumption reported from the C-form • 23 respondents failed reporting using Excel version: • Incorrectly formatted diskette • Bad diskette
Differential Response Rates • As in the case of other establishment surveys, the problem of nonresponse in the MECS has been growing. Possible causes: • Downsizing and greater emphasis on “core functions” • Attitude toward surveys • On the other hand, the budgets necessary to conduct large government surveys are also being squeezed • To best use the resources available to us, the non-response follow-up for the 2002 MECS was targeted to have the greatest effect on the survey data quality
Conclusions and 2006 MECS • 2006 MECS will shift to an Internet-based electronic reporting method for its main mode of collection • Much of the 2002 MECS late- and non-response was due to not having the right contact persons at the establishment in the beginning • Status flags will be amended to further define the occurrence and type of analyst intervention
Questions for the Committee • Given that only 10 percent of the eligible respondents chose to use the electronic Excel form in 2002, have we introduced bias into the comparisons? Are we still justified in reporting the results? • How do we boost participation in the Internet electronic form, given that respondents will have a back-up available to them?
Questions for the Committee - continued • What other performance statistics would be the most useful to track? • What are the risks associated with targeting non-response follow-up to larger establishments to minimize a non-response adjustment?