Using Performance Statistics to Design the 2006 Manufacturing Energy Consumption Survey - PowerPoint PPT Presentation

Using performance statistics to design the 2006 manufacturing energy consumption survey l.jpg
Download
1 / 22

  • 228 Views
  • Uploaded on
  • Presentation posted in: Pets / Animals

Using Performance Statistics to Design the 2006 Manufacturing Energy Consumption Survey. April 6, 2006 Robert Adler (202-586-1134) Robert.Adler@eia.doe.gov Tom Lorenz (202-586-3442) Thomas.Lorenz@eia.doe.gov. Acknowledgements. Census Bureau staff: Richard Hough Vicki Haitot

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Using Performance Statistics to Design the 2006 Manufacturing Energy Consumption Survey

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Using performance statistics to design the 2006 manufacturing energy consumption survey l.jpg

Using Performance Statistics to Design the 2006 Manufacturing Energy Consumption Survey

April 6, 2006

Robert Adler

(202-586-1134)

Robert.Adler@eia.doe.gov

Tom Lorenz

(202-586-3442)

Thomas.Lorenz@eia.doe.gov


Acknowledgements l.jpg

Acknowledgements

Census Bureau staff:

Richard Hough

Vicki Haitot

Stacey Cole


Objective l.jpg

Objective

  • To improve data quality in the Manufacturing Energy Consumption Survey by examining performance statistics on:

    • Data status flags

    • Differential response rates


Discussion topics l.jpg

Discussion Topics

  • Boosting participation in Internet electronic form

  • Other performance statistics that would be useful to track

  • Risks in targeting non-response follow-up to larger establishments


Organization l.jpg

Organization

  • Background on MECS

  • Examination of data status flags to evaluate effectiveness of questionnaire

  • Comparison of status flag data: respondents who completed an Excel version vs a written version

  • Examination of differential response rate to suggest other changes to improve data quality in the MECS


Background l.jpg

Background

MECS is a quadrennial survey of manufacturing industries based on the North American Industry Classification System (NAICS)

  • 3 survey forms

    • “A”: Mailed to most manufacturers

    • “B”: Petroleum refineries only

    • “C”: Sent to Chemical, Paper, Iron and Steel, and other NAICS industries that are energy intensive or have complicated energy flows

  • Self-administered written questionnaire


Background continued l.jpg

Background - continued

  • Designed and sponsored by EIA and conducted by the U.S. Census Bureau because data are confidential by Title 13 of the U.S. Code

  • Mandatory by federal law

  • Statistical sample from list frame (15,500 sample cases in 2002)


Data status flags l.jpg

Data Status Flags

R:Reported data; either through questionnaire or follow-up telephone or e-mail

A:Analyst correction of reporting error

I:Imputed data; missing data imputed by price or other information (rare)

S:Source data: An entirely different set of data was used instead of MECS for the item

E:System edit; data changed due to systematic edit because of common mistake in reporting


Slide9 l.jpg

3

2

1

Figure 1: Status Flags of Selected Data Items, Manufacturing Energy Consumption Survey, 2002


Slide10 l.jpg

3

2

1

Table 1: Type of Status Flag as Percentage of Total, Manufacturing Energy Consumption Survey, 2002


Results for selected items l.jpg

Results for Selected Items

  • Natural gas used as a fuel one of the most reported data items in the MECS

  • Important to minimize costly analyst intervention

  • The 22 percent “A” flags represent mostly conversions of the reporting units after failure of a price range check

  • Most of those analyst corrections could have been avoided if the respondents were warned of an edit failure while at the establishment site


Results for selected items lpg non fuel l.jpg

Results for Selected Items - LPG Non-fuel

  • “S” flag for LPG nonfuel represents the substitution of Economic Census—Manufacturing (ECM) data for MECS

  • Data editors discovered that Chemical respondents did not interpret the MECS reporting requirements correctly

  • “Non-fuel” or “feedstock” use was not seen as a “material input,” the ECM term. Thus, as part of the 2006 MECS, we intend to use that term along side the other traditional MECS terms


Creation of excel version l.jpg

Creation of Excel Version

  • EIA developed an Excel workbook version of the 2002 MECS C-Form

  • Excel version looked like the written questionnaire

  • Added capabilities to Excel version include:

    • price and value range checks of major energy sources

    • automatically calculates derived data items (e.g., total consumption of electricity)

    • automatically copies reported and derived data to later sections


Limitations l.jpg

Limitations

Time and other constraints prevented it from optimal development

  • Not all desired onsite edits or screeners could be included because respondents had to save data to a CD and/or diskette

  • Data from the Excel version was not fully integrated with Census data capture routines

    • CD and/or diskette had to be mailed to the same Census address as the paper questionnaires

    • Responses were keyed the same as with the normal paper-and-pencil mode


Participation rate l.jpg

Participation Rate

  • 323 responses from the Excel version (8.9% of the total number of C-form responders)

  • Excel responders accounted for 12.4% of the total consumption reported from the C-form

  • 23 respondents failed reporting using Excel version:

    • Incorrectly formatted diskette

    • Bad diskette


Slide16 l.jpg

Electricity: Electronic vs. Paper


Slide17 l.jpg

Natural Gas: Electronic vs. Paper


Differential response rates l.jpg

Differential Response Rates

  • As in the case of other establishment surveys, the problem of nonresponse in the MECS has been growing. Possible causes:

    • Downsizing and greater emphasis on “core functions”

    • Attitude toward surveys

  • On the other hand, the budgets necessary to conduct large government surveys are also being squeezed

  • To best use the resources available to us, the non-response follow-up for the 2002 MECS was targeted to have the greatest effect on the survey data quality


Slide19 l.jpg

Table 4: Response and Coverage Rates by Cell Size and Type.


Conclusions and 2006 mecs l.jpg

Conclusions and 2006 MECS

  • 2006 MECS will shift to an Internet-based electronic reporting method for its main mode of collection

  • Much of the 2002 MECS late- and non-response was due to not having the right contact persons at the establishment in the beginning

  • Status flags will be amended to further define the occurrence and type of analyst intervention


Questions for the committee l.jpg

Questions for the Committee

  • Given that only 10 percent of the eligible respondents chose to use the electronic Excel form in 2002, have we introduced bias into the comparisons? Are we still justified in reporting the results?

  • How do we boost participation in the Internet electronic form, given that respondents will have a back-up available to them?


Questions for the committee continued l.jpg

Questions for the Committee - continued

  • What other performance statistics would be the most useful to track?

  • What are the risks associated with targeting non-response follow-up to larger establishments to minimize a non-response adjustment?


  • Login