Workshop 4c data collection
1 / 25

tonto.eia.doe - PowerPoint PPT Presentation

  • Updated On :

Workshop 4c Data Collection Stephanie J. Battles Energy Information Administration Dhaka, Bangladesh September 22, 1999 Data Collection Mail Survey Personal Interview Telephone Interview: Computer-Assisted Telephone Survey (CATI) Other Modes of Data Collection

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'tonto.eia.doe' - adamdaniel

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Workshop 4c data collection l.jpg

Workshop 4cData Collection

Stephanie J. Battles

Energy Information Administration

Dhaka, Bangladesh

September 22, 1999

Data collection l.jpg
Data Collection

  • Mail Survey

  • Personal Interview

  • Telephone Interview: Computer-Assisted Telephone Survey (CATI)

  • Other Modes of Data Collection

  • Multiple Modes of Data Collection

  • Survey Supplement Using A Subsample

  • Nonsampling Error

    • Nonresponse

    • Response

Mail survey self reported data l.jpg


Cost Effectiveness

Small Staff

Same Mailing Cost for 1 Question as it Costs for 100

Easily Target Population of Interest

Studies Show That More Answers Are Provided


Lower Response Rates

Need to Have Good Planning

Need Complete Mailing List

No Opportunity for Follow-up

Need Questions that are Simple and Short

Legible/Easy to Follow

Correct Language

Can Respondent Read?

Mail Survey(Self-Reported Data)

  • Design and Format Crucial

  • Future--CSAQ

Mail survey example 1998 mecs l.jpg
Mail SurveyExample: 1998 MECS

  • Three Different Forms, Forms A,B, and C

    • Form A: Collects Basic Data

    • Form B: Petroleum Refining

    • Form C: Establishments Who Ship Energy Offsite

  • 18,000 Establishments

  • Average Burden per Response--8.24 Hours

  • U.S. Census Bureau--Contractor

  • Two Mail Follow-Ups

  • Additional Telephone Follow-Up for the Largest Establishments Not Responding

Personal interview face to face l.jpg


High Response Rates Are Possible

Survey Questions Can be More Complex

Opportunity for Follow-Up Questions

Can Assist Respondents Who are Disabled, Aged or Other Problem



Need a Large Staff or Contract Personnel

Natural Disaster

Lose Portion of Sample

May Not Get Person Most Capable of Answering the Questions

Sample May Be in Restricted area

Personal Interview(Face-to-Face)

Personal interview face to face6 l.jpg
Personal Interview(Face-to-Face)

  • Interviewer Training (Classroom/Self-study/Both)

  • Introductory Letter Before Initial Contact

    • Purpose of the Survey

    • Why Participation is Important

    • Confidential Responses

  • Exhibit Cards

  • Multicontacts

    • Callbacks at Different Times of the Day

  • Multiwave

    • Reassigned Different Interviewers

  • Use of Incentives

Personal interview example 1997 recs l.jpg
Personal InterviewExample-1997 RECS

  • Trainer Training

    • Three Days

  • Interviewer Training

    • Three Days (2 Sessions)/220 Interviewers

      • Washington D.C.

    • Topics Covered:

      • General Interviewing Techniques

      • RECS Background

      • Introduction to the CAPI system

      • Practice With Mock Interviews

      • Review of Administrative Requirements

All Training Sessions Were Monitored by EIA Staff

Personal interview example 1997 recs8 l.jpg
Personal InterviewExample-1997 RECS

  • Training Manual

    • Locating Exercises

    • Identifying Respondents and Getting In the Door

    • Mock Exercises (one person is the interviewer..the other is the respondent)

  • Instruction Manual

    • Administrative Procedures

    • Identifying RECS Households

    • Conducting the Interview

    • Question by Question Discussion

  • Training Used Computer (CAPI)

  • Letter Sent Before Initial Contact

  • Personal interview example 1997 recs9 l.jpg
    Personal InterviewExample-1997 RECS

    • Data Collection Results

      • Average Time: 29 Minutes

      • Interviewers Conducted an Average of 27 Interviews

      • Median Number of Visits--3

      • 20% of the Interviews Were Were Verified by Mail or Telephone

      • Used Multicontact Approach

        • Different Time of Day

      • Used Multiwave Approach

        • Changed Interviewer

      • Mail Follow-Ups for Nonresponsive Households

        • Short Self-Administered Questionnaire

        • Sent Pen as an Incentive

    Personal interview example 1997 recs10 l.jpg
    Personal InterviewExample-1997 RECS

    • Data Collection Response

      • 8,319 Listed Units--7,285 Eligible

        • Ineligible

          • Out of Scope, e.g. Business, Not Habitable

          • Vacant or Seasonal

      • 5,902 Completed (81% Response Rate)

      • 5% of the Personal Interviews Completed After the Second Wave (299)

      • 3 % of the Interviews Completed After the Mail Follow-up

        • 1,421 Households--Returned 10 % (181)

    Personal interview example 1997 recs11 l.jpg

    High Response Rates

    South (81.1%)

    Midwest (80.2%)

    Rural Areas (85%)

    Single Family and Mobile Homes (80.5%)

    Low Response Rates

    Northeast (73.1%)

    Urban/Suburbs (76.4%)

    Large Apartment Buildings (72.1%)

    Personal InterviewExample-1997 RECS

    Response Rates and Household Characteristics

    Telephone interview cati l.jpg


    Increase Quality


    Programmed to Correctly Skip Automatically

    Edit Replies

    Produce Statistics Quickly

    Eliminates the Keying of the Data



    Savings When the Survey is Large and/or Frequent and Questions Remain the Same

    Lower Response Rates

    Answering Machines

    No Telephone

    Up Front Costs


    Telephone Interview(CATI)

    Most Telephone Surveys are CATI

    (Computer-Assisted Telephone Interviews)

    Other modes of data collection l.jpg
    Other Modes of Data Collection

    BLS: Current



    • Touchtone Data Entry (TDE)

      • Alternative to Mail Collection

      • Questionnaire is on a Computer

      • Respondent Answers Prerecorded Questions Using the Touchtone Buttons

      • Useful for a Few Numbers

      • Eliminates Key Entry

    • Voice Recognition (VR)

      • Recognizes Numbers “0 to 9”, “Yes” and “No”

      • Useful for A Few Numbers

      • Accent Recognition Problem


    Other modes of data collection14 l.jpg
    Other Modes of Data Collection

    • Facsimile Transmission (FAX)

      • Eliminates Mail Delays

    • Electronic Data Interchange (EDI) Collect Large Volumes of Data

      • Electronic Transmission is in a Standard Format

      • Errors are Reduced

      • Data are Instantly Available

    • Email Transmission

      • Paperless Collection

      • Eliminates Mail Delays

      • Errors/Costs Are Reduced

      • Confidentiality Issues

    EIA’s Financial Reporting System

    Multiple modes of data collection l.jpg
    Multiple Modes of Data Collection

    • Survey Requires the Collection of Data from Different Respondent

    • Increase Survey Response Rates

      • Respondents May Favor One Mode Over Another

    • Certify Data Collected Are Correct

    • Reduce Costs

    Multiple modes of data collection example 1993 recs l.jpg
    Multiple Modes of Data CollectionExample: 1993 RECS

    • Personal Interview

      • Household/Housing Unit Data

      • Authorization Form

    • Mail Survey

      • Household Mail Survey

        • Obtain Data for Households Who Refused to Participate in the Personal Interview

      • Energy Supplier Survey

        • Natural Gas, Electricity, LPG, and Fuel Oil

        • Obtain Energy Consumption and Expenditure Data from the Household’s Energy Suppliers

    Multiple modes of data collection example 1993 recs17 l.jpg
    Multiple Modes of Data CollectionExample: 1993 RECS

    • Telephone Survey

      • Rental Agent Survey

        • Verify Information Received From Certain Households Who Did not Pay for All Their Energy or Paid A Third Party and Rented or Owned Living Quarters in A Multi-Unit Building

    Survey supplement using a subsample l.jpg
    Survey Supplement Using A Subsample

    • Obtain Detailed Data

    • Use if Too Costly for the Full Sample

    • Use if Response Burden is a Concern

    • Use Data to Develop a Model Based on Data Available From the Full Survey

    Survey supplement using a subsample example 1993 recs lighting supplement l.jpg
    Survey Supplement Using A SubsampleExample: 1993 RECS Lighting Supplement

    • Purpose of the Supplement

      • Collect Data to Develop a Model to Predict Electricity Consumption for Lighting

    • Proportional Sample of All U.S. Households

    • 7% of the Completed Interviews

    • Lighting Model Used in Electricity Regression Equation Along Other End-Use Models, e.g. Space Heating.

    Nonsampling error unit nonresponse error l.jpg
    Nonsampling ErrorUnitNonresponse Error

    Unit Nonresponse Error: Unit nonresponse error occurs when a sampling unit does not participate in the survey , e.g. household, building or establishment.

    • Unit Nonresponse

      • Refusal/Failure to Return Questionnaire

        • Sensitive Information

        • Incorrect Address

        • Not Have Resources or Time To Reply

        • Not Interested

        • Not Like the Sponsor

      • Could Not Be Located

      • Natural Disaster

      • Recently Vacant/Destroyed

      • Telephone Disconnected

    Nonsampling error unit nonresponse control of error l.jpg
    Nonsampling ErrorUnit Nonresponse :Control of Error

    • Sampling Unit’s Record Carefully Labeled and Transmitted

    • Prior Notice

    • Follow-Up

    • Incentives

    • Questionnaire Design

    • Confidentially Ensured

    • Reweighting the Sample

    Nonsampling error item nonresponse error l.jpg
    Nonsampling ErrorItemNonresponse Error

    Item Nonresponse Error: Item nonresponse error occurs when a particular item of interest is missing from a questionnaire.

    • Sensitive Question

      • Confidentiality Issue

    • Recall

      • Aged or Disabled

    • Fatigue/Burden

      • Question Long and Complex

      • Open Question

    • Records Not Available

      • No Advance Notice

    • Interviewer Error

    • Programming Problems

    Nonsampling error item nonresponse control of error l.jpg
    Nonsampling ErrorItemNonresponse:Control of Error

    • Interviewer Training

    • Confidentiality Reminder

    • Use Memory Aids and Cues

    • Question Shorts and Simple

    • Questionnaire-Reasonable Length

      • Personal Interview, e.g. 60 Minutes

    • Understandable Skip Patterns

    • Limit Open Questions

    • Imputation

    Nonsampling error response error l.jpg
    Nonsampling ErrorResponse Error

    Response Error: Response error occurs when the item

    response is answered incorrectly.

    • Satisficing Answer

      • Doesn’t Care/Time Burden

    • Inability to Answer--A Guess

    • Deliberate Incorrect Response

      • Sensitive Question

      • Respondent Knows the “Correct Answer”

    • Invalid/Out of Date Records

    • Interviewer Incorrectly Records Response

    • Interviewer Bias

    Nonsampling error response error control of error l.jpg
    Nonsampling ErrorResponse Error:Control of Error

    • Importance of the Survey and Accurate Responses

    • Use Memory Aids/Examples

    • Ensure Confidentiality of Responses

    • Advance Notice

    • Interviewer Training

    • Locate the Respondent With the Proper Knowledge

    • Quality Control of the Interviewing Process

    • Allow Respondents to Ask Questions