1 / 33

Quality Control for Field Operations

Quality Control for Field Operations. Overview. Goal To ensure the quality of survey field work Purpose To detect and deter interviewer errors and falsification. Overview (cont.). What do we check in QC for field work? Interviewer procedures Forms Interviewer productivity Interviews

Download Presentation

Quality Control for Field Operations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Quality Control for Field Operations

  2. Overview • Goal • To ensure the quality of survey field work • Purpose • To detect and deter interviewer errors and falsification

  3. Overview(cont.) What do we check in QC for field work? • Interviewer procedures • Forms • Interviewer productivity • Interviews • Address listing

  4. Overview(cont.) What are the tools for performing QC on field work? • Interviewer observations • Formal / Informal reviews • Monitoring reports • Reinterview • Address listing QC

  5. Interviewer Observations What are the goals of interviewer observations? • To assess effectiveness of interviewer training • To ensure interviewers understand and follow prescribed interviewing procedures • To provide interviewers with feedback to encourage continuous improvement

  6. InterviewerObservations(cont.) What are interviewer observations? • Supervisor “shadows” interviewer during initial assignments • Observe interviewer’s work to ensure appropriate procedures are followed • Evaluate interviewer’s work and provide positive and negative feedback to interviewer • See Handout on Field Control Form Information

  7. Interviewer Observations(cont.) What are the results of observations? • Supervisor decides if interviewer’s performance is acceptable • If so, allow interviewer to continue work independently • If not, re-train interviewer on procedures or release interviewer from the operation

  8. Formal / Informal Reviews What are the goals of reviews? • To ensure the quality of the data collected on forms and questionnaires • To ensure interviewers and clerks understand and follow procedures for completing forms and questionnaires

  9. Formal / Informal Reviews(cont.) How are informal reviews performed? • Supervisor meets with employees to review forms and/or questionnaires • Review for legibility, completeness, and accuracy • Provide positive and negative feedback to employee for continuous improvement • Repair errors where necessary

  10. Formal / Informal Reviews(cont.) How are formal reviews performed? • Usually performed in a field office by office staff • Review questionnaires or listings for legibility, completeness, and accuracy • Repair errors or return materials to the field when necessary

  11. Formal / Informal Reviews(cont.) What work products are subject to reviews? • Any form or questionnaire that requires a quality check (e.g., payroll forms, questionnaires, address listing forms, etc.)

  12. Formal / Informal Reviews(cont.) What are the results of informal reviews? • Supervisor determines if the form or questionnaire meets quality standards • If so, the form/questionnaire is accepted and submitted • If not, repair the form/questionnaire prior to submission

  13. Monitoring Reports What are the goals of monitoring reports? • To track status of operations • To inform management of progress • To identify problems early for timely rectification

  14. Monitoring Reports(cont.) How do we know what reports to monitor for QC purposes? • Define quality goals for the operation (e.g., response rates, productivity rates, cost, etc.) • Identify variables that provide information on quality goals • Design report(s) that present the important variables for quick review and assessment

  15. Monitoring Reports(cont.) How do we monitor the reports for QC purposes? • Determine necessary frequency for monitoring (usually daily) • Look for outlier values (e.g., low response rates, high costs, etc.) • Determine reason(s) for outlier values • Take appropriate action

  16. Reinterview What is reinterview? • Quality check on interviewing activities • Primary tool for ensuring quality of interviewers’ field work • Independent, objective review of interviewing work

  17. Reinterview(cont.) What types of reinterview can be used? • Random reinterview • Administrative reinterview • Supplemental reinterview

  18. Reinterview(cont.) What decisions are made in reinterview? • Falsification (release interviewer) • Interviewer error (retraining/feedback) • Respondent error (no action) • Rework may be necessary for falsification or other serious errors

  19. Random Reinterview What is the purpose of the random reinterview component? • Provides broad protection over entire workload and all interviewers • Serves as an effective deterrent for interviewer falsification

  20. Random Reinterview(cont.) What is the sampling design for the random reinterview component? • Select random sample of cases within every enumerator’s workload, or • Two-stage design: take a sample of interviewers, then a sample of cases within those interviewers’ workloads

  21. Administrative Reinterview What is the purpose of the administrative reinterview component? • Provides “targeted” approach to identifying suspect work • Provides valuable performance data to field supervisors

  22. Administrative Reinterview(cont.) How does the administrative model work? • Accumulates data on work characteristics for all interviewers • Vacant housing units • Partial interviews • Single-person households • Deleted housing units • Average population per housing unit

  23. Administrative Reinterview(cont.) How does the administrative model work? (cont.) • Compare accumulated data for each interviewer to the data for all other interviewers in their assignment area • Identify and flag interviewers who are significantly different than their area average

  24. Administrative Reinterview(cont.) How does the administrative model work? (cont.) • Provide field supervisors with a list of “outlier” interviewers and what variables caused the flag • Field supervisors determine if the interviewers’ “outlier” status is explicable • More of suspect interviewer’s cases put into reinterview

  25. Supplemental Reinterview What is the purpose of the supplemental reinterview? • Provides field supervisors with the capability of checking work for selected interviewers • Tool for use during investigation of suspected falsification

  26. Supplemental Reinterview(cont.) What is the sampling design for the supplemental reinterview? • Select a random sample of cases from the interviewer’s workload • Field supervisor can specify how many cases to check

  27. Address Listing QC What is the goal of address listing QC? • To ensure the quality of address listing work • To protect against undercoverage and overcoverage

  28. Address Listing QC(cont.) What are the tools for performing address listing QC? • Dependent QC • Advance listing • Suppression

  29. Address Listing QC(cont.) What decisions are made in address listing QC? • If errors exceed acceptable limit, recanvass assignment area • Repair minor mistakes • Listers may be released or retrained depending on severity of errors

  30. Dependent QC What is dependent QC? • Random, independent check of addresses listed by survey listers • Field supervisors verify existence of housing units or validity of housing unit deletes

  31. Dependent QC(cont.) What is the sampling design for dependent QC? • Select a random sample of housing units to verify from the listers’ workload • Sample should include all types of listings (e.g., verifications, adds, deletes)

  32. Advance Listing What is advance listing? • Field supervisor does initial listing of assignment areas prior to making assignments to listers • Gives supervisor knowledge of the area so they are able to detect faulty or inadequate work

  33. Suppression What is suppression? • Selected addresses are purposely left off listings • Listers’ ability to find and add the suppressed addresses is assessed • Provides confidence in the listers’ ability to perform adequate canvassing

More Related