1 / 40

Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training

Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training. Grace E. O’Neill Presented by Anne Russell U.S. Census Bureau ICES-III - June 19, 2007. Outline of Presentation. Background Respondent focus groups Refusal avoidance training

Download Presentation

Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.


Presentation Transcript

  1. Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. O’Neill Presented by Anne Russell U.S. Census Bureau ICES-III - June 19, 2007

  2. Outline of Presentation • Background • Respondent focus groups • Refusal avoidance training • Training content • Discussion • Future developments

  3. Background

  4. Respondent contact staff • Interviewers: outgoing calls to gain cooperation and to gather data • Clerks: incoming calls to provide basic information and resend forms • Analysts: professional staff who place outgoing calls concerning data errors or nonresponse follow-up

  5. Respondent contact training • Interviewers: Centralized training on telephone skills and refusal avoidance and conversion • Clerks: Shadow experienced clerks with informal discussion • Analysts: Basic training on respondent contact techniques

  6. Why clerks? • Self-administered paper/ electronic forms • Establishment respondents need less encouragement to participate • Nature of telephone calls

  7. However… • Over time, the role of the clerk has evolved into the role of an interviewer • Less cooperative respondents • More data collected over the telephone • More nonresponse follow-up telephone calls • Yet, their training has not evolved

  8. Respondent Focus Groups

  9. Focus Groups • Monthly Trade Surveys • Wholesale and retail firms • Conducted by outside firm • Six focus groups, 44 participants • Respondent’s views and impressions • Ways to improve survey

  10. Focus Groups • Finding • Inexperienced clerks needed skills to increase participation by respondents, especially on voluntary surveys • Solution • Refusal avoidance training

  11. Refusal Avoidance Training

  12. Refusal Avoidance Training • Groves and McGonagle (2001) • Assemble respondent concerns • Develop responses • Train interviewers to classify concerns • Train interviewers to provide quick and appropriate responses

  13. Refusal Avoidance Training • Interactive • Cooperative learning • Flexible • Survey specific • Provides telephone skills, refusal avoidance techniques, and improves communication

  14. Training specifics • Monthly Trade Surveys (MTS) and Quarterly Services Survey (QSS) • 43 MTS clerks, 14 QSS clerks • Mix of tenure and survey experience • Supervisor and survey manager involvement • Eight hours over two days

  15. Training Preparation • Adapt household- based training to establishment-based survey • Adapt training to the MTS or QSS • Identify respondent concerns and develop solutions • FAQ Job Aid

  16. Training Content

  17. Refusal avoidance training modules • Module 1: Introduction • Module 2: Survey Specific • Module 3: Shared Experience • Module 4: Preparing a Telephone Call • Module 5: Telephone Skills • Module 6: Identify, Analyzing, and Dealing with Reluctance and Refusal • Module 7: Recovering from Negative Calls • Module 8: Wrap-up and Evaluation

  18. Module 1 • Introduction • Introduces training to clerks • Introduces the trainer and clerks to each other • Provides training schedule

  19. Module 2 • Survey Specific • Introduces survey specific content • Conducted by survey manager or survey staff

  20. Module 3 • Shared Experience • Identify clerks’ biggest concerns and difficulties • Develops solutions • Concerns are used in a later module

  21. Module 3 example • Trainer: “What are some of the things respondents say when you talk to them?” • Clerks: “Why should I do this,” “I’m not a wholesaler” etc. • Group: Identify appropriate responses • Trainer: Make sure identified concerns are addressed and adds new concerns to list

  22. Module 4 • Preparing for the Telephone Call • Asks clerks how they prepare for telephone calls • Assess what tools clerks use to find information about company and what tools they might need

  23. Module 5 • Telephone Skills • Assess clerks active listening skills • Discuss tone • Discuss mechanics of placing a telephone call

  24. Module 6 • Identifying, Analyzing and Dealing with Reluctance and Refusal • Classify concerns identified in Module 3 as reluctance and/ or refusal • Further discuss solutions • Paired practice during class • Reviewed updated FAQ Job Aid

  25. Module 7 • Recovering from Negative Calls • Discusses recovering from refusals and other negative calls • Helps clerks to evaluate negative experiences

  26. Module 8 • Wrap-up and Evaluation • Review main training points • Clarify any remaining concerns • Evaluations by clerks

  27. Results

  28. Results • Clerk evaluation • Usefulness of workshop • Usefulness of skills learned • Increased confidence, preparation, and communication

  29. Results • Response rates • Caveats: • Not experimentally tested • Confounded by other survey conditions • Don’t know how many potential refusals clerks prevented

  30. Results: MTS • Initially lower from previous response period • Loss of clerks • Misclassification of refusals

  31. Results: MTS • Over time • Good response rates, decrease in wholesale refusals • Continued improvement • Clerks have more responsibilities

  32. Results: QSS • Decline in response rates • Imputation remained stable • More companies refused • Larger companies continued to report

  33. Results: QSS • Communication has improved • Clerks are open about sharing concerns with survey managers • Bi-weekly telephone calls with call center • Annual refresher training

  34. Discussion

  35. Benefits • Centralized dissemination of skills and information • Practice occurs in a test environment • Provides training at regular intervals • Proactive training instead of reactive training

  36. Benefits • Communication between clerks, supervisors, and survey managers • Clerks feel invested in data collection process • Survey managers gain direct insight into data collection process

  37. Costs • Staff time writing and delivering training • Telephone coverage • Monetary cost of training • Difficult to provide conclusive evidence

  38. Future Developments

  39. Future Developments • Formalized process for clerk training • Data capture of call concerns • Follow-up evaluation by clerks • Analyst training

  40. Thank You • Please feel free to contact the author at: • Grace E. O’Neill • Email: grace.e.oneill@census.gov • Phone: 301-763-3537

More Related