Improving the effectiveness of interviewer administered surveys though refusal avoidance training
Download
1 / 40

improving the effectiveness of interviewer administered surveys though refusal avoidance training - PowerPoint PPT Presentation


  • 300 Views
  • Uploaded on

Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training. Grace E. O’Neill Presented by Anne Russell U.S. Census Bureau ICES-III - June 19, 2007. Outline of Presentation. Background Respondent focus groups Refusal avoidance training

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'improving the effectiveness of interviewer administered surveys though refusal avoidance training' - daniel_millan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Improving the effectiveness of interviewer administered surveys though refusal avoidance training l.jpg

Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training

Grace E. O’Neill

Presented by Anne Russell

U.S. Census Bureau

ICES-III - June 19, 2007


Outline of presentation l.jpg
Outline of Presentation Surveys though Refusal Avoidance Training

  • Background

  • Respondent focus groups

  • Refusal avoidance training

  • Training content

  • Discussion

  • Future developments


Background l.jpg
Background Surveys though Refusal Avoidance Training


Respondent contact staff l.jpg
Respondent contact staff Surveys though Refusal Avoidance Training

  • Interviewers: outgoing calls to gain cooperation and to gather data

  • Clerks: incoming calls to provide basic information and resend forms

  • Analysts: professional staff who place outgoing calls concerning data errors or nonresponse follow-up


Respondent contact training l.jpg
Respondent contact training Surveys though Refusal Avoidance Training

  • Interviewers: Centralized training on telephone skills and refusal avoidance and conversion

  • Clerks: Shadow experienced clerks with informal discussion

  • Analysts: Basic training on respondent contact techniques


Why clerks l.jpg
Why clerks? Surveys though Refusal Avoidance Training

  • Self-administered paper/ electronic forms

  • Establishment respondents need less encouragement to participate

  • Nature of telephone calls


However l.jpg
However… Surveys though Refusal Avoidance Training

  • Over time, the role of the clerk has evolved into the role of an interviewer

  • Less cooperative respondents

  • More data collected over the telephone

  • More nonresponse follow-up telephone calls

  • Yet, their training has not evolved


Respondent focus groups l.jpg
Respondent Focus Groups Surveys though Refusal Avoidance Training


Focus groups l.jpg
Focus Groups Surveys though Refusal Avoidance Training

  • Monthly Trade Surveys

  • Wholesale and retail firms

  • Conducted by outside firm

  • Six focus groups, 44 participants

  • Respondent’s views and impressions

  • Ways to improve survey


Focus groups10 l.jpg
Focus Groups Surveys though Refusal Avoidance Training

  • Finding

  • Inexperienced clerks needed skills to increase participation by respondents, especially on voluntary surveys

  • Solution

  • Refusal avoidance training


Refusal avoidance training l.jpg
Refusal Avoidance Training Surveys though Refusal Avoidance Training


Refusal avoidance training12 l.jpg
Refusal Avoidance Training Surveys though Refusal Avoidance Training

  • Groves and McGonagle (2001)

    • Assemble respondent concerns

    • Develop responses

    • Train interviewers to classify concerns

    • Train interviewers to provide quick and appropriate responses


Refusal avoidance training13 l.jpg
Refusal Avoidance Training Surveys though Refusal Avoidance Training

  • Interactive

  • Cooperative learning

  • Flexible

  • Survey specific

  • Provides telephone skills, refusal avoidance techniques, and improves communication


Training specifics l.jpg
Training specifics Surveys though Refusal Avoidance Training

  • Monthly Trade Surveys (MTS) and Quarterly Services Survey (QSS)

  • 43 MTS clerks, 14 QSS clerks

  • Mix of tenure and survey experience

  • Supervisor and survey manager involvement

  • Eight hours over two days


Training preparation l.jpg
Training Preparation Surveys though Refusal Avoidance Training

  • Adapt household- based training to establishment-based survey

  • Adapt training to the MTS or QSS

  • Identify respondent concerns and develop solutions

  • FAQ Job Aid


Training content l.jpg
Training Content Surveys though Refusal Avoidance Training


Refusal avoidance training modules l.jpg
Refusal avoidance training modules Surveys though Refusal Avoidance Training

  • Module 1: Introduction

  • Module 2: Survey Specific

  • Module 3: Shared Experience

  • Module 4: Preparing a Telephone Call

  • Module 5: Telephone Skills

  • Module 6: Identify, Analyzing, and Dealing with Reluctance and Refusal

  • Module 7: Recovering from Negative Calls

  • Module 8: Wrap-up and Evaluation


Module 1 l.jpg
Module 1 Surveys though Refusal Avoidance Training

  • Introduction

  • Introduces training to clerks

  • Introduces the trainer and clerks to each other

  • Provides training schedule


Module 2 l.jpg
Module 2 Surveys though Refusal Avoidance Training

  • Survey Specific

  • Introduces survey specific content

  • Conducted by survey manager or survey staff


Module 3 l.jpg
Module 3 Surveys though Refusal Avoidance Training

  • Shared Experience

  • Identify clerks’ biggest concerns and difficulties

  • Develops solutions

  • Concerns are used in a later module


Module 3 example l.jpg
Module 3 example Surveys though Refusal Avoidance Training

  • Trainer: “What are some of the things respondents say when you talk to them?”

  • Clerks: “Why should I do this,” “I’m not a wholesaler” etc.

  • Group: Identify appropriate responses

  • Trainer: Make sure identified concerns are addressed and adds new concerns to list


Module 4 l.jpg
Module 4 Surveys though Refusal Avoidance Training

  • Preparing for the Telephone Call

  • Asks clerks how they prepare for telephone calls

  • Assess what tools clerks use to find information about company and what tools they might need


Module 5 l.jpg
Module 5 Surveys though Refusal Avoidance Training

  • Telephone Skills

  • Assess clerks active listening skills

  • Discuss tone

  • Discuss mechanics of placing a telephone call


Module 6 l.jpg
Module 6 Surveys though Refusal Avoidance Training

  • Identifying, Analyzing and Dealing with Reluctance and Refusal

  • Classify concerns identified in Module 3 as reluctance and/ or refusal

  • Further discuss solutions

  • Paired practice during class

  • Reviewed updated FAQ Job Aid


Module 7 l.jpg
Module 7 Surveys though Refusal Avoidance Training

  • Recovering from Negative Calls

  • Discusses recovering from refusals and other negative calls

  • Helps clerks to evaluate negative experiences


Module 8 l.jpg
Module 8 Surveys though Refusal Avoidance Training

  • Wrap-up and Evaluation

  • Review main training points

  • Clarify any remaining concerns

  • Evaluations by clerks


Results l.jpg
Results Surveys though Refusal Avoidance Training


Results28 l.jpg
Results Surveys though Refusal Avoidance Training

  • Clerk evaluation

  • Usefulness of workshop

  • Usefulness of skills learned

  • Increased confidence, preparation, and communication


Results29 l.jpg
Results Surveys though Refusal Avoidance Training

  • Response rates

  • Caveats:

  • Not experimentally tested

  • Confounded by other survey conditions

  • Don’t know how many potential refusals clerks prevented


Results mts l.jpg
Results: MTS Surveys though Refusal Avoidance Training

  • Initially lower from previous response period

  • Loss of clerks

  • Misclassification of refusals


Results mts31 l.jpg
Results: MTS Surveys though Refusal Avoidance Training

  • Over time

  • Good response rates, decrease in wholesale refusals

  • Continued improvement

  • Clerks have more responsibilities


Results qss l.jpg
Results: QSS Surveys though Refusal Avoidance Training

  • Decline in response rates

  • Imputation remained stable

    • More companies refused

    • Larger companies continued to report


Results qss33 l.jpg
Results: QSS Surveys though Refusal Avoidance Training

  • Communication has improved

    • Clerks are open about sharing concerns with survey managers

    • Bi-weekly telephone calls with call center

  • Annual refresher training


Discussion l.jpg
Discussion Surveys though Refusal Avoidance Training


Benefits l.jpg
Benefits Surveys though Refusal Avoidance Training

  • Centralized dissemination of skills and information

  • Practice occurs in a test environment

  • Provides training at regular intervals

  • Proactive training instead of reactive training


Benefits36 l.jpg
Benefits Surveys though Refusal Avoidance Training

  • Communication between clerks, supervisors, and survey managers

  • Clerks feel invested in data collection process

  • Survey managers gain direct insight into data collection process


Costs l.jpg
Costs Surveys though Refusal Avoidance Training

  • Staff time writing and delivering training

  • Telephone coverage

  • Monetary cost of training

  • Difficult to provide conclusive evidence


Future developments l.jpg
Future Developments Surveys though Refusal Avoidance Training


Future developments39 l.jpg
Future Developments Surveys though Refusal Avoidance Training

  • Formalized process for clerk training

  • Data capture of call concerns

  • Follow-up evaluation by clerks

  • Analyst training


Thank you l.jpg
Thank You Surveys though Refusal Avoidance Training

  • Please feel free to contact the author at:

  • Grace E. O’Neill

  • Email: [email protected]

  • Phone: 301-763-3537


ad