1 / 14

Survey Completion Rates and Resource Use at Each Step of a Dillman-Style Multi-Modal Survey

Survey Completion Rates and Resource Use at Each Step of a Dillman-Style Multi-Modal Survey. Andreal Hassol, Holly Harrison, Ricki Jarmon, Brenda Rodriguez, Austin Frakt Abt Associates Inc. Introduction. Designing DC efforts becoming an increasingly complex task.

ping
Download Presentation

Survey Completion Rates and Resource Use at Each Step of a Dillman-Style Multi-Modal Survey

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Survey Completion Rates and Resource Use at Each Step of a Dillman-Style Multi-Modal Survey Andreal Hassol, Holly Harrison, Ricki Jarmon, Brenda Rodriguez, Austin Frakt Abt Associates Inc.

  2. Introduction • Designing DC efforts becoming an increasingly complex task. • Declining participation in surveys • Projects must be competitive in costs • Rely on multi-mode DC efforts to achieve high response rates • Designing DC efforts we must weigh resources against expected returns. • Seek mode(s) of DC yielding high quality data and high response rates. • Minimizing nonresponse bias (increase generalizability of data) • Delivered on schedule, on budget • “Total Design Method” widely known / cited in the industry (Dillman, 1978).

  3. Background: Elements of the Total Design Method (TDM) • Respondent friendly questionnaire • Topic of survey salient to survey recipient • Up to 5 contacts with survey recipients (via mail, poss. with phone follow-up) • Inclusion of a stamped return envelope • Personalized Correspondence • Token Financial incentives

  4. Utilizing the TDM in this Project • Lit. review - sought out resources that demonstrated results of TSD implementation and related costs and returns. • Minimal work tangibly demonstrating effects of reminder post cards or additional mailings over time. • This research seeks to fill that gap.

  5. Project Overview Client: Centers for Medicare and Medicaid Services (CMS) Response Rate Goal: 70% Mode of DC: Mail Survey with Phone Follow-up. Field Period: 21 weeks. (Mid-July to December 2002). Sample: 7,730 Community-dwelling Medicare Beneficiaries Topic: Evaluation of beneficiaries’ experiences with a new Medicare Insurance Option. Design:6 contacts by mail with phone follow-up. No $ incentive.

  6. Methodology • 7,730 cases from Medicare Enrollment database, across 25 states. • Included three subgroups: • Those enrolled in new Private Fee for Service (PFFS) plan • Those remaining in traditional Medicare managed care plans. • Those with Medigap supplemental policy • Proxies allowed (spouse / partner / adult child) • Toll-free number staffed by interviewers where recipients could dial to do survey or ask questions • Conducted English and Spanish • 13 page SAQ / CAPI

  7. Advance Letter Survey Packet I – First Class Mail Reminder Postcard I Survey Packet II – First Class Mail Postcard II Survey III – Priority Mail Phone follow-up (47% of non responders matched) Continued receipt of mail quex Week 1 Week 1 Week 5 Week 6 Week 9 Week 13 Weeks 14-21 Weeks 14-21 Timeline of Project

  8. Sample Management • Integral to success of multi-modal project. • Database designed to fulfill receipting and reporting needs for mail and CATI completes. • Downloads of CATI efforts each night • Receipt of mail completes • Enable us to track progress carefully by mode, sample type, site • Minimized burden on sampled beneficiaries • Avoiding re-mails to already complete or identified ineligible • Avoid phone follow-up to those submitted by mail • Minimized number of duplicate completes (mail and phone)

  9. Completion Rate through the Field Period Quex mailing Weeks 1, 6, 13 Postcards Weeks 5, 9 CATI 14-21

  10. Completes by Week • 87% of completes by mail • 13% of completes by phone

  11. Costs Incurred Through the Field Period • DC Costs • (87% of budget): • 23% on phone • 64% on Mail Remaining 13% budget for DP

  12. Results on Salience of Survey Topic 79% 69% 67% 73%

  13. Conclusions • TDM worked, minor adjustments to tailor to our population and study. • Each remail effort saw spike in returns, peaking within 2 weeks of mail out • Less noticeable returns following postcards • Telephone efforts were 13% of completed surveys but 23% of costs. • Salience of instrument proves true – as highest RR in PFFS group (79%) • Respondent friendly design in quex –skip pattern confusion yielded extra data • Database essential component to success • Minimized burden • Maximize labor resources in phone follow-up effort to nonresponders only. • Over 7 week dual effort – 21 cases completed phone and mail. • Attained a 73% response rate (goal 70%), using AAPOR calc. standards.

  14. Abt Associates Inc.

More Related