survey response rates trends and standards l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Survey Response Rates: Trends and Standards PowerPoint Presentation
Download Presentation
Survey Response Rates: Trends and Standards

Loading in 2 Seconds...

play fullscreen
1 / 17

Survey Response Rates: Trends and Standards - PowerPoint PPT Presentation


  • 390 Views
  • Uploaded on

Survey Response Rates: Trends and Standards. Karen Donelan, ScD Senior Scientist in Health Policy Massachusetts General Hospital/Harvard Medical School AcademyHealth - June 27 th , 2006. Overview. What is a response rate? Why do we care about response rates? Trends

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Survey Response Rates: Trends and Standards' - davin


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
survey response rates trends and standards

Survey Response Rates: Trends and Standards

Karen Donelan, ScD

Senior Scientist in Health Policy

Massachusetts General Hospital/Harvard Medical School

AcademyHealth - June 27th, 2006

overview
Overview
  • What is a response rate?
  • Why do we care about response rates?
  • Trends
  • Does nonresponse = biased response?
  • How should rates be calculated and reported?
  • How can we improve?
what is a response rate
What is a Response Rate?
  • The number of complete interviews with reporting units divided by the number of eligible reporting units in the sample
other rates
Other Rates
  • Cooperation rates - The proportion of all cases interviewed of all eligible units ever contacted.
  • Refusal rates - The proportion of all cases in which a sampled unit or the respondent refuses to be interviewed, or breaks-off an interview, of all potentially eligible cases.
  • Contact rates - The proportion of all cases in which some responsible sampled unit member was reached.
why do we care about response rates
Why Do We Care About Response Rates?
  • Sources of Error in Surveys
    • Coverage, Measurement, Nonresponse, Processing, Sampling
  • Practical implications
    • Generalizability
    • Publishability (often cutoff or major test of quality)
    • Credibility
    • Fundability
current population survey cps
Current Population Survey (CPS)

http://www.census.gov/prod/2000pubs/tp63.pdf

slide9

U Michigan: Survey of Consumers

Source: Groves et al. 2004

why does nonresponse happen
Why Does Nonresponse Happen?
  • Who is in the sample? (demographics, SES, lifestyle)
  • # Contacts
  • Schedule of contacts
  • Mode of contact
  • Respondent selection
  • Respondent cooperation (refusal)
  • Incentives ($, benefit)
  • Respondent burden (time, boredom, frustration)
  • Salience
  • Respondent ability to respond to questions
  • Sponsorship
  • Privacy concerns
how should rates be calculated and reported disclosure
How Should Rates Be Calculated and Reported? Disclosure
  • Units drawn and attempted
    • Addresses mailed
    • Phone numbers called
    • Households approached
    • Patients invited
  • Sample Disposition: what happened?
    • Eligible and ineligible respondents
    • Bad contact information
    • How many attempts, at what intervals
    • # Never reached, # actually contacted
    • # completed, refused, never got a decision
does nonresponse biased response
Does Nonresponse = Biased Response?
  • Low response rates may not be problematic
  • Representativeness doesn’t necessarily increase with response rate
  • Sample representativeness is a function of the difference between respondents and nonrespondents on the statistic(s) of interest
      • High response rates can can yield an unrepresentative sample (high nonresponse bias)
      • Low response rates can still yield a representative sample
        • Keeter et al. (2000), Curtin, Presser and Singer (2000), Merkle and Edelman (2002)
measuring nonresponse bias
Measuring Nonresponse Bias
  • With response rates falling, understanding the impact on study findings is essential
  • Assess quantitatively if possible, but at least consider qualitatively
  • Methods for assessing nonresponse bias:
    • Response rate comparisons across groups
    • Follow-up interviews with non-respondents
    • Comparing early vs. late respondents
    • Comparisons to similar estimates from other sources
    • Post survey adjustment for nonresponse
ways not to approach nonresponse
Ways Not to Approach Nonresponse
  • Ignore it
  • Obscure it
  • Omit it
  • Fail to measure/comment on potential associated bias
  • Give up on it
improving response rates
Improving Response Rates
  • Multiple contacts
  • Multiple modes of contact
  • Interviewer training
  • Advance Notification/Endorsement
  • Incentives
  • Reduce respondent burden
  • Increase relevance to respondent
  • Time and money to get quality
reporting response rates
Reporting Response Rates
  • Know and apply standards (AAPOR, CASRO)
  • Engage and manage with vendors
  • Structure sample disposition
  • Response and nonresponse analysis
  • Publish enough information to allow others to do the calculations
  • Peer review: build awareness of standards