survey design and analysis planning a survey l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Survey Design and Analysis: Planning a Survey PowerPoint Presentation
Download Presentation
Survey Design and Analysis: Planning a Survey

Loading in 2 Seconds...

play fullscreen
1 / 23

Survey Design and Analysis: Planning a Survey - PowerPoint PPT Presentation


  • 229 Views
  • Uploaded on

Survey Design and Analysis: Planning a Survey. Division of MH/DD/SAS Quality Management Team . How to plan a survey. Development of Proposal Instrument Development and Survey Design Data Collection Post-collection Processing Analysis and Dissemination . Development of Proposal.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Survey Design and Analysis: Planning a Survey' - dean


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
survey design and analysis planning a survey

Survey Design and Analysis:Planning a Survey

Division of MH/DD/SAS

Quality Management Team

how to plan a survey
How to plan a survey

Development of Proposal

Instrument Development and Survey Design

Data Collection

Post-collection Processing

Analysis and Dissemination

development of proposal
Development of Proposal
  • Goal: identify a need for survey data and obtain resources for data collection
    • Key tasks:
      • Define questions to be answered by the survey
      • Review existing data and literature
      • Outline survey instrument
      • Set rough timetable for survey operations and analysis
instrument development and survey design
Instrument Development and Survey Design
  • Goal: Develop survey tools to meet the objectives, given resources
    • Key tasks:
      • Develop Request for Proposals for survey vendor bids, if needed
      • Finalize survey timetable
      • Develop sampling procedures
      • Create and edit test survey instrument
      • Develop interviewer training materials
      • Conduct field test of survey
      • Evaluate and revise test instrument based on the field test
      • Develop final instrument
      • Work out a plan for analyzing the survey data
designing a questionnaire
Designing a Questionnaire
  • The place to start in designing a questionnaire is with your data collection goals
    • What information do you need and from whom?
    • What pieces of specific information are needed to satisfy these objectives?
types of questions
Types of Questions

Open-ended

Partially open ended

Closed-ended

Rating/Likert Scale

use appropriate language
Use Appropriate Language

DON’T: In the past 30 days, how often have you used Tina?

DO: In the past 30 days, how often have you used crystal methamphetamines (Tina, speed, ice, glass)?

use simple sentence structures
Use Simple Sentence Structures

DON’T: In the past 30 days, when you used needles for injecting drugs, have you shared them with anyone else?

DO: In the past 30 days, have you shared needles?

avoid double barreled questions
Avoid “Double-Barreled” Questions

DON’T: Are you getting along better with your family and friends?

DO: Are you getting along better with your family ? And Are you getting along better with your friends?

avoid embedding assumptions or opinions
Avoid embedding assumptions or opinions

DON’T: Have you ever inappropriately gone to an Emergency Department when you were in need of mental health services, but your regular provider was not available?

DO: Have you ever gone to an Emergency Department when you were in need of mental health services, but your regular provider was not available?

data collection
Data Collection
  • Goal: Gather raw data in a timely and cost-effective manner
    • Key Tasks:
      • Conduct training
      • Field the survey
      • Data capture
      • Monitoring interviewers
      • Communication between vendor and sponsor to resolve operational issues
post collection processing
Post-Collection Processing
  • Goal: Generate accurate and organized final data set
    • Key Tasks:
      • Clean the data
      • Edit the data
      • Missing data imputation, if desired
      • Weighting, if desired
      • Generate data dictionary and code book
      • Contains data set description, variable names, and basic tabulations
summary
Summary
  • Design questions carefully, paying attention to:
    • Question wording
      • Person-centered language, appropriate vocabulary level
    • Length of questions
      • Ensure questions are not too long and do not contain multiple concepts.
    • Length of questionnaire
      • If long, you may not get meaningful data for questions at the end.
    • Order of questions
    • Appearance of questionnaire
how to plan in quality
How to “plan in” quality
  • Plan in quality to keep respondent mistakes and biases to a minimum.
    • Make sure there are mechanisms for quality checks at every point.
    • Make sure sample is selected according to specifications.
    • Pretest questions to ensure they are getting the data you need.
    • If using interviewers, make sure they are appropriately trained and understand their task.
    • Ensure survey answers are coded correctly.
    • Ensure computer programs for data analysis are working properly.
judging the quality of a survey
Judging the quality of a survey
  • There are many issues that can impact the quality of survey data.
  • This section describes potential issues and remedies for common problems.
survey non response and measurement
Survey non-response and measurement
  • Non-response
    • Potential respondent can not or will not participate in the survey or answer specific questions
  • Measurement error
    • Bias or error when surveys do not measure what they intend to
    • Example: Measuring consumer satisfaction with case manager when you are trying to measure satisfaction of services in their entirety
how do problems affect survey results
How do problems affect survey results?
  • Bias
    • Nonresponse bias is the bias that results when respondents differ in meaningful ways from nonrespondents.
  • Variance
    • Less predictable effect that may cause projections to be higher one time but lower the next time
frequent causes of non response error
Frequent causes of non-response error
  • Refusals
    • Individuals are approached to participate in a survey but refuse to do so. Refusals may result from apathy, fear of invasion of privacy or any number of reasons. Some refusals are partial, where the respondent will answer some questions but not all.
  • Unable to answer
    • This includes persons who are unable to communicate for a variety of reasons. Many times a caregiver or family member can serve as a proxy for these individuals.
  • Not Founds
    • This includes individuals who have moved and left no forwarding address, have no permanent address, are deceased, or are not contacted due to error in the survey procedures.
how to handle non response errors
How to handle non-response errors
  • Generalize to the respondents only. In stating that results are accurate for those surveyed, you can avoid making incorrect inferences about the larger population.
  • Assume there is no response bias and generalize to the population. If you know the population well and perceive the results to be reasonable, this strategy may be reasonable.
  • Recontact nonrespondents. Finding out why people did not respond can help determine the extent of response bias. If none is apparent then generalization to the population can be justified.
how to handle non response errors20
How to handle non-response errors
  • Compare data in hand on respondents and nonrespondents. If data, e.g., sex, age, race, is available, the composition of respondents can be compared with that of nonrespondents to see if there are any differences. The presence of differences indicates response bias and that caution is necessary in making inferences.
  • Increase contact efforts. Although this is probably the most costly and time-consuming strategy, obtaining the highest response rate possible is the best way to reduce response bias.
  • Throw away the data. If you feel the survey results are invalid, a logical response is not to use it. For many reasons, this strategy is not often used.
frequent causes of measurement error
Frequent causes of measurement error
  • Failure to identify the target population
    • Surveying individuals who have only been briefly engaged in service
  • Questionnaire design
  • Interviewer bias
  • Respondent bias
  • Processing errors
  • Misinterpretation of results
frequent causes of measurement error22
Frequent causes of measurement error
  • Failure to identify the target population
  • Questionnaire design
  • Interviewer bias
  • Respondent bias
  • Processing errors
  • Misinterpretation of results
how to prevent measurement errors
How to prevent measurement errors
  • Careful selection of the time and place the survey is conducted
  • Using an up-to-date, accurate sample framework
  • Careful questionnaire design
  • Providing thorough training for interviewers and processing staff
  • Being aware of all the factors affecting the topics that are being addressed by the survey