So now you want to do a survey
Download
1 / 29

so now you want to do a survey... - PowerPoint PPT Presentation


  • 171 Views
  • Uploaded on

So Now You Want to Do a Survey. Words of Advice, Words of Caution Jay P. Paul, PhD UCSF Center for AIDS Prevention Studies. Key Questions To Consider When Asking Questions. Is this item understandable to the participant? Is this item something the participant can answer?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'so now you want to do a survey...' - lotus


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
So now you want to do a survey l.jpg

So Now You Want to Do a Survey...

Words of Advice, Words of Caution

Jay P. Paul, PhD

UCSF Center for AIDS Prevention Studies


Key questions to consider when asking questions l.jpg
Key Questions To Consider When Asking Questions

  • Is this item understandable to the participant?

  • Is this item something the participant can answer?

  • Is this item something the participant will be willing to answer honestly?

  • How important is this item compared to others I have to avoid respondent burden?


Areas to consider in designing your survey l.jpg
Areas to Consider in Designing Your Survey

  • What type of survey methods are out there and what can they do for me?

  • What are the things to consider in maximizing the quality of data with respect to questions I ask?

  • How do I consider using existing scales and putting it all together?



Survey method options l.jpg
Survey Method Options

  • IAQ

  • SAQ

    • Including Diary Methods, Mail Surveys

  • CASI/CAPI (and A-CASI)

  • CATI

  • T-ACASI

  • Internet


Interviewer administered questionnaire iaq l.jpg
Interviewer-Administered Questionnaire (IAQ)

  • Most flexible and responsive to respondent re: comprehension of question, selection of response options.

  • Able to handle skip patterns that could confuse the respondent in a self-administered questionnaire.

  • Human element, rapport & responsiveness also can introduce problems in lack of standardization of delivery, self-presentation.


Self administered questionnaire saq l.jpg
Self-Administered Questionnaire (SAQ)

  • Allows for greater privacy than face-to-face interviewing

  • Tends to be associated with higher rates of reporting of sensitive behaviors

  • Literacy requirements for respondent

  • Cannot be adjusted to be responsive to needs of individual participant

  • Cannot handle complex “skip patterns”

  • Low-cost survey delivery option


Diary methods l.jpg
Diary Methods

  • Provides optimal potential for recording of accurate rates of target behaviors

    • Recall

    • Exploration of variations over time

  • Allows for question design to examine co-occurrence of key behaviors (e.g., sexual behavior and drug use)

  • Burden of consistent completion of information means that data may be missing in possibly non-random pattern, or completed at a later time than designed.

  • NOTE: Can be useful to design for Web-based implementation.


Mail surveys l.jpg
Mail Surveys

  • Many of considerations of SAQ are applicable -- with format even more crucial.

  • Many respondents report liking the flexibility of completing surveys at their own pace.

  • Need for clear and easy format to maintain respondent interest and ease of completion.

  • Critical to have contact information and clear procedures to follow up on respondents to ensure adequate response rates.


Computer assisted self interview personal interview casi capi l.jpg
Computer-Assisted Self-Interview /Personal Interview (CASI/CAPI)

  • Survey is delivered via computer (can be laptops for maximal flexibility), with respondent keying in responses.

  • Usual mode is “Audio-CASI” or A-CASI, which involves respondent listening to audio recording of survey as they see items on-screen.

  • For special PCs with touch screens, can avoid keyboard and use fingers to select choice by touching on-screen.


Casi a casi advantages l.jpg
CASI/A-CASI Advantages (CASI/CAPI)

  • Privacy

  • Branching in complex questionnaires

  • Automated consistency checks

  • Automated range checks

  • Automated adaptation of question wording based upon prior responses

  • Audio-CASI allows for standardized verbal delivery of all questions and does not require respondent literacy


Computer assisted telephone interview cati l.jpg
Computer-Assisted Telephone Interview (CATI) (CASI/CAPI)

  • List samples of “live” residential phone lines are called to screen households and determine eligibility of members.

  • Once consent is obtained, interviewer uses a computer to guide them through asking survey questions, providing prompts, entering data, etc. as in A-CASI.

  • Interviewer also has some flexibility in dealing with respondent comprehension/literacy level.


Cati advantages l.jpg
CATI Advantages (CASI/CAPI)

  • Opportunity to access representative sample (costly with eligibility requirements that are less common).

  • Some advantages of live interviewer, but also anonymity of phone call.

  • Has capacities of computer-assisted technology with respect to branching, adaptation of wording, consistency/range checks, etc.


Telephone audio computer assisted self interview t acasi l.jpg
Telephone Audio Computer-Assisted Self-Interview (T-ACASI) (CASI/CAPI)

  • First a telephone interviewer contacts a household, screens for an eligible respondent, and secures consent.

  • Phone call then transferred to automated system where computer-controlled, pre-recorded questions are read aloud.

  • Respondent provides answers by pressing keys on a touch-tone phone.

  • At end, respondent can be returned to live person to close out interview.


Internet surveys l.jpg
Internet Surveys (CASI/CAPI)

  • Access to large sample at relatively low cost in relatively short time frame.

  • May provide access to difficult-to-reach populations.

  • As with other computer-assisted methods, may result in higher rates of reporting of sensitive information, and provides consistency and validity checks.

  • Rates of internet access are growing (if still disparities exist in different populations).

  • Participation rates of target population hard to gauge

  • Dropout rates

  • Direct data entry into database facilitates cleaning and analysis.


Summary survey delivery considerations l.jpg
Summary: Survey Delivery Considerations (CASI/CAPI)

  • Mode of survey delivery clearly influence:

    • Respondent comfort and likelihood of self-disclosure (due to level of privacy)

    • Respondent ease of navigating through the survey if there are any skip patterns

    • Time and effort expended per respondent

    • The audience you can reach with your survey

    • Quality of data, data entry and analysis

  • While important, how you actually ask the questions is just as critical -- our next focus.


Survey construction l.jpg

Survey Construction (CASI/CAPI)


Developing the survey question organization l.jpg
Developing the Survey: Question Organization (CASI/CAPI)

  • Topics should be generally organized from least to most intimate

  • Initial questions are crucial -- respondent must get “hooked” by at least some questions which are felt to be personally salient

  • Initial questions must also require low effort (cognitive demands) to answer

  • Thus, if possible, avoid demographic questions to start (especially income!)


Developing the survey question organization19 l.jpg
Developing the Survey: Question Organization (CASI/CAPI)

  • A survey -- no matter how delivered -- is a type of conversation.

  • Topics should follow an orderly sequence, with common questions grouped together.

  • This flow allows transitions to more intimate matters without jarring or acute awareness.

  • Transitional phrases can help in shifting set.

  • All this also provides respondent with notion that there is a rationale to questions asked.


Question ordering l.jpg
Question Ordering (CASI/CAPI)

  • Be careful with filter questions: if not worded carefully, respondents will learn that a “no” response means avoiding subsequent questions.

  • Be careful about summary attitudinal questions: more specific queries to begin will influence summary responses by forcing respondent to think about their overall beliefs more carefully.


Question ordering21 l.jpg
Question Ordering (CASI/CAPI)

  • Having similar response choices reduces cognitive burden on respondent; grouping items with such options together is thus useful. It can also be useful to have the same number of options to reduce complexity (e.g., always using a four-point or five-point scale).

  • However, arrange types of questions to provide some variety and reduce response set.

  • Avoid asking unnecessary questions!!



Comprehension interpretation l.jpg
Comprehension/Interpretation (CASI/CAPI)

  • Not just question of literacy, but influenced by cognitive burden and attention required

  • Wording choices

  • Length and complexity of survey items

  • Ordering of clauses

  • Consistency of phrasing

  • Avoid “not’s” or items whose meaning will be lost by inattention to a single word

  • Phrasing also influences response distribution along a Likert-type agree-disagree scale


Cultural considerations l.jpg
Cultural Considerations (CASI/CAPI)

  • Individuals can come up with very different understandings of survey items, based upon their personal experiences.

  • The meanings that respondents bring to the survey experience can be strongly influenced by culture, acculturation and other background variables -- not just education and economic status.

  • These differences are highlighted in the vast literature that exists on different response patterns of different ethnic groups to the same standardized scales. (For a review of some issues, see Pasick, Stewart, Bird & D’Onofrio, 2001.)


Key considerations l.jpg
Key Considerations (CASI/CAPI)

  • Examine all survey items with eye for their appropriateness for target population.

  • Be aware of pre-existing measures so as to not have to reinvent the wheel.

  • When using pre-existing scales, review in same way for clarity & ease of response.

  • Get feedback on item wording and construction from as many colleagues as possible.

  • Pre-test survey, using methods first described by Cannell et al. (1989).

Cannell C, Oksenberg I, Kalton G, Bischoping K & Fowler F. (1989). New Techniques in Pretesting

Survey Questions (NCHSR #HS 05616). Survey Research Center, University of Michigan.


Cognitive burden recall demand l.jpg
Cognitive Burden & Recall Demand (CASI/CAPI)

  • Recall accuracy is influenced by a variety of factors, including:

    • salience/vividness of behavior,

    • time frame,

    • complexity of response information necessary (ever, counts, close-ended categories, etc.),

    • frequency and consistency of behavior,

    • effort respondent is willing to exert,

    • cueing techniques and prompts


Time frame sexual drug use behaviors l.jpg
Time Frame - Sexual/Drug Use Behaviors (CASI/CAPI)

  • 3 month time frame generally considered the outside range of accurate reporting; some suggest limiting to 1 month.

  • Longer time frame may be possible if asking if a behavior ever happened.

  • One important timeframe consideration is dealing with infrequent behaviors -- what do you want to capture with respect to either behavior events or possible behavior change (if longitudinal study)?


Sensitivity l.jpg
Sensitivity (CASI/CAPI)

  • Item sensitivity not consistent across populations (e.g., sexual behavior among MSM vs. general population, drug use behavior among heavy drug abusers/IDUs, discussing sexuality among different ethnic groups).

  • Rationale for sensitive questions helps response rates.

  • Asking if “ever” occurred first, prior to asking about (for example) the last month, may be less charged for respondent.

  • Importance of neutral wording of question and (if IAQ) careful training of interviewer


In closing l.jpg
In Closing ... (CASI/CAPI)

  • Selecting your methods of data collection must be carefully considered based upon target population, study questions and design, and costs.

  • No matter what the method of data collection, your instrument is critical.

  • Survey design is a combination of art and science, with the “art” strongly reliant upon interviewing skills.

  • The science of survey design currently has a wealth of research literature to examine and guide you.


ad