COMM 301: Empirical Research in Communication

1. COMM 301:Empirical Research in Communication Lecture 9 (continuation of lect8) Kwan M Lee

2. Announcement • Survey question critique exercise (lect9_2) - important

3. Overall questionnaire design • Different parts in the questionnaire • Introduction • Instructions • Body

4. Introduction • Introduction • brief, neutral introduction • purpose of study • study’s legitimacy, through sponsoring organization • inform participants of their rights voluntary right of withdrawal anonymity, confidentiality • inform participants of the time required

5. Introduction example • Thank you for taking the time to complete this questionnaire about video games. This questionnaire is part of a series of projects being conducted at the Annenberg School for Communication focusing on why people play videogames, and the role videogames play in people’s lives. Instructions for completing the questionnaire appear through the survey. Please answer each item and don’t skip any unless directed to do so. If you have questions or comments, or if you wish to receive a copy of the results of this project, please contact “Name”, name@usc.edu, phone #.

6. Instructions • Instructions • complete and concise set of instructions • clear indications about the specific form of responses • practical issue: use a different style and font for instructions

7. Instruction example • In the next section we want to explore why you play videogames. Following each statement, please circle the number that matches your feelings—Do you “Strongly Agree,” “Agree,” are “neutral” or have no opinion, “Disagree, or “Strongly Disagree?” with the opinion expressed in each statement? (1 =Strongly Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, 5 = Strongly Agree) Strongly Agree Strongly Disagree Playing video games makes me excited. 1 2 3 4 5

8. Body • Body • contains the set of questions • organization is crucial • start with easy to answer, non-intrusive questions • place sensitive questions near the end • be aware of and avoid a possible response pattern (e.g. A B C D A B C D) that will bias responses • consider possible test sensitization and other interaction among questions • e.g. earlier question on media preference affects later question on advertisement recall

9. Body (continued) • format - general order of the questions • funnel format • begins with broad questions, moving to more defined and specific questions • assists respondent in recall of detailed information • most common • inverted funnel format • begins with specific, detailed questions, moving to more general questions • initial questions set a frame of reference for subsequent questions • Questions about course materials, instructor, and other specifics  general evaluation of the course • Tunnel • Equal specificity and detail

10. Body (continued) • grouping of questions • by topic and context • by response type • Professional-looking instrument • Assist respondent recall; reduce cognitive load; increase pleasant appearance

11. Overall questionnaire design • Practice issues • length • eye appeal through good physical formatting • space and fonts • reduce response error • response format • check typo, grammar errors

12. Pilot testing • Ready to go (almost) • pilot testing • Mini-study designed to test the questionnaire, find out about potential problems • sample of about 20 from population of interest • administer questionnaire • then, when questionnaire is completed, go through with respondents the entire questionnaire

13. Administration techniques • Advantages and Disadvantages of each survey technique • Table 9.1 (p. 114) Mail survey technique • Table 9.2 (p. 115) Telephone survey technique • Table 9.3 (p. 116) Face-to-face interview technique • Table 9.4 (p. 117) Computer-assisted data collection technique

14. Mail Survey • Advantage • Access to large geographic area • Reasonable cost • Anonymity • Considered response • Disadvantage • Low response rate • Slow returns • Clarification impossible • Unknown respondents

15. Telephone Survey • Advantages • Access to large geographic data • Easy probability sampling with random digit calling • Fast data collection • Moderate response rate • Disadvantages • Staffing costs • Refusal – telemarketing • Possible systematic biases in sample representative ness – morning vs. evening calls • No visual aids

16. F-t-F Interview • Advantages • Access to observational data • High response rate • Clarification possible • Disadvantages • Staffing costs • Possible biases due to researcher presence • Many unexpected liabilities

17. Response rate • percentage of respondents solicited for participation who complete the instrument • respondents finished / respondents contacted • need care in computing the pool of solicited participants • E.g. how many respondents were actually contacted? • Disconnected numbers; business numbers; fax; no answer all the time; etc. • acceptable response rate relative • F-t-F: 60- 80%; mail: 40-60%

18. Response rate (cont.) • very important to check for possible response bias patterns • e.g. are your respondents mostly of one type • key: questions about representativeness • ways to improve response rates, especially for mail survey • personalization • return postage paid • rewards • follow-up

19. Planning the Survey • Development of a research problem • Access to the community of scholars (lit review) • Definition of constructs (conceptual and operational) • Development of a survey instrument • Sampling • Collection of data • Analysis • Integration of the findings (writing up!)