Public Health Information Network (PHIN) Series II

1. Public Health Information Network (PHIN) Series II Outbreak Investigation Methods: From Mystery to Mastery

3. Series IISession IV “Designing Questionnaires”

4. Series II Sessions

5. Access Series Files Onlinehttp://www.vdh.virginia.gov/EPR/Training.asp • Session slides • Session activities (when applicable) • Session evaluation forms • Speaker biographies Alternate Web site:http://www.sph.unc.edu/nccphp/phtin/index.htm

6. Site Sign-in Sheet Please submit your site sign-in sheet and session evaluation forms to: Suzi Silverstein Director, Education and Training Emergency Preparedness & Response Programs FAX: (804) 225 - 3888

7. Session Overview • The importance of questionnaire design • Hypothesis generating versus hypothesis testing questionnaires • Preparing for questionnaire design • Question design • Question type • Questionnaire format

8. Today’s Presenters Sarah Pfau, MPH Consultant NC Center for Public Health Preparedness Aaron Wendelboe, MSPH Doctoral Candidate and Graduate Research Assistant, NC Center for Public Health Preparedness

9. Learning Objectives Upon completion of this session, you will: • Understand the role of questionnaire design in an outbreak investigation • Know how to develop a hypothesis generating questionnaire • Recognize key planning strategies for successful questionnaire design

10. Learning Objectives • Recognize key characteristics of well-designed questions • Recognize three broad question types and when to use them • Understand what different question types measure, and the type of data (quantitative versus qualitative) they yield

11. Learning Objectives • Know how to format questionnaires for interviewer administered and self-administered settings

12. Session IV “Designing Questionnaires”

13. Basic Steps of an Outbreak Investigation • Verify the diagnosis and confirm the outbreak • Define a case and conduct case finding • Tabulate and orient data: time, place, person • Take immediate control measures • Formulate and test hypothesis • Plan and execute additional studies • Implement and evaluate control measures • Communicate findings

14. Why is Questionnaire Design Important? “The quality of the data will be no better than the most error-prone feature of the survey design.” - Fowler, F.J. (1993). Survey Research Methods: Second Edition. Sage Publications: Newbury Park.

15. Why is Questionnaire Design Important? With an understanding of good questionnaire design principles, you will ask only aboutwhat you need to meet your research objectives.

16. Ask Only About What You Need. . . Example: You ask respondents to list all vaccinations that their school-aged child has had (difficult in terms of respondent recall / accuracy), when you really only need to know if the child is current on all DTP boosters.

17. Why is Questionnaire Design Important? Survey answers are not of interest intrinsically; rather, the answers are important because of their relationship to what they are supposed to help you measure.

18. Why is Questionnaire Design Important? Question type and response option formatting impact: 1. How you can design an on-screen data entry form and / or analyze variables in your software program 2. How respondents interpret and respond to the questions

19. Hypothesis Generating Questionnaires

20. Which Questionnaire Type Should You Use? • Hypothesis Generating? • Hypothesis Testing?

21. Hypothesis Generating Questionnaire Person, Place, Time Questions: • Demographics • Clinical details of the illness • Health care provider visits • Water exposure • Exposure to other ill persons • Exposure to children in day care • Exposure to a farm or farm animals • Travel outside of the immediate area

22. Hypothesis Generating Questionnaire If the pathogen can be spread through food or beverages, include questions about: • Food eaten in the home • Food eaten in the homes of friends, family • Food eaten at any restaurant • Dates and times of food consumption and any suspicious observations

23. Hypothesis Generating Questionnaire

24. Hypothesis Generating Questionnaire

25. Hypothesis Testing Questionnaires

26. Hypothesis Testing Questionnaire Include detailed questions about the suspected source of infection. Example: The local bakery is suspected as the source of a Hepatitis outbreak in multiple counties. The hypothesis testing questionnaire is used with both cases and non-cases, and includes only a food history for all possible items on the bakery menu to pinpoint the exact food item that is contaminated.

27. Hypothesis Testing Questionnaire

28. Hypothesis Generating Account for consumption of ground beef, lettuce, alfalfa sprouts, un-pasteurized milk or juice, and swimming in or drinking sewage-contaminated water Hypothesis Testing [beef suspected]: Beef brand, date of purchase, and grocery store or restaurant where beef was purchased or eaten Hypothesis Generating versus Testing Questions: E. coli 0157:H7

29. Preparing for Questionnaire Design

30. Preparing for Questionnaire Design There is more to questionnaire design than writing questions. Ideally, you should first: • Have a clear purpose and research objectives • List variables to be measured • Have an analysis plan • Consider cost and other logistical aspects

31. Questionnaire Design • Have a clear purpose and research objectives. Is the purpose of your outbreak investigation survey to generate a hypothesis or to test a refined hypothesis? If testing a hypothesis, who is included in your study sample?

32. Questionnaire Design • List variables to be measured Whether you are generating or testing a hypothesis, determine your variables of interest before you develop questions. You will avoid asking unnecessary questions or asking for unnecessary details.

33. Questionnaire Design • Go to: http://www.cdc.gov • Under “Health & Safety Topics” in the left margin, click on, “Diseases & Conditions” • Select a link to either an alphabetized list of all diseases or diseases by topic [e.g., if you already know that a pathogen is water-borne versus food-borne]

34. Questionnaire Design

35. Questionnaire Design • Have an analysis plan. • Guides the question types and response option categories used on the questionnaire • Helps assure that the data collection leading up to analysis yields variable coding that your analysis software program can use efficiently.

36. Questionnaire Design Analysis Plan • Hypothesis • Variables involved • Response option coding • Continuous versus categorical values

37. Questionnaire Design • Consider cost and other logistical aspects • What is the survey sample size? • What is the geographic distribution of the survey sample? • Will questionnaires be interviewer administered or self-administered? • What is your staff capacity to work within the parameters of a – c above?

38. Question & Answer Opportunity

39. Question Design

40. Elements of Good Question Design • Reliability • Validity • Specificity versus ambiguity • Simplicity • Only one question asked • Mutually exclusive answer choices • Refers respondents to specific dates / times for recall • When feasible, make sure data can be compared to existing sources of information

41. Question Design 1. A question that is designed to be reliable will assure that the words are interpreted the same way in any setting, and that respondents answer the same way in any setting.

42. Reliable Question Design Question: “Are you experiencing diarrhea?” Interviewer then adds: “For the purposes of this survey, we consider diarrhea to be 3 or more loose bowel movements in a 24 hour period.”

43. Question Design 2. A question that is designed to be valid will always yield information that can be used as a true measure of what you, the researcher, are looking for.

44. Valid Question Design Less Useful Which is your source of drinking water at home? • Tap water • Bottled water Better Which is your source of drinking water at home? • Municipal tap water • Municipal tap water with additional filtration • Well water • Commercially bottled water

45. Question Design 3. Avoid ambiguity in question wording. Less useful “When did you have ‘Disease X’?” Better “How old were you when you had ‘Disease X’?”

46. Question Design 3. Avoid ambiguity in question wording. Less useful “Have you been examined by a physician in the past seven days?” Better “Have you been examined by a physician for these symptoms in the past seven days?”

47. Question Design 4. Use simple language and keep questions short. Less useful Were you exposed to the fomite at the dinner party? Better Did you use a shared hand towel at the dinner party?

48. Question Design 5. Ask only one question. Two questions in one Did you eat mashed potatoes and giblet gravy? One question at a time Did you eat mashed potatoes? Yes No If Yes, did you eat them: a. Plain b. With giblet gravy

49. Question Design 6. For closed-ended questions, make sure that response options are mutually exclusive. Not useful What is your age? • < 18 years • >18 years Useful What is your age? • 17 years old or younger • 18 years old or older

50. Question Design 7. Use specific date / time references to improve respondent recall. Less useful Have you been swimming in a public pool recently? Better Did you swim in a public pool any time between Friday, July 1st and Monday, July 4th?