Questionnaire Design

1. Questionnaire Design EPIET Introductory Course 2003 VERYIER DU LAC, FRANCE P McKeown

2. So far……… • Introduction • Outbreak or Incident • Salmonella in Northern Ireland • Steps in investigation • Surveillance  Questionnaire Design Disease Detectives  Looking for clues  Building a case

3. Objectives To understand: • Why we use questionnaires • The objectives of a questionnaire • The advantages and disadvantages of questionnaires • The different types of questions that can be used and the circumstances in which they would be used • The way in which questionnaires can be designed and how to avoid some common problems and pitfalls

4. What is a questionnaire? • An instrument (form) to collect answers to questions • Collects factual data – gathers information or measures • A series of written questions / items in a fixed, rational order • Every item is answered (unless instructed otherwise) • Who completes it? • The respondent – public/professional • Someone on behalf of the respondent • The interviewer

5. Types of Questionnaire • Prospective or retrospective • Completed by participant or researcher • Advantages of self-administered include: • Cheap and easy to administer • Preserve confidentiality • Can be completed at respondent's convenience • Can be administered in a standard manner

6. Types of Questionnaire • Advantages of interview-administered include: • participation by illiterate people • clarification of ambiguity • They can be administered face to face, via the telephone, by post or email

7. What makes an effective questionnaire? An effective questionnaire is highly structured to allow the same types of information to be simplycollected from a large number of people in the same wayso that the data can be analysed quantitatively and systematically

8. And why do we want one? Awell designed questionnaire: • Will give you appropriate data which should allow you to answer your research question • Will minimise potential sources of bias, thus increasing the validity of the questionnaire • People are much more likely to complete a well designed questionnaire

9. Assumptions The use of questionnaires is based on certain assumptions: • That individuals will have had common understanding of particular issues and will share general beliefs • (went to a wedding/ that we are trying to find out more about meningitis because this disease is a threat to our children and young people) • That this understanding and these beliefs are shared and can be clearly expressed and quantified

10. Before starting….. • Questionnaire design should be an integral part of the study, not an “add-on” • Therefore: • Decide on the goals of the study (for yourself and for others) • Know the subject – literature, experts • Know the respondents - (SEG, occupation, special sensitivities, educational, ethnic) • Know how much – cost, time, sample size, response rate

11. Fundamental Principle An epidemiological investigation is a scientific loop that tests a hypothesis: Problem Question Hypothesis Answer Study Results Questionnaire

12. Objectives of a questionnaire • To maximise the proportion of subjects answering the questionnaire –as high a response rate as possible • To ensure that the information obtained is asaccurateand as relevant to our study as possible = as simple and as focussed as possible

13. Advantages of questionnaires • Can reach a large number of people relatively easily and economically (especially postal questionnaires) • Provide quantifiable answers • Relativelyeasy to analyse

14. Disadvantages of questionnaires • May provide only limited insight into the problem: • Limited response allowed by questions • You may not have asked the right questions eg Andorra, norovirus and ice-cubes • Varying response • they may not understand your question and may intrepret it differently • Postal questionnaires  low response rate • Hard to chase after missing data – need to get it right first time

15. Stages in designing a questionnaire • Decide on aim of the study and the purpose of using a questionnaire • Decide on content of the questions • Decide on the format of the questions • Decide on presentation and layout • Formulate coding schedule (if appropriate) • Pilot and refine questionnaire

16. Basic Rules • KISSkeep it short and simple • Appearance is crucial and affects: • Response rate • Ease of data summarisation and analysis • Length of questionnaire: shorter  response rates • Question order is important: • Easy  difficult • General  particular • Factual  abstract • Start with closed format questions. • Start with questions relevant to the main subject. • Try to avoid starting with demographic and personal questions

17. Basic Rules • Number all items and pages • Put an identifying mark on all pages • Put the return address on the questionnaire • Put the study title in bold on the first page • Print directions in bold • Remember the Unique Identifier

18. Basic Rules • Group questions by topic or response options • Don’t put the most important item last • Open with relevant non-threatening questions • It is most likely to be completed if easy on the eye, relevant, logical and as short as possible.

19. Content of Questions • Keep a clear focus on your research question • Literature search; what do we need to know in order to ask the most relevant and probing questions? • Demographic information • Contact information (if non-anonymised)

20. Format of Questions • Ask only for the information you need • Start with simple questions – get them on your side • Appropriatefor responding audience– eg middle class wedding guests vs prisoners • Keep sentences simple and short

21. Format of Questions • Ask for one piece of information at a time: • If you had not passed water for 6 hours but were watching a James Bond movie, and your wife brought you a sandwich would you get up to go to the toilet in the next 20 minutes? Y/N • Remember “don’t know”

22. Format of Questions • Be precise– Do you often have to get up at night to pass water? Y/N • Be appropriate/sensitive – bowel habits • Aim always to minimise bias

23. Bias in Questions • Information Bias = systematic differences in the measurement of a response • Recall bias – eg cases being more likely to remember than controls • Observer bias – inter and intra – a questionnaire is an important way to reduce this • Non-response bias • Those who respond are different from those who do not • Reduce by ensuring high response rate or • Correcting for during analysis (eg age, sex, SEG)

24. Format of questions Two main question formats • Closed format  give a ‘fixed’ response • Open format  allow people to express their views in their own words: • What is your most distressing symptom? Please describe: ________________________________________________________________________________________________

25. Open or Closed? Closed – forced choice • Simple and quick • Reduces discrimination against those who are less literate • Easy to code, record, and analyse results quantitatively • Easy to report results Open format– free text • Allows exploration of issues • Can be used even if a comprehensive range of alternative choices cannot be compiled

26. Closed Questions A number of types: • Straightforward response • What is your age in years? _____ yrs • How long have you worked for Company X? _____ • What is your sex (gender)? M  F  • Did you stay in Hotel X on 23/6/02?Y  N  • Did you eat dinner in the company canteen on 23/6/02?Y  N 

27. Closed Questions 2. Checklist • Which of the following did you eat for dinner on 23/6/02? • Chicken  • Beef  • Ham  • Salad  • Egg mayonnaise 

28. Closed Questions 3. Adjectival/adverbial responses • Are you limited by hip pain during any of the following activities? Not at all Slightly Moderately Severely Walking     Sitting     Lying in bed    

29. Closed Questions 4. Numerical rating scale How useful would you think a community methadone clinic in your area would be? (please circle) 1 2 3 4 5 6 7 Not at all useful Very useful

30. Closed Questions 5. Visual analogue scale At rest, how bad is your hip pain? (please mark where on the line below) No pain Worst pain imaginable

31. Closed Questions 6. Ranking scales Please rank these activities in order of the amount of pain they cause you. (1 would be most pain and 4 would be least) Sitting  Running  Lying in bed  Walking 

32. Closed Questions 7. Scales for measuring attitude eg Lickert A local methadone clinic would be useful for my community No, I strongly disagree No, I disagree quite a lot No, I disagree just a little  I’m not sure about this Yes, I agree just a little Yes, I agree quite a lot Yes, I strongly agree

33. Open Questions • Used to provide fuller information • Open, free-text form • Often used in qualitative research, focus groups, at the preliminary and piloting stage of questionnaires What do you think is the reason you became ill? ________________________________________________ Do you think service X would make your life easier? If so, in what way? If not, why not? ________________________________________________________________________________________________

34. Problems and Pitfalls • Avoid questions that ask two things at once - you won’t know which ‘bit’ people are answering: Have you ever had neck pain and diarrhoea? • Ambiguity..... e.g. Do you got to the toilet a lot?

35. Problems and Pitfalls • Avoid jargon • How often do you get up at night to PU? (pass urine) • Should IVDUs be treated in the community? • Avoid options that are not mutually exclusive when they need to be What age are you? 16-20  20-25  25-30  35-40 

36. Problems and Pitfalls • Avoid leading questions Do you think that the food in the hotel made you sick? Did the hotel staff seem unhygenic to you? Do you agree that the hospital staff were close to exhaustion? • Avoid making the questionnaire too long • Typographical / spelling errors • Questionaire • Diahurria • Vommit • Eppidemiology

37. Ensuring a High Response Rate • Make the questionnaire relevant • Ensure respondents see its potential and usefulness (we are trying to find out why you and your family/people in became ill while holidaying in this resort)

38. Ensuring a High Response Rate • Include a brief covering letter • Who you are • Who you work for • Why you are investigating/researching • Where you obtained the respondent’s name • How and where you can be contacted • Absolute guarantee of confidentiality • Self-addressed envelope!!

39. Questionnaire Validation • Many existing questionnaires can be used or adapted eg SF-36 • Validated • Others e.g. for an outbreak have not been validated and will need to be tested • Piloting is a common way of doing this

40. Piloting and Evaluation • Pilot with a similar group of people to your intended subjects • Piloting a essential • Highlights problems before starting • Often concentrates initially on effects of alternative wording • Then will concentrate on respondents and interviewers overall impressions • With a final polishing after several amendments to ensure response rate

41. Presentation and layout • Decide on the order of the items / questions • Where to place sensitive questions? • Make the layout clear, consistent and easy on the eye • Try to get on overall simple “feel” • Ensure there is adequate space to answer • Choose a large enough font size • Avoid flippancy • Avoid fancy logos/printed on recycled paper/is an equal opportunity employer etc • Do not slip to another page

42. Presentation and layout • Using colour or printing questionnaire on coloured paper may help • Use filter questions, if necessary • Give clear instructions about how to answer the questions

43. Coding Schedule • Questionnaire can be pre-coded • Quicker and easier to hands responses prior to data entry eh Epi-Info or SPSS • Examples: • M=1 F=2 DK=3 • Ill=1 Not ill=2 DK=3 • Single=1 Married=2 Separated=3 Divorced=4 DK=5 • No medical intervention=1 Saw GP=2 Went OPD=3 Admitted Hospital=4 Admitted ICU=5 Died=6 DK=7

44. Summary A well designed questionnaire: • Will give you appropriate data which should allow you to answer your research question • Will minimise potential sources of bias, thus increasing the validity of the questionnaire • People are much more likely to complete a well designed questionnaire

45. Question 764 …“If you are an attractive, blonde woman aged 25-50, can I take you to dinner?” FINALLY, keep your questionnaire short and the questions simple, focussed and appropriate