Download
1 / 39
420 likes | 655 Views
Effective Questionnaire Design. Craig Parylo & Annette Lee. What is a questionnaire?. A series of questions. Gathers information from lots of people. Focussed around a single topic or area of interest. An effective questionnaire…. Asks relevant questions
E N D
Effective Questionnaire Design Craig Parylo & Annette Lee
What is a questionnaire? A series of questions. Gathers information from lots of people. Focussed around a single topic or area of interest.
An effective questionnaire… Asks relevant questions • Questions that will provide the answers to the research topic in question Gets valid responses • Responses given reflect the respondent’s actual opinions Gets a representative response • Makes it more reliable to infer the findings to wider population Survey report
Introduction Anyone can design a questionnaire. An effective questionnaire requires planning and an appreciation of the psychology of respondents: Filling in a questionnaire is a complex process. • The job of a questionnaire designer is to make it easy to respond.
Complex cognitive process 1. Identify subject 4. Decodes 5. Collects info 6. Share? 2. Phrase as question 3. Reads • Do you drink coffee? • Yes • No 7. Encode 8. Option I prefer tea 9. Respond No
Exercise: responding to questions • What is the colour of your car? • What do you usually have for breakfast? • What did you have for breakfast last Friday? • Have you ever not left a tip at a restaurant? • What is your sexuality? • What is the biggest regret in your life?
Photo by Ronit Geller (http://www.sxc.hu/profile/Rogel) Choosing the right approach
Types of questionnaire No method consistently outperforms others
Considerations Your approach should take consideration of: • Your topic • Who you want survey • The type of information you want • Your budget • How quickly you need the information
Sample size vs Depth of info • Better response rates from interviewer-led surveys than from self-response surveys. • Self-response surveys are usually cheaper. • Interviewer-led surveys can produce richer information. • Self-response surveys can reach more people. Don’t confuse quantity for quality.
Children Is it pitched at the right age group?
Timing and convenience • Will it be completed right after the event, or sometime after? • Will it be completed in front of other people or in private? • Many people are poor at estimating numerical info about their lives. • The more common the event the shorter a person’s window of recall is.
Question types • Open / closed • Single / multiple response • Attitudinal / opinion questions • Filtering / routing
Attitude/opinion • Can be difficult to get the phrasing of the question right so it means the same thing to all people. • Can be tempting to put lots of explanation into the question – but sometimes this just confuses things further. • Use clear, simple English/ language and avoid being too general. • Try not to ask ‘Iceberg’ questions.
Mid-point / neutral opinions Use mid-points in questions on attitude. • Historically controversial as it was thought that respondents chose a neutral option to avoid thinking about a question. • Now shown that including a mid-point increases reliability and validity Strongly disagree Strongly agree
‘How important?’ Following your visit to the clinic today, please rate the following – first based on your experience and second how important this is to you?
Common mistakes • What was the first drink you had today? • Tea • Coffee • Water • Juice • Haven’t had a drink today • Other (please state) What might be wrong with this question? Need to cover all options
Common mistakes What might be wrong with this question? What is your age group? Options should be mutually exclusive.
Common mistakes What might be wrong with this question? What did you think about the waiting time? Options should be balanced ‘Good’ responses ‘Bad’ response
Common mistakes What might be wrong with this question? How would you rate the appointment booking service using the: Use opt-out responses appropriately
Question wording • Keep questions as short as possible • Max 16 words. • Consider providing medium length (30 word) introductions to groups of related questions (context is vital!) • Use open-ended questions sparingly
Double questions Is space and storage adequate? ☐ Yes ☐ No Was the consent of the patient obtained and recorded? ☐ Yes ☐ No Was the consent of the patient: ☐ Obtained? ☐ Recorded? If a question can be broken down into parts then it should be?
Double negatives Do you think that it is an unwise policy to continue sending food aid to Africa. ☐ Yes ☐No • If the respondent has to tick ‘No’ to give a positive response then this can be considered a double negative. • Try and phrase questions with a positive slant so that the positive response is ‘Yes’ rather than ‘No’.
Design • Use booklet format with double-sided printing • A4 folded to A5, or, A3 folded to A4 • Front cover: • The title of the questionnaire • Identity of the organisation carrying it out • Clearly explain the purpose of the questionnaire • Keep graphics neutral
Aesthetics • Arial font, size 12 point or larger • Questionnaire length • Shorter is generally better • Also keep plenty of ‘whitespace’ • Be consistent in the use of colour, shapes and location in order to guide the respondent through the questionnaire
Other elements to remember • Make it look professional. • Give an estimated completion time. • Put the deadline for return at the beginning and end of the questionnaire. • Provide a return address – on the questionnaire. • Say ‘Thank you’.
Ordering your questionnaire • Place easy, non-challenging questions first • Key filter questions should go in early. • Use a funnel structure • Go from basic to more in depth. • General questions before specific questions. • Go from the least sensitive to the most sensitive questions. • But – try and put important questions in before the end.
Ordering your questions • Ask for suggestions for improvement at the end. • Check for leading questions. • Add prompts where required. • Order of questions can impact heavily on the amount of time it takes for the respondent to complete the questionnaire.
Filtering • Filtering (or routing) helps identify which respondents should answer a detailed set of questions about a certain aspect of the topic. • Can be difficult to achieve – should be VERY CLEARLY signposted
Filtering Example ✓ ✓ ✓ ✓ ✓
Photos by Sarah Williams & DavideGuglielmo (http://www.sxc.hu/profile/MeHere) (http://www.sxc.hu/profile/brokenarts) Tips for increasing response rate
Improving the response rate (1) • Shorter questionnaires • Good communication • Contact the respondents beforehand • Follow-up contact with non-responders • Monetary incentives • Personal touch • Handwritten address • Including respondent’s name in cover letter
Improving the response rate (2) • Method of delivery • Include a stamped return envelope • Send by first class or recorded delivery • University sponsorship can add credibility • Assure confidentiality
Pilot your tool • This is the MOST IMPORTANT part. • If a range of respondents from different backgrounds will be using the questionnaire make sure you use as many of these different groups as possible in the trial. • Go back to your objectives – does the data help you fulfil your objectives and measure against the standards you set?
