Using Questionnaires Part II

1. Using Questionnaires Part II Dr AyazAfsar

2. Objectives • Ethical issues • Approaching the planning of a questionnaire • Types of questionnaire items • Asking sensitive questions • Avoiding pitfalls in question writing • Sequencing the questions • Questionnaires containing few verbal items • The layout of the questionnaire • Covering letters or sheets and follow-up letters • Piloting the questionnaire • Practical considerations in questionnaire design • Administering questionnaire • Processing questionnaire data

3. Types of Questionnaire Items • ratio data questions • open-ended questions • matrix questions • contingency questions, filters and branches

4. Types of questionnaire items • ratio data questions • open-ended questions • matrix questions • contingency questions, filters and branches

5. Ratio data questions Ratio data questions deal with continuous variables where there is a true zero For example • How much money do you have in the bank?–– • How many times have you been late for school? –– • How many marks did you score in the mathematics test?–– • How old are you (in years)?–– Here no fixed answer to category is provided and the respondent puts in the numerical answer that fits his/her figure

7. Open-ended questions The open ended question is a very attractive option for smaller scale research or for those sections of a questionnaire that invite an honest, personal comment from respondents in addition to ticking numbers and boxes For example: • Please indicate the most important factors that reduce staff participation in decision-making. • Please comment on the strengths and weaknesses of the English course. • Please indicate areas for improvement in the teaching of foreign languages in the school.

8. More examples • A sentence – completion item is a useful adjunct to an open ended question For example: • Please complete the following sentence in your own words: • An effective teacher . . . or • The main things that I find annoying with disruptive students are . . ..

9. Matrix questions Matrix questions are not types of questions but concern the layout of questions. Matrix questions enable the same kind of response to be given to several questions, for example ‘strongly disagree’ to ‘strongly agree’. The matrix layout helps to save space, for example:

10. Example Please complete the following by placing a tick in one space only, as follows: 1 = not at all; 2 = very little; 3 = a moderate amount; 4 = quite a lot; 5 = a very great deal How much do you use the following for assessment purposes?

11. Contingency questions, filters and branches • Contingency questions depend on responses to earlier questions, for example: ‘if your answer to question (1) was ‘‘yes’’ please go to question (4)’. • The earlier question acts as a filter for the later question, and the later question is contingent on the earlier, and is a branch of the earlier question. • Some questionnaires will write in words the number of the question to which to go (e.g.‘please go to question 6’); others will place an arrow to indicate the next question to be answered if your answer to the first question was such-and-such.

12. It is advised that there should be limited use of filtering and branching devices. • It is particularly important to avoid having participants turning pages forwards and backwards in a questionnaire in order to follow the sequence of questions that have had filters and branches following from them. • It is a particular problem in Internet surveys where the screen size is much smaller than the length of a printed page. • One way of overcoming the problem of branches is to sectionalize the questionnaire, keeping together conceptually close items and keeping the branches within that section.

13. Asking sensitive questions Researchers indentify several potentially threatening or sensitive issues and suggest the following strategies to avoid them Open rather than closed questions might be more suitable to elicit information about socially undesirable behaviour, particularly frequencies. • Long rather than short questions might be more suitable for eliciting information about socially undesirable behaviour, particularly frequencies. • Using familiar words might increase the number of reported frequencies of socially undesirable behaviour. • Using data gathered from informants, where possible, can enhance the likelihood of obtaining reports of threatening behaviour. • Deliberately loading the question so that overstatements of socially desirable behaviour and understatements of socially undesirable behaviour are reduced might be a useful means of eliciting information.

14. With regard to socially undesirable behaviour, it might be advisable first to ask whether the respondent has engaged in that behaviour previously, and then move to asking about his or her current behaviour. By contrast, when asking about socially acceptable behaviour the reverse might be true, i.e. asking about current behaviour before asking about everyday behaviour. • In order to defuse threat, it might be useful to locate the sensitive topic within a discussion of other more or less sensitive matters, in order to suggest to respondents that this issue might not be too important.

15. Avoiding pitfalls in question writing • Although there are several kinds of questions that can be used, there are caveats about the framing of questions in a questionnaire • Avoid leading questions, that is, questions that are worded (or their response categories presented) in such a way as to suggest to respondents that there is only one acceptable answer, and that other responses might or might not gain approval or disapproval respectively. • For example: • Do you prefer abstract, academic-type courses, or down-to-earth, practical courses that have some pay-off in your day-to-day teaching?

16. Avoid high brow questions even with sophisticated respondents. For example: • What particular aspects of the current positivistic/interpretive debate would you like to see reflected in a course of developmental psychology aimed at a teacher audience?

17. Avoid complex questions. For example: • Would you prefer a short, non-award-bearing course (3, 4 or 5 sessions) with part-day release (e.g. Wednesday afternoons) and one evening per week attendance with financial reimbursement for travel, or a longer, non-award-bearing course (6, 7 or 8 sessions) with full-day release, or the whole course designed on part-day release without evening attendance?

18. Avoid irritating questions or instructions. For example: • Have you ever attended an in-service course of any kind during your entire teaching career? • If you are over forty, and have never attended an in-service course, put one tick in the box marked NEVER and another in the box marked OLD.

19. Avoid questions that use negatives and double negatives. For example: • How strongly do you feel that no teacher should enrol on the in-service, award-bearing course who has not completed at least two years’ full-time teaching? OR • Do you feel that without a parent/teacher association teachers are unable to express their views to parents clearly?

20. Avoid too many open-ended questions on self-completion questionnaires. • Avoid extremes in rating scales, e.g. ‘never’,‘always’, ‘totally’, ‘not at all’ • Avoid pressuring/biasing by association, for example: • ‘Do you agree with your headteacher that boys are more troublesome than girls?’. • Avoid statements with which people tend either to disagree or agree (i.e. that have built-in skewedness (the ‘base-rate’ problem, in which natural biases in the population affect the sample results). • Finally, avoid ambiguous questions or questions that could be interpreted differently from the way that is intended. The problem of ambiguity in words is intractable; at best it can be minimized rather than eliminated. Take the following examples: • Does your child regularly do homework? • What does ‘regularly’ mean – once a day; once a year; once a term; once a week?

21. How many students are there in the college? • What does this mean: on roll; on roll but absent; marked as present but out of college on a field trip; at this precise moment or this week (there being a difference in attendance between a Monday and a Friday), or between the first term of an academic year and the last term of the academic year for college students as some of them will have left college to go into employment and others will be at home revising for examinations or have completed them?

22. Task 1 Think of any one of the courses you are currently taking at COMSATS, and try to answer the questions below about that course. What faults can you find in these questions? 1. Do you find your classes useful and interesting? • Yes { } • No { }  2. Do you talk a lot in these classes? • A lot { } • A little { } • Not much { } 3. Does the lecturer give enough opportunity for every student to express his/her own views in class? • Yes { } • No { }

23. Cont. 4. Do you prefer a friendly student-centred approach to a rigid teacher-dominated approach? • Yes { } • No { } 5. What qualifications does your tutor have? • Diploma { } • BA/B.Ed { } • MA/M.Ed { } • PhD { }  6. If a student’s utterance is linguistically deviant and peer-correction does not occur, does the tutor usually give a non-metalinguistic response? • Yes { } • No { }

24. 7. Which of the following causes you difficulty in the course? • Understanding lectures { } • Reading textbooks { } • Writing essays { } 8. How much individual consultation with the tutor would you like to have for each assignment? • _________ hrs 9. What is your opinion of the course?

25. Questionnaire - Advantages More cost effective . Geographical coverage can be wider and numbers participating larger. Anonymity can be maintained and thus confidentiality assured (except where you ask their name). Quantitative and qualitative data can be collected

26. Questionnaire - Disadvantages Loss of control – validity. No follow through - due to anonymity. Design flaws and part completion. Low response rates.

27. Purposes of piloting • To check clarity of items/layout/sections/presentation/ instructions • To gain feedback on appearance • To eliminate ambiguities/uncertainty/poor wording • To check readability • To gain feedback on question type (suitability/feasibility/ format (e.g. open/closed/multiple choice) • To gain feedback on appropriateness of question stems • To generate categories for responses in multiple choices • To generate items for further exploration/discussion • To gain feedback on response categories • To gain feedback on length/timing (when to conduct the data collection as well as how long each takes to complete (e.g. each interview/questionnaire)/coverage/ease of completion

28. Cont. • To identify redundant items/questions (those with little discriminability) • To identify irrelevant questions • To identify non‑responses • To identify how motivating/non‑engaging/threatening/ intrusive/offensive items may be • To identify sensitive topics and problems in conducting interviews • To gain feedback on leading questions • To identify items which are too easy/difficult/complex/ remote from experience • To identify commonly misunderstood or non‑completed items.

29. Further Reading Book • Research Methods in Education by Louis Cohen, Lawrence Manion and Keith Morrison, 7th ed. Useful Web Sites • http://www.leeds.ac.uk/iss/documentation/top/top2/top2-5.html • http://www.cc.gatech.edu/classes/cs6751_97_winter/Topics/quest-design/ • www.questionpro.com • Many more can be found through search engines such as ‘Google’. • Note – questionnaire templates on the web may not be as useful a tool as you anticipate.

30. The End