Questionnaire interview item writing part 1
1 / 34

Questionnaire & Interview Item-writing Part 1 - PowerPoint PPT Presentation

  • Uploaded on

Questionnaire & Interview Item-writing Part 1. Dr Desmond Thomas, MA TESOL University of Essex. Useful References. Denscombe, M., 1998, The Good Research Guide for Small-scale Social Research Projects, Open U. Press

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Questionnaire & Interview Item-writing Part 1' - mira-whitehead

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Questionnaire interview item writing part 1

Questionnaire & Interview Item-writing Part 1

Dr Desmond Thomas, MA TESOL

University of Essex

Useful references
Useful References

  • Denscombe, M., 1998, The Good Research Guide for Small-scale Social Research Projects, Open U. Press

  • Foddy, W. 1993, Constructing Questions for Interviews and Questionnaires - Theory and Practice in Social Research, Cambridge: Cambridge University Press

  • Oppenheim, A.N., 1992, Questionnaire Design, Interviewing and Attitude Measurement, London: Pinter

Questionnaire or interview what s the difference
Questionnaire or interview? What’s the difference?

  • Not a great deal, when questionnaire items are read out and responses filled in by the researcher

  • But a world of difference between an anonymous structured survey questionnaire and an unstructured 1-to-1 interview


  • And what kind of interview?

A research questionnaire should
A research questionnaire should …

  • Be designed to collect information which can be used as data for analysis: this can be quantitative, qualitative or a mix

  • Gather information by asking people about issues directly related to a research project (demonstrating validity)

  • Consist of a written list of questions with each respondent reading an identical set and following identical procedures (reliability)

Questionnaires are useful when
Questionnaires are useful when …

  • Used with large numbers of respondents in multiple locations

  • Information required is straightforward

  • Standardized data from identical questions is required

Questionnaire issues sampling
Questionnaire issues: Sampling

  • Is a sample representative of a total population really necessary?

  • Or can the sample in a qualitative survey represent itself?

  • If a representative sample is needed are there ready-made sampling frames?

  • Will sampling be random or in clusters?

  • How important is sampling size?

Quantitative survey issues sampling 1
Quantitative Survey Issues: Sampling 1

  • If the target population is : secondary school teachers of English in Bulgaria

  • How large a sample?

    30 or more people? 5-10%?

  • Problem of self-selection in survey responses. How to solve this?

Quantitative survey issues sampling 2
Quantitative Survey Issues: Sampling 2

  • Sample A: teachers belonging to five different schools (cluster sampling)?

  • Sample B: randomly selected?

  • Sample C: randomly selected within large clusters (eg the five schools)?

  • Sample D: Every 10th teacher on a national register (or other sampling frame)?

Questionnaire issues piloting
Questionnaire issues: Piloting

  • What should be piloted? Everything!

  • Questionnaire layout, length, question types, question wording, order of questions, rubrics all need to be tested

  • It is impossible to get things right at a first attempt; second attempts, in turn, will need to be piloted

Advantages of questionnaires
Advantages of questionnaires

  • Supply a large quantity of data for a relatively low cost: not labour-intensive

  • Standardized pre-coded answers can enable speedy data collection, management and even analysis

  • Eliminate ‘interviewer bias’ – at least to a certain extent

  • Face validity

Questionnaire problem areas
Questionnaire problem areas

  • Low response rates

  • Frustration for the respondent: box-ticking can deter respondents when no answer seems appropriate

  • Frustration for the researcher: no scope for clarification of answers

  • One chance only to ‘get it right’ – especially in terms of item wording

Creating web based questionnaires
Creating web-based questionnaires

  • Go to:

  • Design and edit your survey

  • Send the link by email to potential respondents

  • Collect and analyse the data

  • Note: limitations of the free vs the paid version of this software

Issues for all types of questionnaire
Issues for all types of questionnaire

  • What makes a good questionnaire item?

  • Why is it so difficult to formulate the precise questions that we need to ask in order to obtain the required answers?

  • Why is it sometimes difficult to interpret the answers that we obtain?

Fundamental problem 1
Fundamental Problem 1

  • “It is almost impossible to ask a question without suggesting answers. The very fact that a question is asked implies that the researcher thinks the topic is of interest. Moreover, the way a question is asked inevitably reflects the researcher’s preconceptions. Unstated presuppositions always underlie a question. “ (Foddy 1993: 53-4)

Fundamental problem 2
Fundamental Problem 2

Q: Which soft drink do you usually buy?

(What is a ‘soft drink’ and what isn’t?

How often is ‘usually’?

What is understood by ‘buy’ and by ‘you’?

What if you buy more than one?)

A: Probably the first brand that comes to mind

The question may not be understood as it is intended and results may therefore be invalid.

(Starting point: what information does it seek to elicit?)

Question questions such as
Question questions such as …

  • How many journeys have you made on London Underground over the past month?

  • Do you enjoy going to coffee bars and restaurants in the evening?

  • Do you regularly do your shopping in large supermarkets?

  • The library facilities at Essex University are adequate for my needs.

    Strongly agree ( ) Agree ( )

    Disagree ( ) Strongly disagree ( )

And these as well
And these as well …

  • Complete the following sentence.

    I enjoy studying at Essex because …

    2. Do you think that the library should do more to improve its facilities for Masters students?

    3. How important is exercise as part of your daily routine?

    Very important ( ) Important ( )

    Not sure ( ) Unimportant ( )

Design principles
Design principles

  • Choose the right instrument!

  • Target information required first → afterwards the right questions to ask. Required information depends on aims.

  • Rigorously monitor all items

  • Pilot and re-pilot the questionnaire

  • Provide clear instructions and standard procedures (for reliability)

Open vs closed questions
Open vs closed questions

OPEN: Answers have more depth but data processing is more complex.

CLOSED: Easy to process but many useful insights are lost and respondents can become irritated by being put ‘in boxes’. (Oppenheim 1992, p.115)

Some pitfalls to avoid
Some pitfalls to avoid

  • Respondents often answer questions when they don’t really know the answer.

  • Respondents can deliberately not answer or answer incorrectly (ie they lie)

  • Respondents misinterpret questions

  • Small changes in wording can produce major changes in distribution of responses.

  • Attitudes and opinions can be unstable; circumstances can change

  • Memory can be unreliable.

  • Relationship between what respondents say they do and what they actually do is not strong.

  • Cultural context has an impact on responses (eg attitude scale grades)

  • The format itself can affect responses

  • The order of questions and answers to earlier questions can affect responses.

Next week


Questionnaire Item-Writing Workshop:

Coursebook Feedback Survey

Types of 1 to 1 interviews
Types of 1-to-1 interviews

  • Structured: with tight control over question format and possible answers. Like a face-to-face questionnaire

  • Semi-structured: with a clearly-defined question schedule but some flexibility and more open-ended answers

  • Unstructured: a narrative prompted by one general question, perhaps

1 to 1 interviews are useful when
1-to-1 Interviews are useful when …

  • Detailed information is needed from respondents

  • A smaller number of respondents is acceptable

  • Attitudes or feelings are investigated

  • Sensitive issues are explored (?)

  • ‘Key players’ are targeted

  • Ideas for a questionnaire need to be explored or ‘fine-tuned’

Advantages of 1 to 1 interviews
Advantages of 1-to-1 interviews

  • Depth of information

  • Insights eg gained from key informants

  • Flexibility of formats

  • Validity: direct contact means that data can be checked for accuracy and for relevance

  • High response rate

  • Opportunity for targeted individuals to make their voice heard – element of advocacy

Interview problem areas
Interview problem areas

  • The ‘interviewer effect’ and power relationships: face-to-face contact can directly influence answers

  • The ‘instant position’ effect: respondents feel obliged to supply answers of some sort

  • Investment of time and resources

  • Complexity of data analysis

  • Reliability issues for multiple interviews

Reliability issues
Reliability issues

  • How can we know if respondents consistently understand the question? Or if they have misinterpreted it?

  • How can we know if respondents are consistently telling the truth? Or if they are misleading the interviewer deliberately or at a subconscious level?

Features of interview design
Features of interview design

  • How many questions?

  • In what order?

  • How long/short?

  • Format? (open qqs, statements etc.)

  • Follow-up probes and/or prompts?

  • Wording of questions?

  • Mode of recording/analysing answers?

Conducting an interview
Conducting an interview

  • Establishing a relationship/trust

  • Explaining aims & procedures

  • Low key presentation of self and involvement

  • Active listening

  • Clarifications, probes and prompts

  • Recording, transcribing and analysing

Focus groups
Focus groups

  • Consist of a small group of people who are brought together by a trained ‘moderator’ (usually the researcher) to explore attitudes, feelings and ideas about a particular topic or set of issues

Main characteristics of fgs
Main characteristics of FGs

  • A question schedule is prepared by the moderator to focus the discussion

  • Place value on interaction within the group as a means of eliciting information – a social experience

  • No requirement to reach consensus

  • Less pressure to hold a fixed point of view – in fact opinions can develop

Fgs are useful when
FGs are useful when …

  • Topics or issues need to be explored in depth, or attitudes revealed

  • Typical groups of key informants are to be targeted eg language teachers

Advantages of fgs
Advantages of FGs

  • A less directive interview format

  • ‘Interviewer bias’ can be greatly reduced

  • Provide a more natural social forum for the exchange of ideas. Interviewees are able to take the initiative – not just respond

  • The right to silence

Fg problem areas
FG problem areas

  • Less directive means less predictable

  • Huge amounts of data can be collected, much of which may be unusable

  • Perception that FGs are a market research tool (Tony Blair & New Labour)