Skip this Video
Download Presentation
Field test options instrument, instructions, non-response/ refusals, and Interviewer debriefing

Loading in 2 Seconds...

play fullscreen
1 / 29

Field test options instrument, instructions, non-response/ refusals, and Interviewer debriefing - PowerPoint PPT Presentation

  • Uploaded on

Field test options instrument, instructions, non-response/ refusals, and Interviewer debriefing Washington Group Regional Training Workshop Rio de Janeiro 19 – 20 September 2005 Margie Schneider HSRC, South Africa. Recap. What we have covered so far:

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Field test options instrument, instructions, non-response/ refusals, and Interviewer debriefing' - jody

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Field test options

instrument, instructions,

non-response/ refusals, and

Interviewer debriefing

Washington Group Regional Training Workshop

Rio de Janeiro

19 – 20 September 2005

Margie Schneider

HSRC, South Africa


What we have covered so far:

  • Purpose of the questions and where they came from
  • Core set of questions and extended set and Q X Q specifications
  • Objectives of the test
  • Translation protocol

What we still need to go through:

  • How to design the test and related issues
  • Enumerator training
  • Plan for analysis and report writing
different components of the testing process
Different components of the testing process
  • Pre-testing and expert review
  • Testing internal validity on people with known disability status (quota sampling and linked to cognitive testing)
  • Pilot testing – as for full Census or survey
  • Full field test

Different sampling approaches will be discussed in relation to different testing protocols

field test options
Field Test options
  • 2 questions to be answered:
  • What set of questions should be used in field test?
  • What field test approach should be used?

Consider your own context, need, funding, skills, etc. in deciding

question sets
Question sets
  • Core set – 4 + 2 (6 domains of functioning): Compulsory
  • Core + extended set on core domains : highly recommended
  • Core + extended set + further additional questions: nice to have for analysis
    • 2 additional domains (learning and interpersonal interactions)
    • Psychological distress
  • Country specific question set : nice to have for comparison with prior data
what approach to testing
What approach to testing?
  • What factors to consider in deciding:
    • Country needs
    • Capacity to undertake test
    • Funds available
    • Time available
    • Planned activities, e.g. other surveys
    • ?
census or national survey
Census or national survey
  • Use of question set in planned Census or national survey
  • Advantages:
    • Large sample (5000 – 10 000 or whole pop)
    • Minimal additional cost for disability questions
  • Disadvantages:
    • Not sufficient space for extended set
    • Need to administer extended set and cognitive test to sub-sample (costs of time and additional training)
special study small sample
Special study – small sample
  • Select sample on basis of known ‘disability’ status
  • Allows for construction of 2X2 table for sensitivity and specificity calculations and identifying true and false positives and negatives – use of ‘gold standard’
  • Suggest around 200 true positives and 200 true negatives
  • Interviewer does not know status of respondent (unless visible and obvious)
special study contd
Special study (contd.)
  • Advantages:
    • Larger set of questions
    • Close observation of interview
    • Cost is not exorbitant
  • Disadvantages:
    • Might not get full population representation
    • Cost of establishing true status is high
    • What counts as the true positive? (beyond the easily observable)
    • What is the ‘gold standard’?
special study large sample
Special study – large sample
  • Uses population based sample
  • Sample size based on expected prevalence of the different types of activity limitations
  • Prevalence rates are usually low and so need large sample to yield enough ‘disabled’ respondents
special study large sample contd
Special study – large sample (contd)
  • Advantages:
    • Large data set
    • Detailed set of questions - more than extended set
    • Provides an indication of prevalenceusing core and extended sets
  • Disadvantages:
    • Costly and time consuming
    • Requires capacity to run and analyse
field test and cognitive test
Field test and cognitive test
  • All respondents have core set
  • One sub sample of respondents have cognitive test (core + extended are embedded)
  • A second sub-sample of respondents do
    • Core + extended
    • Core + extended + further Qs
    • Core + extended + country specific questions
  • Some do all ?
non responses and refusals
Non-responses and refusals
  • How do you define these?
  • How do you deal with these?
  • Aim to:
    • Reduce non-response and refusals
    • Manage them when they do arise
reducing nr and refusals
Reducing NR and refusals
  • Use up to date sampling frame
  • Clearly defined reasons for data collection
  • Preparing the way – prior contact, letters, etc.
  • Adequate interviewer training
  • Allow budget for call-backs
  • Plan for follow up of NR and refusals
  • Separate refusals, part refusals, non-contacts and sample loss (e.g. vacant dwellings)
effect of each on analysis
Effect of each on analysis
  • Sample loss: does not create bias but reduces sample size; need large enough initial sample to take these into account
  • Refusals and non-contact: bias where these respondents might be different to those respondents reached
  • Need to understand reasons for refusals and non-contacts – during pre-testing and pilot stage
interviewer debriefing
Interviewer debriefing
  • Interviewers are well placed to evaluate process and content at an early stage
  • Interviewers should note comments in margins of questionnaires
  • Different approaches:
    • Group discussion (focus group technique)
    • Interviewer rating forms
    • Standardised interviewer questionnaires
    • Combination of all three
enumerator training

Enumerator training

Washington Group Regional Training Workshop

Rio de Janeiro

19 – 20 September 2005

Margie Schneider

HSRC, South Africa

general points
General points
  • Select some disabled interviewers
  • Importance of all interviewers having a good understanding of what disability is and is not and how it relates to the questions asked
  • Avoid using term ‘disabled’ or ‘with disabilities’
  • All interviews are face to face
  • Translation training
general points contd
General points (contd.)
  • Confidentiality and understanding what this means
  • Getting informed consent (ethics clearance?)
  • Interviewers must be able to explain purpose of survey
  • Read questions in set order and with set wording
  • Editing in field of completed questionnaires
  • Submitting of completed interviews to head office
types of interviews
Types of interviews
  • Direct: respondent answers for him or herself
  • Interpreted: an interpreter ‘translates’ and respondent answers directly
  • Facilitated: a third party assists in explaining (e.g. intellectually disabled person)
  • Proxy: a person responds for another (e.g. child)
interviewing disabled people
Interviewing disabled people
  • Show respect and treat the person like anyone else
  • Don’t use first names unless permitted
  • Address the person directly (not their attendant)
  • Ask how you can adapt your presentation to make it easier (no need to ask what is wrong with person)
hearing difficulties
Hearing difficulties
  • Lip reading
  • Lighting
  • Face the person
  • Get person’s attention before speaking
  • Reduce background noise
  • Set the context – especially when changing topics
  • Use written communication (literate)
physical difficulties
Physical difficulties
  • Accessibility of building where conducting interviews
  • Presence of attendant and confidentiality issues
  • Get to same level (e.g. sitting for person using wheelchair)
  • Person to be seated comfortably
  • Address person directly
  • Pointing may be difficult
  • Person may need breaks to move around
visual difficulties
Visual difficulties
  • Large print and small print for cue cards
  • Braille versions of cue cards
  • Good contrast printing for pictures and print (black on white or yellow)
  • Identify yourself and others in the room verbally
communication difficulties
Communication difficulties
  • Clarify preferred mode of communication
  • Repeat what you think was said to clarify unclear speech
  • Limit to yes / no questions
specific learning difficulties
Specific learning difficulties
  • Manage problems in spatial orientation, hand-eye coordination
  • Limit auditory, visual and tactile distractions
  • Avoid written text
  • Explain carefully (if verbal language skills are affected)
intellectual difficulties
Intellectual difficulties
  • Be careful with informed consent
  • Explain terms simply
  • Listen carefully
  • Have familiar person (friend or relative) close by
  • Use pictures or role play with little human or animal figures
emotional or mental health difficulties
Emotional or mental health difficulties
  • Side effects of medication
  • Break up interview if too fatigued
  • Give encouragement and support
  • Manage expressions of frustration
  • Manage stress
hidden difficulties
Hidden difficulties
  • Might not come forward with information because of fear of stigma
  • Effect of medication
  • May need to break up interview