Field test options
1 / 29

Field test options instrument, instructions, non-response/ refusals, and Interviewer debriefing - PowerPoint PPT Presentation

  • Uploaded on

Field test options instrument, instructions, non-response/ refusals, and Interviewer debriefing Washington Group Regional Training Workshop Rio de Janeiro 19 – 20 September 2005 Margie Schneider HSRC, South Africa. Recap. What we have covered so far:

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Field test options instrument, instructions, non-response/ refusals, and Interviewer debriefing' - jody

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Field test options

instrument, instructions,

non-response/ refusals, and

Interviewer debriefing

Washington Group Regional Training Workshop

Rio de Janeiro

19 – 20 September 2005

Margie Schneider

HSRC, South Africa


What we have covered so far:

  • Purpose of the questions and where they came from

  • Core set of questions and extended set and Q X Q specifications

  • Objectives of the test

  • Translation protocol

    What we still need to go through:

  • How to design the test and related issues

  • Enumerator training

  • Plan for analysis and report writing

Different components of the testing process
Different components of the testing process

  • Pre-testing and expert review

  • Testing internal validity on people with known disability status (quota sampling and linked to cognitive testing)

  • Pilot testing – as for full Census or survey

  • Full field test

    Different sampling approaches will be discussed in relation to different testing protocols

Field test options
Field Test options

  • 2 questions to be answered:

  • What set of questions should be used in field test?

  • What field test approach should be used?

    Consider your own context, need, funding, skills, etc. in deciding

Question sets
Question sets

  • Core set – 4 + 2 (6 domains of functioning): Compulsory

  • Core + extended set on core domains : highly recommended

  • Core + extended set + further additional questions: nice to have for analysis

    • 2 additional domains (learning and interpersonal interactions)

    • Psychological distress

  • Country specific question set : nice to have for comparison with prior data

What approach to testing
What approach to testing?

  • What factors to consider in deciding:

    • Country needs

    • Capacity to undertake test

    • Funds available

    • Time available

    • Planned activities, e.g. other surveys

    • ?

Census or national survey
Census or national survey

  • Use of question set in planned Census or national survey

  • Advantages:

    • Large sample (5000 – 10 000 or whole pop)

    • Minimal additional cost for disability questions

  • Disadvantages:

    • Not sufficient space for extended set

    • Need to administer extended set and cognitive test to sub-sample (costs of time and additional training)

Special study small sample
Special study – small sample

  • Select sample on basis of known ‘disability’ status

  • Allows for construction of 2X2 table for sensitivity and specificity calculations and identifying true and false positives and negatives – use of ‘gold standard’

  • Suggest around 200 true positives and 200 true negatives

  • Interviewer does not know status of respondent (unless visible and obvious)

Special study contd
Special study (contd.)

  • Advantages:

    • Larger set of questions

    • Close observation of interview

    • Cost is not exorbitant

  • Disadvantages:

    • Might not get full population representation

    • Cost of establishing true status is high

    • What counts as the true positive? (beyond the easily observable)

    • What is the ‘gold standard’?

Special study large sample
Special study – large sample

  • Uses population based sample

  • Sample size based on expected prevalence of the different types of activity limitations

  • Prevalence rates are usually low and so need large sample to yield enough ‘disabled’ respondents

Special study large sample contd
Special study – large sample (contd)

  • Advantages:

    • Large data set

    • Detailed set of questions - more than extended set

    • Provides an indication of prevalenceusing core and extended sets

  • Disadvantages:

    • Costly and time consuming

    • Requires capacity to run and analyse

Field test and cognitive test
Field test and cognitive test

  • All respondents have core set

  • One sub sample of respondents have cognitive test (core + extended are embedded)

  • A second sub-sample of respondents do

    • Core + extended

    • Core + extended + further Qs

    • Core + extended + country specific questions

  • Some do all ?

Non responses and refusals
Non-responses and refusals

  • How do you define these?

  • How do you deal with these?

  • Aim to:

    • Reduce non-response and refusals

    • Manage them when they do arise

Reducing nr and refusals
Reducing NR and refusals

  • Use up to date sampling frame

  • Clearly defined reasons for data collection

  • Preparing the way – prior contact, letters, etc.

  • Adequate interviewer training

  • Allow budget for call-backs

  • Plan for follow up of NR and refusals

  • Separate refusals, part refusals, non-contacts and sample loss (e.g. vacant dwellings)

Effect of each on analysis
Effect of each on analysis

  • Sample loss: does not create bias but reduces sample size; need large enough initial sample to take these into account

  • Refusals and non-contact: bias where these respondents might be different to those respondents reached

  • Need to understand reasons for refusals and non-contacts – during pre-testing and pilot stage

Interviewer debriefing
Interviewer debriefing

  • Interviewers are well placed to evaluate process and content at an early stage

  • Interviewers should note comments in margins of questionnaires

  • Different approaches:

    • Group discussion (focus group technique)

    • Interviewer rating forms

    • Standardised interviewer questionnaires

    • Combination of all three

Enumerator training

Enumerator training

Washington Group Regional Training Workshop

Rio de Janeiro

19 – 20 September 2005

Margie Schneider

HSRC, South Africa

General points
General points

  • Select some disabled interviewers

  • Importance of all interviewers having a good understanding of what disability is and is not and how it relates to the questions asked

  • Avoid using term ‘disabled’ or ‘with disabilities’

  • All interviews are face to face

  • Translation training

General points contd
General points (contd.)

  • Confidentiality and understanding what this means

  • Getting informed consent (ethics clearance?)

  • Interviewers must be able to explain purpose of survey

  • Read questions in set order and with set wording

  • Editing in field of completed questionnaires

  • Submitting of completed interviews to head office

Types of interviews
Types of interviews

  • Direct: respondent answers for him or herself

  • Interpreted: an interpreter ‘translates’ and respondent answers directly

  • Facilitated: a third party assists in explaining (e.g. intellectually disabled person)

  • Proxy: a person responds for another (e.g. child)

Interviewing disabled people
Interviewing disabled people

  • Show respect and treat the person like anyone else

  • Don’t use first names unless permitted

  • Address the person directly (not their attendant)

  • Ask how you can adapt your presentation to make it easier (no need to ask what is wrong with person)

Hearing difficulties
Hearing difficulties

  • Lip reading

  • Lighting

  • Face the person

  • Get person’s attention before speaking

  • Reduce background noise

  • Set the context – especially when changing topics

  • Use written communication (literate)

Physical difficulties
Physical difficulties

  • Accessibility of building where conducting interviews

  • Presence of attendant and confidentiality issues

  • Get to same level (e.g. sitting for person using wheelchair)

  • Person to be seated comfortably

  • Address person directly

  • Pointing may be difficult

  • Person may need breaks to move around

Visual difficulties
Visual difficulties

  • Large print and small print for cue cards

  • Braille versions of cue cards

  • Good contrast printing for pictures and print (black on white or yellow)

  • Identify yourself and others in the room verbally

Communication difficulties
Communication difficulties

  • Clarify preferred mode of communication

  • Repeat what you think was said to clarify unclear speech

  • Limit to yes / no questions

Specific learning difficulties
Specific learning difficulties

  • Manage problems in spatial orientation, hand-eye coordination

  • Limit auditory, visual and tactile distractions

  • Avoid written text

  • Explain carefully (if verbal language skills are affected)

Intellectual difficulties
Intellectual difficulties

  • Be careful with informed consent

  • Explain terms simply

  • Listen carefully

  • Have familiar person (friend or relative) close by

  • Use pictures or role play with little human or animal figures

Emotional or mental health difficulties
Emotional or mental health difficulties

  • Side effects of medication

  • Break up interview if too fatigued

  • Give encouragement and support

  • Manage expressions of frustration

  • Manage stress

Hidden difficulties
Hidden difficulties

  • Might not come forward with information because of fear of stigma

  • Effect of medication

  • May need to break up interview