1 / 29

Field test options instrument, instructions, non-response/ refusals, and Interviewer debriefing

Field test options instrument, instructions, non-response/ refusals, and Interviewer debriefing Washington Group Regional Training Workshop Rio de Janeiro 19 – 20 September 2005 Margie Schneider HSRC, South Africa. Recap. What we have covered so far:

jody
Download Presentation

Field test options instrument, instructions, non-response/ refusals, and Interviewer debriefing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Field test options instrument, instructions, non-response/ refusals, and Interviewer debriefing Washington Group Regional Training Workshop Rio de Janeiro 19 – 20 September 2005 Margie Schneider HSRC, South Africa

  2. Recap What we have covered so far: • Purpose of the questions and where they came from • Core set of questions and extended set and Q X Q specifications • Objectives of the test • Translation protocol What we still need to go through: • How to design the test and related issues • Enumerator training • Plan for analysis and report writing

  3. Different components of the testing process • Pre-testing and expert review • Testing internal validity on people with known disability status (quota sampling and linked to cognitive testing) • Pilot testing – as for full Census or survey • Full field test Different sampling approaches will be discussed in relation to different testing protocols

  4. Field Test options • 2 questions to be answered: • What set of questions should be used in field test? • What field test approach should be used? Consider your own context, need, funding, skills, etc. in deciding

  5. Question sets • Core set – 4 + 2 (6 domains of functioning): Compulsory • Core + extended set on core domains : highly recommended • Core + extended set + further additional questions: nice to have for analysis • 2 additional domains (learning and interpersonal interactions) • Psychological distress • Country specific question set : nice to have for comparison with prior data

  6. What approach to testing? • What factors to consider in deciding: • Country needs • Capacity to undertake test • Funds available • Time available • Planned activities, e.g. other surveys • ?

  7. Census or national survey • Use of question set in planned Census or national survey • Advantages: • Large sample (5000 – 10 000 or whole pop) • Minimal additional cost for disability questions • Disadvantages: • Not sufficient space for extended set • Need to administer extended set and cognitive test to sub-sample (costs of time and additional training)

  8. Special study – small sample • Select sample on basis of known ‘disability’ status • Allows for construction of 2X2 table for sensitivity and specificity calculations and identifying true and false positives and negatives – use of ‘gold standard’ • Suggest around 200 true positives and 200 true negatives • Interviewer does not know status of respondent (unless visible and obvious)

  9. Special study (contd.) • Advantages: • Larger set of questions • Close observation of interview • Cost is not exorbitant • Disadvantages: • Might not get full population representation • Cost of establishing true status is high • What counts as the true positive? (beyond the easily observable) • What is the ‘gold standard’?

  10. Special study – large sample • Uses population based sample • Sample size based on expected prevalence of the different types of activity limitations • Prevalence rates are usually low and so need large sample to yield enough ‘disabled’ respondents

  11. Special study – large sample (contd) • Advantages: • Large data set • Detailed set of questions - more than extended set • Provides an indication of prevalenceusing core and extended sets • Disadvantages: • Costly and time consuming • Requires capacity to run and analyse

  12. Field test and cognitive test • All respondents have core set • One sub sample of respondents have cognitive test (core + extended are embedded) • A second sub-sample of respondents do • Core + extended • Core + extended + further Qs • Core + extended + country specific questions • Some do all ?

  13. Non-responses and refusals • How do you define these? • How do you deal with these? • Aim to: • Reduce non-response and refusals • Manage them when they do arise

  14. Reducing NR and refusals • Use up to date sampling frame • Clearly defined reasons for data collection • Preparing the way – prior contact, letters, etc. • Adequate interviewer training • Allow budget for call-backs • Plan for follow up of NR and refusals • Separate refusals, part refusals, non-contacts and sample loss (e.g. vacant dwellings)

  15. Effect of each on analysis • Sample loss: does not create bias but reduces sample size; need large enough initial sample to take these into account • Refusals and non-contact: bias where these respondents might be different to those respondents reached • Need to understand reasons for refusals and non-contacts – during pre-testing and pilot stage

  16. Interviewer debriefing • Interviewers are well placed to evaluate process and content at an early stage • Interviewers should note comments in margins of questionnaires • Different approaches: • Group discussion (focus group technique) • Interviewer rating forms • Standardised interviewer questionnaires • Combination of all three

  17. Enumerator training Washington Group Regional Training Workshop Rio de Janeiro 19 – 20 September 2005 Margie Schneider HSRC, South Africa

  18. General points • Select some disabled interviewers • Importance of all interviewers having a good understanding of what disability is and is not and how it relates to the questions asked • Avoid using term ‘disabled’ or ‘with disabilities’ • All interviews are face to face • Translation training

  19. General points (contd.) • Confidentiality and understanding what this means • Getting informed consent (ethics clearance?) • Interviewers must be able to explain purpose of survey • Read questions in set order and with set wording • Editing in field of completed questionnaires • Submitting of completed interviews to head office

  20. Types of interviews • Direct: respondent answers for him or herself • Interpreted: an interpreter ‘translates’ and respondent answers directly • Facilitated: a third party assists in explaining (e.g. intellectually disabled person) • Proxy: a person responds for another (e.g. child)

  21. Interviewing disabled people • Show respect and treat the person like anyone else • Don’t use first names unless permitted • Address the person directly (not their attendant) • Ask how you can adapt your presentation to make it easier (no need to ask what is wrong with person)

  22. Hearing difficulties • Lip reading • Lighting • Face the person • Get person’s attention before speaking • Reduce background noise • Set the context – especially when changing topics • Use written communication (literate)

  23. Physical difficulties • Accessibility of building where conducting interviews • Presence of attendant and confidentiality issues • Get to same level (e.g. sitting for person using wheelchair) • Person to be seated comfortably • Address person directly • Pointing may be difficult • Person may need breaks to move around

  24. Visual difficulties • Large print and small print for cue cards • Braille versions of cue cards • Good contrast printing for pictures and print (black on white or yellow) • Identify yourself and others in the room verbally

  25. Communication difficulties • Clarify preferred mode of communication • Repeat what you think was said to clarify unclear speech • Limit to yes / no questions

  26. Specific learning difficulties • Manage problems in spatial orientation, hand-eye coordination • Limit auditory, visual and tactile distractions • Avoid written text • Explain carefully (if verbal language skills are affected)

  27. Intellectual difficulties • Be careful with informed consent • Explain terms simply • Listen carefully • Have familiar person (friend or relative) close by • Use pictures or role play with little human or animal figures

  28. Emotional or mental health difficulties • Side effects of medication • Break up interview if too fatigued • Give encouragement and support • Manage expressions of frustration • Manage stress

  29. Hidden difficulties • Might not come forward with information because of fear of stigma • Effect of medication • May need to break up interview

More Related