Disability questions in censuses and surveys Methodological issues: proxy response, non-response and mode of administr

1. Disability questions in censuses and surveys Methodological issues: proxy response, non-response and mode of administration 21-23 September 2005 Montserrat López-Cobo Fifth Meeting of the Washington Group

2. Proxy response • Are proxy-responses different from self-responses? Why? • Do all proxies provide the same quality responses? Who is the best proxy? • Is there an effect on the quality of data? Can we measure it? Can we correct it?

3. Proxy responseLevel differences 1 It is generally accepted that proxy- and self- responses differ • Many studies found that proxy-respondents underreport disability-related information: • Disability • Impairments of emotion and Pain • Limitation of activities, long-term disability, need for assistance with ADL • Chronic condition, days of activity restriction, bed disability days • General health events and conditions

4. Proxy responseLevel differences 2 • ... While some other studies found that proxies overreport: • Cognitive status, Mild Cognitive Impairment • ADL disability • Functional health, when the self-respondent is demented

5. Proxy responseUnderstanding self-proxy differences 1 • Differences in cognitive procedures (1/2) • Type and level of available information • Proxies report impairments or conditions that are more observable or mentioned to them • Proxy reports are more accurate for conditions that are serious, painful, persistent or potentially life threatening • If person under 65: proxies underreport • If person 65 and over: proxies overreport • Proxy information is more stable. Self information is more dynamic • Self-responses are less consistent across time

6. Proxy responseUnderstanding self-proxy differences 2 • Differences in cognitive procedures (2/2) • Proxies engage in less extensive cognitive process. Provide heuristic-based responses • Proxies rely more on inferences and estimations. • Proxies overreport disabilities aparently related to a previously reported disability and underreport disabilities seemingly unrelated to the previously reported. • Proxies are less affected by social desirability concerns • Proxies are more willing to report disability

7. Proxy responseThe best proxy-respondent • Social relationship with the selected person • The closest the relationship, the most accurate the proxy-response • Spouse’s responses are better than other proxies’ • Proffesionals and caregivers provide more accurate information than lay proxies • Length of relationship • Positive effect in convergence self-proxy • Memory capacity of respondent • Affect to data quality

8. Proxy responseMeasuring and correcting bias • Types of methods: • 1) Traditional • First step: Measure the bias by comparing proxy- estimates with self- estimates • Second step: Introduce statistical control adjusting for demographic or health-related variables • 2) New strategy • First: Set assumptions about the nature of differences between self- and proxy-responses • Second: Model and estimate these differences based on the assumptions

9. Mode of administration

10. Mode of administrationSources of differences among modes 1 • Differences in the sample • Population coverage • Response rates • Differences due to social context within which data are collected • Contact with interviewer • Non-verbal cues • Social desirability bias: Reduced with self-administered questionnaires

11. Mode of administrationSources of differences among modes 2 • Differences intrinsic to the mode • Visual materials cannot be used by phone • Complexity of the questionnaire is limited by the mode • Stimuli in a visual mode  Primacy effects Stimuli in a hearing mode  Recency effects • Differences in responses involving long lists • Order effects • Effects of the mode on the interviewer

12. Mode of administrationResults from comparative studies 1 Telephone interviews • Do not underrepresent people with disabilities • Sometimes is not as well accepted by population as face-to-face interviews • Proved useful to assess mental health using recommended instruments (GHQ-12, CIS-R, CIDIS), the Expanded Disability Status Sacle Telephone vs Mail • Mail responses report poorer health and more chronic conditions than Telephone • Differential non-response rates by age: Elder: NR (Telephone) > NR (Mail) Young: NR (Telephone) < NR (Mail) General: Item NR (Telephone) < Item NR (Mail)

13. Mode of administrationResults from comparative studies 2 Face-to-Face vs Mail • Health differences between respondents and non-respondents by mode: non-response bias • Non-respondents and late respondents to Mail are more cognitively impaired and more disabled than respondents (among elderly). • Non-respondents to F-t-F are similar to respondents.

14. Mode of administrationResults from comparative studies 3 Telephone/CATI vs Face to Face • Differential non-response rates: NR (CATI) > NR (F-t-F) • Measurement bias: MBias (CATI) < MBias (F-t-F) • No differences between CATI and F-t-F for reports on chronic conditions, activity limitations and disability rates CATI vs CAPI • Differential non-response rates by age: Elder: NR (CATI) > NR (CAPI) Young: NR (CATI) < NR (CAPI)

15. Mode of administrationResults from comparative studies 4 CASI • Reduces social desirability bias • Enhances the feeling of privacy • Respondents generally like CASI • Technological possibilities have a positive influence on data quality (minimizing errors) • Respondent’s self-disclosure is higher in CASI

16. Mode of administrationAn experience interviewing disabled people by telephone • Disabilities: Physical, sensory, mental illness and mental retardation • Challenges addressed: communication, fatigue and cognitive issues • Questionnaire design • Eliminate soft consonant sounds (s, z, t, f and g) to overcome high-frequency hearing loss • Build in "breaks" for respondents to let them rest • Incorporate neutral encouragement to avoid drop-outs • Design checks for unexpected responses • Use structured probes for questions that might be difficult to understand

17. Mode of administrationAn experience interviewing disabled people by telephone • Interviewer training and supervision • Usual background and purpose of study • Training on challenges likely to face • Sensitive exercise regarding the treatment to the disabled person • Guidance to overcome each of the challenges • Support to interviewers and reduction of stress • Other recommendations • Interviews take longer  Multiple sessions may be required

18. Non-response

19. Non-response and Disability • Components of non-response • Non-contact • Non-cooperation • Non-contact: the household pattern of disabled • Are older and likely to live in non-metropolitan areas  higher probability of being contacted • Less likely to live with children  lower prob. of contact • May be more fearful of opening doors to strangers  lower prob. of contact

20. Non-response and Disability • Non-cooperation: exchange theory vs social isolation theory • Exchange theory: Persons who feel that the survey sponsor has provided (or could provide) benefits to them are more likely to cooperate • Persons with disabilities might be expected to be more cooperative with a government-sponsored survey • Social isolation theory: People who are isolated from the mainstream society feel less responsibility toward government and are less likely to cooperate • Persons with disabilities might be expected to be less cooperative

21. Non-response and DisabilityResults from studies 1 • Persons with severe disability are more likely than persons with less severe disability to be contacted and to cooperate, but when they are interviewed they are less likely to answer for themselves; instead, proxy and assistant respondents tend to answer for them. • Elder people with disabilities respond sooner than young and non-disabled to mail surveys. • These findings support Exchange theory • Self-perceived memory problems increase item non-response and “I don’t know” answers. Not preceived memory deficits implies inaccurate information on items requiring recall.

22. Non-response and DisabilityResults from studies 2 • Health of non-respondents is worse than respondents’ in terms of: stroke, Basic ADL, mobility disabilities, self-rated health and mortality rates. • Respondents with poorer physical functioning and/or limiting long-term illness have higher non-response rates. • These findings support Social isolation theory

23. Recommendations for including disabled people in interview surveys

24. Bias due to exclusion is amplified in surveys where disability is a key measure of interest • Recommendations: • Include institutionalised population in samples • Carry out the interview in a private environment • Provide adaptative technologies and procedures (of primary importance in visual, hearing and speech impairment). • Use simple and clear questions

25. Recommendations: • Special training for interviewers • Provide alternative modes of administration • If proxy is used: • Respondent’s assessments of proxy’s answer is valuable • The proxy should be nominated by the selected respondent

26. Final conclusion • Proxy response validity depends on factors such as: proxy choice and distinction, topic investigated, health condition of the selected person... • Mode of administration (unique or a mix-mode) should be decided taking into account: topic investigated, population objective of the survey, expected acceptability of the mode by the population, technological possibilities...

27. Final conclusion • Different patterns of response might be indicative of bias which can affect estimates. • Non-response is one of the sources of data error. But not the unique... Emphasis should be given not only to minimazing non-response rates but also to estimation and control of measurement error.

28. Fifth Meeting of the Washington Group 21-23 September 2005