Washington Group Cognitive Test

1. Kristen Miller Questionnaire Design Research Lab National Center for Health Statistics, USA Purpose of cognitive test Structure of protocol Types of cognitive test data & possible analyses Things to keep in mind Washington Group Cognitive Test

2. Purpose of Cognitive Testing • Interviews: Designed to examine the stages of the question response process • Comprehension, retrieval, judgment, response • Analysis: Identifies… • potential response errors: (e.g. vague concepts, recall problems, inadequate response options) • patterns of interpretation

3. Traditional Cognitive Testing • Protocol Semi-structured • Data collected are qualitative • Qualitative methodologies for analysis • Small samples: approx. 10 – 30 participants

4. Washington Group Protocol • Structured Questionnaire • Designed to illustrate: • Administration ease • Interpretations • Factors considered by respondent • Degree of consistency with physical abilities • Differences between subgroups

5. Cognitive Interview Protocol • Demographic Section: Country, language, gender, age, SES • Question Testing Sections • Core Question • Interviewer Coding • Open-ended Follow-up Probe • Cognitive Follow-up Probe • Functioning Follow-up Probe • Health Questions: subjective health, chronic condition list • Interviewer Debriefing

6. Interviewer Code: Did the respondent… • need you to repeat any part of the question? Yes/No • have any difficulty using the response options? Yes/No • ask for clarification or qualify their answer? Yes/No

7. Percent of respondents with no response problems Vision 83.89% Hearing 88.47% Cognitive 85.72% Mobility 91.03% Self-care 93.65% Communication 86.70% N=1,287

8. Number of response problems by age

9. Open-ended Follow-up Probe • Can you tell me what you were thinking? Why did you answer [fill with respondent’s answer to the core question]?

10. Qualitative Data • Varied in quality • A lot missing • Very brief • Provides some insight • E.g. “Because I wear glasses”

11. Cognitive Follow-up Probes: • How often they had difficulty • The amount of effort they needed to put into the activity • The degree of concern or worry they had regarding the difficulty • Other domain-specific questions • Communication: Shyness question • Cognitive: Reason question

12. Cognition Cognitive Probe Do you believe that your memory or concentration difficulties are… [Mark all that apply] • because you have too many things to do? • because you getting older? • or, because of something else? Only busy: 28.78% Aging: 25.84% Something else: 43.07%

13. Reason by degree of problem

14. Functioning Follow-up Probes • Presence of a condition • Use of an assistive device • Degree of difficulty doing related activities

15. Analysis of Functioning Data • Examine the consistency between: • Washington Group Question responses and • Follow-up functioning questions responses • Goal: explain the discrepancies • Misunderstood word? • Cultural difference? • Translation problem? • Interviewer error? • Error in the follow-up questions? • Other data assist in explanation

16. Response patterns: Vision

17. Response patterns: Hearing