Partnering with Community in Survey Design

1. Partnering with Community in Survey Design Kathleen Thiede Call, Ph.D. AcademyHealth, Seattle WA June 27, 2006 Supported by a grant from the Minnesota Department of Human Services

2. Overview of Presentation • Context • Problem of mistrust • Role of community partnerships in solving this problem • Example: Disparities in Minnesota Health Care Programs • Principal Findings: Challenges and Rewards of Partnership • Lessons learned and challenge to field

3. Problem: Mistrust • Community mistrust of Medical/Health care system Medical research Survey research

4. Solution: Community “Involvement” in Research “Community-based participatory research in public health is a collaborative approach to research that equitably involves… community members, organizational representatives, and researchers in all aspects of the research process.” (Israel et al., 2000) Academic Research Community Based

5. From Theory to Practice • Project: Disparities and barriers to utilization among Minnesota health care program enrollees • Task: Creation and implementation of statewide survey of program enrollees (e.g., Medicaid, MinnesotaCare) stratified by race/ethnicity

6. Principal Findings: Challenges and Rewards of Partnership • Constraints to participatory research • Survey development • Conceptual equivalence and interviewer quality • Survey administration and response rates

7. Constraints: Project Timeline and Milestones Background research Jan-Feb 03 Focus groups Feb 03 Sample drawn Feb 03 Instrument development Feb-June 03 Survey translation June 03 Data collection July-Sep 03 Data cleaning, weighting Sep 03 Data analysis/reporting Sep-Nov 03 Final report submitted Dec 03

8. Participatory Components of Disparities Project • Administrative structure • Project Management Team • Subcommittees • Communication • Resource distribution

9. Project Administration • Project Management Team (PMT) comprised of key academic/institutional and community researchers charged with project oversight • Stratis Health – Fiscal Agent (EQRO, federal match) • University of Minnesota – PI • The Urban Coalition • Community Researchers (Hmong, African American, Somali, Hispanic, American Indian) • DHS project director • Project subcommittees responsible for specific project tasks

10. Organizational Structure of Research Team Project Management Team Focus Group Instrument Development Survey Administration Data Analysis Dissemination

11. Communication • Continuous intentional communication between PMT and subcommittee members • Monthly meetings of PMT, weekly meetings of subcommittees • Communication via telephone, e-mail • Communication facilitated by overlapping subcommittee membership • Focus on process as well as tasks • Decision-making through consensus where possible, given time constraints

12. Resource Distribution • Although limited, budget allowed for support of academic/institutional and community researchers • Distribution flow: Stratis Health University of Minnesota The Urban Coalition (facilitated community contracting)

13. Survey Creation • Focus groups to inform survey content • Understand community definitions of prevention, service seeking and barriers • Mixed mode survey administration: • Created English mail version • Translated for telephone follow-up: English, Spanish, Somali, Hmong • Face page included instructions in all 4 languages and phone numbers for those wishing to complete phone survey • Translation and pre-testing overseen by multi-lingual, multi-cultural PMT members

14. Conceptual Equivalence and Interviewer Quality • Multi-lingual members participation in design of English survey ensured conceptual equivalence in translation. Two major problems in all 3 languages: • Translations too literal • Spanish: “Indian Health Center” word for “Indian” that could be confused with “from India” instead of American Indian • Hmong: “tus kws kho mob” (the one who cures diseases) instead of “doctor” which is term used • Errors • Spanish back translation yielded “It could be said...” instead of “Would you say…”[response options]. For example “In general, how would you rate your overall health? Would you say it is excellent, very good, good, fair or poor? • Hmong language-- multiple ways of saying “Yes/No” and the response code selected must correspond to the question asked. Sonetimes instead of using “Yes/No” the equivalent of “I believe/do not believe,” or “can do/cannot do,” (Pab, Txhawj, Ua Tau, Mus, etc) is more appropriate. • Somali translation yielded inappropriately “loaded” phrasing of questions or simply did not capture meaning of question.

15. PMT Process and Focus Group Influence on Survey • Expansions to survey • Sources of preventive care (turn to help keep from getting sick) • Expanded response categories beyond doctor or clinic • Spiritual or traditional healer, shaman • Chiropractor • Acupuncturist or herbalist • Beliefs: “To what extent do you agree or disagree with the following statement: There is little doctors can do to keep you from getting sick” • Discrimination • Economic and class based-- Being enrolled in a Minnesota Health Care Program such as Medicaid, Medical Assistance or MinnesotaCare • Barriers • Worry about going to the doctor or clinic for a check-up because you might get bad news

16. Expansion of Survey: Confidence in Providers

17. Survey Administration • Hiring and interviewer quality control overseen by multi-lingual, multi-cultural PMT members • Sample drawn February; Fielded April – July 2003 • Initial mailing to 9,350 enrollees • De-duplication at household level • All ages, with parent responding for sampled child

18. Response and Cooperation Rates(overall response rate 54%) Total of 4,953 surveys completed

19. Summary of Value Added Through Community Partnership • Mail survey in English only (Cost savings) • Instrument design • Content of survey somewhat expanded • Accessible wording • Quality of translations • Involvement of multi-lingual, multi-cultural researchers in survey design increased conceptual equivalence • Quality of interviewing • Hiring and training of interviewers

20. Lessons Learned in Two Areas • Participatory process • Lose a little….gain a lot! • Mistakes are the most important part • Conflict happens! Intentional communication allows us to build relationships, partnerships and most importantly TRUST • Sharing power challenges everyone • Forward momentum can help counter personal politics • Process is as important as product! • Survey research community • No need to compromise methodological rigor • Improved quality and serves larger purpose—ownership and application of results vs subjects of research

21. Collaborators • Tim Beebe, University of Minnesota • Heather Britt, The Urban Coalition • Valeng Cha, Cha Consulting • Charity Kreider, University of Minnesota • Jennifer Lundblad, Stratis Health • Donna McAlpine, University of Minnesota • Jim McRae, Minnesota Department of Human Services • Betty Moore, Independent Consultant (formerly with the Indian Health Board) • Sirad Osman, New Americans Community Services • Walter Suarez, Midwest Center for HIPAA Education

22. For More Information • Main page: http://www.dhs.state.mn.us/HealthCare/pmqi/default.htm#Research-Evaluation • Full report: http://www.dhs.state.mn.us/HealthCare/pmqi-pdfs/Disparities-final-report-Dec2003.pdf • Focus group report: http://edocs.dhs.state.mn.us/lfserver/Legacy/DM-0171-ENG

24. Potential Strengths of Participatory Research • Builds or rebuilds trust with communities • Improves quality of research • Enhances research capacity • Empowering for all partners • (e.g., decision-making, resources, use of results) • Benefit from diverse perspectives/skills • Relevance and applicability of research

25. Potential Weaknesses/Challenges of Participatory Research • Threats to traditional scientific rigor • Lack of standardization • Time consuming endeavor • Research questions and methodology often specified after project initiation • Not always rewarded in an academic environment • Appropriate funding mechanisms difficult