Using Qualitative Methods to Improve Cardiovascular Care

1. Using Qualitative Methods for Improving Cardiovascular Care Elizabeth H. Bradley, PhD Professor of Public Health Yale School of Medicine American Heart Association Pre-Conference Workshop May 19, 2010

2. Disclosure Research used in this presentation were funded in part by the Agency for Healthcare Research and Quality, the National Heart, Lung, and Blood Institute, the Commonwealth Fund, and the Donaghue Medical Research Foundation No conflicts of interest to disclose

3. Objectives of Workshop Develop competencies to: Define qualitative methods and know when to use them Critically evaluate the methodology of qualitative studies (study design and sampling, data collection, analysis)

4. Workshop Outline 1:00 -2:00 PM What are qualitative methods and when should we use them? (Examples of Results from Qualitative Studies) 2:00-2:15 PM Break 2:15-3:15 PM Sampling and Data Collection with Examples from Studies 3:15-3:30 PM Break 3:30-4:30 PM Data Analysis with Examples from Studies 4:30-5:00 PM Standards of Rigor and Addressing Limitations of Qualitative Studies

5. Research cycle we know from quantitative reseach Pick Topic Act Focus research objectives Report Results Define study design + sample Analyze Data Collect Data

6. What are qualitative methods? Qualitative methods are a set of research approaches used with the following objectives: To describe a phenomenon To generate hypotheses To develop grounded theory

7. Quantitative versus qualitative objectives Estimate prevalence/incidence of phenomenon, y (rather than describe phenomenon) Test(rather than generate) hypothesis concerning predictors, correlates, or consequences of y (for instance: x  y or y  y1) Test (rather than develop) a theory

8. Products of qualitative research ObjectiveOutput Describe phenomenon, y Key domains of y Taxonomy of y Generate hypotheses Recurrent themes Testable hypotheses Develop theory Conceptual model/theory (boxes w/ arrows)

9. Describing a phenomenon Not everything that can be counted counts, and not everything that counts can be counted. - Einstein

10. When is a phenomenon best described with qualitative methods? When the phenomenon that is multifaceted and complex to understand and interpret When the interest is not just the objective event but also how the event is experienced When social interaction and context are important Examples?

11. Why is describing phenomenon useful? Distilling a complex phenomenon to its key dimensions or component parts can help develop taxonomy Taxonomy help us compare apples to apples in evaluation (QI interventions, HMOs, etc.) Lays groundwork for valid measurement (improves instrument design and fielding)

12. Example: Taxonomy for Quality Improvement Efforts DomainDimensionsExemplar concepts or range Goals ContentHigh quality; low cost, market share SpecificityVery specific to nonspecific ChallengeVery challenging to “easy” goals SharednessWidely shared to poorly shared ________________________________________________________________ Administrative Support PhilosophyInnovation first; safety net provider; etc. ResourcesHuman, capital, technical _________________________________________________________________ PI initiatives TypeClinical pathway; standing orders; Style of impl.Top down; participatory; blameless _________________________________________________________________ Bradley et al., JAMA, 2002

13. Generating hypotheses Identify causal links that seem to be at work What causes what? What is the consequence? Under what conditions? Can be from what one observes directly in sequencing or from how participants talk about their experiences Remember that these arehypotheses (only)

14. Example: Recurrent themes/hypotheses in D2B improvement 1. Explicit organizational goal-setting 2. Visible senior management support 3. Innovative, standardized protocols 4. Flexibility in implementation 5. Uncompromising clinical leaders 6. Collaborative, interdisciplinary teams 7. Specific data feedback 8. Non-blaming culture  PARADOX Bradley et al., Circ, 2006

15. Developing grounded theory Theory: a set of premises or hypotheses about how the world works; a conceptual model Grounded versus axiomatic theory

16. How is grounded theory useful? Can be interesting in its own right And sets up hypotheses for testing Identifies x and y variables; direction of effects Identifies mediating effects Guides statistical model Helps avoid “fishing” exercise Helps in making sense of observed results

17. Example: Conceptual model for long-term care use Predisposing Attitudes Toward Services Intended Use of Long-term Care Actual Use of Long-term Care Need Social Norms About Caregiving Perceived Control Enabling Factors Bradley et al., HSR 2002

18. When to use qualitative methods? When your research objective matches what qualitative methods can accomplish - Describe a phenomenon - Generate (not test) hypotheses - Develop grounded theory When the phenomenon is complex and difficult to measure with existing quantitative approaches When literature and hypotheses are lacking

19. When not to use qualitative methods When you really want to know and publish how often something occurs Because you think qualitative methods will be easier, cheaper, or faster Because “there is no literature in the area”  still has to be a topic that requires qualitative inquiry

20. Can mixed methods help? Mixed methods: The integration of qualitative and quantitative methods to improve understanding Can be simultaneous or sequential (qualitative  quantitative; quantitative  qualitative

21. Positive deviance approach is a mixed methods approach (qualitative leads to quantitative) A “positive deviance” approach • Identify top performing hospitals • Study them qualitatively • Generate hypotheses about top performance • Test hypotheses quantitatively; random sample • Disseminate evidence in national campaign Bradley et al., Impl Sci 2009

22. Summary Qualitative methods provide an approach to understanding what may not have been previously examined and what defies quantitative measurement Qualitative methods must fit with the research objectives to be valid and useful

23. Break

24. Sampling and data collection

25. Common qualitative study designs In-depth interviewing Focus groups Participant or non-participant observation Case study; ethnography Hybrids

26. Sampling techniques in quantitative research What are key concepts that govern sampling? - Representative-ness of population (valid) - Big enough (precise) What are different sampling strategies? How does one determine sample size?

27. Sampling techniques in qualitative research What are key concepts that govern sampling? - Participants have the experience under inquiry - Diversity; find all dimensions that might matter - If you heard it once, you have heard it

28. Sampling strategy in qualitative research Purposeful sampling (sometimes called purposive)  Theoretical sampling as the highest standard May have a random component, which is augmented with purposeful selections Inclusion criteria (defining “key informants”) - Must have the experience in question - Must be willing to talk about it Looking for broad representation, but not in the proportions that are in the population

29. Sample size in qualitative research Concept of “theoretical saturation”  When no new concepts emerge from successive interviews  Judgment call to some extent  Typically small samples

30. Sampling for focus groups Typically 6-10 participants per group Usually 2-5 groups per “strata” Must share some common experience or trait Homogeneity v heterogeneity issues Avoid power differential within group if you can

31. Principles in data collection for quantitative research Closed-ended measures No room for interpretation by investigator Consistency and reproducibility is goal Premise that abstract can capture truth (example of “age” or “race”)

32. Principles of data collection for qualitative research Open-ended questions Investigator interpretation is expected (disclosure) Depth, validity is goal  TRUST IS #1 CONCERN Premise that truth is in rich detail, not abstraction Always need consent (usually oral is fine, per IRB)

33. Spend time establishing trust and safety Body language, facial expressions Describe goals, consent process, data integrity Do not judge anything, no matter how small, ensure that there are no right or wrong answers Important distinction between professional researcher and colleague or friend

34. Techniques in depth interviewing Discussion guide versus survey instrument Few questions (5), all open-ended, with probes Open-ended interviewing Be authentically curious Practice passive listening Be alertfor jargon, unclear links, new concepts Delve into things that do not make sense Keep your own views to yourself, interrupt judiciously Practice the 5-second pause; let silence happen

35. Typical questions Grand tour question: Tell me about your experience with… (whatever your inquiry is) Can you tell me more about that? (Think back) What was that like for you? What happened next? You used the term “physician champion,” can you tell me more about that? What did you mean when you used that term?

36. More approaches to interviewing Wait the participant out…eyebrow raises, etc. Seek examples, but be careful how you ask this. It can cause defensiveness; you want vignettes, stories, etc. You do not want their packaging (abstracting) of concept but rather the rich detail so you can interpret Summary question: is their anything I should have asked you that would help me understand xyz?

37. Special concerns in focus groups: establishing ground rules 1. Expect differences of opinions 2. Interested in positive and negative comments 3. Want to hear from everyone (“if you are talking a lot, I may asked you to give others a chance, and if you are not talking, I may call on you”) 4. Speak one at a time 5. Respect confidentiality of members 6. Use first names only (no identifying information) 7. We will stick to time limit (1 hour – 90 minutes)

38. Recording data Multiple approaches For in-depth interviews or focus groups, audio-taping is often successful (helps reliability) and forgotten by participants soon after you begin Informal note-taker other than the lead interviewer In field work (ethnography, observation studies), one often just has field notes and maybe archival data

39. In-depth interviewing and focus group moderator skills Strong interviewing skills Keen observational skills Ability to control and guide discussion Ability to suppress own personal views Has and projects authentic respect for participants

40. How to develop skills Watch someone experienced Do some yourself and have your work critiqued Practice (toss early attempts); you learn as you go and analyze your own work Recognize that it is not for everyone!

41. Summary Sampling and data collection rules of thumb in qualitative methods are almost the opposite from these rules in quantitative methods Authentic, open curiosity is key element – of any good scientist, especially one in qualitative research Self-awareness and practice improves skills

42. Break

43. Data analysis

44. Open-ended data Quantitative analysis of open-ended data - Counting frequency of different statements, ideas, etc. - Report: “20% said x is a problem” - “Content analysis” Qualitative analysis of open-ended data - Develop concepts and themes and models - Report themes and illustrative quotations - Populate the “x-axis”

45. Steps in implementing the qualitative analysis 1. Prepare the data 2. Read the data for general understanding 3. Code the data 4. Integrate the data 5. Develop taxonomy, themes, and theory Bradley et al., HSR 2008

46. Codes for organizing qualitative data Codes are tags or labels for assigning meaning to descriptive information Coding is the process of organizing the data into “chunks” that are alike, moving from words and sentences to “incidents” that depict a particular concept

47. Approaches to developing codes 1. Provisional “start list” of codes 2. Purely inductive, or grounded, codes 3. Between “start list” and inductive approaches to coding

48. Major types of codes Concepts of importance - Could be key variables emerging - Could be reasons why or how something works Characteristics of participant or setting - Often emerge as correlates or medicating factors Potentially non-causal links among concepts - Explicit or implicit evidence of inferences about how different coded concepts may interrelate

49. Beginning coding Read the transcripts or notes for overall understanding (best to occur with team) Note in words the key concepts (each separately), write memos for the file as needed, mark up margins of transcripts or use software to note Come together in group to review transcripts line-by-line, using a constant comparative method

50. Negotiating codes The group will not agree and that is the beginning of the analysis process…negotiate, talk out the concepts, fleshing out their properties Develop code list from those meetings Read a few more, same process Refine code list, recoding as needed