Chapter 10

1. Chapter 10 Standards of Validation and Evaluation Creswell Qualitative Inquiry 2e

2. Key Questions • What are some of the qualitative perspectives on validation? • What are some alternative procedures useful in establishing validation? • How is reliability used in qualitative research? • What are some alternative stances on evaluating the quality of qualitative research? • How do these stances differ by types of approaches to qualitative inquiry? Creswell Qualitative Inquiry 2e

3. Perspectives on Validity • Qualitative parallel equivalents to quantitative (LeCompte & Goetz, 1982) • Perspective: Parallel qualitative equivalents to counterparts in experimental and survey research • Terms: Internal Validity, External Validity, Reliability, Objectivity • New terms should be used (Lincoln & Guba, 1985) • Perspective: Alternative terms that apply more to naturalistic axioms • Terms: Credibility, Transferability, Dependability, Confirmability Creswell Qualitative Inquiry 2e

4. Perspectives and Terms About Verification (cont.) • New terms focused on credibility (Eisner, 1991) • Perspective: Alternative terms that focus on reasonable standards for judging the credibility of qualitative research • Terms: Structural Corroboration, Consensual Validation, Referential Adequacy • Feminist perspective (Lather, 1993) • Perspective: Reconceptualized: Four frames of validity • Terms: Iconic Validity, Paralogic Validity, Rhizomatic Validity, Situated/Embedded Voluptuous Validity Creswell Qualitative Inquiry 2e

5. Perspectives and Terms About Verification (cont.) • Metaphor for (Richardson, 1990) • Perspective: Reconceptualized: Metaphor of a crystal • Terms: Crystals: Grow, change, alter, reflect externalities, refract within themselves • Distraction (Wolcott, 1994) • Perspective: Distraction: Neither guides nor informs • Terms: Understanding (better term to use than validity) Creswell Qualitative Inquiry 2e

6. Types of Validation • Interpretivist approach (Angen, 2000) • Ethical Validation: All research agendas must question their underlying moral assumptions, their political and ethical implications, and the equitable treatment of diverse voices • Provides practical answers to questions • Stimulates new dialogue • Substantive Validation: Researchers understand their own understanding of the topic that is derived from other sources and document this in the study • Self-reflection process • Interaction with the subject matter to co-create interpretations • Written accounts must resonate with audiences - Creswell Qualitative Inquiry 2e

7. Types of Validation (cont.) • Synthesis of approaches (Whittemore, Chase, & Mendele, 2001). Found four primary criteria: • Credibility: the results are an accurate interpretation of the participant’s meaning • Authenticity: different voices are heard • Criticality: critical appraisal of all aspects of the research • Integrity: the investigators are self-critical Creswell Qualitative Inquiry 2e

8. Validation Strategies (cont.) • Prolonged engagement in the field • Builds trust with the participant • The researcher learns from the culture • The researcher can check for misinformation • Triangulation • Using multiple sources • Using multiple research methods • Using multiple investigators • Using multiple theories to provide corroborating evidence Creswell Qualitative Inquiry 2e

9. Validation Strategies (cont.) • Peer Review • Provides an external check of the research process • Asks questions about methods, meanings, and interpretations • Provides a record of notes by both peer and researcher • Negative Case Analysis • Uses disconfirming evidence to get alternative points of view • Enables the researcher to revise the initial hypothesis until all the cases fit Creswell Qualitative Inquiry 2e

10. Validation Strategies (cont.) • Rich, Thick Description • Involves descriptive detail of the participants and the setting • Allows the reader to make decisions regarding any transferability of findings • External Audits • Uses an external consultant who examines both the product, process, and account to assess accuracy • Examines whether or not the findings and interpretation are supported by evidence • Uses an auditor not connected to the study or the researcher Creswell Qualitative Inquiry 2e

11. Validation Strategies • Clarifying researcher bias: The researcher comments on past experiences that have likely shaped the interpretation and approach to the study • In member checking • Solicits participants’ views on the credibility of the findings and interpretations • A critical technique for establishing credibility • A focus group could be used • The preliminary analysis of descriptions and themes is given to participants NOT transcripts or raw data Creswell Qualitative Inquiry 2e

12. Reliability Perspectives in Qualitative Research • Observations will enhance reliability through: • Use of multiple data analysts • Use of fieldnote conventions such as double quotation marks to indicate verbatim quotes • Transcript analysis can enhance reliability through: • Use of multiple data analysts • Use of fieldnote conventions such as double quotation marks to indicate verbatim quotes Creswell Qualitative Inquiry 2e

13. Reliability Perspectives in Qualitative Research (cont.) • Textual analysis can enhance reliability through: • Use multiple coders and calculate intercoder reliability • Use agreed upon standardized categories • Interview procedures can enhance reliability through: • Pretest interview protocols • Train interviewers • Use of fixed-choice answers • Reliability checks of open-ended questions Creswell Qualitative Inquiry 2e

14. Evaluation Criteria: A Methods Approach Howe & Eisenhardt 1990 • The data collection and analysis are driven by the research questions • Data collection and analysis techniques are completely applied in a technical sense • The researcher’s assumptions are made explicit including the researcher’s own subjectivity Creswell Qualitative Inquiry 2e

15. Evaluation Criteria: A Methods Approach Howe & Eisenhardt 1990 (cont.) • The study has overall warrant including being robust and using respected theoretical explanations • The study has value both in informing and improving practice Creswell Qualitative Inquiry 2e

16. Evaluation Criteria: A Postmodern, Interpretive Approach Lincoln 1995 • Standards set in inquiry community (guidelines for publication) • Positionality (“text” honest and authentic) • Community (serves community purposes) • Voice (participants heard) • Critical subjectivity (researcher heightened self-awareness/creates social transformation) Creswell Qualitative Inquiry 2e

17. Evaluation Criteria: A Postmodern, Interpretive Approach From Lincoln 1995 (cont.) • Reciprocity (between researcher and participants) • Sacredness of relationships (respect for participants) • Sharing privileges (sharing of rewards with participants) Creswell Qualitative Inquiry 2e

18. Evaluation Criteria: General, Interpretive Richardson & St. Pierre 2005 • Substantive contribution (significant understanding of social life) • Aesthetic merit (practices open up text, artistically shaped, not boring) • Reflexivity (adequate self-awareness, self-exposure to reader) • Impact (affects the reader emotionally, intellectually, moved to action) Creswell Qualitative Inquiry 2e

19. Evaluation Criteria: Methodological Approach Creswell 2007 • Rigorous data collection (multiple forms, extensive data) • Consistent with philosophical assumptions of qualitative research (evolving design, multiple perspectives) • Employs tradition of inquiry (e.g., Case Study, Grounded Theory, Narrative) • Starts with focus on central phenomenon Creswell Qualitative Inquiry 2e

20. Evaluation Criteria: Methodological Approach Creswell 2007 (cont.) • Written persuasively • Multiple levels of analysis • Narrative engages the reader • Includes strategies to confirm accuracy Creswell Qualitative Inquiry 2e

21. Evaluation Guidelines for Narrative Research • The researcher focuses on single individual (or 2 - 3) • The researcher collects stories about a significant issue related to this individual’s life • The researcher develops a chronology that connects different phrases or aspects of a story • The researcher tells a story that restories the story of the study participants Creswell Qualitative Inquiry 2e

22. Evaluation Guidelines forNarrative Research(cont.) • The researcher tells a persuasive story told in a literary way • The researcher reports themes that build from the story to tell a broader analysis • The researcher brings reflexivity into the study Creswell Qualitative Inquiry 2e

23. Evaluation Guidelines for Phenomenological Research • The researcher conveys an understanding of the philosophical tenants of phenomenology • The researcher has a phenomenon to study that is concisely articulated • The researcher uses procedures of data analysis in phenomenology • The researcher conveys the overall essence of the experience of the participants, which includes a description of the experience and the context in which it occurred Creswell Qualitative Inquiry 2e

24. Evaluation Guidelines for Grounded Theory Research • The researcher studies a process, action, or interaction as the key element of the study • The researcher uses a coding process that contains the three central areas of coding found in grounded theory, open coding, axial coding, and selective coding • The researcher presents a theoretical model in the form of a visual diagram Creswell Qualitative Inquiry 2e

25. Evaluation Guidelines for Grounded Theory Research (cont.) • The researcher develops a storyline or propositions that connect the categories in the theoretical model and that present further questions to be answered • The researcher uses reflexivity or self-disclosure about his or her stance in the study Creswell Qualitative Inquiry 2e

26. Evaluation Guidelines for Ethnographic Research • The researcher identifies a culture-sharing group • The researcher provides a detailed description of the culture-sharing group • The researcher examines themes related to the culture-sharing group Creswell Qualitative Inquiry 2e

27. Evaluation Guidelines for Ethnographic Research • The researcher addresses field issues that have surfaced • The researcher explains how the culture-sharing group works • The researcher self-discloses and is reflexive about his/her position Creswell Qualitative Inquiry 2e

28. Evaluation Guidelines for Case Study Research • The researcher provides a clear identification of the case or cases • The researcher uses multiple data sources • The researcher identifies the type of case study procedures (e.g., intrinsic, instrumental) • The researcher provides a clear description of the case Creswell Qualitative Inquiry 2e

29. Evaluation Guidelines for Case Study Research • The researcher identifies themes in the case • The researcher makes assertions or generalizations from the case analysis • The researcher is reflexive and self-disclosing Creswell Qualitative Inquiry 2e

30. Chapter 10 Standards of Validation and Evaluation Creswell Qualitative Inquiry 2e