achieving appropriate rigor in qualitative research l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Achieving Appropriate Rigor in Qualitative Research PowerPoint Presentation
Download Presentation
Achieving Appropriate Rigor in Qualitative Research

Loading in 2 Seconds...

play fullscreen
1 / 36

Achieving Appropriate Rigor in Qualitative Research - PowerPoint PPT Presentation


  • 292 Views
  • Uploaded on

Achieving Appropriate Rigor in Qualitative Research. Research Day, February 4, 2011 Mary Katherine O’Connor, Ph.D. School of Social Work, Virginia Commonwealth University mkoconno@vcu.edu. Presentation Goals.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Achieving Appropriate Rigor in Qualitative Research' - gauri


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
achieving appropriate rigor in qualitative research

Achieving Appropriate Rigor in Qualitative Research

Research Day, February 4, 2011

Mary Katherine O’Connor, Ph.D.

School of Social Work,

Virginia Commonwealth University

mkoconno@vcu.edu

presentation goals
Presentation Goals
  • Propose a multi-paradigmatic heuristic for understanding variety in qualitative research
  • Detail differential standards for research quality depending upon paradigmatic perspective
  • Aid in matching research questions, analytical needs and computer-based qualitative analysis packages
two dimensions of analysis
Two Dimensions of Analysis
  • Subjective/objective relates to assumptions about the nature of knowing (and, therefore, of science)
  • Regulation/radical change relates to assumptions about the nature of society (and, therefore, appropriate goals for science)
burrell morgan s framework

Sociology of Radical Change

Radical

Structuralist

Radical

Humanist

Subjectivity

Objectivity

Interpretive

Functionalist

Sociology of Regulation

Burrell & Morgan’s Framework
multiple paradigms
Multiple Paradigms
  • Fundamentally different perspectives for analysis of social phenomena
  • Generating different concepts and analytical tools
  • Different standards for good research practice
  • Four different types of qualitative research building from different types of research questions and research goals
slide7

Critical Interpretive

RadicalChange

Critical Positivist

Subjective

Objective

Interpretive

Positivist/

Post Positivist

Regulation

slide9

RadicalChange

Subjective

Objective

Positivist/

Post Positivist

Regulation

positivist research understanding for theory building that can result in theory testing
Positivist Research:UNDERSTANDING FOR THEORY BUILDING THAT CAN RESULT IN THEORY TESTING
  • Assumptions: Rational, pragmatic methods of natural science are appropriate to study human affairs
  • Questions: Useful for theory building in the early stages of theory development
  • Standards: understanding for generalizability including random assignment, control groups, valid and reliable data collection, standardized data analysis
slide11

RadicalChange

Critical Positivist

Subjective

Objective

Regulation

critical positivist research consciousness raising and change at the system level
Critical Positivist Research:CONSCIOUSNESS RAISING AND CHANGE AT THE SYSTEM LEVEL
  • Assumptions: Knowledge is for radical/transformative change, emancipation and potentiality; realist but concentrating on structural relationships to understand and generate fundamental conflicts that will be the basis for radical change at the class/structural level
  • Questions: Focus on structural relationships seeking to provide explanations of the basic interrelationship within the context of total social formation for the purpose of consciousness raising
  • Designs: critical ethnographies, cooperative or collaborative inquiry, participatory action research, appreciative inquiry, empowerment evaluation
  • Standards: same as functionalist qualitative inquiry, but in addition, measurable presence of increased sophistication with change or change potential
slide13

RadicalChange

Subjective

Objective

Interpretive

Regulation

interpretive research meaning making
Interpretive Research:MEANING MAKING
  • Assumptions: Understanding at the level of subjective experience from perspective of participants, not observer. Relativist in that the world, though ordered, is an emergent social enterprise that is continually being created.
  • Questions: seeking explanation within individual consciousness and subjectivity from standpoint of the participants rather than the observer
  • Designs: ethnographies, case studies, phenomenological studies, constructivist studies
  • Standards: sampling for maximum variation, emergent design, multiple data collection approaches, triangulation, reflexivity, inductive data analysis, thick description of research product
slide15

Critical Interpretive

RadicalChange

Subjective

Objective

Regulation

critical interpretive research consciousness raising and change at the individual level
Critical Interpretive Research: CONSCIOUSNESS RAISING AND CHANGE AT THE INDIVIDUAL LEVEL
  • Assumptions: Critical interpretive with radical change focus, but from a subjectivist, individualistic standpoint. Concentrating on human consciousness and the social structures that inhibit true fulfillment. Goal is release of constraints that hamper individual human development.
  • Questions: how individual makes sense of life and experiences of societal constraints
  • Designs: interview studies, narrative research, life histories, autoethnographies, collective biographies
  • Standards: more artistic or aesthetic including creativity, aesthetic quality, interpretive vitality, and degree of stimulation for change
research goals implications
Research Goals: Implications
  • All qualitative research is not based on the same ontological and epistemological assumptions
  • All qualitative research cannot be expected to be held to a universal standard of rigor, quality or worth
  • Key to justification of qualitative processes and products is selection of appropriate research design including data collection and analysis techniques able to answer research question lodged within specific paradigmatic perspective

QUESTIONS?

positivist design
Positivist Design:
  • An appropriate research design could include structured interviewing, focus group, grounded theory, etc. and should be identified and justified along with a data collection mechanism.
  • The sampling plan is preferably randomized or at least looking for maximum variation along identifiable dimensions.
  • An “n” of at least 30 or a justification for a lesser number that is similar to a justification for a lesser sample number in a quantitative project.
  • A data analytic plan (thematic, content, constant comparison, linguistic, etc.) that matches the question, including whether or not computer based text analysis is being used. When computer analysis is present, the selected program must be able to do the analysis proposed. In most cases the variables for analysis should be identified beforehand. The exception would be in grounded theory designs.
  • Inter-rater reliability should also be addressed.
critical positivist design
Critical Positivist Design:
  • An appropriate research design could include empowerment, appreciative, focus group, etc. and should be identified and justified along with a data collection mechanism.
  • The sampling plan is preferably randomized or at least looking for maximum variation along identifiable dimensions.
  • An “n” of at least 30 or a justification for a lesser number that is similar to a justification for a lesser number in a quantitative project.
  • A data analytic plan (thematic, content, constant comparison, linguistic, etc.) that matches the question, including whether or not computer based text analysis is being used. When computer analysis is present, the selected program must be able to do the analysis proposed. In most cases the variables for analysis should be identified beforehand.
  • Inter-rater reliability should also be addressed.
  • Change resulting from the research will need to be measured.
interpretive design
Interpretive Design:
  • An appropriate design could include ethnomethodological, constructivist, phenomenological, etc. and should be identified and justified. If the design is emergent, benchmarks for the emergence should be clear including how sampling and data collection will begin and might change.
  • Sampling for maximum variation is preferred and should identify the dimensions of variation of interest, the stakeholding groups to be sampled, and what might constitute saturation in determining the end of sampling and data collection.
  • A data analytic plan and the type of analysis should match the question and if computer-based analysis is being used, the program should be geared to interpretive analysis. Variables for analysis will rarely be identified beforehand; therefore, content analysis and constant comparison would be preferred.
  • Inter-rater reliability is not favored. Instead, member checking with participants is preferred.
critical interpretive design
Critical Interpretive Design:
  • An appropriate design could include an interview study, narrative research, life history, etc. and should be identified and justified. If the design is emergent, benchmarks for the emergence should be clear including how sampling and data collection will begin and might change.
  • Sampling will tend to be for a single case; but if the “n” is greater, maximum variation should identify the dimensions of variation of interest, the stakeholding groups to be sampled, and what might constitute saturation in determining the end of sampling and data collection.
  • A data analytic plan and the type of analysis should match the question and if computer-based analysis is being used, the program should be geared to interpretive analysis. Variables for analysis will rarely be identified beforehand; therefore, content analysis and constant comparison would be preferred.
  • Inter-rater reliability is not favored. Instead, member checking with participants is preferred.
  • Plans for documenting individual change as a result of the process can be noted.
paradigmatic implications
Paradigmatic Implications
  • Content Analysis

Thematic Analysis

Linguistic Analysis

Semiotics

Constant Comparison

  • Reporting

Count

Case Study

Narratives

content analysis
Content Analysis

Thematic Analysis - reading “chunks” of data for meaning and attaching a meaning label (can then move to word count/distribution analysis)

Linguistic Analysis - assessing word usage, message style (can then move to word count/distribution analysis)

Semiotics - reading for meaning through use of metaphor (can then move to word count/distribution analysis)

Constant Comparison - deconstruction and reconstruction through unitization and categorization (can then move to word count/distribution analysis)

reporting
Reporting
  • Count- data display, distribution tables, conceptual or causal maps
  • Case Study – to chronicle (to record temporally and sequentially as in a history); to render (as in a description or to provide a vicarious experience); to teach (as instructional material); to test (using case as a trial for certain theories or hypotheses)
  • Narrative – factual (he said/she said); interpretive (novel like); evaluative (elaborative judgments for evaluative testing)
computer assisted qualitative data analysis software caqdas
Computer Assisted Qualitative Data Analysis Software (CAQDAS)

Misconceptions about software:

  • A primary research instrument
  • A separate form of analysis
  • Enhances rigor
  • Packages have similar paradigmatic assumptions
data analysis with caqdas
Data Analysis with CAQDAS

Practical Considerations

  • Available technology
  • Skills and abilities
  • Time

Political Considerations

  • Institutional expectations (IRB, etc.)
  • Peer review/professional/funding expectations

Paradigmatic Considerations

  • Basic design assumptions
  • Package similarities and nuanced differences
caqdas disciplinary backgrounds
CAQDAS Disciplinary Backgrounds
  • Atlas.ti – Interdisciplinary, Psychology, Linguistics, Computer Science
  • NVivo – Sociology
  • MAXqda – Political Science
caqdas similarities
CAQDAS Similarities
  • Unitizing
  • Coding
  • Annotating
  • Retrieving
  • Querying
  • Graphic representation
  • Writing Support
caqdas differences
CAQDAS Differences

Types of data

Coding structure

Contextualization

However…

slide32

…When considering qualitative data analysis software:

Access + Awareness 

Paradigmatic Congruence

slide34

When there is congruence between paradigmatic perspective, research goal and selected processes, articulation of what can and should be expected for accountability standards are possible and usefulness can be asserted.

Positivist work cannot provide deep, individualized meaning, but can be basis for appropriately targeted quantitative work or provide evidence to support causal hypotheses.

Critical Positivist work must engender class or structural changes from objective, generalizable data.

Interpretive work is not generalizable, but can provide complex, context-based deep understanding.

Critical Interpretive work must engender individual, subjective changes at the intersection of art, spirituality and scientific ways of knowing.

protocol reviews
Protocol Reviews

(or…How do you defend what you are doing? Or assess student work?)

Try to determine underlying assumptions in order to establish the paradigmatic location of the research by evaluating the aims or goals.

Look for mixing of assumptions that suggest an incorrect understanding that qualitative research is always interpretive research

Apply the standards for design and analysis quality consistent with the paradigm.

words to the wise for the researcher
Words to the Wise for the Researcher
  • Be aware of underlying assumptions in project that establish the paradigmatic location of the research, generally best articulated by project aims or goals.
  • Avoid mixing of assumptions that suggest an incorrect understanding that qualitative research is always interpretive research.
  • Articulate and demonstrate the standards for design and analysis quality consistent with the paradigmatic perspective of the project…then choose your software