Chapter 12
1 / 79

Chapter 12 - PowerPoint PPT Presentation

  • Uploaded on

Chapter 12. Critical Appraisal of Quantitative and Qualitative Research for Nursing Practice. Intellectual Research Critique. A careful examination of all aspects of a study to judge: Merits Limitations Meaning Significance. Intellectual Critique Questions.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Chapter 12' - aladdin-francis

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Chapter 12

Critical Appraisal of Quantitative and Qualitative Research for Nursing Practice

Intellectual research critique
Intellectual Research Critique

A careful examination of all aspects of a study to judge:





Intellectual critique questions
Intellectual Critique Questions

Was the research problem significant?

What are the major strengths of the study?

What are the major weaknesses of the study?

Did the researchers use sound methodology?

Do the findings accurately reflect reality?

Are the findings consistent with those from previous studies?

Intellectual critique questions cont d
Intellectual Critique Questions (cont’d)

Can the study be replicated by others?

What are the implications of the findings?

Intellectual critique guidelines
Intellectual Critique Guidelines

Read and critique the entire study.

Examine the organization and presentation of the research report.

Examine the significance of the probe, studied for nursing practice.

Identify strengths and weaknesses of a study.

Be objective and realistic in identifying the study’s strengths and weaknesses.

Intellectual critique guidelines cont d
Intellectual Critique Guidelines (cont’d)

Provide specific examples of the strengths and weaknesses.

Provide a rationale for your critique.

Suggest modifications for future studies.

Discuss feasibility of replication of the study.

Discuss usefulness of findings for practice.

Critique process for quantitative studies
Critique Process for Quantitative Studies

Phase 1—Comprehension

Phase 2—Comparison

Phase 3—Analysis

Phase 4—Evaluation

Phase 1 comprehension
Phase 1—Comprehension

Read the article carefully.

Identify terms you do not understand and determine their meaning in a dictionary or the glossary of Burns and Grove.

Read the article a second time.

Highlight each step of the research process.

Phase 2 comparison
Phase 2—Comparison

Requires knowledge of what each step of the research process should be like

The ideal is compared with the real.

Must examine the extent to which the researcher followed the rules for an ideal study

Phase 3 analysis
Phase 3—Analysis

Involves a critique of logical links connecting one study element with another

Overall logical development of the study must be examined.

Critique guidelines or comparison and analysis
Critique Guidelines or Comparison and Analysis

Review research text(s).

Compare the steps in the study you are analyzing with the criteria in the research text(s).

Analyze the logical links among the steps of the study.

Research problem and purpose
Research Problem and Purpose

Is the problem sufficiently delimited in scope without being trivial?

Is the problem significant to nursing?

Is there evidence of researcher biases?

Does the purpose narrow and clarify the aim of the study?

Was the study feasible in terms of funding, expertise, subjects, facility, equipment, and ethical considerations?

Literature review
Literature Review

Does it demonstrate progressive development of ideas through previous research?

Is a theoretical knowledge base developed for the problem and purpose?

Does the literature review provide rationale and direction for the study?

Is a clear, concise summary presented of the current empirical and theoretical knowledge in the area of study?

Study framework
Study Framework

Is the framework presented with clarity?

If a map or model is presented, is it adequate to explain the phenomenon of concern?

Is the framework linked to the research purpose?

Would another framework fit more logically with the study?

Study framework cont d
Study Framework (cont’d)

Is the framework related to nursing knowledge?

If a proposition from a theory is tested, is the proposition clearly identified and linked to the study hypotheses?

Research objectives questions or hypotheses
Research Objectives, Questions, or Hypotheses

Are the objectives, questions, or hypotheses expressed clearly?

Are the objectives, questions, or hypotheses logically linked to the research purpose and framework?

Are the research objectives, questions, or hypotheses linked to concepts and relationships from the framework?


Do the variables reflect the concepts identified in the framework?

Are the variables clearly defined?

Is the conceptual definition of a variable consistent with the operational definition?


Is the design used the most appropriate to obtain the needed data?

Does the design provide a means to examine all of the objectives, questions, or hypotheses?

Have threats to design validity been minimized?

Is the design logically linked to the sampling method and statistical analyses?

Design cont d
Design (cont’d)

Is the treatment clearly described?

Was a protocol developed to promote consistent implementation of the treatment?

Did the researcher monitor the implementation of the treatment to ensure consistency?

Sample population and setting
Sample, Population, and Setting

Is the sampling method adequate to produce a representative sample?

What are the potential biases in the sampling method?

Are any subjects excluded from the study based on age, socioeconomic status, or race, without a sound rationale?

Sample population and setting cont d
Sample, Population, and Setting (cont’d)

Were the sampling criteria appropriate for the type of study conducted?

Is the sample size sufficient to avoid a type II error?

If more than one group is used, do the groups appear equivalent?

Sample population and setting cont d1
Sample, Population, and Setting (cont’d)

Are the rights of human subjects protected?

Is the setting used in the study typical of clinical settings?

Was sample mortality a problem? If so, how might this influence the findings?


Do the instruments adequately measure the study variables?

Are the instruments sufficiently sensitive to detect small differences?

Does the instrument have adequate validity and reliability?

Scales and questionnaires
Scales and Questionnaires

Are the instruments clearly described?

Are techniques to complete and score the instruments provided?

Are validity and reliability of the instruments described?

If the instrument was developed for the study, is the instrument development process described?


Are the phenomena to be observed clearly identified and defined?

Is interrater and intrarater reliability described?

Are the techniques for recording observations described?


Do the interview questions address concerns expressed in the research problem?

Are the interview questions relevant for the research purpose and objectives, questions, or hypotheses?

Does the design of the questions tend to bias subjects’ responses?

Does the sequence of questions tend to bias subjects’ responses?

Physiological measures
Physiological Measures

Are the physiological measures or instruments clearly described? If appropriate, are brand names of instruments identified?

Are the accuracy, selectivity, precision, sensitivity, and error of the instruments discussed?

Physiological measures cont d
Physiological Measures (cont’d)

Are the physiological measures appropriate for the research purpose and objectives, questions, or hypotheses?

Are the methods for recording data from the physiological measures clearly described?

Is the recording of data consistent?

Data collection
Data Collection

Is the data collection process clearly described?

Is the training of data collectors clearly described and adequate?

Data collection cont d
Data Collection (cont’d)

Is the data collection process conducted in a consistent manner?

Are the data collection methods ethical?

Do the collected data address the research objectives, questions, or hypotheses?

Data analysis
Data Analysis

Are data analysis procedures appropriate to the type of data collected?

Are data analysis procedures clearly described?

Are the results presented in an understandable way?

Do data analyses address each research objective, question, or hypothesis?

Are the analyses interpreted appropriately?

Data analysis cont d
Data Analysis (cont’d)

Are the statistical analyses logically linked to the design?

Is the sample size sufficient to detect significant differences?

Was power analysis used to determine sample size?

Interpretation of findings
Interpretation of Findings

Are findings discussed in relation to each objective, question, or hypothesis?

Are the findings clinically significant?

Do the conclusions fit the findings from the analyses?

Are conclusions based on statistically significant and clinically significant results?

Are there limitations the researcher did not identify?

Phase 4 evaluation
Phase 4—Evaluation

Involves determining the meaning and significance of the study by examining the links among the study process, study findings, and previous studies

Study findings are examined in light of previous study findings.

Evaluation builds on conclusions reached during the first three stages of the critique and provides the basis for the fifth step—conceptual clustering.

Phase 4 evaluation cont d
Phase 4—Evaluation (cont’d)

The steps of the study are evaluated based on previous studies.

Present hypotheses are based on previous hypotheses.

Present design is based on previous designs.

Present methods of measurement are based on previous measurement.

Critique guidelines for evaluation
Critique Guidelines for Evaluation

What rival hypotheses can be suggested for the findings?

How much confidence can be placed in the study findings?

To what populations can the findings be generalized?

What questions emerge from the findings, and are these identified by the researcher?

Critique guidelines for evaluation cont d
Critique Guidelines for Evaluation (cont’d)

What future research can be envisioned?

Could the limitations of the study have been corrected?

When the findings are examined based on previous studies, what is now known and not known about the phenomenon under study?

Critique guidelines for evaluation examination of previous studies
Critique Guidelines for Evaluation—Examination of Previous Studies

Are the findings of previous studies used to generate the research problem and purpose?

Do the findings build on findings of previous studies?

Is the design an advance over previous designs?

Do sampling strategies show an improvement over previous studies?

Critique guidelines for evaluation examination of previous studies cont d
Critique Guidelines for Evaluation—Examination of Previous Studies (cont’d)

Does the sample selection have the potential for adding diversity to samples previously studied?

Does the current research build on previous measurement strategies so that measurement is more precise or more reflective of the variables?

How do statistical analysis techniques compare with those used in previous studies?

Critique guidelines for evaluation examination of previous studies cont d1
Critique Guidelines for Evaluation—Examination of Previous Studies (cont’d)

Is the current knowledge in this area identified?

Does the author indicate the implication of the findings for practice?

Skills needed to critique qualitative studies
Skills Needed to Critique Qualitative Studies Studies (cont’d)

Context flexibility

Inductive reasoning

Conceptualization, theoretical modeling, and theory analysis

Transforming ideas across levels of abstraction

Context flexibility
Context Flexibility Studies (cont’d)

Definition: the capacity to switch from one context or worldview to another, to shift perception so as to see things from a different perspective

It is not necessary to become committed to a perspective to follow or apply its logical structure.

Context flexibility cont d
Context Flexibility (cont’d) Studies (cont’d)

All scholarly work requires a willingness and ability to examine and evaluate works from diverse perspectives.

For example, analysis of the internal structure of a theory requires context flexibility.

Inductive reasoning skills
Inductive Reasoning Skills Studies (cont’d)

Necessary so as to follow the logic of a qualitative researcher

Used in the transformation process during data analysis

Revealed in the move from concrete descriptions to the abstract level of science

Standards for qualitative critique
Standards for Qualitative Critique Studies (cont’d)

Standard 1: Descriptive vividness

Standard 2: Methodological congruence

Standard 3: Analytical and interpretative


Standard 4: Philosophical or theoretical


Standard 5: Heuristic relevance

Standard 1 descriptive vividness
Standard 1—Descriptive Vividness Studies (cont’d)

Description of the site and subjects, the experience of collecting the data, and the thinking of the researcher during the process need to be presented so clearly that the reader has the sense of personally experiencing the event.

Because one of the assumptions of qualitative research is that all data are context specific, the evaluator of a study must understand the context of that study.

Descriptive vividness cont d
Descriptive Vividness (cont’d) Studies (cont’d)

A contextual understanding of the whole is essential and prerequisite to the capability of the reviewer to evaluate the study in the light of the other four standards.

Threats to descriptive vividness
Threats to Descriptive Vividness Studies (cont’d)

Failure to include essential descriptive information

Lack of clarity and/or depth of description

Inadequate skills in writing descriptive narrative

Reluctance to reveal self in written material

Standard 2 methodological congruence
Standard 2—Methodological Congruence Studies (cont’d)

Reviewer must have knowledge of philosophy and methodological approach used by the researcher.

Four dimensions:

Rigor in documentation

Procedural rigor

Ethical rigor


Rigor in documentation
Rigor in Documentation Studies (cont’d)

The reviewer examines if the researcher clearly and concisely presents study elements.

The reviewer examines the study elements for completeness and clarity.

The reviewer identifies any threats to rigor in documentation.

Qualitative study elements
Qualitative Study Elements Studies (cont’d)



Research question

Significance of phenomenon

Identification of assumptions

Identification of philosophy

Researcher credentials

Qualitative study elements cont d
Qualitative Study Elements (cont’d) Studies (cont’d)


Role of researcher

Ethical implications

Sampling and subjects

Data-gathering strategies

Data analysis strategies

Qualitative study elements cont d1
Qualitative Study Elements (cont’d) Studies (cont’d)

Theoretical development



Suggestions for further study

Suggestions for practice

Literature review

Threats to rigor in documentation
Threats to Rigor in Documentation Studies (cont’d)

Failure to present all elements or steps of the study

Failure to present all elements or steps of the study accurately or clearly

Procedural rigor
Procedural Rigor Studies (cont’d)

Definition: the rigor of the researcher in applying selected procedures for the study

The researcher needs to make clear the steps taken to ensure data were accurately recorded and that the data obtained are representative of data as a whole.

When critiquing a qualitative study, the reviewer examines the description of the data collection process and study findings for threats to procedural rigor.

Threats to procedural rigor
Threats to Procedural Rigor Studies (cont’d)

The researcher asked the wrong questions. The questions need to tap the subjects’ experiences, not their theoretical knowledge of the phenomenon.

The questions included terminology from the theoretical orientation of the researcher.

Threats to procedural rigor cont d
Threats to Procedural Rigor (cont’d) Studies (cont’d)

The informant might have misinformed the researcher.

Ulterior motive

Other individuals present might have inhibited the informant.

The informant might have wanted to impress the researcher by giving the most desirable response.

Threats to procedural rigor cont d1
Threats to Procedural Rigor (cont’d) Studies (cont’d)

The informant did not observe the details requested or was not able to recall the event and substituted instead what he or she supposed happened.

The researcher placed more weight on data obtained from well-informed, articulate, high-status individuals than on data from those who were less articulate, obstinate, or of lower status.

Threats to procedural rigor cont d2
Threats to Procedural Rigor (cont’d) Studies (cont’d)

The presence of the researcher distorted the event being observed.

The researcher’s involvement with the subject-participants distorted the data.

Atypical events were interpreted as typical

The informants lacked credibility.

Threats to procedural rigor cont d3
Threats to Procedural Rigor (cont’d) Studies (cont’d)

An insufficient amount of data were gathered.

An insufficient length of time was spent in data gathering.

Approaches for gaining access to the site and/or subjects were inappropriate.

The researcher failed to keep in-depth field notes.

Ethical rigor
Ethical Rigor Studies (cont’d)

Requires recognition and discussion by researcher of ethical implications related to study

Consent is obtained from subjects and documented.

Report must indicate that the researcher ensured rights of subjects were protected.

Reviewer examines data-gathering process and identifies potential threats to ethical rigor.

Threats to ethical rigor
Threats to Ethical Rigor Studies (cont’d)

Researcher failed to inform subjects of their rights.

Researcher failed to obtain consent from subjects.

Researcher failed to ensure protection of subjects’ rights.

Auditability Studies (cont’d)

Definition: the rigorous development of a decision trail

To achieve this, the researcher must report all decisions involved in transformation of data to theoretical schema.

A second researcher, using original data and decision trail, should be able to arrive at conclusions similar to those of original researcher.

Reviewer examines decision trail for threats to auditability.

Threats to auditability
Threats to Auditability Studies (cont’d)

Description of data collection process was inadequate.

Researcher failed to develop or identify decision rules for arriving at ratings or judgments.

Researcher failed to record nature of decisions, data on which they were based, and reasoning that entered into decisions.

Threats to auditability cont d
Threats to Auditability (cont’d) Studies (cont’d)

Evidence for conclusions was not presented.

Other researchers were unable to arrive at similar conclusions after applying decision rules to the data.

Standard 3 analytical and interpretative preciseness
Standard 3—Analytical and Interpretative Preciseness Studies (cont’d)

Requires researcher to make intense efforts to identify and record decision-making processes through which data transformations are made

Analytical preciseness
Analytical Preciseness Studies (cont’d)

Premature patterning may occur before the researcher can logically fit all of the data within the emerging schema.

The consequence may be a poor fit between data and theoretical schema.

It is critical to test the schema by rechecking the fit between the schema and the original data.

When critiquing study, the reviewer examines decision-making processes and theoretical schema to detect threats to analytical preciseness.

Threats to analytical preciseness
Threats to Analytical Preciseness Studies (cont’d)

Interpretive statements do not correspond with findings.

Categories, themes, or common elements are not logical.

Samples are not representative of the class of joint acts referred to by the researcher.

Set of categories, themes, or common elements fails to set forth a whole picture.

Set of categories, themes, or common elements is not inclusive of data that exist.

Threats to analytical preciseness cont d
Threats to Analytical Preciseness (cont’d) Studies (cont’d)

Data are inappropriately assigned to categories, themes, or common elements.

Inclusion and exclusion criteria for categories, themes, or common elements are not consistently followed.

Working hypotheses or propositions are not identified or cannot be verified by data.

Various sources of evidence fail to provide convergence.

Threats to analytical preciseness cont d1
Threats to Analytical Preciseness (cont’d) Studies (cont’d)

There is incongruence of evidence.

Subject-participants fail to validate findings.

Conclusions are not data based or do not encompass all the data.

Data are made to appear more patterned, regular, or congruent than they are.

Standard 4 philosophical or theoretical connectedness
Standard 4—Philosophical or Theoretical Connectedness Studies (cont’d)

Requires that theoretical schema developed from the study be:

Clearly expressed

Logically consistent

Reflective of the data

Compatible with knowledge base of nursing

Threats to theoretical connectedness
Threats to Theoretical Connectedness Studies (cont’d)

Findings are trivialized.

Concepts are inadequately refined.

Concepts are not validated by data.

Set of concepts lacks commonality.

Relationships between concepts are not clearly expressed.

Threats to theoretical connectedness cont d
Threats to Theoretical Connectedness (cont’d) Studies (cont’d)

Proposed relationships among concepts are not validated by data.

Working propositions are not validated by data.

Data are distorted during development of the theoretical schema.

Theoretical schema fails to yield a meaningful picture of the phenomenon studied.

A conceptual framework or map is not derived from the data.

Standard 5 heuristic relevance
Standard 5—Heuristic Relevance Studies (cont’d)

Reflects reader’s capacity to recognize phenomenon described in the study, its theoretical significance, applicability to nursing practice situations, and influence on future research activities

Intuitive recognition: when individuals are confronted with theoretical schema derived from the data, it has meaning within their personal knowledge base

Threats to intuitive recognition
Threats to Intuitive Recognition Studies (cont’d)

Reader is unable to recognize phenomenon.

Description is not consistent with common meanings.

Theoretical connectedness is lacking.

Relationship to existing body of knowledge
Relationship to Existing Body of Knowledge Studies (cont’d)

There should be intersubjectivity with existing theoretical knowledge in nursing and previous research.

When critiquing, reviewer examines strength of the link of study findings to existing knowledge.

Threats to relationship to existing body of knowledge
Threats to Relationship to Existing Body of Knowledge Studies (cont’d)

Researcher failed to examine existing body of knowledge.

Process studied was not related to nursing and health.

There is a lack of correspondence with existing knowledge base in nursing.

Applicability to nursing practice research or education
Applicability to Nursing Practice, Research, or Education Studies (cont’d)

Applicability: nurses need to be able to integrate the findings into their knowledge base and apply them to nursing practice situations

Findings also need to contribute to theory development.

Threats to applicability
Threats to Applicability Studies (cont’d)

Findings are not significant for discipline of nursing.

There is a failure to achieve methodological congruence.

There is a failure to achieve analytical precision.

There is a failure to achieve theoretical connectedness.