1 / 50

Critical Appraisal of Health Care Intervention Studies

Critical Appraisal of Health Care Intervention Studies. Dr. Maureen Markle-Reid, RN, MScN, PhD Associate Professor and Acting Assistant Dean (Research), School of Nursing, Associate Member, Clinical Epidemiology and Biostatistics Nursing Graduate Seminars October 4, 2011. Objectives.

michel
Download Presentation

Critical Appraisal of Health Care Intervention Studies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Critical Appraisal of Health Care Intervention Studies Dr. Maureen Markle-Reid, RN, MScN, PhD Associate Professor and Acting Assistant Dean (Research), School of Nursing, Associate Member, Clinical Epidemiology and Biostatistics Nursing Graduate Seminars October 4, 2011

  2. Objectives • Define evidence-informed decision-making and describe its importance to nursing practice • Identify the parts of a relevant, answerable question • Describe the different types of questions relevant to nursing practice • Define the term critical appraisal and discuss its relevance to nursing practice, policy and research • Identify criteria appraisal criteria for health care intervention studies • Develop skill in applying the criteria for critical appraisal of an intervention study to determine the quality and applicability of the research.

  3. What is evidence-informed decision-making? “The conscientious, explicit and judicious use of the current best evidence in making decisions about the care of individual patients” (Sackett et al, 1996)

  4. A Model for Evidence-Based Clinical Decisions Health care resources Patient preferences and actions Clinical Expertise Clinical Decision Research evidence Clinical state, setting, and circumstances [Haynes, R.B., Devereaux, P.J., & Guyatt, G.H. (2002). Clinical expertise in the era of evidence-based medicine and patient choice. ACP J Club, 136, A11-14]

  5. Why is evidence-based practice important? • Demonstrate that nursing actions and decisions are clinically appropriate and result in positive outcomes for clients, their families and health care system as a whole • Demonstrate professional accountability to clients • Provide evidence for the cost-effectiveness of nursing care

  6. Steps to Evidence-Informed Decision-Making: • Define: Formulate a focused answerable question from the practice situation based on your information needs • Search: Efficiently search for research • Appraise: Critically and efficiently appraise the research sources • Synthesize: Interpret/form recommendations for practice based on the literature • Apply: Apply the results to the client/population • Implement: Decide whether (and plan how) to implement the adapted evidence into practice • Evaluate: Evaluate the effectiveness of implementation efforts

  7. Formulate a Focused Answerable Question • Population / Situation • Intervention / Exposure • Counter intervention • Outcome • Timeframe

  8. Population/Situation: • Client group or clinical scenario of interest • Single patient or group of patients with a particular condition or health care problem, e.g.,: • A person with a health condition • People with hypertension (a group of people with a particular condition • Primary health care for the elderly (an aspect of health care delivery)

  9. Intervention/Exposure: • Interventions can be: • Therapeutic • Preventive • Diagnostic • Organizational • If exploring the meaning of a phenomena, the question may involve a situation rather than an intervention

  10. Counter Intervention: • Clinical decisions involve choosing between alternative courses of action (or no action)

  11. Outcome: • What is the outcome, or the effect we are hoping to achieve by the using the intervention? • May be more than one outcome that is important to the question

  12. Time Frame: • What is the period of time over which the question occurs? • What is the optimal time to measure a change in the outcome(s)?

  13. Step 1:Formulate a focused answerable question • Formulating Answerable Clinical Questions (Centre for Evidence-Based Medicine, Mount Sinai Hospital) • The Well-Built Clinical Question (Duke University Medical Center Library and Health Sciences Library, UNC-Chapel Hill) • Formulating Patient Centered Questions (University Library, University of Illinois at Chicago) • Asking Focused Questions (Centre for Evidence-Based Medicine, Oxford)

  14. Types of questions relevant to nursing practice • Therapy/Health care interventions: What is the effectiveness of different interventions? • Causation and harm:What might be causing disease/ill health/adverse effects? • Diagnosis or assessment: Does a diagnostic test differentiate between people with and without a condition? • Prognosis:What are potential future outcomes of a condition? • Economic evaluation: What is the cost-effectiveness of different interventions? • Meaning: Describing, exploring and explaining aspects of health and illness.

  15. Different clinical questions require evidence from different research designs Nursing Graduate Seminar 2009

  16. What type of research design will you use? All studies Q1 Theoretical (PS) Descriptive (PO) Analytic (PICO or PECO) Q2 Experimental Observational analytic Qualitative Survey (cross sectional) Q3 (Randomised) Parallel group Cohort study (Randomised) crossover Cross-sectional (analytic) Case-control study [Glasziou, P., & Heneghan, C. (2009). A spotter’s guide to study designs. EBN Notebook, 12, 71-72]

  17. Step 2: Collect the best evidence: Hierarchy of Pre-Processed Evidence EXAMPLES: SYSTEMS: Computerized decision support SUMMARIES:: Evidence-Based Guidelines ~ Evidence-Based Texts SYNOPSES OF SYNTHESES: DARE ~ Evidence-Based Abstract Journals SYNTHESES (Systematic Reviews): Cochrane Database of Systematic Reviews SYNOPSES OF SINGLE STUDIES: Evidence-Based Abstract Journals SINGLE STUDIES: Clinical Queries [Adapted from Haynes, R.B. (2007). Of studies, summaries, synopses, and systems: The “5S” evolution of information services for evidence-based healthcare decisions. Evidence-Based Nursing, 10, 6-7]

  18. Step 3: Critically appraise the literature for validity and applicability What is critical appraisal? • Critical appraisal is the process of assessing and interpreting evidence by systematically considering its validity, results and relevance to an individual's work.

  19. Relevance of Critical Appraisal to Nursing Practice, Policy and Research • Use the literature more effectively in answering clinical questions to guide clinical practice • Distinguish stronger evidence from weaker evidence – identify high quality research • Identify the methodological strengths and limitations, results and relevance of the studies in answering a clinical question

  20. Three Steps in Using an Article from the Health Care Literature • Are the results valid? • What are the results? • Are the results applicable (and useful) to my client care population? [DiCenso, A., Guyatt, G., & Ciliska, D. (2005). Evidence-based nursing: A guide to clinical practice]

  21. Are the results valid? • Are the study methods sufficiently rigorous to ensure that the study results represent an unbiased estimate of the true effect? OR • Are the study methods sufficiently biased to lead to a false conclusion?

  22. Are the results valid? • Final assessment of validity is never a yes/no decision • Validity as a continuum ranging from strong studies to weak studies • Evaluation of the validity of a study involves some subjectivity

  23. What are the results? • Size and precision of the estimate of effect

  24. Are the results applicable (and useful) to my client care population? • Can you apply the results to patients in your clinical setting? • Were all important outcomes considered? • Are the likely intervention benefits worth the potential harm and costs?

  25. Applying appropriate criteria • User Guides to Evidence-Based Practice • 1993-2000: Evidence based medicine working group: “User guides to the Medical Literature” in JAMA • 2002: Guyatt et al. “User guides to the medical literature: A manual for evidence based clinical practice” • 2005: DiCenso, Guyatt, & Ciliska. “Evidence-based nursing: A guide to clinical practice”

  26. Applying appropriate criteria • Critical Appraisal Forms: • Critical Appraisal of Articles on CAUSATION • Critical Appraisal of Articles on THERAPY/INTERVENTIONS • Critical Appraisal of SYSTEMATIC REVIEWS • Critical Appraisal of Articles on PREVALENCE AND INCIDENCE • Critical Appraisal of QUALITATIVE RESEARCH • Critical Appraisal of Articles on PROGNOSIShttp://www.cche.net/usersguides/prognosis.asp • Critical Appraisal of GUIDELINES [AGREE: Appraisal of Guidelines Research and Evaluation Instrument] http://www.agreecollaboration.org/pdf/agreeinstrumentfinal.pdf

  27. Example: The Effectiveness of a Nurse-Led Interprofessional Team Approach to Fall Prevention in Older Home Care Clients at Risk of Falling Markle-Reid, M., Browne, G., Gafni, A., Roberts, J., Weir, R., Thabane, L., Miles, M., Vaitonis, V., Hecimovich, C., Baxter, P., & Henderson, S. (2010). The effects and costs of a multifactorial and interdisciplinary team approach to fall prevention for older home care clients “at risk” for falling: A randomized controlled trial. Canadian Journal on Aging, 29(1), 139-161

  28. Critical Appraisal Criteria for Health Care Intervention Studies Screening Questions: • Did the study ask a clearly focused question? Consider if the question is ‘focused’ in terms of: • The population studied • The intervention given • The outcomes considered

  29. Research Question: • `What is the effectiveness of a 6-month nurse-led interprofessional team approach to fall prevention compared with usual home care services in older home care clients “at risk” of falling with respect to the number of falls and fall risk factors (slip or trip, health-related quality of life and function, depressive symptoms, nutritional status, gait and balance, cognitive function, fear of falling)? • PFrail older adults using home care services at risk for falling • INurse-led interprofessional team approach to fall prevention • CUsual home care services • O Number of falls, slip or trip, health-related quality of life and function, depressive symptoms, nutritional status, gait and balance, cognitive function, fear of falling • T 6 months

  30. Critical Appraisal Criteria for Health Care Intervention Studies Screening Questions: • Was this a randomized controlled trial (RCT) and was it appropriately so? Consider: • Why this study was carried out as an RCT • If this was the right research approach for the question being asked

  31. Critical Appraisal Criteria for Health Care Intervention Studies • Is it worth continuing?

  32. Critical Appraisal Criteria for Health Care Intervention Studies • How were participants allocated to intervention and control groups? Consider: • How participants were allocated to intervention and control groups. Was the process truly random? • Whether the method of allocation was described • How the randomization schedule was generated • Were the groups comparable at baseline on characteristics that might explain the outcome?

  33. Population Ineligible Eligible Sample Baseline Measures R n= Control Treatment Follow-up Measures Follow-up Measures

  34. Strengths • Random allocation of participants to groups ensures that groups are similar in all respects except exposure to the outcome; • Prospective design ensures that exposure to the intervention precedes the development of the outcome; • Greater likelihood that participants, health care providers, and outcome assessors can be blinded.

  35. Limitations • May not be ethical or feasible; • Volunteer bias limits generalizability; • Problems associated with a longitudinal design: • Time • Expense • Dropouts • Final results may not be available for several years.

  36. Critical Appraisal Criteria for Health Care Intervention Studies • Were participants, staff and study personnel ‘blind’ to participants’ study group? Consider: • The fact that blinding is not always possible • If every effort was made to achieve blinding • If you think this matters to the study • The fact that we are looking for ‘observer bias’

  37. Critical Appraisal Criteria for Health Care Intervention Studies • Was follow-up complete? Consider: • If any intervention-group participants got a control group option or vice versa • If all participants were followed up in each study group • Was there loss to follow-up bias? • If all of the participants’ outcomes were analysed by the groups to which they were originally allocated (intention to treat analysis)

  38. Eligible Clients n=267 Refused 141 Unable to Contact 13 Language 4 158 Informed consent from 40.8% of eligible applicants n=109 R Group 1 n=54 Group 2 n=55 Group Size Interprofessional Falls Prevention Team Usual Home Care Services Interventions n=5 (9.3%) n=12 (21.8%) Dropouts @ 6 month follow-up: 17 clients (15.6%) Analyzed @ 6 month follow-up: 92 clients (84.4%) n=43 (78.2%) n=49 (90.7%) Study Flow

  39. Critical Appraisal Criteria for Health Care Intervention Studies • Were participants in both groups followed up and data collected in the same way? Consider: • Was data collected in the same way and at the same time interval for both groups?

  40. Data Collection • Multiple sources of data: • In-person interviews • CCAC data • RAI-HC data • Data from service provider agencies • Measurement of Clinical Outcomes: Baseline and 6-months

  41. Outcomes Effects: Frequency and Context of Falls (Falls Surveillance Report) Functional Health Status and Quality of Life (SF-36 Health Survey) Cognitive Status (SMMSE) Depressive Symptoms (CES-D) Gait and Balance (POMA) Nutritional Status (SCREEN II) Environmental Safety (HOME FAST) Perceived Self-Efficacy (MFES) Costs: Health Services Utilization (HSSU and CCAC Utilization Data)

  42. Critical Appraisal Criteria for Health Care Intervention Studies • How are the results presented and what is the main result? Consider: • What are the results? • How large this size of result and how meaningful it is • How you would sum up the bottom-line result of the trial in a few sentences

  43. Results • At 6 months, there was nodifference in the mean number of falls between groups; • Subgroup analyses showed that the intervention was effective in reducing falls in men, 75-84 years of age, with a fear of falling, or a negative history of falls; • Greater reduction in number of slips and trips in the intervention group; • Greater improvement in role functioning related to emotional health in the intervention group; • No additional cost from a societal perspective, e.g. both interventions cost the same.

  44. Critical Appraisal Criteria for Health Care Intervention Studies • How precise are these results? Consider: • If the result is precise enough to make a decision • If a confidence interval were reported. Would your decision about whether or not to use this intervention be the same at the upper confidence limit as at the lower confidence limit?

  45. Critical Appraisal Criteria for Health Care Intervention Studies • Were all important outcomes considered so the results can be applied? Consider whether: • How the participants could be different from your population in ways that would produce different results • Your local setting differs from that of the trial • Is the intervention feasible in your setting? Consider outcomes from the point of view of the: • Individual • Policy maker and professional • Family caregivers • Wider community

  46. Critical Appraisal Criteria for Health Care Intervention Studies • Were all important outcomes considered so the results can be applied? Consider whether: • Any benefit reported outweighs any harm and/or cost • Policy or practice should change as a result of the results of this trial

  47. Methodological Strengths • Study design: randomized controlled trial • Follow-up period was appropriate • Follow-up was complete (<20% loss to follow-up) • Groups comparable at baseline

  48. Methodological Limitations • Small sample size • Volunteer bias • Potential for recall bias related to reporting falls • Strategies for collecting data regarding the number of falls

  49. Other Resources • Evidence-Informed Decision-Making Resource Modules posted on Avenue to Learn • Bandolier Evidence-Based Health Care http://www.medicine.ox.ac.uk/bandolier/ • Tutorials: • Introduction to Evidence-Based Medicine (Duke University Medical Center Library) http://www.hsl.unc.edu/services/tutorials/ebm/index.htm • Directories: • Evidence Based Medicine (EBM) Tookit (New York University School of Medicine. The Frederick L. Ehrman Medical Library): • www.urmc.rochester.edu/hslt/miner/digital_library/evidence_based_resources.cfm • Resource Guide for Evidence-Based Practice (University of Alberta Libraries) http://www.library.ualberta.ca/subject/evidence/guide/index.cfm • McKibbon, A. (1999). PDQ: Evidence-based principles and practice. Hamilton, Ontario: B.C. Decker Inc. • Evidence Based Practice Tips http://www.ebmtips.net/risk001.asp

More Related