1 / 83

Literature review designs

Learn about the three types of literature review designs, including narrative reviews and systematic reviews, and understand the biases and methodologies involved in conducting systematic reviews for evidence-based chiropractic.

fclayton
Download Presentation

Literature review designs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Literature review designs Narrative ReviewSystematic ReviewMeta-analysis

  2. Literature review • Defined as a systematic, explicit, and reproducible way of identifying, evaluating, and interpreting all of the research findings and scholarly work available on a topic • A high-quality review is not haphazard • Ideally, all of the existing work should be included • Considered descriptive or observational Evidence-based Chiropractic

  3. The three types of literature review designs Evidence-based Chiropractic

  4. Narrative reviews • Summarize in general what is in the literature on a given topic • Often written by experts in a given field • A good source for background information • Do not follow strict systematic methods like the other literature review designs • Therefore, they are prone to bias • Lower in the hierarchy of evidence Evidence-based Chiropractic

  5. Narrative reviews (cont.) • Authors like to write them because they are relatively easy to carry out and compose • Practitioners like to read them because they are easier to comprehend than more complex designs • Do not require a lot of background knowledge to understand the message Evidence-based Chiropractic

  6. Narrative reviews are prone to bias • They do not employ many of the safeguards needed to control against bias • Authors may be selective as to which articles are included • They may include articles that support their hypothesis and exclude those that do not • Rigorous appraisal methods are not used to evaluate included articles Evidence-based Chiropractic

  7. Prone to bias (cont.) • During the literature search • Authors have their own opinions on the topic and may try to find studies that support their viewpoint and overlook conflicting studies • During the synthesis of the literature • The approach to analyzing the collected information is often subjective and disorganized Evidence-based Chiropractic

  8. Prone to bias (cont.) • In the discussion and conclusion • The authors’ opinions may be mixed together with evidence • Authors may simply count the number of studies on each side of an issue and then espouse the view presented by the majority without considering the strength of each study • e.g., quality, research design used, the effect size, and sample size Evidence-based Chiropractic

  9. Selection bias innarrative reviews • a.k.a., reference bias • Occurs when authors choose articles that support their own conclusions and exclude articles with conflicting views • Results in an erroneous representation of the literature • The review may lead undiscerning readers astray Evidence-based Chiropractic

  10. Systematic reviews • Use strict methods to locate, appraise and synthesize all research on a topic • Similar to narrative reviews, but with improved procedural quality • Designed to answer specificclinical questions where several primary studies exist • Thus, a good source of clinical evidence Evidence-based Chiropractic

  11. Systematic reviews (cont.) • Articles are evaluated using appraisal instruments • In an attempt to achieve impartiality • More reproducible • Dissimilarities between the findings of studies are investigated • Multiple reviewers are usually involved • Any disagreements are resolved Evidence-based Chiropractic

  12. Systematic reviews (cont.) • The search and selection criteria for articles are well defined • Multiple databases should be searched • Explicit inclusion criteria • The results of the included studies are qualitatively or quantitatively synthesized • Qualitative – written information is merged • Quantitative – data are merged Evidence-based Chiropractic

  13. Systematic reviews (cont.) • Because of strict methodology and thoroughness, conclusions are typically less biased than narrative reviews • Nevertheless, they may still be influenced by the authors’ opinions • Still a potential for selection bias • Criteria may be applied differently when appraising included studies • Therefore, must be critically appraised Evidence-based Chiropractic

  14. Systematic reviews (cont.) • No widely accepted method exists for assessing the validity of studies • i.e., the process is a judgment call • Thus, reviewers sometimes disagree • Information derived from a systematic review may be too narrow to completely answer a specific clinical question • Given that they are designed to answer focused questions about patient care Evidence-based Chiropractic

  15. Evidence-based Chiropractic

  16. Systematic reviews typically only include RCTs • Studies are included primarily based on their quality • Most systematic reviews only include RCTs because it is the only design that adequately controls for confounding variables and biases • The potential for studies to overrate the treatment effect is higher when bias is present • Studies using other less rigorous designs are usually eliminated Evidence-based Chiropractic

  17. Conducting systematic reviews Evidence-based Chiropractic

  18. Search strategy • Should be described in enough detail so that another researcher could replicate the results, including: • Database(s) searched • Date the search was performed • Time-frame encompassed by the search • A list of search terms used • Languages Evidence-based Chiropractic

  19. Search strategy (cont.) • Conference proceedings, unpublished studies, and hand-searching of journals are sometimes included in the search • If so, the procedures involved should be described • Authors must justify using unpublished work Evidence-based Chiropractic

  20. Weighting of studies • Articles may be rejected in a systematic review due to their poor quality • Alternatively, studies are assigned weights in relation to their assessed validity • Studies that are more valid will have more influence on the review’s final results • Based on methodological quality, width of the confidence intervals, and external validity Evidence-based Chiropractic

  21. Publication bias • Studies with statistically significant results are more likely to get published than those with non-significant results • Causes of publication bias: • The author or funding source does not consider a “failed study” worthy of submission • Journals are less likely to publish studies that fail to show positive results Evidence-based Chiropractic

  22. Publication bias (cont.) • Reviews affected by this bias tend to give an overoptimistic view of the effectiveness of the therapy • The chance of this bias occurring is reduced when authors of systematic reviews search sources other than journals • Publication bias in situ • A type of bias where a portion of a study’s results are suppressed Evidence-based Chiropractic

  23. Meta-analysis • a.k.a., quantitative systematic review • A type of systematic review that statistically combines the results from a number of studies • Capable of producing a single estimate of the effect of a treatment • Represents the “average” treatment effect • An estimate of the true treatment effect size Evidence-based Chiropractic

  24. Meta-analysis (cont.) • The same explicit methods as systematic reviews are utilized • Systematic reviews and meta-analyses are at the top of the hierarchy of evidence because of their strict methodology Evidence-based Chiropractic

  25. Weighted average • Meta-analyses typically produce a weighted average for the treatment effect estimate • Small samples are more susceptible to chance variations than larger studies • Thus, they are given less weight than larger studies so they will have less influence on the final estimate Evidence-based Chiropractic

  26. Weighting (cont.) • Weighting is also based on study quality • The quality of the individual studies is rated and resulting numeric scores are calculated • A corresponding weight is assigned for each study prior to analysis Evidence-based Chiropractic

  27. Meta-analyses can increase power • Data from individual studies are combined, which in effect increases sample size • Chiropractic studies commonly involve too few subjects to detect true differences between the groups • Pooling data reduces the potential for type II error • More likely to detect a treatment effect, if there actually is one Evidence-based Chiropractic

  28. Homogeneity and heterogeneity • Homogeneity • Similarities of included studies that allow them to be compared • Homogeneity is preferred in meta-analyses • Achieved by using suitable inclusion criteria • Heterogeneity • Dissimilarities of studies that hamper or even prevent a realistic comparison of studies Evidence-based Chiropractic

  29. Factors that contribute to heterogeneity • Heterogeneity in the study samples • Caused by conflicting inclusion and exclusion criteria, differences in patients’ baseline health status, dissimilar geographical locations of groups, etc. • Heterogeneity in the study design • e.g., the way dropouts were managed in the statistical analysis or the length of time allowed for patient follow-up Evidence-based Chiropractic

  30. Factors that contribute to heterogeneity (cont.) • The way patients were handled • Regarding comorbid conditions, handling of complications, the control practitioners had in patient care, or the outcome measures used • Statistical heterogeneity • When the observed treatment effects of studies are more dissimilar than what would be expected by chance Evidence-based Chiropractic

  31. Consequences of heterogeneity • When the results of studies in a meta-analysis are inconsistent, it reduces confidence in its conclusions • The meta-analysis may actually be worthless if too dissimilar • For instance, combining studies that used different types of comparison groups • Or outcomes that were dissimilar Evidence-based Chiropractic

  32. Forest plot • A type of graph often used in meta-analyses to illustrate the treatment effect sizes of the studies • Each study is represented by a black square that is an estimate of their effect sizes • A horizontal line extends to either side of the squares, the 95% confidence interval Evidence-based Chiropractic

  33. Forest plot Evidence-based Chiropractic

  34. Interpreting a forest plot • If a study’s 95% CI crosses over the vertical line, it is not statistically significant • A diamond with a CI line is sometimes presented at the bottom of the forest plot to represent an overall estimate • The black squares may vary in size representing the weights of the studies Evidence-based Chiropractic

  35. Weighting and overall effect Evidence-based Chiropractic

  36. Effect size • The difference between the means of the treatment and control groups • When studies are combined in a meta-analysis, the units of measurement are not always comparable • Effect sizes are standardized to resolve this problem producing the standardized mean difference Evidence-based Chiropractic

  37. Standardized mean difference • The effect size divided by the pooled standard deviation • Pooled standard deviation has been adjusted for the differences in the sizes of the groups • Represents the standardized difference between group means • i.e., the relative magnitude of the experimental treatment Evidence-based Chiropractic

  38. Cohen’s d Evidence-based Chiropractic

  39. Odds ratio (OR) • Cohen’s d is appropriate with continuous data • An OR is appropriate when the study’s outcome measure is dichotomous • e.g., pain versus no pain • OR is a comparison of the odds of the outcome being present in the treatment group against the control group Evidence-based Chiropractic

  40. Relative risk (RR) • a.k.a., risk ratio • A comparison of the risk of having the outcome in the treatment group with that of the control group • Sometimes OR is reported and sometimes RR • Experts do not agree on which is most appropriate Evidence-based Chiropractic

  41. OR vs. RR • Consider a hypothetical lower back pain study with 25 patients in each group • 5 in treatment group and 10 in control group are still in pain at the study’s end OR = = = .38 RR = = = .5 Evidence-based Chiropractic

  42. Meta-analyses are most valid with RCTs • However, about half of meta-analyses include observational studies • Primarily cohort and case-control • Observational studies are much more susceptible to biases and confounding than RCTs • Therefore, it is usually inappropriate to statistically combine the results of such studies Evidence-based Chiropractic

  43. Spinal motion palpation: A comparison of studies that assessed intersegmental end-feel versus excursion • The objective of this review was to classify and compare studies based on method of MP utilized • i.e., excursion versus end-feel methods • When only high-quality studies were considered, • 3 out of 24 end-feel studies reported good reliability compared • 1 out of 15 excursion studies. • No statistical support for a difference between the two groupings Evidence-based Chiropractic

  44. Evidence-based Chiropractic

  45. Evidence-based Chiropractic

  46. Subgroup analysis • Meta-analyses typically include patients with a variety of characteristics • e.g., age, gender, condition severity, patient history, etc. • Patients in these subgroups may respond to treatment differently • e.g., low back pain patients with leg pain may respond to treatment differently than low back pain only patients Evidence-based Chiropractic

  47. Subgroup analysis (cont.) • Carried out to identify variation between patient groups regarding certain outcomes or findings • The process helps readers to distinguish the effects of a treatment between subgroups • The statistical power of the subgroups will decline as a result Evidence-based Chiropractic

  48. Meta-regression • A statistical procedure that adjusts for differences between studies in meta-analyses • May be used in subgroup analyses • Similar to simple regression • Predictor variables: characteristics of the studies • Outcome variable: treatment effect estimate Evidence-based Chiropractic

  49. Sensitivity analysis • A type of subgroup analysis that considers non-patient characteristics, e.g., treatment variations or study methodology • Determines the extent heterogeneity affected the results of a meta-analysis • If the results are weak, sensitivity analysis may reveal significant treatment effects when different methods are used Evidence-based Chiropractic

  50. Narrative versus systematic reviews and meta-analyses • There are no strict rules regarding the creation of either type • Therefore, it may be difficult to decide if a given review is systematic or narrative • Narrative reviews do not typically use systematic methods • They tend to be subjective and prone to bias • Cover broader topics than systematic reviews Evidence-based Chiropractic

More Related