1 / 42

An introduction to systematic reviews and meta-analyses

An introduction to systematic reviews and meta-analyses. Colin Josephson Assistant Professor of Neurology University of Calgary. Faculty/Presenter Disclosure. Faculty: Colin Josephson Relationships with commercial interests: Grants/Research Support: Nil Speakers Bureau/Honoraria: Nil

rhannah
Download Presentation

An introduction to systematic reviews and meta-analyses

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. An introduction to systematic reviews and meta-analyses Colin Josephson Assistant Professor of Neurology University of Calgary

  2. Faculty/Presenter Disclosure • Faculty: Colin Josephson • Relationships with commercial interests: • Grants/Research Support: Nil • Speakers Bureau/Honoraria: Nil • Consulting fees: Nil • Other: Nil

  3. Disclosure of Commercial Support • This program has received financial support from: Nil • This program has received in-kind support from: Nil • Potential conflict(s) of interest: • Nil

  4. “That’s so meta…” • Greek: prefix for ‘after’ or ‘beyond’ • English: abstraction form a concept that is used to complete or add to said concept • Unfortunately now a ‘hipsterism’

  5. Pyramid of evidence

  6. A lot of work…but for what? • Traditional ‘narrative’ reviews are at high risk of bias • Systematic reviews address a specific question in a standardised, reproducible manner • If low heterogeneity (statistical inconsistency) then aggregating data is possible

  7. What we want to avoid… Publication bias (selective reporting) Cochrane Handbook

  8. The ideal result Cochrane Handbook

  9. Normal distribution

  10. Sampling error

  11. The ‘systematic’ approach PICOST JAMA Users Guide to the Medical Literature

  12. Judge a man by his questions… • What was the question of the review? • Was it sensible, clinically relevant, and focused? • PICOST • A narrow, precise question diminishes the risk of heterogeneity, thus providing accurate conclusions

  13. Examples • What is the effect of surgery on epilepsy outcomes (e.g. seizure-freedom and quality of life)? • What is the effect of resective surgery on seizure freedom? • In patients with TLE secondary to MTS (P), what is the effect of an ATL (I) compared to medical management (C) on seizure-freedom (O) in randomised controlled trials (S) measured at one-year (T)

  14. The ‘systematic’ approach Statistical plan (PROSPERO) JAMA Users Guide to the Medical Literature

  15. Heterogeneity • Clinical: • Population, intervention, outcome (systematic bias) • Methodology: • Study design (systematic bias) • Statistical • Variability in intervention effects (random error)

  16. Means of addressing heterogeneity • Refrain from performing a meta-analysis • Explore heterogeneity through subgroup analyses or meta-regression • Pre-specify heterogeneity level (fixed-effects) • Perform a random-effects meta-analysis • Change the effect measure • Exclude studies after careful consideration

  17. Fixed effects meta-analyses • Assumes one true treatment effect • Thus, differences across trials can only be due to one source of error (i.e. random error) e.g. TLE secondary to MTS Borenstein et al., 2007

  18. Random effects meta-analyses • Assumes many treatment effects of which there is a ‘mean’ true effect e.g. epilepsy type

  19. Random effects meta-analyses • Assumes many effects of which there is a ‘mean’ true effect • Two sources of error (random error around the true effect and random error around the mean of true effects) e.g. epilepsy type Borenstein et al., 2007

  20. The ‘systematic’ approach Conducting the review JAMA Users Guide to the Medical Literature

  21. Conducting the review • Comprehensive search strategy using multiple databases • Some argue that searching MEDLINE, EMBASE, and Cochrane Central is the bare minimum • Additional sources and grey literature: • Trial registries • Reference lists • Personal communications

  22. ‘Searching ain’t easy…’ Josephson et al., Cochrane Database of Systematic Reviews 2014

  23. Databases…just a sampler • MEDLINE • PUBMED • EMBASE • Cochrane Central • TRIP • DARE • WHO ICTRP • Google Scholar

  24. Publication bias Cochrane Handbook

  25. Duplication,duplication • Involvement of two or more reviewers and abstractors ensures reproducibility of the process • Why? Because it is impossible to avoid some degree of subjectivity and, thus, error • The kappa statistic is highly informative (greater IRR suggests more confidence in the process)

  26. Kappa Byrt, Epidemiology, 1996

  27. The ‘systematic’ approach Performing the analyses JAMA Users Guide to the Medical Literature

  28. See the forest for the trees Josephson et al., Neurology, 2013

  29. Pooling the effect

  30. Mantel-Haenszelfixed effect Contingency table M-H weighted average

  31. Mantel-Haenszelfixed effect ssRR= 0.88 ssRR= 1.16 M-H weighted average: ((400*400) + (7*5))/(1000+20) 160035/1020 156.8 RR= 1 1 2 2 1 2 = = = 0.89 ((300*600) + (2*15))/(1000+20) 180030/1020 176.5

  32. DerSimonian and Laird Contingency table DSL Estimate: v = within study variance t = between study variance

  33. Was it correct to pool? Josephson et al., Neurology, 2013

  34. Measuring heterogeneity – Cochran Q Nomenclature: wi = individual study’s weight (1/v) Ti = individual study’s effect size T-bar = mean effect size • Form of chi-square test • p-value of 0.1 is typically used for significance • Caveat: • Low power = insensitive • High power = too sensitive Cochrane Handbook

  35. Measuring heterogeneity – I2 statistic • % variability in the effect estimate that is more than chance alone (i.e. due to heterogeneity rather than random error) • 0-40% = might not be important • 30-60% = may represent moderate heterogeneity • 50-90% = may represent substantial heterogeneity • 75-100% = considerable heterogeneity Cochrane Handbook

  36. Confidence = ‘GRADE’ the results GRADE Type of evidence (RCT vs. observational) Quality (blinding, allocation, f/u, sparse data, methodology) Consistency of results (within or between studies) Directness (generalisability) Effect size (e.g. >5 or <0.2 for all studies)

  37. Additional quality scales Jadad: RCTs Newcastle-Ottawa Scale: non-RCTs

  38. Quality summary table(e.g. QUADAS scale) Josephson et al., Cochrane Database of Systematic Reviews 2014

  39. Summary of findings table Cochrane Handbook

  40. Required for all studies! • Checklist outlining all essential steps of the review to facilitate quality control • www.prisma-statement.org

  41. Conclusions PICOST Statistical plan (PROSPERO) Performing the analyses JAMA Users Guide to the Medical Literature

  42. Thank you

More Related