Split-mouth studies and systematic reviews

1. ICEBOH Split-mouth studies and systematic reviews Ian Needleman1 & Helen Worthington2 1Unit of Periodontology UCL Eastman Dental Institute International Centre for Evidence-Based Oral Health, London UK 2School of Dentistry, University of Manchester, UK

2. Split-mouth studies and systematic reviews • What is the issue? • Why include them? • How to include them • Examples

3. Split-mouth studies and systematic reviews – the analysis issue • As we know, the analysis of split-mouth and parallel group studies is not the same. • As a result, if a meta-analysis includes both types of trials without considering the differences, the result might be unreliable • The confidence interval will be incorrect, possibly leading to; • An inappropriate conclusion on clinical importance (and statistical significance) • Distortion of impact of clinical heterogeneity

4. Why include split-mouth studies in systematic reviews? • Because of the totality of the evidence • Possible advantages of split-mouth trials over parallel group; • Each participant acts as own control • Therefore, fewer participants are required to obtain same study power as parallel group • Every participant receives each intervention, therefore good for determining preferences

5. How to include split-mouth studies in systematic reviews Designing the systematic review: • Is split-mouth an appropriate design to answer this question? • Are carry-over effects a risk?

6. How to include split-mouth studies in systematic reviews Conducting the systematic review: • Fundamental question: Is meta-analysis justified in principle? • Are the trials similar enough in chief characteristics: • Types of populations • Types of interventions • Types of outcomes

7. How to include split-mouth studies in systematic reviews Possible ways: • Narrative (qualitative) summary in evidence tables only. • Advantage: split-mouth studies contribute to totality of evidence and analytic issues (may be) avoided • Disadvantage: Do not contribute to summary estimate or to investigation of heterogeneity • Analyse as if parallel group • Not recommended due to potentially unreliable meta-analysis summary estimate 

8. How to include split-mouth studies in systematic reviews Possible ways: • Meta-analyse those split-mouth trials with adequate data separately from parallel group trials and ignore those without such data • Advantage: More information • Disadvantage: Selection bias • Incorporate data from first intervention side if reported separately • Advantage: More information • Disadvantage: May be biased sample where trialists identified carry-over effects

9. How to include split-mouth studies in systematic reviews Possible ways: • Approximate a paired analysis by inputing a measure describing the similarity of outcomes within each participant. • Advantages: Makes use of all trials • Disadvantages: • May make assumptions about data (that can be tested) • May need statistical support

10. Approximating a paired analysis - you will need one of the following • Individual patient data in publication or from contact with trialist • Mean and SD/SE of patient specific differences between intervention A and B measurement

11. Approximating a paired analysis - you will need one of the following • Mean difference (or difference between means) and one of: • t-statistic (paired t-test) • P-value from paired t-test • Confidence interval from paired analysis • Graph of measurement of intervention A and B from which matched individual data values can be extracted.

12. Approximating a paired analysis • Step one. Calculate the correlation coefficient (r) for each study. • The correlation coefficient describes how similar the measurement of intervention A and B were within a participant • Assumes that mean and SD for each intervention treatment side/period would be the same • If r is inconsistent between studies, then caution on proceeding further. • If r cannot be calculated for a trial, use representative value from other trials as the trials should be very similar for a particular intervention in a systematic review

13. Approximating a paired analysis • Step two. Calculate SE of the mean difference between the interventions • Step three. Enter the data into meta-analysis software. Generic inverse variance method of Cochrane Software - RevMan - particularly flexible (free download). • Step four. Conduct sensitivity analyses employing different values for r to investigate robustness of estimates especially for studies were r could not be calculated

14. How to include split-mouth studies in systematic reviews Further issues: • Generally advisable to meta-analyse split-mouth and parallel group trials separately as sub-groups to investigate systematic differences • Authors of systematic reviews: please state explicitly how data from split-mouth studies has been managed

15. How to include split-mouth studies in systematic reviews Example of meta-analysis of split mouth and parallel group trials.

16. Needleman, Worthington, Giedrys-Leeper, & Tucker 2006, Cochrane Database of Systematic Reviews

17. Split-mouth studies and systematic reviews • Conclusions • Split-mouth studies should be included in systematic reviews when appropriate • It is possible to combine split-mouth and parallel group studies in meta-analysis • Our observation within the Cochrane Oral Health Group is that differences in effect sizes exist between split-mouth and parallel group studies of the same intervention

18. ICEBOH Split-mouth studies and systematic reviews • Resources • Cochrane Handbook and RevMan: www.cochrane.org • Elbourne et al. 2002. Int J Epidem, 31: 140-149 • Follman et al. 1992. J Clin Epidem, 45: 769-773 • Contact: i.needleman@eastman.ucl.ac.uk