Dr Shona Fielding

1. How useful is a reminder system in collection of follow-up quality of life data in clinical trials? Dr Shona Fielding

2. Outline • Background • What is quality of life (QoL)? • Missing data • Missing data mechanism • Ways of dealing with missing data • Example datasets • The reminder-system • Comparison of methods using reminder data • Conclusions

3. What is quality of life? • Measure of health status • Dimensions includes physical functioning, mental functioning, social well-being, cognitive functioning, pain • Generic measures • EuroQoL (EQ5D): overall health status (mobility, self-care, usual activities, pain/discomfort, anxiety) • Disease specific measures • Oxford Knee Score (OKS) • QLQ-C30 cancer specific questionnaire

4. Missing data • Data you expect to collect: e.g. non-response to a postal questionnaire • Missing forms or Missing Items • Why is it a problem? • Loss of power due to reduced sample sizes • Introduce bias to results • Serious problem in analysis of QoL outcomes, as missing data likely to be informative

5. Missing data mechanism Three missing data mechanisms • Missing completely at random (MCAR) • Missing at random (MAR) • Missing not at random (MNAR) Table 1: Simple overview of missing data mechanisms

6. Dealing with missing data • Complete case analysis • Imputation • Simple imputation: single value e.g. last value carried forwards (LVCF), mean imputation • Multiple imputation: several values, incorporates uncertainty • Model-based procedures • e.g. pattern mixture model (not discussed here)

7. Example Trials • RECORD trial (N=5292): • Vitamin supplementation in the elderly to prevent re-fracture • Treatment comparisons: Calcium versus No calcium • QoL: EQ5D • Analysis of covariance (ANCOVA) at 24 months adjusting for baseline QoL plus other patient variables • REFLUX (N=357) • Comparison of surgery with medical management for gastro-oesophageal reflux disease • ANCOVA at 12 months adjusting for baseline QoL, sex, age, BMI • QoL: Reflux Quality of Life Score (RQLS)

8. The reminder system • Each trial used reminder system for follow-up questionnaires, leading to three types of responders • Immediate responders (respond to initial mailing) • Reminder responders (respond following reminder) • Non-responders (not sent the questionnaire or did not return questionnaire)

9. Response Rates Table 2: Response rate foe example trials • Overall response rate (including reminders) • RECORD – 65% ; REFLUX – 89%

10. What did we do? • Use the ‘extra’ data (from reminder responses) to test procedures for dealing with missing data • Investigated the missing data mechanism • Investigated suitable imputation procedures

11. Missing data mechanism • Several methods used: • Two hypothesis tests • Two logistic regression procedures • RECORD • Non-response: MAR • Reminder response: MAR • REFLUX • Non-response: MCAR • Reminder response: MCAR Ref: Shona Fielding, Peter M Fayers, Craig R Ramsay. Investigating the missingness mechanism in quality of life data: A comparison of approaches. Health and Quality of Life Outcomes2009, 7: 57.

12. Investigating suitable imputation methods • Subset of responders was identified • Reminder-responses removed → imputation carried out → ANCOVA on imputed dataset • Result compared to the original ANCOVA result • Identify the most suitable method of imputation

13. Imputation of reminder responses • RECORD • LVCF was ‘best’ simple imputation method • Predictive mean match model was ‘best’ multiple imputation model • Imputation method did impact on the trial conclusion • REFLUX • LVCF was ‘best’ simple imputation method • Predictive mean match model was ‘best’ multiple imputation model • Imputation method affected the magnitude of treatment difference estimate but not the trial conclusion

14. Comparing analysis strategies • How does the choice of analysis strategy affect the result? • Different strategies considered • ANCOVA on immediate responses • ANCOVA on all responses (including reminder) • Repeated measures on immediate responses • Repeated measures on all responses • LVCF following immediate responses • LVCF following all responses • MI (predictive mean match) following immediate responses • MI (predictive mean match) following all responses

15. RECORD – different analysis strategies Table 3: Treatment Difference (95% CI) in EQ5D scores for different analysis strategies

16. RECORD

17. REFLUX– different analysis strategies * All p<0.001 Table 4: Treatment Difference (95% CI) in RQLS scores for different analysis strategies

18. REFLUX

19. What should you do? • No single way of dealing with missing data that is applicable in all situations • Plan study to minimise missing data • Use REMINDERS for follow up questionnaires

20. What should you do? • If still have missing data then • Identify the missing data mechanism • MCAR – complete case or simple imputation may be used • MAR – repeated measures or multiple imputation • MNAR – model-based strategy • Make use of the reminder data to help inform which particular method of imputation (if any) is appropriate

21. Conclusion • A reminder system extremely useful way of recovering data originally missing • It is a cost effective use of resources to maintain the sample size • Using reminders to minimize the amount of missing data also reduces the threat of bias • Data collected by reminders enables a more informed selection of imputation methods, which again reduces the risk of bias

22. Acknowledgements • Health Services Research Unit (HSRU) for providing the data • HSRU is funded by the Chief Scientist Office (CSO) of the Scottish Government Health Directorate • CSO for funding my Research Training Fellowship (CZF/1/31) • Project supervisors Professor Peter Fayers and Dr Craig Ramsay • Others: Jonathan Cook, Graeme Maclennan, Cynthia Fraser, Luke Vale, Samanthan Wileman, Janice Cruden, Gladys McPherson, Alison Macdonald, Seonaidh Cotton (All University of Aberdeen)

23. ANY QUESTIONS?