Laureate Professor Rob Sanson-Fisher University of Newcastle

1. Mixed Methods in Prevention and Health Services Research:Evaluation of prevention in public health Laureate Professor Rob Sanson-Fisher University of Newcastle

2. Why Evaluate? • Need to ensure interventions are: Cost-effective Effective Equitable Not harmful

3. Lobotomy Egas Monitz and Walter Freeman 1930s – 1950’s • Used to treat a range of mental illnesses • Not based on scientific evidence • Resulted in significant harm to thousands

4. Efforts to improve general practice care: the last 30 years… Training and education Staffing changes System changes • Education about smoking and alcohol in undergraduate medical curriculum • Vocational registration • Compulsory Continuing Professional Development • Move from solo to group practices • Practice managers • Practice nurses • Computerization of medical records Financial incentives Policies and guidelines Treatment changes • RACGP Redbook guidelines for preventive care • Divisions of General Practice and Medicare Locals with a focus on chronic disease prevention and management • Focus on SNAP risk factors • Practice Incentive Program (PIP)-incentives related to smoking, diabetes etc • Practice nurse incentives • Payments for preparation of chronic disease care plans, preventive health assessments • Better availability of treatment options- NRT and Champix for smoking cessation

5. Accuracy of GP detection Consent rates in parentheses % At-risk patients correctly detected by GP

6. Outcomes for Indigenous Australians • Despite efforts and investment with the aim of improving the health of Aboriginal and Torres Strait Islander people, a notable gap still exists in health outcomes between Indigenous and non-Indigenous Australians • Chronic diseases (e.g. circulatory disease, cancer) are the main contributors to the mortality gap • Need for effective interventions for prevention of chronic diseases for this vulnerable group AIHW, 2014

7. Life Expectancy AIHW, 2014

8. Mortality AIHW, 2014

9. Three Fundamental Questions Research designs must be able to answer: 1 2 3 Has a change occurred? Did the change occur as a result of the intervention? Is the degree of change significant and important to stakeholders?

10. Theories of Human Behaviour • Historically, the field was dominated by theories with an individual and interpersonal focus • Freud (psycho-analytic theory) • Rogers (humanistic theory) • Jung (analytical psychology) • Later models of Human behaviour have a social focus • Social Learning Theory • Health Belief Model • Theory of Planned Behaviour

11. Changes in Theories of Human Behaviour • Recognise the influence of health systems and social factors on health behaviour • Shift towards multi-faceted approaches that address factors at the individual, social, and system levels • PRECEDE-PROCEED model (L Green) • Behaviour Change Wheel (S Michie)

12. Need for system-level interventions • Health behaviour is complex • Whole-system, multilayered interventions are needed in order to change behaviour • Need a research design with the complexity and flexibility to evaluate system-level behaviour change interventions

13. Need to minimise bias • Bias is anything that introduces systematic variation in the data, resulting in deviation from ‘truth’ • Sources of bias include: • Selection bias • Performance bias • Detection bias • Attrition bias • Reporting bias • Contamination

14. Randomised Controlled Trial (RCT) • Considered the ‘gold standard’ for evaluating intervention effectiveness • Individuals randomly intervention or control conditions • Randomisation eliminates selection bias and variation between conditions Assessed for Eligibility Participant Consent Baseline Outcome Measurement Randomisation Intervention Condition Control Condition Follow-up Outcome Measurement

15. Advantages and Disadvantages

16. Reducing smoking among pregnant Aboriginal and Torres Strait Islander Women • Randomised controlled trial • Aim: To determine the effectiveness of an intensive quit-smoking intervention on smoking rates among pregnant Aboriginal and Torres Strait Islander women (n=263) • No significant differences found between groups Eades et al. 2012

17. Assessed for Eligibility ATSI women attending first antenatal appointment, <20 weeks gestation, current smokers/recent quitters Participant Consent Baseline Outcome Measurement Social and demographic status, smoking history Randomisation by week Control Usual Care Intervention Tailored quit smoking advice from GP Evidence-based communication skills Engaging significant others NRT after two failed quit attempts Follow-up Outcome Measurement Self-reported smoking status (validated with urine cotinine measurement) Between 36 weeks gestation and delivery

18. Reducing smoking among pregnant Aboriginal and Torres Strait Islander Women • Key challenges: • More recent quitters in the intervention group than control • High rate of loss to follow-up (>30%) • Lack of allocation concealment • Contamination between groups • Poor adherence to study protocol – high staff turnover

19. Cluster RCT Assessed for Eligibility • Type of RCT • Groups/settings randomly intervention or control conditions Groups consent Groups matched Baseline Outcome Measurement Randomisation Intervention Condition Control Condition Follow-up Outcome Measurement

20. Advantages and Disadvantages

21. Alcohol Action in Rural Communities • Matched-pair cluster RCT • Aim: To evaluate the effectiveness of a community action strategy aimed at reducing alcohol misuse and alcohol-related harm • Randomization of 20 pair-matched rural communities • Thirteen individual interventions were implemented systematically over five years • Harm reduction strategies • Demand reduction strategies Shakeshaft et al. 2013

22. Alcohol Action in Rural Communities • Intervention communities showed: • 20% reduction in average alcohol consumption • 42% reduction in experience of alcohol fuelled verbal abuse • 33% reduction in alcohol-related street offences • 30% reduction in the number of people drinking at high-risk levels • Key challenges: • High cost • High level of expertise required • Long-term commitment required from researchers and community partners

23. Alternative Research Designs • Pragmatic research designs can be more useful in evaluating public health interventions • When selecting research designs, need to consider trade-offs between: • Internal validity • External validity • Cost effectiveness • Acceptability • Feasibility

24. Stepped Wedge • Intervention implemented sequentially in all groups • Order in which groups receive the intervention is randomized • May be implemented in multiple sites at same time • Outcomes are measured pre-interventionand post-intervention (prior to implementation at the next site) Sanson-Fisher et al. 2014

25. Advantages and Disadvantages

26. Interrupted Time Series • Intervention delivered to a single group/site • Multiple observations pre- and post-intervention • Group acts as own control • Change in level = Change in outcome post-intervention • Change in slope = A different trend in outcome post-intervention Sanson-Fisher et al. 2014

27. Advantages and Disadvantages

28. Multiple Baseline • Intervention implemented in all sites in a phased approach • Time of intervention implementation is randomised • Multiple observations pre- and post-intervention Sanson-Fisher et al. 2014

29. Advantages and Disadvantages

30. Chronic Care Service Enhancements Program Evaluation • Matched-pair multiple baseline design • Aim: To reduce the burden of chronic disease amongst NSW Aboriginal people by partnering with ACCHSs to develop, implement and evaluate improvement strategies 1. Improving chronic disease management 2. Improving preventative health screening 3. Increasing reach of ACCHSs 4. Improving data quality Developing diabetes register and follow-up protocol Training in diabetes care Implementation across 6 matched ACCHSs Training in screening protocols Routine health assessments Establishing a reminder system Implementation across 5 matched ACCHSs ACCHS open day (presentations, health stalls, BBQ) Tailored re-engagement strategies and care plans Implementation across 8 matched ACCHSs Data quality audit and training Implementation across 6 matched ACCHSs

31. “Blue Highways” on the NIH Roadmap Westfall et al. 2007

32. Summary Research designs must be able to answer: • Has change occurred? • Did the change occur as a result of the intervention? • Is the degree of change significant and important to stakeholders? • Need a research design with the complexity and flexibility to evaluate system-level behaviour change interventions • Individual RCT • Cluster RCT • Stepped Wedge • Interrupted Time Series • Multiple Baseline Design