CONTEXT & PROCESS Meeting of the Cochrane Public Health Editorial Group

1. The following slides were presented at a meeting of potential editors and methods advisors for the proposed Cochrane review group in February 2008. The slides were designed to promote discussion rather than represent the views and directions of this group.

2. Population Health Intervention Research Centre ww.ucalgary.ca/phirc CONTEXT & PROCESSMeeting of the Cochrane Public Health Editorial Group Penny Hawe, Professor, AHFMR Health Scientist & Markin Chair in Health and Society

3. PREMISE REGARDING CONTEXT • Effectiveness of some interventions will depend on the interaction between the intervention and the context into which it is implemented … … context is an effect modifier • Context includes the social, economic, political, and organisational characteristics in host setting • Need to disentangle program effects from (program x context) interaction effects to discern ‘under what circumstances’

4. PREMISE REGARDING PROCESS FAILURE TO SHOW AN EFFECT • Evaluation failure • failed to show an effect when one was there • Programme failure • Inadequacy of intervention theory • Poor implementation (fidelity, intensity, consistency) • Contextual factors • Process measures need to look beyond programme reach (messages delivered and received) to consider programme stability, intensity, and consistency with theory

5. MEASUREMENT OF CONTEXT • Importance of context is recognised in health promotion / health education literature especially in sustainability / institutionalisation literature … but mainly qualitative • Largely unrecognised in RCTs … even in cluster RCTs, which in theory could wash out context level effects • Recent acknowledgement of importance of measuring and reporting context effects to improve external validity of evidence (call for improved reporting standards)

6. CONTEXTUAL FACTORS (HPPH Field) • Availability, accessibility of health-promoting / health harming factors (price of fruit, number of liquor outlets in a neighbourhood, worksites that ban smoking) • Aspects of host organisation (staff numbers, skills, morale, competing priorities, history of innovation, status of programs within organisational hierarchy, resources made available, what programs are displaced) • Aspects of system (reward structures, discretion, autonomy) • Characteristics of population (literacy, prevalence of problem) • Characteristics of social or organisational networks (size, reach, density, centrality of key agencies) into which the intervention is placed

7. EXTERNAL VALIDITY (Green & Glasgow) • OUTCOMES FOR DECISION MAKING • Significance to decision makers • Adverse consequences • Moderator effects including sub-group analysis • Sensitivity: dose-response, threshold effects • Costs: economic vs accounting • MAINTENANCE AND INSTITUTIONALISATION • Long term outcomes 12 months post intervention • Institutionalisation: sustainability of intervention post evaluation • Attrition

8. EXTERNAL VALIDITY (Green & Glasgow) • REACH AND REPRESENTATIVENESS • Participation rate • Target audience stated • Representativeness - settings to target settings and to refusers • Representativeness - individuals to target population and refusers • PROGRAM OR POLICY IMPLEMENTATION • Consistent implementation • Staff expertise • Program adaptation • Mechanisms (processes and mediating variables)

10. Oral and written narratives from field workers • Key informant interviews • Event or impact logs • Organisational network analysis before and after • Additional resources provided to project

11. Modo Pergite Natare