INDEPTH Network Effectiveness and Safety Studies Platform (INESS)

1. INDEPTH Network Effectiveness and Safety Studies Platform (INESS) Introduction to Systems Effectiveness Modules Don de Savigny INDEPTH Scientific Advisory Committee Swiss Tropical Institute, Basel Pune, 28 October, 2009

2. INESS systems effectiveness objective • To assess the effectiveness, and determinants of effectiveness, of new malaria treatments in real world health systems.

3. Challenge • For INDEPTH DSS Sites… • To move beyond population health observatories to include a health system observatory function • To link population health and health behaviours to health services and to health system behaviours

4. Driving with the brakes on:How interventions lose traction in health systemsExample of ACT anti-malarial treatment in Rufiji DSS in 2006 Efficacy 98% X Access X 60% X 40% Health system factors X Diagnostics X 95% X 90% X Provider compliance X 95% X 75% X Patient adherence X 70% Averages mask inequities X 60% Effectiveness = 37% Poorest quintile = 16% Data source: IMPACT Tanzania. Effectiveness data are actual. Poorest quintile estimates are hypothetical

5. What does this mean? • Presently more traction can be gained by removing health system bottlenecks than by improving the efficacy of new drugs.

6. INESS Technical approach for systems effectiveness • Seven linked study modules provide the ingredients for the effectiveness estimation:

7. INESS: Understanding barriers to effectiveness Costs Therapeutic efficacy Compliance Adherence Access Targeting Actual Practice Effectiveness HH HF HF HH

8. Module 1. Access • Main purpose: • determine proportion of cases needing to seek care that actually gain physical access to a point of provision Quick overview: • Household surveys of fevers in prior two weeks • Determines who was able to access authorized provider within 24h • Determines reasons for choices and failed access • Analyzes across time, space, socio-economic quintiles and provider characteristics

9. Module 1. Access More details: • DSS Total Population Monitoring via three core questions for every DSS household • Any fever in prior two weeks • If yes, who (name, permanent ID) • Did he/she take an antimalarial • Provides annual pattern of fever burden • DSS Household Access Sample Survey for in-depth assessment of care seeking and access on sample of those with fever (on PDAs) • Sample size ~ 21,000 per year • 2 hh per routine DSS enumerator per week requiring full interview • Modified Malaria Indicator Survey instrument to identify: • ACT provider • Delay and sequence of care seeking • Whether any diagnostic test done for the ACT • Whether and what treatment(s) obtained • Whether full ACT course continuing or completed • Costs of episode • RDT conducted and referral if needed • Sample size ~ 1,690 per year • Powered to provide estimate +/-5% of proportion of RDT +ve febrile individuals having access to a source of ACT within 24 and 48h in both rainy and dry seasons by equity quintile.

10. Module 2 & 3. Diagnostic targeting & Provider compliance • Main purpose: • determine the proportion of cases having access that are correctly diagnosed / classified • determine the proportion of correctly diagnosed cases that are provided with the correct treatment Quick overview: • Health facility / provider surveys • Sampled at peak and low seasons • Exit interviews with gold standard diagnostic • Determines the drug and instructions provided or prescribed • Assesses stock-outs and quality of drugs on hand • Identifies cohorts for adherence follow-up survey at home

11. Module 2 & 3. Diagnostic targeting & Provider compliance More details: • Stratified sampling of ACT providers • Sample size: 1,750 patients per year over two seasons • All patients presenting as initial illness to sampled ACT provider on day of survey • Gold standard diagnostic • Patient exit interview • Pharmacy and supply inventory • Health worker interview

12. Module 4. Patient adherence • Main purpose: • Estimate proportion of patients who receive treatment who use it as intended; and the proportion who are satisfied with the treatment Quick overview: • Household survey • Sample of exit subjects from Module 3 followed at home on day after last scheduled dose (plus filter paper blood sample) • Standard interviews for adherence and acceptability • Further follow-up and filter paper blood at day 28 (and 42 depending on ACT) • Gold standard diagnostic available

13. Sample size: Adherence Three levels of adherence: • High (complete):65% of users with treatment failure of 5% • Medium: 25% of users with treatment failure of 30% • Low: 10% of users with treatment failure of 50% Calculations • Sample size required to detect a difference in treatment failure rate between the two smallest groups, medium and low. • With following parameters • Confidence level: 95% • Power 80% • Ratio unexposed (medium adherence)/ Exposed (low adherence) = 25% / 10% = 2.1 • Prevalence of disease (treatment failure rate) in Exposed group: 50% • Rate ratio = 50% / 30% = 1.67 • We need 175 in medium adherence group and 70 in low adherence group. • As the low group is expected to be 10% of all, we will need a total of 700 patients to be followed through to the last day. • This would have to be corrected upwards to account for the losses. • Perhaps to 1000 patients per treatment for each drug.

14. Module 4. Patient adherence More details: • Sample size: 400 per season • Visited one day after expected end of treatment course • Asked about: • Doses taken on each day, individually • Time specificity limited to morning, noon, afternoon, night • How drugs were taken (with food, drink, etc) • Vomiting and specifics • Pills remaining and packaging examined • Filter paper blood sample taken • 28 day interview and filter paper blood sampling visit scheduled

15. Module 5. Community acceptance • Main purpose: • Examine the social, cultural and behavioural factors that facilitate or impede uptake and adherence to new ACTs when introduced Quick overview: • Community survey of three different populations • Persons having a recent malaria fever episode (45-50 interviews) • Adult men & women living in DSS area (15 FGDs per year) • ACT providers (15-20 interviews) • Two communities <5km and two communities >5km from ACT

16. Module 6. Contexts and additional effects • Main purpose: • Estimate the contribution to reduced morbidity & mortality. • HMIS document reviews for trends and patterns in: • Proportion of fevers recorded as malaria (OPD, IPD) • Severe anemia • Incidence of severe malaria • Proportion requiring transfusion • DSS database and VA review for trends in: • All cause and malaria specific mortality • Health seeking prior to malaria death from verbal autopsy • ITNs and IRS coverage • District plan and budget reviews for trends in: • Health system changes • Malaria expenditures as a share of all expenditure • Other contextual data (rainfall, EIR, molecular markers for resistance) • Repeat therapeutic efficacy (100 patients)

17. Module 7. Overall effectiveness and costs • Main purpose: • Determine the effectiveness, and the determinants of effectiveness • Putting it all together • Determine overall population effectiveness by equity quintile • Determine the efficacy losses, and where the greatest losses occur • Determine the costs of change, comparative financial costs, and expected cost-effectiveness

18. Systems effectiveness: 20 Indicators

19. Systems Effectiveness Task Teams • Will assist with: • Development of field protocols • Piloting protocols in initial sites / countries • Developing training and capacity strengthening approaches • General oversight on module performance • Data management & analysis

20. Thank you

21. District expenditure shares – all strategies

22. District absolute annual per capita expenditure – communicable diseasesRufiji District 2007

23. Estimating District ACT requirements