Global Health Decisions: a web tool for HIV decision-making

1. Global Health Decisions: a web tool for HIV decision-making Mohsen Malekinejad MD DrPH 1 Elliot Marseille DrPH, MPP 2 James G. Kahn MD, MPH1 (PI) 1 - Philip R Lee Institute for Health Policy Studies University of California San Francisco, San Francisco, CA 2 - Health Strategies International, Oakland, CA

2. Overview • First section (~20 min) • Background • Methods • GHD website conceptual model • Q&A (10 min) • Second section (~15 min) • GHD live website tour • Case study: Ghana • Next steps

3. Background: Policy-making in HIV global settings – Challenges • Efficacy (risk reduction in ideal setting) essential but insufficient alone • Evidence on intervention effectiveness often scarce- reliance on research from less-relevant settings • Complexity and inconsistency of data reported in scientific journals is daunting • Policy-makers often lack advanced epidemiological or cost-effectiveness training

4. GHD – Goal • To Provide health policymakers around the globe with an evidence-based and easy-to-use web application that translates scientific data into the likely costs and health benefits of prevention and treatment portfolios.

5. GHD: Specific Aims • Assess evidence of effectiveness (vs. Efficacy) of HIV prevention and treatment: How well does the intervention evidence base reflect real-world practice? • Translate the effect of specified interventions implemented at large scale into reduced population-level disease burden measured in DALYs • Calculate the cost-effectiveness of sets of interventions, incorporating effectiveness, burden reduction, and cost • Make findings easily accessible to decision-makers and other technical and non-technical end users thorough a web-based resource

6. Methods – Overview Studies search & screening: systematic review of selected HIV prevention and treatment interventions Data Extraction: relevant HIV outcome and cost data Data analysis: translate outcome data into a standard metric of RRR and conduct meta-analysis Assess internal validity: rate scientific quality of evidence Assess external validity: rate relevance of evidence to a specific target setting Epi model: develop parsimonious model to evaluate intervention effects Cost-effectiveness model: calculate incremental cost-effectiveness ratio

7. Methods – Search & Screening • Search: conduct searches of systematic reviews in major biomedical and public health databases • Interventions: HIV testing and counseling, needle and syringe exchange program, opioid substitution treatment, sex worker programs, prevention of mother to child transition, antiretroviral treatment. • Screening: dual independent screening based on pre-defined inclusion and exclusion criteria in three levels: • Systematic reviews: stepwise (Title & Abstract, Full text) • hand search references of SRs to extract individual studies • Individual studies: stepwise (Title, Abstract, Full text) • Expert consultation. contact topic experts for studies or documents potentially missed by search process.

8. Methods – IOPT Extraction • extract all relevant data points at the level of Intervention, Outcome, Population Trio (IOPT) • different than conventional systematic reviews with narrow scope • Why IOPT? Studies report >1 useful data point share some overall aspects of the study (e.g., settings, investigators, etc.), but vary in respect to other aspects: • Intervention: subjects may receive different intervention due to variations in dose, frequency, and content of interventions. • Outcomes: different time-interval, or type (e.g., morbidity, mortality, behavioral) • Population: sub-analysis of findings by severity of underlying diseases (e.g., HIV serostatus)

9. Methods – Data Analysis • Studies report data in various type (RR, OR, pre-post prevalence) • transformed all data type to standardized outcome metric of “relative risk reduction” - RRR • RRR: the proportionate reduction in risk of negative health outcomes associated with the intervention – e.g.: RRR = control event rate (CER) – experimental event rate (EER) control event rate (CER) RRR = 1- RR • RRR facilitates comparison across studies and intervention types with varying levels of baseline risk • Meta-analysis: random effect model using inverse of variance to calculate summary effect measure and 95% CI

10. GHD Internal Validity (GIV) Scoring System • Objective: To assess and translates methodological aspects underlying data points into a simple, transparent, and intuitively meaningful ordinal score (1 – 6) • Modified version and hybrid product of two existing tools EPHPP and Cochrane GRADE • GIV score generated at IOPT level in three steps: • Assign an initial score based on study design (1–5) • Rate the potential risks of bias to adjust initial score • Assign an overall GIV score (0 – 6)

11. GIV - Domains • Study design (used to assign the initial score) • Comparability of study arms • Performance of intervention provider • Performance of outcome assessor • Performance of participants • Accuracy of measurement tools • Withdrawal and differential drop-out • Intervention contamination • Conflict of interest

12. GIV - Advantages Four important features representing improvement over earlier efforts: • More precise capture of differences in study design and risk of bias • Assessment of the risk of bias at the level of an intervention-outcome pair • Relatively easy to implement by research staff • Sophisticated algorithm to automate calculation of internal validity score

13. Methods – GHD External Validity (GEV) • Objective: Allow user to consider external validity in assessing body of effectiveness evidence. • Definition: Set of indicators that describe how likely it is that the results found in a study will be replicated in a target setting.

14. Methods – GEV • Indicators. Reflect geography, population, implementation details, ability to scale and legal and cultural context. • Indicator weights. Assigned weights to reflect relative importance of each indicator. • Delphi two rounds • Principle component analysis (PCA) • Meta-regression • Scoring. GEV instrument generates a score for each IOPT and for intervention as a whole.

15. Methods – Epidemiologic Model Parsimonious models that relate epidemiology, RRR and coverage, to change in disease burden for each intervention; integrated with other intervention models. Inputs • Epidemiologic data – population by risk group, prevalence and incidence. • Estimates of RRR for different outcomes. Outputs • Estimate key health outcomes, e.g., averted infections, deaths, morbidity, DALYs. Calibrated parsimonious models for each intervention against full, validated models.

16. Methods – Cost GHD developed a methodology for rating published costing studies, and used that data in a costing tool that generates costs in US $ today, adjusted from other years, other currencies, and across geographic locations. • Costs of components (e.g., personnel, goods and services) were identified and adjusted using a hierarchy of sources. • GHD generated costs of interventions • By units (per person served) • By duration (per person year of service) • By method of intervention delivery

17. Methods – Cost-Effectiveness Determining CE involves: • Summing cost across time and interventions; Calculating averted disease burden; • Comparing across intervention sets (e.g. current vs proposed); and • Calculating Incremental Cost-Effectiveness Ratio (ICER).

18. GHD – Web Conceptual Model

19. Q&A

20. www.globalhealthdecisions.org Detour – GHD LIVE

21. What’s Next for GHD? To complete data extraction and analysis (RRR, GIV, GEV) for 3-4 more interventions NIDA proposal – focusing on integration of HIV,HCV, TB, and drug abuse services for injection drug users NIMH proposal – expand data extraction for more HIV interventions and other disease areas Field test GHD with policy-makers in Africa to assess usability and uptake

22. GHD Team • Investigators: James G. Khan MD, MPH (PI) Mohsen Malekinejad MD, DrPH Elliot A. Marseille DrPH MPP Jonathan Showstack PhD MPH Ali Mirzazadeh MD PhD Senior researchers: Sabina S. Alistar PhD MS Stephane Verguet MS PhD MPP Research support: Devon McCabe MA Alex J. Goodell Pam DeCarlo BA Erin Barker MLIS Justina Wu MPH Jeff Loi MS Leon Traister BS Grant Storey Lena Libatique

23. Thank You GlobalHealthDecisions.org