Longitudinal Methods for Pharmaceutical Policy Evaluation

1. Longitudinal Methods for Pharmaceutical Policy Evaluation Dennis Ross-Degnan, ScD Harvard Medical School and Harvard Pilgrim Health Care WHO Collaborating Center in Pharmaceutical Policy Boston, USA

2. Session Objectives • Touch on key methodological issues in longitudinal studies to evaluate: • Pharmaceutical policy changes • Planned interventions • Hear experiences of researchers who have used longitudinal data in a range of settings • Introduce commonly-used statistical methods • Interrupted time series and survival analysis • Discuss • Other experiences and perspectives • Best practices and areas for methods development

3. Using Routine Data for Pharmaceutical Policy Research • Pharmacy procurement and sales • Public, mission, private sector • Centralized, supply chain, institutional • Volume, cost • Clinical care and pharmacy dispensing • Inpatient, outpatient, retail pharmacy • Electronic records • Manual systems • Insurance reimbursement • Claims, adjudicated payments • Critical Issues • Completeness • Consistency • Coding

4. Common Methodological Issues in Longitudinal Policy Evaluations • Time • Study design • Sample selection • Data quality • Data organization • Statistical approach

5. Issues Related to Time • Key analytic variable for longitudinal research • Errors common: recording, coding • Importance of definitions (e.g., medication gaps) • Defining policy change point • Single point in time, instantaneous effects • Implementation spread over time • Co-interventions • Dynamics of policy impacts • Anticipatory changes, lagged response • Non-linear changes • Study period and unit of aggregation • Depends on data source and sample size • Optimal number of data points per policy period?

6. Issues in Study Design • Appropriate study units • Whose behavior will change? • External policy influences • Timing of implementation (prospective) • Opportunity for randomization? • Staggered implementation? • Comparisons and contrasts • Challenge of identifying similar groups or behaviors unaffected by intervention • Intended and unintended effects • High vs. low risk

7. Issues in Sample Selection • Facilities, prescribers, patients • Optimal sample structure? • Importance of denominators, continuity • Defining prevalent and incident diagnoses • Medications • Trade-offs among therapeutic alternatives • All vs. selected categories • How many is enough? • Representativeness • Need for precision • Problem of clustering

8. Issues in Data Quality • Many challenges in using routine data • Usually not collected for research • Changes in data systems or routines • Common data quality issues • Combining data across facilities • Missingness • Unusual patterns, wild data points • Importance of diagnostics • Graphical display • Evaluating patterns of variability, missingness • Comparing baseline patterns in subgroups

9. Issues in Data Organization • Choice of level of analysis • Aggregated across all units • Separately by logical units (facility, prescriber) • Patient-level analysis • Patient subgroups • Continuing vs. new patients • Clinical risk subgroups • Medication data • Therapeutic classification and organization • Policy-induced switching (market share analysis)

10. Issues in Statistical Approach • Study design, sampling, and statistical approach must go hand in hand • Duration of available data is key factor • Level of analysis • Validity in longitudinal policy change models • Baseline serves as counterfactual • Co-intervention is the major confounder • Need to understand context and stability of system

11. Presenters • Christine Lu, USA • Market utilization or sales data (Abstract 878) • SauwakonRatanawijitrasin, Thailand • Electronic clinical and pharmacy data (Abstract 811) • Ricardo Perez-Cuevas, Mexico • Electronic medical record data (Abstract 1118) • Joshua Kayiwa, Uganda • Routine data from manual systems (Abstract 505) • Mike Law, Canada • Overview of common analytic approaches

12. Listen, participate, enjoy… Thank you!