Why Local Investigations?

1. Design of Local Investigations in Community Practice Settings: An Effectiveness / Cost-Effectiveness Framework with Finite Patient HorizonNaihua Duan, Ph.D.Columbia UniversityJoint work with Ken Cheung (Columbia University) and Jeff Cully (Houston VA HSRD Center) Finite Patient Horizon NYSPI

2. Why Local Investigations? • Think globally, act locally • Quality improvement programs in community practice settings often need to adapt and/or improvise in order to accommodate local conditions • Impact of adaption is usually unknown • Decision-making often based on expert judgment • Empirical evaluation might be warranted, but not widely used • “Small n problem” (Small N, not n) Finite Patient Horizon NYSPI

3. External Program Evaluation vs. Internal Quality Improvement Evaluation • Program evaluation usually takes an external perspective: does the implementation program work? • Does the engine work overall? • Internal evaluation for nuts and bolts might be warranted for quality improvement • How do we do intake calls, case referrals, follow-up reminders, care management (maybe IT enhanced), etc., most effectively for a specific clinical setting? • One size might not fit all (HTE; Kravitz-Duan 2004) Finite Patient Horizon NYSPI

4. Statistical Literature • Statistical literature focused mainly on global investigations, not on local investigations • Zelen (1969) an unusual exception • “New” statistical perspective warranted for local investigations in implementation studies • Global knowledge for external consumption and local knowledge for internal consumption call for distinct statistical frameworks Finite Patient Horizon NYSPI

5. Patient Horizon (Population Size) • Essentially infinite for usual clinical studies aimed to produce generalizable knowledge for external consumption • Large Phase III trial, sample size << N • Strong accuracy (small , high power) warranted to protect welfare of future patients • Can be very limited (hundreds, even fewer) for local investigations aimed to produce local knowledge for internal consumption Finite Patient Horizon NYSPI

6. Hypothetical Example • N = 500 patients • Randomize 2n patients to novel procedure vs. standard procedure (50/50) • Apply local knowledge gained to remaining patients (N – 2 n) • d = 0.25,  = 5%, 80% power ==> n = 250 • No patients left to consume knowledge gained! Finite Patient Horizon NYSPI

7. Cost Effectiveness Framework • Criterion: incremental net benefit per capita • INB = b  – c •  = effect size, say, symptom reduction resulting from new procedure (unknown) • b = value for each unit of treatment effect (known) • c = incremental cost for novel procedure vs. standard procedure (known) • Effectiveness framework if c = 0 Finite Patient Horizon NYSPI

8. Cost Effectiveness Framework (II) • Bayesian framework to incorporate existing knowledge among local experts •  ~ N(0, 2) •  measures local experts’ uncertainty • Cost-effectiveness equipoise: • b 0 = c • Effectiveness equipoise: 0 = 0 (c = 0) Finite Patient Horizon NYSPI

9. Non-Empirical Adoption Strategies • Based adoption decision on expert opinion, without empirical local investigation • Never adopt strategy (bench mark) • Always adopt strategy • Expected gain compared to never adopt: • G1 = N × (b 0 – c) Finite Patient Horizon NYSPI

10. Evidence-Based Adoption Strategy • Randomize 2n patients to trial (evaluation phase) • One-sided test H0: INB  0 vs. HA: INB > 0 with significance threshold  • Apply finding to remaining N – 2n patients (consumption phase) • Each combination of design parameters (n, ) is a possible design for the local investigation • Find the design that maximizes expected net gain across N patients • Compare with non-empirical strategies Finite Patient Horizon NYSPI

11. Evidence-Based Adoption Strategy • Expected gain: G2 = {n + (N – 2n) A} (b 0 – c) + (N – 2n) B A  (0,1); B > 0 • Compared to non-empirical strategies: G0 = 0 G1 = N × (b 0 – c) Finite Patient Horizon NYSPI

12. Optimal Design for Local Investigation Finite Patient Horizon NYSPI

13. Finite Patient Horizon NYSPI

14. Noteworthy Results • Under equipoise: • Any local investigation is better than non-empirical strategies • Optimal  = 50% • Optimal n = N / {3 + sqrt(9 + 4 R)}  N / 6 • R = N 2 / (2 2) • Under mild optimism: optimal  > 50% • Under strong optimism: forthright adoption without local investigation Finite Patient Horizon NYSPI

15. Finite Patient Horizon NYSPI

16. Discussions • More emphasis on local investigations might enhance quality improvement programs, leading to improved patient safety, quality of care, and health outcomes • More emphasis on local investigations might help transform current top-down organization for health care knowledge production, and facilitate the development of learning communities, and motivate more comprehensive data acquisition Finite Patient Horizon NYSPI

17. Discussions (II) • Appropriate statistical methodologies (not the 5%-80% ritual) can facilitate wider use of local investigations in quality improvement studies • Human subjects and publication issues Finite Patient Horizon NYSPI