Daniel Weston, M.B.A . The Ohio Colleges of Medicine: Government Resource Center

1. Improving Estimates for Electronic Health Record Take up in Ohio: A Small Area Estimation Technique Daniel Weston, M.B.A. The Ohio Colleges of Medicine: Government Resource Center

2. Outline • Background • Details on the Electronic Health Records Survey • Initial Estimates • Regression Models • Bootstrap Methods • Conclusions • Recommendations for the next survey and future work

3. Background • To estimate the EHR take up rates for Ohio • Medical practitioners, primary care physicians, medical specialists, dentists, nurse practitioners, and nurse midwives with a Medicaid patient volume above 30%, and pediatricians with a Medicaid patient volume above 20%. • The motivation for these estimates is that every practitioner above the required volume thresholds will be eligible to apply for financial assistance for the adoption and implementation of an EHR system

4. Details on the Electronic Health Records Survey • The data used for this thesis come from the 2010 Ohio Electronic Health Records Survey (EHRS). • To determine Electronic Health Records (EHR) take up by Ohio medical practitioners enrolled as Medicaid providers. • To help reveal key barriers to EHR adoption. • Estimate the proportion of EHR adoption by 2015.

5. Follow-up faxing approach • To all known fax numbers among sample practices. • To encourage response, a blanket approach was employed by faxing all non-responding medical practices the cover letter and instrument to the office manager. • Faxing and return by fax vastly improved total response rates in two days. • 19.23% Response rate • Possible problems • Non response Bias • Surveyed practices not practitioners

6. Initial Stratified Estimates Sahr et al. (2010) • No confidence interval reported • Assumed practitioner per practice was constant across the state • County-level estimates not provided

7. Small Area Estimation • Obtained county attributes from the 2010 Census data • Obtained county-level health attributes from the Ohio Family Health Survey (OFHS) • Fit a linear regression model using responses from the EHRS, Census Data, and the OFHS to determine a county-level estimate of a physician’s probability for EHR take up; • Fit a linear regression model using responses from the EHRS, Census Data, and the OFHS to determine a county-level estimate of doctors per practice; and • Using these estimates, the number of doctors per practice per county was estimated and new estimates for total EHR take up were determined.

8. Small Area Estimation is the px1 vector of regression coefficients for the auxiliary data is the px1 vector of regression coefficients for the variables used in the EHRS is the intercept term are area-specific random errors assumed to be independent and identically normally distributed with mean = 0 and variance ≥ 0

9. Regression Model 1: Log Sum of Practice Size logsumpracsize = 0.000110 (veterans) + 14.2 (Medicaid) - 0.000016 (Black_population)+ 11.6 (Self_rated_health_%_cnty)+ 8.80 (%_County_unmethealthcare) -0.000047 (Median_household_inc)+ 1.28 (%_practice_in_cnty_responding_early) - 9.70 (%_County_poverty) + 0.912(%_prac_in_county_ ehr)- 7.92

10. Regression Model 1: Log Sum of Practice Size

11. Regression Model 1: Log Sum of Practice Size • The parameter estimates suggest that the log in number of practitioners increases with: • The number veterans in a county • A higher percent of Medicaid recipients in the county • Higher proportion of the county with excellent or good self-rated-health-status • A higher proportion of county practitioners responding to the EHRS early • And a higher proportion of the county with an unmet-health-care-need • It decreases with • The number of black individuals in the county • The median home price in the county • Proportion of people below 100% FPL

12. Regression Model 2: Log EHR Status • logehruptake = - 2.40 - 0.00308 (Land total square miles)+ 0.00000005 (Wholesale trade sales)+ 0.000004 ( total farmable acres)+ 0.00970 practice size + 0.000085 (Per capita income )+ 2.90 (% of cntyunmethealthcare)- 0.0925 (Housing change)+ 0.0734 (population,percent change )- 0.154 (AGE u5)+ 0.131 (CLF) - 0.787 (er_mean)

13. Regression Model 2: Log EHR Status

14. Regression Model 2: Log EHR Status • The parameter estimates suggest that the log in EHR take up increases with: • Total sales of wholesale trade • Farmable acres • Larger average doctor practice group sizes • Per capita income • Higher county percent of unmet healthcare needs • County population • And the size of the civilian labor force • Log EHR decreases with: • Total square miles in county • Housing change increases • Total number of children five years old or under • And average emergency room visits per county

15. State-wide Estimate and problems • Two major flaws of the survey instrument are revealed • No way to decipher how many of the practitioners serving in a group practice are above the MPIP thresholds individually. • And secondly, there is no way to decipher if a practitioner was reported by multiple practices • We will approximate the number of eligible MPIP applicants at 25% of the estimated total number of practitioners with an EHR working in practices serving over 200 Medicaid recipients

16. State-wide Estimate and problems • The estimated total numbers of practitioners with an EHR who are working in practices serving over 200 Medicaid recipients over all counties is 5,455 estimated cost $347,756,250 • 95% confidence interval for the number of practitioners working in offices serving 200 or more Medicaid patients is (0.00066, 613449633044.38) • We gain nothing here

17. Non Parametric Bootstrap • Bootstrapping is a technique that treats the original county data used to fit Models 1 and 2 as a pseudopopulation and samples counties randomly with replacement (srs wr) m=999 times. • We use these random subsamples of counties and their corresponding data points are used to re-fit our Models 1 and 2

18. Non Parametric Bootstrap • Non Parametric Bootstrapping was less sensitive to extreme outliers

19. Recommendations for future work • The instrument should ask for National Provider Identification (NPI) numbers of all practitioners accounted for in the practice size • The instrument should include a question which will allow researchers to determine full time equivalents for practitioners. • The instrument should ask Medicaid volume for each separate NPI number and obtain the specialties of each individual practitioner. • The instrument should clearly ask if all NPI numbers listed fully use the EHR system, if not which do not. • The instrument should ask questions relating to the effectiveness of the EHR if on is installed. • If at all possible the EHRS should be done at the practitioner level to avoid responses from practitioners not in the sampling frame. This could be done by sampling from the NPI list instead of the Medicaid Provider list.

20. Conclusions • The goal of this thesis was to accurately estimate the EHR take up rates for Ohio’s medical practitioners (with a Medicaid patient volume above 30%, and pediatricians with a Medicaid patient volume above 20%) using Small Area Estimation. • Small Area Estimation combined data from the EHRS, OFHS, and the 2010 Census and, using linear regression, estimated that 5,455 practitioners were eligible for MPIP funding. SAE’s large variance produced a wide 95% confidence interval was which is not very useful (bounded by zero and the estimated number of practitioners licensed to practice in Ohio (0, 41964)). • Employing the non-parametric bootstrap method, we obtained a new estimate for practitioners eligible for MPIP funding to be 4,566, with a 95% confidence interval of (635, 47,652). Lower confidence bounds were calculated to establish a minimum for Ohio to budget for the MPIP

21. Conclusions • Although our models are based on imperfect data, Small Area Estimation is the preferred technique, because it provides county-level estimates for all counties, (even for counties with no responses from the EHRS – although caution must be used with these estimates for counties without representation in the EHRS) • Non-parametric Bootstrapping with Small Area Estimation is also recommended in future iterations of the EHRS because is less sensitive to outliers. • The non-parametric bootstrap allowed us to calculate lower confidence bounds, allowing a baseline of MPIP eligibility to be established for budgetary purposes which the original estimates did not allow. • Therefore, this thesis has shown Small Area Estimation with bootstrapping the preferred technique to the original method used by Sahr et al. (2010).

22. Acknowledgements • Ohio Department of Job and Family Services/ Ohio Medicaid • Dr. Elizabeth Stasny, The Ohio State University Department of Statistics • Dr. Eloise Kaizar, The Ohio State University Department of Statistics • Timothy R. Sahr, The Ohio Colleges of Medicine, Government Resource Center, The Ohio State University Office of Health Sciences • Lorin Ranbom, The Ohio Colleges of Medicine, Government Resource Center, The Ohio State University Office of Health Sciences