Modeling Diabetic Hospitalizations for the TennCare Population

1. Modeling Diabetic Hospitalizations for the TennCare Population • Application of Predictive Modeling for Care Management Panel • AcademyHealth Annual Research Meeting • June 28, 2005 Boston • Avery Ashby MS • Soyal Momin MS, MBA • Raymond Phillippi PhD • Allen Naidoo PhD • Judy Slagle RN, MPA

2. Background

3. Management Programs • BlueCross BlueShield of Tennessee provides care management programs for members with certain chronic illnesses or conditions. • Care managers are licensed nurses. • Diabetes is a prevalent chronic illness affecting our managed TennCare population. • Modeling of diabetic inpatient hospitalizations can help in identifying and directing those members at higher risk to care management.

4. Methodology

5. Time Period Year 1 July 1, 2001 – June 30, 2002 Member Specific Data Year 2 July 1, 2002 – June 30, 2003 Diabetic Hospitalization? Study Design • Diabetic members were identified using member level claims data. • Data were collected for continuously enrolled diabetic members for the time period of July 1, 2001 through June 30, 2003. • Year 1 member specific data were used to model whether a diabetic hospitalization occurred in Year 2. • Logistic regression was employed to model the probability of a diabetic hospitalization in Year 2.

6. Data Elements

7. Demographics • Gender • Age • Zip Code • Metropolitan & Rural • Region • Multiple Regions • Eligibility • Medicaid subcategories not including dual-eligible members

8. Utilization • Diabetic Hospitalizations • Emergency Room Encounters • Ophthalmologist Encounters • Primary Care Physician (PCP) Encounters • Endocrinologist Encounters • Total Specialist Encounters

9. Pharmacy • Insulin Prescriptions • Prescribed or Not • Misc. Anti-diabetic Prescriptions • Prescribed or Not • Sulfonylurea Prescriptions • Prescribed or Not • Caloric Agents • Prescribed or Not • Total Prescriptions (Any variety)

10. Evidence Based Guidelines • Cholesterol Screening • Received or Not • Eye Examination • Received or Not • Microalbuminuria Screening • Received or Not • HbA1c Screening • Received or Not

11. Diagnosis and Risk Score • Insulin Dependency • Dependent or Not • Total Co-morbidities • Diagnostic Cost Grouper (DCG) Risk Score

12. General Data Characteristics • Members: 11,002 (313 Year 2 Hospitalizations) • Gender: Female 64.7% • Age: Mean 47 Median 50

13. Predictive Model

14. Model Specifics • Probability of hospitalization = 1/(1+e-z) • Where z = -2.160 + ( 1.164 * Diabetic Hospitalizations) – ( 0.328 * No Insulin prescribed) – ( 0.038 * Age) + ( 0.092 * Diagnostic Cost Grouper Risk Score) + ( 0.199 * No Misc. Anti-diabetic prescribed) + ( 0.208 * Ophthalmologist Encounters) – ( 0.015 * Primary Care Physician Encounters) – ( 0.361 * Non-Insulin Dependent) + ( 0.054 * Emergency Room Encounters) – ( 0.031 * Total Specialist Encounters)

15. Sensitivity vs. Specificity

16. Odds Ratio Estimates

17. Diagnostics *Tolerance is 1- R2x, where R2x is the variance in each covariate, X, explained by all of the other covariates.

18. Goodness of Fit

19. Actual Stay No Stay Totals Stay 38 72 Prediction 34 No Stay 275 10,655 10,930 Totals 10,689 11,002 313 Correct Prediction Rate 97.2% Sensitivity 12.1% Specificity 99.7% Positive Predictive Value (PPV) 52.8% Negative Predictive Value (NPV) 97.5% Pseudo-R2 0.223 Model Performance

20. Rational Artificial Intelligence

21. Initial RAI Results • An artificial Neural Network (ANN) was trained and validated on the entire data set. • Problematic because the ANN tried to maximize the overall correct prediction rate. • Similar results to logistic regression models.

22. RAI Model Performance Actual Stay No Stay Totals Prediction Stay 34 7 41 No Stay 279 10,682 10,961 Totals 313 10,689 11,002 Correct Prediction Rate 97.4% Sensitivity 10.9% Specificity 99.9% Positive Predictive Value (PPV) 82.9% Negative Predictive Value (NPV) 97.5% Pseudo-R2 N/A

23. Forced Learning Solution • Collect equal samples from hospitalized and non-hospitalized members. • Build ANN based on this 1:1 (150:150) training data set. • Validate ANN on remaining Out-of-Sample members. • Repeat process to ensure that the overall pattern is accounted for. • Develop credibility intervals for sensitivity, specificity, PPV, and NPV based on this repeated process.

24. Forced Learning Model Performance • Results of repeated forced learning method were collected. • 95% credibility intervals were derived from MCMC simulation using WinBUGS 1.4. Sensitivity Specificity Positive Predictive Value (PPV) Negative Predictive Value (NPV) [66.00%,70.80%] [76.06%,78.13%] [4.11%,4.49%] [98.36%,98.73%]

25. Research Implications

26. Finding a Balance • Begins with the question of allocated resources. • Logistic regression model and ANN identified a small percentage of members with an actual Year 2 hospitalization with a “reasonable” PPV. • ANN using the Forced Learning Method identified a much larger • percentage of members with an actual Year 2 hospitalization with a low PPV.

27. Predicted hospitalization Predicted hospitalization No hospitalization No hospitalization hospitalization hospitalization Coverage Logistic Regression Model Forced Learning ANN

28. Future Considerations • Other covariates like lab values, Health Risk Assessments (HRAs), and psychological indicators. • Using a meta-model where clusters of homogenous sub-groups are modeled separately [and possibly] with differing methods. • Model probability of co-morbid condition related hospitalizations instead of diabetic hospitalizations.

29. Contact Information Avery Ashby MS Senior Research Analyst Health Intelligence Group 801 Pine Street – 3E Chattanooga, TN 37402 423.763.7482 p 423.785.8083 f avery_ashby@healthintelgroup.com