1 / 37

Secondary Database Analysis II

Secondary Database Analysis II. Case-Mix Adjustment. What is the Purpose of Health Services Research?. Income Ethnicity EMR CPOE. Utilization A1c Immunization Medical Errors. Last Time – Uses. Secondary Data:

Download Presentation

Secondary Database Analysis II

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Secondary Database Analysis II Case-Mix Adjustment

  2. What is the Purpose of Health Services Research? • Income • Ethnicity • EMR • CPOE Utilization A1c Immunization Medical Errors

  3. Last Time – Uses Secondary Data: • Can provide new information on health care delivery (quality; geographic variation; post-marketing adverse events; cost), on the natural history of disease, and on regulatory matters • Can save data-collection resources: time, money, personnel, participant burden • Can save researcher resources: your time, your money • May permit you to build your CV without anyone’s help or funds – just your time (but co-authors are a good thing) • Can provide preliminary data for a grant proposal • May enable research on rare events or difficult populations

  4. Last Time – Precautions • Take note of the study design; is it suitable? • Consider: What are inclusion & exclusion criteria? Why? How does that affect the sample and generalizability?

  5. This Time – Precautions • What bias may be inherent in the database – perhaps from the population or from the method of measurement? • Selection bias: who got included • Migration bias: who was lost/gained and why • Other sources of systematic error (bias)?

  6. This Time – Precautions & Solutions • Control for bias – • Restrict • Match • Stratify • Adjust with covariates

  7. Validity Nesting/Clustering of data Risk Adjustment infer infer Truth in the Universe Truth in the Study Findings in the Study ERRORS ERRORS RESEARCH QUESTION STUDY PLAN ACTUAL STUDY Intended Sample Actual Subjects Target Population Phenomenon of Interest Intended Variables Actual Measurements EXTERNAL VALIDITY INTERNAL VALIDITY

  8. Bias • Systematic error in measurement or a systematic difference (other than the one of interest) between groups • Selection • For cohorts, assembly, migration, contamination, and referral bias • Measurement • Confounding

  9. Bias: Anticipate and Control • Restriction (may lose generalizability) • Matching (limited # factors) • Stratification • Standardization • Multivariate Adjustment • Assuming the worst (sensitivity analyses) • If needed, conduct sensitivity analyses on multivariable models • >> Always discuss potential impact of uncorrected bias on your results

  10. Bias: Systematic Error • Selection Bias • Case-Mix/Disease Severity • Nesting/clustering • Measurement Bias • Recall Bias • Investigator Bias

  11. Example: Hospital Mortality Report Cards • Originally unadjusted • Hospitals without trauma centers, doing primarily elective surgery, etc., looked really good • Made hospitals who took care of the sickest of the sick look bad

  12. Quality Assessment • Data Quality: Garbage in, garbage out • Risk Adjustment: To remove the confounding effect of different institutions providing care to patients with dissimilar severity of illness and case complexity

  13. Risk Adjustment • Controls for patient characteristics that are related to the outcomes of interest • Removes the confounding effect, e.g., of different institutions providing care to patients with dissimilar severity of illness and case complexity • Addresses regional variations • Inadequate case-mix adjustment can lead to misclassification of outlier status

  14. Essential Elements of Risk Adjustment • Outcome-specific • Contains specification of the principal diagnosis • Contains demographics as proxies for preexisting physiological reserve • Measures # of comorbidities and allows all the most important comorbidities to assume their own empirically derived coefficients

  15. Additional considerations • – outcome-specific: is one available or do you need to construct & validate one yourself? • – an index incorporates many predictors – do you need to study some separately? • – established comorbidity indices for case-mix adjustment include Charlson, Elixhauser, Selim, developed on various patient pops – suitable for yours? • – other predictors based on your reading of the literature – do you need omitted constructs? • – propensity scores (for treatment choice; good for small datasets, for non-randomized studies of treatment effect)

  16. Risk Adjustment & Outcomes • Primary data collection vs. administrative data • Disease-specific vs. generic • Commercial vs. developed for your study • Predictors vary by outcomes being predicted

  17. Classification of Disease States • ICD-9: too many specific codes (n~10,000) • Clinical Classifications for Health Policy Research (CCHPR): good for chronic illness and longitudinal care [http://www.ahrq.gov/data/hcup/his.htm] • Primary diagnosis: good for studies that focus on a single episode of care

  18. Famous Methods of Risk Adjustment • DRGs: diagnosis related groups • Used by Medicare to set hospital reimbursement • APACHE III • Adult ICU • PRISM • Pediatric ICU • Charlson Score • Adult 1 year survival after hospitalization

  19. Risk Adjustment: Charlson • Advantages: • Commonly used case-mix classification system in the health care industry • System with which most clinicians and reviewers are familiar

  20. Risk Adjustment: Charlson • Disadvantages • Principal diagnosis not differentiated • Original work did not specify ICD-9 codes that went into the disease categories • Developed on inpatients predicting mortality; may not be well suited to outpatients at low risk of death • Not good for longitudinal care / chronic illness

  21. Demographic Factors in Risk Adjusters • Age (e.g., age-adjusted Charlson) • Proxies of Social Support • Marital status • Race • Gender • SES (occupation, employment status, education) • Proxies of Socioeconomic Status • Health insurance status • Home address zip code average income

  22. Race and Gender • Don’t adjust for automatically • Ideally adjust for variation in the patients’ physiological reserve and disease burden but not for variation in care rendered to patients

  23. Propensity Scores • Useful when sample size is small, to conserve power • An alternative to including a lot of covariates • Ask: propensity for what? • Include as many predictors as possible to get predicted probability of group membership (Rosenbaum & Rubin) • Published schema may include predictors you want to study separately • Best for non-randomized studies of treatment effect where you want to adjust for the factors that may have influenced the treatment choices

  24. Study Design: Minimize Bias Experimental Study Apply intervention, observe effect on outcome Decision #1: Alter events under study? yes no Cross-Sectional Study Each subject examined on only one occasion Observational Study no Longitudinal Study Each subject followed over a period of time Decision #2: Make measurement on more than one occasion? yes Case-control Cohort

  25. Risk adjustment is needed… • When subjects are not randomly assigned • People do not randomly distribute by • Setting • Provider • Risk Adjust • Outcomes = f (intrinsic pt factors; treatment applied; quality of treatment; random chance)

  26. On What Factors Should We Risk Adjust? • Risk of what outcome? • Ejection Fraction • Readmission • Activities of Daily Living (feeding, dressing, etc.) • For what population? • Inpatient • Outpatient • Nursing Home • For what purpose? • Clinical • Quality • Payment

  27. Classes of potential risk factors • Demographics: age, sex, etc. • Physiologic status • Number and type of medical diagnoses • Cognitive and mental status • Sensory function • Social, economic, environmental factors • Functional/overall health status

  28. An Important Distinction • Disease Severity • Case-mix or co-morbidity • What is the difference?

  29. Disease Severity v. Case Mix

  30. Nesting or Clustering of Observations A threat to validity

  31. Nesting/Clustering of Observations • Traditional methods of analysis assume observations are independent: people who see same MD are not! • By setting • Multicenter studies • By clinic • By clinician • Key issue is understanding SOURCE of variance in the observed data • Within group (clinic) • Between groups (clinics)

  32. Survival by OR Team

  33. Common Terminology • Intra-class Correlation Coefficient (ICC) • The extent to which individuals within the same group are more similar to each other than they are to individuals in different groups • The proportion of the true variation in the outcome that can be attributed to differences between the clusters

  34. ICC Values • 20 primary care clinics • 30-32 patients with type 2 DM per clinic • For process of care measures (foot exam, labs ordered etc.) ICC = 0.32 • For A1c values, ICC = 0.12 Question: Which is more dependent upon site of care: processes of care or outcomes of care (A1c control)?

  35. Sample Size Implication • Design Effect = 1 + (n-1)ICC (n= number of subjects per cluster) • So if estimated sample size of 300 per group for an intervention study, what sample size would you need for 30 subjects per cluster (10 clinics) and ICC of 0.12?

  36. Assignment • Design a study: • Experimental • Cross-sectional • Prospective cohort • Case-Control • What factors you would measure to “risk adjust” and how measure them? • How would you would adjust for nesting/clustering of data?

  37. Questions? Common Research Design Issues in Health Services Research

More Related