1 / 21

Quantitative Methods: Reliability, Validity, and Use of SEM to Assess Psychometric Equivalence

Quantitative Methods: Reliability, Validity, and Use of SEM to Assess Psychometric Equivalence. Ron D. Hays, Ph.D. UCLA Department of Medicine Los Angeles, CA November 18, 2005. Measurement Range for Health Outcome Measures. Nominal. Ordinal. Interval. Ratio. Indicators of Acceptability.

mseeley
Download Presentation

Quantitative Methods: Reliability, Validity, and Use of SEM to Assess Psychometric Equivalence

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Quantitative Methods: Reliability, Validity, and Use of SEM to AssessPsychometric Equivalence Ron D. Hays, Ph.D. UCLA Department of Medicine Los Angeles, CA November 18, 2005

  2. Measurement Range for Health Outcome Measures Nominal Ordinal Interval Ratio

  3. Indicators of Acceptability • Response rate • Administration time • Missing data (item, scale)

  4. Variability • All scale levels are represented • Distribution approximates bell-shaped “normal”

  5. Measurement Error Observed = true + systematic + random score error error (bias)

  6. Flavors of Reliability • Test-retest (administrators) • Intra-rater (raters) • Internal consistency (items)

  7. Intraclass Correlation and Reliability

  8. 2.9 - 1.1 = 1.8 = 0.62 Alpha = 2.9 2.9 Cronbach’s Alpha

  9. Reliability Minimum Standards • 0.70 or above (for group comparisons) • 0.90 or higher (for individual assessment) • SEM = SD (1- reliability)1/2

  10. Construct Validity • Does measure relate to other measures in ways consistent with hypothesis? • Responsiveness to change

  11. Responsiveness to Change and Minimally Important Difference • HRQOL measures should be responsive to interventions that changes HRQOL • Evaluating responsiveness requires assessment of HRQOL • pre-post intervention of known efficacy • at two times in tandem with anchor • HRQOL change among people who changed on anchor

  12. Self-Report Anchor • Overall has there been any change in your asthma since the beginning of the study? • Much improved; Moderately improved; Minimally improved • No change • Much worse; Moderately worse; Minimally worse

  13. Clinical Anchor • “changed” group = seizure free (100% reduction in seizure frequency) • “unchanged” group = < 50% change in seizure frequency

  14. Responsiveness Indices (1) Effect size (ES) = D/SD (2) Standardized Response Mean (SRM) = D/SD† (3) Guyatt responsiveness statistic (RS) = D/SD‡ D = raw score change in “changed” group; SD = baseline SD; †SD = SD of D; ‡ SD = SD of D among “unchanged”

  15. Effect Size Benchmarks • Small: 0.20->0.49 • Moderate: 0.50->0.79 • Large: 0.80 or above

  16. Hypothetical Multitrait/Multi-Item Correlation Matrix

  17. Confirmatory Factor Analysis • Compares observed covariances with covariances generated by hypothesized model • Statistical and practical tests of fit • Factor loadings • Correlations between factors • Regression coefficients

  18. Fit Indices 2 2  -  • Normed fit index: • Non-normed fit index: • Comparative fit index: null model 2 2 2   null - model null df df null model 2  null - 1 df null 2  - df 1 - model model 2  - df null null

  19. Acknowledgments • Supported in whole by UCLA Center for Health Improvement in Minority Elders/Resource Centers for Minority Aging Research, National Institute on Aging (AG-02-004)

More Related