1 / 12

Reliability

This guide explores the concept of reliability in testing, focusing on key aspects such as consistency, dependability, and replicability of measures. It discusses threats to reliability, including issues related to administration and scoring. Various approaches for establishing reliability, such as test-retest and parallel forms, are elaborated upon alongside the split-half method for internal consistency. The document emphasizes statistical formulas like KR20 and Cronbach's alpha for calculating reliability coefficients, enhancing the understanding of inter-rater reliability and various implications in educational and psychological assessments.

justin
Download Presentation

Reliability

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reliability • Consistent • Dependable • Replicable • Stable

  2. Threats to reliability • Construction • Administration (tester, testee, environment) • Scoring • Interpretation

  3. Two Administrations • Test/retest same test - same group • Parallel/Alternate forms different versions of test - same group

  4. Two administration procedure: • Administer tests (in two sessions) • Convert to z scores (if necessary) • Correlate (Pearson or Spearman)

  5. Two administration issues: • Problems????? • Duration between???? • Type of variable?????

  6. One Administration • One test • One group • One administration

  7. One administration procedure: • Administer test to one group • Divide questions to score • Split Half • first/second or odd/even halves???? • Correlate scores from halves • Apply Spearman-Brown formula • estimate changes in length

  8. Uses of one administration: • Internal consistency of items • Appropriate for homogenous tests • Not appropriate for heterogeneous tests • (may have several measures within test)

  9. Inter item consistency • Statistical estimation • Kuder-Richardson Formula 20 (KR20) = dichotomous questions • Cronbach alpha (alpha coefficient) = all questions • (factor analysis)

  10. Reliability coefficient • 0.00 - 1.00 • higher is better • score is relative, not absolute

  11. Inter-rater reliability • Consensus between raters • Percentage of agreement • Kappa statistic (2 or many raters)

  12. Project homework • Which approach(s) would you use to determine the reliability of your measure? -- Why did you select those approach(s)?

More Related