Chapter 3. Reliability:. As my grand pappy, Old Reliable, used to say . . . Who is this famous bloodhound? What was he noted for saying? . What CU?. Reliability Topics:. The Basic Notion of Reliability Factors Affecting Reliability Methods of Determining Reliability
As my grand pappy, Old Reliable,
used to say . . .
Who is this famous bloodhound?
What was he noted for saying?
Reliability refers to the reliability of a test score or set of test scores, not the reliability of the test.
Reliability questions ask: “Are the scores consistent?” “Are they stable?”
Reliability is a matter of degree; it is NOT all-or-none.
Reliability is not the same as validity – validity asks “Does a test measure what is suppose to?” (reliability is necessary for, but not a sufficient condition for, validity) .
Reliability deals with unsystematic error in assessment. Systematic error (examples, “I test well because I am ‘test-wise’” or “I do not test well because English is not my first language”) will not be uncovered through tests of reliability
Test Scoring –
difference between two scorers judgments
one scorer over time (fatigue) and/or halo effect
Test Content –
the sample of test items is too small
the sample of test items is not evenly selected across material
Test Administration –
noise, time limits not consistent, physical conditions
Personal Conditions –
temporary ups and downs
(chronic test anxiety would be a systematic error and thus undetectable through measures of reliability)
Note: None of these factors automatically result in unreliability, but as we build our assessments, we hope to reduce the impact of these factors. The extent to which these factors may be affecting test scores is an empirical question and we can and will address this as we continue.
rsb = 2rxy /(1+rxy)
where rsb is the split-half reliability coefficient