Lab 4 alpha and standard error of measurement
Download
1 / 16

Lab 4: Alpha and Standard Error of Measurement - PowerPoint PPT Presentation


  • 74 Views
  • Uploaded on

Lab 4: Alpha and Standard Error of Measurement. Reliability. Reliability refers to consistency Types of reliability estimates Test-retest reliability Interobserver (or interrater) reliability Internal consistency Alpha measures the consistency of responses across items.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Lab 4: Alpha and Standard Error of Measurement' - rigel-mcclain


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Reliability
Reliability

  • Reliability refers to consistency

  • Types of reliability estimates

    • Test-retest reliability

    • Interobserver (or interrater) reliability

    • Internal consistency

  • Alpha measures the consistency of responses across items


Calculating alpha of the extraversion scale by hand
Calculating Alpha of the Extraversion Scale by Hand

  • k: number of items

    Extraversion scale: k = 10

  • rij: average inter-item correlation

    Extraversion scale: rij = .46

  • How large would alpha be if the scale had 16 items?


Calculating alpha of the extraversion scale for the time 2 ipip
Calculating Alpha of theExtraversion Scale for the Time 2 IPIP

  • SPSS: Analyze  Scale  Reliability Analysis

  • Put the following items in the items box:

    • e1_t2, e2_r_t2, e3_t2, e4_r_t2, e5_t2, e6_r_t2, e7_t2, e8_r_t2, e9_t2, & e10_r_t2

    • Make sure you use the reverse scored items!

  • Click on Statistics

    • Inter-Item: Correlations

    • Summaries: Correlations

    • Then hit Continue

  • Then click OK


Questions to answer from output
Questions to Answer from Output

  • 1. What was the average inter-item correlation?

  • 2. How many items are on the scale?

  • 3. What was the standardized alpha?


Calculating alphas for the other four scales a c n and o
Calculating Alphas for theother four scales (A, C, N, and O)

  • Agreeableness: a1_r_t2, a2_t2, a3_r_t2, a4_t2, a5_r_t2, a6_t2, a7_r_t2, a8_t2, a9_t2, & a10_t2

  • Conscientiousness: c1_t2, c2_r_t2, c3_t2, c4_r_t2, c5_t2, c6_r_t2, c7_t2, c8_r_t2, c9_t2, & c10_t2

  • Neuroticism: n1_t2, n2_r_t2, n3_t2, n4_r_t2, n5_t2, n6_t2, n7_t2, n8_t2, n9_t2, & n10_t2

  • Openness: o1_t2, o2_r_t2, o3_t2, o4_r_t2, o5_t2, o6_r_t2, o7_t2, o8_t2, o9_t2, & o10_t2


What happens if you forget to reverse score the items
What happens if you forget to reverse score the items?

  • Calculate alpha for the Extraversion scale again, but now use the following items:

    • e1_t2, e2_t2, e3_t2, e4_t2, e5_t2, e6_t2, e7_t2, e8_t2, e9_t2, & e10_t2

  • What is the new alpha?

  • How can you tell that you did something wrong?


What happens if you calculate alpha of two scales at the same time
What happens if you calculate alpha of two scales at the same time?

  • Calculate alpha for the following eight items:

    • e1_t2, e2_r_t2, e3_t2, e5_t2 (Extraversion items)

    • c1_t2, c3_t2, c5_t2, c7_t2 (Conscientiousness items)

  • Can you interpret your result? What might clue you into your mistake?

  • Remember: Alpha does not indicate unidimensionality!


Standard error of measurement
Standard Error of Measurement same time?

  • Standard deviation of the error around any individual’s true score

  • If a student were to take the same test repeatedly, with no change in his level of knowledge and preparation, it is possible that some of the resulting scores would be slightly higher or slightly lower than the score that precisely reflects the student’s actual level of knowledge and ability.

  • The difference between a student’s actual score and his highest or lowest hypothetical score is known as the standarderror of measurement.


Standard error of measurement1
Standard Error of Measurement same time?

  • False Negative

    • A small percentage of students may score lower than anticipated on a test

  • False Positive

    • A small percentage of students may score higher than otherwise would have been expected


Standard error of measurement2
Standard Error of Measurement same time?

  • Using a reliability coefficient and the test’s standard deviation, we can calculate this value:

  • Where:

    • sx = the standard deviation for the test

    • rxx = the reliability coefficient (alpha) for the test


Standard error of measurement3
Standard Error of Measurement same time?

  • Standard Error of Measurement for Agreeableness Scale

  • To obtain rxx: Repeat steps from earlier in Lab

    • Analyze  Scale  Reliability Analysis

    • Enter all Agreeableness items for Time 2

  • Click on Statistics

    • Inter-Item: Correlations

    • Summaries: Correlations

    • Then hit Continue

  • Then click OK


Standard error of measurement4
Standard Error of Measurement same time?

  • To obtain standard deviation:

    • Analyze  Descriptive  Descriptives

    • Enter a_t2 (non-standardized)

    • Check standard deviation in options

    • Click Ok

  • Hence:

    • SEM= .55√ 1-.86 = .20


Sem confidence interval
SEM Confidence Interval same time?

  • 96% Confidence:

    • Z = 2.0

  • 68% Confidence:

    • Z = 1.0

  • 96% Band = Score  (2 *SEM)

    • CI = 4  (2 * .20) = 3.6 - 4.4

  • 68% Band = Score  (1 *SEM)

    • CI = 4  (.20) = 3.8 – 4.2


Homework questions 1 2
Homework Questions 1 & 2 same time?

1. Report the standardized alphas for all five IPIP scales at Time 2. Also report the average inter-item correlation and number of items. (2 point)

2. Imagine that the average inter-item correlation for a scale is .40. Compute the standardized alphas for tests with the following numbers of items: 5, 10, 15, 20, 25. (3 points)


Homework questions 3 4
Homework Questions 3 & 4 same time?

3. Report the SEM for all 5 IPIP scales at Time 2 with interpretation. (3 points)

4. Report the confidence interval for the mean conscientiousness at time 2 at the 96% CI with interpretation. (2pts.)


ad