Schmidt hunter approach to r
1 / 39

Schmidt & Hunter Approach to r - PowerPoint PPT Presentation

  • Uploaded on

Schmidt & Hunter Approach to r. Statistical Artifacts. Extraneous factors that influence observed effect Sampling error Reliability Range restriction Computational error Dichotomization of variables. Bare Bones r. Find weighted mean and variance: Note sample size weight.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Schmidt & Hunter Approach to r' - sydney-ayers

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Statistical artifacts
Statistical Artifacts

  • Extraneous factors that influence observed effect

    • Sampling error

    • Reliability

    • Range restriction

    • Computational error

    • Dichotomization of variables

Bare bones r
Bare Bones r

  • Find weighted mean and variance:

  • Note sample size weight.

  • Note that for unit weights, the weighted variance estimator is the sample, not population, estimate.

Confidence interval for mean
Confidence Interval for Mean

There are k studies, with Ni observations.

This is not the only formula they use, but it’s the best one IMHO.

Estimated sampling error variance
Estimated Sampling Error Variance

  • The variance of r

Estimated variance for a study.

Estimated sampling variance for a meta-analysis. Note mean r is constant.

Variance of rho
Variance of Rho

Classical Test Theory

Sampling Error

A definition

Estimated variance of rho
Estimated Variance of rho


To find the variance of infinite-sample correlations, find the variance of r in the meta-analysis and subtract sampling error variance. Schmidt would be quick to add that part of the estimated variance is artifactual.

Credibility interval
Credibility Interval

The credibility interval and the confidence interval are quite different things. The CI is a standard statistical estimate (intended to contain rho). The CR is a Bayesian estimate (intended to contain a percentage of the values of a random variable).

Bare bones example 1
Bare-Bones Example (1)

Ignore the last 3 columns for now.

Bb example 3
BB Example (3)

Recall unwighted or unit weighted mean = .30.

Why are they different?

Psychometric meta analysis
Psychometric Meta-Analysis

Disattenuation for reliability

Correction for both

Correction for IV

Correction for DV

Suppose rxy = .30, rxx = ryy = .80. Then:

Range restriction enhancement
Range Restriction/Enhancement

These are examples of direct RR.

Direct range restriction enhancement
Direct Range Restriction/enhancement

Suppose rxy = .33, SD1=12, SD2 = 20. Then:

Can also invert by uX = 1/UX

Indirect rr
Indirect RR

Reliability of IV in restricted sample (job incumbents in I/O validation study).

Reliability of IV in unrestricted sample (job applicants in I/O validation study).

Ratio of SD of true scores; analogous to uX.

You will need rxxa for DIRECT range restriction correction.

You will need uT AND rxxi for INDIRECT range restriction correction.

Meta analysis of corrected r
Meta-Analysis of corrected r

  • If information is available can correct r for each study

  • Compute M-A on the corrected values

  • Can also be done with assumed distributions, but I don’t recommend it.

Steps 1
Steps (1)

Record data (N, r, artifact values rxx, etc.)

Compute the corrected correlation for each study:

If there is only 1 kind of artifact, disattenuation is simple:

Where a is the disattenuation factor.

Note ro is observed and rC is corrected.

If there is range restriction, things are tricky. If INDIRECT range restriction, then use Ut instead of Ux and disattenuate for reliability before adjusting for range restriction. Use reliabilities from the restricted group.

If DIRECT range restriction, adjust for ryy, then range restriction, then rxx, but rxxa, the reliability in the unrestricted group.

Steps 1b
Steps (1b)

For each study, compute compound attenuation factor:

Compute sampling variance of uncorrected r:

Note this is sampling variance for one study.

Steps 2
Steps (2)

Compute sampling variance of disattenuated r:

If there is range restriction, then do the following 2 steps.

Compute adjustment for range restriction:

Adjust sampling variance of disattenuated r:

Compute weights:

Note A is the compound attenuation factor.

Steps 3
Steps (3)

Compute the weighted mean:

Compute the weighted variance:

Compute average corrected r sampling error:

Compute variance of rho:

Psychometric m a data
Psychometric M-A data

We’ve already done the bare-bones analysis of these data. Now we’ll analyze 3 ways: (1) just criterion reliability, (2) all artifacts with INDIRECT RR, (3) all artifacts DIRECT rr.

Correct r yy only 1
Correct ryy only (1)

Suppose we only wish to correct for criterion unreliability.

Study 1 r = .20, rxx = .90, ryy = .80, Ux = 1.5

Disattenuation ryy : rC = .2/sqrt(.8) = .223607.

Compound attenuation factor A = .20/.223607 = .894.

Correct r yy only 2
Correct ryy only (2)

Correct r yy only 3
Correct ryy only (3)

Correct r yy only 4
Correct ryy only (4)

Correct r yy only 5
Correct ryy only (5)

All corrections indirect rr
All corrections, Indirect RR

Already know bare-bones mean.