1 / 23

Nonequivalent Groups: Linear Methods

Nonequivalent Groups: Linear Methods. Equating. Common Population. Anchor Test. Observed Score. True Score: CTT/IRT. Observed Score. True Score: CTT/IRT. The Synthetic Population. The design involves two populations

feivel
Download Presentation

Nonequivalent Groups: Linear Methods

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Nonequivalent Groups:Linear Methods

  2. Equating Common Population Anchor Test Observed Score True Score: CTT/IRT Observed Score True Score: CTT/IRT

  3. The Synthetic Population • The design involves two populations • The equating function is defined for a single population (synthetic), where each population is weighted by w1 and w2 where w1 + w2 = 1 and w1 , w2 0

  4. The linear conversion is defined by setting standardized deviation scores equal

  5. Linear Equating Function Where s indicates the synthetic population

  6. Synthetic Population Parameters s(X) = w11(X) + w22(X) s(Y) = w11(Y) + w22(Y)

  7. Synthetic Population Parameters • In population 1, Y is not administered • In population 2, X is not administered • The following parameters cannot be estimated directly:

  8. Parameters Estimated from Data • Form X administered to population 1 • Form Y administered to population 2

  9. Parameters Estimated from Assumptions • Form X moments in Population 2 • Form Y moments in Population 1

  10. Tucker Method • Assumption 1: regression of total scores on common-item scores • The regression of X on V is the same linear function for both populations 1 and 2 • The same assumption is made for the regression of Y on V The slope and regression intercept are assumed to be the same for the observed data with each population and the unobserved parameters in the other population

  11. Tucker Method • Assumption 2: conditional variances of total scores given common-item scores • Conditional variance of X given V is the same for population 1 and 2 • The same assumption is made for the conditional variance of Y given V

  12. Tucker Method • The conditional mean score on the new form increases linearly with scores on the anchor • Use a simple formula to estimate the conditional mean in the synthetic population • The conditional standard error is the same at all levels of the anchor score • Estimate a single value for the conditional sd • Need: Mean and SD of anchor scores in synthetic population

  13. Tucker Method • Result: the synthetic population means and variances for X and Y are adjusted to directly observable quantities. • The adjustment is a function of the differences in means and variances for the common items across the two populations. • If 1(V) = 2(V) and the synthetic parameters would equal observable means and variances.

  14. Tucker Issues • Problems in the equating can occur if • the ability distributions between those who take the different forms differ a great deal • When the anchor is not strongly correlated with the test scores • The test scores and the anchor scores do not yield near perfect reliabilities

  15. Levine Method • Assumes that X, Y, and V are all measuring the same thing so that TX and TV as well as TY and TV are perfectly correlated in both populations. • Assumptions about true scores are made in terms of the linear regression of X on V and Y on V • Assumptions about the error variances (measurement error) are made similarly

  16. Levine True Score Method • Similar assumptions about true scores are made in this method • Instead of equating observed scores, true scores are equated • Although the derivations employ true scores, the equating is actually done on observed scores.

  17. Nonequivalent Groups:Equipercentile Methods

  18. Frequency Estimation • f (x, v ) is the joint distribution; the probability that X = x and V = v. • f (x) is the marginal distribution of scores on X; the probability of obtaining a score of x on X. • h( v ) is the marginal distribution of scores on V. • f (x|v ) is the conditional distribution of scores on Form X for examinees with a particular score on V.

  19. The conditional expectation is It follows that

  20. Synthetic Populations • fs(x) = w1f1(x) + w2f2(x) • gs(y) = w1g1(y) + w2g2(y) • Because form X is not administered to population 2, f2(x) is not directly estimable • Because form Y is not administered to population 1, g1(y) is not directly estimable

  21. Assumptions • For both Form X and Y, the conditional distribution of total score given each score, V = v, is the same in both populations.

  22. Estimation • These assumptions can be used to find expressions for f2(x) and g1(y) using quantities for which direct estimates are available.

  23. Chained Equipercentile Equating • Angoff (1971) • Form X scores are converted to scores on the common items using examinees from population 1 • Scores on the common items are equated to form Y scores using examinees from Population 2. • These conversions are chained together to produce a conversion of Form X to Form Y

More Related