large scale testing uses and abuses n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Large-scale testing: Uses and abuses PowerPoint Presentation
Download Presentation
Large-scale testing: Uses and abuses

Loading in 2 Seconds...

play fullscreen
1 / 20

Large-scale testing: Uses and abuses - PowerPoint PPT Presentation


  • 140 Views
  • Uploaded on

Large-scale testing: Uses and abuses. Richard P. Phelps Universidad Finis Terrae, Santiago, Chile January 7, 2014. Large-scale testing: Uses and abuses. 3 types of large-scale tests Measuring test quality A chronology of mistakes E conomists misunderstand testing How SIMCE is affected.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Large-scale testing: Uses and abuses' - minnie


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
large scale testing uses and abuses

Large-scale testing: Uses and abuses

Richard P. Phelps

Universidad Finis Terrae, Santiago, Chile

January 7, 2014

large scale testing uses and abuses1

Large-scale testing: Uses and abuses

3 types of large-scale tests

Measuring test quality

A chronology of mistakes

Economists misunderstand testing

How SIMCE is affected

achievement tests

Achievement tests

J.M. Rice - systematically analyzed test structures & effects

E.L. Thorndike - developed scoring

scales

Historically, were larger versions of classroom tests

~ 1900 - “scientific” achievement tests developed (Germany & USA)

SOURCE: Phelps, Standardized Testing Primer, 2007

achievement t ests

Achievement tests

Purpose: to measure how much you know and can recall

Developed using: content coverage analysis

How validated: retrospective or concurrent validity

(correlation with past measures, such as high school grades)

Requires a mastery of content prior to test.

Fairness assumes that all have same opportunity to learn content

Coachable – specific content is known in advance

SOURCE: Phelps, Standardized Testing Primer, 2007

aptitude t ests

1890s – A. Binet & T. Simon (France)

  • Pre-school children with mental disabilities
  • - achievement test not possible
  • - developed content-free test of mental abilities
  • (association, attention, memory, motor skills, reasoning)

Aptitude tests

1917 – Adapted by U.S. Army to select, assign soldiers in World War 1

1930s – Harvard University president J. Conant

wanted new admission test to identify students from lower social classes with the potential to succeed at Harvard

developed the first Scholastic Aptitude Test (SAT)

SOURCE: Phelps, Standardized Testing Primer, 2007

aptitude tests

Aptitude tests

Purpose: predict how much can be learned

Developed using: skills/job analysis

How validated: predictive validity, correlation with future activity (e.g., university or job evaluations)

Content independent. Measures:

… what student does with content provided

… how student applies skills & abilities developed over a lifetime

Not easily coachable – the content is either…

… not known in advance,

… basic, broad, commonly known by all, curriculum-free;

… less dependent on the quality of schools

SOURCE: Phelps, Standardized Testing Primer, 2007

aptitude tests1

Aptitude tests

Aptitude tests can identify:

- Students bored in school who study what interests them on their own

- Students not well adapted to high school, but well adapted to university

- Students of high ability stuck in poor schools

SOURCE: Phelps, Standardized Testing Primer, 2007

non cognitive tests

Non-cognitive tests

More recently developed

– measure values, attitudes, preferences

Types: integrity tests

career exploration

matchmaking

employment “fit”

non cognitive tests1

Non-cognitive tests

Purpose: to identify “fit” with others or a situation

Developed using: surveys, personal interviews

How validated? success rate in future activities

Content is personal, not learned

“Faking” can be an issue (e.g., “honesty” tests)

2 measuring test quality

Test reports can be “data dumps”

2. Measuring test quality

3 measures are important:

1. Predictive validity

2. Content coverage

3. Sub-group differences

p redictive validity values from 1 0 to 1 0

Predictive validity(values from -1.0 to +1.0)

…measures how well higher scores on admission test match better outcomes at university (e.g., grades, completion)

A test with low predictive validity provides a little information.

slide15

A positive correlation between two measures

Source: NIST, Engineering Statistics Handbook

slide16

A negative correlation between two measures

Source: NIST, Engineering Statistics Handbook

slide17

No correlation between two measures

Source: NIST, Engineering Statistics Handbook

slide18

Howdoesonemeasurepredictivecapacity?CorrelationCoefficient: I--------------------------------------------I-1 0 1

slide19

Predictive validities: SAT and PSU

SOURCE: Pearson, Final Report Evaluation of the Chile PSU, January 2013

slide20

Predictive validities: SAT and PSU

(faculty: Administracion)

SOURCE: Pearson, Final Report Evaluation of the Chile PSU, January 2013