SAT: Gender Gap & Bias in Standardized Testing

1. SAT: Gender Gap & Bias in Standardized Testing Heather Leigh Clark Ball State University November 22, 2004

2. SAT: Gender Gap & Testing Bias Although girls earn higher grades in high school and college, they consistently receive lower scores on the SAT 2001: 35 points lower in math, 3 points lower in verbal 2002: 34 points lower in math, 5 points lower in verbal Gender gap favoring males persists across demographic characteristics including family income, parental education, size of high school, course work, grade point average, etc.

3. SAT: Gender Gap & Testing Bias SAT is validated for one purpose only, and that is to predict first-year college grades However, SAT consistently underpredicts performance of females in college, while overpredicting the performance of males Testmakers acknowledge that high school GPA and class rank are much better predictors of future college success, despite large variation among high schools and courses

4. Confirmatory Research Evidence Gallagher (1998) Boys are socialized to be risk-takers, independent thinkers, and “take matters into their own hands” Girls tend to prefer cooperation vs. competition and are socialized to be more cautious and compliant Timed nature of standardized tests evokes competitive atmosphere, while unfamiliarity of some testing material requires more “risky”, less rule-governed problem-solving styles Tests are more conducive to masculine culture

5. Confirmatory Research Evidence Luthy (1997) Investigated two questions about standardized tests -GRE scores valid predictors of grad school success? -Do GRE scores predict success similarly for different groups of students? Results indicated existence of gender bias as well as age bias in prediction of graduate GPA from GRE scores Specifically, GRE scores over-predicted graduate performance of male students and under-predicted graduate performance of female students

6. SAT 2005: Will This TestBe Less Biased? Addition of new essay question should slightly reduce gender gap, but elimination is unlikely since underlying causes will remain in place biased test questions multiple-choice format highly-speeded pace rewards for strategic guessing

7. Problematic Test Features:Biased Test Questions Males – perform better on questions dealing with sports, physical sciences, business Females – perform better on questions dealing with relationships, aesthetics, humanities Vast majority of questions with large gender differences in correct answer rates seem to be biased in favor of males “Balancing” – gender differences can be minimized by selecting different test items

8. Problematic Test Features:Multiple-Choice Format ETS and College Board issued joint study, in which multiple-choice format was found to be biased against females Study found that, with exception of multiple-choice, gender gap narrowed or disappeared completely on all other types of questions short answer written essay constructed response

9. Problematic Test Features: Highly Speeded Pace Research suggests that girls approach problem-solving differently than boys more likely to work problem out completely more likely to consider more than one answer more likely to double check their answers These are desirable traits in school and life, but they work against girls on timed tests When time constraint is lifted, female scores improve, while male scores remain the same

10. Problematic Test Features:Rewards for Strategic Guessing SAT is scored with “guessing penalty”, which makes random guessing inadvisable Penalty = 1/4 point deducted for each incorrect answer Yet, one or two answer choices can usually be eliminated as obviously incorrect, so it is often best to make an educated guess Boys are more likely to take testing “risks” Girls tend to answer only when they are confident (omitted questions receive 0 points)

11. Testing Bias Ramifications Testmakers admit that student scores must differ by at least 125 points to be considered reliably “different” due to measurement error SAT scores are not precise enough to serve as sole basis for high-stakes decision making, yet rigid cutoff scores are often used in this way Natinal Merit Scholarship Selection NCAA Athletic Scholarships & Membership Gifted and Talented Program Admission

12. Testing Bias Ramifications National Merit Scholarship Selection Preliminary SAT scores serve as sole criterion in semifinalist selection process Result = predominantly male semifinalist pool (boys score higher on PSAT even though girls have higher grades in high school) Lawsuit by FairTest charged testmakers with illegally perpetuating gender discrimination ETS added writing component to PSAT - increased % of female semifinalists but did not eliminate gender gap

13. The Other Side of the Story Critics argue that discrepancy in standardized test scores is not the result of gender bias Greater percentage of girls considered to be “at-risk” or of low socioeconomic status take SAT “Spread” phenomenon – male scores tend to be at both ends of spectrum suggesting they are more likely to exhibit extremes of high ability as well as disability Regardless of test scores, girls are still more engaged in their classes, more invested in their education, and more interested in taking leadership roles in school Females are also the majority sex in higher education

14. Contradictory Research Evidence Boldt (2000) Noncognitive Questionnaire (NCQ) was developed to address criticism concerning predictive potential of SAT NCQ was thought to tap into psycho-social variables linked to academic ability and college success for women, ethnic minorities, and non-traditional students Research results suggested that NCQ variables were not as strong or consistent predictors as were SAT scores

15. Contradictory Research Evidence Wright, Palmer, & Miller (1996) Investigated the predictive validity of SAT scores by comparing academic performance of males and females in college marketing class No significant differences were found indicating that gender based variations in predictive ability of SAT may be due to factors other than gender bias in test Benbow & Wolins (1996) Results suggested that the math section of SAT did not contain item bias and overall instrument contamination regarding gender bias was not supported

16. Does a Solution Exist? Due to concern regarding bias in standardized testing, nearly 400 universities no longer use SAT or ACT to make admissions decisions for substantial number of applicants Since 1998, applicants to public universities in Texas are not required to submit standardized test scores, if they are ranked in top 10% of high school class Those still convinced of value of standardized testing are encouraged to use ACT or SAT II, which are more closely aligned to school curricula and less susceptible to bias

17. References Benbow, C. P. & Wolins, L. (1996). The utility of out-of-level testing for gifted seventh and eighth graders using the SAT-M: An examination of item bias. Intellectual Talent: Psychometric and social issues. Baltimore, Maryland: Johns Hopkins University Press. Boldt, K. E. (2000). Predicting academic performance of high- risk college student using Scholastic Aptitude Test scores and noncognitive variables. Dissertation Abstracts International, 61, p. 2181. Fair Test: The National Center for Fair & Open Testing. www.FairTest.org. Updated August 28, 2001.

18. References Gallagher, A. (1998). Gender and antecedents of performance in mathematics testing. Teachers College Record, 100. Hoff-Sommers, C. (2000). The war against boys: How misguided feminism is harming our young men. New York, NY: Simon & Schuster. Luthy, T. L. (1997). Validity and prediction bias in grade performance from Graduate Record Examination scores for students at Northern Illinois University: Age and gender considerations. Dissertation Abstracts International, 57, p. 5018.

19. References Orenstein, P. (2000). Schoolgirls: Young women, self-esteem, and the confidence gap. New York, NY: Random House. Paludi, M. A. (1998). The psychology of women. Upper Saddle River, NJ: Prentice Hall, Inc. Sattler, J. M. (2001). Assessment of Children: Cognitive Applications (4th ed.). San Diego, CA: Jerome M. Sattler, Publisher, Inc. Wright, R. E., Palmer, J. C., & Miller, J. C. (1996). An examination of gender-based variations in the predictive ability of the SAT. College Student Journal, 30, p. 81-84.