1 / 34

English Assessment Validity February 2012

English Assessment Validity February 2012. Michael Orkin, Jo Ann Phillips, Hui Zhang, Sheryl Queen Office of Institutional Research Peralta Community College District. English Assessment Test. New Peralta students take assessment tests for placement in English classes.

xaria
Download Presentation

English Assessment Validity February 2012

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. English Assessment Validity February 2012 Michael Orkin, Jo Ann Phillips, Hui Zhang, Sheryl Queen Office of Institutional Research Peralta Community College District

  2. English Assessment Test • New Peralta students take assessment tests for placement in English classes. • Course placement from test scores is recommended but not mandatory. • A student can take courses without placement tests if prerequisites are met.

  3. ACT Compass Test • Peralta currently uses the ACT Compass test, CENG, for English placement. • CENG tests English proficiency with separate writing and reading tests. • This will soon change...

  4. Statewide Online Assessment • From Student Success Task Force, January, 2012: “… all students will have access to common online assessment tools for English, mathematics and ESL and to pre-testing programs that help improve assessment outcomes. They also will be able to take the results of their tests to any community college in the state.”

  5. For now: ACT Compass TestPlacement in English Courses A decision rule based on the two test scores selects an English course: • transfer level (English 1A) • degree applicable (English 201A) • Pre-collegiate (Basic Skills).

  6. Placement in English Course • If the writing score is less than 22, the student is not assessed into a course and must see a counselor. • 699 students in the sample data had low writing scores and were excluded.

  7. Data • Students taking CENG-recommended English courses in Fall 2008, Spring 2009, Fall 2009, Spring 2010. • Took test immediately preceding semester when they took English course. • Never took test prior to semester when they took English course.

  8. Validity Validity measured in three ways: • Relationship between assessment scores and grades. • Comparison of assessed and non-assessed. • Comparison of students who took the “right” course (recommended by CENG) and “wrong” course (not recommended by CENG).

  9. Assessed and Took Right Course * English 252AB, 267AB, 268AB, 269AB

  10. Assessed - Grade %

  11. Placement and Grades • 57% were placed in English 1A. • 47% overall got A’s or B’s. • 25% overall withdrew from class.

  12. Grades vs. Test Scores Are assessment test scores positively correlated with grades? We use ANOVA (Analysis of Variance).

  13. Reading Test Scores by Grade

  14. All Test Scores by Grade

  15. Grades vs. Test Scores Analysis • Reading test scores most sensitive to grades. • Students who get A’s have higher average test scores than those who have lower grades. • Writing test and Combined Tests only significant for English 1A.

  16. Predicted Chance of Success As test scores get higher, does predicted chance of success increase? We use logistic regression.

  17. Reading Test Predicted Chance of Success

  18. Reading Test Predicted Chance of Success

  19. Reading Test Predicted Chance of Success

  20. Predicted Chance of Success As test scores get higher, predicted chance of success increases but … • Only for reading test scores. • Different courses are selected across range of scores so much is filled in by software. • Results aren’t statistically significant.

  21. Assessment vs. No Assessment Do assessed students do better in their assigned courses than students who don’t take the assessment test? We use ANOVA and Chi Square tests.

  22. Counts * English 252AB, 267AB, 268AB, 269AB

  23. Not Assessed - Grade % (Similar to Assessed)

  24. Assessed vs. Not Assessed GPA

  25. Assessed vs. Not Assessed GPA No significant results: • Students who are assessed have similar class GPA as those who are not assessed.

  26. Assessed vs. Not Assessed Success

  27. Success - Assessed vs. Not Assessed Significant only at basic skills level: • Students who are assessed have greater success than those who are not assessed.

  28. Assessed – “Right” vs. “Wrong” Course Do assessed students who take the right course do better than assessed students who take the wrong course? We use Chi Square and ANOVA.

  29. Counts - Right vs. Wrong Course

  30. Success - Right vs. Wrong

  31. Success - Right vs. Wrong Significant only for English 1A: • Assessed students who take right course have greater success than those who take wrong course.

  32. GPA - Right vs. Wrong

  33. GPA - Right vs. Wrong Significant only for English 1A: • Assessed students who take right course have higher GPA than those who take wrong course.

  34. Conclusions • Validity confirmed. • Useful for counselors. • Assessed results aren’t much different from non-assessed, pre-requisite method. • Most students take English 1A. • Students assessed into English 1A do significantly better than “wrong” students.

More Related