Using Demographic Data to Predict Students’ Achievement in DSPM Courses

1. Using Demographic Data to Predict Students’ Achievement in DSPM Courses Daryl Stephens, ETSU stephen@etsu.edu TNADE October 27, 2005

2. Students take developmental studies program (DSP) courses for many reasons: • Forgot material learned in high school by the time they took entrance tests • Long gap between high school and matriculation • Didn’t take high school seriously • Didn’t consider going to college until later • First generation college student (Salter & Noblett, 1994)

3. Most students (~90%) at ETSU take MATH 1530, Probability and Statistics, as their mathematics course to satisfy core curriculum requirements. • In previous years, this course had a high failure rate.

4. The Problem • Placement procedures for TBR schools have changed. • Pre-2000: ACT < 19 or age > 22 → take AAPP • 2000-2002: COMPASS replaces AAPP • 2002: ACT score alone determines placement for students with score < 3 years old; COMPASS alone for others

5. The Problem • State funding is static. • Placement decisions based on one test. • Enrollment caps may happen in the future. • Is there a way to augment the placement process by predicting a student’s chance of success or failure in developmental or core math classes?

6. Purpose • Develop models to predict success of students in • DSPM 0800 (Elementary Algebra) • DSPM 0850 (Intermediate Algebra) • MATH 1530 (Probability and Statistics) • Use multiple regression to develop the models using readily obtainable information

7. Importance • MATH 1530 traditionally had high failure rate • Developmental math students are at a greater risk of failure and dropping out • Specific developmental studies program advisors done away with in budget crunch of 2003

8. Importance • Previous similar studies on developmental students used additional instruments which cost money. • Very few universities require probability and statistics for math credit for graduation. Most similar studies on core math deal with college algebra, precalculus, or math survey courses.

9. Assumptions • Prediction is possible • Self-reported data are correct • Information in SIS is correct

10. Limitations • Different demographics and course requirements from other institutions, so work only applies to ETSU • Data only collected for fall; spring and summer grades are probably different

11. Definitions • DSPM 0800: Elementary algebra • Arithmetic review, algebraic representations, linear equations in one and two variables • DSPM 0850: Intermediate algebra • Exponents, polynomials, factoring, • MATH 1530: Probability and statistics • “Stat mansion” and “stat cave”

12. Definitions • COMPASS (Computerized Adaptive Placement Assessment and Support System • Prealgebra and algebra sections • Reading and writing sections • Replaced AAPP

13. Previous Research

14. Developmental Studies History • Special programs at Harvard in 17th century • Preparatory department at University of Wisconsin, 1849 • Morrill Acts, late 19th century, establish land grant colleges and extend access to higher education to more people

15. Developmental Studies History • Preparatory departments widespread in early 20th century (350 in 1915) • GI Bill of Rights brings in veterans with needs for auxiliary services • 1960s-70s: Increase of women, students of color, first-generation students, students with learning disabilities; open-access community colleges established

16. Tennessee DSP History • TBR establishes formal developmental studies program in 1984. • Defining Our Future (2001): “operate more efficiently and with more limited resources” • Move 0700-level courses to community colleges

17. Related Research • Developmental Math • Core Math (almost nothing on P&S) • What factors are related to success in the courses? Three broad categories: • Academic • Demographic • Affective

18. What variables predict course success in dev. math? • High school GPA, at least for traditional students • Scores on the ACT or SAT — sometimes • Placement tests (e.g. COMPASS, ASSET, Accuplacer, CPT, PTT, CLAST) — sometimes • GED math scores — as adjunct to other placement scores

19. Course success in dev. math • Number of high school mathematics courses taken — sometimes • High school math GPA – some courses • College GPA • Attendance • Study habits • Age or length of time since last math class

20. Course success in dev. math (cont’d) • Gender (some studies) • Race • Math anxiety level not related to grade! • Attitude • Perception of success, engagement in class • Paying attention and interacting with instructor

21. Success in core classes • Very little research on success in classes like MATH 1530, so other courses examined • ACT/SAT math scores (usually) • COMPASS, TASP, local placement tests • High school GPA for college algebra (multiple studies) and calculus (but not precalculus)

22. Success in core classes (contd.) • HS math GPA – mixed results • HS percentile rank • Number and difficulty of HS math classes taken • Whether a math class taken senior year • Students who didn’t take intermediate algebra scored sig. higher in college alg.

23. Success in core classes (contd.) • Age in some cases • Time since last math course • Gender? Yes in 3, no in 5 studies • Full-time vs. part-time (1 study) • Class meeting time (1 study) • No difference in resident vs. commuter, campus activity

24. Success in core classes (contd.) • Attitude • Learning styles • Self-concept

25. Research Questions

26. Relationship Questions Is there a relation between course grade and … • ACT (DSPP) math scores? • ACT (DSPP) reading scores? • COMPASS intermediate algebra scores? • COMPASS reading scores? • Number of college preparatory math classes taken in high school? • High school GPA?

27. From SIS: ACT composite ACT math ACT reading ACT English High school GPA Age on first day of class From Survey: Number of high school math classes taken Number of years since last HS math class Regression QuestionsCan a regression equation be found to predict final course grade based on these items?

28. From SIS: COMPASS writing COMPASS reading COMPASS prealgebra COMPASS intermediate algebra Age on first day of class From Survey: Number of high school math classes taken Number of years since last HS math class Regression QuestionsCan a regression equation be found to predict final course grade based on these items?

29. Method • Collect information about courses taken in high school and year of last high school math class from students via survey • Obtain other information from SIS • Use Pearson correlation for relationship questions • Use stepwise multiple regression for grade prediction questions

30. Initial Placement • ACT (DSPP) Math section • < 17 (SAT < 440): DSPM 0800 • 18 (SAT 450): DSPM 0850 • >19 (SAT > 450): college level math • COMPASS • Prealgebra score < 29: DSPM 0700 • Prealgebra 30-99 and algebra 20-27: DSPM 0800 • Algebra 28-49: DSPM 0850 • Algebra 50-99: college level math

31. Data considerations • Not counted: grades of W, WF, I • FN grade not counted to be consistent with some other ETSU studies • Online sections, RODP sections not included

32. Surveys returned • DSPM 0800: 149 / 304 (49%) • No night, off-campus, or ITV sections • DSPM 0850: 214 / 455 (47%) • Included Kingsport, night, ITV • MATH 1530: 631 / 1074 (59%)

33. Results

34. Grade Distribution Table 3 * Grades of C-, D+, and D are not allowed in developmental studies courses. **Not used in answering the research questions.

35. ACT Math vs. Grade

36. ACT Reading vs. Grade

37. COMPASS Int. Alg. vs. Grade

38. COMPASS Reading vs. Grade

39. High School Math Courses(Most Common)

40. Course Grade vs. Number of College Prep Classes Taken

41. Course Grade vs. Number of College Prep Classes Taken

42. Course Grade vs. Number of College Prep Classes Taken

43. HSGPA vs. Grade

44. Regression Using ACT DSPM 0800 (95 students) • Model 1: ŷ = –.589(HSMATH) + 4.599 (p = .001) • Model 2: ŷ  = –.765(HSMATH) + 1.009(HSOGPA) + 2.298 (p < .001)

45. Regression Using ACT DSPM 0850 (160 students) • Model 1: ŷ  = .364(ACTM) – 3.238 (p < .001) • Model 2: ŷ = .301(ACTM) + .662(HSOGPA) – 4.111 (p < .001)

46. Regression Using ACT MATH 1530 (475 students) Four models (p < .001 in each case; r2 ranging from .233 to .353): • ŷ  = .134(ACTM) – .521 • ŷ  = .092(ACTM) + .771(HSOGPA) – 2.181 • ŷ  = .103(ACTM) + .910(HSOGPA)  +  .108(AGE) –  4.953 • ŷ  = .089(ACTM) + .855(HSOGPA)   +  .105(AGE) + .025(ACTE) – 4.9

47. Regression Using COMPASS • DSPM 0800 (33 students) – no equation possible • DSPM 0850 (23 students with all sections) ŷ = .020 (COMPASS writing) + 1.374 (r2 = .221, p = .024) • DSPM 0850 (44 students with math scores) ŷ = .023 (COMPASS arithmetic) + 1.730 (r2 = .190, p = .033)

48. Regression Using COMPASS MATH 1530 • 22 students with all COMPASS sections: no equation possible • 51 students with just math sections: ŷ = .027(COMPASS arithmetic) + .671 (r2 = .276, p < .001)

49. Observations, Conclusions, and Recommendations

50. Observations: 0800 • Elementary algebra: only high school GPA showed a significant correlation with course grade; math preparation in high school showed a little bit of predictive value. • Finding agrees with Long (2003) – ACT math score not significantly correlated with E.A. course grade