1 / 20

Andrew LaManque, PhD Supervisor, Institutional Research De Anza College

Evaluating a NonRandomized Trial: A Case Study of a Pilot to Increase Pre-Collegiate Math Course Success Rates. Andrew LaManque, PhD Supervisor, Institutional Research De Anza College. Outline. Overview of the Pilot Purpose of the Evaluation Results from Quantitative Analysis

taing
Download Presentation

Andrew LaManque, PhD Supervisor, Institutional Research De Anza College

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluating a NonRandomized Trial: A Case Study of a Pilot to Increase Pre-Collegiate Math Course Success Rates Andrew LaManque, PhD Supervisor, Institutional Research De Anza College De Anza Math Pilot Results, 7-9-07

  2. Outline • Overview of the Pilot • Purpose of the Evaluation • Results from Quantitative Analysis • Demographics of Students • End of Class Survey • Course Success • Findings De Anza Math Pilot Results, 7-9-07

  3. A Partnership was formed to increase the success rates in Pre-Collegiate math courses Components: Math and Counseling faculty EnableMath – Course Software Noel Levitz – College Student Inventory Using technology, provide students timely information on their progress as well as information about their motivations Overview of the Pilot De Anza Math Pilot Results, 7-9-07

  4. Developmental math sequence consists of pre, elementary, and intermediate algebra Offer 175 class sections per year of developmental math (37% of all math department courses) Historically, a 55% success rate in developmental math courses 31 full-time instructors, 45 part-time instructors Class size is 40 Overview of the Pilot De Anza Math Pilot Results, 7-9-07

  5. Program Evaluation • Given that the pilot program provides an additional learning choice for students, the question from the Dean is: • “do the results demonstrate that pilot students do at least as well as other students” • Using multiple forms of evaluation (e.g. student grades, student surveys) provides feedback to instructors as well as administration on implementation De Anza Math Pilot Results, 7-9-07

  6. Program Evaluation • Limitations • Not a randomized trial – possible instructor (although, 12 different instructors participated) and student self selection bias • No independent measure of success • More research is needed to determine the relative effect of each intervention De Anza Math Pilot Results, 7-9-07

  7. Results: Demographics of Students Twice the % of Latino students as college, similar to non-pilot sections De Anza Math Pilot Results, 7-9-07

  8. Results: Demographics of Students Many first-generation college students De Anza Math Pilot Results, 7-9-07

  9. Results: Demographics of Students • Similar with regard to Gender, Initial Education Goal and Ethnicity • although Math114 EnableMath had a higher percentage of minority students (54%) compared with the other sections (34%). • EnableMath students are slightly older on average (45% vs 37% born prior to 1987). De Anza Math Pilot Results, 7-9-07

  10. Results: End of Class Survey De Anza Math Pilot Results, 7-9-07

  11. Results: End of Class Survey 77% Agree or Strongly Agree De Anza Math Pilot Results, 7-9-07

  12. Results: End of Class Survey 61% Agree or Strongly Agree De Anza Math Pilot Results, 7-9-07

  13. De Anza College Math Pilot Sections / Students, Fall 05 - Spr 07 Course Group # Sec HC * Pre-Algebra Pilot 14 428 Other 55 1,755 Elem Algebra Pilot 21 723 Other 88 3,045 Interm Algebra Pilot 27 853 Other 112 3,573 Total Pilot 62 2,004 Other 255 8,373 * HC for the Pilot only includes students completing the CSI. Results: Course Success Approx 1 of 5 students De Anza Math Pilot Results, 7-9-07

  14. Course Success of Fall 2005-Spring 2007 EnableMath / Noel Levitz Sections Compared to All Other Sections, De Anza College Pass Did Not Pass Withdrew Total Course Group Grades Percent Grades Percent Grades Percent Grades Percent Pre Algebra Pilot 308 72% 76 18% 44 10% 428 100% Other 1,012 58% 391 22% 352 20% 1,755 100% Elementary Algebra Pilot 514 71% 138 19% 71 10% 723 100% Other 1,660 55% 664 22% 721 24% 3,045 100% Intermediate Algebra Pilot 641 75% 140 16% 72 8% 853 100% Other 1,977 55% 675 19% 921 26% 3,573 100% Includes end of term grades of students enrolled at census (3rd week). Note: Students in the Pilot group used the EnableMath Software and completed the CSI. Results: First Course Success De Anza Math Pilot Results, 7-9-07

  15. De Anza College EnableMath Pilot, Fall 2005-Winter2007 Second Course Pass Rate for Students Successful in First Course First Course Pass Did Not Pass W Total Course Group Grades % Row Grades % Row Grades % Row Grades % Row Pre to Elementary Algebra Pilot 94 45% 59 28% 57 27% 210 100% Other 305 55% 106 19% 148 26% 559 100% Elementary to Intermediate Algebra Pilot 168 64% 45 17% 49 19% 262 100% Other 545 66% 137 17% 146 18% 828 100% Intermediate Algebra to College Pilot 85 50% 36 21% 48 28% 169 100% Other 301 56% 97 18% 142 26% 540 100% Note: Considers only the first attempt of both courses. Results: Second Course Success De Anza Math Pilot Results, 7-9-07

  16. De Anza College EnableMath Pilot, Fall 2005-Winter2007 Second Course Pass Rate for Students Successful in First Course by Group First Course Second Course Pass Did Not Pass W Total Course Group Group Grades % Row Grades % Row Grades % Row Grades % Row Pre to Elementary Algebra Pilot Pilot 60 52% 36 31% 20 17% 116 100% Control 34 36% 23 24% 37 39% 94 100% Control Pilot 47 64% 14 19% 12 16% 73 100% Control 258 53% 92 19% 136 28% 486 100% Elementary to Intermediate Algebra Pilot Pilot 97 80% 17 14% 7 6% 121 100% Control 71 50% 28 20% 42 30% 141 100% Control Pilot 95 74% 24 19% 9 7% 128 100% Control 450 64% 113 16% 137 20% 700 100% Results: Second Course Success De Anza Math Pilot Results, 7-9-07

  17. Findings • Students currently in the project want to take next class in sequence • These classes are filled before other math classes • Participating instructors & counselors want to continue • More faculty are wanting to participate De Anza Math Pilot Results, 7-9-07

  18. Findings • No increased costs – taking existing resources and using them differently • Regular class period / time • Counseling time in the class instead of counseling center De Anza Math Pilot Results, 7-9-07

  19. Findings • Overall, the data suggests that there is value in offering a different approach to students, but claims of improved learning, especially the long term retention of material, can not be substantiated. • Difficult to evaluate a non-randomized trial • Math department might focus on improving the transition in pilot sections from Pre-Algebra to Elementary Algebra De Anza Math Pilot Results, 7-9-07

  20. Questions For a copy of this presentation please email Andrew LaManque at: lamanqueandrew@fhda.edu De Anza Math Pilot Results, 7-9-07

More Related