The Creation of Two Comparison Groups to Facilitate Evidence-Based Decision Making

1. The Creation of Two Comparison Groups to Facilitate Evidence-Based Decision Making Prepared and Presented by Keith Wurtz Dean, Institutional Effectiveness, Research and Planning Crafton Hills College

2. Session Objectives • Provide a brief introduction to the Left Lane Program • Explain where the idea for two comparison groups came from • Describe the two comparison groups and the limitations of the study • Describe the findings • Review the process for Continuous Quality Improvement and how the results were used to inform decision-making

3. The Left Lane Program (LLP) • Purpose of the LLP is to reduce the amount of time it takes students to earn an AA/AS Degree at Crafton from 5.3 years to 4 years • Program funded by the San Bernardino Community College District Chancellor • First Cohort in Fall 2012

4. The Left Lane Program (LLP) • Students who participate in Left Lane receive Priority Registration B • To keep the Priority Registration B status students are required to do the following • Attend a Summer Bridge Program • Participate in SOA3R – Student Orientation, Application, Assessment, Advisement, and Registration • Complete and closely follow their Student Education Plan • Enroll in math and English as stated in their SEP • Attend all tutoring meetings and check-in appointments • Maintain a 2.0 GPA • Enroll in 12 or more units per semester (depending on SEP)

5. The Left Lane Program (LLP) • How are students selected for the LLP? • Need to place into one level below transfer English or one level below transfer level math • Need to apply

6. The Idea for Two Comparison Groups Collaborative Model of Institutional Research Kirsten Colvey Dean of Student Services, Counseling, and Matriculation Raju Hegde Dean of Math, English, Reading, and Instructional Support Primary Responsibility Faculty/Staff/Student Researcher Joint Activity • Key Features: • Dialogue-rich • Jointly-driven processes Source: Dr. Rob Johnstone and Priya Chaplot, RP Group Conference Presentation

7. The Two Comparison Groups • Left Lane Students • 258 out of 276 (93%) earned a GOR in Fall 2012 • 82% graduated high school in 2012 • 63% Female • 49% Hispanic • First Comparison Group (Last Year High School was 2012) • Second Comparison Group (Last Year High School was 2012 and did not participate in LLS)

8. The Two Comparison Groups

9. The Two Comparison Groups

10. Limitations • The success rate comparisons do not control for instructor or by when the section was offered • The comparison groups do not control for skill level (i.e. math and English placements)

11. Percent of Students Earning a Grade on Record in English and Math Left Lane students were substantially (Effect Sizes were > .67) and statistically significantly (p values were < .001) more likely to earn GOR in English and Math.

12. Percent of Students Successfully Completing English and Math • Left Lane students were slightly more likely to successfully complete English (ES = .10, p = .308) and math (ES = .11, p = .196) than students in the first comparison group • Left Lane students were substantially and statistically significantly more likely to successfully complete English (ES = .32, p = .020) and math (ES = .27, p = .017) than students in the secondcomparison group

13. Percent of Students Successfully Completing English, Math, and/or Reading • Left Lane female (ES = .20, p < .001, 81% to 73%), African American (ES = .43, p = .007, 81% to 61%), and Hispanic (ES = .19, p < .001, 79% to 70%) students were substantially and statistically significantly more likely to successfully complete courses than students in the first comparison group • Left Lane female (ES = .31, p < .001, 81% to 68%), African American (ES = .77, p < .001, 81% to 45%), and Hispanic (ES = .27, p < .001, 79% to 66%) students were substantially and statistically significantly more likely to successfully complete courses than students in the second comparison group

14. English, Math, and Overall Retention (i.e. Persistence – earning GOR) from Fall to Spring • Left Lane English (ES = .25, p = .007), Math(ES = .19, p = .034), and overall (ES = .19, p < .012) retention was substantially and statistically significantly higher for Left Lane students than students in the first comparison group • Left Lane English (ES = .39, p = .002), Math(ES = .27, p = .018), and overall (ES = .31, p < .001) retention was substantially and statistically significantly higher for Left Lane students than students in the second comparison group

15. English and Math Improvement Rates from Fall to Spring Left Lane students did not perform as well in their subsequent English and/or math course after successfully completing an English and/or math course in their first semester

16. Process for Continuous Quality Improvement Communicate Results Communicate Improvements Continuous Quality Improvement Communicate Strategies Communicate Feedback

17. Left Lane Program

18. Left Lane Program

19. A Note on Facilitating Continuous Quality Improvement • Traditional approach to facilitating evidence-based decision making • Provide the research results to constituents only in reports, briefs, and dashboards • Discuss the possible implications and interpretations verbally • Example • Left Lane students were substantially (ES = 1.10) and statistically significantly (p < .001) more likely to earn a GOR in a math course their first semester (89%) at Crafton than non-Left Lane students (34%) • Left Lane students were substantially (ES = 1.03) and statistically significantly (p < .001) more likely to earn a GOR in an English course their first semester (77%) at Crafton than non-Left Lane students (26%)

20. A Note on Facilitating Continuous Quality Improvement New approach to facilitating evidence-based decision making • Provide the research results to constituents in reports, briefs, and dashboards, and… • Provide the research results with an interpretation that focuses on the possible implications of the research Left Lane Comparison Groups and Possible Improvement Areas When preparing the comparison groups to examine the relationship between Left Lane Students and student outcomes one of the most interesting findings was the number of Fall 2012 first-time Crafton students who graduated from high school in 2012 who did not participate in Left Lane, but had participated in one of the Left Lane Strategies (LLS). Thirty-nine percent (n = 226) of these students participated in SOA3R, a learning community, or in supplemental instruction. As a result, the first comparison group of incoming students had success rates that were 70% or higher. Moreover, the Left Lane students often did not have a statistically significantly higher likelihood of achieving the outcomes examined than the first comparison group. Suggesting the possibility that one method for improving the LLP is to focus on ensuring that as many students as possible participate in the LLS rather than solely on LLP. One of the strategies for decreasing the average amount of time it takes students to complete an AA/AS Degree involves increasing how quickly students are able to successfully complete transfer level English and math. The initial results of the research illustrated here indicates that Left Lane students are more likely to enroll in math and English courses in the fall, but in some instances they were not as likely to enroll in math and English in the subsequent semester and/or successfully complete the subsequent English and/or math course in the summer semester. LLP may need to require students to participate in the LLS in the spring semester and devote additional resources to services in the spring semester.

21. Left Lane Program

22. Left Lane Program

23. Additional Questions