1 / 26

The Effects of Student Coaching: An Evaluation of a Randomized Experiment in Student Mentoring

The Effects of Student Coaching: An Evaluation of a Randomized Experiment in Student Mentoring. October 3, 2012. Eric Bettinger, Stanford University Rachel Baker, Stanford University. Defining the problem: Trends in college attendance and completion. More students are taking classes online.

zenda
Download Presentation

The Effects of Student Coaching: An Evaluation of a Randomized Experiment in Student Mentoring

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Effects of Student Coaching: An Evaluation of a Randomized Experiment in Student Mentoring October 3, 2012 Eric Bettinger, Stanford University Rachel Baker, Stanford University

  2. Defining the problem: Trends in college attendance and completion

  3. More students are taking classes online SOURCE: Going the Distance: Online Education in the United States, 2011 Survey by Babson Survey Research Group

  4. College attendance in the United States has consistently increased over the last four decades SOURCE: The College Board, Trends in College Pricing 2010, Figure 17A and Figure 17B.

  5. College completion has not SOURCE: Turner 2004.

  6. Voicing concerns about completion • President Obama (2011): “This country needs and values the talents of every American. That is why we will provide the support necessary for you to complete college and meet a new goal by 2020: America will once again have the highest proportion of college graduates in the world.” • Vice President Biden (2011): “We have to make the same commitment to getting folks across the graduation state that we did to getting them to the registrar’s office.”

  7. Why do students not complete college? • Financial barriers/liquidity constraints (e.g. Dynarski & Deming 2010) • No access to appropriate channels of information (e.g. Bettinger, Long, Oreopolous and Sanbonmatsu 2010) • Weak academic preparation and performance (e.g. Adelman & Gonzalez 2006) • Lack of social and academic integration (e.g. Bloom & Sommo 2005, Tinto 1975)

  8. Today’s Focus is on Student Coaching • What is coaching? • Individualized instruction/guidance aimed at helping students overcome barriers • Why coaching? • Help students to build study skills • “Nudge” students to complete complex tasks • Provide information related to college success • Previous studies have looked at similar interventions

  9. InsideTrack • Student coaching service • Business model focuses on being an external, third-party advising service • Claim to build an economy of scale for counseling services • Partners with a number of types of institutions • Most students are studying in vocational tracks.

  10. InsideTrack’s Coaching • Emphasis on training and hiring coaches • Coaching takes place via phone, email, and text. • Coaching is “Active” not “Passive” Our key goal is to identify the effects of this coaching on student retention.

  11. Our Experiment • InsideTrack wanted to “prove” itself to college partners. They used randomized trials to show colleges their impact.

  12. Selection into Randomization • Colleges selected the number of students to be treated and submitted lists of students to InsideTrack. • InsideTrack randomly divided college lists into two groups.

  13. Selection into Randomization, con’t • InsideTrack presented the list to the schools. • Colleges chose which group would receive treatment.

  14. Basic Descriptive Statistics and Balance

  15. Distributions of Treatment and Control Groups Age Distributions SAT Score Distributions HS GPA Distributions

  16. Significant Differences by Lottery

  17. Methodology • Basic Regression Analysis Y = α + βTreatment + γ1Lottery1 +. . . + γ17Lottery17 +Xδ + ε Y is an outcome of interest focusing on retention after 6, 12, 18 or 24 months Treatment is a binary variable for being coached. Lottery# is a binary variable indicating student participation in a specific lottery. X is a vector of student characteristics

  18. Baseline Results with Covariates

  19. Robustness: Effects in Each Lottery

  20. Robustness:Effects Across Years

  21. Robustness:50/50 Splits

  22. Effects on Subgroups Effects by Gender

  23. Effects by Age Group Effects on Subgroups

  24. Cost-Benefit Analysis • Most studied intervention focused on retention is financial aid • Effect sizes are usually around 3 percentage points per $1000 in aid. • Effect is contemporaneous and doesn’t extend into future years. • InsideTrack cost about $1000 per year per student • Contemporaneous effect was about 5 percentage points • Effects persisted into subsequent year (3 percentage points)

  25. Conclusion/Discussion • College advisement is a widespread intervention • “Adult” learners are becoming an increasingly important group of students in higher education; effects were symmetric across age. • Online education is also rising; multiple campuses in our study were online campuses. • InsideTrack offers 3rd party advising/coaching • Attempts to exploit economy of scale. • Loosely affiliated with college. • Active rather than passive coaching. • Effects were large and cost effective

  26. Degree Completion • Degree completion information come from 3 lotteries • Definition of degree is generally four-year degree. It could include some two-year degrees. • Control Group Graduation Rate = 31.2% • Treatment Effect = 4.0% with standard error of (2.4%)

More Related