Student Quality and Reported Student Quality: Higher Education Admissions Decisions

1. Student Quality and Reported Student Quality: Higher Education Admissions Decisions Michael Conlin Michigan State University Stacy Dickert-Conlin Michigan State University Jeffrey Wooldridge Michigan State University Gabrielle Chapman Syracuse University IUPUI – October 4, 2007

2. Overview • Introduction to policy and goals • Data • Reduced form • Model • Future Work

3. Optional SAT Policies • Whether they get 1300 or 1250 doesn’t really tell you anything about them as a person or a student” says Ken Himmelman, Bennington dean of admissions. All the attention to numbers “becomes so crazy it’s almost a distraction.” • - Bruno in USA Today (2006)

4. Optional SAT Policies • “I SOMETIMES think I should write a handbook for college admission officials titled “How to Play the U.S. News & World Report Ranking Game, and Win!” I would devote the first chapter to a tactic called “SAT optional.” • The idea is simple: tell applicants that they can choose whether or not to submit their SAT or ACT scores.Predictably, those applicants with low scores or those who know that they score poorly on standardized aptitude tests will not submit. Those with high scores will submit. When the college computes the mean SAT or ACT score of its enrolled students, voilà! its average will have risen. And so too, it can fondly hope, will its status in the annual U.S. News & World Report’s college rankings.” • Colin Driver, President of Reed College, New York Times, 2006

5. Optional SAT Policies • The thesis, first stated last year by The New Republic, is that colleges are being less than honest about why they abolish requirements that applicants submit their SAT scores. Behind the rhetoric about "enhancing diversity" and creating a more "holistic approach" to admissions, the theory goes, many colleges "go optional" on the SAT to improve their rankings. The logic is rather simple: At an SAT-optional college, students with higher scores are far more likely to submit them, raising the institution's mean SAT score and hence the heavily test-influenced rankings. • Brownstein (2001) in The Chronicle of Higher Education

6. Prevalence of Optional Policy • As of Spring 2007, more than 700 colleges have SAT- or ACT- optional policies. • 24 of the top 100 liberal arts colleges ranked by U.S. News & World are SAT- or ACT- optional. (Bruno, 2006)

7. Research Questions • How much does a school attempt to maximize the quality of their student body compared to the reported quality of their student body? • How would the composition of the student body change if the schools did not consider rankings when making admission decisions? • Key: Use optional SAT Policy as means to identify • How does this policy inform the voluntary disclosure literature?

8. Data • Application data for 2 liberal arts schools in the north east • Each with approximately 1800 students enrolled. • Both report a typical SAT I score in the upper 1200s/1600 • College X: 2 years ≈ 5 years after the optional policy was instituted. • College Y the year after the optional policy was instituted.

9. College Board Data • SAT scores for those who elected not to submit them to the college. • Student Descriptive Questionnaire (SDQ) • SAT II Scores • Self Reported income • High school GPA • High school activities

10. Optional SAT I policies • College X required applicants to choose between submitting • the ACT scores • or three SAT II: Subject Tests • must submit one of the above if do submit SATI scores • 15.3 percent of the 7023 applicants choose option • At College Y, must submit • SAT II • ACT • three Advanced Placement (AP) exams or • a combination of the above testing requirements. • 24.1 percent of the 3054 applicants choose option

11. Voluntary Disclosure Models with Zero Disclosure Costs • Example: • Student i with given characteristics has the following probability distribution in term of SAT I scores: • School knows distribution of SAT I scores and applies Bayes Rule when inferring an SAT score for a student who doesn’t report. • Bayesian Nash Equilibrium results in every type except the worst revealing and the worst being indifferent between revealing and not revealing.

12. Voluntary Disclosure Models with Zero Disclosure Costs • Comments: • Distribution depends on student characteristics that are observable to the school such as high school GPA. • With positive disclosure costs, the “unraveling” is not complete and only the types with the lowest SAT I scores do not disclose. • Eyster and Rabin (Econometrica, 2005) propose a new equilibrium concept which they call cursed equilibrium.

13. Table 1

14. Table 1

15. Table 2

16. Reduced Form • Student’s Decisions: • Submit SAT I and Apply Early • Enroll • College’s Decisions: • Admit • Financial Aid Grant • Followup – Freshman GPA

17. Table 3Linear Probability - SUR(Dependent variable Don’t Consider SAT = 1)(Dependent variable Apply Early = 1)

18. Table 3Linear Probability (Dependent variable Don’t Consider SAT = 1)(Dependent variable Apply Early = 1)

19. Table 3Linear Probability (Dependent variable Don’t Consider SAT = 1)(Dependent variable Apply Early = 1)

20. Table 3Linear Probability (Dependent variable Don’t Submit SAT = 1)(Dependent variable Apply Early = 1)

21. Table 3Linear Probability - SUR(Dependent variable Don’t Consider SAT = 1)(Dependent variable Apply Early = 1)

22. Table 3Linear Probability (Dependent variable Don’t Consider SAT = 1)(Dependent variable Apply Early = 1)

23. Table 3Linear Probability (Dependent variable Don’t Consider SAT = 1)(Dependent variable Apply Early = 1)

24. TABLE 4Linear Probability (Dependent variable Accepted = 1)

25. Possible Explanations for Negative Coefficient Estimate Associated with Submit SAT1 • For those who don’t submit, school might be “overestimating” their score • Not submitting may be correlated with error term – applicants who do not submit are “more mature” or are athletes. • School is behaving strategically when deciding who to admit.

26. TABLE 4Linear Probability (Dependent variable Accepted = 1)

27. TABLE 4Linear Probability (Dependent variable Accepted = 1)

28. TABLE 4Linear Probability (Dependent variable Accepted = 1)

29. TABLE 5 – Tobit – College Y only(Financial Aid Award|Accepted)

30. TABLE 5 – Tobit – College Y only (Financial Aid Award|Accepted)

31. TABLE 5 – Tobit – College Y only (Financial Aid Award|Accepted)

32. Enrolled|Accepted • SAT Optional: negatively correlated (College Y) • Ability: negatively correlated (safety school) • Diversity: • race negatively correlated • Income positively correlated

33. Freshman GPA • Is there additional info (unobserved to us) contained in decision to withhold score that manifests itself in first year performance • Cannot reject 0 coefficient on SAT optional variable • + correlations: SATI (for College X), SAT 2, HS GPA, women • - correlations: minorities, lower income (College Y)

34. Literature on College Objective Function • Ehrenberg (1999) single well-defined objective function may explain “fairly well the behavior of small liberal arts colleges…” (page 101). • Epple, Romano, and Seig (2006) • GE model • students are matched with institutions of higher education, • how financial aid packages are selected for different students, and • how educational expenditures vary across schools. • assume a school maximizes quality(average quality of the student body, school expenditure per student, and the mean income of the student body) • s.t. balanced budget constraint and a fixed student body size. • Ours • allows uncertainty associated with student body size • Our objective function • reported average student body quality • gender and racial compositions of the student body

35. College’s Objective Function • The Liberal Art Colleges make decisions to maximize the mean quality of the student body in current and subsequent years • The school decides whether to admit each student i=1…N and how much financial aid to offer student i (fai) in order to maximize the objective function

36. Ψi is probability student i is admitted and attends • P, sticker price of college • fai, financial aid offered to student i • ΛP is the perceived average ability of the incoming students, • ΛR is the reported average ability of the incoming students • γ is the weight the college places on perceived quality relative to reported quality • f(Ω) is a function of the demographic characteristics of the student body.

38. Student’s maximization problem • Whether or not to apply early decision • Whether or not to submit SAT I score

39. Ψi is probability student i is admitted and attends • WTPi is student i’s willingness to pay to attend the school • Csub,I is i’s cost of submitting her SAT I score

40. Ψi (early decision, submit SAT I)

41. Conclusions • Students are strategic: Lower actual SAT I scores more likely to choose option • Same students have higher predicted scores • Colleges are strategic: Colleges are less likely to admit students who submit their SAT I scores, unless their SAT I scores are particularly high • Suggestive of maximizing reported quality • College goals appear to include diversity

42. Future Work • Estimate Model to determine the weight the college places on perceived quality relative to reported quality (γ). • Simulate the Model to determine how admissions would change if γ = 1 and how the admissions changes affect quality of student body. • Estimate how school infers SAT score for a student who doesn’t report (Bayesian Nash Equilibrium or Cursed Equilibrium?)

43. TABLE 6 - Linear Probability (Dependent variable Enrolled = 1|Accepted)

44. TABLE 6 - Linear Probability (Dependent variable Enrolled = 1|Accepted)

45. TABLE 6 - Linear Probability (Dependent variable Enrolled = 1|Accepted)

46. TABLE 6 - Linear Probability (Dependent variable Enrolled = 1|Accepted)

47. TABLE 7 – OLS Freshman GPA

48. TABLE 7 – OLS Freshman GPA

49. TABLE 7 – OLS Freshman GPA