U s news world report s american s best colleges ranking methodology
This presentation is the property of its rightful owner.
Sponsored Links
1 / 34

U.S. News & World Report’s American’s Best Colleges Ranking Methodology PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on
  • Presentation posted in: General

U.S. News & World Report’s American’s Best Colleges Ranking Methodology. Office of Institutional Research Song Yan, Kristy Maxwell, Mark A. Byrd Associate Director Senior Research Analyst AVP . Wayne State University. Agenda. U.S. News Best Colleges ranking background

Download Presentation

U.S. News & World Report’s American’s Best Colleges Ranking Methodology

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


U s news world report s american s best colleges ranking methodology

U.S. News & World Report’s American’s Best Colleges Ranking Methodology

Office of Institutional Research

Song Yan, Kristy Maxwell, Mark A. Byrd

Associate Director Senior Research Analyst AVP

Wayne State University


Agenda

Agenda

  • U.S. News Best Colleges ranking background

  • Methodology:

    • Details on how the Best Colleges rankings are calculated

    • How each ranking variable is computed and how each estimate is calculated

  • New changes to the rankings

  • Criticism of the rankings


Background

Background

  • First ranking published in 1983

  • It’s purpose was to be a starting point for families searching for a college

  • Allows for comparison of relative cost, “quality,” retention and graduation rates

  • In September 2011, U.S. News Education section had 4.5 million unique visitors (announced by U.S. News)

  • Rankings published by U.S News in 2012:

    • Best Colleges, September 2012

    • Best Grad Schools, March, 2012

    • Best High Schools, May 2012

    • Top online Programs, January, 2012 (first time)

    • World’s Best Universities, October 2012


Best colleges ranking categories

Best Colleges Ranking Categories

  • Ranking categories:

    Based on Carnegie Basic Classification

    • National Universities: (Offer a full range of undergraduate majors, plus master’s and Ph.D. programs, and faculty emphases on research)

    • National Liberal Arts Colleges (focus almost exclusively on undergraduate education)

    • Regional Universities (offer a broad scope of undergraduate degrees and some master’s degree programs, but a few, if any, doctoral programs)

    • Regional Colleges (focus on undergraduate but grant fewer than 50% of their degrees in liberal arts disciplines)


Ranking methodology

Ranking Methodology

  • Quantitative measures that education experts have proposed as reliable indicators of academic quality

  • Gather data from each college on up to 16 indictors of academic excellence

  • Each factor is assigned a weight.

  • Colleges and universities in each category are ranked against their peers, based on their composite weighted score


Ranking category and weight

Ranking Category and Weight


Undergraduate academic reputation

Undergraduate Academic Reputation


Undergraduate academic reputation1

Undergraduate Academic Reputation

  • Equation*:

    Academic Reputation= Z avg. Peer assessment score * (15%) + Z avg. HS Counselor Score * (7.5%)

    Estimates for missing: None

    * For National Universities and National Liberal Arts Colleges


Student selectivity

Student Selectivity


Student selectivity1

Student Selectivity

  • Equation*:

    Z Student selectivity= Z test score * (50%) + Z high school class standing * (40%) + Z acceptance rate * (10%)

    Estimates for missing: one standard deviation less category’s mean

* For National Universities and National Liberal Arts Colleges


Faculty resources

Faculty Resources


Faculty resources1

Faculty Resources


Faculty resources2

Faculty Resources

  • Equation:

    Z Faculty resources= Z avg. salaries * (35%) + Z avg. w/term degree * (15%) + Z faculty ft * (5%) + Z student faculty ratio * (5%) +Z % class <20 * (30%) + Z % class 50 or more * (10%)

    Estimates for missing: one standard deviation less category’s mean


Graduation and retention rates

Graduation and Retention Rates


Graduation and retention rates1

Graduation and Retention Rates

  • Equation:

    Z Graduation and retention= Z avg. 6-yr grad rate * (80%) + Z avg. fresh retention rate * (20%)

    Estimates for missing: IPEDS or NCAA data; Otherwise, one standard deviation less category’s mean


Financial resources

Financial Resources


Alumni giving

Alumni Giving


Graduation rate performance

Graduation Rate Performance


Recent changes

Recent Changes

  • Rankings categories have been updated

    • Update to Carnegie basic classification (2010)

  • For-profit institutions are included (2011)

    • All for-profit colleges and universities grant bachelor’s degree and regionally accredited.

  • Non-responders are handled differently

    • For schools refused to fill out U.S. News survey for at least two years and for school news to the ranking, made extensive use of data from NCES.


Recent changes continued

Recent Changes (continued)

  • Now collecting graduation rates of students who received a federal Pell grant, a subsidized Stafford loan but not a Pell grant, and students who received neither.

  • May add collection of data for affordability and technology

    • Graduation rates by race

    • Net price of attending by EFC

    • Average loan size

    • # of degrees awarded per student

    • Ways colleges using technology (computer accessibility, wireless Internet, technical support, etc.)


Data sources

Data Sources

  • Most data come from the college, U.S News Main Survey, Financial Aid Survey

  • In 2011, 93% of the 1,378 ranked college and universities returned the survey

  • Obtain missing data from AAUP, National Collegiate Athletics Association, the Council for Aid to Education, and the NCES

  • Estimates are used in the ranking calculation when schools fail to report particular data points.


Schools excluded from the ranking

Schools Excluded from the Ranking

Schools are unranked and listed separately by category if

  • They do Not use the SAT or ACT test scores in admissions decisions for FTIAC, or

  • Too few respondents to the peer assessment survey, or

  • Have fewer than 200 total enrollment, or

  • Large proportion of nontraditional students and no first-year students

  • Also a few highly specialized schools in arts, business, and engineering


Ir and u s news rankings

IR and U.S. News Rankings

  • AIR members could (& have) provide feedback and suggestions to U.S. News

  • Two weeks prior to the release of rankings, email alert sent to survey respondents to Main and Fin Aid survey

  • Free one-month short-terms passes for College Compass

  • One day prior to the official rankings release date, survey respondents were sent notification and be able to access to two years of published ranking tables in PDF on a non-public website

  • AIR members can receive U.S. News published ranking table in Excel format (need to send an email to request the Excel spreadsheets to [email protected] or [email protected]

  • Unpublished rankings available by request (send email to request)


Top 10 criticisms of us news college ranking

Top 10 Criticisms of US News College Ranking

  • #10 Oversimplification: Comes from Stanford University’s FUNC (Forget US News Coalition) & Reed College who in the mid-90s argued “that ranking something as complex and variable as a college education with a single number is an oversimplification. “ FUNC claimed “that the process makes college administrations focus on numerical rankings rather than on educating students.“ Casper, Gerhard (18 April 1997)


Top 10 criticisms of us news college ranking continued

Top 10 Criticisms of US News College Ranking (continued)

  • #9 Peer assessments:

    • College Presidents may have outdated or no knowledge of institutions they are ranking

    • College Participation among Presidents was 43% in 2011, down 5 percentage points overall.

    • May rank competitors low

    • Low participation rates, amongst high school counselors


U s news world report s american s best colleges ranking methodology

Top 10 Criticisms of US News College Ranking (continued)

  • #8 Data manipulation: As early as 1994 when the Wall Street Journal disclosed that institutions flagrantly manipulating data in order to move up in the rankings in U.S.


U s news world report s american s best colleges ranking methodology

Top 10 Criticisms of US News College Ranking (continued)

  • #7Not much weight given to cost of attendance


U s news world report s american s best colleges ranking methodology

Top 10 Criticisms of US News College Ranking (continued)

  • #6 Academic quality not captured


Top 10 criticisms of us news college ranking continued1

Top 10 Criticisms of US News College Ranking (continued)

  • #5 Institutional differences not taken into account

  • Diverse student body

  • Institutional mission


Top 10 criticisms of us news college ranking continued2

Top 10 Criticisms of US News College Ranking (continued)

  • #4 Lack of data definition standards in the data collection process.


Top 10 criticisms of us news college ranking continued3

Top 10 Criticisms of US News College Ranking (continued)

  • #3 Proliferation of data collected


Top 10 criticisms of us news college ranking continued4

Top 10 Criticisms of US News College Ranking (continued)

  • #2 Cost / Benefit


Top 10 criticisms of us news college ranking continued5

Top 10 Criticisms of US News College Ranking (continued)

  • #1 ____________________


U s news world report s american s best colleges ranking methodology

Thank you for attending.


  • Login