Revised gre general test a prelaunch update
Advertisement
This presentation is the property of its rightful owner.
1 / 64

Revised GRE General Test: A Prelaunch Update PowerPoint PPT Presentation

CGS Annual Meeting December 8, 2006. Revised GRE General Test: A Prelaunch Update. David G. Payne, PhD Educational Testing Service. The Revised GRE General Test. Brief Background Reasons for Change Value of the Revised Test Key Aspects of Changes to the General Test

Related searches for Revised GRE General Test: A Prelaunch Update

Download Presentation

Revised GRE General Test: A Prelaunch Update

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Revised gre general test a prelaunch update

CGS Annual Meeting

December 8, 2006

Revised GRE General Test: A Prelaunch Update

David G. Payne, PhD

Educational Testing Service


The revised gre general test

The Revised GRE General Test

  • Brief Background

    • Reasons for Change

    • Value of the Revised Test

  • Key Aspects of Changes to the General Test

  • Contingency Planning for Launch

  • Communications Efforts

  • Additional Slides for Deans to Use on Campus


What is the value of the revised gre general test

What Is the Value of the Revised GRE General Test?

  • Improved security – Administration plan that directly addresses current and potential future security challenges over the long term

  • Improved predictive validity – The revised test reduces the possible effects of memorization.

  • Improved construct validity - Measures skills more directly related to graduate study

  • Linear Testing Format is more test taker friendly than current Computer Adaptive Test (CAT)


Why are we changing the general test

Why Are We Changing the General Test?

  • Test delivery is changing from continuous testing to fixed administrations in order to address security issues

    • Eliminate the potential security risk that exists with continuous testing regarding the exposure of test questions

  • Test content is changing to increase the validity of the test by reducing the effects of memorization

    • Eliminate single-word verbal questions

    • Introduce new question types and tasks

    • Reduce possibility of non-original essay materials


Additional goals for the revision of the gre general test

Additional Goals for the Revision of the GRE General Test

  • Provide faculty with better information on applicants’ performance

    • Essay distribution

    • New Verbal and Quantitative Score Scales

  • Better measurement of skills

    • New question formats other than traditional multiple choice

    • Allow calculator use in order to reduce emphasis on computation in Quantitative section

  • Improved test access for examinees worldwide


Verbal reasoning and quantitative reasoning sections

Verbal Reasoning andQuantitative Reasoning Sections


Value of the new verbal reasoning section

Emphasis on skills related to graduate work such as complex reasoning

Increased emphasis on inferential reasoning

Increased emphasis on verbal reasoning in context

Increased number of reading comprehension questions based on a greater variety of reading passages

Reduced emphasis on vocabulary out of context

No Antonyms or Analogies

Inclusion of new question formats other than traditional multiple choice (e.g., highlighting a sentence in a passage that serves the function described in the question)

Value of the New Verbal Reasoning Section


Value of the new quantitative reasoning section

Quantitative reasoning skills that are similar to skills typically used in graduate school

Increased emphasis on questions involving “real-life” scenarios

Increased emphasis on data interpretation questions

Reduced emphasis on Geometry

On-screen 4-function calculator with square root

Reduced emphasis on computation

Inclusion of new question formats other than traditional multiple choice (such as entering a numeric answer via the keyboard)

Value of the New Quantitative Reasoning Section


Why are we changing the gre verbal and quantitative score scales

Why Are We Changing the GRE Verbal and Quantitative Score Scales?

  • Sound measurement practice and professional standards (developed by the AERA, APA, and NCME) recommend a change in the score scale when significant changes to test specifications are made.

  • The ways in which we are measuring the verbal and quantitative reasoning skills (i.e., question types) and the relative emphasis among the component skills in the revised Verbal and Quantitative sections are sufficiently different from those on the current test, so that it is appropriate to use a new scale for the revised test.


New verbal and quantitative score scales

New Verbal and Quantitative Score Scales

  • Change will allow us to address two situations that have developed over the life of the current test:

    • Mean Verbal scores (470) and Quantitative scores (593) have drifted apart over time

    • Quantitative scores cluster at the top of the scale


Ranked preference for new score ranges

Ranked Preference for New Score Ranges

Preferred Most

Preferred Least


Comparison of old and new verbal and quantitative score scales

Comparison of Old and New Verbal and Quantitative Score Scales

NOTE: The Critical Thinking and Analytical Writing score scale will not be changed.


Advantages of the new score scale

Advantages of the New Score Scale

  • The new scale will make more apparent the differences between candidates and will facilitate more appropriate comparisons between candidates.

    • Score users will be less likely to interpret small score differences as meaningful differences between candidates.

  • The Verbal and Quantitative means will be aligned which will allow score users to see relative strengths for a particular candidate.


Information to assist gre score recipients

Information to Assist GRE Score Recipients

  • Percentiles (based on first 3 administrations) will be available on the GRE Web site (www.ets.org/gre) and printed on score reports to assist score users in interpreting scores

  • Aconcordance table to assist score users in determining the relationship between old and new Verbal and Quantitative scores will be available on the GRE Web site.

  • For Verbal and Quantitative scores earned prior to September 2007 and reported on score reports, approximate score equivalents on the new scale will be included on score reports to assist Score Recipients in comparing old and new scores.

  • Broad major field score distributions will be available on the GRE Web site.


Sample revised general test score report

Sample Revised General Test Score Report

*For tests taken beginning in September 2007, scores will be printed in the “New Score” column and dashes will be printed in the “Original Score” column. For tests taken before September 2007, scores will be printed in the “Original Score” column and approximate equivalents on the new scale will be printed in the “New Score” column. These equivalents are based on a concordance study; the full table showing the relationship between the two scales will be posted on the GRE Web site at www.ets.org/gre.


Analytical writing section

Analytical Writing Section


Familiarity with skills measured in sections of the gre general test

Familiarity with Skills Measured in Sections of the GRE General Test

Very Familiar

Not at all Familiar


Appropriateness of alternative names

Appropriateness of Alternative Names

Very appropriate

Not at all appropriate


Value of the new critical thinking and analytical writing section

A performance-based measure that integrates the assessment of critical thinking and analytical writing

Consists of two complementary analytical writing tasks:

30-minute “Analyze an Issue" task

30-minute “Analyze an Argument" task

New, more focused prompts

Requires a more specific response from the test taker

Will reduce possibility of reliance on memorized materials

Limited or no reuse of prompts

Value of the New Critical Thinking and Analytical Writing Section


The new critical thinking and analytical writing section

The New Critical Thinking and Analytical Writing Section

Measures the ability to:

  • Articulate complex ideas clearly and effectively

  • Examine claims and accompanying evidence

  • Support ideas with relevant reasons and examples

  • Sustain a well-focused, coherent discussion

  • Control the elements of standard written English

    Note: Essay responses are typed on computer, with a basic word processor, and no spell checker or grammar checker.


Essay distribution

Essay Distribution

  • Essay responses will be made available electronically to designated score recipients.

    • The details of operation of this new service will be announced in early 2007.

  • It is critical that faculty understand that test takers construct their essay responses, which are essentially first drafts, under these conditions:

    • Each essay must be completed in 30 minutes

    • It is a high stakes testing situation

    • Test takers use an elementary word processing system with no spell check or grammar check


Contingency planning for the revised gre general test

Contingency Planningfor theRevised GRE General Test


Rgre contingency planning

rGRE Contingency Planning

  • Issues

    • Need to set the new Score Scales for Verbal and Quantitative

      • Requires 30,000 representative test takers in first 3 administrations

        • Only possible to do this in the Fall

      • Not possible to do a phased launch like the one used with TOEFL iBT

    • Constraints on Score Reporting Timelines

    • Concerns about capacity in new IBT Network

      • Primarily in large domestic urban areas


Rgre contingency planning1

rGRE Contingency Planning

  • Strategies

    • Actively Recruiting Test Centers – Apply now!

    • Exploring options for alternative testing sites

      • Large Flexible Testing Centers (e.g., Commercial sites)

      • Mobile Test Centers

    • Exploring options for increasing the number of administration dates

    • Goal is to allow every test taker to be able to schedule a test within 30 days in their preferred area for testing.


Rgre communications

rGRE Communications


Key communications activities

Key Communications Activities

  • Print Materials

  • Web Seminars

  • rGRE eNewsletter

  • Annotated PowerPoint Presentations

  • Outreach to Academic Discipline Societies

  • Conference Presentations

    • Academic Discipline Societies

    • CGS Regional Meetings, CHBGS, NAGAP

  • DVD/Video Presentations

    • General Overview

    • Specific topics (e.g., Using the Revised GRE Scores for Verbal and Quantitative)


Key communications activities1

Key Communications Activities

  • “Just-in-time” communications to Score Recipients, or ”What happened to the 200-800 scores and what the heck do these new scores mean?”

    • Reason for Score Scale Changes

    • Nature of New Score Scales

    • How to Use Score Scales


Need more information

Need More Information?

To get the most up-to-date information about the changes to the GRE General Test:

  • Visit the GRE Web site at www.ets.org/gre/revgentest.html

  • Subscribe to the quarterly GRE eNewsletter (see link on www.ets.org/gre)

    If you have questions:

  • E-mail David Payne at [email protected]


Test details and additional support information

Test Details and Additional Support Information


Benefits of using gre test scores

Benefits of Using GRE Test Scores

  • Measure skills faculty and graduate deans have identified as essential to graduate school success, including verbal reasoning, quantitative reasoning, critical thinking and analytical writing

  • Provide the only common measures of these skills for comparing the qualifications of applicants from different educational backgrounds and countries of origin


Benefits of using gre test scores cont d

Benefits of Using GRE Test Scores (cont’d)

  • Furnish independent information to supplement the evaluation of grades and recommendations

  • Offer a valid predictor of graduate school performance, as confirmed by a recent independent study using 82,000 graduate students and over 1,700 studies containing validity data for GRE tests

  • Ensure that essay responses are the original work of the examinee


Verbal reasoning and quantitative reasoning sections1

Verbal Reasoning and Quantitative Reasoning Sections


The new verbal reasoning section

The New Verbal Reasoning Section

Measures the ability to:

  • Understand the meanings of words, sentences, and entire texts; understand relationships among words and among concepts

  • Select important points; distinguish major from minor or irrelevant points; summarize text; understand the structure of a text

  • Analyze and draw conclusions from discourse; reason from incomplete data; identify author’s/speaker’s assumptions and/or perspective; understand multiple levels of meaning (such as literal, figurative, text’s intent, etc.)


Sample verbal question

Sample Verbal Question

While chocolate was highly esteemed in Mesoamerica, where it originated, its adoption in Europe was initially slow. There is a common belief that Europeans needed to “transform” chocolate to make it appetizing. However, while Spaniards did put sugar, which was unknown to indigenous Americans, into chocolate beverages, this additive was not completely innovative. Mesoamericans were already sweetening chocolate with honey, and the step from honey to sugar– increasingly more available than honey because of expanding sugar plantations in the Americas– is a small one. Likewise, although Spaniards adjusted Mesoamerican recipes by using European spices, the spices chosen suggest an attempt to replicate harder-to-find native flowers. There is no indication the Spaniards deliberately tried to change the original flavor of chocolate.

Click on the sentence that presents a misconception that the passage challenges.


The new quantitative reasoning section

The New Quantitative Reasoning Section

Measures the ability to:

  • Understand quantitative information

  • Interpret and analyze quantitative information

  • Solve problems using mathematical models

  • Apply basic mathematical skills and elementary mathematical concepts of arithmetic, algebra, geometry, probability, and statistics


Sample quantitative question

Sample Quantitative Question

The table shows the distribution of prices of 45 houses for sale in a certain region. Select two of the following choices and place them in the blanks below so that the resulting statement is true.

$175,000 $185,000 $190,000 at most at least

$42,000 $57,000

If the highest price of the 45 houses is , then the range of the prices of the 45 houses is .

Click on a choice, then click on a blank.


Best ways to use revised general test scores v and q

Best Ways to Use Revised General Test Scores (V and Q)

  • The new scaled scores (and new score equivalents) are a good way to evaluate all applicants.

  • Percentile information provides information about the relative standing of the applicant compared to the fall 2007 test-taking population.

  • Broad intended graduate major field score distributions provide a good way to evaluate applicants within a particular group of disciplines.


Best ways to use revised general test scores continued

Best Ways to Use Revised General Test Scores (continued)

  • For applicants with multiple scores (both old and new), use the new score scales (for old General Test scores, use the approximate score equivalent new scores); whichever method used (e.g., average, most recent, or highest) should be used consistently

  • Programs should use multiple pieces of information in making admissions decisions (e.g., GPA, GRE scores, letters of recommendation).


Best ways to use revised general test scores continued1

Best Ways to Use Revised General Test Scores (continued)

  • When there is a mismatch between the level of skill in application writing samples (e.g., personal statement) and the Critical Thinking and Analytical Writing scores, it may be helpful to view the applicant’s essay responses on the Critical Thinking and Analytical Writing section.


If you have score use or score interpretation questions

If You Have Score Use or Score Interpretation Questions

  • The GRE Board has a long history of encouraging appropriate score use

    • See the GRE Board Score Use Guidelines on the GRE Web site at www.ets.org/gre/edupubs.html for a fuller description

  • Visit the URL above to view score interpretative information for the General Test and Subject Tests

  • The GRE Program is happy to answer score interpretation questions from GRE score users. Individuals should contact the GRE Program:

    • By telephone at 609-683-2002

    • Via email at [email protected]


Critical thinking and analytical writing

Critical Thinking and Analytical Writing


Sample argument topic

Sample Argument Topic

“Hospital statistics regarding people who go to the emergency room after roller-skating accidents indicate the need for more protective equipment. Within that group of people, 75 percent of those who had accidents in streets or parking lots had not been wearing any protective clothing (helmets, knee pads, etc.) or any light-reflecting material (clip-on lights, glow-in-the-dark wrist pads, etc.). Clearly, the statistics indicate that by investing in high-quality protective gear and reflective equipment, roller skaters will greatly reduce their risk of being severely injured in an accident.”

Write a response in which you examine the argument’s unstated assumptions making sure to explain how the argument depends on the assumptions and what the implications are if the assumptions prove unwarranted.


Sample issue task reliance on technology

Sample Issue Task:Reliance on Technology

"As people rely more and more on technology to solve problems, the ability of humans to think for themselves will surely deteriorate.“

Discuss the extent to which you agree or disagree with the statement above and explain your reasoning for the position you take. In developing and supporting your position, you should consider ways in which the statement might or might not hold true and explain how those considerations shape your position.


How essay responses are evaluated

How Essay Responses Are Evaluated

GRE readers, who are college and university faculty, read each essay response and evaluate its overall quality according to how well the test taker:

  • Responds to the specific instructions on the issue

  • Considers the complexities of the issue

  • Organizes, develops, and expresses his/her ideas

  • Supports his/her position with relevant reasons and/or examples

  • Controls the elements of standard written English


Understanding how the critical thinking and analytical writing section is scored

Understanding How the Critical Thinking and Analytical Writing Section is Scored

  • Each essay is scored by two trained readers, using a 6-point holistic scale

  • Two scores that differ by more than one point are adjudicated by a third reader

  • Scores from the two readings of an essay are averaged

  • The final scores on the two essays are then averaged and rounded up to the nearest half-point interval

  • A single score is reported for the section

  • Scoring guides and score level descriptions are available on the GRE Web site at www.ets.org/gre


Selected score level descriptions

Selected Score Level Descriptions

5.5 and 6 – Sustains insightful, in-depth analyses of complex ideas; develops and supports main points with logically compelling reasons and/or highly persuasive examples; is well focused and well organized; skillfully uses sentence variety and precise vocabulary to convey meaning effectively; demonstrates superior facility with sentence structure and language usage but may have minor errors that do not interfere with meaning

3.5 and 4 – Provides competent analysis of complex ideas; develops and supports main points with relevant reasons and/or examples; is adequately organized; conveys meaning with reasonable clarity; demonstrates satisfactory control of sentence structure and language usage but may have some errors that affect clarity.

1.5 and 2 – Displays serious weaknesses in analytical writing. The writing is seriously flawed in at least one of the following ways: serious lack of analysis or development; lack of organization; serious and frequent problems in sentence structure or language usage, with errors that obscure meaning.


Revised general test overall information

Revised General Test:Overall Information


Test structure

Test Structure

*Questions in this section are either being tried out for possible use in future tests or are being used to ensure that scores on new editions of the test are comparable to scores on earlier test editions.


Test administration changes

Test Administration Changes

*For statistical equating purposes, some test questions

need to be reused across test editions.


Test administration changes continued

Test Administration Changes (continued)


Timeline for major events

Timeline for Major Events

  • Fall 2006

    • Sample questions available on GRE Web site (www.ets.org/gre)

    • Score use information available

    • New format for score reports described

  • Spring 2007

    • Test preparation materials available

    • Test administration schedule available


Timeline for major events continued

Timeline for Major Events (continued)

  • July 2007

    • Registration for the revised General Test opens

  • September 2007

    • Revised GRE General Test launched

    • Internet-Based testing in a new network

  • October 2007

    • New Verbal and Quantitative score scales finalized


Timeline for major events continued1

Timeline for Major Events (continued)

  • November 2007

    • Scores reported for first 3 administrations (approx 4-5 weeks after 3rd administration)

    • Percentiles available on GRE Web site and printed on score reports

    • Concordance table available on the GRE Web site

    • Approximate score equivalents on the new scale included on score reports for Verbal and Quantitative scores earned prior to September 2007

    • Essay responses made available to Score Recipients

  • January 2008

    • Broad major field score distributions available on the GRE Web site.


Gre validity research

GRE Validity Research


Independent gre validity research

Independent GRE Validity Research

  • In order to examine the predictive validity of the GRE General Test and Subject Tests, Nathan R. Kuncel, Sarah A. Hezlett, and Deniz S. Ones conducted a meta-analysis of all available GRE validity studies that had been conducted by independent researchers over the past several years.

  • Their study included:

    • 1,753 independent samples of students

    • Over 80,000 graduate students

  • Refinements in study included

    • A very large database of studies

    • Statistical corrections for restriction of range

    • Statistical corrections for unreliable criterion measures


Kuncel hezlett ones

Kuncel, Hezlett & Ones

  • General Test was found to be a “generalizably valid predictor” of

    • First-year graduate grade point average

    • Overall graduate grade point average

    • Comprehensive exam scores

    • Publication citation counts

    • Faculty ratings

  • Correlations with degree attainment and research productivity were consistently positive, but not statistically significant in all cases.


Results of kuncel et al research

Results of Kuncel et al. Research

  • A follow-up study by Kuncel indicated the GRE General Test has similar levels of predictive validity for both master's and doctoral level programs.

  • The GRE General Test has better predictive validity than undergraduate grades or letters of recommendation.

  • Subject Tests were found to be better predictors of success than were either the General Test or undergraduate grade point average.

    These data strongly suggest that the GRE is a valid predictor of graduate student performance across disciplines, degree levels, age of students, etc.


Results of kuncel et al research continued

Results of Kuncel et al. Research (continued)

For more information about the Kuncel et al. research study, see:

“A Comprehensive Meta-Analysis of the Predictive Validity of the Graduate Record Examinations: Implications for Graduate Student Selection and Performance”

Nathan R. Kuncel

Sarah A. Hezlett

Deniz S. Ones

Psychological Bulletin 2001, Vol. 127, No. 1, 162-181


Gre communications

GRE Communications


Rgre enewsletter

rGRE eNewsletter

  • Content

    • Articles about aspects of the revised General Test and/or GRE tests/services

    • Links to upcoming conferences

    • Links to important information on GRE Web site

  • Quarterly publication


Web seminars

Web Seminars

  • Online, web-based seminars about the revised General Test

  • Conducted by GRE Program Staff

  • Multiple Sessions, multiple topics

    • Overview of the Revised General Test

    • How to Use RGRE Scores in Admissions

    • Overview of the Analytical Writing Measure

  • Email invitations and online registration

  • Multiple Audiences

    • Score recipients/faculty/administrators

    • Undergraduate faculty

    • Test Prep trainers

  • Promote the Website and rGRE eNewsletter


Powerpoint presentations

PowerPoint Presentations

  • Presentations about rGRE will be available on the GRE Web site

    • Multiple topics

    • Include detailed speaker notes

    • Presentations for various audiences

      • Administrators

      • Faculty

      • Test prep trainers

    • Promotes the Website and eNewsletter


  • Login