Revised gre general test a prelaunch update
Download
1 / 64

Revised GRE General Test: A Prelaunch Update - PowerPoint PPT Presentation


  • 2232 Views
  • Updated On :

CGS Annual Meeting December 8, 2006. Revised GRE General Test: A Prelaunch Update. David G. Payne, PhD Educational Testing Service. The Revised GRE General Test. Brief Background Reasons for Change Value of the Revised Test Key Aspects of Changes to the General Test

Related searches for Revised GRE General Test: A Prelaunch Update

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Revised GRE General Test: A Prelaunch Update' - oshin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Revised gre general test a prelaunch update

CGS Annual Meeting

December 8, 2006

Revised GRE General Test: A Prelaunch Update

David G. Payne, PhD

Educational Testing Service


The revised gre general test
The Revised GRE General Test

  • Brief Background

    • Reasons for Change

    • Value of the Revised Test

  • Key Aspects of Changes to the General Test

  • Contingency Planning for Launch

  • Communications Efforts

  • Additional Slides for Deans to Use on Campus


What is the value of the revised gre general test
What Is the Value of the Revised GRE General Test?

  • Improved security – Administration plan that directly addresses current and potential future security challenges over the long term

  • Improved predictive validity – The revised test reduces the possible effects of memorization.

  • Improved construct validity - Measures skills more directly related to graduate study

  • Linear Testing Format is more test taker friendly than current Computer Adaptive Test (CAT)


Why are we changing the general test
Why Are We Changing the General Test?

  • Test delivery is changing from continuous testing to fixed administrations in order to address security issues

    • Eliminate the potential security risk that exists with continuous testing regarding the exposure of test questions

  • Test content is changing to increase the validity of the test by reducing the effects of memorization

    • Eliminate single-word verbal questions

    • Introduce new question types and tasks

    • Reduce possibility of non-original essay materials


Additional goals for the revision of the gre general test
Additional Goals for the Revision of the GRE General Test

  • Provide faculty with better information on applicants’ performance

    • Essay distribution

    • New Verbal and Quantitative Score Scales

  • Better measurement of skills

    • New question formats other than traditional multiple choice

    • Allow calculator use in order to reduce emphasis on computation in Quantitative section

  • Improved test access for examinees worldwide


Verbal reasoning and quantitative reasoning sections

Verbal Reasoning andQuantitative Reasoning Sections


Value of the new verbal reasoning section

Emphasis on skills related to graduate work such as complex reasoning

Increased emphasis on inferential reasoning

Increased emphasis on verbal reasoning in context

Increased number of reading comprehension questions based on a greater variety of reading passages

Reduced emphasis on vocabulary out of context

No Antonyms or Analogies

Inclusion of new question formats other than traditional multiple choice (e.g., highlighting a sentence in a passage that serves the function described in the question)

Value of the New Verbal Reasoning Section


Value of the new quantitative reasoning section

Quantitative reasoning skills that are similar to skills typically used in graduate school

Increased emphasis on questions involving “real-life” scenarios

Increased emphasis on data interpretation questions

Reduced emphasis on Geometry

On-screen 4-function calculator with square root

Reduced emphasis on computation

Inclusion of new question formats other than traditional multiple choice (such as entering a numeric answer via the keyboard)

Value of the New Quantitative Reasoning Section


Why are we changing the gre verbal and quantitative score scales
Why Are We Changing the GRE Verbal and typically used in graduate schoolQuantitative Score Scales?

  • Sound measurement practice and professional standards (developed by the AERA, APA, and NCME) recommend a change in the score scale when significant changes to test specifications are made.

  • The ways in which we are measuring the verbal and quantitative reasoning skills (i.e., question types) and the relative emphasis among the component skills in the revised Verbal and Quantitative sections are sufficiently different from those on the current test, so that it is appropriate to use a new scale for the revised test.


New verbal and quantitative score scales
New Verbal and Quantitative Score Scales typically used in graduate school

  • Change will allow us to address two situations that have developed over the life of the current test:

    • Mean Verbal scores (470) and Quantitative scores (593) have drifted apart over time

    • Quantitative scores cluster at the top of the scale


Ranked preference for new score ranges
Ranked Preference for New Score Ranges typically used in graduate school

Preferred Most

Preferred Least


Comparison of old and new verbal and quantitative score scales
Comparison of Old and New Verbal and Quantitative Score Scales

NOTE: The Critical Thinking and Analytical Writing score scale will not be changed.


Advantages of the new score scale
Advantages of the New Score Scale Scales

  • The new scale will make more apparent the differences between candidates and will facilitate more appropriate comparisons between candidates.

    • Score users will be less likely to interpret small score differences as meaningful differences between candidates.

  • The Verbal and Quantitative means will be aligned which will allow score users to see relative strengths for a particular candidate.


Information to assist gre score recipients
Information to Assist ScalesGRE Score Recipients

  • Percentiles (based on first 3 administrations) will be available on the GRE Web site (www.ets.org/gre) and printed on score reports to assist score users in interpreting scores

  • Aconcordance table to assist score users in determining the relationship between old and new Verbal and Quantitative scores will be available on the GRE Web site.

  • For Verbal and Quantitative scores earned prior to September 2007 and reported on score reports, approximate score equivalents on the new scale will be included on score reports to assist Score Recipients in comparing old and new scores.

  • Broad major field score distributions will be available on the GRE Web site.


Sample revised general test score report
Sample Revised General Test Score Report Scales

*For tests taken beginning in September 2007, scores will be printed in the “New Score” column and dashes will be printed in the “Original Score” column. For tests taken before September 2007, scores will be printed in the “Original Score” column and approximate equivalents on the new scale will be printed in the “New Score” column. These equivalents are based on a concordance study; the full table showing the relationship between the two scales will be posted on the GRE Web site at www.ets.org/gre.



Familiarity with skills measured in sections of the gre general test
Familiarity with Skills Measured in ScalesSections of the GRE General Test

Very Familiar

Not at all Familiar


Appropriateness of alternative names
Appropriateness of Alternative Names Scales

Very appropriate

Not at all appropriate


Value of the new critical thinking and analytical writing section

A performance-based measure that integrates the assessment of critical thinking and analytical writing

Consists of two complementary analytical writing tasks:

30-minute “Analyze an Issue" task

30-minute “Analyze an Argument" task

New, more focused prompts

Requires a more specific response from the test taker

Will reduce possibility of reliance on memorized materials

Limited or no reuse of prompts

Value of the New Critical Thinking and Analytical Writing Section


The new critical thinking and analytical writing section
The New Critical Thinking and of critical thinking and analytical writingAnalytical Writing Section

Measures the ability to:

  • Articulate complex ideas clearly and effectively

  • Examine claims and accompanying evidence

  • Support ideas with relevant reasons and examples

  • Sustain a well-focused, coherent discussion

  • Control the elements of standard written English

    Note: Essay responses are typed on computer, with a basic word processor, and no spell checker or grammar checker.


Essay distribution
Essay Distribution of critical thinking and analytical writing

  • Essay responses will be made available electronically to designated score recipients.

    • The details of operation of this new service will be announced in early 2007.

  • It is critical that faculty understand that test takers construct their essay responses, which are essentially first drafts, under these conditions:

    • Each essay must be completed in 30 minutes

    • It is a high stakes testing situation

    • Test takers use an elementary word processing system with no spell check or grammar check


Contingency planning for the revised gre general test

Contingency Planning of critical thinking and analytical writingfor theRevised GRE General Test


Rgre contingency planning
rGRE Contingency Planning of critical thinking and analytical writing

  • Issues

    • Need to set the new Score Scales for Verbal and Quantitative

      • Requires 30,000 representative test takers in first 3 administrations

        • Only possible to do this in the Fall

      • Not possible to do a phased launch like the one used with TOEFL iBT

    • Constraints on Score Reporting Timelines

    • Concerns about capacity in new IBT Network

      • Primarily in large domestic urban areas


Rgre contingency planning1
rGRE Contingency Planning of critical thinking and analytical writing

  • Strategies

    • Actively Recruiting Test Centers – Apply now!

    • Exploring options for alternative testing sites

      • Large Flexible Testing Centers (e.g., Commercial sites)

      • Mobile Test Centers

    • Exploring options for increasing the number of administration dates

    • Goal is to allow every test taker to be able to schedule a test within 30 days in their preferred area for testing.


Rgre communications

rGRE of critical thinking and analytical writingCommunications


Key communications activities
Key Communications Activities of critical thinking and analytical writing

  • Print Materials

  • Web Seminars

  • rGRE eNewsletter

  • Annotated PowerPoint Presentations

  • Outreach to Academic Discipline Societies

  • Conference Presentations

    • Academic Discipline Societies

    • CGS Regional Meetings, CHBGS, NAGAP

  • DVD/Video Presentations

    • General Overview

    • Specific topics (e.g., Using the Revised GRE Scores for Verbal and Quantitative)


Key communications activities1
Key Communications Activities of critical thinking and analytical writing

  • “Just-in-time” communications to Score Recipients, or ”What happened to the 200-800 scores and what the heck do these new scores mean?”

    • Reason for Score Scale Changes

    • Nature of New Score Scales

    • How to Use Score Scales


Need more information
Need More Information? of critical thinking and analytical writing

To get the most up-to-date information about the changes to the GRE General Test:

  • Visit the GRE Web site at www.ets.org/gre/revgentest.html

  • Subscribe to the quarterly GRE eNewsletter (see link on www.ets.org/gre)

    If you have questions:

  • E-mail David Payne at [email protected]


Test details and additional support information

Test Details and of critical thinking and analytical writingAdditional Support Information


Benefits of using gre test scores
Benefits of Using GRE Test Scores of critical thinking and analytical writing

  • Measure skills faculty and graduate deans have identified as essential to graduate school success, including verbal reasoning, quantitative reasoning, critical thinking and analytical writing

  • Provide the only common measures of these skills for comparing the qualifications of applicants from different educational backgrounds and countries of origin


Benefits of using gre test scores cont d
Benefits of Using GRE Test Scores (cont’d) of critical thinking and analytical writing

  • Furnish independent information to supplement the evaluation of grades and recommendations

  • Offer a valid predictor of graduate school performance, as confirmed by a recent independent study using 82,000 graduate students and over 1,700 studies containing validity data for GRE tests

  • Ensure that essay responses are the original work of the examinee


Verbal reasoning and quantitative reasoning sections1

Verbal Reasoning and Quantitative Reasoning Sections of critical thinking and analytical writing


The new verbal reasoning section
The New Verbal Reasoning Section of critical thinking and analytical writing

Measures the ability to:

  • Understand the meanings of words, sentences, and entire texts; understand relationships among words and among concepts

  • Select important points; distinguish major from minor or irrelevant points; summarize text; understand the structure of a text

  • Analyze and draw conclusions from discourse; reason from incomplete data; identify author’s/speaker’s assumptions and/or perspective; understand multiple levels of meaning (such as literal, figurative, text’s intent, etc.)


Sample verbal question
Sample Verbal Question of critical thinking and analytical writing

While chocolate was highly esteemed in Mesoamerica, where it originated, its adoption in Europe was initially slow. There is a common belief that Europeans needed to “transform” chocolate to make it appetizing. However, while Spaniards did put sugar, which was unknown to indigenous Americans, into chocolate beverages, this additive was not completely innovative. Mesoamericans were already sweetening chocolate with honey, and the step from honey to sugar– increasingly more available than honey because of expanding sugar plantations in the Americas– is a small one. Likewise, although Spaniards adjusted Mesoamerican recipes by using European spices, the spices chosen suggest an attempt to replicate harder-to-find native flowers. There is no indication the Spaniards deliberately tried to change the original flavor of chocolate.

Click on the sentence that presents a misconception that the passage challenges.


The new quantitative reasoning section
The New Quantitative Reasoning Section of critical thinking and analytical writing

Measures the ability to:

  • Understand quantitative information

  • Interpret and analyze quantitative information

  • Solve problems using mathematical models

  • Apply basic mathematical skills and elementary mathematical concepts of arithmetic, algebra, geometry, probability, and statistics


Sample quantitative question
Sample Quantitative Question of critical thinking and analytical writing

The table shows the distribution of prices of 45 houses for sale in a certain region. Select two of the following choices and place them in the blanks below so that the resulting statement is true.

$175,000 $185,000 $190,000 at most at least

$42,000 $57,000

If the highest price of the 45 houses is , then the range of the prices of the 45 houses is .

Click on a choice, then click on a blank.


Best ways to use revised general test scores v and q
Best Ways to Use of critical thinking and analytical writingRevised General Test Scores (V and Q)

  • The new scaled scores (and new score equivalents) are a good way to evaluate all applicants.

  • Percentile information provides information about the relative standing of the applicant compared to the fall 2007 test-taking population.

  • Broad intended graduate major field score distributions provide a good way to evaluate applicants within a particular group of disciplines.


Best ways to use revised general test scores continued
Best Ways to Use of critical thinking and analytical writingRevised General Test Scores (continued)

  • For applicants with multiple scores (both old and new), use the new score scales (for old General Test scores, use the approximate score equivalent new scores); whichever method used (e.g., average, most recent, or highest) should be used consistently

  • Programs should use multiple pieces of information in making admissions decisions (e.g., GPA, GRE scores, letters of recommendation).


Best ways to use revised general test scores continued1
Best Ways to Use of critical thinking and analytical writingRevised General Test Scores (continued)

  • When there is a mismatch between the level of skill in application writing samples (e.g., personal statement) and the Critical Thinking and Analytical Writing scores, it may be helpful to view the applicant’s essay responses on the Critical Thinking and Analytical Writing section.


If you have score use or score interpretation questions
If You Have Score Use or of critical thinking and analytical writingScore Interpretation Questions

  • The GRE Board has a long history of encouraging appropriate score use

    • See the GRE Board Score Use Guidelines on the GRE Web site at www.ets.org/gre/edupubs.html for a fuller description

  • Visit the URL above to view score interpretative information for the General Test and Subject Tests

  • The GRE Program is happy to answer score interpretation questions from GRE score users. Individuals should contact the GRE Program:


Critical thinking and analytical writing

Critical Thinking and of critical thinking and analytical writingAnalytical Writing


Sample argument topic
Sample Argument Topic of critical thinking and analytical writing

“Hospital statistics regarding people who go to the emergency room after roller-skating accidents indicate the need for more protective equipment. Within that group of people, 75 percent of those who had accidents in streets or parking lots had not been wearing any protective clothing (helmets, knee pads, etc.) or any light-reflecting material (clip-on lights, glow-in-the-dark wrist pads, etc.). Clearly, the statistics indicate that by investing in high-quality protective gear and reflective equipment, roller skaters will greatly reduce their risk of being severely injured in an accident.”

Write a response in which you examine the argument’s unstated assumptions making sure to explain how the argument depends on the assumptions and what the implications are if the assumptions prove unwarranted.


Sample issue task reliance on technology
Sample Issue Task: of critical thinking and analytical writingReliance on Technology

"As people rely more and more on technology to solve problems, the ability of humans to think for themselves will surely deteriorate.“

Discuss the extent to which you agree or disagree with the statement above and explain your reasoning for the position you take. In developing and supporting your position, you should consider ways in which the statement might or might not hold true and explain how those considerations shape your position.


How essay responses are evaluated
How Essay Responses Are Evaluated of critical thinking and analytical writing

GRE readers, who are college and university faculty, read each essay response and evaluate its overall quality according to how well the test taker:

  • Responds to the specific instructions on the issue

  • Considers the complexities of the issue

  • Organizes, develops, and expresses his/her ideas

  • Supports his/her position with relevant reasons and/or examples

  • Controls the elements of standard written English


Understanding how the critical thinking and analytical writing section is scored
Understanding How the Critical Thinking and of critical thinking and analytical writingAnalytical Writing Section is Scored

  • Each essay is scored by two trained readers, using a 6-point holistic scale

  • Two scores that differ by more than one point are adjudicated by a third reader

  • Scores from the two readings of an essay are averaged

  • The final scores on the two essays are then averaged and rounded up to the nearest half-point interval

  • A single score is reported for the section

  • Scoring guides and score level descriptions are available on the GRE Web site at www.ets.org/gre


Selected score level descriptions
Selected Score Level Descriptions of critical thinking and analytical writing

5.5 and 6 – Sustains insightful, in-depth analyses of complex ideas; develops and supports main points with logically compelling reasons and/or highly persuasive examples; is well focused and well organized; skillfully uses sentence variety and precise vocabulary to convey meaning effectively; demonstrates superior facility with sentence structure and language usage but may have minor errors that do not interfere with meaning

3.5 and 4 – Provides competent analysis of complex ideas; develops and supports main points with relevant reasons and/or examples; is adequately organized; conveys meaning with reasonable clarity; demonstrates satisfactory control of sentence structure and language usage but may have some errors that affect clarity.

1.5 and 2 – Displays serious weaknesses in analytical writing. The writing is seriously flawed in at least one of the following ways: serious lack of analysis or development; lack of organization; serious and frequent problems in sentence structure or language usage, with errors that obscure meaning.


Revised general test overall information

Revised General Test: of critical thinking and analytical writingOverall Information


Test structure
Test Structure of critical thinking and analytical writing

*Questions in this section are either being tried out for possible use in future tests or are being used to ensure that scores on new editions of the test are comparable to scores on earlier test editions.


Test administration changes
Test Administration Changes of critical thinking and analytical writing

*For statistical equating purposes, some test questions

need to be reused across test editions.


Test administration changes continued
Test Administration Changes (continued) of critical thinking and analytical writing


Timeline for major events
Timeline for Major Events of critical thinking and analytical writing

  • Fall 2006

    • Sample questions available on GRE Web site (www.ets.org/gre)

    • Score use information available

    • New format for score reports described

  • Spring 2007

    • Test preparation materials available

    • Test administration schedule available


Timeline for major events continued
Timeline for Major Events (continued) of critical thinking and analytical writing

  • July 2007

    • Registration for the revised General Test opens

  • September 2007

    • Revised GRE General Test launched

    • Internet-Based testing in a new network

  • October 2007

    • New Verbal and Quantitative score scales finalized


Timeline for major events continued1
Timeline for Major Events (continued) of critical thinking and analytical writing

  • November 2007

    • Scores reported for first 3 administrations (approx 4-5 weeks after 3rd administration)

    • Percentiles available on GRE Web site and printed on score reports

    • Concordance table available on the GRE Web site

    • Approximate score equivalents on the new scale included on score reports for Verbal and Quantitative scores earned prior to September 2007

    • Essay responses made available to Score Recipients

  • January 2008

    • Broad major field score distributions available on the GRE Web site.


Gre validity research

GRE Validity Research of critical thinking and analytical writing


Independent gre validity research
Independent GRE Validity Research of critical thinking and analytical writing

  • In order to examine the predictive validity of the GRE General Test and Subject Tests, Nathan R. Kuncel, Sarah A. Hezlett, and Deniz S. Ones conducted a meta-analysis of all available GRE validity studies that had been conducted by independent researchers over the past several years.

  • Their study included:

    • 1,753 independent samples of students

    • Over 80,000 graduate students

  • Refinements in study included

    • A very large database of studies

    • Statistical corrections for restriction of range

    • Statistical corrections for unreliable criterion measures


Kuncel hezlett ones
Kuncel, Hezlett & Ones of critical thinking and analytical writing

  • General Test was found to be a “generalizably valid predictor” of

    • First-year graduate grade point average

    • Overall graduate grade point average

    • Comprehensive exam scores

    • Publication citation counts

    • Faculty ratings

  • Correlations with degree attainment and research productivity were consistently positive, but not statistically significant in all cases.


Results of kuncel et al research
Results of Kuncel et al. Research of critical thinking and analytical writing

  • A follow-up study by Kuncel indicated the GRE General Test has similar levels of predictive validity for both master's and doctoral level programs.

  • The GRE General Test has better predictive validity than undergraduate grades or letters of recommendation.

  • Subject Tests were found to be better predictors of success than were either the General Test or undergraduate grade point average.

    These data strongly suggest that the GRE is a valid predictor of graduate student performance across disciplines, degree levels, age of students, etc.


Results of kuncel et al research continued
Results of Kuncel et al. Research (continued) of critical thinking and analytical writing

For more information about the Kuncel et al. research study, see:

“A Comprehensive Meta-Analysis of the Predictive Validity of the Graduate Record Examinations: Implications for Graduate Student Selection and Performance”

Nathan R. Kuncel

Sarah A. Hezlett

Deniz S. Ones

Psychological Bulletin 2001, Vol. 127, No. 1, 162-181


Gre communications

GRE Communications of critical thinking and analytical writing


Rgre enewsletter
rGRE eNewsletter of critical thinking and analytical writing

  • Content

    • Articles about aspects of the revised General Test and/or GRE tests/services

    • Links to upcoming conferences

    • Links to important information on GRE Web site

  • Quarterly publication


Web seminars
Web Seminars of critical thinking and analytical writing

  • Online, web-based seminars about the revised General Test

  • Conducted by GRE Program Staff

  • Multiple Sessions, multiple topics

    • Overview of the Revised General Test

    • How to Use RGRE Scores in Admissions

    • Overview of the Analytical Writing Measure

  • Email invitations and online registration

  • Multiple Audiences

    • Score recipients/faculty/administrators

    • Undergraduate faculty

    • Test Prep trainers

  • Promote the Website and rGRE eNewsletter


Powerpoint presentations
PowerPoint Presentations of critical thinking and analytical writing

  • Presentations about rGRE will be available on the GRE Web site

    • Multiple topics

    • Include detailed speaker notes

    • Presentations for various audiences

      • Administrators

      • Faculty

      • Test prep trainers

    • Promotes the Website and eNewsletter


ad