Creating a multiple measures placement system
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

Creating A Multiple Measures Placement System PowerPoint PPT Presentation


  • 35 Views
  • Uploaded on
  • Presentation posted in: General

Creating A Multiple Measures Placement System. An Exercise With Ron Gordon & Armand Brunhoeber. The Concept. Although test scores may predict failure, they do not necessarily predict success.

Download Presentation

Creating A Multiple Measures Placement System

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Creating a multiple measures placement system

Creating A Multiple Measures Placement System

An Exercise

With

Ron Gordon & Armand Brunhoeber


The concept

The Concept

  • Although test scores may predict failure, they do not necessarily predict success.

  • Research shows that students’ backgrounds, environments, and personal habits may have more influence on their potential academic success than their residual academic skills.

  • Find a way to factor that information into the placement decision at testing time.


Decision process

Decision Process

  • Involve Faculty

  • Select Questions

    • How many

    • What variety

  • Select Weight Values

    • Use negative weights?

    • How much possible total weight


Select questions

Select Questions

  • Use enough questions that any one does not have undue influence

  • Must be manageable

    • Each weighted answer choice requires an additional line in the placement rule

    • Weights must be repeated in each rule segment

  • Questions must, in some way, relate to student success.


Limits

Limits

  • Limit total weight so that background information does not allow students to skip a course level

  • Limit number of questions to a manageable number

    • More questions adds to testing time

    • Placement rules can become unmanageable

  • Answer choices must be mutually exclusive and all inclusive


Example

Example

  • (If Arithmetic, plus all weighted choices is >= 75 OR

    Algebra, plus all weighted choices is >= 48)AND

    (Algebra, plus all weighted choices is < 65 OR Algebra Not Taken) AND

    (CLM, plus all weighted choices is <62 OR

    CLM Not Taken)

    Then Placement is Elementary Algebra


Example continued

Example, Continued

  • If this rule had 5 questions with 4 weighted answer choices each, there would be 80 lines just for weights.

  • With too many questions, or too many choices per question, rules can become unmanageable


Conditional weights

Conditional Weights

  • High school accomplishments have limited shelf life

  • How much does it matter that a 25-year-old student had 2 years of high school algebra?

  • Does it matter that the same 25-year-old student works for a surveyor and uses algebra daily?


Conditional weights1

Conditional Weights

  • Young students have not had time to build skill usage experience, but their high school accomplishments are relevant

  • What older students accomplished in high school is less relevant than what they do now.


Example1

Example

  • How long has it been since you were enrolled in high school or other formal educational process?

    • Less than 2 years or still enrolled

    • 2 to 5 years

    • More than 5 but less than 7 years

    • 7 years or more

  • Use high school data for up to 5 years, experience for more than 5 years.


Assigning weights

Assigning Weights

  • Total possible weight should not move student more than one level in either direction

  • Set maximum possible weight so a student who scores near or above the midpoint of a placement range could move up, but one who scores below the midpoint could not.

  • Use faculty to select BGQ and assign weight

  • Guide them


Multiple measures movement model

Multiple Measures Movement Model


Sample question with weights

Sample Question With Weights

  • Which choice below best describes you when you read textbooks or other complex information?

    • I usually need to read material several times before I understand it well-.01

    • Sometimes I can understand what I read the first time, but often I must reread it .00

    • I usually understand what I read if I take notes or highlight passages.+.01

    • I always understand what I read the first time through+.02


Preparing to build the system

Preparing to Build the System

  • Assign numeric codes to course names

  • Determine which tests will be used for each course in each discipline

  • Create cut score Table

  • Create a BGQ weight Matrix


Building the system

Building the System

  • Create Background questions

  • Assign BGQ to groups

  • Create branching profiles

  • Create course groups

  • Create courses and assign to groups List

  • Create majors if used

  • Create placement rules Edit1Edit2


Verify

Verify

  • Write most complex rule first

  • Run verify function in branching profile

  • Use several BGQ and score combinations to test the placement rule

  • Compute weighted score for each run

  • Try to hit cut scores to test for bad weight or answer choice selections


Common errors

Common Errors

  • Unequal weights between rules

    • E.G. A response has .01 weight in one rule and -.01 weight in the next rule

  • Misplaced Parentheses

    • The multiple measures weights make the rule larger and more difficult to visualize

  • Misuse of AND/OR

  • Misuse of arithmetic operators

  • Wrong answer choice in rule line


Troubleshooting

Troubleshooting

  • From the score report, determine what the student’s weight should be from the BGQ responses

  • Using the weight, compute the weighted score

  • Determine what the placement should be

  • Examine the appropriate rule for errors


Computing the weight 1

Computing the Weight 1


Computing the weight 2

Computing the Weight 2


Computation

Computation

  • Score is multiplied by 1 plus the accumulated weight.

    • 85 * (1+.04) = 88.4

    • Placement will be based on a score of 88.

  • Example 2

    • 85 * (1+ [-.03]) = 82.45

    • Placement will be based on a score of 82


Creating a multiple measures placement system1

Creating a Multiple Measures Placement System

An exercise with

Ron Gordon &

Armand Brunhoeber

Thank you for not throwing things at the presenters


  • Login