Testing Transitivity (and other Properties) Using a True and Error Model - PowerPoint PPT Presentation

Testing transitivity and other properties using a true and error model
1 / 34

  • Uploaded on
  • Presentation posted in: General

Testing Transitivity (and other Properties) Using a True and Error Model. Michael H. Birnbaum. Testing Algebraic Models with Error-Filled Data. Algebraic models assume or imply formal properties such as stochastic dominance, coalescing, transitivity, gain-loss separability, etc.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Testing Transitivity (and other Properties) Using a True and Error Model

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Testing transitivity and other properties using a true and error model

Testing Transitivity (and other Properties) Using a True and Error Model

Michael H. Birnbaum

Testing algebraic models with error filled data

Testing Algebraic Models with Error-Filled Data

  • Algebraic models assume or imply formal properties such as stochastic dominance, coalescing, transitivity, gain-loss separability, etc.

  • But these properties will not hold if data contain “error.”

Some proposed solutions

Some Proposed Solutions

  • Neo-Bayesian approach (Myung, Karabatsos, & Iverson.

  • Cognitive process approach (Busemeyer)

  • “Error” Theory (“Error Story”) approach (Thurstone, Luce) combined with algebraic models.

Variations of error models

Variations of Error Models

  • Thurstone, Luce: errors related to separation between subjective values. Case V: SST (scalability).

  • Harless & Camerer: errors assumed to be equal for certain choices.

  • Today: Allow each choice to have a different rate of error.

  • Advantage: we desire error theory that is both descriptive and neutral.

Basic assumptions

Basic Assumptions

  • Each choice in an experiment has a true choice probability, p, and an error rate, e.

  • The error rate is estimated from (and is the “reason” given for) inconsistency of response to the same choice by same person over repetitions

One choice two repetitions

One Choice, Two Repetitions

Solution for e

Solution for e

  • The proportion of preference reversals between repetitions allows an estimate of e.

  • Both off-diagonal entries should be equal, and are equal to:

Estimating e

Estimating e

Estimating p

Estimating p

Testing if p 0

Testing if p = 0

Ex stochastic dominance

Ex: Stochastic Dominance

122 Undergrads: 59% repeated viols (BB)

28% Preference Reversals (AB or BA)

Estimates: e = 0.19; p = 0.85

170 Experts: 35% repeated violations

31% Reversals

Estimates: e = 0.196; p = 0.50

Chi-Squared test reject H0: p < 0.4

Testing 3 choice properties

Testing 3-Choice Properties

  • Extending this model to properties using 2, 3, or 4 choices is straightforward.

  • Allow a different error rate on each choice.

  • Allow a true probability for each choice pattern.

Response combinations

Response Combinations

Weak stochastic transitivity

Weak Stochastic Transitivity

Wst can be violated even when everyone is perfectly transitive

WST Can be Violated even when Everyone is Perfectly Transitive

Model for transitivity

Model for Transitivity

A similar expression is written for the

other seven probabilities. These can in turn be expanded to predict the probabilities of showing each pattern repeatedly.

Starmer 1999 data

Starmer (1999) data

  • A = ($15, 0.2; $0, 0.8)

  • B = ($8; 0.3; $0, 0.7)

  • C = ($8, 0.15; $7.75; 0.15; $0, .7)

  • Starmer predicted intransitivity from Prospect Theory and the dominance detection (editing) mechanism.

Starmer best data

Starmer (Best) Data

Transitive solution to starmer data

Transitive Solution to Starmer Data

Full model is underdetermined. One error

Fixed to zero; but other errors not equal.

Most people recognized dominance.

Expand and simplify

Expand and Simplify

  • There are 8 X 8 data patterns in an experiment with 2 repetitions.

  • However, most of these have very small probabilities.

  • Examine probabilities of each of 8 repeated patterns.

  • Probability of showing each of 8 patterns in one replicate OR the other, but NOT both. Mutually exclusive, exhaustive partition.

New studies of transitivity

New Studies of Transitivity

  • Work currently under way testing transitivity under same conditions as used in tests of other decision properties.

  • Participants view choices via the WWW, click button beside the gamble they would prefer to play.

Some recipes being tested

Some Recipes being Tested

  • Tversky’s (1969) 5 gambles.

  • LS: Preds of Priority Heuristic

  • Starmer’s recipe

  • Additive Difference Model

  • Birnbaum, Patton, & Lott (1999) recipe.

Tversky gambles

Tversky Gambles

  • Some Sample Data, using Tversky’s 5 gambles, but formatted with tickets instead of pie charts.

  • Data as of May 5, 2005, n = 123.

  • No pre-selection of participants.

  • Participants served in other studies, prior to testing (~1 hr).

Three of the gambles

Three of the Gambles

  • A = ($5.00, 0.29; $0, 0.79)

  • C = ($4.50, 0.38; $0, 0.62)

  • E = ($4.00, 0.46; $0, 0.54)

Results ace


Test of wst

Test of WST



  • Preliminary results were surprisingly transitive.

  • Difference: no pre-test, selection

  • Probability represented by # of tickets (100 per urn)

  • Participants have practice with variety of gambles, & choices.

  • Tested via Computer

Test of gain loss separability

Test of Gain-Loss Separability

  • Same Structure as Transitivity

  • Property implied by CPT, RSDU

  • Property violated by TAX.

  • Loss Aversion: people do not like fair bets to win or lose.

  • CPT: Loss Aversion due to utility function for gains and losses.

Gain loss separability

Gain-Loss Separability



Birnbaum bahra f

Birnbaum & Bahra--% F

Birnbaum bahra

Birnbaum & Bahra

Summary gls

Summary GLS

  • Wu & Markle (2004) found evidence of violation of GLS. Modified CPT.

  • Birnbaum & Bahra (2005) also find evidence of violation of GLS, violations of modified CPT as well.

  • TAX: In mixed gambles, losses get greater weight. Data do not require kink in the utility function at zero.



  • True & Error model with different error rates seems a reasonable “null” hypothesis for testing transitivity and other properties.

  • Requires data with replications so that we can use each person’s self-agreement or reversals to estimate whether response patterns are “real” or due to “error.”

  • Login