Testing transitivity and other properties using a true and error model
1 / 34

Testing Transitivity (and other Properties) Using a True and Error Model - PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Testing Transitivity (and other Properties) Using a True and Error Model. Michael H. Birnbaum. Testing Algebraic Models with Error-Filled Data. Algebraic models assume or imply formal properties such as stochastic dominance, coalescing, transitivity, gain-loss separability, etc.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Testing Transitivity (and other Properties) Using a True and Error Model

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Testing Transitivity (and other Properties) Using a True and Error Model

Michael H. Birnbaum

Testing Algebraic Models with Error-Filled Data

  • Algebraic models assume or imply formal properties such as stochastic dominance, coalescing, transitivity, gain-loss separability, etc.

  • But these properties will not hold if data contain “error.”

Some Proposed Solutions

  • Neo-Bayesian approach (Myung, Karabatsos, & Iverson.

  • Cognitive process approach (Busemeyer)

  • “Error” Theory (“Error Story”) approach (Thurstone, Luce) combined with algebraic models.

Variations of Error Models

  • Thurstone, Luce: errors related to separation between subjective values. Case V: SST (scalability).

  • Harless & Camerer: errors assumed to be equal for certain choices.

  • Today: Allow each choice to have a different rate of error.

  • Advantage: we desire error theory that is both descriptive and neutral.

Basic Assumptions

  • Each choice in an experiment has a true choice probability, p, and an error rate, e.

  • The error rate is estimated from (and is the “reason” given for) inconsistency of response to the same choice by same person over repetitions

One Choice, Two Repetitions

Solution for e

  • The proportion of preference reversals between repetitions allows an estimate of e.

  • Both off-diagonal entries should be equal, and are equal to:

Estimating e

Estimating p

Testing if p = 0

Ex: Stochastic Dominance

122 Undergrads: 59% repeated viols (BB)

28% Preference Reversals (AB or BA)

Estimates: e = 0.19; p = 0.85

170 Experts: 35% repeated violations

31% Reversals

Estimates: e = 0.196; p = 0.50

Chi-Squared test reject H0: p < 0.4

Testing 3-Choice Properties

  • Extending this model to properties using 2, 3, or 4 choices is straightforward.

  • Allow a different error rate on each choice.

  • Allow a true probability for each choice pattern.

Response Combinations

Weak Stochastic Transitivity

WST Can be Violated even when Everyone is Perfectly Transitive

Model for Transitivity

A similar expression is written for the

other seven probabilities. These can in turn be expanded to predict the probabilities of showing each pattern repeatedly.

Starmer (1999) data

  • A = ($15, 0.2; $0, 0.8)

  • B = ($8; 0.3; $0, 0.7)

  • C = ($8, 0.15; $7.75; 0.15; $0, .7)

  • Starmer predicted intransitivity from Prospect Theory and the dominance detection (editing) mechanism.

Starmer (Best) Data

Transitive Solution to Starmer Data

Full model is underdetermined. One error

Fixed to zero; but other errors not equal.

Most people recognized dominance.

Expand and Simplify

  • There are 8 X 8 data patterns in an experiment with 2 repetitions.

  • However, most of these have very small probabilities.

  • Examine probabilities of each of 8 repeated patterns.

  • Probability of showing each of 8 patterns in one replicate OR the other, but NOT both. Mutually exclusive, exhaustive partition.

New Studies of Transitivity

  • Work currently under way testing transitivity under same conditions as used in tests of other decision properties.

  • Participants view choices via the WWW, click button beside the gamble they would prefer to play.

Some Recipes being Tested

  • Tversky’s (1969) 5 gambles.

  • LS: Preds of Priority Heuristic

  • Starmer’s recipe

  • Additive Difference Model

  • Birnbaum, Patton, & Lott (1999) recipe.

Tversky Gambles

  • Some Sample Data, using Tversky’s 5 gambles, but formatted with tickets instead of pie charts.

  • Data as of May 5, 2005, n = 123.

  • No pre-selection of participants.

  • Participants served in other studies, prior to testing (~1 hr).

Three of the Gambles

  • A = ($5.00, 0.29; $0, 0.79)

  • C = ($4.50, 0.38; $0, 0.62)

  • E = ($4.00, 0.46; $0, 0.54)


Test of WST


  • Preliminary results were surprisingly transitive.

  • Difference: no pre-test, selection

  • Probability represented by # of tickets (100 per urn)

  • Participants have practice with variety of gambles, & choices.

  • Tested via Computer

Test of Gain-Loss Separability

  • Same Structure as Transitivity

  • Property implied by CPT, RSDU

  • Property violated by TAX.

  • Loss Aversion: people do not like fair bets to win or lose.

  • CPT: Loss Aversion due to utility function for gains and losses.

Gain-Loss Separability


Birnbaum & Bahra--% F

Birnbaum & Bahra

Summary GLS

  • Wu & Markle (2004) found evidence of violation of GLS. Modified CPT.

  • Birnbaum & Bahra (2005) also find evidence of violation of GLS, violations of modified CPT as well.

  • TAX: In mixed gambles, losses get greater weight. Data do not require kink in the utility function at zero.


  • True & Error model with different error rates seems a reasonable “null” hypothesis for testing transitivity and other properties.

  • Requires data with replications so that we can use each person’s self-agreement or reversals to estimate whether response patterns are “real” or due to “error.”

  • Login