conjoint adaptive ranking database system cards n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Conjoint Adaptive Ranking Database System ( CARDS ) PowerPoint Presentation
Download Presentation
Conjoint Adaptive Ranking Database System ( CARDS )

Loading in 2 Seconds...

play fullscreen
1 / 29

Conjoint Adaptive Ranking Database System ( CARDS ) - PowerPoint PPT Presentation


  • 116 Views
  • Uploaded on

Good, Fast, Cheap and Easy?. Conjoint Adaptive Ranking Database System ( CARDS ). Ely Dahan. Michael Yee, John Hauser & Jim Orlin. EXPLOR Award Winning Presentation – September 22, 2004. The Problem. Current methods require many questions for few answers

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Conjoint Adaptive Ranking Database System ( CARDS )' - eddy


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
conjoint adaptive ranking database system cards

Good, Fast, Cheap and Easy?

ConjointAdaptive Ranking Database System (CARDS)

Ely Dahan

Michael Yee, John Hauser & Jim Orlin

EXPLOR Award Winning Presentation – September 22, 2004

the problem
The Problem
  • Current methods require manyquestions for few answers
  • Respondents must rate products they don’t like
  • Simplifying rules to narrowchoices not typically captured
  • Respondents make mistakesdue to fatigue, causing inconsistency
  • Is there a better way?
example smart phone

Small

Service

Phone Brand

Mini Keyboard

Flip

Example: Smart Phone
  • Respondent: Alex Bell
  • How does Alex choosea smart phone?

5

6

7

Utility Scores:

Alex makes tradeoffs

9

10

example smart phone1

Small

Service

Phone Brand

Mini Keyboard

Flip

Example: Smart Phone
  • Respondent: Alex Bell
  • How does Alex choosea smart phone?

1

2

4

Process of Elimination:Focus on key features

8

16

slide6

Consider this tough task:Rank 32 Smart Phones based on your preferences

Are wesurprised thatrespondents becomefatigued and makemistakes!?

There are a billion, billion, billion, billion,

ways for a respondent to rank 32 smartphones!

prior research on adaptive questioning
Prior Research on Adaptive Questioning
  • Johnson (1987, 1991) & Orme and King (‘02) Sawtooth ACA
  • Huber and Zwerina (1996), Aggregate utility balance
  • Arora and Huber (2001), Aggregate customization
  • Sandor and Wedel (2001), Aggregate + prior beliefs
  • Louviere, Hensher, and Swait (2000), Aggregate CBC

Prior Research on Fast & Frugal Rules

  • Tversky (1969, 1972), lexicographic semi-order, elimination by aspects
  • Dawes and Corrigan (1974), unit weights, linear models
  • Montgomery and Svenson (1976), 2-stage processing
  • Thorngate (1980), efficient decision heuristics
  • Shugan (1980), cost of thinking (pair wise comparisons)
  • Johnson, Meyer, et. al. (1984, 1989), protocol anal., choice models can fail
  • Roberts and Lattin (1991, 1997), two-stage w/greedy
  • Gigerenzer and Goldstein (1996), Take the Best & others
  • Bettman, Luce, Payne (1996, 1998), Accuracy vs. effort, lexicography
  • Martignon and Hoffrage (2002), fast and frugal is robust
two new ideas
Two new ideas:
  • IDEA 1: We can now measure Alex’s process of elimination
  • IDEA 2: We can help Alex avoid inconsistent answers
slide9

IDEA 1:

Phone Brand

Mini Keyboard

Flip

Customer Insight:Respondents may be using a simpleprocess of eliminationto narrow choices for consideration

“I will only considerflip phones, with mini-keyboards, from Blackberry”

customer insight respondents may be using a simple process of elimination

IDEA 1:

Customer Insight:Respondents may be using a simpleprocess of elimination

How hard is it to identify each respondent’s simplifying rule?

AGB

how can we identify each respondent s process of elimination

IDEA 1:

How can we identify each respondent’sprocess of elimination

?

  • Tougher than it seems, because they may be using one of a huge number of possible rules
  • We solved this problem with a new computer technique (speedy)
  • We tested our theory and it works!
the big benefit of identifying respondents process of elimination
The big benefit of identifying respondents’process of elimination

?

  • Good Accuracy, customer insight
  • Fast 1 minute for them, quick for us
  • Cheap Pack more into the same study
  • Easy Reduce drudgery
slide13

Process of elimination Benefit: Cheap

RankSome

RankAll

7 minutes

2 minutes

2

7

Fast

Could you use 5 extra minutes of survey time?

slide14

Benefit: Easy

kind of fun

okay

about right

long

Ranksome

Rankall

Somewhatinteresting

slide18

IDEA 2: Avoiding inconsistent answers

The consistency criterion,a new approach

Reduce response error by “guiding” respondents towards consistent answers

Each choice must be 100% consistent with at least one set of utility scores

slide19

IDEA 2: Consistency

Show product features

Click on favorite cards

Inconsistentcards just“disappear”

GetUtilityscores

Save lotsof clicks

Keeping people consistent:

Conjoint Adaptive Ranking Database System (CARDS)

slide20

IDEA 2: Consistency

10

9

7

6

5

Small

Service

Phone Brand

Mini Keyboard

Flip

How do we keep people consistent?

Imagine we knew Alex Bell’s utility scores…

We would know how he would rank all 32 phones

slide21

IDEA 2: Consistency

How do we keep people consistent?

Imagine we knew every possible set of scores…

Each set of utility scores are consistent witha unique ranking of all 32 phones

Surprise: Consistent rankings are atiny percentage of the possible answers

Eliminate rankings not on the consistent list

the big benefit of keeping respondents consistent
The big benefit of keeping respondentsconsistent

?

  • Good Accuracy, consistent
  • Fast minutes for them, quick for us
  • Cheap Pack more into the same study
  • Easy 50% to 75% effort reduction
slide23

Consistency Benefit: Easy

7

cards

17

cards

Without consistency

With consistency

Consistency reduced effort 73%!

slide24

Extra benefits of consistency

  • Scalable
  • Utility Scores as you go
  • Emphasizes likes
  • Measures uncertainty
key takeaways
Key Takeaways:
  • Good Predictive; Customer insight
  • Fast For them and for us
  • Cheap Pack more into one study
  • Easy Reduce drudgery & mistakes
slide29

Thank you for this exciting award!

edahan@ucla.edu

Good, Fast, Cheap, Easy demonstrations:http://orc-pumba.mit.edu/~myee/CARDS/conjoint.php Use email: nodebug7@

http://wow.mit.edu