Loading in 5 sec....

Prediction Markets and the Wisdom of CrowdsPowerPoint Presentation

Prediction Markets and the Wisdom of Crowds

- 397 Views
- Updated On :

Prediction Markets and the Wisdom of Crowds. David Pennock Yahoo! Research NYC. Outline. Prediction Markets Survey What is a prediction market? Examples Some research findings The Wisdom of Crowds: A Story. Bet = Credible Opinion. Hillary Clinton will win the election.

Related searches for Prediction Markets and the Wisdom of Crowds

Download Presentation
## PowerPoint Slideshow about 'Prediction Markets and the Wisdom of Crowds' - Roberta

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Outline

- Prediction Markets Survey
- What is a prediction market?
- Examples
- Some research findings

- The Wisdom of Crowds:A Story

Bet = Credible Opinion

Hillary Clinton will win the election

- Which is more believable?More Informative?
- Betting intermediaries
- Las Vegas, Wall Street, Betfair, Intrade,...
- Prices: stable consensus of a large number of quantitative, credible opinions
- Excellent empirical track record

“I bet $100 Hillary will win at 1 to 2 odds”

A Prediction Market

- Take a random variable, e.g.
- Turn it into a financial instrument payoff = realized value of variable

Bird Flu Outbreak US 2007?(Y/N)

I am entitled to:

Bird FluUS ’07

Bird FluUS ’07

$1 if

$0 if

Why?

Get information

- price probability of uncertain event(in theory, in the lab, in the field, ...more later)
- Is there some future event you’d like to forecast?A prediction market can probably help

6

= 6 ?

= 6

I am entitled to:

$1 if

$0 if

A Prediction Market- Take a random variable, e.g.
- Turn it into a financial instrument payoff = realized value of variable

2008 CAEarthquake?

US’08Pres =Dem?

Aside: Terminology

- Key aspect: payout is uncertain
- Called variously: asset, security, contingent claim, derivative (future, option), stock, prediction market, information market, gamble, bet, wager, lottery
- Historically mixed reputation
- Esp. gambling aspect
- A time when options were frowned upon

- But when regulated serve important social roles...

6

= 6

I am entitled to:

$1 if

$0 if

Getting Information- Non-market approach: ask an expert
- How much would you pay for this?

- A: $5/36 $0.1389
- caveat: expert is knowledgeable
- caveat: expert is truthful
- caveat: expert is risk neutral, or ~ RN for $1
- caveat: expert has no significant outside stakes

6

= 6

= 6

Getting Information- Non-market approach:pay an expert
- Ask the expert for his report r of the probabilityP( )
- Offer to pay the expert
- $100 + log r if
- $100 + log (1-r) if

- It so happens that the expert maximizes expected profit by reporting r truthfully
- caveat: expert is knowledgeable
- caveat: expert is truthful
- caveat: expert is risk neutral, or ~ RN
- caveat: expert has no significant outside stakes

“logarithmic scoring rule”,

a “proper” scoring rule

6

= 6

= 6

$1 if

$0 if

Getting Information- Market approach: “ask” the public—experts & non-experts alike—by opening a market:
- Let any person i submit a bid order: an offer to buy qi units at price pi
- Let any person j submit an ask order: an offer to sell qj units at price pj(if you sell 1 unit, you agree to pay $1 if )
- Match up agreeable trades (many poss. mechs...)

Sampling

No incentive to be truthful

Equally weighted information

Hard to be real-time

Ask Experts

Identifying experts can be hard

Incentives

Combining opinions can be difficult

Prediction Markets

Self-selection

Monetary incentive and more

Money-weighted information

Real-time

Self-organizing

Non-Market Alternatives vs. MarketsHistorical data

Past and future are related

Hard to incorporate recent new information

Prediction Markets

No need for data

No assumption on past and future

Immediately incorporate new information

Non-Market Alternatives vs. MarketsFunction of Markets 2: Risk Management

- If is bad for me,
I buy a bunch of

- If my house is struck by lightening, I am compensated.

Risk Management Examples

- Insurance
- I buy car insurance to hedge the risk of accident

- Futures
- Farmers sell soybean futures to hedge the risk of price drop

- Options
- Investors buy options to hedge the risk of stock price changes

What you can say/learn% chance that

Hillary wins

GOP wins Texas

YHOO stock > 30

Duke wins tourney

Oil prices fall

Heat index rises

Hurricane hits Florida

Rains at place/time

Where

IEM, Intrade.com

Intrade.com

Stock options market

Las Vegas, Betfair

Futures market

Weather derivatives

Insurance company

Weatherbill.com

Giving/Getting Informationhttp://www.hedgestreet.com/

Screen capture 2007/05/18

Screen capture 2007/05/18

Play money;Real predictions

http://www.hsx.com/

Yahoo!/O’Reilly Tech Buzz Game

http://buzz.research.yahoo.com/

More Prediction Market Games

- BizPredict.com
- CasualObserver.net
- FTPredict.com
- InklingMarkets.com
- ProTrade.com
- StorageMarkets.com
- TheSimExchange.com
- TheWSX.com
- Alexadex, Celebdaq, Cenimar, BetBubble, Betocracy, CrowdIQ, MediaMammon,Owise, PublicGyan, RIMDEX, Smarkets, Trendio, TwoCrowds
- http://www.chrisfmasse.com/3/3/markets/#Play-Money_Prediction_Markets

Catalysts

- Markets have long history of predictive accuracy: why catching on now as tool?
- No press is bad press: Policy Analysis Market (“terror futures”)
- Surowiecki's “Wisdom of Crowds”
- Companies:
- Google, Microsoft, Yahoo!; CrowdIQ, HSX, InklingMarkets, NewsFutures

- Press: BusinessWeek, CBS News, Economist, NYTimes, Time, WSJ, ...http://us.newsfutures.com/home/articles.html

Does it work?

- Yes, evidence from real markets, laboratory experiments, and theory
- Racetrack odds beat track experts [Figlewski 1979]
- Orange Juice futures improve weather forecast [Roll 1984]
- I.E.M. beat political polls 451/596 [Forsythe 1992, 1999][Oliven 1995][Rietz 1998][Berg 2001][Pennock 2002]
- HP market beat sales forecast 6/8 [Plott 2000]
- Sports betting markets provide accurate forecasts of game outcomes [Gandar 1998][Thaler 1988][Debnath EC’03][Schmidt 2002]
- Market games work [Servan-Schreiber 2004][Pennock 2001]
- Laboratory experiments confirm information aggregation[Plott 1982;1988;1997][Forsythe 1990][Chen, EC’01]
- Theory: “rational expectations” [Grossman 1981][Lucas 1972]
- More later …

[Source: Berg, DARPA Workshop, 2002]

Example: IEM 1992[Source: Berg, DARPA Workshop, 2002]

Example: IEM[Source: Berg, DARPA Workshop, 2002]

Example: IEMSpeed: TradeSports

[Source: Wolfers 2004]

Contract: Pays $100 if Cubs win game 6 (NLCS)

Price of contract

(=Probability that Cubs win)

Fan reaches over and spoils Alou’s catch. Still 1 out.

Cubs are winning 3-0 top of the 8th1 out.

The Marlins proceed to hit 8 runs in the 8th inning

Time (in Ireland)

MP1-37

2003 NFL Season

ProbabilitySports.com Online football forecasting competition

Contestants assess probabilities for each game

Quadratic scoring rule

~2,000 “experts”, plus:

NewsFutures (play $)

Tradesports (real $)

Used “last trade” prices

Results:

Play money and real money performed similarly

6th and 8th respectively

Markets beat most of the ~2,000 contestants

Average of experts came 39th (caveat)

Does money matter? Play vs real, head to headElectronic Markets, Emile Servan-Schreiber, Justin Wolfers, David Pennock and Brian Galebach

Real marketsvs. market games

probabilisticforecasts

HSX

FX, F1P6

forecast source avg log score

F1P6 linear scoring -1.84

F1P6 F1-style scoring -1.82

betting odds -1.86

F1P6 flat scoring -2.03

F1P6 winner scoring -2.32

expectedvalueforecasts489 movies

MP1-42

1/7

Story

Survey

Research

A Wisdom of Crowds StoryOpinion

- ProbabilitySports.com
- Thousands of probability judgments for sporting events
- Alice: Jets 67% chance to beat Patriots
- Bob: Jets 48% chance to beat Patriots
- Carol, Don, Ellen, Frank, ...

- Reward: Quadratic scoring rule:Best probability judgments maximize expected score

Individuals

- Most individuals are poor predictors
- 2005 NFL Season
- Best: 3747 points
- Average: -944 Median: -275
- 1,298 out of 2,231 scored below zero(takes work!)

Poorly calibrated (too extreme)

Teams given < 20% chance actually won 30% of the time

Teams given > 80% chance actually won 60% of the time

IndividualsThe Crowd

- Create a crowd predictor by simply averaging everyone’s probabilities
- Crowd = 1/n(Alice + Bob + Carol + ... )
- 2005: Crowd scored 3371 points(7th out of 2231) !

- Wisdom of fools: Create a predictor by averaging everyone who scored below zero
- 2717 points (62nd place) !
- (the best “fool” finished in 934th place)

The Crowd: How Big?

More:http://blog.oddhead.com/2007/01/04/the-wisdom-of-the-probabilitysports-crowd/http://www.overcomingbias.com/2007/02/how_and_when_to.html

Can We Do Better?: ML/Stats

[Dani et al. UAI 2006]

- Maybe Not
- CS “experts algorithms”
- Other expert weights
- Calibrated experts
- Other averaging fn’s (geo mean, RMS, power means, mean of odds, ...)
- Machine learning (NB, SVM, LR, DT, ...)

- Maybe So
- Bayesian modeling + EM
- Nearest neighbor (multi-year)

Download Presentation

Connecting to Server..