crowdsourcing and all pay auctions n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Crowdsourcing and All-Pay Auctions PowerPoint Presentation
Download Presentation
Crowdsourcing and All-Pay Auctions

Loading in 2 Seconds...

play fullscreen
1 / 44

Crowdsourcing and All-Pay Auctions - PowerPoint PPT Presentation


  • 143 Views
  • Uploaded on

Crowdsourcing and All-Pay Auctions. Milan Vojnović Microsoft Research Joint work with Dominic DiPalantino. UC Berkeley, July 13, 2009. Examples of Crowdsourcing. Crowdsourcing = soliciting solutions via open calls to large-scale communities Coined in a Wired article (’06) Taskcn

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Crowdsourcing and All-Pay Auctions' - hoshiko


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
crowdsourcing and all pay auctions

Crowdsourcing and All-Pay Auctions

Milan Vojnović

Microsoft Research

Joint work with Dominic DiPalantino

UC Berkeley, July 13, 2009

examples of crowdsourcing
Examples of Crowdsourcing
  • Crowdsourcing = soliciting solutions via open calls to large-scale communities
    • Coined in a Wired article (’06)
  • Taskcn
    • 530,000 solutions posted for 3,100 tasks
  • Innocentive
    • Over $3 million awarded
  • Odesk
    • Over $43 million brokered
  • Amazon’s Mechanical Turk
    • Over 23,000 tasks
examples of crowdsourcing cont d
Examples of Crowdsourcing (cont’d)
  • Yahoo! Answers
    • Lunched Dec ’05
    • 60M users / 65M answers (as of Dec ’06)
  • Live QnA
    • Lunched Aug ’06 / closed May ’09
    • 3M questions / 750M answers
  • Wikipedia
incentives for contribution
Incentives for Contribution
  • Incentives
    • Monetary$$$
    • Non-momentarySocial gratification and publicityReputation pointsCertificates and “levels”
  • Incentives for both participation and quality
incentives for contribution cont d
Incentives for Contribution (cont’d)
  • Ex. Taskcn

Contest duration

Number of submissions

Number of registrants

Number of views

Reward range (RMB)

100 RMB  $15 (July 09)

incentives for contribution cont d1
Incentives for Contribution (cont’d)
  • Ex. Yahoo! Answers

Levels

Points

Source: http://en.wikipedia.org/wiki/Yahoo!_Answers

questions of interest
Questions of Interest
  • Understanding of the incentive schemes
    • How do contributions relate to offered rewards?
  • Design of contests
    • How do we best design contests?
    • How do we set rewards?
    • How do we best suggest contests to players and rewards to contest providers?
strategic user behavior
Strategic User Behavior
  • From empirical analysis of Taskcn by Yang et al (ACM EC ’08) – (i) users respond to incentives, (ii) users learn better strategies
    • Suggests a game-theoretic analysis

User Strategies on Taskcn.com

User Strategies on Taskcn.com

outline
Outline
  • Model of Competing Contests
  • Equilibrium Analysis
    • Player-Specific Skills
    • Contest-Specific Skills
  • Design of Contests
  • Experimental Validation
  • Conclusion
single contest competition
Single Contest Competition

c1

c2

R

c3

c4

contest offeringreward R

players

ci = cost per unit effort or quality produced

all pay auction
All-Pay Auction

v1

b1

v2

b2

v3

b3

v4

b4

Everyone pays their bid

competing contests
Competing Contests

1

R1

2

R2

...

...

u

Rj

...

...

RJ

N

users

contests

incomplete information assumption
Incomplete Information Assumption

Each user u knows

= total number of users

= his own skill

= skills are randomly drawn from F

We assume F is an atomless distribution with finite support [0,m]

assumptions on user skill
Assumptions on User Skill

1) Player-specific skill random i.i.d. across u(ex. contests require similar skills or skill determined by player’s opportunity cost)

2) Contest-specific skill random i.i.d. across u and j(ex. contests require diverse skills)

bayes nash equilibrium
Bayes-Nash Equilibrium
  • Mixed strategy
  • Equilibrium

Select contest of highest expected profit where expectation with respect to “beliefs” about other user skills

= prob. of selecting a contest of class j

= bid

Contest class = set of contests that offer same reward

user expected profit
User Expected Profit
  • Expected profit for a contest of class j

= prob. of selecting a contest of class j

= distribution of user skillconditional on having selected contest class j

outline1
Outline
  • Model of Competing Contests
  • Equilibrium Analysis
    • Player-Specific Skills
    • Contest-Specific Skills
  • Design of Contests
  • Experimental Validation
  • Conclusion
equilibrium contest selection
Equilibrium Contest Selection

m

1

1

v2

2

2

v3

3

3

v4

4

4

0

5

contestclasses

skill

levels

threshold reward
Threshold Reward
  • Only K highest-reward contest classes selected with strictly positive probability

= number of contests of class k

partitioning over skill levels
Partitioning over Skill Levels
  • User of skill v is of skill level l if

where

contest selection
Contest Selection
  • User of skill l, i.e. with skill selects a contest of class j with probability
participation rates
Participation Rates
  • A contest of class j selected with probability
  • Prior-free – independent of the distribution F
large system limit
Large-System Limit
  • For positive constants

where K is a finite number of contest classes

skill levels for large system
Skill Levels for Large System
  • User of skill v is of skill level l if

where

participation rates for large system
Participation Rates for Large System
  • Expected number of participants for a contest of class j
  • Prior-free – independent of the distribution F
contest selection in large system
Contest Selection in Large System
  • User of skill l, i.e. with skill selects a contest of class j with probability

m

1

1

2

2

3

3

4

4

0

5

1/3

  • For large systems, what matters is which contests are selected for given skill

1/3

1/3

proof hint for player specific skills
Proof Hint for Player-Specific Skills

g1(v)

  • Key property – equilibrium expected payoffs as showed

g2(v)

g3(v)

g4(v)

v

0

v3

v2

v1

m

outline2
Outline
  • Model of Competing Contests
  • Equilibrium Analysis
    • Player-Specific Skills
    • Contest-Specific Skills
  • Design of Contests
  • Experimental Validation
  • Conclusion
contest specific skills
Contest-specific Skills
  • Results established only for large-system limit
  • Same equilibrium relationship between participation and rewards as for player-specific skills
proof hints
Proof Hints
  • Limit expected payoff – For each
  • Balancing – Whenever
  • Asserted relations for follow from above
outline3
Outline
  • Model of Competing Contests
  • Equilibrium Analysis
    • Player-Specific Skills
    • Contest-Specific Skills
  • Design of Contests
  • Experimental Validation
  • Conclusion
system optimum rewards
System Optimum Rewards
  • Set the rewards so as to optimize system welfare

SYSTEM

maximise

over

subject to

example 1 zero costs non monetary rewards
Example 1: zero costs(non monetary rewards)

Assume are increasing strictly concave functions. Under player-specific skills, system optimum rewards:

for any c > 0 where  is unique solution of

  • Rewards unique up to a multiplicative constant – only relative setting of rewards matters
example 1 cont d
Example 1 (cont’d)
  • For large systems

Assume are increasing strictly concave functions. Under player-specific skills, system optimum rewards:

for any c > 0 where  is unique solution of

example 2 optimum effort
Example 2: optimum effort
  • Consider SYSTEM with

Utility:

{

exerted effort

Cost:

{

{

prob. contest attended

cost of

giving Rj

(budget constraint)

outline4
Outline
  • Model of Competing Contests
  • Equilibrium Analysis
    • Player-Specific Skills
    • Contest-Specific Skills
  • Design of Contests
  • Experimental Validation
  • Conclusion
taskcn
Taskcn
  • Analysis of rewards and participation across tasks as observed on Taskcn
    • Tasks of diverse categories: graphics, characters, miscellaneous, super challenge
    • We considered tasks posted in 2008
taskcn cont d
Taskcn (cont’d)

reward

number of views

number of registrants

number of submissions

submissions vs reward
Submissions vs. Reward
  • Diminishing increase of submissions with reward

Graphics

Characters

Miscellaneous

linear regression

submissions vs reward for subcategory logos
Submissions vs. Rewardfor Subcategory Logos
  • Conditional on the rate at which users submit solutions
  • Conditioning on the more experienced users, the better the prediction by the model

any rate

once a month

every fourth day

every second day

model

same for the subcategory 2 d
Same for the Subcategory 2-D

any rate

once a month

every fourth day

every second day

model

conclusion
Conclusion
  • Crowdsourcing as a system of competing contests
  • Equilibrium analysis of competing contests
    • Explicit relationship between rewards and participations
      • Prior-free
    • Diminishing increase of participation with reward
      • Suggested by the model and data
  • Framework for design of crowdsourcing / contests
  • Base results for strategic modelling
    • Ex. strategic contest providers
more information
More Information
  • Paper: ACM EC ’09
  • Version with proofs: MSR-TR-2009-09
    • http://research.microsoft.com/apps/pubs/default.aspx?id=79370