Create Presentation
Download Presentation

Download Presentation
## Performance Evaluation

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Performance Evaluation**Sponsored Search Markets Giovanni Neglia INRIA – EPI Maestro 4 February 2013**Google**• A class of games for which there is a function P(s1,s2,…sN) such that • For each i Ui(s1,s2,…xi,…sN)>Ui(s1,s2,…yi,…sN) if and only if P(s1,s2,…xi,…sN)>P(s1,s2,…yi,…sN) • Properties of potential games: Existence of a pure-strategy NE and convergence to it of best-response dynamics • The routing games we considered are particular potential games**How itworks**• Companies bid for keywords • On the basis of the bids Google puts their link on a given position (first ads get more clicks) • Companies are charged a given cost for each click (the cost depends on all the bids)**Somenumbers**• ≈ 95% of Google revenues (46 billions$) fromads • investor.google.com/financial/tables.html • 87% of Google-Motorola revenues (50 billions$) • Costs • "calligraphypens" $1.70 • "Loan consolidation" $50 • "mesothelioma" $50 per click • Click fraudproblem**Outline**• Preliminaries • Auctions • Matchingmarkets • Possible approaches to adspricing • Google mechanism • References • Easley, Kleinberg, "Networks, Crowds and Markets", ch.9,10,15**Types of auctions**• 1stprice & descendingbids • 2ndprice & ascendingbids**Game Theoretic Model**• N players (the bidders) • Strategies/actions: biisplayeri’sbid • For player i the good has value vi • piisplayeri’spayment if hegets the good • Utility: • vi-piif player i gets the good • 0 otherwise • Assumptionhere: values vi are independentand private • i.e. veryparticulargoods for whichthereis not a referenceprice**Game Theoretic Model**• N players (the bidders) • Strategies: biisplayeri’sbid • Utility: • vi-bi if player i gets the good • 0 otherwise • Difficulties: • Utilities of otherplayers are unknown! • Better to model the strategyspaceas continuous • Most of the approacheswestudied do not work!**2ndpriceauction**• Player with the highestbidgets the good and pays a priceequal to the 2ndhighestbid • There is a dominant strategies • I.e. a strategythatis more convenientindependentlyfromwhat the otherplayers do • Be truthful, i.e. bid how muchyouevaluate the good (bi=vi) • Social optimality: the bidderwho value the good the mostgetsit!**bi=viis the highestbid**bids bids bi’>bi bi bk bk Ui=vi-bk>vi-bi=0 Ui’=vi-bk bh bh bn bn Bidding more than viis not convenient**bi=viis the highestbid**bids bids bi bk bk bi’<bi Ui=vi-bk>vi-bi=0 Ui’=0 bh bh bn bn Biddinglessthan viis not convenient (maybeunconvenient)**bi=viis not the highestbid**bids bids bi’>bi Ui’=vi-bk<vi-bi=0 bk bk bi Ui=0 bh bh bn bn Bidding more than viis not convenient (maybeunconvenient)**bi=viis not the highestbid**bids bids bk bk bi bi’<bi Ui’=0 Ui=0 bh bh bn bn Bidding more than viis not convenient**Seller revenue**• N bidders • Values are independentrandom values between 0 and 1 • Expectedithlargest utility is(N+1-i)/(N+1) • Expected seller revenue is(N-1)/(N+1)**1stpriceauction**• Player with the highestbidgets the good and pays a priceequal to her/hisbid • Beingtruthfulis not a dominant strategyanymore! • How to studyit?**1stpriceauction**• Assumption: for eachplayer the other values are i.i.d. random variables between 0 and 1 • to overcome the factthat utilities are unknown • Player i’sstrategyis a function s() mapping value vi to a bid bi • s() strictlyincreasing, differentiablefunction • 0≤s(v)≤v s(0)=0 • Weinvestigate if thereis a strategy s() common to all the playersthat leads to a Nash equilibrium**1stpriceauction**• Assumption: for eachplayer the other values are i.i.d. random variables between 0 and 1 • Player i’sstrategyis a function s() mapping value vi to a bid bi • Expectedpayoff of player i if all the playersplays s(): • Ui(s,…s,…s) = viN-1 (vi-s(vi)) i’spayoff if he/shewins prob. i wins**1stpriceauction**• Expectedpayoff of player i if all the playersplay s(): • Ui(s,…s,…s) = viN-1 (vi-s(vi)) • What if i plays a differentstrategyt()? • If all playersplaying s() is a NE, then : • Ui(s,…s,…s) = viN-1(vi-s(vi)) ≥ viN-1 (vi-t(vi)) = = Ui(s,…t,…s) • Difficult to check for all the possible functionst() differentfrom s() • Help from the revelationprinciple**The RevelationPrinciple**• All the strategies are equivalent to bidder i supplying to s() a different value of vi s() vi bi t() s() vi' vi bi' bi'**1stpriceauction**• Expectedpayoff of player i if all the playersplays s(): • Ui(v1,…vi,…vN) = Ui(s,…s,…s) = viN-1 (vi-s(vi)) • What if i plays a differentstrategyt()? • By the revelationprinciple: • Ui(s,…t,…s) = Ui(v1,…v,…vN)= vN-1(vi-s(v)) • If viN-1 (vi-s(vi)) ≥ vN-1(vi-s(v)) for each v (and for each vi) • Then all playersplaying s() is a NE**1stpriceauction**• If viN-1 (vi-s(vi)) ≥ vN-1(vi-s(v)) for each v (and for each vi) • Then all playersplaying s() is a NE • f(v)=viN-1 (vi-s(vi)) - vN-1 (vi-s(v)) isminimized for v=vi • f’(v)=0 for v=vi, • i.e. (N-1) viN-2 (vi-s(v)) + viN-1 s’(vi) = 0 for each vi • s’(vi) = (N-1)(1 – s(vi)/vi), s(0)=0 • Solution: s(vi)=(N-1)/N vi**1stpriceauction**• All playersbiddingaccording to s(v) = (N-1)/N v is a NE • Remarks • They are not truthful • The more they are, the highertheyshouldbid • Expected seller revenue • (N-1)/N E[vmax] = (N-1)/N N/(N+1) = (N-1)/(N+1) • Identical to 2ndpriceauction! • A general revenue equivalenceprinciple**Outline**• Preliminaries • Auctions • Matchingmarkets • Possible approaches to adspricing • Google mechanism • References • Easley, Kleinberg, "Networks, Crowds and Markets", ch.9,10,15**MatchingMarkets**goods buyers v11,v21, v31 1 1 v12, v22, v32 2 2 v12, v22, v32 3 3 vij: value thatbuyer j gives to good i How to match a set of differentgoods to a set of buyerswithdifferentevaluations**MatchingMarkets**p1=2 12, 4, 2 1 1 8, 7, 6 p2=1 2 2 7, 5, 2 p3=0 3 3 Whichgoodsbuyerslikemost? Preferred seller graph How to match a set of differentgoods to a set of buyerswithdifferentevaluations**MatchingMarkets**• Given the prices, look for a perfectmatching on the preferred seller graph • There is no suchmatching for this graph p1=2 12, 4, 2 1 1 8, 7, 6 p2=1 2 2 7, 5, 2 p3=0 3 3 Whichgoodsbuyerslikemost? Preferred seller graph**MatchingMarkets**• But withdifferentprices, thereis p1=3 12, 4, 2 1 1 8, 7, 6 p2=1 2 2 7, 5, 2 p3=0 3 3 Whichgoodsbuyerslikemost? Preferred seller graph**MatchingMarkets**• But withdifferentprices, thereis • Suchprices are market clearing prices p1=3 12, 4, 2 1 1 8, 7, 6 p2=1 2 2 7, 5, 2 p3=0 3 3 Whichgoodsbuyerslikemost? Preferred seller graph**Market Clearing Prices**• Theyalwaysexist • And canbeeasilycalculated if valuations are known • They are socially optimal in the sensethattheymaximize the sum of all the payoffs in the network (bothsellers are buyers)**Outline**• Preliminaries • Auctions • Matchingmarkets • Possible approaches to adspricing • Google mechanism • References • Easley, Kleinberg, "Networks, Crowds and Markets", ch.9,10,15**Adspricing**Ads positions companies r1 v1 1 1 v2 r2 2 2 v3 r3 3 3 vi: value thatcompany i gives to a click ri: click rate for an ad in position i (assumed to beindependent from the ad and known a priori) How to rankadsfromdifferentcompanies**Adspricing as a matchingmarket**Ads positions companies • Problem: Valuations are not known! • … but wecould look for something as 2ndpriceauctions r1 v1r1, v1r2, v1r3 1 1 v2r1, v2r2, v2r3 r2 2 2 v3r1, v3r2, v3r3 r3 3 3 vi: value thatcompany i gives to a click ri: click rate for an ad in position i (assumed to beindependent from the ad and known a priori)**The VCG mechanism**• The correct way to generalize 2ndpriceauctions to multiple goods • Vickrey-Clarke-Groves • Everybuyersshouldpay a priceequal to the social value loss for the othersbuyers • Example: consider a 2ndpriceauctionwith v1>v2>…vN • With 1 present the othersbuyersget 0 • Without 1, 2 would have got the good with a value v2 • then the social value loss for the othersis v2**The VCG mechanism**• The correct way to generalize 2ndpriceauctions to multiple goods • Vickrey-Clarke-Groves • Everybuyersshouldpay a priceequal to the social value loss for the othersbuyers • If VBSis the maximum total valuation over all the possible perfectmatchings of the set of sellers S and the set of buyers B, • If buyer j gets good i, he/sheshouldbecharged VB-jS- VB-jS-i**VCG example**Ads positions companies r1=10 v1=3 1 1 v2=2 r2=5 2 2 v3=1 r3=2 3 3 vi: value thatcompany i gives to a click ri: click rate for an ad in position i (assumed to beindependent from the ad and known a priori)**VCG example**Ads positions companies 30, 15, 6 1 1 20, 10, 4 2 2 10, 5, 2 3 3**VCG example**Ads positions companies • This is the maximum weightmatching • 1 gets 30, 2 gets 10 and 3 gets 2 30, 15, 6 1 1 20, 10, 4 2 2 10, 5, 2 3 3**VCG example**Ads positions companies • If 1 weren’tthere, 2 and 3 wouldget 25 instead of 12, • Then 1 shouldpay 13 30, 15, 6 1 1 20, 10, 4 2 2 10, 5, 2 3 3**VCG example**Ads positions companies • If 2 weren’tthere, 1 and 3 wouldget 35 instead of 32, • Then 2 shouldpay 3 30, 15, 6 1 1 20, 10, 4 2 2 10, 5, 2 3 3**VCG example**Ads positions companies • If 3 weren’tthere, nothingwould change for 1 and 2, • Then 3 shouldpay 0 30, 15, 6 1 1 20, 10, 4 2 2 10, 5, 2 3 3**The VCG mechanism**• Everybuyersshouldpay a priceequal to the social value loss for the othersbuyers • If VBSis the maximum total valuation over all the possible perfectmatchings of the set of sellers S and the set of buyers B, • If buyer j gets good i, he/sheshouldbecharged VB-jS- VB-jS-i • Under thispricemechanism, truth-tellingis a dominant strategy**Outline**• Preliminaries • Auctions • Matchingmarkets • Possible approaches to adspricing • Google mechanism • References • Easley, Kleinberg, "Networks, Crowds and Markets", ch.9,10,15**Google’s GSP auction**• Generalized Second Price • Once all the bids are collected b1>b2>…bN • Company i pays bi+1 • In the case of a single good (position), GSP isequivalent to a 2ndpriceauction, and also to VCG • But why Google wanted to implementsomethingdifferent???**GSP properties**• Truth-tellingmay not be an equilibrium**GSP example**Ads positions companies • If eachplayerbidsitstrueevaluation, 1 gets a payoffequal to 10 • If 1 bids 5, 1 gets a payoffequal to 24 r1=10 v1=7 1 1 v2=6 r2=4 2 2 v3=1 r3=0 3 3 vi: value thatcompany i gives to a click ri: click rate for an ad in position i (assumed to beindependent from the ad and known a priori)**GSP properties**• Truth-tellingmay not be an equilibrium • There isalwaysat least 1 NE maximizing total advertiservaluation**GSP example**Ads positions companies • Multiple NE • 1 bids 5, 2 bids 4 and 3 bids 2 • 1 bids3, 2 bids5 and 3 bids1 r1=10 v1=7 1 1 v2=6 r2=4 2 2 v3=1 r3=0 3 3 vi: value thatcompany i gives to a click ri: click rate for an ad in position i (assumed to beindependent from the ad and known a priori)**GSP properties**• Truth-tellingmay not be an equilibrium • There isalwaysat least 1 NE maximizing total advertiservaluation • Revenues canbehigher or lowerthan VCG • Attention: the revenue equivalenceprincipledoes not hold for auctionswith multiple goods! • Google wastargetinghigherrevenues… • … not clear if theydid the right choice.**GSP example**Ads positions companies • Multiple NE • 1 bids 5, 2 bids 4, 3 bids 2 google’s revenue=48 • 1 bids3, 2 bids5, 3 bids1 google’s revenue=34 • With VCG, google’s revenue=44 r1=10 v1=7 1 1 v2=6 r2=4 2 2 v3=1 r3=0 3 3**Other issues**• Click rates are unknown and depend on the ad! • Concreterisk: low-qualityadvertiserbiddinghighmayreduce the searchengine’s revenue • Google’s solution: introduce and ad-quality factor takingintoaccountactual click rate, relevance of the page and itsranking • Google isverysecretive about how to calculateit => the marketis more opaque • Complexqueries, nobodypaid for • Usuallyenginesextrapolatefromsimplerbids