1 / 45

Experiments for Market Design

Experiments for Market Design. Al Roth Market Design. Multiple roles for experiments in practical market design. In diagnosing and understanding market failures, and successes In designing new markets In communicating results to policy makers.

lilika
Download Presentation

Experiments for Market Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Experiments for Market Design Al Roth Market Design

  2. Multiple roles for experiments in practical market design • In diagnosing and understanding market failures, and successes • In designing new markets • In communicating results to policy makers

  3. Handbook of Experimental Economics V.2, John Kagel and Alvin Roth, eds., in preparation • Market Design • Early Plott/Smith e.g. posted price versus double auction • public goods experiments • emissions trading, • auctions: FCC, eBay,reputation markets, procurement, combinatorial • Contract design • Matching markets: doctors, lawyers, schools, kidneys

  4. Lessons from market failures and successes • To achieve efficient outcomes, marketplaces need make markets sufficiently • Thick • Enough potential transactions available at one time • Uncongested • Enough time for offers to be made, accepted, rejected… • Incentive compatible • Safe to reveal relevant preferences • Some kinds of transactions are repugnant… • This can be an important constraint on market design • Experiments play a critical role in learning and communicating design lessons

  5. What makes a clearinghouse successful or unsuccessful? • A matching is “stable” if there aren’t a doctor and residency program, not matched to each other, who would both prefer to be. • Hypothesis: successful clearinghouses produce stable matchings. • How to test this?

  6. Market Stable Still in use (halted unraveling) • NRMP yes yes (new design in ’98) • Edinburgh ('69) yes yes • Cardiffyes yes • Birmingham no no • Edinburgh ('67) no no • Newcastleno no • Sheffield no no • Cambridge no yes • London Hospital no yes • Medical Specialties yes yes (~30 markets, 1 failure) • Canadian Lawyers yes yes (Alberta, no BC, Ontario) • Dental Residencies yes yes (5 ) (no 2) • Osteopaths (< '94) no no • Osteopaths (> '94) yes yes • Pharmacists yes yes • Reform rabbis yes (first used in ‘97-98) yes • Clinical psych yes (first used in ‘99) yes So stability looks like an important feature of a centralized labor market clearinghouse.

  7. The need for experiments • How to know if the difference between stable and unstable matching mechanisms is the key to success? • There are other differences between e.g. Edinburgh and Newcastle • The policy question is whether the new clearinghouse needs to produce stable matchings (along with all the other things it needs to do like handle couples, etc. ) • E.g. rural hospital question…

  8. A matching experiment(Kagel and Roth, QJE 2000) • 6 firms, 6 workers (half "High productivity" half "low productivity") • It is worth $15 plus or minus at most 1 to match to a high • It is worth $5 plus or minus at most 1 to match to a low • There are three periods in which matches can be made:-2, -1, 0. • Your payoff is the value of your match, minus $2 if made in • period -2, minus $1 if made in period -1 • Decentralized match technology: firms may make one offer at any period if they are not already matched. Workers may accept at most one offer. Each participant learns only of his own offers and responses until the end of period 0. • After experiencing ten decentralized games, a centralized matching technology was introduced for period 0 (periods -2 and -1 were organized as before). • Centralized matching technology: participants who are still unmatched at period 0 submit rank order preference lists, and are matched by a centralized matching algorithm. • Experimental variable: Newcastle (unstable) or Edinburgh (stable) algorithm.

  9. Offers and acceptances

  10. Market Stable Still in use (halted unraveling) • NRMP yes yes (new design in ’98) • Edinburgh ('69) yes yes • Cardiffyes yes • Birmingham no no • Edinburgh ('67) no no • Newcastleno no • Sheffield no no • Cambridge no yes • London Hospital no yes • Medical Specialties yes yes (~30 markets, 1 failure) • Canadian Lawyers yes yes (Alberta, no BC, Ontario) • Dental Residencies yes yes (5 ) (no 2) • Osteopaths (< '94) no no • Osteopaths (> '94) yes yes • Pharmacists yes yes • Reform rabbis yes (first used in ‘97-98) yes • Clinical psych yes (first used in ‘99) yes • Lab experiments yes yes. (Kagel&Roth QJE 2000) no no Lab experiments fit nicely on the list, just more of a variety of observations that increase our confidence in the robustness of our conclusions, the lab observations are the smallest but most controlled of the markets on the list…

  11. Abdominal Transplant Surgery (2005) Child & Adolescent Psychiatry (1995) Colon & Rectal Surgery (1984) Combined Musculoskeletal Matching Program (CMMP) Hand Surgery (1990) Medical Specialties Matching Program (MSMP) Cardiovascular Disease (1986) Gastroenterology (1986-1999; rejoined in 2006) Hematology (2006) Hematology/Oncology (2006) Infectious Disease (1986-1990; rejoined in 1994) Oncology (2006) Pulmonary and Critical Medicine (1986) Rheumatology (2005) Minimally Invasive and Gastrointestinal Surgery (2003) Obstetrics/Gynecology Reproductive Endocrinology (1991) Gynecologic Oncology (1993) Maternal-Fetal Medicine (1994) Female Pelvic Medicine & Reconstructive Surgery (2001) Ophthalmic Plastic & Reconstructive Surgery (1991) Pediatric Cardiology (1999) Pediatric Critical Care Medicine (2000) Pediatric Emergency Medicine (1994) Pediatric Hematology/Oncology (2001) Pediatric Rheumatology (2004) Pediatric Surgery (1992) Primary Care Sports Medicine (1994) Radiology Interventional Radiology (2002) Neuroradiology (2001) Pediatric Radiology (2003) Surgical Critical Care (2004) Thoracic Surgery (1988) Vascular Surgery (1988) NRMP also runs Fellowship matches through the Specialties Matching Service (SMS) (using the Roth-Peranson algorithm since 1998)

  12. The market for gastroenterologists Why did the market for gastroenterologists collapse? With what consequences? How to get it back on its feet? Science/persuasion/politics… Experiments play a role in each of these, and complement other kinds of economic investigation (in this case theory, surveys, analysis of historical data...)

  13. Niederle, Muriel and Alvin E. Roth, “Unraveling reduces mobility in a labor market: Gastroenterology with and without a centralized match,” Journal of Political Economy, vol. 111, no. 6, December 2003, 1342-1352. Niederle, Muriel and Alvin E. Roth, “Relationship Between Wages and Presence of a Match in Medical Fellowships,” JAMA. Journal of the American Medical Association, vol. 290, No. 9, September 3, 2003, 1153-1154. Niederle, Muriel and Alvin E. Roth, “The Gastroenterology Fellowship Match: How it failed, and why it could succeed once again,” Gastroenterology, 127, 2 August 2004, 658-666. C. Nicholas McKinney, Muriel Niederle, and Alvin E. Roth, “The collapse of a medical labor clearinghouse (and why such failures are rare),” American Economic Review, 95, 3, June, 2005,878-889. Niederle, Muriel and Alvin E. Roth, “The Gastroenterology Fellowship Market: Should there be a Match?,” American Economic Review, Papers and Proceedings, 95,2, May, 2005, 372-375. Niederle, Muriel, and Alvin E. Roth, “Re-starting the Gastroenterology Match,” letter, American Journal of Gastroenterology, vol. 100, no. 5, May, 2005, 1202-1203. Niederle, Muriel, Deborah D. Proctor, and Alvin E. Roth, “What will be needed for the new GI fellowship match to succeed?” Gastroenterology, January, 2006, 130, 218-224. Niederle, Muriel and Alvin E. Roth, “Making Markets Thick: Designing Rules for Offers and Acceptances,” working paper.

  14. The demise of the Match Within 4 years the Match collapsed. After 1997 the match was effectively abandoned, and in 2000 it was formally abandoned

  15. What have been the consequences of the demise of the match? • The market has become • Less thick in time and space • Earlier and more dispersed in time • Less mobility • No changes in wages • Each of these things can be observed in the field data…

  16. Why did the Gastro Match break down?And why are such failures so rare? Some Hypotheses • A centralized Match only works when there are more applicants than positions. • There may not be enough “high quality” applicants to fill high quality positions. • The match failed because of the shock that reduced the demand of positions below the supply. • The match failed because, when this shock occurred, fellowship programs were aware of it, but applicants were not. • Programs could update their priors when they saw how many applications they received.

  17. How to sort among these hypotheses? • Each of them is consistent with the historical data. • And, since stable match failures are rare, there isn’t a good possibility of a cross-market comparison. • But in the laboratory, we can shock a market in different ways, and try to make a stable matching mechanism fail.

  18. A simple experimental environment • 2 types of firms and workers; "High" and "low” productivity • Matching to a High type is worth 150 points + private value: in [-10,10]). • To a low type is worth 50 (+ private value) • There are three periods in which to match: -2, -1, 0. • Your payoff is the value of your match, minus 20 points if made in period -2, minus 10 points if made in period -1 • Decentralized match technology: firms may make one offer at any period if they are not already matched. Workers may accept at most one offer. Each participant learns only of his own offers and responses until the end of period 0. (no period 0 eq.) • After experiencing ten decentralized games, a centralized matching technology was introduced for period 0 (periods -2 and -1 were organized as before). • Centralized matching technology: participants who are still unmatched at period 0 submit rank order preference lists, and are matched by a centralized (stable) matching algorithm.

  19. Experimental conditions • Different supply and demand (between subjects variable) • Shocks that change the long side of the market (within subject variable) • Decentralized and Centralized matching (within subject) • Different information conditions (symmetric or asymmetric between firms and workers) (between subject variable)

  20. Experimental Results • Simple demand/supply imbalances have no effect. • Shocks disturb the match, especially when applicants are not aware of them. • Gradual adoption of the match…

  21. Full Information

  22. After the shock firms start making more early offers to workers. Workers: • Before the shock: high type workers eagerly accept early (-1) high offers. • After the shock: firms are in excess supply, workers have no big incentive to accept early offers. Workers only know this when they are informed of the shock, and indeed in that case accept early offers with much lower propensity. The ability of applicants to reject early offers that prevents unraveling. No other specialty experienced such shocks: Cardiovascular: from 1990 to 1998 the ratio of applicants to positions offered varied from a high of 1.6 to a low of 1.3. Pulmonary disease those ratios varied from a high of 1.5 to a low of 1.1, For Infectious disease (from 1994 to 1998) those ratios vary from a low of .68 to a high of .92.

  23. Gastro market—preliminary advice to the Gastroenterologists • It appears that the collapse of the match was due to a shock, and the match could succeed if re-started. • This would increase the mobility of GI fellows. • Not all GI programs would do better: some small programs enjoy having a captive market (particularly if they are at a hospital with a strong internal medicine residency program)

  24. Transition to a later market • In May 2005, the American Gastroenterological Association (AGA), the American College of Gastroenterology (ACG), the American Society for Gastrointestinal Endoscopy (ASGE) and the American Association for the Study of Liver Diseases (AASLD) decided to reintroduce a GI fellowship match, starting in 2006, for positions beginning in July 2007. • How to manage the transition? • Rates of participation • Concern among programs about whether their chief competitors will participate. • Change of dates to June 2006 (from as early as July 2005) • How to reassure programs that other programs will wait for Match?

  25. In June 2005, our colleague Debbie Proctor, the gastroenterologist who took the lead in reorganizing the match, sent us an email saying, in part • “I’m answering 3-4 emails per day especially on this issue. ‘I want to make sure MY competition is in the match and that they don’t cheat.’ Well, this is another way of saying that if they cheat, then I will too!...Have you ever seen this before? The distrust amongst program directors? I find it hard to believe that we are unique. Maybe this is [a] social science phenomenon?”

  26. Managing the decentralized market prior to a match(Niederle and Roth 2007) • Different markets have different cultures: e.g. rules, norms, and expectations about how and when offers will be made, accepted, rejected. • In some markets, an exploding offer is easy to enforce and rely on, in other markets, making such offers would be frowned on, and other market participants wouldn’t help in enforcing the resulting contract. • Many markets have organizations that try to change and influence these norms in order to promote more orderly markets and control the timing of the markets.

  27. Organizations concerned with how and when offers are made, accepted, rejected • Council of Graduate Schools (CGS): graduate admissions, • National Association for College Admission Counseling (NCAC): undergraduate admissions, • National Resident Matching Program (NRMP): entry level medical residencies, (also Canadian Resident Matching Service – CaRMS – and various regional matches in Britain) • Specialty Matching Services (SMS): advanced medical residencies and fellowships, • Association of Psychology Postdoctoral and Internship Centers (APPIC): clinical psychology positions, • National Association for Law Placement (NALP) for positions in law firms, • Judicial Conference of the United States and various ad hoc committees of judges for federal judicial clerkships, • Provincial Law Societies in Canada. • National Association of Colleges and Employers (NACE) for US college undergraduates, • NCAA: formerly for postseason college football bowls, now regulated by the Bowl Championship series (BCS), • National Panhellenic Conference for sorority matching • The Japan Federation of Employers’ Associations (Nikkeiren) for japanese university graduates • National Collegiate Athletic Association (NCAA) for recruitment of college athletes, and various drafts…

  28. Which of these markets are unraveled? • Of these markets, it appears that markets in which transactions are made at early, uncoordinated times are markets in which there are both • Exploding offers • Binding commitments • Example: Graduate admissions has a nicely designed set of market rules, sent to all admitted students, that suppresses exploding offers (in part by facilitating “reneges” of premature acceptances).

  29. “Students are under no obligation to respond to offers of financial support prior to April 15; earlier deadlines for acceptance of such offers violate the intent of this Resolution. In those instances in which a student accepts an offer before April 15, and subsequently desires to withdraw that acceptance, the student may submit in writing a resignation of the appointment at any time through April 15.

  30. The need for experiments • Note that a simple experimental environment will be quite different from the markets in Table 1, and from the gastroenterology market. • The laboratory environment, because it is so simple, is different from each of these markets in more transparent ways than they are different from one another. • while it is always somewhat risky to draw inferences about the effect of a rule change in one market from the effects in a different market, the inferences may be clearer when one of the markets is simple. • And in the experiment, the rules are an exogenous experimental variable, so that their influence can be readily observed.

  31. An experiment allows us to view different offer regimes in a controlled environment • 5 firms, 6 applicants, 9 periods. • In each period, a firm may make an offer to at most one applicant. Firms make offers, applicants decide upon the offers they receive. • Firms and applicants are assigned “qualities.” • If firm of quality x hires an applicant of quality y, both firm and applicant will receive a payoff of xy points each. • Firms’ qualities are simply their assigned participant number, 1,2,3,4,5. • Uncertainty about applicants’ quality is only resolved over time: • In periods 1, 4 and 7, each applicant receives a “signal,” an integer between 1 and 10, each equally likely. • In period 7, the relative ranking of the sum of the 3 signals determines the applicants’ quality. The applicant with the highest sum of 3 signals has a quality of 6… Ties are broken randomly. • Firms see all signals, applicants see only their own signals (as they become available over time) and their ranking in period 7. (Note that in this experiment the cost of early matching is bad matches due to uncertainty about quality.)

  32. Experimental treatments: 3 “market cultures” Exploding offers: Firms can make exploding offers and acceptances are binding. Renege: Firms can make exploding offers, but applicants can renege on their acceptance, for a small fee (1 point). Open offers: Firms can only make open offers. (Many equilibria: One in which all matches are agreed upon inefficiently early. All environments have a perfect equilibrium with efficient late matching. But the late matching equilibrium is more fragile when offers are exploding and acceptances are binding.)

  33. Timing of final offers (offers that were accepted and not reneged upon for the renege treatment) in terms of the number of signals that were observed.

  34. In last five periods…

  35. Not everyone loses…(so it can be hard to get consensus on reversing unraveling:)

  36. So…How should Gastro transition from early to late? • AGA Executive Committee • Adopted a resolution on the conduct of the market prior to the match, regarding early offers. • Published in Jan 2006 (Niederle, Proctor, Roth)

  37. Resolution Regarding Gastroenterology Fellowship Applicants, Including all Research and Clinical Fellowship Applicants and Positions This resolution concerns the conditions surrounding gastroenterology fellowship offers to applicants, acceptance by applicants of such offers, and participation by applicants and programs in the gastroenterology fellowship Match. The general spirit of this resolution is that each applicant should have an opportunity to consider all programs before making a decision and be able to participate in the Match. … The intention of this resolution is to ensure uniformity so that everyone participates fairly and to establish the principle that all positions should be filled through the Match or after Match Day. It therefore seeks to create rules that give both programs and applicants the confidence that applicants and positions will remain available to be filled through the Match and not withdrawn in advance of it. This resolution addresses the issue that some applicants may be persuaded or coerced to make commitments prior to, or outside of, the Match. Early offers and acceptances, and offers outside of the Match, are violations of the rules and of this resolution and are not condoned. Any applicant may participate in the matching process by registering for the Match to interview and consider match-participating programs; however, an applicant who accepts a position prior to, or outside of, the Match must comply with the National Resident Matching Program/Specialty Matching Services (NRMP/SMS) Match Participation Agreement by either resigning the accepted position if he/she wishes to submit a rank order list of programs or by withdrawing from the Match prior to the rank order list certification deadline, which is the first week in June. In addition, no program may withdraw a position from the Match after the quota change deadline to offer that position outside the matching process. … The spirit of this resolution is to make it unprofitable for program directors to press applicants to accept early offers, and to give applicants an opportunity to consider all offers as well as to provide uniform and widely acceptable rules that protect both applicants and fellowship programs.

  38. The gastroenterology match for 2007 fellows was held June 21, 2006, and succeeded in attracting 147 of the 151 eligible fellowship programs, 13 of which withdrew before the match. The final participation rate: 89% 98% of the positions offered in the match were filled through the match. Early movers couldn’t impose a big negative externality on those who waited for the match, since pre-match exploding offers would not necessarily remove candidates from the market. This made it easier for everyone to wait for the match. The second year of the centralized match was held in June 2007…very similar participation rates.

  39. Orthopedic surgery has the same problem • But aspiring surgeons may not be able to turn down early offers after accepting them, even if the orthopedic organizations were to adopt a policy like the gastroenterologists'. • However, while the (15) orthopedic professional organizations also cannot directly prevent employers from making early offers, unlike the gastroenterologists, they feel they can effectively punish employers who make early offers, by not allowing them to present papers at professional meetings. • So it looks like the orthopedic organizations are going to try to adopt a clearinghouse by instituting a series of penalties for employers who don't participate according to the rules.

  40. Federal judges hiring law clerks have a similar problem • This market has also unraveled. • Like orthopedic surgeons, law students can't change their minds no matter how early the offer (law students are not in a position to break promises to federal judges). • Like the gastroenterology organizations, the judicial conferences have no way to prevent judges from hiring early, or from making exploding offers, or punishing those who do. • There are now discussions underway among judges and law schools about the possibility of instituting a clearinghouse like the ones used by doctors. • But until some way is found to address the issue of early and exploding offers before a clearinghouse, the chances of success are small. • It appears that judges may have to become willing to do some "community enforcement" of norms against early exploding offers before a clearinghouse will work for them.

  41. Exploding offers are a particular problem for judges and law clerks: • Avery, Christopher, Christine Jolls, Richard A. Posner, and Alvin E. Roth, “The New Market for Federal Judicial Law Clerks”, University of Chicago Law Review, 74, Spring 2007, 447-486. Representative quotes: • “I received the offer via voicemail while I was in flight to my second interview. The judge actually left three messages. First, to make the offer. Second, to tell me that I should respond soon. Third, to rescind the offer. It was a 35 minute flight.” • “I had 10 minutes to accept.” • “I asked for an hour to consider the offer. The judge agreed; however thirty minutes later [the judge] called back and informed me that [the judge] wanted to rescind my offer.”

  42. Clinical Neuropsychology • Presently unraveling in front of a match • We proposed a policy like Gastro, to allow students to change their minds about an early, pre-match acceptance. • The president of the association replied as follows: “I have said it once, and I will say it again: Two wrongs do not make a right. To state it another way: The end does not justify the means. I will be strongly opposed to any attempt at an APPCN policy that allows candidates to accept an offer outside of the match, participate in the match anyway, and then renege on their earlier "acceptance".” • We’ll talk more about repugnance as a constraint on markets when we talk about kidneys…

  43. Conclusions: • Markets don’t always spring up like weeds, some of them are hothouse orchids that need care and attention.

  44. Experiments have multiple roles to play • They allow us to investigate questions that the field data cannot answer • E.g. why did the GI match fail in ’96? • They allow us to investigate hypotheses suggested by the field data • E.g. is stability the key to understanding the successful and failed British medical matches? • E.g. does the ability to renege on early acceptances reduce exploding offers and unraveling? • They play a role in the considerable amounts of discovery, demonstration, and persuasion that are necessary to coordinate market participants to move from one equilibrium to a better one.

More Related