Lessons From An Oops At Consumer Reports Consumer Follow Experts; Ignore Invalid Information

1. Uri Simonsohn The Wharton School Lessons From An Oops At Consumer ReportsConsumer Follow Experts; Ignore Invalid Information

2. The paper in one slide: • Jan 4th 2007: Consumer Reports on carseats • Jan 18th: Retraction • Unique opportunity: Do consumers continue using Jan 4th info? • Test on 6,000+ eBay auctions for carseats • Main finding: • Full return to baseline • My interpretation: voluntarily ignored info. • Alt explanations • Information ‘depreciates’ • Post-retraction buyers didn’t know • Kind-of alternative: Sellers’ behavior

3. Outline • Background • New information: release and retraction • Auction data • Main results • Alternative specifications • Conclusions

4. Can people voluntarily ignore information they possess? Existing evidence: Debriefing paradigm Hindsight bias Anchoring Mock juries and inadmissible evidence

5. Debriefing ParadigmRoss, Lepper & Colleageus (JPSP 1975;1980) • Critique of false feedback in Psych Paradigm: • Give false feedback on personality test • Debrief: “feedback was false” • Ask their beliefs • …still influenced by retracted feedback

6. Anchoring • Subjects asked to make numerical estimate • Length of Mississippi river • WTP for keyboard. • Asked first: is the amount larger or smaller than anchor. • Final estimate is correlated with anchor. • Even when anchor is roulette or SS#

7. OPIM 690 • Write down the last 2 digits of your SS#:__ • Would you be willing to pay that amount for yearly access to NYTimes.com? • What is the most you would pay? _____

8. Hindsight Bias • People told some outcome • Asked to estimate what those without information would predict. • Finding: estimates are biased towards the to-be-ignored outcome. • Next: results from Fischhoff (1975)

10. Mock Juries & inadmissible evidence • Dozens of studies • Random assignment across “jurors” • Control: baseline evidence • T1: control + extra evidence • T2: T1 + extra evidence is inadmissible. • Decisions by T2 fall between control and T1.

11. In sum: experimental evidence strongly predicts consumers won’t ignore retracted info.

12. Outline • Background • New information: release and retraction • Auction data • Main results • Alternative specifications • Conclusions

13. January 4th, 2007 • Corr( rank2007,rank 2005) = -.08

14. Retraction and empirical strategy • Jan 18th : Oops! • Outsourced, 30 vs 38 vs 70 MPH • Unique opportunity to study: • Causal effects of expert advice Contributions: • Individual level measures of WTP • Simple identification strategy (wrong info) Compared to • Discontinuities around discrete scores • Differences across sites • Timing 2) Ability of consumers to ignore retracted information.

15. How would people learn of a new Consumer Rerports carseat rating? Important because: • 1) Face validity of quick market reactions • 2) Post-retraction awareness.

16. From CR to consumers. • CR in print • Subscribers: slow • Library got it 01/11 • They claim: letter for retraction • Otherwise, not till May • Newstands: slower • No retraction till May • cr.org • Comscore 100k users • 15% of carseat buyers visit within 30 • 5% same day Not a direct source of info

17. How about the media? In short: CR unlikely to have direct influence.

18. Number of stories about“Consumer Reports” and “Carseats”sources: newsbank+lexisnexis 50+ Newspapers 600+ Stories

19. Internet coverage • Can’t do same search for web-coverage • Can use web.archive.org to check specific sites. • All major sites covered it

20. In Sum • CR info indirectly received via media • Fast  Retracted information remained available following retraction I’d argue: Post-retraction buyers probably read stories before being retracted.

21. Outline • Background • New information: release and retraction • Auction data • Main results • Alternative specifications • Conclusions

22. Why auctions • Retailers don’t change prices often • Few decision makers behind them • Auctions: • 1000s of DMs interacting • Prices change continuously • Aside: • Unexploited side to eBay data: pulse on demand shocks.

23. Auctions Data • 6 months: 3 before & 3 after • Many analyses focus on: • Before: 3 weeks • During: 2 weeks • After: 3 weeks • Auctions: 6k • Bids: 35k

24. Descriptive Stats:

25. Descriptive statistics

26. Annoyance: • Shipping is only observed for sold items. • Estimate OLS for sold items (w/shipping) • Estimate Tobit for all (w.o./shipping)

27. Outline or regression specifications • Y: (tot.pricei/Avg.Pricei,k) i:auction, k:carseat model • Time variables (dummies): • Primarily: before, during, after. • Also: biweekly dummies (next slide) • Also: 3-day-dummies • Key predictor • Primarily: ΔRanking • Also: carseat-model-dummies e.g. Y=OLS(during*ΔRanking , after* ΔRanking, controls)

28. First: bird’s eye view • Estimate Y=OLS(biweekly*ΔRanking) • 1 observation every 14 days. • Plot point estimates 3.98 SD

29. Next: more fine grained look Time: before, during, after

31. Plotting time*dranking betas

32. So far: just before-after • How quick are the reactions? • Y=OLS(3-day-dummies* Δranking)

33. price=OLS(3-day dummies * Δrank)omitted cat.: two previous weeks

34. How about non-winning bids? • Camerer et al (1989) “Curse of Knowledge” • Market forces reduce it • Rational agents trade more • Same here? • Are non winning bidders ‘cursed’? • Unit of observation: auction  bid • Quantile Regression

35. Specification • Bids are unit of observation. • If more than one bid by same bidder, take highest only. • Estimate quantile regressions of: • bid $ = f(Time*ΔRanking) • With quantiles at 20% ,40% ,60% ,80%.

37. Plotting the betas

38. Dividing point estimates by average bid % at quantile

39. Next: Beyond ΔRanking

40. From ΔRanking to model-dummies • Previous analyses: • Impose • Δ%price=b* ΔRanking • Don’t allow for heterogeneity in effect • Next: estimates by model. • Plot avg(OLS,Tobit)

43. Price=f(demand AND supply) Ehem.

44. NEXT: Plotting seller decisions over time

45. Starting Price Number of paid features

46. # of items for sale% New

47. Why would sellers not respond?

48. Summary of evidence • Biweekly: biggest price drop in 6 months • During vs. After: • Market responded to information • Ceased to once retracted • 3-day: • Market respond virtually immediately • Quantile regressions • Bidders across the full spectrum do so. • Carseat dummies • Every carseat (6/6) exhibits the pattern • Supply: • No evidence of changes in supply

49. Interpretation • Consumers successfully ignored information they possessed once it was retracted.

50. Alternative Explanations 1) Knowledge depreciates • …& coincides w/retraction • But: 3-day graphs 2) Buyers never knew • Retracted information still available online - Evenflo