Does Media Concentration Lead to Biased Coverage? Evidence from Movie Reviews

1. Does Media Concentration Lead to Biased Coverage? Evidence from Movie Reviews Stefano DellaVigna, UC Berkeley and NBER Alec Kennedy, San Francisco Fed Moscow Media Conference October 29, 2011 Preliminary, comments most welcome

2. Introduction • Dec. 13, 2007: News Corp. acquires from Bancroft family Dow Jones & Company  Wall Street Journal • Unlike Bancroft family, Murdoch's holdings include: • Cable channels (i.e., Fox Sports and Fox News) • Satellite television (Sky group) • Movie distributor (20th Century Fox) • … • Media conglomerate implies conflict of interest  Coverage of such businesses (or their competitors) in the Wall Street Journal may be biased.

3. Introduction • Wall Street Journal situation hardly unique • Media conglomerates: • Comcast owns NBC • Hearst Corporation owns ESPN and print outlets • Time Warner owns AOL and Time magazine •  Pervasive conflict of interest • But should media consolidation lead to distortion? • Cost: Loss of reputation if bias is revealed • Benefit: Can persuade audience (if audience naïve about bias) • Generally, persuasion rates of 5-10% from the media (DellaVigna-Gentzkow, ARE2010)

4. Introduction • This paper: Focus on two conglomerates • News Corp. • Time-Warner • Measure how media outlets in these groups review movies distributed by an affiliate • 20th Century Fox • Warner Bros. • Identification of bias transparent (diff-in-diff): • Compare review of 20th Century Fox movie by WSJ and NYT • Further control group: review of Paramount movie by WSJ and NYT

5. Introduction • Why movie reviews? • Frequent (500,000+ reviews in data set) • Easily quantifiable (coded on 0-100 scale by metacritic.com) • Large industry ($60bn annual revenue) • Some evidence that movie reviews influence movie attendance (Reinstein and Snyder (2005) benefits to the distributor from increased ticket sales

6. Data • First data set of reviews from metacritic.com, scored 0-100

7. Data • Second data set of reviews from rottentomatoes.com, scored on a 0-1 “freshness” scale

8. Data • Merged data set contains 548,764 reviews from 336 media from 1985 to July 2011

9. Data • Studios which distribute movies in the sample: Major, Independent, and Other

10. Average Bias: News Corp.

11. Average Bias: Time Warner

12. Average Bias: Statistical Test • Is the bias for News Corp. statistically significant? • Is it robust to introducing controls? • OLS regression • captures the effect of conflict of interest in Fox • captures the effect of conflict of interest in Time Warner • Sample of 473,727 reviews (qualitative reviews in RT have no 0-100 score) • Standard errors clustered at the movie level

13. Average Bias: Statistical Test

14. Average Bias: Statistical Test

15. Average Bias: Statistical Test • In favorite specification, conflict of interest increases rating for News Corp. by 2.6 points out of 100 • Estimate of bias increases with extra controls  Unobservablesunlikely to bias coefficient upward (Altonji, Elder, and Taber, 2005) • Magnitude of bias: • Equivalent to 1 extra star every ten reviews • Small but still economically significant impact • Reinsteinand Snyder (2005) estimate 25% higher revenue for two thumbs up by Roger Ebert • Are all media conglomerates the same? • No evidence of bias for Time Warner, can reject bias of 0.9 points

16. Average Bias: Robustness • Evidence of bias using RT 0-1 freshness indicator

17. Calculation of benefits • Back-of-the-envelope calculation of estimated benefits from distortion of movie review • Case of New York Post (NYP) • Suppose that NYP gives one extra star per Fox movie • 500,000 average readers • Persuades an extra 1% of readers to watch movie • Ticket sales increase by 5,000, or $8*5,000=$40,000 • Studio receives about half of increased sales, plus another half from higher rights from DVD licensing  $40,000 • About 400 20th Century Fox movies since 1985 •  Potential benefits to NewsCorp. from one extra star per Fox review by NYP over 25 years: $16,ooo,ooo

18. Average Bias: Explanations • Three main explanations for the results: • (E)Explicit editorial policy conveyed to journalists • (J) Bias by a journalist ultimately due to the conflict of interest, but lacking editorial pressure • (T) Correlation in taste between the media reviewer (or the media audience) and the affiliated studio • To separate explanations, we present evidence on: • Clustering of bias within a conglomerate (E and T, maybe J) • Editorial policies (E only) • Selective bias by type of movie (E or J, not T) • Omission of reviews (E or J)

19. Clustering of Bias: By Media • News Corp. media: Statistical evidence for NYP, 2-3 point bias for all 6 media • No evidence of bias for any of the Time Warner media

20. Clustering of Bias: By Journalist • Most media have a small number of journalists reviewing movies • Chicago Sun-Times, News of the World, Wall Street Journal and CNN.com have essentially only one

21. Clustering of Bias: By Journalist • News Corp: Statistical evidence of bias for 3 out of 4 NYP journalists, and one of TV Guide journalists • Time Warner: No evidence of positive bias • Significant Clustering: Pattern suggestive of editorial bias, but could also be correlated tastes, or similar journalists

22. Editorial Policies • Two tests of editorial policies • Personnel Policy: Change of reviewers at change of ownership – No evidence • Bias in Assignment: Assign affiliated movies to more generous reviewers • Estimate average reviewer generosity in score: • Reviewers differ significantly in their generosity • Are movies by affiliate studios more likely to be assigned to more generous reviewer? • No evidence– assignment quasi-random • Bias at Newscorp. is unlikely of editorial origin • Could be correlation in journalist bias or in tastes

23. Editorial Policies

24. Selective Bias • If bias is due to conflict of interest, should be optimal: • Bias when marginal return (persuasion) is highest • Maximize impact on revenue • Proposition 2. Small or no bias for very low quality movies • If bias is due to correlated tastes, no such prediction • Assumption for this test: • Movies with negative reviews by others are unlikely to benefit from a lone positive review • Movies with other positive reviews more likely to benefit from more positive review

25. Selective Bias: News Corp.

26. Selective Bias: News Corp.

27. Selective Bias: New York Post

28. Selective Bias: Time Warner

29. Selective Bias: Statistical Evidence • Evidence of selective bias for NYPost and qualitatively for WSJ – Bias due to conflict of interest • No evidence for other media

30. Bias by Omission • If bias is due to conflict of interest, and audience not fully rational, bias by omission: • Review high-quality affiliated movies • Omit review of low-quality affiliated movies • Do not expect this pattern if correlated tastes • Rare setting to separate bias by omission and by commission • Outlets differ substantially in probability of review  Use matching procedure • For each media, find ten matching media in terms of average probability of review of a movie • Plot local polynomial regression of dummy for review on average review score

31. Bias by Omission: New York Post

32. Bias by Omission: Time

33. Bias by Omission: Entertainment Weekly

34. Bias by Omission: Statistical Test • Newscorp.: No systematic evidence of omission bias • Time Warner: Evidence for 2 outlets • Bias by omission and by commission substitutes, not complement

35. Bias in Movie Aggregator • So far focus on most obvious conflict, for reviewers • Conflict of interest hardly stops there: Rottentomatoes.com, also independent when launched in 1998, was acquired by IGN Entertainment in June 2004, and IGN was purchased by News Corp. in September 2005. IGN, and hence RottenTomatoes, was then sold in January of 2010 by Newscorp. • Conflict of interest for RT: more positive reviews of the 20th Century Fox movies in 2006-2009 • Control for rating of same review in MC

36. Bias in Movie Aggregator • Remarkably, no evidence of bias, even for qualitative reviews, where bias is easier to hide • Can reject small bias

37. Bias in Movie Aggregator • Local polynomial regression of ‘fresh’ indicator on average movie score – no evidence of bias

38. Bias in Movie Aggregator • Event study comparing residual freshness (after controlling for score) for FOX and non-FOX movies

39. Summary of Results • We documented the extent of bias due to conflict of interest for two media conglomerates • Average bias: • 2.6 points bias out of 100 for Newscorp. outlets • No bias for Time Warner outlets, can reject even small bias • Bias is clustered within a media conglomerate • No evidence of editorial policy to assign movies • Selective Bias: Evidence for New York Post, not for other media • Omission Bias: Evidence for two Time Warner outlets • Interpretation: • Best fits with bias due to conflict of interest for journalists, with clustering of such bias

40. Conclusion • Overall, remarkably little evidence of distortion from conflict of interest: • No distortion in review for Time Warner • No distortion for Rottentomatoes • No distortion in editorial assignment • However, bias still does occur: • Small, but significant, bias for Newscorp. Outlets • Some omission bias for Time Warner outlets • Suggests that transparency and emphasis on reputation (for example because of competition) critical to keep media honest • Paper allowed us to decompose potential media bias in novel ways • Relates to Conflict of interest in other settings: • Significantly less distortion than for analysts • Less distortion than for advertising

42. Average Bias

43. Average Bias

44. Selective Bias II • Different types of movies can have different returns to positive reviews • Snyder and Reinstein: Larger effect of movie reviews for independent movies • However: Independent movies also have smaller revenue, so bias may be less worthwhile • Examine the effect of conflict of interest separately

45. Selective Bias: Indy movies • Significant bias for News Corp. only for mainstream movies