1 / 21

The Interviewer Fallacy: Evidence from a Decade of MBA Interviews

This paper delves into the concept of the interviewer fallacy, the reluctance to create subsets of judgments that deviate too much from the expected distribution. Analyzing data from 10 years of MBA interviews, the study explores the impact of subsetting on ratings and examines alternative explanations like contrast effects and non-random sequencing. With a focus on average interview scores, the research challenges traditional beliefs and proposes new insights into the dynamics of interviewer judgments.

blondelle-kayson
Download Presentation

The Interviewer Fallacy: Evidence from a Decade of MBA Interviews

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Interviewer Fallacy:Evidence from 10 years of MBA interviews Photo not necessary Francesca Gino HBS Uri Simonsohn

  2. Motivation • How is a journal editor like a venture capitalist? • Continuous flow of judgments “random” “daily” subsets. • Research question: Impact of subsetting? Narrow bracketing +Belief in law of small numbers interviewer fallacy Definition. Reluctance to create subsets of judgments that differ too much from expected distribution.

  3. Paper in one slide • Data: 1-5 Rating of MBA interviewees • Handful per day. • corr[avg(so far), this interview]<0 • Ruled out alternatives: • Contrast effects • Non-random sequence

  4. Data Description • A business school gave us data • 10 years: N=9,323, k=31 ***INTERRUPT THIS TALK TO COMMENT ON ANOTHER PROJECT*** False-Positive (PsychScience2011): “list all your variables” Naysayers: “love to, have too many” Authors of False-Positive: “really?” Uri: “watch me.”

  5. Note: another 22 variables are listed in this page

  6. Other side of that single sheet of paper

  7. Note: The .pdf weighs 13Kb. The Wharton logo from slide 1: 11kb A hardliner may say: Only reason to choose not to post is to hide information from readers.

  8. Back to this talkData Description • A business school gave us data • 10 years: N=9,323, k=31* • Interviews per day M=4.5, SD=1.9 • Cluster SE [repeated measures] • Info on: • Applicant (e.g, GMAT scores, experience, race, gender) • Interviewer identity • Interview: time, date • Ratings (1-5 likert) • 5 subscores: communication, leader, etc. • Overall score (M=2.9, SD=0.9)

  9. Would like to analyze like gambler fallacy • HHHHpr(T)↑ • Problem • Non-binary data • Covariates • Different interviewers

  10. Instead: Scorek,i = OLS(average score so fari, covariates) k: Interviewee, 1 to N that day. i : Interviewer Prediction: <0

  11. Effect Size • Average interview 1 point higher, • Equivalent to losing: • 40 GMAT points, or • 30 months of experience.

  12. Alternative Explanations • Contrast effects • Non-random sequencing of interviews

  13. Contrast vs. Interviewer Fallacy Two divergent predictions: • Same effect on the interview subscores? Explanation Prediction Contrast: yes, and stronger Int.Fallacy: no, or at least weaker. Data: • Every one of five subscores:n.s. • Average a-la Robyn Dawes:n.s. • Biggest point estimate, ¼ as big • one is >0

  14. Contrast vs. Interviewer Fallacy Two divergent predictions: 2) Effect as end of day approaches. Explanation Prediction Contrast: weaker (arguably) Int.Fallacy: stronger (absolutely) Data: Estimate same regressions for: • last interview of day • 1 interview left • 2 interviews left

  15. Effect of previous interviews as day’s end approaches

  16. Alternative Explanations • Contrast effects • Non-random sequencing of interviews

  17. If better candidates follow bad ones or vice-versa  spurious finding. • Can we predict objective quality with average-interview-score-so-far? • Test: GMAT=OLS(avg.score) Job Experience = OLS(avg.score)

  18. Same table + 2 new columns

  19. Possible Mechanisms • Gambler fallacy + confirmation bias • Mental Accounting • Accountability

  20. A note on the internal validity of non-lab data • In the lab: hard to study interviewer fallacy • Participants could be learning about • Scale use • Distribution of underlying stimuli quality • Some psychological questions are better studied outside the lab. • This seems likes one of them.

More Related