Identification and Analysis of Extremes: Economic Forecasts and CEO Compensation

1. Identification and Analysis of Extremes with Two Examples:Executives’ Economic Forecasts after 9/11; CEO Compensation Ehsan S. Soofi, Paul C. Nystrom Sheldon B. Lubar School of Business University of Wisconsin-Milwaukee & Masoud Yasai-Ardekani School of Management George Mason University

2. Preamble: Extreme? • What is meant by extreme? • Semantically, extremes refer to cases farthest from the center.  • Technically, extremes are plausible outcomes of the probability distributions of the farthest statistics from the sample center.  

3. Preamble: Concepts of extreme & rare • Extremes(main focus of our study) • Farthest from the center of data • Rare events • Far from the mass of population (Heavy tail) • Relate to our study if have extreme impacts • Dot plots of simulated samples (n=100) Extreme Rare Extreme classification error rate, a • Horizontal scales are different

4. Preamble: Variations of sample central values and extremes • Example: Uniform distribution P(an X in an interval) = Area under the density Sample: n=15 Extreme Min Extreme Min Random order X1,X2, …X15 Random order X1,X2, …X15 Order statistics Y1<Y2< …<Y8<…<Y15 Order statistics Y1<Y2< …<Y8<…<Y15 Center Mean Center Mean Center Median Center Median Extreme Max Extreme Max P=.2 P=.2 P=.2 Mean .55 Mean .53

5. Outline • Why study extremes?  • Distribution theory • Thresholds for extremes • Example: survey of executives after 9/11 • Bayesian inference, small number of observations • Example: Components of CEO compensation • Thresholds for low and high percentiles • Deciles, quartiles, thirds • Example: Edmondson et al (2001)

6. Why study extremes?

7. Learning • What has been learned? • From the best ones • Firm performance (Galunic & Eisenhardt 1996) • Product development cycle (Clark & Fujimoto 1991) • Inventory control (Wal-Mart) • Rapid organization crises and turnarounds (Hedberg, Nystrom & Starbuck 1976, Nystrom & Starbuck 1984) • From the worst ones • Decision errors (Starbuck & Milliken 1988) • What more can be learned? • Comparing relationship between a variable with • Extremes and non-extremes on another variable • Compare relationship between a pair of variables • For extremes of the two variables • Overall relationship

8. Distribution theory

9. Order statistics • Standard assumption for statistical analysis of a sample of measurements x1 , x2 , …, xn • Are observations generated on random variables X1 ,…,Xn • Independent and have identical probability distribution FX(x) = Pr(Xi< x), for all i=1,…,n • Density functionfX(x)=derivative ofFX(x) • The order statistics of the sample • The sampled measurements arrayed in ascending order y1< y2< … < yn • Are observations from the set of random variables Y1< Y2< … < Yn

10. Distributions of order statistics • The probability distribution of Yi Gi(yi)= Pr(Yi< yi) • The probability density function is • A function of n, the order index i, and FX • FX is referred to as the parent distribution • gi(yi)are notidentical • Y1< Y2< … < Yn are not independent

11. Distributions of extremes • Minimum • Distribution function G1(y1) = Pr(Y1< y1)=1-[1- FX(y1)]n • Density function g1(y1) = [1- FX(y1)]n-1 fX(y1) •  Maximum • Distribution function Gn(yn) = Pr(Yn< yn)=[FX(yn)]n • Density function gn(yn) = [FX(yn)]n-1 fX(yn)

12. Density functions of extremes of normal distribution • Normal (Mean=0, Standard deviation=1) • Parent is symmetric, extremes are mirror image • Three distributions are more separated for the larger n

13. Density functions of extremes of Weibull distribution • Weibull (Shape=2, Scale=1) • Parent is not symmetric, extremes are not mirror image • Three distributions are more separated for the larger n

14. Extreme value distribution • As the sample size increases, distribution of maximum, suitably standardized, approaches an extreme value distribution having a parameter, referred to as the extreme value index (EVI) • Three domains of attraction (set of distributions) • EVI>0 • EVI<0 • EVI=0 Includes Weibull

15. Discrepancy between distributions • As n increases, the distributions of extremes separate more from • the parent distribution • the median at a faster rate • each other at a much faster rate • The discrepancies do not depend on the parent distribution Extreme-Parent Extreme-median Extreme-Extreme

16. Dependence of extremes • As the sample size increases, dependence between • each extreme and its neighbor increases • the two extremes decreases • The dependence does not depend on the parent distribution

17. How to identify extremes?

18. Thresholds for extremes • Definition: A measurement xi is an extreme if we can infer with high probability that it is a sample from the distribution of an extreme • Inference with probability .95 • 95% threshold for minimum is the solution to Pr(Y1< ymin)= G1(ymin) = 1-[1- FX(ymin)]n=.95 • 95% threshold for maximum is the solution to Pr(Yn > ymax)=1- Pr(Yn< ymax)=1- Gn(ymax) = 1-[FX(yn)]n=.95

19. Thresholds for extremes of normal distribution • Normal (Mean=0, SD=1), n= 93 • Pr(Y1< -1.89) = .95 • Threshold for minimum is -1.89, any observation less than 1.89 is an extreme • Pr(Yn > 1.89) = .95 • Threshold for maximum is 1.89, any observation greater than 1.89 is an extreme

20. Thresholds for extremes of Weibull distribution • Weibull (Shape=0, Scale=1), n= 100 • Pr(Y1< .17) = .95 • Threshold for minimum is .17, any observation less than .17 is an extreme • Pr(Yn > 1.88) = .95 • Threshold for maximum is 1.86, any observation greater than 1.86 is an extreme

21. Thresholds for extremes of several distributions

22. Remarks on thresholds • Monotone transformation of variables • Threshold for log-normal is log of thresholds for normal • Threshold for Weibull is power of threshold for exponential • Thresholds for all distributions can be obtained by thresholds for uniform data • Probability integral transformation • Thresholds are functions of the parameters of the parent distribution FX(x;q) • In data analysis, the parameters must be estimated • Maximum likelihood method • Bayesian method

23. Survey of ExecutivesSoon After 9/11

24. Variables chosen for studying extremes • Executives’ economic forecast accuracy • Executives’ perceived environmental uncertainty

25. Forecast accuracy • A paragraph described the U.S. economy's GDP (Gross Domestic Product) and its rates of change in recent years and quarters. • Likelihoods for three outcomes of the future economy (GDP growth by end of September 2002): • Full recovery (GDP>2%) • Modest recovery (0% < GDP<2%) • Further decline (GDP<0%) • Best estimates of GDP growth choosing from • GDP > 2% • 1% < GDP < 2% • 0% < GDP < 1% • -1% < GDP< 0% • -2% < GDP < -1% • GDP < -2%

26. Forecast distribution • Constructed piece-wise uniform distribution for the economic forecast (W) of each executive • Actual GDP growth by the end of September 2002 was 3.4% • For 87 of 93 respondents, the best estimate was assigned the highest probability (modal category) Most accurate Least accurate More accurate Less accurate

27. Forecast accuracy • The mean squared error (MSE) of each forecast distribution MSE(Wi)= E[(Wi - 3.4)2] =Variance(Wi) + [mean(Wi)– 3.4]2 • Quadratic loss function • Measures forecast inaccuracy • Our results are robust against variants of distribution for W and the loss function

28. Environmental uncertainty Probability P(S) • Type of uncertainty (Milliken 1987) • State (S): Outcome of the future economy • Effect (E): Impacts of each Son the organization • Highly positive • Positive • Neutral • Negative • Highly negative • Multiplicative rule of probability • Concept: Uniformity of the joint distribution (Argote 1982) • Measure: Shannon entropy (Leblebici and Salancik 1981) Conditional probability P(E | S) Bivariate distribution P(S & E) = P(E|S)P(S)

29. Distributions of the variables across the sample • Forecast inaccuracy • Weibull (shape=3.93, scale 10.14, location=-2.47) • Environmental uncertainty: Normal (mean 1.43, SD=0.54) • Probability plots and fit statistic (Anderson-Darling) AD=0.493 P=0.157 AD=0.466 P=0.247

30. Thresholds for extremes • Example: n= 93 (9/11 sample size) • Forecast inaccuracy: Weibull (shape=3.93, scale 10.14, location=-2.47) • Extreme high

31. Thresholds for extremes • n= 93 (9/11 sample size) • Maximum likelihood method

32. Forecast accuracy and strategic type • Strategic types (Miles and Snow 1978) • Prospector • Analyzer • Defender • Reactor • Each executive in the 9/11 study read four paragraphs that characterize four different strategic approaches • None of the paragraphs bore the type label • Each executive selected the paragraph that best described his/her organization

33. Hypotheses and data • Two hypotheses of interest: • H1: The probability of extreme accurate forecasting is higher for Prospector/Analyzer than for Defender/Reactor. • H2: The probability of extreme inaccurate forecasting is lower for Prospector/Analyzer than for Defender/Reactor. • Data: Cross-tabulation

34. Forecast accuracy: Bayesian analysis • Prior and posterior distributions for probability of being extremely accurate and extremely inaccurate (for each strategy type separately) • Prior: Uniform distribution (Bayes-Laplace) • Posterior Beta distributions for prospector • Posterior Beta distributions for reactors

35. Forecast accuracy: Bayesian analysis • Difference between the posterior distributions for the two strategy types • No analytical solution • Simulation results H1 holds H2 holds H2 Doesn’t hold H1 Doesn’t hold

36. Forecast accuracy: Bayesian analysis • Prior and posterior distributions for probability of being extremely accurate and extremely inaccurate (for each strategy type separately) • Prior: A highly skewed Beta distribution • Posterior Beta distributions for prospector/analyzer • Posterior Beta distributions for reactors/defender

37. Forecast accuracy: Bayesian analysis • Difference between the posterior distributions for the two strategy types • Results are robust H1 Doesn’t hold H2 Doesn’t hold

38. Forecast accuracy: Bayesian analysis • Prior probability of hypotheses: P(H true)=P(H false) • Posterior probability and posterior odds of hypothesis

39. Forecast accuracy & Environmentaluncertainty • Traditional: correlation and a regression analysis • Line of averages • Below on both variables • Above on both variables • Relationship between extremes on the two variables? Regression assumptions hold Thresholds for low Thresholds for high Positive association, mostly purple Regression to the mean

40. Bayesian analysis Not worth more than a bare mention

41. CEO Compensation

42. Data • The amounts of total compensation received by CEOs of America’s largest companies is controversial discussions • Data on 50 of Forbe’s 1991 US large firms (Frees 1996) • Two variables • Salary and bonuses • Additional compensations (mainly stock options exercised)

43. Distributions of variables • Dot plots of data • Models for the variables • Log-normal fits salary and bonus (AD test =.53, P=.16) • Weibull fits additional comp (AD test = .61, P=.12) Heavy tail

44. Relationship between extremes of two variables • Do CEOs with extreme salary and bonus get extreme additional compensation? • The traditional analysis • Positive association between salary and bonus and additional compensation • Correlation and a regression analysis • Regression assumptions do not hold • Transform data; assumptions hold • Line of averages • Both above average • Both below average • One above, one below • Positive Association Mostly blue kind 16 8 22 8

45. Relationship between extremes of two variables • Bonferroni type adjustment for two variables • 95% thresholds for two variable • 97.5% threshold for each variable • Positive association between extremes on the two variables? 3 0 • Extreme on both • Extreme on only one • Neither • Regression line 42 5

46. Bayesian analysis • Inference for probability of extreme on only one or both variables • Data: Cross-tabulation • Prior: Uniform distribution (or skewed Beta) • Posterior Beta distribution for extreme on both variables • Posterior Beta distribution for extreme on only one variable • Posterior odds in favor of being extreme in only one variable: 832to 1 Robust against the choice of prior

47. Thresholds for percentiles

48. Generalization: Thresholds for Lower & Upper • Upper and lower observations in sample (e.g., Edmonton et al 2001) • n= 93 (9/11 sample size) • Raw count creates ties • Use distribution of respective order statistics Thresholds avoid tie

49. An example from the literature • Edmondson et al (2001) • Comparison of low and high performing hospitals (n = 16) • “The middle two scores are too close” • Normal model fits (Mean=21.5, SD=10.1, AD Test=.38, P-value=.37) Edmondson et al’s High Edmondson et al’s Low