What Makes A Good Model in Economics and Finance

1. What Makes A Good Model in Economics and Finance Ioanid Rosu University of Chicago ASE, December 21, 2006

2. Introduction • What is a goodtheoretical model in economics? • Why is it that most famous models in economics are relatively simple? • We will not discuss here what makes a model important (i.e. useful).Typically, such a model • Explains an important fact (e.g. the Phillips curve: inflation vs. unemployment) • Gives a framework to think about important issues (e.g. Arrow’s social choice theory) • Provides an important methodology or economic intuition (e.g. Akerlof’s lemons model – asymmetric information)

3. Good = True = Credible • Instead, we want to know what a true model is • In economics true is usually replaced by credible or appropriate • We do not test an economic model, but rather we validate it, i.e. we decide whether it is credible or not • In general, a model: • Assumptions ) Conclusions (via causal chains) • Naively, one may propose two criteria for a credible economic model: • Internal consistency (mathematically true) • Verifiable conclusions (empirically true) • This works for physics • Assumptions and causal chains are very reliable (physical laws and mathematical derivations) • Conclusions can be verified by controlled experiments

4. Economics vs. Physics • Can economics imitate physics? Some say no: • “Economics suffers from physics envy. […] We would love to have 3 laws that explain 99 percent of economic behavior; instead, we have about 99 laws that explain maybe 3 percent of economic behavior!” (A.W. Lo) • Nevertheless, are economic assumptions reliable? • For example in many models agents are assumed to be rational, even though there is ample evidence to the contrary • Milton Friedman (Nobel prize 1976): it doesn’t matter as long as the conclusions are verified • E.g. agents are assumed to behave as if they were rational, even if individually they can be irrational • And how about the structure of the model? Can we have as many assumptions and causal chains as we want? • Mathematical economists seem to believe so

5. Why Economics Is Hard • In fact things are not so simple with economic models • Assumptions and causal chains are noisy, and controlled experiments are rare (i.e. one cannot eliminate the outside noise) • Each causal chain amplifies (compounds) the noise, so one must focus only on a few causal chains • As a consequence, a typical economic model is not a complete system, but only a window through which we study reality • Moreover, economic models themselves may affect human behavior, which creates two sides of economics: • Positive (describe reality as it is) • Normative (describe reality as it “should” be) • The economic truth is a partial, moving target • Often when a model fails empirically, economists still use the model as a guide • They rely on the normative aspect: if only the economics agents were rational and listened to us…

6. How to Think of Economic Models • Because of the noise, an economic model is not a complete system (e.g. Newtonian mechanics), but a window which connects a few economic variables • So we should think of a model as • A few causal chains between a small set of variables • A plausible explanation for a set of conclusions • Now, when we think of a good model, we want to know whether the window through which we observe reality does a credible job • Let us call an economic model credible (good) if • The assumptions are plausible, although possibly imperfect (noisy) • The explanation itself is plausible, i.e. the results are strong enough to be distinguished from noise • The conclusions are not too sensitive to the assumptions

7. The 3R of Economic Research & the ABC of Model Verification • The 3R of economic research: a good (credible) economic model must be 1. Realistic 2. Relevant 3. Robust • Moreover, a (positive) economic model should be 4. Verifiable The ABC of model verification: a good model has A. Alternative B. Byproducts C. Calibration

8. Example: CAPM(Capital Asset Pricing Model) • Assumptions • Agents only care about risk and return • Their utility depends only on the mean of the portfolio ( E ) and its volatility ( ) • Agents have the same beliefs about asset returns • E(R) and  (R) are the same for all agents • The market is in equilibrium • Conclusions • Agents invest the same relative percentages of their wealth in the risky assets (stocks) • Everybody invests only in the riskless asset (= the “bond”) and the collection of all stocks (= “the market”) • Main conclusion: the expected return of a stock satisfies the CAPM formula E(Ri) – Rf= i ( E(RM) – Rf )

9. The 3R of Economic Research • A model approximates reality, and tries to reliably explain an economic phenomenon • Since we start with noisy assumptions and we draw noisy inferences (causal chains), we need to make sure that conclusions are not too noisy • In some sense, the noise of the conclusions equals the noise of assumptions, compounded by the noise of the causal chain • The model therefore must be • Realistic: the assumptions are plausible, i.e. they are a good approximation of reality • The assumption noise is small • Relevant: The results are strong, i.e. first order • The causal chain noise outside the model is relatively small • Robust: The conclusions are not too sensitive to the assumptions • The causal chain noise inside the model is small • Then the conclusion noise is also relatively small

10. 1. Realistic • Assumptions must be realistic, i.e., they should represent a reasonable approximation of reality • Contradiction with Friedman’s thesis? • No, because “plausible” is not the same as “true”. Assumptions need not be true per se. • Realistic assumptions are the foundation of good research: Suppose you find a conclusion is false • Then one of the assumptions is false • If the assumptions are not realistic (plausible), it is hard to learn from the failure of the conclusion • Example: When testing CAPM, we find that agents are far from holding the market portfolio (e.g. home bias) • Then one of the assumptions is false, but which one? • Maybe that E(R) and  (R) are the same for all agents • Since the assumption is plausible, it is clear how to modify it: e.g. agents have different information about asset returns and volatilities • The Black–Litterman model modifies exactly this assumption and suggests how agents with different beliefs should invest

11. 2. Relevant • The results must be relevant, i.e. the assumptions provide a plausible explanation of the conclusions • Since a model is only a window to reality, one has to make sure the causal chains are strong enough compared with the outside noise • For example, many models can predict that in the summer it’s hot, but do they provide a plausible explanation? • Relevance is the key feature of a good model. Typically a relevant model satisfies the 3S rule: • Has a story, i.e. a clear intuition behind it • Is simple: too many assumptions weaken the value of a conclusion (each assumption comes with an inherent error) • The sizes implied by the model are plausible (e.g. from a calibration) • Example: CAPM is relevant for explaining expected returns by betas (Fama & MacBeth 1973) • Even if the CAPM formula today is empirically less true (APT), the model remains relevant: the normative aspect!

12. 3. Robust • A model is robust when small modifications of the assumptions do not change conclusions in an essential way • It is very rare that assumptions are perfectly plausible, so we must make sure that the inherent error in choosing them does not affect the results significantly • Example: In one version of CAPM it is assumed that agents have quadratic utility • This is a strong assumption: it implies a saturation point • Do the conclusions of CAPM depend essentially on this assumption? • No, the conclusions also hold if stock returns are normally distributed • In reality, the distribution of stock returns is not exactly normal, but has “fat tails” (it is leptokurtic) • If we modify this assumption, it turns out that CAPM is still approximately true

13. 4. Verifiable • Karl Popper’s criterion: a (positive) scientific model must be verifiable, i.e.falsifiable • Verification of a single conclusion can be done by • Intuition or anecdotal evidence: Is the conclusion reasonable? Do we have arguments or examples that support it? • E.g. in corporate finance (moral hazard, risk shifting) • Statistics: Test the conclusion by bringing it to the data • E.g. to test CAPM, one can check whether a = 0in the linear regression E(Ri) – Rf= a + b£i + ei One can also check whether b = E(RM) – Rf • It is not enough to verify just one conclusion of the model. One also needs to verify the model itself, since in economics conclusions are model dependent • The ABC of model verification: a good model has A. Alternative B. Byproducts C. Calibration

14. A. Alternative • One must have an alternative (H1) to the main conclusion (H0) of a model • In many cases, the alternative is obtained by negating H0 • Example: for CAPM, H0: a = 0 , H1: a 0 • Ideally, one would like to have an alternative model, to provide an alternative explanation for the main conclusion • Example: One can compare CAPM to the alternative model, APT (Arbitrage Pricing Theory) in its Fama–French version: E(Ri) – Rf= i ( E(RM) – Rf ) + hiE(RHML) + si E(RSMB) where HML is a factor that represents “value – growth” firms, and SMB represents “small – big” firms

15. B. Byproducts • Besides an economic model’s main conclusion, there are also secondary conclusions (byproducts) • Since other alternative models may have the same main prediction, one can check that the given model provides the correct explanation by also testing the byproducts • Example: The main conclusion of CAPM is that expected stock returns only depend on betas • It also has the byproduct that agents invest in the same way in stocks – the market portfolio • This may explain the popularity of index funds (low cost copies of the market portfolio) • But there are empirical problems (e.g. home bias) • Again, one can refer to the normative side of the model

16. C. Calibration • One should be able to calibrate the model (both the assumptions and the conclusions) • Many models can predict that in the summer it is hot • The question is: exactly how hot is it in the summer? • Otherwise, we cannot tell if our explanation is correct • Example: CAPM not only predicts that the higher the risk – the higher the expected return, but also gives a precise estimate of the expected return ( E(R) ) • The parameters of the calibration should be plausible • For this, it is important that the model be realistic (otherwise, what does “plausible parameter” mean?)

17. In Conclusion • A good economic model should be (the 3R) • Realistic • Plausible assumptions • Relevant • Results should be of first order (plausible explanations) • Robust • Small changes in assumptions do not change conclusions in an essential way • Verifiable (the ABCof verifying a model) • Alternative • Compare the given explanation with an alternative one • Byproducts • The other conclusions of the model should also hold • Calibration • One should calibrate the model (with plausible parameters) and compare it with the data