Skip this Video
Download Presentation
The Kelly criterion and its variants: theory and practice in sports, lottery, futures & options trading

Loading in 2 Seconds...

play fullscreen
1 / 90

The Kelly criterion and its variants: theory and practice in sports, lottery, futures & options trading - PowerPoint PPT Presentation

  • Uploaded on

The Kelly criterion and its variants: theory and practice in sports, lottery, futures & options trading .

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'The Kelly criterion and its variants: theory and practice in sports, lottery, futures & options trading' - marcel

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
The Kelly criterion and its variants: theory and practicein sports, lottery, futures & options trading

Professor William T ZiembaAlumni Professor of Financial Modeling and Stochastic Optimization (Emeritus), UBCICMA Financial Markets Centre, University of ReadingVisiting Professor Mathematical Institute, Oxford University andPresident, William T Ziemba Investment Management Inc

Kaist Lecture Program

August 2011

who i am
Who I Am:

RAB/Canyon 2005-07

PIWM 2007 to present


the two trading routes available for cash equity and equity futures hedge funds
The two trading routes available for cash, equity and equity Futures hedge funds
  • The search for positive  and the generation of returns from  risk in smart index funds and active equity portfolio management

I=excess mean return

i=leveraging factor for long market exposure

  • The search for absolute returns using research on market imperfections, security market biases, mispriced derivative securities, arbitrage, risk arbitrage, and superior investment criteria
  • The strategy is to win in all markets - up, down and even, and to achieve a smooth wealth path with few drawdowns.


to win consistently you must
To win consistently: you must
    • Get the “mean right” that is the direction of the market - adjusted for various types of hedging and positioning. This must consider your risk tolerance.
    • You must diversify so that regardless of the market path (scenario) the positions are not overbet.
    • You must bet (portfolio allocate) well.

I argue that getting the mean right is themost important ingredient in winning strategies.

  • This was especially crucial in the equity markets the past 10+ years: the mean was frequently negative.


The Importance of getting the mean right. The mean dominates if the two distributions cross only once.
  • Thm: Hanoch and Levy (1969)
  • If X~F( ) and Y~G( ) have CDF’s that cross only once, but are otherwise arbitrary, then F dominates G for all concave u.
  • The mean of F must be at least as large as the mean of G to have dominance.
  • Variance and other moments are unimportant. Only the means count.
  • With normal distributions X and Y will cross only once iff the variance of X does not exceed that of Y
  • That’s the basic equivalence of Mean-Variance analysis and Expected Utility Analysis via second order (concave, non-decreasing) stochastic dominance.
Mean Percentage Cash Equivalent Loss Due to Errors in Inputs

Risk tolerance is the reciprocal of risk aversion.

When RA is very low such as with log u, then the errors in means become 100 times as important.

Conclusion: spend your money getting good mean estimates and use historical variances and covariances

Average turnover: percentage of portfolio sold (or bought) relative to preceding allocation
  • Moving to (or staying at) a near-optimal portfolio may be preferable to incurring the transaction costs of moving to the optimal portfolio
  • High-turnover strategies are justified only by dramatically different forecasts
  • There are a large number of near-optimal portfolios
  • Portfolios with similar risk and return characteristics can be very different in composition
  • In practice (Frank Russell for example) only change portfolio weights when they change considerably 10, 20 or 30%.
  • Tests show that leads to superior performance, see Turner-Hensel paper in ZM (1998).
log utility bernoulli 1732
Log Utility: Bernoulli (1732)
  • In the theory of optimal investment over time, it is not quadratic (the utility function behind the Sharpe ratio) but log that yields the most long term growth.
  • But the elegant results on the Kelly (1956) criterion, as it is known in the gambling literature and the capital growth theory as it is known in the investments literature, see the survey by Hakansson and Ziemba (1995) and MacLean and Ziemba (2006), that were proved rigorously by Breiman (1961) and generalized by Algoet and Cover (1988) are long run asymptotic results.
  • However, the Arrow-Pratt absolute risk aversion of the log utility criterion is essentially zero, where u is the utility function of wealth w,, and primes denote differentiation.
  • The Arrow-Pratt risk aversion index.
  • is essentially zero, where u is the utility function of wealth w, and primes denote differentiation.
  • Hence, in the short run, log can be an exceedingly risky utility function with wide swings in wealth values.
Thus the criterion of maximizing the long run exponential rate of asset growth is equivalent to maximizing the one period expected logarithm of wealth. So an optimal policy is myopic.
  • Max G(f) = p log (1+f) + q log (1-f)  f* = p-q
  • The optimal fraction to bet is the edge p-q
Slew O’ Gold, 1984 Breeders Cup Classic

f*=64% for place/show; suggests fractional Kelly.

maximizing long run growth
Maximizing long run growth
  • Thus the criterion of maximizing the long run exponential rate of asset growth is equivalent to maximizing the one period expected logarithm of wealth.
  • So an optimal policy is myopic - future and past do not affect current optimal decisions
  • Max G(f) = p log (1+f) + q log (1-f)  f* = p-q
  • The optimal fraction to bet is the edge p-q (the mean)
  • So if the edge is large, the bet is larger
  • p = .99, q = .01  f*= 98% of wealth


what does the theory tell us about long term hedge fund trading and overbetting
What does the theory tell us about long term hedge fund trading and overbetting?

Kelly and fractional Kelly - explaining the overbetting that leads to hedge fund disasters: you cannot ever bet more than full Kelly and usually you should bet less


mohnish pabrai investing in stewart enterprises thorp 2010 in our world scientific book
Mohnish Pabrai, investing in Stewart Enterprises - Thorp (2010) in our World Scientific book

Hedge fund manager won bidding for 2008 lunch with Warren Buffett for $600K+

Stewart Enterprises, Payoff ≤ 24 months

Prob Net Return

0.8 >100%

0.19 zero

0.01 lose all investment

Pabrai bet 10% of his fund

What’s the full Kelly bet?

f* =0.975; half Kelly 0.3875; quarter Kelly=0.24375


pabrai bet issues should he have bet more
Pabrai bet issues: should he have bet more?

Other opportunities: must compute against all options (nonlinear or stochastic optimization) for the available wealth

Risk tolerance: what fractional Kelly to use?

Black Swans: we call them bad scenarios

Long vs short run planning


kelly betting at pimco
Kelly betting at PIMCO
  • During an interview in the Wall Street Journal (March 22-23, 2008) Bill Gross and Ed Thorp discussed turbulence in the markets, hedge funds and risk management.
    • Bill considered the question of risk management after he read Beat the Dealer in 1966.
    • That summer he was off to Las Vegas to beat blackjack.
    • Just as Ed did some years earlier, he sized his bets in proportion to his advantage, following the Kelly Criterion as described in Beat the Dealer, and ran his $200 bankroll up to $10,000 over the summer.
    • Bill has gone from managing risk for his tiny bankroll to managing risk for Pacific Investment Management Company’s (PIMCO) investment pool of almost $1 trillion.
    • He still applies lessons he learned from the Kelly Criterion.
    • As Bill said, “Here at PIMCO it doesn’t matter how much you have, whether it’s $200 or $1 trillion … Professional blackjack is being played in this trading room from the standpoint of risk management and that’s a big part of our success.”.


top 10 equity holdings of soros fund management and berkshire hathaway sept 30 2008 sec filings
Top 10 equity holdings of Soros Fund Management and Berkshire Hathaway, Sept 30, 2008 (SEC filings)
The wealth levels from December 1985 to April 2000 for the Windsor Fund of George Neff, the Ford Foundation, the Tiger Fund of Julian Robertson, the Quantum Fund of George Soros and Berkshire Hathaway, the fund run by Warren Buffett, as well as the S&P500 total return index.
berkshire hathaway versus ford foundation monthly returns distribution january 1977 to april 2000
Berkshire Hathaway versus Ford Foundation, monthly returns distribution, January 1977 to April 2000
return distributions of all the funds quarterly returns distribution december 1985 to march 2000
Return distributions of all the funds, quarterly returns distribution, December 1985 to March 2000
Kelly and half Kelly medium time simulations: Ziemba-Hausch (1986)

These were independent

The good, the bad and the ugly

166 times out of 1000 the wealth is more than 100 times the initial wealth with full Kelly but only once with half Kelly does the investor gain this much

But probability of being ahead is higher with half Kelly, 87% vs 95.4%

Min wealth is 18 and only 145 with half Kelly

700 bets all independent with a 14% edge: the result, you can still lose over 98% of your fortune with bad scenarios

With half Kelly, lose half of wealth only 1% of the time but is is 8.40% with full Kelly

So even after 700 plays, the strategy is still risky

See  Simulation paper  for more on this and two more examples

Kentucky Derby 1934-1998
  • Use inefficient market system in Hausch, Ziemba, Rubinstein (1981) and Ziemba-Hausch books
  • Place/show wagers made when prices off sufficiently and EX≥ 1.10
  • w0 = $2500 63 years 72 wagers with 45 (62.5%) successful
Typical wealth level histories with one scenario (the actual results) from place and show betting (Dr Z system) on the Kentucky Derby, 1934-1994 with Kelly, half Kelly and betting on the favorite strategies

Probability of doubling and quadrupling before halving and relative growth rates versus fraction of wealth wagered for Blackjack (2% advantage, p=0.51 and q=0.49

Should you ever be above 0.02 that is positive power utility like

It is growth-security dominated.

Betting more than the Kelly bet is non-optimal as risk increases and growth decreases; betting double the Kelly leads to a growth rate of zero plus the riskfree asset.

LTCM was at this level or more, see AIMR, 2003.

Several similar blowouts are discussed in Ziemba and Ziemba (2007) including Amaranth and Niederhoffer.

Fractional Kelly and negative power utility
  • u(w) =-w <0
  •  0 u  log

f=1/(1- ) = fraction (Kelly) in log optimal portfolio, rest in cash

=0 f=1 full Kelly

=-1 f=1/2 1/2 Kelly

=--3 f=1/4 1/4 Kelly futures trading down here

This is exact with log normality and approximate otherwise but it can be way off.

samuelson s critique of kelly betting
Samuelson’s critique of Kelly betting

Correspondence: Nov 16, 2005 to Elwyn Berlekamp

Dec 13, 2006 to WTZ

Samuelson postulated three investors, all risk averse and concave

WTZ adds two more: Ida (the most risk averse) and Victor (the most risk accepting)

Cash return zero, Stock 50-50: $4 or $0.25 for $1 bet (for every period)

the investors
The Investors
  • Tom, I believe, is overbetting and dominated and will go bankrupt
  • Harriet has a limited degree of risk tolerance, fits well with lots of empirical Wall St equity premium data
  • Victor Niederhoff: always had high returns of blowouts leveraging the S&P500
  • Too close to the money for proper risk control
  • See Chapter 12 of Ziemba and Ziemba (2007)
  • There are more blowouts since then!
  • This is correct. I have no disagreement.
  • The fully Kelly strategy gives very high final wealth most of the time
  • But it is possible to have low final wealth with no leveraging (-98%) and many times W0 lost with leveraging
  • See examples in Simulation Section
  • market in miniature
  • fundamental and technical systems
  • returns and odds are determined by
  • 1) participants -- like stock market, unlike roulette
  • 2) transaction costs -- track take (17%), breakage;
  • rebates now plus Betfair (long short)
  • bet to
  • 1) win -- must be 1st
  • 2) place -- must be 1st or 2nd
  • 3) show -- must be 1st, 2nd or 3rd
Place market in horseracing

Inefficiencies are possible since:

1) more complex wager

2) prob(horse places) > prob(horse wins) ==> favorites may be good bets

To investigate place bets we need:

1) determine place payoffs 

2) their likelihood 

3) expected place payoffs

4) betting strategy, if expected payoffs are positive

Bettors do not like place and show bets.

The Idea

Use data in a simple market (win) to generate probabilities of outcomes

Then use those in a complex market (place and show) to find positive expectation bets

Then bet on them following the capital growth theory to maximize long run wealth

Effect of transactions costs, calculation of optimal place and show Kelly bets

Non concave program but it seems to converge.

In practice, adjust q’s to replicate biases.

Victor Lo research on this in his thesis and Hausch, Lo, Ziema 1994, 2008 books

Use in a calculator

What we do in the system is to reduce the non-convex log optimization problem down to four numbers: Wi,, W, and Si, S or Pi, P,

Thousands of race results regress the expected value and the optimal Kelly bet as a function of these four variables.

Hence, you just find horses where the relative amount bet to place or show is below the bet in the win pool.

The calculator tells you when the expected value is say 1.10 or better and calculates the optimal Kelly bet.

So this can be done in say 15 seconds.

Expected value approximation equations
  • Expected value (and optimal wager) are functions of only four numbers - the totals and the horse in question.
  • These equations approximate the full optimized optimal growth model.
  • Solving the complex NLP: too much work and too much data for most people.
  • This is used in the calculators, and Hausch-Ziemba (1985, Management Science), differing track take, etc.
simulations in 2004 5
Simulations in 2004-5

Real results April 2005-March 2006

Up ~ 36,000 ~ 2% on bets ~ 1.5 M,

System -7%, rebate ~ 9%, edge ~ +2%

We keep doing this searching for good bets at 80 tracks

Before going more into Samuelson and the simulations, let’s look at when the Kelly bets are very large or very small
Commodity Trading: Turn of Year Effect

Small cap stocks have outperformed large cap stocks in January on a regular basis since 1926

Average excess returns of smallest minus largest decile of US stocks, 1926-93, Source: Ibbotson Associates

january effect historical and recent move to december
January effect: historical and recent, move to December

January effect, 1926-1995. January size premium = R(10th)-R(1st).




Futures play with anticipation, mid December to mid January, this is a typical year in the mid 90s, Value Line versus S&P, 1992-3

Turn of the year effect: using past data to get the probability distributons of returns

Probability of reaching $10 million before ruin for Kelly, half Kelly and quarter Kelly strategies

Relative growth rate and probability of doubling, tripling or tenfolding before halving for various Kelly strategies

Turn of the year effect, recent developments

Futures markets - much more violent

Russell 2000 - has more volume than Value Line

Effect moved into December

Textbooks and finance experts say effect is not there

Graphs in Hensel-Ziemba paper in Keim-Ziemba (2000) Worldwide security market imperfections, Cambridge University Press.

Doing this trade is like driving a dynamite truck smoking a cigar. You do it carefully.

Rendon-Ziemba (2007) update to 2005 turn of the year Value Line/S&P500 and Russell 2000/S&P500 spread trades

Ziemba (2011) new paper updating to 2011, showing the trade still worked in 2009/10 and 2010/11

my experiences trading the toy
My experiences trading the TOY
  • I started trading the January turn-of-the-year effect for the 1982/83 TOY and for the next 13 TOYs
  • During that time the cash small stock advantage was in the first half of January
  • But with futures anticipation, the effect covered the mid-December to mid-January period
  • A rule we devised in 82/83 was to go long the Value Line short the S&P500 during this period
  • However, the effect moved to be fully in the second half of December
  • I managed to make gains in all 14 of these years and be 7% of the market
  • But for two reasons stopped trading it:
    • Low Value Line volume, and
    • Teaching the trade to Morgan Stanley as a consultant to Peter Muller’s group
  • I continued to do research and write papers on this and returned to trade the effect the past two TOYs with the Russell2000 replacing the Value Line


toy graphs
TOY graphs

The paper Investing in the Turn-of-the-Year hasgraphs and tables of the trades for various years

The next slide has four years of VL/S&P trades and the following slide has four years of the R2000/S&P trades

Each year was a little different but it was possible to win each year.

Finally, the third set of graphs after the monthly R2000-S&P500 futures spread, 1993-2009 has 2009/2010 and 2010/11 where we entered on the dots and exited on the squares as the market turned.


russell2000 s p500 futures spread average returns during the moy 1993 2010
Russell2000 - S&P500 Futures Spread Average Returns during the MOY, 1993-2010

Observe December!


Probability of reaching the goal of $10 million before falling to $25,000 with various initial wealth levels for Kelly, 1/2 Kelly and 1/4 Kelly wagering strategies

The downside of the analysis is that the expected time to win a lot is in the millions of years.

Calculating the optimal Kelly fraction: staying above an exogenous wealth path

To stay above a wealth path using a Kelly strategy is very difficult

Kelly fractions and path achievement

  • the more attractive the investment opportunity,
  • the larger the bet size and
  • hence the larger is the chance of falling below the path.

MSZZ (2004) using a continuous time lognormally distributed asset model calculate that function to stay above a path at various points in time to stay with a high exogenously specified value at risk probability.

Convex case like Geyer-Ziemba (2008) Siemens Vienna pension model - can do on a computer; in MacLean, Zhao, Ziemba (2009)

With initial wealth W(1)=1, the value at risk is a. The optimal investment decisions and optimal growth rate for a, the secured average annual growth rate and 1-a, the security level are shown in the table.
Brief Guide to Capital Growth Theory and Kelly Criterion Literature

Bernoulli (1732) translated in 1954; original idea of log utility; marginal utility, current wealth proportion to St Petersburg Paradox

MacLean, Thorp, Ziemba (2009) book with main articles reprinted plus new ones

Some properties of the Capital Growth Theory (cont’d)

See Ziemba and Hausch (1996), Aucamp (1993), Browne (1997) and MacLean, Thorp, Ziemba (2010) for more on this.


Essentially all of the material in this talk is in the following books plus the papers listed above

Ziemba, The Stochastic Programming Approach to Asset Liability Management, AIMR, 2003

Ziemba-Hausch, Dr Z’s Beat the Racetrack, William Morrow, 1987 (has UK betting system)

Hausch-Lo-Ziemba, Efficiency of Racetrack Betting Systems, Academic Press, 1994. Classic new and reprinted articles, bible for Hong Kong professional betting teams. Originals sell for huge prices as high as $12,000 I am told, I sold one for $1400. 2008 2nd edition from World Scientific in Singapore at a low price.

Ziemba-Vickson, Stochastic Optimization Models in Finance, Academic Press, 1975. Classic articles, new articles, huge collection of portfolio theory, problems.Reprinted by World Scientific, Singapore, 2006.

Ziemba et al, 6/49 Lotto Guidebook, 1986

Ziemba-Hausch, Betting at the Racetrack, 1986, exotic bet pricing

Ziemba and Ziemba (2007) Scenarios for Risk Management and Global Invetment Strategies, Wiley

MacLean, L.C., E. O. Thorp, Ziemba, W.T., Eds., The Kelly Capital Growth Criterion: Theory and Practice, World Scientific

Books all available, [email protected] for information. Amazon has them at low prices.