Deriving Kelly Criterion for Speculation in Stock Markets by Jane Hung

Introduction

Gambling in all forms, whether it be in blackjack, sports, or the stock market, must begin with a bet. In this paper, we summarize Kelly’s criterion for determining the fraction of capital to wager in a gamble. We also test Kelly’s criterion by running simulations.

In his original paper, Kelly proposed a different criterion for gamblers. The classic gambler thought to maximize expected value of wealth, which meant she would need to invest 100% of her capital for every bet. Rather than maximizing expected value of capital, Kelly maximized the expected value of the utility function. Utility functions are used by economists to value money and are increasing as a function of wealth under the assumption that more money can never be worse than less [1]. Kelly took the base 2 logarithm of capital as his utility function [2], but we will use the base e logarithm (the natural log) instead.

Kelly Criterion

The following derivation is modified from Thorp [1]. We assume that the probability of events are known and independent and that the probability of a win is p (1 > p > 1/2) and the probability of a loss is q = 1 − p. Suppose a fraction f (0 < f < 1) of the capital is bet each turn and Wn and Ln represent the number of wins and losses after n bets, respectfully. Rather than even payoff (i.e., a win of 1 unit per unit bet per win), we consider the more general scenario that b units are won per unit bet per win and a units are lost per unit bet per loss. Given initial capital X0, the capital after n bets is

Xn = X0(1 − af)Ln(1 + bf)Wn.

Now define,

21

the exponential rate of increase per trial. The expected value of g(f) is

G(f) = E(g(f)) = q · log(1 − af) + p · log(1 + bf)

because the ratio of expected wins or losses to trials is given by the probabilities p and q, respectively. We want to maximize G(f) because ,

22

so maximizing G(f) would in turn maximize E(log(Xn)), the expected value of the logarithm of wealth. A critical point of G(f) can be found by setting the derivative to 0:

23

Figure 1: Expected Value of Logarithm of Wealth vs. Bet as a Fraction of Wealth

2425

so the critical point is at

.26

Notice that) is defined at f, and the critical point is a zero of

G0(f) there. Since

27

fis a local maximum. And because G(0) = 0 and limf→1G(f) = −∞, the maximum of G(f) is at f. Figure 1 shows the plot of G(f) as a function of f. For this function, we set p = 0.8, q = 0.2, and a = b = 1. The maximum occurs at f= p q = 0.6.

The Stock Market

Kelly criterion can be applied to the stock market. In the stock market, money is invested in securities that have high expected return [3]. The following derivation is modified from Thorp [1]. Since there is not a finite number of outcomes of a bet on a security, we must use continuous probability distributions. Let X be a random variable that denotes the return per unit, and suppose

.28

Then the expected value E(X)=µ, and the variance of X is σ2 (with standard deviation σ). Suppose the initial capital is Y0 and the bet as a fraction of wealth is f. Then the capital Y (f) is given by

Y (f) = Y0(1 + (1 − f)r + fX),

where r is the rate of return of capital invested elsewhere. Using the probability assumptions, this means

.29

If there are n time steps of equal length in the time interval, then we have X at each of those steps, Xi, with i=1, 2,…, n. Also,

30

given that we want the same total µ, σ2 and r. Then we have

31

and

32.png

Now we expand Gn(f) as a Taylor series around f = 0. Calculating the derivatives of Gn(f), we get

33

and

34

for k ≥ 3. Then Gn can be expressed as

.35

To make this continuous, we allow n → ∞; thus Gn becomes

.36

Notice that f < 0 is allowed and is equivalent to taking a short position. This Gis an instantaneous growth rate, so adjustments must be made when Yn undergoes a change. Using the method in section 2, we find that the optimal betting fraction, f, is

.38

(where expected value E(X)=µr is the rate of return of capital invested elsewhere, and the variance of X is σ2 (with standard deviation σ))

Simulations

Using MATLAB, we simulated betting with two different strategies: one using the Kelly Criterion and another with constant betting. The scenario is simplified such that the probability of a win and a loss are known and constant. This may be realistic in the case of a very consistent sports team for example. The parameters given are

  • probability of winning the bet p = 0.55,
  • probability of losing the bet 1 − p = q = 0.45,
  • units won per unit bet per win b = 10/11,
  • units lost per unit bet per loss a = 1,
  • number of trials n = 5000, and initial capital X0 = $100.

The rand command in MATLAB was used to generate random numbers for determining the outcome of each trial; this command returns pseudorandom numbers from a uniform distribution. The results are shown in Figure 2.

Figure 2: Capital Through 5000 Bets: Betting with the Kelly Criterion vs. with constant bets.

39.png

From the graph, betting with the Kelly Criterion clearly has an advantage over constant betting. After 5000 bets, betting with the Kelly Criterion yields a total capital of between $5000 and $10000 (a percent increase of capital of over 4900%) while constant betting yields a total capital of around $2500 (a percent increase of capital of about 2400%). However, unlike the Kelly Criterion curve, constant betting showed a roughly linear trend line; the fluctuations cannot be measured readily by glance. With the Kelly Criterion, the fluctuation is orders of magnitude different though the overall upward trend is above that of constant betting. Noticeable drops and gains of thousands of dollars within 100 bets are evident from looking at the Kelly Criterion graph. In addition, betting with the Kelly Criterion may occasionally be worse than constant betting even after several thousand bets.

The number of bets considered here should also be discussed. Betting 5000 times may be unrealistic for most. If 3 bets were made every week, it would take around 32 years to reach 5000. During this time, even a consistent team would likely not carry the same win percentage! For the short term, it may be better to look at the performance of betting with the Kelly Criterion through 150 bets (1 year’s worth of betting). In this interval, the Kelly Criterion seems virtually identical to constant betting (flat betting). There does not seem to be a significant increase in capital during that time with either method. Appreciable differences are seen only at around 1000 bets, so in order to experience the advantage of using the Kelly Criterion, a bettor should start with more capital, make more bets, or be willing to wait a long time. From this simulation, we see that betting with the Kelly Criterion is effective after many trials but also quite volatile.

40

Use of the Kelly Criterion is further investigated through application to the stock market. The closing stock prices of Goldman Sachs Group, Inc. (GS) from May 30, 1999 to May 24, 2010 were obtained [4] and used as the data. In this period, the stock rose from 64.19 to 136.69. Since stocks typically experience many highs and lows, one single mean and standard deviation value cannot represent the behavior of the stock through 11 years accurately. Thus, the data was split into nineteen 146 day blocks, and the mean and standard deviation of each block was found. The optimal fraction for each block could then be calculated. The parameters given are

  • return rate of other investments r = 0.00,
  • number of days = 2774,
  • initial investment Y0 = $10000.

The stock price is the price per share, so the number of shares for day k was given by the investment for that day as suggested by the Kelly Criterion divided by that day’s stock price. Fractional shares were allowed. The subsequent value of those shares was the product of the number of shares for day k and the stock price on day k + 1. To simplify matters, the rate of return of the uninvested wealth was set to zero. Hence the total wealth was the sum of the uninvested wealth and the value of the invested wealth. This yielded the net return when subtracted by the initial investment. It should be noted that the fraction of wealth to invest was limited to ≤ 1 so that we did not have to deal with short selling or debt. The results can be seen in Figure 3 above.

The results are similar to those found in the case of sport betting. The net return after 11 years is about $10000, which is 100% of the initial investment. While investing higher fractions of wealth would increase the net return slightly, that is an extremely risky strategy when the future stock price is unknown. The Kelly Criterion clearly involves nontrivial risk, as evidenced by the negative return within the first 100 days; however, the risk is reduced by the changing of fraction of wealth invested.

This was a simplified example, so the actual outcome would differ if, e.g., the uninvested wealth were put into a risk free security, or if short selling or debt were considered so that the fraction of wealth invested could be above 1. Still, this simulation provides insight into how the Kelly Criterion might perform when used on the stock market.

Conclusion

The Kelly Criterion can be utilized to find the optimal bet size for a wager. Not only can Kelly Criterion be used for sports betting and casino games, it can also be used in the stock market. We derived the optimal bet size expression for a situation with only two outcomes and discrete time steps. Furthermore, we used continuous probability distributions to find the optimal bet size expression in a situation where securities may be bought or sold. Finally, we simulated a betting situation using MATLAB and compared the results of betting with the Kelly Criterion to constant betting. This was expanded to investing in the stock market. We found that the Kelly Criterion is effective, but initial capital should be high and/or a great deal of time should be allowed for the final capital to reach substantial amounts. In this way, the Kelly Criterion is impractical and so is not applied in many situations.

References

  • Thorp, E. The Kelly Criterion in Blackjack, Sports Betting, and the Stock Market. Paper presented at: The 10th International Conference on Gambling and Risk Taking. 1997.
  • Kelly, J.L. A New Interpretation of Information Rule. Bell System Technical Journal. 35. (1956): 917-926
  • Thorp, E. O. Optimal Gambling Systems for Favorable Games. Revue De L’Institut International De Statistique. 37. 3 (1969): 273-293.
  • Goldman Sachs Group, Inc. Historical Prices. Google Finance. http://www.google.com/finance/historical?cid=663137&startdate=May+3 0%2C+1999&enddate=May+29%2C+2010. May 29, 2010.

Detailed analysis fogs the future

As you probably know, derivatives trading is a multibillion dollar industry, both online and offline. Derivatives trading attracts all sorts of traders, from the casual weekend wager on the favorite index on Friday afternoon to those that become experts or even professionals at derivatives trading. Despite the size of this industry, until recently there has not been a lot of significant research on how to make the right bet. In other words, is there a real strategy that one who wants to wager on a trading event can use? Or is it really all up to the luck of the draw?

You may have noticed that we are literally swimming in data lately. If you want to find out any detail, no matter how minute, about a scrip, chances are that data is available somewhere. Yes, there may be a price attached to some data, but the information is there somewhere. Yet, the availability of all this data might be masking the real problem.

Here is what you may not know, the other side of the story, the real truth about all this information. Maybe you don’t actually need all that data, all those detailed analytical reports and projections after all. Yes, this may sound like heresy to some in the derivatives trading world, but you will want to pay real close attention to the next few lines here.

In fact, recent research by Song-Oh Yoon and colleagues at the Korea University Business School suggests that when you zero in on the details of a team or event (e.g., RBIs, unforced errors, home runs), you may weigh one of those details too heavily. For example, you might consider the number of games won by a team in a recent streak, and lose sight of the total games won this season. As a result, your judgment of the likely winner of the game is skewed, and you are less accurate in predicting the outcome of the game than someone who takes a big picture approach. In other words, it is easy to lose sight of the forest for the trees.

Yoon and his research team explored the optimal process of prediction in a series of studies examining bets made on soccer matches and baseball games. In their first study, they reviewed more than one billion (yes, billion) bets placed in 2008-2010 through Korea’s largest sports-betting company, “Sports ToTo.” They characterized the bets in one of two ways: (a) bets that involved a general prediction (i.e., win or lose), and (b) bets that involved a specific prediction (i.e., a precise score). Critically, they wanted to know which type of bet was more likely to result in an accurate prediction of the overall winner. Despite the fact that the specific bets were arguably more difficult and involved greater effort than general bets, they led to diminished success in predicting the global outcome of the game (i.e., which team won). This disadvantage was especially pronounced for games in which the favored team won.

These findings suggest that adopting a holistic approach when predicting outcomes, even for multi-faceted events like sporting competitions, may be more effective than dwelling in the details. However, because these findings reflect performance in a natural setting, they are open to alternative interpretations. For example, different kinds of people (e.g., risk-averse versus risk-seeking) may be more prone to placing different kinds of bets (e.g., general versus specific). In addition, different opportunities for reward may influence betting behavior, thus encouraging those making specific bets to take risks on unlikely outcomes. To control for these factors, Yoon’s team examined betting behavior in a controlled laboratory paradigm.

In three different experiments, participants were asked to make predictions about upcoming sporting events. In each study, half of the participants were randomly selected to make general win/lose predictions, while the other half were asked to make specific score predictions. The dependent measure was the same for both groups: Could they predict the winners?

The pattern of performance across the three studies was remarkably consistent: Participants who made general win/lose predictions were reliably better at projecting the winners of the sporting events than those who made specific score predictions. This advantage was evident regardless of whether reward opportunities were relative (i.e., only the participant with the highest overall performance received cash) or individual (participants received cash for every correct prediction).

Notably, experts did outperform novices. Nonetheless, even experts were reliably better in predicting winners when making general bets than when making specific bets. It seems that even in cases where greater knowledge may offer an advantage, the act of focusing on that knowledge can disrupt decision-making. Thus, while a lifelong baseball fan is more likely to pick the winning team than someone who has never watched a game, for either person a quick prediction about the winner is likely to be more accurate than one that follows deep reflection.

Yoon’s team confirmed this notion by assessing the kinds of information participants were using to make their predictions. As you might expect, those assigned to the general win/lose group reported relying on global assessments (e.g., overall impression of the teams, performance of the teams in years past) to a greater extent than those assigned to the specific score group. In addition, reliance on global information significantly predicted success for all participants. Even for those in the specific score group, use of detailed knowledge (e.g., strength of the defense, coaching talent) was not associated with better performance, while use of global information was.

These data align with lessons learned from research on basic personal decisions. Whether choosing a jelly bean flavor, rating the attractiveness of a face, or selecting a poster to hang in a room, people are more satisfied with their selection and less likely to change their minds when they make their decisions quickly, without systematically analyzing their options or mulling over the reasons for their choice. The advice is thus the same whether considering complex scenarios or simple situations: Don’t overthink it.

This research goes on to indicate that by focusing on minute details, a trader may lose sight of the big picture. Here is an easy to understand illustration, straight from their research findings. Those who wager on simple uptrend/downtrend scenarios using a pre-calculated stop, significantly win more than those who wager on specific target levels.

Bottom line: Don’t get lost in the details. The ‘gut’ feeling you have about a specific trade is more likely to be a winner than bundles of pages of statistics.

( Loosely based on an article published by Scientific American )

 

Punters’ Paradise: Adjusting Full Kelly for Catastrophic Loss

Adjusting Full Kelly for Catastrophic Loss, with examples taken from Options Trading

Kelly’s Criterion Revisted

Following Kelly’s Criterion if we would make repeated bets on the same positive expectation gamble, how much of our bankroll should we risk each time?

From before the fraction to bet is:

1

or as some put it

2
Example
Say the gamble is to flip a fair coin, the payoff odds are 2:1. How much should we bet?
The expected or average payoff per play is

3

Upon a win, for a $1 bet we get our bet back and $2 besides, so the gain per unit bet is 2; hence the Kelly fraction is

4.png

of our bankroll.

Virtues of Kelly

our investment is compounded so we get exponential growth (or decay)
our growth of capital is maximized
we can not be wiped out – e.g. for an even money 60/40 investment, f = 0.2, upon a loss we still have 80% of our bankroll.

Example

Starting with $10,000, after 11 losses in a row we would still have $860, or 8.6% of our initial bankroll, the probability of 11 losses in a row is 1/25,000 or 0.00004, (conversely the probability of 11 consecutive losses not happening is 99.99996%).

Kelly applied to Option Trading

While stock investments are more free-form, the option trading has common grounds with gambling:

  • fixed terms
  • a definite time horizon
  • a payoff settlement at expiration

Hence with the proper statistics, we can use the Kelly criterion to determine optimal investment levels while protecting against a string of reverses.

Gathering Statistics
We need to identify the characteristic features of a specific option play we customarily make. Then record the particulars and results of that play over many implementations. Steve Lenz and  Steve Papale of OptionVue provided this data.

z.png

OptionVue Credit Spread
The data we receied was for over 75 trades

  • number of trades that gained money was 66, and 9 lost money
  • average gain per winning trade: $659.12
  • average loss per losing trade: $1,799.06

I regard the average loss per losing trade as the “bet size”.

Credit Spread Risk Fraction
We calculate the following needed for “edge over odds”
win probability.

5

Therefore  gain per unit bet

6


Hence expectation = (0.366) ∗ 0.88 (1) ∗ 0.12 = 0.202.
And so the Kelly risk fraction is

7

Accounting for Catastrophic Loss

But we can do more with the data. Note that we have “maximum loss” information. This can be regarded as catastrophic loss and taken into account. Let p be the probability of a win, q the probability of a loss, and r the probability of a catastrophic loss. Let γ (greek letter gamma) be the gain per unit bet and λ (greek letter lambda) the size of the catastrophic loss per unit bet.

We now rederive the Kelly fraction.

Kelly Fraction for Catastrophic Loss

The expectation is now

E = γp – q – λr.

The expected growth rate (from the previous talk) is

E(g) = p log(1 + γf) + q log(1 – f) + r log(1 – λf)

And the optimal fraction is the root of the quadratic equation

0 = E – f (pγ(1 + λ) + q(γ – λ) + rλ(γ – 1)) + γλf2

Credit Spreads with Catastrophic Loss

The new parameters are now

  • prob. of a win p = .88
  • prob. of an avg loss q
    8
  • prob. of a catastrophic loss r
    9
  • gain per unit bet γ
    10
  • catastrophic loss per unit bet λ
    11

OptionVue Credit Spreads with Catastrophic Loss

Substituting the values and solving the quadratic equation

E – f (pγ(1 + λ) + q(γ – λ) + rλ(γ – 1)) + γλf2 = 0

we get

f = .458; or about 46%.

Courtesy: Ron Shonkwiler (shonkwiler@math.gatech.edu)

Money Management vs Risk Management vs Position Sizing™

Money management is a term used often in the trading and gambling community. It involves deciding how large a position to take on trade entry, how to scale into or out of a position, etc. It also involves setting aside enough capital that you can make more than a few mistakes and still not blow up. Sometimes it involves averaging in when price goes against you (very dangerous), other times it involves taking profit on part of a position and keeping the rest of the position in case you have a “runner”. Done well, money management can amplify the value of your entry/exit signals.

Risk management is more of an institutional term. It involves statistical models for such metrics as VaR (value at risk), CVaR (conditional value at risk) and many others. Risk managers at banks, hedge funds, brokers, etc. will estimate portfolio risk across managers, across instruments, etc. to comply with financial regulations, and to reduce the chance that they’ll be exposed to deep losses in case of a big event in the market. They will also look at subjective things like counter-party risk, technical risk, model misspecification risk, etc. In this sense, Risk management attempts to manage all the material risks a firm faces.

Position Sizing™ is a trademarked term invented and used, mainly by Van Tharp in his writings.Position sizing is the part of your trading system that tells you how many shares or contracts to take per trade. Poor position sizing is the reason behind almost every instance of account blowouts. In other words, position sizing basically refers to the size of a position within a particular portfolio, or the dollar amount that an investor is going to trade.

So, basically all three terms mean the same thing, except that Money Management is a very general term, used by Investors, Traders and Gamblers, and also by people not connected with any of these, who view money management as the process of managing money which includes expense tracking, investment, budgeting, banking and taxes. Risk Control is a specific term, with an underlying institutional context, whereas Position Sizing™ is a specific term used in the context of trading and investment only.
What I intend to discuss here, is therefore all the above three in the context of Investing, and Trading.

William Ziemba: Fractional Kelly (Part 1)

Notes from the seminar on Mathematical Finance, held at University of Chicago, April 2007.

  1. The Kelly or capital growth criteria maximizes the expected logarithm as its utility function period by period.
  2. It has many desirable properties such as being myopic in that today’s optimal decision does not depend upon yesterday’s or tomorrow’s data,

    Success in investments has two key pillars: (1) devising a strategy with positive expectation and (2) betting the right amount to balance growth of one’s fortune against the risk of losses

  3. It asymptotically maximizes long run wealth almost surely and it attains arbitrarily large wealth goals faster than any other strategy.
  4. Also in an economy with one log bettor and all other essentially different strategy wagers, the log bettor will eventually get all the economy’s wealth.
  5. The drawback of log with its essentially zero Arrow-Pratt absolute risk aversion is that in the short run it is the most risky utility function one would ever consider.
  6. Since there is essentially no risk aversion, the wagers it suggests are very large and typically undiversified.
  7. Simulations show that log bettors have much more final wealth most of the time than those using other strategies but can essentially go bankrupt a small percentage of the time, even facing very favorable investment choices.
  8. One way to modify the growth-security profile is to use either ad hoc or scientifically computed fractional Kelly strategies that blend the log optimal portfolio with cash. to keep one above the highest possible wealth path with high probability or to risk adjust the wealth with convex penalties for being below the path.

    Additionally legendary investors such as John Maynard Keynes (0.8 Kelly) running the King’s College Cambridge endowment, George Soros (? Kelly) running the Quantum funds and Warren Buffett (full Kelly) running Berkshire Hathaway had similarly good results but had much more variable wealth paths

  9. For log normally distributed assets this simply means using a negative power utility function whose risk aversion coefficient is 1:1 determined by the fraction and vice versa.
  10. For other asset returns this is an approximate solution.
  11. Thus one moves the risk aversion away from zero to a higher level.
  12. This results in a smoother wealth path but usually has less growth.
  13. This seminar is a review of the good and bad properties of the Kelly and fractional Kelly strategies and a discussion of their use in practice by great investors and speculators most of whom have become centi-millionaires or billionaires by isolating profitable anomalies and betting on them well with these strategies.
  14. The latter include Bill Bentor the Hong Kong racing guru, Ed Thorp , the inventor of blackjack card counting who compiled one of the finest hedge fund records.
  15. Both of these gamblers had very smooth, low variance wealth paths.
  16. Additionally legendary investors such as John Maynard Keynes (0.8 Kelly) running the King’s College Cambridge endowment, George Soros (? Kelly) running the Quantum funds and Warren Buffett (full Kelly) running Berkshire Hathaway had similarly good results but had much more variable wealth paths.
  17. The difference seems to be in the choice of fraction and other risk control measures that relate to true diversification and position size relative to liquid assets under management.
  18. Success in investments has two key pillars: (1) devising a strategy with positive expectation and (2) betting the right amount to balance growth of one’s fortune against the risk of losses
  19. A strategy which has wonderful asymptotic long run properties (1) the log bettor will dominate other strategies with probability one and (2) accumulate unbounded amount more wealth.  But in the short run the strategy can be very risky since it has very low Arrow-Pratt risk aversion.  Z.PNG

Quantitative Introduction to Kelly Criterion

Part I — Expected Value vs Expected Growth

A question I’m often asked is how exactly expected value differs from expected growth. The difference is somewhat subtle but understanding it is essential to risk management in general and the Kelly criterion in particular.

The question frequently arises (as it did here in the the next-to-last paragraph) in the context of the idea that betting one’s entire bankroll implies -100% bankroll growth. (That’s 100% bankroll shrinkage — a bankroll that shrinks to $0.) What’s more, if you bet your entire bankroll in one go, 100% bankroll shrinkage is implied regardless of both the probability of the bet winning (as long as it wins less than 100% of time) and the odds paid out on the bet (as long as the odds are less than infinity).

Think about that for a moment, because it’s an important point: If when you bet you wager your entire bankroll each time then you expect your bankroll to eventually shrink to zero.

Well, at least it should be important, but the truth is doesn’t really get us any closer to understanding what exactly bankroll growth is and how it differs from expected value. We’ll get back to this later.

Let’s start with a brief review of expected value. I described it in relative depth here. And I quote:

The notion of expectation is central to probability and statistics and may be thought of as an average with an extra syllable. If you were to flip a coin 10 times then you could expect it would land on heads 5 times and you could expect it would land on tails 5 times. In reality of course the coin’s not always going to land on heads exactly 5 times out of 10 (in fact it would only do so about 24.6% of the time), but if you were to repeat the experiment (flipping a coin ten times) many, many times over then on average it would land on heads 5 times each trial.The same thought process is also applicable to sports. If the Yankees can be expected to win a particular game 60% of the time, then this would mean that if the exact same game were repeated under the exact same conditions across many, many parallel universes, we would expect the Yankees to win 60% of those encounters.

So let’s say you bet $1 straight up that the Yankees are going to win that game. Now that’s quite obviously a good bet. But just how “good” is it?

That’s where expectations come in with sports betting. If you made the same bet in each of those parallel universes you’d win $1 60% of the time, and lose $1 40% of the time. Now let’s say that there are actually 1,000,000 of these universes. Exactly how much money would you make? Well, in 600,000 of those universes you’d make $1 for a total of $600,000 dollars, and in the remaining 400,000 of those universes you’d lose $1 in each game for a total of $400,000 dollars. So you’d receive $600,000 and would pay out $400,000 meaning that your total profit would be $200,000. Winning $200,000 across 1,000,000 means on average you would have won $200,000 / 1,000,000 games = $.20 per game.

Now of course 1,000,000 is just a made up number in this context. There aren’t really 999,999 other universes where we could make such a bet. This bet can only be made once. But that doesn’t actually matter in the world of statistics. Whether you can make this bet only one time or you can make it multiple times the expectation per game is precisely the same, namely 20%.

So to summarize, the expected value of a bet is the amount we would receive on average if we were to repeat the exact same bet a very large number of times. As such, the expected value of a bet is a a metric by which one might judge the relative attractiveness of that bet. If one bet has an expected value of 5% (meaning that for every $10,000 we bet we would expect to win $500) and another has an expected value of 10% (meaning that for every $10,000 we bet we would expect to win $1,0000), then we would tend to think that one would prefer the latter bet to the former.

But there’s a bit of a difficulty here — namely, expected value ignores any consideration of the relative likelihoods of given outcomes alone. For example a $10,000 bet on a 0.0000000000000000000000000000000000001% likelihood event paying out at +110,000,000,000,000,000,000,000,000,000 ,000,000,000,000 odds corresponds to an expected value of 10% (+$1,000). But who among us would be willing to essentially throw away $10,000 on such a long shot? To put it in perspective you’d be about 1,870 times more likely to win the the New Jersey State Lottery five times in a row, than you would be to win this particular bet. Does it really matter that if by some fluke of nature you actually did win you’d have an unfathomably huge amount of money? If you’re like most people, the answer is probably not.

So now here’s the difficulty … there’s no way whatsoever to account for this very real phenomenon of preferences by appealing to the theory of expected value alone.

(Enter stage right, expected bankroll growth.)

One major problem with the proposed bet is that for most people, $10,000 represents a rather large chunk of one’s bankroll to be throwing away on a bet that’s nearly certain to lose. But while a $10,000 bet is probably too large a quantity to risk on this bet, there’s still a sufficiently small dollar amount that most people would be willing to risk to make this bet. Granted, for most people that dollar amount would be somewhere in the neighborhood of a tiny fraction of a penny, but it nevertheless would still be a positive dollar amount.

The fundamental issue with bets such as these is that, despite being positive EV, placing them is an excellent way to go broke. The apparent contradiction is easily reconciled. If you were to repeat this bet once in each of a gigantically huge number of parallel universes, in nearly all of the universes you’d lose your bet, but in a tiny, tiny, tiny, tiny, tiny fraction of those universes you’d have win the bet and that win quantity would make up for all the losses plus an additional 10% of the amount risked.

The fact is that most people just aren’t willing to live through billions of trillions worth of bets just to have a vanishingly minuscule probability of winning a huge odds bet once. So while the bet may have positive expected value, the expected outcome is for your bankroll to shrink by $10,000 each time the bet’s made. If your bankroll were $1,000,000 and you made the bet 100 times, you could expect to be broke after the 100th bet (even though your expected value would be 10% × $1,000,000 = +$100,000).

So let’s look at some more practical numbers. Assume you’re considering at a bet that wins with 50% probability and pays out at odds of +200. Further assume your total bankroll is $100,000 and that you want to place 1% of your bankroll on this wager.

Question: Where do you expect your bankroll to be after 2 wagers?

Answer: There are 4 possible outcomes after placing two wagers:

  1. Win both bets.
  2. Win 1st bet, lose 2nd bet
  3. Lose 1st bet, win 2nd bet
  4. Lose both bets

Now because winning and losing the bet are both equally likely, all 4 outcomes occur with equal probability, namely 25%. Recall that you’d be betting 1% of your bankroll on each bet and would be paid off at odds of +200. Therefore, your ending bankroll under each of the 4 outcomes would be:

  1. B = $100,000 × (1 + 2×1%) × (1 + 2×1%) = $104,040
  2. B = $100,000 × (1 + 2×1%) × (1 – 1%) = $100,980
  3. B = $100,000 × (1 – 1%) × (1 + 2×1%) = $100,980
  4. B = $100,000 × (1 – 1%) × (1 – 1%) = $98,010

(The derivation of these equations is simple. Every time you win your bankroll would grow to 102% of its previous value, and every time you lose your bankroll would shrink to 99%.) The expected value from betting in this manner would be 25%×$104,040 + 25%×$100,980 + 25%×$100,980 + 25%×$98,010 = $101,002.50. To calculate expected growth, we would first need to recognize that given our 50% win probability, our expected outcome would be to win a bet and to lose a bet (# of wins = 50% × 2 bets, # of losses = 50% × 2 bets). Therefore our expected growth would be that associated with that outcome (with expected growth, the relative ordering of wins/losses is irrelevant), namely $100,980.

Therefore, the expected value from the two bets is $1,002.50 or 1.0025%, and the expected growth is $980 or 0.9800%. Notice that expected value is higher than expected growth — this is what you’re always going to see. Expected value will always be higher than expected growth (except for probabilities of 0 or 100%, we’ll they’ll be equal) because a few relatively large, relatively uncommon outcomes will increase EV. Another way to think about this is by realizing that the worst case scenario is losing everything one time over., while the best case scenario would be winning your bankroll infinity times over – in other words you while your maximum possible profit is unlimited, your maximum possible loss is limited to your bankroll.

So in this instance our expected outcome would be a bankroll of:

B* = $100,000 × (1 + 2×1%)2×50% × (1 – 1%)2×50% = $100,980,

implying expected bankroll growth of

E(G) = $100,980/$100,000 = 0.9800%

It should be readily apparent our expected outcome after n bets would be a bankroll of:

B* = $100,000 × (1 + 2×1%)n×50% × (1 – 1%)n×50% = $100,000 × (100.48881%)n,

implying expected bankroll growth of

E(G) = (100.48881%)n -1.

By extension, our expected outcome after just 1 bet would be:

B* = $100,000 × (1 + 2×1%)50% × (1 – 1%)50% = $100,488.81

And our expected bankroll growth would be

E(G)= (1 + 2×1%)50% × (1 – 1%)50% – 1 = 0.48881%

(This last result bears a little discussion. We can talk about expected growth after only 1 bet in the same manner as we can talk about expected value after just one bet. In the same way as we’d never see a real result equal to our expected value, we’d never actually see growth after one bet equal to expected growth. This should cause absolutely no concern.)

So let’s generalize our results with expected outcomes and growth. Given a starting bankroll of B0, decimal odds of O, a win probability of p, and a bet size of X (as a percentage of starting bankroll, B0), the bankroll associated with the expected outcome from placing the bet would be:

B* = B0 * (1 + (O-1) * X)p * (1 – X)1-p

And expected growth would be:

E(G) = (1 + (O-1) * X)p * (1 – X)1-p – 1

Expected value, you’ll recall, would be:

EV = p*(O-1)*X – (1-p)*X = (pO – 1)*X

Q: So let’s look at a concrete example: What are the expected value and bankroll growth associated with a bet equal to 1% of bankroll paying out at -110 and winning with probability 54%?

A:

EV = 1% × (54% × 1.909091 – 1) = 0.03091% of bankroll
E(G) = (1 + 0.909091 × 1%)54% × (1 – 1%)46% – 1 = 0.02638% bankroll growth.

Q: Now let’s consider the same terms, but in the case of a player placing a bet 25% of bankroll. What would expected value and growth be in this case?

A:

EV = 25% × (54% × 1.909091 – 1) = 0.7727% of bankroll
E(G) = (1 + 0.909091 × 25%)54% × (1 – 25%)46% – 1 = 2.1510% bankroll growth = 2.1510% bankroll shrinkage

So think about these results for a moment. We have a positive expectation bet and hence, quite naturally, the more we bet on it the more we expect to make. However, if we were to wager too much on this bet then we’d expect our bankroll to shrink by 2.1510% per wager (were we to place this positive expectation bet 32 times, for example, we’d expect our bankroll to depreciate roughly a half).

So this should help elucidate the huge odds bet above. No matter how positive EV a bet might be, if you bet too much on it then you expect your bankroll to shrink. This is the concept to which people are referring when they talk about “money management”. Even if you could pick NFL spreads at 75% (which you can’t), were you to bet too much, you’d expect to head towards bankruptcy.

So as a limiting case let’s look at one more example, the example of betting one’s entire bankroll mentioned at this start of this article: win probability = p, bet size = 100% of bankroll.

EV = 100% × (pO – 1) = pO – 1 (EV > 0 for p > 1/O)
E(G) = (1 + (O – 1))p × (0)1-p – 1 = -100% (for p < 1 and O < ∞)

So what does this tell us? Well for one thing it tells us that even if you were the “best handicapper ever”, were you to risk your entire bankroll on every bet, you would expect to go broke. More generally, it illustrates the concept that looking solely at expected value as a metric for the attractiveness of a given bet is not the proper way to maintain long term growth.

In the next part of this article we’ll discuss how one might use the concept of expected growth to determine bet size. This is the essence of the Kelly criterion.

Part II — Maximizing Expected Growth

In Part I of this article we introduced the concept of expected growth, where we discussed why a bettor might reasonably choose to gauge the relative attractiveness of a given bet by considering its expected growth. In Part II of the series we’ll look at how a bettor might use the notion of expected growth to determine how large a bet to place on a given event. This is the very essence of the Kelly criterion.

There are two extremes when it comes to placing positive expectation bets. On the one hand you have people like my aunt, who’s so afraid of risk that I doubt she’d even bet the sun would rise tomorrow (“But what if it didn’t? I could lose a lot of money!”). On the other hand you have people like my old college buddy Will, whose gambling motto was “Get an advantage, and then push it.”

One Saturday night during the spring term of my sophomore year, Will decided he was going to run a craps game. He put the word out to a number of the bigger trust fund kids and associated hangers-on and let the dice fly. After maybe 4 or 5 hours, Will was up close to $8,000, which was far from an insignificant amount for us at the time. One player, an uppity gap-toothed British guy named Dudley, whose own losses accounted for most of that $8K, loudly proclaimed that he was sick of playing for small stakes and wanted some “real” action. He told Will he was looking to bet $15,000 on one series of rolls. Will paused for a moment and then quickly agreed. He just couldn’t back down from the challenge. It didn’t matter that this represented all of Will’s spending money for the entire semester — the odds were in his favor and he knew it and as far as he was concerned the choice was clear.

So what happened? Well to make a long story short, the guy picked up the dice and without a word silently rolled himself an 11. Will paid him the next Monday and wound up having to work at the campus bookstore for the rest of the semester. I remember a few weeks later I ran into Will at work and we got to talking while he moved boxes around trying to look busy. I asked him if he and Dudley and were still friends.

“Sure,” he said, “But the guy’s a moron. Didn’t he realize the odds were in my favor?”

So there you have it. Will was quick to label Dudley a moron because he made a negative EV bet. What Will failed to realize, however, was that this guy certainly had the means to make $15,000 bets, and ultimately wouldn’t have been all that impacted by the result were he to have lost. Will on the other hand, had no business making a $15,000 bet that he stood to lose close to half the time. It didn’t matter that if he made the same bet 10,000 times over he’d almost certainly have come out well ahead, it only took making the bet one time to bankrupt him for the semester and render him incapable of staking any more craps games at all.

Dudley might very well have been foolish for having offered to make the negative EV bet, but Will on the other hand was foolish for having risked such a large chunk of his bankroll on the positive EV bet in the first place. Never mind that losing the bet forced Will to work in the bookstore, never mind that losing the bet forced Will to switch from his Heineken bottles to Milwaukee’s Best cans, losing the bet had probably the worst effect possible on an advantage bettor – decimating his bankroll.

Hopefully, this example helps illustrate a key concept that was touched on in the last article. Specifically, that expected value and expected growth are both key components of proper long-term wagering. Most bettors instinctively recognize the importance of expected value – most everyone realizes that betting 2-1 odds on a fair coin flip is “smart”, while betting 1-2 odds on a fair coin flip is not. But very few people consider as much as they should the expected growth of their bankroll due to their wagers they make. When a bettor places too much importance on the expected value and not enough on expected growth, he puts himself in danger of winding up in the same predicament as Will – pushing around boxes at the Brown Bookstore and trying to look busy, despite having made a indisputably “smart” bet when only considering EV alone.

But let’s go back to Will’s initial decision to make the $15,000 bet. Certainly it’s pretty clear that making the bet was a mistake, but it should also be clear that because the bet had positive EV there was obviously a certain (lower) risk amount for which Will would have been making the right decision in accepting the wager. For a person with unlimited access to funds, the decision of how much to bet on a positive EV wager is easy – bet as much as possible. But for a person with a limited bankroll who wants to survive until the next day so he can continue staking craps games, the decision isn’t quite so obvious. That’s where Kelly comes in.

You’ll recall from Part I of this article the equation for expected growth:

E(G) = (1 + (O-1) * X)p * (1 – X)1-p – 1

Where X represents the percentage of bankroll wagered on the given bet and O the decimal odds.

For a player like Will, who has his basic necessities already paid for (food, shelter, clothing), his only real goal is to grow his bankroll as much and as quickly as possible. As such, Will’s objective would be to maximize the expected growth of his bankroll. The size of the bet (always given as a percentage of the player’s total bankroll) is known as the “Kelly Stake” and is a function of the bet’s payout odds and either win probability or edge1.

Mathematically , the formula for the Kelly stake is derived using calculus2. The actual mechanics are rather unimportant, but the result is that in order to maximize the growth of one’s bankroll when placing only one bet at a time, one should bet a percentage of bankroll equal to edge divided by decimal odds minus 1. (This is assuming the player has a positive edge. If he doesn’t his optimal bet is zero.) In other words:

Kelly Stake as percentage of bankroll = Edge / (Odds – 1) for Edge ≥ 0

Put in terms of win probability the equation becomes:3

Kelly Stake as percentage of bankroll = (Prob * Odds – 1) / (Odds – 1) for Probability * Odds ≥ 1

Let’s take a look at a few examples:

  1. Given a bankroll of $10,000 and an edge of 5%, then on a bet at odds of +100 one should wager 5% / (2-1) = 5% of bankroll, or $500.
  2. Given a bankroll of $10,000 and a win probability of 55%, then on a bet at odds of -110, one should wager $10,000 * (55% * 1.909091 – 1) / (1.909091-1) = 5.5% of bankroll, or $550.
  3. Given a bankroll of $10,000 and a win probability of 25% then on a bet at odds of +350, one should wager $10,000 * (25% * 4.5 – 1) / (4.5-1) ≈ 3.57% of bankroll, or about $357.
  4. Given a bankroll of $10,000 and a win probability of 70% then on a bet at odds of -250, one should not wager anything because edge = win prob*odds = 70%*1.4 = 98% < 1.

Let’s look at all this a little more closely. Consider a bet at even odds (decimal: 2.0000) — in this case, the bankroll growth maximizing Kelly equation simplifies to:

K(even odds) = Edge/(2-1) = Edge for Edge ≥ 0

In other words, when betting at even odds, the expected bankroll growth maximizing bet is equal to the percent edge on that bet. So if you have an edge of 5% on a bet at +100, then you should be wagering 5% of your bankroll. If your edge were only 2.5% then you should be wagering 2.5% of your bankroll. Now let’s consider a bet at -200, or decimal odds of 1.5:

K(-200 odds) = Edge/(1.5-1) = 2*Edge for Edge ≥ 0

So this means that for a bet at -200, the expected bankroll growth maximizing bet size would be twice the edge on the bet. Similarly, for a bet at -300, one should bet three times the edge, and for a bet at -1,000 one should bet ten times the edge.

This fits rather well with the manner in which many players size their relative bets on favorites. For a bet at a given edge if they were to bet $100 at +100, they’d bet $150 at -150, $200 at -200, $250 at -250, etc.

Now let’s consider bets on underdogs (that is, bets on money line underdogs — bets paying greater than even odds). In the case of a bet at +200:

K(+200 odds) = Edge/(3-1) = ½*Edge for Edge ≥ 0

The optimal bet size is only half the edge. Similarly at a line of +300, the optimal bet size would be a third of edge, at +400 a quarter the edge, etc.

Now this is quite different from the manner in which many players choose to structure their underdog bets. If they were to bet $100 on a line of +100, they might also bet $100 on a bet with the same edge at +400. For a player wanting to maximize his bankroll growth, this is inappropriate behavior because it attributes, relatively , excessively large amounts to underdog bets. Assuming constant EV an expected growth maximizing player should only bet half of his +100 bet size at +200, and only a quarter his +100 bet size at +4004.

So what we see in the case of any bet (be it on an underdog or a favorite) is that the player should bet an amount such that the percentage of his bankroll he stands to win is the same as his percent edge. In other words, a player betting at an edge of 2% should place a bet to win 2% of his bankroll. This means that at -200 he’d be risking 4% of his bankroll, while at +200 he’d only be risking 1% of his bankroll. The rationale behind this should be clear when you consider the following example:

For a player betting at an edge of 5% and odds of -200, the proper Kelly stake is 10%. Over 100 bets, he has an expected return of 64.7% with a 36.7% probability of not turning a profit and a 3.4% probability of losing two-thirds or more of his stake.

For a player betting at the same 5% edge but at odds of +400, were he to bet the 10% stake of the -200 player, while he’d have the identical 64.7% expectation, he’d have a 73.5% probability of no profit, while his probability of losing two-thirds or more of his stake would be 55.8%.

Generalizing, for two same-sized bets of equivalent (positive) EV repeatedly made over time, there’s a higher probability associated with losing a given amount of money when making the longer odds bet.

Once again, we keep returning to the same simple but often overlooked point – expected value isn’t everything. Due to the fact that longer odds (for a given edge) imply greater a probability of loss, the Kelly bettor will bet less on longer odds and more on shorter odds. Any time an advantage player loses money he’s giving up opportunity cost as that represents money he can’t wager on +EV propositions down the line. As such the Kelly player will (for a given edge) always seek to minimize his loss probability over time by selecting the shorter odds bet, even though that necessitates risking more to win the same amount.

Taking the logic a step further, a Kelly player should be willing to even accept lower edge in order to play at shorter odds. For example:

  • At odds of -200 (decimal:1.500) and an edge of 4%, the win probability would be p = (1+4%)/1.5 ≈ 69.33%, and Kelly stake would be K = 4%/(1.5-1) = 8%. This represents expected bankroll growth of:
    (1+(1.5-1)*8%)69.33%*(1-8%)1-69.33% -1 ≈ 0.1624%
  • At odds of +400 (decimal: 5.0000) and an edge of 10%, the win probability would be p = (1+10%)/5 = 22%, and Kelly stake would be K = 10%/(5-1) = 2.5%. This represents expected bankroll growth of:
    (1+(5-1)*2.5%)22%*(1-2.5%)1-22% -1 ≈ 0.1221%

So what this tells us is that a Kelly player would prefer (and by a decent margin) 4% edge at -200 to 10% edge at +400.

In this article we’ve introduced Kelly staking. This represents a methodology for sizing bets in order to maximize the expected future growth rate of a bankroll5. The bet sizes determined by Kelly will necessarily not maximize expected value, because doing so would require betting one’s entire bankroll on every positive EV wager that presented itself. This would eventually lead to bankruptcy and the inability to place further positive EV wagers.

We’ve seen that Kelly may also be utilized to gauge the relative attractiveness of several bets. What we see is that for a given edge, an expected growth maximizing bettor will prefer the bet with shorter odds (in other words, the bigger favorite). This result, derived entirely from first principles, may be surprising to some advantage players who’ve come to find wagers on underdogs generally more profitable than bets on favorites. While our conclusion in no way precludes the possibility that underdogs may in general provide superior return opportunities than favorites, the fact that for two bets of equal expected return the bet on the favorite will yield greater expected bankroll growth is indisputable and needs to be acknowledged by all those seeking to manage bankroll risk.

In Part III of this series we’ll discuss how one may generalize Kelly so it may be applied to a greater range of circumstances including multiple simultaneous bets, multi-way mutually exclusive outcomes, and hedging.


FootNotes:

  1. Technically, because odds, edge, and win probability are linked by way of the equality Odds * Prob = 1 + Edge, any two of these variables could be used to determine the Kelly stake.
  2. The calculus is rather simple. We need to maximize E(G) = (1 + (O-1) * X)p * (1 – X)1-p – 1 with respect to X, subject to X lying on the unit interval [0,1]. To simplify the analysis, however, we can take the natural log of both sides of the equality and seek to maximize the log of expected growth. This is equivalent because the log function is monotonically increasing. So our problem becomes:
    Maximize wrt X:
    log(Growth) = p*log(1 + (O-1) * X) + (1-p)*log(1 – X)
    s.t. 0 ≤ X ≤ 1which gives us:
    dlog(G)/dX = p*(O-1)/(1 + (O-1) * X) – (1-p)/(1 – X)

    setting to zero and solving yields:
    X = (Op-1)/(O-1)

    with d2log(G)/dX2 ≤ 0
    for all feasible 0 ≤ X < 1

  3. This may also be extended to include bets that include a third push outcome where the at-risk amount is returned to the bettor in full (such as in the case of an integer spread or total). In order to generalize this article to include bets with ternary outcomes, one need only consider the “probability of winning conditioned on not pushing” instead of pure “win probability”.In general, given a win probability of PW, a loss probability of PL, and a push probability of PT (where PW + PL + PT = 1), then the probability of winning conditioned on not pushing would be:
    P*W = PW / (1 – PT)

    and the probability of losing conditioned on not pushing would be:

    P*L = PL / (1 – PT)

    So assuming decimal odds of O, Edge would be:

    Edge = O × PW / (1 – PT) – 1
    -or-
    Edge = O × PW – (1 – PT)

    which in either case is just the same as:

    Edge = O × P*W – 1

    And the Kelly stake would remain unchanged as:

    Kelly Stake as percentage of bankroll = Edge / (Odds – 1) for Edge ≥ 0
  4. So why do so few players do this? It’s my opinion that the only explanation for this inconsistent behavior (risking the same amount on all underdogs while betting to win the same amount on favorites) is the manner in which US-style odds are quoted. Odds of -200 imply one would need to bet $200 to win $100 so it would seem to make sense to bet in increments of that $200. Odds of +200 imply one would need to bet $100 to win $200, and so it would seem to make sense to bet in increments of that $100. What if, however, US odds on under dogs were also quotes as negative numbers? What if a +200 underdog were written as a -50 underdog (meaning a player would need to risk $50 to win $100) and a +400 dog as a -25 dog? The two methods for expressing odds are obviously identical, but it’s my belief that if odds were quoted in this manner you’d have far fewer bettors undertaking the questionable practice of betting an equivalent dollar amount on all underdogs.
  5. An equivalent way of looking at this is that Kelly maximizes both the bettor’s median and modal future bankroll over a large number of bets. In other words, applying expected bankroll growth to the current bankroll yields both most likely bankroll outcome (the mode) and the outcome which has an equal likelihood of being outperformed and underperformed.
  6. Courtesy: SBRForum.com

Applying Kelly Criterion by Ray Dillinger

Introduction to Kelly Criterion

Kelly’s formula is a theoretical benchmark for deciding the appropriate position size when investing, trading or gambling. A divergence in attitude towards this theory illustrates the disconnect between academicians and practitioners, and the necessity of closer collaboration between the two circles.

To understand the essence of Kelly’s formula, let us consider the question: Can one lose money in a game in which one has a favorable probability of winning? The answer is, absolutely yes. To see why, think of the simple game of tossing a biased coin: heads means that the player wins the bet, and tail that he loses. Even when the player has a 90% winning probability, if he bets all he has every time, then sooner or later he will inevitably encounter one losing game and go bankrupt. This is a simplified example of gambler’s ruin. Yet, since the odds are so much in favor of the player, it is unreasonable not to play. The reasonable middle ground between not playing and playing is to come up with an optimal bet size. Kelly’s formula determines such an optimal size.

Sketching the log return per play in such a game as a function of the bet size (as a percentage f of the total wealth of the player), we get the picture in the figure below.

Z.PNG

The log return function is essentially what Kelly employed to solve for the unique optimal betting percentage. In our example we can observe the theoretical optimal bet size to be 80%. In general, in such a coin-toss game, if the probability of winning and losing are p and q=1-p, respectively, then Kelly’s formula tells us the theoretical optimal betting size should be

p-q,

or 2p-1.

More interesting is that Kelly was motivated to explain Shannon’s information rate in information theory. As a result, the average optimal log return at the Kelly’s bet size is actually proportional to the information rate, and can be viewed as the information in this game favorable to the player. The subsequent development of this important result within the circles of practitioners and academicians is rather interesting.

Practitioners regard the Kelly bet size as a red line that should never be crossed. In fact, experienced traders and investors have long known the importance of being conservative in allocating capital into risky assets, even without knowing the Kelly’s formula. They have learned from trial and error — although many seem to have forgotten this principle during the past two decades.

Coming back to our coin toss game, if we used the bet size of 80%, then (with probability 0.1) in one losing game you could loss 80% of your total wealth. If you are unlucky and encounter two losing games consecutively, the total loss will be 96%. Gamblers with a little common sense don’t need any formula to know this is too risky. Thus, for practitioners, Kelly’s formula provides a useful guide for the upper bound of allocating capital to the risky assets. The emphasis has always been how to reduce the risk from there.

Edward Thorp, a mathematics professor turned legendary blackjack player and the pioneer of the basic system for playing blackjack, was a leading practitioner of the Kelly’s formula. He first applied Kelly’s formula in managing bet size in blackjack and later generalized the principle to money management in trading. Thorp’s view is representative of that of a practitioner. Recently, answering a question on whether he uses the Kelly’s formula in asset allocation, he replied,”.. if you bet half the Kelly amount, you get about three-quarters of the return with half the volatility. So it is much more comfortable to trade. I believe that betting half Kelly is psychologically much better…”

Indeed, recent research of Vince and Zhu shows that when incorporating the practical consideration of adjusting for risk, and realizing that one only gambles over a finite horizon of time, Kelly’s suggested bet size needs to be adjusted down considerably. This confirms what practitioners have long been doing in the real world.

For many decades, Kelly’s formula was dismissed as irrelevant by leading academics. These included Nobel Prize Laureate Paul Samuelson, who went as far as calling the Kelly criterion a fallacy, and derided it in an article consisting entirely of one-syllable words (presumably so that his benighted opponents could understand it). This led to the bitter Samuelson controversy, which positioned Markowitz’s portfolio theory against the Growth Optimal Portfolio (GOP), each side claiming superiority over the other.

This controversy seems fueled more by partisanship (economists vs. mathematicians) than by practical considerations. Some of the most successful investors have reportedly applied the Kelly criterion, including Warren Buffett and Bill Gross. The caveat is that to construct and to manage the growth portfolio (or alternatively the Markowitz portfolio), one needs to know the joint probability distribution of the price processes of all the assets involved. This is of course a tall order. In practice, all one has to work with is the price history of the assets involved, which is used to estimate the true joint probability distribution. However, searching for optimal strategies through historical simulations leads to the trap of backtest overfitting  (see the paper Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance).

These two tales of Kelly’s formula illustrate the dangers of naively following theoretical financial results in investment practice. This is not to say that theoretical results are not useful. For example, the Sharpe ratio, which was developed in the context of Markowitz’s portfolio theory, has become an industry standard in measuring the (past) performance of mutual funds and hedge funds. Similar usage of the idea of the growth portfolio is also useful, especially given the close relationship between the Kelly formula and information theory. As always, caveat emptor!

Courtesy: Two tales of the Kelly Formula

Analysis of Multiple Simultaneous Non-Independent Investment Opportunities with Multiple Possible Outcomes

There are a lot of papers that have been published about figuring out how much money to invest in a business opportunity given your level of wealth, its odds of success, and its estimated returns in the events of success and failure. Usually the further assumption is made that your decision to invest or not doesn’t affect the price of the issue or the odds of success.

Frequently I’ve seen papers analyzing two or more such opportunities in terms of finding an optimal strategy for investing in both, and these are good work too as far as they go; they’re accurate in the way that physicists models that involve frictionless surfaces and model planetary masses as a point are accurate; within the limitations of their simplifying assumptions.

The problem is, that’s not how business opportunities look in the real world.

In this paper I’m going to review the math that governs the simple cases and then move on to introduce the math that governs the more complicated cases.

Everybody knows, or everybody should know, the Kelly Criterion for making optimal investments in a single two-valued opportunity. It’s very simple; you invest your wealth, times the edge, divided by the variance of the bet.

“The edge”, in this case, is the return you’d get per dollar invested in the average case.

So let’s say I offer you a simple coin toss where I offer to double the money you bet on it if you win the toss. Since you win or lose exactly the amount bet, the edge is zero. The Kelly Criterion says you don’t take this bet because there is no long-term edge to justify the risk of losing your money.

So I’ll get a little stupid in order to make a point, and make it sweeter. Let’s say I triple your money if you win and take your money if you lose. Now you’ll win one bet of every two and lose one bet of every two, winning twice the amount bet and losing the amount bet. So, you come out ahead by the amount bet every two bets, and the edge is 1/2. The variance though is bigger; if you can be up two or down one times the amount bet, the difference between them, or the variance, is three. Since 1/2 divided by 3 is 1/6, the Kelly Criterion says you bet one-sixth of your money on this bet.

Let’s make it even sweeter. Now I offer to quadruple your money if you win (I must be really amazingly stupid to make you such an offer) and take your money if you lose. Now your edge is 3/2 since, on average, you get back one and a half times the amount bet. But the variance is 5. Since 3/2 divided by five is 3/10, the Kelly Criterion says you bet three-tenths of your money on this bet.

Now let’s get ridiculous. Let’s say I offer you a hundred times your money back if you win a coin toss, and take your money if you lose. Now how much should you risk? Well, your edge is now 99/2 and the variance is 100. So you wind up betting 99/200 of your money on this bet.

By now you’ve probably noticed the point I’m heading for: No matter how big your edge gets, the Kelly Criterion says you never *EVER* invest a fraction of your wealth greater than the probability of losing it. Even if you are fifty percent likely to win a billion dollars for every dollar bet, you don’t bet more than half your money. State lotteries that cost a dollar and have chances of winning of one in ten million are bad investments for anyone who has less than ten million dollars, no matter how many billions large the jackpot may be.

It turns out that the Kelly Criterion tells you EXACTLY how much of your wealth you should invest in order to maximize long-term growth of your money. If you bet more, you have more risk but don’t make as much money. If you bet less, you have less risk and don’t make as much money. Now, if you need to actually take income out of your investing wealth every so often, then you should be investing less than the Kelly Criterion says; the money you take out isn’t contributing to longer term growth, so it doesn’t justify as much risk as the Kelly Criterion accepts. But there is never, under any circumstances, any reason to bet more than the Kelly Criterion suggests; in terms of money management, the Kelly Criterion is the bright clear line between aggressive long-term investing that undertakes exactly as much risk as necessary to absolutely maximize growth, and insane investing that accepts more risk than needed and by doing so impairs the long-term growth of funds.

All this is very simple when there are just two possible outcomes. More gamblers than investors know the Kelly Criterion, because they’re more familiar with the simplified, two-valued kind of investment opportunities where it’s easy to calculate.

But real business opportunities don’t look like that.

How do you calculate the Kelly Criterion when you’re looking at Acme Widgets and the government is seeking bids on a big widget contract, and you figure they have about

  • a 15% chance of landing the contract and making a 50% return,
  • a 20% chance of being a supplier to the company that lands the contract and making a 30% return,
  • a 55% chance of having no contract awarded and doing business as usual making a 10% return, and
  • a 10% chance of having a competitor get the contract and losing 70% of the money invested?

What’s the most you should put into this company? The math is a bit more complicated now, and there isn’t a straightforward way to find an answer. But there is a straightforward way (well, only mildly complicated) to check how good a possible answer is.

What the Kelly Criterion does is to maximize the logarithm of the expected wealth. By maximizing the logarithm repeatedly and compounding your earnings, maximum growth of wealth is achieved. So, while it’s no longer straightforward to directly calculate the Kelly threshold for this more complicated situation, you can still iteratively maximize the logarithm of expected wealth to find the optimal Kelly-Criterion investment. Here’s an example.

Let’s say you have a million dollars to manage.

The natural logarithm of 1000000 is 13.8155, so that’s the benchmark for making no investment at all.

Now, if you contemplate putting all of your money into acme widgets, then you have to figure the different outcomes and likelihoods and take the weighted average of their logarithms. So….

0.15 * ln(1000000 * 1.50) +

0.20 * ln(1000000 * 1.30) +

0.55 * ln(1000000 * 1.10) +

0.10 * ln(1000000 * 0.30) = 13.8608.

Since making no investment at all gave a logarithm of 13.8155, investing all your money in Acme Widgets is seen as being better than investing none of it.

But is that all there is to the story? What if you only invest half your money?

0.15 * ln (500000 + 500000 * 1.50) +

0.20 * ln (500000 + 500000 * 1.30) +

0.55 * ln (500000 + 500000 * 1.10) +

0.10 * ln (500000 + 500000 * 0.30) = 13.8607.

This is very close to being as good as investing all your money. Let’s try three-quarters:

0.15 * ln (250000 + 750000 * 1.50) +

0.20 * ln (250000 + 750000 * 1.30) +

0.55 * ln (250000 + 750000 * 1.10) +

0.10 * ln (250000 + 750000 * 0.30) = 13.8692.

That’s better than either all or half, so let’s see what happens if we invest seven-eighths of our wealth, which is halfway between the two best scores we’ve seen so far:

0.15 * ln (125000 + 875000 * 1.50) +

0.20 * ln (125000 + 875000 * 1.30) +

0.55 * ln (125000 + 875000 * 1.10) +

0.10 * ln (125000 + 875000 * 0.30) = 13.8679.

That’s not as good as investing three-quarters of our wealth, so it’s too much. We can back off a little bit and try investing 13/16 of our wealth:

0.15 * ln (187500 + 812500 * 1.50) +

0.20 * ln (187500 + 812500 * 1.30) +

0.55 * ln (187500 + 812500 * 1.10) +

0.10 * ln (187500 + 812500 * 0.30) = 13.8691.

And this is the best score we’ve seen so far. The optimal amount to invest is going to be right around here somewhere; we could carry this regression out ten more steps and have it correct to within one part in about 20000. But we don’t have to; I’ve just shown the first few steps to illustrate the process.

The problem is real business opportunities don’t look like that either.

In the real world, you’re never looking at a situation where you’re deciding how much money to put into your only investment opportunity. At the very least, the money you don’t invest in that opportunity may usefully be placed in a risk-free investment like gold or a low-risk investment like Treasury Bills. You should also be considering Acme Widgets’ competitor, the Klein Brush & Bottle Company, because if Acme doesn’t get that contract, Klein is a whole lot more likely than otherwise to get it. You know they won’t both get the contract, although they may both be suppliers if someone else gets it. And you also know that if you invest in both companies, you run less risk because the downside risk at Acme is coincident with a much higher probability of a high return at Klein and vice versa. And finally, since Acme is a fairly small company and you’re looking at a medium-sized fund, the amount you invest may drive up the price you have to pay for its stock, which will drive down your effective return.

But, the technique above turns out to be something you can generalize:

The General Form of the Kelly Criterion is:

Sum for all X of (probability of X * ln (ending wealth if X happens))

This is how you can calculate the degree to which your growth opportunities are being maximized. Now, if you’ve done a lot of math, you’re already looking at the generalized form of the Kelly Criterion, above, and setting up integrals in your head to deal with continuous probability distribution functions and reward levels and differentials to help find the optimum points, but it turns out that depending on what the probability and return formulas are like, it may not be generally or easily integrable or differentiable. In fact it’s usually not.

This is a most excellent formula, because you can use it to evaluate investment strategies involving making lots of different investments simultaneously: For example, you might try different investment levels in Acme and Klein and Gold, and account for such things as the difference in the tax bite that depend on how well you do.

But this complicates things, because if we substitute in complicated formulas that depend on our investment for the probability of X, and we substitute in complicated formulas that depend on our investment for the rate of return if X happens, we wind up with nonmonotonic functions in multiple variables.

And with nonmonotonic functions in multiple variables, you can’t easily just “home in on it” the way I did above, because the function may have several local maxima, local minima, and discontinuities.

Optimization as Search and Simplifying Assumptions

In this case, optimization becomes a search, and the more complex the set of outcomes you’re looking at and the greater the number of investments you’re trying to optimize the distribution of money between, the harder the search becomes. Here is where you make a lot of simplifying assumptions, aggregating companies into industries and risk profiles to try to reduce the number of variables you have to work with. Here is where you assume things operate independently, even though sometimes they may not, because analysis of independent variables can be carried out separately from each other.

But very complex search spaces are in fact, what genetic algorithms, stochastic searches, and multivariate regressions are for, and computer code can be your tool to cut through a whole lot of fog here seeking the best investment levels in all these different opportunities.

Usually you have to pick and choose which simplifying assumptions you’re making and which you’re throwing out. Analyzing the situation of Acme and Klein and that big contract, clearly you shouldn’t assume that they’re independent. But analyzing, say, an oil-rig firefighting company and a boot and shoe dealer, you can be pretty comfortable assuming that their relative performances have nothing to do with each other. It may turn out that the Boot and Shoe Company makes a lot of its money manufacturing protective boots for the rig firefighters so it may not be true — but it’s a pretty comfortable assumption, and I’d make it in a heartbeat. There’s really no way you can capture every detail of every possible interdependence; you just have to wind up ignoring some of them.

The Value of Conservative Assumptions

Remember what I said earlier about the Kelly Criterion being the clear bright line between aggressive investing and insane investing?

Z.PNG

If you overestimate the amount you should invest, you expose yourself to more risk, and simultaneously reduce the long-term growth of your wealth. That is insane. If you underestimate the amount you should invest, you make less money, which is bad, but you also expose yourself to less risk and adjust for eventually taking some income from your wealth, which is good. Investing more than the Kelly Criterion says is clearly insane, but there are good reasons why most people should want to invest less.

That is why conservative assumptions — those which would lead you to invest less — are generally better than assumptions which would lead you to invest more. It’s clear that accurate assumptions are the best assumptions of all, but when you are forced to deviate from accuracy, it’s best to deviate in a conservative direction.

The Art of Picking Your Assumptions

And this is where hard math meets art, science, and experience. We have a tool we can use to analyze any investment strategy, under a given set of assumptions. But we have to make assumptions which we know aren’t completely true all the time to control the complexity of the analysis, and the assumptions we make and don’t make govern the accuracy of our analysis. And this, traditionally, is the part of securities analysis you can’t automate; there’s no way an automaton can adjust for things it just plain doesn’t know.

Closing The Loop

Or is there? We have been talking about optimizing systems based on predictions; and we’re already used to the idea of optimizing ex ante prediction systems based on ex post performance. What if we make a dozen different systems that use a dozen different sets of assumptions, and turn them all loose trying to figure out optimal investment strategies in hundreds or thousands of ex ante scenarios drawn from real life? Then we could come back with the ex post performance numbers from those scenarios and figure out how well each of the robot portfolio managers would have done.

Clearly, the robot whose portfolio did the best must, ipso facto, have been the one whose predictions were most useful on the scenarios presented.

Now, what if we do it iteratively, performing a cluster analysis on the ex-ante information and ex-post performance pairs to find out which set of assumptions tends to do best in what clusters? As a cluster analysis on regression criteria, it’s going to be an expensive computation; it could tie up a good workstation for several weeks. But the results would continue to be useful for years.

Conclusion

But anyway, that is a topic for a different paper. In this paper I have introduced the Kelly Criterion itself and how to apply it in complicated situations. This creates a need to make simplifying assumptions, but they don’t have to be the same assumptions that the authors of so many other papers have made. My point is that you have to pick which simplifying assumptions you make based on the business situations you’re presented with, and you can frequently do better in some situations by using different sets of assumptions more appropriate to those situations.

But picking the assumptions is not, as usually presented, a problem that completely defies analysis either. There is a possibility of “closing the loop” and creating a system that picks and chooses its assumptions without human input, based on the business situations presented.

-By Ray Dillinger

The Expert Gambler, Risk Intelligence and Dylan Evans

“The race is not always to the swift, nor the battle to the strong, but that’s the way to bet.”

Disclaimer: If you have been trading for any length of time and have never heard about Risk Intelligence, and an Expert Gambler’s attitude, this article would change your life.

First of all watch this video, featuring Dylan Evans PhD.

Now here is an excerpt from a rare interview that Dylan Evans gave to Alistair Evans of QBasis Invest.

A discussion of risk, intelligence and trading

Are some individuals simply better at understanding and analysing risks and estimating probabilities than others? Alistair Evans of Qbasis Invest interviews Dr Dylan Evans, a pioneer in the concept of risk intelligence in individuals, who argues that this is the case and that firms that actively manage risk can benefit from understanding why. The subject of risk is a fascinating one. So much of our lives is dictated by risk: relationships, choice of career, who we marry, where we live, even such trivial questions such as whether to take an umbrella to work in case it  might rain later. Not only are many of us unaware of the influence of risk in our lives, it remains a somewhat intangible term that we often struggle to define. Hence, it may be one of the reasons we often struggle to manage it effectively.

Psychologists and philosophers have long tried to separate different types of intelligence, as many are not satisfied that measures such as IQ provide a full picture of how intelligent an individual may be. Whilst some argue that it is the most appropriate measure available for assessing general intelligence, others put forward the notion that IQ fails to account for equally important things such as emotional (EQ) or interpersonal intelligence. Psychologist Howard Gardner, for example, believes that there are 8 different types of intelligence: bodilykinesthetic, interpersonal, verbal-linguistic, logical-mathematical, naturalistic, intrapersonal, visual-spatial and musical. Psychologist Dylan Evans proposes that there is a 9th form of intelligence: a special type for thinking about and dealing with risk, something he calls ‘risk intelligence’. The reason for this, in his opinion, is that popular measures of intelligence such as IQ and EQ both fail to assess how good people are at judging risks and weighting probabilities.

Dr Dylan Evans received his PhD in Philosophy from the London School of Economics and has gone on to write several books on cognitive psychology, emotional intelligence and most recently on risk intelligence in the book ‘Risk Intelligence: How to live with uncertainty’. He is also the co-founder of Projection Point, a company that helps businesses, including some high profile private equity and hedge fund groups, improve their risk intelligence. The aim is that they become more risk-aware and therefore manage crucial business risks more effectively.

Evans is a colourful character and has spent the last few years interviewing some of the best gamblers in the world, many of whom have made gambling a highly profitable full-time occupation. Whilst the word ‘gamble’ often has very negative connotations, especially when linked to investments and trading, it is his assertion that we can actually learn a lot about the decision-making, probability estimation and risk management process from expert speculators and gamblers. The negative connotations are due to the over-emphasis on problem gamblers that, in his words, “is a bit like studying obesity and then neglecting haute cuisine”. Making decisions about uncertain events is something humans have been doing for centuries and this process manifests itself elegantly in financial markets. The subject of risk has become a heavily debated topic in recent years and it is arguably due to poor risk management and probability estimation that so many high profile investment firms have blown up so spectacularly within the last two decades. More recently, during the credit crisis of 2008, risk managers were once again placed under the microscope thanks to the proliferation and subsequent collapse of mortgage-backed securities that lead to numerous bank bailouts.

Risk can never be eliminated but some, such as Evans, believe it can be managed more effectively. In this interview, Alistair Evans of Qbasis Invest (a systematic trend-following managed futures or ‘CTA’ investment firm) seeks to relate the subject of risk to the trading and investment universe and indeed to understand how we can all become better at managing risk in order to be more successful in our pursuit of profit. Is IQ or EQ a misleading or, in some cases, even a dangerous measure of how good a trader or speculator may be?

Q: After reading “Risk Intelligence” it struck me that there are some obvious influences in your work such as Nassim Taleb. Who else are you influenced by?
A: Danny Kahneman is a big influence, a psychologist who won the Nobel Prize for economics in 2002 and has never actually studied economics! He has a book called ‘Thinking Fast and Slow’ which is all about the decisionmaking process. He is one of the pioneers of the study of judgement and decision-making. I also follow the work of the evolutionary psychologist Steven Pinker.

Q: A recent Deutsche Bank report defined risk as ‘exposure to change’. How would you define risk in terms of investing/trading?
A: I think the problem with definitions of risk such as this is that they are too vague, too general. I think risk is something that isn’t entirely under your control so it has some sort of random element to it that can leave you worse off or it can leave you better off. Unfortunately I think many people forget the latter! The term risk “management” is quite specific because many people make the mistake of thinking it is something they can eliminate it altogether.

Q: Perhaps that is why in the investment industry, there is a gravitation towards straight-line equity curve / ”1% a month” type return streams that appear to be “low risk”. What does this say about risk intelligence and the emotional make-up of typical investors?
A: I think that shows a bias towards trying to be risk averse but it’s based on this flawed idea that if we’re scientific enough and have enough rules, we can somehow eliminate risk altogether, which is nonsense. As I quote Clint Eastwood at the end of my book “if you want a guarantee, buy a toaster” and that’s about as good as it gets. Risk follows the waterbed principle, if you squash it down in one area it will just pop up in another area and if you follow all these seemingly wonderful rules such as VaR (value at risk) they will give you your nice straight line curves for a while but more often than not, when it does go wrong it usually goes horrendously wrong. I have a preference towards investments where the risk is more obviously acknowledged up front because then you’re not under any illusions and won’t be surprised by some black swan event if and when it emerges.

“..lack of experienced gamblers / speculators (who were familiar with assessing risks and probabilities) were largely absent from key risk management positions on Wall Street prior to and during the financial crisis of 2008”

Q: Let’s talk about asymmetric payoffs or return then. There is one particular hedge fund manager who expects that 95% of the time he will be wrong with his “bets”, yet he is still highly profitable in the long run. What does the fact that most investors / traders seek a high winning trade percentage say about risk intelligence and a need to be right all the time?
A: Well it just shows they’re not paying attention to the proper variable; that is the expected value and people need to move away from thinking that it’s about how often you are right but rather how much you win when you win. If you are only right 5% of the time but your average pay-off ratio is more than 20 to 1 then you’re going to be profitable in the long run. If you’re right 70% of the time and your average ratio is only 8 to 7 then you’re going to lose money. It’s amazing how many people don’t understand that, even people who really, really should!

Q: This leads me on to your recent interview with Aaron Brown of AQR, where he alluded to the fact that the lack of experienced gamblers / speculators (who were familiar with assessing risks and probabilities) were largely absent from key risk management positions on Wall Street prior to and during the financial crisis. Given that risks were generally more quantitatively assessed during those periods, do you think that active, as opposed to passive, risk management is more important than ever?
A: The danger in this case was that it relies on thinking that you can just reduce everything to a few simple rules and one number. In the 1980s for example, if a trader went to his boss and said “I’ve already exceeded my exposure for the month or week but I want to take a bigger position in X, can I have a bigger limit?” He would maybe have looked it up and down, thought about it more objectively, spoken to a few of the other guys on the desk and would then say yes or no based on a number of different inputs. In 1995 or 1998 however, he would have most probably simply said “what’s your VAR?”. Added to this, he would probably have been a different person, i.e. an MBA in quantative finance and the guy from the 1980s, who may not have had the same academic credentials, would likely be actively managing risk in a hedge fund or something outside of the investment banking world altogether.

Q: A number of writers and indeed some of the chapters in your book seem to insinuate that heuristics or “gut feel” can lead to accurate judgements. When it comes to investing, surely because of the broad spectrum of often conflicting “predictive” information perpetuated by the mass media, the availability heuristic is somewhat corrupted and therefore less effective when it comes to making investment decisions?
A: Yes, absolutely and I think some of these books are hugely successful because they appeal to our narcissism. It makes us feel good and almost encourages us to be lazy about using our intuition and that we can think or make decisions without actually thinking which is nonsense. There may be some circumstances where it’s effective,  such as a firefighter running into a burning building but that’s more about instinct than intuition. For the kinds of decisions we’re talking about in investing and speculating, it may have the opposite effect and it’s often the more considered approach that works where we have to check our instincts and, in some cases, go against them. Danny Kahneman distinguishes between System 1 and System 2 in the way that humans make decisions. System one is the snap judgement or gut feel system and System 2 is the more conscious deliberative system where we apply our knowledge and reason and in the speculative arena, I think it’s definitely more system 2 that counts.

Q: Indeed, some of the best traders are able to alter what may have previously been strong opinions within a very short time frame, often to something that is entirely the opposite. Why is that such a rare quality?
A: Good question, we all have something psychologists call “belief perseverance”. Beliefs have inertia and a lot of people find it difficult to change their minds. Why that is?…who knows…but it’s just a fundamental part of human nature in that we like to feel right. So, it does take a big effort of will for people to look for conflicting evidence, move away from their confirmation bias and accept that you might actually be wrong. For example, how many people do you know who, if you give them compelling evidence that an opinion they hold may well be wrong, will say “actually you’re right, thank you very much for proving me wrong”?

Q: This is linked to something you also talk about called “imagination inflation” and how people often only focus on the positive expected outcomes. Would you say that this, in turn, forces people to take excessive risks when it should force you to be more cautious about estimating probabilities?
A: Imagination inflation means that just the fact that you think about a certain outcome or event a lot, it can make you think it’s going to be more likely to occur. This is obviously a dangerous bias to have. Whenever you think how likely an event is, you have to stop and question whether your judgement is biased by the fact that maybe you just thought about it more than you thought about the less palatable alternative. It’s often the people who have the ability to think about scenarios that most people cannot conceive….whether that’s the price of oil eventually breaking $100 when it traded around $20-$30 for years on end, or the stock market declining over 80% in the 1920s….who can be hugely profitable in such instances.

This can lead to over-confidence and my favourite analogy is of a socially inept, awkward young man who believes he’s a real Casanova but doesn’t even realise that people are actually just laughing at him.

Q: Surely this comes down to an innate awareness of what’s really going on in the world. What is the most striking thing you have found in people’s lack of self-awareness of their ability to know their own world and surroundings? Do people just not accept it or since releasing the book, are people more willing to try?
A: There’s a real paradox here. It’s the people who are aware of their own limitations are the people who will really be able to get a long way down the road to improving their risk intelligence and are therefore the ones who will read the book. Conversely, the people who most need to improve their risk intelligence are least likely to want to read my book. Unfortunately most people suffer from something known as the Dunning-Kruger effect, so not only are they not aware of X, they don’t even know that they’re not aware of it. It all comes down to something known as “metacognition”. The world is full of people who are not just ignorant but also un-aware of just how ignorant they are! This can lead to over-confidence and my favourite analogy is of a socially inept, awkward young man who believes he’s a real Casanova but doesn’t even realise that people are actually just laughing at him.

Q: I think that is generally demonstrated by unsuccessful investors who seek the wrong type of asymmetric return, ie they take huge risks, most often unbeknown to them, in order to generate relatively little in return. Do you find that most of the successful, “risk intelligent” individuals you have observed target the other end of the asymmetric spectrum….ie small risks and potentially large pay-offs?
A: I don’t know, I think ultimately it’s all about your expected value or expected return and as long as you have positive long-term expected value then in the long run, by the law of large numbers, then you should generate profit. How you mix and match the kinds of bets you’re making so essentially, so you create a portfolio of bets. Using horse racing as an analogy you have a proportion of long shots and a proportion where you back the favourites. I think specific speculative activities lend themselves more to the asymmetric, call option-like pay-off than others. For example, financial markets most likely do whereas in certain types of sports betting, you’re often better off backing the favourite and it’s actually the people who bet on the long shots who lose money in the long run.

Q: Humans have a very poor record of predicting anything. What is your view on predictions and is the term “predict” not a bit of a misnomer in itself?
A: As Yogi Berra famously said “Prediction is very hard, especially about the future” and I suppose it depends on how you define prediction. If you define it as saying that something WILL happen then yes, it is a misnomer but if you define it as making a probabilistic estimate, which I suppose is more of a forecast, then that’s fine and I think there is compelling evidence, some of which I explore in the book, that we can get better at this. We can obviously never be perfect but we can get more towards what we call the “optimum prediction frontier”. If you imagine a calibration curve with the optimal line down the middle. You are never going to be right on the line but there are definitely ways of getting closer to this over time if you follow certain procedures.

Q: When reading the book, the most obvious question to me was: why you do not draw a distinction between gambling and speculation?
A: I think that when people do draw those distinctions they are usually prompted more by some sort of concern to avoid a moral association with the bias that gambling is a bad thing

There’s no fundamental difference  (between gambling and trading) because I think everything has an element of luck…speculating has an element of luck. You can make a more intelligent bet or a less intelligent one just as you can make a more or less intelligent gamble. People always just condemn something to “oh that’s just gambling!” where they are implicitly just defining gambling as nothing more than making completely reckless decisions.

Q: ……surely though, the term gambling suggests the element of luck or recklessness whereas speculation (from its Latin meaning) is more about observing and making judgements based on estimated probabilities?
I don’t. There’s no fundamental difference because I think everything has an element of luck…speculating has an element of luck. You can make a more intelligent bet or a less intelligent one just as you can make a more or less intelligent gamble. People always just condemn something to “oh that’s just gambling!” where they are implicitly just defining gambling as nothing more than making completely reckless decisions. I have a very positive view of gambling and if you look at the whole origin of decision theory, they all go back to the analysis of gambling behavior by mathematicians such Pascal in the 17th Century or John von Neumann analyzing poker and the theory of games. Furthermore, statistical theory originated as a way of calculating the odds for dice games originally. Gambling is a brilliant way of studying the decision-making process. You will find that a lot of the good traders are experts in game theory.

Q: This is possibly why investment firms such as ours, who exist predominantly to manage risk, are very keen to distance ourselves from gamblers. If that is your assertion and obscure as this question may sound, what can traders and investors learn about the probability assessment process from some of the successful gamblers
you’ve come across?
A: I would say that they should try and assign numeric probabilities to their estimates and keep records of those estimates and how they played out. Looking for times they’re wrong and learning from mistakes is what distinguishes that group from the majority of unsuccessful speculators. Good successful traders and investors are obviously most likely to be doing this already.

Q: Would you therefore agree that the best speculators and gamblers you’ve come across have a more systematic or rules-based approach?
A: They’re systematic, yes, but that doesn’t necessarily mean they are following an explicit set of rules. They are probably some combination of a set of rules and an inherent intuition but their intuitions are systematic and they’re systematic about the way they manage and use that information and often in how they execute it. I would distinguish, for example, between Blackjack players who are proper system players and are literally just following a set of rules where there is almost no room for maneuver and someone such as a horse-racing bettor where they are exercising more discretion.

Expert gmablers and speculators, “..They’re systematic, yes, but that doesn’t necessarily mean they are following an explicit set of rules. “

Q: If only a small percentage of people are risk-intelligent, would you agree that systematizing at least the probability estimation process (and therefore investment process) is a better solution?
A: For most people, I would say very much so. I think that unless you’ve got the time, resources, skill and energy to become really good at being highly risk intelligent in one particular domain, most of the time you would be better off following a robust process.

Q: Do you think in general that most people are too risk averse and should embrace risk more?
A: For risk intelligence, you can objectively measure people who are better at estimating probabilities but with risk appetite, it ultimately comes down to personal preference. It doesn’t mean it’s better but I, for example, am ‘riskloving’ but I find it sad when I see people who are so risk averse they are never willing to try or do anything new. It also depends on the circumstances. You often find that more risk-averse people can make better investors in a mundane market whereas someone who is more of a risk taker is probably more likely to profit from a crash or a raging bull market. But some prominent risk-takers in history such as Livermore or Paulson have made more money on the way down, perhaps because the market has a gravitational human characteristic.

Q: What would you say is more damaging as a trader; overconfidence or under-confidence?
A: They are both equally damaging and will make you less profitable but as a matter of empirical fact, overconfidence is just far more common. So, given that it’s more common you should be more worried about it. Polar Bears and cars are probably equally dangerous to humans but which are most people worried about?!

Q: In your study of professional gamblers and investors, how would you characterize their use and analysis of risk in making probability-based decisions versus non-professionals?
A: In terms of the successful professional gamblers, of which there are very few for a start, they tend to be far better at it than the overwhelming majority of people who think about predictions simply in terms of questioning whether it will or won’t happen, i.e. will it or won’t it rain tomorrow. For most people it doesn’t even cross their mind to think about assigning probabilities to certain events occurring. Getting them to think in
terms of probabilities at all is a much higher level challenge in the first instance, let alone before you can work on their ability to assign numeric probabilities to events successfully.
Q: In one of your other books on emotional intelligence, you were critical of the idolization of supposedly unemotional characters such as Star Trek’s Dr Spock. Would you therefore disagree that un-emotional, logical people are better speculators than more emotional or emotionally intelligent people?
A: Well, at the time I was criticising the view that the best decision maker would be someone completely devoid of emotion but that’s a bit of a straw man because the sort of emotionless Dr Spock character doesn’t exist. There are situations where it’s useful to have emotions but, having said that, one of the characteristics of good investors and gamblers is they can put emotions to one side when they need to. Good gamblers are cold and emotionless at times and do this naturally when they really need to. If you think about the process of a good speculative decision, you need to take two factors into account;

1) the numeric probability and

2) the expected value.

In a way you need to multiply them together. When you’re assessing the probability (number 1), you need to be as emotion-less as possible about the cold hard facts, hence why systematizing this part can solve the problem most people have in this regard. However when you’re considering number 2, the monetary award, you
need to work out how important that is to you and how it would make you feel if it worked out in your favour.

Q: Given that on the risk-intelligence test you have in your book and on your website
(http://www.projectionpoint.com), if you assigned a 50% probability to every question, you’d emerge perfectly calibrated. Does that suggest that your probability estimates should always be relatively constant?
A: Not necessarily because you can “game” the test that way. But we can measure how often people use intermediate categories such as 10/20/30/40/60/70/80/90% etc and we give people a point each time they use those intermediate categories, whereas every time they use 0/50/100% they get zero so that forms an index of how reliable their risk intelligence score is. So you can score 100 but get a K score that is very low so you can
say that someone has scored well on the test but their risk intelligence isn’t necessarily reliable. On the other hand, if you scored 80 on the test and had a K score of 35-40, that would be very impressive as it forms a much more reliable guide to how risk-intelligent that person is likely to be. If you’re making predictions about geopolitics, assigning a 50% probability on a civil war for example, that wouldn’t really help us prioritise our
resources very effectively!

In terms of the successful professional gamblers, of which there are very few for a start, they tend to be far better at it than the overwhelming majority of people who think about predictions simply in terms of questioning whether it will or won’t happen

Q: How would you say most traders and speculators can improve their risk intelligence and get closer to this optimum frontier?
A: I have to say a logical process and plan is vital. Track progress, write down your numeric probability estimates and then monitor how they played out. Score yourself. There is even a spreadsheet that you can download for free from my website so that you can make up your own risk intelligence test and base it on your investment strategy. Keep a trading diary, write down your estimates and then review your performance. By doing this on a regular basis, you can become better at estimating probabilities and hopefully therefore trading as a result.

Q: I quote from the book “By transforming low probability events into complete certainties, especially when the events are particularly scary, worst-case thinking leads to terrible decision-making”. Can you elaborate on this and relate it to trading?
A: This is the Cheney doctrine where he stated that even if there’s only a 1% chance that Iraq is going to acquire nuclear weapons, we’d better regard it as a 100% chance for the purposes of our reaction which is ludicrous. I think far too many people hugely oversize their bets on a worst case or best case scenario whether it’s a horse winning a race or a view on where the stock market will end up 12 months down the line. Basing your decisions, whether it be a bet, investment or anything else, worst-case scenarios are just as damaging as best case scenarios. It is far better to make a variety of scenarios and assign a numeric probability to each event occurring.

Keep a trading diary, write down your estimates and then review your performance. By doing this on a regular basis, you can become better at estimating probabilities and hopefully therefore trading as a result.

Dylan Evans’ Blog

The Right Expectations

EXPECT THE UNEXPECTED

There is a world of risk out there, and managing it is a lifetime endeavor. Every time you get in the car, you risk the possibility of having an automobile accident. When you walk in a thunderstorm, you risk getting hit by lightning (a remote risk, but a risk nonetheless). You may face certain medical risks (which vary, depending on genetic and family history) or risk of losing your job—the list can go on and on.

Typically, in our society, we attempt to control or manage these risks by obtaining insurance, or by using greater care in our day-to-day behavior and choices. For example, you may very well currently have an insurance policy that protects you from auto theft, collision, and bodily injury. You might have medical, life, homeowners, or unemployment insurance. And if you are really responsible, you probably exercise, eat right, and look both ways before you cross the street. All these precautions and procedures are designed to reduce, not eliminate, the possibility of being devastated by a variety of unexpected circumstances. And that is just the point: These circumstances are unexpected.

Our job as traders is to make a habit of expecting and being prepared for the unexpected. In addition, we need to avoid a state of “trader paralysis” that can be created by unexpected events. If we are well prepared we are better equipped to combat the fear that trading can trigger.

SIX TYPES OF RISK TO MANAGE IN TRADING

In the spirit of expecting the unexpected, we can always attempt to plan for what might happen. In doing so, there are half a dozen primary types of risk for you to consider every time you place a trade. We will cover each of these in detail in the coming posts, but ponder the following top six for now:

1. Trade risk

2. Market risk

3. Margin risk

4. Liquidity risk

5. Overnight risk

6. Volatility risk

(Excerpts from A Trader′s Money Management System: How to Ensure Profit and Avoid the Risk of Ruin (Wiley Trading, 2008) by Steve Nison and Bennett A. McDowell)