Not Another Investment Podcast
Understand investing beyond the headlines with Edward Finley, sometime Professor of Finance at the University of Virginia and veteran Wall Street investor.
Not Another Investment Podcast
Navigating the Waves of Finance: The Role of Probability Theory in Investment Decision-Making (S1 E8)
Dive headfirst into the unpredictable currents of finance with me, Edward Finley, as we chart a course through the complex seas of probability theory and its indispensable role in understanding capital markets. Venture into the treacherous waters where historical price data is both a beacon and a riddle.
We'll begin by navigating some of the basic assumptions of probability theory, uncovering the meaning of the basic statistical properties in easy-to-understand language and applying them to real world data to give them life.
The seas can get pretty rough, and we'll measure that market volatility, as we dissect the patterns that shape our understanding of investment risk through the lens of variance and standard deviation. Learn how volatility clustering makes it simpler to forecast than return, and why the long-term view of market returns may be clearer than the short-term blur. We'll navigate the stormy spells that rocked the markets in the past and glean insights for weathering future financial squalls.
Wrapping up our odyssey, we confront the harsh truths about the limitations of statistical tools in understanding markets fraught with uncertainty. Join us for a candid discussion on the perils of relying too heavily on statistics in predicting market behavior, illustrated with cautionary tales like the "lost decade." By journey's end, you'll emerge with a fortified understanding of how probability theory can—and cannot—guide your investment decisions in the tumultuous tides of finance.
Notes - https://1drv.ms/p/s!AqjfuX3WVgp8uSLfASdaN8dlmtfE?e=XDpWKX
Thanks for listening! Please be sure to review the podcast or send your comments to me by email at info@not-another-investment-podcast.com. And tell your friends!
Hi, I'm Edward Finley, a Sum-Time Professor at the University of Virginia and a Veteran Wall Street Investor, and you're listening to Not Another Investment Podcast. Here we explore topics and markets and investing that every educated person should understand to be a good citizen. Welcome to the podcast. I'm Edward Finley.
Speaker 1:Well, last time we sought to answer the question what is it about capital markets that we think makes it so good at allocating capital to risky uses? And the answer we came up with in a nutshell is the efficient markets hypothesis, which tells us that capital markets with a sufficient number of participants all seeking to understand what the correct price of a security is will, over the long term, incorporate all accurate information into price. Well, today we're going to talk a little bit about probability theory. Even if markets are not entirely random in the short run and we don't think they are and even if markets are not a stationary process which, you'll recall, means like flipping a coin, where we know all the possible outcomes and we can compute the probabilities of every outcome, even if we don't think markets are stationary process and we don't think they are probability theory is still useful as a tool, albeit imperfect, for us to understand risk and return, interpret risk and return in market data and express some strength of our conviction about what we think assets returns will be in the future, how imperfect our assumption about those returns might be, how much risk therefore there is in an asset and how those assets interact with each other. So we're going to do some basics of probability theory Now. Health warning there will be numbers. Don't panic, you're not going to need to understand probability theory like a statistician. You're not going to have to compute anything. Just listen and follow along, because my hope is that and I think it will be the case that this will all be very intuitive and make a lot of sense. But it's important to just get the vocabulary down in our heads.
Speaker 1:So let's talk first just about some of the basics of probability theory in markets. Well, first we make a really big assumption before we start using probability theory, and the assumption is, as I mentioned just a moment ago we assume that markets are a stationary process. That is, using historical prices, we can calculate periodic, usually monthly, returns and create a probability distribution. A probability distribution is just our classic bell curve. That's a normal distribution in a stationary process and, so long as we have sufficient historical data, that is enough monthly returns in our time period, then a normal distribution of an asset's returns can reasonably approximate what the future will be. That's really the heart of probability theory in markets and, as I said, we don't think it's a stationary process and so it's an imperfect tool, but it's one which I think we can still use to make some sense.
Speaker 1:Well, what's the first thing you compute when you are applying probability theory to say monthly returns of assets? Well, the first thing you're going to compute is the mean. The mean, if you can picture a bell curve, is going to be the center and top of the bell. The mean is where all the monthly returns are centered. Now it's really important to use the phrase that I just did where the monthly returns are centered. It is not the case that the mean is the most likely value. In fact, if you want to be really precise, statistically, it is not the most likely value. In fact, it has no higher probability of being true than any other observation in the distribution. Instead, what the mean tells us is where we think future returns will be.
Speaker 1:Now we should distinguish the mean, first of all, from something called the median. The median is the middle number in a sequence of returns that are lined up from smallest to largest. So if I took all the monthly returns and I lined them up from the smallest monthly return to the largest monthly return and I divided that sample directly in half, whatever that monthly return in is called the median. Medians are really good for dividing something into two categories. Where you're trying to understand those categories. The most common example is we really care about median income because if we're talking about the population, we want to kind of divide the population into two groups and understand what's the dividing line between the two groups. The thing about the median is that it ignores outliers. The median doesn't take into account whether some people have a lot of the wealth this is what we're talking about income for now and that some people don't have very, very little of the wealth. And so if we want to take into account outliers, then we really have to think in terms of mean, not median. So in finance we very rarely talk about median. That's really the province of social sciences and when economics is talking about policy.
Speaker 1:In addition, there are two kinds of means that you can calculate. There's something called the arithmetic mean. That's what you and I all know to be the average. The arithmetic mean is you just add up all the periodic monthly returns, divide by the number of observations and you get the arithmetic mean or the average. The arithmetic mean tends to be very well suited to things that have a linear relationship, like how many shirts will a factory produce tomorrow? If I know the average number of shirts that the factory produces, I can make a pretty good prediction of how many shirts the factory will produce tomorrow. It's also really useful to predict what the future return will be. If I know the average monthly returns, or if I know the average annual return, and you ask me what do I think the return on this asset will be next year, the arithmetic mean, or the average, is going to be a pretty good estimate of what that return will be. It can be affected by outliers, which we care about, because we might have next year one of those years that are outliers. The thing that the arithmetic mean is not particularly good at is it's not particularly good at telling us what the future value of something will be, because the future value of something is not additive, it's multiplicative. If I earn return next month and I earn return the following month, the return that I earn in the second month is also return on the return I earned in the first month.
Speaker 1:The arithmetic mean doesn't capture that concept. For that we use something called the geometric mean. The geometric mean is something that is time certain. That means it takes into account the compounding in a multi-period situation. It's less affected by outliers, though it is affected by outliers to some extent. It scales returns based on the previous period's returns, making it a much better predictor of the expected future value of something In finance. Whenever we want to think about what the expected return for an asset will be in the future, we want to use the arithmetic mean. Whenever we want to understand what we think something will be worth in the future, we want to use the geometric mean. On the website I've got some slides with charts that you can look at and see some of the things I'm talking about, but it isn't necessary to look at them because I'll be able to describe them here in pretty clear terms.
Speaker 1:Let's apply what we just learned to real life. Let's look at the returns for US equities. Here I'm going to use something called the Wilshire 5000. That's an index of all US equities 5,000 US publicly traded stocks. Let's also look at the 10-year US Treasury. What's the data? If I look at the monthly returns from 1971 through the end of 2022, the Wilshire 5000 had an arithmetic mean of 11.4% per year. That's a reasonably good estimate of what the return on US equities might be in any future year. You wouldn't want to use 11.4% to compute what the value of your 401k might be when you retire. That would be an imprecise number For that. You want to take a look at the geometric mean. The geometric mean for the US equity market was 10.62%, about 120 basis points less per year. That would be not terribly accurate if you were trying to predict when any future year's return might be, but it would be much more accurate if I wanted to estimate what the future value of your investment would be if you owned all of US equities. It's also important to note how the mean masks the variations in the return. The returns are not, in our example just now, 11.4% per year. They're an average of 11.4% per year.
Speaker 1:We can look decade by decade and see that there's huge variation over time. In the 1970s, for example, and in the 2000s, equity returns were lower. Then the long-term average in the 1970s there were about 10.8%. In the 2000s they were 3.89%. That's far lower than the 11.4% average for the full period, and in the 1980s and 90s and 2010s we can see that the average is also quite varied, but it's higher. In the 80s it was 13%, 17% in the 90s and 14% in the 2010s.
Speaker 1:Tenure Treasury had an arithmetic mean, or an arithmetic return, or an average return, of 5.48% per year. Now that's going to be a reasonably good estimate of what the future return on tenure treasuries might be in any given year. But if we look at the geometric return, that was 5.28%. That's going to be a better estimate of what our investment in tenure treasuries might be over a long period of time. When we look at the decades in the tenure treasury market, we also see variation, but not quite as much variation as we see in equity returns. So in the tenure treasury market, the 1970s, the 2000s and the 2010s all saw returns lower than the mean, while in the 1980s and 1990s returns were much higher than the mean.
Speaker 1:Now, because mean returns in financial markets vary significantly over time, it's really impossible to predict returns of asset classes in anything except very long time horizons. This should spark a memory to our conversation last time about efficient markets hypothesis. The view there is that all accurate information is incorporated into price in the long run and in the short run. We had the strong view that said it's random and we had the weak view that said it's driven by behaviors. And this sort of makes that point quite nicely. It illustrates how the ability of our predicting returns with so much variation decade by decade, is really hard to do.
Speaker 1:But we should want to try to quantify how wrong the estimate is, how varied are the returns really? And we compute something called the variance and its close cousin standard deviation, which in finance speak we often refer to as volatility. So what are these things? What are these concepts? It's essentially measures of how dispersed the returns are around the mean. So again, picture in your mind a standard bell curve. If the returns monthly are all very, very close to the mean, you can picture that the bell will be very narrow and very tall. Very close to the mean Means there's not much variation in monthly returns. They're very stable. In contrast, if they're widely dispersed from the mean, that bell curve is going to look shorter and stouter. That means that more of the observations vary from the mean. We can compute the measure that's called variance and its cousin standard deviation, in order to give some real expression to how dispersed they are around the mean, and how dispersed they are is going to tell us how likely are we to be wrong in our estimate.
Speaker 1:What I should point out here that I think is the most important is that standard deviation is always going to be talking about volatility and it's always going to be telling us the degree to which our measurements might be wrong. But it's also, if you think about it, going to be telling us something about the risk in owning that asset. If the returns are widely dispersed and there's high volatility, that is risk, risk measured as uncertainty. I can't be sure what my return is going to be in any one month. One standard deviation you sometimes read this in the news One standard deviation just means that 68.27% of all the observations are plus or minus the standard deviation from the mean. I'll give you a numeric example in a minute, but the takeaway is that's a lot. 68% of the monthly returns are going to be in the range of that standard deviation, plus or minus the mean. Two standard deviations represents about 95% of all the observations. Three standard deviations is 99.7% of the observations.
Speaker 1:Standard deviation and volatility is always a positive number and it's always expressed as a percentage. So you'll see, the volatility will be expressed in terms of 16%, 17% and so on, and since the volatility tells us only the degree of wrongness, that's why we have to add and subtract the standard deviation to and from the mean to understand how much uncertainty there is. So let's take an example. Let's say that the standard deviation of a probability distribution is 10%, and that means that 68.3% of the time the future return will be plus or minus the mean, that's, minus 10% from the mean. The problem is that volatility, also time, varies. It's not obvious that 10% variation from the mean over a long period of time will be what we expect in a shorter time horizon.
Speaker 1:But what we've learned through research is that volatility tends to cluster. So only a portion of investors sometimes are trading and incorporating new information into prices, and so they're slow to react, except when there are periods of turmoil. And when there are periods of turmoil there are many more traders all trying to incorporate information into price, and so we tend to see much more volatility in periods of turmoil than in periods of calm. It tends to mean revert over a long time horizon the level uncertainty about the prospects of an asset will be more reliable, more constant. The moves in our confidence in the short run are really the product of new information and market participants trying to incorporate that information into price. So it's a lot more volatile in the short run. It tends to increase as prices go down and that's usually explained as the product of when prices go down. It makes a firm more leveraged, and remember what we learned about leverage it amplifies the upside and the downside. So if prices of a firm go down, making the firm more leverage, it hasn't changed how much debt there is. Then we expect the future price movements are going to be a lot more wide swinging than they were previously.
Speaker 1:All right, let's apply this to something concrete to see if we can make some sense. Remember that I told you the average return for US equities since 1971 was 11.4%. The standard deviation for equities during that period that's the average annual dispersion of returns was 16.34%. So that means that about 68% of the time returns were going to be plus or minus 16% from the mean. Now think about that. That means that we're saying 68% of the time annual equity returns will be somewhere from about negative 6% to about positive 26%. That is a massively wide range, to say 68% of the time. So what do we take away from that? What we take away from that is that equities are a highly uncertain asset class. There is great variation in those returns and therefore our estimate for what the return might be next year is going to be very uncertain, very unreliable.
Speaker 1:How about treasuries? Until the 10-year treasury, average return was 5.5%, more or less. The standard deviation was 9%. So that means that 68% of the time you might expect annual returns for 10-year treasuries to be somewhere between negative 4% and positive 14%. Again, that's a pretty wide birth to fit through, but not nearly as wide as equities. So treasuries also vary uncertain, just not nearly as uncertain as equity returns.
Speaker 1:We can also look in both cases at the standard deviation in individual decades, and what we see here is again this notion that if time varies, it's not 16% in the case of equities every 10-year period. In fact, there are some periods where it's lower, like the 1990s and the 2010s, and there's some periods where it's higher, like the 1970s and the 1980s and the 2000s. But the difference, the range of differences by decade, isn't terribly much. It's only ever as low as 12%, where the long-term average is 16, and it's only ever as high as 20%. Likewise, the same is true for 10-year treasuries 9% is the long-term average dispersion of returns, and you see, by decade, that it's only ever as low as about 6% in the 2010s and only ever as high as about 11%. So it time varies, but it time varies much less so than returns, and that's because of what I mentioned a moment ago.
Speaker 1:Volatility tends to cluster, it tends to mean revert, and so you get something that's just a little bit more predictable. All right, so we have our mean, that gives us an estimate of what we think future returns will be, and we have our volatility, which gives us some measure of the uncertainty of that estimate. How wrong might it be? Well, we have another measure to help us understand the uncertainty of the estimate, two in particular, and those have to do with the shape of the bell curve, just like standard deviation. Did you recall standard deviation's shape is? If they're all very tightly gathered around the mean, it's going to be tall and skinny, and if they're widely dispersed, it's going to be fat and stout. Here we're going to look at two other measures called skew and kurtosis.
Speaker 1:Skew let's start with skewness tells us how symmetrical the dispersion of returns is, and that's very important, of course, because it can help us understand how reliable volatility is as a measure of uncertainty. Picture in your mind a standard bell curve. It is perfectly symmetrical. That's a normal distribution. But distributions can be skewed to the left or skewed to the right, that is to say, more of the monthly returns can be lower than the mean, higher or more of the monthly returns can be higher than the mean, than lower.
Speaker 1:Financial markets are historically not symmetrical like a normal distribution. More of the returns tend to be lower than the mean. Now let's think about our behavioral explanations for markets from last time. Think about some behaviors that might tell you why normally, financial markets have more observations that are lower than the mean, than higher than the mean, and I hope what you come to is you'll remember things like loss aversion and representativeness and framing. These are things that would tend to lead market participants to be more sensitive and cause more downside observations than upside observations. So if we want to understand the risk of our return estimate being wrong, volatility alone doesn't tell us the story.
Speaker 1:We have to look at skewness as well. Volatility assumes symmetrical distribution around the mean and, as we just said a moment ago, financial markets tend not to be. If more of the returns are higher than the mean, that's called positive skew then volatility overstates the risk that our estimate return is wrong. Generally speaking, if a distribution has a skewness between 0.5 and negative 0.5, so anywhere in that range then we're going to say that that's a normal distribution. If the distribution has skewness, that's between negative 1 and negative 0.5. If it's negatively skewed, or between 0.5 and 1, if it's positively skewed, we're going to say it's moderately skewed. And if the distribution has a skew of less than negative 1, so that's a bigger negative number. Negative 2, negative 3, or greater than 1, we're going to say it's highly skewed.
Speaker 1:One thing about skew that's important to point out here is that measure is very, very fragile if you have fewer than 3,000 observations. And in general in financial markets, when we're looking even at long data like 1971 to 2022, we are generally going to have fewer than 3,000 observations, many fewer. And so whenever we look at things like skew and, in a moment, kurtosis, we're going to want to take it with a grain of salt. All right, so let's take a look at skew. How wrong will our volatility estimates be of uncertainty, taking into consideration the shape of the distribution. So in US equity markets from 1971 to 2022, the skew was negative 0.52. And so you'll recall that means we would just say that there is a negative, moderately negative skew. The skew for 10-year treasuries was 0.13. And so we would say that treasuries therefore had a normal distribution with respect to the shape. Whether it was skewed right or left, it was more normal, more symmetric. If we break down equities by decade, we also see that skewness varies. So, for equities, returns were normally distributed in the 1970s and the 2010s, but they otherwise had moderate, negative skew. Treasuries, likewise, were normally distributed in most decades, but were moderately negative in the 2000s. And so we have this notion that this measure is not some magic explanation. We don't compute it and then suddenly have clear answers. It's just all there to help us interpret when someone tells us numbers, how to interpret the confidence level that we have in that number.
Speaker 1:The second measure of shape that I mentioned is kurtosis. Kurtosis describes for us how many outliers there are, both high and low, and these are very low probability events. If you picture again the bell curve, kurtosis talks about the ends of the bell curve, or what are sometimes called the tails of the bell curve, and so those are real outliers. Those are things that happen very rarely, but they happen, and in a normal distribution the tails of the curve are pretty thin, pretty skinny, we would say.
Speaker 1:Financial markets, though, don't have skinny tails like a normal distribution. In fact, they are prone to booms and busts, and booms and busts mean statistically lots more tails than a normal distribution, and we can understand why. From our earlier discussions about how securities trade in markets, we know that markets tend to be subject to optimism. We have trading mechanisms like shorts to try to tame that, but markets tend to become very abominant and then quickly reverse. We also know from our discussion of other financial periods in history that markets are sometimes prone to panic. People panic when we talked about the liquidity crisis in money market funds, or people panic when we talked about mortgage securities and the mortgage crisis, and when people panic, then the value of all securities declines very precipitously and we have tails in the other direction. So booms or busts in markets are very typical, and that means we don't ever expect to see markets have normal tails. We expect them to have fat tails.
Speaker 1:A normal distribution has a kurtosis of three Most kurtosis figures. When we look at kurtosis figures, they're always going to be reporting excess kurtosis, so that's the kurtosis over and above what normal is, and so we're really just looking at fatter or thinner. So you don't have to worry about three. We look at the kurtosis number and it just tells us. If it's positive, that means fatter tails. If it's zero, it means normal and if it's negative, it means thinner tails. Like SKU, kurtosis is a very unreliable measure on its own when there are fewer than 3,000 observations, and we won't ever look at 3,000 observations. So we've got to take great care in looking at SKU. So let's take a look at it.
Speaker 1:So in the period from 1971 to 2022, we see that US equity markets had significantly fatter tails than a normal distribution. The kurtosis was 2.02. And likewise, the 10-year treasury market had fatter tails than a normal distribution 1.88. I will tell you that that's very typical in the literature that most financial markets these and bonds are no particular distinction generally have fatter tails to the tune of about an excess kurtosis of 2. Decade by decade, though, that varies quite a bit, and so we can see that in equities, for instance, the tails were noticeably thinner, though not still normal, but noticeably thinner in the 2000s or even in the 1970s, but they were noticeably fatter in the 1980s because of events that occurred that created much, much wider tails.
Speaker 1:Interestingly, however, I think when we look at the 2000s, where we had the global financial crisis, that's an amazing outlier. The kurtosis there is 0.87, which is to say it's kind of thinner tails than over the long-term average. And that brings me to another point that's really important. Kurtosis doesn't tell you the distribution in the tails, it's just a statistical measure to tell you that the outliers are more than normal. And so we can't look at kurtosis and say, ah, there's going to be high kurtosis when there's the global financial crisis or high kurtosis when there's COVID, because there might not be. And that's a really good illustration of the point where, in the decade that included the global financial crisis, the kurtosis of US equities was 0.87, so much thinner than long-term kurtosis, though still fatter than normal. And I'll just remind again that, because kurtosis requires 3,000 observations or more to be reliable, we have to be very careful about darning too many conclusions.
Speaker 1:Another question that we might want to address when thinking about the statistical properties of asset returns is we might ask how related are the returns on two or more assets, and the reason why we might want to know about how related are the returns is important because it helps us understand whether the average return that we're talking about, if we put multiple assets together, is the average return of the portfolio. We create just the addition of the proportional amounts of each asset's average return, or is it something different and it turns out punchline. It's something different. How different is going to depend on the covariance or its cousin, statistically correlation. What those measures seek to do is answer the question how related are the returns or two or more assets, and in turn, that helps us figure out how the volatility of combining assets is usually lower than the average volatility would be of just each of the assets proportionately, and it helps us understand that the returns will be different than just the additive proportion of the returns.
Speaker 1:A way to think about covariance and correlation is to think in terms of assets zigging and zagging. They don't necessarily move in the same direction and to the same degree, and so we want to have a way of computing their relative relatedness. In practice, of course, we don't own just one asset, even if we have just stocks. We're not going to have one stock. We're going to have a lot of stocks If we have a portfolio. We're not going to have only stocks. We're going to have stocks and bonds or maybe other asset classes. And because asset prices don't move in unison, we can't just add each asset's volatility to evaluate how much risk there is, and we can't just add each asset's return to figure out what the expected return should be. We've got to figure out the degree to which those assets are related in their movement, and so we compute these two measures.
Speaker 1:Correlation is probably the most typical one that you'll hear about. Correlation is just a scaled coefficient of covariance. Its value is always between negative one that's perfect co-movement in the opposite direction and positive one perfect co-movement in the same direction. Zero means the asset's returns are completely independent. Sometimes correlation is expressed as a percentage, and so instead of negative one, you would see negative 100%, and instead of one you would see 100%. And so the way to think about correlation is to say that if something has correlation of negative one, it means that for every change in price of the first asset, the second asset is going to have exactly the same change in price, but in the opposite direction. And a perfect correlation of one positive one means that for every change in price in one asset, the other asset is going to increase in price by exactly the same amount.
Speaker 1:Correlation tends to time vary, as we've seen in all of our statistical measures, but it also changes sign and that's not something we've seen in these other statistical measures. We have noticed that average returns will vary from time to time, but if we're looking at the long enough time periods, it's unusual for it to change sign. It can, but it's unusual and volatility never changes sign. So correlation tends to be a little bit more difficult to predict among assets. Again, if you're interested, I've got some charts on the website that show you the average correlation among the stocks in the S&P 500. And you'll see the time variation. It sort of goes up and down over time, but it tends to vary between about 10% correlation and can be as high as 60% or 70% correlation. But the takeaway is that it's always positive. It's very unusual for equities to have negative correlation with each other.
Speaker 1:I also have a chart there that shows the correlation between stocks and bonds, namely US equities, the Wilshire 5000, and 10-year Treasuries. Over the full period 1971 to 2022, the correlation was 7.42%, or in the lingo we're adopting here, we would say that their returns were roughly independent. But when we look by decade, we see that during the 70s, 80s and 90s the correlation was positive low but positive 17%, 24%, 36%. So for a given change in equity prices, we would expect Treasury prices to move in the same direction, but just not by quite as much. Only in the 2000s and the 2010s do we see negative correlation between stocks and bonds negative 35% and positive 35%.
Speaker 1:What I think is important to note here is that you shouldn't mistake positive correlation with meaning the returns will be the same. It's just about the direction of returns in monthly intervals. Likewise, negative also means that they're not returns moving in the opposite direction, and I'll give you two good illustrations of that. So in the full period with 7% correlation should we would say that's nearly independent Stocks geometric returns were about 10.6% and the 10-year Treasury was about 5.28%. So the fact that they're not related doesn't mean that their returns are going to have a related or unrelated relationship to each other. In addition, if we look at the 1970s and we see again positive relationship, 17% correlation, higher correlation it doesn't mean that they're moving more closely together. In fact, equities are 9.7% and Treasuries earned 3.5%, so both less than their long-term average, but not proportionately less. And in the same vein, taking the opposite side, in the 2000s, stocks and bonds had negative 35% correlation. That doesn't mean, though, that their returns one's positive and one's negative. It just means that their movements were not in the same direction. The average return for equities was 2.5% in the 2000s, and the average return for 10-year Treasuries was 4%, both positive. They're just about the individual movements.
Speaker 1:We're going to come up with two more measures, and then we're going to talk a little bit about some of the problems of using probability theory and take a look at a case study of what's sometimes called the lost decade. The two more measures I'm going to do very, very quickly. The first is auto correlation. We have to have a way to measure how independent monthly returns are, because, again, in probability theory, we assume just like the strong view in efficient markets hypothesis, we assume that monthly returns are completely independent of each other, and it turns out, of course, in capital markets they are not independent of each other. In order to measure that, we can compute something called the auto correlation. Sometimes it's called the serial correlation. It's just like correlation between two assets, except you're computing the correlation between the same assets monthly return in month one and its monthly return in month two, if you have, say, a one-month lag in your correlation.
Speaker 1:When an asset's monthly returns are independent, the auto correlation should be close to zero and that would tell you that the assets monthly returns are independent and therefore probability theory is probably reasonably reliable to look at. But where there's positive or auto correlation, it suggests that monthly returns exhibit some momentum, and remember our discussion from momentum. That's really a weak view that behaviors could tend to cause prices to continue to go up if they have been going up, or down if they have been going down. And in this case a positive or auto correlation tells us that if a monthly return is larger than average, it's more likely that the next month's return will also be larger than average. That's a violation of the strong efficient markets hypothesis view and is more consistent with the behavioralists. It's also the case that it has an effect on the measure of volatility, because when we compute volatility statistically, we assume that the monthly returns are independent of each other, but they're not, and so we can adjust volatility for that lack of independence. And so on a lot of the slides you'll see a reference to standard deviation as adjusted standard deviation, and that's just to adjust for the effects of auto correlation.
Speaker 1:And last but not least, auto correlation could tell us something, and often does tell us something, about the liquidity or illiquidity in a particular market. Because, of course, if a market is rather illiquid that is, there are very few buyers and sellers and there may be there for times when it's difficult to sell an asset we could expect to see that monthly returns will be not independent of each other. They will be related in some way, and so some people use auto correlation as a reasonably interesting way to evaluate the liquidity in a particular market. I'll point out that in the long term data, the auto correlation for US equities is 3%, so that's basically independent, and the long term auto correlation for the tenure treasury is about 13%. It's a bit higher, but still that's very modest auto correlation. So we could say A that that auto correlation tells us that we don't really have to adjust volatility very much, that those monthly returns are pretty independent of each other, and, second, it tells us that these are big liquid markets. There's very little illiquidity in those two markets.
Speaker 1:The last measure that I'll mention is something called the information ratio or the sharp ratio. How do we evaluate the relative benefit of owning assets in a portfolio one to another? And the answer there is we look at something called the information or the sharp ratio. If the risk that our return estimate is wrong is measured by volatility, we have our caveats from above. But let's just take it as a given that volatility will tell us something about uncertainty and about risk, and by combining multiple assets whose returns don't move in unison we can reduce the volatility. Then we should want a way to evaluate the relative improvement of adding non-correlated assets in a portfolio, and the information ratio is one such measure. It's simply the average return divided by the average volatility. The sharp ratio is the same thing, except it's a little more fancy, because the sharp ratio is the average return in excess of the risk-free rate divided by the average volatility.
Speaker 1:In this class we will typically talk about the information ratio, but sometimes, if I refer to academic studies, they more typically use the sharp ratio. It's really basically the same idea. It tells us the return earned for one unit of volatility. The higher the information ratio, the more the portfolio has succeeded in being more efficient. Us equities have, over the long period, an information ratio of 0.65. That is to say, for every unit of volatility you earn 65 basis points in return. Okay, that tells us something about the efficiency of owning an equity portfolio. How about the efficiency of owning a bond portfolio? Well, in owning a bond portfolio, for every unit of volatility you earn 0.58% return. So a bond portfolio for the last several decades, from 1971 to 2022, paid us less in return per unit of volatility. That means it's a slightly less efficient portfolio at earning return than equities. Notice, I'm not saying it's more risky or less risky, because this is per unit of risk, it's per unit of uncertainty, and so here, what we're really just commenting on is the efficiency of owning that kind of risk.
Speaker 1:Okay, what are the problems with applying probability theory to markets? I've already alluded to some of these, but let's just knock them out First. Independence we have to assume that asset prices move independently of each other and that their monthly returns are random. That's what's thought of as a stationary process, meaning that we know the rules that apply. We know all the possible outcomes. Like a coin toss, markets regularly exhibit a breakdown of independence. There is a long-term trend in equity markets that is up, which is arguably not really random. In the short term, there's evidence of momentum in equity prices. We saw in our data how it's possible to observe that in equities. All of which tells us that using probability theory is prone to pretty big error, because in forecasting market returns and market risks, we're using a tool that assumes independence when independence doesn't exist.
Speaker 1:Second, radical uncertainty. Remember we talked about the book by Mervyn King and his co-author that in a non-stationary system like capital markets, there are events which occur and that they cannot be described in advance. We don't even know what those events will be. They can't be described and they certainly therefore can't be predicted. So it's impossible to assign probabilities to events that we haven't even thought of yet, and so probability theory simply cannot contemplate that notion. It cannot contemplate the notion of radical uncertainty. Instead, when we talk about using probability in a non-stationary system like markets, what we're really doing is just expressing the strength of our belief. We're not saying that probability theory tells us what the returns of equities will be in the future, because we simply know it won't. But if we say returns of equities in the future are going to be 10%, that means I'm expressing a level of confidence that that's the general direction of equity returns over a long period of time, but I can't be certain what they're going to be, and by using probability theory, we can then think about narratives. We can look at the historical data and construct a narrative of what's going on here in order to try to develop a theory for ourselves about what a particular asset class will or won't earn in the future.
Speaker 1:Third and lastly, easy misapplication. It's really easy to apply statistics incorrectly. So, for instance, in the beginning I mentioned that the mean is not the most likely outcome. I predict that a very large portion of you listening were surprised by that statement. The average person thinks that the average is the most likely outcome, and why? Well, that's because in English, when we say average, we mean most likely. We don't mean a statistical measure of the number around which all the observations are clustered, which is what we say in statistics, and so the difference in those meanings can create misapplication and confusion.
Speaker 1:It also is the case that we can't assign probabilities to unique events because we hadn't thought of them yet, and so to use statistical theory is to assume that the future will look exactly like the past, and we all know that that's not going to be true. How many people before 2020 could have predicted COVID? How many people before the 1990s could have predicted the dot com bubble? How many people before 2008 could have even anticipated the global financial crisis? All of these events that happen in our world are because we live in a non-stationary system, and in a non-stationary system, statistics that are used to express the strength of our conviction make perfect sense, but when we start using statistics to describe or predict the future, we get into very, very dicey territory, right? So let's look at an illustration that I think will kind of pull all of these ideas together for us, and the case study that I want to consider is the so-called lost decade. The lost decade is generally thought of to be 2001 through 2010.
Speaker 1:And again, on the website, you'll find some charts if you're interested, but no need to find them if you're not that interested. I can just describe them to you. If you look at the returns on the S&P 500 during that period, what we see is that they started the period and ended the period at roughly the same amount. When we look at that chart, we can see return, because we see negative return where the value is declining. We can see positive return where the line is increasing. We can also sort of see volatility because we see the line moving up and down. We can see how unstable the value is over time and we can also see that there was very little compensation for owning equity risk during that decade, on average about 1.4%.
Speaker 1:In contrast, I can put up a chart where I show both the S&P 500 and Apple during the same period and we randomly say let's make them both valued at 100 in 2001, just so that the chart can lay one on top of the other. What happens suddenly when you do that is that compared to the volatility of Apple, you don't really see any volatility in the S&P 500. It looks like a straight line. It's not straight, but it looks like it's a straight line and that's because Apple is so much more volatile that the chart masks its volatility. And this is sort of one important lesson when reading the newspaper or looking at statistics that someone is sharing with you to be very suspicious of data sets where people are putting two different items together and one seems to be very, very flat and the other seems to be not so flat, because, generally speaking, what they're doing is really masking the volatility of one and minimizing, in a sense, the volatility of the other.
Speaker 1:We could see the significant return and volatility of Apple, but we stopped being able to see the volatility in the S&P 500. And how much did we get paid for owning the risk of that uncertainty, of that volatility for Apple? Well, it turns out that we earned 45.8% per year on average. That's quite a lot for owning the volatility of Apple. What was the information ratio? Information ratio was 1.06. We earned 1% return for every percent of volatility. And what was the information ratio for the S&P 500 during that decade? 0.09. We earned 9.00 of a percent for every 1% volatility.
Speaker 1:Owning Apple was a far more efficient investment, but of course, be careful. It was efficient because it paid us more, but that's because we know how the story ended. X-anti, you can't know how the story ends. What was the correlation between Apple and the S&P 500 during that period? It was 55%. Now again, if we look at that chart, it's very, very difficult to see any correlation in the chart at all. But by computing the correlation we can see that Apple's price tended to co-move in the same direction, just not to the same extent as equities as a whole.
Speaker 1:What accounts for the difference between Apple and the S&P 500? How do we make any sense of this? Apple operates in the same economy as all the other companies in the S&P 500. It's much riskier than owning all of the companies in the S&P 500 because you've just got one. There's a lot more leverage to Apple because it has more debt as a measure of the firm's overall finances than the average S&P 500 company, and Apple is more sensitive to product changes in the economy. If people stop wanting to use Apple devices, that's going to really hurt Apple significantly. If people like using Apple devices, it will help. So what accounts for the difference is that Apple has what's called idiosyncratic risk. It has risk in addition to just the risk of being a stock. There is some risk of being a stock that you might say we can measure, and in a moment I'll describe how we can measure it. But there's some risk of being a stock which tells us well, that's just something about the way in which stocks are moving, and then there's going to be some risk that's unique to a company and in this case, what makes for the difference between Apple and the S&P 500 is all idiosyncratic risk.
Speaker 1:The last item in our illustration of how to use probability theory and statistics in evaluating investments is would it matter if we change the time period? So I was looking at the lost decade, 2001 to 2010. What if I expanded it to 1993 to 2022? So I incorporate the upside of the tech boom when I start in 2001,. I'm at the end of the bursting of the tech bubble. What if I back up and I pick up the upside in the tech boom as well as the recovery from the financial crisis, as well as the recovery from COVID? What if I put in what we sometimes say are multiple cycles? Does that really change the results?
Speaker 1:And it turns out it changes the results a bit. The average return for the S&P 500 from 1993 to 2022 was about 9.5%. So that's a lot better than the lost decade. And the return for Apple during the same period was a lot less. It was about 21% as opposed to 46%. But what's fascinating is that the volatility just by changing the time period, the volatility doesn't change much at all. During the 2001 to 2010 period, the volatility for the S&P 500 was 16% and in 1993 to 2022, the volatility was 15%. In the case of Apple, during the lost decade, volatility was 43% and during the longer period it was 42.5%.
Speaker 1:Likewise, you can look at skew and kurtosis and you see that they're roughly the same. They're not terribly different, and so does that tell us? It tells us that changing time horizons massively changes what the average returns are, but it doesn't really change so much the volatility, the skew, their kurtosis or the correlation for that matter. They remain roughly the same. That's it. That's what I've got for you today on probability theory. When we come back in our next episode, we're now going to pivot away from talking about markets and we're now going to start talking about investments, and we'll begin with equities. We have equity markets in order to raise capital, long-term capital, for firms in their business enterprises. As investors, what are we buying when we buy equities? Come back, and we'll listen next time to find out, as always. Thanks for listening.
.jpeg)