How did you come to write about skill versus luck?
If I tell you that luck and skill contribute to success, you get it. But your brain doesnt. There is a part of our left hemisphere, dubbed the Interpreter, that creates a narrative every time something happens that explains the cause and effect. Your Interpreter doesnt know anything about luck, so there is a creeping determinism. You believe the thing that just happened is the only thing that could have happened. It plays out in our love of stories. If I present a person with a statistic and a story, the story will swamp the stat. A physician friend says that if he tells a patient that a treatment works 50 percent of the time but the last patient who took the treatment is doing splendidly, then 90 percent of the time people go with it. Why do I care about skill and luck? Because I want to be able to predict or think about the future more intelligently than I do now.
Your new book has a whole chapter on reversion to the mean. Why?
Everyone in the investment world nods affirmatively when you ask about it, like they know what you mean. But collectively, people dont behave as if they understand it. Ive written about this so many times, and I still dont understand it. In situations where skill dictates outcomes and there is no luck, there is no reversion to the mean. And if you have pure luck and no skill a roulette wheel, say there is no reversion to the mean. The best prediction about the outcome will just be an average. But the rate of reversion to the mean is a function of where you lie on the spectrum of skill and luck.
What was the most surprising thing you learned while writing the book?
Its the paradox of skill, and it goes against intuition. If you have an outcome that is a combination of skill and luck, in many fields and by the way, this is [true of] investing as skill becomes greater, luck becomes more important in determining the outcome. I learned this from Stephen Jay Gould, who wrote about Ted Williams, the last player to hit over .400 in Major League Baseball. In 1941 he hit .406. Why has no one been able to replicate this feat? The answer is that the quality of the players is uniformly better than it was in the past, which is to say the distance between the very best player and the average player is much shorter today. The standard deviation of batting averages has gone down. If Ted Williamss batting average was a 4-standard-deviation event in 1941, then in 2011 he would have hit .380.
What does that mean for investing?
This idea, the paradox of skill, is everywhere. This is what Charlie Ellis wrote about in his article The Losers Game in 1975. In the 1950s and 1960s, institutions competed against mom and pop. That is like being at a poker game with a patsy. You can get the money from them all day, so my positive alpha is an individuals negative alpha. In 1975, Ellis said that things had evolved so it was institution against institution; there was no patsy at the table anymore. Things had gotten much more efficient. Now think about the state of affairs in 2013, with even more skilled investors, on average. There has been a steady decline in standard deviation of excess returns in the mutual fund world from 1962 to 2008. It also applies to corporations, which means that the difference between the highest quality and the average quality has narrowed over time. The key is that skills are offsetting. The hitters are much better than they used to be, but so are the pitchers.
So investors have to constantly find new playgrounds where they can be the best?
David Swensen, head of Yale Universitys endowment, said that 20 years ago: If I look at these new asset classes where there is less liquidity and so forth, then I can be the first guy in, I can be more skillful, and I can get excess returns. That has proved to be true. But as everybody else tries to do the same thing, it negates that effect. There is always some frontier that an investor can navigate toward.