A new report from Rockford, Ill.-based financial advisor Savant Capital Management and advocate of passive investment strategies Zero Alpha Group, of which Savant is a member, attempts to cast doubt on the validity of widely-known averages, which overstate the performance of actively-managed mutual funds because those dead funds are conveniently forgotten. This survivor bias, according to the report, boosts the apparent returns for 41 of 42 categories in the Morningstar database. It also shows that mutual funds in all nine of Morningstar's famous style boxes, on averaged, lagged behind their respective indices.
Survivor bias occurs when data providers – and mutual fund companies – strike no-longer-extant mutual funds, which generally have pretty terrible track records, from their cumulative data. On the micro-level, when a fund company, for instance, merges one fund into another, it no longer has to report the lesser of the two historical performances, only that of the succeeding fund. On the macro level, it means that the average return for mutual funds taken collectively tends to be higher, according to the Savant report by an average of 160 basis points per year, than it is in reality, since the bad funds have gone away.
The former problem, of managers essentially hiding bad performance certainly seems worthy of reproach. But what of the latter? Are "inflated" mutual fund average performance numbers a problem? That's much less clear.
Pat Beaird, founder of ZAG member firm BHCO Capital Management, argues that the report "really shows that indexes win hands-down" and the "active managers don't have a leg to stand on." Fair enough, but didn't we already know that? Standard & Poor's has made something of a quarterly rite of showing that, when adjusted for survivorship bias, actively-managed equity funds of all kinds, on average, regularly get creamed by the corresponding S&P indices. "The evidence," report author and Savant managing director Brent Brodeski says, "is very damning that active management doesn't work regardless of the category."
A fair assessment, as the S&P data has shown again and again, but one that, in a sense, is missing the trees for the forest. On an average basis, as with all things, the lesser performers drag down the apparent performance of the group. But that doesn't necessarily mean that there aren't good managers out there, just that a lot of investors – though not, say, Bill Miller's – would be better off in an index fund.
Indeed, the survivor bias that the report demonstrates in Morningstar data, far from being "a kind of grade inflation for mutual funds" that "make the remaining managers look better," as Brodeski claims, actually makes those managers look worse compared to the category average. For instance, if, as the report shows, large-cap blend funds collectively returned 190 basis points less than reported by Morningstar over an annualized period, don't the funds within the band between the lower survivor-bias-adjusted number and the standard Morningstar number have as much to gripe about as index-based investing proponents? Brodeski concedes that, since the survivors tend to be the better funds, this is correct.
"Fund that survive actually end up looking worse compared to peer averages," he said. "[Investors] are looking at a subset of above-average funds to start with, and in fact, the surviving funds are not given a fair deal when compared to their peer averages."
Basically, then, the problem the ZAG members have is if investors, of whom Brodeski says "very few" know about survivorship bias, are using category averages to decide how to invest their money, and use it to pick active management over passive management. That argument is specious, at best, because even without survivor bias, category averages tend to lag indices.
In many cases, survivor bias is just extending the indices' lead. The report also provided no evidence that investors actually use survivor-biased category averages to pick active management over passive management. "The way most people use category averages is not to demonstrate the theoretical superiority of indexing," posits Morningstar managing director Don Phillips. "They use it to help compare their fund to other funds."
Phillips takes no issue with the argument that adjusting for survivor bias is a legitimate way to measure performance. In fact, he says Morningstar plans to begin publishing survivor-bias adjusted category averages later this year alongside the ones that are already there. He notes, and the report's authors agree that the report's findings were "not an issue about the accuracy of Morningstar's data or the total returns we calculate on funds."
So what of the conspiracy theory outlined in the first sentence of the report: "The mutual fund industry systematically and significantly overstates fund performance in a way that falsely makes actively managed mutual funds occasionally look competitive with indexes." Morningstar, apparently, isn't behind the conspiracy. Neither are the individual fund companies, which are as likely to be hurt by the numbers as helped.
"It's more about the fund industry in general hanging their hat on a database that shows in some categories that managers appear to beat the market," Brodeski says.
When asked about this, Chris Wloszczyna, a spokesman for the industry association Investment Company Institute, joked, "I don't think that we're able to exert much control over Morningstar's decisions." In the final analysis, he believes "shareholders benefit" from survivorship, if not survivor bias, "because they're in a fund that would presumably have the better performance."
Survivor bias in the averages is real. The question is, does it matter?