Much of the outperformance promised by smart beta indexes may just be the result of backtest bias, according to Research Affiliates.
About two-thirds of smart-beta track records are “backward-looking, frictionless results” that can “bias investors’ live return expectations higher than may be realistic,” Feifei Li, the firm’s head of investment management, and John West, its client strategy head, wrote in a note posted on Research Affiliates’ website.
While backtesting may be useful for gaining a better understanding of risks tied to an investment strategy, there’s reason to be cautious. The results can be influenced by data mining and tend to ignore transaction costs and other fees that nearly obliterate “outperformance” when real money is actually put to work, according to Li and West.
Relying heavily on backtesting can be “a harmful activity if investors are not fully aware of the limitations related to the simulated results,” they wrote in the note.
Few “live” smart-beta track records extend any further back than a decade, and even these live results are not always based on the performance of substantial levels of assets, potentially misleading investors, according to Research Affiliates.
[II Deep Dive: Back-Testing Won’t Help You]
Morningstar data on 125 U.S equity smart beta indexes show they earned an average annualized excess return of 2.8 percent during backtested periods. Once indexes were live, however, that figure dropped to 0.7 percent over 5 years, and 0.5 percent over 10 years.
Given this disparity, Li and West said smart beta investors “shouldn’t expect the perfect alpha production promised by a simulated backtest.” Instead, they argued that investors should expect returns to be lower than what the backtest produced, and “dig deeper” into the results.
“In order to achieve the superior investment outcomes promised by smart beta strategies, investors need to make decisions cautiously and request asset managers provide out-of-sample test results as well as return estimates that incorporate implementation costs,” they said. “Most importantly, we recommend that investors select strategies built on strong underlying economic theory and that have a simple, transparent, and intuitive methodology.”
