How to Understand the Limits of Financial Models

Few people understand the rise of quantitative financial techniques, and their limits, as well as Emanuel Derman does. The South African–born scientist earned a Ph.D. in theoretical particle physics from Columbia University and spent 12 years doing research in academia and at AT&T Bell Laboratories before joining Goldman, Sachs & Co. in 1985. Over the next 18 years, he did pioneering work in derivatives and developed the Black-Derman-Toy interest rate options model with William Toy and Fischer Black. In the process he helped transform Wall Street into a place where rocket scientists rather than street-smart traders were the ultimate power brokers. Derman left Goldman in 2002 to become co-director of the financial engineering program at Columbia and head of risk at Prisma Capital Partners.Derman, 66, has long been aware of the dangers of relying too heavily on mathematical models, having written an early paper on model risk back in 1996. Physicists can determine the laws that govern the motion of atomic particles, he notes, but the human moods that move markets are much more elusive. “You can’t do finance without models, but you have to realize their limitations,” Derman says. “There’s no model that will really capture people’s panic.”  — Tom Buerkle

All that is solid melts into air, all that is holy is profaned, and man is at last compelled to face with sober senses his real conditions of life, and his relations with his kind,” wrote Marx and Engels in The Communist Manifesto in 1848. They were referring to modern capitalism, a way of life in which all the standards of the past are supposedly subservient to the goal of efficient, timely production.

With the phrase “melts into air,” Marx and Engels were evoking sublimation, the chemists’ name for the process by which a solid transmutes directly into a gas. They used sublimation as a metaphor to describe the way capitalism’s endless urge for new sources of profits results in the destruction of traditional values. Solid-to-vapor is an apt summary of the evanescence of value, financial and ethical, that has taken place throughout the great and ongoing financial crisis that commenced in 2007.

The crisis has been marked by the failure of models both qualitative and quantitative. During the past two decades, the U.S. has suffered the decline of manufacturing; the ballooning of the financial sector; that sector’s capture of the regulatory system; stimulus whenever the economy has wavered; taxpayer-funded bailouts of large capitalist corporations; crony capitalism; private profits and public losses; the redemption of the rich and powerful by the poor and weak; companies that shorted stock for a living being legally protected from the shorting of their own stock; compromised yet unpunished rating agencies; government policies that tried to cure insolvency by branding it as illiquidity; and, on the quantitative side, the widespread use of obviously poor security valuation models for the purpose of marketing.

People and models and theories have been behaving badly, and there has been a frantic attempt to prevent loss, to restore the status quo ante at all cost. What is to be done?

I began my professional life as a physicist, studying fundamentals and mastering theory. Then in 1985, I migrated to the center of the quant world at Goldman Sachs. My colleagues were as smart as academics, but more interesting. The work was an interdisciplinary mix of modeling, mathematics, statistics and programming, all aimed at trying to value securities for trading desks. Quants were the theorists, traders were the experimentalists, and we collaborated to develop and explore our models. Though the aim was moneymaking, the environment was collegial and the techniques were remarkably similar to those I’d used in physics. Within a few months I had met and begun to collaborate with Fischer Black, co-inventor of the Black-Scholes option pricing model, which regards markets as equilibrium-seeking systems and models them by analogy with the physics of heat diffusion.

I plunged into work, reading and modeling and programming. I learned the elegant logic behind Black’s work and witnessed the depth of his thinking. I worked closely with traders who survived and prospered by using models. Soon I began to believe it was possible to apply the methods of physics successfully to economics and finance, perhaps even to build a grand unified theory of securities.

After 20 years on Wall Street, I’m a disbeliever in grand financial theories. We do have a variety of financial models, but financial modeling is not the physics of markets. The similarity of physics and finance lies more in their syntax than in their semantics. In physics you’re playing against God, and He doesn’t change His laws very often. In finance you’re playing against God’s creatures, agents who value assets based on their ephemeral opinions. The truth therefore is that there is no grand unified theory of everything in finance; there are only models of specific things. I say “models” because finance relies on modeling the mental qualities of stocks and markets by comparing them to physical realities — the diffusion of smoke, for example. Such analogies, though not unfounded, are partial and flawed. That doesn’t mean that modeling in finance is a waste of time; it means that you have to understand what models are best used for and then be very careful not to discard your common sense.

Physics theories use mathematics to describe abstractions such as position, mass, electric charge and atoms, but there is a unity between the abstractions and the realities of the external world they represent so accurately. The best physics models make predictions accurate to ten decimal places. Finance’s objects of interest — markets, money, assets and securities — are also abstractions, and the aim of financial models, like that of physics, is to find not only the relationships among the abstractions themselves but also the relationships between abstractions and the realities they represent. Unfortunately, financial models are only models, not reality or even close to it.

If you are someone who cannot distinguish between God’s creations and man’s idols, you may mistake models for deep laws. Many economists are such people. If you open up the prestigious Journal of Finance, one of the select number of journals in which finance professors must publish to get tenure, many of the papers resemble those in a mathematics journal. Replete with axioms, theorems and lemmas, they have a degree of rigor that is inversely proportional to their minimal usefulness. Economists for the most part have never seen a genuinely successful theory. The simple models they work with fail to reflect the complex reality of the world around them. That lack of success is not the fault of economists, for people, unlike matter, are difficult to theorize about. But it is the economists’ fault that they take their simple models so seriously.

What Is to Be Done?

At the end of the cold war, we imagined a future with no more history, a smooth stroll into the sunrise accompanied by democracy, privatization and free markets. It hasn’t worked out that way. Authoritarian versions of capitalism have spread. Privatization has become oligarchy. The gaps between rich and poor, managers and workers, and owners and employees have widened. Economic models have misfired, and financial models have proved to be enormously inaccurate. More recently, the prescribed cure of a Keynesian stimulus to jump-start spending and employment has had only a muted effect. Low interest rates, the Federal Reserve’s cure for past crises and the progenitor of future ones, are being prescribed again. Lessons have not been learned.

I wasn’t surprised by the failure of economic models to make accurate forecasts. Any assurance economists pretend to with regard to cause and effect is merely a pose. They whistle in the dark while they write their regressions that ignore the humans behind the equations. I was similarly unsurprised by the failure of financial models. Sensible people don’t forecast with financial models; they use a model to transform forecasts of future parameters (price per square foot, for example) into present value. Everyone should understand the difference between a model and reality, and no one should be astonished by the inability of one- or two-inch equations to represent the convolutions of people and markets.

What did shock and disturb me was the abandonment of the principle that everyone had paid lip service to: the link between democracy and capitalism. We were told not to expect reward without risk, gain without the possibility of loss. Now we have been forced to accept crony capitalism, private profits and socialized losses, and corporate welfare, in the hope of restoring the status quo ante at any cost. We have seen corporations treated with the kindness owed to individuals, in the hope, perhaps, that their well-being would trickle down to individuals, and individuals treated with the kindness owed to objects. We have forgotten what Arthur Young, an 18th-century English writer, noted at the start of the French Revolution: Only the poor can consume in numbers sufficient to sustain other trades.

Capitalism’s problems will not be solved by models. But just as certainly, financial models are not going to disappear. Data alone doesn’t tell you anything. Theorizing and modeling are what humans do and will continue to do. So how do we use models wisely and well?

First, one must recognize that there are no genuine theories in finance. In physics Newton’s laws and Maxwell’s equations are facts of nature, equivalent and identical to the phenomena of mechanics and electromagnetism that they describe. In finance the Efficient Market Model’s assumption that stock prices behave like smoke diffusing through a room is not even remotely a fact. It is a metaphor, entirely approximate and limited, as are all financial models.

Wise practitioners know that the point of a model in finance is not the same as the point of a model in physics. In physics one wants to predict or control the future. In finance one wants to determine present value and goes about it by forming opinions about the future, about the interest rates or defaults or volatilities or housing prices or prices per square foot that will come to pass. Models are used to interpolate or extrapolate from the current known prices of liquid securities to the estimated values of illiquid securities — estimating the value of a stock option, for example, by thinking of it, as the Black-Scholes model does, as a hybrid, part stock and part bond.

Given the inevitable unreliability of models and the limited truth or likely falseness of the assumptions they’re based on, the best strategy is to use them sparingly and to make as few assumptions as possible when you do. Here are some other observations I’ve found useful in modeling:

• Avoid axiomatization. Axioms and theorems are suitable for mathematics, but finance is concerned with the real world. Every financial axiom is pretty much wrong; the most-relevant questions in creating a model are, how wrong and in what way?

• Be vulgar in a sophisticated way. In physics it pays to drop down deep — several levels below what you can observe — formulate an elegant principle, then rise back to the surface to work out the observable consequences. Think of Newton or Maxwell. Finance lacks deep scientific principles, so it’s better to stay shallow and use models that have as direct as possible a path between the securities whose prices you know (that is, “vulgar” securities that are liquid and common) and the security whose value you want to estimate.

Markets are by definition vulgar, and the most useful models are vulgar too, using variables (such as price per square foot) that crowds use to describe the value of the assets they trade. One should build vulgar models in a sophisticated way. Some of the best and most practical models involve interpolation, not in prices but rather in the intuitive variables sophisticated users employ to estimate value — volatility, for example.

• Sweep dirt under the rug, but let users know about it. One should be humble in applying mathematics to markets and wary of overly ambitious theories. Whenever we make a model of something involving human beings, we are trying to force the ugly stepsister’s foot into Cinderella’s pretty glass slipper — it doesn’t fit without cutting off some essential parts. Financial models, because of their incompleteness, inevitably mask risk. You must start with models, but then overlay them with common sense and experience.

Whenever one uses a model, one should know exactly what has been assumed in its creation and, equally important, exactly what has been swept out of view. A robust model allows a user to qualitatively adjust for those omissions. The Black-­Scholes model is robust: Its main assumptions are that the risk of an option is related to the risk of the underlying stock and that the market will be in equilibrium when both option and stock provide the same excess return per unit of risk. That’s a sensible idea, no matter how naively you define risk. The dangerous part of Black-Scholes is the further assumption that the sole risk of a stock is the risk of diffusion, which isn’t true. But the more realistically you can define risk, the better the model will become.

• Use imagination. The perfect axiom or model doesn’t exist, so we have to use imperfect ones intelligently. Smart traders know that you have to combine quantitative models with heuristics. When people build models — when options modelers assume stock prices diffuse like smoke, for example — they make all sorts of imaginative assumptions that they then formulate mathematically.

When someone shows you an economic or financial model that involves mathematics, you should understand that, despite the confident appearance of the equations, what lies beneath is a substrate of great simplification and — only sometimes — great imagination, perhaps even intuition. Even the best financial model can never be truly valid because, despite the fancy mathematics, a model is a toy. No wonder it often breaks down and causes havoc.

• Think of models as gedanken experiments. No model is correct, but models can provide immensely helpful ways to estimate value. I like to think of financial models as gedanken, or thought, experiments, like those Einstein carried out when he pictured himself surfing a light wave, or Schrödinger when he pictured a macroscopic cat subject to quantum interference. I believe that’s the right way to use mathematical models in finance, and the way experienced practitioners do use them. Models are only models, not the thing in itself. Models are better regarded as a collection of parallel thought universes to explore. Each universe should be consistent, but, unlike the world of matter, the world of financial concepts and the minds of the humans that interact with them are going to be infinitely more complex than any model you make.

• Beware of idolatry. The greatest conceptual danger is idolatry: believing that someone can write down a theory that encapsulates human behavior and thereby free you of the obligation to think for yourself. A model may be entrancing, but no matter how hard you try, you will not be able to breathe life into it. To confuse a model with a theory is to believe that humans obey mathematical rules, and to invite future disaster. Financial modelers must therefore compromise. They must decide what small part of the financial world is of greatest current interest to them, describe its key features and then mock up only those features. A successful financial model must have limited scope and must work with simple analogies. In physics there may one day be a theory of everything; in finance and the social sciences, you have to work hard to come up with a usable model of anything.

The Financial Modeler’s Manifesto

Back in 2009, in response to the financial crisis, fellow quant Paul Wilmott and I came up with several principles for scientists to follow when applying their skills to finance:

• I will remember that I didn’t make the world and that it doesn’t satisfy my equations.

• Though I will use the models that I or others create to boldly estimate value, I will always look over my shoulder and never forget that the model is not the world.

• I will not be overly impressed by mathematics. I will never sacrifice reality for elegance without explaining to end users why I have done so.

• I will not give the people who use my models false comfort about their accuracy. I will make the assumptions and oversights explicit to all who use them.

• I understand that my work may have enormous effects on society and the economy, many beyond my apprehension.

Similarly, I believe that the designers of financial products should create securities whose purpose, exposure and risks are clear. Unnecessarily bundled complex products whose risks are obscure are often more profitable for the firms creating them than simple ones because their value is hard to estimate. If products were transparent, good modeling would be easier.

I am deeply disillusioned by the West’s response to the recent financial crisis. Though chance doesn’t treat everyone fairly, what makes the intrinsic brutalities of capitalism tolerable is the principle that links risk and return: If you want to have a shot at the upside, you must be willing to suffer the down. In the past few years, that principle has been violated. When Wall Street and the U.S. economy were threatened, the ethical principles of capitalism took a backseat. After the bailout of the financial sector, after the provision of cheap government loans to banks at taxpayers’ expense and after the banks’ rapid rebound from taxpayer life support to record profits and bonuses, I am ashamed of the hypocrisies of the system. If you want to benefit from the seven fat years, then you must suffer the seven lean years too, even the catastrophically lean ones. We need free markets, but we need them to be principled.

Excerpted from Models. Behaving. Badly: Why Confusing Illusion with Reality Can Lead to Disaster, on Wall Street and in Life by Emanuel Derman. © 2011 by Emanuel Derman. Used by arrangement with John Wiley & Sons. All rights reserved.