Theres a lesson in that too. Even as a country, city or industry struggles to make it through a crisis unscathed, the desire to look forward can never completely eclipse the question, what might we have to get through next?
On the morning of September 11, 2001, there was a slight chill in the air as Damian Handzy rode his bike from his home in New Jersey to the local PATH station. There he caught a train to lower Manhattan, exiting through the World Trade Center at about 8:40. Once outside he walked the five blocks south to his office at 80 Broad Street. Handzy didnt hear American Airlines Flight 11 hit the World Trade Centers north tower his best guess is that he was in the elevator en route to his office on the 23rd floor. The first of his colleagues to arrive at work that day, he made a pot of coffee and switched on National Public Radio. Thats when he found out that a plane had crashed into the tower. Handzy walked to the window; he recalls wondering how such a thing could have happened on such a clear, blue day. Then the second plane zoomed by, right in front of the window where he stood. The plane was so close to him, he says, that he was able to see individual expressions on the faces of its passengers an image that will never leave him, much as hed like to let it go. Little did he know how many more strange and horrific images would be tattooed on his memory that day. Back at his office window a few minutes later, looking north up Broad Street, Handzy saw that the air was so thick with papers from the damaged towers that it looked like it was snowing on the roof of the New York Stock Exchange. He heard the next image before he could see it: a crowd of what seemed like thousands of people running down Wall Street, followed by a rolling cloud of black smoke. Handzy sprinted down the stairs of his building into the lobby, and he saw through the glass doors that it was now dark outside. People filed off the street into the building, some of them limping, some of them bloody. Someone mentioned that one of the twin towers had fallen and that it was knocking down other buildings around it. Handzy estimates that at this point, 150 people were sitting around him, but he remembers the lobby was gripped by silence. We were all waiting for the building to fall down on top of us, he says. It was absolutely surreal. After about an hour the black dust began to settle. Handzy and a few of his colleagues ventured outside and began to make their way north toward Midtown. He phoned his father, and while they spoke, a vivid realization washed over him: the discovery of a new awareness he wasnt sure he wanted to have. My reaction was, everything has changed. Everything, recalls the 43-year-old Handzy. Yesterday was a different world; nothing I believed yesterday can be assured today. I felt like Id never be surprised again. He was undergoing a loss of innocence, he realizes now. Handzy categorizes that moment in his memory with two other particularly punctuated points in his life: the instant he understood that Santa Claus doesnt exist and the moment he admitted to himself that he doesnt believe in God. The shock of that day and the new perspective it left behind also changed the way he thinks about his job. Handzy is CEO of Investor Analytics, a risk consulting firm he co-founded in 1999. In the same way that Americans after 9/11 had to consider as very real the once-unimaginable threat of terrorists crashing planes into buildings, he says, the finance industry would have to reconsider whether it was properly taking account of all the risks confronting it. Indeed, the Wall Streetcentered financial sector grappled with how to confront its own loss of innocence in the wake of the 9/11 terrorist attacks. A frightened urgency gripped the industry as regulators and firms hurried to put in place new practices meant to shore up the system against 9/11-size risks. Spending by Wall Street firms on contingency planning and security measures spiked after the attacks, hitting a high of $3.8 billion in 2005 up from $1.8 billion in 2001 before tapering off, according to estimates by Needham, Massachusettsbased research and consulting firm Tower Group. Sixty percent of those funds were dedicated to information technology improvements; the rest was divided between the construction of new building sites and the hiring of security-focused personnel. Some risk experts, however, including Handzy, contend that the financial sector was so focused on fortifying business operations and drawing up documents around whats known in the industry as business continuity planning, or BCP, that it gave short shrift to lessons gleaned from 9/11 about another type of risk: the kind lurking within investment portfolios. From an operational point of view, the industry did a lot of what it should have done after 9/11 disaster recovery planning, technology disruption planning and a lot of those responses were useful, says Dan Borge, a risk expert who spent most of his career at Bankers Trust Co., where he helped design the first enterprise risk system. But another thing we should have done post-9/11 was think to ourselves, Wow, anything can happen. Not just operationally, but on a systemic level, lots of things can go wrong at the same time. I dont think that happened. It was this first type of risk physical, operational risk that executives from Depository Trust & Clearing Corp. had on their minds when in October 2001, just three weeks after the terrorist attacks, they presented their board with a plan. One major breakdown in the financial system that immediately followed the 9/11 attacks was in the clearance and settlement system, which enables buyers and sellers of securities to complete their trades. Employees at clearing banks with backup sites situated near their original sites (a setup that would have been helpful in the event of a building fire) were unable to access either of their facilities. Until the clearance and settlement operations were completed, traders had no way of knowing what their positions or risk exposures were, making it difficult to continue trading. The system seized up. The U.S. stock exchanges remained shuttered for four days the longest suspension since the Great Depression. DTCC is the worlds largest provider of posttrade services and a critical participant in the clearance and settlement area. Internally, DTCC itself didnt undergo any real problems on 9/11 in fact, CEO Donald Donahue says one of his most striking and heartening memories from that day is the unbroken flurry of activity that pervaded DTCC headquarters on 55 Water Street as employees maintained a rapt focus on keeping the company functioning and the financial system stable, even as the streets outside their office backed up with people fleeing lower Manhattan. But the system DTCC helps oversee suffered a blow because firms all around it had been incapacitated. And that was something the firms extensive contingency plans didnt cover. Our focus pre-9/11 was, what happens if something goes wrong with us? says Donahue. But after 9/11, for the first time, we appreciated very concretely that the clearing and settlement system is an ecosystem. Its not something where you can be totally focused on the hubs; you have to also understand whats happening with the spokes. The course of action DTCC immediately undertook foreshadowed the steps the larger banking and finance industry would follow in the coming months. Hubs like DTCC needed a new layer of protection, and that was Donahues first priority. When he met with his firms board in October, the plans he offered were preliminary, but his goal was clear: to spread out geographically DTCCs data, people and operations. Immediately, the board said, Do it, Donahue explains. There was no hesitation, even though it was clear the price tag was going to be in the multiple tens of millions of dollars. By spring 2003, DTCC had dispersed its data-processing and data-retention capabilities, and at the end of 2004, the firm opened an office in Tampa, Florida. It has since opened offices in Shanghai and London. On September 11, 2001, only 12 DTCC staff members worked somewhere other than 55 Water Street. Today more than 800 employees work elsewhere. Geographic redistribution became a key priority for firms across the financial services industry as they sought to parlay the disorientation and shock felt on 9/11 into changes that would help rebuild a sense of safety and control over operations. Morgan Stanley, which lost 1.2 million square feet of office space in the south tower of the World Trade Center, moved employees to Westchester County, New York, while keeping its headquarters in Midtown. (In 2005, Morgan Stanley moved about 2,300 of its employees from various locations back to lower Manhattan, into six floors of leased space at One New York Plaza.) Insurance giant Marsh & McLennan Cos. moved workers to Midtown and New Jersey. Firms built backup sites in suburbs all over the New York City region where managers found fully operational office space from which to run their companies though most dont like to offer details about their locations for obvious reasons. Other firms simply got out of lower Manhattan: Now-defunct Lehman Brothers Holdings, whose headquarters in the World Trade Center was destroyed in the attacks, settled in Midtown, as did Cantor Fitzgerald, the bond trading powerhouse that lost 658 of its then 960 employees in the attack everyone who had been in its office in the north tower of the World Trade Center that day. After DTCC had addressed its internal concerns, Donahue and his team moved on to the spokes. In 2004, DTCC began requiring that its 40 most critical member firms engage in an annual test to demonstrate a full capability to interact with it from their backup sites. The wider financial industry was making a similar shift from hub to whole-wheel thinking. In April 2003 the Securities and Exchange Commission, Federal Reserve and Office of the Comptroller of the Currency released an interagency paper directed at two types of market entities: core clearance and settlement bodies, and significant firms (defined as those that account for 5 percent or more of the volume in a critical market). The report specified that regulators expected these entities to ensure their ability to recover clearing and settlement activities the same day as a widespread disruption. But 9/11 had demonstrated that the various components of the larger financial system had to be able to communicate better too. Roger Mahach, chief security officer at the OCC, says that in the same way government intelligence agencies were forced to question whether gaps in communication had resulted in the failure to ascertain al-Qaedas plans for 9/11, the financial industry had to find a way to better identify risks that applied to the whole. After 9/11, Mahach explains, there was a real awareness within the government, as well as within private industry, that we needed systems that could aggregate, consolidate and integrate different data streams. As that awareness seeped into both government agencies and the financial industry, Mahach says, he began to notice the widespread use of military and intelligence terms like situational awareness and defense in depth in briefings and marketing literature coming from finance firms and financial risk professionals. Those are the types of words and concepts that started to move in and are just taken for granted now in my discipline, he adds. As the years passed, the sense of exigency displayed by the industry about business continuity planning began to ease. In 2004, Dushyant Shahrawat, a senior research director at Tower Group, wrote that the severe urgency that prevailed around BCP post-9/11 had been replaced with a more measured approach that recognizes the actual environment in which firms operate. Much of this early urgency had been funneled into huge IT investments, especially in the case of bulge-bracket broker-dealers and top-tier investment management firms: Shahrawat estimates that the U.S. securities industrys IT-related BCP spending jumped to about $2.5 billion in 2003 (from $1.3 billion in 2000), after which it backed off. Firms were pouring the majority of their money into data storage and retention. Though funding and general levels of urgency did begin to soften, approaches to BCP still looked much different from their pre-9/11 days. Wrote Shahrawat, When the history of business continuity planning is written, the events of 9/11 will stand out for their stark significance and represent a major inflection point when industry perceptions changed significantly . . . and the very definition of continuity planning underwent a transformation. Some of that transformation can be seen in the way firms are incorporating business continuity plans into their personnel structures. Mahach says that at the OCC, security and BCP divisions were reorganized in an effort to break down what had been a more vertical chain of command and encourage conversations between experts in physical security, cybersecurity, privacy and identity management. Morgan Stanley organized a BCP committee made up of the presidents and chief operating officers of major business lines. Firms also sought to distinguish themselves to clients as the most prepared and dependable among their competitors by rolling out new and innovative approaches to BCP. Valley Forge, Pennsylvaniabased investment manager Vanguard Group created a BCP product in the form of a dashboard (think of an airplane console) that shows in real time the status of major businesses and even key employees. If a critical application or person becomes incapacitated, the console registers the change and sends out an alert to all of its users. Another indication of the post-9/11 change in systemwide attitudes toward protection and preparedness crops up once a year, on a Saturday in October, when the financial industry has a chance to test its ability to stay interconnected and communicate in the midst of a disruption. In an industrywide version of the fire drill DTCC holds each year with its member firms, the Securities Industry and Financial Markets Association trade group convenes hundreds of market participants regulators, utilities, exchanges, market-data vendors and others to send dummy data to one anothers backup sites to ensure that they can effectively accept orders. Perhaps the most visible change on Wall Street ten years after 9/11 is the quiet that now pervades the trading room of the New York Stock Exchange a sharp contrast to the pandemonium of traders and brokers on the floor a decade ago. Of course, the move to a more automated, and therefore less noisy, trading floor was already in the works before the twin towers fell. Two years earlier Regulation Alternative Trading System had gone into effect, allowing electronic communications networks to compete with traditional exchanges for trading volume. Ten years ago the vast majority of listed U.S. equities were traded on the floor of the NYSE. Today, thanks to Reg ATS and other regulatory changes designed to increase competition among exchanges, only about one quarter of all U.S. equity trading occurs through the now publicly held NYSE Euronext. But Michael Rutigliano, vice president and broker liaison at the NYSE, says the greater industry emphasis on decentralizing operations and ensuring data redundancy post-9/11 has undoubtedly helped push NYSE floor operations to where they are today. The events of 9/11 caused firms and regulators to reexamine ways to ensure redundancy, he says. We were less electronic then, and were much more electronic now; theres clearly a connection to 9/11. That day played a pretty big role in the market structure changes that followed. Although the infrastructural changes that 9/11 wrought have been large, some prominent financial risk experts, like Handzy, believe that the lessons the financial industry gleaned were incomplete. In general, its really difficult to figure out what causes what and whether or not something is meaningful, says Handzy, who has a Ph.D. in nuclear physics from Michigan State University. Humans are really good at seeing patterns where they dont exist. But one of the things you learn when you study physics is that random events happen all the time, and you start trying to learn how to see the real patterns and not the false patterns. The financial industrys reaction to the events of 9/11 clearly demonstrated that this type of pattern-crafting and association-building was in full effect as firms struggled with what to learn from the tragedy, Handzy explains. An attack had been made on physical infrastructure, so rebuilding efforts and system improvements were largely constrained to infrastructure as well. But 9/11 a textbook example of whats known as fat-tail risk, a low-probability, high-impact event might also have left behind lessons about how to prepare for and deal with other market risks, if industry participants had looked harder for them. And those lessons would have come in mighty handy seven years later, when the industry found itself in the epicenter of another massive crisis. The September 2008 bankruptcy filing by Lehman Brothers, after that firms highly leveraged investments in housing-related assets went deeply south, set off a chain of events that rocked global markets. Firms that had been happy business partners the day before suddenly wouldnt loan to one another for fear of counterparty risk, and the credit market ground to a halt. Handzy says the lack of preparation within the industry for such a bankruptcy revealed deep misunderstandings about fat-tail risk. Leading up to it, it seemed impossible that Lehman would collapse, the same way it seemed impossible for either of the World Trade Center towers to collapse, Handzy says. And then the sense just afterward is that the probability of these types of things happening again just went up. That is patently not true. The key, Handzy says, is to be aware of the very human biases that encourage us to think this way and to learn when to overrule our hunches. But its also important to keep an eye out for risks as theyre forming and head them off at the pass when possible. After 9/11, though government agencies and financial firms took very seriously the need to improve communications among personnel and within operations to sniff out things like terrorist plots, that lesson didnt translate as well into financial risk management. The OCCs Mahach believes that despite the fact that technology designed to pull together disparate types of data and analysis across systems to get a full picture of risk did improve after 9/11, it wasnt used the way it should have been in the context of the financial crisis. The information systems that could provide early warnings either were not being utilized by the right folks, or that technology was ignored, he says. For his part, risk expert Borge wonders whether that collective failure was not necessarily in spite of what the markets went through on 9/11 but, in some sense, because of it. I cant prove this, but 9/11 might have given us a false sense of security, he says. We might have said, If we can get through that, we can get through anything.