Just two weeks before Hurricane Sandy barreled up the east coast, leaving behind her a ghostly, flooded and blacked-out Lower Manhattan, Wall Street’s major trade organization, the Securities Industry and Financial Markets Association, held its annual industry-wide disaster-plan drill. On a Saturday in October every year since 2003, SIFMA has convened hundreds of market participants — regulators, utilities, exchanges, market-data vendors and others — to send dummy data to one another’s backup sites to ensure that they can effectively accept orders.
The annual drill was one of many steps Wall Street took after the terrorist attacks on September 11, 2001, in an attempt to ensure that the financial epicenter would be able to maintain operations during just about any type of disaster, whether it be a once-in-a-century superstorm or the type of attack that seemed inconceivable on September 10, 2001 but became terrifyingly real the next day.
In the week since the arrival of Hurricane Sandy — which forced U.S. stock markets to close Monday and Tuesday — some say the storm has made it clear that Wall Street still has a ways to go in shoring up its business continuity plans.
“Gotta say, in the era of electronic trading and distributed computing, it’s pretty silly that local storm closes stock markets for whole USA,” said Henry Blodget on social media platform Twitter on October 29. Blodget is the former head of the global Internet research team at Merrill Lynch who was banned from the securities industry for lying in his published stock reports and the current editor and CEO of The Business Insider.
Other echoed his sentiment, arguing that the New York Stock Exchange would have been able to stay open both days had it reverted to its electronic-only trading system, a scenario for which the market wasn’t sufficiently prepared.
Joe Mecane, an executive vice president at NYSE Euronext and the co-head of the U.S. listing and cash execution businesses, says that while obviously “the bias from all venues is to open no matter what,” it’s not his exchange’s tip-top priority to open in every circumstance, come hell or — very literally — high water.
“I don’t think there’s a mandate that no matter what is happening in the world, or in a particular geographic region, that the markets have to open no matter what,” he says. “Frankly, anyone that’s saying that, I think, either has a differing philosophical view or is acting out of self-interest. If every 100 years we have one of these events and we close the markets, I’m not quite sure why that’s terribly controversial.”
But Damian Handzy, for one, believes NYSE and the rest of Wall Street could have done better. Handzy is CEO of Investor Analytics, a risk management firm headquartered at 55 Broad Street, part of the Lower Manhattan flood zone. He says firms should be able to maintain market operations in Sandy-type situations by coming up with one-size-fits-most-disasters gameplans. He believes his firm has managed this.
“Our reports have come out every day this week,” Handzy says. “I’ve got to tell you, some clients expressed real surprise. But we do risk management, and if we weren’t up, it’d be kind of embarrassing.”
Granted, Investor Analytics employs only 36 people, a far cry from the hundreds of employees and dozens of departments and functions that comprise Wall Street’s major banks. But from where he stands, Handzy says the disaster preparedness solutions that seemed most obvious after 9/11 aren’t necessarily the most flexible ones.
After the attack on the Twin Towers over a decade ago, major firms scrambled to organize backup sites outside the New York City metro area where critical staff could evacuate and continue firm operations as necessary.
“Larger firms have backup facilities probably fifty to eighty miles away from their primary bases of operation,” explains Karl Schimmeck, vice president of financial services operations at SIFMA, the trade organization. “The sites are pre-setup, ready to go; they have desks, computers, everything — they can just walk in, turn them on and get connected. They have people identified who will go to those facilities and conduct their normal functions, whether it’s trading operations, clearance, et cetera. Other groups, who might not be in critical function, will work from home.”
Handzy believes this approach is flawed, and expects that Sandy will force in it some major tweaks.
Institutional Investor reporter Katie Gilbert caught Handzy by phone while he worked in a Dunkin Donuts near his home in New Jersey — the one spot in the area where he could access cell phone coverage and Internet — to talk to him about the thinking that went into Investor Analytics’ contingency planning, how the NYSE should try to be more like London and what the Street can learn from Sandy.
How has your firm’s disaster contingency plan worked out for you?
This actually fell kind of according to plan. When we designed our disaster recovery procedures and locations, we took a look around and asked ourselves what best-in-breed disaster recovery solutions were out there. The first thing we latched onto was the Internet. The Internet was designed by the U.S. government as a military communication tool so that they would never lose touch with each other. The whole point is it’s not a spoke and hub system. It’s distributed. Any spoke can be taken out and the network still runs.
We said, “Okay, how can we take advantage of this?” We said, “Well, we have to have an office, which puts a hub in one place, but our data center doesn’t have to be in the office, so we now have two data hubs — one is in New Jersey and other is in Pennsylvania.” We did that so we’d have them on different electrical grids, and one stands as a backup for the other.
Now, how do employees interact with the data center? We decided to set everybody up [in their homes] with a secure tunnel connection to both the alpha site [the data center in New Jersey] and beta site [in Pennsylvania]. So everybody can work from home and connect to their computers and hard drives in the office, and they can connect to computers in New Jersey and Pennsylvania.
The whole point is that if there is a disaster — not knowing what type — some of us will have electricity, some will have phones, some will have nothing at all. It doesn’t matter who does or doesn’t; as long as a core number of people have access to the site, it’s far better to have people working from home than to pick a disaster recovery site, like some firms did. Because then people have to get to that site. And what if the roads are closed, and what if the trains aren’t running and what if things are flooded?
As long as a skeletal staff of people can access the office remotely, we’re up and running.
From what you’ve seen, how are other firms’ disaster plans panning out?
All those firms that decided to, after 9/11, rent disaster recovery sites somewhere 100 miles from NYC that are now sitting on mothballs waiting to be used in case of a disaster — well, this is the week when they have to use those. The challenge now is getting there. And once they get there, what do they do with their employees? Those employees have houses in the NYC area that need looking after, and they’re going to put them in a bus or carpool to get them upstate or to Connecticut or Pennsylvania? In hotels? Those people want to go home and make sure the basement isn’t flooded, the food isn’t rotting in the fridge. One of the lessons that will come out of this disaster is that just having a disaster recovery site far away from your primary office isn’t necessarily good enough when the rubber meets the road. When you’re in the real world, people have to take care of their families first.
I wish I could say we were clever enough to take that into account when we developed our disaster recovery process. We happened to be fortunate enough that we’re taking advantage of it, because people are able to do both at the same time by working from home.
Is the fact that the NYSE had to close for two days a sign of a bigger systemic weakness?
The NYSE is the last big exchange that still has floor trading and where people physically have to be there. Since that seems to be a priority for them, they are at the mercy of getting enough people there. When exchanges can be mobile and electronic, by definition you take out that big requirement of getting lots of people to the spot.
[Mayor] Bloomberg made a big deal of going down and ringing the bell this morning, and [the message was] “Look, it only took a few days.” After 9/11, the exchange was closed four days. Here, people can say, “Ah, this time we did it in two days, so a big improvement.” Yeah, but they always have to move heaven and earth to get that done. It takes generators and getting the people there.
As far as I can tell, [opening up today] really is a for-show, message-sending endeavor. They did that specifically because it’s sending a bad signal if the stock exchange is out.
Just the fact that the NYSE is the last one who still has this model answers the question [of whether it’s an antiquated setup]. Look, even the London stock exchange — as traditional as traditional gets — even they’ve gone electronic.
This may very well be the last straw. And it may not. But I wouldn’t be surprised if they decide to go all electronic. Because at that point they can have data centers in various locations. Any one location is vulnerable.
Do you think Wall Street has gotten better at handling disasters over the past decade or so?
Yeah, I do. Just the fact that folks trade with multiple prime brokers; that’s a lesson from the Lehman collapse, when assets were frozen in multiple jurisdictions. Just trading with multiple prime brokers is already a step in the right direction. Having a disaster recovery site — prior to 9/11, most didn’t have DR sites. At least now they have one.
After Hurricane Sandy, maybe they’re going to think about having multiple DR sites or, if they can, adopt this model where their people can work from home.
