If 2013 was the year postcrisis financial regulations transformed the hedge fund industry’s strategic landscape, 2014 may see this trend go a step farther as the gulf widens between managers who can succeed in the new environment and those who cannot. The biggest risk many funds face is strategic: the need to grow assets under management to support margins that have been under pressure from rising operational and compliance costs and investor complaints about fees.

Apart from this operational risk, there are two market risks that could cause managers headaches in 2014. First, the stepwise withdrawal of the Federal Reserve’s quantitative easing program, or other events the markets find unsettling, could trigger a rise in equity correlations and a fall in dispersion, reversing trends that made 2013 a profitable year for long-short and other equity strategies. Whereas the Fed’s December 18 announcement to taper QE did not meet with this type of reaction, talk of a taper back in June did.

The worst-case scenario would be a return to the high-risk environment of 2010, 2011 and part of 2012, when there were huge and costly correlation spikes, says Sebastián Ceria, CEO of New York–based buyside risk analysis firm Axioma. “Things went well this year because hedge funds’ ability to control risk has been facilitated by the fact that correlations have come down significantly,” he says. Axioma, he adds, is currently concerned about what it calls the correlation roller coaster. “When [correlation] spikes, it spikes in a hurry,” he says.

Second, the reduction in bank proprietary trading mandated by the Volcker Rule could affect liquidity in thinly traded markets, causing prices to rise and generating losses for some hedge funds. “The banks argued that the markets would suffer if prop trading went away, back when they were lobbying against Volcker,” says the chief risk officer of a New York multistrategy fund management company, who asked not to be named. “That could be, but it’s a hard risk to quantify at this point.”

By contrast, the regulatory risk faced by small and medium-size hedge funds is already having an effect, causing them to urgently attempt to boost assets under management (AUM).

An annual survey of hedge fund managers and investors by Ernst & Young, conducted in July and August 2013, found that growth was the top strategic priority for more than two thirds of participating managers.

“We’ve seen the cost of business go up significantly, and in order to survive and grow their margins, hedge funds have been looking for different ways to grow,” says Ernst & Young partner Natalie Deak Jaros, a member of the firm’s global hedge fund practice. The rollout of registered funds under the Investment Company Act of 1940 with low fees and minimums; low-fee, long-only products; and private equity vehicles is an example of attempts by hedge fund managers to tap new sources of investor demand.

This scramble for assets favors large firms. “It brings a whole host of risks, such as additional regulatory scrutiny and operational risk,” she says. Investments in personnel, technology and other infrastructure needed to manage greater AUM are scalable, so funds large enough to develop and sell many of these products can make the investment pay off. These tend to be funds with at least $10 billion in assets, she says.

The problem is particularly acute for start-ups. “The barriers to entry in terms of setup operational costs — onshore regulatory, compliance, due diligence, systems requirements, personnel and other infrastructure costs — are only going to grow,” says Ian Gobin, partner and global head of the funds and investment services practice at law firm Appleby in the Cayman Islands. “And for a start-up hedge fund manager, this is problematic.”

Small funds are also at a disadvantage in the pursuit of institutional money, as investors increasingly prefer large funds with best-in-class compliance infrastructure. “If you intend to attract institutional money, a manager needs a meaningful front-and-back office operation and a strong track record,” Gobin says. “And even so, a lot of new hedge funds get stuck around the $50 million assets under management mark. Institutional investors tell them to come back when they reach $100 million or more, but they can’t get there without institutional money.”

One development that could help small funds build up their assets is the Securities and Exchange Commission’s new regulations permitting general solicitation of investors, mandated by the Jumpstart Our Business Startups (JOBS) Act of 2012. “The change in the rules on general solicitation is the biggest development in fundraising in a long time,” says Jeffrey Blomberg, a partner at law firm Withers Bergman in Greenwich, Connecticut, who advises funds and institutions on private investment transactions. Whereas funds will have to provide more information to the SEC, they could see the greater scrutiny as a fair trade-off for the ability to advertise and solicit a wide range of potential investors.

The final SEC general solicitation regulations are not yet out, but if they do facilitate growth by smaller funds, they could also reduce the risk of extinction for them. That could be a good thing. “The biggest risk for the U.S. hedge fund market as a whole is the loss of the start-up entrepreneurial hedge fund manager,” Gobin says. If that happens, he adds, “where will investors get their best returns?”

Get more on hedge funds and regulation.