"Investors should see this as a vote of confidence for the economy," said Kristina Hooper, head of U.S. investment strategies for Allianz Global Investors
The Dow jumped 292 points, or nearly 2 percent, to 16,167.97 — another all-time high for the blue-chip index. Shortly before the Fed announcement at 2 p.m., the Dow was up just 47 points.
The broader Standard & Poor's 500 index rose 29 points, also 2 percent, to 1,810.65 and the Nasdaq composite rose 46 points, or 1 percent, to 4,070.06.
The rally adds to what has already been a historic run for stocks. The S&P 500 is up nearly 27 percent, its best yearly performance since the dot-com boom of the late 1990s.
The Fed's decision removes a huge amount of uncertainty for investors, something they hate. The fate of the stimulus program had hung over investors' heads since May. Now that investors have an outline for how the Fed will pull back, investors can move forward.
"The Fed believed the market could handle it," said Art Steinmetz, president and chief investment officer at OppenheimerFunds.
With no economic downturn on the horizon, stocks are expected to continue their rise in 2014, with market strategists predicting gains of 6 percent to 8 percent from current levels.
Starting in January, the Fed will reduce its bond-buying program to $75 billion a month from $85 billion now. The so-called tapering will be the first step toward winding down a program that has been in place, in one form or another, since the financial crisis.
By purchasing bonds and depressing yields, the Fed has kept interest rates low and encouraged borrowing and lending. But all that buying has also led investors to shift money into stocks. That's because the Fed's purchases made bond prices artificially more expensive in comparison to stocks.
The program has given the Fed a big role in the current bull market. The S&P 500 index has surged about 26 percent since the Fed announced a year ago that it would buy the $85 billion in bonds each month. And since the central bank's first round of bond buying at the end of 2008, stocks have soared about 124 percent.
In the last month, as signs emerged of that hiring was picking up, the housing market was improving and manufacturing was strengthening, investors grew more confident that markets could gain traction without stimulus.
The Fed also said Wednesday it is likely to keep cutting its bond purchases. Fed Chairman Ben Bernanke, who is nearing the end of his tenure, said the bank will likely vote for "measured reductions" in upcoming meetings, as long as the economy shows improvement.
Bernanke also said Janet Yellen, who becomes Fed chairwoman once she is confirmed by the Senate, "fully supports" the bank's decision to start reducing the stimulus.
Wednesday's pullback was a surprise to the market, as most strategists and economists expected the Fed to wait until March before pulling back. But investors say it is a welcome surprise.
"We finally have something in motion," said Frank Davis, director of trading at LEK Securities "and we have details on how it's going to work, so it's something the market can embrace."