The losses put the Standard & Poor's 500 index on track for its fifth loss in a row, matching its longest losing streak since September.
"If they do cut the bond purchases, the knee-jerk reaction for the market will be to move down," said Chris Gaffney, a senior market strategist at EverBank.
The S&P 500 index dropped eight points, or 0.5 percent, to 1,784, as of 3:12 p.m. Eastern. The Dow Jones industrial average fell 69 points, or 0.4 percent, to 15,819. The Nasdaq composite was down 11 points, or 0.3 percent, at 4,035.
Earlier in the week, there were strong reports on manufacturing and construction. Investors will get more insight into how the U.S. economy is doing on Friday, when the government releases its monthly jobs report.
While few investors think that the Fed will announce that it will cut its bond purchases at its meeting this month, many believe policy makers could make the move in March next year.
Several retailers fell after reporting disappointing results. L Brands, the owner of Victoria's Secret, Bath & Body Works and other retailers, lost 93 cents, or 1.5 percent, to $62.34 after reporting that its sales dropped 5 percent last month.
Gaming company Electronic Arts was the biggest decliner in the S&P 500 index after Forbes reported that the company had been forced to delay future games from one of its developers due to ongoing problems with its Battlefield 4 game. The company's stock fell $1.18, or 5.3 percent, to $21.15.
The S&P 500 index has dipped 1.2 percent since the start of the month and is on course to log its first weekly decline in nine weeks. The losses have pared this year's advance for the index to 25.1 percent.
Stocks have surged this year as the Fed's stimulus has helped keep the economic recovery on track and as corporations produced record profits. Low interest rates have also made stocks more attractive in comparison to bonds.
The stock market may be sliding this month as investors sell some of their best-performing holdings given the strong returns this year, said Natalie Trunow, chief investment officer at Calvert Investments, an asset management company.
"I just don't know if folks will try to squeeze another percentage point (out of the market), or just sell and go home," said Trunow.
In government bond trading, the yield on the 10-year Treasury note rose to 2.85 percent from 2.83 percent Wednesday. The yield is the highest it's been in more than two months as traders expect the Fed to reduce its bond purchases. The Fed has been keeping long-term interest rates low with their monthly bond purchases.
In commodities trading, the price of oil rose 18 cents, or 0.2 percent, to $97.38 a barrel. Gold fell $15.30, or 1.2 percent, to $1,231.90 an ounce.
Among other stocks making big moves:
— Microsoft fell $1.02, or 2.6 percent, to $37.92 after Bloomberg reported a Ford company director as saying that CEO Alan Mulally was staying at the automaker until the end of next year. Mulally is considered one of the leading candidates to take the top job at the software company.