The IRS has shed about 8,000 jobs since 2010, the report said. It comes as the IRS is taking on big new responsibilities implementing President Barack Obama's new health care law, even as congressional Republicans want to cut the agency's budget even more.
"The IRS is facing many new challenges while operating with fewer resources and employees," George said. "Several indicators showed the effect of this, including a decrease in enforcement revenue and a continued increase in accounts receivable."
Overall, federal tax collections are on the rise as the economy slowly improves. That's because the vast majority of Americans pay the taxes they owe, according to IRS statistics.
The IRS said budget cuts are hurting the agency's ability to go after people who can't or won't pay. In June, acting IRS head Danny Werfel told Congress that additional dollars for enforcement would yield a return of better than 6 to 1.
The agency said other factors were also at play. For example, the IRS said enforcement collections were unusually high in 2010 and 2011 because of a voluntary disclosure program for people hiding money overseas. More than 38,000 people disclosed offshore accounts under a program that offered reduced penalties and no jail time.
The sluggish economy also slowed collections, the IRS said, noting that most of the delinquent taxes collected in 2012 were from cases that were opened in previous years.
Like many federal agencies, the IRS has seen its budget and workforce shrink since 2010, when the agency was allotted $12.1 billion. This year, the IRS is expected to spend $11.2 billion. Obama's proposed budget for next year is $12.9 billion — a 14 percent increase over current spending.
As funding declined, the IRS shifted resources from auditing individuals to auditing businesses, according to agency statistics.
In the 2010 budget year, the IRS conducted nearly 1.6 million audits of individuals, including those done by mail. In 2012, the number dropped below 1.5 million.
During the same period, the number of audits for businesses increased, from 58,000 to more than 70,000.
The IRS budget for next year will likely be decided as Congress deals with two financial deadlines this fall. Funding for the federal government is scheduled to run out at the end of September. Unless Congress acts, the government could face a partial shutdown. Later in the fall, the government will reach the legal limit of its ability to borrow money.
Many Republicans in Congress want to use both deadlines to negotiate spending cuts with Obama and his Democratic allies in Congress.