Goldman will pay about $1 billion to Fannie and $2.15 billion to Freddie for the securities it sold.
"We are pleased to have resolved these matters," said Goldman Sachs Group Inc. general counsel Gregory Palm in a statement.
The settlement is the latest federal government agreement over actions related to the financial crisis that struck in 2008. The crisis, triggered by vast sales of risky mortgage securities, plunged the economy into the deepest recession since the Great Depression.
The securities soured after the housing bubble burst in 2007, losing billions in value.
The government rescued Fannie and Freddie at the height of the crisis in September 2008 when both were on the verge of collapse. The companies received taxpayer aid totaling $187 billion. They have since become profitable and repaid the full bailouts.
The FHFA sued 18 financial institutions in 2011 over their sales of mortgage securities to Fannie and Freddie. The total price for the securities sold was $196 billion.
A number of big banks, including Goldman, JPMorgan, Bank of America and Citigroup, previously have been accused of abuses in sales of securities linked to mortgages in the years leading up to the crisis. Together, they have paid hundreds of millions in penalties to settle civil charges brought by the Securities and Exchange Commission, which accused them of deceiving investors about the quality of the bonds they sold.
Goldman agreed in 2010 to pay $550 million to settle the SEC's charges, the largest penalty against a Wall Street firm in the agency's history.
In recent months, Justice and state regulators have reached multibillion-dollar agreements over mortgage securities with JPMorgan, Citigroup and Bank of America. The most recent was announced Thursday with Bank of America, the second-largest U.S. bank, which is paying a record $16.65 billion — $7 billion of it earmarked for consumer relief.