TD Bank is paying a $37.5 million penalty to the OCC and a $15 million penalty to the SEC.
The bank neither admitted nor denied wrongdoing in its settlement with the SEC. But it did agree to refrain from future violations of securities laws.
The OCC, a Treasury Department agency, said TD Bank's failure to file the suspicious activity reports was "significant and egregious."
TD Bank said in a statement that it "is pleased to resolve these regulatory concerns and to put the Rothstein matter behind us."
TD Bank Group, based in Toronto, is Canada's second-largest bank and ranks among the top 10 banks in terms of assets in the U.S.
The SEC also charged Frank Spinosa, a former TD Bank regional vice president in Florida, with misleading the bank's investors with documents and statements about Rothstein's accounts.
TD Bank, through a regional vice president, produced false documents on bank letterhead and told outright lies to investors, failing in its gatekeeper role," Andrew Ceresney, co-director of the SEC's enforcement division, said in a statement.
Spinosa refuted the allegations through an attorney and vowed to contest them in court.
Spinosa didn't know about Rothstein's fraud and never acted to advance his schemes, the attorney, Samuel Rabin, said in a statement. Spinosa "is now being vilified for discharging his duties as a banker and for various deficiencies within TD Bank's internal administrative and compliance functions," Rabin said.
A federal jury decided in January 2012 that TD Bank owed an investment firm $67 million for the bank's role in Rothstein's scheme. The verdict came in a lawsuit filed by Coquina Investments, based in Corpus Christi, Texas. It was one of several lawsuits filed by investors against the bank.
According to the OCC, TD Bank ultimately paid about $600 million in restitution to investors affected by Rothstein's use of the bank for his scheme.
Testimony and court documents in the Coquina trial showed that Rothstein used his TD Bank accounts as an integral part of the Ponzi scheme. Conspirators in his scheme allegedly posed as TD Bank employees.
Rothstein pleaded guilty in 2010 to a scam involving investments in phony legal settlements. It was one of the largest frauds in South Florida history and triggered the failure of the law firm Rothstein Rosenfeldt Adler.