The Browns' attorney, Jonathan Turley, said the family will take their fight to the U.S. Supreme Court.
"While disappointed, the Brown family remains committed to this case and the struggle for equal rights for all families in Utah," said Turley, who is a law professor at George Washington University in Washington, D.C.
Brown and his wives sued Utah and the Utah County attorney's office in Salt Lake City's U.S. District Court in 2011, alleging the state's bigamy law violated their constitutional right to privacy by prohibiting them from living together and that the law criminalized their sexual relationships.
The family, which practices polygamy for religious reasons, moved from Lehi to Nevada to avoid prosecution after being investigated by the police and the Utah County attorney's office.
In a landmark 2013 decision, U.S. District Judge Clark Waddoups struck down the cohabitation section of the bigamy law, saying it forced the Browns — who are members of the "Sister Wives" reality TV family — to leave Utah and violated their rights to privacy and religious freedom.
On April 11, the 10th Circuit panel ruled that Waddoups should not have considered such constitutional questions. Instead, the panel said, the family's lawsuit should have been thrown out after the Utah County attorney's office announced that it had adopted a policy only to target polygamists for bigamy when there is also evidence of other collateral crimes, such as fraud or abuse.
That ruling prompted the Browns to ask for a rehearing by the entire court.
Turley said the family left Utah because of "abusive treatment" by the government and that the case centers on the right to be heard in a federal court.
"The 10th Circuit panel ruled that a prosecutor can publicly declare a family to be felons, keep them under criminal investigation, and denounce them for their religious beliefs without fear of being held accountable in a court of law," he said.
The Utah attorney general's office, which declined to comment Friday, has previously said that the appeals court made the right call in their initial denial in April.
Under Utah law, people are guilty of third-degree felony bigamy if they hold multiple marriage licenses or if, when already married, they cohabit with another consenting adult in a marriage-like relationship.