The federal government imposed a $58,000 penalty on the Utah State Office of Education in August for defying testing rules in order to proceed with an experimental testing program.

Though Utah won't lose the money -- the state office must shift it to school districts instead of use it at the state level -- the U.S. Department of Education doesn't often financially punish states. The sanction came after the state failed to give a certain test in two Utah school districts that are piloting a new testing system for Utah, a system that an independent evaluation shows is yielding mixed results.

Normally, Utah schools must give Criterion Referenced Tests (CRTs) each year to students in grades two through 11. The federal government uses those results to determine whether state schools are meeting annual No Child Left Behind goals. Late last year, however, the state asked the feds for permission to not give the test in two pilot school districts -- Juab and Sevier. The feds said no, but the state skipped the tests anyway as part of the pilot program, which uses other tests.

Patti Harrington, who was state superintendent at the time, said she had hoped the new federal education administration would approve of the change even though the previous administration did not.

"We haven't ignored it, and we certainly aren't trying to show rebellion," Harrington said. "Rather we were trying to migrate toward a system that gives meaningful results for parents."

And the penalty doesn't represent a net loss, said current state superintendent Larry Shumway. The penalty money, which would have been used for state-level Title 1 training and administration, will go to districts for Title 1 programs, which serve poor students.

Also, because of a change in a federal distribution formula, the state office of education still got more than $60,000 in Title 1 money for the state level.

"It doesn't mean any change in total Title 1 dollars at the state level," Shumway said of the penalty. "This is one of those situations where the state and U.S. Department of Education didn't see it in the same way."

Shumway said not giving CRTs saved students in those two districts from too much testing. He said the state office is working to gain federal permission to skip the tests in the future.

In Juab and Sevier, instead of taking CRTs, students took computer-adaptive tests, which adapt in difficulty as students take them. The adaptive tests, which students took three times last school year, are part of a pilot program that many educators would like to see spread statewide. Many argue that the computer adaptive tests, which provide teachers and students with immediate results during the school year, are more effective tools than CRTs. Students take CRTs at the end of each school year, and results aren't available until school is over.

Also, as part of the pilot program, students no longer take the Iowa Test of Basic Skills or the Utah Basic Skills Competency Test. Instead, they take the computer adaptive tests, a computer writing assessment, the ACT and other college prep tests.

An independent evaluation of the pilot program presented Friday found that educators in the two districts are enthusiastic about the computer-adaptive tests.

"Teachers continually praised the immediate feedback and results the tests provided them," said Reino Makkonen, with WestEd, which conducted the evaluation. "They could then set goals with individual students and group students more effectively."

But Makkonen said districts also faced technological problems, such as tests freezing on computer screens and inadequate computer calculators. Also, the testing ate up computer lab time, which could have been used for instruction. Some teachers complained the tests were too long for elementary school students and some secondary teachers saw the tests as less relevant to their instruction.

Also, questions on the computer-adaptive tests did not align to Utah curriculum as well as current state tests and students didn't perform the same on them as on CRTs, said Stanley Rabinowitz, also with WestEd.

Still, state education leaders say they're eager to move forward with the pilot program with a newer, better-aligned version of the adaptive tests. Dixie Allen, a state board of education member, said the state office plans to ask lawmakers to consider lengthening the pilot program, which is slated to end after this school year, and expanding it to more schools.

She said she believes the evaluation isn't totally accurate because the schools didn't give CRTs and adaptive tests at the same time, making it difficult to compare the two. Also, she said if Utah wins federal Race to the Top money, that could help the state pay for technological advances needed to give the adaptive tests.

"We believe we need informed instruction for parents, teachers and students, and we're not doing that with the CRT," Allen said.