Prominent Testing experts caution the U.S.Department of Education in a letter written October 5th to “pursue vigorously the use of multiple indicators of what students know and can do,” said The Board of Testing and Assessments.

U.S. Department of Education and Education Secretary Arne Duncan will soon have finalized the official rules for Race to the Top and on how they propose to use assessments to measure student achievement and teacher-quality improvements under the initiative. $4 billion dollars will be allocated to states that participate.

Follow up:

Congress approved earlier this year, $787 billion under the American Recovery and Reinvestment Act.

The 13-member Board on Testing and Assessment said they could not meet the Education Department's August 28th deadline to comment on the Race to the Top proposed regulations.
The agency requires that all public documents must be reviewed by a group of independent experts before releasing an opinion.

The chairman of the testing panel proceeded to review the draft of rules. Edward H. Haertel said “it was too important an opportunity” to weigh in on the massive federal investment in public schools.

In an e-mail to Education Week, Education Department spokesman Justin Hamilton wrote that the department welcomes “vigorous evaluations” of the Race to the Top program. “The best impact that [the program] can have is if we create a road map for reforms into the future that have solid research behind them.”

Evaluation Concerns

The Race to the Top program is one of two high-profile discretionary grant programs that are part of up to $100 billion in education aid under the economic-stimulus program. On Tuesday, the department issued proposed ground rules for the other program, the $650 million Investing in Innovation, or i3, fund.

In its letter, the testing experts warned against using a single test, such as the National Assessment of Educational Progress, or NAEP, to measure growth in student achievement, and also suggested that the department’s plans to use student growth data to evaluate teachers could be premature.

In its draft regulations, the department proposed using NAEP to “monitor” overall increases in student achievement, as well as progress in closing achievement gaps, saying that the test “provides a way to report consistently across Race to the Top grantees as well as within a state over time.”

But Mr. Haertel said in an interview that while NAEP would be one good way to monitor the various strategies funded by the Race to the Top program, the national exam should not be used as a way to do objective evaluations of those same initiatives.

“Part of this is the respect that we have for the value of the NAEP as a low-stakes auditing tool,” said Mr. Haertel, an education professor at Stanford University. “We don’t need one more high-stakes test to drive curriculum and instruction nearly as badly as we need the long-term trend lines that we get with the NAEP.”

If high-stakes decisions were to be attached to NAEP results, he warned, it could become “just another test that people would start teaching to.”

In its letter to Secretary Duncan, the board also pointed out that only students in grades 4, 8, and 12 take NAEP, and do so every other year. The test, the board’s letter to Duncan says, is not aligned with any states’ academic content standards or curricula, and consequently would not “fully reflect improvement taking place at the state level.”

The board suggests, instead, that evaluations of the various Race to the Top education improvement strategies should be crafted around the “theories of action” of each initiative.
Questions on ‘Value-Added’

Mr. Haertel and his colleagues also raised numerous concerns about the department’s plans to use individual students’ progress over the course of each academic year, the so-called value-added model, as a way to evaluate the effectiveness of teachers. While they expressed support for linking student test scores with their teachers for the purposes of research, they said it would be “premature” to use the “value-added” approach in decisions on actions such as firing teachers or rewarding them. Too little is known about the accuracy of such methods, the board members said.

The board also pointed out practical difficulties in using data to judge teachers. Among them: teachers in grades and subjects that are not tested could not be evaluated in that way; estimates of how much value is added by a teacher can vary widely year to year; and students often receive instruction from multiple teachers, which would make it difficult to figure out who should get credit for learning gains.

The board also weighed in on the department’s proposed requirement that local school districts use data to improve instruction on a constant basis. It cautioned that multiple-choice assessments that can be graded rapidly are not the best tools for figuring out how to tweak what teachers do in the classroom.

Mr. Haertel said that using tests that can be graded within 72 hours “bumps up against the concern that many of us have about using assessments that really measure the full range of knowledge and skills that we want children to acquire.

“What we really need are forms of assessment that require children to construct their own answers and not just select answers from prefabricated choices,” Mr. Haertel said. “But those can’t be graded in a fast fashion. This is a case of where the hopes and dreams of policymakers are getting ahead of realities.”

10/12/09 12:59