No Child Left Behind’s Incentive Game

At a recent education policy gathering of top social scientists from fifteen universities and several policy research organizations, the best line of the day came from a reporter at the back of the room.

Various longitudinal studies of the No Child Left Behind Act (NCLB) had been presented at a conference dedicated to evaluating the 2001 Act by the time a young reporter raised his hand, stood up, and asked with a straight face: “Could any of today’s speakers please tell us whether or not the No Child Left Behind Act is working?”

Everyone laughed. One panelist even refused to respond. From the perspective of the academics, the whole message of the day was that the opaque results from their methodologically-imperfect research made it an impossible question.

But for the policymakers, educators, and parents in the audience, the question was the elephant in the room. Given, the extent to which NCLB affects test scores is unclear; the newly released results are mixed. As Professor Thomas Dee of Swarthmore College quipped, it is a classic “glass is half-full / half-empty” situation. But we already knew that, didn’t we?

The fascinating take-home from the research is that there is consensus about the relationship between test design, incentives, teaching strategies, and the types of academic results generated. How we design the accountability tests that our students take will affect how students are prepared for those tests and thus which groups of scores go up.

Intuitively, this might seem obvious. But the conference’s studies provide consensus evidence for this intuition.

Thus, the reporter’s question is not so impossible after all. Everyone laughed because we all understood his question to mean “have test scores improved overall?” But there is a different way of evaluating whether a particular test regime “works.” An accountability test creates incentives to change how education is administered. On this point, the consensus is that NCLB works. As one presenter remarked tongue-in-cheek: “Test score-based accountability will raise someone’s scores.”

This broad conclusion of the day’s studies was offered as a joke. But if this conclusion is as far as research consensus goes, then let’s stop there. It is at this critical point that difficult choices begin.

What incentives do we want a testing regime to generate? Which groups of students do we want those incentives to address? Who do we want the incentives to encourage—just teachers (as is the case with NCLB) or students and parents, too?

NCLB uses a binary metric: schools either fail or pass based on whether a certain number of students within different racial and socioeconomic subgroups are deemed proficient or not in math and reading. According to several of the conference’s papers, this aspect of the test design leads to a redistribution of school resources.

A study by Professor John Krieg of Western Washington University, for example, found that students of races deemed proficient one year might produce worse scores the next year if they attend a school where students of a different race failed. The reason is that NCLB’s incentives encourage schools to withdraw resources from the proficient group and redirect them to the failing group. Similarly, teachers give more attention to students on the margin of passing and, as a corollary, less attention to low- and high-achieving students.

An alternative value-added metric might measure progress at multiple thresholds (such as low pass, pass, high pass); this type of test would reward improvement by each student rather than improvement by only certain groups of students on the margin of passing. Such a metric would change the incentive structure for schools. Assuming that all students respond to teacher engagement in more or less the same way, there would be less incentive than under the NCLB regime to focus disproportionate efforts on average students and more incentive to engage low- and high-achieving students as well. As Lauren Resnick and her colleagues point out in a recent book, Improving on No Child Left Behind, a single performance standard is too high for some students and too low for others (p. 9).

The recent research clarifies that NCLB reform efforts should focus on how different incentives alter the way students are taught and—ultimately—the degrees to which different students benefit. There is an empirical link between test design, incentives, teaching strategies, and student achievement.

Back to the reporter’s question: is NCLB working? Yes, in the very broad sense that its incentive structure is altering test scores. But expectations have differed greatly regarding what scores are changing and by how much. Those of us whose expectations have been disappointed might want to keep the incentive model but change the incentive structure.

In any event, there is no “improve education overall” button. Tough choices are unavoidable. Education reform is not a zero-sum game, but—given limited resources—there is no way around the reality of trade-offs.