It’s hard to overestimate the importance teacher quality has not only on students’ academic outcomes, but even their outcomes once they leave school. A study out of Harvard, conducted by the economist Raj Chetty, found that those placed in kindergarten classes run by more experienced and better teachers not only were more likely to perform well in later grades, but also were more likely to graduate high school, enroll in college, earn more and save more for retirement. A good teacher, defined by Eric Hanushek of Stanford as landing in the 84th percentile in quality, could add between $22,000 and $46,000 to a student’s lifetime earnings.

It is just such findings that have education reform advocates calling for a more widespread adoption of the “merit pay” system, where teachers’ evaluation scores are linked to their salaries and chances for promotion.

This has met opposition from teachers’ unions and testing skeptics, who argue that it would result in teaching-to-the-test at the expense of actual learning. For a long time, the data has been mixed on merit pay. Two studies from Mathematica Policy Research in 2010 that found little benefit, while a study in Nashville found mild benefits for fifth graders but none for other students.

However, a recent paper, authored by Roland Fryer of Harvard University, Steven Levitt and John List of University of Chicago, and Sally Sadoff of UC San Diego reaches much more definitive conclusions on the merit of merit pay. And those conclusions are that teachers operating under the merit pay system do perform better than their colleagues who aren’t. Furthermore, the authors also determined that how the merit pay system is designed has an enormous impact on how much of an increase in student achievement could be obtained.

The authors split teachers in the study into a control group, who were not offered any rewards, a “gain” group, which was promised rewards of up to $8,000 at the end of the school year, and a “loss” group, which was given $4,000 upfront and asked to pay back any rewards they did not earn. The idea behind the latter group was that loss aversion should motivate teachers to perform better than they would if they only stood to gain more money. Additionally, the gain and loss groups were split, with a “team” group being rewarded on the basis of theirs and fellow teachers’ test scores, and the “individual” group being reward only on the basis of their own scores. The conclusion: it worked, and it worked almost twice as well when the money was given at the start and then taken away.

Using the Hanushek formula, test gains by the students of the teachers either in the Individual Gain or Team Gain groups translated into increase of lifetime earnings of $20,000 to $42,000, while gains of students in Individual Loss and Team Loss classrooms amounted to between $37,000 to $78,000. But aside from that, the authors also determined that teachers participating in the zero-sum incentive program did about as well as those participating in the group incentive plan. This means that testing gains could be obtained without putting teachers in each school in competition with each other.