Joseph Rogan: Value-Added Evals are Disastrous for Ed Reform


President Barack Obama’s “Race to the Top’’ (RTT) initiative motivated states to develop teacher evaluation systems based on how well students perform on standardized tests. One dozen states were funded in the first two rounds of the competition. Pennsylvania’s application will be considered in the third round.

Joseph Rogan, Ed.D.

One of the outcomes of RTT has been various “value-added measure systems’’ (VAMs) that use students’ past test scores to predict future scores, and to also measure teaching and learning. If students perform as predicted, teachers are seen as “average.’’ Teachers can be rated “above average’’ or “highly effective’’ if their students beat predicted outcomes, or “below average’’ if students do worse.

Tennessee was the first state to receive RTT funding. Its VAM system has received a lot of attention, largely because of its flaws. For example, two-thirds of its educators teach subjects that are not tested. Social studies, art, music, science and physical education teachers, as well as special educators, specialists who work with challenged or gifted students and teachers who team-teach must be evaluated based on other teachers’ reading or math scores.

Civil rights groups challenged the District of Columbia’s system. Highly-rated teachers are concentrated in affluent schools, while schools in less affluent neighborhoods are staffed by lower-rated educators. The reality is that teachers in impoverished areas find it much more difficult to eke out gains. Teachers that are assigned to the city’s impoverished schools are far more likely to be seen as inferior, denied tenure and dismissed no matter how well prepared or caring they are in the classroom.

In New York, recently, 700 principals – the people charged with doing the ratings – protested that state’s VAM system. They observed that standardized tests are woefully inaccurate and unreliable.

The move to improve evaluation is based on the assumption that we are falling behind the rest of the world because our teachers are not good enough. Indeed, although America’s schools are showing improvement, our relative rankings have dipped. Our competitors around the world have poured resources into improving their systems. Most of these nations have nationalized education. Many cynics want to abolish our U.S. Department of Education and trash its efforts to develop a globally-competitive curriculum. Other nations respect educators, while ours are the latest targets of those who want to eliminate unions.

In a recent paper, Laura Goe of the Education Testing Service (ETS) — the producer of many standardized tests and thus a major beneficiary of the testing movement — discussed the pros and cons of VAM systems. One reason to support them, she says, is that they are inexpensive. Performance-based assessments are more effective, but are costly. For example, in the New Haven, Conn., Change Project, teachers helped to develop a comprehensive system that considers scores, but also focused on excellent teaching. To frequently and thoroughly evaluate its nearly 2,000 teachers, the district needed an ample cadre of dedicated principals. Real evaluation is impossible without evaluators. Many Pennsylvania schools do not have a full-time principal.

Dr. Goe’s concerns outweigh her support. She urged “extreme caution” because, she argued, VAM systems alone are “insufficient” to determine the impact of teaching, thus “it is difficult, if not impossible to isolate an individual teacher’s contributions to student achievement” relative to other factors such as school characteristics, peers, effort, parents, student mobility, curriculum quality, and access to materials and resources.

Methodological issues abound. For example, students who have had a series of either effective or ineffective teachers may demonstrate either better or worse achievement in later years. Is it fair then that achievement or failure is attributed solely to their current teachers? Also, a teacher whose students show gains may be an expert at helping students master a range of worthwhile knowledge, but also might teach to the test instead.

VAMs’ house of cards falls when we examine the integrity of standardized tests, especially those built on the cheap. Last summer, the Los Angeles Times published elementary teachers’ test-based rankings. The National Education Policy Center found that the newspaper’s procedures were “demonstrably inadequate” and its rankings a “disservice to the teachers, students, and parents,” according to the article. When the researchers applied a different research tool, they found a completely different result.

Teachers do far more than fill students’ buckets. They contribute to important outcomes, such as socio-emotional wellness, civic engagement, moral character, open-mindedness, and motivation for continued learning — all of which are not measured by standardized tests. What test considers what it takes to get and keep a job, solve a social problem, settle an argument, have a successful marriage, raise a family, volunteer to help flood victims, or participate in a community?

When teachers complain about such matters, they appear defensive. However, if done properly, teachers would welcome the results of evaluations. When done poorly, to save money or for political purposes, evaluations are unfair and a waste of resources.

Teachers should be concerned. In September, President Obama effectively cancelled the ridiculous “No Child Left Behind’’ (NCLB) mandate that all students be “proficient” by 2014. Teachers have not rejoiced because the standardized tests remain. However, instead of holding states and districts responsible, the new arrangement places the whole burden of success or failure at the hands of individual teachers.

VAM systems seem so simple, but teaching and learning and thus the evaluation of teachers and learners is complex. As we discovered with NCLB, one-size-fits-all systems are disastrous. Educational reform cannot fit on a bumper sticker.

Joseph Rogan, Ed.D., is a professor of teacher education at Misericordia University in Dallas, Pa. Contact the author at jrogan@misericordia.edu.