Mike Petrilli is Vice President for National Programs and Policy at the Thomas B. Fordham Foundation, where he oversees the Foundation's research projects and publications, including The Education Gadfly. He is also research fellow at Stanford University's Hoover Institution and Executive Editor of Education Next. Petrilli is author, with Frederick M. Hess, of No Child Left Behind: A Primer, the most comprehensive overview of the law and its implementation to date. He comes to the Foundation from the U.S. Department of Education, where he served as Associate Assistant Deputy Secretary in the Office of Innovation and Improvement. In that role, he oversaw approximately two-dozen discretionary grant programs that support a variety of education reforms, including alternate routes to certification, charter schools, and more, and helped to lead the implementation of No Child Left Behind's public school choice and supplemental services provisions. Before working at the Department of Education, he was Vice President of Community Partnerships at K12, an Internet education company. He started his career as a teacher at the Joy Outdoor Education Center in Clarksville, Ohio. Mr. Petrilli holds a Bachelor's degree in Honors Political Science from the University of Michigan and a teaching certificate in high school The above bio is taken from his web site. In this interview, he responds to questions about "The Proficiency Illusion". This report, is a collaboration of the Thomas B. Fordham Institute and the Northwest Evaluation Association, and contains several major findings. In this interview, he addresses some of the findings and implications.

1) You have recently been involved with the NWEA and the Fordham Institute and released a report entitled " The Proficiency Illusion" What were the basic goals that you were trying to accomplish?

We set out to determine whether states' proficiency "cut scores" on their reading and math tests are high, low, or in between, whether they've been rising or falling (i.e., whether it's been getting harder or easier to pass the state test), and whether they're internally consistent as between, say, reading and math, or fourth and eighth grade. Because achieving "proficiency" in reading and math is the foundation for the No Child Left Behind act, we thought it was important for the public to know what "proficiency" means—state by state, subject by subject, and grade by grade.

2) In one summary I read, it was indicated that states seem to be aiming somewhat low when it comes to expectations for younger children (some of whom may be developmentally delayed). Is this appropriate, or is there more variance in young children?

Many states are internally inconsistent, with higher expectations in seventh and eighth grade than in third and fourth (though occasionally it goes the other way), differences that are far greater than could be explained by conscious curricular decisions and children's levels of intellectual development. This means that millions of parents are being

told that their eight- and nine-year-olds are doing fine in relation to state standards, only to discover later that (assuming normal academic progress) they are nowhere near being prepared to succeed at the end of middle school. It means that too little is being expected of millions of younger kids and/or that states may erroneously think their middle schools are underperforming.

3) You seem to feel that a central problem in NCLB is that it allows each and every state to define "proficiency". In a sense, this is the issue of states' rights- but isn't there a logic also to allowing a very rich state to set different standards than a poor one?

Your "logic" equates to the "soft bigotry of low expectations." At the heart of the standards movement is the idea that all kids deserve to be challenged to high standards, no matter their zip code or family background.

4) A summary of some of the findings indicate a "walk to the middle" in terms of expectations seems to be dropping. Is this a lowering of standards or a regression to the mean effect?

The trends suggest a walk to the middle more than a race to the bottom. The states with the greatest declines in estimated cut scores were those with very high standards. At the same time, some states with low standards saw their cut score estimates increase. Though many factors could explain these changes, it is possible that these states are reacting to the 100 percent proficiency requirement of the No Child Left Behind Act. States with very high standards may be moderating them, while states with very low standards may be raising them.

5) Now, math tests seem to be more difficult than reading- is that because of the quality of math instruction or the fact that math is more objective, while reading is open to subjective interpretation?

What we found is that many states are internally inconsistent, with more demanding expectations in math than in reading. In seven states (Colorado, Idaho, Delaware, Washington, New Mexico, Montana, and Massachusetts), the difference between the eighth-grade reading and mathematics cut scores was greater than 10 percentile points. So, for example, a state might set the proficiency passing score at the 30^th percentile for reading, but the 40^th percentile for math. This means Americans may wrongly think their children are doing better in reading than in math—when in fact less is expected in the former subject.

6) The report also indicates that 8th grade tests are more difficult to pass than those in earlier grades. In my mind, 8th grade is somewhat of a cut off point for those about to enter high school. Should there not be some type of gate keeping mechanism or at least evaluative time?

All students should be held to high expectations, regardless of grade. What many states are doing, however, is holding students to a lower standard in lower grades, giving parents and students a false impression of student performance. If Susie scores at "proficient: in reading and math in third grade, her parents should be able to safely presume she will continue to score proficient on subsequent tests with regular progress. But if the state in which Susie lives holds elementary students to lower standard than middle schoolers, Susie could be setup for failure in later grades.

7) How consistent are the various states in terms of their expectations for reading and math. Is Montana different than Rhode Island, for example?

States vary greatly in terms of expectations and cut scores, ranging from the 6th to 77th percentile. Rhode Island typically scored in the top 10 (out of 26 states studied) in terms of difficulty in both reading and math. Montana had similar rankings in math, but uses a below-average cut score in reading, ranking mostly in the third quartile.

8) NWEA (Northwest Evaluation Association) is often mentioned. What role do they play in this evaluation?

NWEA played a critical role. Its researchers authored the report, and all of the study's data come from NWEA assessments.

9) There are two sides to every coin. It seems that improvements in test scores can be easily explained by the fact that the difficulty of the test has decreased. True or False?

True, a decrease in difficulty on exams may explain some improvements in test scores. However, this is just one potential influence. It's also possible, for example, that schools are drilling students in how to take the state test, and thus improvements on the state test aren't showing up on other exams like the NWEA's because students are learning test-taking skills, not reading and math.

10) Why in your mind, are math tests more difficult to "pass" than reading tests?

Generally, states appear to set cut scores for math tests higher than for reading tests. For example, in Idaho, "proficient" eighth graders must score in the 47th percentile in math but in just the 36th percentile in reading. It's nothing inherent in the subject that makes it harder; states are just making a different decision in terms of where to set the bar.

11) How closely are states proficiency scores or standards similar across states?

States vary greatly in terms of expectations and cut scores, ranging from the 6th to 77th percentile. Colorado and Wisconsin were toward the bottom, with cut scores ranging from the 6th to 29th percentile. South Carolina and California were at the top, with cut scores ranging from the 43rd to 75th percentile.

12) In this study, an equiperecntile equating procedure was used. It has been years since I studied statistics with Dr. Plake. Can you explain this procedure?

To estimate proficiency-level cut scores, the researchers found the proportion of students within the sample who achieved at the proficient level or better on the state assessment. Following the equipercentile method, they then found the score on the NWEA scale that would produce an equivalent proportion of students. For example, if 75 percent of the students in the sample achieved proficient performance on their state assessment, then the score of the 25th percentile student in the sample (100 percent of the group minus the 75 percent of the group who achieved proficiency) would represent the minimum score on MAP associated with proficiency on the state test.

13) We have the year 2014 on the horizon. What do you see happening as we approach the year that mandates "proficiency"?

Sadly, the 2014 provision in NCLB seems to be scaring some states into lowering standards as they scramble to have all students "proficient" in time. If, instead, the provision were scrapped, states with lower expectations (like Wisconsin and Colorado) might have stronger incentives to raise the bar.

14) One other perhaps non related question is this--are teachers really adequately prepared to help all students across America reach proficiency? Are they adequately trained to work with the heterogeneity that they find in the classroom?

We didn't address this question in the study. But I doubt anyone would tell you that all teachers are adequately prepared, or that classroom heterogeneity isn't a huge challenge!

Published december 31, 2007