(Mis)Understanding the NAEP Results
3.27.10 – by Chad Aldeman – Stories from the NY Times, Mother Jones, and the Washington Post bemoaned the flat National Assessment of Education Progress (NAEP) reading scores released Wednesday. Jay Matthews called it the epitaph of the No Child Left Behind era. The results aren’t quite so simple.
Stories from the NY Times, Mother Jones, and the Washington Post bemoaned the flat National Assessment of Education Progress (NAEP) reading scores released Wednesday. Jay Matthews called it the epitaph of the No Child Left Behind era. The results aren’t quite so simple.
See, NAEP is different than most standardized tests. It takes a sample of the current population in every state, so this year’s population of kids is compared to the last time the test was administered. There’s an automatic correction for changing demographics, so as America has gotten less white, so has NAEP. In statistical terms this creates something called Simpson’s Paradox, which makes trend lines seem worse than they really are because of a hidden variable, in this case, race (Matthew Yglesias touched on this point yesterday).
To show how this impacts NAEP scores, here are the results of the long-term trend NAEP results for fourth-grade reading from 1975 to 2008 (I’m using the long-term trend version of NAEP, because it’s been largely unchanged since its first administrations in the 1970s. Its had one significant format change, in 2004, when NAEP administered both the new and old formats. Hence the dotted and solid lines in all of the following graphs). As the chart below shows, average fourth-grade reading scores have risen only modestly, from 210 in 1975 to 220 in 2008.
This is basically the same thing that showed up in yesterday’s results. There’s been some small gains over time, but year-to-year progress has been small.
But that’s not the whole story. See, these overall trend lines are a sampling of America. As we’ve become more diverse, NAEP has changed its sampling ratios to reflect our changing society. This chart shows the percentage of students drawn from racial/ ethnic categories over time. In 1975, NAEP test-takers were 80 percent white. By 2008, only 56 percent were. There were three percent more blacks in 2008 than in 1975, and Hispanics had quadrupled from five to 20 percent.
So, because NAEP has gradually included more black and Hispanic students, and black and Hispanic students score lower, on average, than white students, the total score doesn’t reflect the true gains made by each group. The chart below shows scores taken from the same testing years, this time disaggregated by race.
Each group has actually made greater gains over time than the overall total. White students increase 11 points, one more than the national average. Black students scored 23 points higher, and Hispanic students were scoring 24 points higher in 2008 than they were in 1975 despite quadrupling in size. In other words, the white-black and white-Hispanic gaps are closing and every group is scoring higher, but the national score is showing more modest improvements because of demographic changes.
This is an important distinction to make, because it means the test score results are not just a matter of classroom teaching and learning (to be completely clear, I don’t think NAEP results can be easily attributed to national education policies like NCLB either). The students themselves have changed in important ways, and to break even or to make small achievement gains as society becomes more diverse is an accomplishment worth celebrating. At the very least it’s worth understanding.
For more background reading on NAEP, try Education Sector’s NAEP Explainer.
Enter your email to subscribe to daily Education News!
- Education Technology
- Online Education
- California Education
- Charter Schools
- Teachers Unions
- Education Research
- New York Education
- School Choice
- Education Funding
- STEM Education
- Parent Involvement
- UK Education
- Cost of College
- New York City Schools
- Florida Education
- Julia Steiny
- Common Core
- Texas Education
- Math Education
- School Health
- Los Angeles Schools
- Obama Administration
- Education Reform
- Pennsylvania Education
- Louisiana Education
- Chicago Schools
- College Admissions
- Online Courses
- Teacher Training
- New Jersey Education
- C. M. Rubin
- Ohio Education
- Early Childhood Education
- UK Higher Education
- Health Education
- Illinois Education
- Tennessee Education
- 2012 Election
- iPads in the Classroom
- Massachusetts Education
- Teacher Evaluation
Plan your career as an educator using our free online datacase of useful information.
- Select a City Subject
- Business Schools in Belleville
- Business Schools in Bourbonnais
- Business Schools in Carterville
- Business Schools in Chicago Heights
- Business Schools in Cicero
- Business Schools in Collinsville
- Business Schools in Crystal Lake
- Business Schools in Danville
- Business Schools in Glen Ellyn
- Business Schools in Greenville
- Business Schools in Lincoln
- Business Schools in Macomb
- Business Schools in Malta
- Business Schools in Moline
- Business Schools in Mount Carmel
- Business Schools in Mount Prospect
- Business Schools in Naperville
- Business Schools in North Aurora
- Business Schools in Oglesby
- Business Schools in Palatine
- Business Schools in Rock Island
- Business Schools in Springfield
- Business Schools in Ullin