Five "Honorees" of Bunkum Awards Announced for their Contributions to Sub-Par Education Research
BOULDER, Colo. and TEMPE, Ariz. (February 15, 2010) — State education agencies and local school districts are increasingly asked to make evidence-based decisions about school reform initiatives, often assuming that all evidentiary claims are the result of high-quality research.
Five “Honorees” of Bunkum Awards Announced for their Contributions to Sub-Par Education Research
High-Production Values and Eye-Catching Charts and Graphs Can Never Replace Strong Methodology and Sound Research Practices
BOULDER, Colo. and TEMPE, Ariz. (February 15, 2010) — State education agencies and local school districts are increasingly asked to make evidence-based decisions about school reform initiatives, often assuming that all evidentiary claims are the result of high-quality research. Unfortunately, much of the evidence offered in policy debates is based on research reports that have bypassed the quality control mechanisms of academic research.
In an effort to help education policy makers separate the wheat from the chaff, expert third party reviews are provided by the Think Tank Review Project, a collaboration of the Education and Public Interest Center (EPIC) at the University of Colorado at Boulder and the Education Policy Research Unit (EPRU) at Arizona State University. Each year the reports identified by experts as the worst of the worst are awarded a “Bunkum.” The Think Tank Review Project today announced five “honorees” for 2009.
While the social science of the winning reports was sub-par, they typically had very high production values, glossy paper, multi-color printing, and artful layouts. “Given the bibliographies, footnotes, charts and tables, policymakers or laypeople may be forgiven for thinking that these honoree reports are based on the highest quality research. We hope that our expert reviews have helped to correct that impression,” said EPIC director Kevin Welner.
The 2009 Bunkum Award honorees:
The Time Machine Award
Weighted Student Formula Yearbook 2009
Authored by Lisa Snell, and published by the Reason Foundation.
In a truly breathtaking innovation, the report enters its time machine and attributes positive reform outcomes to policy changes that had not yet been implemented.
The Data Dodger Award
How New York City’s Charter Schools Affect Achievement
Authored by Caroline M. Hoxby, Sonali Murarka & Jenny Kang, and published by the National Bureau of Economic Research.
New York City’s charter schools might genuinely be improving student outcomes; however, this study — because of the information it withheld and its methodological shortcomings — does not and cannot resolve the issue.
The Misdirection Award: Keep Our Eyes Off What Works
Reroute the Preschool Juggernaut
Authored by Checker Finn and published by the Hoover Institution.
The report misdirects readers from a mountain of empirical, peer-reviewed and widely accepted evidence, and instead cherry-picks a few weak studies to critique proposals for universal preschool.
The Innovations in Promoting Alternative Certification Award
An Evaluation of Teachers Trained Through Different Routes to Certification: Final Report
Authored by Jill Constantine, Daniel Player, Tim Silva, Kristin Hallgren, Mary Grider & John Deke, and published by the U.S. Department of Education, Institute of Education Sciences.
The authors report ‘no evidence’ that traditionally trained teachers provided better student scores than alternatively trained teachers. The report does not bother to set forth caveats to the ‘no evidence’ conclusion, but there should, in fact, have been many, many caveats — including small sample size, sampling methods, and a failure to distinguish the treatments. Also interesting: the study actually included many analyses that found traditionally trained teachers outperformed their alternative route counterparts. It’s just that the authors chose not to fully report and acknowledge these findings in the report’s conclusions.
The Annual Friedman Foundation Johnny One-Note Award
Multiple Reports by the Friedman Foundation
Multiple authors, all published by the Friedman Foundation.
The Friedman Foundation has, over the past three years, cloned the same study on the cost of drop-outs in at least seven states, a tax credit voucher report in at least six states, and opinion polls on school choice in 15 states. Amazingly, all these reports lead to the same conclusion: vouchers and other forms of school choice will save money and improve student outcomes. The basic technique used by Friedman researchers is to take the same report, change the name of the state, plug in some state-specific data, vary the title a bit, and come up with the predetermined conclusion.
This year’s honorees were selected following expert third-party reviews of research reports published by think tanks and other research organizations. Reports reviewed by the Think Tank Review Project are carefully selected. Every day the web sites of prominent think tanks are visited to identify new research publications for possible review. If a report is deemed of sufficient importance, it is then assigned for review to an independent scholar with expertise in the area of inquiry.
A complete analysis of this year’s Bunkum Award winners can be found at: http://epicpolicy.org/think-tank/bunkum-awards.
About the Bunkum Awards
The term ‘bunkum,’ meaning essentially ‘nonsense,’ came about because of a long-winded and pointless speech given in 1820 on the House floor by Congressman Felix Walker of Buncombe County, North Carolina. The Bunkum Awards help to highlight nonsensical, confusing, and disingenuous reports produced by education think tanks.
Enter your email to subscribe to daily Education News!
- Education Technology
- Teachers Unions
- Charter Schools
- California Education
- Education Research
- New York Education
- Online Education
- UK Education
- STEM Education
- School Choice
- Cost of College
- Education Funding
- New York City Schools
- Julia Steiny
- Florida Education
- Education Reform
- Parent Involvement
- Texas Education
- Math Education
- C. M. Rubin
- Los Angeles Schools
- Obama Administration
- Chicago Schools
- 2012 Election
- New Jersey Education
- Pennsylvania Education
- Tennessee Education
- Teacher Training
- UK Higher Education
- Early Childhood Education
- Louisiana Education
- College Admissions
- Ohio Education
- Teacher Evaluations
- School Health
- Illinois Education
- Arne Duncan
- UK Politics
- Michigan Education
Plan your career as an educator using our free online datacase of useful information.
- Select a City Subject
- Criminal Justice Schools in Aurora
- Criminal Justice Schools in Belleville
- Criminal Justice Schools in Bridgeview
- Criminal Justice Schools in Calumet City
- Criminal Justice Schools in Carbondale
- Criminal Justice Schools in Carlinville
- Criminal Justice Schools in Cicero
- Criminal Justice Schools in Danville
- Criminal Justice Schools in Des Plaines
- Criminal Justice Schools in Dixon
- Criminal Justice Schools in Elmhurst
- Criminal Justice Schools in Glen Ellyn
- Criminal Justice Schools in Grayslake
- Criminal Justice Schools in Harrisburg
- Criminal Justice Schools in Ina
- Criminal Justice Schools in Jacksonville
- Criminal Justice Schools in Kankakee
- Criminal Justice Schools in Lincoln
- Criminal Justice Schools in Lisle
- Criminal Justice Schools in Macomb
- Criminal Justice Schools in Malta
- Criminal Justice Schools in Moline
- Criminal Justice Schools in Mount Prospect
- Criminal Justice Schools in Normal
- Criminal Justice Schools in Oak Brook
- Criminal Justice Schools in Olney
- Criminal Justice Schools in Orland Park
- Criminal Justice Schools in Palos Hills
- Criminal Justice Schools in Peoria
- Criminal Justice Schools in River Grove
- Criminal Justice Schools in Robinson
- Criminal Justice Schools in Romeoville
- Criminal Justice Schools in Sugar Grove
- Criminal Justice Schools in Ullin