Steve Raudenbush Investigates Scientifically-based evidence
In February 2002, Assistant Secretary Neuman invited her UM SOE colleague, Steve
Raudenbush to present at a seminar in Washington, DC, which focused on the
topic of scientifically-based evidence or research. Professor Raudenbush, known
internationally as one of the pioneers of hierarchical linear modeling, a statistical
method for analyzing complex data sets, explained to the audience that it was
indeed possible, and sometimes quite useful, to conduct ethical, randomized
experiments in education. Raudenbush argued that “our best ideas about
how to improve teaching ought to be tested scientifically.”
Once Raudenbush began studying the evaluative aspects of NCLB, however, he found that the measures currently being used for high stakes decision-making were “scientifically indefensible.” His study will soon be published in a monograph, “Schooling, Statistics, and Poverty: Can We Measure School Improvement?” which will be distributed to 10,000 educators by ETS, a non-governmental organization that constructs some of the standardized tests used by NCLB to measure school progress.
Based on an empirical analysis of four data sets spanning elementary and high school years, he concludes that “A reassessment of approaches to accountability appears essential….Holding educators accountable for their contributions to student learning is a laudable goal and one potentially powerful lever for school improvement. But the amount and quality of data must be reasonably aligned with the uses of data.” He calls for combining “information on student learning with information on school organizational and instructional practice, allowing a triangulation of evidence that would supply greater confidence in inferences about the functioning of particular schools."
This article appears in the Fall 2004 edition of Innovator.
