Science education is the angsty teenager of the scientific research field. Assaulted on all sides with strict demands from the “patriarchal” hard sciences, education research holds its ground by echoing cries of “you don’t understand me!” and basing its claims on past literature, much of which was probably subject to the same criticisms that present-day educational research is! Where does the vicious cycle end?
A little background first—as regular readers of my blog may be vaguely aware (assuming these “readers” exist), Science has begun to include science education studies in its pages lately. This is a very good thing. Not only does it put important research in the spotlight, it also attracts science educators to the journal, and an army of science educators who read Science is much better than one that does not! A recent educational research paper by Deslauriers, Schelew, and Wieman published in the pages of Science made some sweeping claims about improved learning in a large physics class thanks to a course intervention based on the idea of “deliberate practice.” Exam scores and attendance were both higher in the section that used deliberate practice; the “old school” section’s scores and attendance were lower. The sections were “matched” using several metrics, including the Brief Electricity and Magnetism Assessment and Colorado Learning Attitudes about Science Survey. Matched sections, differing only in the presence or absence of deliberate practice…everything seems peachy, right?
Not according to Derting et al. and Torgerson, who both wrote letters to Science criticizing the study. The bulk of Torgerson’s argument is that the study is not properly controlled, and does not take into account teacher effects (maybe the control group teacher just sucks in general, in addition to using “bad, old school” methods), selection bias, whether students knew they were being treated differently, etc. Derting et al. echo many of these points. One of their more intriguing ideas in common is that, really, the original study is the equivalent of a “single data point,” or a clinical trial involving a single placebo patient and a single treatment patient. Replication, echo the throngs of hard scientists, is needed.
The original authors responded by supplying evidence that their experimental design was good enough to be generalized. Randomized, hyper-controlled trials are not, they claim, necessary in collegiate science courses. Teacher personalities tend to not affect the amount of learning that occurs in collegiate courses (?!). Finally, they raise the point that replications of their experiment may introduce ethical issues, as investigators should expect to replicate their result, which would involve putting the control group at an intrinsic disadvantage.
Where to fall on this debate? It’s tough for me to decide. Both sides advance good arguments. Theoretical ideas and educational psychology research do support the practices used by the experimental section from the original paper. It would have been very bad if the authors’ results had not supported this existing literature, and what they did was almost certainly good from an educational perspective. However, like a sparrow sitting on a giant’s shoulder, the work does little to advance the field of physics education. There are some very subtle issues at play in the classroom, not all of which can be addressed by sweeping labels like “deliberate practice” and even “active learning.” Practical ideas that real educators can take away are hard to find in the paper, and that lowers its value. It’s a shame, because one can tell that their hearts are in it, but the long-term usefulness of the work just doesn’t stand up to scrutiny! The most valuable literature in education (at least to me) has always been the stuff with the most practical value. This paper will at best fade away and be remembered as little more than a blip on the radar—and at worst have a negative effect on the practicing scientist’s view of education research.