New Study Raises Important Point – How Helpful Is Ed Research?

Clker-Free-Vector-Images / Pixabay

CAUSE, EFFECT, AND THE STRUCTURE OF THE SOCIAL WORLD, a new paper/chapter by MEGAN T. STEVENSON seems to be making a minor stir in research circles this week.

I first heard about via a Kevin Drum post, Nothing works. Now with proof!

It’s a very long study – 47 pages of narrative. It’s also not easy for a layperson – at least, this layperson – to understand. I even tried – for the first time – to use an AI tool to help me understand a piece of research.

Basically, it seems she says that most studies in the area of criminal justice don’t prove anything and that those that claim to can’t be replicated. And that this conclusion can be applied to most other fields that primarily relate to behavioral science, including education, though they can’t be applied to medical and other more hard science topics.

The researcher exempts more direct interventions (if someone is hungry, they will become less hungry if you give them food) but, apart from those, pretty much dismisses any more nuanced efforts. Without providing specific examples she, instead, suggests that only more systemic changes can really result in substantial impacts.

In the field of education, some of her points have been echoed by Dylan Wiliam, the respected education researcher:

The importance of context by @dylanwiliam via @Andy__Buck #SLTchat #Honk #ukedchat pic.twitter.com/YI4mkgiTEc

— Stephen Logan (@Stephen_Logan) October 9, 2016

He elaborates on this point at Getting educational research right.

And her points seem to relate to the common discussion about if teaching is an art or a science.

The replication issue is clearly one in education research (see THIS IS INTERESTING & DEPRESSING: ONLY .13% OF EDUCATION RESEARCH EXPERIMENTS ARE REPLICATED).

This new study seems to suggest that we teachers should be especially cautious about research that is more correlational than directly causal, and it seems to me that most research falls into that correlational category (see The Best Online Resources For Teaching The Difference Between Correlation & Causation).

Please let me know if I’m off the mark on my take from this new study.

My bottom line perspective on most education research comes from veteran educator Joanne Yatvin’s discussion of the Hawthorne Effect.  That experiment found that it didn’t matter what changes the experimenters made at a factory – every change resulted in increased productivity.

Here’s what Joanne wrote about it:

What it really shows is that, when people believe they are important in a project, anything works, and, conversely, when they don’t believe they are important, nothing works.

I’m adding this post to The Best Resources For Understanding How To Interpret Education Research.