Tackling Educational Research in Schools

The following article comes from Joe Nutt, an international educational consultant and author.

Whenever I speak at a conference I make every effort to allow ample time for questions. In such a fraught arena as education it seems not just polite, but necessary. But I was unprepared for a reaction when I recently spoke about difficulties with educational research design, at researchED’s national conference. The very first thing a young member of the audience said was, “I wish I’d heard you seven years ago. I wouldn’t have wasted the last seven years.”

However gratifying it was at the time, on reflection it was actually quite depressing.

One of the most significant shifts in education in recent years has been the rise of educational research and the idea that schools and classroom teachers should be using it to inform what they do. This apparently simple and sensible principle has been a key element of education policy, certainly in the UK. After reflecting on that seven years comment I decided I really didn’t like the idea that there might be thousands of other teachers like my audience member out there wasting valuable time, so what follows is a highly compressed version of my presentation.

Most of what is labelled “research” in education is really marketing or lobbying and media organisations that publicise it are entirely undiscerning in how they evaluate it. No media outlet, even one’s dedicated to education, are free from this weakness. The BBC is perfectly capable of publicising research that claims, “Comics are a better educational resource than traditional textbooks” even though the researcher described himself as a “comic fanatic” and his research covered only 90 adults. Education is a playground for all kinds of interested parties, from individual philanthropists to global technology businesses masquerading as philanthropists. All kinds of other businesses: NGOs, charities, government agencies and think tanks, publish marketing collateral, under the umbrella term “research.” It really isn’t and shouldn’t be regarded by busy, professional teachers who have a demanding day job to do, as useful.

Teachers don’t have time to do what I have to do when clients employ me to read research and ask me to distil value from it. It’s impossible to read any single paper without also reading many others it cites. This is not only hugely time consuming but demands a lot of determination and self-discipline. There is no way I would waste the kind of time necessary unless someone was paying me to do it.

There is a simple and practical solution to this problem for teachers and I never read anything until I’ve done this first. Research the organisation publishing the material, before you even consider glancing at the research itself. Nine times out of ten you will not need to read the research because the organisation’s agenda will become crystal clear. They are rarely coy about it.

Across the whole spectrum of research, from solo academics to the expensively commissioned research from organisations like the OECD and the Nuffield Trust you will find the same weaknesses of design. Researchers frequently work from nothing but data and even when they do interact with children, teachers and schools, they often fail to align their design with the real, messy world of schools and classrooms. It’s easy to forget how the entire fascination with data found its way into schools. Selling data as truth was the shrewd sales pitch that fuelled the entire tech revolution and education is not exceptional in buying into this delusion. But data is never unadulterated truth, it is always generated, edited, packaged and delivered by employees and as such is subject to very human flaws.

In my presentation I provided a number of high profile examples, research that received a lot of publicity but when you really study it, was seriously flawed. The CPD programme that took no account of what teachers who were trained actually did when they went back to their schools, but judged the programme’s effectiveness solely on the basis of maths and English GCSE results in those schools. The lengthy, hugely expensive report by a leading organisation that concluded state schools make better use of human resource investments than private schools, but which relied largely on data from a 110 page workplace questionnaire designed for businesses which contained pages of questions utterly irrelevant in schools. Or the OECD graph which was selected out of a presentation that was 71 slides long and widely shared online, but which made no sense whatsoever, to myself or any of the expert professional educational research colleagues I shared it with.

There are of course, researchers and even organisations who do their utmost to avoid these kinds of weaknesses, but they really are needles in a huge haystack. So classroom teachers would be wise to read widely about research in the educational press and even on social media, and use their own professional judgement. Much better that than to try and engage with the original research itself – unless of course they actually like chasing needles.