Outside the Box: Why are Cochrane reviews so boring?

Be honest. Have you ever read one? Have you ever met anyone who has ever read one? Have you ever got farther than halfway down the abstract? There's an elephant in the room here, and it's time to talk about it. Archie Cochrane is one of my all-time heroes. Born in 1909, he was passionately committed to social justice and what we would now recognise as evidence-based medicine. He trained in the Welsh Valleys at a time when unmet need was high and effective treatments few. He once marched on Whitehall carrying a placard that said, ‘All Effective Treatments Must Be Free’.

Cochrane famously wrote in 1979:

‘It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials.’1

I have seen the Cochrane Collaboration grow from a small group of bohemian academics gathered around a kitchen table in north Oxford to a powerful international network overseeing six electronic databases which together hold over a million studies, a 300-page methodological handbook, and over 50 subject-specific Review Groups whose written approval of a detailed protocol is required before a systematic review may begin.

All of this I applaud, up to a point. It works well for the simple question Archie had in mind: ‘What is the efficacy of drug X for indication Y?’. The problem is that the Cochrane machinery is built on the assumption that by summarising the findings around tightly focused questions we will build a meaningful knowledge base.

For today's complex and multifaceted health challenges, this may not be the case. Take your pick: dementia, arthritis, depression/unhappiness, obesity, heart failure, or anything smoking-related. The low-hanging fruit of single-drug therapies has largely been picked. Designer drugs (‘biologics’) wait in the wings but threaten to bankrupt the health service unless restricted to second-line use; cost-effectiveness studies are typically contested or absent. A growing proportion of questions concern not drug therapies but lifestyle and/or educational interventions for chronic disease or risk factors. And what is a risk factor anyway?

Five thousand (mostly) high-quality Cochrane reviews notwithstanding, the troubling aspect of this enterprise is not the few narrow questions that the reviews answer but the many broad ones they leave unanswered. Lifestyle diseases require lifestyle interventions, and these require attention to people's identities, values, families, and communities. Every ‘complex intervention’ for obesity raises questions about what it means to be obese, the upstream ‘obesogenic environment’, and the myriad combinations of interventions which, if time and resources were infinite, may be tested.

The reason why Cochrane reviews are boring — and sometimes unimplementable in practice — is that the technical process of stripping away all but the bare bones of a focused experimental question removes what practitioners and policymakers most need to engage with: the messy context in which people get ill, seek health care (or not), receive and take treatment (or not), and change their behaviour (or not).

Which gives me an idea for another database: the Cochrane Database of Editorials, Ideas, and Opinion Pieces. It is surely as outrageous in this rationalist age as Cochrane's placard was in 1938.

• © British Journal of General Practice 2012