By Dr. Andy Haig
July 22, 2018
Whether it’s the TV show NCIS or CSI: Miami, it’s always about the evidence. Same thing in healthcare: Good care is driven by the evidence. But what evidence? And how much evidence is enough to act on? And who says so?
Cochrane is an international organization that creates standards for evidence-based medicine throughout the world and works to disseminate these good practices. Cochrane has commissioned a group of international leaders to form Cochrane Rehabilitation. As a member, I am excited about our mission to explore these issues in rehabilitation, an especially challenging area. In July 2018, the group met for an intensive two days in Paris, where we began to map out the challenges and solutions.
The biggest challenge: Those twentieth century standards for evidence – the randomized double-blinded, controlled research trials – that brought us to where we are now. They provide compelling evidence that simple interventions like drugs or surgery work or do not work. But the dirty secret is that the standard is both inaccurate and, in many areas like rehabilitation, quite inappropriate.
Average Doesn’t Fit
Inaccurate? To be precise, research studies have strict inclusion and exclusion criteria, and they typically exclude anyone with problems other than exactly and only the one problem being studied. Most real people have many problems, and many people don’t fit the standards but still may have the problem.
Results are typically described in averages for the group, not the individual. I’ve never met the “average” patient. Each person is different, so an 80 percent response rate sounds great, unless you’re one of the 20 percent.
Most studies look at a single intervention at a single point in time, yet the choice of timing for treatment may be critical. Think of a research trial involving surgery vs. no surgery for spinal stenosis in older people. Those who qualify are randomized. But it could be that some patients are better served by qualifying and randomization later during the course of their disease.
Inappropriate? It’s easy to give people a blue vs. a red pill and assume they can’t tell the difference. But a therapist named Carlos and another named Jane and a third named Shinichi can’t be interchanged as easily. Rehabilitation interventions ranging from exercise to cognitive training require many talents:
· Technical skill on the part of the clinician (Maybe Carlos is better at mobilizing the sacroiliac joint).
· Motivational skills on the part of the clinician (Maybe Jane just has a way with old people).
· Professional biases (Perhaps after years of Mackenzie training Shinichi doesn’t much believe in mobilizing the sacroiliac joint, so he’s not a very good placebo).
It’s also impractical and often unethical to withhold treatments that are unproven but well-accepted. Would we really want to withhold range of motion exercises on one person with spasticity just to see if they worked in another?
Rehabilitation cases are often unusual diseases or disability. A drug trial for brain injury may show impact in week 3 for 23-year old Rancho, a level 6 male with diffuse axonal damage, but not for 45-year old Delores, a female in a vegetative state in Week One.
Considering Options
Cochrane Rehabilitation is looking at solutions:
· Single-case design studies with on-off-on treatments can provide evidence.
· Prospective trials without control groups can augment basic science and other evidence of effectiveness of an intervention.
· Qualitative methodologies can support complex interventions.
I’m on the committee in part because of the work of Haig Consulting. We’re widely recognized for our ability to create reproducible team processes, so that an intelligent multidisciplinary team can randomize treatments to receiving or not receiving pain the team assessment.
Funded by the Robert Wood Johnson Foundation, the National Institutes for Disability and Rehabilitation Research, and most recently by Blue Cross Blue Shield of Vermont, we’re constantly refining the protocols and the science so that health systems can reproduce good interventions without discarding the creative brilliance of their clinicians.
The work of Cochrane Rehabilitation is critical to rehabilitation and indeed to health systems overall. Until scientists in the field (who may have an easier time with red and blue pills) recognize and respect the validity of rehabilitation methodologies – as well as the frailties and wrong assumptions of 20th century clinical research – rehabilitation scientists will have less chance to obtain funding, less respect for their work, and thus less influence to create an optimal health system where function and quality of life are optimized.
It’s time to turn that around.