- The story of Clever Hans
- The real meaning of “anecdote” and “placebo”
- Things that you’ve probably tried but don’t work (and why)
(This post is extracted from a book that isn’t available yet, but I wanted to share with you because people need this now, not in 12 months’ time. If you want to actually get everything right now, free, the only way is to join us at https://psychologybestpractice.substack.com where I’m bit by bit putting up all the tutorials. Please do that if you haven’t already.
Alrighty, let’s get started. And let’s deal with what might be the two biggest obstacles to understanding what works and what doesn’t: the meaning of two troublesome words, “anecdote” and “placebo”. These two closely-related words keep me awake at night far too often, worrying that people misunderstand them and continue to get taken in by them.
An anecdote is basically a story, so when someone tells you that they took a certain supplement or medication, or they did a certain thing, used a certain intervention, or changed their diet, that their pain improved, that’s an anecdote. This word also applies to “stories” we tell ourselves about what works and what doesn’t. And the problem with anecdotes is that they’re notoriously unreliable. Probably more than 99.9 times out of 100 they don’t reveal the truth and are simply a misunderstanding. And yet on platforms like Facebook people share these kinds of stories in their masses, and whole industries are based on nothing but these false stories.
How do we know the stories are false? Because when we take away the misunderstanding and errors of judgement by actually testing these things in proper clinical trials, and honestly analyse the statistical data (instead of fraudulently manipulating date, or trying to assert that insignificant statistical noise is evidence), they fail miserably.
The story of Clever Hans
This is a true story, and Clever Hans is a real name. Clever Hans was a horse who lived in Germany in the late 19th and early 20th century. The story of Clever Hans is just one of many stories that show us that no matter how certain we are of the evidence of our own eyes or our own experience, what we think is happening, and even what we absolutely know is happening, is not necessarily so.
Clever Hans was paraded all over the country, billed as a horse who was skilled at mathematics and spelling amongst other things. He used hoof tapping to provide answers, or would select out letters to spell words.
People were amazed at the intelligence of this beautiful horse! You can read the full story here if you want to check it out: https://en.wikipedia.org/wiki/Clever_Hans.
Anyway the German board of education created a commission to investigate this seemingly amazing horse. That commission, a group of 13 people including a veterinarian and other professional people, analysed Clever Hans’ performance and concluded that “no tricks were involved”. So you could say that even the experts agreed that Hans could do exactly what was claimed.
The task of investigating Clever Hans then went to comparative biologist and psychologist Oskar Pfungst, who applied a scientific and methodical examination. First he tried testing Hans without an audience who might be giving him cues, and also when someone other than his trainer was asking Hans to solve maths problems. Hans was perfect! What a horse!
Then he tried using blinders to test whether there was any difference in Clever Hans’ accuracy dependent on whether or not he could see the speaker. Unsurprisingly, Hans was suddenly not at all hot with his answers!
Finally Pfungst tested Clever Hans’ accuracy when the speaker himself didn’t know the answer to the maths problem. Clever Hans was not at all clever with those questions either.
For Clever Hans to provide accurate answers, the speaker had to know the answer, and also Clever Hans had to be able to see the speaker clearly. Can you guess what was happening? It turned out that the speaker had been unconsciously giving Clever Hans cues to start and stop tapping with his hoof. Even the testers had been unknowingly making little movements with their heads that Clever Hans had learned meant to start or stop tapping with his hoof.
Even though the myth of Clever Hans had been well and truly debunked, the act continued to travel widely and still drew large, amazed crowds.
You might be asking yourself at this stage, “But Christine, what’s the harm in that?” and in a way I agree because on the surface it looks fun and entertaining, and from a science perspective it’s actually led to some very important research on human impact on experiments in animal behaviour and has been very useful to improve animal training strategies. But here’s the problem with accepting stuff on face value, or just trusting that what we think is happening is true, and it’s a pretty tragic and often even fatal problem.
We humans are very easy to fool, and we even frequently fool ourselves, because we judge things by comparing what we think is a “before” and an “after”. We believe that if two things happen together in a close sequence of time, that one of those things actually caused the other. We are also massively prone to think that if we do “something” that “something” will produce a real result. And we see patterns where there are no actual patterns, remaining stubbornly blind to information which doesn’t fit the pattern that we want to see. And then we are at risk of wasting time, energy, money, hope, and even life, not only for ourselves but for others as well. Our assumptions and beliefs can keep us stuck, can keep us from trusting real experts, and can even be fatal.
This weakness in our human logic is something that scientists are very interested in. A well-known study investigating this very human behaviour tested people with asthma, giving them either no treatment, a fake treatment, or a drug (albuterol) known to improve airway function.
(See: Wechsler ME, Kelley JM, Boyd IO, Dutile S, Marigowda G, Girsch I, Israel E, Captchuk TJ: Active albuterol or placebo, sham acupuncture, or no intervention in asthma. New England Journal of Medicine.2011 Jul 14;365(2):119-26.)
Participants reported that they could breathe more easily regardless of the treatment given, however those given fake treatment had no improvement whatsoever in their actual lung function. They remained compromised and at risk of dying even though they believed they were better, because they were unmedicated but had fooled themselves.
Let me repeat that. These people honestly believed their lung function was better, but it was not, and without medication they remained at risk of an asthma attack serious enough to kill them.
“Belief” has no place in health care of any kind, and that is why I dedicate so much of my time to debunking false treatments that are so rife in the world of chronic pain (and in the world of health in general). Sometimes they’re even rife amongst health professionals, especially psychologists, as you’ll see shortly.
Sorry psychologists, I honestly love you but someone has to say it. Don’t hate me.
More examples – homeopathy, naturopathy, CBT, mindfulness, “medical” cannabis
Let’s look at homeopathy as another example of how easily we humans can fool ourselves. A lot of people swear by homeopathy, especially homeopaths (no surprise there). But whenever you put homeopathy to the test with a group of people who don’t know whether they’re taking a homeopathic preparation or just a sugar pill (for example) there is no difference in outcome whatsoever. Homeopathy gets the same result as the placebo (I’ll explain that word shortly), which is precisely nil.
Likewise acupuncture. It doesn’t matter in properly-designed trials whether actual needles are used, or fake needles, or where the needles are placed, or whether non-acupuncturists or qualified acupuncturists are used, the results are the same, zero.
And in case you think I’m just picking on alternative therapies, it’s the same with CBT (cognitive behaviour therapy) frequently used by psychologists and erroneously billed as “gold standard”. CBT is not better than placebo. There may be extremely short term benefits observed in the therapist’s office (naturally – the person has just had a very nice chat with an empathetic therapist) but any benefits disappear basically as soon as the client walks out the door and back to normal life. Despite earnestly seeking, I have not been able to find a single study, let alone a meta-analysis, which shows significant long-term benefit from CBT (or any other psychological strategy) compared to the therapeutic alliance alone, and yet many psychologists refer to it as the “gold standard” of therapy and claim that it has strong evidence of effectiveness for depression and anxiety. I’m sorry, it does not.
And another one, mindfulness. So far most researchers avoid testing mindfulness against more beneficial activities, perhaps because they suspect deep down that mindfulness is basically a con job. You’d be better off listening to music, watching television, socializing with friends, reading a book, or going for a walk. In fact good research shows that playing a very short game of Tetris on your mobile phone is more beneficial than hours of practicing mindfulness. None of this stops the unwarranted marketing hype. And no-one seems to want to talk about the potential dangers of mindfulness, such as increased depression and anxiety that has been indentified amongst school children. And yet it’s still being pushed in schools. It’s also being using in the corporate world to try to help executives cope with environments they shouldn’t have to cope with – a cynical application of a dud intervention.
There are many, many more, but here I’ll finish with one that’s growing in popularity, and that’s medical cannabis in all its forms. There is massive hype around cannabis for chronic pain, even though as I’ve already demonstrated in my comprehensive paper on pain treatment, it’s completely unwarranted. One infamous study on chronic pain reported that some 70% of participants claimed their pain had reduced, and yet the pain scores they gave were the same or actually larger, and they required the same or more pain medication, had no improvement to function, and reported less ability to cope.
So the stories (anecdotes) are most often completely and utterly wrong.
That last example above gives us a nice little segue into the word “placebo”. People talk about “placebo effect” as it if were a real thing, but it’s not at all. A placebo is simply something that is given to a participant in a clinical trial so that they don’t know whether they’re getting the real treatment or nothing.
You see, if a trial of something just compares to a waiting list, it’s pretty much guaranteed to show improvement because just about anything achieves better results than nothing. That’s an effect that’s meaningless because you’d get the same result from anything.
So the placebo is basically just a decoy that is there to avoid the comparison with “nothing” and there is no “effect”. What people call a placebo effect is always regression to the mean (the normal ebb and flow of symptoms that would have occurred with or without the intervention, so it’s just a co-incidence), false attribution (something else was going on that was actually the real cause of the change) or error of judgement, or even plain delusion. This false awareness doesn’t last long, because nothing has actually changed.
Now I’m well aware that there are studies that claim biological effect from placebo. These studies are deeply flawed, manipulated, often “p-hacked” (a dirty statistical trick) which unfortunately hasn’t prevented some of them being published in scientific journals.
Understanding the current state of chronic pain treatment
OK, let’s get into the detail of what’s NOT working, and why, including the absolute proof of uselessness for most people. This bit is maybe a bit dry (sorry) and depressing (sorry again) so feel free to skim if you just don’t have the patience or want to read happier stuff. The main thing is that you’re armed with knowledge about what to avoid, and for more detail oriented or scientifically minded people, or people who have too much time on their hands, there are heaps of citations to enjoy.
Up until a paper I wrote in 2019/2020 there seems to have been no aggregated, comprehensive analysis of the effectiveness of the very wide range of treatments currently utilised for non-malignant chronic pain without sufficient explanatory pathology.
To write up that paper I did a wide review of scientific literature (study papers and analyses in respected journals) because that’s the one and only way to get to the truth. Not blogs, not advertisements by celebrities, not YouTube videos, etc etc etc. Certainly not cesspits of meaningless anecdotes and false information like Facebook.
The purpose of this literature review was to investigate the best possible evidence for and against the widest possible range of pharmacological, physical/mechanical, and psychological treatments currently in use by health professionals, kosher and not so kosher (standard stuff and quack stuff alike) in the treatment of chronic pain.
This meant investigating a whole bunch of medications, supplements, physical therapy and exercise programs, surgery, chiropractic, acupuncture, TENS, psychological therapies such as cognitive behaviour therapy (CBT and its variants), EMDR, mindfulness, and the newer multi-disciplinary/comprehensive approaches with respect to efficiency, efficacy, effect sizes, and cost. Although you may already realise that some of these don’t work and can be regarded as scams, I felt it was important to include them in that review because they continue to be used and promoted by front-line health professionals, and are sometimes even wildly popular because of (useless) anecdotes. Some of them even attract health fund rebates.
I found that without exception, all studies of all treatments in current use demonstrate minimal to no effectiveness in reducing reported pain levels, and demonstrate only marginal effectiveness (and often no effectiveness) in reducing comorbidity like depression or anxiety or sleep issues, or increasing quality of life.
Finally I looked at evidence for a new way to understand non-malignant chronic pain without sufficient explanatory pathology which both explains the failure rate of current treatment, and points clearly to an entirely new direction of therapy. That “new direction” is what you’ll learn in the main text, but not before I share the evidence for it because I don’t want you just taking my word for it. Remain skeptical!
And if you want lots more information and scientific references, continue HERE!