On the stage at TEDGlobal 2011, Ben Goldacre delivered an incredibly fast-paced and informative talk on the subtle ways nutritionists, pharmaceutical companies, and others distort scientific evidence. TED’s Ben Lillie caught up with him to talk about how to read newspapers, the power of the placebo effect, and how people really want to learn if given the chance.
What should someone who just reads the newspaper and wants to learn something about health do?
Don’t start with newspapers! Newspapers have a very unhelpful set of strategies for deciding what to write about: they select stories by what’s eye-catching and melodramatic, but that’s the opposite of what science is about. Science is about carefully documenting and making sure you see all the evidence, rather than just small portions of it. Huge amounts of effort now are about making sure that we do systematic reviews of the evidence that’s available to us, so that we see all of the studies, the positive and the negative ones.
What’s surprising, in some respects, is that this is a fairly recent phenomenon, only about 25 years old. It used to be that if you were writing a review — on treating diabetes, for example — you would stroke your chin and you’d say, “Well, I quite like this paper and I like that paper, and this one’s written by my friend, and this one validates my pre-existing prejudices. So I’ll just put those into a chapter and write about it.” And that was a bizarre state of affairs — we’re so obsessed in science with making sure that our individual experiments are carefully controlled, that we remove any bias, we exclude any extraneous explanations for any phenomena that we observe. We’re incredibly cautious for individual experiments. But then, at the crucial second step — the part of the process where you bring together all the evidence to produce a summary that’s useful in the real world — we suddenly threw all these principles out the window.
So, 25 years ago in science there was a very important change. We started doing systematic reviews rather than allowing ourselves to cherry-pick. And that’s the opposite of what you get in newspapers. What you get there is, “Oh wow, tomatoes cure cancer.”
So, this overwhelming barrage of shrill health stories in the media is a problem. The individual stories are a bit silly, of course, and misleading in their own corner, but they can also have more destructive effects that you might not think of immediately. The World Cancer Research Fund did a fascinating survey of 2,500 people, and half of the respondents agreed with the statement, something like, “Doctors and scientists are constantly changing their minds about what a healthy lifestyle looks like.” Actually doctors and scientists haven’t changed their mind about what a healthy lifestyle looks like for decades now. We know the basics: eat fruit and veg, don’t smoke, don’t drink, get some exercise, and don’t be too fat. The evidence on this hasn’t changed much in two decades. And yet, people are misled by this constant barrage of spurious, overstated, marginal claims into thinking that this information is in a constant state of flux, that a healthy lifestyle is complicated and difficult. This is disempowering.
Do you think this is a solvable problem, or is it fundamental to how newspapers are?
I think newspapers are useful for lots of things — for telling you the results of sports games, for giving you a first draft of recent historical events. But I don’t think they’re very useful for giving you health information.
The sense I get from having done this for about ten years now is that the more abstract a piece of science coverage is, the more likely it is to be accurate. It’s likely to be written by somebody who doesn’t have any huge motivation for distorting it. It’s just going to be written by somebody who’s interested in describing the fact.
So you’re happy with the high-energy particle physics coverage more than the health coverage?
Yeah, there are fewer factors driving a distortion of an article about the space telescope than there are for health risk stories. The closer, tragically, that a story is to an actual decision that you might have to make about your own health, the more likely it is to be foolish —it’s either going to be cherry-picked and selected into an extreme and unrepresentative story, or it’s going to be actively distorted.
But here’s what’s most important: the flaws and holes that you can find in the misrepresentations of scientific evidence from mainstream media, and quacks, are a great way to learn about science, because they’re a very extreme version of the more subtle kinds of flaws in interpretation that we see in proper academic science. So you see people overstating the results of observational findings in newspapers, or quack sales literature, but you also see people doing that — to a less extreme extent — in the proper academic epidemiology literature. That’s why I think the mess people make can be such an interesting teaching tool for explaining the basics of how evidence-based medicine works.
In fact, you know, people worry endlessly about how we can make science more interesting or exciting to the general public, or to children, and I think epidemiology really is the answer. It’s right there in front of you. Half of all the science stories you read in newspapers are about what’s going to kill you or cure you. And that’s evidence-based medicine. That’s epidemiology. I believe that people aren’t just interested in the answers from those stories. I believe many people are also interested in the mechanisms behind that knowledge, in the question “how do we know that something is good for you?”
You talked about a ‘force-field of tediousness’. Is it that we’re just presenting it in a tedious way?
When I talked about the force-field of tediousness, I’m talking about the tricks that the pharmaceutical industry uses to distort evidence, because those are much more complicated and difficult to unpick than the distortions that the quacks and mainstream media use. They’re protected by how technical their tricks are. Mainstream media and quacks use very obvious, very day-glo tricks.
My impression is that, for a lot of people reading, even the simple ones are tedious and uninteresting.
I think they become more interesting when they’re shined up by the fact that you’re, well, you’re taking the piss out of someone, and making them look like an idiot. There’s a certain sadistic pleasure that comes from that, which I can’t entirely force myself to shy away from. There’s a pleasure in the schadenfreude of exposing idiots.
And everybody wants to uncover the secret reality behind the news. That impulse is getting misdirected with the 9/11 truthers, but it’s usefully directed towards trying to unpick the science behind health stories in newspapers.
One thing I would say from my own experience, of working in extremely impoverished parts of London, is that I’ve never met any stupid people. They differ in how motivated they are to understand something, the extent to which they want to get in and understand the nuts and bolts. But I can honestly say it’s vanishingly rare to encounter somebody who doesn’t have the intellectual horsepower to understand the basics of evidence based medicine. People really aren’t stupid. They vary on how much they care, not on how clever they are.
You’ve also had great success with the Infinite Monkey Cage, the comedy show you’re part of. You recently sold out all across the UK.
What can I say? A mathematician, an epidemiologist, and a particle physicist sold out the Manchester Apollo, a rock venue, on a Friday night. We went on and people were really going for it — a drunk, shouty room of 2000 people laughing at statistics jokes. It was a very strange thing, and another illustration that there are many, many more clever people out there than people give credit to.
The other fascinating thing about your talk is the placebo effect. Do we know much about the mechanism of why a shot is better than a pill?
No, and what’s really interesting is, it’s all about belief and expectation. The placebo effect is a cultural phenomenon; It’s about the meaning of treatment.
One very powerful example of this is a four-way study where they gave people either a pain-killer or a dummy sugar pill, and it was packaged either in plain packaging, or in full on brand-name packaging that would have been very familiar to all of the people in the study. As you’d expect, there was a difference between whether you got a pain killer or a dummy pill, but also, people had better pain relief when the pill came in all of the fancy brand-name packaging, than in the brown package. And that really is a cultural phenomenon.
If you went back to the stone age, and you handed some neolithic chap one packet of super glossy aspirin, or whatever, and the same drug in a plain package, then unlike us, I suspect he wouldn’t notice any difference in pain relief between those two options. The effect of the packaging isn’t about what’s written on the box, it triggers your memory of the TV adverts where the woman is sitting there and she has a bright red pulsing thing on the graphic of her brain, and then she eats the pill and the blue arrows go into her blood supply from her stomach and fly around and kill the pain, and in the next shot she’s on roller-skates on the seafront being pulled along by dogs, or something. All of that is part of the placebo effect: the meaning of the treatment and our beliefs and expectations when we take it.
Is there a way to harness this clinically, in an effective, ethical way?
Absolutely. What the placebo effect tells us is that the setting in which treatment is delivered, and the behavior of the doctor, are extremely important, and they have a significant impact on outcomes. So you make sure that your clinic room is a therapeutic environment; you make sure that when you’re seeing patients, whether you’re a nurse or a physiotherapist or a doctor, that you are as reassuring as you can be — while delivering evidence based treatments.
I think the real danger here is to imagine that, because the placebo effect works, then it’s somehow acceptable to lie to patients and give them dummy pills, as quacks do; to mislead them into believing that these are effective treatments. That’s a dangerous path to go down ethically, because doctors shouldn’t lie to patients. I mean, I don’t think I could give a placebo with a straight face, I’m not a very good liar, but even if I was, I don’t think I would be able to hold my head up high as I did it. Even if it improved the outcome for that one patient, the side-effects, culturally, are very serious — ultimately you’re undermining the credibility of doctors around the world. I don’t think that’s melodramatic. The public has a very reasonable expectation that doctors will tell them the truth; that doctors won’t lie to them.
That hasn’t always been the way. Fifty years ago doctors didn’t tell people that they were dying of cancer. They lied to them routinely. But now we’re very clear about the fact that we should be honest with our patients. We want informed consent, and we want to respect our patients’ autonomy and ability to make informed decisions. I don’t think we can lie to patients and still say that we respect their autonomy.
So, the results of all these placebo experiments, we should use them to enhance the benefits of the treatments we know are evidence-based. We can use them to improve the manner, the context in which health care is delivered. What you don’t do is use them as a spurious justification for giving people dummy sugar pills.
Were there one or two things you didn’t have time to get to in the talk, but wished you had?
Oh, my lord…
I said one or two.
Okay, here’s one thing I find interesting, about the criticisms made of big pharma by quacks. You know, whenever you look at the rhetoric of alternative therapists, naturopaths, and anti-vaccination campaigners, you see people talking very colourfully about the problems with big pharma.
Now, a central part of my worldview, as I say in the talk, is that quacks and big pharma use the same tricks to sell pills, but big pharma just use slightly more sophisticated ones. I think big pharma have had a slightly easy ride because the issues are a bit complicated and difficult.
So what we’ve had instead are cheap throw-away conspiracy theory criticisms, often driven by the quacks.The real problems are complicated, and difficult, and technical. They require serious regulatory dimensions.
But what I see with the rhetoric of naturopaths is: “Big pharma is evil, therefore sugar pills cure cancer. Big pharma is evil, therefore vitamin pills are good for you. Big pharma is evil, therefore vaccines give you autism.” Those are logical non-sequiturs, they feel like poetic cultural responses to this big unaddressed problem that we have, but they don’t make sense, and more importantly than anything, they’re not a substantive response to the biggest ethical problem we face in medicine today.
And that is the very simple issue that the pharmaceutical industry is still allowed to run a whole bunch of trials, and when they get a result they don’t like, they can hide those results and not give them to the doctors and the patients who need all of the information to make a meaningful treatment decision.
This pulls the rug out from the whole of evidenced based medicine. I was talking about systematic reviews earlier. If we don’t have access to all of the data, if we only see a part of it, we simply do not know what the true effect is of the treatment’s we’re giving will be.
Now, that’s such a huge problem, that I think it’s a real tragedy if the only mainstream cultural response to that is, “Big pharma is evil, therefore buy my sugar pills.” That sounds to me like a very insubstantial political response to a very serious regulatory failure.
It’s a problem in lots of fields of science: How do you spin a narrative that people can latch onto out of something that’s truly complicated?
Yeah, it is tricky. Some things can’t be explained in two minutes. Some take 10, or 30, or 60, and maybe we can’t get around that.
But one thing that I’ve found interesting about watching how people can get more informed about evidence based medicine over the last ten years — by watching conversations on blog and twitter and so on — is that I don’t think you have to convince everybody of your case straight off with one single document. We have to realize that ideas flow through communities virally, and there are lots and lots of different publics.
One thing you can do, that’s very powerful: you can put the tools for arguing into the hands of the 5% of the population who are very motivated nerds, and they will act as natural advocates in their own community. I guess maybe that’s just a poncey way of saying I get a lot of emails from people who say, “Hey, I won an argument last night at a dinner party using stuff I learned from your book.” I love the idea that I’m destroying friendships around the country.
So, you know, you don’t have to change hearts and minds and public understanding about epidemiology by going directly to everybody and saying, “Hey, all of you, seriously, shut up, switch off what you’re doing and listen to this for half an hour.” You can just rely on the fact that some people will find this really interesting, and they’ll talk about it, where it’s relevant to peoples lives.
One of the really great things about the last five years of watching the explosion of great science writing online has been seeing that there are a huge number of people, nerds, scientists, who wouldn’t self-identify as professional communicators, but who are incredibly good at explaining stuff. There are also a huge number of people who are really interested in and excited by that stuff and who are going out and finding it.
That makes me really optimistic. I don’t mind if newspapers have lots of misleading and dumbed down stuff. I mean, I slightly mind, but I can live with it. As long as there’s something else, as long as there’s something meaty out there, for the people who want it. So I feel weirdly optimistic for the future.
No comments:
Post a Comment