What doctors don't know about the drugs they prescribe
-
0:01 - 0:03Hi. So, this chap here,
-
0:03 - 0:06he thinks he can tell you the future.
-
0:06 - 0:08His name is Nostradamus, although here the Sun have
-
0:08 - 0:11made him look a little bit like Sean Connery. (Laughter)
-
0:11 - 0:14And like most of you, I suspect, I don't really believe
-
0:14 - 0:15that people can see into the future.
-
0:15 - 0:18I don't believe in precognition, and every now and then,
-
0:18 - 0:21you hear that somebody has been able to predict something that happened in the future,
-
0:21 - 0:24and that's probably because it was a fluke, and we only
-
0:24 - 0:27hear about the flukes and about the freaks.
-
0:27 - 0:31We don't hear about all the times that people got stuff wrong.
-
0:31 - 0:33Now we expect that to happen with silly stories
-
0:33 - 0:36about precognition, but the problem is,
-
0:36 - 0:40we have exactly the same problem in academia
-
0:40 - 0:44and in medicine, and in this environment, it costs lives.
-
0:44 - 0:48So firstly, thinking just about precognition, as it turns out,
-
0:48 - 0:50just last year a researcher called Daryl Bem conducted
-
0:50 - 0:52a piece of research where he found evidence
-
0:52 - 0:56of precognitive powers in undergraduate students,
-
0:56 - 0:58and this was published in a peer-reviewed academic journal
-
0:58 - 1:00and most of the people who read this just said, "Okay, well,
-
1:00 - 1:03fair enough, but I think that's a fluke, that's a freak, because I know
-
1:03 - 1:05that if I did a study where I found no evidence
-
1:05 - 1:08that undergraduate students had precognitive powers,
-
1:08 - 1:11it probably wouldn't get published in a journal.
-
1:11 - 1:14And in fact, we know that that's true, because
-
1:14 - 1:17several different groups of research scientists tried
-
1:17 - 1:20to replicate the findings of this precognition study,
-
1:20 - 1:23and when they submitted it to the exact same journal,
-
1:23 - 1:26the journal said, "No, we're not interested in publishing
-
1:26 - 1:31replication. We're not interested in your negative data."
-
1:31 - 1:33So this is already evidence of how, in the academic
-
1:33 - 1:38literature, we will see a biased sample of the true picture
-
1:38 - 1:42of all of the scientific studies that have been conducted.
-
1:42 - 1:46But it doesn't just happen in the dry academic field of psychology.
-
1:46 - 1:51It also happens in, for example, cancer research.
-
1:51 - 1:55So in March, 2012, just one month ago, some researchers
-
1:55 - 1:58reported in the journal Nature how they had tried
-
1:58 - 2:01to replicate 53 different basic science studies looking at
-
2:01 - 2:05potential treatment targets in cancer,
-
2:05 - 2:08and out of those 53 studies, they were only able
-
2:08 - 2:11to successfully replicate six.
-
2:11 - 2:15Forty-seven out of those 53 were unreplicable.
-
2:15 - 2:19And they say in their discussion that this is very likely
-
2:19 - 2:22because freaks get published.
-
2:22 - 2:24People will do lots and lots and lots of different studies,
-
2:24 - 2:26and the occasions when it works they will publish,
-
2:26 - 2:27and the ones where it doesn't work they won't.
-
2:27 - 2:31And their first recommendation of how to fix this problem,
-
2:31 - 2:35because it is a problem, because it sends us all down blind alleys,
-
2:35 - 2:36their first recommendation of how to fix this problem
-
2:36 - 2:40is to make it easier to publish negative results in science,
-
2:40 - 2:43and to change the incentives so that scientists are
-
2:43 - 2:47encouraged to post more of their negative results in public.
-
2:47 - 2:51But it doesn't just happen in the very dry world
-
2:51 - 2:55of preclinical basic science cancer research.
-
2:55 - 2:58It also happens in the very real, flesh and blood
-
2:58 - 3:02of academic medicine. So in 1980,
-
3:02 - 3:05some researchers did a study on a drug called lorcainide,
-
3:05 - 3:07and this was an anti-arrhythmic drug,
-
3:07 - 3:10a drug that suppresses abnormal heart rhythms,
-
3:10 - 3:12and the idea was, after people have had a heart attack,
-
3:12 - 3:13they're quite likely to have abnormal heart rhythms,
-
3:13 - 3:16so if we give them a drug that suppresses abnormal heart
-
3:16 - 3:19rhythms, this will increase the chances of them surviving.
-
3:19 - 3:22Early on its development, they did a very small trial,
-
3:22 - 3:24just under a hundred patients.
-
3:24 - 3:28Fifty patients got lorcainide, and of those patients, 10 died.
-
3:28 - 3:31Another 50 patients got a dummy placebo sugar pill
-
3:31 - 3:34with no active ingredient, and only one of them died.
-
3:34 - 3:36So they rightly regarded this drug as a failure,
-
3:36 - 3:39and its commercial development was stopped, and because
-
3:39 - 3:44its commercial development was stopped, this trial was never published.
-
3:44 - 3:49Unfortunately, over the course of the next five, 10 years,
-
3:49 - 3:53other companies had the same idea about drugs that would
-
3:53 - 3:55prevent arrhythmias in people who have had heart attacks.
-
3:55 - 3:57These drugs were brought to market. They were prescribed
-
3:57 - 4:01very widely because heart attacks are a very common thing,
-
4:01 - 4:04and it took so long for us to find out that these drugs
-
4:04 - 4:07also caused an increased rate of death
-
4:07 - 4:10that before we detected that safety signal,
-
4:10 - 4:16over 100,000 people died unnecessarily in America
-
4:16 - 4:20from the prescription of anti-arrhythmic drugs.
-
4:20 - 4:23Now actually, in 1993,
-
4:23 - 4:27the researchers who did that 1980 study, that early study,
-
4:27 - 4:31published a mea culpa, an apology to the scientific community,
-
4:31 - 4:34in which they said, "When we carried out our study in 1980,
-
4:34 - 4:36we thought that the increased death rate that occurred
-
4:36 - 4:39in the lorcainide group was an effect of chance."
-
4:39 - 4:41The development of lorcainide was abandoned for commercial reasons,
-
4:41 - 4:43and this study was never published;
-
4:43 - 4:45it's now a good example of publication bias.
-
4:45 - 4:47That's the technical term for the phenomenon where
-
4:47 - 4:51unflattering data gets lost, gets unpublished, is left
-
4:51 - 4:55missing in action, and they say the results described here
-
4:55 - 4:59"might have provided an early warning of trouble ahead."
-
4:59 - 5:03Now these are stories from basic science.
-
5:03 - 5:07These are stories from 20, 30 years ago.
-
5:07 - 5:11The academic publishing environment is very different now.
-
5:11 - 5:14There are academic journals like "Trials," the open access journal,
-
5:14 - 5:17which will publish any trial conducted in humans
-
5:17 - 5:20regardless of whether it has a positive or a negative result.
-
5:20 - 5:24But this problem of negative results that go missing in action
-
5:24 - 5:28is still very prevalent. In fact it's so prevalent
-
5:28 - 5:34that it cuts to the core of evidence-based medicine.
-
5:34 - 5:37So this is a drug called reboxetine, and this is a drug
-
5:37 - 5:39that I myself have prescribed. It's an antidepressant.
-
5:39 - 5:42And I'm a very nerdy doctor, so I read all of the studies
-
5:42 - 5:45that I could on this drug. I read the one study that was published
-
5:45 - 5:48that showed that reboxetine was better than placebo,
-
5:48 - 5:50and I read the other three studies that were published
-
5:50 - 5:53that showed that reboxetine was just as good as any other antidepressant,
-
5:53 - 5:56and because this patient hadn't done well on those other antidepressants,
-
5:56 - 5:58I thought, well, reboxetine is just as good. It's one to try.
-
5:58 - 6:01But it turned out that I was misled. In fact,
-
6:01 - 6:04seven trials were conducted comparing reboxetine
-
6:04 - 6:07against a dummy placebo sugar pill. One of them
-
6:07 - 6:09was positive and that was published, but six of them
-
6:09 - 6:13were negative and they were left unpublished.
-
6:13 - 6:15Three trials were published comparing reboxetine
-
6:15 - 6:17against other antidepressants in which reboxetine
-
6:17 - 6:19was just as good, and they were published,
-
6:19 - 6:23but three times as many patients' worth of data was collected
-
6:23 - 6:25which showed that reboxetine was worse than
-
6:25 - 6:30those other treatments, and those trials were not published.
-
6:30 - 6:33I felt misled.
-
6:33 - 6:36Now you might say, well, that's an extremely unusual example,
-
6:36 - 6:38and I wouldn't want to be guilty of the same kind of
-
6:38 - 6:41cherry-picking and selective referencing
-
6:41 - 6:42that I'm accusing other people of.
-
6:42 - 6:44But it turns out that this phenomenon of publication bias
-
6:44 - 6:46has actually been very, very well studied.
-
6:46 - 6:49So here is one example of how you approach it.
-
6:49 - 6:51The classic model is, you get a bunch of studies where
-
6:51 - 6:53you know that they've been conducted and completed,
-
6:53 - 6:55and then you go and see if they've been published anywhere
-
6:55 - 6:58in the academic literature. So this took all of the trials
-
6:58 - 7:00that had ever been conducted on antidepressants
-
7:00 - 7:04that were approved over a 15-year period by the FDA.
-
7:04 - 7:08They took all of the trials which were submitted to the FDA as part of the approval package.
-
7:08 - 7:11So that's not all of the trials that were ever conducted on these drugs,
-
7:11 - 7:13because we can never know if we have those,
-
7:13 - 7:17but it is the ones that were conducted in order to get the marketing authorization.
-
7:17 - 7:19And then they went to see if these trials had been published
-
7:19 - 7:22in the peer-reviewed academic literature. And this is what they found.
-
7:22 - 7:25It was pretty much a 50-50 split. Half of these trials
-
7:25 - 7:28were positive, half of them were negative, in reality.
-
7:28 - 7:33But when they went to look for these trials in the peer-reviewed academic literature,
-
7:33 - 7:35what they found was a very different picture.
-
7:35 - 7:40Only three of the negative trials were published,
-
7:40 - 7:44but all but one of the positive trials were published.
-
7:44 - 7:48Now if we just flick back and forth between those two,
-
7:48 - 7:51you can see what a staggering difference there was
-
7:51 - 7:54between reality and what doctors, patients,
-
7:54 - 7:57commissioners of health services, and academics
-
7:57 - 8:00were able to see in the peer-reviewed academic literature.
-
8:00 - 8:05We were misled, and this is a systematic flaw
-
8:05 - 8:08in the core of medicine.
-
8:08 - 8:11In fact, there have been so many studies conducted on
-
8:11 - 8:14publication bias now, over a hundred, that they've been
-
8:14 - 8:17collected in a systematic review, published in 2010,
-
8:17 - 8:20that took every single study on publication bias
-
8:20 - 8:21that they could find.
-
8:21 - 8:24Publication bias affects every field of medicine.
-
8:24 - 8:28About half of all trials, on average, go missing in action,
-
8:28 - 8:31and we know that positive findings are around twice as likely
-
8:31 - 8:34to be published as negative findings.
-
8:34 - 8:39This is a cancer at the core of evidence-based medicine.
-
8:39 - 8:42If I flipped a coin 100 times but then
-
8:42 - 8:46withheld the results from you from half of those tosses,
-
8:46 - 8:49I could make it look as if I had a coin that always came up heads.
-
8:49 - 8:51But that wouldn't mean that I had a two-headed coin.
-
8:51 - 8:53That would mean that I was a chancer
-
8:53 - 8:56and you were an idiot for letting me get away with it. (Laughter)
-
8:56 - 8:59But this is exactly what we blindly tolerate
-
8:59 - 9:03in the whole of evidence-based medicine.
-
9:03 - 9:08And to me, this is research misconduct.
-
9:08 - 9:10If I conducted one study and I withheld
-
9:10 - 9:13half of the data points from that one study,
-
9:13 - 9:18you would rightly accuse me, essentially, of research fraud.
-
9:18 - 9:21And yet, for some reason, if somebody conducts
-
9:21 - 9:2510 studies but only publishes the five that give the result that they want,
-
9:25 - 9:28we don't consider that to be research misconduct.
-
9:28 - 9:31And when that responsibility is diffused between
-
9:31 - 9:34a whole network of researchers, academics,
-
9:34 - 9:37industry sponsors, journal editors, for some reason
-
9:37 - 9:39we find it more acceptable,
-
9:39 - 9:42but the effect on patients is damning.
-
9:42 - 9:48And this is happening right now, today.
-
9:48 - 9:50This is a drug called Tamiflu. Tamiflu is a drug
-
9:50 - 9:53which governments around the world have spent billions
-
9:53 - 9:55and billions of dollars on stockpiling,
-
9:55 - 9:59and we've stockpiled Tamiflu in panic,
-
9:59 - 10:02in the belief that it will reduce the rate of complications of influenza.
-
10:02 - 10:05Complications is a medical euphemism for pneumonia
-
10:05 - 10:10and death. (Laughter)
-
10:10 - 10:13Now when the Cochrane systematic reviewers
-
10:13 - 10:16were trying to collect together all of the data from all
-
10:16 - 10:19of the trials that had ever been conducted on whether Tamiflu actually did this or not,
-
10:19 - 10:22they found that several of those trials were unpublished.
-
10:22 - 10:24The results were unavailable to them.
-
10:24 - 10:28And when they started obtaining the writeups of those trials through various different means,
-
10:28 - 10:30through Freedom of Information Act requests, through
-
10:30 - 10:35harassing various different organizations, what they found was inconsistent.
-
10:35 - 10:37And when they tried to get a hold of the clinical study reports,
-
10:37 - 10:40the 10,000-page long documents that have
-
10:40 - 10:44the best possible rendition of the information,
-
10:44 - 10:47they were told they weren't allowed to have them.
-
10:47 - 10:49And if you want to read the full correspondence
-
10:49 - 10:53and the excuses and the explanations given by the drug company,
-
10:53 - 10:55you can see that written up in this week's edition
-
10:55 - 11:00of PLOS Medicine.
-
11:00 - 11:04And the most staggering thing of all of this, to me,
-
11:04 - 11:07is that not only is this a problem, not only do we recognize
-
11:07 - 11:11that this is a problem, but we've had to suffer fake fixes.
-
11:11 - 11:14We've had people pretend that this is a problem that's been fixed.
-
11:14 - 11:16First of all, we had trials registers, and everybody said,
-
11:16 - 11:20oh, it's okay. We'll get everyone to register their trials, they'll post the protocol,
-
11:20 - 11:22they'll say what they're going to do before they do it,
-
11:22 - 11:24and then afterwards we'll be able to check and see if all the trials which
-
11:24 - 11:26have been conducted and completed have been published.
-
11:26 - 11:29But people didn't bother to use those registers.
-
11:29 - 11:31And so then the International Committee of Medical Journal Editors came along,
-
11:31 - 11:33and they said, oh, well, we will hold the line.
-
11:33 - 11:35We won't publish any journals, we won't publish any trials,
-
11:35 - 11:38unless they've been registered before they began.
-
11:38 - 11:42But they didn't hold the line. In 2008, a study was conducted
-
11:42 - 11:45which showed that half of all of trials published by journals
-
11:45 - 11:47edited by members of the ICMJE
-
11:47 - 11:52weren't properly registered, and a quarter of them weren't registered at all.
-
11:52 - 11:55And then finally, the FDA Amendment Act was passed
-
11:55 - 11:57a couple of years ago saying that everybody who conducts
-
11:57 - 12:01a trial must post the results of that trial within one year.
-
12:01 - 12:05And in the BMJ, in the first edition of January, 2012,
-
12:05 - 12:08you can see a study which looks to see if people kept
-
12:08 - 12:11to that ruling, and it turns out that only one in five
-
12:11 - 12:14have done so.
-
12:14 - 12:17This is a disaster.
-
12:17 - 12:21We cannot know the true effects of the medicines
-
12:21 - 12:24that we prescribe if we do not have access
-
12:24 - 12:27to all of the information.
-
12:27 - 12:31And this is not a difficult problem to fix.
-
12:31 - 12:36We need to force people to publish all trials
-
12:36 - 12:39conducted in humans, including the older trials,
-
12:39 - 12:43because the FDA Amendment Act only asks that you publish the trials conducted after 2008,
-
12:43 - 12:46and I don't know what world it is in which we're only
-
12:46 - 12:50practicing medicine on the basis of trials that completed in the past two years.
-
12:50 - 12:53We need to publish all trials in humans,
-
12:53 - 12:56including the older trials, for all drugs in current use,
-
12:56 - 12:59and you need to tell everyone you know
-
12:59 - 13:02that this is a problem and that it has not been fixed.
-
13:02 - 13:05Thank you very much. (Applause)
-
13:05 - 13:08(Applause)
- Title:
- What doctors don't know about the drugs they prescribe
- Speaker:
- Ben Goldacre
- Description:
-
When a new drug gets tested, the results of the trials should be published for the rest of the medical world -- except much of the time, negative or inconclusive findings go unreported, leaving doctors and researchers in the dark. In this impassioned talk, Ben Goldacre explains why these unreported instances of negative data are especially misleading and dangerous.
- Video Language:
- English
- Team:
- closed TED
- Project:
- TEDTalks
- Duration:
- 13:29
Thu-Huong Ha edited English subtitles for What doctors don't know about the drugs they prescribe | ||
Orr Schlesinger accepted English subtitles for What doctors don't know about the drugs they prescribe | ||
Orr Schlesinger edited English subtitles for What doctors don't know about the drugs they prescribe | ||
Anna Bublinska edited English subtitles for What doctors don't know about the drugs they prescribe | ||
Anna Bublinska edited English subtitles for What doctors don't know about the drugs they prescribe | ||
Anna Bublinska edited English subtitles for What doctors don't know about the drugs they prescribe | ||
Retired user edited English subtitles for What doctors don't know about the drugs they prescribe | ||
Retired user edited English subtitles for What doctors don't know about the drugs they prescribe |