Question everything

Can bad scientific practices be fixed? Part of the problem is that no-one is incentivised to be right. Instead, scientists are incentivised to be productive and innovative. Would a Hippocratic Oath for science help? Certainly don't add more layers of research red-tape. Instead of changing incentives, perhaps one could remove incentives altogether. Or insist on replicability statements in grant applications and research papers. Or emphasise collaboration, not competition. Or insist on preregistration of protocols. Or reward better pre and post publication peer review. Or improve research training and mentorship. Or implement the recommendations from our Series on increasing research value, published last year. One of the most convincing proposals came from outside the biomedical community. Tony Weidberg is a Professor of Particle Physics at Oxford. Following several high-profile errors, the particle physics community now invests great effort into intensive checking and re-checking of data prior to publication. By filtering results through independent working groups, physicists are encouraged to criticise. Good criticism is rewarded. The goal is a reliable result, and the incentives for scientists are aligned around this goal. Weidberg worried we set the bar for results in biomedicine far too low. In particle physics, significance is set at 5 sigma—a p value of 3 × 10–7 or 1 in 3·5 million (if the result is not true, this is the probability that the data would have been as extreme as they are). The conclusion of the symposium was that something must be done. Indeed, all seemed to agree that it was within our power to do that something. But as to precisely what to do or how to do it, there were no firm answers. Those who have the power to act seem to think somebody else should act first. And every positive action (eg, funding well-powered replications) has a counterargument (science will become less creative). The good news is that science is beginning to take some of its worst failings very seriously. The bad news is that nobody is ready to take the first step to clean up the system.

Тhe pharmaceutical industry has gained enormous control over how doctors evaluate and use its own products. Its extensive ties to physicians, particularly senior faculty at prestigious medical schools, affect the results of research, the way medicine is practiced, and even the definition of what constitutes a disease.

In view of this control and the conflicts of interest that permeate the enterprise, it is not surprising that industry-sponsored trials published in medical journals consistently favor sponsors’ drugs—largely because negative results are not published, positive results are repeatedly published in slightly different forms, and a positive spin is put on even negative results. A review of seventy-four clinical trials of antidepressants, for example, found that thirty-seven of thirty-eight positive studies were published.8 But of the thirty-six negative studies, thirty-three were either not published or published in a form that conveyed a positive outcome. It is not unusual for a published paper to shift the focus from the drug’s intended effect to a secondary effect that seems more favorable.

Many drugs that are assumed to be effective are probably little better than placebos, but there is no way to know because negative results are hidden. One clue was provided six years ago by four researchers who, using the Freedom of Information Act, obtained FDA reviews of every placebo-controlled clinical trial submitted for initial approval of the six most widely used antidepressant drugs approved between 1987 and 1999—Prozac, Paxil, Zoloft, Celexa, Serzone, and Effexor. They found that on average, placebos were 80 percent as effective as the drugs. The difference between drug and placebo was so small that it was unlikely to be of any clinical significance. The results were much the same for all six drugs: all were equally ineffective. But because favorable results were published and unfavorable results buried (in this case, within the FDA), the public and the medical profession believed these drugs were potent antidepressants.
Clinical trials are also biased through designs for research that are chosen to yield favorable results for sponsors. For example, the sponsor’s drug may be compared with another drug administered at a dose so low that the sponsor’s drug looks more powerful. Or a drug that is likely to be used by older people will be tested in young people, so that side effects are less likely to emerge. A common form of bias stems from the standard practice of comparing a new drug with a placebo, when the relevant question is how it compares with an existing drug. In short, it is often possible to make clinical trials come out pretty much any way you want, which is why it’s so important that investigators be truly disinterested in the outcome of their work.

In recent years, drug companies have perfected a new and highly effective method to expand their markets. Instead of promoting drugs to treat diseases, they have begun to promote diseases to fit their drugs. The strategy is to convince as many people as possible (along with their doctors, of course) that they have medical conditions that require long-term drug treatment. Sometimes called “disease-mongering,”

To promote new or exaggerated conditions, companies give them serious-sounding names along with abbreviations. Thus, heartburn is now “gastro-esophageal reflux disease” or GERD; impotence is “erectile dysfunction” or ED; premenstrual tension is “premenstrual dysphoric disorder” or PMMD; and shyness is “social anxiety disorder” (no abbreviation yet). Note that these are ill-defined chronic conditions that affect essentially normal people, so the market is huge and easily expanded. For example, a senior marketing executive advised sales representatives on how to expand the use of Neurontin: “Neurontin for pain, Neurontin for monotherapy, Neurontin for bipolar, Neurontin for everything.” It seems that the strategy of the drug marketers—and it has been remarkably successful—is to convince Americans that there are only two kinds of people: those with medical conditions that require drug treatment and those who don’t know it yet. While the strategy originated in the industry, it could not be implemented without the complicity of the medical profession.

Given its importance, you might think that the DSM (Diagnostic and Statistical Manual of Mental Disorders - библия психиатров) represents the authoritative distillation of a large body of scientific evidence. But Lane, using unpublished records from the archives of the American Psychiatric Association and interviews with the princi-pals, shows that it is instead the product of a complex of academic politics, personal ambition, ideology, and, perhaps most important, the influence of the pharmaceutical industry. What the DSM lacks is evidence. Lane quotes one contributor to the DSM-III task force:

There was very little systematic research, and much of the research that existed was really a hodgepodge—scattered, inconsistent, and ambiguous. I think the majority of us recognized that the amount of good, solid science upon which we were making our decisions was pretty modest.

Some of the biggest blockbusters are psychoactive drugs. The theory that psychiatric conditions stem from a biochemical imbalance is used as a justification for their widespread use, even though the theory has yet to be proved. Children are particularly vulnerable targets. What parents dare say “No” when a physician says their difficult child is sick and recommends drug treatment? We are now in the midst of an apparent epidemic of bipolar disease in children (which seems to be replacing attention-deficit hyperactivity disorder as the most publicized condition in childhood), with a forty-fold increase in the diagnosis between 1994 and 2003.18 These children are often treated with multiple drugs off-label, many of which, whatever their other properties, are sedating, and nearly all of which have potentially serious side effects.

Мany reforms would be necessary to restore integrity to clinical research and medical practice that they cannot be summarized briefly. Many would involve congressional legislation and changes in the FDA, including its drug approval process. But there is clearly also a need for the medical profession to wean itself from industry money almost entirely. Although industry–academic collaboration can make important scientific contributions, it is usually in carrying out basic research, not clinical trials, and even here, it is arguable whether it necessitates the personal enrichment of investigators. Members of medical school faculties who conduct clinical trials should not accept any payments from drug companies except research support, and that support should have no strings attached, including control by drug companies over the design, interpretation, and publication of research results.

Medical schools and teaching hospitals should rigorously enforce that rule, and should not enter into deals with companies whose products members of their faculty are studying. Finally, there is seldom a legitimate reason for physicians to accept gifts from drug companies, even small ones, and they should pay for their own meetings and continuing education.

After much unfavorable publicity, medical schools and professional organizations are beginning to talk about controlling conflicts of interest, but so far the response has been tepid. They consistently refer to “potential” conflicts of interest, as though that were different from the real thing, and about disclosing and “managing” them, not about prohibiting them. In short, there seems to be a desire to eliminate the smell of corruption, while keeping the money. Breaking the dependence of the medical profession on the pharmaceutical industry will take more than appointing committees and other gestures. It will take a sharp break from an extremely lucrative pattern of behavior. But if the medical profession does not put an end to this corruption voluntarily, it will lose the confidence of the public, and the government (not just Senator Grassley) will step in and impose regulation. No one in medicine wants that.