Let’s Be Negative

There is a dark joke among scientists that if an experiment works right the first time, there must be something wrong. This joke stems from the reality that the conjectures and hypotheses that drive science forward also produce an enormous number of ideas that are simply wrong.

Confirming that ideas are wrong has always been less glamorous than proving ideas correct. Negative results—which do not confirm a hypothesis about how something works—do not make good scientific papers. Scientists, especially those who do biomedical research, rarely share such negative results with their competitors and collaborators. This lack of sharing leads to wasted resources and repetition of experiments when one group of scientists does not know what another group has already done.

But in a move that could be a boon for biomedical scientists everywhere, a Harvard Medical School professor has started the “Journal of Negative Results in Biomedicine” (JNRBM) which is dedicated to publishing the results that could save scientists from doing years of work just to get the same negative result that others have already found. In an introductory editorial, Hersey Professor of Cell Biology Bjorn R. Olsen, who edits the journal, and Visiting Research Fellow in Pediatrics Christian Pfeffer wrote that “it is useful and important to publish well documented failures, such as with drugs that show no benefit for which the shortcomings have not been publicized.” Publishing such failures, the Olsen and Pfeffer argue, can make significant contributions to the advancement of science. But Olsen and Pfeffer’s attempt to publish negative results does not account for a crucial aspect of scientific inquiry. Although the idea of publishing negative results would benefit science as a whole, individual scientists gain very little until a critical mass of negative results is published in their field.

Indeed, the small number of scientists willing to publish negative results may be a function of self-interest. For example, the Journal of Universal Computer Science started a section for negative results in 1997. Since then, the journal has received no submissions on negative results, leading one editorial board member to quip that the only known negative result was the failure of the section itself. Owning up to a failed hypothesis is something few scientists would be willing to do unless their peers are doing so as well. After all, admitting to getting something wrong does little to help a scientist’s reputation among colleagues.

Likewise, Olsen’s JNRBM itself has had only a handful of submissions since it was launched last fall. In their introductory editorial, Olsen and Pfeffer address some of the most important reasons why scientists may be wary of publishing negative results. The first reason is that it could give crucial information to competitors who may be able to use those negative results to beat everyone else to an important positive result. This problem is magnified for scientists who are the first to publish negative results. Those researchers benefit very little from the sparse numbers of negative results published by others.

The second objection is that publishing negative results could lead to shoddy science—when things do not work, there are often several possible explanations, and scientists are hardly going to track down what the cause of their problem was when there are always more fruitful research avenues to pursue. Negative results can be helpful in planning further experiments or overthrowing old ways of thinking about problems, but they add very little to the knowledge of how the world really works. Until a large number of negative papers are published in a given field, scientists will gain little by publishing their negative results since the only people to benefit will likely be competitors.

But publishing negative results may not always be altruistic. Some negative results can be used as weapons against competitors. There are few lines in scientific papers more chilling than “we could not account for the serious discrepancy between so-and-so’s data and our own.” Such open questioning of a competitor’s work could make journals of negative results into sparring grounds where rivals try to prove each other wrong.

Nevertheless, the resistance to publishing negative results in scientific journals speaks to larger issues about the culture of science. That problem is the widespread belief that publication of big results in a handful of prestigious journals is the primary goal of scientists. In a commentary in the journal Nature last month, a prominent editor of scientific journals wrote about how the obsession to publish in the top journals such as Nature, Science or Cell has sometimes eroded the quality of work produced. The narrow self-interest that pushes scientists to focus on a handful of prestigious journals also prevents them from publishing negative results. Until publishing negative results helps scientists advance their careers, we can expect them to continue focusing on the positive.

Jonathan H. Esensten ’04 is a biochemical sciences concentrator in Lowell House. His column appears on alternate Mondays.