The revelations struck a chord with the scientific community at large: A recent essay by Ioannidis simply entitled “Why most published research findings are false” has been downloaded more than 100,000 times; the Boston Globe called it “an instant cult classic.” Now in a Möbius-strip-like twist, there is a growing body of research that is investigating, analyzing, and suggesting causes and solutions for faulty research.
Two papers published this spring in the open-access journal PLoS Medicine by Benjamin Djulbegovic from the University of South Florida and Ramal Moonesinghe from the CDC have delved into the issues raised by Ioannidis and suggested possible ways to mitigate this apparent failure of scientific enterprise.
One of the suggestions is to ensure that experimental results are independently replicable. “More often than not, genuine replication is not done, and what we end up with in the literature is corroboration or indirect supporting evidence,” says Moonesinghe.
The culprits appear to be the proverbial suspects: lies, damn lies, and statistics. Jonathan Sterne and George Smith, a statistician and an epidemiologist from the university of Bristol in the UK, point out in a study in British Medical Journal (BMJ) that “the widespread misunderstanding of statistical significance is a fundamental problem” in medical research.
What’s more, the scientist’s bias may distort statistics. Pressure to publish can lead to “selective reporting”; the implication is that attention-seeking scientists are exaggerating their results far more often than the occasional, spectacular science fraud would suggest.
Cash-for-science practices between the nutrition and drug companies and the academics that conduct their research may also be playing a role. A survey of published results on beverages earlier this year found that research sponsored by industry is much more likely to report favorable findings than papers with other sources of funding. Although not a direct indication of bias, findings like these feed suspicion that the cherry-picking of data, hindrance of negative results, or adjustment of research is surreptitiously corrupting accuracy. In his essay, Ioannidis wrote, “The greater the financial and other interest and prejudices in a scientific field, the less likely the research findings are to be true.”
Academic bias could also be to blame. As Ioannidis puts it, “Prestigious investigators may suppress via the peer-review process the appearance and dissemination of findings that refute their findings, thus condemning their field to perpetuate false dogma.” Advocates of prevailing paradigms have been observed to band together in opposition against alternative ideas with perhaps more antagonism than one might expect from objective scientific debate. And the opposition isn’t limited to publication of new science; jobs and grants are also more easily allocated to those affiliated with the scientific party in power.
Ioannidis is adamant that the problem is widespread. “I have heard from scientists from many different fields who think that the problems are the same in their fields as well,” he says. “This is a potentially severe crisis, unless we realize the issue and try to address it.”
With the debate over the causes and solutions of high rates of falsifiable research findings ongoing, how the problem is seen in the eyes of a skeptical public may be another issue altogether.
Virginia Barbour, managing editor of PLoS Medicine, puts it simply: “In terms of perception, the point is that science doesn’t emerge from single new findings that become ‘breakthrough’ stories in the media, but rather from developments that mature over months or years, with different sources of experimental validation.”
Source:
“Dirty Little Secret”
João Medeiros, Seed Magazine, May 21, 2007
