Why Most Published Research Findings Are False
John Ioannidis
Summary and citation · read the original at the source
In one page
John Ioannidis is an epidemiologist at Stanford, and this 2005 essay is the most-read article PLoS Medicine has published. His argument is arithmetic rather than complaint. Whether a published finding is actually true depends on four things: how large the study was, how much bias crept in, how many other teams are working the same question, and — above all — how likely the hypothesis was to be true before anybody tested it. Put realistic numbers into that model and, for most study designs in most fields, a claim that clears the usual significance threshold is more likely to be false than true. Ioannidis then lists the conditions that make it worse, and several are uncomfortable reading: small studies, small effects, many relationships tested with little preselection, flexible definitions and analyses, strong financial interests, and, surprisingly, a hot field with many teams racing one another.
Why it matters hereChapter 1 teaches readers to weigh a result rather than count results, and this is where the arithmetic comes from — a single published finding is a prior plus a measurement, not a verdict. It is also why chapter 13 keeps a ledger of what would settle each claim instead of a tally of papers.
What it claims
01The probability that a research claim is true depends on statistical power, the level of bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field.Abstract; Modeling the Framework for False Positive Findings
Published and peer-reviewed02The smaller the studies conducted in a scientific field, and the smaller the effect sizes, the less likely the research findings are to be true.Corollaries 1 and 2
Published and peer-reviewed03The greater the number and the lesser the selection of tested relationships, and the greater the flexibility in designs, definitions, outcomes and analytical modes, the less likely the research findings are to be true.Corollaries 3 and 4
Published and peer-reviewed04The greater the financial and other interests and prejudices in a field, and the hotter the field in the sense of more scientific teams being involved, the less likely the research findings are to be true.Corollaries 5 and 6
Published and peer-reviewed05Simulations show that for most study designs and settings it is more likely for a research claim to be false than true, so a single positive result crossing a significance threshold is not by itself strong evidence.Most Research Findings Are False for Most Research Designs and for Most Fields
Published and peer-reviewed06Claimed research findings may often be simply accurate measures of the prevailing bias in a field; the improvements Ioannidis proposes are better-powered evidence, registration of hypotheses and study standards, and explicit attention to the pre-study odds rather than reliance on a single p-value.Claimed Research Findings May Often Be Simply Accurate Measures of the Prevailing Bias; How Can We Improve the Situation?
Published and peer-reviewed
The way in
https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124Open access. © 2005 John P. A. Ioannidis, distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution and reproduction in any medium provided the original work is properly cited — so the full text may be carried here once the rights pass is done.
How to cite it
John Ioannidis (2005) Why Most Published Research Findings Are False. doi:10.1371/journal.pmed.0020124
Where it sits in the curriculum