Title |
An investigation of the false discovery rate and the misinterpretation of p-values
|
---|---|
Published in |
Royal Society Open Science, November 2014
|
DOI | 10.1098/rsos.140216 |
Pubmed ID | |
Authors |
David Colquhoun |
Abstract |
If you use p=0.05 to suggest that you have made a discovery, you will be wrong at least 30% of the time. If, as is often the case, experiments are underpowered, you will be wrong most of the time. This conclusion is demonstrated from several points of view. First, tree diagrams which show the close analogy with the screening test problem. Similar conclusions are drawn by repeated simulations of t-tests. These mimic what is done in real life, which makes the results more persuasive. The simulation method is used also to evaluate the extent to which effect sizes are over-estimated, especially in underpowered experiments. A script is supplied to allow the reader to do simulations themselves, with numbers appropriate for their own work. It is concluded that if you wish to keep your false discovery rate below 5%, you need to use a three-sigma rule, or to insist on p≤0.001. And never use the word 'significant'. |
X Demographics
Geographical breakdown
Country | Count | As % |
---|---|---|
United Kingdom | 333 | 23% |
United States | 236 | 16% |
Canada | 56 | 4% |
Australia | 56 | 4% |
Germany | 22 | 2% |
Spain | 21 | 1% |
France | 21 | 1% |
Netherlands | 17 | 1% |
Ireland | 15 | 1% |
Other | 155 | 11% |
Unknown | 518 | 36% |
Demographic breakdown
Type | Count | As % |
---|---|---|
Members of the public | 804 | 55% |
Scientists | 477 | 33% |
Practitioners (doctors, other healthcare professionals) | 99 | 7% |
Science communicators (journalists, bloggers, editors) | 66 | 5% |
Unknown | 4 | <1% |
Mendeley readers
Geographical breakdown
Country | Count | As % |
---|---|---|
United Kingdom | 66 | 3% |
United States | 54 | 3% |
Germany | 17 | <1% |
Brazil | 10 | <1% |
Netherlands | 7 | <1% |
France | 5 | <1% |
Australia | 5 | <1% |
Italy | 4 | <1% |
Spain | 4 | <1% |
Other | 49 | 2% |
Unknown | 1876 | 89% |
Demographic breakdown
Readers by professional status | Count | As % |
---|---|---|
Researcher | 476 | 23% |
Student > Ph. D. Student | 462 | 22% |
Student > Master | 233 | 11% |
Student > Bachelor | 164 | 8% |
Other | 133 | 6% |
Other | 424 | 20% |
Unknown | 205 | 10% |
Readers by discipline | Count | As % |
---|---|---|
Agricultural and Biological Sciences | 514 | 25% |
Psychology | 204 | 10% |
Medicine and Dentistry | 196 | 9% |
Biochemistry, Genetics and Molecular Biology | 147 | 7% |
Neuroscience | 100 | 5% |
Other | 626 | 30% |
Unknown | 310 | 15% |