Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The . gov means it’s official. Federal government websites often end in . gov or . mil. Before sharing sensitive information, make sure you’re on a federal government site VSports app下载. .

Https

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely V体育官网. .

. 2015 Mar 13;13(3):e1002106.
doi: 10.1371/journal.pbio.1002106. eCollection 2015 Mar.

The extent and consequences of p-hacking in science

Megan L Head  1 , Luke Holman  1 , Rob Lanfear  2 , Andrew T Kahn  1 , Michael D Jennions  1
Affiliations

The extent and consequences of p-hacking in science

Megan L Head et al. PLoS Biol. .

Abstract (V体育安卓版)

A focus on novel, confirmatory, and statistically significant results leads to substantial bias in the scientific literature. One type of bias, known as "p-hacking," occurs when researchers collect or select data or statistical analyses until nonsignificant results become significant VSports手机版. Here, we use text-mining to demonstrate that p-hacking is widespread throughout science. We then illustrate how one can test for p-hacking when performing a meta-analysis and show that, while p-hacking is probably common, its effect seems to be weak relative to the real effect sizes being measured. This result suggests that p-hacking probably does not drastically alter scientific consensuses drawn from meta-analyses. .

PubMed Disclaimer

Conflict of interest statement

The authors have declared that no competing interests exist.

Figures (VSports app下载)

Fig 1
Fig 1. The effect of publication bias on the distribution of p-values around the significance threshold of 0.05.
A) Black line shows distribution of p-values when there is no evidential value and the red line shows how publication bias influences this distribution. B) Black line shows distribution of p-values when there is evidential value and red line shows how publication bias influences this distribution. Tests for publication bias due to a file-drawer effect often compare the number of p-values in each of the bins either side of 0.05.
Fig 2
Fig 2. The effect of p-hacking on the distribution of p-values in the range of significance.
A) Black line shows distribution of p-values when there is no evidential value and the red line shows how p-hacking influences this distribution. B) Black line shows distribution of p-values when there is evidential value and the red line shows how p-hacking influences this distribution. Tests for p-hacking often compare the number of p-values in two adjacent bins just below 0.05.
Fig 3
Fig 3. Evidence for p-hacking across scientific disciplines.
A) Evidence for p-hacking from p-values obtained from Results sections. B) Evidence for p-hacking from p-values obtained from Abstracts. The strength of p-hacking is presented as the proportion of p-values in the upper bin (0.045 < p < 0.05) with one-tailed 95% confidence intervals (calculated following Clopper and Pearson [47] using the binom.test function in R). Only disciplines where text-mining of the Results sections returned more than 25 p-values between 0.04 and 0.05 are presented. Marker colour is shaded according to the sample size: with white indicating low samples sizes and red indicating larger sample sizes.
Fig 4
Fig 4. The distribution of p-values associated with the meta-analysis conducted by Jiang et al. (2013).
The p-curve shows evidence for evidential value (strong right skew) and p-hacking (rise in p-values just below 0.05).
See this image and copyright information in PMC

Comment in

"V体育官网入口" References

    1. Barch DM, Yarkoni T (2013) Introduction to the special issue on reliability and replication in cognitive and affective neuroscience research. Cogn Affect Behav Neurosci 13: 687–689. 10.3758/s13415-013-0201-7 - DOI - PubMed
    1. Ioannidis JPA (2005) Why most published research findings are false. PLoS Med 2: e124 - PMC - PubMed
    1. Jager LR, Leek JT (2014) An estimate of the science-wise false discovery rate and application to the top medical literature. Biostatistics 15: 1–12. 10.1093/biostatistics/kxt007 - DOI - PubMed
    1. Begg CB, Berlin JA (1988) Publication bias—a problem in interpreting medical data. J R Stat Soc Ser A Stat Soc 151: 419–463.
    1. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan A-W, et al. (2008) Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE 3: e3081 10.1371/journal.pone.0003081 - DOI - PMC - PubMed

VSports手机版 - Publication types