A surge of p-values between 0.040 and 0.049 in recent decades (but negative results are increasing rapidly too)
- Published
- Accepted
- Subject Areas
- Science Policy, Statistics
- Keywords
- significant differences, bias, physical sciences, biological sciences, social sciences, science policy
- Copyright
- © 2014 de Winter et al.
- Licence
- This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ PrePrints) and either DOI or URL of the article must be cited.
- Cite this article
- 2014. A surge of p-values between 0.040 and 0.049 in recent decades (but negative results are increasing rapidly too) PeerJ PrePrints 2:e447v3 https://doi.org/10.7287/peerj.preprints.447v3
Abstract
It is known that statistically significant results are more likely to be published than results that are not statistically significant. However, it is unclear whether negative results are disappearing from papers, and whether there exists a ‘hierarchy of sciences’ with the social sciences publishing more positive results than the physical sciences. Using Scopus, we conducted a search in the abstracts of papers published between 1990 and 2014, and calculated the percentage of papers reporting marginally positive results (i.e., p-values between 0.040 and 0.049) versus the percentage of papers reporting marginally negative results (i.e., p-values between 0.051 and 0.060). The results indicate that negative results are not disappearing, but have actually become 4.3 times more prevalent since 1990. Positive results, on the other hand, have become 13.9 times more prevalent since 1990. We found no consistent support for a ‘hierarchy of sciences’. However, we did find large differences in reporting practices between disciplines, with the reporting of p-values being 60.6 times more frequent in the biological sciences than in the physical sciences. We argue that the observed longitudinal trends may be caused by negative factors, such as an increase of questionable research practices, but also by positive factors, such as an increasingly quantitative research focus.
Author Comment
We improved the legends of the tables, to make clear that the table contains the percentage of papers reporting a significant result over the sum of the papers reporting a significant or non-significant result (S/(S+NS)).