Please use this identifier to cite or link to this item: doi:10.22028/D291-32422
Volltext verfügbar? / Dokumentlieferung
Title: p-Hacking and publication bias interact to distort meta-analytic effect size estimates
Author(s): Friese, Malte
Frankenbach, Julius
Language: English
Title: Psychological methods : published quarterly by the American Psychological Association
Volume: 25
Issue: 4
Startpage: 456
Endpage: 471
Publisher/Platform: American Psychological Association
Year of Publication: 2020
Publikation type: Journal Article
Abstract: Science depends on trustworthy evidence. Thus, a biased scientific record is of questionable value because it impedes scientific progress, and the public receives advice on the basis of unreliable evidence that has the potential to have far-reaching detrimental consequences. Meta-analysis is a technique that can be used to summarize research evidence. However, meta-analytic effect size estimates may themselves be biased, threatening the validity and usefulness of meta-analyses to promote scientific progress. Here, we offer a large-scale simulation study to elucidate how p-hacking and publication bias distort meta-analytic effect size estimates under a broad array of circumstances that reflect the reality that exists across a variety of research areas. The results revealed that, first, very high levels of publication bias can severely distort the cumulative evidence. Second, p-hacking and publication bias interact: At relatively high and low levels of publication bias, p-hacking does comparatively little harm, but at medium levels of publication bias, p-hacking can considerably contribute to bias, especially when the true effects are very small or are approaching zero. Third, p-hacking can severely increase the rate of false positives. A key implication is that, in addition to preventing p-hacking, policies in research institutions, funding agencies, and scientific journals need to make the prevention of publication bias a top priority to ensure a trustworthy base of evidence.
DOI of the first publication: 10.1037/met0000246
URL of the first publication:
Link to this record: hdl:20.500.11880/29795
ISSN: 1082-989X
Date of registration: 1-Oct-2020
Faculty: HW - Fakultät für Empirische Humanwissenschaften und Wirtschaftswissenschaft
Department: HW - Psychologie
Professorship: HW - Prof. Dr. Malte Friese
Collections:SciDok - Der Wissenschaftsserver der Universität des Saarlandes

Files for this record:
There are no files associated with this item.

Items in SciDok are protected by copyright, with all rights reserved, unless otherwise indicated.