Testing for Questionable Research Practices in a Meta-Analysis: An Example from Experimental Parapsychology.

ABSTRACT: We describe a method of quantifying the effect of Questionable Research Practices (QRPs) on the results of meta-analyses. As an example we simulated a meta-analysis of a controversial telepathy protocol to assess the extent to which these experimental results could be explained by QRPs. Our simulations used the same numbers of studies and trials as the original meta-analysis and the frequencies with which various QRPs were applied in the simulated experiments were based on surveys of experimental psychologists. Results of both the meta-analysis and simulations were characterized by 4 metrics, two describing the trial and mean experiment hit rates (HR) of around 31%, where 25% is expected by chance, one the correlation between sample-size and hit-rate, and one the complete P-value distribut

SUBMITTER: Bierman DJ 

PROVIDER: S-EPMC4856278 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature