How large should the next study be? Predictive power and sample size requirements for replication studies.

ABSTRACT: We use information derived from over 40K trials in the Cochrane Collaboration database of systematic reviews (CDSR) to compute the replication probability, or predictive power of an experiment given its observed (two-sided) P$$ P $$ -value. We find that an exact replication of a marginally significant result with P=.05$$ P=.05 $$ has less than 30% chance of again reaching significance. Moreover, the replication of a result with P=.005$$ P=.005 $$ still has only 50% chance of significance. We also compute the probability that the direction (sign) of the estimated effect is correct, which is closely related to the type S error of Gelman and Tuerlinckx. We find that if an estimated effect has P=.05$$ P=.05 $$ , there is a 93% probability that its sign is correct. If P=.005$$ P=.005 $$ , then

SUBMITTER: van Zwet EW 

PROVIDER: S-EPMC9325423 | biostudies-literature | 2022 Jul

REPOSITORIES: biostudies-literature