Ranking versus rating in peer review of research grant applications.

ABSTRACT: The allocation of public funds for research has been predominantly based on peer review where reviewers are asked to rate an application on some form of ordinal scale from poor to excellent. Poor reliability and bias of peer review rating has led funding agencies to experiment with different approaches to assess applications. In this study, we compared the reliability and potential sources of bias associated with application rating with those of application ranking in 3,156 applications to the Canadian Institutes of Health Research. Ranking was more reliable than rating and less susceptible to the characteristics of the review panel, such as level of expertise and experience, for both reliability and potential sources of bias. However, both rating and ranking penalized early career investi

SUBMITTER: Tamblyn R 

PROVIDER: S-EPMC10553257 | biostudies-literature | 2023

REPOSITORIES: biostudies-literature