On the agreement between bibliometrics and peer review: Evidence from the Italian research assessment exercises.

ABSTRACT: This paper analyzes the concordance between bibliometrics and peer review. It draws evidence from the data of two experiments of the Italian governmental agency for research evaluation. The experiments were performed by the agency for validating the adoption in the Italian research assessment exercises of a dual system of evaluation, where some outputs were evaluated by bibliometrics and others by peer review. The two experiments were based on stratified random samples of journal articles. Each article was scored by bibliometrics and by peer review. The degree of concordance between the two evaluations is then computed. The correct setting of the experiments is defined by developing the design-based estimation of the Cohen's kappa coefficient and some testing procedures for assessing the h

SUBMITTER: Baccini A 

PROVIDER: S-EPMC7673579 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature