Inter-rater reliability of AMSTAR is dependent on the pair of reviewers.

ABSTRACT:

Background

Inter-rater reliability (IRR) is mainly assessed based on only two reviewers of unknown expertise. The aim of this paper is to examine differences in the IRR of the Assessment of Multiple Systematic Reviews (AMSTAR) and R(evised)-AMSTAR depending on the pair of reviewers.

Methods

Five reviewers independently applied AMSTAR and R-AMSTAR to 16 systematic reviews (eight Cochrane reviews and eight non-Cochrane reviews) from the field of occupational health. Responses were dichotomized and reliability measures were calculated by applying Holsti's method (r) and Cohen's kappa (κ) to all potential pairs of reviewers. Given that five reviewers participated in the study, there were ten possible pairs of reviewers.

Results

Inter-rater reliability varied for AMSTAR be

SUBMITTER: Pieper D 

PROVIDER: S-EPMC5504630 | biostudies-literature | 2017 Jul

REPOSITORIES: biostudies-literature