Agreement Lambda for Weighted Disagreement With Ordinal Scales: Correction for Category Prevalence.

ABSTRACT: Weighted inter-rater agreement allows for differentiation between levels of disagreement among rating categories and is especially useful when there is an ordinal relationship between categories. Many existing weighted inter-rater agreement coefficients are either extensions of weighted Kappa or are formulated as Cohen's Kappa-like coefficients. These measures suffer from the same issues as Cohen's Kappa, including sensitivity to the marginal distributions of raters and the effects of category prevalence. They primarily account for the possibility of chance agreement or disagreement. This article introduces a new coefficient, weighted Lambda, which allows for the inclusion of varying weights assigned to disagreements. Unlike traditional methods, this coefficient does not assume random assi

SUBMITTER: Almehrizi RS 

PROVIDER: S-EPMC12602299 | biostudies-literature | 2025 Nov

REPOSITORIES: biostudies-literature