Modeling Rating Order Effects Under Item Response Theory Models for Rater-Mediated Assessments.

ABSTRACT: Rater effects are commonly observed in rater-mediated assessments. By using item response theory (IRT) modeling, raters can be treated as independent factors that function as instruments for measuring ratees. Most rater effects are static and can be addressed appropriately within an IRT framework, and a few models have been developed for dynamic rater effects. Operational rating projects often require human raters to continuously and repeatedly score ratees over a certain period, imposing a burden on the cognitive processing abilities and attention spans of raters that stems from judgment fatigue and thus affects the rating quality observed during the rating period. As a result, ratees' scores may be influenced by the order in which they are graded by raters in a rating sequence, and the r

SUBMITTER: Huang HY 

PROVIDER: S-EPMC10240569 | biostudies-literature | 2023 Jun

REPOSITORIES: biostudies-literature