The impact of inconsistent human annotations on AI driven clinical decision making.
Ontology highlight
ABSTRACT: In supervised learning model development, domain experts are often used to provide the class labels (annotations). Annotation inconsistencies commonly occur when even highly experienced clinical experts annotate the same phenomenon (e.g., medical image, diagnostics, or prognostic status), due to inherent expert bias, judgments, and slips, among other factors. While their existence is relatively well-known, the implications of such inconsistencies are largely understudied in real-world settings, when supervised learning is applied on such 'noisy' labelled data. To shed light on these issues, we conducted extensive experiments and analyses on three real-world Intensive Care Unit (ICU) datasets. Specifically, individual models were built from a common dataset, annotated independently by 11 Gl
SUBMITTER: Sylolypavan A
PROVIDER: S-EPMC9944930 | biostudies-literature | 2023 Feb
REPOSITORIES: biostudies-literature
ACCESS DATA