A novel, standardised approach to balancing effectiveness, efficiency and utility of surveillance AI prediction models for hospitalised patients using sepsis prediction as an exemplar.

ABSTRACT:

Objective

To introduce a novel, standardised approach to evaluating AI prediction models in balancing effectiveness, efficiency and utility, using a sepsis prediction model case study.

Materials and methods

Retrospective patient data from electronic medical records of 7 public hospitals was used to retrain and evaluate a machine learning sepsis prediction model. Four conventional metrics-area under the receiver operating curve (AUROC), sensitivity, positive predictive value, and specificity-were compared with a novel graphical display integrating metrics of predictive accuracy (effectiveness), alert burden (efficiency) and lead time of alerts relative to clinical events (utility) for different alert thresholds.

Results

The dataset comprised 977,506 inpatient admission

SUBMITTER: van der Vegt AH 

PROVIDER: S-EPMC12844580 | biostudies-literature | 2026 Feb

REPOSITORIES: biostudies-literature