Stability of clinical prediction models developed using statistical or machine learning methods.

ABSTRACT: Clinical prediction models estimate an individual's risk of a particular health outcome. A developed model is a consequence of the development dataset and model-building strategy, including the sample size, number of predictors, and analysis method (e.g., regression or machine learning). We raise the concern that many models are developed using small datasets that lead to instability in the model and its predictions (estimated risks). We define four levels of model stability in estimated risks moving from the overall mean to the individual level. Through simulation and case studies of statistical and machine learning approaches, we show instability in a model's estimated risks is often considerable, and ultimately manifests itself as miscalibration of predictions in new data. Therefore, we

SUBMITTER: Riley RD 

PROVIDER: S-EPMC10952221 | biostudies-literature | 2023 Dec

REPOSITORIES: biostudies-literature