Statistical variability in comparing accuracy of neuroimaging based classification models via cross validation.

ABSTRACT: Machine learning (ML) has significantly transformed biomedical research, leading to a growing interest in model development to advance classification accuracy in various clinical applications. However, this progress raises essential questions regarding how to rigorously compare the accuracy of different ML models. In this study, we highlight the practical challenges in quantifying the statistical significance of accuracy differences between two neuroimaging-based classification models when cross-validation (CV) is performed. Specifically, we propose an unbiased framework to assess the impact of CV setups (e.g., the number of folds) on the statistical significance. We apply this framework to three publicly available neuroimaging datasets to re-emphasize known flaws in current computation of

SUBMITTER: Jafrasteh B 

PROVIDER: S-EPMC12328793 | biostudies-literature | 2025 Aug

REPOSITORIES: biostudies-literature