Biased binomial assessment of cross-validated estimation of classification accuracies illustrated in diagnosis predictions.

ABSTRACT: Multivariate classification is used in neuroimaging studies to infer brain activation or in medical applications to infer diagnosis. Their results are often assessed through either a binomial or a permutation test. Here, we simulated classification results of generated random data to assess the influence of the cross-validation scheme on the significance of results. Distributions built from classification of random data with cross-validation did not follow the binomial distribution. The binomial test is therefore not adapted. On the contrary, the permutation test was unaffected by the cross-validation scheme. The influence of the cross-validation was further illustrated on real-data from a brain-computer interface experiment in patients with disorders of consciousness and from an fMRI stud

SUBMITTER: Noirhomme Q 

PROVIDER: S-EPMC4053638 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature