100% classification accuracy considered harmful: the normalized information transfer factor explains the accuracy paradox.

ABSTRACT: The most widely spread measure of performance, accuracy, suffers from a paradox: predictive models with a given level of accuracy may have greater predictive power than models with higher accuracy. Despite optimizing classification error rate, high accuracy models may fail to capture crucial information transfer in the classification task. We present evidence of this behavior by means of a combinatorial analysis where every possible contingency matrix of 2, 3 and 4 classes classifiers are depicted on the entropy triangle, a more reliable information-theoretic tool for classification assessment. Motivated by this, we develop from first principles a measure of classification performance that takes into consideration the information learned by classifiers. We are then able to obtain the entro

SUBMITTER: Valverde-Albacete FJ 

PROVIDER: S-EPMC3888391 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature