Adaptive Confidence Intervals for the Test Error in Classification.

ABSTRACT: The estimated test error of a learned classifier is the most commonly reported measure of classifier performance. However, constructing a high quality point estimator of the test error has proved to be very difficult. Furthermore, common interval estimators (e.g. confidence intervals) are based on the point estimator of the test error and thus inherit all the difficulties associated with the point estimation problem. As a result, these confidence intervals do not reliably deliver nominal coverage. In contrast we directly construct the confidence interval by use of smooth data-dependent upper and lower bounds on the test error. We prove that for linear classifiers, the proposed confidence interval automatically adapts to the non-smoothness of the test error, is consistent under fixed and lo

SUBMITTER: Laber EB 

PROVIDER: S-EPMC3285493 | biostudies-literature | 2011 Sep

REPOSITORIES: biostudies-literature