Predicting sample size required for classification performance.

ABSTRACT:

Background

Supervised learning methods need annotated data in order to generate efficient models. Annotated data, however, is a relatively scarce resource and can be expensive to obtain. For both passive and active learning methods, there is a need to estimate the size of the annotated sample required to reach a performance target.

Methods

We designed and implemented a method that fits an inverse power law model to points of a given learning curve created using a small annotated training set. Fitting is carried out using nonlinear weighted least squares optimization. The fitted model is then used to predict the classifier's performance and confidence interval for larger sample sizes. For evaluation, the nonlinear weighted curve fitting method was applied to a set of learning

SUBMITTER: Figueroa RL 

PROVIDER: S-EPMC3307431 | biostudies-literature | 2012 Feb

REPOSITORIES: biostudies-literature