An open-source framework for large-scale, flexible evaluation of biomedical text mining systems.

ABSTRACT:

Background

Improved evaluation methodologies have been identified as a necessary prerequisite to the improvement of text mining theory and practice. This paper presents a publicly available framework that facilitates thorough, structured, and large-scale evaluations of text mining technologies. The extensibility of this framework and its ability to uncover system-wide characteristics by analyzing component parts as well as its usefulness for facilitating third-party application integration are demonstrated through examples in the biomedical domain.

Results

Our evaluation framework was assembled using the Unstructured Information Management Architecture. It was used to analyze a set of gene mention identification systems involving 225 combinations of system, evaluation corpus

SUBMITTER: Baumgartner WA 

PROVIDER: S-EPMC2276192 | biostudies-literature | 2008 Jan

REPOSITORIES: biostudies-literature