Evaluating active learning methods for annotating semantic predications.

ABSTRACT:

Objectives

This study evaluated and compared a variety of active learning strategies, including a novel strategy we proposed, as applied to the task of filtering incorrect semantic predications in SemMedDB.

Materials and methods

We evaluated 8 active learning strategies covering 3 types-uncertainty, representative, and combined-on 2 datasets of 6,000 total semantic predications from SemMedDB covering the domains of substance interactions and clinical medicine, respectively. We also designed a novel combined strategy called dynamic β that does not use hand-tuned hyperparameters. Each strategy was assessed by the Area under the Learning Curve (ALC) and the number of training examples required to achieve a target Area Under the ROC curve. We also visualized and compared

SUBMITTER: Vasilakes J 

PROVIDER: S-EPMC6367018 | biostudies-literature | 2018 Oct

REPOSITORIES: biostudies-literature