Fine-Tuning Bidirectional Encoder Representations From Transformers (BERT)-Based Models on Large-Scale Electronic Health Record Notes: An Empirical Study.

ABSTRACT:

Background

The bidirectional encoder representations from transformers (BERT) model has achieved great success in many natural language processing (NLP) tasks, such as named entity recognition and question answering. However, little prior work has explored this model to be used for an important task in the biomedical and clinical domains, namely entity normalization.

Objective

We aim to investigate the effectiveness of BERT-based models for biomedical or clinical entity normalization. In addition, our second objective is to investigate whether the domains of training data influence the performances of BERT-based models as well as the degree of influence.

Methods

Our data was comprised of 1.5 million unlabeled electronic health record (EHR) notes. We first fine-tuned B

SUBMITTER: Li F 

PROVIDER: S-EPMC6746103 | biostudies-literature | 2019 Sep

REPOSITORIES: biostudies-literature