Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction.

ABSTRACT: Deep learning (DL)-based predictive models from electronic health records (EHRs) deliver impressive performance in many clinical tasks. Large training cohorts, however, are often required by these models to achieve high accuracy, hindering the adoption of DL-based models in scenarios with limited training data. Recently, bidirectional encoder representations from transformers (BERT) and related models have achieved tremendous successes in the natural language processing domain. The pretraining of BERT on a very large training corpus generates contextualized embeddings that can boost the performance of models trained on smaller datasets. Inspired by BERT, we propose Med-BERT, which adapts the BERT framework originally developed for the text domain to the structured EHR domain. Med-BERT is a

SUBMITTER: Rasmy L 

PROVIDER: S-EPMC8137882 | biostudies-literature | 2021 May

REPOSITORIES: biostudies-literature