TransformEHR: transformer-based encoder-decoder generative model to enhance prediction of disease outcomes using electronic health records.

ABSTRACT: Deep learning transformer-based models using longitudinal electronic health records (EHRs) have shown a great success in prediction of clinical diseases or outcomes. Pretraining on a large dataset can help such models map the input space better and boost their performance on relevant tasks through finetuning with limited data. In this study, we present TransformEHR, a generative encoder-decoder model with transformer that is pretrained using a new pretraining objective-predicting all diseases and outcomes of a patient at a future visit from previous visits. TransformEHR's encoder-decoder framework, paired with the novel pretraining objective, helps it achieve the new state-of-the-art performance on multiple clinical prediction tasks. Comparing with the previous model, TransformEHR improves

SUBMITTER: Yang Z 

PROVIDER: S-EPMC10687211 | biostudies-literature | 2023 Nov

REPOSITORIES: biostudies-literature