De-identification of clinical notes with pseudo-labeling using regular expression rules and pre-trained BERT.

ABSTRACT:

Background

De-identification of clinical notes is essential to utilize the rich information in unstructured text data in medical research. However, only limited work has been done in removing personal information from clinical notes in Korea.

Methods

Our study utilized a comprehensive dataset stored in the Note table of the OMOP Common Data Model at Seoul National University Bundang Hospital. This dataset includes 11,181,617 radiology and 9,282,477 notes from various other departments (non-radiology reports). From this, 0.1% of the reports (11,182) were randomly selected for training and validation purposes. We used two de-identification strategies to improve performance with limited and few annotated data. First, a rule-based approach is used to construct regular expression

SUBMITTER: An J 

PROVIDER: S-EPMC11831849 | biostudies-literature | 2025 Feb

REPOSITORIES: biostudies-literature