Dataset Information

Learning to predict in-hospital mortality risk in the intensive care unit with attention-based temporal convolution network.

ABSTRACT:

Background

Dynamic prediction of patient mortality risk in the ICU with time series data is limited due to high dimensionality, uncertainty in sampling intervals, and other issues. A new deep learning method, temporal convolution network (TCN), makes it possible to deal with complex clinical time series data in ICU. We aimed to develop and validate it to predict mortality risk using time series data from MIMIC III dataset.

Methods

A total of 21,139 records of ICU stays were analysed and 17 physiological variables from the MIMIC III dataset were used to predict mortality risk. Then we compared the model performance of the attention-based TCN with that of traditional artificial intelligence (AI) methods.

Results

The area under receiver operating characteristic (AUCROC) and area under precision-recall curve (AUC-PR) of attention-based TCN for predicting the mortality risk 48 h after ICU admission were 0.837 (0.824 -0.850) and 0.454, respectively. The sensitivity and specificity of attention-based TCN were 67.1% and 82.6%, respectively, compared to the traditional AI method, which had a low sensitivity (< 50%).

Conclusions

The attention-based TCN model achieved better performance in the prediction of mortality risk with time series data than traditional AI methods and conventional score-based models. The attention-based TCN mortality risk model has the potential for helping decision-making for critical patients.

Trial registration

Data used for the prediction of mortality risk were extracted from the freely accessible MIMIC III dataset. The project was approved by the Institutional Review Boards of Beth Israel Deaconess Medical Center (Boston, MA) and the Massachusetts Institute of Technology (Cambridge, MA). Requirement for individual patient consent was waived because the project did not impact clinical care and all protected health information was deidentified. The data were accessed via a data use agreement between PhysioNet, a National Institutes of Health-supported data repository (https://www.physionet.org/), and one of us (Yu-wen Chen, Certification Number: 28341490). All methods were carried out in accordance with the institutional guidelines and regulations.

SUBMITTER: Chen YW

PROVIDER: S-EPMC9034533 | biostudies-literature | 2022 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Learning to predict in-hospital mortality risk in the intensive care unit with attention-based temporal convolution network.

Chen Yu-Wen YW Li Yu-Jie YJ Deng Peng P Yang Zhi-Yong ZY Zhong Kun-Hua KH Zhang Li-Ge LG Chen Yang Y Zhi Hong-Yu HY Hu Xiao-Yan XY Gu Jian-Teng JT Ning Jiao-Lin JL Lu Kai-Zhi KZ Zhang Ju J Xia Zheng-Yuan ZY Qin Xiao-Lin XL Yi Bin B

BMC anesthesiology 20220423 1

<h4>Background</h4>Dynamic prediction of patient mortality risk in the ICU with time series data is limited due to high dimensionality, uncertainty in sampling intervals, and other issues. A new deep learning method, temporal convolution network (TCN), makes it possible to deal with complex clinical time series data in ICU. We aimed to develop and validate it to predict mortality risk using time series data from MIMIC III dataset.<h4>Methods</h4>A total of 21,139 records of ICU stays were analys ...[more]

PMID: 35461225

Similar Datasets

Project description:AimsAlthough some scores based on traditional statistical methods are available for risk stratification in patients hospitalized in cardiac intensive care units (CICUs), the interest of machine learning (ML) methods for risk stratification in this field is not well established. We aimed to build an ML model to predict in-hospital major adverse events (MAE) in patients hospitalized in CICU.Methods and resultsIn April 2021, a French national prospective multicentre study involving 39 centres included all consecutive patients admitted to CICU. The primary outcome was in-hospital MAE, including death, resuscitated cardiac arrest, or cardiogenic shock. Using 31 randomly assigned centres as an index cohort (divided into training and testing sets), several ML models were evaluated to predict in-hospital MAE. The eight remaining centres were used as an external validation cohort. Among 1499 consecutive patients included (aged 64 ± 15 years, 70% male), 67 had in-hospital MAE (4.3%). Out of 28 clinical, biological, ECG, and echocardiographic variables, seven were selected to predict MAE in the training set (n = 844). Boosted cost-sensitive C5.0 technique showed the best performance compared with other ML methods [receiver operating characteristic area under the curve (AUROC) = 0.90, precision-recall AUC = 0.57, F1 score = 0.5]. Our ML score showed a better performance than existing scores (AUROC: ML score = 0.90 vs. Thrombolysis In Myocardial Infarction (TIMI) score: 0.56, Global Registry of Acute Coronary Events (GRACE) score: 0.52, Acute Heart Failure (ACUTE-HF) score: 0.65; all P < 0.05). Machine learning score also showed excellent performance in the external cohort (AUROC = 0.88).ConclusionThis new ML score is the first to demonstrate improved performance in predicting in-hospital outcomes over existing scores in patients admitted to the intensive care unit based on seven simple and rapid clinical and echocardiographic variables.Trial registrationClinicalTrials.gov Identifier: NCT05063097.

Project description:ObjectiveRisk stratification of patients with congestive heart failure (HF) is vital in clinical practice. The aim of this study was to construct a machine learning model to predict the in-hospital all-cause mortality for intensive care unit (ICU) patients with HF.MethodseXtreme Gradient Boosting algorithm (XGBoost) was used to construct a new prediction model (XGBoost model) from the Medical Information Mart for Intensive Care IV database (MIMIC-IV) (training set). The eICU Collaborative Research Database dataset (eICU-CRD) was used for the external validation (test set). The XGBoost model performance was compared with a logistic regression model and an existing model (Get with the guideline-Heart Failure model) for mortality in the test set. Area under the receiver operating characteristic cure and Brier score were employed to evaluate the discrimination and the calibration of the three models. The SHapley Additive exPlanations (SHAP) value was applied to explain XGBoost model and calculate the importance of its features.ResultsThe total of 11,156 and 9,837 patients with congestive HF from the training set and test set, respectively, were included in the study. In-hospital all-cause mortality occurred in 13.3% (1,484/11,156) and 13.4% (1,319/9,837) of patients, respectively. In the training set, of 17 features with the highest predictive value were selected into the models with LASSO regression. Acute Physiology Score III (APS III), age and Sequential Organ Failure Assessment (SOFA) were strongest predictors in SHAP. In the external validation, the XGBoost model performance was superior to that of conventional risk predictive methods, with an area under the curve of 0.771 (95% confidence interval, 0.757-0.784) and a Brier score of 0.100. In the evaluation of clinical effectiveness, the machine learning model brought a positive net benefit in the threshold probability of 0%-90%, prompting evident competitiveness compare to the other two models. This model has been translated into an online calculator which is accessible freely to the public (https://nkuwangkai-app-for-mortality-prediction-app-a8mhkf.streamlit.app).ConclusionThis study developed a valuable machine learning risk stratification tool to accurately assess and stratify the risk of in-hospital all-cause mortality in ICU patients with congestive HF. This model was translated into a web-based calculator which access freely.

Project description:BackgroundsThe in-hospital mortality in lung cancer patients admitted to intensive care unit (ICU) is extremely high. This study intended to adopt machine learning algorithm models to predict in-hospital mortality of critically ill lung cancer for providing relative information in clinical decision-making.MethodsData were extracted from the Medical Information Mart for Intensive Care-IV (MIMIC-IV) for a training cohort and data extracted from the Medical Information Mart for eICU Collaborative Research Database (eICU-CRD) database for a validation cohort. Logistic regression, random forest, decision tree, light gradient boosting machine (LightGBM), eXtreme gradient boosting (XGBoost), and an ensemble (random forest+LightGBM+XGBoost) model were used for prediction of in-hospital mortality and important feature extraction. The AUC (area under receiver operating curve), accuracy, F1 score and recall were used to evaluate the predictive performance of each model. Shapley Additive exPlanations (SHAP) values were calculated to evaluate feature importance of each feature.ResultsOverall, there were 653 (24.8%) in-hospital mortality in the training cohort, and 523 (21.7%) in-hospital mortality in the validation cohort. Among the six machine learning models, the ensemble model achieved the best performance. The top 5 most influential features were the sequential organ failure assessment (SOFA) score, albumin, the oxford acute severity of illness score (OASIS) score, anion gap and bilirubin in random forest and XGBoost model. The SHAP summary plot was used to illustrate the positive or negative effects of the top 15 features attributed to the XGBoost model.ConclusionThe ensemble model performed best and might be applied to forecast in-hospital mortality of critically ill lung cancer patients, and the SOFA score was the most important feature in all models. These results might offer valuable and significant reference for ICU clinicians' decision-making in advance.

Project description:BackgroundIntensive care unit (ICU) patients demand continuous monitoring of several clinical and laboratory parameters that directly influence their medical progress and the staff's decision-making. Those data are vital in the assistance of these patients, being already used by several scoring systems. In this context, machine learning approaches have been used for medical predictions based on clinical data, which includes patient outcomes.AimTo develop a binary classifier for the outcome of death in ICU patients based on clinical and laboratory parameters, a set formed by 1087 instances and 50 variables from ICU patients admitted to the emergency department was obtained in the "WiDS (Women in Data Science) Datathon 2020: ICU Mortality Prediction" dataset.MethodsFor categorical variables, frequencies and risk ratios were calculated. Numerical variables were computed as means and standard deviations and Mann-Whitney U tests were performed. We then divided the data into a training (80%) and test (20%) set. The training set was used to train a predictive model based on the Random Forest algorithm and the test set was used to evaluate the predictive effectiveness of the model.ResultsA statistically significant association was identified between need for intubation, as well predominant systemic cardiovascular involvement, and hospital death. A number of the numerical variables analyzed (for instance Glasgow Coma Score punctuations, mean arterial pressure, temperature, pH, and lactate, creatinine, albumin and bilirubin values) were also significantly associated with death outcome. The proposed binary Random Forest classifier obtained on the test set (n = 218) had an accuracy of 80.28%, sensitivity of 81.82%, specificity of 79.43%, positive predictive value of 73.26%, negative predictive value of 84.85%, F1 score of 0.74, and area under the curve score of 0.85. The predictive variables of the greatest importance were the maximum and minimum lactate values, adding up to a predictive importance of 15.54%.ConclusionWe demonstrated the efficacy of a Random Forest machine learning algorithm for handling clinical and laboratory data from patients under intensive monitoring. Therefore, we endorse the emerging notion that machine learning has great potential to provide us support to critically question existing methodologies, allowing improvements that reduce mortality.

Project description:IntroductionSeptic patients requiring intensive care unit (ICU) readmission are at high risk of mortality, but research focusing on the association of ICU readmission due to sepsis and mortality is limited. The aim of this study was to develop and validate a machine learning (ML) model for predicting in-hospital mortality in septic patients readmitted to the ICU using routinely available clinical data.MethodsThe data used in this study were obtained from the Medical Information Mart for Intensive Care (MIMIC-IV, v1.0) database, between 2008 and 2019. The study cohort included patients with sepsis requiring ICU readmission. The data were randomly split into a training (75%) data set and a validation (25%) data set. Nine popular ML models were developed to predict mortality in septic patients readmitted to the ICU. The model with the best accuracy and area under the curve (A.C.) in the validation cohort was defined as the optimal model and was selected for further prediction studies. The SHAPELY Additive explanations (SHAP) values and Local Interpretable Model-Agnostic Explanation (LIME) methods were used to improve the interpretability of the optimal model.ResultsA total of 1117 septic patients who had required ICU readmission during the study period were enrolled in the study. Of these participants, 434 (38.9%) were female, and the median (interquartile range [IQR]) age was 68.6 (58.4-79.2) years. The median (IQR) ICU interval duration was 2.60 (0.64-5.78) days. After feature selection, 31 of 47 clinical factors were ultimately chosen for use in model construction. Of the nine ML models tested, the best performance was achieved with the random forest (RF) model, with an A.C. of 0.81, an accuracy of 85% and a precision of 62% in the validation cohort. The SHAP summary analysis revealed that Glasgow Coma Scale score, urine output, blood urea nitrogen, lactate, platelet count and systolic blood pressure were the top six most important factors contributing to the RF model. Additionally, the LIME method demonstrated how the RF model works in terms of explaining risk of death prediction in septic patients requiring ICU readmission.ConclusionThe ML models reported here showed a good prognostic prediction ability in septic patients requiring ICU readmission. Of the features selected, the parameters related to organ perfusion made the largest contribution to outcome prediction during ICU readmission in septic patients.

Project description:BackgroundAlthough the machine learning model developed on electronic health records has become a promising method for early predicting hospital mortality, few studies focus on the approaches for handling missing data in electronic health records and evaluate model robustness to data missingness. This study proposes an attention architecture that shows excellent predictive performance and is robust to data missingness.MethodsTwo public intensive care unit databases were used for model training and external validation, respectively. Three neural networks (masked attention model, attention model with imputation, attention model with missing indicator) based on the attention architecture were developed, using masked attention mechanism, multiple imputation, and missing indicator to handle missing data, respectively. Model interpretability was analyzed by attention allocations. Extreme gradient boosting, logistic regression with multiple imputation and missing indicator (logistic regression with imputation, logistic regression with missing indicator) were used as baseline models. Model discrimination and calibration were evaluated by area under the receiver operating characteristic curve, area under precision-recall curve, and calibration curve. In addition, model robustness to data missingness in both model training and validation was evaluated by three analyses.ResultsIn total, 65,623 and 150,753 intensive care unit stays were respectively included in the training set and the test set, with mortality of 10.1% and 8.5%, and overall missing rate of 10.3% and 19.7%. attention model with missing indicator had the highest area under the receiver operating characteristic curve (0.869; 95% CI: 0.865 to 0.873) in external validation; attention model with imputation had the highest area under precision-recall curve (0.497; 95% CI: 0.480-0.513). Masked attention model and attention model with imputation showed better calibration than other models. The three neural networks showed different patterns of attention allocation. In terms of robustness to data missingness, masked attention model and attention model with missing indicator are more robust to missing data in model training; while attention model with imputation is more robust to missing data in model validation.ConclusionsThe attention architecture has the potential to become an excellent model architecture for clinical prediction task with data missingness.

Dataset Information

Learning to predict in-hospital mortality risk in the intensive care unit with attention-based temporal convolution network.

Background

Methods

Results

Conclusions

Trial registration

Publications

Learning to predict in-hospital mortality risk in the intensive care unit with attention-based temporal convolution network.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets