A comparison of general and disease-specific machine learning models for the prediction of unplanned hospital readmissions.
Ontology highlight
ABSTRACT:
SUBMITTER: Sutter T
PROVIDER: S-EPMC7973448 | biostudies-literature | 2021 Mar
REPOSITORIES: biostudies-literature
Publications
A comparison of general and disease-specific machine learning models for the prediction of unplanned hospital readmissions.
Sutter Thomas T Roth Jan A JA Chin-Cheong Kieran K Hug Balthasar L BL Vogt Julia E JE
Journal of the American Medical Informatics Association : JAMIA 20210301 4
Unplanned hospital readmissions are a burden to patients and increase healthcare costs. A wide variety of machine learning (ML) models have been suggested to predict unplanned hospital readmissions. These ML models were often specifically trained on patient populations with certain diseases. However, it is unclear whether these specialized ML models-trained on patient subpopulations with certain diseases or defined by other clinical characteristics-are more accurate than a general ML model train ...[more]
Similar Datasets
Project description:ImportanceHospital readmissions are associated with patient harm and expense. Ways to prevent hospital readmissions have focused on identifying patients at greatest risk using prediction scores.ObjectiveTo identify the type of score that best predicts hospital readmissions.Design, setting, and participantsThis prognostic study included 14 062 consecutive adult hospital patients with 16 649 discharges from a tertiary care center, suburban community hospital, and urban critical access hospital in Maryland from September 1, 2016, through December 31, 2016. Patients not included as eligible discharges by the Centers for Medicare & Medicaid Services or the Chesapeake Regional Information System for Our Patients were excluded. A machine learning rank score, the Baltimore score (B score) developed using a machine learning technique, for each individual hospital using data from the 2 years before September 1, 2016, was compared with standard readmission risk assessment scores to predict 30-day unplanned readmissions.Main outcomes and measuresThe 30-day readmission rate evaluated using various readmission scores: B score, HOSPITAL score, modified LACE score, and Maxim/RightCare score.ResultsOf the 10 732 patients (5605 [52.2%] male; mean [SD] age, 54.56 [22.42] years) deemed to be eligible for the study, 1422 were readmitted. The area under the receiver operating characteristic curve (AUROC) for individual rules was 0.63 (95% CI, 0.61-0.65) for the HOSPITAL score, which was significantly lower than the 0.66 for modified LACE score (95% CI, 0.64-0.68; P < .001). The B score machine learning score was significantly better than all other scores; 48 hours after admission, the AUROC of the B score was 0.72 (95% CI, 0.70-0.73), which increased to 0.78 (95% CI, 0.77-0.79) at discharge (all P < .001). At the hospital using Maxim/RightCare score, the AUROC was 0.63 (95% CI, 0.59-0.69) for HOSPITAL, 0.64 (95% CI, 0.61-0.68) for Maxim/RightCare, and 0.66 (95% CI, 0.62-0.69) for modified LACE score. The B score was 0.72 (95% CI, 0.69-0.75) 48 hours after admission and 0.81 (95% CI, 0.79-0.84) at discharge. In directly comparing the B score with the sensitivity at cutoff values for modified LACE, HOSPITAL, and Maxim/RightCare scores, the B score was able to identify the same number of readmitted patients while flagging 25.5% to 54.9% fewer patients.Conclusions and relevanceAmong 3 hospitals in different settings, an automated machine learning score better predicted readmissions than commonly used readmission scores. More efficiently targeting patients at higher risk of readmission may be the first step toward potentially preventing readmissions.
| S-EPMC6484642 | biostudies-literature
Project description:Unplanned readmission to the intensive care unit (ICU) confers excess morbidity and mortality. We explore whether machine learning models can outperform the current standard, the Stability and Workload Index for Transfer (SWIFT) score, in assessing 7-day ICU readmission risk at discharge. Logistic regression, random forest, support vector machine, and gradient boosting models were trained and validated on Stanford Hospital data (2009-2019), externally validated on Beth Israel Deaconess Medical Center (BIDMC) data (2008-2019) and benchmarked against SWIFT. The best performing model was gradient boosting, with AUROC of 0.85 and 0.60 and F1-score of 0.43 and 0.14 on internal and external validation, respectively. SWIFT had an AUROC of 0.67 and 0.51 and F1-score of 0.33 and 0.10 on Stanford and BIDMC data, respectively. Machine learning models predicting 7-day ICU readmission risk can improve current ICU discharge risk assessment standards, but performance may be limited without local training.
| S-EPMC9285156 | biostudies-literature
Project description:ObjectiveTo assess both the feasibility and potential impact of predicting preventable hospital readmissions using causal machine learning applied to data from the implementation of a readmissions prevention intervention (the Transitions Program).Data sourcesElectronic health records maintained by Kaiser Permanente Northern California (KPNC).Study designRetrospective causal forest analysis of postdischarge outcomes among KPNC inpatients. Using data from both before and after implementation, we apply causal forests to estimate individual-level treatment effects of the Transitions Program intervention on 30-day readmission. These estimates are used to characterize treatment effect heterogeneity and to assess the notional impacts of alternative targeting strategies in terms of the number of readmissions prevented.Data collection1 539 285 index hospitalizations meeting the inclusion criteria and occurring between June 2010 and December 2018 at 21 KPNC hospitals.Principal findingsThere appears to be substantial heterogeneity in patients' responses to the intervention (omnibus test for heterogeneity p = 2.23 × 10-7 ), particularly across levels of predicted risk. Notably, predicted treatment effects become more positive as predicted risk increases; patients at somewhat lower risk appear to have the largest predicted effects. Moreover, these estimates appear to be well calibrated, yielding the same estimate of annual readmissions prevented in the actual treatment subgroup (1246, 95% confidence interval [CI] 1110-1381) as did a formal evaluation of the Transitions Program (1210, 95% CI 990-1430). Estimates of the impacts of alternative targeting strategies suggest that as many as 4458 (95% CI 3925-4990) readmissions could be prevented annually, while decreasing the number needed to treat from 33 to 23, by targeting patients with the largest predicted effects rather than those at highest risk.ConclusionsCausal machine learning can be used to identify preventable hospital readmissions, if the requisite interventional data are available. Moreover, our results suggest a mismatch between risk and treatment effects.
| S-EPMC7704477 | biostudies-literature
Project description:Predicting unplanned rehospitalizations has traditionally employed logistic regression models. Machine learning (ML) methods have been introduced in health service research and may improve the prediction of health outcomes. The objective of this work was to develop a ML model to predict 30-day all-cause rehospitalizations based on the French hospital medico-administrative database.This was a retrospective cohort study of all discharges in the year 2015 from acute-care inpatient hospitalizations in a tertiary-care university center comprising 4 French hospitals. The study endpoint was unplanned 30-day all-cause rehospitalization. Logistic regression (LR), classification and regression trees (CART), random forest (RF), gradient boosting (GB), and neural networks (NN) were applied to the collected data. The predictive performance of the models was evaluated using the H-measure and the area under the ROC curve (AUC).Our analysis included 118,650 hospitalizations, of which 4127 (3.5%) led to rehospitalizations via emergency departments. The RF model was the most performant model according to the H-measure (0.29) and the AUC (0.79). The performances of the RF, GB and NN models (H-measures ranged from 0.18 to 0. 29, AUC ranged from 0.74 to 0.79) were better than those of the LR model (H-measure = 0.18, AUC = 0.74); all P values <.001. In contrast, LR was superior to CART (H-measure = 0.16, AUC = 0.70), P < .0001.The use of ML may be an alternative to regression models to predict health outcomes. The integration of ML, particularly the RF algorithm, in the prediction of unplanned rehospitalization may help health service providers target patients at high risk of rehospitalizations and propose effective interventions at the hospital level.
| S-EPMC7717815 | biostudies-literature
Project description:BackgroundAlthough multiple prediction models have been developed to predict hospital admission to emergency departments (EDs) to address overcrowding and patient safety, only a few studies have examined prediction models for prehospital use. Development of institution-specific prediction models is feasible in this age of data science, provided that predictor-related information is readily collectable.ObjectiveWe aimed to develop a hospital admission prediction model based on patient information that is commonly available during ambulance transport before hospitalization.MethodsPatients transported by ambulance to our ED from April 2018 through March 2019 were enrolled. Candidate predictors were age, sex, chief complaint, vital signs, and patient medical history, all of which were recorded by emergency medical teams during ambulance transport. Patients were divided into two cohorts for derivation (3601/5145, 70.0%) and validation (1544/5145, 30.0%). For statistical models, logistic regression, logistic lasso, random forest, and gradient boosting machine were used. Prediction models were developed in the derivation cohort. Model performance was assessed by area under the receiver operating characteristic curve (AUROC) and association measures in the validation cohort.ResultsOf 5145 patients transported by ambulance, including deaths in the ED and hospital transfers, 2699 (52.5%) required hospital admission. Prediction performance was higher with the addition of predictive factors, attaining the best performance with an AUROC of 0.818 (95% CI 0.792-0.839) with a machine learning model and predictive factors of age, sex, chief complaint, and vital signs. Sensitivity and specificity of this model were 0.744 (95% CI 0.716-0.773) and 0.745 (95% CI 0.709-0.776), respectively.ConclusionsFor patients transferred to EDs, we developed a well-performing hospital admission prediction model based on routinely collected prehospital information including chief complaints.
| S-EPMC7655472 | biostudies-literature
Project description:The objective of this study is twofold. First, we seek to understand the characteristics of the multimorbid population that needs hospital care by using all diagnoses information (ICD-10 codes) and two aggregated multimorbidity and frailty scores. Second, we use machine learning prediction models on these multimorbid patients characteristics to predict rehospitalization within 30 and 365 days and their length of stay. This study was conducted on 8 882 anonymized patients hospitalized at the University Hospital of Saint-Étienne. A descriptive statistical analysis was performed to better understand the characteristics of the patient population. Multimorbidity was measured using raw diagnoses information and two specific scores based on clusters of diagnoses: the Hospital Frailty Risk Score and the Calderon-Larrañaga index. Based on these variables different machine learning models (Decision Tree, Random forest and k-nearest Neighbors) were used to predict near future rehospitalization and length of stay (LoS). The use of random forest algorithms yielded better performance to predict both 365 and 30 days rehospitalization and using the diagnoses ICD-10 codes directly was significantly more efficient. However, using the Calderon-Larrañaga's clusters of diagnoses can be used as an efficient substitute for diagnoses information for predicting readmission. The predictive power of the algorithms is quite low on length of stay indicator. Using machine learning techniques using patients' diagnoses information and Calderon-Larrañaga's score yielded efficient results to predict hospital readmission of multimorbid patients. These methods could help improve the management of care of multimorbid patients in hospitals.
| S-EPMC9779015 | biostudies-literature
Project description:ObjectiveTo compare machine learning methods for predicting inpatient seizures risk and determine the feasibility of 1-h screening EEG to identify low-risk patients (<5% seizures risk in 48 h).MethodsThe Critical Care EEG Monitoring Research Consortium (CCEMRC) multicenter database contains 7716 continuous EEGs (cEEG). Neural networks (NN), elastic net logistic regression (EN), and sparse linear integer model (RiskSLIM) were trained to predict seizures. RiskSLIM was used previously to generate 2HELPS2B model of seizure predictions. Data were divided into training (60% for model fitting) and evaluation (40% for model evaluation) cohorts. Performance was measured using area under the receiver operating curve (AUC), mean risk calibration (CAL), and negative predictive value (NPV). A secondary analysis was performed using Monte Carlo simulation (MCS) to normalize all EEG recordings to 48 h and use only the first hour of EEG as a "screening EEG" to generate predictions.ResultsRiskSLIM recreated the 2HELPS2B model. All models had comparable AUC: evaluation cohort (NN: 0.85, EN: 0.84, 2HELPS2B: 0.83) and MCS (NN: 0.82, EN; 0.82, 2HELPS2B: 0.81) and NPV (absence of seizures in the group that the models predicted to be low risk): evaluation cohort (NN: 97%, EN: 97%, 2HELPS2B: 97%) and MCS (NN: 97%, EN: 99%, 2HELPS2B: 97%). 2HELPS2B model was able to identify the largest proportion of low-risk patients.InterpretationFor seizure risk stratification of hospitalized patients, the RiskSLIM generated 2HELPS2B model compares favorably to the complex NN and EN generated models. 2HELPS2B is able to accurately and quickly identify low-risk patients with only a 1-h screening EEG.
| S-EPMC6649418 | biostudies-literature
Project description:It is expected but unknown whether machine-learning models can outperform regression models, such as a logistic regression (LR) model, especially when the number and types of predictor variables increase in electronic health records (EHRs). We aimed to compare the predictive performance of gradient-boosted decision tree (GBDT), random forest (RF), deep neural network (DNN), and LR with the least absolute shrinkage and selection operator (LR-LASSO) for unplanned readmission. We used EHRs of patients discharged alive from 38 hospitals in 2015-2017 for derivation and in 2018 for validation, including basic characteristics, diagnosis, surgery, procedure, and drug codes, and blood-test results. The outcome was 30-day unplanned readmission. We created six patterns of data tables having different numbers of binary variables (that ≥5% or ≥1% of patients or ≥10 patients had) with and without blood-test results. For each pattern of data tables, we used the derivation data to establish the machine-learning and LR models, and used the validation data to evaluate the performance of each model. The incidence of outcome was 6.8% (23,108/339,513 discharges) and 6.4% (7,507/118,074 discharges) in the derivation and validation datasets, respectively. For the first data table with the smallest number of variables (102 variables that ≥5% of patients had, without blood-test results), the c-statistic was highest for GBDT (0.740), followed by RF (0.734), LR-LASSO (0.720), and DNN (0.664). For the last data table with the largest number of variables (1543 variables that ≥10 patients had, including blood-test results), the c-statistic was highest for GBDT (0.764), followed by LR-LASSO (0.755), RF (0.751), and DNN (0.720), suggesting that the difference between GBDT and LR-LASSO was small and their 95% confidence intervals overlapped. In conclusion, GBDT generally outperformed LR-LASSO to predict unplanned readmission, but the difference of c-statistic became smaller as the number of variables was increased and blood-test results were used.
| S-EPMC11335098 | biostudies-literature
Project description:QuestionSevere asthma and COPD exacerbations requiring hospitalization are linked to increased disease morbidity and healthcare costs. We sought to identify Electronic Health Record (EHR) features of severe asthma and COPD exacerbations and evaluate the performance of four machine learning (ML) and one deep learning (DL) model in predicting readmissions using EHR data.Study design and methodsObservational study between September 30, 2012, and December 31, 2017, of patients hospitalized with asthma and COPD exacerbations.ResultsThis study included 5,794 patients, 1,893 with asthma and 3,901 with COPD. Patients with asthma were predominantly female (n = 1288 [68%]), 35% were Black (n = 669), and 25% (n = 479) were Hispanic. Black (44 vs. 33%, p = 0.01) and Hispanic patients (30 vs. 24%, p = 0.02) were more likely to be readmitted for asthma. Similarly, patients with COPD readmissions included a large percentage of Blacks (18 vs. 10%, p < 0.01) and Hispanics (8 vs. 5%, p < 0.01). To identify patients at high risk of readmission index hospitalization data of a subset of 2,682 patients, 777 with asthma and 1,905 with COPD, was analyzed with four ML models, and one DL model. We found that multilayer perceptron, the DL method, had the best sensitivity and specificity compared to the four ML methods implemented in the same dataset.InterpretationMultilayer perceptron, a deep learning method, had the best performance in predicting asthma and COPD readmissions, demonstrating that EHR and deep learning integration can improve high-risk patient detection.
| S-EPMC10720134 | biostudies-literature
Project description:BackgroundPatient safety in the intensive care unit (ICU) is one of the most critical issues, and unplanned extubation (UE) is considered the most adverse event for patient safety. Prevention and early detection of such an event is an essential but difficult component of quality care.ObjectiveThis study aimed to develop and validate prediction models for UE in ICU patients using machine learning.MethodsThis study was conducted in an academic tertiary hospital in Seoul, Republic of Korea. The hospital had approximately 2000 inpatient beds and 120 ICU beds. As of January 2019, the hospital had approximately 9000 outpatients on a daily basis. The number of annual ICU admissions was approximately 10,000. We conducted a retrospective study between January 1, 2010, and December 31, 2018. A total of 6914 extubation cases were included. We developed a UE prediction model using machine learning algorithms, which included random forest (RF), logistic regression (LR), artificial neural network (ANN), and support vector machine (SVM). For evaluating the model's performance, we used the area under the receiver operating characteristic curve (AUROC). The sensitivity, specificity, positive predictive value, negative predictive value, and F1 score were also determined for each model. For performance evaluation, we also used a calibration curve, the Brier score, and the integrated calibration index (ICI) to compare different models. The potential clinical usefulness of the best model at the best threshold was assessed through a net benefit approach using a decision curve.ResultsAmong the 6914 extubation cases, 248 underwent UE. In the UE group, there were more males than females, higher use of physical restraints, and fewer surgeries. The incidence of UE was higher during the night shift as compared to the planned extubation group. The rate of reintubation within 24 hours and hospital mortality were higher in the UE group. The UE prediction algorithm was developed, and the AUROC for RF was 0.787, for LR was 0.762, for ANN was 0.763, and for SVM was 0.740.ConclusionsWe successfully developed and validated machine learning-based prediction models to predict UE in ICU patients using electronic health record data. The best AUROC was 0.787 and the sensitivity was 0.949, which was obtained using the RF algorithm. The RF model was well-calibrated, and the Brier score and ICI were 0.129 and 0.048, respectively. The proposed prediction model uses widely available variables to limit the additional workload on the clinician. Further, this evaluation suggests that the model holds potential for clinical usefulness.
| S-EPMC8387891 | biostudies-literature