Intensive care unit scoring systems outperform emergency department scoring systems for mortality prediction in critically ill patients: a prospective cohort study.
ABSTRACT: Multiple scoring systems have been developed for both the intensive care unit (ICU) and the emergency department (ED) to risk stratify patients and predict mortality. However, it remains unclear whether the additional data needed to compute ICU scores improves mortality prediction for critically ill patients compared to the simpler ED scores.We studied a prospective observational cohort of 227 critically ill patients admitted to the ICU directly from the ED at an academic, tertiary care medical center. We compared Acute Physiology and Chronic Health Evaluation (APACHE) II, APACHE III, Simplified Acute Physiology Score (SAPS) II, Modified Early Warning Score (MEWS), Rapid Emergency Medicine Score (REMS), Prince of Wales Emergency Department Score (PEDS), and a pre-hospital critical illness prediction score developed by Seymour et al. (JAMA 2010, 304(7):747-754). The primary endpoint was 60-day mortality. We compared the receiver operating characteristic (ROC) curves of the different scores and their calibration using the Hosmer-Lemeshow goodness-of-fit test and visual assessment.The ICU scores outperformed the ED scores with higher area under the curve (AUC) values (p?=?0.01). There were no differences in discrimination among the ED-based scoring systems (AUC 0.698 to 0.742; p?=?0.45) or among the ICU-based scoring systems (AUC 0.779 to 0.799; p?=?0.60). With the exception of the Seymour score, the ED-based scoring systems did not discriminate as well as the best-performing ICU-based scoring system, APACHE III (p?=?0.005 to 0.01 for comparison of ED scores to APACHE III). The Seymour score had a superior AUC to other ED scores and, despite a lower AUC than all the ICU scores, was not significantly different than APACHE III (p?=?0.09). When data from the first 24 h in the ICU was used to calculate the ED scores, the AUC for the ED scores improved numerically, but this improvement was not statistically significant. All scores had acceptable calibration.In contrast to prior studies of patients based in the emergency department, ICU scores outperformed ED scores in critically ill patients admitted from the emergency department. This difference in performance seemed to be primarily due to the complexity of the scores rather than the time window from which the data was derived.
Project description:BACKGROUND:Specific prognostic models for intracerebral hemorrhage (ICH) have short and simple features, whereas intensive care unit (ICU) severity scales include more complicated parameters. Even though newly developed ICU severity scales have disease-specific properties, they still lack radiologic parameters, which is crucial for ICH. AIMS:To compare the performance of the Simplified Acute Physiology Score (SAPS) III, Acute Physiology and Chronic Health Evaluation (APACHE) IV, Logistic Organ Dysfunction Score (LODS), ICH, max-ICH, ICH functional outcome score (ICH-FOS), and Essen-ICH for prediction of in-hospital and one-year mortality of patients with ICH. METHODS:A single-center analysis of 137 patients with ICH was conducted over 5 years. The performance of scoring systems was evaluated with receiver operating characteristic analysis. The independent predictors of one-year mortality were investigated with a multivariate logistic regression analysis. The SAPS-III score was calculated both in the emergency department (ED) and ICU. RESULTS:Among the independent variables, the need for mechanical ventilation, hematoma volume, the presence of intraventricular hemorrhage, and hematoma originating from both lobar and nonlobar regions were found as the strongest predictor of one-year mortality. For in-hospital mortality, the discriminative power of SAPS-II, APACHE-IV, and LODS was excellent, and for SAPS-III-ICU and SAPS-III-ED, it was good. For one-year mortality, the discriminative power of SAPS-II, APACHE-IV, LODS, and SAPS-III-ICU was good, and for SAPS-III-ED, Essen-ICH, ICH, max-ICH, and ICH-FOS, it was fair. CONCLUSIONS:Although all three ICH-specific prognostic scales performed satisfactory results for predicting one-year mortality, the common intensive care severity scoring showed better performance. SAPS-III scores may be recommended for use in EDs after proper customization.
Project description:The emergency department (ED) serves as the first point of hospital contact for most septic patients. Early mortality risk stratification using a quick and accurate triage tool would have great value in guiding management. The mortality in emergency department sepsis (MEDS) score was developed to risk stratify patients presenting to the ED with suspected sepsis, and its performance in the literature has been promising. We report in this study the first utilization of the MEDS score in a Singaporean cohort.In this retrospective observational cohort study, adult patients presenting to the ED with suspected sepsis and fulfilling systemic inflammatory response syndrome (SIRS) criteria were recruited. Primary outcome was 30-day in-hospital mortality (IHM) and secondary outcome was 72-hour mortality. MEDS, acute physiology and chronic health evaluation II (APACHE II), and sequential organ failure assessment (SOFA) scores were compared for prediction of primary and secondary outcomes. Receiver operating characteristic (ROC) analysis was conducted to compare predictive performance.Of the 249 patients included in the study, 46 patients (18.5%) met 30-day IHM. MEDS score achieved an area under the ROC curve (AUC) of 0.87 (95% confidence interval [CI], 0.82-0.93), outperforming the APACHE II score (0.77, 95% CI 0.69-0.85) and SOFA score (0.78, 95% CI 0.71-0.85). On secondary analysis, MEDS score was superior to both APACHE II and SOFA scores in predicting 72-hour mortality, with AUC of 0.88 (95% CI 0.82-0.95), 0.81 (95% CI 0.72-0.89), and 0.79 (95% CI 0.71-0.87), respectively. In predicting 30-day IHM, MEDS score ≥12, APACHE II score ≥23, and SOFA score ≥5 performed at sensitivities of 76.1%, 67.4%, and 76.1%, and specificities of 83.3%, 73.9%, and 65.0%, respectively.The MEDS score performed well in its ability for mortality risk stratification in a Singaporean ED cohort.
Project description:Outcome prediction of critically ill patients is an integral part of care in an Intensive Care Unit (ICU). Acute Physiology and Chronic Health Evaluation (APACHE) scoring systems provide an objective means of mortality prediction in ICU. The aim of this study was to compare the performance of APACHE II and IV scoring system in our ICU.All patients admitted to the ICU between January and June 2014 and who met the inclusion criteria were evaluated. APACHE II and IV score were calculated during the first 24 h of ICU stay based on the worst values. All patients were followed up till discharge from the hospital or death. Statistical analysis was performed using SPSS version 19.0. Discrimination of the model for mortality was assessed using receiver operating characteristic curve and calibration was assessed using the Hosmer-Lemeshow goodness-of-fit test.Of a total 1268, 1003 patients were included in this study. The mean (±standard deviation) admission APACHE II score was 19.4 ± 8.9, and APACHE IV score was 59.1 ± 27.2. The APACHE scores were significantly higher among nonsurvivors than survivors (P < 0.001). The overall crude hospital mortality rate was 17.6%. APACHE IV had better discriminative power area under the ROC curve ([AUC] -0.82) than APACHE II (AUC-0.75). Both APACHE II and APACHE IV had poor calibration.APACHE IV showed better discrimination compared to APACHE II in our ICU population. Both APACHE II and APACHE IV had poor calibration. However, APACHE II calibrated better compared to APACHE IV.
Project description:AIMS:To develop and validate a novel score for prediction of 3-month functional outcome in neurocritically ill patients. METHODS:The development of the novel score was based on two widely used scores for general critical illnesses (Acute Physiology and Chronic Health Evaluation II, APACHE II; Simplified Acute Physiology Score II, SAPS II) and consideration of the characteristics of neurocritical illness. Data from consecutive patients admitted to neurological ICU (N-ICU) between January 2013 and June 2016 were used for the validation. The modified Rankin Scale (mRS) was used to evaluate 3-month functional outcomes. APACHE II scores, SAPS II scores, and our novel scores at 24 hours and 72 hours in N-ICU were obtained. We compared the prognostic performance of our score with APACHE II and SAPS II. RESULTS:We developed a 44-point scoring system named the INCNS score, and it includes 19 items which were categorized into five parts: inflammation (I), nutrition (N), consciousness (C), neurological function (N), and systemic function (S). We validated the INCNS score with a cohort of 941 N-ICU patients. The 72-hours INCNS score achieved an area under the receiver operating characteristic curve (AUC) of 0.828 (95% CI: 0.802-0.854), and the 24-hours INCNS score achieved an AUC of 0.788 (95% CI: 0.759-0.817). The INCNS score exhibited significantly better discriminative and prognostic performance than APACHE II and SAPS II at both 24 hours and 72 hours in N-ICU. CONCLUSION:We developed an INCNS score with superior predictive power for functional outcome of neurocritically ill patients.
Project description:BACKGROUND:There are many prognostic models and scoring systems in use to predict mortality in ICU patients. The only general ICU scoring system developed and validated for patients after cardiac surgery is the APACHE-IV model. This is, however, a labor-intensive scoring system requiring a lot of data and could therefore be prone to error. The SOFA score on the other hand is a simpler system, has been widely used in ICUs and could be a good alternative. The goal of the study was to compare the SOFA score with the APACHE-IV and other ICU prediction models. METHODS:We investigated, in a large cohort of cardiac surgery patients admitted to Dutch ICUs, how well the SOFA score from the first 24?h after admission, predict hospital and ICU mortality in comparison with other recalibrated general ICU scoring systems. Measures of discrimination, accuracy, and calibration (area under the receiver operating characteristic curve (AUC), Brier score, R2, and ?-statistic) were calculated using bootstrapping. The cohort consisted of 36,632 Patients from the Dutch National Intensive Care Evaluation (NICE) registry having had a cardiac surgery procedure for which ICU admission was necessary between January 1st, 2006 and June 31st, 2018. RESULTS:Discrimination of the SOFA-, APACHE-IV-, APACHE-II-, SAPS-II-, MPM24-II - models to predict hospital mortality was good with an AUC of respectively: 0.809, 0.851, 0.830, 0.850, 0.801. Discrimination of the SOFA-, APACHE-IV-, APACHE-II-, SAPS-II-, MPM24-II - models to predict ICU mortality was slightly better with AUCs of respectively: 0.809, 0.906, 0.892, 0.919, 0.862. Calibration of the models was generally poor. CONCLUSION:Although the SOFA score had a good discriminatory power for hospital- and ICU mortality the discriminatory power of the APACHE-IV and SAPS-II was better. The SOFA score should not be preferred as mortality prediction model above traditional prognostic ICU-models.
Project description:INTRODUCTION: The aim of this study was to evaluate the usefulness of the APACHE II (Acute Physiology and Chronic Health Evaluation II), SAPS II (Simplified Acute Physiology Score II) and SOFA (Sequential Organ Failure Assessment) scores compared to simpler models based on age and Glasgow Coma Scale (GCS) in predicting long-term outcome of patients with moderate-to-severe traumatic brain injury (TBI) treated in the intensive care unit (ICU). METHODS: A national ICU database was screened for eligible TBI patients (age over 15 years, GCS 3-13) admitted in 2003-2012. Logistic regression was used for customization of APACHE II, SAPS II and SOFA score-based models for six-month mortality prediction. These models were compared to an adjusted SOFA-based model (including age) and a reference model (age and GCS). Internal validation was performed by a randomized split-sample technique. Prognostic performance was determined by assessing discrimination, calibration and precision. RESULTS: In total, 1,625 patients were included. The overall six-month mortality was 33%. The APACHE II and SAPS II-based models showed good discrimination (area under the curve (AUC) 0.79, 95% confidence interval (CI) 0.75 to 0.82; and 0.80, 95% CI 0.77 to 0.83, respectively), calibration (P?>?0.05) and precision (Brier score 0.166 to 0.167). The SOFA-based model showed poor discrimination (AUC 0.68, 95% CI 0.64 to 0.72) and precision (Brier score 0.201) but good calibration (P?>?0.05). The AUC of the SOFA-based model was significantly improved after the insertion of age and GCS (?AUC +0.11, P?<?0.001). The performance of the reference model was comparable to the APACHE II and SAPS II in terms of discrimination (AUC 0.77; compared to APACHE II, ?AUC -0.02, P?=?0.425; compared to SAPS II, ?AUC -0.03, P?=?0.218), calibration (P?>?0.05) and precision (Brier score 0.181). CONCLUSIONS: A simple prognostic model, based only on age and GCS, displayed a fairly good prognostic performance in predicting six-month mortality of ICU-treated patients with TBI. The use of the more complex scoring systems APACHE II, SAPS II and SOFA added little to the prognostic performance.
Project description:Emergency medical services (EMS) personnel frequently use the Glasgow Coma Scale (GCS) to assess injured and critically ill patients. This study assesses the accuracy of EMS providers' GCS scoring, as well as the improvement in GCS score assessment with the use of a scoring aid.This randomized, controlled study was conducted in the emergency department (ED) of an urban academic trauma center. Emergency medical technicians or paramedics who transported a patient to the ED were randomly assigned one of 9 written scenarios, either with or without a GCS scoring aid. Scenarios were created by consensus of expert attending emergency medicine, EMS, and neurocritical care physicians, with universal consensus agreement on GCS scores. ?(2) And Student's t tests were used to compare groups.Of 180 participants, 178 completed the study. Overall, 73 of 178 participants (41%) gave a GCS score that matched the expert consensus score. GCS score was correct in 22 of 88 (25%) cases without the scoring aid. GCS was correct in 51 of 90 (57%) cases with the scoring aid. Most (69%) of the total GCS scores fell within 1 point of the expert consensus GCS score. Differences in accuracy were most pronounced in scenarios with a correct GCS score of 12 or below. Subcomponent accuracy was eye 62%, verbal 70%, and motor 51%.In this study, 60% of EMS participants provided inaccurate GCS score estimates. Use of a GCS scoring aid improved accuracy of EMS GCS score assessments.
Project description:Background: Clinical assessment of disease severity is an important part of medical practice for prediction of mortality and morbidity in Intensive Care Unit (ICU). A disease severity scoring system can be used as guidance for clinicians for objective assessment of disease outcomes and estimation of the chance of recovery. This study aimed to evaluate the hypothesis that the mortality and length of stay in emergency ICUs predicted by APACHE-IV is different to the real rates of mortality and length of stay observed in our emergency ICU in Iran. Methods: This was a retrospective cohort study conducted on the data of 839 consecutive patients admitted to the emergency ICU of Nemazi Hospital, Shiraz, Iran, during 2012-2015. The relevant variables were used to calculate APACHE-IV. Length of stay and death or discharge, Glasgow coma score, and acute physiology score were also evaluated. Moreover, the accuracy of APACHE-IV for mortality was assessed using area under the Receiver Operator Characteristic (ROC) curve. Results: Of the studied patients, 157 died and 682 were discharged (non-survivors and survivors, respectively). The length of stay in the ICU was 10.98±14.60, 10.22 ± 14.21 and 14.30±15.80 days for all patients, survivors, and non-survivors, respectively. The results showed that APACHE-IV model underestimated length of stay in our emergency ICU (p<0.001). In addition, the overall observed mortality was 17.8%, while the predicted mortality by APACHE-IV model was 21%. Therefore, there was an overestimation of predicted mortality by APACHE-IV model, with an absolute difference of 3.2% (p=0.036). Conclusion: The findings showed that APACHE-IV was a poor predictor of length of stay and mortality rate in emergency ICU. Therefore, specific models based on big sample sizes of Iranian patients are required to improve accuracy of predictions.
Project description:This study assesses the performance of National Early Warning Score (NEWS), Quick Sepsis-related Organ Failure Assessment (qSOFA), Modified Early Warning Score (MEWS), Rapid Emergency Medicine Score (REMS), and Rapid Acute Physiology Score (RAPS) in predicting emphysematous pyelonephritis (EPN) patients' need for intensive care unit (ICU) admission. A retrospective analysis was conducted at four training and research hospitals' emergency departments (EDs) on all EPN adult patients from January 2007 to August 2017. Data extracted were used to calculate raw scores for five physiologic scoring systems. Mann-Whitney U tests and ?2 tests were done for numerical and categorical variables respectively to examine differences between characteristics of ICU and non-ICU patient populations. Predictability of ICU admission was evaluated with AUROC analysis. ICU patients had lower GCS scores, SpO2, platelet counts, and estimated glomerular filtration rate; and higher bands, blood urea nitrogen, creatinine, and incidences of septic shock and nephrectomy. NEWS performed best, with 73.85% accuracy at optimal cut-off of 3. In this multicentre ED EPN series, we recommend using NEWS in early identification of critical EPN patients and advance planning for ICU admission. This would reduce delays in ICU transfer and ultimately improve patient outcomes.
Project description:INTRODUCTION: Due to the increasing burden on hospital systems, most elderly patients with non-surgical sepsis are admitted to general internal medicine departments. Disease-severity scoring systems are used for stratification of patients for utilization management, performance assessment, and clinical research. Some widely used scoring systems for septic patients are inappropriate when rating non-surgical patients in a non-intensive care unit (ICU) environment mainly because their calculations require types of data that are frequently unavailable. This study aimed to assess the fitness of four scoring systems for septic patients hospitalized in general internal medicine departments: modified early warning score (MEWS), simple clinical score (SCS), mortality in emergency department sepsis (MEDS) score, and rapid emergency medicine score (REMS). METHODS: We prospectively collected computerized data of septic patients admitted to general internal medicine departments in our community-based university hospital. We followed 28-day in-hospital mortality, overall in-hospital mortality, and 30- and 60-day mortality. Using a logistic regression procedure we calculated the area under ROC curve (AUC) for every scoring system. RESULTS: Between February 1st, 2008 and April 30th, 2009 we gathered data of 1,072 patients meeting sepsis criteria on admission to general internal medicine departments. The 28-day mortality was 19.4%. The AUC for the MEWS was 0.65-0.70, for the SCS 0.76-0.79, for the MEDS 0.73-0.75, and for the REMS, 0.74-0.79. Using Hosmer-Lemeshow statistics, a lack of fit was found for the MEDS model. All scoring systems performed better than calculations based on sepsis severity. CONCLUSIONS: The SCS and REMS are the most appropriate clinical scores to predict the mortality of patients with sepsis in general internal medicine departments.