Validation of The 4C Deterioration Model for COVID-19 in a UK Teaching Hospital During Wave 2.
Ontology highlight
ABSTRACT:
SUBMITTER: Cowan SL
PROVIDER: S-EPMC8381635 | biostudies-literature | 2021 Nov
REPOSITORIES: biostudies-literature
Publications
Validation of The 4C Deterioration Model for COVID-19 in a UK Teaching Hospital During Wave 2.
Cowan Sarah L SL Wiegand Martin M Preller Jacobus J Goudie Robert J B RJB
The American journal of the medical sciences 20210823 5
Similar Datasets
Project description:Prognostic models to predict the deterioration and mortality risk in COVID-19 patients are utterly needed to assist in informed decision making. Most of these models, however, are at high risk of bias, model overfitting, and unclear reporting. Here, we aimed to externally validate the modified (urea was omitted) 4C Deterioration Model and 4C Mortality Score in a cohort of Swiss COVID-19 patients and, second, to evaluate whether the inclusion of the neutrophil-to-lymphocyte ratio (NLR) improves the predictive performance of the models. We conducted a retrospective single-centre study with adult patients hospitalized with COVID-19. Both prediction models were updated by including the NLR. Model performance was assessed via the models' discriminatory performance (area under the curve, AUC), calibration (intercept and slope), and their performance overall (Brier score). For the validation of the 4C Deterioration Model and Mortality Score, 546 and 527 patients were included, respectively. In total, 133 (24.4%) patients met the definition of in-hospital deterioration. Discrimination of the 4C Deterioration Model was AUC = 0.78 (95% CI 0.73-0.82). A total of 55 (10.44%) patients died in hospital. Discrimination of the 4C Mortality Score was AUC = 0.85 (95% CI 0.79-0.89). There was no evidence for an incremental value of the NLR. Our data confirm the role of the modified 4C Deterioration Model and Mortality Score as reliable prediction tools for the risk of deterioration and mortality. There was no evidence that the inclusion of NLR improved model performance.
| S-EPMC9139628 | biostudies-literature
Project description:BackgroundPrognostic models to predict the risk of clinical deterioration in acute COVID-19 cases are urgently required to inform clinical management decisions.MethodsWe developed and validated a multivariable logistic regression model for in-hospital clinical deterioration (defined as any requirement of ventilatory support or critical care, or death) among consecutively hospitalised adults with highly suspected or confirmed COVID-19 who were prospectively recruited to the International Severe Acute Respiratory and Emerging Infections Consortium Coronavirus Clinical Characterisation Consortium (ISARIC4C) study across 260 hospitals in England, Scotland, and Wales. Candidate predictors that were specified a priori were considered for inclusion in the model on the basis of previous prognostic scores and emerging literature describing routinely measured biomarkers associated with COVID-19 prognosis. We used internal-external cross-validation to evaluate discrimination, calibration, and clinical utility across eight National Health Service (NHS) regions in the development cohort. We further validated the final model in held-out data from an additional NHS region (London).Findings74 944 participants (recruited between Feb 6 and Aug 26, 2020) were included, of whom 31 924 (43·2%) of 73 948 with available outcomes met the composite clinical deterioration outcome. In internal-external cross-validation in the development cohort of 66 705 participants, the selected model (comprising 11 predictors routinely measured at the point of hospital admission) showed consistent discrimination, calibration, and clinical utility across all eight NHS regions. In held-out data from London (n=8239), the model showed a similarly consistent performance (C-statistic 0·77 [95% CI 0·76 to 0·78]; calibration-in-the-large 0·00 [-0·05 to 0·05]); calibration slope 0·96 [0·91 to 1·01]), and greater net benefit than any other reproducible prognostic model.InterpretationThe 4C Deterioration model has strong potential for clinical utility and generalisability to predict clinical deterioration and inform decision making among adults hospitalised with COVID-19.FundingNational Institute for Health Research (NIHR), UK Medical Research Council, Wellcome Trust, Department for International Development, Bill & Melinda Gates Foundation, EU Platform for European Preparedness Against (Re-)emerging Epidemics, NIHR Health Protection Research Unit (HPRU) in Emerging and Zoonotic Infections at University of Liverpool, NIHR HPRU in Respiratory Infections at Imperial College London.
| S-EPMC7832571 | biostudies-literature
Project description:ObjectivesTo develop a disease stratification model for COVID-19 that updates according to changes in a patient's condition while in hospital to facilitate patient management and resource allocation.DesignIn this retrospective cohort study, we adopted a landmarking approach to dynamic prediction of all-cause in-hospital mortality over the next 48 hours. We accounted for informative predictor missingness and selected predictors using penalised regression.SettingAll data used in this study were obtained from a single UK teaching hospital.ParticipantsWe developed the model using 473 consecutive patients with COVID-19 presenting to a UK hospital between 1 March 2020 and 12 September 2020; and temporally validated using data on 1119 patients presenting between 13 September 2020 and 17 March 2021.Primary and secondary outcome measuresThe primary outcome is all-cause in-hospital mortality within 48 hours of the prediction time. We accounted for the competing risks of discharge from hospital alive and transfer to a tertiary intensive care unit for extracorporeal membrane oxygenation.ResultsOur final model includes age, Clinical Frailty Scale score, heart rate, respiratory rate, oxygen saturation/fractional inspired oxygen ratio, white cell count, presence of acidosis (pH <7.35) and interleukin-6. Internal validation achieved an area under the receiver operating characteristic (AUROC) of 0.90 (95% CI 0.87 to 0.93) and temporal validation gave an AUROC of 0.86 (95% CI 0.83 to 0.88).ConclusionsOur model incorporates both static risk factors (eg, age) and evolving clinical and laboratory data, to provide a dynamic risk prediction model that adapts to both sudden and gradual changes in an individual patient's clinical condition. On successful external validation, the model has the potential to be a powerful clinical risk assessment tool.Trial registrationThe study is registered as 'researchregistry5464' on the Research Registry (www.researchregistry.com).
| S-EPMC9445230 | biostudies-literature
Project description:In this cross-sectional study, we studied performance of the International Severe Acute Respiratory and Emerging Infections Consortium mortality and deterioration scores in a cohort of 410 hospitalized patients (51.2% fully vaccinated). area under the receiver operating characteristic curves were 0.778 and 0.764, respectively, comparable to originally published validation cohorts. Subgroup analysis showed equally good performance in vaccinated and partially or unvaccinated patients.
| S-EPMC8903389 | biostudies-literature
Project description:ObjectiveTo create and validate a simple and transferable machine learning model from electronic health record data to accurately predict clinical deterioration in patients with covid-19 across institutions, through use of a novel paradigm for model development and code sharing.DesignRetrospective cohort study.SettingOne US hospital during 2015-21 was used for model training and internal validation. External validation was conducted on patients admitted to hospital with covid-19 at 12 other US medical centers during 2020-21.Participants33 119 adults (≥18 years) admitted to hospital with respiratory distress or covid-19.Main outcome measuresAn ensemble of linear models was trained on the development cohort to predict a composite outcome of clinical deterioration within the first five days of hospital admission, defined as in-hospital mortality or any of three treatments indicating severe illness: mechanical ventilation, heated high flow nasal cannula, or intravenous vasopressors. The model was based on nine clinical and personal characteristic variables selected from 2686 variables available in the electronic health record. Internal and external validation performance was measured using the area under the receiver operating characteristic curve (AUROC) and the expected calibration error-the difference between predicted risk and actual risk. Potential bed day savings were estimated by calculating how many bed days hospitals could save per patient if low risk patients identified by the model were discharged early.Results9291 covid-19 related hospital admissions at 13 medical centers were used for model validation, of which 1510 (16.3%) were related to the primary outcome. When the model was applied to the internal validation cohort, it achieved an AUROC of 0.80 (95% confidence interval 0.77 to 0.84) and an expected calibration error of 0.01 (95% confidence interval 0.00 to 0.02). Performance was consistent when validated in the 12 external medical centers (AUROC range 0.77-0.84), across subgroups of sex, age, race, and ethnicity (AUROC range 0.78-0.84), and across quarters (AUROC range 0.73-0.83). Using the model to triage low risk patients could potentially save up to 7.8 bed days per patient resulting from early discharge.ConclusionA model to predict clinical deterioration was developed rapidly in response to the covid-19 pandemic at a single hospital, was applied externally without the sharing of data, and performed well across multiple medical centers, patient subgroups, and time periods, showing its potential as a tool for use in optimizing healthcare resources.
| S-EPMC8850910 | biostudies-literature
Project description:Risk prediction scores are important tools to support clinical decision-making for patients with coronavirus disease (COVID-19). The objective of this paper was to validate the 4C mortality score, originally developed in the United Kingdom, for a Canadian population, and to examine its performance over time. We conducted an external validation study within a registry of COVID-19 positive hospital admissions in the Kitchener-Waterloo and Hamilton regions of southern Ontario between March 4, 2020 and June 13, 2021. We examined the validity of the 4C score to prognosticate in-hospital mortality using the area under the receiver operating characteristic curve (AUC) with 95% confidence intervals calculated via bootstrapping. The study included 959 individuals, of whom 224 (23.4%) died in-hospital. Median age was 72 years and 524 individuals (55%) were male. The AUC of the 4C score was 0.77, 95% confidence interval 0.79-0.87. Overall mortality rates across the pre-defined risk groups were 0% (Low), 8.0% (Intermediate), 27.2% (High), and 54.2% (Very High). Wave 1, 2 and 3 values of the AUC were 0.81 (0.76, 0.86), 0.74 (0.69, 0.80), and 0.76 (0.69, 0.83) respectively. The 4C score is a valid tool to prognosticate mortality from COVID-19 in Canadian hospitals and can be used to prioritize care and resources for patients at greatest risk of death.
| S-EPMC8452633 | biostudies-literature
Project description:On Mar 11th, 2020, the World Health Organization (WHO) stated in its Situation Report - 51 Coronavirus disease 2019 (COVID-19) as a pandemic. In early April 2020, a teaching hospital underwent shutdown and quarantine due to an outbreak of infection in accordance with Section 6 of the Infection Protection Act (index patient and 5 infected nursing staff). The complete staff (physicians, nurses and nonmedical personnel [NMP]) underwent COVID-19 testing within two phases: (1) between Apr 3rd and 5th, 2020 [n=1170], followed by (2) between Apr 8th and 9th, 2020 [n=953] with COVID-19 silent carrier positivity rates in accordance to testing phases of (1) n=19 (1.6%) and (2) n=25 (2.6%). The cumulative infection rate for NMP (1.6%), doctors (3.8%) and nurses (9.7%) was connected to type and extent of COVID-19 patient contact. Despite COVID-19 positivity of 34.8% (46 of 132 beds), a risk-free management of hospital operation is possible to a certain extent if hygiene regulations and strict patient selection are followed. However, a COVID-19-free clinic cannot be expected due to silent carriers.
| S-EPMC7220628 | biostudies-literature
Project description:From early March through mid-May 2020, the COVID-19 pandemic overwhelmed hospitals in New York City. In anticipation of ventilator shortages and limited ICU bed capacity, hospital operations prioritized the development of prognostic tools to predict clinical deterioration. However, early experience from frontline physicians observed that some patients developed unanticipated deterioration after having relatively stable periods, attesting to the uncertainty of clinical trajectories among hospitalized patients with COVID-19. Prediction tools that incorporate clinical variables at one time-point, usually on hospital presentation, are suboptimal for patients with dynamic changes and evolving clinical trajectories. Therefore, our study team developed a machine-learning algorithm to predict clinical deterioration among hospitalized COVID-19 patients by extracting clinically meaningful features from complex longitudinal laboratory and vital sign values during the early period of hospitalization with an emphasis on informative missing-ness. To incorporate the evolution of the disease and clinical practice over the course of the pandemic, we utilized a time-dependent cross-validation strategy for model development. Finally, we validated our prediction model on an external validation cohort of COVID-19 patients served in a demographically distinct population from the training cohort. The main finding of our study is the identification of risk profiles of early, late and no clinical deterioration during the course of hospitalization. While risk prediction models that include simple predictors at ED presentation and clinical judgement are able to identify any deterioration vs. no deterioration, our methodology is able to isolate a particular risk group that remain stable initially but deteriorate at a later stage of the course of hospitalization. We demonstrate the superior predictive performance with the utilization of laboratory and vital sign data during the early period of hospitalization compared to the utilization of data at presentation alone. Our results will allow efficient hospital resource allocation and will motivate research in understanding the late deterioration risk group.
| S-EPMC8084618 | biostudies-literature
Project description:COVID-19 Genomics UK (COG-UK) consortium
| PRJEB37886 | ENA
Project description:ObjectivesTo develop a COVID-19-specific deterioration index for hospitalized patients: the COVID Hospitalized Patient Deterioration Index (COVID-HDI). This index builds on the proprietary Epic Deterioration Index, which was not developed for predicting respiratory deterioration events among patients with COVID-19.Study designA retrospective observational cohort was used to develop and validate the COVID-HDI model to predict respiratory deterioration or death among hospitalized patients with COVID-19. Deterioration events were defined as death or requiring high-flow oxygen, bilevel positive airway pressure, mechanical ventilation, or intensive-level care within 72 hours of run time. The sample included hospitalized patients with COVID-19 diagnoses or positive tests at Kaiser Permanente Southern California between May 3, 2020, and October 17, 2020.MethodsMachine learning models and 118 candidate predictors were used to generate benchmark performance. Logit regression with least absolute shrinkage and selection operator and physician input were used to finalize the model. Split-sample cross-validation was used to train and test the model.ResultsThe area under the receiver operating curve was 0.83. COVID-HDI identifies patients at low risk (negative predictive value [NPV] > 98.5%) and borderline low risk (NPV > 95%) of an event. Of all patients, 74% were identified as being at low or borderline low risk at some point during their hospitalization and could be considered for discharge with or without home monitoring. A high-risk group with a positive predictive value of 51% included 12% of patients. Model performance remained high in a recent cohort of patients.ConclusionsCOVID-HDI is a parsimonious, well-calibrated, and accurate model that may support clinical decision-making around discharge and escalation of care.
| S-EPMC10843847 | biostudies-literature