Dataset Information

Development and validation of early warning score systems for COVID-19 patients.

ABSTRACT: COVID-19 is a major, urgent, and ongoing threat to global health. Globally more than 24 million have been infected and the disease has claimed more than a million lives as of November 2020. Predicting which patients will need respiratory support is important to guiding individual patient treatment and also to ensuring sufficient resources are available. The ability of six common Early Warning Scores (EWS) to identify respiratory deterioration defined as the need for advanced respiratory support (high-flow nasal oxygen, continuous positive airways pressure, non-invasive ventilation, intubation) within a prediction window of 24 h is evaluated. It is shown that these scores perform sub-optimally at this specific task. Therefore, an alternative EWS based on the Gradient Boosting Trees (GBT) algorithm is developed that is able to predict deterioration within the next 24 h with high AUROC 94% and an accuracy, sensitivity, and specificity of 70%, 96%, 70%, respectively. The GBT model outperformed the best EWS (LDTEWS:NEWS), increasing the AUROC by 14%. Our GBT model makes the prediction based on the current and baseline measures of routinely available vital signs and blood tests.

SUBMITTER: Youssef A

PROVIDER: S-EPMC8239612 | biostudies-literature | 2021 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development and validation of early warning score systems for COVID-19 patients.

Youssef Alexey A Kouchaki Samaneh S Shamout Farah F Armstrong Jacob J El-Bouri Rasheed R Taylor Thomas T Birrenkott Drew D Vasey Baptiste B Soltan Andrew A Zhu Tingting T Clifton David A DA Eyre David W DW

Healthcare technology letters 20210527 5

COVID-19 is a major, urgent, and ongoing threat to global health. Globally more than 24 million have been infected and the disease has claimed more than a million lives as of November 2020. Predicting which patients will need respiratory support is important to guiding individual patient treatment and also to ensuring sufficient resources are available. The ability of six common Early Warning Scores (EWS) to identify respiratory deterioration defined as the need for advanced respiratory support ...[more]

PMID: 34221413

Similar Datasets

Project description:BackgroundThe emergence of new SARS-CoV-2 variants with significant immune-evasiveness, the relaxation of measures for reducing the number of infections, the waning of immune protection (particularly in high-risk population groups), and the low uptake of new vaccine boosters, forecast new waves of hospitalizations and admission to intensive care units. There is an urgent need for easily implementable and clinically effective Early Warning Scores (EWSs) that can predict the risk of complications within the next 24-48 hr. Although EWSs have been used in the evaluation of COVID-19 patients, there are several clinical limitations to their use. Moreover, no models have been tested on geographically distinct populations or population groups with varying levels of immune protection.MethodsWe developed and validated COVID-19 Early Warning Score (COEWS), an EWS that is automatically calculated solely from laboratory parameters that are widely available and affordable. We benchmarked COEWS against the widely used NEWS2. We also evaluated the predictive performance of vaccinated and unvaccinated patients.ResultsThe variables of the COEWS predictive model were selected based on their predictive coefficients and on the wide availability of these laboratory variables. The final model included complete blood count, blood glucose, and oxygen saturation features. To make COEWS more actionable in real clinical situations, we transformed the predictive coefficients of the COEWS model into individual scores for each selected feature. The global score serves as an easy-to-calculate measure indicating the risk of a patient developing the combined outcome of mechanical ventilation or death within the next 48 hr.The discrimination in the external validation cohort was 0.743 (95% confidence interval [CI]: 0.703-0.784) for the COEWS score performed with coefficients and 0.700 (95% CI: 0.654-0.745) for the COEWS performed with scores. The area under the receiver operating characteristic curve (AUROC) was similar in vaccinated and unvaccinated patients. Additionally, we observed that the AUROC of the NEWS2 was 0.677 (95% CI: 0.601-0.752) in vaccinated patients and 0.648 (95% CI: 0.608-0.689) in unvaccinated patients.ConclusionsThe COEWS score predicts death or MV within the next 48 hr based on routine and widely available laboratory measurements. The extensive external validation, its high performance, its ease of use, and its positive benchmark in comparison with the widely used NEWS2 position COEWS as a new reference tool for assisting clinical decisions and improving patient care in the upcoming pandemic waves.FundingUniversity of Vienna.

Project description:We sought to validate prognostic scores in coronavirus disease 2019 including National Early Warning Score, Modified Early Warning Score, and age-based modifications, and define their performance characteristics.DesignWe analyzed prospectively collected data from the Adaptive COVID-19 Treatment Trial. National Early Warning Score was collected daily during the trial, Modified Early Warning Score was calculated, and age applied to both scores. We assessed prognostic value for the end points of recovery, mechanical ventilation, and death for score at enrollment, average, and slope of score over the first 48 hours.SettingA multisite international inpatient trial.PatientsA total of 1,062 adult nonpregnant inpatients with severe coronavirus disease 2019 pneumonia.InterventionsAdaptive COVID-19 Treatment Trial 1 randomized participants to receive remdesivir or placebo. The prognostic value of predictive scores was evaluated in both groups separately to assess for differential performance in the setting of remdesivir treatment.Measurements and main resultsFor mortality, baseline National Early Warning Score and Modified Early Warning Score were weakly to moderately prognostic (c-index, 0.60-0.68), and improved with addition of age (c-index, 0.66-0.74). For recovery, baseline National Early Warning Score and Modified Early Warning Score demonstrated somewhat better prognostic ability (c-index, 0.65-0.69); however, National Early Warning Score+age and Modified Early Warning Score+age further improved performance (c-index, 0.68-0.71). For deterioration, baseline National Early Warning Score and Modified Early Warning Score were weakly to moderately prognostic (c-index, 0.59-0.69) and improved with addition of age (c-index, 0.63-0.70). All prognostic performance improvements due to addition of age were significant (p < 0.05).ConclusionsIn the Adaptive COVID-19 Treatment Trial 1 cohort, National Early Warning Score and Modified Early Warning Score demonstrated moderate prognostic performance in patients with severe coronavirus disease 2019, with improvement in predictive ability for National Early Warning Score+age and Modified Early Warning Score+age. Area under receiver operating curve for National Early Warning Score and Modified Early Warning Score improved in patients receiving remdesivir versus placebo early in the pandemic for recovery and mortality. Although these scores are simple and readily obtainable in myriad settings, in our data set, they were insufficiently predictive to completely replace clinical judgment in coronavirus disease 2019 and may serve best as an adjunct to triage, disposition, and resourcing decisions.

Project description:ObjectivesIdentifying patients with a possible SARS-CoV-2 infection in the emergency department (ED) is challenging. Symptoms differ, incidence rates vary and test capacity may be limited. As PCR-testing all ED patients is neither feasible nor effective in most centres, a rapid, objective, low-cost early warning score to triage ED patients for a possible infection is developed.DesignCase-control study.SettingSecondary and tertiary hospitals in the Netherlands.ParticipantsThe study included patients presenting to the ED with venous blood sampling from July 2019 to July 2020 (n=10 417, 279 SARS-CoV-2-positive). The temporal validation cohort covered the period from July 2020 to October 2021 (n=14 080, 1093 SARS-CoV-2-positive). The external validation cohort consisted of patients presenting to the ED of three hospitals in the Netherlands (n=12 061, 652 SARS-CoV-2-positive).Primary outcome measuresThe primary outcome was one or more positive SARS-CoV-2 PCR test results within 1 day prior to or 1 week after ED presentation.ResultsThe resulting 'CoLab-score' consists of 10 routine laboratory measurements and age. The score showed good discriminative ability (AUC: 0.930, 95% CI 0.909 to 0.945). The lowest CoLab-score had high sensitivity for COVID-19 (0.984, 95% CI 0.970 to 0.991; specificity: 0.411, 95% CI 0.285 to 0.520). Conversely, the highest score had high specificity (0.978, 95% CI 0.973 to 0.983; sensitivity: 0.608, 95% CI 0.522 to 0.685). The results were confirmed in temporal and external validation.ConclusionsThe CoLab-score is based on routine laboratory measurements and is available within 1 hour after presentation. Depending on the prevalence, COVID-19 may be safely ruled out in over one-third of ED presentations. Highly suspect cases can be identified regardless of presenting symptoms. The CoLab-score is continuous, in contrast to the binary outcome of lateral flow testing, and can guide PCR testing and triage ED patients.

Project description:BackgroundEarly warning scores (EWS) have been developed as clinical prognostication tools to identify acutely deteriorating patients. In the past few years, there has been a proliferation of studies that describe the development and validation of novel machine learning-based EWS. Systematic reviews of published studies which focus on evaluating performance of both well-established and novel EWS have shown conflicting conclusions. A possible reason is the heterogeneity in validation methods applied. In this review, we aim to examine the methodologies and metrics used in studies which perform EWS validation.MethodsA systematic review of all eligible studies from the MEDLINE database and other sources, was performed. Studies were eligible if they performed validation on at least one EWS and reported associations between EWS scores and inpatient mortality, intensive care unit (ICU) transfers, or cardiac arrest (CA) of adults. Two reviewers independently did a full-text review and performed data abstraction by using standardized data-worksheet based on the TRIPOD (Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) checklist. Meta-analysis was not performed due to heterogeneity.ResultsThe key differences in validation methodologies identified were (1) validation dataset used, (2) outcomes of interest, (3) case definition, time of EWS use and aggregation methods, and (4) handling of missing values. In terms of case definition, among the 48 eligible studies, 34 used the patient episode case definition while 12 used the observation set case definition, and 2 did the validation using both case definitions. Of those that used the patient episode case definition, 18 studies validated the EWS at a single point of time, mostly using the first recorded observation. The review also found more than 10 different performance metrics reported among the studies.ConclusionsMethodologies and performance metrics used in studies performing validation on EWS were heterogeneous hence making it difficult to interpret and compare EWS performance. Standardizing EWS validation methodology and reporting can potentially address this issue.

Dataset Information

Development and validation of early warning score systems for COVID-19 patients.

Publications

Development and validation of early warning score systems for COVID-19 patients.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets