Dataset Information

Development and validation of the COVID-19 Hospitalized Patient Deterioration Index.

ABSTRACT:

Objectives

To develop a COVID-19-specific deterioration index for hospitalized patients: the COVID Hospitalized Patient Deterioration Index (COVID-HDI). This index builds on the proprietary Epic Deterioration Index, which was not developed for predicting respiratory deterioration events among patients with COVID-19.

Study design

A retrospective observational cohort was used to develop and validate the COVID-HDI model to predict respiratory deterioration or death among hospitalized patients with COVID-19. Deterioration events were defined as death or requiring high-flow oxygen, bilevel positive airway pressure, mechanical ventilation, or intensive-level care within 72 hours of run time. The sample included hospitalized patients with COVID-19 diagnoses or positive tests at Kaiser Permanente Southern California between May 3, 2020, and October 17, 2020.

Methods

Machine learning models and 118 candidate predictors were used to generate benchmark performance. Logit regression with least absolute shrinkage and selection operator and physician input were used to finalize the model. Split-sample cross-validation was used to train and test the model.

Results

The area under the receiver operating curve was 0.83. COVID-HDI identifies patients at low risk (negative predictive value [NPV] > 98.5%) and borderline low risk (NPV > 95%) of an event. Of all patients, 74% were identified as being at low or borderline low risk at some point during their hospitalization and could be considered for discharge with or without home monitoring. A high-risk group with a positive predictive value of 51% included 12% of patients. Model performance remained high in a recent cohort of patients.

Conclusions

COVID-HDI is a parsimonious, well-calibrated, and accurate model that may support clinical decision-making around discharge and escalation of care.

SUBMITTER: Nau C

PROVIDER: S-EPMC10843847 | biostudies-literature | 2023 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development and validation of the COVID-19 Hospitalized Patient Deterioration Index.

Nau Claudia C Butler Rebecca K RK Huang Cheng-Wei CW Khang Vang Kou VK Chen Aiyu A Creekmur Beth B Broder Benjamin B Subject Christopher C Sharp Adam L AL Moreta-Sainz Luis M LM Park Joon S JS Manek Akshay J AJ Cooper Robert M RM Mendoza Sergio M SM Luo Gang G Gould Michael K MK

The American journal of managed care 20231201 12

<h4>Objectives</h4>To develop a COVID-19-specific deterioration index for hospitalized patients: the COVID Hospitalized Patient Deterioration Index (COVID-HDI). This index builds on the proprietary Epic Deterioration Index, which was not developed for predicting respiratory deterioration events among patients with COVID-19.<h4>Study design</h4>A retrospective observational cohort was used to develop and validate the COVID-HDI model to predict respiratory deterioration or death among hospitalized ...[more]

PMID: 38170527

Similar Datasets

Project description:ImportanceThe Deterioration Index (DTI), used by hospitals for predicting patient deterioration, has not been extensively validated externally, raising concerns about performance and equitable predictions.ObjectiveTo locally validate DTI performance and assess its potential for bias in predicting patient clinical deterioration.Design, setting, and participantsThis retrospective prognostic study included 13 737 patients admitted to 8 heterogenous Midwestern US hospitals varying in size and type, including academic, community, urban, and rural hospitals. Patients were 18 years or older and admitted between January 1 and May 31, 2021.ExposureDTI predictions made every 15 minutes.Main outcomes and measuresDeterioration, defined as the occurrence of any of the following while hospitalized: mechanical ventilation, intensive care unit transfer, or death. Performance of the DTI was evaluated using area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC). Bias measures were calculated across demographic subgroups.ResultsA total of 5 143 513 DTI predictions were made for 13 737 patients across 14 834 hospitalizations. Among 13 918 encounters, the mean (SD) age of patients was 60.3 (19.2) years; 7636 (54.9%) were female, 11 345 (81.5%) were White, and 12 392 (89.0%) were of other ethnicity than Hispanic or Latino. The prevalence of deterioration was 10.3% (n = 1436). The DTI produced AUROCs of 0.759 (95% CI, 0.756-0.762) at the observation level and 0.685 (95% CI, 0.671-0.700) at the encounter level. Corresponding AUPRCs were 0.039 (95% CI, 0.037-0.040) at the observation level and 0.248 (95% CI, 0.227-0.273) at the encounter level. Bias measures varied across demographic subgroups and were 14.0% worse for patients identifying as American Indian or Alaska Native and 19.0% worse for those who chose not to disclose their ethnicity.Conclusions and relevanceIn this prognostic study, the DTI had modest ability to predict patient deterioration, with varying degrees of performance at the observation and encounter levels and across different demographic groups. Disparate performance across subgroups suggests the need for more transparency in model training data and reinforces the need to locally validate externally developed prediction models.

Project description:ImportanceIdentifying which patients with COVID-19 are likely to benefit from COVID-19 convalescent plasma (CCP) treatment may have a large public health impact.ObjectiveTo develop an index for predicting the expected relative treatment benefit from CCP compared with treatment without CCP for patients hospitalized for COVID-19 using patients' baseline characteristics.Design, setting, and participantsThis prognostic study used data from the COMPILE study, ie, a meta-analysis of pooled individual patient data from 8 randomized clinical trials (RCTs) evaluating CCP vs control in adults hospitalized for COVID-19 who were not receiving mechanical ventilation at randomization. A combination of baseline characteristics, termed the treatment benefit index (TBI), was developed based on 2287 patients in COMPILE using a proportional odds model, with baseline characteristics selected via cross-validation. The TBI was externally validated on 4 external data sets: the Expanded Access Program (1896 participants), a study conducted under Emergency Use Authorization (210 participants), and 2 RCTs (with 80 and 309 participants).ExposureReceipt of CCP.Main outcomes and measuresWorld Health Organization (WHO) 11-point ordinal COVID-19 clinical status scale and 2 derivatives of it (ie, WHO score of 7-10, indicating mechanical ventilation to death, and WHO score of 10, indicating death) at day 14 and day 28 after randomization. Day 14 WHO 11-point ordinal scale was used as the primary outcome to develop the TBI.ResultsA total of 2287 patients were included in the derivation cohort, with a mean (SD) age of 60.3 (15.2) years and 815 (35.6%) women. The TBI provided a continuous gradation of benefit, and, for clinical utility, it was operationalized into groups of expected large clinical benefit (B1; 629 participants in the derivation cohort [27.5%]), moderate benefit (B2; 953 [41.7%]), and potential harm or no benefit (B3; 705 [30.8%]). Patients with preexisting conditions (diabetes, cardiovascular and pulmonary diseases), with blood type A or AB, and at an early COVID-19 stage (low baseline WHO scores) were expected to benefit most, while those without preexisting conditions and at more advanced stages of COVID-19 could potentially be harmed. In the derivation cohort, odds ratios for worse outcome, where smaller odds ratios indicate larger benefit from CCP, were 0.69 (95% credible interval [CrI], 0.48-1.06) for B1, 0.82 (95% CrI, 0.61-1.11) for B2, and 1.58 (95% CrI, 1.14-2.17) for B3. Testing on 4 external datasets supported the validation of the derived TBIs.Conclusions and relevanceThe findings of this study suggest that the CCP TBI is a simple tool that can quantify the relative benefit from CCP treatment for an individual patient hospitalized with COVID-19 that can be used to guide treatment recommendations. The TBI precision medicine approach could be especially helpful in a pandemic.

Project description:BackgroundThe emergence of new SARS-CoV-2 variants with significant immune-evasiveness, the relaxation of measures for reducing the number of infections, the waning of immune protection (particularly in high-risk population groups), and the low uptake of new vaccine boosters, forecast new waves of hospitalizations and admission to intensive care units. There is an urgent need for easily implementable and clinically effective Early Warning Scores (EWSs) that can predict the risk of complications within the next 24-48 hr. Although EWSs have been used in the evaluation of COVID-19 patients, there are several clinical limitations to their use. Moreover, no models have been tested on geographically distinct populations or population groups with varying levels of immune protection.MethodsWe developed and validated COVID-19 Early Warning Score (COEWS), an EWS that is automatically calculated solely from laboratory parameters that are widely available and affordable. We benchmarked COEWS against the widely used NEWS2. We also evaluated the predictive performance of vaccinated and unvaccinated patients.ResultsThe variables of the COEWS predictive model were selected based on their predictive coefficients and on the wide availability of these laboratory variables. The final model included complete blood count, blood glucose, and oxygen saturation features. To make COEWS more actionable in real clinical situations, we transformed the predictive coefficients of the COEWS model into individual scores for each selected feature. The global score serves as an easy-to-calculate measure indicating the risk of a patient developing the combined outcome of mechanical ventilation or death within the next 48 hr.The discrimination in the external validation cohort was 0.743 (95% confidence interval [CI]: 0.703-0.784) for the COEWS score performed with coefficients and 0.700 (95% CI: 0.654-0.745) for the COEWS performed with scores. The area under the receiver operating characteristic curve (AUROC) was similar in vaccinated and unvaccinated patients. Additionally, we observed that the AUROC of the NEWS2 was 0.677 (95% CI: 0.601-0.752) in vaccinated patients and 0.648 (95% CI: 0.608-0.689) in unvaccinated patients.ConclusionsThe COEWS score predicts death or MV within the next 48 hr based on routine and widely available laboratory measurements. The extensive external validation, its high performance, its ease of use, and its positive benchmark in comparison with the widely used NEWS2 position COEWS as a new reference tool for assisting clinical decisions and improving patient care in the upcoming pandemic waves.FundingUniversity of Vienna.

Dataset Information

Development and validation of the COVID-19 Hospitalized Patient Deterioration Index.

Objectives

Study design

Methods

Results

Conclusions

Publications

Development and validation of the COVID-19 Hospitalized Patient Deterioration Index.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets