Dataset Information

CheXED: Comparison of a Deep Learning Model to a Clinical Decision Support System for Pneumonia in the Emergency Department.

ABSTRACT:

Purpose

Patients with pneumonia often present to the emergency department (ED) and require prompt diagnosis and treatment. Clinical decision support systems for the diagnosis and management of pneumonia are commonly utilized in EDs to improve patient care. The purpose of this study is to investigate whether a deep learning model for detecting radiographic pneumonia and pleural effusions can improve functionality of a clinical decision support system (CDSS) for pneumonia management (ePNa) operating in 20 EDs.

Materials and methods

In this retrospective cohort study, a dataset of 7434 prior chest radiographic studies from 6551 ED patients was used to develop and validate a deep learning model to identify radiographic pneumonia, pleural effusions, and evidence of multilobar pneumonia. Model performance was evaluated against 3 radiologists' adjudicated interpretation and compared with performance of the natural language processing of radiology reports used by ePNa.

Results

The deep learning model achieved an area under the receiver operating characteristic curve of 0.833 (95% confidence interval [CI]: 0.795, 0.868) for detecting radiographic pneumonia, 0.939 (95% CI: 0.911, 0.962) for detecting pleural effusions and 0.847 (95% CI: 0.800, 0.890) for identifying multilobar pneumonia. On all 3 tasks, the model achieved higher agreement with the adjudicated radiologist interpretation compared with ePNa.

Conclusions

A deep learning model demonstrated higher agreement with radiologists than the ePNa CDSS in detecting radiographic pneumonia and related findings. Incorporating deep learning models into pneumonia CDSS could enhance diagnostic performance and improve pneumonia management.

SUBMITTER: Irvin JA

PROVIDER: S-EPMC8940736 | biostudies-literature | 2022 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

CheXED: Comparison of a Deep Learning Model to a Clinical Decision Support System for Pneumonia in the Emergency Department.

Irvin Jeremy A JA Pareek Anuj A Long Jin J Rajpurkar Pranav P Eng David Ken-Ming DK Khandwala Nishith N Haug Peter J PJ Jephson Al A Conner Karen E KE Gordon Benjamin H BH Rodriguez Fernando F Ng Andrew Y AY Lungren Matthew P MP Dean Nathan C NC

Journal of thoracic imaging 20210923 3

<h4>Purpose</h4>Patients with pneumonia often present to the emergency department (ED) and require prompt diagnosis and treatment. Clinical decision support systems for the diagnosis and management of pneumonia are commonly utilized in EDs to improve patient care. The purpose of this study is to investigate whether a deep learning model for detecting radiographic pneumonia and pleural effusions can improve functionality of a clinical decision support system (CDSS) for pneumonia management (ePNa) ...[more]

PMID: 34561377

Similar Datasets

Project description:ObjectiveTo demonstrate the incremental benefit of using free text data in addition to vital sign and demographic data to identify patients with suspected infection in the emergency department.MethodsThis was a retrospective, observational cohort study performed at a tertiary academic teaching hospital. All consecutive ED patient visits between 12/17/08 and 2/17/13 were included. No patients were excluded. The primary outcome measure was infection diagnosed in the emergency department defined as a patient having an infection related ED ICD-9-CM discharge diagnosis. Patients were randomly allocated to train (64%), validate (20%), and test (16%) data sets. After preprocessing the free text using bigram and negation detection, we built four models to predict infection, incrementally adding vital signs, chief complaint, and free text nursing assessment. We used two different methods to represent free text: a bag of words model and a topic model. We then used a support vector machine to build the prediction model. We calculated the area under the receiver operating characteristic curve to compare the discriminatory power of each model.ResultsA total of 230,936 patient visits were included in the study. Approximately 14% of patients had the primary outcome of diagnosed infection. The area under the ROC curve (AUC) for the vitals model, which used only vital signs and demographic data, was 0.67 for the training data set, 0.67 for the validation data set, and 0.67 (95% CI 0.65-0.69) for the test data set. The AUC for the chief complaint model which also included demographic and vital sign data was 0.84 for the training data set, 0.83 for the validation data set, and 0.83 (95% CI 0.81-0.84) for the test data set. The best performing methods made use of all of the free text. In particular, the AUC for the bag-of-words model was 0.89 for training data set, 0.86 for the validation data set, and 0.86 (95% CI 0.85-0.87) for the test data set. The AUC for the topic model was 0.86 for the training data set, 0.86 for the validation data set, and 0.85 (95% CI 0.84-0.86) for the test data set.ConclusionCompared to previous work that only used structured data such as vital signs and demographic information, utilizing free text drastically improves the discriminatory ability (increase in AUC from 0.67 to 0.86) of identifying infection.

Project description:IntroductionThe burden of mental health-related visits to emergency departments (EDs) is growing, and agitation episodes are prevalent with such visits. Best practice guidance from experts recommends early assessment of at-risk populations and pre-emptive intervention using de-escalation techniques to prevent agitation. Time pressure, fluctuating work demands, and other systems-related factors pose challenges to efficient decision-making and adoption of best practice recommendations during an unfolding behavioural crisis. As such, we propose to design, develop and evaluate a computerised clinical decision support (CDS) system, Early Detection and Treatment to Reduce Events with Agitation Tool (ED-TREAT). We aim to identify patients at risk of agitation and guide ED clinicians through appropriate risk assessment and timely interventions to prevent agitation with a goal of minimising restraint use and improving patient experience and outcomes.Methods and analysisThis study describes the formative evaluation of the health record embedded CDS tool. Under aim 1, the study will collect qualitative data to design and develop ED-TREAT using a contextual design approach and an iterative user-centred design process. Participants will include potential CDS users, that is, ED physicians, nurses, technicians, as well as patients with lived experience of restraint use for behavioural crisis management during an ED visit. We will use purposive sampling to ensure the full spectrum of perspectives until we reach thematic saturation. Next, under aim 2, the study will conduct a pilot, randomised controlled trial of ED-TREAT at two adult ED sites in a regional health system in the Northeast USA to evaluate the feasibility, fidelity and bedside acceptability of ED-TREAT. We aim to recruit a total of at least 26 eligible subjects under the pilot trial.Ethics and disseminationEthical approval by the Yale University Human Investigation Committee was obtained in 2021 (HIC# 2000030893 and 2000030906). All participants will provide informed verbal consent prior to being enrolled in the study. Results will be disseminated through publications in open-access, peer-reviewed journals, via scientific presentations or through direct email notifications.Trial registration numberNCT04959279; Pre-results.

Project description:BackgroundEmergency departments (EDs) frequently care for individuals with opioid use disorder (OUD). Buprenorphine (BUP) is an effective treatment option for patients with OUD that can safely be initiated in the ED. At present, BUP is rarely initiated as a part of routine ED care. Clinical decision support (CDS) could accelerate adoption of ED-initiated BUP into routine emergency care.ObjectiveThis study aimed to design and formatively evaluate a user-centered decision support tool for ED initiation of BUP for patients with OUD.MethodsUser-centered design with iterative prototype development was used. Initial observations and interviews identified workflows and information needs. The design team and key stakeholders reviewed prototype designs to ensure accuracy. A total of 5 prototypes were evaluated and iteratively refined based on input from 26 attending and resident physicians.ResultsEarly feedback identified concerns with the initial CDS design: an alert with several screens. The timing of the alert led to quick dismissal without using the tool. User feedback on subsequent iterations informed the development of a flexible tool to support clinicians with varied levels of experience with the intervention by providing both one-click options for direct activation of care pathways and user-activated support for critical decision points. The final design resolved challenging navigation issues through targeted placement, color, and design of the decision support modules and care pathways. In final testing, users expressed that the tool could be easily learned without training and was reasonable for use during routine emergency care.ConclusionsA user-centered design process helped designers to better understand users' needs for a Web-based clinical decision tool to support ED initiation of BUP for OUD. The process identified varying needs across user experience and familiarity with the protocol, leading to a flexible design supporting both direct care pathways and user-initiated decision support.

Project description:BackgroundCoprescribing naloxone with opioid analgesics is a Centers for Disease Control and Prevention (CDC) best practice to mitigate the risk of fatal opioid overdose, yet coprescription by emergency medicine clinicians is rare, occurring less than 5% of the time it is indicated. Clinical decision support (CDS) has been associated with increased naloxone prescribing; however, key CDS design characteristics and pragmatic outcome measures necessary to understand replicability and effectiveness have not been reported.ObjectiveThis study aimed to rigorously evaluate and quantify the impact of CDS designed to improve emergency department (ED) naloxone coprescribing. We hypothesized CDS would increase naloxone coprescribing and the number of naloxone prescriptions filled by patients discharged from EDs in a large health care system.MethodsFollowing user-centered design principles, we designed and implemented a fully automated, interruptive, electronic health record-based CDS to nudge clinicians to coprescribe naloxone with high-risk opioid prescriptions. "High-risk" opioid prescriptions were defined as any opioid analgesic prescription ≥90 total morphine milligram equivalents per day or for patients with a prior diagnosis of opioid use disorder or opioid overdose. The Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework was used to evaluate pragmatic CDS outcomes of reach, effectiveness, adoption, implementation, and maintenance. Effectiveness was the primary outcome of interest and was assessed by (1) constructing a Bayesian structural time-series model of the number of ED visits with naloxone coprescriptions before and after CDS implementation and (2) calculating the percentage of naloxone prescriptions associated with CDS that were filled at an outpatient pharmacy. Mann-Kendall tests were used to evaluate longitudinal trends in CDS adoption. All outcomes were analyzed in R (version 4.2.2; R Core Team).UnlabelledBetween November 2019 and July 2023, there were 1,994,994 ED visits. CDS reached clinicians in 0.83% (16,566/1,994,994) of all visits and 15.99% (16,566/103,606) of ED visits where an opioid was prescribed at discharge. Clinicians adopted CDS, coprescribing naloxone in 34.36% (6613/19,246) of alerts. CDS was effective, increasing naloxone coprescribing from baseline by 18.1 (95% CI 17.9-18.3) coprescriptions per week or 2,327% (95% CI 3390-3490). Patients filled 43.80% (1989/4541) of naloxone coprescriptions. The CDS was implemented simultaneously at every ED and no adaptations were made to CDS postimplementation. CDS was maintained beyond the study period and maintained its effect, with adoption increasing over time (τ=0.454; P<.001).ConclusionsOur findings advance the evidence that electronic health record-based CDS increases the number of naloxone coprescriptions and improves the distribution of naloxone. Our time series analysis controls for secular trends and strongly suggests that minimally interruptive CDS significantly improves process outcomes.

Dataset Information

CheXED: Comparison of a Deep Learning Model to a Clinical Decision Support System for Pneumonia in the Emergency Department.

Purpose

Materials and methods

Results

Conclusions

Publications

CheXED: Comparison of a Deep Learning Model to a Clinical Decision Support System for Pneumonia in the Emergency Department.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets