Dataset Information

Deep learning for distinguishing normal versus abnormal chest radiographs and generalization to two unseen diseases tuberculosis and COVID-19.

ABSTRACT: Chest radiography (CXR) is the most widely-used thoracic clinical imaging modality and is crucial for guiding the management of cardiothoracic conditions. The detection of specific CXR findings has been the main focus of several artificial intelligence (AI) systems. However, the wide range of possible CXR abnormalities makes it impractical to detect every possible condition by building multiple separate systems, each of which detects one or more pre-specified conditions. In this work, we developed and evaluated an AI system to classify CXRs as normal or abnormal. For training and tuning the system, we used a de-identified dataset of 248,445 patients from a multi-city hospital network in India. To assess generalizability, we evaluated our system using 6 international datasets from India, China, and the United States. Of these datasets, 4 focused on diseases that the AI was not trained to detect: 2 datasets with tuberculosis and 2 datasets with coronavirus disease 2019. Our results suggest that the AI system trained using a large dataset containing a diverse array of CXR abnormalities generalizes to new patient populations and unseen diseases. In a simulated workflow where the AI system prioritized abnormal cases, the turnaround time for abnormal cases reduced by 7-28%. These results represent an important step towards evaluating whether AI can be safely used to flag cases in a general setting where previously unseen abnormalities exist. Lastly, to facilitate the continued development of AI models for CXR, we release our collected labels for the publicly available dataset.

SUBMITTER: Nabulsi Z

PROVIDER: S-EPMC8410908 | biostudies-literature | 2021 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Deep learning for distinguishing normal versus abnormal chest radiographs and generalization to two unseen diseases tuberculosis and COVID-19.

Nabulsi Zaid Z Sellergren Andrew A Jamshy Shahar S Lau Charles C Santos Edward E Kiraly Atilla P AP Ye Wenxing W Yang Jie J Pilgrim Rory R Kazemzadeh Sahar S Yu Jin J Kalidindi Sreenivasa Raju SR Etemadi Mozziyar M Garcia-Vicente Florencia F Melnick David D Corrado Greg S GS Peng Lily L Eswaran Krish K Tse Daniel D Beladia Neeral N Liu Yun Y Chen Po-Hsuan Cameron PC Shetty Shravya S

Scientific reports 20210901 1

Chest radiography (CXR) is the most widely-used thoracic clinical imaging modality and is crucial for guiding the management of cardiothoracic conditions. The detection of specific CXR findings has been the main focus of several artificial intelligence (AI) systems. However, the wide range of possible CXR abnormalities makes it impractical to detect every possible condition by building multiple separate systems, each of which detects one or more pre-specified conditions. In this work, we develop ...[more]

PMID: 34471144

Similar Datasets

Project description:BackgroundThere is interest in using convolutional neural networks (CNNs) to analyze medical imaging to provide computer-aided diagnosis (CAD). Recent work has suggested that image classification CNNs may not generalize to new data as well as previously believed. We assessed how well CNNs generalized across three hospital systems for a simulated pneumonia screening task.Methods and findingsA cross-sectional design with multiple model training cohorts was used to evaluate model generalizability to external sites using split-sample validation. A total of 158,323 chest radiographs were drawn from three institutions: National Institutes of Health Clinical Center (NIH; 112,120 from 30,805 patients), Mount Sinai Hospital (MSH; 42,396 from 12,904 patients), and Indiana University Network for Patient Care (IU; 3,807 from 3,683 patients). These patient populations had an age mean (SD) of 46.9 years (16.6), 63.2 years (16.5), and 49.6 years (17) with a female percentage of 43.5%, 44.8%, and 57.3%, respectively. We assessed individual models using the area under the receiver operating characteristic curve (AUC) for radiographic findings consistent with pneumonia and compared performance on different test sets with DeLong's test. The prevalence of pneumonia was high enough at MSH (34.2%) relative to NIH and IU (1.2% and 1.0%) that merely sorting by hospital system achieved an AUC of 0.861 (95% CI 0.855-0.866) on the joint MSH-NIH dataset. Models trained on data from either NIH or MSH had equivalent performance on IU (P values 0.580 and 0.273, respectively) and inferior performance on data from each other relative to an internal test set (i.e., new data from within the hospital system used for training data; P values both <0.001). The highest internal performance was achieved by combining training and test data from MSH and NIH (AUC 0.931, 95% CI 0.927-0.936), but this model demonstrated significantly lower external performance at IU (AUC 0.815, 95% CI 0.745-0.885, P = 0.001). To test the effect of pooling data from sites with disparate pneumonia prevalence, we used stratified subsampling to generate MSH-NIH cohorts that only differed in disease prevalence between training data sites. When both training data sites had the same pneumonia prevalence, the model performed consistently on external IU data (P = 0.88). When a 10-fold difference in pneumonia rate was introduced between sites, internal test performance improved compared to the balanced model (10× MSH risk P < 0.001; 10× NIH P = 0.002), but this outperformance failed to generalize to IU (MSH 10× P < 0.001; NIH 10× P = 0.027). CNNs were able to directly detect hospital system of a radiograph for 99.95% NIH (22,050/22,062) and 99.98% MSH (8,386/8,388) radiographs. The primary limitation of our approach and the available public data is that we cannot fully assess what other factors might be contributing to hospital system-specific biases.ConclusionPneumonia-screening CNNs achieved better internal than external performance in 3 out of 5 natural comparisons. When models were trained on pooled data from sites with different pneumonia prevalence, they performed better on new pooled data from these sites but not on external data. CNNs robustly identified hospital system and department within a hospital, which can have large differences in disease burden and may confound predictions.

Project description:IntroductionTuberculosis (TB) stands as a paramount global health concern, contributing significantly to worldwide mortality rates. Effective containment of TB requires deployment of cost-efficient screening method with limited resources. To enhance the precision of resource allocation in the global fight against TB, this research proposed chest X-ray radiography (CXR) based machine learning screening algorithms with optimization, benchmarking and tuning for the best TB subclassification tasks for clinical application.MethodsThis investigation delves into the development and evaluation of a robust ensemble deep learning framework, comprising 43 distinct models, tailored for the identification of active TB cases and the categorization of their clinical subtypes. The proposed framework is essentially an ensemble model with multiple feature extractors and one of three fusion strategies-voting, attention-based, or concatenation methods-in the fusion stage before a final classification. The comprised de-identified dataset contains records of 915 active TB patients alongside 1,276 healthy controls with subtype-specific information. Thus, the realizations of our framework are capable for diagnosis with subclass identification. The subclass tags include: secondary tuberculosis/tuberculous pleurisy; non-cavity/cavity; secondary tuberculosis only/secondary tuberculosis and tuberculous pleurisy; tuberculous pleurisy only/secondary tuberculosis and tuberculous pleurisy.ResultsBased on the dataset and model selection and tuning, ensemble models show their capability with self-correction capability of subclass identification with rendering robust clinical predictions. The best double-CNN-extractor model with concatenation/attention fusion strategies may potentially be the successful model for subclass tasks in real application. With visualization techniques, in-depth analysis of the ensemble model's performance across different fusion strategies are verified.DiscussionThe findings underscore the potential of such ensemble approaches in augmenting TB diagnostics with subclassification. Even with limited dataset, the self-correction within the ensemble models still guarantees the accuracies to some level for potential clinical decision-making processes in TB management. Ultimately, this study shows a direction for better TB screening in the future TB response strategy.

Project description:ObjectivesThe goal of this study was to assess whether a deep learning estimate of age from a chest radiograph image (CXR-Age) can predict longevity beyond chronological age.BackgroundChronological age is an imperfect measure of longevity. Biological age, a measure of overall health, may improve personalized care. This paper proposes a new way to estimate biological age using a convolutional neural network that takes as input a CXR image and outputs a chest x-ray age (in years) as a measure of long-term mortality risk.MethodsCXR-Age was developed using CXR from 116,035 individuals and validated in 2 held-out testing sets: 1) 75% of the CXR arm of PLCO (Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial) (N = 40,967); and 2) the CXR arm of NLST (National Lung Screening Trial) (N = 5,414). CXR-Age was compared to chronological age and a multivariable regression model of chronological age, risk factors, and radiograph findings to predict all-cause and cardiovascular mortality with a maximum 23 years and 13 years of follow-up, respectively. The primary outcome was observed mortality; results are provided for the testing datasets only.ResultsIn the PLCO testing dataset, a 5-year increase in CXR-Age carried a higher risk of all-cause mortality than a 5-year increase in chronological age (CXR-Age hazard ratio [HR]: 2.26 [95% confidence interval (CI): 2.24 to 2.29] vs. chronological age HR: 1.77 [95% CI: 1.75 to 1.78]; p < 0.001). A similar pattern was found for cardiovascular mortality (CXR-Age cause-specific HR: 2.45 per 5 years [95% CI: 2.34 to 2.56] vs. chronological age HR: 1.82 per 5 years [95% CI: 1.74 to 1.90]). Similar results were seen for both outcomes in the NLST external testing dataset. Adding CXR-Age to the multivariable model resulted in significant improvements for predicting both outcomes in both testing datasets (p < 0.001 for all comparisons).ConclusionsBased on a CXR image, CXR-Age predicted long-term all-cause and cardiovascular mortality.

Dataset Information

Deep learning for distinguishing normal versus abnormal chest radiographs and generalization to two unseen diseases tuberculosis and COVID-19.

Publications

Deep learning for distinguishing normal versus abnormal chest radiographs and generalization to two unseen diseases tuberculosis and COVID-19.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets