Dataset Information

Self-evolving vision transformer for chest X-ray diagnosis through knowledge distillation.

ABSTRACT: Although deep learning-based computer-aided diagnosis systems have recently achieved expert-level performance, developing a robust model requires large, high-quality data with annotations that are expensive to obtain. This situation poses a conundrum that annually-collected chest x-rays cannot be utilized due to the absence of labels, especially in deprived areas. In this study, we present a framework named distillation for self-supervision and self-train learning (DISTL) inspired by the learning process of the radiologists, which can improve the performance of vision transformer simultaneously with self-supervision and self-training through knowledge distillation. In external validation from three hospitals for diagnosis of tuberculosis, pneumothorax, and COVID-19, DISTL offers gradually improved performance as the amount of unlabeled data increase, even better than the fully supervised model with the same amount of labeled data. We additionally show that the model obtained with DISTL is robust to various real-world nuisances, offering better applicability in clinical setting.

SUBMITTER: Park S

PROVIDER: S-EPMC9252561 | biostudies-literature | 2022 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Self-evolving vision transformer for chest X-ray diagnosis through knowledge distillation.

Park Sangjoon S Kim Gwanghyun G Oh Yujin Y Seo Joon Beom JB Lee Sang Min SM Kim Jin Hwan JH Moon Sungjun S Lim Jae-Kwang JK Park Chang Min CM Ye Jong Chul JC

Nature communications 20220704 1

Although deep learning-based computer-aided diagnosis systems have recently achieved expert-level performance, developing a robust model requires large, high-quality data with annotations that are expensive to obtain. This situation poses a conundrum that annually-collected chest x-rays cannot be utilized due to the absence of labels, especially in deprived areas. In this study, we present a framework named distillation for self-supervision and self-train learning (DISTL) inspired by the learnin ...[more]

PMID: 35789159

Similar Datasets

Project description:Abstract The goal of diagnosing the coronavirus disease 2019 (COVID‐19) from suspected pneumonia cases, that is, recognizing COVID‐19 from chest X‐ray or computed tomography (CT) images, is to improve diagnostic accuracy, leading to faster intervention. The most important and challenging problem here is to design an effective and robust diagnosis model. To this end, there are three challenges to overcome: (1) The lack of training samples limits the success of existing deep‐learning‐based methods. (2) Many public COVID‐19 data sets contain only a few images without fine‐grained labels. (3) Due to the explosive growth of suspected cases, it is urgent and important to diagnose not only COVID‐19 cases but also the cases of other types of pneumonia that are similar to the symptoms of COVID‐19. To address these issues, we propose a novel framework called Unsupervised Meta‐Learning with Self‐Knowledge Distillation to address the problem of differentiating COVID‐19 from pneumonia cases. During training, our model cannot use any true labels and aims to gain the ability of learning to learn by itself. In particular, we first present a deep diagnosis model based on a relation network to capture and memorize the relation among different images. Second, to enhance the performance of our model, we design a self‐knowledge distillation mechanism that distills knowledge within our model itself. Our network is divided into several parts, and the knowledge in the deeper parts is squeezed into the shallow ones. The final results are derived from our model by learning to compare the features of images. Experimental results demonstrate that our approach achieves significantly higher performance than other state‐of‐the‐art methods. Moreover, we construct a new COVID‐19 pneumonia data set based on text mining, consisting of 2696 COVID‐19 images (347 X‐ray + 2349 CT), 10,155 images (9661 X‐ray + 494 CT) about other types of pneumonia, and the fine‐grained labels of all. Our data set considers not only a bacterial infection or viral infection which causes pneumonia but also a viral infection derived from the influenza virus or coronavirus.

Dataset Information

Self-evolving vision transformer for chest X-ray diagnosis through knowledge distillation.

Publications

Self-evolving vision transformer for chest X-ray diagnosis through knowledge distillation.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets