Dataset Information

Evaluation of an artificial intelligence support system for breast cancer screening in Chinese people based on mammogram.

ABSTRACT:

Background

To evaluate the diagnostic performance of radiologists on breast cancer with or without artificial intelligence (AI) support.

Methods

A retrospective study was performed. In total, 643 mammograms (average age: 54 years; female: 100%; cancer: 62.05%) were randomly allocated into two groups. Seventy-five percent of mammograms in each group were randomly selected for assessment by two independent radiologists, and the rest were read once. Half of the 71 radiologists could read mammograms with AI support, and the other half could not. Sensitivity, specificity, Youden's index, agreement rate, Kappa value, the area under the receiver operating characteristic curve (AUC) and the reading time of radiologists in each group were analyzed.

Results

The average AUC was higher if the AI support system was used (unaided: 0.84; with AI support: 0.91; p < 0.01). The average sensitivity increased from 84.77% to 95.07% with AI support (p < 0.01), but the average specificity decreased (p = 0.07). Youden's index, agreement rate and Kappa value were larger in the group with AI support, and the average reading time was shorter (p < 0.01).

Conclusions

The AI support system might contribute to enhancing the diagnostic performance (e.g., higher sensitivity and AUC) of radiologists. In the future, the AI algorithm should be improved, and prospective studies should be conducted.

SUBMITTER: Bao C

PROVIDER: S-EPMC9939225 | biostudies-literature | 2022 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluation of an artificial intelligence support system for breast cancer screening in Chinese people based on mammogram.

Bao Chengzhen C Shen Jie J Zhang Yue Y Zhang Yan Y Wei Wei W Wang Ziteng Z Ding Jia J Han Lili L

Cancer medicine 20220909 3

<h4>Background</h4>To evaluate the diagnostic performance of radiologists on breast cancer with or without artificial intelligence (AI) support.<h4>Methods</h4>A retrospective study was performed. In total, 643 mammograms (average age: 54 years; female: 100%; cancer: 62.05%) were randomly allocated into two groups. Seventy-five percent of mammograms in each group were randomly selected for assessment by two independent radiologists, and the rest were read once. Half of the 71 radiologists could ...[more]

PMID: 36082949

Similar Datasets

Project description:ImportanceContemporary approaches to artificial intelligence (AI) based on deep learning have generated interest in the application of AI to breast cancer screening (BCS). The US Food and Drug Administration (FDA) has approved several next-generation AI products indicated for BCS in recent years; however, questions regarding their accuracy, appropriate use, and clinical utility remain.ObjectivesTo describe the current FDA regulatory process for AI products, summarize the evidence used to support FDA clearance and approval of AI products indicated for BCS, consider the advantages and limitations of current regulatory approaches, and suggest ways to improve the current system.Evidence reviewPremarket notifications and other publicly available documents used for FDA clearance and approval of AI products indicated for BCS from January 1, 2017, to December 31, 2021.FindingsNine AI products indicated for BCS for identification of suggestive lesions and mammogram triage were included. Most of the products had been cleared through the 510(k) pathway, and all clearances were based on previously collected retrospective data; 6 products used multicenter designs; 7 products used enriched data; and 4 lacked details on whether products were externally validated. Test performance measures, including sensitivity, specificity, and area under the curve, were the main outcomes reported. Most of the devices used tissue biopsy as the criterion standard for BCS accuracy evaluation. Other clinical outcome measures, including cancer stage at diagnosis and interval cancer detection, were not reported for any of the devices.Conclusions and relevanceThe findings of this review suggest important gaps in reporting of data sources, data set type, validation approach, and clinical utility assessment. As AI-assisted reading becomes more widespread in BCS and other radiologic examinations, strengthened FDA evidentiary regulatory standards, development of postmarketing surveillance, a focus on clinically meaningful outcomes, and stakeholder engagement will be critical for ensuring the safety and efficacy of these products.

Project description:ImportanceMammography screening currently relies on subjective human interpretation. Artificial intelligence (AI) advances could be used to increase mammography screening accuracy by reducing missed cancers and false positives.ObjectiveTo evaluate whether AI can overcome human mammography interpretation limitations with a rigorous, unbiased evaluation of machine learning algorithms.Design, setting, and participantsIn this diagnostic accuracy study conducted between September 2016 and November 2017, an international, crowdsourced challenge was hosted to foster AI algorithm development focused on interpreting screening mammography. More than 1100 participants comprising 126 teams from 44 countries participated. Analysis began November 18, 2016.Main outcomes and measurementsAlgorithms used images alone (challenge 1) or combined images, previous examinations (if available), and clinical and demographic risk factor data (challenge 2) and output a score that translated to cancer yes/no within 12 months. Algorithm accuracy for breast cancer detection was evaluated using area under the curve and algorithm specificity compared with radiologists' specificity with radiologists' sensitivity set at 85.9% (United States) and 83.9% (Sweden). An ensemble method aggregating top-performing AI algorithms and radiologists' recall assessment was developed and evaluated.ResultsOverall, 144 231 screening mammograms from 85 580 US women (952 cancer positive ≤12 months from screening) were used for algorithm training and validation. A second independent validation cohort included 166 578 examinations from 68 008 Swedish women (780 cancer positive). The top-performing algorithm achieved an area under the curve of 0.858 (United States) and 0.903 (Sweden) and 66.2% (United States) and 81.2% (Sweden) specificity at the radiologists' sensitivity, lower than community-practice radiologists' specificity of 90.5% (United States) and 98.5% (Sweden). Combining top-performing algorithms and US radiologist assessments resulted in a higher area under the curve of 0.942 and achieved a significantly improved specificity (92.0%) at the same sensitivity.Conclusions and relevanceWhile no single AI algorithm outperformed radiologists, an ensemble of AI algorithms combined with radiologist assessment in a single-reader screening environment improved overall accuracy. This study underscores the potential of using machine learning methods for enhancing mammography screening interpretation.

Dataset Information

Evaluation of an artificial intelligence support system for breast cancer screening in Chinese people based on mammogram.

Background

Methods

Results

Conclusions

Publications

Evaluation of an artificial intelligence support system for breast cancer screening in Chinese people based on mammogram.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets