Dataset Information

Improving the diagnostic performance of inexperienced readers for thyroid nodules through digital self-learning and artificial intelligence assistance.

ABSTRACT:

Background

Data-driven digital learning could improve the diagnostic performance of novice students for thyroid nodules.

Objective

To evaluate the efficacy of digital self-learning and artificial intelligence-based computer-assisted diagnosis (AI-CAD) for inexperienced readers to diagnose thyroid nodules.

Methods

Between February and August 2023, a total of 26 readers (less than 1 year of experience in thyroid US from various departments) from 6 hospitals participated in this study. Readers completed an online learning session comprising 3,000 thyroid nodules annotated as benign or malignant independently. They were asked to assess a test set consisting of 120 thyroid nodules with known surgical pathology before and after a learning session. Then, they referred to AI-CAD and made their final decisions on the thyroid nodules. Diagnostic performances before and after self-training and with AI-CAD assistance were evaluated and compared between radiology residents and readers from different specialties.

Results

AUC (area under the receiver operating characteristic curve) improved after the self-learning session, and it improved further after radiologists referred to AI-CAD (0.679 vs 0.713 vs 0.758, p<0.05). Although the 18 radiology residents showed improved AUC (0.7 to 0.743, p=0.016) and accuracy (69.9% to 74.2%, p=0.013) after self-learning, the readers from other departments did not. With AI-CAD assistance, sensitivity (radiology 70.3% to 74.9%, others 67.9% to 82.3%, all p<0.05) and accuracy (radiology 74.2% to 77.1%, others 64.4% to 72.8%, all p <0.05) improved in all readers.

Conclusion

While AI-CAD assistance helps improve the diagnostic performance of all inexperienced readers for thyroid nodules, self-learning was only effective for radiology residents with more background knowledge of ultrasonography.

Clinical impact

Online self-learning, along with AI-CAD assistance, can effectively enhance the diagnostic performance of radiology residents in thyroid cancer.

SUBMITTER: Lee SE

PROVIDER: S-EPMC11249553 | biostudies-literature | 2024

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Improving the diagnostic performance of inexperienced readers for thyroid nodules through digital self-learning and artificial intelligence assistance.

Lee Si Eun SE Kim Hye Jung HJ Jung Hae Kyoung HK Jung Jing Hyang JH Jeon Jae-Han JH Lee Jin Hee JH Hong Hanpyo H Lee Eun Jung EJ Kim Daham D Kwak Jin Young JY

Frontiers in endocrinology 20240702

<h4>Background</h4>Data-driven digital learning could improve the diagnostic performance of novice students for thyroid nodules.<h4>Objective</h4>To evaluate the efficacy of digital self-learning and artificial intelligence-based computer-assisted diagnosis (AI-CAD) for inexperienced readers to diagnose thyroid nodules.<h4>Methods</h4>Between February and August 2023, a total of 26 readers (less than 1 year of experience in thyroid US from various departments) from 6 hospitals participated in th ...[more]

PMID: 39015174

Similar Datasets

Project description:PurposeThyroid nodules are highly prevalent in the general population, posing a clinical challenge in accurately distinguishing between benign and malignant cases. This study aimed to investigate the diagnostic performance of different strategies, utilizing a combination of a computer-aided diagnosis system (AmCAD) and shear wave elastography (SWE) imaging, to effectively differentiate benign and malignant thyroid nodules in ultrasonography.MethodsA total of 126 thyroid nodules with pathological confirmation were prospectively included in this study. The AmCAD was utilized to analyze the ultrasound imaging characteristics of the nodules, while the SWE was employed to measure their stiffness in both transverse and longitudinal thyroid scans. Twelve diagnostic patterns were formed by combining AmCAD diagnosis and SWE values, including isolation, series, parallel, and integration. The diagnostic performance was assessed using the receiver operating characteristic curve and area under the curve (AUC). Sensitivity, specificity, accuracy, missed malignancy rate, and unnecessary biopsy rate were also determined.ResultsVarious diagnostic schemes have shown specific advantages in terms of diagnostic performance. Overall, integrating AmCAD with SWE imaging in the transverse scan yielded the most favorable diagnostic performance, achieving an AUC of 72.2% (95% confidence interval (CI): 63.0-81.5%), outperforming other diagnostic schemes. Furthermore, in the subgroup analysis of nodules measuring <2 cm or 2-4 cm, the integrated scheme consistently exhibited promising diagnostic performance, with AUCs of 74.2% (95% CI: 61.9-86.4%) and 77.4% (95% CI: 59.4-95.3%) respectively, surpassing other diagnostic schemes. The integrated scheme also effectively addressed thyroid nodule management by reducing the missed malignancy rate to 9.5% and unnecessary biopsy rate to 22.2%.ConclusionThe integration of AmCAD and SWE imaging in the transverse thyroid scan significantly enhances the diagnostic performance for distinguishing benign and malignant thyroid nodules. This strategy offers clinicians the advantage of obtaining more accurate clinical diagnoses and making well-informed decisions regarding patient management.

Project description:BackgroundUltrasonography (US) is the most common method of identifying thyroid nodules, but US images require an experienced surgeon for identification. Many artificial intelligence (AI) techniques such as computer-aided diagnostic systems (CAD), deep learning (DL), and machine learning (ML) have been used to assist in the diagnosis of thyroid nodules, but whether AI techniques can improve the diagnostic accuracy of thyroid nodules still needs to be explored.ObjectiveTo clarify the accuracy of AI-based thyroid nodule US images for differentiating benign and malignant thyroid nodules.MethodsA search strategy of "subject terms + key words" was used to search PubMed, Cochrane Library, Embase, Web of Science, China Biology Medicine (CBM), and China National Knowledge Infrastructure (CNKI) for studies on AI-assisted diagnosis of thyroid nodules based on US images. The summarized receiver operating characteristic (SROC) curve and the pooled sensitivity and specificity were used to assess the performance of the diagnostic tests. The quality assessment of diagnostics accuracy studies-2 (QUADAS-2) tool was used to assess the methodological quality of the included studies. The Review Manager 5.3 and Stata 15 were used to process the data. Subgroup analysis was based on the integrity of data collection.ResultsA total of 25 studies with 17,429 US images of thyroid nodules were included. AI-assisted diagnostic techniques had better diagnostic efficacy in the diagnosis of benign and malignant thyroid nodules: sensitivity 0.88 (95% CI: (0.85-0.90)), specificity 0.81 (95% CI: 0.74-0.86), diagnostic odds ratio (DOR) 30 (95% CI: 19-46). The SROC curve indicated that the area under the curve (AUC) was 0.92 (95% CI: 0.89-0.94). Threshold effect analysis showed a Spearman correlation coefficient: 0.17 < 0.5, suggesting no threshold effect for the included studies. After a meta-regression analysis of 4 different subgroups, the results showed a statistically significant effect of mean age ≥50 years on heterogeneity. Compared with studies with an average age of ≥50 years, AI-assisted diagnostic techniques had higher diagnostic performance in studies with an average age of <50 years (0.89 (95% CI: 0.87-0.92) vs. 0.80 (95% CI: 0.73-0.88)), (0.83 (95% CI: 0.77-0.88) vs. 0.73 (95% CI: 0.60-0.87)).ConclusionsAI-assisted diagnostic techniques had good diagnostic efficacy for thyroid nodules. For the diagnosis of <50 year olds, AI-assisted diagnostic technology was more effective in diagnosis.

Project description:BackgroundA dynamic artificial intelligence (AI) ultrasonic intelligent assistant diagnosis system (dynamic AI) is a joint application of AI technology and medical imaging, which can conduct real-time synchronous dynamic analysis of nodules from multiple sectional views with different angles. This study explored the diagnostic value of dynamic AI for benign and malignant thyroid nodules in patients with Hashimoto thyroiditis (HT) and its significance in guiding surgical treatment strategies.MethodsData of 487 patients (154 with and 333 without HT) with 829 thyroid nodules who underwent surgery were collected. Differentiation of benign and malignant nodules was performed using dynamic AI, and diagnostic effects (specificity, sensitivity, negative predictive value, positive predictive value, accuracy, misdiagnosis rate and missed diagnosis rate) was assessed. Differences in diagnostic efficacy were compared among AI, preoperative ultrasound based on the American College of Radiology (ACR) Thyroid Imaging Reporting and Data System (TI-RADS), and fine needle aspiration cytology (FNAC) diagnoses.ResultsThe accuracy, specificity and sensitivity of dynamic AI reached 88.06%, 80.19%, and 90.68%, respectively; besides, there was consistency with postoperative pathological consequences (κ=0.690; P<0.001). The diagnostic efficacy of dynamic AI was equivalent between patients with and without HT, and there were no significant differences in sensitivity, specificity, accuracy, positive predictive value, negative predictive value, missed diagnosis rate, and misdiagnosis rate. In patients with HT, dynamic AI had significantly higher specificity and a lower misdiagnosis rate than did preoperative ultrasound based on the ACR TI-RADS (P<0.05). Compared with FNAC diagnosis, dynamic AI had a significantly higher sensitivity and a lower missed diagnosis rate (P<0.05).ConclusionsDynamic AI possessed an elevated diagnostic worth of malignant and benign thyroid nodules in patients with HT, which can provide a new method and valuable information for the diagnosis and development of management strategy of patients.

Project description:BackgroundA previously trained deep learning-based smartphone app provides an artificial intelligence solution to help diagnose biliary atresia from sonographic gallbladder images, but it might be impractical to launch it in real clinical settings. This study aimed to redevelop a new model using original sonographic images and their derived smartphone photos and then test the new model's performance in assisting radiologists with different experiences to detect biliary atresia in real-world mimic settings.MethodsA new model was first trained retrospectively using 3659 original sonographic gallbladder images and their derived 51,226 smartphone photos and tested on 11,410 external validation smartphone photos. Afterward, the new model was tested in 333 prospectively collected sonographic gallbladder videos from 207 infants by 14 inexperienced radiologists (9 juniors and 5 seniors) and 4 experienced pediatric radiologists in real-world mimic settings. Diagnostic performance was expressed as the area under the receiver operating characteristic curve (AUC).ResultsThe new model outperformed the previously published model in diagnosing BA on the external validation set (AUC 0.924 vs 0.908, P = 0.004) with higher consistency (kappa value 0.708 vs 0.609). When tested in real-world mimic settings using 333 sonographic gallbladder videos, the new model performed comparable to experienced pediatric radiologists (average AUC 0.860 vs 0.876) and outperformed junior radiologists (average AUC 0.838 vs 0.773) and senior radiologists (average AUC 0.829 vs 0.749). Furthermore, the new model could aid both junior and senior radiologists to improve their diagnostic performances, with the average AUC increasing from 0.773 to 0.835 for junior radiologists and from 0.749 to 0.805 for senior radiologists.ConclusionsThe interpretable app-based model showed robust and satisfactory performance in diagnosing biliary atresia, and it could aid radiologists with limited experiences to improve their diagnostic performances in real-world mimic settings.

Dataset Information

Improving the diagnostic performance of inexperienced readers for thyroid nodules through digital self-learning and artificial intelligence assistance.

Background

Objective

Methods

Results

Conclusion

Clinical impact

Publications

Improving the diagnostic performance of inexperienced readers for thyroid nodules through digital self-learning and artificial intelligence assistance.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets