Dataset Information

Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic Review.

ABSTRACT:

Background

Deep learning (DL)-based models have demonstrated an ability to automatically diagnose clinically significant prostate cancer (PCa) on MRI scans and are regularly reported to approach expert performance. The aim of this work was to systematically review the literature comparing deep learning (DL) systems to radiologists in order to evaluate the comparative performance of current state-of-the-art deep learning models and radiologists.

Methods

This systematic review was conducted in accordance with the 2020 Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Studies investigating DL models for diagnosing clinically significant (cs) PCa on MRI were included. The quality and risk of bias of each study were assessed using the checklist for AI in medical imaging (CLAIM) and QUADAS-2, respectively. Patient level and lesion-based diagnostic performance were separately evaluated by comparing the sensitivity achieved by DL and radiologists at an identical specificity and the false positives per patient, respectively.

Results

The final selection consisted of eight studies with a combined 7337 patients. The median study quality with CLAIM was 74.1% (IQR: 70.6-77.6). DL achieved an identical patient-level performance to the radiologists for PI-RADS ≥ 3 (both 97.7%, SD = 2.1%). DL had a lower sensitivity for PI-RADS ≥ 4 (84.2% vs. 88.8%, p = 0.43). The sensitivity of DL for lesion localization was also between 2% and 12.5% lower than that of the radiologists.

Conclusions

DL models for the diagnosis of csPCa on MRI appear to approach the performance of experts but currently have a lower sensitivity compared to experienced radiologists. There is a need for studies with larger datasets and for validation on external data.

SUBMITTER: Roest C

PROVIDER: S-EPMC9605624 | biostudies-literature | 2022 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic Review.

Roest Christian C Fransen Stefan J SJ Kwee Thomas C TC Yakar Derya D

Life (Basel, Switzerland) 20220926 10

<h4>Background</h4>Deep learning (DL)-based models have demonstrated an ability to automatically diagnose clinically significant prostate cancer (PCa) on MRI scans and are regularly reported to approach expert performance. The aim of this work was to systematically review the literature comparing deep learning (DL) systems to radiologists in order to evaluate the comparative performance of current state-of-the-art deep learning models and radiologists.<h4>Methods</h4>This systematic review was c ...[more]

PMID: 36294928

Similar Datasets

Project description:ObjectiveAssess radiologists' contribution to variation in clinically significant prostate cancer (csPCa) detection in patients with elevated prostate-specific antigen (PSA) and multiparametric MRI (mpMRI).MethodsThis institutional review board-approved, retrospective cohort study was performed at a tertiary, academic, National Cancer Institute-designated Comprehensive Cancer Center with a multidisciplinary prostate cancer program. Men undergoing mpMRI examinations from January 1, 2015, to December 31, 2019, with elevated PSA (≥4 ng/mL) and biopsy within 6 months pre- or post-MRI or prostatectomy within 6 months post-mpMRI were included. Univariate and multivariable hierarchical logistic regression assessed impact of patient, provider, mpMRI examination, mpMRI report, and pathology factors on the diagnosis of Grade Group ≥ 2 csPCa.ResultsStudy cohort included 960 MRIs in 928 men, mean age 64.0 years (SD ± 7.4), and 59.8% (555 of 928) had csPCa. Interpreting radiologist was not significant individually (P > .999) or combined with mpMRI ordering physician and physician performing biopsy or prostatectomy (P = .41). Prostate Imaging Reporting and Data System (PI-RADS) category 2 (odds ratio [OR] 0.18, P = .04), PI-RADS category 4 (OR 2.52, P < .001), and PI-RADS category 5 (OR 4.99, P < .001) assessment compared with no focal lesion; PSA density of 0.1 to 0.15 ng/mL/cc (OR 2.46, P < .001), 0.15 to 0.2 ng/mL/cc (OR 2.77, P < .001), or ≥0.2 ng/mL/cc (OR 4.52, P < .001); private insurance (reference = Medicare, OR 0.52, P = .001), and unambiguous extraprostatic extension on mpMRI (OR 2.94, P = .01) were independently associated with csPCa. PI-RADS 3 assessment (OR 1.18, P = .56), age (OR 0.99, P = .39), and African American race (OR 0.90, P = .75) were not.DiscussionAlthough there is known in-practice variation in radiologists' interpretation of mpMRI, in our multidisciplinary prostate cancer program we found no significant radiologist-attributable variation in csPCa detection.

Project description:PurposeTo establish machine learning(ML) models for the diagnosis of clinically significant prostate cancer (csPC) using multiparameter magnetic resonance imaging (mpMRI), texture analysis (TA), dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) quantitative analysis and clinical parameters and to evaluate the stability of these models in internal and temporal validation.MethodsThe dataset of 194 men was split into training (n = 135) and internal validation (n = 59) cohorts, and a temporal dataset (n = 58) was used for evaluation. The lesions with Gleason score ≥ 7 were defined as csPC. Logistic regression (LR), stepwise regression (SR), classical decision tree (cDT), conditional inference tree (CIT), random forest (RF) and support vector machine (SVM) models were established by combining mpMRI-TA, DCE-MRI and clinical parameters and validated by internal and temporal validation using the receiver operating characteristic (ROC) curve and Delong's method.ResultsEight variables were determined as important predictors for csPC, with the first three related to texture features derived from the apparent diffusion coefficient (ADC) mapping. RF, LR and SR models yielded larger and more stable area under the ROC curve values (AUCs) than other models. In the temporal validation, the sensitivity was lower than that of the internal validation (p < 0.05). There were no significant differences in specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV) and AUC (p > 0.05).ConclusionsEach machine learning model in this study has good classification ability for csPC. Compared with internal validation, the sensitivity of each machine learning model in temporal validation was reduced, but the specificity, accuracy, PPV, NPV and AUCs remained stable at a good level. The RF, LR and SR models have better classification performance in the imaging-based diagnosis of csPC, and ADC texture-related parameters are of the highest importance.

Project description:Standardized reporting of multiparametric prostate MRI (mpMRI) is widespread and follows international standards (Pi-RADS). However, quantitative measurements from mpMRI are not widely comparable. Although T2 mapping sequences can provide repeatable quantitative image measurements and extract reliable imaging biomarkers from mpMRI, they are often time-consuming. We therefore investigated the value of quantitative measurements on a highly accelerated T2 mapping sequence, in order to establish a threshold to differentiate benign from malignant lesions. For this purpose, we evaluated a novel, highly accelerated T2 mapping research sequence that enables high-resolution image acquisition with short acquisition times in everyday clinical practice. In this retrospective single-center study, we included 54 patients with clinically indicated MRI of the prostate and biopsy-confirmed carcinoma (n = 37) or exclusion of carcinoma (n = 17). All patients had received a standard of care biopsy of the prostate, results of which were used to confirm or exclude presence of malignant lesions. We used the linear mixed-effects model-fit by REML to determine the difference between mean values of cancerous tissue and healthy tissue. We found good differentiation between malignant lesions and normal appearing tissue in the peripheral zone based on the mean T2 value. Specifically, the mean T2 value for tissue without malignant lesions was (151.7 ms [95% CI: 146.9-156.5 ms] compared to 80.9 ms for malignant lesions [95% CI: 67.9-79.1 ms]; p < 0.001). Based on this assessment, a limit of 109.2 ms is suggested. Aditionally, a significant correlation was observed between T2 values of the peripheral zone and PI-RADS scores (p = 0.0194). However, no correlation was found between the Gleason Score and the T2 relaxation time. Using REML, we found a difference of -82.7 ms in mean values between cancerous tissue and healthy tissue. We established a cut-off-value of 109.2 ms to accurately differentiate between malignant and non-malignant prostate regions. The addition of T2 mapping sequences to routine imaging could benefit automated lesion detection and facilitate contrast-free multiparametric MRI of the prostate.

Project description:BackgroundRadiomics promises to enhance the discriminative performance for clinically significant prostate cancer (csPCa), but still lacks validation in real-life scenarios. This study investigates the classification performance and robustness of machine learning radiomics models in heterogeneous MRI datasets to characterize suspicious prostate lesions for non-invasive prediction of prostate cancer (PCa) aggressiveness compared to conventional imaging biomarkers.MethodsA total of 142 patients with clinical suspicion of PCa underwent 1.5T or 3T biparametric MRI (7 scanner types, 14 institutions) and exhibited suspicious lesions [prostate Imaging Reporting and Data System (PI-RADS) score ≥3] in peripheral or transitional zones. Whole-gland and index-lesion segmentations were performed semi-automatically. A total of 1,482 quantitative morphologic, shape, texture, and intensity-based radiomics features were extracted from T2-weighted and apparent diffusion coefficient (ADC)-images and assessed using random forest and logistic regression models. Five-fold cross-validation performance in terms of area under the ROC curve was compared to mean ADC (mADC), PI-RADS and prostate-specific antigen density (PSAD). Bias mitigation techniques targeting the high-dimensional feature space and inherent class imbalance were applied and robustness of results was systematically evaluated.ResultsTrained models showed mean area under the curves (AUCs) ranging from 0.78 to 0.83 in csPCa classification. Despite using mitigation techniques, high performance variability of results could be demonstrated. Trained models achieved on average numerically higher classification performance compared to clinical parameters PI-RADS (AUC =0.78), mADC (AUC =0.71) and PSAD (AUC =0.63).ConclusionsRadiomics models' classification performance of csPCa was numerically but not significantly higher than PI-RADS scoring. Overall, clinical applicability in heterogeneous MRI datasets is limited because of high variability of results. Performance variability, robustness and reproducibility of radiomics-based measures should be addressed more transparently in future research to enable broad clinical application.

Dataset Information

Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic Review.

Background

Methods

Results

Conclusions

Publications

Comparative Performance of Deep Learning and Radiologists for the Diagnosis and Localization of Clinically Significant Prostate Cancer at MRI: A Systematic Review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets