Dataset Information

Comparing two artificial intelligence software packages for normative brain volumetry in memory clinic imaging.

ABSTRACT:

Purpose

To compare two artificial intelligence software packages performing normative brain volumetry and explore whether they could differently impact dementia diagnostics in a clinical context.

Methods

Sixty patients (20 Alzheimer's disease, 20 frontotemporal dementia, 20 mild cognitive impairment) and 20 controls were included retrospectively. One MRI per subject was processed by software packages from two proprietary manufacturers, producing two quantitative reports per subject. Two neuroradiologists assigned forced-choice diagnoses using only the normative volumetry data in these reports. They classified the volumetric profile as "normal," or "abnormal", and if "abnormal," they specified the most likely dementia subtype. Differences between the packages' clinical impact were assessed by comparing (1) agreement between diagnoses based on software output; (2) diagnostic accuracy, sensitivity, and specificity; and (3) diagnostic confidence. Quantitative outputs were also compared to provide context to any diagnostic differences.

Results

Diagnostic agreement between packages was moderate, for distinguishing normal and abnormal volumetry (K = .41-.43) and for specific diagnoses (K = .36-.38). However, each package yielded high inter-observer agreement when distinguishing normal and abnormal profiles (K = .73-.82). Accuracy, sensitivity, and specificity were not different between packages. Diagnostic confidence was different between packages for one rater. Whole brain intracranial volume output differed between software packages (10.73%, p < .001), and normative regional data interpreted for diagnosis correlated weakly to moderately (r_s = .12-.80).

Conclusion

Different artificial intelligence software packages for quantitative normative assessment of brain MRI can produce distinct effects at the level of clinical interpretation. Clinics should not assume that different packages are interchangeable, thus recommending internal evaluation of packages before adoption.

SUBMITTER: Zaki LAM

PROVIDER: S-EPMC9177657 | biostudies-literature | 2022 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparing two artificial intelligence software packages for normative brain volumetry in memory clinic imaging.

Zaki Lara A M LAM Vernooij Meike W MW Smits Marion M Tolman Christine C Papma Janne M JM Visser Jacob J JJ Steketee Rebecca M E RME

Neuroradiology 20220115 7

<h4>Purpose</h4>To compare two artificial intelligence software packages performing normative brain volumetry and explore whether they could differently impact dementia diagnostics in a clinical context.<h4>Methods</h4>Sixty patients (20 Alzheimer's disease, 20 frontotemporal dementia, 20 mild cognitive impairment) and 20 controls were included retrospectively. One MRI per subject was processed by software packages from two proprietary manufacturers, producing two quantitative reports per subjec ...[more]

PMID: 35032183

Similar Datasets

Project description:ImportanceAmyloid-related imaging abnormalities (ARIA) are brain magnetic resonance imaging (MRI) findings associated with the use of amyloid-β-directed monoclonal antibody therapies in Alzheimer disease (AD). ARIA monitoring is important to inform treatment dosing decisions and might be improved through assistive software.ObjectiveTo assess the clinical performance of an artificial intelligence (AI)-based software tool for assisting radiological interpretation of brain MRI scans in patients monitored for ARIA.Design, setting, and participantsThis diagnostic study used a multiple-reader multiple-case design to evaluate the diagnostic performance of radiologists assisted by the software vs unassisted. The study enrolled 16 US Board of Radiology-certified radiologists to perform radiological reading with (assisted) and without the software (unassisted). The study encompassed 199 retrospective cases, where each case consisted of a predosing baseline and a postdosing follow-up MRI of patients from aducanumab clinical trials PRIME, EMERGE, and ENGAGE. Statistical analysis was performed from April to July 2023.ExposuresUse of icobrain aria, an AI-based assistive software for ARIA detection and quantification.Main outcomes and measuresCoprimary end points were the difference in diagnostic accuracy between assisted and unassisted detection of ARIA-E (edema and/or sulcal effusion) and ARIA-H (microhemorrhage and/or superficial siderosis) independently, assessed with the area under the receiver operating characteristic curve (AUC).ResultsAmong the 199 participants included in this study of radiological reading performance, mean (SD) age was 70.4 (7.2) years; 105 (52.8%) were female; 23 (11.6%) were Asian, 1 (0.5%) was Black, 157 (78.9%) were White, and 18 (9.0%) were other or unreported race and ethnicity. Among the 16 radiological readers included, 2 were specialized neuroradiologists (12.5%), 11 were male individuals (68.8%), 7 were individuals working in academic hospitals (43.8%), and they had a mean (SD) of 9.5 (5.1) years of experience. Radiologists assisted by the software were significantly superior in detecting ARIA than unassisted radiologists, with a mean assisted AUC of 0.87 (95% CI, 0.84-0.91) for ARIA-E detection (AUC improvement of 0.05 [95% CI, 0.02-0.08]; P = .001]) and 0.83 (95% CI, 0.78-0.87) for ARIA-H detection (AUC improvement of 0.04 [95% CI, 0.02-0.07]; P = .001). Sensitivity was significantly higher in assisted reading compared with unassisted reading (87% vs 71% for ARIA-E detection; 79% vs 69% for ARIA-H detection), while specificity remained above 80% for the detection of both ARIA types.Conclusions and relevanceThis diagnostic study found that radiological reading performance for ARIA detection and diagnosis was significantly better when using the AI-based assistive software. Hence, the software has the potential to be a clinically important tool to improve safety monitoring and management of patients with AD treated with amyloid-β-directed monoclonal antibody therapies.

Project description:Background/objectivesThe purpose of this study was to compare the differences in tooth movements when implementing the same virtual setup on the following four different software packages: ClinCheck® Pro, Ortho Analyzer®, SureSmile®, and Ortho Insight 3D®.Materials/methodsTwenty-five adult patients treated with Invisalign® at the Case School of Dental Medicine (CWRU)'s department of orthodontics were retrospectively collected. Initial stereolithography (STL) files were obtained and imported into three software packages. The teeth were moved in order to replicate the virtual setup from ClinCheck® Pro. The final outcomes were exported from each software package. ClinCheck® Pro STL files were used as the reference while STL files produced by the other software packages were used as the targets. Best fit superimpositions were performed using Geomagic® Control X. Based on the results, tooth position was adjusted in the three software packages until the virtual setups from ClinCheck® Pro were replicated. Once confirmed, the tables containing the tooth movements were compared. The number of aligners and number of attachments automatically generated from each of the software packages were also evaluated.ResultsExtrusion/intrusion (p ≤ 0.0001) and translation buccal/lingual (p ≤ 0.0004) were significantly different among the software packages. ClinCheck® Pro and SureSmile® (p ≤ 0.000), SureSmile® and Ortho Insight 3D® (p ≤ 0.014), SureSmile® and Ortho Analyzer® (p ≤ 0.009), and Ortho Insight 3D® and Ortho Analyzer® (p ≤ 0.000) generated a significantly different number of maxillary aligners. The results varied slightly for mandibular aligners, with only ClinCheck® Pro and Ortho Insight 3D® (p ≤ 0.000), SureSmile® and Ortho Insight 3D® (p ≤ 0.000), and Ortho Insight 3D® and Ortho Analyzer® (p ≤ 0.000) exhibiting a significant difference. ClinCheck® Pro and SureSmile® (p ≤ 0.000) differed significantly in the number of attachments produced.ConclusionsThere are statistically significant differences in extrusion/intrusion, translation buccal/lingual, the number of aligners, and the number of attachments when implementing the same virtual setup on different software packages. Clinicians may need to consider this when utilizing software programs for digital diagnosis and treatment planning.

Project description:This study aimed to build automated detection models-one by brain regional volume (V-model), and the other by radiomics features of the whole brain (R-model)-to differentiate mild cognitive impairment (MCI) from cognitive normal (CN), and Alzheimer's Disease (AD) from mild cognitive impairment (MCI). The objectives are to compare the models and identify whether radiomics or volumetry can provide a better prediction for differentiating different types of dementia. 582 MRI T1-weighted images were retrieved from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database, which is a multicenter operating open source database for AD. In total, 97 images of AD, 293 images of MCI patient and 192 images of cognitive normal were divided into a training, a validation and a test group at a ratio of 70:15:15. For each T1-weighted image, volumetric segmentation was performed with the image analysis software FreeSurfer, and radiomics features were retrieved by imaging research software 3D slicers. Brain regional volume and radiomics features were used to build the V-model and R-model, respectively, using the random forest algorithm by R. The receiver operating characteristics (ROC) curve of both models were used to evaluate their diagnostic accuracy and reliability to differentiate AD, MCI and CN. To differentiate MCI and CN, both V-model and R-model achieved excellent performance, with an AUC of 0.9992 ± 0.0022 and 0.9850 ± 0.0032, respectively. No significant difference was found between the two AUCs, indicating both models attained similar good performance. In MCI and AD differentiation, the V-model and R-model yielded AUC of 0.9986 ± 0.0013 and 0.9714 ± 0.0175, respectively. The best performance was to differentiate AD from CN, where the V-model and R-model yielded AUC of 0.9994 ± 0.0019 and 0.9830 ± 0.009, respectively. The results suggested that both volumetry and radiomics approaches could be used in differentiating AD, MCI and CN, based on T1 weighted MR images using random forest algorithm successfully. This study showed that the radiomics features from T1-weighted MR images achieved excellence performance in differentiating AD, MCI and CN. Compared to the volumetry method, the accuracy, sensitivity and specificity are slightly lower in using radiomics, but still attained very good and reliable classification of the three stages of neurodegenerations. In view of the convenience and operator independence in feature extraction, radiomics can be a quantitative biomarker to differentiate the disease groups.

Dataset Information

Comparing two artificial intelligence software packages for normative brain volumetry in memory clinic imaging.

Purpose

Methods

Results

Conclusion

Publications

Comparing two artificial intelligence software packages for normative brain volumetry in memory clinic imaging.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets