Dataset Information

Simulated MRI Artifacts: Testing Machine Learning Failure Modes.

ABSTRACT: Objective. Seven types of MRI artifacts, including acquisition and preprocessing errors, were simulated to test a machine learning brain tumor segmentation model for potential failure modes. Introduction. Real-world medical deployments of machine learning algorithms are less common than the number of medical research papers using machine learning. Part of the gap between the performance of models in research and deployment comes from a lack of hard test cases in the data used to train a model. Methods. These failure modes were simulated for a pretrained brain tumor segmentation model that utilizes standard MRI and used to evaluate the performance of the model under duress. These simulated MRI artifacts consisted of motion, susceptibility induced signal loss, aliasing, field inhomogeneity, sequence mislabeling, sequence misalignment, and skull stripping failures. Results. The artifact with the largest effect was the simplest, sequence mislabeling, though motion, field inhomogeneity, and sequence misalignment also caused significant performance decreases. The model was most susceptible to artifacts affecting the FLAIR (fluid attenuation inversion recovery) sequence. Conclusion. Overall, these simulated artifacts could be used to test other brain MRI models, but this approach could be used across medical imaging applications.

SUBMITTER: Wang NC

PROVIDER: S-EPMC10521705 | biostudies-literature | 2022

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Simulated MRI Artifacts: Testing Machine Learning Failure Modes.

Wang Nicholas C NC Noll Douglas C DC Srinivasan Ashok A Gagnon-Bartsch Johann J Kim Michelle M MM Rao Arvind A

BME frontiers 20221101

<i>Objective</i>. Seven types of MRI artifacts, including acquisition and preprocessing errors, were simulated to test a machine learning brain tumor segmentation model for potential failure modes. <i>Introduction</i>. Real-world medical deployments of machine learning algorithms are less common than the number of medical research papers using machine learning. Part of the gap between the performance of models in research and deployment comes from a lack of hard test cases in the data used to tr ...[more]

PMID: 37850164

Similar Datasets

Project description:Background:To evaluate the association of multiparametric and multiregional MRI-features with key molecular characteristics in patients with newly-diagnosed glioblastoma. Methods:Retrospective data evaluation was approved by the local ethics committee of the University of Heidelberg (ethics approval number: S-320/2012) and informed consent was waived. Preoperative MRI-features were correlated with key molecular characteristics within a single-institutional cohort of 152 patients with newly-diagnosed glioblastoma. Preoperative MRI-features (n=31) included multiparametric (anatomical, diffusion-, perfusion-, and susceptibility-weighted images) and multiregional (contrast enhancing and non-enhancing FLAIR-hyperintense) information with (histogram) quantification of tumor volumes, volume ratios, apparent diffusion coefficients, cerebral blood flow / volume (CBF / CBV) and intratumoral susceptibility signals. Molecular characteristics determined with the Illumina Infinium HumanMethylation450 array included global DNA-methylation subgroups (e.g. mesenchymal (MES), RTK I “PGFRA”, RTK II “classic”), MGMT-promoter methylation status and hallmark copy-number-variations (EGFR-, PDGFRA-, MDM4- and CDK4-amplification; PTEN-, CDKN2A-, NF1- and RB1-loss). Univariate analyses (voxel-lesion-symptom-mapping for tumor location, Wilcoxon-test for all other MRI-features) as well as machine-learning models were applied to study the strength of association and discriminative value of MRI-features for predicting underlying molecular characteristics. Results: There was no tumor location predilection for any of the assessed molecular parameters (permutation-adjusted p>0.05 each). Univariate imaging parameter associations were noted for EGFR amplification and CDKN2A loss, both demonstrating increased nrCBV and nrCBF values (performance of these parameters, as assessed by the area under the ROC curve ranged from 63 to 69%, FDR-adjusted p<0.05, respectively). Subjecting all MRI-features to machine-learning-based classification allowed to predict EGFR amplification status and the RTK II “classic” GB subgroup with a moderate, yet significantly greater accuracy (63% for EGFR [p<0.01] and 61% for RTK II [p=0.01]) than the prediction by chance, whereas prediction accuracy for all other molecular parameters was non-significant (p>0.05, all models). Conclusions: In summary, we found univariate associations between established MRI-features and molecular characteristics, however not of sufficient strength to allow the generation of machine-learning classification models for reliable and clinically meaningful prediction of the assessed molecular characteristics in patients with newly-diagnosed glioblastoma.

Project description:BackgroundHeart failure (HF) hospitalization is a dominant contributor of morbidity and healthcare expenditures in patients with systolic HF. Cardiovascular magnetic resonance (CMR) imaging is increasingly employed for the evaluation of HF given capacity to provide highly reproducible phenotypic markers of disease. The combined value of CMR phenotypic markers and patient health information to deliver predictions of future HF events has not been explored. We sought to develop and validate a novel risk model for the patient-specific prediction of time to HF hospitalization using routinely reported CMR variables, patient-reported health status, and electronic health information.MethodsStandardized data capture was performed for 1,775 consecutive patients with chronic systolic HF referred for CMR imaging. Patient demographics, symptoms, Health-related Quality of Life, pharmacy, and routinely reported CMR features were provided to both machine learning (ML) and competing risk Fine-Gray-based models (FGM) for the prediction of time to HF hospitalization.ResultsThe mean age was 59 years with a mean LVEF of 36 ± 11%. The population was evenly distributed between ischemic (52%) and idiopathic non-ischemic cardiomyopathy (48%). Over a median follow-up of 2.79 years (IQR: 1.59-4.04) 333 patients (19%) experienced HF related hospitalization. Both ML and competing risk FGM based models achieved robust performance for the prediction of time to HF hospitalization. Respective 90-day, 1 and 2-year AUC values were 0.87, 0.83, and 0.80 for the ML model, and 0.89, 0.84, and 0.80 for the competing risk FGM-based model in a holdout validation cohort. Patients classified as high-risk by the ML model experienced a 34-fold higher occurrence of HF hospitalization at 90 days vs. the low-risk group.ConclusionIn this study we demonstrated capacity for routinely reported CMR phenotypic markers and patient health information to be combined for the delivery of patient-specific predictions of time to HF hospitalization. This work supports an evolving migration toward multi-domain data collection for the delivery of personalized risk prediction at time of diagnostic imaging.

Dataset Information

Simulated MRI Artifacts: Testing Machine Learning Failure Modes.

Publications

Simulated MRI Artifacts: Testing Machine Learning Failure Modes.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets