Dataset Information

Quantifying the unknown impact of segmentation uncertainty on image-based simulations.

ABSTRACT: Image-based simulation, the use of 3D images to calculate physical quantities, relies on image segmentation for geometry creation. However, this process introduces image segmentation uncertainty because different segmentation tools (both manual and machine-learning-based) will each produce a unique and valid segmentation. First, we demonstrate that these variations propagate into the physics simulations, compromising the resulting physics quantities. Second, we propose a general framework for rapidly quantifying segmentation uncertainty. Through the creation and sampling of segmentation uncertainty probability maps, we systematically and objectively create uncertainty distributions of the physics quantities. We show that physics quantity uncertainty distributions can follow a Normal distribution, but, in more complicated physics simulations, the resulting uncertainty distribution can be surprisingly nontrivial. We establish that bounding segmentation uncertainty can fail in these nontrivial situations. While our work does not eliminate segmentation uncertainty, it improves simulation credibility by making visible the previously unrecognized segmentation uncertainty plaguing image-based simulation.

SUBMITTER: Krygier MC

PROVIDER: S-EPMC8440761 | biostudies-literature | 2021 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Quantifying the unknown impact of segmentation uncertainty on image-based simulations.

Krygier Michael C MC LaBonte Tyler T Martinez Carianne C Norris Chance C Sharma Krish K Collins Lincoln N LN Mukherjee Partha P PP Roberts Scott A SA

Nature communications 20210914 1

Image-based simulation, the use of 3D images to calculate physical quantities, relies on image segmentation for geometry creation. However, this process introduces image segmentation uncertainty because different segmentation tools (both manual and machine-learning-based) will each produce a unique and valid segmentation. First, we demonstrate that these variations propagate into the physics simulations, compromising the resulting physics quantities. Second, we propose a general framework for ra ...[more]

PMID: 34521853

Similar Datasets

Project description:BackgroundUncertainty quantification in deep learning is an important research topic. For medical image segmentation, the uncertainty measurements are usually reported as the likelihood that each pixel belongs to the predicted segmentation region. In potential clinical applications, the uncertainty result reflects the algorithm's robustness and supports the confidence and trust of the segmentation result when the ground-truth result is absent. For commonly studied deep learning models, novel methods for quantifying segmentation uncertainty are in demand.PurposeTo develop a U-Net segmentation uncertainty quantification method based on spherical image projection of multi-parametric MRI (MP-MRI) in glioma segmentation.MethodsThe projection of planar MRI data onto a spherical surface is equivalent to a nonlinear image transformation that retains global anatomical information. By incorporating this image transformation process in our proposed spherical projection-based U-Net (SPU-Net) segmentation model design, multiple independent segmentation predictions can be obtained from a single MRI. The final segmentation is the average of all available results, and the variation can be visualized as a pixel-wise uncertainty map. An uncertainty score was introduced to evaluate and compare the performance of uncertainty measurements. The proposed SPU-Net model was implemented on the basis of 369 glioma patients with MP-MRI scans (T1, T1-Ce, T2, and FLAIR). Three SPU-Net models were trained to segment enhancing tumor (ET), tumor core (TC), and whole tumor (WT), respectively. The SPU-Net model was compared with (1) the classic U-Net model with test-time augmentation (TTA) and (2) linear scaling-based U-Net (LSU-Net) segmentation models in terms of both segmentation accuracy (Dice coefficient, sensitivity, specificity, and accuracy) and segmentation uncertainty (uncertainty map and uncertainty score).ResultsThe developed SPU-Net model successfully achieved low uncertainty for correct segmentation predictions (e.g., tumor interior or healthy tissue interior) and high uncertainty for incorrect results (e.g., tumor boundaries). This model could allow the identification of missed tumor targets or segmentation errors in U-Net. Quantitatively, the SPU-Net model achieved the highest uncertainty scores for three segmentation targets (ET/TC/WT): 0.826/0.848/0.936, compared to 0.784/0.643/0.872 using the U-Net with TTA and 0.743/0.702/0.876 with the LSU-Net (scaling factor = 2). The SPU-Net also achieved statistically significantly higher Dice coefficients, underscoring the improved segmentation accuracy.ConclusionThe SPU-Net model offers a powerful tool to quantify glioma segmentation uncertainty while improving segmentation accuracy. The proposed method can be generalized to other medical image-related deep-learning applications for uncertainty evaluation.

Project description:Problem. Image biomarker analysis, also known as radiomics, is a tool for tissue characterization and treatment prognosis that relies on routinely acquired clinical images and delineations. Due to the uncertainty in image acquisition, processing, and segmentation (delineation) protocols, radiomics often lack reproducibility. Radiomics harmonization techniques have been proposed as a solution to reduce these sources of uncertainty and/or their influence on the prognostic model performance. A relevant question is how to estimate the protocol-induced uncertainty of a specific image biomarker, what the effect is on the model performance, and how to optimize the model given the uncertainty. Methods. Two non-small cell lung cancer (NSCLC) cohorts, composed of 421 and 240 patients, respectively, were used for training and testing. Per patient, a Monte Carlo algorithm was used to generate three hundred synthetic contours with a surface dice tolerance measure of less than 1.18 mm with respect to the original GTV. These contours were subsequently used to derive 104 radiomic features, which were ranked on their relative sensitivity to contour perturbation, expressed in the parameter η. The top four (low η) and the bottom four (high η) features were selected for two models based on the Cox proportional hazards model. To investigate the influence of segmentation uncertainty on the prognostic model, we trained and tested the setup in 5000 augmented realizations (using a Monte Carlo sampling method); the log-rank test was used to assess the stratification performance and stability of segmentation uncertainty. Results. Although both low and high η setup showed significant testing set log-rank p-values (p = 0.01) in the original GTV delineations (without segmentation uncertainty introduced), in the model with high uncertainty, to effect ratio, only around 30% of the augmented realizations resulted in model performance with p < 0.05 in the test set. In contrast, the low η setup performed with a log-rank p < 0.05 in 90% of the augmented realizations. Moreover, the high η setup classification was uncertain in its predictions for 50% of the subjects in the testing set (for 80% agreement rate), whereas the low η setup was uncertain only in 10% of the cases. Discussion. Estimating image biomarker model performance based only on the original GTV segmentation, without considering segmentation, uncertainty may be deceiving. The model might result in a significant stratification performance, but can be unstable for delineation variations, which are inherent to manual segmentation. Simulating segmentation uncertainty using the method described allows for more stable image biomarker estimation, selection, and model development. The segmentation uncertainty estimation method described here is universal and can be extended to estimate other protocol uncertainties (such as image acquisition and pre-processing).

Project description:PurposeHemodynamics play an important role in the assessment of intracranial aneurysm (IA) development and rupture risk. The purpose of this study was to examine the impact of complex vasculatures onto the intra-vessel and intra-aneurysmal blood flow.MethodsComplex segmentation of a subject-specific, 60-outlet and 3-inlet circle of Willis model captured with 7T magnetic resonance imaging was performed. This model was trimmed to a 10-outlet model version. Two patient-specific IAs were added onto both models yielding two pathological versions, and image-based blood flow simulations of the four resulting cases were carried out. To capture the differences between complex and trimmed model, time-averaged and centerline velocities were compared. The assessment of intra-saccular blood flow within the IAs involved the evaluation of wall shear stresses (WSS) at the IA wall and neck inflow rates (NIR).ResultsLower flow values are observed in the majority of the complex model. However, at specific locations (left middle cerebral artery 0.5 m/s, left posterior cerebral artery 0.25 m/s), higher flow rates were visible when compared to the trimmed counterpart. Furthermore, at the centerlines the total velocity values reveal differences up to 0.15 m/s. In the IAs, the reduction in the neck inflow rate and WSS in the complex model was observed for the first IA (IA-A δNIRmean = - 0.07ml/s, PCA.l δWSSmean = - 0.05 Pa). The second IA featured an increase in the neck inflow rate and WSS (IA-B δNIRmean = 0.04 ml/s, PCA.l δWSSmean = 0.07 Pa).ConclusionBoth the magnitude and shape of the flow distribution vary depending on the model's complexity. The magnitude is primarily influenced by the global vessel model, while the shape is determined by the local structure. Furthermore, intra-aneurysmal flow strongly depends on the location in the vessel tree, emphasizing the need for complex model geometries for realistic hemodynamic assessment and rupture risk analysis.

Dataset Information

Quantifying the unknown impact of segmentation uncertainty on image-based simulations.

Publications

Quantifying the unknown impact of segmentation uncertainty on image-based simulations.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets