Dataset Information

Test-Retest Data for Radiomics Feature Stability Analysis: Generalizable or Study-Specific?

ABSTRACT: Radiomics is an objective method for extracting quantitative information from medical images. However, in radiomics, standardization, overfitting, and generalization are major challenges to be overcome. Test-retest experiments can be used to select robust radiomic features that have minimal variation. Currently, it is unknown whether they should be identified for each disease (disease specific) or are only imaging device-specific (computed tomography [CT]-specific). Here, we performed a test-retest analysis on CT scans of 40 patients with rectal cancer in a clinical setting. Correlation between radiomic features was assessed using the concordance correlation coefficient (CCC). In total, only 9/542 features have a CCC > 0.85. Furthermore, results were compared with the test-retest results on CT scans of 27 patients with lung cancer with a 15-minute interval. Results show that 446/542 features have a higher CCC for the test-retest analysis of the data set of patients with lung cancer than for patients with rectal cancer. The importance of controlling factors such as scanners, imaging protocol, reconstruction methods, and time points in a radiomics analysis is shown. Moreover, the results imply that test-retest analyses should be performed before each radiomics study. More research is required to independently evaluate the effect of each factor.

SUBMITTER: van Timmeren JE

PROVIDER: S-EPMC6037932 | biostudies-literature | 2016 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Test-Retest Data for Radiomics Feature Stability Analysis: Generalizable or Study-Specific?

van Timmeren Janna E JE Leijenaar Ralph T H RTH van Elmpt Wouter W Wang Jiazhou J Zhang Zhen Z Dekker André A Lambin Philippe P

Tomography (Ann Arbor, Mich.) 20161201 4

Radiomics is an objective method for extracting quantitative information from medical images. However, in radiomics, standardization, overfitting, and generalization are major challenges to be overcome. Test-retest experiments can be used to select robust radiomic features that have minimal variation. Currently, it is unknown whether they should be identified for each disease (disease specific) or are only imaging device-specific (computed tomography [CT]-specific). Here, we performed a test-ret ...[more]

PMID: 30042967

Similar Datasets

Project description:Besides basic measurements as maximum standardized uptake value (SUV)max or SUVmean derived from 18F-FDG positron emission tomography (PET) scans, more advanced quantitative imaging features (i.e. "Radiomics" features) are increasingly investigated for treatment monitoring, outcome prediction, or as potential biomarkers. With these prospected applications of Radiomics features, it is a requisite that they provide robust and reliable measurements. The aim of our study was therefore to perform an integrated stability analysis of a large number of PET-derived features in non-small cell lung carcinoma (NSCLC), based on both a test-retest and an inter-observer setup.Eleven NSCLC patients were included in the test-retest cohort. Patients underwent repeated PET imaging within a one day interval, before any treatment was delivered. Lesions were delineated by applying a threshold of 50% of the maximum uptake value within the tumor. Twenty-three NSCLC patients were included in the inter-observer cohort. Patients underwent a diagnostic whole body PET-computed tomography (CT). Lesions were manually delineated based on fused PET-CT, using a standardized clinical delineation protocol. Delineation was performed independently by five observers, blinded to each other. Fifteen first order statistics, 39 descriptors of intensity volume histograms, eight geometric features and 44 textural features were extracted. For every feature, test-retest and inter-observer stability was assessed with the intra-class correlation coefficient (ICC) and the coefficient of variability, normalized to mean and range. Similarity between test-retest and inter-observer stability rankings of features was assessed with Spearman's rank correlation coefficient.Results showed that the majority of assessed features had both a high test-retest (71%) and inter-observer (91%) stability in terms of their ICC. Overall, features more stable in repeated PET imaging were also found to be more robust against inter-observer variability.Results suggest that further research of quantitative imaging features is warranted with respect to more advanced applications of PET imaging as being used for treatment monitoring, outcome prediction or imaging biomarkers.

Project description:BackgroundRadiomic features extracted from breast MRI have potential for diagnostic, prognostic, and predictive purposes. However, before they can be used as biomarkers in clinical decision support systems, features need to be repeatable and reproducible.ObjectiveIdentify repeatable radiomics features within breast tissue on prospectively collected MRI exams through multiple test-retest measurements.Study typeProspective.Population11 healthy female volunteers.Field strength/sequence1.5 T; MRI exams, comprising T2-weighted turbo spin-echo (T2W) sequence, native T1-weighted turbo gradient-echo (T1W) sequence, diffusion-weighted imaging (DWI) sequence using b-values 0/150/800, and corresponding derived ADC maps.Assessment18 MRI exams (three test-retest settings, repeated on 2 days) per healthy volunteer were examined on an identical scanner using a fixed clinical breast protocol. For each scan, 91 features were extracted from the 3D manually segmented right breast using Pyradiomics, before and after image preprocessing. Image preprocessing consisted of 1) bias field correction (BFC); 2) z-score normalization with and without BFC; 3) grayscale discretization using 32 and 64 bins with and without BFC; and 4) z-score normalization + grayscale discretization using 32 and 64 bins with and without BFC.Statistical testsFeatures' repeatability was assessed using concordance correlation coefficient(CCC) for each pair, i.e. each MRI was compared to each of the remaining 17 MRI with a cut-off value of CCC > 0.90.ResultsImages without preprocessing produced the highest number of repeatable features for both T1W sequence and ADC maps with 15 of 91 (16.5%) and 8 of 91 (8.8%) repeatable features, respectively. Preprocessed images produced between 4 of 91 (4.4%) and 14 of 91 (15.4%), and 6 of 91 (6.6%) and 7 of 91 (7.7%) repeatable features, respectively for T1W and ADC maps. Z-score normalization produced highest number of repeatable features, 26 of 91 (28.6%) in T2W sequences, in these images, no preprocessing produced 11 of 91 (12.1%) repeatable features.Data conclusionRadiomic features extracted from T1W, T2W sequences and ADC maps from breast MRI exams showed a varying number of repeatable features, depending on the sequence. Effects of different preprocessing procedures on repeatability of features were different for each sequence.Level of evidence2 TECHNICAL EFFICACY STAGE: 1.

Dataset Information

Test-Retest Data for Radiomics Feature Stability Analysis: Generalizable or Study-Specific?

Publications

Test-Retest Data for Radiomics Feature Stability Analysis: Generalizable or Study-Specific?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets