Impact of inter-observer variability on first axillary level dosimetry in breast cancer radiotherapy: An AIRO multi-institutional study.
Ontology highlight
ABSTRACT:
SUBMITTER: Leonardi MC
PROVIDER: S-EPMC10702305 | biostudies-literature | 2023 Dec
REPOSITORIES: biostudies-literature
Publications
Impact of inter-observer variability on first axillary level dosimetry in breast cancer radiotherapy: An AIRO multi-institutional study.
Leonardi Maria Cristina MC Pepa Matteo M Zaffaroni Mattia M Vincini Maria Giulia MG Luraschi Rosa R Vigorito Sabrina S Morra Anna A Dicuonzo Samantha S Mazzola Giovanni Carlo GC Gerardi Marianna Alessandra MA Zerella Maria Alessia MA Cante Domenico D Petrucci Edoardo E Borzì Giuseppina G Marrocco Maristella M Chieregato Matteo M Iadanza Luciano L Lobefalo Francesca F Valenti Marco M Cavallo Anna A Russo Serenella S Guernieri Marika M Malatesta Tiziana T Meaglia Ilaria I Liotta Marco M Palumbo Isabella I Marcantonini Marta M Mezzenga Emilio E Falivene Sara S Arrichiello Cecilia C Barbero Maria Paola MP Ivaldi Giovanni Battista GB Catalano Gianpiero G Vidali Cristiana C Giannitto Caterina C Ciabattoni Antonella A Meattini Icro I Aristei Cynthia C Orecchia Roberto R Cattani Federica F Jereczek-Fossa Barbara Alicja BA
Tumori 20230909 6
This study quantified the incidental dose to the first axillary level (L1) in locoregional treatment plan for breast cancer. Eighteen radiotherapy centres contoured L1-L4 on three different patients (P1,2,3), created the L2-L4 planning target volume (single centre planning target volume, SC-PTV) and elaborated a locoregional treatment plan. The L2-L4 gold standard clinical target volume (CTV) along with the gold standard L1 contour (GS-L1) were created by an expert consensus. The SC-PTV was then ...[more]
Similar Datasets
Project description:ObjectivesTo determine interobserver variability in axillary nodal contouring in breast cancer (BC) radiotherapy (RT) by comparing the clinical target volume of participating single centres (SC-CTV) with a gold-standard CTV (GS-CTV).MethodsThe GS-CTV of three patients (P1, P2, P3) with increasing complexity was created in DICOM format from the median contour of axillary CTVs drawn by BC experts, validated using the simultaneous truth and performance-level estimation and peer-reviewed. GS-CTVs were compared with the correspondent SC-CTVs drawn by radiation oncologists, using validated metrics and a total score (TS) integrating all of them.ResultsEighteen RT centres participated in the study. Comparative analyses revealed that, on average, the SC-CTVs were smaller than GS-CTV for P1 and P2 (by -29.25% and -27.83%, respectively) and larger for P3 (by +12.53%). The mean Jaccard index was greater for P1 and P2 compared to P3, but the overlap extent value was around 0.50 or less. Regarding nodal levels, L4 showed the highest concordance with the GS. In the intra-patient comparison, L2 and L3 achieved lower TS than L4. Nodal levels showed discrepancy with GS, which was not statistically significant for P1, and negligible for P2, while P3 had the worst agreement. DICE similarity coefficient did not exceed the minimum threshold for agreement of 0.70 in all the measurements.ConclusionsSubstantial differences were observed between SC- and GS-CTV, especially for P3 with altered arm setup. L2 and L3 were the most critical levels. The study highlighted these key points to address.Advances in knowledgeThe present study compares, by means of validated geometric indexes, manual segmentations of axillary lymph nodes in breast cancer from different observers and different institutions made on radiotherapy planning CT images. Assessing such variability is of paramount importance, as geometric uncertainties might lead to incorrect dosimetry and compromise oncological outcome.
| S-EPMC8248216 | biostudies-literature
Project description:INTRODUCTION:Strong correlation has been demonstrated between tumor dose and response and between healthy liver dose and side effects. Individualized dosimetry is increasingly recommended in the current clinical routine. However, hepatic and tumor segmentations could be complex in some cases. The aim of this study is to assess the reproducibility of the tumoral and non-tumoral liver dosimetry in selective internal radiation therapy (SIRT). MATERIAL AND METHODS:Twenty-three patients with hepatocellular carcinoma (HCC) who underwent SIRT with glass microspheres were retrospectively included in the study. Tumor (TV) and total liver volumes (TLV), and mean absorbed doses in tumoral liver (TD) and non-tumoral liver (THLD) were determined on the 90Y PET/CT studies using Simplicit90YTM software, by three independent observers. Dosimetry datasets were obtained by a medical physicist helped by a nuclear medicine (NM) physician with 10 years of experience (A), by a NM physician with 4-year experience (B), and by a resident who first performed 10 dosimetry assessments as a training (C). Inter-observer agreement was evaluated using intra-class correlation coefficients (ICC), coefficients of variation (CV), Bland-Altman plots, and reproducibility coefficient (RDC). RESULTS:A strong agreement was observed between all three readers for estimating TLV (ICC 0.98) and THLD (ICC 0.97). Agreement was lower for TV delineation (ICC 0.94) and particularly for TD (ICC 0.73), especially for the highest values. Regarding TD, the CV (%) was 26.5, 26.9, and 20.2 between observers A and B, A and C, and B and C, respectively, and the RDC was 1.5. Regarding THLD, it was 8.5, 12.7, and 9.4, and the RDC was 1.3. CONCLUSION:Using a standardized methodology, and regardless of the different experiences of the observers, the estimation of THLD is highly reproducible. Although the reproducibility of the assessment of tumor irradiation is overall quite high, large variations may be observed in a limited number of patients.
| S-EPMC7218042 | biostudies-literature
Project description:The NIMO TEMPO (Lambda-X, Nivelles, Belgium) is a novel, user-friendly and compact device designed for in vitro optical analysis of refractive and diffractive intraocular lenses (IOLs). This device analyzes the IOL wavefront and generates a synthetic eye model for numerical computation. The objective of this study was to evaluate the precision of this innovative device. Intra- and inter-observer variability were calculated using a two-way analysis of variance (ANOVA) after conducting ten measurements of eight different IOL models, with each measurement being repeated by three distinct operators (resulting in a total of 30 measurements for each IOL). The device demonstrated satisfactory intra- and inter-observer variability in evaluating IOL power and modulation transfer function (MTF) profiles, with values of 0.066 and 0.078 diopters for IOL power and 0.018 and 0.019 for MTF measurements, respectively. Furthermore, this hybrid optical and numerical in vitro IOL wavefront analyzer appears to have several advantages over conventional optical bench devices. It reduces the need for operator manipulation, and allows for numerical modeling of various optical environments, including cornea models and apertures. In conclusion, this novel metrology device designed for refractive and diffractive IOLs appears to provide a satisfactory precision, making it a promising tool in the field of IOL metrology.
| S-EPMC10813881 | biostudies-literature
Project description:ObjectiveTo determine inter-observer and inter-examination variability of manual attenuation measurements of the vertebrae in low-dose unenhanced chest computed tomography (CT).MethodsThree hundred and sixty-seven lung cancer screening trial participants who underwent baseline and repeat unenhanced low-dose CT after 3 months because of an indeterminate lung nodule were included. The CT attenuation value of the first lumbar vertebrae (L1) was measured in all CTs by one observer to obtain inter-examination reliability. Six observers performed measurements in 100 randomly selected CTs to determine agreement with limits of agreement and Bland-Altman plots and reliability with intraclass correlation coefficients (ICCs). Reclassification analyses were performed using a threshold of 110 HU to define osteoporosis.ResultsInter-examination reliability was excellent with an ICC of 0.92 (p < 0.001). Inter-examination limits of agreement ranged from -26 to 28 HU with a mean difference of 1 ± 14 HU. Inter-observer reliability ICCs ranged from 0.70 to 0.91. Inter-examination variability led to 11.2 % reclassification of participants and inter-observer variability led to 22.1 % reclassification.ConclusionsVertebral attenuation values can be manually quantified with good to excellent inter-examination and inter-observer reliability on unenhanced low-dose chest CT. This information is valuable for early detection of osteoporosis on low-dose chest CT.Key points• Vertebral attenuation values can be manually quantified on low-dose unenhanced CT reliably. • Vertebral attenuation measurements may be helpful in detecting subclinical low bone density. • This could become of importance in the detection of osteoporosis.
| S-EPMC4972882 | biostudies-literature
Project description:PurposeSeveral efforts are being undertaken toward MRI-based treatment planning for ocular proton therapy for uveal melanoma (UM). The interobserver variability of the gross target volume (GTV) on magnetic resonance imaging (MRI) is one of the important parameters to design safety margins for a reliable treatment. Therefore, this study assessed the interobserver variation in GTV delineation of UM on MRI.Methods and materialsSix observers delineated the GTV in 10 different patients using the Big Brother contouring software. Patients were scanned at 3T MRI with a surface coil, and tumors were delineated separately on contrast enhanced 3DT1 (T1gd) and 3DT2-weighted scans with an isotropic acquisition resolution of 0.8 mm. Volume difference and overall local variation (median standard deviation of the distance between the delineated contours and the median contour) were analyzed for each GTV. Additionally, the local variation was analyzed for 4 interfaces: sclera, vitreous, retinal detachment, and tumor-choroid interface.ResultsThe average GTV was significantly larger on T1gd (0.57cm3) compared with T2 (0.51cm3, P = .01). A not significant higher interobserver variation was found on T1gd (0.41 mm) compared with T2 (0.35 mm). The largest variations were found at the tumor-choroid interface due to peritumoral enhancement (T1gd, 0.62 mm; T2, 0.52 mm). As a result, a larger part of this tumor-choroid interface appeared to be included on T1gd-based GTVs compared with T2, explaining the smaller volumes on T2.ConclusionsThe interobserver variation of 0.4 mm on MRI are low with respect to the voxel size of 0.8 mm, enabling small treatment margins. We recommend delineation based on the T1gd-weighted scans, as choroidal tumor extensions might be missed.
| S-EPMC9860418 | biostudies-literature
Project description:ObjectivesEstablishing the reproducibility of expert-derived measurements on CTA exams of aortic dissection is clinically important and paramount for ground-truth determination for machine learning.MethodsFour independent observers retrospectively evaluated CTA exams of 72 patients with uncomplicated Stanford type B aortic dissection and assessed the reproducibility of a recently proposed combination of four morphologic risk predictors (maximum aortic diameter, false lumen circumferential angle, false lumen outflow, and intercostal arteries). For the first inter-observer variability assessment, 47 CTA scans from one aortic center were evaluated by expert-observer 1 in an unconstrained clinical assessment without a standardized workflow and compared to a composite of three expert-observers (observers 2-4) using a standardized workflow. A second inter-observer variability assessment on 30 out of the 47 CTA scans compared observers 3 and 4 with a constrained, standardized workflow. A third inter-observer variability assessment was done after specialized training and tested between observers 3 and 4 in an external population of 25 CTA scans. Inter-observer agreement was assessed with intraclass correlation coefficients (ICCs) and Bland-Altman plots.ResultsPre-training ICCs of the four morphologic features ranged from 0.04 (-0.05 to 0.13) to 0.68 (0.49-0.81) between observer 1 and observers 2-4 and from 0.50 (0.32-0.69) to 0.89 (0.78-0.95) between observers 3 and 4. ICCs improved after training ranging from 0.69 (0.52-0.87) to 0.97 (0.94-0.99), and Bland-Altman analysis showed decreased bias and limits of agreement.ConclusionsManual morphologic feature measurements on CTA images can be optimized resulting in improved inter-observer reliability. This is essential for robust ground-truth determination for machine learning models.Key points• Clinical fashion manual measurements of aortic CTA imaging features showed poor inter-observer reproducibility. • A standardized workflow with standardized training resulted in substantial improvements with excellent inter-observer reproducibility. • Robust ground truth labels obtained manually with excellent inter-observer reproducibility are key to develop reliable machine learning models.
| S-EPMC10017115 | biostudies-literature
Project description:BackgroundPET-based tumor delineation is an error prone and labor intensive part of image analysis. Especially for patients with advanced disease showing bulky tumor FDG load, segmentations are challenging. Reducing the amount of user-interaction in the segmentation might help to facilitate segmentation tasks especially when labeling bulky and complex tumors. Therefore, this study reports on segmentation workflows/strategies that may reduce the inter-observer variability for large tumors with complex shapes with different levels of user-interaction.MethodsTwenty PET images of bulky tumors were delineated independently by six observers using four strategies: (I) manual, (II) interactive threshold-based, (III) interactive threshold-based segmentation with the additional presentation of the PET-gradient image and (IV) the selection of the most reasonable result out of four established semi-automatic segmentation algorithms (Select-the-best approach). The segmentations were compared using Jaccard coefficients (JC) and percentage volume differences. To obtain a reference standard, a majority vote (MV) segmentation was calculated including all segmentations of experienced observers. Performed and MV segmentations were compared regarding positive predictive value (PPV), sensitivity (SE), and percentage volume differences.ResultsThe results show that with decreasing user-interaction the inter-observer variability decreases. JC values and percentage volume differences of Select-the-best and a workflow including gradient information were significantly better than the measurements of the other segmentation strategies (p-value<0.01). Interactive threshold-based and manual segmentations also result in significant lower and more variable PPV/SE values when compared with the MV segmentation.ConclusionsFDG PET segmentations of bulky tumors using strategies with lower user-interaction showed less inter-observer variability. None of the methods led to good results in all cases, but use of either the gradient or the Select-the-best workflow did outperform the other strategies tested and may be a good candidate for fast and reliable labeling of bulky and heterogeneous tumors.
| S-EPMC7105134 | biostudies-literature
Project description:Background and purposeThe delineation of the gross tumor volume (GTV) is a critical step for radiation therapy treatment planning. The delineation procedure is typically performed manually which exposes two major issues: cost and reproducibility. Delineation is a time-consuming process that is subject to inter- and intra-observer variability. While methods have been proposed to predict GTV contours, typical approaches ignore variability and therefore fail to utilize the valuable confidence information offered by multiple contours.Materials and methodsIn this work we propose an automatic GTV contouring method for soft-tissue sarcomas from X-ray computed tomography (CT) images, using deep learning by integrating inter- and intra-observer variability in the learned model. Sixty-eight patients with soft tissue and bone sarcomas were considered in this evaluation, all underwent pre-operative CT imaging used to perform GTV delineation. Four radiation oncologists and radiologists performed three contouring trials each for all patients. We quantify variability by defining confidence levels based on the frequency of inclusion of a given voxel into the GTV and use a deep convolutional neural network to learn GTV confidence maps.ResultsResults were compared to confidence maps from the four readers as well as ground-truth consensus contours established jointly by all readers. The resulting continuous Dice score between predicted and true confidence maps was 87% and the Hausdorff distance was 14 mm.ConclusionResults demonstrate the ability of the proposed method to predict accurate contours while utilizing variability and as such it can be used to improve clinical workflow.
| S-EPMC8934266 | biostudies-literature
Project description:Evaluating error that arises through the aggregation of data recorded by multiple observers is a key consideration in many metric and geometric morphometric analyses of stone tool shape. One of the most common approaches involves the convergence of observers for repeat trails on the same set of artefacts: however, this is logistically and financially challenging when collaborating internationally and/or at a large scale. We present and evaluate a unique alternative for testing inter-observer error, involving the development of 3D printed copies of a lithic reference collection for distribution among observers. With the aim of reducing error, clear protocols were developed for photographing and measuring the replicas, and inter-observer variability was assessed on the replicas in comparison with a corresponding data set recorded by a single observer. Our results demonstrate that, when the photography procedure is standardized and dimensions are clearly defined, the resulting metric and geometric morphometric data are minimally affected by inter-observer error, supporting this method as an effective solution for assessing error under collaborative research frameworks. Collaboration is becoming increasingly important within archaeological and anthropological sciences in order to increase the accessibility of samples, encourage dual-project development between foreign and local researchers and reduce the carbon footprint of collection-based research. This study offers a promising validation of a collaborative research design whereby researchers remotely work together to produce comparable data capturing lithic shape variability.Supplementary informationThe online version contains supplementary material available at 10.1007/s12520-022-01676-2.
| S-EPMC9525927 | biostudies-literature
Project description:Background and purposeDaily target re-delineation in online adaptive radiotherapy (oART) introduces uncertainty. The aim of this study was to evaluate artificial intelligence (AI) generated contours and inter-observer target variation among radiotherapy technicians in cone-beam CT (CBCT) guided oART of bladder cancer.Materials and methodsFor each of 10 consecutive patients treated with oART for bladder cancer, one CBCT was randomly selected and retrospectively included. The bladder (CTV-T) was AI-segmented (CTV-TAI). Seven radiotherapy technicians independently reviewed and edited CTV-TAI, generating CTV-TADP. Contours were benchmarked against a ground truth contour (CTV-TGT) delineated blindly from scratch. CTV-TADP and CTV-TAI were compared to CTV-TGT using volume, dice similarity coefficient, and bidirectional local distance. Dose coverage (D99%>95 %) of CTV-TGT was evaluated for treatment plans optimized for CTV-TAI and CTV-TADP with clinical margins. Inter-observer variation among CTV-TADP was assessed using coefficient of variation and generalized conformity index.ResultsCTV-TGT ranged from 48.7 cm3 to 211.6 cm3. The median [range] volume difference was 4.5 [-17.8, 42.4] cm3 for CTV-TADP and -15.5 [-54.2, 4.3] cm3 for CTV-TAI, compared to CTV-TGT. Corresponding dice similarity coefficients were 0.87 [0.71, 0.95] and 0.84 [0.64, 0.95]. CTV-TGT was adequately covered in 68/70 plans optimized on CTV-TADP and in 6/10 plans optimized on CTV-TAI with clinical margins. The median [range] coefficient of variation was 0.08 [0.05, 0.11] and generalized conformity index was 0.78 [0.71, 0.88] among CTV-TADP.ConclusionsTarget re-delineation in CBCT-guided oART of bladder cancer demonstrated non-isotropic inter-observer variation. Manual adjustment of AI-generated contours was necessary to cover ground truth targets.
| S-EPMC11407955 | biostudies-literature