Clinical Implementation and Evaluation of Auto-Segmentation Tools for Multi-Site Contouring in Radiotherapy
Ontology highlight
ABSTRACT:
SUBMITTER:
PROVIDER: S-EPMC10726238 | biostudies-literature | 2023 Oct
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundStructure delineation is a necessary, yet time-consuming manual procedure in radiotherapy. Recently, convolutional neural networks have been proposed to speed-up and automatise this procedure, obtaining promising results. With the advent of magnetic resonance imaging (MRI)-guided radiotherapy, MR-based segmentation is becoming increasingly relevant. However, the majority of the studies investigated automatic contouring based on computed tomography (CT).PurposeIn this study, we investigate the feasibility of clinical use of deep learning-based automatic OARs delineation on MRI.Materials and methodsWe included 150 patients diagnosed with prostate cancer who underwent MR-only radiotherapy. A three-dimensional (3D) T1-weighted dual spoiled gradient-recalled echo sequence was acquired with 3T MRI for the generation of the synthetic-CT. The first 48 patients were included in a feasibility study training two 3D convolutional networks called DeepMedic and dense V-net (dV-net) to segment bladder, rectum and femurs. A research version of an atlas-based software was considered for comparison. Dice similarity coefficient, 95% Hausdorff distances (HD95), and mean distances were calculated against clinical delineations. For eight patients, an expert RTT scored the quality of the contouring for all the three methods. A choice among the three approaches was made, and the chosen approach was retrained on 97 patients and implemented for automatic use in the clinical workflow. For the successive 53 patients, Dice, HD95 and mean distances were calculated against the clinically used delineations.ResultsDeepMedic, dV-net and the atlas-based software generated contours in 60 s, 4 s and 10-15 min, respectively. Performances were higher for both the networks compared to the atlas-based software. The qualitative analysis demonstrated that delineation from DeepMedic required fewer adaptations, followed by dV-net and the atlas-based software. DeepMedic was clinically implemented. After retraining DeepMedic and testing on the successive patients, the performances slightly improved.ConclusionHigh conformality for OARs delineation was achieved with two in-house trained networks, obtaining a significant speed-up of the delineation procedure. Comparison of different approaches has been performed leading to the succesful adoption of one of the neural networks, DeepMedic, in the clinical workflow. DeepMedic maintained in a clinical setting the accuracy obtained in the feasibility study.
| S-EPMC7216473 | biostudies-literature
Project description:BackgroundThe delineation of clinical target volumes (CTVs) for radiotherapy for nasopharyngeal cancer is complex and varies based on the location and extent of disease.PurposeThe current study aimed to develop an auto-contouring solution following one protocol guidelines (NRG-HN001) that can be adjusted to meet other guidelines, such as RTOG-0225 and the 2018 International guidelines.MethodsThe study used 2-channel 3-dimensional U-Net and nnU-Net framework to auto-contour 27 normal structures in the head and neck (H&N) region that are used to define CTVs in the protocol. To define the CTV-Expansion (CTV1 and CTV2) and CTV-Overall (the outer envelope of all the CTV contours), we used adjustable morphological geometric landmarks and mimicked physician interpretation of the protocol rules by partially or fully including select anatomic structures. The results were evaluated quantitatively using the dice similarity coefficient (DSC) and mean surface distance (MSD) and qualitatively by independent reviews by two H&N radiation oncologists.ResultsThe auto-contouring tool showed high accuracy for nasopharyngeal CTVs. Comparison between auto-contours and clinical contours for 19 patients with cancers of various stages showed a DSC of 0.94 ± 0.02 and MSD of 0.4 ± 0.4 mm for CTV-Expansion and a DSC of 0.83 ± 0.02 and MSD of 2.4 ± 0.5 mm for CTV-Overall. Upon independent review, two H&N physicians found the auto-contours to be usable without edits in 85% and 75% of cases. In 15% of cases, minor edits were required by both physicians. Thus, one physician rated 100% of the auto-contours as usable (use as is, or after minor edits), while the other physician rated 90% as usable. The second physician required major edits in 10% of cases.ConclusionsThe study demonstrates the ability of an auto-contouring tool to reliably delineate nasopharyngeal CTVs based on protocol guidelines. The tool was found to be clinically acceptable by two H&N radiation oncology physicians in at least 90% of the cases.
| S-EPMC11492310 | biostudies-literature
Project description:Background and purposeDeep learning-based models have been actively investigated for various aspects of radiotherapy. However, for cervical cancer, only a few studies dealing with the auto-segmentation of organs-at-risk (OARs) and clinical target volumes (CTVs) exist. This study aimed to train a deep learning-based auto-segmentation model for OAR/CTVs for patients with cervical cancer undergoing radiotherapy and to evaluate the model's feasibility and efficacy with not only geometric indices but also comprehensive clinical evaluation.Materials and methodsA total of 180 abdominopelvic computed tomography images were included (training set, 165; validation set, 15). Geometric indices such as the Dice similarity coefficient (DSC) and the 95% Hausdorff distance (HD) were analyzed. A Turing test was performed and physicians from other institutions were asked to delineate contours with and without using auto-segmented contours to assess inter-physician heterogeneity and contouring time.ResultsThe correlation between the manual and auto-segmented contours was acceptable for the anorectum, bladder, spinal cord, cauda equina, right and left femoral heads, bowel bag, uterocervix, liver, and left and right kidneys (DSC greater than 0.80). The stomach and duodenum showed DSCs of 0.67 and 0.73, respectively. CTVs showed DSCs between 0.75 and 0.80. Turing test results were favorable for most OARs and CTVs. No auto-segmented contours had large, obvious errors. The median overall satisfaction score of the participating physicians was 7 out of 10. Auto-segmentation reduced heterogeneity and shortened contouring time by 30 min among radiation oncologists from different institutions. Most participants favored the auto-contouring system.ConclusionThe proposed deep learning-based auto-segmentation model may be an efficient tool for patients with cervical cancer undergoing radiotherapy. Although the current model may not completely replace humans, it can serve as a useful and efficient tool in real-world clinics.
| S-EPMC10175826 | biostudies-literature
Project description:Proper delineation of both target volumes and organs at risk is a crucial step in the radiation therapy workflow. This process is normally carried out manually by medical doctors, hence demanding timewise. To improve efficiency, auto-contouring methods have been proposed. We assessed a specific commercial software to investigate its impact on the radiotherapy workflow on four specific disease sites: head and neck, prostate, breast, and rectum. For the present study, we used a commercial deep learning-based auto-segmentation software, namely Limbus Contour (LC), Version 1.5.0 (Limbus AI Inc., Regina, SK, Canada). The software uses deep convolutional neural network models based on a U-net architecture, specific for each structure. Manual and automatic segmentation were compared on disease-specific organs at risk. Contouring time, geometrical performance (volume variation, Dice Similarity Coefficient-DSC, and center of mass shift), and dosimetric impact (DVH differences) were evaluated. With respect to time savings, the maximum advantage was seen in the setting of head and neck cancer with a 65%-time reduction. The average DSC was 0.72. The best agreement was found for lungs. Good results were highlighted for bladder, heart, and femoral heads. The most relevant dosimetric difference was in the rectal cancer case, where the mean volume covered by the 45 Gy isodose was 10.4 cm3 for manual contouring and 289.4 cm3 for automatic segmentation. Automatic contouring was able to significantly reduce the time required in the procedure, simplifying the workflow, and reducing interobserver variability. Its implementation was able to improve the radiation therapy workflow in our department.
| S-EPMC9782080 | biostudies-literature
Project description:PurposeContouring inconsistencies are known but understudied in clinical radiation therapy trials. We applied auto-contouring to the Radiation Therapy Oncology Group (RTOG) 0617 dose escalation trial data. We hypothesized that the trial heart doses were higher than reported due to inconsistent and insufficient heart segmentation. We tested our hypothesis by comparing doses between deep-learning (DL) segmented hearts and trial hearts.Methods and materialsThe RTOG 0617 data were downloaded from The Cancer Imaging Archive; the 442 patients with trial hearts and dose distributions were included. All hearts were resegmented using our DL pipeline and quality assured to meet the requirements for clinical implementation. Dose (V5%, V30%, and mean heart dose) was compared between the 2 sets of hearts (Wilcoxon signed-rank test). Each dose metric was associated with overall survival (Cox proportional hazards). Lastly, 18 volume similarity metrics were assessed for the hearts and correlated with |DoseDL - DoseRTOG0617| (linear regression; significance: P ≤ .0028; corrected for 18 tests).ResultsDose metrics were significantly higher for DL hearts compared with trial hearts (eg, mean heart dose: 15 Gy vs 12 Gy; P = 5.8E-16). All 3 DL heart dose metrics were stronger overall survival predictors than those of the trial hearts (median, P = 2.8E-5 vs 2.0E-4). Thirteen similarity metrics explained |DoseDL - DoseRTOG0617|; the axial distance between the 2 centers of mass was the strongest predictor (CENTAxial; median, R2 = 0.47; P = 6.1E-62). CENTAxial agreed with the qualitatively identified inconsistencies in the superior direction. The trial's qualitative heart contouring score was not correlated with |DoseDL - DoseRTOG0617| (median, R2 = 0.01; P = .02) or with any of the similarity metrics (median, Rs = 0.13 [range, -0.22 to 0.31]).ConclusionsUsing a coherent heart definition, as enabled through our open-source DL algorithm, the trial heart doses in RTOG 0617 were found to be significantly higher than previously reported, which may have led to an even more rapid mortality accumulation. Auto-segmentation is likely to reduce contouring and dose inconsistencies and increase the quality of clinical RT trials.
| S-EPMC8729313 | biostudies-literature
Project description:Background and purposeAutomatic delineations are often used as a starting point in the radiotherapy contouring workflow, after which they are manually reviewed and adapted. The purpose of this work was to quantify the geometric differences between automatic and manually edited breast clinical target volume (CTV) contours and evaluate the dosimetric impact of such differences.Materials and methodsEighty-seven automatically generated and manually edited contours of the left breast were retrieved from our clinical database. The automatic contours were obtained with a commercial auto-segmentation toolbox. The geometrical comparison was performed both locally and globally using the Dice score and the 95% Hausdorff distance (HD). Two treatment plans were generated for each patient and the obtained dosimetric differences were quantified using dose-volume histogram (DVH) parameters in the lungs, heart and planning target volume (PTV). An inter-observer variability study with four observers was performed on a subset of ten patients.ResultsA median Dice score of 0.95 and a median 95% HD of 9.7 mm were obtained. Larger breasts were consistently under-contoured. Cranial under-contouring resulted in more than 5% relative decrease in PTV coverage in 15% of the patients while lateroposterior over-contouring increased the lung V20Gy by a maximum of 2%. The inter-observer variability of the PTV coverage was smaller than the difference between PTV coverage achieved by the automatic and the consensus contours.ConclusionsCranial under-contouring resulted in under-treatment, while lateroposterior over-contouring resulted in an increased lung dosage that is clinically irrelevant, showing the need to consider dose distributions to assess the clinical impact of local geometrical differences.
| S-EPMC7807634 | biostudies-literature
Project description:PurposeWe recently described the validation of deep learning-based auto-segmented contour (DC) models for organs at risk (OAR) and clinical target volumes (CTV). In this study, we evaluate the performance of implemented DC models in the clinical radiotherapy (RT) planning workflow and report on user experience.Methods and materialsDC models were implemented at two cancer centers and used to generate OAR and CTVs for all patients undergoing RT for a central nervous system (CNS), head and neck (H&N), or prostate cancer. Radiation Therapists/Dosimetrists and Radiation Oncologists completed post-contouring surveys rating the degree of edits required for DCs (1 = minimal, 5 = significant) and overall DC satisfaction (1 = poor, 5 = high). Unedited DCs were compared to the edited treatment approved contours using Dice similarity coefficient (DSC) and 95% Hausdorff distance (HD).ResultsBetween September 19, 2019 and March 6, 2020, DCs were generated on approximately 551 eligible cases. 203 surveys were collected on 27 CNS, 54 H&N, and 93 prostate RT plans, resulting in an overall survey compliance rate of 32%. The majority of OAR DCs required minimal edits subjectively (mean editing score ≤ 2) and objectively (mean DSC and 95% HD was ≥ 0.90 and ≤ 2.0 mm). Mean OAR satisfaction score was 4.1 for CNS, 4.4 for H&N, and 4.6 for prostate structures. Overall CTV satisfaction score (n = 25), which encompassed the prostate, seminal vesicles, and neck lymph node volumes, was 4.1.ConclusionsPreviously validated OAR DC models for CNS, H&N, and prostate RT planning required minimal subjective and objective edits and resulted in a positive user experience, although low survey compliance was a concern. CTV DC model evaluation was even more limited, but high user satisfaction suggests that they may have served as appropriate starting points for patient specific edits.
| S-EPMC8186196 | biostudies-literature
Project description:Purpose/objectivesAuto-segmentation with artificial intelligence (AI) offers an opportunity to reduce inter- and intra-observer variability in contouring, to improve the quality of contours, as well as to reduce the time taken to conduct this manual task. In this work we benchmark the AI auto-segmentation contours produced by five commercial vendors against a common dataset.Methods and materialsThe organ at risk (OAR) contours generated by five commercial AI auto-segmentation solutions (Mirada (Mir), MVision (MV), Radformation (Rad), RayStation (Ray) and TheraPanacea (Ther)) were compared to manually-drawn expert contours from 20 breast, 20 head and neck, 20 lung and 20 prostate patients. Comparisons were made using geometric similarity metrics including volumetric and surface Dice similarity coefficient (vDSC and sDSC), Hausdorff distance (HD) and Added Path Length (APL). To assess the time saved, the time taken to manually draw the expert contours, as well as the time to correct the AI contours, were recorded.ResultsThere are differences in the number of CT contours offered by each AI auto-segmentation solution at the time of the study (Mir 99; MV 143; Rad 83; Ray 67; Ther 86), with all offering contours of some lymph node levels as well as OARs. Averaged across all structures, the median vDSCs were good for all systems and compared favorably with existing literature: Mir 0.82; MV 0.88; Rad 0.86; Ray 0.87; Ther 0.88. All systems offer substantial time savings, ranging between: breast 14-20 mins; head and neck 74-93 mins; lung 20-26 mins; prostate 35-42 mins. The time saved, averaged across all structures, was similar for all systems: Mir 39.8 mins; MV 43.6 mins; Rad 36.6 min; Ray 43.2 mins; Ther 45.2 mins.ConclusionsAll five commercial AI auto-segmentation solutions evaluated in this work offer high quality contours in significantly reduced time compared to manual contouring, and could be used to render the radiotherapy workflow more efficient and standardized.
| S-EPMC10436522 | biostudies-literature
Project description:Background and purposeArtificial Intelligence (AI)-based auto-contouring for treatment planning in radiotherapy needs extensive clinical validation, including the impact of editing after automatic segmentation. The aims of this study were to assess the performance of a commercial system for Clinical Target Volumes (CTVs) (prostate/seminal vesicles) and selected Organs at Risk (OARs) (rectum/bladder/femoral heads + femurs), evaluating also inter-observer variability (manual vs automatic + editing) and the reduction of contouring time.Materials and methodsTwo expert observers contoured CTVs/OARs of 20 patients in our Treatment Planning System (TPS). Computed Tomography (CT) images were sent to the automatic contouring workstation: automatic contours were generated and sent back to TPS, where observers could edit them if necessary. Inter- and intra-observer consistency was estimated using Dice Similarity Coefficients (DSC). Radiation oncologists were also asked to score the quality of automatic contours, ranging from 1 (complete re-contouring) to 5 (no editing). Contouring times (manual vs automatic + edit) were compared.ResultsDSCs (manual vs automatic only) were consistent with inter-observer variability (between 0.65 for seminal vesicles and 0.94 for bladder); editing further improved performances (range: 0.76-0.94). The median clinical score was 4 (little editing) and it was <4 in 3/2 patients for the two observers respectively. Inter-observer variability of automatic + editing contours improved significantly, being lower than manual contouring (e.g.: seminal vesicles: 0.83vs0.73; prostate: 0.86vs0.83; rectum: 0.96vs0.81). Oncologist contouring time reduced from 17 to 24 min of manual contouring time to 3-7 min of editing time for the two observers (p < 0.01).ConclusionAutomatic contouring with a commercial AI-based system followed by editing can replace manual contouring, resulting in significantly reduced time for segmentation and better consistency between operators.
| S-EPMC10618761 | biostudies-literature
Project description:BackgroundAccurate delineation of clinical target volume of tumor bed (CTV-TB) is important but it is also challenging due to surgical effects and soft tissue contrast. Recently a few auto-segmentation methods were developed to improve the process. However, those methods had comparatively low segmentation accuracy. In this study the prior information was introduced to aid auto-segmentation of CTV-TB based on a deep-learning model.MethodsTo aid the delineation of CTV-TB, the tumor contour on preoperative CT was transformed onto postoperative CT via deformable image registration. Both original and transformed tumor contours were used for prior information in training an auto-segmentation model. Then, the CTV-TB contour on postoperative CT was predicted by the model. 110 pairs of preoperative and postoperative CT images were used with a 5-fold cross-validation strategy. The predicted contour was compared with the clinically approved contour for accuracy evaluation using dice similarity coefficient (DSC) and Hausdorff distance.ResultsThe average DSC of the deep-learning model with prior information was improved than the one without prior information (0.808 vs. 0.734, P < 0.05). The average DSC of the deep-learning model with prior information was higher than that of the traditional method (0.808 vs. 0.622, P < 0.05).ConclusionsThe introduction of prior information in deep-learning model can improve segmentation accuracy of CTV-TB. The proposed method provided an effective way to automatically delineate CTV-TB in postoperative breast cancer radiotherapy.
| S-EPMC10577969 | biostudies-literature