Dataset Information

Application of computer-aided detection for NCCN-based follow-up recommendation in subsolid nodules: Effect on inter-observer agreement.

ABSTRACT:

Rationale and objectives

Computer-aided detection (CAD) of pulmonary nodules reduces the impact of observer variability, improving the reliability and reproducibility of nodule assessments in clinical practice. Therefore, this study aimed to assess the impact of CAD on inter-observer agreement in the follow-up management of subsolid nodules.

Materials and methods

A dataset comprising 60 subsolid nodule cases was constructed based on the National Cancer Center lung cancer screening data. Five observers independently assessed all low-dose computed tomography scans and assigned follow-up management strategies to each case according to the National Comprehensive Cancer Network (NCCN) guidelines, using both manual measurements and CAD assistance. The linearly weighted Cohen's kappa test was used to measure agreement between paired observers. Agreement among multiple observers was evaluated using the Fleiss kappa statistic.

Results

The agreement of the five observers for NCCN follow-up management categorization was moderate when measured manually, with a Fleiss kappa score of 0.437. Utilizing CAD led to a notable enhancement in agreement, achieving a substantial consensus with a Fleiss kappa value of 0.623. After using CAD, the proportion of major and substantial management discrepancies decreased from 27.5% to 15.8% and 4.8% to 1.5%, respectively (p < 0.01). In 23 lung cancer cases presenting as part-solid nodules, CAD significantly elevates the average sensitivity in detecting lung cancer cases presenting as part-solid nodules (overall sensitivity, 82.6% vs. 92.2%; p < 0.05).

Conclusion

The application of CAD significantly improves inter-observer agreement in the follow-up management strategy for subsolid nodules. It also demonstrates the potential to reduce substantial management discrepancies and increase detection sensitivity in lung cancer cases presenting as part-solid nodules.

SUBMITTER: Quanyang W

PROVIDER: S-EPMC10832308 | biostudies-literature | 2024 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Application of computer-aided detection for NCCN-based follow-up recommendation in subsolid nodules: Effect on inter-observer agreement.

Quanyang Wu W Lina Zhou Z Yao Huang H Jiawei Wang W Wei Tang T Linlin Qi Q Zewei Zhang Z Donghui Hou H Hongjia Li L Shuluan Chen C Jiaxing Zhang Z Shijun Zhao Z

Cancer medicine 20240101 2

<h4>Rationale and objectives</h4>Computer-aided detection (CAD) of pulmonary nodules reduces the impact of observer variability, improving the reliability and reproducibility of nodule assessments in clinical practice. Therefore, this study aimed to assess the impact of CAD on inter-observer agreement in the follow-up management of subsolid nodules.<h4>Materials and methods</h4>A dataset comprising 60 subsolid nodule cases was constructed based on the National Cancer Center lung cancer screening ...[more]

PMID: 38348960

Similar Datasets

Project description:BackgroundFollow-up management of pulmonary nodules is a crucial component of lung cancer screening. Consistency in follow-up recommendations is essential for effective lung cancer screening. This study aimed to assess inter-observer agreement on National Comprehensive Cancer Network (NCCN) guideline-based follow-up recommendation for subsolid nodules from low-dose computed tomography (LDCT) screening.MethodsA retrospective collection of LDCT reports from 2014 to 2017 for lung cancer screening was conducted using the Radiology Information System and keyword searches, focusing on subsolid nodules. A total of 110 LDCT cases containing subsolid nodules were identified. Two senior radiologists provided standardized follow-up recommendation. Follow-up recommendation was categorized into four groups (0-, 3-, 6-, and 12-month). To ensure overall balance and representativeness of the follow-up categories, 60 scans from 60 participants were included (distribution ratio 1:1:2:2). Cases were categorised into follow-up recommendation groups by five observers following NCCN guidelines. Fleiss' kappa statistic was used to evaluate inter-observer agreement.ResultsOverall accuracy rate for follow-up recommendation among five observers was 72.3%. Chest radiologists' overall agreement was significantly higher than radiology residents (P<0.01). The overall agreement among the five observers was moderate, with a Fleiss' kappa of 0.437. For all paired readers, the mean Cohen's kappa value was 0.603, with 95% confidence interval (CI) from 0.489 to 0.716. Chest radiologists demonstrated substantial agreement, evidenced by a Cohen's kappa of 0.655 (95% CI: 0.503-0.807). In contrast, the mean Cohen's kappa among radiology residents was 0.533 (95% CI: 0.501-0.565). The majority of cases with discrepancies, accounting for 73.5%, were associated with the same risk-dominant nodules. A higher proportion of part-solid nodule was a risk factor for discrepancies. Of the 600 paired readings, major discrepancies and substantial discrepancies were observed in 27.5% and 4.8% (29/600) of the cases.ConclusionsIn subsolid nodules, category evaluation of observer follow-up recommendation based on NCCN guidelines achieved moderate consistency. Disagreements were mainly caused by measurement and type disagreements of identical risk-dominant nodules. Part-solid nodule was a contributor for discrepancies in follow-up recommendation. Major and substantial management discrepancies were 27.5% and 4.8% in the paired evaluations.

Project description:BackgroundRecent developments in diagnostic techniques for malaria, particularly DNA probes and sero-immunology, have raised questions as to how these techniques might be used to facilitate malaria diagnosis at the most peripheral levels of the primary health care system. At present, malaria diagnosis is based on the standard microscopic examination of blood films in most field epidemiologic studies and is likely to remain so in the immediate future in Africa. The objective of this study was to assess inter-observer agreement for the examination of Giemsa-stained slides for Plasmodium falciparum parasites.MethodsChildren aged 0 to 10 years were enrolled yearly in Bancoumana village (West Africa), mainly during the transmission season (June to October). The blood smears obtained from the persistently negative children in June 1996, August 1996, October 1996 and March 1997 were systematically re-examined. A stratified random sample (10%) proportional to the following parasite density classes 1-100, 101-5000, and 5001 and over was taken from the slides collected. The kappa statistics and the intra-class correlation were used as measures of agreement the first and the second slide examinations.ResultsThe weighted kappa statistic, widely used as a chance-corrected measure for nominal agreement, showed excellent inter-observer agreement (κ(w)=0.7926; 95% CI [0.7588, 0.8263]; p=0.01). The intra-class correlation co-efficient had the same value of 0.7926 confirming the appropriateness of the weighted kappa statistic. Inter-observer agreement for slides read as negative by one observer, or as containing more than 100 parasites per μl, was excellent: 97% (493/506) and 92% (145/158), respectively. In contrast, the inter-observer agreement for slides read by one observer as containing 1-100 parasites/μl was poor, 36% (96/268).ConclusionsIn field conditions in Mali, there was a high reproducibility for slides reported as negative or as having more than 100 parasites per μl. However, smears with readings of 1-100 parasites per μl were less reproducible and should be re-examined carefully.

Project description:Invasive lobular breast carcinoma (ILC) is the second most common breast carcinoma (BC) subtype and is mainly driven by loss of E-cadherin expression. Correct classification of BC as ILC is important for patient treatment. This study assessed the degree of agreement among pathologists for the diagnosis of ILC. Two sets of hormone receptor (HR)-positive/HER2-negative BCs were independently reviewed by participating pathologists. In set A (61 cases), participants were provided with hematoxylin/eosin (HE)-stained sections. In set B (62 cases), participants were provided with HE-stained sections and E-cadherin immunohistochemistry (IHC). Tumor characteristics were balanced. Participants classified specimens as non-lobular BC versus mixed BC versus ILC. Pairwise inter-observer agreement and agreement with a pre-defined reference diagnosis were determined with Cohen's kappa statistics. Subtype calls were correlated with molecular features, including CDH1/E-cadherin mutation status. Thirty-five pathologists completed both sets, providing 4,305 subtype calls. Pairwise inter-observer agreement was moderate in set A (median κ = 0.58, interquartile range [IQR]: 0.48-0.66) and substantial in set B (median κ = 0.75, IQR: 0.56-0.86, p < 0.001). Agreement with the reference diagnosis was substantial in set A (median κ = 0.67, IQR: 0.57-0.75) and almost perfect in set B (median κ = 0.86, IQR: 0.73-0.93, p < 0.001). The median frequency of CDH1/E-cadherin mutations in specimens classified as ILC was 65% in set A (IQR: 56-72%) and 73% in set B (IQR: 65-75%, p < 0.001). Cases with variable subtype calls included E-cadherin-positive ILCs harboring CDH1 missense mutations, and E-cadherin-negative ILCs with tubular elements and focal P-cadherin expression. ILCs with trabecular growth pattern were often misclassified as non-lobular BC in set A but not in set B. In conclusion, subtyping of BC as ILC achieves almost perfect agreement with a pre-defined reference standard, if assessment is supported by E-cadherin IHC. CDH1 missense mutations associated with preserved E-cadherin protein expression, E- to P-cadherin switching in ILC with tubular elements, and trabecular ILC were identified as potential sources of discordant classification.

Dataset Information

Application of computer-aided detection for NCCN-based follow-up recommendation in subsolid nodules: Effect on inter-observer agreement.

Rationale and objectives

Materials and methods

Results

Conclusion

Publications

Application of computer-aided detection for NCCN-based follow-up recommendation in subsolid nodules: Effect on inter-observer agreement.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets