Dataset Information

AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancer.

ABSTRACT: The Ki-67 proliferation index (PI) guides treatment decisions in breast cancer but suffers from poor inter-rater reproducibility. Although AI tools have been designed for Ki-67 assessment, their impact on pathologists' work remains understudied. 90 international pathologists were recruited to assess the Ki-67 PI of ten breast cancer tissue microarrays with and without AI. Accuracy, agreement, and turnaround time with and without AI were compared. Pathologists' perspectives on AI were collected. Using AI led to a significant decrease in PI error (2.1% with AI vs. 5.9% without AI, p < 0.001), better inter-rater agreement (ICC: 0.70 vs. 0.92; Krippendorff's α: 0.63 vs. 0.89; Fleiss' Kappa: 0.40 vs. 0.86), and an 11.9% overall median reduction in turnaround time. Most pathologists (84%) found the AI reliable. For Ki-67 assessments, 76% of respondents believed AI enhances accuracy, 82% said it improves consistency, and 83% trust it will improve efficiency. This study highlights AI's potential to standardize Ki-67 scoring, especially between 5 and 30% PI-a range with low PI agreement. This could pave the way for a universally accepted PI score to guide treatment decisions, emphasizing the promising role of AI integration into pathologist workflows.

SUBMITTER: Dy A

PROVIDER: S-EPMC10787826 | biostudies-literature | 2024 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancer.

Dy Amanda A Nguyen Ngoc-Nhu Jennifer NJ Meyer Julien J Dawe Melanie M Shi Wei W Androutsos Dimitri D Fyles Anthony A Liu Fei-Fei FF Done Susan S Khademi April A

Scientific reports 20240113 1

The Ki-67 proliferation index (PI) guides treatment decisions in breast cancer but suffers from poor inter-rater reproducibility. Although AI tools have been designed for Ki-67 assessment, their impact on pathologists' work remains understudied. 90 international pathologists were recruited to assess the Ki-67 PI of ten breast cancer tissue microarrays with and without AI. Accuracy, agreement, and turnaround time with and without AI were compared. Pathologists' perspectives on AI were collected. ...[more]

PMID: 38218973

Similar Datasets

Project description:BackgroundThe percentage of cells staining positive for Ki67 is sometimes used for decision-making in patients with early invasive breast cancer (IBC). However, there is uncertainty regarding the most appropriate Ki67 cut points and the influence of interlaboratory measurement variability. We examined the relationship between breast cancer mortality and Ki67 both before and after accounting for interlaboratory variability and 8 patient and tumor characteristics.MethodsA multicenter cohort study of women with early IBC diagnosed during 2009-2016 in more than 20 NHS hospitals in England and followed until December 31, 2020.ResultsKi67 was strongly prognostic of breast cancer mortality in 8212 women with estrogen receptor (ER)-positive, human epidermal growth factor receptor 2 (HER2)-negative early IBC (Ptrend < .001). This relationship remained strong after adjustment for patient and tumor characteristics (Ptrend < .001). Standardization for interlaboratory variability did little to alter these results. For women with Ki67 scores of 0%-5%, 6%-10%, 11%-19%, and 20%-29% the corresponding 8-year adjusted cumulative breast cancer mortality risks were 3.3% (95% confidence interval [CI] = 2.8% to 4.0%), 3.7% (95% CI = 3.0% to 4.4%), 3.4% (95% CI = 2.8% to 4.1%), and 3.4% (95% CI = 2.8% to 4.1%), whereas for women with Ki67 scores of 30%-39% and 40%-100%, these risks were higher, at 5.1% (95% CI = 4.3% to 6.2%) and 7.7% (95% CI = 6.6% to 9.1) (Ptrend < .001). Similar results were obtained when the adjusted analysis was repeated with omission of pathological information about tumor size and nodal involvement, which would not be available preoperatively for patients being considered for neoadjuvant therapy.ConclusionOur findings confirm the prognostic value of Ki67 scores of 30% or more in women with ER-positive, HER2-negative early IBC, irrespective of interlaboratory variability. These results also suggest that Ki67 may be useful to aid decision-making in the neoadjuvant setting.

Project description:A common problem in the assessment of Ki67 proliferative index in well-differentiated gastrointestinal neuroendocrine tumors is distinguishing tumor from non-tumor. This is because background stromal lymphocytes, entrapped non-neoplastic glands, and the delicate vascular network characteristic of neuroendocrine tumors frequently contain a subset of proliferating cells. Furthermore, in small biopsies, crush and cautery artifact can alter the morphologic appearance of tumor cells, making the Ki67 proliferative index more difficult to assess. To address these issues, we developed a synaptophysin-Ki67 double stain using a commercially available immunohistochemistry kit, allowing simultaneous visualization of tumor and proliferating nuclei. To test this method, three gastrointestinal pathologists individually graded 50 gastrointestinal neuroendocrine tumors first using synaptophysin-Ki67 double-stained slides and then, after a washout period, using Ki67-only stained slides (along with routine hematoxylin- and eosin-stained slides). Interobserver agreement on Ki67 proliferative index was moderate using the Ki67-only stained slides (intraclass correlation 0.51, 95% confidence interval: 0.35-0.66) and improved using the synaptophysin-Ki67 double stain (intraclass correlation 0.79, 95% confidence interval: 0.69-0.86). Similarly, interobserver agreement on tumor grade was fair with Ki67-only stained slides (κ=0.39, P<0.001) and improved with the double stain (κ=0.58, P<0.001). Analysis of individual pathologists' scores revealed that fewer total number of tumor cells counted correlated with higher grade designation and appeared to contribute to grade discordance. In tumors cited as particularly challenging to assess by the pathologists, three of four tumors were grade discordant with the Ki67-only stain, whereas all four tumors were grade concordant with the synaptophysin-Ki67 stain. The synaptophysin-Ki67 double stain is the first technique to address specifically the histomorphologic challenges of evaluating Ki67 proliferative index in well-differentiated gastrointestinal neuroendocrine tumors. Although further validation is needed, this study provides evidence that the synaptophysin-Ki67 double stain can improve interobserver agreement.

Project description:BackgroundArtificial intelligence (AI) systems are proposed as a replacement of the first reader in double reading within mammography screening. We aimed to assess cancer detection accuracy of an AI system in a Danish screening population.MethodsWe retrieved a consecutive screening cohort from the Region of Southern Denmark including all participating women between Aug 4, 2014, and August 15, 2018. Screening mammograms were processed by a commercial AI system and detection accuracy was evaluated in two scenarios, Standalone AI and AI-integrated screening replacing first reader, with first reader and double reading with arbitration (combined reading) as comparators, respectively. Two AI-score cut-off points were applied by matching at mean first reader sensitivity (AIsens) and specificity (AIspec). Reference standard was histopathology-proven breast cancer or cancer-free follow-up within 24 months. Coprimary endpoints were sensitivity and specificity, and secondary endpoints were positive predictive value (PPV), negative predictive value (NPV), recall rate, and arbitration rate. Accuracy estimates were calculated using McNemar's test or exact binomial test.ResultsOut of 272,008 screening mammograms from 158,732 women, 257,671 (94.7%) with adequate image data were included in the final analyses. Sensitivity and specificity were 63.7% (95% CI 61.6%-65.8%) and 97.8% (97.7-97.8%) for first reader, and 73.9% (72.0-75.8%) and 97.9% (97.9-98.0%) for combined reading, respectively. Standalone AIsens showed a lower specificity (-1.3%) and PPV (-6.1%), and a higher recall rate (+ 1.3%) compared to first reader (p < 0.0001 for all), while Standalone AIspec had a lower sensitivity (-5.1%; p < 0.0001), PPV (-1.3%; p = 0.01) and NPV (-0.04%; p = 0.0002). Compared to combined reading, Integrated AIsens achieved higher sensitivity (+ 2.3%; p = 0.0004), but lower specificity (-0.6%) and PPV (-3.9%) as well as higher recall rate (+ 0.6%) and arbitration rate (+ 2.2%; p < 0.0001 for all). Integrated AIspec showed no significant difference in any outcome measures apart from a slightly higher arbitration rate (p < 0.0001). Subgroup analyses showed higher detection of interval cancers by Standalone AI and Integrated AI at both thresholds (p < 0.0001 for all) with a varying composition of detected cancers across multiple subgroups of tumour characteristics.ConclusionsReplacing first reader in double reading with an AI could be feasible but choosing an appropriate AI threshold is crucial to maintaining cancer detection accuracy and workload.

Dataset Information

AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancer.

Publications

AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancer.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets