Dataset Information

Computer-aided interpretation of chest radiography reveals the spectrum of tuberculosis in rural South Africa.

ABSTRACT: Computer-aided digital chest radiograph interpretation (CAD) can facilitate high-throughput screening for tuberculosis (TB), but its use in population-based active case-finding programs has been limited. In an HIV-endemic area in rural South Africa, we used a CAD algorithm (CAD4TBv5) to interpret digital chest x-rays (CXR) as part of a mobile health screening effort. Participants with TB symptoms or CAD4TBv5 score above the triaging threshold were referred for microbiological sputum assessment. During an initial pilot phase, a low CAD4TBv5 triaging threshold of 25 was selected to maximize TB case finding. We report the performance of CAD4TBv5 in screening 9,914 participants, 99 (1.0%) of whom were found to have microbiologically proven TB. CAD4TBv5 was able to identify TB cases at the same sensitivity but lower specificity as a blinded radiologist, whereas the next generation of the algorithm (CAD4TBv6) achieved comparable sensitivity and specificity to the radiologist. The CXRs of people with microbiologically confirmed TB spanned a range of lung field abnormality, including 19 (19.2%) cases deemed normal by the radiologist. HIV serostatus did not impact CAD4TB's performance. Notably, 78.8% of the TB cases identified during this population-based survey were asymptomatic and therefore triaged for sputum collection on the basis of CAD4TBv5 score alone. While CAD4TBv6 has the potential to replace radiologists for triaging CXRs in TB prevalence surveys, population-specific piloting is necessary to set the appropriate triaging thresholds. Further work on image analysis strategies is needed to identify radiologically subtle active TB.

SUBMITTER: Fehr J

PROVIDER: S-EPMC8253848 | biostudies-literature | 2021 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Computer-aided interpretation of chest radiography reveals the spectrum of tuberculosis in rural South Africa.

Fehr Jana J Konigorski Stefan S Olivier Stephen S Gunda Resign R Surujdeen Ashmika A Gareta Dickman D Smit Theresa T Baisley Kathy K Moodley Sashen S Moosa Yumna Y Hanekom Willem W Koole Olivier O Ndung'u Thumbi T Pillay Deenan D Grant Alison D AD Siedner Mark J MJ Lippert Christoph C Wong Emily B EB

NPJ digital medicine 20210702 1

Computer-aided digital chest radiograph interpretation (CAD) can facilitate high-throughput screening for tuberculosis (TB), but its use in population-based active case-finding programs has been limited. In an HIV-endemic area in rural South Africa, we used a CAD algorithm (CAD4TBv5) to interpret digital chest x-rays (CXR) as part of a mobile health screening effort. Participants with TB symptoms or CAD4TBv5 score above the triaging threshold were referred for microbiological sputum assessment. ...[more]

PMID: 34215836

Similar Datasets

Project description:BackgroundComputer-aided detection (CAD) can help identify people with active tuberculosis left undetected. However, few studies have compared the performance of commercially available CAD products for screening in high tuberculosis and high HIV settings, and there is poor understanding of threshold selection across products in different populations. We aimed to compare CAD products' performance, with further analyses on subgroup performance and threshold selection.MethodsWe evaluated 12 CAD products on a case-control sample of participants from a South African tuberculosis prevalence survey. Only those with microbiological test results were eligible. The primary outcome was comparing products' accuracy using the area under the receiver operating characteristic curve (AUC) against microbiological evidence. Threshold analyses were performed based on pre-defined criteria and across all thresholds. We conducted subgroup analyses including age, gender, HIV status, previous tuberculosis history, symptoms presence, and current smoking status.FindingsOf the 774 people included, 516 were bacteriologically negative and 258 were bacteriologically positive. Diverse accuracy was noted: Lunit and Nexus had AUCs near 0·9, followed by qXR, JF CXR-2, InferRead, Xvision, and ChestEye (AUCs 0·8-0·9). XrayAME, RADIFY, and TiSepX-TB had AUC under 0·8. Thresholds varied notably across these products and different versions of the same products. Certain products (Lunit, Nexus, JF CXR-2, and qXR) maintained high sensitivity (>90%) across a wide threshold range while reducing the number of individuals requiring confirmatory diagnostic testing. All products generally performed worst in older individuals, people with previous tuberculosis, and people with HIV. Variations in thresholds, sensitivity, and specificity existed across groups and settings.InterpretationSeveral previously unevaluated products performed similarly to those evaluated by WHO. Thresholds differed across products and demographic subgroups. The rapid emergence of products and versions necessitates a global strategy to validate new versions and software to support CAD product and threshold selections.FundingGovernment of Canada.

Project description:Diagnostic tools for paediatric tuberculosis remain limited, with heavy reliance on clinical algorithms which include chest x-ray. Computer aided detection (CAD) for tuberculosis on chest x-ray has shown promise in adults. We aimed to measure and optimise the performance of an adult CAD system, CAD4TB, to identify tuberculosis on chest x-rays from children with presumptive tuberculosis. Chest x-rays from 620 children <13 years enrolled in a prospective observational diagnostic study in South Africa, were evaluated. All chest x-rays were read by a panel of expert readers who attributed each with a radiological reference of either 'tuberculosis' or 'not tuberculosis'. Of the 525 chest x-rays included in this analysis, 80 (40 with a reference of 'tuberculosis' and 40 with 'not tuberculosis') were allocated to an independent test set. The remainder made up the training set. The performance of CAD4TB to identify 'tuberculosis' versus 'not tuberculosis' on chest x-ray against the radiological reference read was calculated. The CAD4TB software was then fine-tuned using the paediatric training set. We compared the performance of the fine-tuned model to the original model. Our findings were that the area under the receiver operating characteristic curve (AUC) of the original CAD4TB model, prior to fine-tuning, was 0.58. After fine-tuning there was an improvement in the AUC to 0.72 (p = 0.0016). In this first-ever description of the use of CAD to identify tuberculosis on chest x-ray in children, we demonstrate a significant improvement in the performance of CAD4TB after fine-tuning with a set of well-characterised paediatric chest x-rays. CAD has the potential to be a useful additional diagnostic tool for paediatric tuberculosis. We recommend replicating the methods we describe using a larger chest x-ray dataset from a more diverse population and evaluating the potential role of CAD to replace a human-read chest x-ray within treatment-decision algorithms for paediatric tuberculosis.

Project description:BackgroundChest radiography to diagnose and screen for pulmonary tuberculosis has limitations, especially due to inter-reader variability. Automating the interpretation has the potential to overcome this drawback and to deliver objective and reproducible results. The CAD4TB software is a computer-aided detection system that has shown promising preliminary findings. Evaluation studies in different settings are needed to assess diagnostic accuracy and practicability of use.MethodsCAD4TB was evaluated on chest radiographs of patients with symptoms suggestive of pulmonary tuberculosis enrolled in two cohort studies in Tanzania. All patients were characterized by sputum smear microscopy and culture including subsequent antigen or molecular confirmation of Mycobacterium tuberculosis (M.tb) to determine the reference standard. Chest radiographs were read by the software and two human readers, one expert reader and one clinical officer. The sensitivity and specificity of CAD4TB was depicted using receiver operating characteristic (ROC) curves, the area under the curve calculated and the performance of the software compared to the results of human readers.ResultsOf 861 study participants, 194 (23%) were culture-positive for M.tb. The area under the ROC curve of CAD4TB for the detection of culture-positive pulmonary tuberculosis was 0.84 (95% CI 0.80-0.88). CAD4TB was significantly more accurate for the discrimination of smear-positive cases against non TB patients than for smear-negative cases (p-value<0.01). It differentiated better between TB cases and non TB patients among HIV-negative compared to HIV-positive individuals (p<0.01). CAD4TB significantly outperformed the clinical officer, but did not reach the accuracy of the expert reader (p = 0.02), for a tuberculosis specific reading threshold.ConclusionCAD4TB accurately distinguished between the chest radiographs of culture-positive TB cases and controls. Further studies on cost-effectiveness, operational and ethical aspects should determine its place in diagnostic and screening algorithms.

Project description:BackgroundIn 2022, fewer than half of persons with tuberculosis (TB) had access to molecular diagnostic tests for TB due to their high costs. Studies have found that the use of artificial intelligence (AI) software for chest X-ray (CXR) interpretation and sputum specimen pooling can each reduce the cost of testing. We modeled the combination of both strategies to estimate potential savings in consumables that could be used to expand access to molecular diagnostics.MethodsWe obtained Xpert testing and positivity data segmented into deciles by AI probability scores for TB from the community- and healthcare facility-based active case finding conducted in Bangladesh, Nigeria, Viet Nam, and Zambia. AI scores in the model were based on CAD4TB version 7 (Zambia) and qXR (all other countries). We modeled four ordinal screening and testing approaches involving AI-aided CXR interpretation to indicate individual and pooled testing. Setting a false negative rate of 5%, for each approach we calculated additional and cumulative savings over the baseline of universal Xpert testing, as well as the theoretical expansion in diagnostic coverage.ResultsIn each country, the optimal screening and testing approach was to use AI to rule out testing in deciles with low AI scores and to guide pooled vs individual testing in persons with moderate and high AI scores, respectively. This approach yielded cumulative savings in Xpert tests over baseline ranging from 50.8% in Zambia to 57.5% in Nigeria and 61.5% in Bangladesh and Viet Nam. Using these savings, diagnostic coverage theoretically could be expanded by 34% to 160% across the different approaches and countries.ConclusionsUsing AI software data generated during CXR interpretation to inform a differentiated pooled testing strategy may optimize TB diagnostic test use, and could extend molecular tests to more people who need them. The optimal AI thresholds and pooled testing strategy varied across countries, which suggests that bespoke screening and testing approaches may be needed for differing populations and settings.Supplementary informationThe online version contains supplementary material available at 10.1186/s44263-024-00081-2.

Project description:Computer-aided detection (CAD) was recently recommended by the WHO for TB screening and triage based on several evaluations, but unlike traditional diagnostic tests, software versions are updated frequently and require constant evaluation. Since then, newer versions of two of the evaluated products have already been released. We used a case control sample of 12,890 chest X-rays to compare performance and model the programmatic effect of upgrading to newer versions of CAD4TB and qXR. We compared the area under the receiver operating characteristic curve (AUC), overall, and with data stratified by age, TB history, gender, and patient source. All versions were compared against radiologist readings and WHO’s Target Product Profile (TPP) for a TB triage test. Both newer versions significantly outperformed their predecessors in terms of AUC: CAD4TB version 6 (0.823 [0.816–0.830]), version 7 (0.903 [0.897–0.908]) and qXR version 2 (0.872 [0.866–0.878]), version 3 (0.906 [0.901–0.911]). Newer versions met WHO TPP values, older versions did not. All products equalled or surpassed the human radiologist performance with improvements in triage ability in newer versions. Humans and CAD performed worse in older age groups and among those with TB history. New versions of CAD outperform their predecessors. Prior to implementation CAD should be evaluated using local data because underlying neural networks can differ significantly. An independent rapid evaluation centre is necessitated to provide implementers with performance data on new versions of CAD products as they are developed. Author summary The World Health Organization recommended the use of artificial intelligence (AI)-powered computer-aided detection (CAD) for TB screening and triage in 2021. One year on, we comprehensively compare the performance of the newest versions of two CAD (CAD4TB and qXR) to their WHO-evaluated predecessors. We found that both newer versions significantly improved upon their predecessor’s ability to detect TB, performing better than the human readers. We also showed that the AI underlying new software versions can differ remarkably from the old and resemble an entirely new product altogether. We further demonstrate that, unlike laboratory diagnostic tools, CAD software updates could significantly impact the selection of appropriate threshold scores, the number of people with TB detected and cost-effectiveness. With newer CAD versions being rolled out almost annually, our results therefore underscore the need for rapid evidence generation to evaluate newer CAD versions in the fast-growing medical AI industry.

Project description:In general, chest radiographs (CXR) have high sensitivity and moderate specificity for active pulmonary tuberculosis (PTB) screening when interpreted by human readers. However, they are challenging to scale due to hardware costs and the dearth of professionals available to interpret CXR in low-resource, high PTB burden settings. Recently, several computer-aided detection (CAD) programs have been developed to facilitate automated CXR interpretation. We conducted a retrospective case-control study to assess the diagnostic accuracy of a CAD software (qXR, Qure.ai, Mumbai, India) using microbiologically-confirmed PTB as the reference standard. To assess overall accuracy of qXR, receiver operating characteristic (ROC) analysis was used to determine the area under the curve (AUC), along with 95% confidence intervals (CI). Kappa coefficients, and associated 95% CI, were used to investigate inter-rater reliability of the radiologists for detection of specific chest abnormalities. In total, 317 cases and 612 controls were included in the analysis. The AUC for qXR for the detection of microbiologically-confirmed PTB was 0.81 (95% CI: 0.78, 0.84). Using the threshold that maximized sensitivity and specificity of qXR simultaneously, the software achieved a sensitivity and specificity of 71% (95% CI: 66%, 76%) and 80% (95% CI: 77%, 83%), respectively. The sensitivity and specificity of radiologists for the detection of microbiologically-confirmed PTB was 56% (95% CI: 50%, 62%) and 80% (95% CI: 77%, 83%), respectively. For detection of key PTB-related abnormalities 'pleural effusion' and 'cavity', qXR achieved an AUC of 0.94 (95% CI: 0.92, 0.96) and 0.84 (95% CI: 0.82, 0.87), respectively. For the other abnormalities, the AUC ranged from 0.75 (95% CI: 0.70, 0.80) to 0.94 (95% CI: 0.91, 0.96). The controls had a high prevalence of other lung diseases which can cause radiological manifestations similar to PTB (e.g., 26% had pneumonia, 15% had lung malignancy, etc.). In a tertiary hospital in India, qXR demonstrated moderate sensitivity and specificity for the detection of PTB. There is likely a larger role for CAD software as a triage test for PTB at the primary care level in settings where access to radiologists in limited. Larger prospective studies that can better assess heterogeneity in important subgroups are needed.

Dataset Information

Computer-aided interpretation of chest radiography reveals the spectrum of tuberculosis in rural South Africa.

Publications

Computer-aided interpretation of chest radiography reveals the spectrum of tuberculosis in rural South Africa.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets