Dataset Information

Clinical deployment environments: Five pillars of translational machine learning for health.

ABSTRACT: Machine Learning for Health (ML4H) has demonstrated efficacy in computer imaging and other self-contained digital workflows, but has failed to substantially impact routine clinical care. This is no longer because of poor adoption of Electronic Health Records Systems (EHRS), but because ML4H needs an infrastructure for development, deployment and evaluation within the healthcare institution. In this paper, we propose a design pattern called a Clinical Deployment Environment (CDE). We sketch the five pillars of the CDE: (1) real world development supported by live data where ML4H teams can iteratively build and test at the bedside (2) an ML-Ops platform that brings the rigour and standards of continuous deployment to ML4H (3) design and supervision by those with expertise in AI safety (4) the methods of implementation science that enable the algorithmic insights to influence the behaviour of clinicians and patients and (5) continuous evaluation that uses randomisation to avoid bias but in an agile manner. The CDE is intended to answer the same requirements that bio-medicine articulated in establishing the translational medicine domain. It envisions a transition from "real-world" data to "real-world" development.

SUBMITTER: Harris S

PROVIDER: S-EPMC9437594 | biostudies-literature | 2022

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Clinical deployment environments: Five pillars of translational machine learning for health.

Harris Steve S Bonnici Tim T Keen Thomas T Lilaonitkul Watjana W White Mark J MJ Swanepoel Nel N

Frontiers in digital health 20220819

Machine Learning for Health (ML4H) has demonstrated efficacy in computer imaging and other self-contained digital workflows, but has failed to substantially impact routine clinical care. This is no longer because of poor adoption of Electronic Health Records Systems (EHRS), but because ML4H needs an infrastructure for development, deployment and evaluation within the healthcare institution. In this paper, we propose a design pattern called a Clinical Deployment Environment (CDE). We sketch the f ...[more]

PMID: 36060542

Similar Datasets

Project description:BackgroundAttention-deficit/hyperactivity disorder (ADHD) is a prevalent mental disorder characterized by hyperactivity, impulsivity, and inattention. This study aims to develop a verifiable and interpretable machine learning model to identify ADHD and its subtypes in children using clinical assessment scales data.MethodsThis study utilized the ADHD-200 dataset, including demographic data, Behavioral Rating Scale, and Wechsler Intelligence Scale assessments, to train and validate our models. The model's performance was evaluated using 10-fold cross-validation within the internal dataset, and the best model will be used for external validation. Seven machine learning models were evaluated. The SHapley Additive exPlanations (SHAP) method was employed for model interpretation. Finally, a web application will deploy the prediction model to provide ADHD probabilities based on user input.ResultsThe Random Forest (RF) model performing best in identifying ADHD and the Support Vector Machine (SVM) model excelling in distinguishing ADHD subtypes. The RF model achieved an AUC of 0.99 in 10-fold cross-validation and an AUC of 0.99 in external validation, and the SVM model achieved a micro-average AUC of 0.96 and an accuracy of 0.83 in internal validation and a micro-average AUC of 0.96 and an accuracy of 0.85 in external validation. We used SHAP to interpret the models, revealing that higher ADHD Index pushed the model towards ADHD classification. Additionally, lower IQ scores were correlated with a higher likelihood of ADHD, consistent with previous studies. The dependency analysis found that the model can identify different behavioral scales. We deployed the final model online using a web application and showed users how the model made decisions.ConclusionsOur findings highlight the potential of using machine learning and clinical assessment scales to support the diagnosis and subtype identification of ADHD in children, offering a practical solution for improving diagnostic accuracy and efficiency in clinical settings.

Project description:ObjectivesTo simulate clinical deployment, evaluate performance, and establish quality assurance of a deep learning algorithm (U-Net) for detection, localization, and segmentation of clinically significant prostate cancer (sPC), ISUP grade group ≥ 2, using bi-parametric MRI.MethodsIn 2017, 284 consecutive men in active surveillance, biopsy-naïve or pre-biopsied, received targeted and extended systematic MRI/transrectal US-fusion biopsy, after examination on a single MRI scanner (3 T). A prospective adjustment scheme was evaluated comparing the performance of the Prostate Imaging Reporting and Data System (PI-RADS) and U-Net using sensitivity, specificity, predictive values, and the Dice coefficient.ResultsIn the 259 eligible men (median 64 [IQR 61-72] years), PI-RADS had a sensitivity of 98% [106/108]/84% [91/108] with a specificity of 17% [25/151]/58% [88/151], for thresholds at ≥ 3/≥ 4 respectively. U-Net using dynamic threshold adjustment had a sensitivity of 99% [107/108]/83% [90/108] (p > 0.99/> 0.99) with a specificity of 24% [36/151]/55% [83/151] (p > 0.99/> 0.99) for probability thresholds d3 and d4 emulating PI-RADS ≥ 3 and ≥ 4 decisions respectively, not statistically different from PI-RADS. Co-occurrence of a radiological PI-RADS ≥ 4 examination and U-Net ≥ d3 assessment significantly improved the positive predictive value from 59 to 63% (p = 0.03), on a per-patient basis.ConclusionsU-Net has similar performance to PI-RADS in simulated continued clinical use. Regular quality assurance should be implemented to ensure desired performance.Key points• U-Net maintained similar diagnostic performance compared to radiological assessment of PI-RADS ≥ 4 when applied in a simulated clinical deployment. • Application of our proposed prospective dynamic calibration method successfully adjusted U-Net performance within acceptable limits of the PI-RADS reference over time, while not being limited to PI-RADS as a reference. • Simultaneous detection by U-Net and radiological assessment significantly improved the positive predictive value on a per-patient and per-lesion basis, while the negative predictive value remained unchanged.

Dataset Information

Clinical deployment environments: Five pillars of translational machine learning for health.

Publications

Clinical deployment environments: Five pillars of translational machine learning for health.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets