Dataset Information

Development and validation of a machine learning-supported strategy of patient selection for osteoarthritis clinical trials: the IMI-APPROACH study.

ABSTRACT:

Objectives

To efficiently assess the disease-modifying potential of new osteoarthritis treatments, clinical trials need progression-enriched patient populations. To assess whether the application of machine learning results in patient selection enrichment, we developed a machine learning recruitment strategy targeting progressive patients and validated it in the IMI-APPROACH knee osteoarthritis prospective study.

Design

We designed a two-stage recruitment process supported by machine learning models trained to rank candidates by the likelihood of progression. First stage models used data from pre-existing cohorts to select patients for a screening visit. The second stage model used screening data to inform the final inclusion. The effectiveness of this process was evaluated using the actual 24-month progression.

Results

From 3500 candidate patients, 433 with knee osteoarthritis were screened, 297 were enrolled, and 247 completed the 2-year follow-up visit. We observed progression related to pain (P, 30%), structure (S, 13%), and combined pain and structure (P + S, 5%), and a proportion of non-progressors (N, 52%) ∼15% lower vs an unenriched population. Our model predicted these outcomes with AUC of 0.86 [95% CI, 0.81-0.90] for pain-related progression and AUC of 0.61 [95% CI, 0.52-0.70] for structure-related progression. Progressors were ranked higher than non-progressors for P + S (median rank 65 vs 143, AUC = 0.75), P (median rank 77 vs 143, AUC = 0.71), and S patients (median rank 107 vs 143, AUC = 0.57).

Conclusions

The machine learning-supported recruitment resulted in enriched selection of progressive patients. Further research is needed to improve structural progression prediction and assess this strategy in an interventional trial.

SUBMITTER: Widera P

PROVIDER: S-EPMC10463256 | biostudies-literature | 2023 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development and validation of a machine learning-supported strategy of patient selection for osteoarthritis clinical trials: the IMI-APPROACH study.

Widera Paweł P Welsing Paco M J PMJ Danso Samuel O SO Peelen Sjaak S Kloppenburg Margreet M Loef Marieke M Marijnissen Anne C AC van Helvoort Eefje M EM Blanco Francisco J FJ Magalhães Joana J Berenbaum Francis F Haugen Ida K IK Bay-Jensen Anne-Christine AC Mobasheri Ali A Ladel Christoph C Loughlin John J Lafeber Floris P J G FPJG Lalande Agnès A Larkin Jonathan J Weinans Harrie H Bacardit Jaume J

Osteoarthritis and cartilage open 20230818 4

<h4>Objectives</h4>To efficiently assess the disease-modifying potential of new osteoarthritis treatments, clinical trials need progression-enriched patient populations. To assess whether the application of machine learning results in patient selection enrichment, we developed a machine learning recruitment strategy targeting progressive patients and validated it in the IMI-APPROACH knee osteoarthritis prospective study.<h4>Design</h4>We designed a two-stage recruitment process supported by mach ...[more]

PMID: 37649530

Similar Datasets

Project description:BackgroundSepsis is a heterogeneous syndrome, and enrollment of more homogeneous patients is essential to improve the efficiency of clinical trials. Artificial intelligence (AI) has facilitated the identification of homogeneous subgroups, but how to estimate the uncertainty of the model outputs when applying AI to clinical decision-making remains unknown.ObjectiveWe aimed to design an AI-based model for purposeful patient enrollment, ensuring that a patient with sepsis recruited into a trial would still be persistently ill by the time the proposed therapy could impact patient outcome. We also expected that the model could provide interpretable factors and estimate the uncertainty of the model outputs at a customized confidence level.MethodsIn this retrospective study, 9135 patients with sepsis requiring vasopressor treatment within 24 hours after sepsis onset were enrolled from Beth Israel Deaconess Medical Center. This cohort was used for model development, and 10-fold cross-validation with 50 repeats was used for internal validation. In total, 3743 patients with sepsis from the eICU Collaborative Research Database were used as the external validation cohort. All included patients with sepsis were stratified based on disease progression trajectories: rapid death, recovery, and persistent ill. A total of 148 variables were selected for predicting the 3 trajectories. Four machine learning algorithms with 3 different setups were used. We estimated the uncertainty of the model outputs using conformal prediction (CP). The Shapley Additive Explanations method was used to explain the model.ResultsThe multiclass gradient boosting machine was identified as the best-performing model with good discrimination and calibration performance in both validation cohorts. The mean area under the receiver operating characteristic curve with SD was 0.906 (0.018) for rapid death, 0.843 (0.008) for recovery, and 0.807 (0.010) for persistent ill in the internal validation cohort. In the external validation cohort, the mean area under the receiver operating characteristic curve (SD) was 0.878 (0.003) for rapid death, 0.764 (0.008) for recovery, and 0.696 (0.007) for persistent ill. The maximum norepinephrine equivalence, total urine output, Acute Physiology Score III, mean systolic blood pressure, and the coefficient of variation of oxygen saturation contributed the most. Compared to the model without CP, using the model with CP at a mixed confidence approach reduced overall prediction errors by 27.6% (n=62) and 30.7% (n=412) in the internal and external validation cohorts, respectively, as well as enabled the identification of more potentially persistent ill patients.ConclusionsThe implementation of our model has the potential to reduce heterogeneity and enroll more homogeneous patients in sepsis clinical trials. The use of CP for estimating the uncertainty of the model outputs allows for a more comprehensive understanding of the model's reliability and assists in making informed decisions based on the predicted outcomes.

Project description:In the Innovative Medicine's Initiative Applied Public-Private Research enabling OsteoArthritis Clinical Headway (IMI-APPROACH) knee osteoarthritis (OA) study, machine learning models were trained to predict the probability of structural progression (s-score), predefined as >0.3 mm/year joint space width (JSW) decrease and used as inclusion criterion. The current objective was to evaluate predicted and observed structural progression over 2 years according to different radiographic and magnetic resonance imaging (MRI)-based structural parameters. Radiographs and MRI scans were acquired at baseline and 2-year follow-up. Radiographic (JSW, subchondral bone density, osteophytes), MRI quantitative (cartilage thickness), and MRI semiquantitative [SQ; cartilage damage, bone marrow lesions (BMLs), osteophytes] measurements were obtained. The number of progressors was calculated based on a change exceeding the smallest detectable change (SDC) for quantitative measures or a full SQ-score increase in any feature. Prediction of structural progression based on baseline s-scores and Kellgren-Lawrence (KL) grades was analyzed using logistic regression. Among 237 participants, around 1 in 6 participants was a structural progressor based on the predefined JSW-threshold. The highest progression rate was seen for radiographic bone density (39%), MRI cartilage thickness (38%), and radiographic osteophyte size (35%). Baseline s-scores could only predict JSW progression parameters (most P>0.05), while KL grades could predict progression of most MRI-based and radiographic parameters (P<0.05). In conclusion, between 1/6 and 1/3 of participants showed structural progression during 2-year follow-up. KL scores were observed to outperform the machine-learning-based s-scores as progression predictor. The large amount of data collected, and the wide range of disease stage, can be used for further development of more sensitive and successful (whole joint) prediction models. Trial Registration: Clinicaltrials.gov number NCT03883568.

Project description:ObjectivesThe IMI-APPROACH knee osteoarthritis study used machine learning (ML) to predict structural and/or pain progression, expressed by a structural (S) and pain (P) predicted-progression score, to select patients from existing cohorts. This study evaluates the actual 2-year progression within the IMI-APPROACH, in relation to the predicted-progression scores.MethodsActual structural progression was measured using minimum joint space width (minJSW). Actual pain (progression) was evaluated using the Knee injury and Osteoarthritis Outcomes Score (KOOS) pain questionnaire. Progression was presented as actual change (Δ) after 2 years, and as progression over 2 years based on a per patient fitted regression line using 0, 0.5, 1 and 2-year values. Differences in predicted-progression scores between actual progressors and non-progressors were evaluated. Receiver operating characteristic (ROC) curves were constructed and corresponding area under the curve (AUC) reported. Using Youden's index, optimal cut-offs were chosen to enable evaluation of both predicted-progression scores to identify actual progressors.ResultsActual structural progressors were initially assigned higher S predicted-progression scores compared with structural non-progressors. Likewise, actual pain progressors were assigned higher P predicted-progression scores compared with pain non-progressors. The AUC-ROC for the S predicted-progression score to identify actual structural progressors was poor (0.612 and 0.599 for Δ and regression minJSW, respectively). The AUC-ROC for the P predicted-progression score to identify actual pain progressors were good (0.817 and 0.830 for Δ and regression KOOS pain, respectively).ConclusionThe S and P predicted-progression scores as provided by the ML models developed and used for the selection of IMI-APPROACH patients were to some degree able to distinguish between actual progressors and non-progressors.Trial registrationClinicalTrials.gov, https://clinicaltrials.gov, NCT03883568.

Project description:BackgroundThere are multiple measures for assessment of physical function in knee osteoarthritis (OA), but each has its strengths and limitations. The GaitSmart® system, which uses inertial measurement units (IMUs), might be a user-friendly and objective method to assess function. This study evaluates the validity and responsiveness of GaitSmart® motion analysis as a function measurement in knee OA and compares this to Knee Injury and Osteoarthritis Outcome Score (KOOS), Short Form 36 Health Survey (SF-36), 30s chair stand test, and 40m self-paced walk test.MethodsThe 2-year Innovative Medicines Initiative-Applied Public-Private Research enabling OsteoArthritis Clinical Headway (IMI-APPROACH) knee OA cohort was conducted between January 2018 and April 2021. For this study, available baseline and 6 months follow-up data (n = 262) was used. Principal component analysis was used to investigate whether above mentioned function instruments could represent one or more function domains. Subsequently, linear regression was used to explore the association between GaitSmart® parameters and those function domains. In addition, standardized response means, effect sizes and t-tests were calculated to evaluate the ability of GaitSmart® to differentiate between good and poor general health (based on SF-36). Lastly, the responsiveness of GaitSmart® to detect changes in function was determined.ResultsKOOS, SF-36, 30s chair test and 40m self-paced walk test were first combined into one function domain (total function). Thereafter, two function domains were substracted related to either performance based (objective function) or self-reported (subjective function) function. Linear regression resulted in the highest R2 for the total function domain: 0.314 (R2 for objective and subjective function were 0.252 and 0.142, respectively.). Furthermore, GaitSmart® was able to distinguish a difference in general health status, and is responsive to changes in the different aspects of objective function (Standardized response mean (SRMs) up to 0.74).ConclusionGaitSmart® analysis can reflect performance based and self-reported function and may be of value in the evaluation of function in knee OA. Future studies are warranted to validate whether GaitSmart® can be used as clinical outcome measure in OA research and clinical practice.

Project description:BackgroundThe IMI-APPROACH cohort is an exploratory, 5-centre, 2-year prospective follow-up study of knee osteoarthritis (OA). Aim was to describe baseline multi-tissue semiquantitative MRI evaluation of index knees and to describe change for different MRI features based on number of subregion-approaches and change in maximum grades over a 24-month period.MethodsMRIs were acquired using 1.5 T or 3 T MRI systems and assessed using the semi-quantitative MRI OA Knee Scoring (MOAKS) system. MRIs were read at baseline and 24-months for cartilage damage, bone marrow lesions (BML), osteophytes, meniscal damage and extrusion, and Hoffa- and effusion-synovitis. In descriptive fashion, the frequencies of MRI features at baseline and change in these imaging biomarkers over time are presented for the entire sample in a subregional and maximum score approach for most features. Differences between knees without and with structural radiographic (R) OA are analyzed in addition.ResultsTwo hundred eighty-nine participants had readable baseline MRI examinations. Mean age was 66.6 ± 7.1 years and participants had a mean BMI of 28.1 ± 5.3 kg/m2. The majority (55.3%) of included knees had radiographic OA. Any change in total cartilage MOAKS score was observed in 53.1% considering full-grade changes only, and in 73.9% including full-grade and within-grade changes. Any medial cartilage progression was seen in 23.9% and any lateral progression on 22.1%. While for the medial and lateral compartments numbers of subregions with improvement and worsening of BMLs were very similar, for the PFJ more improvement was observed compared to worsening (15.5% vs. 9.0%). Including within grade changes, the number of knees showing BML worsening increased from 42.2% to 55.6%. While for some features 24-months change was rare, frequency of change was much more common in knees with vs. without ROA (e.g. worsening of total MOAKS score cartilage in 68.4% of ROA knees vs. 36.7% of no-ROA knees, and 60.7% vs. 21.8% for an increase in maximum BML score per knee).ConclusionsA wide range of MRI-detected structural pathologies was present in the IMI-APPROACH cohort. Baseline prevalence and change of features was substantially more common in the ROA subgroup compared to the knees without ROA.Trial registrationClinicaltrials.gov identification: NCT03883568.

Dataset Information

Development and validation of a machine learning-supported strategy of patient selection for osteoarthritis clinical trials: the IMI-APPROACH study.

Objectives

Design

Results

Conclusions

Publications

Development and validation of a machine learning-supported strategy of patient selection for osteoarthritis clinical trials: the IMI-APPROACH study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets