Dataset Information

Improving clinical trial efficiency using a machine learning-based risk score to enrich study populations.

ABSTRACT:

Aims

Prognostic enrichment strategies can make trials more efficient, although potentially at the cost of diminishing external validity. Whether using a risk score to identify a population at increased mortality risk could improve trial efficiency is uncertain. We aimed to assess whether Machine learning Assessment of RisK and EaRly mortality in Heart Failure (MARKER-HF), a previously validated risk score, could improve clinical trial efficiency.

Methods and results

Mortality rates and association of MARKER-HF with all-cause death by 1 year were evaluated in four community-based heart failure (HF) and five HF clinical trial cohorts. Sample size required to assess effects of an investigational therapy on mortality was calculated assuming varying underlying MARKER-HF risk and proposed treatment effect profiles. Patients from community-based HF cohorts (n = 11 297) had higher observed mortality and MARKER-HF scores than did clinical trial patients (n = 13 165) with HF with either reduced ejection fraction (HFrEF) or preserved ejection fraction (HFpEF). MARKER-HF score was strongly associated with risk of 1-year mortality both in the community (hazard ratio [HR] 1.48, 95% confidence interval [CI] 1.44-1.52) and clinical trial cohorts with HFrEF (HR 1.41, 95% CI 1.30-1.54), and HFpEF (HR 1.74, 95% CI 1.53-1.98), per 0.1 increase in MARKER-HF. Using MARKER-HF to identify patients for a hypothetical clinical trial assessing mortality reduction with an intervention, enabled a reduction in sample size required to show benefit.

Conclusion

Using a reliable predictor of mortality such as MARKER-HF to enrich clinical trial populations provides a potential strategy to improve efficiency by requiring a smaller sample size to demonstrate a clinical benefit.

SUBMITTER: Jering KS

PROVIDER: S-EPMC9388618 | biostudies-literature | 2022 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Improving clinical trial efficiency using a machine learning-based risk score to enrich study populations.

Jering Karola S KS Campagnari Claudio C Claggett Brian B Adler Eric E Klein Liviu L Ahmad Faraz S FS Voors Adriaan A AA Solomon Scott S Yagil Avi A Greenberg Barry B

European journal of heart failure 20220522 8

<h4>Aims</h4>Prognostic enrichment strategies can make trials more efficient, although potentially at the cost of diminishing external validity. Whether using a risk score to identify a population at increased mortality risk could improve trial efficiency is uncertain. We aimed to assess whether Machine learning Assessment of RisK and EaRly mortality in Heart Failure (MARKER-HF), a previously validated risk score, could improve clinical trial efficiency.<h4>Methods and results</h4>Mortality rate ...[more]

PMID: 35508918

Similar Datasets

Project description:ObjectiveEfficiently identifying eligible patients is a crucial first step for a successful clinical trial. The objective of this study was to test whether an approach using electronic health record (EHR) data and an ensemble machine learning algorithm incorporating billing codes and data from clinical notes processed by natural language processing (NLP) can improve the efficiency of eligibility screening.MethodsWe studied patients screened for a clinical trial of rheumatoid arthritis (RA) with one or more International Classification of Diseases (ICD) code for RA and age greater than 35 years, from a tertiary care center and a community hospital. The following three groups of EHR features were considered for the algorithm: 1) structured features, 2) the counts of NLP concepts from notes, 3) health care utilization. All features were linked to dates. We applied random forest and logistic regression with least absolute shrinkage and selection operator penalty against the following two standard approaches: 1) one or more RA ICD code and no ICD codes related to exclusion criteria (ScreenRAICD1 +EX ) and 2) two or more RA ICD codes (ScreenRAICD2 ). To test the portability, we trained the algorithm at one institution and tested it at the other.ResultsIn total, 3359 patients at Brigham and Women's Hospital (BWH) and 642 patients at Faulkner Hospital (FH) were studied, with 461 (13.7%) eligible patients at BWH and 84 (13.4%) at FH. The application of the algorithm reduced ineligible patients from chart review by 40.5% at the tertiary care center and by 57.0% at the community hospital. In contrast, ScreenRAICD2 reduced patients for chart review by 2.7% to 11.3%; ScreenRAICD1+EX reduced patients for chart review by 63% to 65% but excluded 22% to 27% of eligible patients.ConclusionThe ensemble machine learning algorithm incorporating billing codes and NLP data increased the efficiency of eligibility screening by reducing the number of patients requiring chart review while not excluding eligible patients. Moreover, this approach can be trained at one institution and applied at another for multicenter clinical trials.

Project description:BackgroundAccurate estimation of operative case-time duration is critical for optimizing operating room use. Current estimates are inaccurate and earlier models include data not available at the time of scheduling. Our objective was to develop statistical models in a large retrospective data set to improve estimation of case-time duration relative to current standards.Study designWe developed models to predict case-time duration using linear regression and supervised machine learning. For each of these models, we generated an all-inclusive model, service-specific models, and surgeon-specific models. In the latter 2 approaches, individual models were created for each surgical service and surgeon, respectively. Our data set included 46,986 scheduled operations performed at a large academic medical center from January 2014 to December 2017, with 80% used for training and 20% for model testing/validation. Predictions derived from each model were compared with our institutional standard of using average historic procedure times and surgeon estimates. Models were evaluated based on accuracy, overage (case duration > predicted + 10%), underage (case duration < predicted - 10%), and the predictive capability of being within a 10% tolerance threshold.ResultsThe machine learning algorithm resulted in the highest predictive capability. The surgeon-specific model was superior to the service-specific model, with higher accuracy, lower percentage of overage and underage, and higher percentage of cases within the 10% threshold. The ability to predict cases within 10% improved from 32% using our institutional standard to 39% with the machine learning surgeon-specific model.ConclusionsOur study is a notable advancement toward statistical modeling of case-time duration across all surgical departments in a large tertiary medical center. Machine learning approaches can improve case duration estimations, enabling improved operating room scheduling, efficiency, and reduced costs.

Project description:Background: Previous studies have constructed prediction models for type 2 diabetes mellitus (T2DM), but machine learning was rarely used and few focused on genetic prediction. This study aimed to establish an effective T2DM prediction tool and to further explore the potential of genetic risk scores (GRS) via various classifiers among rural adults. Methods: In this prospective study, the GRS for a total of 5,712 participants from the Henan Rural Cohort Study was calculated. Cox proportional hazards (CPH) regression was used to analyze the associations between GRS and T2DM. CPH, artificial neural network (ANN), random forest (RF), and gradient boosting machine (GBM) were used to establish prediction models, respectively. The area under the receiver operating characteristic curve (AUC) and net reclassification index (NRI) were used to assess the discrimination ability of the models. The decision curve was plotted to determine the clinical-utility for prediction models. Results: Compared with the individuals in the lowest quintile of the GRS, the HR (95% CI) was 2.06 (1.40 to 3.03) for those with the highest quintile of GRS (P trend < 0.05). Based on conventional predictors, the AUCs of the prediction model were 0.815, 0.816, 0.843, and 0.851 via CPH, ANN, RF, and GBM, respectively. Changes with the integration of GRS for CPH, ANN, RF, and GBM were 0.001, 0.002, 0.018, and 0.033, respectively. The reclassifications were significantly improved for all classifiers when adding GRS (NRI: 41.2% for CPH; 41.0% for ANN; 46.4% for ANN; 45.1% for GBM). Decision curve analysis indicated the clinical benefits of model combined GRS. Conclusion: The prediction model combined with GRS may provide incremental predictions of performance beyond conventional factors for T2DM, which demonstrated the potential clinical use of genetic markers to screen vulnerable populations. Clinical Trial Registration: The Henan Rural Cohort Study is registered in the Chinese Clinical Trial Register (Registration number: ChiCTR-OOC-15006699). http://www.chictr.org.cn/showproj.aspx?proj=11375.

Dataset Information

Improving clinical trial efficiency using a machine learning-based risk score to enrich study populations.

Aims

Methods and results

Conclusion

Publications

Improving clinical trial efficiency using a machine learning-based risk score to enrich study populations.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets