Dataset Information

Variable importance analysis with interpretable machine learning for fair risk prediction.

ABSTRACT: Machine learning (ML) methods are increasingly used to assess variable importance, but such black box models lack stability when limited in sample sizes, and do not formally indicate non-important factors. The Shapley variable importance cloud (ShapleyVIC) addresses these limitations by assessing variable importance from an ensemble of regression models, which enhances robustness while maintaining interpretability, and estimates uncertainty of overall importance to formally test its significance. In a clinical study, ShapleyVIC reasonably identified important variables when the random forest and XGBoost failed to, and generally reproduced the findings from smaller subsamples (n = 2500 and 500) when statistical power of the logistic regression became attenuated. Moreover, ShapleyVIC reasonably estimated non-significant importance of race to justify its exclusion from the final prediction model, as opposed to the race-dependent model from the conventional stepwise model building. Hence, ShapleyVIC is robust and interpretable for variable importance assessment, with potential contribution to fairer clinical risk prediction.

SUBMITTER: Ning Y

PROVIDER: S-EPMC11244764 | biostudies-literature | 2024 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Variable importance analysis with interpretable machine learning for fair risk prediction.

Ning Yilin Y Li Siqi S Ng Yih Yng YY Chia Michael Yih Chong MYC Gan Han Nee HN Tiah Ling L Mao Desmond Renhao DR Ng Wei Ming WM Leong Benjamin Sieu-Hon BS Doctor Nausheen N Ong Marcus Eng Hock MEH Liu Nan N

PLOS digital health 20240712 7

Machine learning (ML) methods are increasingly used to assess variable importance, but such black box models lack stability when limited in sample sizes, and do not formally indicate non-important factors. The Shapley variable importance cloud (ShapleyVIC) addresses these limitations by assessing variable importance from an ensemble of regression models, which enhances robustness while maintaining interpretability, and estimates uncertainty of overall importance to formally test its significance ...[more]

PMID: 38995879

Similar Datasets

Project description:Background and purposeMechanical thrombectomy greatly improves stroke outcomes. Nonetheless, some patients fall short of full recovery despite good reperfusion. The purpose of this study was to develop machine learning (ML) models for the pre-interventional prediction of functional outcome at 3 months of thrombectomy in acute ischemic stroke (AIS), using clinical and auto-extractable radiological information consistently available upon first emergency evaluation.Materials and methodsA two-center retrospective cohort of 293 patients with AIS who underwent thrombectomy was analyzed. ML models were developed to predict dichotomized modified Rankin score at 90 days (mRS-90) using clinical and imaging features, both separately and combined. Conventional and experimental imaging biomarkers were quantified using automated image-processing software from non-contract computed tomography (CT) and computed tomography angiography (CTA). Shapley Additive Explanation (SHAP) was applied for model interpretability and predictor importance analysis of the optimal model.ResultsMerging clinical and imaging features returned the best results for mRS-90 prediction. The best performing classifier was Extreme Gradient Boosting (XGB) with an area under the receiver operating characteristic curve (AUC) = 84% using selected features. The most important classifying features were age, baseline National Institutes of Health Stroke Scale (NIHSS), occlusion side, degree of brain atrophy [primarily represented by cortical cerebrospinal fluid (CSF) volume and lateral ventricle volume], early ischemic core [primarily represented by e-Alberta Stroke Program Early CT Score (ASPECTS)], and collateral circulation deficit volume on CTA.ConclusionMachine learning that is applied to quantifiable image features from CT and CTA alongside basic clinical characteristics constitutes a promising automated method in the pre-interventional prediction of stroke prognosis. Interpretable models allow for exploring which initial features contribute the most to post-thrombectomy outcome prediction overall and for each individual patient outcome.

Project description:BackgroundDrug-induced long-QT syndrome (diLQTS) is a major concern among patients who are hospitalized, for whom prediction models capable of identifying individualized risk could be useful to guide monitoring. We have previously demonstrated the feasibility of machine learning to predict the risk of diLQTS, in which deep learning models provided superior accuracy for risk prediction, although these models were limited by a lack of interpretability.ObjectiveIn this investigation, we sought to examine the potential trade-off between interpretability and predictive accuracy with the use of more complex models to identify patients at risk for diLQTS. We planned to compare a deep learning algorithm to predict diLQTS with a more interpretable algorithm based on cluster analysis that would allow medication- and subpopulation-specific evaluation of risk.MethodsWe examined the risk of diLQTS among 35,639 inpatients treated between 2003 and 2018 with at least 1 of 39 medications associated with risk of diLQTS and who had an electrocardiogram in the system performed within 24 hours of medication administration. Predictors included over 22,000 diagnoses and medications at the time of medication administration, with cases of diLQTS defined as a corrected QT interval over 500 milliseconds after treatment with a culprit medication. The interpretable model was developed using cluster analysis (K=4 clusters), and risk was assessed for specific medications and classes of medications. The deep learning model was created using all predictors within a 6-layer neural network, based on previously identified hyperparameters.ResultsAmong the medications, we found that class III antiarrhythmic medications were associated with increased risk across all clusters, and that in patients who are noncritically ill without cardiovascular disease, propofol was associated with increased risk, whereas ondansetron was associated with decreased risk. Compared with deep learning, the interpretable approach was less accurate (area under the receiver operating characteristic curve: 0.65 vs 0.78), with comparable calibration.ConclusionsIn summary, we found that an interpretable modeling approach was less accurate, but more clinically applicable, than deep learning for the prediction of diLQTS. Future investigations should consider this trade-off in the development of methods for clinical prediction.

Dataset Information

Variable importance analysis with interpretable machine learning for fair risk prediction.

Publications

Variable importance analysis with interpretable machine learning for fair risk prediction.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets