Dataset Information

Comparisons of the prediction models for undiagnosed diabetes between machine learning versus traditional statistical methods.

ABSTRACT: We compared the prediction performance of machine learning-based undiagnosed diabetes prediction models with that of traditional statistics-based prediction models. We used the 2014-2020 Korean National Health and Nutrition Examination Survey (KNHANES) (N = 32,827). The KNHANES 2014-2018 data were used as training and internal validation sets and the 2019-2020 data as external validation sets. The receiver operating characteristic curve area under the curve (AUC) was used to compare the prediction performance of the machine learning-based and the traditional statistics-based prediction models. Using sex, age, resting heart rate, and waist circumference as features, the machine learning-based model showed a higher AUC (0.788 vs. 0.740) than that of the traditional statistical-based prediction model. Using sex, age, waist circumference, family history of diabetes, hypertension, alcohol consumption, and smoking status as features, the machine learning-based prediction model showed a higher AUC (0.802 vs. 0.759) than the traditional statistical-based prediction model. The machine learning-based prediction model using features for maximum prediction performance showed a higher AUC (0.819 vs. 0.765) than the traditional statistical-based prediction model. Machine learning-based prediction models using anthropometric and lifestyle measurements may outperform the traditional statistics-based prediction models in predicting undiagnosed diabetes.

SUBMITTER: Choi SG

PROVIDER: S-EPMC10421881 | biostudies-literature | 2023 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparisons of the prediction models for undiagnosed diabetes between machine learning versus traditional statistical methods.

Choi Seong Gyu SG Oh Minsuk M Park Dong-Hyuk DH Lee Byeongchan B Lee Yong-Ho YH Jee Sun Ha SH Jeon Justin Y JY

Scientific reports 20230811 1

We compared the prediction performance of machine learning-based undiagnosed diabetes prediction models with that of traditional statistics-based prediction models. We used the 2014-2020 Korean National Health and Nutrition Examination Survey (KNHANES) (N = 32,827). The KNHANES 2014-2018 data were used as training and internal validation sets and the 2019-2020 data as external validation sets. The receiver operating characteristic curve area under the curve (AUC) was used to compare the predicti ...[more]

PMID: 37567907

Similar Datasets

Project description:ObjectivesTo evaluate whether postpartum haemorrhage (PPH) can be predicted using both machine learning (ML) and traditional statistical models.DesignDiagnostic systematic review and meta-analysis of observational and clinical studies, prospectively registered on PROSPERO, performed accordingly to the Preferred Reporting Items for Systematic Reviews and Meta-analysis and Prediction model risk of bias assessment tool for studies developing, validating or updating prediction models, with the use of an independent analysis by a large language model (GPT-4 Open AI).Data sourcesMEDLINE/PubMed, LILACS-BVS, Cochrane Library, Scopus-Elsevier, Embase-Elsevier and Web of Science.Eligibility criteria for selected studiesThe literature search was conducted on 4 January 2024 and included observational studies and clinical trials published in the past 10 years that assessed early PPH and PPH prediction and that applied accuracy metrics for outcomes evaluation. We excluded studies that did not define PPH or had exclusive PPH subgroups evaluation.Primary and secondary outcome measuresThe primary outcome is the accuracy of PPH prediction using both ML and conventional statistical models. A secondary outcome is to describe the strongest risk factors of PPH identified by ML and traditional statistical models.ResultsOf 551 citations screened, 35 studies were eligible for inclusion. The synthesis gathered 383 648 patients in 24 studies conducted with conventional statistics (CS), 9 studies using ML models and 2 studies using both methods. Multivariate regression was a preferred modelling approach to predict PPH in CS studies, while ML approaches used multiple models and a myriad of features. ML comparison to CS was only performed in two studies, and ML models demonstrated a 95% higher likelihood of PPH prediction compared with CS when applied to the same dataset (OR 1.95, 95% CI 1.88 to 2.01, p<0.001). The I² had a value of 54%, p=0.14, indicating moderate heterogeneity between the studies.ConclusionsML models are promising for predicting PPH. Nevertheless, they often require a large number of predictors, which may limit their applicability or necessitate automation through digital systems. This poses challenges in resource-scarce settings where the majority of PPH complications occur.Prospero registration numberCRD42024521059.

Project description:BackgroundPredicting survival of recipients after liver transplantation is regarded as one of the most important challenges in contemporary medicine. Hence, improving on current prediction models is of great interest.Nowadays, there is a strong discussion in the medical field about machine learning (ML) and whether it has greater potential than traditional regression models when dealing with complex data. Criticism to ML is related to unsuitable performance measures and lack of interpretability which is important for clinicians.MethodsIn this paper, ML techniques such as random forests and neural networks are applied to large data of 62294 patients from the United States with 97 predictors selected on clinical/statistical grounds, over more than 600, to predict survival from transplantation. Of particular interest is also the identification of potential risk factors. A comparison is performed between 3 different Cox models (with all variables, backward selection and LASSO) and 3 machine learning techniques: a random survival forest and 2 partial logistic artificial neural networks (PLANNs). For PLANNs, novel extensions to their original specification are tested. Emphasis is given on the advantages and pitfalls of each method and on the interpretability of the ML techniques.ResultsWell-established predictive measures are employed from the survival field (C-index, Brier score and Integrated Brier Score) and the strongest prognostic factors are identified for each model. Clinical endpoint is overall graft-survival defined as the time between transplantation and the date of graft-failure or death. The random survival forest shows slightly better predictive performance than Cox models based on the C-index. Neural networks show better performance than both Cox models and random survival forest based on the Integrated Brier Score at 10 years.ConclusionIn this work, it is shown that machine learning techniques can be a useful tool for both prediction and interpretation in the survival context. From the ML techniques examined here, PLANN with 1 hidden layer predicts survival probabilities the most accurately, being as calibrated as the Cox model with all variables.Trial registrationRetrospective data were provided by the Scientific Registry of Transplant Recipients under Data Use Agreement number 9477 for analysis of risk factors after liver transplantation.

Project description:BackgroundIn health research, several chronic diseases are susceptible to competing risks (CRs). Initially, statistical models (SM) were developed to estimate the cumulative incidence of an event in the presence of CRs. As recently there is a growing interest in applying machine learning (ML) for clinical prediction, these techniques have also been extended to model CRs but literature is limited. Here, our aim is to investigate the potential role of ML versus SM for CRs within non-complex data (small/medium sample size, low dimensional setting).MethodsA dataset with 3826 retrospectively collected patients with extremity soft-tissue sarcoma (eSTS) and nine predictors is used to evaluate model-predictive performance in terms of discrimination and calibration. Two SM (cause-specific Cox, Fine-Gray) and three ML techniques are compared for CRs in a simple clinical setting. ML models include an original partial logistic artificial neural network for CRs (PLANNCR original), a PLANNCR with novel specifications in terms of architecture (PLANNCR extended), and a random survival forest for CRs (RSFCR). The clinical endpoint is the time in years between surgery and disease progression (event of interest) or death (competing event). Time points of interest are 2, 5, and 10 years.ResultsBased on the original eSTS data, 100 bootstrapped training datasets are drawn. Performance of the final models is assessed on validation data (left out samples) by employing as measures the Brier score and the Area Under the Curve (AUC) with CRs. Miscalibration (absolute accuracy error) is also estimated. Results show that the ML models are able to reach a comparable performance versus the SM at 2, 5, and 10 years regarding both Brier score and AUC (95% confidence intervals overlapped). However, the SM are frequently better calibrated.ConclusionsOverall, ML techniques are less practical as they require substantial implementation time (data preprocessing, hyperparameter tuning, computational intensity), whereas regression methods can perform well without the additional workload of model training. As such, for non-complex real life survival data, these techniques should only be applied complementary to SM as exploratory tools of model's performance. More attention to model calibration is urgently needed.

Dataset Information

Comparisons of the prediction models for undiagnosed diabetes between machine learning versus traditional statistical methods.

Publications

Comparisons of the prediction models for undiagnosed diabetes between machine learning versus traditional statistical methods.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets