Dataset Information

AlphaML: A clear, legible, explainable, transparent, and elucidative binary classification platform for tabular data.

ABSTRACT: Leveraging the potential of machine learning and recognizing the broad applications of binary classification, it becomes essential to develop platforms that are not only powerful but also transparent, interpretable, and user friendly. We introduce alphaML, a user-friendly platform that provides clear, legible, explainable, transparent, and elucidative (CLETE) binary classification models with comprehensive customization options. AlphaML offers feature selection, hyperparameter search, sampling, and normalization methods, along with 15 machine learning algorithms with global and local interpretation. We have integrated a custom metric for hyperparameter search that considers both training and validation scores, safeguarding against under- or overfitting. Additionally, we employ the NegLog2RMSL scoring method, which uses both training and test scores for a thorough model evaluation. The platform has been tested using datasets from multiple domains and offers a graphical interface, removing the need for programming expertise. Consequently, alphaML exhibits versatility, demonstrating promising applicability across a broad spectrum of tabular data configurations.

SUBMITTER: Nasimian A

PROVIDER: S-EPMC10801203 | biostudies-literature | 2024 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

AlphaML: A clear, legible, explainable, transparent, and elucidative binary classification platform for tabular data.

Nasimian Ahmad A Younus Saleena S Tatli Özge Ö Hammarlund Emma U EU Pienta Kenneth J KJ Rönnstrand Lars L Kazi Julhash U JU

Patterns (New York, N.Y.) 20231213 1

Leveraging the potential of machine learning and recognizing the broad applications of binary classification, it becomes essential to develop platforms that are not only powerful but also transparent, interpretable, and user friendly. We introduce alphaML, a user-friendly platform that provides clear, legible, explainable, transparent, and elucidative (CLETE) binary classification models with comprehensive customization options. AlphaML offers feature selection, hyperparameter search, sampling, ...[more]

PMID: 38264719

Similar Datasets

Project description:Classification algorithms assign observations to groups based on patterns in data. The machine-learning community have developed myriad classification algorithms, which are used in diverse life science research domains. Algorithm choice can affect classification accuracy dramatically, so it is crucial that researchers optimize the choice of which algorithm(s) to apply in a given research domain on the basis of empirical evidence. In benchmark studies, multiple algorithms are applied to multiple datasets, and the researcher examines overall trends. In addition, the researcher may evaluate multiple hyperparameter combinations for each algorithm and use feature selection to reduce data dimensionality. Although software implementations of classification algorithms are widely available, robust benchmark comparisons are difficult to perform when researchers wish to compare algorithms that span multiple software packages. Programming interfaces, data formats, and evaluation procedures differ across software packages; and dependency conflicts may arise during installation. To address these challenges, we created ShinyLearner, an open-source project for integrating machine-learning packages into software containers. ShinyLearner provides a uniform interface for performing classification, irrespective of the library that implements each algorithm, thus facilitating benchmark comparisons. In addition, ShinyLearner enables researchers to optimize hyperparameters and select features via nested cross-validation; it tracks all nested operations and generates output files that make these steps transparent. ShinyLearner includes a Web interface to help users more easily construct the commands necessary to perform benchmark comparisons. ShinyLearner is freely available at https://github.com/srp33/ShinyLearner. This software is a resource to researchers who wish to benchmark multiple classification or feature-selection algorithms on a given dataset. We hope it will serve as example of combining the benefits of software containerization with a user-friendly approach.

Project description:BackgroundThe performance of a classification algorithm eventually reaches a point of diminishing returns, where the additional sample added does not improve the results. Thus, there is a need to determine an optimal sample size that maximizes performance while accounting for computational burden or budgetary concerns.ObjectiveThis study aimed to determine optimal sample sizes and the relationships between sample size and dataset-level characteristics over a variety of binary classification algorithms.MethodsA total of 16 large open-source datasets were collected, each containing a binary clinical outcome. Furthermore, 4 machine learning algorithms were assessed: XGBoost (XGB), random forest (RF), logistic regression (LR), and neural networks (NNs). For each dataset, the cross-validated area under the curve (AUC) was calculated at increasing sample sizes, and learning curves were fit. Sample sizes needed to reach the observed full-dataset AUC minus 2 points (0.02) were calculated from the fitted learning curves and compared across the datasets and algorithms. Dataset-level characteristics, minority class proportion, full-dataset AUC, number of features, type of features, and degree of nonlinearity were examined. Negative binomial regression models were used to quantify relationships between these characteristics and expected sample sizes within each algorithm. A total of 4 multivariable models were constructed, which selected the best-fitting combination of dataset-level characteristics.ResultsAmong the 16 datasets (full-dataset sample sizes ranging from 70,000-1,000,000), median sample sizes were 9960 (XGB), 3404 (RF), 696 (LR), and 12,298 (NN) to reach AUC stability. For all 4 algorithms, more balanced classes (multiplier: 0.93-0.96 for a 1% increase in minority class proportion) were associated with decreased sample size. Other characteristics varied in importance across algorithms-in general, more features, weaker features, and more complex relationships between the predictors and the response increased expected sample sizes. In multivariable analysis, the top selected predictors were minority class proportion among all 4 algorithms assessed, full-dataset AUC (XGB, RF, and NN), and dataset nonlinearity (XGB, RF, and NN). For LR, the top predictors were minority class proportion, percentage of strong linear features, and number of features. Final multivariable sample size models had high goodness-of-fit, with dataset-level predictors explaining a majority (66.5%-84.5%) of the total deviance in the data among all 4 models.ConclusionsThe sample sizes needed to reach AUC stability among 4 popular classification algorithms vary by dataset and method and are associated with dataset-level characteristics that can be influenced or estimated before the start of a research study.

Dataset Information

AlphaML: A clear, legible, explainable, transparent, and elucidative binary classification platform for tabular data.

Publications

AlphaML: A clear, legible, explainable, transparent, and elucidative binary classification platform for tabular data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets