Dataset Information

Beyond scores: A machine learning approach to comparing educational system effectiveness.

ABSTRACT: Studies comparing large-scale assessment data among educational systems have been an important tool for understanding the differences in how education is delivered worldwide. Many of these studies do not go beyond reporting average student scores in a particular educational system. A more unbiased analysis would avoid the simple use of gross performance and consider educational system contexts. A common approach is to estimate effectiveness by the residuals of parametric linear models. These models rely upon strong assumptions regarding the data-generating process, and are limited to handling extensive datasets. To address this issue, our paper provides a new approach based on machine learning models. The new approach is flexible, allows paired comparison, and is model-independent. An analysis conducted in Brazil verifies the suitability of the method to explore differences in effectiveness between Brazilian educational administrative units at the regional and state levels from 2009 to 2019. Our results are consistent with the existing literature, but the methodology produced a number of new findings that were not observed in studies using more traditional approaches.

SUBMITTER: Cardoso Silva Filho RL

PROVIDER: S-EPMC10602239 | biostudies-literature | 2023

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Beyond scores: A machine learning approach to comparing educational system effectiveness.

Cardoso Silva Filho Rogério Luiz RL Garg Anvit A Brito Kellyton K Adeodato Paulo Jorge Leitão PJL Carnoy Martin M

PloS one 20231026 10

Studies comparing large-scale assessment data among educational systems have been an important tool for understanding the differences in how education is delivered worldwide. Many of these studies do not go beyond reporting average student scores in a particular educational system. A more unbiased analysis would avoid the simple use of gross performance and consider educational system contexts. A common approach is to estimate effectiveness by the residuals of parametric linear models. These mod ...[more]

PMID: 37883478

Similar Datasets

Project description:There is no consensus on which statistical model estimates school value-added (VA) most accurately. To date, the two most common statistical models used for the calculation of VA scores are two classical methods: linear regression and multilevel models. These models have the advantage of being relatively transparent and thus understandable for most researchers and practitioners. However, these statistical models are bound to certain assumptions (e.g., linearity) that might limit their prediction accuracy. Machine learning methods, which have yielded spectacular results in numerous fields, may be a valuable alternative to these classical models. Although big data is not new in general, it is relatively new in the realm of social sciences and education. New types of data require new data analytical approaches. Such techniques have already evolved in fields with a long tradition in crunching big data (e.g., gene technology). The objective of the present paper is to competently apply these "imported" techniques to education data, more precisely VA scores, and assess when and how they can extend or replace the classical psychometrics toolbox. The different models include linear and non-linear methods and extend classical models with the most commonly used machine learning methods (i.e., random forest, neural networks, support vector machines, and boosting). We used representative data of 3,026 students in 153 schools who took part in the standardized achievement tests of the Luxembourg School Monitoring Program in grades 1 and 3. Multilevel models outperformed classical linear and polynomial regressions, as well as different machine learning models. However, it could be observed that across all schools, school VA scores from different model types correlated highly. Yet, the percentage of disagreements as compared to multilevel models was not trivial and real-life implications for individual schools may still be dramatic depending on the model type used. Implications of these results and possible ethical concerns regarding the use of machine learning methods for decision-making in education are discussed.

Project description:Current treatments for Alzheimer's disease (AD) focus on slowing memory and cognitive decline, but none offer curative outcomes. This study aims to explore and curate the common properties of active, drug-like molecules that modulate glycogen synthase kinase 3β (GSK-3β), a well-documented kinase with increased activity in tau hyperphosphorylation and neurofibrillary tangles-hallmarks of AD pathology. Leveraging quantitative structure-activity relationship (QSAR) data from the PubChem and ChEMBL databases, we employed seven machine learning models: logistic regression (LogR), k-nearest neighbors (KNN), random forest (RF), support vector machine (SVM), extreme gradient boosting (XGB), neural networks (NNs), and ensemble majority voting. Our goal was to correctly predict active and inactive compounds that inhibit GSK-3β activity and identify their key properties. Among the six individual models, the NN demonstrated the highest performance with a 79% AUC-ROC on unbalanced external validation data, while the SVM model was superior in accurately classifying the compounds. The SVM and RF models surpassed NN in terms of Kappa values, and the ensemble majority voting model demonstrated slightly better accuracy to the NN on the external validation data. Feature importance analysis revealed that hydrogen bonds, phenol groups, and specific electronic characteristics are important features of molecular descriptors that positively correlate with active GSK-3β inhibition. Conversely, structural features like imidazole rings, sulfides, and methoxy groups showed a negative correlation. Our study highlights the significance of structural, electronic, and physicochemical descriptors in screening active candidates against GSK-3β. These predictive features could prove useful in therapeutic strategies to understand the important properties of GSK-3β candidate inhibitors that may potentially benefit non-amyloid-based AD treatments targeting neurofibrillary tangles.

Dataset Information

Beyond scores: A machine learning approach to comparing educational system effectiveness.

Publications

Beyond scores: A machine learning approach to comparing educational system effectiveness.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets