Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

A Model for Improving the Learning Curves of Artificial Neural Networks.

ABSTRACT: In this article, the performance of a hybrid artificial neural network (i.e. scale-free and small-world) was analyzed and its learning curve compared to three other topologies: random, scale-free and small-world, as well as to the chemotaxis neural network of the nematode Caenorhabditis Elegans. One hundred equivalent networks (same number of vertices and average degree) for each topology were generated and each was trained for one thousand epochs. After comparing the mean learning curves of each network topology with the C. elegans neural network, we found that the networks that exhibited preferential attachment exhibited the best learning curves.

SUBMITTER: Monteiro RL

PROVIDER: S-EPMC4763452 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

A Model for Improving the Learning Curves of Artificial Neural Networks.

Monteiro Roberto L S RL Carneiro Tereza Kelly G TK Fontoura José Roberto A JR da Silva Valéria L VL Moret Marcelo A MA Pereira Hernane Borges de Barros HB

PloS one 20160222 2

In this article, the performance of a hybrid artificial neural network (i.e. scale-free and small-world) was analyzed and its learning curve compared to three other topologies: random, scale-free and small-world, as well as to the chemotaxis neural network of the nematode Caenorhabditis Elegans. One hundred equivalent networks (same number of vertices and average degree) for each topology were generated and each was trained for one thousand epochs. After comparing the mean learning curves of eac ...[more]

PMID: 26901646

Similar Datasets

Deconvoluting Kinetic Rate Constants of Catalytic Substrates from Scanning Electrochemical Approach Curves with Artificial Neural Networks.

Project description:Extracting information from experimental measurements in the chemical sciences typically requires curve fitting, deconvolution, and/or solving the governing partial differential equations via numerical (e.g., finite element analysis) or analytical methods. However, using numerical or analytical methods for high-throughput data analysis typically requires significant postprocessing efforts. Here, we show that deep learning artificial neural networks can be a very effective tool for extracting information from experimental data. As an example, reactivity and topography information from scanning electrochemical microscopy (SECM) approach curves are highly convoluted. This study utilized multilayer perceptrons and convolutional neural networks trained on simulated SECM data to extract kinetic rate constants of catalytic substrates. Our key findings were that multilayer perceptron models performed very well when the experimental data were close to the ideal conditions with which the model was trained. However, convolutional neural networks, which analyze images as opposed to direct data, were able to accurately predict the kinetic rate constant of Fe-doped nickel (oxy)hydroxide catalyst at different applied potentials even though the experimental approach curves were not ideal. Due to the speed at which machine learning models can analyze data, we believe this study shows that artificial neural networks could become powerful tools in high-throughput data analysis.

| S-EPMC10120032 | biostudies-literature

Mapping the learning curves of deep learning networks.

Project description:There is an important challenge in systematically interpreting the internal representations of deep neural networks (DNNs). Existing techniques are often less effective for non-tabular tasks, or they primarily focus on qualitative, ad-hoc interpretations of models. In response, this study introduces a cognitive science-inspired, multi-dimensional quantification and visualization approach that captures two temporal dimensions of model learning: the "information-processing trajectory" and the "developmental trajectory." The former represents the influence of incoming signals on an agent's decision-making, while the latter conceptualizes the gradual improvement in an agent's performance throughout its lifespan. Tracking the learning curves of DNNs enables researchers to explicitly identify the model appropriateness of a given task, examine the properties of the underlying input signals, and assess the model's alignment (or lack thereof) with human learning experiences. To illustrate this method, we conducted 750 runs of simulations on two temporal tasks: gesture detection and sentence classification, showcasing its applicability across different types of deep learning tasks. Using four descriptive metrics to quantify the mapped learning curves-start, end - start, max, tmax-, we identified significant differences in learning patterns based on data sources and class distinctions (all p's < .0001), the prominent role of spatial semantics in gesture learning, and larger information gains in language learning. We highlight three key insights gained from mapping learning curves: non-monotonic progress, pairwise comparisons, and domain distinctions. We reflect on the theoretical implications of this method for cognitive processing, language models and representations from multiple modalities.

| S-EPMC11841907 | biostudies-literature

Brain-inspired replay for continual learning with artificial neural networks.

Project description:Artificial neural networks suffer from catastrophic forgetting. Unlike humans, when these networks are trained on something new, they rapidly forget what was learned before. In the brain, a mechanism thought to be important for protecting memories is the reactivation of neuronal activity patterns representing those memories. In artificial neural networks, such memory replay can be implemented as 'generative replay', which can successfully - and surprisingly efficiently - prevent catastrophic forgetting on toy examples even in a class-incremental learning scenario. However, scaling up generative replay to complicated problems with many tasks or complex inputs is challenging. We propose a new, brain-inspired variant of replay in which internal or hidden representations are replayed that are generated by the network's own, context-modulated feedback connections. Our method achieves state-of-the-art performance on challenging continual learning benchmarks (e.g., class-incremental learning on CIFAR-100) without storing data, and it provides a novel model for replay in the brain.

| S-EPMC7426273 | biostudies-literature

Dendritic normalisation improves learning in sparsely connected artificial neural networks.

Project description:Artificial neural networks, taking inspiration from biological neurons, have become an invaluable tool for machine learning applications. Recent studies have developed techniques to effectively tune the connectivity of sparsely-connected artificial neural networks, which have the potential to be more computationally efficient than their fully-connected counterparts and more closely resemble the architectures of biological systems. We here present a normalisation, based on the biophysical behaviour of neuronal dendrites receiving distributed synaptic inputs, that divides the weight of an artificial neuron's afferent contacts by their number. We apply this dendritic normalisation to various sparsely-connected feedforward network architectures, as well as simple recurrent and self-organised networks with spatially extended units. The learning performance is significantly increased, providing an improvement over other widely-used normalisations in sparse networks. The results are two-fold, being both a practical advance in machine learning and an insight into how the structure of neuronal dendritic arbours may contribute to computation.

| S-EPMC8407571 | biostudies-literature

Peak learning of mass spectrometry imaging data using artificial neural networks.

Project description:Mass spectrometry imaging (MSI) is an emerging technology that holds potential for improving, biomarker discovery, metabolomics research, pharmaceutical applications and clinical diagnosis. Despite many solutions being developed, the large data size and high dimensional nature of MSI, especially 3D datasets, still pose computational and memory complexities that hinder accurate identification of biologically relevant molecular patterns. Moreover, the subjectivity in the selection of parameters for conventional pre-processing approaches can lead to bias. Therefore, we assess if a probabilistic generative model based on a fully connected variational autoencoder can be used for unsupervised analysis and peak learning of MSI data to uncover hidden structures. The resulting msiPL method learns and visualizes the underlying non-linear spectral manifold, revealing biologically relevant clusters of tissue anatomy in a mouse kidney and tumor heterogeneity in human prostatectomy tissue, colorectal carcinoma, and glioblastoma mouse model, with identification of underlying m/z peaks. The method is applied for the analysis of MSI datasets ranging from 3.3 to 78.9 GB, without prior pre-processing and peak picking, and acquired using different mass spectrometers at different centers.

| S-EPMC8452737 | biostudies-literature

Learning spin liquids on a honeycomb lattice with artificial neural networks.

Project description:Machine learning methods provide a new perspective on the study of many-body system in condensed matter physics and there is only limited understanding of their representational properties and limitations in quantum spin liquid systems. In this work, we investigate the ability of the machine learning method based on the restricted Boltzmann machine in capturing physical quantities including the ground-state energy, spin-structure factor, magnetization, quantum coherence, and multipartite entanglement in the two-dimensional ferromagnetic spin liquids on a honeycomb lattice. It is found that the restricted Boltzmann machine can encode the many-body wavefunction quite well by reproducing accurate ground-state energy and structure factor. Further investigation on the behavior of multipartite entanglement indicates that the residual entanglement is richer in the gapless phase than the gapped spin-liquid phase, which suggests that the residual entanglement can characterize the spin-liquid phases. Additionally, we confirm the existence of a gapped non-Abelian topological phase in the spin liquids on a honeycomb lattice with a small magnetic field and determine the corresponding phase boundary by recognizing the rapid change of the local magnetization and residual entanglement.

| S-EPMC8371168 | biostudies-literature

Comparison of multiple modalities for drug response prediction with learning curves using neural networks and XGBoost

Project description: Not available

| S-EPMC10812874 | biostudies-literature

Improving the performance of mutation-based evolving artificial neural networks with self-adaptive mutations.

Project description:Neuroevolution is a promising approach for designing artificial neural networks using an evolutionary algorithm. Unlike recent trending methods that rely on gradient-based algorithms, neuroevolution can simultaneously evolve the topology and weights of neural networks. In neuroevolution with topological evolution, handling crossover is challenging because of the competing conventions problem. Mutation-based evolving artificial neural network is an alternative topology and weights neuroevolution approach that omits crossover and uses only mutations for genetic variation. This study enhances the performance of mutation-based evolving artificial neural network in two ways. First, the mutation step size controlling the magnitude of the parameter perturbation is automatically adjusted by a self-adaptive mutation mechanism, enabling a balance between exploration and exploitation during the evolution process. Second, the structural mutation probabilities are automatically adjusted depending on the network size, preventing excessive expansion of the topology. The proposed methods are compared with conventional neuroevolution algorithms using locomotion tasks provided in the OpenAI Gym benchmarks. The results demonstrate that the proposed methods with the self-adaptive mutation mechanism can achieve better performance. In addition, the adjustment of structural mutation probabilities can mitigate topological bloat while maintaining performance.

| S-EPMC11249216 | biostudies-literature

Representation learning in the artificial and biological neural networks underlying sensorimotor integration.

Project description:The integration of deep learning and theories of reinforcement learning (RL) is a promising avenue to explore novel hypotheses on reward-based learning and decision-making in humans and other animals. Here, we trained deep RL agents and mice in the same sensorimotor task with high-dimensional state and action space and studied representation learning in their respective neural networks. Evaluation of thousands of neural network models with extensive hyperparameter search revealed that learning-dependent enrichment of state-value and policy representations of the task-performance-optimized deep RL agent closely resembled neural activity of the posterior parietal cortex (PPC). These representations were critical for the task performance in both systems. PPC neurons also exhibited representations of the internally defined subgoal, a feature of deep RL algorithms postulated to improve sample efficiency. Such striking resemblance between the artificial and biological networks and their functional convergence in sensorimotor integration offers new opportunities to better understand respective intelligent systems.

| S-EPMC9166289 | biostudies-literature

A Semi-supervised Learning-Based Diagnostic Classification Method Using Artificial Neural Networks.

Project description:The purpose of cognitive diagnostic modeling (CDM) is to classify students' latent attribute profiles using their responses to the diagnostic assessment. In recent years, each diagnostic classification model (DCM) makes different assumptions about the relationship between a student's response pattern and attribute profile. The previous research studies showed that the inappropriate DCMs and inaccurate Q-matrix impact diagnostic classification accuracy. Artificial Neural Networks (ANNs) have been proposed as a promising approach to convert a pattern of item responses into a diagnostic classification in some research studies. However, the ANNs methods produced very unstable and unappreciated estimation unless a great deal of care was taken. In this research, we combined ANNs with two typical DCMs, the deterministic-input, noisy, "and" gate (DINA) model and the deterministic-inputs, noisy, "or" gate (DINO) model, within a semi-supervised learning framework to achieve a robust and accurate classification. In both simulated study and real data study, the experimental results showed that the proposed method could achieve appreciated performance across different test conditions, especially when the diagnostic quality of assessment was not high and the Q-matrix contained misspecified elements. This research study is the first time of applying the thinking of semi-supervised learning into CDM. Also, we used the validating test to choose the appropriate parameters for the ANNs instead of using typical statistical criteria.

| S-EPMC7856146 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data