Next generation reservoir computing.
Ontology highlight
ABSTRACT:
SUBMITTER: Gauthier DJ
PROVIDER: S-EPMC8455577 | biostudies-literature | 2021 Sep
REPOSITORIES: biostudies-literature
Publications
Next generation reservoir computing.
Gauthier Daniel J DJ Bollt Erik E Griffith Aaron A Barbosa Wendson A S WAS
Nature communications 20210921 1
Reservoir computing is a best-in-class machine learning algorithm for processing information generated by dynamical systems using observed time-series data. Importantly, it requires very small training data sets, uses linear optimization, and thus requires minimal computing resources. However, the algorithm uses randomly sampled matrices to define the underlying recurrent neural network and has a multitude of metaparameters that must be optimized. Recent results demonstrate the equivalence of re ...[more]
Similar Datasets
Project description:Real-time monitoring of key quality variables is essential and crucial for stable and safe operations of wastewater treatment plants (WWTPs). Next generation reservoir computing (NG-RC) has recently garnered significant attention in quality prediction, such as COD and BOD, as an effective alternative to traditional reservoir computing (RC), then is able to act as a data-driven soft sensor to twin a hardware sensor for quality variable measurements. Unlike RC, NG-RC does not require random sampling matrices to define the weights of recurrent neural networks and has fewer hyperparameters. However, NG-RC is usually used online but trained offline, thus leading to model degradation under dynamic scenarios. This paper proposes a sparse online NG-RC approach to meet the real-time requirements of WWTPs and mitigate the impact of measurement noise on the model. First, inspired by the Woodbury matrix identity, an incremental strategy is designed, using sequentially arriving data blocks to learn the output weights of NG-RC online. Then, an ensemble sparse strategy is combined to alleviate overfitting issues of the prediction model. Moreover, a soft sensor based on the ensemble sparse online NG-RC is developed to perform real-time prediction of quality indicators in wastewater treatment processes. Finally, two datasets from actual WWTPs are used to validate the effectiveness of the proposed model.
| S-EPMC11613165 | biostudies-literature
Project description:Reservoir computing is a recently introduced, highly efficient bio-inspired approach for processing time dependent data. The basic scheme of reservoir computing consists of a non linear recurrent dynamical system coupled to a single input layer and a single output layer. Within these constraints many implementations are possible. Here we report an optoelectronic implementation of reservoir computing based on a recently proposed architecture consisting of a single non linear node and a delay line. Our implementation is sufficiently fast for real time information processing. We illustrate its performance on tasks of practical importance such as nonlinear channel equalization and speech recognition, and obtain results comparable to state of the art digital implementations.
| S-EPMC3286854 | biostudies-literature
Project description:Albeit the conceptual simplicity of hardware reservoir computing, the various implementation schemes that have been proposed so far still face versatile challenges. The conceptually simplest implementation uses a time delay approach, where one replaces the ensemble of nonlinear nodes with a unique nonlinear node connected to a delayed feedback loop. This simplification comes at a price in other parts of the implementation; repetitive temporal masking sequences are required to map the input information onto the diverse states of the time delay reservoir. These sequences are commonly introduced by arbitrary waveform generators which is an expensive approach when exploring ultra-fast processing speeds. Here we propose the physical generation of clock-free, sub-nanosecond repetitive patterns, with increased intra-pattern diversity and their use as masking sequences. To that end, we investigate numerically a semiconductor laser with a short optical feedback cavity, a well-studied dynamical system that provides a wide diversity of emitted signals. We focus on those operating conditions that lead to a periodic signal generation, with multiple harmonic frequency tones and sub-nanosecond limit cycle dynamics. By tuning the strength of the different frequency tones in the microwave domain, we access a variety of repetitive patterns and sample them in order to obtain the desired masking sequences. Eventually, we apply them in a time delay reservoir computing approach and test them in a nonlinear time-series prediction task. In a performance comparison with masking sequences that originate from random values, we find that only minor compromises are made while significantly reducing the instrumentation requirements of the time delay reservoir computing system.
| S-EPMC7988145 | biostudies-literature
Project description:BACKGROUND:Using next-generation sequencing (NGS) to guide cancer therapy has created challenges in analyzing and reporting large volumes of genomic data to patients and caregivers. Specifically, providing current, accurate information on newly approved therapies and open clinical trials requires considerable manual curation performed mainly by human "molecular tumor boards" (MTBs). The purpose of this study was to determine the utility of cognitive computing as performed by Watson for Genomics (WfG) compared with a human MTB. MATERIALS AND METHODS:One thousand eighteen patient cases that previously underwent targeted exon sequencing at the University of North Carolina (UNC) and subsequent analysis by the UNCseq informatics pipeline and the UNC MTB between November 7, 2011, and May 12, 2015, were analyzed with WfG, a cognitive computing technology for genomic analysis. RESULTS:Using a WfG-curated actionable gene list, we identified additional genomic events of potential significance (not discovered by traditional MTB curation) in 323 (32%) patients. The majority of these additional genomic events were considered actionable based upon their ability to qualify patients for biomarker-selected clinical trials. Indeed, the opening of a relevant clinical trial within 1 month prior to WfG analysis provided the rationale for identification of a new actionable event in nearly a quarter of the 323 patients. This automated analysis took <3 minutes per case. CONCLUSION:These results demonstrate that the interpretation and actionability of somatic NGS results are evolving too rapidly to rely solely on human curation. Molecular tumor boards empowered by cognitive computing could potentially improve patient care by providing a rapid, comprehensive approach for data analysis and consideration of up-to-date availability of clinical trials. IMPLICATIONS FOR PRACTICE:The results of this study demonstrate that the interpretation and actionability of somatic next-generation sequencing results are evolving too rapidly to rely solely on human curation. Molecular tumor boards empowered by cognitive computing can significantly improve patient care by providing a fast, cost-effective, and comprehensive approach for data analysis in the delivery of precision medicine. Patients and physicians who are considering enrollment in clinical trials may benefit from the support of such tools applied to genomic data.
| S-EPMC5813753 | biostudies-literature
Project description:The connectivity of Artificial Neural Networks (ANNs) is different from the one observed in Biological Neural Networks (BNNs). Can the wiring of actual brains help improve ANNs architectures? Can we learn from ANNs about what network features support computation in the brain when solving a task? At a meso/macro-scale level of the connectivity, ANNs' architectures are carefully engineered and such those design decisions have crucial importance in many recent performance improvements. On the other hand, BNNs exhibit complex emergent connectivity patterns at all scales. At the individual level, BNNs connectivity results from brain development and plasticity processes, while at the species level, adaptive reconfigurations during evolution also play a major role shaping connectivity. Ubiquitous features of brain connectivity have been identified in recent years, but their role in the brain's ability to perform concrete computations remains poorly understood. Computational neuroscience studies reveal the influence of specific brain connectivity features only on abstract dynamical properties, although the implications of real brain networks topologies on machine learning or cognitive tasks have been barely explored. Here we present a cross-species study with a hybrid approach integrating real brain connectomes and Bio-Echo State Networks, which we use to solve concrete memory tasks, allowing us to probe the potential computational implications of real brain connectivity patterns on task solving. We find results consistent across species and tasks, showing that biologically inspired networks perform as well as classical echo state networks, provided a minimum level of randomness and diversity of connections is allowed. We also present a framework, bio2art, to map and scale up real connectomes that can be integrated into recurrent ANNs. This approach also allows us to show the crucial importance of the diversity of interareal connectivity patterns, stressing the importance of stochastic processes determining neural networks connectivity in general.
| S-EPMC9710781 | biostudies-literature
Project description:Reservoir computing is a neuromorphic architecture that may offer viable solutions to the growing energy costs of machine learning. In software-based machine learning, computing performance can be readily reconfigured to suit different computational tasks by tuning hyperparameters. This critical functionality is missing in 'physical' reservoir computing schemes that exploit nonlinear and history-dependent responses of physical systems for data processing. Here we overcome this issue with a 'task-adaptive' approach to physical reservoir computing. By leveraging a thermodynamical phase space to reconfigure key reservoir properties, we optimize computational performance across a diverse task set. We use the spin-wave spectra of the chiral magnet Cu2OSeO3 that hosts skyrmion, conical and helical magnetic phases, providing on-demand access to different computational reservoir responses. The task-adaptive approach is applicable to a wide variety of physical systems, which we show in other chiral magnets via above (and near) room-temperature demonstrations in Co8.5Zn8.5Mn3 (and FeGe).
| S-EPMC10769874 | biostudies-literature
Project description:Reservoir computing (RC) offers efficient temporal information processing with low training cost. All-ferroelectric implementation of RC is appealing because it can fully exploit the merits of ferroelectric memristors (e.g., good controllability); however, this has been undemonstrated due to the challenge of developing ferroelectric memristors with distinctly different switching characteristics specific to the reservoir and readout network. Here, we experimentally demonstrate an all-ferroelectric RC system whose reservoir and readout network are implemented with volatile and nonvolatile ferroelectric diodes (FDs), respectively. The volatile and nonvolatile FDs are derived from the same Pt/BiFeO3/SrRuO3 structure via the manipulation of an imprint field (Eimp). It is shown that the volatile FD with Eimp exhibits short-term memory and nonlinearity while the nonvolatile FD with negligible Eimp displays long-term potentiation/depression, fulfilling the functional requirements of the reservoir and readout network, respectively. Hence, the all-ferroelectric RC system is competent for handling various temporal tasks. In particular, it achieves an ultralow normalized root mean square error of 0.017 in the Hénon map time-series prediction. Besides, both the volatile and nonvolatile FDs demonstrate long-term stability in ambient air, high endurance, and low power consumption, promising the all-ferroelectric RC system as a reliable and low-power neuromorphic hardware for temporal information processing.
| S-EPMC10275999 | biostudies-literature
Project description:Detection in high fidelity of tipping points, the emergence of which is often induced by invisible changes in internal structures or/and external interferences, is paramountly beneficial to understanding and predicting complex dynamical systems (CDSs). Detection approaches, which have been fruitfully developed from several perspectives (e.g., statistics, dynamics, and machine learning), have their own advantages but still encounter difficulties in the face of high-dimensional, fluctuating datasets. Here, using the reservoir computing (RC), a recently notable, resource-conserving machine learning method for reconstructing and predicting CDSs, we articulate a model-free framework to accomplish the detection only using the time series observationally recorded from the underlying unknown CDSs. Specifically, we encode the information of the CDS in consecutive time durations of finite length into the weights of the readout layer in an RC, and then we use the learned weights as the dynamical features and establish a mapping from these features to the system's changes. Our designed framework can not only efficiently detect the changing positions of the system but also accurately predict the intensity change as the intensity information is available in the training data. We demonstrate the efficacy of our supervised framework using the dataset produced by representative physical, biological, and real-world systems, showing that our framework outperforms those traditional methods on the short-term data produced by the time-varying or/and noise-perturbed systems. We believe that our framework, on one hand, complements the major functions of the notable RC intelligent machine and, on the other hand, becomes one of the indispensable methods for deciphering complex systems.
| S-EPMC10317016 | biostudies-literature
Project description:In this paper, we prove the existence of a reservoir that has a finite-dimensional output and makes the reservoir computing model universal. Reservoir computing is a method for dynamical system approximation that trains the static part of a model but fixes the dynamical part called the reservoir. Hence, reservoir computing has the advantage of training models with a low computational cost. Moreover, fixed reservoirs can be implemented as physical systems. Such reservoirs have attracted attention in terms of computation speed and energy consumption. The universality of a reservoir computing model is its ability to approximate an arbitrary system with arbitrary accuracy. Two sufficient reservoir conditions to make the model universal have been proposed. The first is the combination of fading memory and the separation property. The second is the neighborhood separation property, which we proposed recently. To date, it has been unknown whether a reservoir with a finite-dimensional output can satisfy these conditions. In this study, we prove that no reservoir with a finite-dimensional output satisfies the former condition. By contrast, we propose a single output reservoir that satisfies the latter condition. This implies that, for any dimension, a reservoir making the model universal exists with the output of that specified dimension. These results clarify the practical importance of our proposed conditions.
| S-EPMC11006892 | biostudies-literature
Project description:Reservoir computing is a brain-inspired machine learning framework that employs a signal-driven dynamical system, in particular harnessing common-signal-induced synchronization which is a widely observed nonlinear phenomenon. Basic understanding of a working principle in reservoir computing can be expected to shed light on how information is stored and processed in nonlinear dynamical systems, potentially leading to progress in a broad range of nonlinear sciences. As a first step toward this goal, from the viewpoint of nonlinear physics and information theory, we study the memory-nonlinearity trade-off uncovered by Dambre et al. (2012). Focusing on a variational equation, we clarify a dynamical mechanism behind the trade-off, which illustrates why nonlinear dynamics degrades memory stored in dynamical system in general. Moreover, based on the trade-off, we propose a mixture reservoir endowed with both linear and nonlinear dynamics and show that it improves the performance of information processing. Interestingly, for some tasks, significant improvements are observed by adding a few linear dynamics to the nonlinear dynamical system. By employing the echo state network model, the effect of the mixture reservoir is numerically verified for a simple function approximation task and for more complex tasks.
| S-EPMC5579006 | biostudies-literature