Net-Net Auto Machine Learning (AutoML) Prediction of Complex Ecosystems.
ABSTRACT: Biological Ecosystem Networks (BENs) are webs of biological species (nodes) establishing trophic relationships (links). Experimental confirmation of all possible links is difficult and generates a huge volume of information. Consequently, computational prediction becomes an important goal. Artificial Neural Networks (ANNs) are Machine Learning (ML) algorithms that may be used to predict BENs, using as input Shannon entropy information measures (Shk) of known ecosystems to train them. However, it is difficult to select a priori which ANN topology will have a higher accuracy. Interestingly, Auto Machine Learning (AutoML) methods focus on the automatic selection of the more efficient ML algorithms for specific problems. In this work, a preliminary study of a new approach to AutoML selection of ANNs is proposed for the prediction of BENs. We call it the Net-Net AutoML approach, because it uses for the first time Shk values of both networks involving BENs (networks to be predicted) and ANN topologies (networks to be tested). Twelve types of classifiers have been tested for the Net-Net model including linear, Bayesian, trees-based methods, multilayer perceptrons and deep neuronal networks. The best Net-Net AutoML model for 338,050 outputs of 10 ANN topologies for links of 69 BENs was obtained with a deep fully connected neuronal network, characterized by a test accuracy of 0.866 and a test AUROC of 0.935. This work paves the way for the application of Net-Net AutoML to other systems or ML algorithms.
Project description:Discoveries in biological neural networks (BNNs) shaped artificial neural networks (ANNs) and computational parallels between ANNs and BNNs have recently been discovered. However, it is unclear to what extent discoveries in ANNs can give insight into BNN function. Here, we designed and trained an ANN to perform heat gradient navigation and found striking similarities in computation and heat representation to a known zebrafish BNN. This included shared ON- and OFF-type representations of absolute temperature and rates of change. Importantly, ANN function critically relied on zebrafish-like units. We furthermore used the accessibility of the ANN to discover a new temperature-responsive cell type in the zebrafish cerebellum. Finally, constraining the ANN by the C. elegans motor repertoire retuned sensory representations indicating that our approach generalizes. Together, these results emphasize convergence of ANNs and BNNs on stereotypical representations and that ANNs form a powerful tool to understand their biological counterparts.
Project description:Artificial Neural Networks (ANNs) model and Adaptive Neuro-Fuzzy Inference System (ANFIS) were used to estimate and predict the removal efficiency of tetracycline (TC) using the adsorption process from aqueous solutions. The obtained results demonstrated that the optimum condition for removal efficiency of TC were 1.5 g L-1 modified zeolite (MZ), pH of 8.0, initial TC concentration of 10.0 mg L-1, and reaction time of 60 min. Among the different back-propagation algorithms, the Marquardt-Levenberg learning algorithm was selected for ANN Model. The log sigmoid transfer function (log sig) at the hidden layer with ten neurons in the first layer and a linear transfer function were used for prediction of the removal efficiency. Accordingly, a correlation coefficient, mean square error, and absolute error percentage of 0.9331, 0.0017, and 0.56% were obtained for the total dataset, respectively. The results revealed that the ANN has great performance in predicting the removal efficiency of TC.•ANNs used to estimate and predict tetracycline antibiotic removal using the adsorption process from aqueous solutions.•The model's predictive performance evaluated by MSE, MAPE, and R2.
Project description:Artificial neural networks (ANNs) are powerful computational tools that are designed to replicate the human brain and adopted to solve a variety of problems in many different fields. Fault tolerance (FT), an important property of ANNs, ensures their reliability when significant portions of a network are lost. In this paper, a fault/noise injection-based (FIB) genetic algorithm (GA) is proposed to construct fault-tolerant ANNs. The FT performance of an FIB-GA was compared with that of a common genetic algorithm, the back-propagation algorithm, and the modification of weights algorithm. The FIB-GA showed a slower fitting speed when solving the exclusive OR (XOR) problem and the overlapping classification problem, but it significantly reduced the errors in cases of single or multiple faults in ANN weights or nodes. Further analysis revealed that the fit weights showed no correlation with the fitting errors in the ANNs constructed with the FIB-GA, suggesting a relatively even distribution of the various fitting parameters. In contrast, the output weights in the training of ANNs implemented with the use the other three algorithms demonstrated a positive correlation with the errors. Our findings therefore indicate that a combination of the fault/noise injection-based method and a GA is capable of introducing FT to ANNs and imply that the distributed ANNs demonstrate superior FT performance.
Project description:Drug resistance in HIV treatment is still a worldwide problem. Predicting resistance to antiretrovirals (ARVs) before starting any treatment is important. Prediction accuracy is essential, as low-accuracy predictions increase the risk of prescribing sub-optimal drug regimens leading to patients developing resistance sooner. Artificial Neural Networks (ANNs) are a powerful tool that would be able to assist in drug resistance prediction. In this study, we constrained the dataset to subtype B, sacrificing generalizability for a higher predictive performance, and demonstrated that the predictive quality of the ANN regression models have definite improvement for most ARVs.Trained regression ANNs were optimized for eight protease inhibitors, six nucleoside reverse transcriptase (RT) inhibitors and four non-nucleoside RT inhibitors by experimenting combinations of rare variant filtering (none versus 1 residue occurrence) and ANN topologies (1-3 hidden layers with 2, 4, 6, 8 and 10 nodes per layer). Single hidden layers (5-20 nodes) were used for training where overfitting was detected. 5-fold cross-validation produced mean R2 values over 0.95 and standard deviations lower than 0.04 for all but two antiretrovirals.Overall, higher accuracies and lower variances (compared to results published in 2016) were obtained by experimenting with various preprocessing methods, while focusing on the most prevalent subtype in the raw dataset (subtype B).We thus highlight the need to develop and make available subtype-specific datasets for developing higher accuracy in drug-resistance prediction methods.
Project description:Spiking neural networks (SNNs) are potentially highly efficient models for inference on fully parallel neuromorphic hardware, but existing training methods that convert conventional artificial neural networks (ANNs) into SNNs are unable to exploit these advantages. Although ANN-to-SNN conversion has achieved state-of-the-art accuracy for static image classification tasks, the following subtle but important difference in the way SNNs and ANNs integrate information over time makes the direct application of conversion techniques for sequence processing tasks challenging. Whereas all connections in SNNs have a certain propagation delay larger than zero, ANNs assign different roles to feed-forward connections, which immediately update all neurons within the same time step, and recurrent connections, which have to be rolled out in time and are typically assigned a delay of one time step. Here, we present a novel method to obtain highly accurate SNNs for sequence processing by modifying the ANN training before conversion, such that delays induced by ANN rollouts match the propagation delays in the targeted SNN implementation. Our method builds on the recently introduced framework of streaming rollouts, which aims for fully parallel model execution of ANNs and inherently allows for temporal integration by merging paths of different delays between input and output of the network. The resulting networks achieve state-of-the-art accuracy for multiple event-based benchmark datasets, including N-MNIST, CIFAR10-DVS, N-CARS, and DvsGesture, and through the use of spatio-temporal shortcut connections yield low-latency approximate network responses that improve over time as more of the input sequence is processed. In addition, our converted SNNs are consistently more energy-efficient than their corresponding ANNs.
Project description:Humans and most animals can learn new tasks without forgetting old ones. However, training artificial neural networks (ANNs) on new tasks typically causes them to forget previously learned tasks. This phenomenon is the result of "catastrophic forgetting," in which training an ANN disrupts connection weights that were important for solving previous tasks, degrading task performance. Several recent studies have proposed methods to stabilize connection weights of ANNs that are deemed most important for solving a task, which helps alleviate catastrophic forgetting. Here, drawing inspiration from algorithms that are believed to be implemented in vivo, we propose a complementary method: adding a context-dependent gating signal, such that only sparse, mostly nonoverlapping patterns of units are active for any one task. This method is easy to implement, requires little computational overhead, and allows ANNs to maintain high performance across large numbers of sequentially presented tasks, particularly when combined with weight stabilization. We show that this method works for both feedforward and recurrent network architectures, trained using either supervised or reinforcement-based learning. This suggests that using multiple, complementary methods, akin to what is believed to occur in the brain, can be a highly effective strategy to support continual learning.
Project description:Phages play critical roles in the survival and pathogenicity of their hosts, via lysogenic conversion factors, and in nutrient redistribution, via cell lysis. Analyses of phage- and viral-encoded genes in environmental samples provide insights into the physiological impact of viruses on microbial communities and human health. However, phage ORFs are extremely diverse of which over 70% of them are dissimilar to any genes with annotated functions in GenBank. Better identification of viruses would also aid in better detection and diagnosis of disease, in vaccine development, and generally in better understanding the physiological potential of any environment. In contrast to enzymes, viral structural protein function can be much more challenging to detect from sequence data because of low sequence conservation, few known conserved catalytic sites or sequence domains, and relatively limited experimental data. We have designed a method of predicting phage structural protein sequences that uses Artificial Neural Networks (ANNs). First, we trained ANNs to classify viral structural proteins using amino acid frequency; these correctly classify a large fraction of test cases with a high degree of specificity and sensitivity. Subsequently, we added estimates of protein isoelectric points as a feature to ANNs that classify specialized families of proteins, namely major capsid and tail proteins. As expected, these more specialized ANNs are more accurate than the structural ANNs. To experimentally validate the ANN predictions, several ORFs with no significant similarities to known sequences that are ANN-predicted structural proteins were examined by transmission electron microscopy. Some of these self-assembled into structures strongly resembling virion structures. Thus, our ANNs are new tools for identifying phage and potential prophage structural proteins that are difficult or impossible to detect by other bioinformatic analysis. The networks will be valuable when sequence is available but in vitro propagation of the phage may not be practical or possible.
Project description:BACKGROUND:The prognosis for patients with severe acute lower intestinal bleeding (ALIB) may be assessed by complex artificial neural networks (ANNs) or user-friendly regression-based models. Comparisons between these modalities are limited, and predicting the need for surgical intervention remains elusive. We hypothesized that ANNs would outperform the Strate rule to predict severe bleeding and would also predict the need for surgical intervention. METHODS:We performed a 4-y retrospective analysis of 147 adult patients who underwent endoscopy, angiography, or surgery for ALIB. Baseline characteristics, Strate risk factors, management parameters, and outcomes were analyzed. The primary outcomes were severe bleeding and surgical intervention. ANNs were created in SPSS. Models were compared by area under the receiver operating characteristic curve (AUROC) with 95% confidence intervals. RESULTS:The number of Strate risk factors for each patient correlated significantly with the outcome of severe bleeding (r = 0.29, P < 0.001). However, the Strate model was less accurate than an ANN (AUROC 0.66 [0.57-0.75] versus 0.98 [0.95-1.00], respectively) which incorporated six variables present on admission: hemoglobin, systolic blood pressure, outpatient prescription for Aspirin 325 mg daily, Charlson comorbidity index, base deficit ?5 mEq/L, and international normalized ratio ?1.5. A similar ANN including hemoglobin nadir and the occurrence of a 20% decrease in hematocrit was effective in predicting the need for surgery (AUROC 0.95 [0.90-1.00]). CONCLUSIONS:The Strate prediction rule effectively stratified risk for severe ALIB, but was less accurate than an ANN. A separate ANN accurately predicted the need for surgery by combining risk factors for severe bleeding with parameters quantifying blood loss. Optimal prognostication may be achieved by integrating pragmatic regression-based calculators for quick decisions at the bedside and highly accurate ANNs when time and resources permit.
Project description:BACKGROUND:Accurate prediction of operative transfusions is essential for resource allocation and identifying patients at risk of postoperative adverse events. This research examines the efficacy of using artificial neural networks (ANNs) to predict transfusions for all inpatient operations. METHODS:Over 1.6 million surgical cases over a two year period from the NSQIP-PUF database are used. Data from 2014 (750937 records) are used for model development and data from 2015 (885502 records) are used for model validation. ANN and regression models are developed to predict perioperative transfusions for surgical patients. RESULTS:Various ANN models and logistic regression, using four variable sets, are compared. The best performing ANN models with respect to both sensitivity and area under the receiver operator characteristic curve outperformed all of the regression models (p < .001) and achieved a performance of 70-80% specificity with a corresponding 75-62% sensitivity. CONCLUSION:ANNs can predict >75% of the patients who will require transfusion and 70% of those who will not. Increasing specificity to 80% still enables a sensitivity of almost 67%. The unique contribution of this research is the utilization of a single ANN model to predict transfusions across a broad range of surgical procedures.
Project description:This is a survey of neural network applications in the real-world scenario. It provides a taxonomy of artificial neural networks (ANNs) and furnish the reader with knowledge of current and emerging trends in ANN applications research and area of focus for researchers. Additionally, the study presents ANN application challenges, contributions, compare performances and critiques methods. The study covers many applications of ANN techniques in various disciplines which include computing, science, engineering, medicine, environmental, agriculture, mining, technology, climate, business, arts, and nanotechnology, etc. The study assesses ANN contributions, compare performances and critiques methods. The study found that neural-network models such as feedforward and feedback propagation artificial neural networks are performing better in its application to human problems. Therefore, we proposed feedforward and feedback propagation ANN models for research focus based on data analysis factors like accuracy, processing speed, latency, fault tolerance, volume, scalability, convergence, and performance. Moreover, we recommend that instead of applying a single method, future research can focus on combining ANN models into one network-wide application.