Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Deep reinforcement learning for optimal experimental design in biology.

ABSTRACT: The field of optimal experimental design uses mathematical techniques to determine experiments that are maximally informative from a given experimental setup. Here we apply a technique from artificial intelligence-reinforcement learning-to the optimal experimental design task of maximizing confidence in estimates of model parameter values. We show that a reinforcement learning approach performs favourably in comparison with a one-step ahead optimisation algorithm and a model predictive controller for the inference of bacterial growth parameters in a simulated chemostat. Further, we demonstrate the ability of reinforcement learning to train over a distribution of parameters, indicating that this approach is robust to parametric uncertainty.

SUBMITTER: Treloar NJ

PROVIDER: S-EPMC9721483 | biostudies-literature | 2022 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Deep reinforcement learning for optimal experimental design in biology.

Treloar Neythen J NJ Braniff Nathan N Ingalls Brian B Barnes Chris P CP

PLoS computational biology 20221121 11

The field of optimal experimental design uses mathematical techniques to determine experiments that are maximally informative from a given experimental setup. Here we apply a technique from artificial intelligence-reinforcement learning-to the optimal experimental design task of maximizing confidence in estimates of model parameter values. We show that a reinforcement learning approach performs favourably in comparison with a one-step ahead optimisation algorithm and a model predictive controlle ...[more]

PMID: 36409776

Similar Datasets

Optimal Hierarchical Learning Path Design With Reinforcement Learning.

Project description:E-learning systems are capable of providing more adaptive and efficient learning experiences for learners than traditional classroom settings. A key component of such systems is the learning policy. The learning policy is an algorithm that designs the learning paths or rather it selects learning materials for learners based on information such as the learners' current progresses and skills, learning material contents. In this article, the authors address the problem of finding the optimal learning policy. To this end, a model for learners' hierarchical skills in the E-learning system is first developed. Based on the hierarchical skill model and the classical cognitive diagnosis model, a framework to model various mastery levels related to hierarchical skills is further developed. The optimal learning path in consideration of the hierarchical structure of skills is found by applying a model-free reinforcement learning method, which does not require any assumption about learners' learning transition processes. The effectiveness of the proposed framework is demonstrated via simulation studies.

| S-EPMC7711250 | biostudies-literature

Deep reinforcement learning for de novo drug design.

Project description:We have devised and implemented a novel computational strategy for de novo design of molecules with desired properties termed ReLeaSE (Reinforcement Learning for Structural Evolution). On the basis of deep and reinforcement learning (RL) approaches, ReLeaSE integrates two deep neural networks-generative and predictive-that are trained separately but are used jointly to generate novel targeted chemical libraries. ReLeaSE uses simple representation of molecules by their simplified molecular-input line-entry system (SMILES) strings only. Generative models are trained with a stack-augmented memory network to produce chemically feasible SMILES strings, and predictive models are derived to forecast the desired properties of the de novo-generated compounds. In the first phase of the method, generative and predictive models are trained separately with a supervised learning algorithm. In the second phase, both models are trained jointly with the RL approach to bias the generation of new chemical structures toward those with the desired physical and/or biological properties. In the proof-of-concept study, we have used the ReLeaSE method to design chemical libraries with a bias toward structural complexity or toward compounds with maximal, minimal, or specific range of physical properties, such as melting point or hydrophobicity, or toward compounds with inhibitory activity against Janus protein kinase 2. The approach proposed herein can find a general use for generating targeted chemical libraries of novel compounds optimized for either a single desired property or multiple properties.

| S-EPMC6059760 | biostudies-literature

Molecular de-novo design through deep reinforcement learning.

Project description:This work introduces a method to tune a sequence-based generative model for molecular de novo design that through augmented episodic likelihood can learn to generate structures with certain specified desirable properties. We demonstrate how this model can execute a range of tasks such as generating analogues to a query structure and generating compounds predicted to be active against a biological target. As a proof of principle, the model is first trained to generate molecules that do not contain sulphur. As a second example, the model is trained to generate analogues to the drug Celecoxib, a technique that could be used for scaffold hopping or library expansion starting from a single molecule. Finally, when tuning the model towards generating compounds predicted to be active against the dopamine receptor type 2, the model generates structures of which more than 95% are predicted to be active, including experimentally confirmed actives that have not been included in either the generative model nor the activity prediction model. Graphical abstract .

| S-EPMC5583141 | biostudies-literature

Near-optimal experimental design for model selection in systems biology.

Project description:MotivationBiological systems are understood through iterations of modeling and experimentation. Not all experiments, however, are equally valuable for predictive modeling. This study introduces an efficient method for experimental design aimed at selecting dynamical models from data. Motivated by biological applications, the method enables the design of crucial experiments: it determines a highly informative selection of measurement readouts and time points.ResultsWe demonstrate formal guarantees of design efficiency on the basis of previous results. By reducing our task to the setting of graphical models, we prove that the method finds a near-optimal design selection with a polynomial number of evaluations. Moreover, the method exhibits the best polynomial-complexity constant approximation factor, unless P = NP. We measure the performance of the method in comparison with established alternatives, such as ensemble non-centrality, on example models of different complexity. Efficient design accelerates the loop between modeling and experimentation: it enables the inference of complex mechanisms, such as those controlling central metabolic operation.AvailabilityToolbox 'NearOED' available with source code under GPL on the Machine Learning Open Source Software Web site (mloss.org).

| S-EPMC3789540 | biostudies-literature

Optimal adaptive allocation using deep reinforcement learning in a dose-response study.

Project description:Estimation of the dose-response curve for efficacy and subsequent selection of an appropriate dose in phase II trials are important processes in drug development. Various methods have been investigated to estimate dose-response curves. Generally, these methods are used with equal allocation of subjects for simplicity; nevertheless, they may not fully optimize performance metrics because of nonoptimal allocation. Optimal allocation methods, which include adaptive allocation methods, have been proposed to overcome the limitations of equal allocation. However, they rely on asymptotics, and thus sometimes cannot efficiently optimize the performance metric with the sample size in an actual clinical trial. The purpose of this study is to construct an adaptive allocation rule that directly optimizes a performance metric, such as power, accuracy of model selection, accuracy of the estimated target dose, or mean absolute error over the estimated dose-response curve. We demonstrate that deep reinforcement learning with an appropriately defined state and reward can be used to construct such an adaptive allocation rule. The simulation study shows that the proposed method can successfully improve the performance metric to be optimized when compared with the equal allocation, D-optimal, and TD-optimal methods. In particular, when the mean absolute error was set to the metric to be optimized, it is possible to construct a rule that is superior for many metrics.

| S-EPMC9298337 | biostudies-literature

Multi-Objective Optimal Trajectory Planning for Robotic Arms Using Deep Reinforcement Learning.

Project description:This study investigated the trajectory-planning problem of a six-axis robotic arm based on deep reinforcement learning. Taking into account several characteristics of robot motion, a multi-objective optimization approach is proposed, which was based on the motivations of deep reinforcement learning and optimal planning. The optimal trajectory was considered with respect to multiple objectives, aiming to minimize factors such as accuracy, energy consumption, and smoothness. The multiple objectives were integrated into the reinforcement learning environment to achieve the desired trajectory. Based on forward and inverse kinematics, the joint angles and Cartesian coordinates were used as the input parameters, while the joint angle estimation served as the output. To enable the environment to rapidly find more-efficient solutions, the decaying episode mechanism was employed throughout the training process. The distribution of the trajectory points was improved in terms of uniformity and smoothness, which greatly contributed to the optimization of the robotic arm's trajectory. The proposed method demonstrated its effectiveness in comparison with the RRT algorithm, as evidenced by the simulations and physical experiments.

| S-EPMC10346668 | biostudies-literature

Deep reinforcement learning extracts the optimal sepsis treatment policy from treatment records.

Project description:BackgroundSepsis is one of the most life-threatening medical conditions. Therefore, many clinical trials have been conducted to identify optimal treatment strategies for sepsis. However, finding reliable strategies remains challenging due to limited-scale clinical tests. Here we tried to extract the optimal sepsis treatment policy from accumulated treatment records.MethodsIn this study, with our modified deep reinforcement learning algorithm, we stably generated a patient treatment artificial intelligence model. As training data, 16,744 distinct admissions in tertiary hospitals were used and tested with separate datasets. Model performance was tested by t test and visualization of estimated survival rates. We also analyze model behavior using the confusion matrix, important feature extraction by a random forest decision tree, and treatment behavior comparison to understand how our treatment model achieves high performance.ResultsHere we show that our treatment model's policy achieves a significantly higher estimated survival rate (up to 10.03%). We also show that our models' vasopressor treatment was quite different from that of physicians. Here, we identify that blood urea nitrogen, age, sequential organ failure assessment score, and shock index are the most different factors in dealing with sepsis patients between our model and physicians.ConclusionsOur results demonstrate that the patient treatment model can extract potential optimal sepsis treatment policy. We also extract core information about sepsis treatment by analyzing its policy. These results may not apply directly in clinical settings because they were only tested on a database. However, they are expected to serve as important guidelines for further research.

| S-EPMC11584651 | biostudies-literature

Molecular Design in Synthetically Accessible Chemical Space via Deep Reinforcement Learning.

Project description:The fundamental goal of generative drug design is to propose optimized molecules that meet predefined activity, selectivity, and pharmacokinetic criteria. Despite recent progress, we argue that existing generative methods are limited in their ability to favorably shift the distributions of molecular properties during optimization. We instead propose a novel Reinforcement Learning framework for molecular design in which an agent learns to directly optimize through a space of synthetically accessible drug-like molecules. This becomes possible by defining transitions in our Markov decision process as chemical reactions and allows us to leverage synthetic routes as an inductive bias. We validate our method by demonstrating that it outperforms existing state-of-the-art approaches in the optimization of pharmacologically relevant objectives, while results on multi-objective optimization tasks suggest increased scalability to realistic pharmaceutical design problems.

| S-EPMC7774092 | biostudies-literature

NLoed: A Python Package for Nonlinear Optimal Experimental Design in Systems Biology.

Project description:Modeling in systems and synthetic biology relies on accurate parameter estimates and predictions. Accurate model calibration relies, in turn, on data and on how well suited the available data are to a particular modeling task. Optimal experimental design (OED) techniques can be used to identify experiments and data collection procedures that will most efficiently contribute to a given modeling objective. However, implementation of OED is limited by currently available software tools that are not well suited for the diversity of nonlinear models and non-normal data commonly encountered in biological research. Moreover, existing OED tools do not make use of the state-of-the-art numerical tools, resulting in inefficient computation. Here, we present the NLoed software package and demonstrate its use with in vivo data from an optogenetic system in Escherichia coli. NLoed is an open-source Python library providing convenient access to OED methods, with particular emphasis on experimental design for systems biology research. NLoed supports a wide variety of nonlinear, multi-input/output, and dynamic models and facilitates modeling and design of experiments over a wide variety of data types. To support OED investigations, the NLoed package implements maximum likelihood fitting and diagnostic tools, providing a comprehensive modeling workflow. NLoed offers an accessible, modular, and flexible OED tool set suited to the wide variety of experimental scenarios encountered in systems biology research. We demonstrate NLoed's capabilities by applying it to experimental design for characterization of a bacterial optogenetic system.

| S-EPMC9765746 | biostudies-literature

A value-based deep reinforcement learning model with human expertise in optimal treatment of sepsis.

Project description:Deep Reinforcement Learning (DRL) has been increasingly attempted in assisting clinicians for real-time treatment of sepsis. While a value function quantifies the performance of policies in such decision-making processes, most value-based DRL algorithms cannot evaluate the target value function precisely and are not as safe as clinical experts. In this study, we propose a Weighted Dueling Double Deep Q-Network with embedded human Expertise (WD3QNE). A target Q value function with adaptive dynamic weight is designed to improve the estimate accuracy and human expertise in decision-making is leveraged. In addition, the random forest algorithm is employed for feature selection to improve model interpretability. We test our algorithm against state-of-the-art value function methods in terms of expected return, survival rate, action distribution and external validation. The results demonstrate that WD3QNE obtains the highest survival rate of 97.81% in MIMIC-III dataset. Our proposed method is capable of providing reliable treatment decisions with embedded clinician expertise.

| S-EPMC9894526 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data