Dataset Information

Efficient computation of optimal actions.

ABSTRACT: Optimal choice of actions is a fundamental problem relevant to fields as diverse as neuroscience, psychology, economics, computer science, and control engineering. Despite this broad relevance the abstract setting is similar: we have an agent choosing actions over time, an uncertain dynamical system whose state is affected by those actions, and a performance criterion that the agent seeks to optimize. Solving problems of this kind remains hard, in part, because of overly generic formulations. Here, we propose a more structured formulation that greatly simplifies the construction of optimal control laws in both discrete and continuous domains. An exhaustive search over actions is avoided and the problem becomes linear. This yields algorithms that outperform Dynamic Programming and Reinforcement Learning, and thereby solve traditional problems more efficiently. Our framework also enables computations that were not possible before: composing optimal control laws by mixing primitives, applying deterministic methods to stochastic systems, quantifying the benefits of error tolerance, and inferring goals from behavioral data via convex optimization. Development of a general class of easily solvable problems tends to accelerate progress--as linear systems theory has done, for example. Our framework may have similar impact in fields where optimal choice of actions is relevant.

SUBMITTER: Todorov E

PROVIDER: S-EPMC2705278 | biostudies-literature | 2009 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Efficient computation of optimal actions.

Todorov Emanuel E

Proceedings of the National Academy of Sciences of the United States of America 20090702 28

Optimal choice of actions is a fundamental problem relevant to fields as diverse as neuroscience, psychology, economics, computer science, and control engineering. Despite this broad relevance the abstract setting is similar: we have an agent choosing actions over time, an uncertain dynamical system whose state is affected by those actions, and a performance criterion that the agent seeks to optimize. Solving problems of this kind remains hard, in part, because of overly generic formulations. He ...[more]

PMID: 19574462

Similar Datasets

Project description:Clot formation is a crucial process that prevents bleeding, but can lead to severe disorders when imbalanced. This process is regulated by the coagulation cascade, a biochemical network that controls the enzyme thrombin, which converts soluble fibrinogen into the fibrin fibers that constitute clots. Coagulation cascade models are typically complex and involve dozens of partial differential equations (PDEs) representing various chemical species' transport, reaction kinetics, and diffusion. Solving these PDE systems computationally is challenging, due to their large size and multi-scale nature. We propose a multi-fidelity strategy to increase the efficiency of coagulation cascade simulations. Leveraging the slower dynamics of molecular diffusion, we transform the governing PDEs into ordinary differential equations (ODEs) representing the evolution of species concentrations versus blood residence time. We then Taylor-expand the ODE solution around the zero-diffusivity limit to obtain spatiotemporal maps of species concentrations in terms of the statistical moments of residence time, [Formula: see text], and provide the governing PDEs for [Formula: see text]. This strategy replaces a high-fidelity system of N PDEs representing the coagulation cascade of N chemical species by N ODEs and p PDEs governing the residence time statistical moments. The multi-fidelity order (p) allows balancing accuracy and computational cost providing a speedup of over N/p compared to high-fidelity models. Moreover, this cost becomes independent of the number of chemical species in the large computational meshes typical of the arterial and cardiac chamber simulations. Using a coagulation network with N = 9 and an idealized aneurysm geometry with a pulsatile flow as a benchmark, we demonstrate favorable accuracy for low-order models of p = 1 and p = 2. The thrombin concentration in these models departs from the high-fidelity solution by under 20% (p = 1) and 2% (p = 2) after 20 cardiac cycles. These multi-fidelity models could enable new coagulation analyses in complex flow scenarios and extensive reaction networks. Furthermore, it could be generalized to advance our understanding of other reacting systems affected by flow.

Dataset Information

Efficient computation of optimal actions.

Publications

Efficient computation of optimal actions.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets