Combining brain-computer interfaces with deep reinforcement learning for robot training: a feasibility study in a simulation environment
Ontology highlight
ABSTRACT:
SUBMITTER: Vukelic M
PROVIDER: S-EPMC10790930 | biostudies-literature | 2023 Jan
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Quadruped robots require robust and general locomotion skills to exploit their mobility potential in complex and challenging environments. In this work, we present an implementation of a robust end-to-end learning-based controller on the Solo12 quadruped. Our method is based on deep reinforcement learning of joint impedance references. The resulting control policies follow a commanded velocity reference while being efficient in its energy consumption and easy to deploy. We detail the learning procedure and method for transfer on the real robot. We show elaborate experiments. Finally, we present experimental results of the learned locomotion on various grounds indoors and outdoors. These results show that the Solo12 robot is a suitable open-source platform for research combining learning and control because of the easiness in transferring and deploying learned controllers.
| S-EPMC10366154 | biostudies-literature
Project description:In a brain-computer interface (BCI) system, the testing of decoding algorithms, tasks, and their parameters is critical for optimizing performance. However, conducting human experiments can be costly and time-consuming, especially when investigating broad sets of parameters. Attempts to utilize previously collected data in offline analysis lack a co-adaptive feedback loop between the system and the user present online, limiting the applicability of the conclusions obtained to real-world uses of BCI. As such, a number of studies have attempted to address this cost-wise middle ground between offline and live experimentation with real-time neural activity simulators. We present one such system which generates motor imagery electroencephalography (EEG) via forward modeling and novel motor intention encoding models for conducting sensorimotor rhythm (SMR)-based continuous cursor control experiments in a closed-loop setting. We use the proposed simulator with 10 healthy human subjects to test the effect of three decoder and task parameters across 10 different values. Our simulated approach produces similar statistical conclusions to those produced during parallel, paired, online experimentation, but in 55% of the time. Notably, both online and simulated experimentation expressed a positive effect of cursor velocity limit on performance regardless of subject average performance, supporting the idea of relaxing constraints on cursor gain in online continuous cursor control. We demonstrate the merits of our closed-loop motor imagery EEG simulation, and provide an open-source framework to the community for closed-loop SMR-based BCI studies in the future. All code including the simulator have been made available on GitHub.
| S-EPMC9428352 | biostudies-literature
Project description:Biologically inspired jumping robots exhibit exceptional movement capabilities and can quickly overcome obstacles. However, the stability and accuracy of jumping movements are significantly compromised by rapid changes in posture. Here, we propose a stable jumping control algorithm for a locust-inspired jumping robot based on deep reinforcement learning. The algorithm utilizes a training framework comprising two neural network modules (actor network and critic network) to enhance training performance. The framework can control jumping by directly mapping the robot's observations (robot position and velocity, obstacle position, target position, etc.) to its joint torques. The control policy increases randomness and exploration by introducing an entropy term to the policy function. Moreover, we designed a stage incentive mechanism to adjust the reward function dynamically, thereby improving the robot's jumping stability and accuracy. We established a locus-inspired jumping robot platform and conducted a series of jumping experiments in simulation. The results indicate that the robot could perform smooth and non-flip jumps, with the error of the distance from the target remaining below 3%. The robot consumed 44.6% less energy to travel the same distance by jumping compared with walking. Additionally, the proposed algorithm exhibited a faster convergence rate and improved convergence effects compared with other classical algorithms.
| S-EPMC11430585 | biostudies-literature
Project description:In the severity of myocardial infarction, Toll-like receptor (TLR) 9 plays a pivotal role in the inflammatory responses induced through damage-associated molecular patterns involving mitochondrial DNA and HMGB1. However, the therapeutic agents currently available for myocardial infarction substantially lack any association with the effects on immune responses. In this study, we applied a comprehensive drug discovery approach, which integrates in silico, in vitro, and in vivo processes, as well as determined therapeutic effects and mechanisms free from side effects. In an animal model of myocardial infarction, conditioned based on pharmacological properties, improved effects compared to the FDA-approved Rosuvastatin were detected. In summary, a small molecular inhibitor, ETA53 (endosomal Toll-like receptor antagonist 53), which selectively acts on TLR9 in nano-molar units, was developed. ETA53 was found to be effective for treating myocardial infarction caused by permanent ligation of the left anterior descending coronary artery, based on indicators such as cardiac function, inflammation, infarct size, and fibrosis. The results offered insights that varied from those of conventional drug development perspectives; consequently, the novel inhibitor may provide an alternative treatment with a mechanism different from that of the commercially available drugs for myocardial infarction.
2023-11-12 | GSE247311 | GEO
Project description:Bio-inspired solutions devised for autonomous underwater robots are currently being investigated by researchers worldwide as a way to improve propulsion. Despite efforts to harness the substantial potential payoffs of marine animal locomotion, biological system performance still has far to go. In order to address this very ambitious objective, the authors of this study designed and manufactured a series of ostraciiform swimming robots over the past three years. However, the pursuit of the maximum propulsive efficiency by which to maximize robot autonomy while maintaining acceptable maneuverability ultimately drove us to improve our design and move from ostraciiform to carangiform locomotion. In order to comply with the tail motion required by the aforementioned swimmers, the authors designed a transmission system capable of converting the continuous rotation of a single motor in the travelling wave-shaped undulations of a multijoint serial mechanism. The propulsive performance of the resulting thruster (i.e., the caudal fin), which constitutes the mechanism end effector, was investigated by means of computational fluid dynamics techniques. Finally, in order to compute the resulting motion of the robot, numerical predictions were integrated into a multibody model that also accounted for the mass distribution inside the robotic swimmer and the hydrodynamic forces resulting from the relative motion between its body and the surrounding fluid. Dynamic analysis allowed the performance of the robotic propulsion to be computed while in the cruising condition.
| S-EPMC7709489 | biostudies-literature
Project description:We present a novel approach based on deep learning for decoding sensory information from non-invasively recorded Electroencephalograms (EEG). It can either be used in a passive Brain-Computer Interface (BCI) to predict properties of a visual stimulus the person is viewing, or it can be used to actively control a BCI application. Both scenarios were tested, whereby an average information transfer rate (ITR) of 701 bit/min was achieved for the passive BCI approach with the best subject achieving an online ITR of 1237 bit/min. Further, it allowed the discrimination of 500,000 different visual stimuli based on only 2 seconds of EEG data with an accuracy of up to 100%. When using the method for an asynchronous self-paced BCI for spelling, an average utility rate of 175 bit/min was achieved, which corresponds to an average of 35 error-free letters per minute. As the presented method extracts more than three times more information than the previously fastest approach, we suggest that EEG signals carry more information than generally assumed. Finally, we observed a ceiling effect such that information content in the EEG exceeds that required for BCI control, and therefore we discuss if BCI research has reached a point where the performance of non-invasive visual BCI control cannot be substantially improved anymore.
| S-EPMC6730910 | biostudies-literature
Project description:Traditionally, precision medicine involves classifying patients to identify subpopulations that respond favorably to specific therapeutics. We pose precision medicine as a dynamic feedback control problem, where treatment administered to a patient is guided by measurements taken during the course of treatment. We consider sepsis, a life-threatening condition in which dysregulation of the immune system causes tissue damage. We leverage an existing simulation of the innate immune response to infection and apply deep reinforcement learning (DRL) to discover an adaptive personalized treatment policy that specifies effective multicytokine therapy to simulated sepsis patients based on systemic measurements. The learned policy achieves a dramatic reduction in mortality rate over a set of 500 simulated patients relative to standalone antibiotic therapy. Advantages of our approach are threefold: (1) the use of simulation allows exploring therapeutic strategies beyond clinical practice and available data, (2) advances in DRL accommodate learning complex therapeutic strategies for complex biological systems, and (3) optimized treatments respond to a patient's individual disease progression over time, therefore, capturing both differences across patients and the inherent randomness of disease progression within a single patient. We hope that this work motivates both considering adaptive personalized multicytokine mediation therapy for sepsis and exploiting simulation with DRL for precision medicine more broadly.
| S-EPMC6590719 | biostudies-literature
Project description:Membrane-integral pyrophosphatases (mPPases) couple the hydrolysis of pyrophosphate (PPi) to the pumping of Na+, H+, or both these ions across a membrane. Recently solved structures of the Na+-pumping Thermotoga maritima mPPase (TmPPase) and H+-pumping Vigna radiata mPPase revealed the basis of ion selectivity between these enzymes and provided evidence for the mechanisms of substrate hydrolysis and ion-pumping. Our atomistic molecular dynamics (MD) simulations of TmPPase demonstrate that loop 5-6 is mobile in the absence of the substrate or substrate-analogue bound to the active site, explaining the lack of electron density for this loop in resting state structures. Furthermore, creating an apo model of TmPPase by removing ligands from the TmPPase:IDP:Na structure in MD simulations resulted in increased dynamics in loop 5-6, which results in this loop moving to uncover the active site, suggesting that interactions between loop 5-6 and the imidodiphosphate and its associated Mg2+ are important for holding a loop-closed conformation. We also provide further evidence for the transport-before-hydrolysis mechanism by showing that the non-hydrolyzable substrate analogue, methylene diphosphonate, induces low levels of proton pumping by VrPPase.
| S-EPMC5336470 | biostudies-literature
Project description:This paper presents a gamified motor imagery brain-computer interface (MI-BCI) training in immersive virtual reality. The aim of the proposed training method is to increase engagement, attention, and motivation in co-adaptive event-driven MI-BCI training. This was achieved using gamification, progressive increase of the training pace, and virtual reality design reinforcing body ownership transfer (embodiment) into the avatar. From the 20 healthy participants performing 6 runs of 2-class MI-BCI training (left/right hand), 19 were trained for a basic level of MI-BCI operation, with average peak accuracy in the session = 75.84%. This confirms the proposed training method succeeded in improvement of the MI-BCI skills; moreover, participants were leaving the session in high positive affect. Although the performance was not directly correlated to the degree of embodiment, subjective magnitude of the body ownership transfer illusion correlated with the ability to modulate the sensorimotor rhythm.
| S-EPMC6775193 | biostudies-literature
Project description:BackgroundLimited research has been conducted on how healthcare simulation can mitigate clinician stress. Stress exposure training (SET) has been shown to decrease stress's impact on performance. Combining SET with virtual reality (VR) simulation training has not yet been explored in the context of stress inoculation. The primary purpose of this pilot study was to determine if a VR module could induce stress. The secondary purpose was to determine if repeated exposure to stressors could decrease stress response in a simulated environment.MethodsMedical students were recruited to partake in VR simulation modules aimed at treatment of malignant hyperthermia (MH). Those in the SET group were exposed to stressful stimuli during training modules, while those in the Control group were not. Both groups then completed a Test Module with the presence of stressful stimuli. Objective and subjective indicators of stress were measured after each module.ResultsBoth groups indicated increases in perceived stress and module stressfulness after Training Module 1 and decreases after Training Module 2. After the Test Module, the Control group experienced significant elevation in perceived stress (p = .05), and the SET group had a significant decrease in perceived module stressfulness (p < .05). Both groups had a decrease in perceived competence after Training Module 1 (p < .001) and an increase after Training Module 2 (p < .001), with the SET group having significant elevation after the Test Module (p < .01). Both groups found the VR module to be feasible as a teaching tool. Objectively, the SET group showed an upward trend in electrodermal activity (EDA) from the Tutorial to Test Modules (p < .05), with the Control group showing a decrease after Training Module 2 (p = .05) and an increase after the Test Module (p < .01).ConclusionsA VR module targeting treatment of MH successfully induced stress and was regarded favorably by participants. Those in the SET group perceived less stress and more competence after the Test Module than those in the Control. Findings suggest that repeated exposure to stressors through VR may desensitize participants from future stress in a simulated environment.
| S-EPMC11330138 | biostudies-literature