Data-driven body-machine interface for the accurate control of drones.
Ontology highlight
ABSTRACT:
SUBMITTER: Miehlbradt J
PROVIDER: S-EPMC6077744 | biostudies-literature | 2018 Jul
REPOSITORIES: biostudies-literature
Publications
Data-driven body-machine interface for the accurate control of drones.
Miehlbradt Jenifer J Cherpillod Alexandre A Mintchev Stefano S Coscia Martina M Artoni Fiorenzo F Floreano Dario D Micera Silvestro S
Proceedings of the National Academy of Sciences of the United States of America 20180716 31
The accurate teleoperation of robotic devices requires simple, yet intuitive and reliable control interfaces. However, current human-machine interfaces (HMIs) often fail to fulfill these characteristics, leading to systems requiring an intensive practice to reach a sufficient operation expertise. Here, we present a systematic methodology to identify the spontaneous gesture-based interaction strategies of naive individuals with a distant device, and to exploit this information to develop a data-d ...[more]
Similar Datasets
Project description:Gaze estimation is long been recognised as having potential as the basis for human-computer interaction (HCI) systems, but usability and robustness of performance remain challenging . This work focuses on systems in which there is a live video stream showing enough of the subjects face to track eye movements and some means to infer gaze location from detected eye features. Currently, systems generally require some form of calibration or set-up procedure at the start of each user session. Here we explore some simple strategies for enabling gaze based HCI to operate immediately and robustly without any explicit set-up tasks. We explore different choices of coordinate origin for combining extracted features from multiple subjects and the replacement of subject specific calibration by system initiation based on prior models. Results show that referencing all extracted features to local coordinate origins determined by subject start position enables robust immediate operation. Combining this approach with an adaptive gaze estimation model using an interactive user interface enables continuous operation with the 75th percentile gaze errors of 0.7 ∘ , and maximum gaze errors of 1.7 ∘ during prospective testing. There constitute state-of-the-art results and have the potential to enable a new generation of reliable gaze based HCI systems.
| S-EPMC11111737 | biostudies-literature
Project description:OBJECTIVE:Making mistakes is inevitable, but identifying them allows us to correct or adapt our behavior to improve future performance. Current brain-machine interfaces (BMIs) make errors that need to be explicitly corrected by the user, thereby consuming time and thus hindering performance. We hypothesized that neural correlates of the user perceiving the mistake could be used by the BMI to automatically correct errors. However, it was unknown whether intracortical outcome error signals were present in the premotor and primary motor cortices, brain regions successfully used for intracortical BMIs. APPROACH:We report here for the first time a putative outcome error signal in spiking activity within these cortices when rhesus macaques performed an intracortical BMI computer cursor task. MAIN RESULTS:We decoded BMI trial outcomes shortly after and even before a trial ended with 96% and 84% accuracy, respectively. This led us to develop and implement in real-time a first-of-its-kind intracortical BMI error 'detect-and-act' system that attempts to automatically 'undo' or 'prevent' mistakes. The detect-and-act system works independently and in parallel to a kinematic BMI decoder. In a challenging task that resulted in substantial errors, this approach improved the performance of a BMI employing two variants of the ubiquitous Kalman velocity filter, including a state-of-the-art decoder (ReFIT-KF). SIGNIFICANCE:Detecting errors in real-time from the same brain regions that are commonly used to control BMIs should improve the clinical viability of BMIs aimed at restoring motor function to people with paralysis.
| S-EPMC5742283 | biostudies-literature
Project description:Several groups have developed brain-machine-interfaces (BMIs) that allow primates to use cortical activity to control artificial limbs. Yet, it remains unknown whether cortical ensembles could represent the kinematics of whole-body navigation and be used to operate a BMI that moves a wheelchair continuously in space. Here we show that rhesus monkeys can learn to navigate a robotic wheelchair, using their cortical activity as the main control signal. Two monkeys were chronically implanted with multichannel microelectrode arrays that allowed wireless recordings from ensembles of premotor and sensorimotor cortical neurons. Initially, while monkeys remained seated in the robotic wheelchair, passive navigation was employed to train a linear decoder to extract 2D wheelchair kinematics from cortical activity. Next, monkeys employed the wireless BMI to translate their cortical activity into the robotic wheelchair's translational and rotational velocities. Over time, monkeys improved their ability to navigate the wheelchair toward the location of a grape reward. The navigation was enacted by populations of cortical neurons tuned to whole-body displacement. During practice with the apparatus, we also noticed the presence of a cortical representation of the distance to reward location. These results demonstrate that intracranial BMIs could restore whole-body mobility to severely paralyzed patients in the future.
| S-EPMC4776675 | biostudies-literature
Project description:BackgroundBody-machine interfaces map movements onto commands to external devices. Redundant motion signals derived from inertial sensors are mapped onto lower-dimensional device commands. Then, the device users face two problems, a) the structural problem of understanding the operation of the interface and b) the performance problem of controlling the external device with high efficiency. We hypothesize that these problems, while being distinct are connected in that aligning the space of body movements with the space encoded by the interface, i.e. solving the structural problem, facilitates redundancy resolution towards increasing efficiency, i.e. solving the performance problem.MethodsTwenty unimpaired volunteers practiced controlling the movement of a computer cursor by moving their arms. Eight signals from four inertial sensors were mapped onto the two cursor's coordinates on a screen. The mapping matrix was initialized by asking each user to perform free-form spontaneous upper-limb motions and deriving the two main principal components of the motion signals. Participants engaged in a reaching task for 18 min, followed by a tracking task. One group of 10 participants practiced with the same mapping throughout the experiment, while the other 10 with an adaptive mapping that was iteratively updated by recalculating the principal components based on ongoing movements.ResultsParticipants quickly reduced reaching time while also learning to distribute most movement variance over two dimensions. Participants with the fixed mapping distributed movement variance over a subspace that did not match the potent subspace defined by the interface map. In contrast, participant with the adaptive map reduced the difference between the two subspaces, resulting in a smaller amount of arm motions distributed over the null space of the interface map. This, in turn, enhanced movement efficiency without impairing generalization from reaching to tracking.ConclusionsAligning the potent subspace encoded by the interface map to the user's movement subspace guides redundancy resolution towards increasing movement efficiency, with implications for controlling assistive devices. In contrast, in the pursuit of rehabilitative goals, results would suggest that the interface must change to drive the statistics of user's motions away from the established pattern and toward the engagement of movements to be recovered.Trial registrationClinicalTrials.gov, NCT01608438, Registered 16 April 2012.
| S-EPMC7216597 | biostudies-literature
Project description:Recent development of the chemical inhibitors specific to oncogenic KRAS (Kirsten Rat Sarcoma 2 Viral Oncogene Homolog) mutants revives much interest to control KRAS-driven cancers. Here, we report that AIMP2-DX2, a variant of the tumor suppressor AIMP2 (aminoacyl-tRNA synthetase-interacting multi-functional protein 2), acts as a cancer-specific regulator of KRAS stability, augmenting KRAS-driven tumorigenesis. AIMP2-DX2 specifically binds to the hypervariable region and G-domain of KRAS in the cytosol prior to farnesylation. Then, AIMP2-DX2 competitively blocks the access of Smurf2 (SMAD Ubiquitination Regulatory Factor 2) to KRAS, thus preventing ubiquitin-mediated degradation. Moreover, AIMP2-DX2 levels are positively correlated with KRAS levels in colon and lung cancer cell lines and tissues. We also identified a small molecule that specifically bound to the KRAS-binding region of AIMP2-DX2 and inhibited the interaction between these two factors. Treatment with this compound reduces the cellular levels of KRAS, leading to the suppression of KRAS-dependent cancer cell growth in vitro and in vivo. These results suggest the interface of AIMP2-DX2 and KRAS as a route to control KRAS-driven cancers.
| S-EPMC9095880 | biostudies-literature
Project description:Motor cortex plays a substantial role in driving movement, yet the details underlying this control remain unresolved. We analyzed the extent to which movement-related information could be extracted from single-trial motor cortical activity recorded while monkeys performed center-out reaching. Using information theoretic techniques, we found that single units carry relatively little speed-related information compared with direction-related information. This result is not mitigated at the population level: simultaneously recorded population activity predicted speed with significantly lower accuracy relative to direction predictions. Furthermore, a unit-dropping analysis revealed that speed accuracy would likely remain lower than direction accuracy, even given larger populations. These results suggest that the instantaneous details of single-trial movement speed are difficult to extract using commonly assumed coding schemes. This apparent paucity of speed information takes particular importance in the context of brain-machine interfaces (BMIs), which rely on extracting kinematic information from motor cortex. Previous studies have highlighted subjects' difficulties in holding a BMI cursor stable at targets. These studies, along with our finding of relatively little speed information in motor cortex, inspired a speed-dampening Kalman filter (SDKF) that automatically slows the cursor upon detecting changes in decoded movement direction. Effectively, SDKF enhances speed control by using prevalent directional signals, rather than requiring speed to be directly decoded from neural activity. SDKF improved success rates by a factor of 1.7 relative to a standard Kalman filter in a closed-loop BMI task requiring stable stops at targets. BMI systems enabling stable stops will be more effective and user-friendly when translated into clinical applications.
| S-EPMC4064402 | biostudies-literature
Project description:OBJECTIVE:Intracortical brain-machine interfaces (BMIs) are a promising source of prosthesis control signals for individuals with severe motor disabilities. Previous BMI studies have primarily focused on predicting and controlling whole-arm movements; precise control of hand kinematics, however, has not been fully demonstrated. Here, we investigate the continuous decoding of precise finger movements in rhesus macaques. APPROACH:In order to elicit precise and repeatable finger movements, we have developed a novel behavioral task paradigm which requires the subject to acquire virtual fingertip position targets. In the physical control condition, four rhesus macaques performed this task by moving all four fingers together in order to acquire a single target. This movement was equivalent to controlling the aperture of a power grasp. During this task performance, we recorded neural spikes from intracortical electrode arrays in primary motor cortex. MAIN RESULTS:Using a standard Kalman filter, we could reconstruct continuous finger movement offline with an average correlation of ???=??0.78 between actual and predicted position across four rhesus macaques. For two of the monkeys, this movement prediction was performed in real-time to enable direct brain control of the virtual hand. Compared to physical control, neural control performance was slightly degraded; however, the monkeys were still able to successfully perform the task with an average target acquisition rate of 83.1%. The monkeys' ability to arbitrarily specify fingertip position was also quantified using an information throughput metric. During brain control task performance, the monkeys achieved an average 1.01 bits s-1 throughput, similar to that achieved in previous studies which decoded upper-arm movements to control computer cursors using a standard Kalman filter. SIGNIFICANCE:This is, to our knowledge, the first demonstration of brain control of finger-level fine motor skills. We believe that these results represent an important step towards full and dexterous control of neural prosthetic devices.
| S-EPMC5737665 | biostudies-literature
Project description:Medically-induced coma is a drug-induced state of profound brain inactivation and unconsciousness used to treat refractory intracranial hypertension and to manage treatment-resistant epilepsy. The state of coma is achieved by continually monitoring the patient's brain activity with an electroencephalogram (EEG) and manually titrating the anesthetic infusion rate to maintain a specified level of burst suppression, an EEG marker of profound brain inactivation in which bursts of electrical activity alternate with periods of quiescence or suppression. The medical coma is often required for several days. A more rational approach would be to implement a brain-machine interface (BMI) that monitors the EEG and adjusts the anesthetic infusion rate in real time to maintain the specified target level of burst suppression. We used a stochastic control framework to develop a BMI to control medically-induced coma in a rodent model. The BMI controlled an EEG-guided closed-loop infusion of the anesthetic propofol to maintain precisely specified dynamic target levels of burst suppression. We used as the control signal the burst suppression probability (BSP), the brain's instantaneous probability of being in the suppressed state. We characterized the EEG response to propofol using a two-dimensional linear compartment model and estimated the model parameters specific to each animal prior to initiating control. We derived a recursive Bayesian binary filter algorithm to compute the BSP from the EEG and controllers using a linear-quadratic-regulator and a model-predictive control strategy. Both controllers used the estimated BSP as feedback. The BMI accurately controlled burst suppression in individual rodents across dynamic target trajectories, and enabled prompt transitions between target levels while avoiding both undershoot and overshoot. The median performance error for the BMI was 3.6%, the median bias was -1.4% and the overall posterior probability of reliable control was 1 (95% Bayesian credibility interval of [0.87, 1.0]). A BMI can maintain reliable and accurate real-time control of medically-induced coma in a rodent model suggesting this strategy could be applied in patient care.
| S-EPMC3814408 | biostudies-literature
Project description:Motor brain-machine interfaces (BMIs) decode neural signals to help people with paralysis move and communicate. Even with important advances in the last two decades, BMIs face key obstacles to clinical viability. Invasive BMIs achieve proficient cursor and robotic arm control but require neurosurgery, posing significant risk to patients. Non-invasive BMIs do not have neurosurgical risk, but achieve lower performance, sometimes being prohibitively frustrating to use and preventing widespread adoption. We take a step toward breaking this performance-risk tradeoff by building performant non-invasive BMIs. The critical limitation that bounds decoder performance in non-invasive BMIs is their poor neural signal-to-noise ratio. To overcome this, we contribute (1) a novel EEG decoding approach and (2) artificial intelligence (AI) copilots that infer task goals and aid action completion. We demonstrate that with this "AI-BMI," in tandem with a new adaptive decoding approach using a convolutional neural network (CNN) and ReFIT-like Kalman filter (KF), healthy users and a paralyzed participant can autonomously and proficiently control computer cursors and robotic arms. Using an AI copilot improves goal acquisition speed by up to 4.3× in the standard center-out 8 cursor control task and enables users to control a robotic arm to perform the sequential pick-and-place task, moving 4 randomly placed blocks to 4 randomly chosen locations. As AI copilots improve, this approach may result in clinically viable non-invasive AI-BMIs.
| S-EPMC11482823 | biostudies-literature
Project description:In this study, we consider a non-invasive body-machine interface that captures body motions still available to people with spinal cord injury (SCI) and maps them into a set of signals for controlling a computer user interface while engaging in a sustained level of mobility and exercise. We compare the effectiveness of two decoding algorithms that transform a high-dimensional body-signal vector into a lower dimensional control vector on six subjects with high-level SCI and eight controls. One algorithm is based on a static map from current body signals to the current value of the control vector set through principal component analysis (PCA), the other on dynamic mapping a segment of body signals to the value and the temporal derivatives of the control vector set through a Kalman filter. SCI and control participants performed straighter and smoother cursor movements with the Kalman algorithm during center-out reaching, but their movements were faster and more precise when using PCA. All participants were able to use the BMI's continuous, two-dimensional control to type on a virtual keyboard and play pong, and performance with both algorithms was comparable. However, seven of eight control participants preferred PCA as their method of virtual wheelchair control. The unsupervised PCA algorithm was easier to train and seemed sufficient to achieve a higher degree of learnability and perceived ease of use.
| S-EPMC5472505 | biostudies-literature