Learned parametrized dynamic movement primitives with shared synergies for controlling robotic and musculoskeletal systems.
ABSTRACT: A salient feature of human motor skill learning is the ability to exploit similarities across related tasks. In biological motor control, it has been hypothesized that muscle synergies, coherent activations of groups of muscles, allow for exploiting shared knowledge. Recent studies have shown that a rich set of complex motor skills can be generated by a combination of a small number of muscle synergies. In robotics, dynamic movement primitives are commonly used for motor skill learning. This machine learning approach implements a stable attractor system that facilitates learning and it can be used in high-dimensional continuous spaces. However, it does not allow for reusing shared knowledge, i.e., for each task an individual set of parameters has to be learned. We propose a novel movement primitive representation that employs parametrized basis functions, which combines the benefits of muscle synergies and dynamic movement primitives. For each task a superposition of synergies modulates a stable attractor system. This approach leads to a compact representation of multiple motor skills and at the same time enables efficient learning in high-dimensional continuous systems. The movement representation supports discrete and rhythmic movements and in particular includes the dynamic movement primitive approach as a special case. We demonstrate the feasibility of the movement representation in three multi-task learning simulated scenarios. First, the characteristics of the proposed representation are illustrated in a point-mass task. Second, in complex humanoid walking experiments, multiple walking patterns with different step heights are learned robustly and efficiently. Finally, in a multi-directional reaching task simulated with a musculoskeletal model of the human arm, we show how the proposed movement primitives can be used to learn appropriate muscle excitation patterns and to generalize effectively to new reaching skills.
Project description:Humans can adapt their motor commands in response to alterations in the movement environment. This is achieved by tuning different motor primitives, generating adaptations that can be generalized also to relevant untrained scenarios. A theory of motor primitives has shown that natural movements can be described as combinations of muscle synergies. Previous studies have shown that motor adaptations are achieved by tuning the recruitment of robust synergy modules. Here we tested if: 1) different synergistic tunings can be achieved in response to the same perturbations applied with different orders of exposure; 2) different synergistic tunings can explain different patterns of generalization of adaptation. We found that exposing healthy individuals to two visuomotor rotation perturbations covering different parts of the same workspace in a different order resulted in different tunings of the activation of the same set of synergies. Nevertheless, these tunings resulted in the same net biomechanical adaptation patterns. We also show that the characteristics of the different tunings correlate with the presence and extent of generalization of adaptation to untrained portions of the workspace. Our results confirm synergies as invariant motor primitives whose recruitment is dynamically tuned during motor adaptations.
Project description:The human body is an outstandingly complex machine including around 1000 muscles and joints acting synergistically. Yet, the coordination of the enormous amount of degrees of freedom needed for movement is mastered by our one brain and spinal cord. The idea that some synergistic neural components of movement exist was already suggested at the beginning of the 20th century. Since then, it has been widely accepted that the central nervous system might simplify the production of movement by avoiding the control of each muscle individually. Instead, it might be controlling muscles in common patterns that have been called muscle synergies. Only with the advent of modern computational methods and hardware it has been possible to numerically extract synergies from electromyography (EMG) signals. However, typical experimental setups do not include a big number of individuals, with common sample sizes of 5 to 20 participants. With this study, we make publicly available a set of EMG activities recorded during treadmill running from the right lower limb of 135 healthy and young adults (78 males and 57 females). Moreover, we include in this open access data set the code used to extract synergies from EMG data using non-negative matrix factorization (NMF) and the relative outcomes. Muscle synergies, containing the time-invariant muscle weightings (motor modules) and the time-dependent activation coefficients (motor primitives), were extracted from 13 ipsilateral EMG activities using NMF. Four synergies were enough to describe as many gait cycle phases during running: weight acceptance, propulsion, early swing, and late swing. We foresee many possible applications of our data that we can summarize in three key points. First, it can be a prime source for broadening the representation of human motor control due to the big sample size. Second, it could serve as a benchmark for scientists from multiple disciplines such as musculoskeletal modeling, robotics, clinical neuroscience, sport science, etc. Third, the data set could be used both to train students or to support established scientists in the perfection of current muscle synergies extraction methods. All the data is available at Zenodo (doi: 10.5281/zenodo.1254380).
Project description:Current models of sensorimotor control posit that motor commands are generated by combining multiple modules which may consist of internal models, motor primitives or motor synergies. The mechanisms which select modules based on task requirements and modify their output during learning are therefore critical to our understanding of sensorimotor control. Here we develop a novel modular architecture for multi-dimensional tasks in which a set of fixed primitives are each able to compensate for errors in a single direction in the task space. The contribution of the primitives to the motor output is determined by both top-down contextual information and bottom-up error information. We implement this model for a task in which subjects learn to manipulate a dynamic object whose orientation can vary. In the model, visual information regarding the context (the orientation of the object) allows the appropriate primitives to be engaged. This top-down module selection is implemented by a Gaussian function tuned for the visual orientation of the object. Second, each module's contribution adapts across trials in proportion to its ability to decrease the current kinematic error. Specifically, adaptation is implemented by cosine tuning of primitives to the current direction of the error, which we show to be theoretically optimal for reducing error. This error-tuned model makes two novel predictions. First, interference should occur between alternating dynamics only when the kinematic errors associated with each oppose one another. In contrast, dynamics which lead to orthogonal errors should not interfere. Second, kinematic errors alone should be sufficient to engage the appropriate modules, even in the absence of contextual information normally provided by vision. We confirm both these predictions experimentally and show that the model can also account for data from previous experiments. Our results suggest that two interacting processes account for module selection during sensorimotor control and learning.
Project description:Fine motor control is achieved through the coordinated activation of groups of muscles, or "muscle synergies." Muscle synergies change after stroke as a consequence of the motor deficit. We investigated the pattern and longitudinal changes in upper limb muscle synergies during therapy in a largely unconstrained movement in patients with a broad spectrum of poststroke residual voluntary motor capacity. Electromyography (EMG) was recorded using wireless telemetry from 6 muscles acting on the more-affected upper body in 24 stroke patients at early and late therapy during formal Wii-based Movement Therapy (WMT) sessions, and in a subset of 13 patients at 6-month follow-up. Patients were classified with low, moderate, or high motor-function. The Wii-baseball swing was analyzed using a non-negative matrix factorization (NMF) algorithm to extract muscle synergies from EMG recordings based on the temporal activation of each synergy and the contribution of each muscle to a synergy. Motor-function was clinically assessed immediately pre- and post-therapy and at 6-month follow-up using the Wolf Motor Function Test, upper limb motor Fugl-Meyer Assessment, and Motor Activity Log Quality of Movement scale. Clinical assessments and game performance demonstrated improved motor-function for all patients at post-therapy (p < 0.01), and these improvements were sustained at 6-month follow-up (p > 0.05). NMF analysis revealed fewer muscle synergies (mean ± SE) for patients with low motor-function (3.38 ± 0.2) than those with high motor-function (4.00 ± 0.3) at early therapy (p = 0.036) with an association trend between the number of synergies and the level of motor-function. By late therapy, there was no significant change between groups, although there was a pattern of increase for those with low motor-function over time. The variability accounted for demonstrated differences with motor-function level (p < 0.05) but not time. Cluster analysis of the pooled synergies highlighted the therapy-induced change in muscle activation. Muscle synergies could be identified for all patients during therapy activities. These results show less complexity and more co-activation in the muscle activation for patients with low motor-function as a higher number of muscle synergies reflects greater movement complexity and task-related phasic muscle activation. The increased number of synergies and changes within synergies by late-therapy suggests improved motor control and movement quality with more distinct phases of movement.
Project description:The central nervous system produces movements by activating specifically programmed muscle synergies that are also altered with injuries in the brain, such as stroke. In this study, we hypothesize that there exists a positive correlation between task-specific muscle synergy and motor functions at joint and task levels in patients following stroke. The purpose here is to define and evaluate neurophysiological metrics based on task-specific muscle synergy for assessing motor functions in patients. A patient group of 10 subjects suffering from stroke and a control group of nine age-matched healthy subjects were recruited to participate in this study. Electromyography (EMG) signals and movement kinematics were recorded in patients and control subjects while performing arm reaching tasks. Muscle synergies of individual patients were extracted off-line from EMG records of each patient, and a baseline pattern of muscle synergy was obtained from the pooled EMG data of all nine control subjects. Peak velocities and movement durations of each reaching movement were computed from measured kinematics. Similarity indices of matching components to those of the baseline synergy were defined by synergy vectors and time profiles, respectively, as well as by a combined similarity of vector and time profile. Results showed that pathological synergies of patients were altered from the characteristics of baseline synergy with missing components, or varied vector patterns and time profiles. The kinematic performance measured by peak velocities and movement durations was significantly poorer for the patient group than the control group. In patients, all three similarity indices were found to correlate significantly to the kinematics of movements for the reaching tasks. The correlation to the Fugl-Meyer score of arm was the highest with the vector index, the lowest with the time profile index, and in between with the combined index. These findings illustrate that the analysis of task-specific muscle synergy can provide valuable insights into motor deficits for patients following stroke, and the task-specific similarity indices are useful neurophysiological metrics to predict the function of neuromuscular control at the joint and task levels for patients.
Project description:Muscle synergies provide a simple description of a complex motor control mechanism. Synergies are extracted from muscle activation patterns using factorisation methods. Despite the availability of several factorisation methods in the literature, the most appropriate method for muscle synergy extraction is currently unknown. In this study, we compared four muscle synergy extraction methods: non-negative matrix factorisation, principal component analysis, independent component analysis, and factor analysis. Probability distribution of muscle activation patterns were compared with the probability distribution of synergy excitation primitives obtained from the four factorisation methods. Muscle synergies extracted using non-negative matrix factorisation best matched the probability distribution of muscle activation patterns across different walking and running speeds. Non-negative matrix factorisation also best tracked changes in muscle activation patterns compared to the other factorisation methods. Our results suggest that non-negative matrix factorisation is the best factorisation method for identifying muscle synergies in dynamic tasks with different levels of muscle contraction.
Project description:Diverse features of motor learning have been reported by numerous studies, but no single theoretical framework concurrently accounts for these features. Here, we propose a model for motor learning to explain these features in a unified way by extending a motor primitive framework. The model assumes that the recruitment pattern of motor primitives is determined by the predicted movement error of an upcoming movement (prospective error). To validate this idea, we perform a behavioural experiment to examine the model's novel prediction: after experiencing an environment in which the movement error is more easily predictable, subsequent motor learning should become faster. The experimental results support our prediction, suggesting that the prospective error might be encoded in the motor primitives. Furthermore, we demonstrate that this model has a strong explanatory power to reproduce a wide variety of motor-learning-related phenomena that have been separately explained by different computational models.
Project description:Selecting the appropriate muscle pattern to achieve a given goal is an extremely complex task because of the dimensionality of the search space and because of the nonlinear and dynamical nature of the transformation between muscle activity and movement. To investigate whether the central nervous system uses a modular architecture to achieve motor coordination we characterized the motor output over a large set of movements. We recorded electromyographic activity from 13 muscles of the hind limb of intact and freely moving frogs during jumping, swimming, and walking in naturalistic conditions. We used multidimensional factorization techniques to extract invariant amplitude and timing relationships among the muscle activations. A decomposition of the instantaneous muscle activations as combinations of nonnegative vectors, synchronous muscle synergies, revealed a spatial organization in the muscle patterns. A decomposition of the same activations as a combination of temporal sequences of nonnegative vectors, time-varying muscle synergies, further uncovered specific characteristics in the muscle activation waveforms. A mixture of synergies shared across behaviors and synergies for specific behaviors captured the invariances across the entire dataset. These results support the hypothesis that the motor controller has a modular organization.
Project description:The plasticity of the human nervous system allows us to acquire an open-ended repository of sensorimotor skills in adulthood, such as the mastery of tools, musical instruments or sports. How novel sensorimotor skills are learned from scratch is yet largely unknown. In particular, the so-called inverse mapping from goal states to motor states is underdetermined because a goal can often be achieved by many different movements (motor redundancy). How humans learn to resolve motor redundancy and by which principles they explore high-dimensional motor spaces has hardly been investigated. To study this question, we trained human participants in an unfamiliar and redundant visually-guided manual control task. We qualitatively compare the experimental results with simulation results from a population of artificial agents that learned the same task by Goal Babbling, which is an inverse-model learning approach for robotics. In Goal Babbling, goal-related feedback guides motor exploration and thereby enables robots to learn an inverse model directly from scratch, without having to learn a forward model first. In the human experiment, we tested whether different initial conditions (starting positions of the hand) influence the acquisition of motor synergies, which we identified by Principal Component Analysis in the motor space. The results show that the human participants' solutions are spatially biased towards the different starting positions in motor space and are marked by a gradual co-learning of synergies and task success, similar to the dynamics of motor learning by Goal Babbling. However, there are also differences between human learning and the Goal Babbling simulations, as humans tend to predominantly use Degrees of Freedom that do not have a large effect on the hand position, whereas in Goal Babbling, Degrees of Freedom with a large effect on hand position are used predominantly. We conclude that humans use goal-related feedback to constrain motor exploration and resolve motor redundancy when learning a new sensorimotor mapping, but in a manner that differs from the current implementation of Goal Babbling due to different constraints on motor exploration.
Project description:In most human movement tasks, the same goal can be achieved by a diversity of coordination patterns. For instance, when learning to juggle, individuals adopt their own unique coordination patterns in the early stages of acquiring the fundamental skills of juggling. These individual differences in the learning paths lead to differences in adaptability to new constraints. However, the reason for these differences in adaptability is still unknown. To address this problem, we quantified these differences in terms of attractor stability of the coordination patterns of expert jugglers using Recurrence Quantification Analysis. Furthermore, we quantified the attractor stability of intermediate jugglers and examined adaptability in a sensorimotor synchronization task. We found differences in attractor stability among coordination patterns of expert jugglers, as well as a difference in attractor stability between intrinsic coordination patterns of intermediate jugglers. Whereas, almost no significant direct correlation between attractor stability and adaptability of intermediate jugglers was found, suggesting a difference in both attractor stability and adaptability between intrinsic coordination patterns such that the difference in attractor stability might affect adaptability to new constraints. We submit that the learning path selected by each learner in the early stages of learning plays an important role in the subsequent development of expertise.