Robust human locomotion and localization activity recognition over multisensory
Ontology highlight
ABSTRACT:
SUBMITTER: Khan D
PROVIDER: S-EPMC10915014 | biostudies-literature | 2024 Jan
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Healthcare services increasingly use the activity recognition technology to track the daily activities of individuals. In some cases, this is used to provide incentives. For example, some health insurance companies offer discount to customers who are physically active, based on the data collected from their activity tracking devices. Therefore, there is an increasing motivation for individuals to cheat, by making activity trackers detect activities that increase their benefits rather than the ones they actually do. In this study, we used a novel method to make activity recognition robust against deceptive behavior. We asked 14 subjects to attempt to trick our smartphone-based activity classifier by making it detect an activity other than the one they actually performed, for example by shaking the phone while seated to make the classifier detect walking. If they succeeded, we used their motion data to retrain the classifier, and asked them to try to trick it again. The experiment ended when subjects could no longer cheat. We found that some subjects were not able to trick the classifier at all, while others required five rounds of retraining. While classifiers trained on normal activity data predicted true activity with ~38% accuracy, training on the data gathered during the deceptive behavior increased their accuracy to ~84%. We conclude that learning the deceptive behavior of one individual helps to detect the deceptive behavior of others. Thus, we can make current activity recognition robust to deception by including deceptive activity data from a few individuals.
| S-EPMC4676610 | biostudies-literature
Project description:Recently, modern smartphones equipped with a variety of embedded-sensors, such as accelerometers and gyroscopes, have been used as an alternative platform for human activity recognition (HAR), since they are cost-effective, unobtrusive and they facilitate real-time applications. However, the majority of the related works have proposed a position-dependent HAR, i.e., the target subject has to fix the smartphone in a pre-defined position. Few studies have tackled the problem of position-independent HAR. They have tackled the problem either using handcrafted features that are less influenced by the position of the smartphone or by building a position-aware HAR. The performance of these studies still needs more improvement to produce a reliable smartphone-based HAR. Thus, in this paper, we propose a deep convolution neural network model that provides a robust position-independent HAR system. We build and evaluate the performance of the proposed model using the RealWorld HAR public dataset. We find that our deep learning proposed model increases the overall performance compared to the state-of-the-art traditional machine learning method from 84% to 88% for position-independent HAR. In addition, the position detection performance of our model improves superiorly from 89% to 98%. Finally, the recognition time of the proposed model is evaluated in order to validate the applicability of the model for real-time applications.
| S-EPMC6263408 | biostudies-other
Project description:Natural objects provide partially redundant information to the brain through different sensory modalities. For example, voices and faces both give information about the speech content, age, and gender of a person. Thanks to this redundancy, multimodal recognition is fast, robust, and automatic. In unimodal perception, however, only part of the information about an object is available. Here, we addressed whether, even under conditions of unimodal sensory input, crossmodal neural circuits that have been shaped by previous associative learning become activated and underpin a performance benefit. We measured brain activity with functional magnetic resonance imaging before, while, and after participants learned to associate either sensory redundant stimuli, i.e. voices and faces, or arbitrary multimodal combinations, i.e. voices and written names, ring tones, and cell phones or brand names of these cell phones. After learning, participants were better at recognizing unimodal auditory voices that had been paired with faces than those paired with written names, and association of voices with faces resulted in an increased functional coupling between voice and face areas. No such effects were observed for ring tones that had been paired with cell phones or names. These findings demonstrate that brief exposure to ecologically valid and sensory redundant stimulus pairs, such as voices and faces, induces specific multisensory associations. Consistent with predictive coding theories, associative representations become thereafter available for unimodal perception and facilitate object recognition. These data suggest that for natural objects effective predictive signals can be generated across sensory systems and proceed by optimization of functional connectivity between specialized cortical sensory modules.
| S-EPMC1570760 | biostudies-literature
Project description:Decision-making in mammals fundamentally relies on integrating multiple sensory inputs, with conflicting information resolved flexibly based on a dominant sensory modality. However, the neural mechanisms underlying state-dependent changes in sensory dominance remain poorly understood. Our study demonstrates that locomotion in mice shifts auditory-dominant decisions toward visual dominance during audiovisual conflicts. Using circuit-specific calcium imaging and optogenetic manipulations, we found that weakened visual representation in the posterior parietal cortex (PPC) leads to auditory-dominant decisions in stationary mice. Prolonged locomotion, however, promotes visual dominance by inhibiting auditory cortical neurons projecting to the PPC (ACPPC). This shift is mediated by secondary motor cortical neurons projecting to the auditory cortex (M2AC), which specifically inhibit ACPPC neurons without affecting auditory cortical projections to the striatum (ACSTR). Our findings reveal the neural circuit mechanisms underlying auditory gating to the association cortex depending on locomotion states, providing insights into the state-dependent changes in sensory dominance during multisensory decision-making.
| S-EPMC11889129 | biostudies-literature
Project description:Walking and running are mechanically and energetically different locomotion modes. For selecting one or another, speed is a parameter of paramount importance. Yet, both are likely controlled by similar low-dimensional neuronal networks that reflect in patterned muscle activations called muscle synergies. Here, we challenged human locomotion by having our participants walk and run at a very broad spectrum of submaximal and maximal speeds. The synergistic activations of lower limb locomotor muscles were obtained through decomposition of electromyographic data via non-negative matrix factorization. We analyzed the duration and complexity (via fractal analysis) over time of motor primitives, the temporal components of muscle synergies. We found that the motor control of high-speed locomotion was so challenging that the neuromotor system was forced to produce wider and less complex muscle activation patterns. The motor modules, or time-independent coefficients, were redistributed as locomotion speed changed. These outcomes show that humans cope with the challenges of high-speed locomotion by adapting the neuromotor dynamics through a set of strategies that allow for efficient creation and control of locomotion.
| S-EPMC7610320 | biostudies-literature
Project description:Powered knee and ankle prostheses can perform a limited number of discrete ambulation tasks. This is largely due to their control architecture, which uses a finite-state machine to select among a set of task-specific controllers. A non-switching controller that supports a continuum of tasks is expected to better facilitate normative biomechanics. This paper introduces a predictive model that represents gait kinematics as a continuous function of gait cycle percentage, speed, and incline. The basis model consists of two parts: basis functions that produce kinematic trajectories over the gait cycle and task functions that smoothly alter the weight of basis functions in response to task. Kinematic data from 10 able-bodied subjects walking at 27 combinations of speed and incline generate training and validation data for this data-driven model. Convex optimization accurately fits the model to experimental data. Automated model order reduction improves predictive abilities by capturing only the most important kinematic changes due to walking tasks. Constraints on a range of motion and jerk ensure the safety and comfort of the user. This model produces a smooth continuum of trajectories over task, an impossibility for finite-state control algorithms. Random sub-sampling validation indicates that basis modeling predicts untrained kinematics more accurately than linear interpolation.
| S-EPMC6358210 | biostudies-literature
Project description:Locomotion assistive devices equipped with a microprocessor can potentially automatically adapt their behavior when the user is transitioning from one locomotion mode to another. Many developments in the field have come from machine learning driven controllers on locomotion assistive devices that recognize/predict the current locomotion mode or the upcoming one. This review synthesizes the machine learning algorithms designed to recognize or to predict a locomotion mode in order to automatically adapt the behavior of a locomotion assistive device. A systematic review was conducted on the Web of Science and MEDLINE databases (as well as in the retrieved papers) to identify articles published between 1 January 2000 to 31 July 2020. This systematic review is reported in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines and is registered on Prospero (CRD42020149352). Study characteristics, sensors and algorithms used, accuracy and robustness were also summarized. In total, 1343 records were identified and 58 studies were included in this review. The experimental condition which was most often investigated was level ground walking along with stair and ramp ascent/descent activities. The machine learning algorithms implemented in the included studies reached global mean accuracies of around 90%. However, the robustness of those algorithms seems to be more broadly evaluated, notably, in everyday life. We also propose some guidelines for homogenizing future reports.
| S-EPMC7664393 | biostudies-literature
Project description:A series of our previous studies explored the use of an abstract visual representation of the amplitude envelope cues from target sentences to benefit speech perception in complex listening environments. The purpose of this study was to expand this auditory-visual speech perception to the tactile domain. Twenty adults participated in speech recognition measurements in four different sensory modalities (AO, auditory-only; AV, auditory-visual; AT, auditory-tactile; AVT, auditory-visual-tactile). The target sentences were fixed at 65 dB sound pressure level and embedded within a simultaneous speech-shaped noise masker of varying degrees of signal-to-noise ratios (-7, -5, -3, -1, and 1 dB SNR). The amplitudes of both abstract visual and vibrotactile stimuli were temporally synchronized with the target speech envelope for comparison. Average results showed that adding temporally-synchronized multimodal cues to the auditory signal did provide significant improvements in word recognition performance across all three multimodal stimulus conditions (AV, AT, and AVT), especially at the lower SNR levels of -7, -5, and -3 dB for both male (8-20% improvement) and female (5-25% improvement) talkers. The greatest improvement in word recognition performance (15-19% improvement for males and 14-25% improvement for females) was observed when both visual and tactile cues were integrated (AVT). Another interesting finding in this study is that temporally synchronized abstract visual and vibrotactile stimuli additively stack in their influence on speech recognition performance. Our findings suggest that a multisensory integration process in speech perception requires salient temporal cues to enhance speech recognition ability in noisy environments.
| S-EPMC9630463 | biostudies-literature
Project description:Context modulates how information is processed in the mammalian brain. For example, brain responses are amplified to contextually unusual stimuli. This phenomenon, known as "deviance detection,"1,2 is well documented in early, primary sensory cortex, where large responses are generated to simple stimuli that deviate from their context in low-order properties, such as line orientation, size, or pitch.2,3,4,5 However, the extent to which neural deviance detection manifests (1) in broader cortical networks and (2) to simple versus complex stimuli, which deviate only in their higher-order, multisensory properties, is not known. Consistent with a predictive processing framework,6,7 we hypothesized that deviance detection manifests in a hierarchical manner across cortical networks,8,9 emerging later and further downstream when stimulus deviance is complex. To test this, we examined brain responses of awake mice to simple unisensory deviants (e.g., visual line gratings, deviating from context in their orientation alone) versus complex multisensory deviants (i.e., audiovisual pairs, deviating from context only in their audiovisual pairing but not visual or auditory content alone). We find that mouse parietal associative area-a higher cortical region-displays robust multisensory deviance detection. In contrast, primary visual cortex exhibits strong unisensory visual deviance detection but weaker multisensory deviance detection. These results suggest that deviance detection signals in the cortex may be conceptualized as "prediction errors," which are primarily fed forward-or downstream-in cortical networks.6,7.
| S-EPMC10529873 | biostudies-literature
Project description:The integration and interaction of vision, touch, hearing, smell, and taste in the human multisensory neural network facilitate high-level cognitive functionalities, such as crossmodal integration, recognition, and imagination for accurate evaluation and comprehensive understanding of the multimodal world. Here, we report a bioinspired multisensory neural network that integrates artificial optic, afferent, auditory, and simulated olfactory and gustatory sensory nerves. With distributed multiple sensors and biomimetic hierarchical architectures, our system can not only sense, process, and memorize multimodal information, but also fuse multisensory data at hardware and software level. Using crossmodal learning, the system is capable of crossmodally recognizing and imagining multimodal information, such as visualizing alphabet letters upon handwritten input, recognizing multimodal visual/smell/taste information or imagining a never-seen picture when hearing its description. Our multisensory neural network provides a promising approach towards robotic sensing and perception.
| S-EPMC7893014 | biostudies-literature