Dataset Information

Determining Exception Context in Assembly Operations from Multimodal Data.

ABSTRACT: Robot assembly tasks can fail due to unpredictable errors and can only continue with the manual intervention of a human operator. Recently, we proposed an exception strategy learning framework based on statistical learning and context determination, which can successfully resolve such situations. This paper deals with context determination from multimodal data, which is the key component of our framework. We propose a novel approach to generate unified low-dimensional context descriptions based on image and force-torque data. For this purpose, we combine a state-of-the-art neural network model for image segmentation and contact point estimation using force-torque measurements. An ensemble of decision trees is used to combine features from the two modalities. To validate the proposed approach, we have collected datasets of deliberately induced insertion failures both for the classic peg-in-hole insertion task and for an industrially relevant task of car starter assembly. We demonstrate that the proposed approach generates reliable low-dimensional descriptors, suitable as queries necessary in statistical learning.

SUBMITTER: Simonic M

PROVIDER: S-EPMC9610822 | biostudies-literature | 2022 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Determining Exception Context in Assembly Operations from Multimodal Data.

Simonič Mihael M Majcen Hrovat Matevž M Džeroski Sašo S Ude Aleš A Nemec Bojan B

Sensors (Basel, Switzerland) 20221019 20

Robot assembly tasks can fail due to unpredictable errors and can only continue with the manual intervention of a human operator. Recently, we proposed an exception strategy learning framework based on statistical learning and context determination, which can successfully resolve such situations. This paper deals with context determination from multimodal data, which is the key component of our framework. We propose a novel approach to generate unified low-dimensional context descriptions based ...[more]

PMID: 36298313

Similar Datasets

Project description:Hemodynamic responses, in general, and the blood oxygenation level-dependent (BOLD) fMRI signal, in particular, provide an indirect measure of neuronal activity. There is strong evidence that the BOLD response correlates well with post-synaptic changes, induced by changes in the excitatory and inhibitory (E-I) balance between active neuronal populations. Typical BOLD responses exhibit transients, such as the early-overshoot and post-stimulus undershoot, that can be linked to transients in neuronal activity, but they can also result from vascular uncoupling between cerebral blood flow (CBF) and venous cerebral blood volume (venous CBV). Recently, we have proposed a novel generative hemodynamic model of the BOLD signal within the dynamic causal modeling framework, inspired by physiological observations, called P-DCM (Havlicek et al., 2015). We demonstrated the generative model's ability to more accurately model commonly observed neuronal and vascular transients in single regions but also effective connectivity between multiple brain areas (Havlicek et al., 2017b). In this paper, we additionally demonstrate the versatility of the generative model to jointly explain dynamic relationships between neuronal and hemodynamic physiological variables underlying the BOLD signal using multi-modal data. For this purpose, we utilized three distinct data-sets of experimentally induced responses in the primary visual areas measured in human, cat, and monkey brain, respectively: (1) CBF and BOLD responses; (2) CBF, total CBV, and BOLD responses (Jin and Kim, 2008); and (3) positive and negative neuronal and BOLD responses (Shmuel et al., 2006). By fitting the generative model to the three multi-modal experimental data-sets, we showed that the presence or absence of dynamic features in the BOLD signal is not an unambiguous indication of presence or absence of those features on the neuronal level. Nevertheless, the generative model that takes into account the dynamics of the physiological mechanisms underlying the BOLD response allowed dissociating neuronal from vascular transients and deducing excitatory and inhibitory neuronal activity time-courses from BOLD data alone and from multi-modal data.

Dataset Information

Determining Exception Context in Assembly Operations from Multimodal Data.

Publications

Determining Exception Context in Assembly Operations from Multimodal Data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets