Dataset Information

Tool-Use Model to Reproduce the Goal Situations Considering Relationship Among Tools, Objects, Actions and Effects Using Multimodal Deep Neural Networks.

ABSTRACT: We propose a tool-use model that enables a robot to act toward a provided goal. It is important to consider features of the four factors; tools, objects actions, and effects at the same time because they are related to each other and one factor can influence the others. The tool-use model is constructed with deep neural networks (DNNs) using multimodal sensorimotor data; image, force, and joint angle information. To allow the robot to learn tool-use, we collect training data by controlling the robot to perform various object operations using several tools with multiple actions that leads different effects. Then the tool-use model is thereby trained and learns sensorimotor coordination and acquires relationships among tools, objects, actions and effects in its latent space. We can give the robot a task goal by providing an image showing the target placement and orientation of the object. Using the goal image with the tool-use model, the robot detects the features of tools and objects, and determines how to act to reproduce the target effects automatically. Then the robot generates actions adjusting to the real time situations even though the tools and objects are unknown and more complicated than trained ones.

SUBMITTER: Saito N

PROVIDER: S-EPMC8510504 | biostudies-literature | 2021

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Tool-Use Model to Reproduce the Goal Situations Considering Relationship Among Tools, Objects, Actions and Effects Using Multimodal Deep Neural Networks.

Saito Namiko N Ogata Tetsuya T Mori Hiroki H Murata Shingo S Sugano Shigeki S

Frontiers in robotics and AI 20210928

We propose a tool-use model that enables a robot to act toward a provided goal. It is important to consider features of the four factors; tools, objects actions, and effects at the same time because they are related to each other and one factor can influence the others. The tool-use model is constructed with deep neural networks (DNNs) using multimodal sensorimotor data; image, force, and joint angle information. To allow the robot to learn tool-use, we collect training data by controlling the r ...[more]

PMID: 34651020

Similar Datasets

Project description:During social interactions, we continuously integrate current and previous information over varying timescales to infer other people's action intentions. Motor cognition theories argue for a hierarchical organization of goal-directed actions based on temporal scales. Accordingly, transient motor primitives are represented at lower levels of the hierarchy, a combination of primitives building motor sequences at subordinate levels, and more stable overarching action goals at superordinate levels. A neural topography of hierarchal timescales for information accumulation was previously shown in the visual and auditory domains. However, whether such a temporal hierarchy can also account for observed goal-directed action representations in motor pathways remains to be determined. Thus, the current study examined the neural architecture underlying the processing of observed goal-directed actions using inter-subject correlation (ISC) of fMRI activity. Observers (n = 24) viewed sequential hand movements presented in their intact order or piecewise scrambled at three timescales pertaining to goal-directed action evolution (Primitives: ± 1.5 s, Sub-Goals: ± 4 s, and High-Goals: ± 10 s). The results revealed differential intrinsic temporal capacities for integrating goal-directed action information across brain areas engaged in action observation. Longer timescales (> ± 10 s) were found in the posterior parietal and dorsal premotor compared to the ventral premotor (± 4 s) and anterior parietal (± 1.5 s) cortex. Moreover, our results revealed a hemispheric bias with more extended timescales in the right MT+, primary somatosensory, and early visual cortices compared to their homotopic regions in the left hemisphere. Our findings corroborate a hierarchical neural mapping of observed actions based on temporal scales of goals and provide further support for a ubiquitous time-dependent neural organization of information processing across multiple modalities.

Project description:Developments in high-throughput sequencing (HTS) result in an exponential increase in the amount of data generated by sequencing experiments, an increase in the complexity of bioinformatics analysis reporting and an increase in the types of data generated. These increases in volume, diversity and complexity of the data generated and their analysis expose the necessity of a structured and standardized reporting template. BioCompute Objects (BCOs) provide the requisite support for communication of HTS data analysis that includes support for workflow, as well as data, curation, accessibility and reproducibility of communication. BCOs standardize how researchers report provenance and the established verification and validation protocols used in workflows while also being robust enough to convey content integration or curation in knowledge bases. BCOs that encapsulate tools, platforms, datasets and workflows are FAIR (findable, accessible, interoperable and reusable) compliant. Providing operational workflow and data information facilitates interoperability between platforms and incorporation of future dataset within an HTS analysis for use within industrial, academic and regulatory settings. Cloud-based platforms, including High-performance Integrated Virtual Environment (HIVE), Cancer Genomics Cloud (CGC) and Galaxy, support BCO generation for users. Given the 100K+ userbase between these platforms, BioCompute can be leveraged for workflow documentation. In this paper, we report the availability of platform-dependent and platform-independent BCO tools: HIVE BCO App, CGC BCO App, Galaxy BCO API Extension and BCO Portal. Community engagement was utilized to evaluate tool efficacy. We demonstrate that these tools further advance BCO creation from text editing approaches used in earlier releases of the standard. Moreover, we demonstrate that integrating BCO generation within existing analysis platforms greatly streamlines BCO creation while capturing granular workflow details. We also demonstrate that the BCO tools described in the paper provide an approach to solve the long-standing challenge of standardizing workflow descriptions that are both human and machine readable while accommodating manual and automated curation with evidence tagging. Database URL: https://www.biocomputeobject.org/resources.

Project description:Understanding of other's actions as goal-directed is considered a fundamental ability underlying cognitive and social development in human infants. A number of studies using the habituation-dishabituation paradigm have shown that the ability to discern intentional relations, in terms of goal-directedness of an action towards an object, appears around 5 months of age. The question of whether non-human species can perceive other's actions as goal-directed has been more controversial, however there is mounting evidence that at least some primates species do. Recently domestic dogs have been shown to be particularly sensitive to human communicative cues and more so in cooperative and intentional contexts. Furthermore, they have been shown to imitate selectively. Taken together these results suggest that dogs may perceive others' actions as goal-directed, however no study has investigated this issue directly. In the current study, adopting an infant habituation-dishabituation paradigm, we investigated whether dogs attribute intentions to an animate (a human) but not an inanimate (a black box) agent interacting with an object. Following an habituation phase in which the agent interacted always with one of two objects, two sets of 3 trials were presented: new side trials (in which the agent interacted with the same object as in the habituation trial but placed in a novel location) and new goal trials (in which the agent interacted with the other object placed in the old location). Dogs showed a similar pattern of response to that shown in infants, looking longer in the new goal than new side trials when they saw the human agent interact with the object. No such difference emerging with the inanimate agent (the black box). Results provide the first evidence that a non-primate species can perceive another individual's actions as goal-directed. We discuss results in terms of the prevailing mentalisitic and non-mentalistic hypotheses regarding goal-attribution.

Dataset Information

Tool-Use Model to Reproduce the Goal Situations Considering Relationship Among Tools, Objects, Actions and Effects Using Multimodal Deep Neural Networks.

Publications

Tool-Use Model to Reproduce the Goal Situations Considering Relationship Among Tools, Objects, Actions and Effects Using Multimodal Deep Neural Networks.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets