Intention-Related Natural Language Grounding via Object Affordance Detection and Intention Semantic Extraction.

ABSTRACT: Similar to specific natural language instructions, intention-related natural language queries also play an essential role in our daily life communication. Inspired by the psychology term "affordance" and its applications in Human-Robot interaction, we propose an object affordance-based natural language visual grounding architecture to ground intention-related natural language queries. Formally, we first present an attention-based multi-visual features fusion network to detect object affordances from RGB images. While fusing deep visual features extracted from a pre-trained CNN model with deep texture features encoded by a deep texture encoding network, the presented object affordance detection network takes into account the interaction of the multi-visual features, and reserves the complem

SUBMITTER: Mi J 

PROVIDER: S-EPMC7238763 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature