Natural language instructions induce compositional generalization in networks of neurons.

ABSTRACT: A fundamental human cognitive feat is to interpret linguistic instructions in order to perform novel tasks without explicit task experience. Yet, the neural computations that might be used to accomplish this remain poorly understood. We use advances in natural language processing to create a neural model of generalization based on linguistic instructions. Models are trained on a set of common psychophysical tasks, and receive instructions embedded by a pretrained language model. Our best models can perform a previously unseen task with an average performance of 83% correct based solely on linguistic instructions (that is, zero-shot learning). We found that language scaffolds sensorimotor representations such that activity for interrelated tasks shares a common geometry with the semantic re

SUBMITTER: Riveland R 

PROVIDER: S-EPMC11537972 | biostudies-literature | 2024 May

REPOSITORIES: biostudies-literature