Dataset Information

Credit assignment to state-independent task representations and its relationship with model-based decision making.

ABSTRACT: Model-free learning enables an agent to make better decisions based on prior experience while representing only minimal knowledge about an environment's structure. It is generally assumed that model-free state representations are based on outcome-relevant features of the environment. Here, we challenge this assumption by providing evidence that a putative model-free system assigns credit to task representations that are irrelevant to an outcome. We examined data from 769 individuals performing a well-described 2-step reward decision task where stimulus identity but not spatial-motor aspects of the task predicted reward. We show that participants assigned value to spatial-motor representations despite it being outcome irrelevant. Strikingly, spatial-motor value associations affected behavior across all outcome-relevant features and stages of the task, consistent with credit assignment to low-level state-independent task representations. Individual difference analyses suggested that the impact of spatial-motor value formation was attenuated for individuals who showed greater deployment of goal-directed (model-based) strategies. Our findings highlight a need for a reconsideration of how model-free representations are formed and regulated according to the structure of the environment.

SUBMITTER: Shahar N

PROVIDER: S-EPMC6689934 | biostudies-literature | 2019 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Credit assignment to state-independent task representations and its relationship with model-based decision making.

Shahar Nitzan N Moran Rani R Hauser Tobias U TU Kievit Rogier A RA McNamee Daniel D Moutoussis Michael M Dolan Raymond J RJ

Proceedings of the National Academy of Sciences of the United States of America 20190718 32

Model-free learning enables an agent to make better decisions based on prior experience while representing only minimal knowledge about an environment's structure. It is generally assumed that model-free state representations are based on outcome-relevant features of the environment. Here, we challenge this assumption by providing evidence that a putative model-free system assigns credit to task representations that are irrelevant to an outcome. We examined data from 769 individuals performing a ...[more]

PMID: 31320592

Similar Datasets

Project description:Alternating between two tasks is effortful and impairs performance. Previous fMRI studies have found increased activity in frontoparietal cortex when task switching is required. One possibility is that the additional control demands for switch trials are met by strengthening task representations in the human brain. Alternatively, on switch trials, the residual representation of the previous task might impede the buildup of a neural task representation. This would predict weaker task representations on switch trials, thus also explaining the performance costs. To test this, male and female participants were cued to perform one of two similar tasks, with the task being repeated or switched between successive trials. Multivoxel pattern analysis was used to test which regions encode the tasks and whether this encoding differs between switch and repeat trials. As expected, we found information about task representations in frontal and parietal cortex, but there was no difference in the decoding accuracy of task-related information between switch and repeat trials. Using cross-classification, we found that the frontoparietal cortex encodes tasks using a generalizable spatial pattern in switch and repeat trials. Therefore, task representations in frontal and parietal cortex are largely switch independent. We found no evidence that neural information about task representations in these regions can explain behavioral costs usually associated with task switching.SIGNIFICANCE STATEMENT Alternating between two tasks is effortful and slows down performance. One possible explanation is that the representations in the human brain need time to build up and are thus weaker on switch trials, explaining performance costs. Alternatively, task representations might even be enhanced to overcome the previous task. Here, we used a combination of fMRI and a brain classifier to test whether the additional control demands under switching conditions lead to an increased or decreased strength of task representations in frontoparietal brain regions. We found that task representations are not modulated significantly by switching processes and generalize across switching conditions. Therefore, task representations in the human brain cannot account for the performance costs associated with alternating between tasks.

Dataset Information

Credit assignment to state-independent task representations and its relationship with model-based decision making.

Publications

Credit assignment to state-independent task representations and its relationship with model-based decision making.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets