Incentives Boost Model-Based Control Across a Range of Severity on Several Psychiatric Constructs.
ABSTRACT: BACKGROUND:Human decision making exhibits a mixture of model-based and model-free control. Recent evidence indicates that arbitration between these two modes of control ("metacontrol") is based on their relative costs and benefits. While model-based control may increase accuracy, it requires greater computational resources, so people invoke model-based control only when potential rewards exceed those of model-free control. We used a sequential decision task, while concurrently manipulating performance incentives, to ask if symptoms and traits of psychopathology decrease or increase model-based control in response to incentives. METHODS:We recruited a nonpatient population of 839 online participants using Amazon Mechanical Turk who completed transdiagnostic self-report measures encompassing symptoms, traits, and factors. We fit a dual-controller reinforcement learning model and obtained a computational measure of model-based control separately for small incentives and large incentives. RESULTS:None of the constructs were related to a failure of large incentives to boost model-based control. In fact, for the sensation seeking trait and anxious-depression factor, higher scores were associated with a larger incentive effect, whereby greater levels of these constructs were associated with larger increases in model-based control. Many constructs showed decreases in model-based control as a function of severity, but a social withdrawal factor was positively correlated; alcohol use and social anxiety were unrelated to model-based control. CONCLUSIONS:Our results demonstrate that model-based control can reliably be improved independent of construct severity for most measures. This suggests that incentives may be a useful intervention for boosting model-based control across a range of symptom and trait severity.
Project description:It has previously been shown that the relative reliability of model-based and model-free reinforcement-learning (RL) systems plays a role in the allocation of behavioral control between them. However, the role of task complexity in the arbitration between these two strategies remains largely unknown. Here, using a combination of novel task design, computational modelling, and model-based fMRI analysis, we examined the role of task complexity alongside state-space uncertainty in the arbitration process. Participants tended to increase model-based RL control in response to increasing task complexity. However, they resorted to model-free RL when both uncertainty and task complexity were high, suggesting that these two variables interact during the arbitration process. Computational fMRI revealed that task complexity interacts with neural representations of the reliability of the two systems in the inferior prefrontal cortex.
Project description:There is accumulating neural evidence to support the existence of two distinct systems for guiding action selection, a deliberative "model-based" and a reflexive "model-free" system. However, little is known about how the brain determines which of these systems controls behavior at one moment in time. We provide evidence for an arbitration mechanism that allocates the degree of control over behavior by model-based and model-free systems as a function of the reliability of their respective predictions. We show that the inferior lateral prefrontal and frontopolar cortex encode both reliability signals and the output of a comparison between those signals, implicating these regions in the arbitration process. Moreover, connectivity between these regions and model-free valuation areas is negatively modulated by the degree of model-based control in the arbitrator, suggesting that arbitration may work through modulation of the model-free valuation system when the arbitrator deems that the model-based system should drive behavior.
Project description:Although procrastination is a widespread phenomenon with significant influence on our personal and professional life, its genetic foundation is somewhat unknown. An important factor that influences our ability to tackle specific goals directly instead of putting them off is our ability to initiate cognitive, motivational and emotional control mechanisms, so-called metacontrol. These metacontrol mechanisms have been frequently related to dopaminergic signaling. To gain deeper insight into the genetic components of procrastination, we examined whether genetically induced differences in the dopaminergic system are associated with interindividual differences in trait-like procrastination, measured as decision-related action control (AOD). Analyzing the data of 278 healthy adults, we found a sex-dependent effect of TH genotype on AOD. Interestingly, only in women, T-allele carriers showed lower AOD values and were therefore more likely to procrastinate. Additionally, we investigated whether differences in the morphology and functional connectivity of the amygdala that were previously associated with AOD happen to be related to differences in the TH genotype and thus to differences in the dopaminergic system. However, there was no significant amygdala volume or connectivity difference between the TH genotype groups. Therefore, this study is the first to suggest that genetic, anatomical and functional differences affect trait-like procrastination independently.
Project description:Emotions play a significant role in internal regulatory processes. In this paper, we advocate four key ideas. First, novelty detection can be grounded in the sensorimotor experience and allow higher order appraisal. Second, cognitive processes, such as those involved in self-assessment, influence emotional states by eliciting affects like boredom and frustration. Third, emotional processes such as those triggered by self-assessment influence attentional processes. Last, close emotion-cognition interactions implement an efficient feedback loop for the purpose of top-down behavior regulation. The latter is what we call 'Emotional Metacontrol'. We introduce a model based on artificial neural networks. This architecture is used to control a robotic system in a visual search task. The emotional metacontrol intervenes to bias the robot visual attention during active object recognition. Through a behavioral and statistical analysis, we show that this mechanism increases the robot performance and fosters the exploratory behavior to avoid deadlocks.
Project description:Humans employ different strategies when making decisions. Previous research has reported reduced reliance on model-based strategies with aging, but it remains unclear whether this is due to cognitive or motivational factors. Moreover, it is not clear how aging affects the metacontrol of decision making, that is the dynamic adaptation of decision-making strategies to varying situational demands. In this cross-sectional study, we tested younger and older adults in a sequential decision-making task that dissociates model-free and model-based strategies. In contrast to previous research, model-based strategies led to higher payoffs. Moreover, we manipulated the costs and benefits of model-based strategies by varying reward magnitude and the stability of the task structure. Compared to younger adults, older adults showed reduced model-based decision making and less adaptation of decision-making strategies. Our findings suggest that aging affects the metacontrol of decision-making strategies and that reduced model-based strategies in older adults are due to limited cognitive abilities.
Project description:Depression is characterized by deficits in the reinforcement learning (RL) process. Although many computational and neural studies have extended our knowledge of the impact of depression on RL, most focus on habitual control (model-free RL), yielding a relatively poor understanding of goal-directed control (model-based RL) and arbitration control to find a balance between the two. We investigated the effects of subclinical depression on model-based and model-free learning in the prefrontal-striatal circuitry. First, we found that subclinical depression is associated with the attenuated state and reward prediction error representation in the insula and caudate. Critically, we found that it accompanies the disrupted arbitration control between model-based and model-free learning in the predominantly inferior lateral prefrontal cortex and frontopolar cortex. We also found that depression undermines the ability to exploit viable options, called exploitation sensitivity. These findings characterize how subclinical depression influences different levels of the decision-making hierarchy, advancing previous conflicting views that depression simply influences either habitual or goal-directed control. Our study creates possibilities for various clinical applications, such as early diagnosis and behavioral therapy design.
Project description:Cognitive control processes are advantageous when routines would not lead to the desired outcome, but this can be ill-advised when automated behavior is advantageous. The aim of this study was to identify neural dynamics related to the ability to adapt to different cognitive control demands - a process that has been referred to as 'metacontrol.' A sample of <i>N</i> = 227 healthy subjects that was split in a 'high' and 'low adaptability' group based on the behavioral performance in a task with varying control demands. To examine the neurophysiological mechanisms, we combined event-related potential (ERP) recordings with source localization and machine learning approaches. The results show that individuals who are better at strategically adapting to different cognitive control demands benefit from automatizing their response processes in situations where little cognitive control is needed. On a neurophysiological level, neither perceptual/attentional selection processes nor conflict monitoring processes paralleled the behavioral data, although the latter showed a descriptive trend. Behavioral differences in metacontrol abilities were only significantly mirrored by the modulation of response-locked P3 amplitudes, which were accompanied by activation differences in insula (BA13) and middle frontal gyrus (BA9). The machine learning result corroborated this by identifying a predictive/classification feature near the peak of the response-locked P3, which arose from the anterior cingulate cortex (BA24; BA33). In short, we found that metacontrol is associated to the ability to manage response selection processes, especially the ability to effectively downregulate cognitive control under low cognitive control requirements, rather than the ability to upregulate cognitive control.
Project description:BACKGROUND:Highly complex tasks generally benefit from increases in cognitive control, which has been linked to dopamine. Yet, the same amount of control may actually be detrimental in tasks with low complexity so that the task-dependent allocation of cognitive control resources (also known as "metacontrol") is key to expedient and adaptive behavior in various contexts. METHODS:Given that dopamine D1 and D2 receptors have been suggested to exert opposing effects on cognitive control, we investigated the impact of 2 single nucleotide polymorphisms in the DRD1 (rs4532) and DRD2 (rs6277) genes on metacontrol in 195 healthy young adults. Subjects performed 2 consecutive tasks that differed in their demand for control (starting with the less complex task and then performing a more complex task rule). RESULTS:We found carriers of the DRD1 rs4532 G allele to outperform noncarriers in case of high control requirements (i.e., reveal a better response accuracy), but not in case of low control requirements. This was confirmed by Bayesian analyses. No effects of DRD2 rs6277 genotype on either task were evident, again confirmed by Bayesian analyses. CONCLUSIONS:Our findings suggest that higher DRD1 receptor efficiency improves performance during high, but not low, control requirements, probably by promoting a "D1 state," which is characterized by highly stable task set representations. The null findings for DRD2 signaling might be explained by the fact that the "D2 state" is thought to enhance flexible switching between task set representations when our task only featured 1 task set at any given time.
Project description:The present study aimed to examine effects of motivational and social cognition constructs on children’s leisure-time physical activity participation alongside constructs representing implicit processes using an extended trans-contextual model. The study adopted a correlational prospective design. Secondary-school students (N = 502) completed self-report measures of perceived autonomy support from physical education (PE) teachers, autonomous motivation in PE and leisure-time contexts, and social cognition constructs (attitudes, subjective norms, perceived behavioral control), intentions, trait self-control, habits, and past behavior in a leisure-time physical activity context. Five weeks later, students (N = 298) self-reported their leisure-time physical activity participation. Bayesian path analyses supported two key premises of the model: perceived autonomy support was related to autonomous motivation in PE, and autonomous motivation in PE was related to autonomous motivation in leisure time. Indirect effects indicated that both forms of autonomous motivation were related to social cognition constructs and intentions. However, intention was not related to leisure-time physical activity participation, so model variables reflecting motivational processes did not account for substantive variance in physical activity participation. Self-control, attitudes, and past behavior were direct predictors of intentions and leisure-time physical activity participation. There were indirect effects of autonomous motivation in leisure time on intentions and physical activity participation mediated by self-control. Specifying informative priors for key model relations using Bayesian analysis yielded greater precision for some model effects. Findings raise some questions on the predictive validity of constructs from the original trans-contextual model in the current sample, but highlight the value of extending the model to incorporate additional constructs representing non-conscious processes.
Project description:BACKGROUND:Public health authorities have been recommending interventions such as physical distancing and face masks, to curtail the transmission of coronavirus disease (COVID-19) within the community. Public perceptions toward such interventions should be identified to enable public health authorities to effectively address valid concerns. The Health Belief Model (HBM) has been used to characterize user-generated content from social media during previous outbreaks, with the aim of understanding the health behaviors of the public. OBJECTIVE:This study is aimed at developing and evaluating deep learning-based text classification models for classifying social media content posted during the COVID-19 outbreak, using the four key constructs of the HBM. We will specifically focus on content related to the physical distancing interventions put forth by public health authorities. We intend to test the model with a real-world case study. METHODS:The data set for this study was prepared by analyzing Facebook comments that were posted by the public in response to the COVID-19-related posts of three public health authorities: the Ministry of Health of Singapore (MOH), the Centers for Disease Control and Prevention, and Public Health England. The comments made in the context of physical distancing were manually classified with a Yes/No flag for each of the four HBM constructs: perceived severity, perceived susceptibility, perceived barriers, and perceived benefits. Using a curated data set of 16,752 comments, gated recurrent unit-based recurrent neural network models were trained and validated for text classification. Accuracy and binary cross-entropy loss were used to evaluate the model. Specificity, sensitivity, and balanced accuracy were used to evaluate the classification results in the MOH case study. RESULTS:The HBM text classification models achieved mean accuracy rates of 0.92, 0.95, 0.91, and 0.94 for the constructs of perceived susceptibility, perceived severity, perceived benefits, and perceived barriers, respectively. In the case study with MOH Facebook comments, specificity was above 96% for all HBM constructs. Sensitivity was 94.3% and 90.9% for perceived severity and perceived benefits, respectively. In addition, sensitivity was 79.6% and 81.5% for perceived susceptibility and perceived barriers, respectively. The classification models were able to accurately predict trends in the prevalence of the constructs for the time period examined in the case study. CONCLUSIONS:The deep learning-based text classifiers developed in this study help to determine public perceptions toward physical distancing, using the four key constructs of HBM. Health officials can make use of the classification model to characterize the health behaviors of the public through the lens of social media. In future studies, we intend to extend the model to study public perceptions of other important interventions by public health authorities.