The Computational, Pharmacological, and Physiological Determinants of Sensory Learning under Uncertainty.
ABSTRACT: The ability to represent and respond to uncertainty is fundamental to human cognition and decision-making. Noradrenaline (NA) is hypothesized to play a key role in coordinating the sensory, learning, and physiological states necessary to adapt to a changing world, but direct evidence for this is lacking in humans. Here, we tested the effects of attenuating noradrenergic neurotransmission on learning under uncertainty. We probed the effects of the ?-adrenergic receptor antagonist propranolol (40 mg) using a between-subjects, double-blind, placebo-controlled design. Participants performed a probabilistic associative learning task, and we employed a hierarchical learning model to formally quantify prediction errors about cue-outcome contingencies and changes in these associations over time (volatility). Both unexpectedness and noise slowed down reaction times, but propranolol augmented the interaction between these main effects such that behavior was influenced more by prior expectations when uncertainty was high. Computationally, this was driven by a reduction in learning rates, with people slower to update their beliefs in the face of new information. Attenuating the global effects of NA also eliminated the phasic effects of prediction error and volatility on pupil size, consistent with slower belief updating. Finally, estimates of environmental volatility were predicted by baseline cardiac measures in all participants. Our results demonstrate that NA underpins behavioral and computational responses to uncertainty. These findings have important implications for understanding the impact of uncertainty on human biology and cognition.
Project description:Current theories of psychosis highlight the role of abnormal learning signals, i.e., prediction errors (PEs) and uncertainty, in the formation of delusional beliefs. We employed computational analyses of behaviour and functional magnetic resonance imaging (fMRI) to examine whether such abnormalities are evident in clinical high risk (CHR) individuals. Non-medicated CHR individuals (n = 13) and control participants (n = 13) performed a probabilistic learning paradigm during fMRI data acquisition. We used a hierarchical Bayesian model to infer subject-specific computations from behaviour - with a focus on PEs and uncertainty (or its inverse, precision) at different levels, including environmental 'volatility' - and used these computational quantities for analyses of fMRI data. Computational modelling of CHR individuals' behaviour indicated volatility estimates converged to significantly higher levels than in controls. Model-based fMRI demonstrated increased activity in prefrontal and insular regions of CHR individuals in response to precision-weighted low-level outcome PEs, while activations of prefrontal, orbitofrontal and anterior insula cortex by higher-level PEs (that serve to update volatility estimates) were reduced. Additionally, prefrontal cortical activity in response to outcome PEs in CHR was negatively associated with clinical measures of global functioning. Our results suggest a multi-faceted learning abnormality in CHR individuals under conditions of environmental uncertainty, comprising higher levels of volatility estimates combined with reduced cortical activation, and abnormally high activations in prefrontal and insular areas by precision-weighted outcome PEs. This atypical representation of high- and low-level learning signals might reflect a predisposition to delusion formation.
Project description:1. The present study was aimed to determine whether propranolol improves contractile function of the ischaemic/reperfused heart through protection of the mitochondrial function during ischaemia. 2. Isolated perfused rat hearts were subjected to 35-min ischaemia followed by 60-min reperfusion. Pre-treatment with propranolol at the concentrations of 10 to 100 microM for the final 3 min of pre-ischaemia resulted in the improvement of ischaemia/reperfusion-induced contractile dysfunction, release of creatine kinase (CK) into perfusate, and decrease in myocardial high-energy phosphates. Propranolol also attenuated ischaemia-induced accumulation in Na+, suggesting that cytosolic sodium overload during ischaemia was prevented by propranolol. 3. The mitochondrial oxygen consumption rate of skinned bundles from the perfused heart decreased at the end of ischaemia and it further decreased at the end of reperfusion. These decreases were cancelled by treatment with propranolol. A release of cytochrome c from the perfused heart was observed during ischaemia, and this release was suppressed by treatment with propranolol. 4. To elucidate the direct effect of propranolol on mitochondria, the mitochondria were isolated from normal hearts and their activities were determined in the presence of various concentrations of Na+ and propranolol. The addition of sodium lactate, which mimicked sodium overload in the ischaemic heart, reduced the state 3 respiration, whereas this reduction was not attenuated by the presence of propranolol. 5. These results suggest that cardioprotection of propranolol may be exerted via attenuating Na+ influx into cardiac cells followed by prevention of the mitochondrial dysfunction in the ischaemic heart, leading to improvement of energy production of the heart during reperfusion.
Project description:Healthy adults flexibly adapt their learning strategies to ongoing changes in uncertainty, a key feature of adaptive behaviour. However, the developmental trajectory of this ability is yet unknown, as developmental studies have not incorporated trial-to-trial variation in uncertainty in their analyses or models. To address this issue, we compared adolescents' and adults' trial-to-trial dynamics of uncertainty, learning rate, and exploration in two tasks that assess learning in noisy but otherwise stable environments. In an estimation task-which provides direct indices of trial-specific learning rate-both age groups reduced their learning rate over time, as self-reported uncertainty decreased. Accordingly, the estimation data in both groups was better explained by a Bayesian model with dynamic learning rate (Kalman filter) than by conventional reinforcement-learning models. Furthermore, adolescents' learning rates asymptoted at a higher level, reflecting an over-weighting of the most recent outcome, and the estimated Kalman-filter parameters suggested that this was due to an overestimation of environmental volatility. In a choice task, both age groups became more likely to choose the higher-valued option over time, but this increase in choice accuracy was smaller in the adolescents. In contrast to the estimation task, we found no evidence for a Bayesian expectation-updating process in the choice task, suggesting that estimation and choice tasks engage different learning processes. However, our modeling results of the choice task suggested that both age groups reduced their degree of exploration over time, and that the adolescents explored overall more than the adults. Finally, age-related differences in exploration parameters from fits to the choice data were mediated by participants' volatility parameter from fits to the estimation data. Together, these results suggest that adolescents overestimate the rate of environmental change, resulting in elevated learning rates and increased exploration, which may help understand developmental changes in learning and decision-making.
Project description:In conditions of constant illumination, the eye pupil diameter indexes the modulation of arousal state and responds to a large breadth of cognitive processes, including mental effort, attention, surprise, decision processes, decision biases, value beliefs, uncertainty, volatility, exploitation/exploration trade-off, or learning rate. Here, I propose an information theoretic framework that has the potential to explain the ensemble of these findings as reflecting pupillary response to information processing. In short, updates of the brain's internal model, quantified formally as the Kullback-Leibler (KL) divergence between prior and posterior beliefs, would be the common denominator to all these instances of pupillary dilation to cognition. I show that stimulus presentation leads to pupillary response that is proportional to the amount of information the stimulus carries about itself and to the quantity of information it provides about other task variables. In the context of decision making, pupil dilation in relation to uncertainty is explained by the wandering of the evidence accumulation process, leading to large summed KL divergences. Finally, pupillary response to mental effort and variations in tonic pupil size are also formalized in terms of information theory. On the basis of this framework, I compare pupillary data from past studies to simple information-theoretic simulations of task designs and show good correspondance with data across studies. The present framework has the potential to unify the large set of results reported on pupillary dilation to cognition and to provide a theory to guide future research.
Project description:Basal forebrain cholinergic neurons constitute a major neuromodulatory system implicated in normal cognition and neurodegenerative dementias. Cholinergic projections densely innervate neocortex, releasing acetylcholine to regulate arousal, attention, and learning. However, their precise behavioral function is poorly understood because identified cholinergic neurons have never been recorded during behavior. To determine which aspects of cognition their activity might support, we recorded cholinergic neurons using optogenetic identification in mice performing an auditory detection task requiring sustained attention. We found that a non-cholinergic basal forebrain population-but not cholinergic neurons-were correlated with trial-to-trial measures of attention. Surprisingly, cholinergic neurons responded to reward and punishment with unusual speed and precision (18 ± 3 ms). Cholinergic responses were scaled by the unexpectedness of reinforcement and were highly similar across neurons and two nuclei innervating distinct cortical areas. These results reveal that the cholinergic system broadcasts a rapid and precisely timed reinforcement signal, supporting fast cortical activation and plasticity.
Project description:Understanding the impact of infectious disease pandemic on stock market volatility is of great concerns for investors and policy makers, especially during recent new coronavirus spreading period. Using an extended GARCH-MIDAS model and a newly developed Infectious Disease Equity Market Volatility Tracker (EMV-ID), we investigate the effects of infectious disease pandemic on volatility of US, China, UK and Japan stock markets through January 2005 to April 2020. The empirical results show that, up to 24-month lag, infectious disease pandemic has significant positive impacts on the permanent volatility of international stock markets, even after controlling the influences of past realized volatility, global economic policy uncertainty and the volatility leverage effect. At different lags of eruptions in infectious disease pandemic, EMV-ID has distinct effects on various stock markets while it has the smallest impact on permanent volatility of China's stock market.
Project description:Paranoia is the belief that harm is intended by others. It may arise from selective pressures to infer and avoid social threats, particularly in ambiguous or changing circumstances. We propose that uncertainty may be sufficient to elicit learning differences in paranoid individuals, without social threat. We used reversal learning behavior and computational modeling to estimate belief updating across individuals with and without mental illness, online participants, and rats chronically exposed to methamphetamine, an elicitor of paranoia in humans. Paranoia is associated with a stronger prior on volatility, accompanied by elevated sensitivity to perceived changes in the task environment. Methamphetamine exposure in rats recapitulates this impaired uncertainty-driven belief updating and rigid anticipation of a volatile environment. Our work provides evidence of fundamental, domain-general learning differences in paranoid individuals. This paradigm enables further assessment of the interplay between uncertainty and belief-updating across individuals and species.
Project description:Current neurocircuitry models of PTSD focus on the neural mechanisms that mediate hypervigilance for threat and fear inhibition/extinction learning. Less focus has been directed towards explaining social deficits and heightened risk of revictimization observed among individuals with PTSD related to physical or sexual assault. The purpose of the present study was to foster more comprehensive theoretical models of PTSD by testing the hypothesis that assault-related PTSD is associated with behavioral impairments in a social trust and reciprocity task and corresponding alterations in the neural encoding of social learning mechanisms. Adult women with assault-related PTSD (n = 25) and control women (n = 15) completed a multi-trial trust game outside of the MRI scanner. A subset of these participants (15 with PTSD and 14 controls) also completed a social and non-social reinforcement learning task during 3T fMRI. Brain regions that encoded the computationally modeled parameters of value expectation, prediction error, and volatility (i.e., uncertainty) were defined and compared between groups. The PTSD group demonstrated slower learning rates during the trust game and social prediction errors had a lesser impact on subsequent investment decisions. PTSD was also associated with greater encoding of negative expected social outcomes in perigenual anterior cingulate cortex and bilateral middle frontal gyri, and greater encoding of social prediction errors in the left temporoparietal junction. These data suggest mechanisms of PTSD-related deficits in social functioning and heightened risk for re-victimization in assault victims; however, comorbidity in the PTSD group and the lack of a trauma-exposed control group temper conclusions about PTSD specifically.
Project description:Anxiety results in sub-optimal motor learning, but the precise mechanisms through which this effect occurs remain unknown. Using a motor sequence learning paradigm with separate phases for initial exploration and reward-based learning, we show that anxiety states in humans impair learning by attenuating the update of reward estimates. Further, when such estimates are perceived as unstable over time (volatility), anxiety constrains adaptive behavioral changes. Neurally, anxiety during initial exploration increased the amplitude and the rate of long bursts of sensorimotor and prefrontal beta oscillations (13-30 Hz). These changes extended to the subsequent learning phase, where phasic increases in beta power and burst rate following reward feedback were linked to smaller updates in reward estimates, with a higher anxiety-related increase explaining the attenuated belief updating. These data suggest that state anxiety alters the dynamics of beta oscillations during reward processing, thereby impairing proper updating of motor predictions when learning in unstable environments.
Project description:The last decade has seen considerable discussion regarding a theoretical account of medial prefrontal cortex (mPFC) function with particular focus on the anterior cingulate cortex. The proposed theories have included conflict detection, error likelihood prediction, volatility monitoring, and several distinct theories of error detection. Arguments for and against particular theories often treat mPFC as functionally homogeneous, or at least nearly so, despite some evidence for distinct functional subregions. Here we used functional magnetic resonance imaging (fMRI) to simultaneously contrast multiple effects of error, conflict, and task-switching that have been individually construed in support of various theories. We found overlapping yet functionally distinct subregions of mPFC, with activations related to dominant error, conflict, and task-switching effects successively found along a rostral-ventral to caudal-dorsal gradient within medial prefrontal cortex. Activations in the rostral cingulate zone (RCZ) were strongly correlated with the unexpectedness of outcomes suggesting a role in outcome prediction and preparing control systems to deal with anticipated outcomes. The results as a whole support a resolution of some ongoing debates in that distinct theories may each pertain to corresponding distinct yet overlapping subregions of mPFC.