Reward-Guided Learning with and without Causal Attribution.
Ontology highlight
ABSTRACT:
SUBMITTER: Jocham G
PROVIDER: S-EPMC4826429 | biostudies-literature | 2016 Apr
REPOSITORIES: biostudies-literature
Publications
Reward-Guided Learning with and without Causal Attribution.
Jocham Gerhard G Brodersen Kay H KH Constantinescu Alexandra O AO Kahn Martin C MC Ianni Angela M AM Walton Mark E ME Rushworth Matthew F S MF Behrens Timothy E J TE
Neuron 20160310 1
When an organism receives a reward, it is crucial to know which of many candidate actions caused this reward. However, recent work suggests that learning is possible even when this most fundamental assumption is not met. We used novel reward-guided learning paradigms in two fMRI studies to show that humans deploy separable learning mechanisms that operate in parallel. While behavior was dominated by precise contingent learning, it also revealed hallmarks of noncontingent learning strategies. The ...[more]
Similar Datasets
Project description:ObjectiveThis study addresses the contribution of genetics-related concerns to reduced childbearing among people with epilepsy.MethodsSurveys were completed by 606 adult patients with epilepsy of unknown cause at our medical center. Poisson regression analysis was used to assess the relations of number of offspring to: (1) genetic attribution (GA: participants' belief that genetics was a cause of their epilepsy), assessed via a novel scale developed from four survey items (Cronbach's alpha = .89), (2) participants' estimates of epilepsy risk in the child of a parent with epilepsy (1%, 5%-10%, 25%, and 50%-100%), and (3) participants' reports of the influence on their reproductive decisions of "the chance of having a child with epilepsy" (none/weak/moderate, strong/very strong). Analyses were adjusted for age, education, race/ethnicity, religion, type of epilepsy (generalized, focal, and both/unclassifiable), and age at epilepsy onset (<10, 10-19, and ≥20 years).ResultsAmong participants 18-45 years of age, the number of offspring decreased significantly with increasing GA (highest vs lowest GA quartile rate ratio [RR] = .5, p < .001), and increasing estimated epilepsy risk in offspring (with 5%-10% as referent because it is closest to the true value, RR for 25%: .7, p = .05; RR for 50%-100%: .6, p = .03). Number of offspring was not related to the reported influence of "the chance of having a child with epilepsy" on reproductive decisions. Among participants >45 years of age, the number of offspring did not differ significantly according to GA quartile or estimated offspring epilepsy risk. However, those reporting a strong/very strong influence on their reproductive decisions of "the chance of having a child with epilepsy" had only 60% as many offspring as others.SignificanceThese findings suggest that overestimating the risk of epilepsy in offspring can have important consequences for people with epilepsy. Patient and provider education about recurrence risks and genetic testing options to clarify risks are critical, given their potential influence on reproductive decisions.
| S-EPMC10308304 | biostudies-literature
Project description:Heating is a major source of air pollution. To improve air quality, a range of clean heating policies were implemented in China over the past decade. Here, we evaluated the impacts of winter heating and clean heating policies on air quality in China using a novel, observation-based causal inference approach. During 2015-2021, winter heating causally increased annual PM2.5, daily maximum 8-h average O3, and SO2 by 4.6, 2.5, and 2.3 μg m-3, respectively. From 2015 to 2021, the impacts of winter heating on PM2.5 in Beijing and surrounding cities (i.e., "2 + 26" cities) decreased by 5.9 μg m-3 (41.3%), whereas that in other northern cities only decreased by 1.2 μg m-3 (12.9%). This demonstrates the effectiveness of stricter clean heating policies on PM2.5 in "2 + 26" cities. Overall, clean heating policies caused the annual PM2.5 in mainland China to reduce by 1.9 μg m-3 from 2015 to 2021, potentially avoiding 23,556 premature deaths in 2021.
| S-EPMC10666544 | biostudies-literature
Project description:The observed increase in extreme weather has prompted recent methodological advances in extreme event attribution. We propose a machine learning-based approach that uses convolutional neural networks to create dynamically consistent counterfactual versions of historical extreme events under different levels of global mean temperature (GMT). We apply this technique to one recent extreme heat event (southcentral North America 2023) and several historical events that have been previously analyzed using established attribution methods. We estimate that temperatures during the southcentral North America event were 1.18° to 1.42°C warmer because of global warming and that similar events will occur 0.14 to 0.60 times per year at 2.0°C above preindustrial levels of GMT. Additionally, we find that the learned relationships between daily temperature and GMT are influenced by the seasonality of the forced temperature response and the daily meteorological conditions. Our results broadly agree with other attribution techniques, suggesting that machine learning can be used to perform rapid, low-cost attribution of extreme events.
| S-EPMC11338235 | biostudies-literature
Project description:Psychedelics are a broad class of drugs defined by their ability to induce an altered state of consciousness. These drugs have been used for millennia in both spiritual and medicinal contexts, and a number of recent clinical successes have spurred a renewed interest in developing psychedelic therapies. Nevertheless, a unifying mechanism that can account for these shared phenomenological and therapeutic properties remains unknown. Here we demonstrate in mice that the ability to reopen the social reward learning critical period is a shared property across psychedelics. Interestingly, the time course of critical period reopening is proportional to the duration of acute subjective effects reported in humans. Furthermore, the ability to reinstate social reward learning in adulthood is paralleled by metaplastic restoration of oxytocin mediated long-term depression (OT-LTD) in the Nucleus Accumbens (NAc). Finally, identification of differentially expressed genes in the ‘open state’ versus ‘closed state’, provides evidence that reorganization of the extracellular matrix (ECM) is a common downstream mechanism underlying psychedelic mediated critical period reopening. Together these results have significant implications for the implementation of psychedelics in clinical practice, as well as the design of novel compounds for the treatment of neuropsychiatric disease.
2023-06-14 | GSE230679 | GEO
Project description:Unlike conventional epidemiological studies that use observational data to estimate "associations" between risk factors and disease, the science of causal inference has identified situations where causal estimates can be made from observational data, using results such as the "backdoor criteria". Here these results are combined with established epidemiological methods, to calculate simple population attribution fractions that estimate the causal influence of risk factors on disease incidence, and can be estimated using conventional proportional hazards methods. A counterfactual argument gives an attribution fraction for individuals. Causally meaningful attribution fractions cannot be constructed for all risk factors or confounders, but they can for the important established risk factors of smoking and body mass index (BMI). Using the new results, the causal attribution of smoking and BMI to the incidence of 226 diseases in the UK Biobank are estimated, and summarised in terms of disease chapters from the International Classification of Diseases (ICD-10). The diseases most strongly attributed to smoking and BMI are identified, finding 11 with attribution fractions greater than 0.5, and a small number with protective associations. The results provide new tools to quantify the causal influence of risk factors such as smoking and BMI on disease, and survey the causal influence of smoking and BMI on the landscape of disease incidence in the UK Biobank population.
| S-EPMC9667855 | biostudies-literature
Project description:ObjectivesRapid advances in genetic research and increased use of genetic testing have increased the emphasis on genetic causes of epilepsy in patient encounters. Research in other disorders suggests that genetic causal attributions can influence patients' psychological responses and coping strategies, but little is known about how epilepsy patients and their relatives will respond to genetic attributions of epilepsy. We investigated the possibility that among members of families containing multiple individuals with epilepsy, depression, the most frequent psychiatric comorbidity in the epilepsies, might be related to the perception that epilepsy has a genetic cause.MethodsA self-administered survey was completed by 417 individuals in 104 families averaging 4 individuals with epilepsy per family. Current depression was measured with the Patient Health Questionnaire. Genetic causal attribution was assessed by three questions addressing the following: perceived likelihood of having an epilepsy-related mutation, perceived role of genetics in causing epilepsy in the family, and (in individuals with epilepsy) perceived influence of genetics in causing the individual's epilepsy. Relatives without epilepsy were asked about their perceived chance of developing epilepsy in the future, compared with the average person.ResultsPrevalence of current depression was 14.8% in 182 individuals with epilepsy, 6.5% in 184 biologic relatives without epilepsy, and 3.9% in 51 individuals married into the families. Among individuals with epilepsy, depression was unrelated to genetic attribution. Among biologic relatives without epilepsy, however, prevalence of depression increased with increasing perceived chance of having an epilepsy-related mutation (p = 0.02). This association was not mediated by perceived future epilepsy risk among relatives without epilepsy.SignificanceDepression is associated with perceived likelihood of carrying an epilepsy-related mutation among individuals without epilepsy in families containing multiple affected individuals. This association should be considered when addressing mental health issues in such families.
| S-EPMC5056147 | biostudies-literature
Project description:The propensity to attribute incentive salience to reward cues, measured by Pavlovian sign-tracking, is strongly associated with addiction-related traits including cocaine self-administration, impulsivity, novelty reactivity, and novelty preference. Despite its critical role in addiction, the genetic underpinnings of incentive salience attribution and its relationship to drug addiction are unknown. Mouse genetics can be a powerful means to discover genetic mechanisms underlying this relationship. However, feasibility of genetic dissection of sign-tracking in mice is unknown as only a single study limited to male C57BL/6J mice has rigorously examined this behavior, and limited sign-tracking was observed. Highly diverse mouse populations such as the Collaborative Cross (CC) and Diversity Outbred population (DO) possess a greater range of behavioral and genetic variation than conventional laboratory strains. In the present study, we evaluated sign-tracking and the related phenotype goal-tracking in mice of both sexes from five inbred CC and DO founder strains. Male CAST/EiJ mice exhibited robust sign-tracking; male NOD, male C57BL/6J, and female A/J mice also exhibited significant sign-tracking. Male and female mice from all strains exhibited significant goal-tracking, and significant strain and sex differences were observed. Sign-tracking in males was genetically correlated with exploration of a novel environment, and heritability of sign-tracking and goal-tracking ranged from .32 to .41. These data highlight the importance of considering genetic diversity when evaluating the occurrence of specific behavioral traits in the laboratory mouse and demonstrate that the CC and DO mouse populations can be used to discover mechanisms underlying genetic relationships among sign-tracking and addiction-related behaviors.
| S-EPMC4558302 | biostudies-literature
Project description:Learning about the expected value of choice alternatives associated with reward is critical for adaptive behavior. Although human choice preferences are affected by the presentation frequency of reward-related alternatives, this may not be captured by some dominant models of value learning, such as the delta rule. In this study, we examined whether reward learning is driven more by learning the probability of reward provided by each option, or how frequently each option has been rewarded, and assess how well models based on average reward (e.g. the delta model) and models based on cumulative reward (e.g. the decay model) can account for choice preferences. In a binary-outcome choice task, participants selected between pairs of options that had reward probabilities of 0.65 (A) versus 0.35 (B) or 0.75 (C) versus 0.25 (D). Crucially, during training there were twice the number of AB trials as CD trials, such that option A was associated with higher cumulative reward, while option C gave higher average reward. Participants then decided between novel combinations of options (e.g., AC). Most participants preferred option A over C, a result predicted by the Decay model, but not the Delta model. We also compared the Delta and Decay models to both more simplified as well as more complex models that assumed additional mechanisms, such as representation of uncertainty. Overall, models that assume learning about cumulative reward provided the best account of the data.
| S-EPMC6814570 | biostudies-literature
Project description:Unobservable mechanisms that tie causes to their effects generate observable events. How can one make inferences about hidden causal structures? This paper introduces the domain-matching heuristic to explain how humans perform causal reasoning when lacking mechanistic knowledge. We posit that people reduce the otherwise vast space of possible causal relations by focusing only on the likeliest ones. When thinking about a cause, people tend to think about possible effects that participate in the same domain, and vice versa. To explore the specific domains that people use, we asked people to cluster artifacts. The analyses revealed three commonly employed mechanism domains: the mechanical, chemical, and electromagnetic. Using these domains, we tested the domain-matching heuristic by testing adults' and children's causal attribution, prediction, judgment, and subjective understanding. We found that people's responses conform with domain-matching. These results provide evidence for a heuristic that explains how people engage in causal reasoning without directly appealing to mechanistic or probabilistic knowledge.
| S-EPMC9106179 | biostudies-literature
Project description:Biased causal attribution is a critical factor in the cognitive model of depression. Whereas depressed patients interpret events negatively, healthy people show a self-serving bias (internal attribution of positive events and external attribution of negative events).Using fMRI, depressed patients (n=15) and healthy controls (n=15) were confronted with positive and negative social events and made causal attributions (internal vs. external). Functional data were analyzed using a mixed effects model.Behaviourally, controls showed a self-serving bias, whereas patients demonstrated a balanced attributional pattern. Analysis of functional data revealed a significant group difference in a fronto-temporal network. Higher activation of this network was associated with non self-serving attributions in controls but self-serving attributions in patients. Applying a psycho-physiological interaction analysis, we observed reduced coupling between a dorsomedial PFC seed region and limbic areas during self-serving attributions in patients compared to controls.Results of the PPI analysis are preliminary given the liberal statistical threshold.The association of the behaviourally less frequent attributional pattern with activation in a fronto-temporal network suggests that non self-serving responses may produce a self-related response conflict in controls, while self-serving responses produce this conflict in patients. Moreover, attribution-modulated coupling between the dorsomedial PFC and limbic regions was weaker in patients than controls. This preliminary finding suggests that depression may be associated with disturbances in fronto-limbic coupling during attributional decisions. Our results implicate that treatment of major depression may benefit from approaches that facilitate reinterpretation of emotional events in a more positive, more self-serving way.
| S-EPMC3565123 | biostudies-literature