{"database":"biostudies-literature","file_versions":[],"scores":null,"additional":{"omics_type":["Unknown"],"volume":["13(1)"],"submitter":["Barfuss W"],"pubmed_abstract":["In this work, we ask for and answer what makes classical temporal-difference reinforcement learning with [Formula: see text]-greedy strategies cooperative. Cooperating in social dilemma situations is vital for animals, humans, and machines. While evolutionary theory revealed a range of mechanisms promoting cooperation, the conditions under which agents learn to cooperate are contested. Here, we demonstrate which and how individual elements of the multi-agent learning setting lead to cooperation. We use the iterated Prisoner's dilemma with one-period memory as a testbed. Each of the two learning agents learns a strategy that conditions the following action choices on both agents' action choices of the last round. We find that next to a high caring for future rewards, a low exploration rate, and a small learning rate, it is primarily intrinsic stochastic fluctuations of the reinforcement learning process which double the final rate of cooperation to up to 80%. Thus, inherent noise is not a necessary evil of the iterative learning process. It is a critical asset for the learning of cooperation. However, we also point out the trade-off between a high likelihood of cooperative behavior and achieving this in a reasonable amount of time. Our findings are relevant for purposefully designing cooperative algorithms and regulating undesired collusive effects."],"journal":["Scientific reports"],"pagination":["1309"],"full_dataset_link":["https://www.ebi.ac.uk/biostudies/studies/S-EPMC9873645"],"repository":["biostudies-literature"],"pubmed_title":["Intrinsic fluctuations of reinforcement learning promote cooperation."],"pmcid":["PMC9873645"],"pubmed_authors":["Barfuss W","Meylahn JM"],"additional_accession":[]},"is_claimable":false,"name":"Intrinsic fluctuations of reinforcement learning promote cooperation.","description":"In this work, we ask for and answer what makes classical temporal-difference reinforcement learning with [Formula: see text]-greedy strategies cooperative. Cooperating in social dilemma situations is vital for animals, humans, and machines. While evolutionary theory revealed a range of mechanisms promoting cooperation, the conditions under which agents learn to cooperate are contested. Here, we demonstrate which and how individual elements of the multi-agent learning setting lead to cooperation. We use the iterated Prisoner's dilemma with one-period memory as a testbed. Each of the two learning agents learns a strategy that conditions the following action choices on both agents' action choices of the last round. We find that next to a high caring for future rewards, a low exploration rate, and a small learning rate, it is primarily intrinsic stochastic fluctuations of the reinforcement learning process which double the final rate of cooperation to up to 80%. Thus, inherent noise is not a necessary evil of the iterative learning process. It is a critical asset for the learning of cooperation. However, we also point out the trade-off between a high likelihood of cooperative behavior and achieving this in a reasonable amount of time. Our findings are relevant for purposefully designing cooperative algorithms and regulating undesired collusive effects.","dates":{"release":"2023-01-01T00:00:00Z","publication":"2023 Jan","modification":"2025-04-04T13:37:26.087Z","creation":"2025-04-04T13:37:26.087Z"},"accession":"S-EPMC9873645","cross_references":{"pubmed":["36693872"],"doi":["10.1038/s41598-023-27672-7"]}}