Two sides of the same coin: Beneficial and detrimental consequences of range adaptation in human reinforcement learning

Sophie Bavard, Aldo Rustichini, Stefano Palminteri

Research output: Contribution to journalArticlepeer-review

Abstract

Evidence suggests that economic values are rescaled as a function of the range of the available options. Although locally adaptive, range adaptation has been shown to lead to suboptimal choices, particularly notable in reinforcement learning (RL) situations when options are extrapolated from their original context to a new one. Range adaptation can be seen as the result of an adaptive coding process aiming at increasing the signal-to-noise ratio. However, this hypothesis leads to a counterintuitive prediction: Decreasing task difficulty should increase range adaptation and, consequently, extrapolation errors. Here, we tested the paradoxical relation between range adaptation and performance in a large sample of participants performing variants of an RL task, where we manipulated task difficulty. Results confirmed that range adaptation induces systematic extrapolation errors and is stronger when decreasing task difficulty. Last, we propose a range-adapting model and show that it is able to parsimoniously capture all the behavioral results.

Original languageEnglish (US)
Article numbereabe0340
JournalScience Advances
Volume7
Issue number14
DOIs
StatePublished - Mar 2021

Bibliographical note

Funding Information:
S.P. is supported by an ATIP-Avenir grant (R16069JS), the Programme Emergence(s) de la Ville de Paris, the Fondation Fyssen, the Fondation Schlumberger pour l?Education et la Recherche, the FrontCog grant (ANR-17-EURE-0017) and the Institut de Recherche en Sant? Publique (IRESP, grant number: 20II138-00). S.B. is supported by MILDECA (Mission Interministerielle de Lutte contre les Drogues et les Conduites Addictives) and the EHESS (Ecole des Hautes Etudes en Sciences Sociales). A.R. thanks the US Army for financial support (contract W911NF2010242). The funding agencies did not influence the content of the manuscript. Author contributions: S.B. and S.P. designed the experiments. S.B. ran the experiments. S.B. and S.P. analyzed the data. S.B., A.R., and S.P. interpreted the results. S.B. and S.P. wrote the manuscript. A.R. edited the manuscript. Competing interests: The authors declare that they have no financial or other competing interests. Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials and are available from Github repository (https://github.com/hrl-team/range). All custom scripts have been made available from Github repository (https://github.com/hrl-team/range). Additional modified scripts can be accessed upon request.

Publisher Copyright:
Copyright © 2021 The Authors, some rights reserved.

PubMed: MeSH publication types

  • Journal Article

Fingerprint

Dive into the research topics of 'Two sides of the same coin: Beneficial and detrimental consequences of range adaptation in human reinforcement learning'. Together they form a unique fingerprint.

Cite this