A gradual temporal shift of dopamine responses mirrors the progression of temporal difference error in machine learning

Ryunosuke Amo, Sara Matias, Akihiro Yamanaka, Kenji F. Tanaka, Naoshige Uchida, Mitsuko Watabe-Uchida

Research output: Contribution to journalArticlepeer-review

Abstract

A large body of evidence has indicated that the phasic responses of midbrain dopamine neurons show a remarkable similarity to a type of teaching signal (temporal difference (TD) error) used in machine learning. However, previous studies failed to observe a key prediction of this algorithm: that when an agent associates a cue and a reward that are separated in time, the timing of dopamine signals should gradually move backward in time from the time of the reward to the time of the cue over multiple trials. Here we demonstrate that such a gradual shift occurs both at the level of dopaminergic cellular activity and dopamine release in the ventral striatum in mice. Our results establish a long-sought link between dopaminergic activity and the TD learning algorithm, providing fundamental insights into how the brain associates cues and rewards that are separated in time.

Original languageEnglish
Pages (from-to)1082-1092
Number of pages11
JournalNature Neuroscience
Volume25
Issue number8
DOIs
Publication statusPublished - 2022 Aug

ASJC Scopus subject areas

  • Neuroscience(all)

Fingerprint

Dive into the research topics of 'A gradual temporal shift of dopamine responses mirrors the progression of temporal difference error in machine learning'. Together they form a unique fingerprint.

Cite this