Biological implementation of the temporal difference algorithm for reinforcement learning: theoretical comment on O'Reilly et al. (2007).

The ability to survive in the world depends critically on the brain's capacity to detect earlier and earlier predictors of reward or punishment. The dominant theoretical perspective for understanding this capacity has been the temporal difference (TD) algorithm for reinforcement learning. In this issue of Behavioral Neuroscience, R. C. O'Reilly, M. J. Frank, T. E. Hazy, and B. Watz (2007) propose a new model dubbed primary value and learned value (PVLV) that is simpler than TD, and they claimed that it is biologically more realistic. In this commentary, the author suggests some slight modifications of a previous biological implementation of TD instead of adopting the new PVLV algorithm.