Frontal theta links prediction errors to behavioral adaptation in reinforcement learning

James F. Cavanagh, Michael J. Frank, Theresa J. Klein, John JB Allen

Research output: Contribution to journalArticle

203 Citations (Scopus)

Abstract

Investigations into action monitoring have consistently detailed a frontocentral voltage deflection in the event-related potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the feedback-related negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single-trial relationship between neural activity and the quanta of expectation violation remains untested. Although ERP methods are not well suited to single-trial analyses, the FRN has been associated with theta band oscillatory perturbations in the medial prefrontal cortex. Mediofrontal theta oscillations have been previously associated with expectation violation and behavioral adaptation and are well suited to single-trial analysis. Here, we recorded EEG activity during a probabilistic reinforcement learning task and fit the performance data to an abstract computational model (Q-learning) for calculation of single-trial reward prediction errors. Single-trial theta oscillatory activities following feedback were investigated within the context of expectation (prediction error) and adaptation (subsequent reaction time change). Results indicate that interactive medial and lateral frontal theta activities reflect the degree of negative and positive reward prediction error in the service of behavioral adaptation. These different brain areas use prediction error calculations for different behavioral adaptations, with medial frontal theta reflecting the utilization of prediction errors for reaction time slowing (specifically following errors), but lateral frontal theta reflecting prediction errors leading to working memory-related reaction time speeding for the correct choice.

Original languageEnglish (US)
Pages (from-to)3198-3209
Number of pages12
JournalNeuroImage
Volume49
Issue number4
DOIs
StatePublished - Feb 15 2010

Fingerprint

Learning
Reaction Time
Reward
Evoked Potentials
Task Performance and Analysis
Prefrontal Cortex
Short-Term Memory
Electroencephalography
Reinforcement (Psychology)
Brain

Keywords

  • Anterior cingulate
  • FRN
  • Prediction error
  • Reinforcement learning
  • Theta

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Neurology

Cite this

Frontal theta links prediction errors to behavioral adaptation in reinforcement learning. / Cavanagh, James F.; Frank, Michael J.; Klein, Theresa J.; Allen, John JB.

In: NeuroImage, Vol. 49, No. 4, 15.02.2010, p. 3198-3209.

Research output: Contribution to journalArticle

Cavanagh, James F. ; Frank, Michael J. ; Klein, Theresa J. ; Allen, John JB. / Frontal theta links prediction errors to behavioral adaptation in reinforcement learning. In: NeuroImage. 2010 ; Vol. 49, No. 4. pp. 3198-3209.
@article{e1c328e9f69d461c80387825c1a49d03,
title = "Frontal theta links prediction errors to behavioral adaptation in reinforcement learning",
abstract = "Investigations into action monitoring have consistently detailed a frontocentral voltage deflection in the event-related potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the feedback-related negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single-trial relationship between neural activity and the quanta of expectation violation remains untested. Although ERP methods are not well suited to single-trial analyses, the FRN has been associated with theta band oscillatory perturbations in the medial prefrontal cortex. Mediofrontal theta oscillations have been previously associated with expectation violation and behavioral adaptation and are well suited to single-trial analysis. Here, we recorded EEG activity during a probabilistic reinforcement learning task and fit the performance data to an abstract computational model (Q-learning) for calculation of single-trial reward prediction errors. Single-trial theta oscillatory activities following feedback were investigated within the context of expectation (prediction error) and adaptation (subsequent reaction time change). Results indicate that interactive medial and lateral frontal theta activities reflect the degree of negative and positive reward prediction error in the service of behavioral adaptation. These different brain areas use prediction error calculations for different behavioral adaptations, with medial frontal theta reflecting the utilization of prediction errors for reaction time slowing (specifically following errors), but lateral frontal theta reflecting prediction errors leading to working memory-related reaction time speeding for the correct choice.",
keywords = "Anterior cingulate, FRN, Prediction error, Reinforcement learning, Theta",
author = "Cavanagh, {James F.} and Frank, {Michael J.} and Klein, {Theresa J.} and Allen, {John JB}",
year = "2010",
month = "2",
day = "15",
doi = "10.1016/j.neuroimage.2009.11.080",
language = "English (US)",
volume = "49",
pages = "3198--3209",
journal = "NeuroImage",
issn = "1053-8119",
publisher = "Academic Press Inc.",
number = "4",

}

TY - JOUR

T1 - Frontal theta links prediction errors to behavioral adaptation in reinforcement learning

AU - Cavanagh, James F.

AU - Frank, Michael J.

AU - Klein, Theresa J.

AU - Allen, John JB

PY - 2010/2/15

Y1 - 2010/2/15

N2 - Investigations into action monitoring have consistently detailed a frontocentral voltage deflection in the event-related potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the feedback-related negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single-trial relationship between neural activity and the quanta of expectation violation remains untested. Although ERP methods are not well suited to single-trial analyses, the FRN has been associated with theta band oscillatory perturbations in the medial prefrontal cortex. Mediofrontal theta oscillations have been previously associated with expectation violation and behavioral adaptation and are well suited to single-trial analysis. Here, we recorded EEG activity during a probabilistic reinforcement learning task and fit the performance data to an abstract computational model (Q-learning) for calculation of single-trial reward prediction errors. Single-trial theta oscillatory activities following feedback were investigated within the context of expectation (prediction error) and adaptation (subsequent reaction time change). Results indicate that interactive medial and lateral frontal theta activities reflect the degree of negative and positive reward prediction error in the service of behavioral adaptation. These different brain areas use prediction error calculations for different behavioral adaptations, with medial frontal theta reflecting the utilization of prediction errors for reaction time slowing (specifically following errors), but lateral frontal theta reflecting prediction errors leading to working memory-related reaction time speeding for the correct choice.

AB - Investigations into action monitoring have consistently detailed a frontocentral voltage deflection in the event-related potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the feedback-related negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single-trial relationship between neural activity and the quanta of expectation violation remains untested. Although ERP methods are not well suited to single-trial analyses, the FRN has been associated with theta band oscillatory perturbations in the medial prefrontal cortex. Mediofrontal theta oscillations have been previously associated with expectation violation and behavioral adaptation and are well suited to single-trial analysis. Here, we recorded EEG activity during a probabilistic reinforcement learning task and fit the performance data to an abstract computational model (Q-learning) for calculation of single-trial reward prediction errors. Single-trial theta oscillatory activities following feedback were investigated within the context of expectation (prediction error) and adaptation (subsequent reaction time change). Results indicate that interactive medial and lateral frontal theta activities reflect the degree of negative and positive reward prediction error in the service of behavioral adaptation. These different brain areas use prediction error calculations for different behavioral adaptations, with medial frontal theta reflecting the utilization of prediction errors for reaction time slowing (specifically following errors), but lateral frontal theta reflecting prediction errors leading to working memory-related reaction time speeding for the correct choice.

KW - Anterior cingulate

KW - FRN

KW - Prediction error

KW - Reinforcement learning

KW - Theta

UR - http://www.scopus.com/inward/record.url?scp=73749083115&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=73749083115&partnerID=8YFLogxK

U2 - 10.1016/j.neuroimage.2009.11.080

DO - 10.1016/j.neuroimage.2009.11.080

M3 - Article

C2 - 19969093

AN - SCOPUS:73749083115

VL - 49

SP - 3198

EP - 3209

JO - NeuroImage

JF - NeuroImage

SN - 1053-8119

IS - 4

ER -