Advertisement

Eyeblink rate, a putative dopamine marker, predicts negative reinforcement learning by tDCS of the dlPFC

  • Michael Prowacki
    Affiliations
    College of Health & Human Sciences, Charles Darwin University, Darwin, Australia
    Search for articles by this author
  • Lee Wei Lim
    Affiliations
    School of Biomedical Sciences, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Pokfulam, Hong Kong, SAR, China
    Search for articles by this author
  • Author Footnotes
    1 Current address: College of Science, Health, Engineering and Education, Discipline of Psychology, Murdoch University, Perth, Australia.
    Luca Aquili
    Correspondence
    Corresponding author.
    Footnotes
    1 Current address: College of Science, Health, Engineering and Education, Discipline of Psychology, Murdoch University, Perth, Australia.
    Affiliations
    College of Health & Human Sciences, Charles Darwin University, Darwin, Australia
    School of Biomedical Sciences, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Pokfulam, Hong Kong, SAR, China
    Search for articles by this author
  • Author Footnotes
    1 Current address: College of Science, Health, Engineering and Education, Discipline of Psychology, Murdoch University, Perth, Australia.
Open AccessPublished:February 24, 2022DOI:https://doi.org/10.1016/j.brs.2022.02.009

      Keywords

      Studies have shown that anodal transcranial direct current stimulation (tDCS) applied to the dorsolateral prefrontal cortex (dlPFC) increases extracellular dopamine (DA) levels in the striatum [
      • Fonteneau C.
      • Redoute J.
      • Haesebaert F.
      • Le Bars D.
      • Costes N.
      • Suaud-Chagny M.-F.
      • et al.
      Frontal transcranial direct current stimulation induces dopamine release in the ventral striatum in human.
      ]. Furthermore, pharmacological and genetic investigations indicate that different DA levels interact with tDCS of the dlPFC to modulate performance in reinforcement learning (RL) tasks [
      • Borwick C.
      • Lal R.
      • Lim L.W.
      • Stagg C.J.
      • Aquili L.
      Dopamine depletion effects on cognitive flexibility as modulated by tDCS of the dlPFC.
      ,
      • Dennison O.
      • Gao J.
      • Lim L.W.
      • Stagg C.J.
      • Aquili L.
      Catecholaminergic modulation of indices of cognitive flexibility: a pharmaco-tDCS study.
      ,
      • Nieratschker V.
      • Kiefer C.
      • Giel K.
      • Krüger R.
      • Plewnia C.
      The COMT val/met polymorphism modulates effects of tDCS on response inhibition.
      ]. The results of these studies suggest an inverted U-shaped relationship between DA concentration and cognitive performance [
      • Cools R.
      • D'Esposito M.
      Inverted-U–shaped dopamine actions on human working memory and cognitive control.
      ].
      To determine differences between positive and negative RL, Frank and colleagues developed a probabilistic selection task (PST). They investigated a sample of Parkinson's disease (PD) patients on and off DA medication. They found those on DA medication displayed impairments in the negative reinforcement trials, whereas those off medication performed poorly in the positive reinforcement trials [
      • Frank M.J.
      • Seeberger L.C.
      • O'Reilly R.C.
      By carrot or by stick: cognitive reinforcement learning in parkinsonism.
      ]. These data support the theory of DA reward prediction error, whereby the DA medication prevents DA dips that are necessary to learn from worse than expected outcomes (i.e., negative prediction error), whereas those without DA medication do not have sufficient DA available to produce bursts of DA signalling when an outcome is better than expected (i.e., positive prediction error) [
      • Schultz W.
      • Dayan P.
      • Montague P.R.
      A neural substrate of prediction and reward.
      ].
      To further test the relationship among DA availability, tDCS, and RL, we utilized putative physiological and psychological markers of DA availability (namely eyeblink rate (EBR) [
      • Jongkees B.J.
      • Colzato L.S.
      Spontaneous eye blink rate as predictor of dopamine-related cognitive function—a review.
      ], extraversion [
      • Smillie L.D.
      • Jach H.K.
      • Hughes D.M.
      • Wacker J.
      • Cooper A.J.
      • Pickering A.D.
      Extraversion and reward-processing: consolidating evidence from an electroencephalographic index of reward-prediction-error.
      ], and impulsivity [
      • Petzold J.
      • Kienast A.
      • Lee Y.
      • Pooseh S.
      • London E.D.
      • Goschke T.
      • et al.
      Baseline impulsivity may moderate L-DOPA effects on value-based decision-making.
      ]) to predict the effects of anodal tDCS of the dlPFC in the PST. In agreement with Frank's findings, we hypothesized that anodal tDCS would be detrimental to those with high DA (as characterized by our physiological/psychological measurements) during the negative reinforcement trials (Fig. 1A).
      Fig. 1
      Fig. 1A. Hypothetical model based on the theory of DA reward prediction error and the findings of Frank et al. on PD patients who completed the PST on and off DA medication [
      • Frank M.J.
      • Seeberger L.C.
      • O'Reilly R.C.
      By carrot or by stick: cognitive reinforcement learning in parkinsonism.
      ]. Based on the above, we predicted the effects of tDCS on PST performance based on low, medium, and high EBR. The black, red, and green lines represent putative DA firing for the three EBR groups depending on whether approach trials (i.e., positive RL: top row) or avoidance trials (i.e., negative RL: bottom row) were presented. During sham tDCS and when a correct response was made in the approach trials, there would be a positive prediction error, which would be greatest in the high EBR participants. During anodal tDCS of the dlPFC, this signal would be amplified across all the three groups. Therefore, the low EBR group would be predicted to benefit from anodal tDCS given its presumed lower baseline DA, and hence, performance in the approach trials would be comparable to the medium and high EBR groups. During sham tDCS and when an incorrect response was made in the avoidance trials, there would be a negative prediction error, which would be greatest in the low EBR group. During anodal tDCS, DA activity would become less inhibitory across all three groups. Therefore, the high EBR group would be negatively affected by anodal tDCS, as DA activity may be too high to produce a negative prediction error required for learning.
      B. Representation of the PST paradigm. Participants are required to select between pairs of visual stimuli (Japanese characters which have been mirrored, flipped, and rotated) which are associated with different probabilities of reinforcement. During training, only three different pairs of stimuli are presented. In AB, A has an 80% chance of being reinforced (correct feedback), whereas B has only a 20% chance; for CD, C has a 70% and D has a 30% chance; and for EF, E has a 60% and F has a 40% chance. During testing, all other stimuli combinations are presented. Of particular interest are accuracy rates and RT during the approach trials, where the most positively reinforced stimulus A is paired in AC, AD, AE, and AF trials. Similarly, of interest in the avoidance trials is the least positive reinforced stimulus B paired in BC, BD, BE, and BF trials. C. Three-way significant interaction effects (Time∗tDCS∗EBR) were broken down into two-way factorial ANOVAs split by low and high EBR. There was a non-significant interaction between Time∗tDCS for avoidance trials in the low EBR group (accuracy). D. There was a significant interaction between time∗tDCS for the avoidance trials in the high EBR group. This was further investigated by planned comparisons. Accuracy rates were significantly lower during anodal tDCS compared to sham tDCS. Error bars as SEM. ∗ represents p < .05. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)
      The present study was a double-blind, crossover, sham-controlled randomized trial with counterbalancing of conditions (registered at ClinicalTrials.gov; identifier: NCT04798105). Thirty-three healthy participants from the university community took part in this study. Anodal tDCS (1.5 mA for 20 minutes) and Sham tDCS (1.5 mA faded in for 30 seconds, then off) of the dlPFC was applied over two experimental sessions. The anode was positioned over the left dlPFC centered on F3 in the 10–20 electroencephalography (EEG) system and the cathode was positioned on the contralateral supraorbital ridge (Fp2). We used the PST to measure positive and negative RL. The PST was administered four times, before and after each of the sham and anodal tDCS. Measures of interest included accuracy and reaction times (RT) in the positive reinforcement ‘approach’ trials and the negative reinforcement ‘avoidance’ trials. The positive reinforcement trials of the PST measure the ability to select the stimulus that has the highest probability of reinforcement (out of six stimuli), whereas the negative reinforcement trials measure the ability to avoid the least reinforced stimulus (see Fig. 1B for more details). Eyeblink rate, as a putative marker of DA concentration, was measured in participants over a 5-min period during the two experimental sessions. The impulsivity and extraversion questionnaires were administered as psychological proxies for DA. The median split for these putative DA markers (low and high) was established, which were then entered in a series of 2∗(2∗2) factorial mixed design ANOVAs (see Supplementary Materials 1.1–1.8).
      We report here the most salient findings. The full results of this study are available in the supplementary files (see Supplementary Materials 1.9). There was a significant Eyeblink∗tDCS∗Time interaction for the avoidance trials [F (1, 31) = 7.44, p = .010, η2p = .194]. This interaction could be broken down by splitting tDCS∗Time into low and high EBR. There was no significant interaction effect in the low EBR group (Fig. 1C), whereas a significant effect was observed in the high EBR group [F (1, 15) = 4.87, p = .043, η2p = .245]. These results were further examined with two planned comparisons. Although there was no difference in the accuracy between pre-anodal tDCS versus post-anodal tDCS, the accuracy scores were significantly lower for anodal tDCS compared with sham tDCS (p = .038, d = 0.57) (Fig. 1D). A similar pattern of results was seen for RT, although none of the planned comparisons were significant. There was no significant three-way interaction in the approach trials.
      There was a significant Impulsivity∗tDCS∗Time interaction in the avoidance trials [F (1, 31) = 5.44, p = .026, η2p = .149], although follow up showed no significant two-way interactions (tDCS∗Time) in the low and high impulsivity groups. Extraversion did not significantly determine the tDCS∗Time interaction for either avoidance or approach trials.
      Overall, we found that neither the psychological trait of impulsivity nor extraversion could act as useful putative DA markers predictive of the tDCS effects on RL. However, low and high EBR participants responded differently to tDCS stimulation during avoidance trials, but not during approach trials. Specifically, there was an impairment in their performance (i.e., lower accuracy for avoidance trials) following anodal tDCS of the dlPFC compared with sham tDCS. These findings parallel those reported by Frank's group on PD patients on and off DA medication [
      • Frank M.J.
      • Seeberger L.C.
      • O'Reilly R.C.
      By carrot or by stick: cognitive reinforcement learning in parkinsonism.
      ]. That is, the higher putative DA of the high EBR group in combination with anodal tDCS of the dlPFC, which has been shown to increase DA release in the striatum [
      • Fonteneau C.
      • Redoute J.
      • Haesebaert F.
      • Le Bars D.
      • Costes N.
      • Suaud-Chagny M.-F.
      • et al.
      Frontal transcranial direct current stimulation induces dopamine release in the ventral striatum in human.
      ], may have blocked DA neurons' ability to produce an inhibitory signal required for learning about stimuli that are poor predictors of reinforcement. This interpretation of the data is based on the theory of DA reward prediction error [
      • Schultz W.
      • Dayan P.
      • Montague P.R.
      A neural substrate of prediction and reward.
      ], although the obvious caveat is that we did not measure DA neuron firing or DA concentrations.
      Eyeblink rate has been shown to be associated with striatal DA activity, and importantly, it has been reported to be a useful predictor of reward-driven behaviour, cognitive flexibility, inhibitory control, and working memory involving DA function [
      • Jongkees B.J.
      • Colzato L.S.
      Spontaneous eye blink rate as predictor of dopamine-related cognitive function—a review.
      ]. Although EBR is a non-distinctive method of measuring DA, it has the advantages of being non-invasive and easy to quantify compared to neuroimaging (e.g., PET) and genetic approaches (e.g., DA gene polymorphisms). As the effects of tDCS on cognitive function are highly variable due to methodological and biological heterogeneity, EBR may act as a useful marker to help identify who will benefit from tDCS in the context of tasks and processes that depend on DA activity. For example, based on the results of our study, we may predict that cathodal (inhibitory) tDCS applied to those with low EBR may impair approach behaviour, as a burst of DA signal is required for learning. This would allow for designing tDCS studies that are better tailored to individual characteristics to achieve more homogenous outcomes.

      Author contributions

      LA designed the study. MP performed the research. LA and MP analysed the data. MP, LWL, and LA wrote the manuscript.

      Declaration of competing interest

      None.

      Acknowledgements

      We would like to thank the College of Health & Human Sciences at Charles Darwin University for funding this research.

      Appendix A. Supplementary data

      The following is the Supplementary data to this article:

      References

        • Fonteneau C.
        • Redoute J.
        • Haesebaert F.
        • Le Bars D.
        • Costes N.
        • Suaud-Chagny M.-F.
        • et al.
        Frontal transcranial direct current stimulation induces dopamine release in the ventral striatum in human.
        Cerebr Cortex. 2018; 28: 2636-2646
        • Borwick C.
        • Lal R.
        • Lim L.W.
        • Stagg C.J.
        • Aquili L.
        Dopamine depletion effects on cognitive flexibility as modulated by tDCS of the dlPFC.
        Brain Stimulation. 2020; 13: 105-108
        • Dennison O.
        • Gao J.
        • Lim L.W.
        • Stagg C.J.
        • Aquili L.
        Catecholaminergic modulation of indices of cognitive flexibility: a pharmaco-tDCS study.
        Brain Stimulation. 2019; 12: 290-295
        • Nieratschker V.
        • Kiefer C.
        • Giel K.
        • Krüger R.
        • Plewnia C.
        The COMT val/met polymorphism modulates effects of tDCS on response inhibition.
        Brain Stimulation. 2015; 8: 283-288
        • Cools R.
        • D'Esposito M.
        Inverted-U–shaped dopamine actions on human working memory and cognitive control.
        Biol Psychiatr. 2011; 69: e113-e125
        • Frank M.J.
        • Seeberger L.C.
        • O'Reilly R.C.
        By carrot or by stick: cognitive reinforcement learning in parkinsonism.
        Science. 2004; 306: 1940-1943
        • Schultz W.
        • Dayan P.
        • Montague P.R.
        A neural substrate of prediction and reward.
        Science. 1997; 275: 1593-1599
        • Jongkees B.J.
        • Colzato L.S.
        Spontaneous eye blink rate as predictor of dopamine-related cognitive function—a review.
        Neurosci Biobehav Rev. 2016; 71: 58-82
        • Smillie L.D.
        • Jach H.K.
        • Hughes D.M.
        • Wacker J.
        • Cooper A.J.
        • Pickering A.D.
        Extraversion and reward-processing: consolidating evidence from an electroencephalographic index of reward-prediction-error.
        Biol Psychol. 2019; 146: 107735
        • Petzold J.
        • Kienast A.
        • Lee Y.
        • Pooseh S.
        • London E.D.
        • Goschke T.
        • et al.
        Baseline impulsivity may moderate L-DOPA effects on value-based decision-making.
        Sci Rep. 2019; 9: 5652