Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2012 Sep;15(9):1281-9.
doi: 10.1038/nn.3188. Epub 2012 Aug 19.

Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value

Affiliations

Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value

Lung-Hao Tai et al. Nat Neurosci. 2012 Sep.

Abstract

In changing environments, animals must adaptively select actions to achieve their goals. In tasks involving goal-directed action selection, striatal neural activity has been shown to represent the value of competing actions. Striatal representations of action value could potentially bias responses toward actions of higher value. However, no study to date has demonstrated the direct effect of distinct striatal pathways in goal-directed action selection. We found that transient optogenetic stimulation of dorsal striatal dopamine D1 and D2 receptor-expressing neurons during decision-making in mice introduced opposing biases in the distribution of choices. The effect of stimulation on choice was dependent on recent reward history and mimicked an additive change in the action value. Although stimulation before and during movement initiation produced a robust bias in choice behavior, this bias was substantially diminished when stimulation was delayed after response initiation. Together, our data suggest that striatal activity is involved in goal-directed action selection.

PubMed Disclaimer

Figures

Figure 1
Figure 1. Design of the probabilistic switching task and animal performance
(a) Sequence of events in a probabilistic switching task. Mice learned to initiate a trial at the center port and to choose a left or right peripheral port for water reinforcement. Only one peripheral port was rewarded at a time. In 25% of trials, neither port was rewarded. The rewarded port switched only following a rewarded trial. (b) Fraction of choices for left port (n=28 subjects) for trials before and after a switch of the rewarded port (at trial 0). (c) Fraction of choices for the left port from one subject for reward histories in which two consecutive choices to either the left or the right port were made during the previous two trials. Data from mixed choice histories are not shown for brevity. All error bars represent s.e.m.
Figure 2
Figure 2. Modeling action value and animals' choice
(a) Contributions of intrinsic bias (brown), rewarded (cyan) and unrewarded (magenta) outcomes in the previous five trials on choices in the current trial derived from logistic regression (n=28 subjects, 22726±2881 trials per subject). (b) The fraction of choices for the left port given the relative action value. Data from each subject were grouped in 10 bins and represented in a distinct color. (c) The fraction of choices for the left port given the fraction of left choices predicted by the regression model. (d) Average median withdrawal time measured from go-light signal to nose-withdrawal from center port (n=28). Withdrawal time is shorter when the relative action value for either port is higher. (e) Example data from fourteen trial blocks. In the top panel, right reward blocks are represented in green and left reward blocks in orange. The dotted line indicates the subject's probability of choosing the left port averaged across four trials while the black line indicates predicted probability of choice based upon action value estimates (bottom panel). Long ticks correspond with rewarded trials while short tics represent unrewarded trials. All error bars represent s.e.m.
Figure 3
Figure 3. Optical stimulation induces opposing biases in an animal's choice
(a) Timing of optical stimulation within the task. In 6% of trials, optical stimulation was delivered to the dorsal striatum during a 500 ms period starting at the same time as the `Go' light cues. Stimulation occurred at 5, 10 or 20 Hz delivering respectively 3, 5 or 10 pulses of 5 ms light stimulation. (b,c) Examples showing the effect of 10 Hz stimulation in the left dorsomedial striatum (DMS) of a D1-Cre animal (b) and a D2-Cre animal (c) expressing ChR2-eYFP. Individual bars represent fraction of left choices for various reward histories in trials when the animal previously made two consecutive responses at the same port. Red bars indicate stimulation trials and blue bars represent trials without stimulation. (d,e) Fraction of left choices with and without stimulation for all possible combinations of choices and outcomes in the previous two trials with more than five total occurrences. (d) Data from the same D1-Cre animal shown in (b). The frequency of trials with a given reward history are indicated by the relative size of the circle. Filled circles represent a significant change in fraction of left choice with stimulation (p<0.05, Fisher's exact test). The red curve relates the probabilities of choice with and without stimulation for a fixed odds ratio (odds ratio=e−1.33±0.20). (e) Data from the same D2-Cre animal shown in (c) (odds ratio=e1.45± 0.18). All error bars represent s.e.m.
Figure 4
Figure 4. Dorsal striatal D1R-expressing neuron activation mimics an increase in relative action value for contralateral choice
Fraction of choices for the left port on trials with different relative action value estimates in D1-Cre animals in the presence (red) or absence (blue) of optical stimulation. (a) Representative data from one subject transduced with AAV-EF1α-DIO-ChR2-eYFP with optical stimulation in the right hemisphere. Logistic regression was used to fit the data from trials with (red line) and without stimulation (blue line). A leftward shift in the logistic curve represents a bias for the left reward port. (b) Summary data for probability of choice for the left port and relative action value pooled from all D1-Cre subjects expressing ChR2-eYFP and stimulated on the right hemisphere (top) or left hemisphere (bottom). A logistic regression fit for the no stimulation (thick blue line) and stimulation (thick red line) trial data was produced for data pooled from all subjects. Curves representing the estimated probability of choice for given relative action values in individual animals are plotted in light blue (no stimulation) or light red (with stimulation). Reported n refers to number of stimulation sites (1 per hemisphere). (c) Summary data from all D1-Cre subjects expressing eYFP only and stimulated on the right hemisphere (top) or left hemisphere (bottom). P values reported for t-tests: H0:ßstim=0 (distance between thick red and blue lines). All error bars represent s.e.m.
Figure 5
Figure 5. Dorsal striatal D2R-expressing neuron activation mimics a decrease in relative action value for contralateral choice
Fraction of choices for the left port on trials with different relative action value estimates in D2-Cre animals in the presence (red) or absence (blue) of optical stimulation. (a) Representative data from one subject transduced with AAV-EF1α-DIO-ChR2-eYFP with optical stimulation in the right hemisphere. Logistic regression was used to fit the data from trials with (red line) and without stimulation (blue line). A rightward shift in the logistic curve represents a bias for the right reward port. (b) Summary data for probability of choice for the left port and relative action value pooled from all D2-Cre subjects expressing ChR2-eYFP and stimulated on the right hemisphere (top) or left hemisphere (bottom). A logistic regression fit for the no stimulation (thick blue line) and stimulation (thick red line) trial data was produced for data pooled from all subjects. Curves representing the estimated probability of choice for given relative action values in individual animals are plotted in light blue (no stimulation) or light red (with stimulation). Reported n refers to number of stimulation sites (1 per hemisphere). (c) Summary data from all D2-Cre subjects expressing eYFP only and stimulated on the right hemisphere (top) or left hemisphere (bottom). P values reported for t-tests: H0:ßstim=0 (distance between thick red and blue lines). All error bars represent s.e.m.
Figure 6
Figure 6. Comparison of D1R- and D2R-expressing neuron activation across stimulation parameters
(a) Estimated change in the relative action value for choosing the port ipsilateral versus contralateral to the site of stimulation averaged across individuals within a group. Positive changes in relative action values correspond with an ipsilateral bias while negative changes correspond to a contralateral bias. Estimates of relative action change were derived from logistic regression analysis (see methods) for 5, 10, and 20 Hz stimulation sessions. Reported n refers to number of stimulation sites. The D1-Cre data set consisted of 6 stimulated animals expressing ChR2-eYFP and 4 control animals expressing only eYFP. The D2-Cre data set consisted of 8 stimulated animals expressing ChR2-eYFP and 5 control animals expressing only eYFP. (b) The median time taken to withdraw from the center port averaged across individual D1-Cre subjects (top) and D2-Cre subjects (bottom) expressing ChR2-eYFP in trials without stimulation (blue) or with stimulation (red) and across different relative action values for choosing the port ipsilateral versus contralateral to the site of stimulation. Positive relative action values correspond to trials in which the value of the port ipsilateral to the site of stimulation is greater than the contralateral port. Median times to withdraw are plotted for 5 Hz (left), 10 Hz (middle), and 20 Hz (right) stimulation sessions. All error bars represent s.e.m. *: p <0.05; **: p <0.01 for Wilcoxon signed-rank test. ***: p<0.001, Wilcoxon rank-sum test.
Figure 7
Figure 7. Significant bias was induced by stimulation limited to a 50 ms period before a `Go' cue
(a) The timing of events in sessions where a `Go' cue sound was delayed 70–110 ms after initiation of trial with center port entry. In these trials, two 5 ms optical pulses separated by 50 ms were delivered just prior and at the onset of the `Go cue' and before withdrawal from the center port. (b, c) Summary data for probability of choice for the ipsilateral port on trials with different relative action value estimates pooled from (b) D1-Cre and (c) D2-Cre subjects expressing ChR2-eYFP with confirmed eYFP expression sites. Positive relative action values correspond to trials in which the value of the port ipsilateral to the site of stimulation is greater than the contralateral port. A logistic regression fit for the no stimulation (thick blue line) and stimulation (thick red line) trial data was produced for data pooled from all subjects. Curves representing the estimated probability of ipsilateral choice for given relative action values in individual animals are plotted in light blue (no stimulation) or light red (with stimulation). P values reported for t-tests: H0:ßstim=0 (distance between thick red and blue lines). (d) Change in estimated relative action value for choosing the port ipsilateral versus contralateral to the site of stimulation averaged across individuals stimulated with two optical pulses prior to movement initiation. All error bars represent s.e.m.
Figure 8
Figure 8. Bias induced by stimulation diminished when stimulation was delayed 150 ms after a `Go' cue
(a) The timing of events in sessions where stimulation occurred at latencies of either 0 ms or 150 ms relative to `Go light' cue onset. (b,c) Summary data for probability of choice for the ipsilateral port and relative action value pooled from (b) D1-Cre and (c) D2-Cre subjects expressing ChR2-eYFP with confirmed eYFP expression sites stimulated with 0 ms latency (left) or 150 ms latency (right) relative to `Go light' onset. A logistic regression fit for the no stimulation (thick blue line) and stimulation (thick red line) trial data was produced for data pooled from D1-Cre or D2-Cre subjects. Curves representing the estimated probability of choice for given relative action values in individual animals are plotted in light blue (no stimulation) or light red (with stimulation). P values reported for t-tests: H0:ßstim=0 (distance between thick red and blue lines). (d) Estimated change in relative action value for choosing the port ipsilateral versus contralateral to the side of stimulation averaged across individuals stimulated at 10 Hz with 0 ms and 150 ms latency within the same session. Estimates of relative action change were derived from logistic regression analysis (see methods). Reported n refers to number of stimulation sites. **: p<0.005, Wilcoxon signed-rank test. All error bars represent s.e.m.

References

    1. Lauwereyns J, Watanabe K, Coe B, Hikosaka O. A neural correlate of response bias in monkey caudate nucleus. Nature. 2002;418:413–417. - PubMed
    1. Cromwell HC, Schultz W. Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol. 2003;89:2823–2838. - PubMed
    1. Samejima K, Ueda Y, Doya K, Kimura M. Representation of action-specific reward values in the striatum. Science. 2005;310:1337–1340. - PubMed
    1. Lau B, Glimcher PW. Value representations in the primate striatum during matching behavior. Neuron. 2008;58:451–463. - PMC - PubMed
    1. Barnes TD, Kubota Y, Hu D, Jin DZ, Graybiel AM. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature. 2005;437:1158–1161. - PubMed

Publication types

MeSH terms