Abstract
Temporal discounting (TD) matures with age, alongside other markers of increased impulse control, and coherent, self-regulated behaviour. Discounting paradigms quantify the ability to refrain from preference of immediate rewards, in favour of delayed, larger rewards. As such, they measure temporal foresight and the ability to delay gratification, functions that develop slowly into adulthood. We investigated the neural maturation that accompanies the previously observed age-related behavioural changes in discounting, from early adolescence into mid-adulthood. We used functional magnetic resonance imaging of a hypothetical discounting task with monetary rewards delayed in the week to year range. We show that age-related reductions in choice impulsivity were associated with changes in activation in ventromedial prefrontal cortex (vmPFC), anterior cingulate cortex (ACC), ventral striatum (VS), insula, inferior temporal gyrus, and posterior parietal cortex. Limbic frontostriatal activation changes were specifically associated with age-dependent reductions in impulsive choice, as part of a more extensive network of brain areas showing age-related changes in activation, including dorsolateral PFC, inferior parietal cortex, and subcortical areas. The maturational pattern of functional connectivity included strengthening in activation coupling between ventromedial and dorsolateral PFC, parietal and insular cortices during selection of delayed alternatives, and between vmPFC and VS during selection of immediate alternatives. We conclude that maturational mechanisms within limbic frontostriatal circuitry underlie the observed post-pubertal reductions in impulsive choice with increasing age, and that this effect is dependent on increased activation coherence within a network of areas associated with discounting behaviour and inter-temporal decision-making.
Introduction
Foresighted adaptive behaviour matures slowly, well into adolescence, with the development of self-regulatory processes, such as cognitive flexibility, motor inhibition, and impulse control (Casey et al., 2008, Luna, 2009, Rubia et al., 2000). Adolescence in particular is a developmental period characterised by hallmarks of impulsive behaviour, and is associated with a disproportionately high risk of psychopathological morbidity (Casey et al., 2008, Kelley et al., 2004). Developmental neuropsychiatric disorders, such as Attention Deficit Hyperactivity Disorder (ADHD), associated with a delay in brain maturation (Shaw et al., 2007), are characterised by reduced impulse control in both motor and cognitive domains (Sagvolden and Sergeant, 1998). These observations add significant weight to the study of the maturation of impulse control mechanisms during this critical developmental period.
Impulsivity is a broad construct, encompassing behaviours such as poor self-control, risk-taking, sensation-seeking, poor temporal foresight, and the inability to delay gratification (Evenden, 1999). Cognitively, impulsiveness has been associated with deficits in motor inhibitory control, attention, reward-related decision making and timing processes (Rubia et al., 2009). Most neuroimaging studies have focused on tasks of motor inhibition and attention to demonstrate the progressive underlying maturation of frontostriatal systems that regulate cognitive control (Bunge and Wright, 2007, Christakou et al., 2009b, Geier et al., 2010, Hardin et al., 2009, Hare et al., 2008, Luna et al., 2001, Ordaz et al., 2010, Rubia et al., 2009, Rubia et al., 2010, Rubia et al., 2000, Rubia et al., 2007, Rubia et al., 2006). Studies of the functional maturation of decision-making aspects of impulsivity have concentrated on ambiguous or risky decision-making, to model aspects of the risk-taking behaviour characteristic of this developmental period. Results suggest that adolescents are more prone to taking risks compared to adults because of neural and psychological hyper-responsiveness to reward (Crone et al., 2005, Crone and van der Molen, 2004, Crone and van der Molen, 2007, Ernst et al., 2009, Galvan, 2010, Van Leijenhorst et al., 2010).
Although experimental decision-making and temporal discounting (TD) paradigms share some cognitive and neural substrates, there is no direct evidence for the specific neurodevelopmental processes related to inter-temporal bridging components of decision-making, as measured by TD.
Temporal discounting describes the observation that the subjective value of rewards decreases as a function of the prospective delay to their attainment (Cardinal, 2006, Dalley et al., 2008). In TD tasks, participants choose between small but immediate, or delayed but larger reward alternatives, based on their perceived relative value. The procedure allows the estimation of the subjective value of a reward as a function of the delay to its presentation. Inversely, it can be interpreted as an index of the subjective tolerance to temporal delays in units of reward (Rubia et al., 2009). The rate (or “steepness”) of discounting (i.e. the relative impact of increasing delay on the subjective devaluation of reward) is considered an experimental proxy of aspects of impulsivity related to delay of gratification and inter-temporal self-control. The rate of discounting by humans and other animals yields a hyperbolic TD curve (Mazur and Vaughan, 1987, Rachlin et al., 1991), a theoretical model that describes the decay of subjective reward value with increasing delay.
The rate of TD of reward decreases with increasing age from childhood to adulthood (Scheres et al., 2006), along with maturational changes in other indices of self-regulation, such as reductions in risk-taking (Galvan et al., 2006, Geier et al., 2010), and progressive improvement in inhibitory control (Luna et al., 2001, Rubia et al., 2000). Other factors may affect the maturation of this behaviour, such as development of the ability to orient to future self-perspectives (Bjornebekk and Gjesme, 2009, Ersner-Hershfield et al., 2009, Steinberg et al., 2009), and late maturing frontal-lobe mediated temporal reflexivity and executive foresight mechanisms (Fuster, 2002, McAuley et al., 2007).
Although this framework suggests a varied and complex underlying neural substrate for discounting behaviour, evidence shows significant convergence with other decision-making paradigms. Both human and non-human studies implicate limbic frontostriatal circuitry in the regulation of reward discounting, with dopaminergic and serotonergic mechanisms playing a key role (Cardinal et al., 2001, Cardinal et al., 2004, Kheramin et al., 2004, Kobayashi and Schultz, 2008, Mobini et al., 2002, Schweighofer et al., 2008, Winstanley et al., 2006b). In humans, ventral striatal activation tracks the value of delayed rewards, and the level of this neural responsivity is inversely correlated with individual rates of discounting (Ballard and Knutson, 2009). Conversely, ventral striatal activation levels during reward presentation correlate with the degree of preference for immediate compared to delayed rewards as measured outside the scanner (Hariri et al., 2006). Further, insular as well as frontal and parietal activation has been associated with choices of delayed over immediate rewards (McClure et al., 2004, Rubia et al., 2009, Wittmann et al., 2007, Xu et al., 2009).
Decision-making research then assigns a context-sensitive evaluative role to the limbic frontostriatal system. Importantly, however, TD behaviour also recruits lateral fronto-parietal areas for the computation of decisions that involve contextual parameters beyond the absolute value of reinforcers, such as their temporal distribution and frequency, their level of abstraction, and the degree of ambiguity associated with their presentation (Christakou et al., 2009a, McClure et al., 2004, Rubia et al., 2009). This is in line with the involvement of these associative regions in working memory (Miller and Cohen, 2001), numerical, and abstract computations (Piazza and Izard, 2009, Pinel et al., 2001). Additionally, animal and human studies show that prefrontal and parietal brain regions are crucial for temporal integrative functions such as inter-temporal bridging (Fuster, 1990, Fuster, 2002, Quintana and Fuster, 1999, Rubia et al., 1998). Thus, the recruitment of fronto-parietal associative systems underpins aspects of discounting related to the representation of the future self in relation to the available choice alternatives, and subserves the use of a future self perspective to moderate myopic, impulsive reactions to the availability of immediate reward, driven by “unsupervised” limbic mechanisms (Bickel et al., 2007).
The development of corticostriatal systems is significantly accelerated with the onset of puberty, through adolescence, and continues well into adulthood (Sowell et al., 2007, Sowell et al., 2003). Both associative and limbic nodes show marked structural developmental changes during this period. Longitudinal anatomical studies show that the prefrontal cortex, mediating higher-order control functions, matures later in life compared to areas specialised for more basic, reactive functions (Casey et al., 2005, Giedd et al., 1999, Gogtay et al., 2004, Sowell et al., 2004). These anatomical changes, comprising maturation of cortical thickness and myelination indices (Huttenlocher, 1979), are accompanied by progressive functional maturation as measured in tasks of cognitive control (Christakou et al., 2009b, Marsh et al., 2006, Ordaz et al., 2010, Rubia et al., 2010, Rubia et al., 2000, Rubia et al., 2007, Rubia et al., 2006, Velanova et al., 2009).
The delayed maturation of prefrontal regulatory regions lags behind significant functional and structural changes in subcortical areas and neurochemical systems that mediate responsivity to motivational, emotional, and social cues (Casey et al., 2008, Galvan et al., 2006). This combination conspires against coherent regulatory control during adolescence (Fareri et al., 2008, Yurgelun-Todd, 2007). Increased impulsive decision-making in younger individuals, reflected in sensation-seeking, risk-taking, and the inability to delay gratification, is believed to stem from this developmental pattern (Spear, 2000). Despite accumulating evidence for the impact of frontostriatal development on the maturation of inhibitory and attentional components associated with impulsivity (Adleman et al., 2002, Bunge and Wright, 2007, Christakou et al., 2009b, Crone et al., 2005, Luna, 2009, Marsh et al., 2006, Rubia et al., 2006), little is known about the development of the neural substrates of TD between late childhood and adulthood.
In this study, we concentrated on the maturation of the neural systems that underlie inter-temporal decision-making, investigating their impact on changes in TD of reward from adolescence into adulthood (Olson et al., 2007, Scheres et al., 2006, Steinberg et al., 2009). We used a functional magnetic resonance imaging (fMRI) adaptation of hypothetical temporal discounting (HTD).
HTD allows the examination of processes that “judge” relative values across time-points, over and above the mechanisms of response selection and motor inhibition (which are more tightly coupled to real discounting or choice paradigms). There is evidence to suggest that while experiential discounting taps into moment-to-moment, state changes in impulsivity (Reynolds et al., 2006, Reynolds and Schiffbauer, 2004), hypothetical discounting taps into trait impulsivity (showing for example good temporal stability; Beck and Triplett, 2009, Kirby, 2009, Ohmura et al., 2006). It is characteristic that acute pharmacological manipulations have been shown to affect real-time discounting performance, but not hypothetical discounting (de Wit, 2009, McDonald et al., 2003, Shiels et al., 2009), while, conversely, genotypic variation in frontostriatal dopamine-handling mechanisms is associated with hypothetical but not experiential discounting (Paloyelis et al., 2010).
In this study, we were particularly interested in brain areas where the effects of increasing age on activation interacted with decreasing rates of discounting in our HTD task. We suggest that this interaction indexes the maturational processes which are crucial for the observed developmental decrease in the steepness of discounting.
Given evidence for the involvement of the limbic corticostriatal system and lateral fronto-parietal areas in TD, and the progressive maturation of these networks during adolescence and into adulthood, we tested the hypothesis that age-dependent activation changes within this system would accompany the predicted age-related reduction in the steepness of discounting rate.
Methods
Participants
Forty (40) healthy males participated in the experiment (mean age 20.15, SD = 5.64; range 11.96–31.77). Because of significant sex differences in brain structure, development, and function (including in related tasks and brain regions; Bobova et al., 2009, Christakou et al., 2009b, Rubia et al., 2010, Tranel et al., 2005), only male participants were included in the study to increase homogeneity. All participants were right-handed, as assessed with the Edinburgh Handedness Inventory (Oldfield, 1971) (mean laterality quotient: 90.40, SD = 14.38). Participant IQ was estimated with the Wechsler Abbreviated Scale of Intelligence (WASI, Harcourt Assessment): mean IQ = 117, SD = 12. There was no significant change in IQ with age across the whole sample (r = −0.11, p = ns), or within the adult (age > 18, n = 21, r = 0.21, p = ns), or adolescent subgroups (age < 18, n = 19, r = −0.23, p = ns).
Exclusion criteria were psychiatric or neurological disorders, learning disability, dyslexia, current or past drug abuse, head injury and psychotropic medication. All participants gave written informed consent and received £30 compensation for their participation. The study was approved by the local research ethics committee, and was undertaken in compliance with the Code of Ethics of the World Medical Association (Declaration of Helsinki).
Temporal discounting task
We used our fMRI adaptation of the hypothetical TD paradigm (Rubia et al., 2009). Participants were required to choose between an amount of money (between £0 and £100) available immediately, and a larger, fixed amount (£100) available after a week, a month, or a year. We used an adjusting-amount procedure, developed by Richards et al. (1997). The value of the immediate reward was adjusted using an algorithm based on the previous choices of the participant for each of the three different delays. This narrowed the range of the immediate values offered for each delay trial type, converging towards the value of the participant's subjective equivalent of the fixed delayed reward (Richards et al., 1999). The adjusting algorithm was employed to ensure equal numbers of immediate and delayed choices for each participant in each delay trial type, in order to facilitate the fMRI analysis.
Choices were presented to the right (fixed, delayed) and left (adjusted, immediate) sides of a computer screen, displayed onto a projector screen, and viewed by the participants through a system of mirrors fixed to the head coil. Choice types were presented on the same side of the screen to minimise the sensorimotor mapping load of the decision. Participants indicated their preference by pressing a left or right button (with the right index or right middle finger), corresponding to the location of their preferred option on the screen. Choices were displayed for 4 s, followed by a blank screen variable period of at least 8 s (serving as the implicit baseline), depending on the participant's reaction time (inter-trial interval = 12 s). Trials of the three different delays were randomly interspersed throughout the testing session (20 trials for each delay). Responses were monitored for consistency during the session, and checked for distribution of preferences after data collection to eliminate potential data skewed by random selections (this was not necessary for any of the participants).
Participants were acclimatised to the scanner environment in advance of the testing session in a “mock” scanner, where they practised the task they were going to perform in an environment similar to the scanner facility. The practice session consisted of 12 pseudo-randomised presentations of hypothetical choices between a small immediate amount and the standard £100 after a week, a month, or a year, such that each delay was presented four times.
Performance data analysis
To estimate the steepness of TD for each participant, we first calculated the point of effective indifference between the immediate amount and the delayed £100 for each delay interval (day, month, or year). This “indifference”, or “switch” point was calculated as the midpoint value between the lowest immediate reward selected by the subject and the next lowest immediate reward available (i.e. the value of immediate reward offered at which the subject began consistently to select the standard £100 delayed reward) (Hariri et al., 2006, Mitchell, 1999). The indifference point is equivalent to the individual's subjective value of £100 when it is available after each delay.
A hyperbolic decay function has been shown to describe the relationship between the subjective value of a reward as a function of the delay to its presentation. The mathematical expression of this relationship is V = A/(1 + kD), where V is the subjective value of a reward of amount A, D is the delay to reward presentation, and k is a constant characterising the individual's rate of discounting (Mazur and Vaughan, 1987, Rachlin et al., 1991). The value of k is frequently used as the main dependent variable of choice impulsivity in the TD paradigm. We estimated the discounting parameter k for our data with an iterative least-squares fitting process using the Solver function of Microsoft Excel (Brown, 2001).
However, in this study, the inherent limitations of fMRI task adaptation, including the relatively small number of trials and the use of only three delay points, limited the goodness-of-fit of the data to a non-linear curve function. In addition, the distribution of k values was not normal across the sample, skewed by very low frequency, very high value outliers, likely a side-effect of the fitting process (there was no systematic developmental direction) (Kolmogorov–Smirnov test Z = 1.75, p < 0.005; any and all subsequent analyses based on the k estimate used the square root transform of k values). Consequently, discounting behaviour was assessed using the area-under-the-curve (AUC) method as described by Myerson et al. (2001). Briefly, we plotted the normalised subjective values of the £100 for each delay against the normalised experimental delays, and calculated the AUC of these plots for each participant, using this value as the main dependent variable of the task. Smaller AUC values therefore denote steeper discounting rates, i.e. increased choice impulsivity. There was a highly significant correlation between the AUC measure and the square root transform of the k parameter as estimated in our population (r = 0.99, p < 0.001), indicating substantial congruency between the two metrics of choice impulsivity. Linear age-dependent changes in AUC were assessed using Pearson's correlation analysis.
Participants were slower overall to make delayed compared to immediate choices (paired samples t test: t = 2.44, p < 0.05), but reaction times did not differ between adults and adolescents (delayed: F = 0.01, p = n.s.; immediate: F = 0.31, p = n.s.), suggesting similar deliberation times.
Functional magnetic resonance imaging
Data acquisition
Gradient echo-planar magnetic resonance imaging data were acquired on a GE Signa 3 T system (General Electric, Milwaukee, WI, USA) at the Centre for Neuroimaging Sciences, King's College London, using a semi-automated image quality control procedure. A quadrature birdcage headcoil was used for RF transmission and reception. In each of 22 non-contiguous planes, 480 T2*-weighted MR images depicting BOLD (Blood Oxygen Level Dependent) contrast covering the whole brain were acquired (TE = 30 ms, TR = 1.5 s, flip angle = 60°, in-plane resolution = 3.75 mm, slice thickness = 5.0 mm, slice skip = 0.5 mm). A whole-brain high resolution structural scan (inversion recovery gradient echo EPI, TE = 40 ms, TR = 3 s, flip angle = 90°, 43 slices, slice thickness = 3.0 mm, slice skip = 0.3 mm) was also acquired, on which to superimpose the activation maps.
Data analysis
The fMRI data were analyzed using the XBAM software developed at the Institute of Psychiatry (http://brainmap.it ). The software uses a non-parametric permutation-based strategy, rather than normal theory based inference, to minimise assumptions, and employs median, rather than mean-based statistics, to control for outlier effects. Finally, its most commonly used test statistic (SSQ ratio, see below) is computed by standardising for individual differences in residual noise before embarking on second level multi-subject testing using robust permutation-based methods. This allows a mixed effects approach to analysis—an approach that has recently been recommended following a detailed analysis of the validity and impact of normal theory based inference in fMRI in large number of subjects (Thirion et al., 2007).
Pre-processing and individual subject analysis
Functional MRI data were realigned to minimise motion-related artefacts (Bullmore et al., 1999), and smoothed using a Gaussian filter (full-width half maximum, 8.82 mm). Time-series analysis of individual subject activation was performed using XBAM v4, with a wavelet-based re-sampling method previously described (Bullmore et al., 2001). Briefly, we first convolved each experimental condition (immediate and delayed choices), with two Poisson model functions (delays of 4 and 8 s). We calculated the weighted sum of these two convolutions that gave the best fit (least-squares) to the time series at each voxel. A goodness-of-fit statistic (the SSQ-ratio) was then computed at each voxel consisting of the ratio of the sum of squares of deviations from the mean intensity value due to the model (fitted time series), divided by the sum of squares due to the residuals (i.e. original time series minus model time series). The appropriate null distribution for assessing significance of any given SSQ-ratio was established using the wavelet-based data re-sampling method (Bullmore et al., 2001), and applying the model-fitting process to the re-sampled data. This process was repeated 20 times at each voxel and the data combined over all voxels, resulting in 20 null parametric maps of SSQ-ratio for each subject, which were combined to give the overall null distribution of SSQ-ratio. The same permutation strategy was applied at each voxel to preserve spatial correlation structure in the data. Instead of relying on asymptotic distributions such as t or F that assume data normality, we use data-driven, permutation-based methods with minimal distributional assumptions that have been shown to be more suitable for fMRI data analysis in sample sizes similar to the ours (Zhang et al., 2009). Individual SSQ-ratio maps were transformed into standard space, first by rigid body transformation of the fMRI data into a high-resolution inversion recovery image of the same subject, and then by affine transformation onto a Talairach template (Talairach and Tournoux, 1988).
Group level analysis
Group activation maps were produced for each experimental condition (immediate > baseline; delayed > baseline, immediate <> delayed) by calculating the median observed SSQ-ratio over all subjects at each voxel in standard space and testing them against the null distribution of median SSQ-ratios computed from the identically transformed wavelet re-sampled (permuted) data (Brammer et al., 1997). The voxel-level threshold was first set to p < 0.05 to give maximum sensitivity and to avoid type II errors. Next, a cluster-mass threshold was computed from the distribution of cluster masses in the wavelet-permuted data, such that the final expected number of type I error clusters under the null hypothesis was < 1 per whole brain (cluster level probability p < 0.01). Cluster mass rather than a cluster extent threshold was used, to minimise discrimination against possible small, but strongly responding foci of activation (Bullmore et al., 1999).
Whole-brain regression analyses
Regression with age
In order to identify brain regions where activation showed progressive or regressive maturational changes, we tested for a linear correlation between whole-brain activation and age: the Pearson product-moment correlation coefficient was first computed at each voxel in standard space between age and signal change over all subjects. The correlation coefficients were recalculated after randomly permuting the subjects' age (but not the fMRI data). Repeating the second step many times (1000 times per voxel, then combining over all voxels) gives the distribution of correlation coefficients under the null hypothesis that there is no association between specific age and specific BOLD effects. This null distribution can then be used to assess the probability of any particular correlation coefficient under the null hypothesis. The critical value of the correlation coefficient at any desired type I error level in the original (non-permuted) data can be determined by reference to this distribution. Statistical analysis was extended to cluster level as described by Bullmore et al. (1999). The cluster probability under the null hypothesis (p < 0.01) was chosen to set the level of expected type I error clusters at an acceptable level (i.e. < 1 per whole brain).
Regression of age-correlated data masks with rate of discounting
We were interested in brain maturation processes which underlie the age-related decreases in discounting as indexed by increases in AUC in the task. Therefore, in order to identify brain areas where activation changes with age were related to age-dependent increases in AUC (i.e. reductions in discounting), we first extracted 3D masks of age-correlated activation (see “Regression with age” section above), and then used these masks to confine a voxel-wise regression analysis with AUC using the same method. This analysis yielded areas where individual differences in age and AUC interacted in their impact on brain activation.
Functional connectivity analyses
The use of age-correlated activation masks in the regression analysis with AUC produced clusters within frontostriatal circuitry, where increasing age and AUC affected the level of activation in line with our hypotheses, including vmPFC and ventral striatum (VS) during immediate choices (see Results section). In light of the involvement of vmPFC and VS in reward-related decision-making and inter-temporal choice behaviour (Ballard and Knutson, 2009, Christakou et al., 2009a, Rubia et al., 2009), and the maturation of frontostriatal circuitry during adolescence into mid-adulthood (Casey et al., 2008, Christakou et al., 2009b, Rubia et al., 2010), we tested the hypothesis that vmPFC connectivity with its ventral striatal projection region would increase as a function of age, as well as age-dependent increases in AUC.
The vmPFC group-level functional cluster was used as a seed in a correlational functional connectivity analysis. Firstly (a, below), we measured the strength of vmPFC connectivity with the VS group-level cluster identified in the same analysis during immediate choices, and, secondly (b, below), we examined whole-brain changes in vmPFC connectivity, to identify other areas where the strength of activation coherence with vmPFC was also associated with age-dependent changes in AUC during both immediate and delayed choices:
(a)
The time-series of activation in the regions of interest (vmPFC and VS) were extracted for each subject, and used to construct the partial time-series for each condition (immediate or delayed choice trials). Pearson correlation analysis was used to quantify the strength of activation coupling between these regions during immediate trials, during which there was a significant effect of both age and AUC on the activation of these areas. We hypothesised that increasing age and AUC would be associated with enhanced functional coupling between vmPFC and VS, which we tested with one-tailed Pearson correlations with age and AUC on the Fischer-transformed functional connectivity z scores across the whole sample during immediate trials (Silver and Dunlap, 1987).
(b)
The vmPFC partial time-series were also used in an intra-subject, voxel-wise, whole-brain, correlation analysis (two-tailed Pearson's correlation), to identify further brain areas where activation was functionally coupled with that of the vmPFC ROI for each experimental condition (immediate or delayed choice trials). In order to quantify the developmental trajectory of the resulting functional connectivity patterns we used whole-brain correlation analysis of the correlation coefficients at each voxel with age (as described in “Whole-brain regression analyses” above). Based on this analysis, we investigated the interaction effect of increasing age and decreasing discounting rate (i.e. increasing AUC), by extracting 3D data masks for each condition from the whole-brain analysis of correlation of connectivity with age, and used these to constrain a correlation with AUC (see “Regression of age-correlated data masks with rate of discounting” above). This analysis yielded areas where strengthening of activation coupling with the regions of interest was associated with age-related reductions in impulsive choice.
Results
Behavioural results
The rate of discounting decreased with age across the sample, as the main discounting measure of the area-under-the-curve (AUC) showed a significant linear increase with age (AUC × age Person's r = 0.42, p < 0.01; Fig. 1A), unaffected by IQ (AUC × age, controlling for IQ, Pearson's r = 0.45, p < 0.005). There was a strong correlation between the AUC and the square root transform of our estimate of the k parameter of the hyperbolic discounting function (Pearson's r = 0.99, p < 0.001), suggesting that our data conform with the proposed hyperbolic decay of subjective value with increasing delay (Mazur and Vaughan, 1987, Rachlin et al., 1991). IQ did not correlate with AUC, either across the whole sample (r = 0.16, p = ns), or within either of the adult (age ≥ 18 years; n = 21, r = 0.28, p = ns), or adolescent (age < 18 years; n = 19, r = 0.18, p = ns) subgroups, in line with previous work with comparable sample sizes, suggesting that TD is independent of IQ in the normal range (Kirby and Petry, 2004, Monterosso et al., 2001) (we note however that sample size and magnitude effects may skew the association of aspects of intelligence with discounting: see (Shamosh et al., 2008) for association of discounting with metrics of fluid intelligence, working memory and prefrontal function in healthy adults, and (Olson et al., 2007) for correlation between discounting and verbal IQ in healthy adolescents).
Fig. 1. Age effects on discounting behaviour: A. Area under the curve (AUC) increases with age across the sample (r = 0.42, p < 0.01). B. The subjective value of £100 decreases as a function of delay, more steeply in adolescents than in adults (group × delay interaction p < 0.05).
We note that it is possible that different discounting patterns may produce similar AUC values, in cases where the relative steepness of discounting is disproportionately distributed along the available delays. In this experiment, age-related AUC analysis would be problematic if the discounting curves of younger and older participants “crossed”; if for example younger participants discounted rewards more steeply against shorter delays, and older participants discounted rewards more steeply against longer delays (or vice versa), this would produce different shapes of discounting curves, but potentially similar AUCs. In order to ensure the validity of the AUC measure we performed an additional categorical analysis, between adolescent (age < 18 years, n = 19) and adult (age > 18 years, n = 21) participants. We checked the relative distribution of the subjective values (i.e. the indifference points) of adolescents and adults by using repeated measures analysis of variance (ANOVA) across the three delays (week, month and year) to examine the developmental nature of the effect of delay on average subjective values, and confirmed that the shape of the discounting curves did not affect the use of the AUC as the main measure of discounting.
Repeated measures ANOVA of the subjective values across the three delays showed that increasing delays had a greater effect on the subjective value of the £100 for adolescent than for adult participants: there was an overall decrease in subjective value with increasing delay (F(2,65) = 116.74, p < 0.001), and although there was no main effect of group (F(1,38) = 2.97, p = ns), there was a group by delay interaction (F(2,76) = 7.85, p < 0.001) with adolescents showing a steeper reduction in subjective value with increasing delay (Fig. 1B).
In addition to the incorporation of the individual adjusting algorithm in the discounting task, we further ensured that the calculation of the AUC for participants of different ages was not affected by a skewed distribution of immediate versus delayed choices (e.g. if adults made less immediate choices than adolescents at certain delays). We performed a repeated measures ANOVA on the percentage of immediate choices for the two groups across the three delays. There was a significant effect of delay (the percentage of immediate choices was larger for shorter delays across the sample; F = 17.33, p < 0.001), but there was no effect of group (F = 1.42, p = n.s.) and no group by delay interaction (F = 2.88, p = n.s.).
fMRI results
Activation differences between immediate and delayed choices across the whole sample
Across the whole sample, there was increased activation during delayed compared to immediate choices in left dlPFC, pre-SMA, left precentral gyrus, left parietal and temporal cortex, bilateral occipital cortex and cerebellum (Fig. 2A, Table 1). Increased activation during immediate compared to delayed choices was observed in right inferior frontal gyrus (BA44), right dorsal striatum and bilateral thalamus (Fig. 2B, Table 1).
Fig. 2. Activation differences between immediate and delayed choices across the whole sample A. Contrast of delayed > immediate choices. B. Contrast of immediate > delayed choices. Activation clusters are detailed in Table 1.Table 1. Activation differences between immediate and delayed choices across the whole sample.
Immediate > delayed choices | ||||||
Cluster location (BA) | Side | Talairach coordinates | Probability | Size | ||
x | y | z | ||||
Insula | R | 43 | 11 | 4 | < 0.005 | 21 |
Caudate body | R | 18 | 7 | 9 | < 0.005 | 47 |
Claustrum | R | 25 | −19 | 20 | < 0.005 | 10 |
Midbrain/thalamus | L/R | −4 | −11 | −2 | < 0.005 | 9 |
Delayed > immediate choices | ||||||
Cluster location (BA) | Side | Talairach coordinates | Probability* | Size | ||
x | y | z | ||||
Inferior frontal gyrus (9) | L | −40 | 7 | 26 | > 0.005 | 72 |
Medial frontal gyrus (6) | L | −4 | 0 | 48 | > 0.001 | 215 |
Precentral gyrus, frontal lobe (6) | L | −40 | −4 | 26 | > 0.001 | 118 |
Postcentral gyrus, parietal lobe (3) | L | −33 | −26 | 48 | > 0.001 | 290 |
Precuneus (7) | L | −25 | −59 | 37 | > 0.005 | 94 |
Cuneus, occipital lobe (30) | L | −22 | −67 | 15 | > 0.005 | 79 |
Supramarginal gyrus (40) | L | −51 | −48 | 31 | > 0.01 | 42 |
Superior temporal gyrus (22) | L | −36 | −52 | 9 | > 0.05 | 26 |
Anterior lobe, cerebellum | R | 25 | −48 | −24 | > 0.01 | 54 |
Posterior lobe, cerebellum | L | −22 | −81 | −13 | > 0.000 | 343 |
Posterior lobe, cerebellum | R | 29 | −74 | −13 | > 0.001 | 288 |
BA: Brodmann's area. Talairach coordinates shown for the peak of each 3D cluster. Probability: cluster-wise probability yielding less than 1 false positive cluster. Size: number of activated voxels per 3D cluster. *Note: The delayed > immediate contrast produced a single contiguous cluster of 1665 voxels. To characterise this activation, we used an in-house de-clustering routine, setting minimum cluster size to 5 and using a search kernel of 5. The probabilities for this contrast are given for sub-cluster maxima.
Age-dependent changes in activation during immediate or delayed choices
Significant age-related linear increases in activation were observed, during both immediate and delayed choices compared to baseline, in left ventromedial and dorsolateral PFC, and bilateral parietal and superior temporal regions. During delayed choices, we additionally observed increases in activation in pre-supplementary motor area, dorsal striatum, thalamus, and cerebellum.
Activation decreases with age were observed in limbic corticostriatal areas, including ventrolateral PFC, ventral striatum, insula, and medial temporal lobe, as well as dorsal frontopolar, posterior cingulate, and parietal cortices. These changes were observed during both immediate and delayed choices. During immediate choices, additional reductions in activation with age were observed in the dorsal anterior cingulate cortex. These results are illustrated and described in more detail in the supplementary material accompanying this article.
Interaction effects of increasing age and decreasing impulsive choice on whole-brain activation
The only brain area where age-dependent activation increase was associated with reductions in discounting (as indexed by increases in AUC) was the left vmPFC during immediate choices (Brodmann area 10; Talairach coordinates: −18, 46, −6; probability: < 0.04; size (voxels): 6; Fig. 3).
Fig. 3. vmPFC activation increases with increasing age and AUC during immediate choices.
Areas where age-dependent activation decreases were associated with reductions in discounting were identified during immediate choices in the ventral striatum, insula, anterior cingulate, occipital, and parietal cortices (Table 2). During delayed choices, activation reductions with age and AUC were observed in the temporal lobe (Table 2), and no areas exhibited significant activation increases. Table 2. Interaction effects of increasing age and decreasing impulsive choice on whole-brain activation decreases.
A. Immediate choices | ||||||
Cluster location (BA) | Side | Talairach coordinates | Probability | Size | ||
x | y | z | ||||
ACC (24) | R | 7 | 4 | 31 | < 0.001 | 29 |
Superior parietal lobule (7) | R | 25 | −63 | 53 | < 0.001 | 20 |
Postcentral gyrus (5) | L | −33 | −44 | 64 | < 0.001 | 10 |
Fusiform gyrus, occipital lobe (19) | R | 33 | −63 | −7 | < 0.001 | 28 |
Ventral striatum (caudate head)/subgenual ACC (25) | L | −7 | 26 | −13 | < 0.001 | 33 |
Putamen/thalamus | L | −22 | −15 | 4 | < 0.001 | 18 |
B. Delayed choices | ||||||
Cluster location (BA) | Side | Talairach coordinates | Probability | Size | ||
x | y | z | ||||
Superior temporal gyrus (38) | L | −54 | 11 | −24 | < 0.001 | 42 |
Superior/middle temporal gyrus (38/21) | L | −33 | 0 | −29 | < 0.001 | 23 |
Superior temporal gyrus (38) | R | 29 | 7 | −29 | < 0.001 | 19 |
Fusiform gyrus, temporal lobe (20) | L | −33 | −26 | −24 | < 0.001 | 28 |
Fusiform gyrus, temporal lobe (20) | R | 54 | −33 | −24 | < 0.005 | 9 |
Brain areas exhibiting reductions in activation with increasing age and AUC. BA: Brodmann's area. Talairach coordinates shown for the peak of each 3D cluster. Probability: cluster-wise probability yielding less than 1 false positive cluster. Size: number of activated voxels per 3D cluster.
Interaction of increasing age and AUC on functional connectivity
Based on previous implication of limbic frontostriatal circuitry in TD, we examined the functional connectivity of two regions of interest (ROI), in the vmPFC and the ventral striatum during immediate choices. The ROIs were defined from the activation clusters identified in the interaction analysis of whole brain correlation with age and AUC (Fig. 3, Table 2).
Maturing age and reduced discounting (i.e. higher AUC measures) were associated with increased activation during immediate choices in the vmPFC cluster, and decreased activation in the ventral striatum. However, correlational connectivity analysis revealed that the two regions exhibited enhanced coupling (p < 0.001) in the majority of subjects (n = 32). Importantly, this effect increased with age (one-tailed Pearson correlation with connectivity z transforms: r = 0.30, p < 0.05) and AUC (r = 0.35, p < 0.05 (Fig. 4); this relationship was also true for the square-root transform of the k parameter (r = 0.35, p < 0.05)). Importantly, the AUC/connectivity correlation was age-dependent (r(controlling for age) = 0.28, p = n.s.), and it was not significant in the adult or adolescent subgroups alone (adult r = 0.14, p = n.s.; adolescent r = 0.16, p = n.s.). These results suggest that the AUC/connectivity relationship was largely, if not exclusively, a developmental effect, and that this effect is at play throughout adolescence and into young adulthood. These suggestions should be considered with respect to the limitations of cross-sectional studies; the exact nature of the developmental trajectory of the AUC/connectivity association can only be addressed by longitudinal investigations.
Fig. 4. Ventromedial frontostriatal functional connectivity strengthens with age (top chart) and AUC (bottom chart): vmPFC and ventral striatal functional connectivity increased during immediate choices with maturing age and reduced impulsive choice. Filled circles denote the participants for whom the ROI functional connectivity analysis showed significant coupling in activation between vmPFC and VS (n = 32, p < 0.001).
Whole-brain analysis of vmPFC connectivity during both immediate and delayed choices showed that age and AUC further interacted to increase vmPFC activation coupling with bilateral insula, dlPFC, parietal and visual cortices, and cerebellum during delayed choices (Fig. 5B, Table 3). There were no changes during immediate choices, with the exception of a visual cortex cluster (Fig. 5A, Table 3).
Fig. 5. Age and AUC interaction effects on vmPFC functional connectivity during immediate (A) and delayed (B) choices. Three-dimensional clusters of age and AUC interaction effect on activation for each trial type are presented superimposed on horizontal slices, marked with the z-coordinate as distance in millimeters from the anterior–posterior commissure.Table 3. Whole-brain vmPFC connectivity analysis during immediate and delayed choices.
A. Immediate choices | ||||||
Cluster location (BA) | Side | Talairach coordinates | Probability | Size | ||
x | y | z | ||||
Inferior occipital gyrus (18) | R | 25 | −89 | −2 | < 0.001 | 8 |
B. Delayed choices | ||||||
Cluster location (BA) | Side | Talairach coordinates | Probability | Size | ||
x | y | z | ||||
Insula, inferior frontal gyrus (47) | L | −29 | 22 | −13 | < 0.001 | 25 |
Insula, inferior frontal gyrus (47) | R | 36 | 26 | −7 | < 0.001 | 40 |
Superior/middle frontal gyrus (6) | L | −7 | −4 | 59 | < 0.001 | 118 |
Middle frontal gyrus (6) | R | 25 | −4 | 48 | < 0.001 | 28 |
Superior frontal gyrus (8) | L | −18 | 48 | 42 | < 0.01 | 6 |
Middle frontal gyrus (10) | L | −33 | 52 | 4 | < 0.001 | 49 |
Middle frontal gyrus (10) | R | 4 | −85 | −29 | < 0.001 | 61 |
Superior frontal gyrus (10) | R | 43 | 52 | 20 | < 0.005 | 4 |
Precuneus (7) | L | −29 | −41 | 48 | < 0.001 | 21 |
Supramarginal gyrus, parietal lobe (40) | L | −54 | −44 | 31 | < 0.001 | 14 |
Angular gyrus, parietal lobe (39) | R | 51 | −63 | 31 | < 0.005 | 5 |
Superior temporal gyrus (38) | L | −36 | 0 | −13 | < 0.001 | 8 |
Middle occipital gyrus (18) | L | −25 | −89 | 9 | < 0.001 | 52 |
Middle occipital gyrus (18) | R | 22 | −89 | 20 | < 0.001 | 32 |
Middle occipital gyrus (18) | R | 43 | −81 | 4 | < 0.01 | 6 |
BA: Brodmann's area. Talairach coordinates shown for the peak of each 3D cluster. Probability: cluster-wise probability yielding less than 1 false positive cluster. Size: number of activated voxels per 3D cluster.
Discussion
This study aimed to identify the neural substrates of maturational reductions in choice impulsivity from early adolescence to mid-adulthood, as indexed in a task of hypothetical TD of monetary rewards, in the week to year range.
During immediate choices, age-dependent changes in activation in the extended limbic corticostriatal system were associated with the observed reduction in TD with increasing age: out of a broad network of areas which exhibited age-related changes in activation, only activation increases in vmPFC, and activation decreases in VS, ACC and temporal lobe were also correlated with increases in the main discounting measure, the area-under-the-curve (AUC), an index of delay-tolerant, foresighted choice in the task.
Further, using these vmPFC and VS clusters as regions of interest (ROIs), we investigated the maturation in functional connectivity of these areas, again in the context of age-depended reductions in TD. Despite the fact that activation increased with age and AUC in the vmPFC, and decreased in the VS, these areas showed significant parallel progressive strengthening in their activation coupling during immediate choices.
The inverse relationship of decreased ventral striatal activation with increasing AUC replicates previous findings of increased VS responsivity to reward presentation associated with steeper individual TD functions outside the scanner (Hariri et al., 2006), and is consistent with a broad literature of human and animal studies that implicate the ventral striatum in discounting processes (Cardinal, 2006, Dalley et al., 2008, Kalenscher et al., 2006, Reynolds, 2006). Here, we also show that maturation of choice behaviour in this task (i.e. reduced discounting with increasing age, as indexed by an increase in the AUC measure) is associated with age-related decreases in VS activation during immediate choices.
Taken together, these results suggest that this effect may be mediated through increased activation coherence within the vmPFC/VS system. Thus, frontal regulatory influence redirects the decision-making process that guides the pursuit of an immediately available reward away from ventral striatal hyper-responsiveness to proximal rewards in younger individuals, towards context-sensitive value-driven evaluation of the available alternatives in older individuals. Context-sensitive evaluation refers to the adaptive incorporation of information, not only about the absolute value of rewards but also about higher-order aspects of their availability, such as their frequency, cost, and longer-term impact, all integrated to calculate its appropriate discounted value. This proposal echoes our recent findings, with an fMRI adaptation of the Iowa Gambling Task (Bechara et al., 1994, Lawrence et al., 2009), where the vmPFC was shown to drive adaptive decision-making through contingency-sensitive outcome evaluation, with its level of activation associated with trait levels of impulsivity (Christakou et al., 2009a). Importantly, it is consistent with current models of adolescent development, suggesting that limbic hyper-responsiveness to reward is a primary feature of risk-taking and related decision-making characteristics during adolescence, progressively tempered by prefrontal control systems which mature later in development (Casey et al., 2008, Spear, 2000). Crone and van der Molen (2004) have specifically demonstrated age-related increases in sensitivity to the future consequences of decisions on a task that relies on intact/mature vmPFC functioning.
The finding of ventromedial PFC involvement in maturation of temporal discounting is in line with recent evidence implicating developmental changes in ventral PFC white matter organisation in discounting behaviour (Olson et al., 2009). Furthermore, studies of patients with lesions in ventromedial frontal lobe who show shortening of personal future time perspective as measured with a questionnaire (Fellows and Farah, 2005). Animal studies have found that lesions of the orbitofrontal cortex impair inter-temporal choice in rats (Cardinal, 2006, Kheramin et al., 2004, Mobini et al., 2002, Winstanley et al., 2004). The ventromedial prefrontal cortex, furthermore, is thought to be important for holding information in representational memory, as well as for incentive motivation, and is thus crucial for comparator operations of future and current rewards (Schoenbaum et al., 2006).
We further demonstrate that maturational changes in the steepness of discounting are associated with progressive strengthening of the correlation of activation between vmPFC and dlPFC, inferior parietal cortex, and insula, during delayed choices. This finding suggests that the involvement of vmPFC in guiding choice behaviour is supported by distinct neural systems depending on the available alternatives. Specifically, the strength of vmPFC functional connectivity with prefrontal and parietal cortices and insula becomes more important with age in choices of the delayed, larger alternative. Dorsolateral PFC and parietal cortex activation has been shown to correlate with the length of the delay of prospective reinforcement in a related task using fMRI (Ballard and Knutson, 2009), and is involved in complex decisions with ambiguous temporal or reward contingencies alongside the vmPFC (Christakou et al., 2009a, McClure et al., 2004).
Dorsolateral prefrontal and inferior parietal cortices have also been shown to be specifically sensitive to temporal delays and thus crucial for inter-temporal bridging and foresighted behaviour, independently of reward contexts (Braver et al., 1997, Rubia et al., 1998). This is in line with animal studies that isolated specific dlPFC neurons for the mediation of temporal bridging in delay tasks, as opposed to dlPFC neurons that were specifically sensitive to the mnemonic components of the task (Constantinidis et al., 2002, Fuster, 1990). The inferior parietal regions are important for encoding duration (Wittmann, 2009) and allocating attention to time (Coull, 2004, Ortuno et al., 2002), as well as quantity representations, and may thus contribute to inter-temporal choice in their role as magnitude comparators of both time and reward (Sandrini et al., 2004).
According to the model of discounting of Laibson (1997), describing a two-factor, quasi-hyperbolic function, immediate rewards are weighted more strongly than later rewards (proximal factor), while reward choices available in the future are discounted exponentially (distal factor). There is evidence to suggest that choosing between rewards that are available in the future is dependent on lateral prefrontal and parietal activation in adults and adolescents (McClure et al., 2004, Rubia et al., 2009).
The findings of progressively increased connectivity with age between vmPFC and bilateral inferior prefrontal cortices in association with foresighted choices are in line with evidence for inferior prefrontal activation being inversely associated with the steepness of delay discounting in adults (Wittmann et al., 2007). The role of inferior prefrontal cortex in the maturation of foresighted behaviour is in line with its role in the development of inhibitory control (Rubia et al., 2003, Rubia et al., 2007) which may be essential for delayed gratification, as it requires the inhibition of the immediate reward in favour of the future reward perspective.
The performance findings suggest that the age-related increase in AUC is driven largely by the distal factor of discounting behaviour, since adolescent subjects deviated in their choices from adults progressively in the month and year range compared to the day delay (Fig. 1B). The neuroimaging findings suggest that, with maturing age, vmPFC processes of reward evaluation leading to delayed choices rely on activation coherence with lateral prefrontal and inferior parietal areas, presumably supporting longer-term foresighted decisions.
Increased connectivity between vmPFC and insula associated with delayed choices is in line with previous observations of increased insular activation during delayed compared to immediate choices in similar hypothetical discounting paradigms in healthy adults (Tanaka et al., 2004, Wittmann et al., 2007) and children (Rubia et al., 2009). Insula activation is further involved in risk-taking (Ernst et al., 2002), and during the anticipation (Critchley et al., 2001) and presentation of rewards (Elliott et al., 2000). The insula is ideally placed to integrate the temporal and affective features of reinforcement, given its involvement in reward (Cardinal, 2006, Cardinal et al., 2002, Elliott et al., 2000, Volz et al., 2003, Wittmann et al., 2007), temporal encoding processes (Wittmann, 2009, Wittmann et al., 2010), and reward-related temporal processes (Rubia et al., 2009, Tanaka et al., 2004, Wittmann et al., 2007).
The insula receives autonomic and primary sensory input, and projects to the ventral PFC and basal ganglia regions of limbic circuitry (Cavada et al., 2000, Chikama et al., 1997, Mesulam and Mufson, 1982). Further, it has been associated particularly with decisions made under ambiguity (Paulus et al., 2003, Volz et al., 2003, Volz and von Cramon, 2006), aiding decision-making by linking autonomic signals of the potential risk of decisions (Bechara, 2001, Critchley et al., 2001, Paulus et al., 2003) with action selection (Huettel, 2006, Van Leijenhorst et al., 2010). There is evidence that, despite generating appropriate autonomic signals (e.g. heart rate and skin conductance) during decisions, children are unable to utilise these signals in adaptive decision-making (Crone et al., 2005, Crone et al., 2004, Crone and van der Molen, 2004, Crone and van der Molen, 2007), in contrast to healthy adults (Bechara et al., 1997). This finding has been linked to the underlying maturation of insular cortex activity from early adolescence to mid-adulthood (Van Leijenhorst et al., 2010). Our findings support and expand the evidence for functional maturation of the insula in reward-related tasks to temporal discounting processes. Furthermore, they extend previous evidence for functional maturation of this region during cognitive control (Christakou et al., 2009b, Rubia et al., 2007, Rubia et al., 2006). Functional maturation of the insula during foresighted choices may hence reflect progressive maturation of the interface between motivation and cognition.
In summary, our results suggest that, in addition to its proposed function in progressively regulating ventral striatal reward responsivity, vmPFC activity further supports the maturation of discounting behaviour by progressively incorporating more appropriate information about the delay-dependent value of the available options, through interaction with dorsal prefrontal, insular, and parietal areas. This interaction brings together cognitive control and temporal foresight processes (supported by the lateral PFC), abstract, numerical and attention processing (supported by the inferior parietal cortex), and processing of autonomic signals and of the temporal characteristics of reward (supported by the insula). Given evidence from both structural and functional developmental studies that these neural systems and cognitive processes develop through adolescence and into adulthood (Casey et al., 2008, Geier et al., 2010, Rubia et al., 2010, Rubia et al., 2007, Rubia et al., 2006, Van Leijenhorst et al., 2010), our results illustrate how this developmental process may drive the progressive maturation of discounting behaviour.
The significance of these findings is exemplified by previous studies showing that individuals with impulse-control disorders, including ADHD, substance abuse, and pathological gambling, exhibit steeper (i.e. more impulsive) reward discounting (Barkley et al., 2001, Crean et al., 2000, Petry, 2001, Rubia et al., 2009, Scheres et al., 2008, Winstanley et al., 2006a, Winstanley et al., 2006b), as well as reduced activation in ventromedial prefrontal cortex and ventral striatum (Plichta et al., 2009, Rubia et al., 2009, Scheres et al., 2007). Importantly, measures of discounting behaviour and brain activation have been associated with symptom severity (Alessi and Petry, 2003, Rubia et al., 2009, Scheres et al., 2008), and there are associations between self-report measures of trait impulsivity and discounting behaviour (Mitchell et al., 2005).
In conclusion, this study supports age-associated progressive control over limbic striatal hyper-responsiveness to reward, via progressive strengthening of functional connections between the vmPFC and striatal, frontoparietal and insular regions, mediating the development of self-controlled, foresighted behaviour through adolescence and into adulthood.