text:
Improvement of jet flow simulations using ZDES mode 3 and silent turbulence generation

In the present extended abstract, the results of the ZDES mode 2 simulation and preliminary results of the ZDES mode 3 simulation are shown. The final paper will provide an extensive cross-comparison between experimental data, ZDES mode 2 and ZDES mode 3 simulations. The issue of the turbulence generation inside the nozzle to feed the ZDES mode 3 will also be tackled, with a focus on the acoustic perturbations associated to the turbulence generation method.

I. Nomenclature

II. Introduction

To improve both the physical understanding of jet noise generation mechanisms and the design of future aircraft engine nozzles, accurate numerical simulations are required to complete wind-tunnel data. The simulation of jet flows involves two major challenges. On the one hand, turbulence production mechanisms need to be accurately reproduced since they are the cornerstone of the jet flow development and noise generation. This leads to the use of eddy-resolving simulations such as LES [START_REF] Bogey | Simulations of Initially Highly Disturbed Jets with Experiment-Like Exit Boundary Layers[END_REF] and hybrid RANS/LES methods which are less expensive and thus applicable to technical configurations at high Reynolds numbers [START_REF] Deck | High-fidelity simulations of unsteady civil aircraft aerodynamics: stakes and perspectives. Application of zonal detached eddy simulation[END_REF]. On the other hand, numerical schemes have to be accurate enough to propagate acoustic waves over large distances with low dissipation and dispersion. This paper is devoted to the first issue at stake, turbulence modelling. In particular, it has been stressed in a number of publications that the resolution of the nozzle boundary layer -in terms of integral quantities and resolved Reynolds stresses -is required to reach the level of agreement with experimental data needed by aircraft makers design teams. Therefore, two approaches are used in the present study. The first one corresponds to a standard RANS/LES simulation with ZDES mode 2 [START_REF] Deck | Recent Improvements of the Zonal Detached Eddy Simulation (ZDES) Formulation[END_REF], where the boundary layer is treated in RANS. Although this rather standard method has proven to be efficient for massively separated flows on complex geometries [START_REF] Deck | High-fidelity simulations of unsteady civil aircraft aerodynamics: stakes and perspectives. Application of zonal detached eddy simulation[END_REF], it fails to capture accurately the jet acoustics due to the RANS/LES transition that occurs near the nozzle exit [START_REF] Gand | Investigation of turbulence development in incompressible jets with zonal detached eddy simulation (ZDES) and synthetic turbulent inflow[END_REF] [START_REF] Gand | Jet noise of a UHBR nozzle using ZDES: external boundary layer thickness and installation effects[END_REF], where the model switches automatically from RANS to LES. To overcome this issue, the second approach investigated in 2 this paper relies on the use of ZDES mode 3 in the nozzle in order to resolve in LES mode the external part of the boundary layer [START_REF] Renard | Improvements in Zonal Detached Eddy Simulation for Wall Modeled Large Eddy Simulation[END_REF]. The mode 3 of ZDES is indeed the WMLES operating mode of ZDES [START_REF] Brès | Large eddy simulation for jet noise: the importance of getting the boundary layer right[END_REF].

The paper is organized as follows. First, the jet flow configuration investigated is described in section III. Then the computational framework is presented in section IV, including the turbulence modelling approach, numerical parameters and grids used. The jet near-field is investigated in section V. Finally, the far-field acoustics predicted by the simulations is investigated, both in terms of directivity and spectral content.

III. Jet flow configuration

The configuration investigated in this paper is an isothermal subsonic jet issued from a round nozzle of exit diameter D= 50 mm. The experiments were performed at the Bruit et Vent jet-noise facility of the Institut Pprime, Poitiers, France. Table 1 summarizes the flow conditions, and details about the experiments and measurements techniques can be found in [START_REF] Brès | Large eddy simulation for jet noise: the importance of getting the boundary layer right[END_REF] and [START_REF] Brès | Importance of the nozzle-exit boundary-layer state in subsonic turbulent jets[END_REF]. Of interest, the experimental data as well as reference numerical results from [START_REF] Brès | Importance of the nozzle-exit boundary-layer state in subsonic turbulent jets[END_REF] have been made available online, which makes this database ideal for the evaluation of numerical simulations. In the final paper, the formulation of ZDES mode 2 and mode 3 will be recalled. Emphasis will also be put on the approach used to initiate turbulent fluctuations in the boundary layer for the ZDES mode 3 simulation, which is a key issue for aeroacoustic simulations. A recent approach published in Ref [START_REF] Deck | A rapid and low noise switch from RANS to WMLES on curvilinear grids with compressible flow solvers[END_REF] is used in the present study. Its setup will be detailed.

External flow

B. Numerical setup

The simulations were performed with the elsA software [START_REF] Cambier | The Onera elsA CFD software: input from research and feedback from industry[END_REF]. Tthe time integration is performed using an implicit LU-SSOR algorithm and a second-order accurate backward Euler Gear scheme. The number of subiterations is adjusted to reach a convergence of one order of magnitude of the inner iteration residuals to achieve second-order time accuracy. For the present simulations, 8 Newton sub-iterations were used. For the spatial integration, the diffusive fluxes are discretized using a second-order-accurate centered scheme. The convective terms are treated with a hybrid centered/upwind second-order-accurate modified AUSM+P scheme [START_REF] Mary | Large Eddy Simulation of flow around an airfoil near stall[END_REF] using a third order MUSCL extrapolation. This version of the AUSM scheme involves a "wiggle" sensor to minimize numerical viscosity by applying some upwinding only in areas where the solution displays numerical oscillations, while the scheme is actually centered everywhere else. In order to compute the far field noise, far-field pressure time signals are reconstructed from the unsteady aerodynamic flow fields stored on surfaces surrounding the jet in the near field, using the Ffowcs-Williams and Hawkings integral formulation [START_REF] Williams | Sound generation by turbulence and surfaces in arbitrary motion[END_REF]. Three surfaces with different radial positions are used to ensure the stability of the noise radiation process (i.e. to check the independence of the results with the location of the surface). Those surfaces are closed at both extremities and radiation is achieved using a formulation with additional flux terms proposed by Rahier et al. [START_REF] Rahier | Additional terms for the use of Ffowcs Williams and Hawkings surface integrals in turbulent flows[END_REF]. This methodology differs from the one traditionally used in jet noise radiation problems, where the surfaces are kept open to avoid spurious noise radiation caused by the turbulence crossing the storage area. With the additional flux terms, spurious radiation from turbulence becomes negligible compared to the physically radiated noise. The advantages of this new formulation are manifold [START_REF] Rahier | Additional terms for the use of Ffowcs Williams and Hawkings surface integrals in turbulent flows[END_REF]. In particular, it improves the stability of the noise computed from the different surfaces, especially in the low frequency part of the spectra. Three different positions of the downstream closing disk are considered in the simulation to ensure radiated pressure signals do not vary with the axial extent of the storage surfaces.

C. Computational domain and grids

Figure 1 illustrates the computational domain size and boundary conditions used. To avoid reflections; the grid is stretched outside the area of interest and the limits of the computational domain are placed far from the jet (40D in the radial direction and 70D in the streamwise direction).

Figure 1. Computational domain

The numerical setup for the ZDES mode 3 simulation involves the zonal definition of the turbulence modelling applied in the nozzle, as depicted in Figure 2. The resolved turbulent content is triggered using the IBC technique following the guidelines suggested in [START_REF] Deck | A rapid and low noise switch from RANS to WMLES on curvilinear grids with compressible flow solvers[END_REF]. The tripping dots included in the nozzle with IBC are shown in Figure 3. The grid sizes were chosen in accordance with the existing literature guidelines and previous experience of the Authors. As shown in Figure 4, the grid stretching in the area of interest is limited to 2% to limit the numerical dissipation. In the azimuthal direction, 260 nodes were used. Overall, the grid contains 52.10 6 nodes for the ZDES mode 2 simulation. The grid was refined only locally inside the nozzle in the streamwise direction in order to reach a resolution of ∆x + =200 as recommended for ZDES mode 3 [6] [7]. Note that the azimuthal resolution has been chosen rather coarse for ZDES mode 3 on purpose (r∆θ + =400) in order to assess the ZDES mode 3 approach on a grid typically made for the jet flow, with a cost equivalent to the ZDES mode 2 simulation. Indeed, the addition of grid points only in the streamwise direction adds 3. 10 6 grid points.

V. Near-field results

In the present extended abstract, only preliminary results are presented and compared with the database. In particular, only instantaneous data are presented for the ZDES mode 3 simulation since it is not completed at the time of the abstract submission.

A. Instantaneous visualizations

Figure 5, Figure 6 and Figure 7 illustrate the simulations investigated. The turbulent structures resolved in the jet appear fairly similar with both approaches, but Figure 6 clearly shows that the some turbulence is resolved inside the boundary layer in the ZDES mode 3 simulation, conversely to the ZDES mode 2 simulation as expected. The nozzle exit boundary layer profiles are plotted in Figure 8 and Figure 9. These quantities are of outmost importance for the prediction of the jet flow development and acoustic sources [START_REF] Brès | Importance of the nozzle-exit boundary-layer state in subsonic turbulent jets[END_REF] [START_REF] Bogey | Simulations of Initially Highly Disturbed Jets with Experiment-Like Exit Boundary Layers[END_REF]. As expected, the ZDES mode 2 acts in RANS mode in the nozzle and therefore is able to recover the mean velocity profile. However, the velocity fluctuations are not present in the ZDES mode 2 simulation. In the final paper, emphasis will be put on the boundary layer characteristics obtained with the ZDES mode 3 simulation. The jet flow development predicted by the ZDES mode 2 simulation is in a fair agreement with the experimental data (see Figure 10 and Figure 11). The velocity fluctuations are slightly underestimated both along the lipline and the centerline, which may be due to a somewhat too coarse grid. A refined grid may be used for the final paper to improve these preliminary results, in particular if no improvement is obtained with the ZDES mode 3 simulation. The RANS to LES transition occurs very quickly downstream of the nozzle exit as depicted in Figure 12. Figure 13 presents the location of the numerical probes used to analyse the spectral distribution of the velocity and pressure fields in the shear layer and in the jet near-field. In Figure 14, the velocity spectra along the shear layer for the ZDES mode 2 simulation further illustrate the RANS to LES process. In the final paper, the comparison with ZDES mode 3 will also to assess the efficiency of this approach in removing this pseudo laminar behaviour induced by a delay in the formation of the instabilities in the mixing layer. Near field pressure spectra plotted in Figure 15 give some preliminary insights on the acoustics predicted by the ZDES mode 2 simulation. The results are in fair agreement with the database.

VI. Far-field acoustic results

As mentioned in section IV, the FWH approach is used to compute the far-field radiated noise. Preliminary results are shown in Figure 16. The sound directivity is in fair agreement with the experimental data, but the sound levels are underestimated by 2 to 4 dB. This is consistent with the velocity fluctuations presented in Figure 11, and will be further analyzed in the final paper.

VII. Conclusion

In the final paper, the comparison of ZDES mode 2 and mode 3 simulations will allow assessing the improvement in the aerodynamic and aeroacoustic results obtained by using WMLES inside the nozzle. If so, some guidelines should be given to carry out similar simulations on different jet cases.

F. Gand 1

 1 and M. Huet 2 DAAA, ONERA, Université Paris Saclay This paper presents hybrid RANS/LES simulations of an isothermal round jet at Mach number M=0.9 and Reynolds number based on the nozzle exit diameter Re = 10 6 . The study is focused on the effect of the RANS to LES transition in the early stages of the jet mixing layer on the properties of the spatial organization of the fluctuating field. In order to capture the salient features of the nozzle internal boundary layer, which is essential to reproduce the jet flow development and acoustics, the Zonal Detached Eddy Simulation mode 3 (i.e. WMLES mode of ZDES) is used to resolve the external part of the boundary layer in LES. The advantage of using this methodology are demonstrated by comparing the results of the simulation with experimental data and another simulation involving ZDES mode 2 (i.e. automatic mode of ZDES) where the whole boundary layer is treated in RANS.

D

  = Nozzle exit section diameter (D = 0.05 m) M = Nozzle exit Mach number (M = 0.9) U j = Jet exhaust velocity

For

  the statistical analysis of the results, aerodynamic simulations were time-averaged over T=0.05 s = 300 D/U j after an initial transient stage of 0.02 s = 140 D/U j . The total cost of one simulation was around 50.10 3 CPU hours. Due to transient parts at the beginning and end of the radiation process, far-field pressure signals are analyzed over a duration of 260 D/U j .

Figure 2 .Figure 3 .

 23 Figure 2. ZDES modes selection for the ZDES mode 3 simulation Figure 3. Visualization of the IBC roughness elements inside the nozzle for the ZDES mode 3 simulation

Figure 4 .

 4 Figure 4. Grid sizes along the lip line and in the jet development area

Figure 5 .Figure 6 .Figure 7 .

 567 Snapshots of the simulations. Iso-Q criterion and gradient of density (a) ZDES mode 2 (b) ZDES mode 3 Snapshots of the simulations. Close-up on the nozzle exit, magnitude of vorticity (a) ZDES2 AUSMPO3 (b) ZDES3 AUSMPO3 Snapshots of the simulations. Cross-jet planes, magnitude of vorticity B. Nozzle exit boundary layer

Figure 8 .Figure 9 .

 89 Figure 8. Nozzle exit velocity profiles at x/D=0.04 Figure 9. Reynolds stresses profiles at the nozzle exit (x/D=0.04)

Figure 10 .Figure 11 .Figure 12 .

 101112 Figure 10. Mean velocity profiles along the centreline and the lipline

Figure 13 .

 13 Figure 13. Location of the numerical sensors

Figure 16 .

 16 Figure 16. Overall sound pressure level directivity on the polar microphone array at 50D from the nozzle exit.experiments ; -ZDES mode 2 ; --Brès et al. simulations[START_REF] Brès | Importance of the nozzle-exit boundary-layer state in subsonic turbulent jets[END_REF]

Table 1 . Aerodynamic conditions of the Pprime jet experiments IV. Computational framework A. Turbulence modelling approach: Zonal Detached Eddy Simulation (ZDES)

 1

	Pressure	p ∞	100 000 Pa
	Temperature	T ∞	293 K
	Mach number	M ∞	0.
	Jet
	Nozzle exit diameter	D	0.05 m
	Total pressure	pt	170 000 Pa
	Total temperature	Tt	337 K
	Mach number	Mj	0.9
	Reynolds number (D)	Re D	1.0 10 6
	Nozzle exit boundary layer
	Thickness	δ	0.004 m

Research scientist, Aerodynamics, aeroelasticity and acoustics department, F-92190 Meudon -France, fabien.gand@onera.fr

Research scientist, Aerodynamics, aeroelasticity and acoustics department, F-92322 Châtillon -France

Acknowledgements

The Authors would like to thank G. Brès (Cascade Technolgies, Inc., Palo Alto, USA) and P. Jordan (Institut Pprime, Poitiers, France) for providing the experimental and numerical database before the online publication of this material.


Juan-Francisco Espinosa-Parrilla

Christelle Baunez

Paul Apicella
email: paul.apicella@univ-amu.fr

Modulation of neuronal activity by reward identity in the monkey subthalamic nucleus

Keywords: basal ganglia, prediction, choice behavior, primate

The subthalamic nucleus (STN) has been argued to be an important component of rewardsensitive basal ganglia circuitry. This view is especially supported by the behavioral changes observed after STN inactivation which could reflect impairments in the motivational control of action. However, it is still unclear how the STN integrates reward information and to what extent such integration correlates with behavior. In this study, we investigated the response properties of STN neurons in monkeys performing reaching movements with a cue predicting the identity of an upcoming liquid reward (juice or water). Although the timing of movements reliably indicated that monkeys had greater motivation for juice than water, rarely did taskrelated changes in neuronal activity depend on the nature of the expected reward. Conversely, when presented with a choice of selecting a response that leads to juice or water delivery, animals showed a clear preference for juice and more than half of the neurons were differentially modulated dependent on the reward obtained, mostly after the monkeys's overt choice of action. Under such circumstances, an increase in activity specifically followed the action outcomes across the population of neurons when monkeys failed to choose the juice reward. These results indicate that STN neurons encode whether or not a preferred reward had been received when a choice between response alternatives is required. This differential neuronal activity might reflect the participation of the STN in evaluating the reward value of chosen actions thus highlighting its contribution to decision-making processes.

Introduction

The subthalamic nucleus (STN), long known to be a key basal ganglia structure involved in motor control, is also considered as influencing cognitive and motivational aspects of behavior. The role of this nucleus in the motivational control of behavior is suggested by its afferent projections from limbic cortical areas, primarily the orbitofrontal and cingulate cortices thought to be critical for reward processing [START_REF] Takada | Organization of inputs from cingulate motor areas to basal ganglia in macaque monkey[END_REF][START_REF] Haynes | The organization of prefrontal-subthalamic inputs in primates provides an anatomical substrate for both functional specificity and integration: implications for Basal Ganglia models and deep brain stimulation[END_REF]. Lesion studies in behaving rats have indicated that disruption of the STN interfere with motivational processes [START_REF] Baunez | Enhanced food-related motivation after bilateral lesions of the subthalamic nucleus[END_REF] possibly contributing to impaired response selection and impulsivity [START_REF] Baunez | Bilateral lesions of the subthalamic nucleus induce multiple deficits in an attentional task in rats[END_REF]. Clinical studies assessing the nonmotor functions of parkinsonian patients who underwent STN deep brain stimulation have reported premature and impulsive response selection, particularly when faced with difficult decisions among competing motor commands [START_REF] Aron | Triangulating a cognitive control network using diffusion-weighted magnetic resonance imaging (MRI) and functional MRI[END_REF]Cavanagh et al., 2011;Coulthard et al., 2012). Additional evidence is provided by neuroimaging experiments in healthy participants reporting that STN activation is associated with slowed response times under decision conflict [START_REF] Aron | Triangulating a cognitive control network using diffusion-weighted magnetic resonance imaging (MRI) and functional MRI[END_REF]. Electrophysiological studies in parkinsonian patients have also shown that enhanced control over decisions and actions is associated with changes in oscillatory activity in local field potentials (LFPs) recorded in the STN (Cavanagh et al., 2011;[START_REF] Brittain | A role for the subthalamic nucleus in response inhibition during conflict[END_REF]Zavala et al., 2014). Recent electrophysiological studies have emphasized the contribution of STN to motivational and emotional processes [START_REF] Huebl | Oscillatory subthalamic nucleus activity is modulated by dopamine during emotional processing in Parkinson's disease[END_REF]Sieger et al., 2015) and a few studies have pointed out specific changes in STN LFP oscillations in parkinsonian patients during reward-based decisions [START_REF] Rosa | Pathological gambling in Parkinson's disease: subthalamic oscillations during economics decisions[END_REF][START_REF] Fumagalli | Subthalamic involvement in monetary reward and its dysfunction in parkinsonian gamblers[END_REF]. It has further been shown that STN stimulation in these patients may lead to mood disturbances, such as depression and hypomania [START_REF] Appleby | Psychiatric and neuropsychiatric adverse events associated with deep brain stimulation: A meta-analysis of ten years' experience[END_REF]Péron et al., 2013), possibly reflecting impaired reward processing.

More direct evidence for the involvement of the STN in motivational processes comes from recordings of individual neurons in behaving animals. In particular, studies in rats have reported differential modulation of STN activity related to changes in the value of the expected reward, including sweetened liquids and drugs of abuse [START_REF] Lardeux | Beyond the reward pathway: coding reward magnitude and error in the rat subthalamic nucleus[END_REF]2013). In addition, neurons in the monkey STN are sensitive to reward prediction and reception [START_REF] Matsumura | Visual and oculomotor functions of monkey subthalamic nucleus[END_REF][START_REF] Darbaky | Reward-related neuronal activity in the subthalamic nucleus of the monkey[END_REF][START_REF] Espinosa-Parrilla | Linking reward processing to behavioral output: motor and motivational integration in the primate subthalamic nucleus[END_REF]. These studies, however, did not examine to what extent neuronal activity in the primate STN can be differentially modulated according to specific reward outcomes. Furthermore, it is unknown as to how STN neurons respond when animals are making decisions between differently valued rewards.

In the present study, we recorded single-neuron activity in the STN of monkeys performing a reaching task to investigate how rewards that were preferred by the animals, as compared with less preferred rewards, influence neuronal activity in the presence and absence of a choice of selecting a response that results in a specific reward. We found that changes in STN activity could provide signals useful for selecting actions based on motivational outcomes in choice context.

Materials and Methods

Behavioral procedures

Experiments were conducted in two adult male Macaca fascicularis monkeys, P and G, in compliance with the National Institutes of Health's Guide for the Care and Use of Laboratory Animals and approved by the Comité d'éthique en Neurosciences . The setup for behavioral testing was similar to that described in [START_REF] Deffains | Modulation of neuronal activity in the monkey putamen associated with changes in the habitual order of sequential movements[END_REF]. Monkeys were seated in a restraining box and faced a panel containing two contactsensitive metal knobs (10x10 mm) positioned 10 cm apart (right, left), at the animal's eye level, and two light-emitting diodes (three-color LEDs red, green, and yellow), one above each knob. A resting bar was mounted in the lower part of the panel at waist level. Two tubes ending in a single spout positioned directly in front of the monkey's mouth dispensed small amounts of apple juice or water (0.3 ml) as a reward, each tube delivering one type of liquid.

The monkeys received liquids during the experimental sessions and had unlimited access to water in their home cage for at least one day each week.

Animals performed a target reaching task in which a visual cue predicted the type of reward obtained in case of correct response. The trial structure is illustrated in Figure 1A.

Each trial began with the animal keeping its hand on the bar. One of the two LEDs was lit for 0.5 s as an instruction cue providing information about the forthcoming reward: a green light corresponded to juice, whereas a yellow (monkey P) or a red light (monkey G) announced water, their left and right locations alternating pseudorandomly between trials. After a fixed delay of 1 s following cue offset, the same LED was lit with a red (monkey P) or yellow color (monkey G), as a triggering stimulus. In response to this stimulus, the animal released the bar, contacted the knob below the illuminated LED and received the liquid reward indicated by the cue. This cue also informed the monkey about the spatial location of the upcoming trigger stimulus and prepared the animal for directing its movement at a specific target. This condition, which is the standard version of the task, allowed us to investigate how the type of reward expected may influence neuronal activity during the preparation, initiation, and execution of the reaching movement. During each trial, the trigger stimulus remained on until the target was contacted or until an upper limit of 1 s was reached. When monkeys had contacted the target and received the liquid reward, either juice or water, they moved immediately their hand back to the bar in preparation for the next trial which could start only if the total duration of the current trial (5 s) had elapsed. Trials in which the monkey released the bar before trigger presentation were aborted, whereas trials in which it failed to release the bar and contact a target in less than 1 s were considered as incorrect and were not rewarded. If the animal failed to respond to the trigger stimulus by reaching the correct target, the same trial was repeated until a rewarded movement is successfully completed, to ensure that the monkeys made movements irrespective of reward preferences.

We also used another version of the task in which monkeys were presented with a choice of selecting the target of the reaching movement using their reward preferences. On each trial, two different color instruction cues for the two liquid rewards appeared simultaneously (same relationships between colors and rewards and same temporal structure of events as in the standard task) their left and right locations alternating semirandomly. After a delay of 1 s following cue offset, two trigger stimuli were presented at the same two locations and the monkeys had to initiate a movement toward one of the two targets. This condition allowed the monkey to choose its reward by making a movement to a specific target according to the color of the preceding cue. Their overt choices of action provided an operational measure of the monkeys' preference for one or the other type of reward. The number of trials in a block for the standard and choice conditions was 40-60 trials. We trained each animal during 3-4 months prior to neuronal recordings. At this stage, they achieved a consistent correct performance rate of >90% in both conditions of the task described above.

Surgery

Monkeys were implanted with a recording chamber and a head-restraining device under general anesthesia maintained with isoflurane (2.5%). An opening was made in the skull over one hemisphere and a stainless steel recording chamber (25 mm OD) was positioned over the hole, its center being aimed at the anterior commissure, approximately 5 mm anterior to the rostral pole of the STN. The recording chamber was filled with an antibiotic solution and sealed with a removable cap. Following surgery, monkeys received antibiotics (Ampicillin, Bristol-Myer Squibb, Paris, France, 17 mg/kg every 12 h) and analgesics (Tolfedine®, Vetoquinol, Lure, France, 2 mg/kg) for a period of 5 days.

Neuronal recordings

During recording sessions, the monkey's head was mechanically immobilized. Single-neuron recordings were performed using custom-made glass-coated tungsten electrodes. The electrode was passed inside a stainless steel guide tube (0.6 mm OD) and was lowered to a position just dorsal to the STN with a manual hydraulic microdrive (MO-95, Narishige).

Neuronal activity was amplified (x5000) and bandpass filtered (0.3-1.5 kHz), the discrimination of individual neurons being performed on-line with a window discriminator (Neurolog, Digitimer, Hertfordshire, UK). Continuous monitoring of the spike waveform on a digital oscilloscope allowed us to check the isolation quality of the recorded neurons. A computer controlled the behavioral task and data acquisition using a custom-made software.

The task relationships of neuronal discharges were assessed on-line in the forms of rasters and perievent time histograms aligned on trigger onset, bar release, and target contact.

Before recording in the STN, we identified the mediolateral extent of the putamen and adjacent pallidal segments accessible through our recording chamber. Electrophysiological mapping of the postcommissural putamen for the purposes of another study [START_REF] Deffains | Modulation of neuronal activity in the monkey putamen associated with changes in the habitual order of sequential movements[END_REF] was very useful in targeting and defining the boundaries of the STN. Parallel electrode tracks were made vertically and neurons in the STN were identified on the basis of previously described electrophysiological characteristics in primates [START_REF] Matsumura | Visual and oculomotor functions of monkey subthalamic nucleus[END_REF][START_REF] Wichmann | The primate subthalamic nucleus. I. Functional properties in intact animals[END_REF][START_REF] Darbaky | Reward-related neuronal activity in the subthalamic nucleus of the monkey[END_REF][START_REF] Isoda | Role for subthalamic nucleus neurons in switching from automatic to controlled eye movement[END_REF][START_REF] Espinosa-Parrilla | Linking reward processing to behavioral output: motor and motivational integration in the primate subthalamic nucleus[END_REF]. In their downward trajectory, electrodes passed through the thalamus, zona incerta, STN, and finally into the substantia nigra pars reticulata, the transition between these structures being obvious because of different baseline activity of neurons. During the recording of any neuron, the activity was sampled during a block of trials using the standard condition. If the isolation could be sustained for a sufficient period of time, the tests were continued in the choice condition. During some recording sessions, we monitored the licking movements of the monkeys using force transducers (strain gauges) attached to the tubes delivering liquids.

Signals from the strain gauge device were digitized at 100 Hz and stored into an analog file.

Data analysis

Performance in task conditions was assessed by measuring the time between the onset of the trigger stimulus and bar release (reaction time, RT) and the time between bar release and target contact (movement time, MT). Trials with excessively short RTs (<100 ms) were excluded from analysis. The RT and MT data were compared using ANOVAs with reward types and target locations as factors. Quantitative analysis of the monkeys' tongue or lips contacts with the spout was made off-line by single-trial analysis, and involved durations of anticipatory licking movements measured from cue onset to trigger onset, latencies of licking movements relative to the presentation of the trigger stimulus, and numbers of licks during a period of 1 s starting at 500 ms after the delivery of liquid. Distributions of duration, latency and number of licking movements in juice and water trials were compared with the one-tailed Wilcoxon test.

The first step of neuronal data analysis involved identifying significant modulations of activity in individual neurons by using a previously established procedure based on a sliding time window analysis. This procedure employed the Wilcoxon signed-rank test (P < 0.05) between a 0.5-s control period immediately preceding cue onset and a 100-ms test window that was moved in steps of 10 ms starting at the onset of the cue, the onset of movement, and the delivery of reward. The times of onset and offset of task-related changes in activity were determined for each neuron. The latency of a significant change in neuronal activity was defined as the beginning of the first of 20 consecutive steps showing a significant difference as against the baseline activity during the control period. Response offset was determined in the same way by searching for a loss of statistically significant differences during 20 steps.

After extensive testing with our sliding time window procedure using different numbers of time steps, we have chosen a threshold of 20 steps because recordings of individual neurons of the STN often showed great variability from trial to trial, making it necessary to exclude detection of transient changes that are not consistent with changes in firing rate visible in the rasters and histograms of individual neurons. The second step of neuronal data analysis assessed the sensitivity of each neuron to the type of reward separately in three task periods: the cue-trigger delay period, the movement period, and the reward period. We did this by computing the spike counts of neurons in 300 ms non-overlapping windows across each task periods, spanning from 100 to 1300 ms after the cue onset (4 successive 300-ms windows), between -200 and 400 ms from movement onset (2 successive 300-ms windows), and from 100 to 1300 ms after the delivery of reward (4 successive 300-ms windows). This led us to analyse neuronal activity in ten time windows. In the third step of neuronal data analysis, changes in neuronal activity were also examined at the level of population average. This analysis was performed in 10 ms bins to identify when the population significantly changed its activity, relative to the control period of 0.5 s immediately preceding cue onset. Because variability in firing rate was less pronounced over the population of neurons, we defined the onset time of a change as the first of 3 consecutive bins (30 ms) for which a significant difference was detected (paired t test, P < 0.05). Differences in proportions of task-related neurons between task periods or task conditions were statistically assessed by using the 2test. All statistical comparisons were conducted with JMP10 (SAS Institute, Cary, NC).

Histology

Recording sites were histologically verified in one animal (monkey P), using small electrolytic lesion marks in the putamen and around the STN as reference points. After the experiments had been completed, this animal was killed with an overdose of pentobarbital and perfused transcardially with 0.9% saline followed by a fixative (4% paraformaldehyde, pH 7.4 phosphate buffer). The brain was cut in 50-m coronal sections, mounted on slides, and stained with cresyl violet. As histological reconstruction was not available for the second animal, we determined its approximated recording sites from the neuronal activity characteristic to the STN and neighboring structures, particularly the thalamus, zona incerta, and substantia nigra pars reticulata. Although the number of proposed divisions of the primate STN is still being debated [START_REF] Keuken | Are there three subdivisions in the primate subthalamic nucleus[END_REF][START_REF] Alkemade | Do we need to revise the tripartite subdivision hypothesis of the human subthalamic nucleus (STN)?[END_REF], they include at least two parts consisting of dorsolateral and ventromedial regions regarded as sensorimotor and associative/limbic divisions of the STN, respectively.

Results

Behavioral data

Both monkeys responded correctly in more than 95% of the trials, i.e., they touched the correct target within the required time whatever the type of the reward obtained (juice or water). Figure 1B shows the mean RTs and MTs for the two outcomes. Two-way ANOVA (reward type x target location) on the RT data revealed a significant main effect of the expected reward type in monkey G (F(1, 408) = 10.31, P < 0.01) and in monkey P (F(1, 542) = 22.76, P < 0.01) with longer RTs in water trials than in juice trials. In contrast, there was no significant influence of the reward type on MT in monkey G (F(1, 408) = 0.16, P > 0.05) and in monkey P (F(1, 542) = 0.005, P > 0.05), suggesting that the execution phase of movement was less sensitive to the specific identity of the expected reward compared to movement initiation phase. As noted in previous studies (Ravel et al., 2006;[START_REF] Deffains | Modulation of neuronal activity in the monkey putamen associated with changes in the habitual order of sequential movements[END_REF], our monkeys were faster to make movements toward targets located on the same side of the moving arm, this spatial location effect being evidenced in both juice and water trials (data not shown). When monkeys were permitted to freely choose between the two reward options, they selected the target associated with juice over the one associated with water in 77% (monkey G) and 75% (monkey P) of the trials, irrespective of left or right locations. This choice behavior indicated that the monkeys discriminated the type of reward predicted by the cues and that the juice had greater appetitive value for them. As also shown in Figure 1B, a lengthening of RT on water trials as compared with juice trials was seen in the choice task in monkey G (F(1, 302) = 8.49, P < 0.01) and in monkey P (F(1, 915) = 16.53, P < 0.01), suggesting a lower level of motivation when the chosen option led to the less-preferred reward. In contrast, monkey G had significantly shorter MT in water trials than in juice trials (F(1, 302) = 13.97, P < 0.01), whereas MT lacked significant changes between reward types in monkey P F(1, 915) = 2.40, P > 0.05)). An effect of the location of the trigger stimulus was still present in the choice task, monkeys being longer to react to the trigger stimulus presented contralaterally to the moving arm than when it was presented ipsilaterally (data not shown).

As a further measure of the effect of reward type on behavior, we looked for variations in mouth movement patterns during the task that would be consistent with the idea that the monkeys were differentially motivated to make movements depending on the liquid available.

For unforeseen reasons, the data collected in monkey G could not be analyzed because of noise in the signal from the strain gauge device and the results only concern monkey P. As illustrated in Figure 2A, the overall pattern of licking movements during the delay period did not show clear differences between juice and water trials. The monkey started licking the tube on or slightly before the presentation of the cue and these movements often stopped in the last 500 ms preceding the presentation of the trigger stimulus. Thereafter, the monkey elicited brief licking reactions immediately after trigger onset. The only noticeable difference in orofacial activity between juice and water trials was observed later in the task, after the delivery of reward, juice eliciting more intense and more persistent licking movements compared with water. Durations of anticipatory licks (Fig. 2B, left) and latencies of lick responses to the trigger stimulus (Fig. 2B, center) were comparable for the two liquid rewards, whereas the number of contacts of the monkey's tongue or lips with the spout during a 1 s period starting 500 ms after reward delivery was higher for juice trials than for water trials (Fig. 2B, right). Although not illustrated here, mouth movement patterns remained essentially unchanged when the monkey had the opportunity to choose the reward.

Neuronal data

A total of 27 and 31 penetrations were carried out in the STN area in monkeys P and G, respectively. As emphasized previously [START_REF] Wichmann | The primate subthalamic nucleus. I. Functional properties in intact animals[END_REF][START_REF] Espinosa-Parrilla | Linking reward processing to behavioral output: motor and motivational integration in the primate subthalamic nucleus[END_REF], it was often difficult to maintain a stable isolation of single STN neurons throughout a testing period because of the high cell packing density in this nucleus. For this reason, a number of recording sessions were discarded, resulting in a total of 33 neurons (14 and 19 in monkeys P and G, respectively) which were thoroughly tested while the monkeys performed the standard task. We also recorded 81 neurons in the choice task and selected for study only 42 neurons (20 and 22 in monkeys P and G, respectively) with a sufficient number of trials to allow comparison between juice and water trials. The electrophysiological characteristics of STN neurons were in accordance with those reported before [START_REF] Matsumura | Visual and oculomotor functions of monkey subthalamic nucleus[END_REF][START_REF] Wichmann | The primate subthalamic nucleus. I. Functional properties in intact animals[END_REF][START_REF] Darbaky | Reward-related neuronal activity in the subthalamic nucleus of the monkey[END_REF][START_REF] Isoda | Role for subthalamic nucleus neurons in switching from automatic to controlled eye movement[END_REF][START_REF] Espinosa-Parrilla | Linking reward processing to behavioral output: motor and motivational integration in the primate subthalamic nucleus[END_REF]. The quantification of firing rates and patterns was based on the neurons recorded in the standard condition (n=33), some of them (n=10) being also recorded in the choice condition, and in the choice condition only (n=32). STN neurons displayed irregular activity patterns with impulses of short duration. Their mean firing rate during the 1-s period prior to the cue onset was 21.7 + 16.7 spikes/s (n=65), which is similar to spontaneous firing rates observed in the primate STN in previous work [START_REF] Matsumura | Visual and oculomotor functions of monkey subthalamic nucleus[END_REF][START_REF] Wichmann | The primate subthalamic nucleus. I. Functional properties in intact animals[END_REF][START_REF] Darbaky | Reward-related neuronal activity in the subthalamic nucleus of the monkey[END_REF][START_REF] Isoda | Role for subthalamic nucleus neurons in switching from automatic to controlled eye movement[END_REF][START_REF] Espinosa-Parrilla | Linking reward processing to behavioral output: motor and motivational integration in the primate subthalamic nucleus[END_REF]. The STN neuronal discharge pattern was also characterized using the mean length of interspike intervals which was 67.7 + 62.6 ms (n=65). We used the interspike interval of each recorded neuron to confirm that it corresponded to a discriminated single neuron, i.e., only those neurons whose the interspike interval histogram displayed a refractory period of at least 3-5 ms were included in the analysis. As detailed later, neurons reported in the present study were recorded over the mediolateral extent of the STN which encompasses most of the two STN subdivisions we will consider here, i.e., the dorsolateral part and the ventromedial part.

We examined temporal profiles of activity changes for each neuron at a high temporal resolution using our sliding time window analysis (see Materials and Methods). This analysis was carried out in three steps, starting at the onset of the cue, the movement onset, and the delivery of reward, both juice and water trials being pooled. As shown in Figure 3A, the group of 33 neurons recorded in the standard task was heterogeneous in terms of the timing of changes in activity during the delay, movement, and reward periods of the task, and the direction of these changes (i.e., increased or decreased in firing rates). We found that 67% (22/33) of neurons were modulated during the three task periods and 24% (8/33) during two task periods, modulations occurring during only one task period being quite rare (3/33).

The proportion of neurons displaying increases or decreases in activity varied across the different task periods (Figure 3B). About half (18/33) of the STN neurons exhibited unidirectional changes in activity during the course of the trial (increases: n = 16 : decreases: n = 2), whereas the other half (15/33) displayed complex modulations that combined increases and decreases. Overall, neurons showing an increase in activity were more frequent in the delay period, as compared to the movement period (χ2 = 6.11, df = 1, p < 0.05), whereas the fractions of increases were not significantly different (P > 0.05) when comparing the delay and movement periods with the reward period. Also, the proportion of neurons showing a decrease in activity was significantly higher in the movement (χ2 = 7.75, df = 1, p < 0.01) and reward (χ2 = 4.24, df = 1, p < 0.05) periods, as compared to the delay period, decreases occurring with similar frequency in the delay and reward periods.

Figure 4 illustrates the variety of changes in STN neuronal activity with respect to task events. Neuron A was activated during the delay period after the presentation of the cue and this activation was maintained until the movement was initiated. The same neuron exhibited an additional slight increase in activity after the delivery of reward. Neuron B decreased its activity immediately after the onset of the cue and its activity increased gradually until movement onset. For this neuron, reward delivery was followed by a strong increase in activity. No changes in activity were observed during the delay period in neuron C, but its activity declined abruptly during the movement period and was followed by an extended phase of enhanced activity after reward delivery. Thus, all three neurons showed an increase in activity after the delivery of reward. Importantly, in our sample of tested neurons, we never observed clear temporal relationship between particular patterns of neuronal activity and orofacial movements, although the possibility that some modulations were related to early or late phases of liquid consumption could not be totally eliminated.

Effect of the identity of the expected reward

To examine whether the type of reward expected at trial end could influence the STN activity, we calculated the average spike counts for each neuron within 300 ms non-overlapping windows spanning each task period (see Materials and Methods for the definition of time windows) and we compared activity between juice and water trials in each time window separately. A neuron was considered to be sensitive to reward type if its activity for one liquid was significantly higher than for the other during at least one time window (one-way ANOVA, p < 0.05). We found that the activity of only 8 neurons (24%) varied significantly depending on the reward type (3 and 5 in monkeys P and G, respectively), being higher with either juice (n=5) or water reward (n=3). Figure 5A shows how the sensitivity to the reward type evolves over the course of a trial across the entire population of 33 neurons tested in the standard task. Plotted are every neuron's activity for every time window across the delay, movement, and reward periods of the task. Four of these neurons displayed reward sensitivity at separate periods of the task (delay-reward, n=3; delay-movement-reward, n=1) and these modulations were always in the same direction (i.e., increase or decrease) for a given neuron.

Overall, significant reward-discriminative activity was found in six, one, and six neurons during the delay, movement, and reward periods, respectively, We also examined whether the direction of the movement associated with target reaching influenced the STN neuronal activity in the same time windows defined above. Eleven neurons (4 and 7 in monkeys P and G, respectively) also showed stronger activity associated with a preferred location in at least one time window (data not shown).

Next, we investigated the impact of the outcome on STN neuronal activity when monkeys made choices between two reaching targets using their reward preferences. Because the monkeys chose the option with the juice most frequently than the other, the analysis was restricted to recording sessions in which there were at least five trials in which the animals selected the action leading to water reward. A total of 42 neurons were thus included in the analysis. As mentioned in the behavioral results, both monkeys continued to respond faster in juice trials than in water trials, indicating that they differentially expected the likely outcomes of their own choices, being more motivated when choice responses lead to juice rewards.

A statistical analysis using the same 300-ms time windows as previously defined was performed to examine, for each neuron in our sample, whether the activity differed between the juice and water trials in choice context. The results are shown in Figure 5B. Out of the 42 neurons recorded, 26 (62%) showed activity that differed significantly between the two reward types in at least one time window (14 and 12 in monkeys P and G, respectively). The size of this subset of neurons (62% of 42) was greater than that observed in the standard task (24% of 33), in which monkeys did not have a choice between two reward options (χ2 = 12.24, df = 1, p < 0.01) indicating that the proportion of reward-discriminating neurons was greatly affected by the presence or absence of response choice.

Of the 26 reward-discriminating neurons found in the choice task, 9 neurons showed greater modulation on juice trials than on water trials, 8 neurons showed the opposite, and 9 neurons showed greater modulation on juice trials or water trials at different periods of the task. It therefore appears that the sensitivity to reward type varied from neuron to neuron and that neurons displaying a preferential sensitivity for either juice or water occurred with similar frequencies. Overall, significant reward-discriminative activity was found in 6, 11, and 21 neurons during the delay, movement, and reward periods, respectively. Nine of these neurons showed reward effects for more than one task period. No differences were observed in the fraction of neurons sensitive to the reward type in the delay and movement periods (χ2 = 1.84, df = 1, p > 0.05), whereas this fraction was significantly higher in the reward period, as compared to the delay (χ2 = 12.28, df = 1, p < 0.01) and movement periods (χ2 = 5.04, df = 1, p < 0.05), indicating that modulation by reward outcome is most prevalent after completion of the response. We also found that target location and movement direction had a significant effect on the activity of 31 neurons (13 and 18 in monkeys P and G, respectively) tested in the choice condition (data not shown).

Figure 6A represents the average activity of the whole sample of 33 neurons recorded in the standard task, separately for trials in which juice or water was delivered. The two curves were superimposed during the delay period with a rise in activity starting 150-220 ms before the trigger onset which reached a peak at the time of movement initiation. An additional increase in the average population activity occurred after the delivery of reward though to a lesser magnitude compared with that around the initiation of movement. The population level analysis revealed that neither the activity associated with the preparation and execution of the movement nor the activity following the delivery of reward was affected by the specific reward expected at trial end. The similarity of the time course of neuronal changes in the juice and water trials seen at the population level therefore indicates that the STN activity was not markedly modulated by the identity of the expected reward.

Two representative examples of such reward-related activity are shown in Figure 6B.

The upper part of the figure shows the activity of a neuron which was activated equally strongly on juice and water trials during the movement period and the reward period of the task, whereas its firing remained unchanged during the delay period. Conversely, the activity of the neuron shown in the lower part of the figure increased gradually during the delay period and subsided after the onset of the movement and subsequent reward delivery, regardless of the type of reward.

Figure 7A provides a description of neuronal activity averaged over all 42 neurons recorded in the choice condition, separately for juice and water trials. During the delay period, when the choice is still covert, the activity levels of these neurons for juice and water trials were nearly identical. A rise in activity occurred slightly before the onset of the trigger stimulus and peaked in close temporal relation to the onset of movement in juice and water trials. On the other hand, the population level analysis revealed that the curves do not overlap after the reward is delivered, a brief increase in firing being clearly present around 500 ms following the delivery of water. To examine this effect more closely, we quantified firing rates for the ensemble activity of all 42 neurons by analyzing the same four 300-ms windows as previously defined. We then compared the mean activity levels of the population of neurons between the two reward types in each time window (insert in the right upper part of Fig. 7A) and found that the population average activity was enhanced significantly for water trials compared with that for juice trials in the second window (i.e., 400-700 ms after reward delivery, t=4.30, p < 0.01, two-tailed). In other words, although the differential activity measurable at a single-neuron resolution level did not suggest a stronger modulation for a particular reward, the population average activity was selectively enhanced for water trials as compared with that for juice trials, indicating that information about the identity of reward is present at the population level after the monkey has selected a movement that led to the lesspreferred reward. Such an increase in activity may reflect neuronal processes that evaluate the result of a completed movement with respect to its predicted outcome.

Figure 7B shows two representative examples of neuron displaying differential activity level in relation to the type of reward obtained. Both neurons showed stronger firing during the reward period when target associated with water was chosen as compared with juice. The neuron in the upper part of the figure showed a relatively sharp, phasic increase in activity, whereas the neuron in the lower part increased its activity in a sustained manner.

Interestingly, in this latter neuron, the level of activity was also increased through the delay period and until the movement was initiated, regardless of the type of reward delivered.

Recording positions

Reconstructions of recording positions revealed that neurons were sampled throughout the mediolateral extent of the STN (Fig. 8). A functional specialization within the STN, based on the distribution of its cortical input, is still debated (Keuken et al., 2013;[START_REF] Alkemade | Do we need to revise the tripartite subdivision hypothesis of the human subthalamic nucleus (STN)?[END_REF] and no reliable electrophysiological marker could help in defining a clear delineation of the STN's putative subdivisions. Despite this, we attempted to parcel the nucleus into two distinct zones, i.e., a dorsolateral region and a ventromedial region, following the subdivision adopted by other authors [START_REF] Eitan | Asymmetric right/left encoding of emotions in the human subthalamic nucleus[END_REF]Keuken et al., 2013) and which we have also used in an earlier study [START_REF] Espinosa-Parrilla | Linking reward processing to behavioral output: motor and motivational integration in the primate subthalamic nucleus[END_REF]. In particular, we took into account the possibility that the limbic part of the STN may be largely confined to the ventromedial region [START_REF] Haynes | The organization of prefrontal-subthalamic inputs in primates provides an anatomical substrate for both functional specificity and integration: implications for Basal Ganglia models and deep brain stimulation[END_REF] and we wanted to see whether there would be more reward sensitive neurons in this region, compared to the dorsolateral region. Because the sample number of neurons displaying reward-discriminating activity in the standard task was too low, we focused on the 42 neurons recorded in the choice task among which 24 showed differential activity. Neurons were categorized according to their preferential sensitivity to a specific reward, namely juice type (higher activity in the juice trials) and water type (higher activity in the water trials). Among these neurons, 25 were considered to be located within the dorsolateral part and 17 within the ventromedial part. Rewarddiscriminating neurons were found in the dorsolateral (16 of 25 neurons; 64%) and ventromedial STN (10 of 17 neurons; 59%). The frequency of neurons with a sensitivity to the type of reward failed to vary significantly between the two regions (χ2 = 0.73, df = 1, p > 0.05).

Discussion

The present study was designed to specifically examine STN neuronal activity while monkeys performed arm movements toward target stimuli that predicted juice or water rewards. Our behavioral results revealed that animals changed their performance depending on the reward outcome, suggesting that their level of motivation varied according to the kind of liquid available on each trial. When no choice was allowed, we found that the activity of STN neurons was rarely modulated by adjustments of behavior mandated by motivational context.

Conversely, when monkeys were presented with a choice of selecting a response that results in a specific reward, a greater proportion of the neurons was sensitive to the reward type and an increase in STN activity occurred at the outcome phase across the population of neurons when the less-preferred reward was chosen. Our results provide evidence that the STN carries information about the identity of reward when monkeys must select particular actions based on the expected reward outcome. This work therefore extends previous descriptions of reward-related activities in the STN by highlighting the sensitivity to the value of the outcome in choice context.

Relationship between STN neuronal activity and expected reward values

In the standard condition, the animals consistently responded faster in juice trials compared with water trials, indicating that they discriminated between reward-predictive cues and that the expected reward affects their motivation to elicit a movement. This is consistent with previous reports in monkeys showing that the latency of instrumental responses changed depending on action outcomes, RT being shortened when a more valued reward is expected (Hollerman et al., 1998;[START_REF] Hassani | Influence of expectation of different rewards on behavior-related neuronal activity in the striatum[END_REF][START_REF] Watanabe | Behavioral reactions reflecting differential expectations of outcomes in monkeys[END_REF]Cromwell & Schultz, 2003;Minamimoto et al., 2005). On the other hand, in the present study, licking movements preceding reward delivery failed to vary systematically between rewards, even if the monkey showed a clear preference for one liquid over the other. Previous studies have shown that anticipatory licking is generally more vigorous for the preferred reward, thus serving as a behavioral measure of the animals' valuation of upcoming rewards [START_REF] Hassani | Influence of expectation of different rewards on behavior-related neuronal activity in the striatum[END_REF][START_REF] Watanabe | Behavioral reactions reflecting differential expectations of outcomes in monkeys[END_REF][START_REF] Fiorillo | The temporal precision of reward prediction in dopamine neurons[END_REF]. In our experiment, the only noticeable difference in orofacial behavior was restricted to the late part of the trial in which licking movements were longer for the preferred reward. However, because our analysis concerns just one monkey, it cannot be excluded that the orofacial characteristics are peculiar to this animal.

Across task periods, very few STN neurons were differentially modulated by the type of reward expected at trial end. Indeed, the level of STN activity remained very similar, even though the latency of movements toward targets that predicted juice or water was different.

Because the information about the expected outcome is seldom incorporated into task-related STN activities, these neurons did not have the properties one would expect of neurons sending signals involved in guiding actions based on value representations. In this regard, the present results may appear to differ from those of previous single-neuron recording studies in rats which stressed the importance of STN in encoding reward information. In these studies, STN neuronal activity is often modulated by the identity of the expected reward, whether it is an appetitive liquid or a drug of abuse, and reward influences are present during various task periods, including detection of conditioned incentive cues, preparation and execution of motor reactions leading to the reward [START_REF] Lardeux | Beyond the reward pathway: coding reward magnitude and error in the rat subthalamic nucleus[END_REF]2013). This is clearly not the case in our experiments in which we saw little change in STN neuronal activity depending on the nature of the expected reward. This apparent discrepancy may be caused by differences in task design, including different time intervals between task events and different behavioral sequences aimed at obtaining reward. Another notable difference is that the impact of reward preferences on rats' performance is evident in the executive part of the task, whereas it occurs before onset of the instrumental response in our experiment, limiting further our ability to make a direct comparison of neuronal sensitivity to expected rewards in tasks used with rats and monkeys. Signals related to the expected reward outcome have previously been identified in monkey single-neuron recordings of STN [START_REF] Matsumura | Visual and oculomotor functions of monkey subthalamic nucleus[END_REF][START_REF] Darbaky | Reward-related neuronal activity in the subthalamic nucleus of the monkey[END_REF][START_REF] Espinosa-Parrilla | Linking reward processing to behavioral output: motor and motivational integration in the primate subthalamic nucleus[END_REF] and we have reported that some neurons can be modulated by the monkey's valuation of reward which decayed as the time to its delivery was delayed [START_REF] Espinosa-Parrilla | Linking reward processing to behavioral output: motor and motivational integration in the primate subthalamic nucleus[END_REF]. It is therefore possible that STN neurons may encode the value of upcoming rewards under specific reinforcement contexts, as demonstrated in rodent studies [START_REF] Lardeux | Beyond the reward pathway: coding reward magnitude and error in the rat subthalamic nucleus[END_REF]2013). Indeed, our findings show that a substantial number of neurons in the STN distinguished between two reward outcomes when monkeys had the opportunity to choose a particular reward, as discussed below.

Influence of a choice between reward outcomes

Changing the conditions of the task from one in which monkeys were simply assigned a single option leading to a specific reward to another in which they had to choose betwen two reward options markedly enhanced the STN sensitivity to the expected reward. Indeed, more than half of the presently tested neurons (62% of 42 neurons) showed differences in taskrelated modulations between the two liquid rewards when the outcome was determined by the animal's own choice, whereas modulations reflecting the reward type were rare outside of a choice context (24% of 33 neurons). In addition, modulations by reward type were most frequently observed after the monkey had reached a target, suggesting that information about the value of the outcome is not reflected in STN activity before an action is selected and executed. It therefore appears unlikely that discriminatory activities in the STN represent meaningful signals that potentially have an influence on the choice about which movement to make. Furthermore, among neurons that displayed reward-discriminating properties, some neurons showed higher task-related activities following water reward as compared with juice reward, whereas others showed opposite changes, both types of modulations being roughly equal in number. Thus, in our experiment, STN neuronal activity did not seem to encode more strongly one reward or the other.

Enhanced reward sensitivity under choice can be explained by the fact that our monkeys presumably attended better on choice trials, compared with standard trials. This potential explanation lies in our behavioral findings: animals did not take longer to decide where to respond in the presence of a choice between two reward options compared to the case with a single option, which could result from an increase in attentional resources needed in a non-routine situation that requires decision making and more controlled processing. It is also possible that a general increase in motivation to perform the task may influence STN activity when animals could deliberately choose the upcoming reward. Another explanation for the enhanced reward sensitivity of STN neurons we observed might be that the choice condition involves some degree of conflict between two positive outcomes relative to the standard condition involving a single outcome. This idea is in line with an influential theory that considers that the STN plays a prominent role in response selection under conditions of conflict between possible action choices (Frank, 2006;Bogacz & Gurney, 2007). Although monkeys could use reward-predictive cues to guide their actions based on their reward preferences, in some instances they chose the target associated with the less-preferred reward.

Obviously, these choice responses cannot be considered as errors in motor execution because the monkeys' movement latencies indicate that animals apparently knew they were responding for the less-preferred reward, i.e., they responded slower in water trials compared with juice trials. It is therefore possible that the value of reward-predictive cues were in conflict, pushing monkeys toward choosing the less desirable option in some instances. This kind of inconsistency in monkeys' choice behavior has been previously interpreted as reflecting an interference with the process of action selection [START_REF] Bromberg-Martin | Lateral habenula neurons signal errors in the prediction of reward information[END_REF]. In keeping with this view, enhanced reward sensitivity of STN neurons might be a consequence of a presumed conflict caused by the small differences in motivational value between two appetitive liquids. A growing number of electrophysiological studies in parkinsonian patients provides evidence that the STN is involved in decision-making under conflict (Cavanagh et al., 2011;[START_REF] Brittain | A role for the subthalamic nucleus in response inhibition during conflict[END_REF][START_REF] Zaghloul | Neuronal activity in the human subthalamic nucleus encodes decision conflict during action selection[END_REF]Zavala et al., 2014). It has been reported that some neurons in the monkey STN are preferentially activated when the animal has to suppress automatic responses and replace them with more controlled responses [START_REF] Isoda | Role for subthalamic nucleus neurons in switching from automatic to controlled eye movement[END_REF]. Neuroimaging work in humans also indicates that STN activation is associated with slowed response times when faced with difficult choices among competing motor actions [START_REF] Aron | Triangulating a cognitive control network using diffusion-weighted magnetic resonance imaging (MRI) and functional MRI[END_REF]. Further work is needed to determine the extent to which a competition between response alternatives might reflect the observed changes in STN activity dependent on the value of the expected reward.

There is evidence from human studies pointing to the contribution of the ventromedial part of the STN to the processing of emotional information, and the dorsolateral part to motor functions (Mallet et al., 2007;[START_REF] Greenhouse | Stimulation at dorsal and ventral electrode contacts targeted at the subthalamic nucleus has different effects on motor and emotion functions in Parkinson's disease[END_REF]Buot et al., 2013;[START_REF] Eitan | Asymmetric right/left encoding of emotions in the human subthalamic nucleus[END_REF].

Although neurons related to body and eye movements have been described in the dorsolateral and ventromedial parts of the primate STN, respectively [START_REF] Delong | Primate globus pallidus and subthalamic nucleus: functional organization[END_REF][START_REF] Matsumura | Visual and oculomotor functions of monkey subthalamic nucleus[END_REF], anatomical studies have also shown that some degree of convergence exists between projections from distinct cortical areas [START_REF] Takada | Organization of inputs from cingulate motor areas to basal ganglia in macaque monkey[END_REF][START_REF] Haynes | The organization of prefrontal-subthalamic inputs in primates provides an anatomical substrate for both functional specificity and integration: implications for Basal Ganglia models and deep brain stimulation[END_REF], suggesting that there is at least partial overlap between putative functional areas within the STN. In the present study, we did not find any difference in the distribution of rewarddiscriminating neurons over the extent of the STN sampled, these neurons being scattered in both the dorsolateral and ventromedial subdivisions of the nucleus. Although the small number of recorded neurons did not allow a definite statement, our results do not seem to lend support to a functional clustering of neuronal populations in specific parts of the STN.

Are STN neurons able to encode the outcome of the chosen action ?

Another main finding obtained in the choice task was that once the movement was completed, a brief increase in STN firing specifically emerged at the population level when monkeys chose the less-preferred reward. Remarkably, this signal was only evident when monkeys are presented with a choice between different rewards and after the behavioral manifestation of this choice, raising the possibility that the STN might evaluate the consequences of chosen actions. It is tempting to suggest that this increased activity may reflect a negative affective state arising from a failure to obtain the preferred reward, but this seems unlikely given that our monkeys apparently noticed the type of reward associated with their choices. It is also conceivable that the observed outcome-related increase in activity simply reflects differences in the taste of specific liquids or movements to consume them. However, the same liquids eliciting the same mouth movement patterns were used in both choice and standard trials thus arguing that the differential modulation is not exclusively due to a physical quality of outcomes or associated consummatory movements, but rather reflects an internal process related to the evaluation of the consequence of the chosen action.

An important feature of our behavioral situation was the opportunity for choosing between response alternatives leading to different outcomes. However, the effect of the outcome of choice responses on STN activity was quite limited, albeit significant, which raises the question of the adequacy of task features for the expression of the differential STN activity. In this regard, one might argue that a decision in which one option has positive value and another has negative value would elicit a stronger differential modulation of STN neuronal activity relative to a decision in which both options have relatively similar positive values (e.g., juice and water). Further experimental work is required to examine the influence of various outcome values on the capacity of STN neurons to encode motivational information and reward.

Although it is still unclear how a signal related to trial outcome is used to affect the behavior, it could be speculated that it might enable the STN to modulate the preparation and execution of subsequent actions when monkeys occasionally choose options that are not in agreement with their reward preference. If so, disruption of STN function should interfere with performance adjustments driven by the consequences of actions, leading to premature and impulsive reactions as observed in rodents [START_REF] Baunez | Bilateral lesions of the subthalamic nucleus induce multiple deficits in an attentional task in rats[END_REF][START_REF] Eagle | Is there an inhibitory-response-control system in the rat? Evidence from anatomical and pharmacological studies of behavioral inhibition[END_REF]) and parkinsonian patients with STN DBS (Hälbig et al., 2009).

One brain region that should be considered with regard to trial outcome is the anterior cingulate cortex (ACC), which sends projections to the STN via the hyperdirect pathway [START_REF] Takada | Organization of inputs from cingulate motor areas to basal ganglia in macaque monkey[END_REF][START_REF] Haynes | The organization of prefrontal-subthalamic inputs in primates provides an anatomical substrate for both functional specificity and integration: implications for Basal Ganglia models and deep brain stimulation[END_REF] and has a well established role in monitoring action outcomes and using this information to adjust behavior [START_REF] Paus | Primate anterior cingulate cortex: where motor control, drive and cognition interface[END_REF][START_REF] Bush | Dorsal anterior cingulate cortex: a role in reward-based decision making[END_REF][START_REF] Rushworth | Action sets and decisions in the medial frontal cortex[END_REF][START_REF] Shenhav | The expected value of control: an integrative theory of anterior cingulate cortex function[END_REF]. Neurophysiological studies in humans provide evidence that coupling of neuronal activity between the midline frontal cortex and the STN may occur under conditions of conflict and response inhibition (Cavanagh et al., 2011;Zavala et al., 2014). One possibility is that ACC may convey information about trial outcome to STN, particularly the ventromedial zone which is the major recipient of afferent information from ACC. However, as mentioned above, we did not observe any obvious clustering of reward-discriminating neurons in a particular region of the STN. In this regard, the possibility that increased activity in STN is driven by specific frontal inputs after inappropriate action selection remains speculative and requires additional testing.

Conclusion

In summary, we have extended the notion of reward processing by the STN. Our results show that individual neurons in this nucleus are able to discriminate between different rewards when actions are initiated in the presence of a choice between response options based on expected reward outcomes. This differential activity might reflect the participation of STN neurons in evaluating the outcomes associated with chosen actions. This study therefore emphasizes the importance of future investigations into the mechanisms underlying the contributions of STN to reward-guided decision making. nucleus oscillations during conflict. J. Neurosci., 34, 7322-7333. A, At the start of each trial, one (standard task) or two (choice task) visual stimuli were presented while the monkey kept its hand on a resting bar, the color of the stimuli (green and/or yellow) indicating the type of liquid reward obtained for correct responding. This instruction cue was extinguished 0.5 s after it came up. After a 1-s delay following cue offset, another one (standard task) or two (choice task) visual stimuli (red) were presented at the same locations as the previous cue. In response to this signal, the monkey had to initiate a movement toward the target associated with the trigger stimulus (standard task) or to choose A, The scheme shows the temporal profile of activity changes for all neurons tested in the standard task (n=33), separately referenced to cue onset (left), movement onset (middle), and reward delivery (right). Each horizontal line indicates the duration of statistically significant changes in activity for a single STN neuron using both juice and water trials. In the delay period, lines are rank-ordered according to the earliest time of significant modulation after the cue onset, except for 6 neurons showing no modulation during this period (dashed lines at the bottom). In the two other task periods (movement and reward), lines are plotted in the same order as in the delay period. Dashed lines indicate a lack of significant change in discharge rate in a given period. The colored lines represent increase (red) or decrease (blue) in neuronal

Figure 1 .

 1 Figure 1. Temporal sequence of task events and reaching performance in the two task

  one of the two targets (choice task). Depending upon the color of the cue corresponding to the target reached, correct movement resulted in the delivery of juice or water reward. The two conditions were run in separate blocks of 40-60 trials. B, Reaching task performance for the two monkeys. Values of reaction time and movement time are means + SEM for the two types of reward. Standard task: Monkey P: n = 262 for juice trials and 282 for water trials; Monkey G: n = 173 for juice trials and 237 for water trials; Choice task: Monkey P: n = 678 for juice trials and 239 for water trials; Monkey G: n = 893 for juice trials and 275 for water trials.

Figure 2 .

 2 Figure 2. Mouth movements during the standard task for the two types of reward.

Figure 3 .

 3 Figure 3. Time course of changes in activity of STN neurons.

  activity. Arrows followed by A, B and C correspond to the three example STN neurons illustrated in Fig. 4. B, Relative proportions of neurons with increased and/or decreased activity among the three task periods. The percentages are calculated from the total number of recorded neurons (n=33).

Figure 4 .

 4 Figure 4. Three examples of STN neurons recorded in the standard task.

Figure 5 .

 5 Figure 5. Modulation of STN neuronal activity by reward type across task periods.

Figure 6 .

 6 Figure 6. Influence of different expected rewards on activity of STN neurons.

Figure 7 .

 7 Figure 7. Influence of different chosen rewards on activity of STN neurons.

Figure 8 .

 8 Figure 8. Locations of reward-discriminating neurons in the STN.

Figure 1 Figure 8

 18 Figure 1

Acknowledgements

This work was supported by the Centre National de la Recherche Scientifique, the Aix Marseille Université, and a French ANR grant of the Young Researcher program to C. Baunez (JC05_48262). We thank Dr. E. Legallet for computer programming and Dr. I. Balansard for assistance with surgery. We also thank Dr. M. Deffains and Dr. S. Ravel for their participation in data collection. The authors declare no competing interests.