Nicolas Clairis
email: nicolas.clairis@protonmail.com

Mathias Pessiglione
email: mathias.pessiglione@gmail.com

Deciding about courses of action involves minimizing costs and maximizing benefits. Decision neuroscience studies have implicated both the ventral and dorsal medial prefrontal cortex (vmPFC and dmPFC) in signaling goal value and action cost, but the precise functional role of these regions is still a matter of debate. Here, we suggest a more general functional partition that applies not only to decisions but also to judgments about goal value (expected reward) and action cost (expected effort). In this conceptual framework, cognitive representations related to options (reward value and effort cost) are dissociated from metacognitive representations (confidence and deliberation) related to solving the task (providing a judgment or making a choice). We used an original approach aiming at identifying consistencies across several preference tasks, from likeability ratings to binary decisions involving both attribute integration and option comparison. fMRI results in human male and female participants confirmed the vmPFC as a generic valuation system, its activity increasing with reward value and decreasing with effort cost. In contrast, more dorsal regions were not concerned with the valuation of options but with metacognitive variables, confidence being reflected in mPFC activity and deliberation time in dmPFC activity. Thus, there was a dissociation between the effort attached to choice options (represented in the vmPFC) and the effort invested in deliberation (represented in the dmPFC), the latter being expressed in pupil dilation. More generally, assessing commonalities across preference tasks might help reaching a unified view of the neural mechanisms underlying the cost/benefit tradeoffs that drive human behavior.

Significance statement

Decision neuroscience studies have implicated the medial prefrontal cortex in forming the cognitive representations that drive human choice behavior. However, different studies using different tasks have suggested somewhat inconsistent links between precise computational variables and specific brain regions. Here, we use fMRI to demonstrate a robust functional partition of the medial PFC that generalizes across tasks involving an estimation of goal value and/or action cost to provide a judgement or make a choice. This general functional partition makes a critical dissociation between neural representations of decisional factors (the expected costs and benefits attached to a given option) and metacognitive estimates (confidence in the judgment or choice, and effort invested in the deliberation process).

Introduction

Standard decision theory assumes that selecting a course of action can be reduced to maximizing a net value function, where expected benefits are discounted by expected costs.

Numerous studies in decision neuroscience have implicated key regions of the medial prefrontal cortex (PFC) in computing the net values of options during choice. While there is a general agreement for a functional dissociation between ventral and dorsal parts of the medial PFC (vmPFC, sometimes called medial OFC, versus dmPFC, sometimes called dACC), the specific roles of these subregions are still a matter of debate. Some accounts insist on the opponency between costs and benefits [START_REF] Rangel | Neural computations associated with goal-directed choice[END_REF][START_REF] Pessiglione | Why not try harder? Computational approach to motivation deficits in neuro-psychiatric diseases[END_REF]: the vmPFC would estimate the expected reward while the dmPFC would estimate the expected effort [START_REF] Bartra | The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value[END_REF][START_REF] Kurniawan | Effort and Valuation in the Brain: The Effects of Anticipation and Execution[END_REF][START_REF] Clithero | Informatic parcellation of the network involved in the computation of subjective value[END_REF][START_REF] Skvortsova | Learning To Minimize Efforts versus Maximizing Rewards: Computational Principles and Neural Correlates[END_REF]. However, this view has been challenged by representations of effort cost found in vmPFC activity and reward value in dmPFC activity [START_REF] Gläscher | Determining a Role for Ventromedial Prefrontal Cortex in Encoding Action-Based Value Signals During Reward-Related Decision Making[END_REF][START_REF] Fouragnan | Two spatiotemporally distinct value systems shape reward-based learning in the human brain[END_REF][START_REF] Klein-Flugge | Neural Signatures of Value Comparison in Human Cingulate Cortex during Decisions Requiring an Effort-Reward Trade-off[END_REF][START_REF] Pisauro | Neural correlates of evidence accumulation during value-based decisions revealed via simultaneous EEG-fMRI[END_REF][START_REF] Arulpragasam | Corticoinsular circuits encode subjective value expectation and violation for effortful goal-directed behavior[END_REF][START_REF] Seaman | Subjective value representations during effort, probability and time discounting across adulthood[END_REF][START_REF] Aridan | Neural correlates of effort-based valuation with prospective choices[END_REF][START_REF] Hogan | Roles of Ventromedial Prefrontal Cortex and Anterior Cingulate in Subjective Valuation of Prospective Effort[END_REF][START_REF] Westbrook | The Subjective Value of Cognitive Effort is Encoded by a Domain-General Valuation Network[END_REF][START_REF] Lopez-Gamundi | The neural basis of effort valuation: A meta-analysis of functional magnetic resonance imaging studies[END_REF]. Other accounts insist on the comparison between options that occurs during choice and suggest that the two regions estimate decision values in opposite fashion [START_REF] Boorman | How Green Is the Grass on the Other Side? Frontopolar Cortex and the Evidence in Favor of Alternative Courses of Action[END_REF][START_REF] Wunderlich | Neural computations underlying action-based decision making in the human brain[END_REF][START_REF] Hunt | Mechanisms underlying cortical activity during value-guided choice[END_REF][START_REF] Jocham | A mechanism for value-guided choice based on the excitation-inhibition balance in prefrontal cortex[END_REF]: the vmPFC would activate while the dmPFC would deactivate with value difference (chosen minus unchosen option value). Yet this other view has been questioned because the correlation with chosen and unchosen option values is not always observed in these regions, and because the value difference may be confounded with other constructs such as default preference, choice confidence and decision time [START_REF] Lim | The Decision Value Computations in the vmPFC and Striatum Use a Relative Value Code That is Guided by Visual Attention[END_REF][START_REF] Qin | Subjective Sense of Memory Strength and the Objective Amount of Information Accurately Remembered Are Related to Distinct Neural Correlates at Encoding[END_REF][START_REF] Martino | Confidence in value-based choice[END_REF][START_REF] Jocham | Dissociable contributions of ventromedial prefrontal and posterior parietal cortex to value-guided choice[END_REF][START_REF] Massar | Separate and overlapping brain areas encode subjective value during delay and effort discounting[END_REF][START_REF] Lopez-Persem | How prior preferences determine decisionmaking frames and biases in the human brain[END_REF][START_REF] Bobadilla-Suarez | Subjective value and decision entropy are jointly encoded by aligned gradients across the human brain[END_REF]. Thus, both types of accounts have received empirical support but also contradictory evidence, such that their validity is still debated.

Here, we intend to take a step aside from these debates and propose a functional partition that would generalize beyond choice tasks. Indeed, contrary to the view that there is no value representation outside of choice contexts [START_REF] Hayden | The case against economic values in the orbitofrontal cortex (or anywhere else in the brain)[END_REF], neural correlates of values in the medial PFC have been found in many tasks that do not involve any choice between the items presented, including likeability rating and distractive tasks or even passive viewing, during which covert likeability ratings are spontaneously generated [START_REF] Lebreton | An Automatic Valuation System in the Human Brain: Evidence from Functional Neuroimaging[END_REF][START_REF] Plassmann | Appetitive and Aversive Goal Values Are Encoded in the Medial Orbitofrontal Cortex at the Time of Decision Making[END_REF][START_REF] Harvey | Monetary favors and their influence on neural responses and revealed preference[END_REF][START_REF] Levy | Choice from Non-Choice: Predicting Consumer Preferences from Blood Oxygenation Level-Dependent Signals Obtained during Passive Viewing[END_REF][START_REF] Abitbol | Neural Mechanisms Underlying Contextual Dependency of Subjective Values: Converging Evidence from Monkeys and Humans[END_REF][START_REF] Martino | Social Information Is Integrated into Value and Confidence Judgments According to Its Reliability[END_REF][START_REF] Shenhav | Dissociable components of the reward circuit are involved in appraisal versus choice[END_REF][START_REF] Lopez-Persem | Four core properties of the human brain valuation system demonstrated in intracranial signals[END_REF]. We therefore reasoned that a general account for the role of the medial PFC in expressing preference should explain the pattern of activity observed during both rating and choice.

The new functional partition that we propose here is based on a metacognitive account [START_REF] Lee | Trading mental effort for confidence in the metacognitive control of valuebased decision-making[END_REF]: the idea is that, whatever the task, the brain invests effort in deliberation until it reaches a satisfactory level of confidence in the intended response. Thus, a second cost/benefit tradeoff would govern the meta-decision about when to make a response, the cost being the amount of time spent in deliberation and the benefit being the level of confidence attained. During this double cost/benefit arbitration, the brain would represent two sorts of variables: 1) at the decisional level, the reward and effort values associated to options proposed for rating or choice, and 2) at the metacognitive level, the expected confidence in the response and the required amount of deliberation. The aim of the present study is to test whether this functional partition can account for the pattern of activity observed in medial prefrontal regions across rating and choice tasks.

Materials and Methods

General overview

To this aim, we reversed the typical logic of standard functional neuroimaging approach, which specifies the roles of brain regions with contrasts that isolate minimal differences between conditions. On the contrary, we intended to generalize our findings across various conditions and tasks, with the aim to reach more robust conclusions. Thus, we employed a series of preference tasks (also called 'value-based' tasks) that enable the investigation of 1) the assignment of reward value or effort cost to a single option, with likeability rating tasks, 2) the comparison between two reward or two effort options with A/B choice tasks, and 3) the integration of reward and effort attributes for one option to accept or reject, with Yes/No choice tasks. In all these tasks, we defined the same key variables of interest as the global stimulus value (Val), which increases with more appetitive reward and/or less aversive effort, the confidence in the response (Conf), which is higher for more extreme ratings and more likely choices, and deliberation time (DT), meaning duration of the effort invested in the valuation process so as to reach a satisfactory response. We then explored the relationships between these three variables at the behavioral level, and their representations in the medial PFC at the neural level.

Subjects

In total, 40 right-handed volunteers participated in this fMRI study, which was approved by the Pitié-Salpêtrière Hospital local ethics committee. Participants were recruited through the RISC (Relais d'Information en Sciences de la Cognition) online platform (https://www.risc.cnrs.fr/) and signed inform consent prior to participation in the study. All participants were screened for the use of psychotropic medications and drugs, history of psychiatric and neurologic disorders, and traumatic brain injury. One participant was excluded from all analyses because of a clear misunderstanding about task instructions, leaving n=39 participants for behavioral data analysis (22 females / 17 males, aged 25.4±4.1 years). Another participant was excluded from the fMRI analysis due to excessive movement inside the scanner (>3mm within-session per direction).

Eleven additional participants were excluded from pupil size analysis, due to poor signal detection in at least one of the sessions (leaving a total of n=27 participants for pupil analysis).

All participants gave informed consent and were paid a fixed amount for their participation. The 15 first subjects were paid 60€ and the 25 other subjects were paid 75€. The difference in payoff corresponds to a difference in scanning protocols, although all participants performed the same tasks. The pilot protocol (n=15) aimed at comparing fMRI data acquisition sequences: regular EPI, EPI with multiband acceleration, EPI with multiband acceleration + multi-echo acquisition. The main protocol (n=25) aimed at addressing the neurocognitive question of interest with the best acquisition sequence. For this main protocol, we kept the regular EPI sequence for all sessions, as we saw no clear advantage for multiband acceleration or multi-echo acquisition in basic contrast images. Therefore, the analyses only include fMRI data using regular EPI acquisition (three sessions for the pilot protocol, all nine sessions for the main protocol).

Behavioral tasks

All tasks were programmed using the Psychtoolbox [START_REF] Brainard | The Psychophysics Toolbox[END_REF] Psychtoolbox-3 running in Matlab (The MathWorks, Inc., Version 2012). Participants were given a 4-button box (fORP 932, Current Designs Inc, Philadelphia, USA) placed under their right hand to provide their responses. Stimuli were projected on a computer screen, their luminance being estimated using standard function of red-green-blue composition (0.299•red + 0.587•green + 0.114•blue, see http://www.w3.org/TR/AERT#color-contrast). Stimuli comprised 144 reward items (72 food and 72 goods) and 72 effort items (36 mental and 36 physical). Half the reward items were presented with text only (Rewt items), and the other half was presented with both text and image (Rewti items). All effort items were only described with text (Efft). For each task, fMRI sessions were preceded by a short training (not included in the analysis), for participants to familiarize with the sort of items they would have to valuate and with the button pad they would use to express their preferences.

Participants all started with a (dis-)likeability rating task (Fig. 1A), performed during the first three fMRI sessions, each divided into three 24-trial blocks corresponding to the three stimulus type (Rti, Rt, Et). The order of blocks within a session was counterbalanced across participants. The items were presented one by one, and participants rated them by moving a cursor along a visual analog scale. They used their index and middle fingers to press buttons corresponding to left and right movements, and validated the final position of the cursor by pressing a third button, which triggered the new trial. The initial position of the cursor, at the beginning of each trial, was randomly placed between 25 and 75% of the 0-100 rating scale.

There was no mark on the scale, giving the impression of a continuous rating, although it was in practice discretized into 100 steps. The left and right extremes of the scale were labeled "I would not care" and "I would like it enormously" for reward items, "I would not mind" and "I would dislike it enormously" for effort items. Note that both reward and effort scales included indifference at one extremity, such that the two scales could form a continuum of increasing likeability from very aversive effort to very appetitive reward. In any case, the situations to be rated were hypothetical: the question was about how much they would like the reward (should it be given to them at the end of the experiment) and how much they would dislike the effort (should it be imposed to them at the end of the experiment). Should the timeout (10 s in rating tasks and 5s in choice tasks) be reached, the message 'too slow' would have been displayed on screen and the trial repeated later, but this remained exceptional.

After the three rating sessions, participants performed a series of binary choices. The A/B choice task (Fig. 1B) involved expressing a preference between two options of a same dimension, presented on the left and right of the screen. The two options were items presented in the rating task, drawn from the same category, regarding both the presentation mode (Rewti vs Rewti, Rewt vs Rewt, Efft vs Efft) and type of stimulus (food vs. food, goods vs. goods, mental vs mental, physical vs physical). Each item was presented twice, following two intermixed pairing schedules: one varied the mean rating (i.e., stimulus value) while controlling for distance (i.e., decision value or choice difficulty), whereas the other varied the distance in rating while controlling the mean. Participants selected the reward they would most like to obtain, or the effort they would least dislike to exert, by pressing the left or right button with their middle or index finger. The chosen option was then highlighted with a red frame, so participants could check that their choice was correctly recorded. The fMRI sessions devoted to the A/B choice task included three 24-trial blocks presenting the three types of options (Rti, Rt, Et), the order of blocks being counterbalanced across participants.

Then participants performed the Yes/No choice task (Fig. 1C), which involved deciding whether to accept exerting a given effort in order to get a given reward. Thus, every trial proposed one option combining two dimensions (one Rewt and one Efft item). Each item was presented twice, following two intermixed pairing schedules: one associating more pleasant reward with more painful effort (thus controlling for decision value or choice difficulty), the other associating more pleasant reward with less painful effort (thus varying choice difficulty).

The mean net value was also balanced across fMRI sessions. Participants selected their response by pressing the button corresponding to 'yes' or 'no' with their index or middle finger. The left/right position of yes/no responses was counterbalanced across trials. To give participants a feedback on their choice, the selected option was highlighted with a red frame. The three fMRI sessions devoted to the Yes/No choice task contained 48 trials each.

Note that, as were ratings, all choices were hypothetical. This was implemented to enable the use of natural reward and effort items that can be encountered in everyday life but are difficult to implement in the lab (such as walking a 1-km distance). Another reason was to allow for a distinction between the estimation of effort cost and motor preparation processes that are triggered when efforts are implemented [START_REF] Hogan | Roles of Ventromedial Prefrontal Cortex and Anterior Cingulate in Subjective Valuation of Prospective Effort[END_REF].

Behavioral data analysis

All data were analyzed using Matlab 2017a (The MathWorks, Inc., USA).

Choices were fitted with logistic regression models of decision value, with intercept and slope parameters.

For A/B choices, the model was:

𝑷 𝒍𝒆𝒇𝒕 = 1 1 + 𝑒 -(𝛽 0 + 𝛽 1 •𝛥𝑉)
Where Pleft is the probability of choosing the left option, ΔV is the decision value, i.e. the difference in likeability rating between left and right options (Vleft -Vright), while β0 and β1 are the intercept and slope parameters capturing potential bias and choice consistency (inverse temperature).

For Yes/No choices, the model was:

𝑷 𝒂𝒄𝒄𝒆𝒑𝒕 = 1 1 + 𝑒 -(𝛽 0 + 𝛽 𝑅𝑒𝑤 .𝑉 𝑅𝑒𝑤 + 𝛽 𝛦𝑓𝑓 .𝑉 𝐸𝑓𝑓 )
Where Paccept is the probability of accepting the offer (make the effort to get the reward), VRew and VEff are the likeability ratings provided for the reward and effort items. Thus, the decision value (or net value) here is a weighted sum of reward and effort likeability (one being positive and the other negative), the parameter weights βRew and βEff serving as both scaling factors and inverse temperature.

The stimulus value (Val) and response confidence (Conf) regressors used in the analysis of deliberation time (DT) and fMRI data were respectively defined as the addition of likeability ratings assigned to the items on screen and the squared distance from the mean response. They were adapted to each task, as follows:

Rating task A/B choice task Yes/No choice task Val V Vleft + Vright βRew•VRew + βEff•VEff Conf [V-mean(V)] ² [Pleft -mean(Pleft)]² [Paccept -mean(Paccept)]²
In each case, V is either the reward or effort likeability provided by z-scored individual rating of the item presented in a given trial, and P is the probability generated for each trial using the logistic model fitted to choices. Note that, by construction (before z-scoring), V is positive for reward items (which are liked) and negative for effort items (which are disliked).

The mean response used in confidence estimation is simply the mean rating over trials, the mean frequency of left choice and the mean frequency of accept choice, depending on the task.

The validity of our confidence proxy had been previously assessed and confirmed in two independent datasets (Fig. 3).

Deliberation time (DT) was defined across tasks as the time between stimulus onset and first button press. A tilted-plane acquisition sequence was used to optimize sensitivity to BOLD signal in the orbitofrontal cortex [START_REF] Deichmann | Optimized EPI for fMRI studies of the orbitofrontal cortex[END_REF][START_REF] Weiskopf | Optimized EPI for fMRI studies of the orbitofrontal cortex: compensation of susceptibility-induced gradients in the readout direction[END_REF]. Note that the number of volumes per session was not predefined, because all responses were self-paced. Volume acquisition was just stopped when the task was completed.

Most subjects (n=25) performed nine fMRI sessions (three per task) using this standard EPI sequence. The pilot subgroup (n=15) also performed nine fMRI sessions, but the fMRI data acquisition sequences were alternated between standard EPI, EPI with multi-band acceleration factor (TR = 1.20 s; TE = 25 ms; flip angle = 66°; number of slices = 44; slice thickness = 2.5 mm; acceleration factor = 2) and EPI with multi-band acceleration factor + multi-echo (TR = 1.28 s; TE = 11.00 ms and 29.89 ms; flip angle = 69°; number of slices = 44; slice thickness = 2.5 mm; acceleration factor = 2). The order of fMRI sequences was counterbalanced across participants. Preliminary analyses of basic contrast images were done using the pilot dataset to select the best acquisition sequence. As there was no clear benefit with the multi-band and multi-echo add-ons, we retained the standard EPI for the main experiment.

fMRI data analysis

Functional MRI data were preprocessed and analyzed with the SPM12 toolbox (Wellcome Trust Center for NeuroImaging, London, UK) running in Matlab 2017a. Preprocessing consisted of spatial realignment, normalization using the same transformation as anatomical images, and spatial smoothing using a Gaussian kernel with a full width at a half-maximum of 8 mm.

Preprocessed data were analyzed with a standard general linear model (GLM) approach at the first (individual) level and then tested for significance at the second (group) level. All GLM included the six movement regressors generated during realignment of successive scans.

In our main GLM, stimulus onset was modeled by a stick function, modulated by the following regressors: 1) fixation cross duration, 2) luminance, 3) text length, 4) Val, 5) Conf, 6) DT. The first three were factors of no interest that were found to significantly impact DT in the linear regression analysis. The regressors of interest (Val, Conf and DT) were defined as explained in the behavioral data analysis section. The different blocks of the rating and A/B choice tasks (presenting reward as text + image, reward as text and effort as text) were modeled in separate regressors. All regressors of interest were z-scored and convolved with the canonical hemodynamic response function and its first temporal derivative. All parametric modulators were serially orthogonalized. At the second level, correlates of Val, Conf and DT were obtained with contrasts tested across tasks of corresponding regression estimates against zero. Note that likeability ratings obtained for effort items were negative in all regressors (meaning that they can only decrease stimulus value).

Several alternative GLM were built to test variants of the main GLM. GLM2 was identical to GLM1 except that orthogonalization was removed such that all native regressors could compete to explain variance in fMRI time series. GLM3 was identical to GLM1, except that instead of a stick function, stimulus onsets were modeled with a boxcar function modeling periods from stimulus onset to first button press. Three additional GLM were built to further explore the choice tasks. In GLM4, the Val regressor (sum of option values) was replaced by the difference between option values (Vc -Vuc) for the two choice tasks. This GLM served to perform a group-level Bayesian model comparison to test which value regressor (sum or difference) best explains the fMRI time series during choice tasks. In GLM5, Conf and DT were removed and Val was replaced by two separate regressors for the chosen and unchosen option values (Vc and Vuc). This GLM was used to test whether regressor estimates for chosen and unchosen values had the same sign (as in a sum) or opposite signs (as in a difference). In GLM6, reward and effort values were split in two separate regressors for all tasks (including the Yes/No choice task). The purpose of this GLM was to distinguish between neural correlates of reward value and effort cost in brain valuation regions. Finally, a last GLM was built with one event per trial, modeled with a stick function, at the time of stimulus onset, with the aim to extract trial-by-trial activity levels in regions of interest, which then served as regressors to explain pupil size data (see next section).

Regions of interest (ROI) were defined as clusters in group-level statistical maps that survived significance threshold of p < 0.05 after family-wise error correction for multiple comparisons at the voxel level. To avoid double dipping [START_REF] Kriegeskorte | Circular analysis in systems neuroscience: the dangers of double dipping[END_REF] in statistical tests, regression estimates were extracted from ROI re-defined for each participant through a leave-one-out procedure. Regarding Bayesian Model Selection, to avoid biasing the comparison in favor of one or the other GLM, an independent ROI was defined as the conjunction between the positive minus negative value contrast in a published meta-analysis [START_REF] Bartra | The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value[END_REF] and the bilateral medial orbitofrontal cortex region from the AAL atlas [START_REF] Tzourio-Mazoyer | Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain[END_REF]. Additionally, we defined twelve 8-mm radius spherical ROI in the medial wall to illustrate the distribution of regression estimates for Val, Conf and DT.

Parameter estimates were extracted from each voxel within these ROI and then averaged across voxels.

Meta-analysis of fMRI studies

The meta-analytic maps were extracted from the online platform Neurosynth (https://www.neurosynth.org/), using the keywords "value" (470 studies), "confidence" (79 studies) and "effort" (204 studies) for "uniformity test", which displays brain regions that are consistently activated across papers mentioning the target keyword. Each map was binarized to visualize clusters surviving a significance threshold of p < 0.01 after false discovery rate (FDR) correction for multiple comparisons.

Pupil size

Pupil diameter was recorded at a sampling rate of 1000Hz, using an EyeLink 1000 plus (SR Research) eye-tracker. The eye-tracker was calibrated before the start of fMRI sessions, once the subject was positioned inside the scanner. A cubicle interpolation was performed to compensate for any period of time when the pupil signal was lost due to blinking. The pupil size time series were subsequently band-pass filtered (1/128 to 1 Hz) and z-scored per session.

Within-trial variations in pupil size was baseline-corrected (by removing the mean signal over the 200 ms preceding stimulus onset) and time-locked either to stimulus onset or button press. Then trial-wise variations in pupil size were fitted with a linear regression model that included factors of no interest (an intercept per block, jitter duration, stimulus luminance and text length), variables of interest (Val, Conf and DT defined as in the behavioral data analysis section) and neural activity (extracted from vmPFC, mPFC and dmPFC ROI clusters).

Within-trial individual time series of regression estimates were then smoothed using a 100ms

kernel. Group-level significant time clusters were identified after correction for multiple comparisons estimated according to random field theory, using the RFT_GLM_contrast.m function of the VBA toolbox (available at http://mbb-team.github.io/VBA-toolbox/).

Results

Behavior

Participants (n=39 in total, 22 females) first performed a series of ratings, divided into three fMRI sessions (Fig. 1A). Each session presented 72 items to be valuated one by one. Within a session, items were grouped into three blocks: one block with 24 reward items presented by text + image (Rewti), one block with 24 reward items presented by text only (Rewt) and one block with 24 effort items presented by text only (Efft). The reason for varying the mode of presentation was to assess the generality of the neural valuation process across different inputs that require more or less imagination, according to previous study [START_REF] Lebreton | A Critical Role for the Hippocampus in the Valuation of Imagined Outcomes Behrens T[END_REF]. For reward, participants were asked to rate how much they would like it, should they be given the item immediately after the experiment. Symmetrically, the instruction for effort was to rate how much they would dislike it, should they be requested to exert it immediately after the experiment. We included both food and non-food (goodies) reward items, and both mental and physical effort items. There was no number on the scale, just labels on endpoints, and ratings were pseudo-continuous, from 'I would not care / mind' to 'I would like / dislike it enormously'.

Thus, the left endpoint corresponded to indifference and the right endpoint to extreme attraction or extreme aversion (Fig. 1A).

The z-scored rating was taken as a proxy for stimulus value (Val) in this task, while the square of z-score rating was taken as a proxy for response confidence (Conf). The quadratic relationship between confidence and rating has been validated empirically and accounted for by a Bayes-optimal model mapping a probabilistic distribution (over likeability) onto a bounded visual scale [START_REF] Lebreton | Automatic integration of confidence in the brain valuation signal[END_REF][START_REF] Lopez-Persem | Four core properties of the human brain valuation system demonstrated in intracranial signals[END_REF]. Under this model, confidence is inversely proportional to the variance of the underlying probability distribution, hence to the variability in likeability rating across presentations of the same item when they are repeated (which was not the case in the present design). The confidence proxy used here is not to be confounded with motivational salience, which would be maximal for very appetitive reward and very aversive effort. Instead, confidence is maximal at the extremes of the rating scale, meaning for both very appetitive and null reward or for both very aversive and null effort (Fig. 2A). Note also that Val and Conf were orthogonal variables by construction (Conf being a Ushaped function of Val for both reward and effort).

Deliberation time (DT) was defined as the time between item onset and the first button press used to move the cursor along the scale. DT was regressed against a linear model that included Val and Conf proxies (Fig. 2B), in addition to factors of no interest (such as jitter duration, stimulus luminance, text length and trial index, see methods). Irrespective of stimulus type, we found a significant effect of both value (Rewti: βVal = -0.21 ± 0.02, p = 4•10 -11 ; Rewt: βVal = -0.17 ± 0.02, p = 6•10 -11 ; Efft: βVal = 0.26 ± 0.03, p = 2•10 -11 ) and confidence (Rewti: βConf = -0.17 ± 0.03, p = 3•10 -8 ; Rewt: βConf = -0.19 ± 0.03, p = 7•10 -8 ; Efft: βConf = -0.13 ± 0.04; p = 0.0024). Thus, participants were faster to provide their rating when the item was more appetitive (or less aversive) and when they were more confident (going towards the extremes of the rating scale). Among the factors of no interest, we observed effects of jitter duration, stimulus luminance and text length, which were therefore included as regressors in subsequent analyses.

However, there was no significant effect of trial index, which discards a possible contamination of DT by habituation or fatigue.

Then participants performed a series of binary choices, either A/B choices or Yes/No choices. The choice tasks were always performed after the rating tasks because the ratings were used to control the difficulty of choices (i.e., the difference in value between the two options).

In the A/B choice task (Fig. 1B), participants were asked to select the reward they would prefer to receive at the end of the experiment, if they were offered one of two options, or the effort they would prefer to exert, if they were forced to implement one of two options. Thus, the two options always pertained to the same dimension (reward or effort), and even to the same subcategory (food or good for reward, mental or physical for effort), to avoid shortcut of deliberation by general preference. The mode of presentation (text or image) was also the same for the two options, to avoid biasing the choice by a difference in salience. To obtain a same number of trials as in the rating task, each item was presented twice, for a total of 72 choices per stimulus type (Rewti, Rewt, Efft) distributed over three fMRI sessions. Within a session, items were grouped into three blocks: one block with 24 choices between reward items presented with text + image (Rewti), one block with 24 choices between reward items presented with text only (Rewt) and one block with 24 choices between effort items presented with text only (Efft). In the Yes/No choice task (Fig. 1C), participants were asked whether they would be willing to exert an effort in order to obtain a reward, at the end of the experiment. Only items described with text were retained for this task (since there was no picture for effort items), each item again appearing twice, for a total of 144 choices divided into three fMRI sessions of 48 trials each.

The A/B choice task was meant to assess value comparison between the two options, within a same dimension. The decision value (ΔV) in this task was defined as the difference in (dis-)likeability rating between the two options. We checked with a logistic regression (Fig. 2A) that ΔV was a significant predictor of choices, irrespective of stimulus type (Rewti: βΔV = 3.38 ± 0.27, p = 7•10 -15 ; Rewt: βΔV = 2.67 ± 0.16, p = 2•10 -19 ; Efft: βΔV = -2.28 ± 0.16, p = 4•10 - 17 ). The Yes/No choice task was meant to assess value integration across two dimensions, for a single option. The decision value (or net value) in this task was defined as a linear combination of reward and effort ratings. Note that it would make no sense to fit an effort discounting function here, because such function is meant to capture the mapping from objective effort levels to subjective effort estimates, which we directly collected (with dislikeability ratings).

We checked with a logistic regression that both reward and effort ratings were significant predictors of choice in this task (βRew = 1.50 ± 0.09, p = 6•10 -20 ; βEff = -1.12 ± 0.08, p = 1•10 - 16 ).

To analyze DT (time between stimulus onset and button press) in choice tasks, we defined proxies for stimulus value and response confidence, as we did for the rating task.

Stimulus value (Val) was defined as the addition of the likeability ratings assigned to the two stimuli on screen. In the A/B choice task, this is simply the sum of the two item ratings. In the Yes/No choice task, this is a weighted sum (with a scaling factor to adjust the unit of reward and effort ratings). In both cases, choice probability was calculated with the logistic regression ). Thus, the relationship between DT and confidence was similar in rating and choice tasks: participants were faster when they were more confident (because of a strong preference for one response or the other). They also tended to be faster when the options were more appetitive (or less aversive), but this trend was not significant in all tasks.

Because we did not measure confidence in the present study, we verified that our proxy could predict confidence ratings in separate datasets. Note that this proxy has already been validated for likeability rating tasks used in previous studies [START_REF] Lebreton | Automatic integration of confidence in the brain valuation signal[END_REF][START_REF] Martino | Social Information Is Integrated into Value and Confidence Judgments According to Its Reliability[END_REF][START_REF] Lopez-Persem | Four core properties of the human brain valuation system demonstrated in intracranial signals[END_REF], a result that we reproduced here (Fig. 3). To test whether the same proxy could also predict confidence in choice tasks, we used another dataset from a published study [START_REF] Lee | Choosing what we like vs liking what we choose: How choice-induced preference change might actually be instrumental to decision-making[END_REF]. In this study, participants provided confidence ratings about having selected the best option in binary A/B choices (between food items presented two by two). Our confidence proxy could significantly predict confidence judgments not only in the likeability rating task but also in the A/B choice task even when including Val and DT as competitors (without orthogonalization) in the same regression model (rating: βConf = 0.49 ± 0.09; p = 8•10 -5 ; choice: βConf = 0.21 ± 0.02; p = 2•10 -11 ).

Neural activity

The aim of fMRI data analysis was to dissociate the first-level variables related to option attributes (reward and effort estimates) from the second-level variables related to metacognition (confidence and deliberation) across value-based tasks (rating and choice). To assess whether these variables can be dissociated on the basis of existing literature, we conducted a metaanalysis of fMRI studies using Neurosynth platform (Fig. 4A) with value, confidence and effort as keywords. Results show that the three keywords are associated to similar activation patterns, with clusters in both vmPFC and dmPFC. To better dissociate the neural correlates of these constructs in our dataset, we built a general linear model where stimulus onset events were modulated by our three variables of interest -Val, Conf and DT (defined as in the behavioral data analysis). Factors of no interest that were found to influence DT (jitter duration, stimulus luminance, text length) were also included as modulators of stimulus onset events, before the variables of interest. Note that by construction, the correlation between regressors of interest was low (between -0.084 and -0.204). Nevertheless, to avoid any confound in the interpretation, we employed serial orthogonalization. Thus, the variables of interest were orthogonalized with respect to factors of no interest, and DT was made orthogonal to all other regressors, including

Val and Conf.

After correction for multiple comparisons at the voxel level, we found only three significant clusters in the prefrontal cortex (Fig. 4B): Val was signaled in vmPFC activity (Table Fig. 4-1), Conf in mPFC activity (Table Fig. 4-2) and DT in dmPFC activity (Table Fig. 4-3). All three correlations were positive, there was no significantly negative correlation in any brain region when correcting for multiple comparisons. With a more lenient threshold (correction at the cluster level), we observed significant positive association with Val in other brain regions, such as the ventral striatum (vS), posterior cingulate cortex (pCC) and primary visual cortex (V1). Note that vS and pCC are standard components of the brain valuation system, whereas V1 activation is likely to be an artifact of gaze position on the rating scale, as it was not observed in the choice tasks. Consistently, positive correlation with Val was found in right V1 activity, and negative correlation in left V1 activity (a pattern that was not observed with other clusters). To provide a more exhaustive depiction, we examined the distribution of regression estimates below statistical thresholds, along a path going from vmPFC to dmPFC within a medial plane (Fig. 6A). Results show that the three associations did not correspond to separate clusters (as was suggested by thresholded maps) but to gradual variations peaking at different positions along the path.

To assess whether the triple association between variables and clusters of interest was robust, we conducted a number of additional analyses using variants of the main GLM (Fig. 5).

The same three clusters were significantly associated with the Val, Conf and DT regressors when 1) removing serial orthogonalization such that regressors could compete for variance and

2) replacing stick functions by boxcar functions extending from stimulus onset to behavioral response (showing a modulation of dmPFC activity by DT in amplitude and not just duration).

In addition, we tested the triple association using different fMRI acquisition sequences in participants of the pilot study (n=15). The fMRI sessions acquired with multiband acceleration sequences (see methods) were not included in the main analysis, since they were not directly comparable to those using our standard EPI sequence. We separately regressed fMRI activity recorded during these sessions against our main GLM, and observed similar trends in this independent dataset. Due to a three times smaller sample, activations did not pass whole-brain corrected thresholds. However, using group-level significant clusters (from the main dataset) as regions of interest (ROI), we observed significant associations of Val and DT with vmPFC and dmPFC, respectively (vmPFC: βVal = 0.164 ± 0.044, p = 0.0024; dmPFC: βDT = 0.236 ± 0.062, p = 0.0021).

We further analyzed the relationship between computational variables and activity in the three medial prefrontal ROI with post-hoc t-tests on regression estimates. To avoid any double-dipping issue, we used a leave-one-out procedure, such that clusters were defined from group-level analyses including all subjects but the one in whom regression estimates were extracted. We first verified that the three main associations were not driven by any particular task (Fig. 6B and6C). Indeed, regression estimates were significant in both rating and choice tasks, more specifically for Val in vmPFC activity (rating: βVal = 0.69 ± 0.13, p = 6•10 -6 ; choice: βVal = 0.47 ± 0.10, p = 3•10 -5 ), for Conf in mPFC activity (rating: βConf = 0.75 ± 0.11, p = 8•10 - 8 ; choice: βConf = 0.31 ± 0.10, p = 0.004) and for DT in dmPFC activity (rating: βDT = 0.39 ± 0.11, p = 9•10 -4 ; choice: βDT = 0.74 ± 0.11, p = 7•10 -8 ). Note that our point was to generalize the associations across different tasks -comparing between tasks would be meaningless because tasks were not designed to be comparable (any possible significant contrast could be due to many differences of no interest).

We also investigated whether each cluster of interest was better associated with the corresponding variable (across tasks), again using a leave-one-out procedure to avoid double dipping (Fig. 6B): Val was better reflected in vmPFC activity (βVal/vmPFC > βVal/mPFC : p = 9•10 -8

; βVal/vmPFC > βVal/dmPFC : p = 4•10 -7 ), Conf in mPFC activity (βConf/mPFC > βConf/vmPFC : p = 0.0043; βConf/mPFC > βConf/dmPFC : p = 3•10 -7 ) and DT in dmPFC activity (βDT/dmPFC > βDT/vmPFC : p = 0.066;

βDT/dmPFC > βDT/mPFC : p = 7•10 -4 ). However, the fact that vmPFC, mPFC and dmPFC better reflected Val, Conf and DT, respectively, does not imply that these regions were not affected by the other variables. In particular, vmPFC activity was also associated with Conf and DT, (βConf = 0.26 ± 0.10, p = 0.012; βDT = 0.40 ± 0.11, p = 0.001), even if it was dominated by Valrelated activity. Nevertheless, all cross-over interactions between regions and variables were significant: from vmPFC to mPFC, the relative encoding of Val and Conf (βVal -βConf)

significantly reversed (0.29 ± 0.11 vs. -0.30 ± 0.10, p=2•10 -8 ) and similarly, from mPFC to dmPFC, the relative encoding of Conf and DT (βConf -βDT) significantly reversed (0.27 ± 0.13 vs. -0.72 ± 0.14, p = 9•10 -6 ). The distant cross-over interaction between vmPFC and dmPFC (βVal -βDT) was also significant (0.15 ± 0.15 vs. -0.30 ± 0.10, p=10 -5 ).

We next looked for further generalization of the valuation signal, not solely across tasks but also across stimuli. In the main analysis, fMRI time series were regressed against a GLM that separated stimulus types (Rewti, Rewt and Efft) into different onset regressors, each modulated by corresponding ratings. Instead of testing the average regression estimates across stimulus categories, we tested regression estimates obtained for each category, separately (Fig. 6D). Regression estimates (extracted using leave-one-out procedure across rating and choice tasks) show that vmPFC activity was positively related to the subjective value of reward items, whether or not they are presented with an image (Rewti: βVal = 0.49 ± 0.13, p = 8•10 -4 ; Rewt: βVal = 0.61 ± 0.13, p = 5•10 -5 ), and negatively correlated to the subjective cost of effort items (Efft: βVal = -0.35 ± 0.13, p = 0.017). Thus, the association between Val and vmPFC activity was independent of the presentation mode, and integrated costs as well as benefits.

On a different note, we questioned the validity of our Val proxy to capture value-related activity in choice tasks. Again, the reason for summing stimulus values in choice tasks instead of taking the difference between chosen and unchosen option values, as is often done, was that we wanted a proxy that could generalize to rating tasks, in which there is no notion of difference, since there is only one stimulus on screen. Note that the value difference regressor (chosen minus unchosen option value) is related to all three variables that we intend to dissociate here as capturing different concepts (stimulus value, response confidence, deliberation effort).

Nevertheless, we wondered whether vmPFC activity in choice tasks would be better captured by the difference (Vc -Vuc) than by the sum (Vc + Vuc). To test this, we simply replaced our partition (Val / Conf / DT) by Vc and Vuc regressors, and fitted the GLM to fMRI activity recorded during choice tasks only (Fig. 6E). The two regression estimates, extracted from the Val cluster in the main analysis, were significantly positive (βVc= 0.42 ± 0.12, p = 9•10 -4 ; βVuc = 0.29 ± 0.07, p = 2•10 -4 ), with no significant difference between the two (p = 0.36), therefore

showing no evidence for a representation of the difference. We completed this simple analysis by a comparison using Bayesian Model Selection at the group level, between two variants of the main GLM where Val was replaced by either the sum (Vc + Vuc) or the difference (Vc -Vuc), competing to explain choice-related activity in a vmPFC ROI defined from the literature (to avoid non-independence issues). Although not formally conclusive, the comparison showed that exceedance probability was in favor of the sum model (Fig. 6F), thus validating our Val proxy as most relevant to capture vmPFC activity, even during choices. Another advantage of this Val proxy is being orthogonal to confidence, whereas the difference between option values is not. The consequence is that the neural correlates of Conf were unaffected by introducing the Val regressor, or by serial orthogonalization (Fig. 5).

Importantly, no consistent association with reward value or effort cost was observed in putative opponent brain regions such as the dmPFC, which was instead systematically reflecting DT. Thus, it appeared that dmPFC activity reflected the metacognitive effort cost invested in the ongoing task (deliberation about the response) rather than the effort cost attached to the option on valuation. Importantly, the association with DT was observed despite the fact that DT was orthogonalized to both value and confidence, suggesting that the dmPFC represents the effort invested above and beyond that induced by the difficulty of value-based judgment or decision. The parametric modulation by DT was also obtained when dmPFC activation was fitted with a boxcar function extending from stimulus response (Fig. 5), suggesting a modulation in amplitude beyond prolonged activity.

However, DT is a very indirect proxy for the effort invested in solving the task, and could be affected by many other factors (such as distraction or mind-wandering). We therefore investigated the relationship between brain activity and another proxy that has been repeatedly related to effort: pupil size. Neural activity was extracted in each ROI by fitting a GLM containing one event (stimulus onset) per trial. Then pupil size at each time point was regressed across trials against a GLM that contained factors of no interest (luminance, jitter duration, text length), variables of interest (Val, Conf, DT) and neural activity (vmPFC, mPFC, dmPFC).

A positive association between pupil size and dmPFC activity was observed in both rating and choice tasks (Fig. 7), about one second before the response. This association was not an artifact of the trial being prolonged (and therefore of the response to luminance being cut at different durations), since it was observed both when locking time courses on stimulus onset and on motor response (button press). Finally, it was specific to the dmPFC ROI, and observed even if dmPFC was made independent (through serial orthogonalization) to all other variables (notably Val, Conf and DT). Thus, the association between dmPFC and pupil size was observed above and beyond DT and factors that could affect DT. In contrast, there was no consistent association between vmPFC and pupil size before the response, suggesting that the correlates of DT observed in vmPFC were not related to effort but to some other factors affecting DT, such as mind-wandering.

Discussion

Exploring the neural correlates of variables that are common to rating and choice tasks, we observed a functional partition within the medial PFC: stimulus value, response confidence and deliberation time were best reflected in vmPFC, mPFC and dmPFC activity, respectively.

Our results confirm the role attributed to the vmPFC as a generic valuation system [START_REF] Levy | The root of all value: a neural common currency for choice[END_REF][START_REF] Bartra | The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value[END_REF]. The subjective value of reward items was reflected in vmPFC activity irrespective of the category (food versus goods), as was reported in many studies [START_REF] Chib | Evidence for a Common Representation of Decision Values for Dissimilar Goods in Human Ventromedial Prefrontal Cortex[END_REF][START_REF] Lebreton | An Automatic Valuation System in the Human Brain: Evidence from Functional Neuroimaging[END_REF][START_REF] Abitbol | Neural Mechanisms Underlying Contextual Dependency of Subjective Values: Converging Evidence from Monkeys and Humans[END_REF][START_REF] Lopez-Persem | Four core properties of the human brain valuation system demonstrated in intracranial signals[END_REF].

Also, vmPFC value signals were observed whether or not reward items were presented with images, suggesting that they can be extracted from both direct perceptual input or from textbased imagination, which was shown to recruit episodic memory systems [START_REF] Lebreton | A Critical Role for the Hippocampus in the Valuation of Imagined Outcomes Behrens T[END_REF]. Critically, our results show that the vmPFC also reflects the effort cost (whether mental

or physical) attached to potential courses of actions. Therefore, they challenge previous suggestions that the vmPFC is involved in stimulus valuation, independently of action costs [START_REF] Rangel | Neural computations associated with goal-directed choice[END_REF][START_REF] Pessiglione | Why not try harder? Computational approach to motivation deficits in neuro-psychiatric diseases[END_REF]. They rather suggest that the vmPFC might compute a net value, its activity increasing with reward benefit and decreasing with effort cost, so as to prescribe whether or not an action is worth engaging. This idea is in line with recent mounting evidence that vmPFC activity decreases with effort demand [START_REF] Aridan | Neural correlates of effort-based valuation with prospective choices[END_REF][START_REF] Hogan | Roles of Ventromedial Prefrontal Cortex and Anterior Cingulate in Subjective Valuation of Prospective Effort[END_REF][START_REF] Westbrook | The Subjective Value of Cognitive Effort is Encoded by a Domain-General Valuation Network[END_REF][START_REF] Lopez-Gamundi | The neural basis of effort valuation: A meta-analysis of functional magnetic resonance imaging studies[END_REF].

The mPFC was not affected by reward value or effort cost, but the confidence in the response. Our notion of confidence (defined as the squared distance from the mean response)

was orthogonal to stimulus value (defined as the addition of reward and/or effort values). This confidence proxy was previously shown to correlate with confidence ratings and to elicit similar neural correlates (De [START_REF] Martino | Social Information Is Integrated into Value and Confidence Judgments According to Its Reliability[END_REF][START_REF] Lopez-Persem | Four core properties of the human brain valuation system demonstrated in intracranial signals[END_REF]. The value proxy is related to the representation of overall value (or 'set liking') assigned to choice options, which was previously observed in vmPFC activity [START_REF] Blair | Choosing the Lesser of Two Evils, the Better of Two Goods: Specifying the Roles of Ventromedial Prefrontal Cortex and Dorsal Anterior Cingulate in Object Choice[END_REF][START_REF] Palminteri | Brain Hemispheres Selectively Track the Expected Value of Contralateral Options[END_REF][START_REF] Hare | Transformation of stimulus value signals into motor commands during simple choice[END_REF][START_REF] Jocham | Dissociable contributions of ventromedial prefrontal and posterior parietal cortex to value-guided choice[END_REF][START_REF] Gluth | Effective Connectivity between Hippocampus and Ventromedial Prefrontal Cortex Controls Preferential Choices from Memory[END_REF][START_REF] Shenhav | Dissociable components of the reward circuit are involved in appraisal versus choice[END_REF]. The two notions are close to the sum and difference signals that may emerge from an attractor network model in which two neuronal populations compete for their favorite option through mutual inhibition [START_REF] Hunt | Mechanisms underlying cortical activity during value-guided choice[END_REF]. Our results suggest a partial dissociation of value and confidence signals (as in [START_REF] Shenhav | Dissociable components of the reward circuit are involved in appraisal versus choice[END_REF]) that is consistent with a previously described ventro-dorsal gradient from value to confidence (De [START_REF] Martino | Social Information Is Integrated into Value and Confidence Judgments According to Its Reliability[END_REF]. The same dissociation applied to the rating task, where there is no comparison between unrelated options.

Note that there could be a covert comparison between current and previous items, with the purpose to adjust the rating, not to select an option and discard the others. We also acknowledge that in a sense, likeability ratings can be conceived as a choice, since one position on the rating scale must be selected. However, this would be choosing between a large number (virtually infinite) of possible responses ordered along a single dimension (likeability). It is highly unlikely that the brain would solve the rating task through a competition mechanism in which each neuronal population would vote for one position on the scale. Thus, observing the same pattern of medial PFC activity across rating and choice tasks suggests that the functional role of this region cannot be reduced to models narrowly applied to the classical case of comparison between two options. It is more compatible with a neural network model [START_REF] Pessiglione | Bridging across functional models: The OFC as a value-making neural network[END_REF] whose function is to generate values (from stimulus features), not to compare them for option selection. As rating and choice tasks both involve valuating the stimuli and selecting the response in which confidence is maximal, it may not be surprising that they share a common representation of stimulus value and response confidence, in the vmPFC and mPFC, respectively. Confidence was the only variable significantly associated to mPFC activity, but was also positively reflected in vmPFC activity, as previously reported [START_REF] Chua | Understanding metamemory: neural correlates of the cognitive process and subjective level of confidence in recognition memory[END_REF][START_REF] Martino | Confidence in value-based choice[END_REF][START_REF] Gherman | Human VMPFC encodes early signatures of confidence in perceptual decisions Donner TH[END_REF]. Indeed, the addition of value and confidence signals in the vmPFC is a pattern that has been already observed in both fMRI and iEEG activity [START_REF] Lebreton | Automatic integration of confidence in the brain valuation signal[END_REF][START_REF] Lopez-Persem | Four core properties of the human brain valuation system demonstrated in intracranial signals[END_REF]. On the contrary, dmPFC activity tended to decrease with confidence, but this trend did not survive significance threshold.

The variable that was robustly associated with dmPFC activity was deliberation time.

This variable was not orthogonal to the others, since it decreased with both stimulus value and response confidence. In some of our analyses, deliberation time was post-hoc orthogonalized with respect to the other variables, meaning that the association with dmPFC activity was observed above and beyond the variance explained by stimulus value and response confidence.

This association alone would not yield a clear-cut interpretation, since many factors may affect response time. However, the systematic link observed between trial-wise dmPFC activation and the increase in pupil size just before the response hints that this association might reflect the cognitive effort invested in the task. Indeed, pupil size has been associated to the intensity of not only physical effort, such as handgrip squeeze [START_REF] Zénon | Comparison between the main GLM (GLM1) where Val is the sum, and an alternative GLM (GLM4) where Val is the difference between option values (Vc -Vuc), for explaining vmPFC activity across the two choice[END_REF] but also mental effort, such as focusing attention to resolve conflict or overcome task difficulty [START_REF] Kahneman | Pupil diameter and load on memory[END_REF][START_REF] Alnaes | Pupil size signals mental effort deployed during multiple object tracking and predicts brain activity in the dorsal attention network and the locus coeruleus[END_REF][START_REF] Van Der Wel | Pupil dilation as an index of effort in cognitive control tasks: A review[END_REF]. By contrast, we did not observe this systematic link with pupil size during deliberation with vmPFC activity. The link between vmPFC and deliberation time might therefore reflect other sources of variance, such as mind-wandering (being slower because of some off-task periods), in accordance with a previous report that elevated baseline vmPFC activity predicts prolonged response time [START_REF] Hinds | Roles of default-mode network and supplementary motor area in human vigilance performance: evidence from real-time fMRI[END_REF]. Regarding dmPFC, our ROI overlaps with clusters that have been labeled dorsal anterior cingulate cortex, or sometimes pre-supplementary motor area, in previous studies [START_REF] Shenhav | The Expected Value of Control: An Integrative Theory of Anterior Cingulate Cortex Function[END_REF][START_REF] Kolling | Value, search, persistence and model updating in anterior cingulate cortex[END_REF][START_REF] Kamiński | Persistently active neurons in human medial frontal and medial temporal lobe support working memory[END_REF]. The association with deliberation time is compatible with a role attributed to this region in the exertion of both physical effort [START_REF] Kurniawan | Effort and Valuation in the Brain: The Effects of Anticipation and Execution[END_REF][START_REF] Skvortsova | Learning To Minimize Efforts versus Maximizing Rewards: Computational Principles and Neural Correlates[END_REF][START_REF] Chong | Neurocomputational mechanisms underlying subjective valuation of effort costs Seymour B[END_REF] and cognitive control [START_REF] Botvinick | Conflict monitoring and cognitive control[END_REF][START_REF] Kerns | Anterior Cingulate Conflict Monitoring and Adjustments in Control[END_REF][START_REF] Sohn | Anticipation of conflict monitoring in the anterior cingulate cortex and the prefrontal cortex[END_REF]. Importantly, this dmPFC region differs from clusters located with the cingulate gyrus that have been more specifically related to physical effort [START_REF] Prevost | Separate Valuation Subsystems for Delay and Effort Decision Costs[END_REF][START_REF] Klein-Flugge | Neural Signatures of Value Comparison in Human Cingulate Cortex during Decisions Requiring an Effort-Reward Trade-off[END_REF].

To recapitulate, we have teased apart the neural correlates of likeability, confidence and deliberation in the medial prefrontal cortex, which have been confused in previous fMRI studies, as shown by meta-analytic maps. The key distinction operated here is perhaps between effort as an attribute of choice option and effort as a resource allocated to solving the task, or in other words, between valuation applied to effort (implicating the vmPFC) and effort invested in valuation (implicating the dmPFC). This dissociation is consistent with the idea that the vmPFC anticipates the aversive value of a potential effort, while the dmPFC represents the intensity of effort when it must be exerted. It could be related to efforts being hypothetical in our design, but previous studies have observed similar effort representation in the vmPFC (not the dmPFC) when efforts were not hypothetical but only implemented later, at the end of the experiment [START_REF] Aridan | Neural correlates of effort-based valuation with prospective choices[END_REF][START_REF] Hogan | Roles of Ventromedial Prefrontal Cortex and Anterior Cingulate in Subjective Valuation of Prospective Effort[END_REF][START_REF] Westbrook | The Subjective Value of Cognitive Effort is Encoded by a Domain-General Valuation Network[END_REF]. At a metacognitive level, our results could be interpreted in the frame of a resource allocation model, where the effort or time invested in the deliberation is meant to increase confidence in the response, whether a rating or a choice [START_REF] Lee | Trading mental effort for confidence in the metacognitive control of valuebased decision-making[END_REF]). Yet our results cannot tell whether the dmPFC signals the need for deliberation effort, monitors the time invested in deliberation, or generates an aversive feeling related to the prolongation of deliberation.

Even if showing robust associations between brain regions and cognitive variables, our approach (looking for robust associations across tasks) also bears limitations. Notably, our design would not allow comparing between conditions, as is traditionally done in neuroimaging studies. One may want for instance to compare between tasks and test whether brain regions are more involved in one or the other, but this would be confounded by several factors, such as the order (choice tasks being performed after rating tasks). A significant contrast would not be interpretable anyway, because there is more than one minimal difference between tasks. Thus, the aim to generalize the role of brain regions across tasks carries the inherent drawback of a limited specificity, but also the promises of a more robust understanding of anatomo-functional relationships. We hope this study will pave the way to further investigations following a similar approach, assessing a same concept across several tasks in a single study, instead of splitting tasks over separate reports, with likely inconsistent conclusions. A] Rating task. In every trial, subjects are shown an item that can be a reward described with both text and image (Rewti), a reward described with text only (Rewt) or an effort described with text only (Efft).

The task for subjects is to rate how much they would like receiving the proposed reward or dislike performing the proposed effort, should it occur, hypothetically, at the end of the experiment. They first move the cursor using left and right buttons on a pad to the position that best reflect their (dis)likeability estimate, then validate their response with a third button and proceed to the next trial.

B] A/B choice task. In every trial, two options belonging to the same category are shown on screen and subjects are asked to select their favorite option, i.e. which reward they would prefer to receive if they were offered the two options or which effort they would prefer to exert if they were forced to implement one of the two options at the end of the experiment (hypothetically). The choice is expressed by selecting between left and right buttons with the index or middle finger. The chosen option is then highlighted in red, and subjects proceed to the next trial.

C] Yes/No choice task. In every trial, one option combining the two dimensions is shown on screen and subjects are asked to state whether they would be willing to exert the effort in order to receive the reward, if they were given the opportunity at the end of the experiment (hypothetically). They select their response ('yes' or 'no', positions counterbalanced across trials) by pressing the left or right button, with their index or middle finger.

Figure 2: Behavioral measures of value, confidence and deliberation.

A] Response rate. For ratings, plots show the average response rate for each bin (portion of the rating scale). Effort items (on the left) are rated between bin 0 ('I would not mind') and bin -10 ('I would dislike it enormously'). Reward items (on the right) are rated between bin 0 ('I would not care') and bin +10 ('I would like it enormously'). Note that the x-axis has been reverted for effort ratings, compared to the visual scale presented in the task, such that it globally indicates increasing values (less aversive effort from -100 to 0 and more appetitive rewards from 0 to +100). For choices, the response rate is plotted as a function of binned decision value (ΔV). In the A /B task, decision value is the difference in likeability rating between left and right options (Vleft -Vright), and choice rate is the frequency of left option being selected. In the Yes/No task, decision value is the addition of weighted reward and effort likeability ratings (βR•VR + βE•VE), which is equivalent to both stimulus value (Val) and to the value difference between yes and no options (net value minus zero). Continuous lines show logistic regression fits of choice rate and dashed lines show variations in the confidence proxy (Conf). Tables Fig. 4-1, 4-2 and 4-3. For clusters obtained using the same GLM without orthogonalization of regressors and using the same GLM with events modeled as boxcar instead of stick functions, see Fig. 5 and Tables 5-1, 5-2, 5-3. For all maps, sagittal slices were taken at the same coordinates as the Neurosynth output (shown in Fig. 4A), and superimposed on the average anatomical scan normalized to canonical (MNI) template. Maps were thresholded at p < 0.05 after voxel-wise family-wise error correction for multiple comparisons.

For all maps, only the main clusters of interest located in the medial prefrontal cortex are shown. For clusters outside the medial prefrontal cortex, please refer to Tables in Figures 5-1, 5-2 and 5-3.

Functional

  and structural brain imaging data was collected using a Siemens Magnetom Prisma 3-T scanner equipped with a Siemens 64 channel Head/Neck coil. Structural T1-weighted images were coregistered to the mean echo planar image (EPI), segmented and normalized to the standard T1 template and then averaged across subjects for anatomical localization of group-level functional activation. Functional T2*-weighted EPIs were acquired with BOLD contrast using the following parameters: repetition time TR = 2.01 seconds, echo time TE = 25 ms, flip angle = 78°, number of slices = 37, slice thickness = 2.5 mm, field of view = 200 mm.

  model (softmax function of decision value). Response confidence (Conf) was defined, by analogy to the rating task, as the square of the difference between choice probability and mean choice rate. Linear regression showed that DT decreased with value in the A/B choice task (Rewti: βVal = -0.06 ± 0.01, p = 3•10 -7 ; Rewt: βVal = -0.06 ± 0.01, p = 3•10 -7 ; Efft: βVal = 0.05 ± 0.01, p = 8•10 -4 ), albeit not in the Yes/No choice task (βVal = 0.033 ± 0.024, p = 0.172). DT also decreased with confidence (Fig.2B) in both the A/B choice task (Rewti: βConf = -1.74 ± 0.20, p = 2•10 -10 ; Rewt: βConf = -1.98 ± 0.18, p = 4•10 -13 ; Efft: βConf = -1.73 ± 0.22, p = 2•10 -9 ) and the Yes/No choice task (βConf = -1.15 ± 0.15, p = 1•10 -9

FiguresFigure 1 .

 1 Figures

Figure 3 :

 3 Figure 3: Validation of the confidence proxy (Conf). Our proxy for confidence (Conf = square of centered likeability rating or choice likelihood) was tested against confidence ratings collected in independent datasets. Left panel: in the likeability rating task (Lopez-Persem et al., 2020), participants first rated the likeability of food, face and painting items and then provided a confidence rating about their own likeability judgment. Right panel: in the A/B binary choice task (Lee and Daunizeau, 2020), participants selected their preferred item between options shown in pairs, and then provided a confidence rating about having made the best choice. The graphs show confidence rating as a function of binned Conf. Dots represent means over participants, error bars are inter-participant standard errors, dotted lines show linear regression fits.

Figure 4 :

 4 Figure 4: Neural mapping of value, confidence and deliberation. A] Meta-analysis of fMRI studies. Statistical maps (sagittal slices) were extracted from the Neurosynth platform with the 'value', 'confidence' and 'effort' keywords. Significant clusters in the medial prefrontal cortex are similar across keywords, being located in both ventral and dorsal regions. B] Neural correlates of value, confidence and deliberation constructs in the present dataset (in red, blue, and green, respectively). Statistical maps were obtained with a GLM including the different variables as parametric modulators of stimulus onset, across rating and choice tasks. Sagittal slice was taken at the same coordinates as the Neurosynth output, and superimposed on the average anatomical scan normalized to canonical (MNI) template. Coronal slices show the extent of the different medial prefrontal clusters. Statistical threshold was set at p < 0.05 after family-wise error for multiple comparisons at the voxel level. For clusters outside the medial prefrontal cortex, see activations in

Figure 5 :

 5 Figure 5: Neural mappings of value, confidence and deliberation obtained with alternative GLM. A] Statistical map (same as in Fig. 4B) obtained with the main GLM is shown for comparison. B] Statistical map obtained with the same GLM when serial orthogonalization was removed.C] Statistical map obtained with the same GLM when events were modeled with a boxcar function encompassing the period from trial onset to first button press.

Figure 6 :

 6 Figure 6: Neural representations of value, confidence and deliberation across behavioral tasks A] Distribution of regression estimates (inter-subject means ± standard errors) obtained for Val, Conf and DT variables along a ventro-dorsal line within the medial prefrontal cortex (sampled in each 8mmradius shown on the average anatomical map). Colored circles show sampled spheres in which correlation with the corresponding variable was maximal (Valred, Confblue and DTgreen). B] Decomposition of regression estimates obtained for each variable of interest, plotted separately for rating and choice tasks (noted R and C) and for the different ROI (vmPFC, mPFC, dmPFC). C] Decomposition of regression estimates obtained for each variable of interest (Val, Conf and DT), plotted separately for each choice task (noted A/B and Y/N) in the different ROI (vmPFC, mPFC, dmPFC). For the three regionvariable associations, there was no significant difference between regression estimates obtained in the A/B and Yes/No choice tasks. D] Regression estimates were extracted across rating and choice tasks, separately for rewards presented as text (Rewt) or text + image (Rewti) and separately for reward (Rew) and effort (Eff) values. The vmPFC ROI was based on group-level cluster activated with Val using GLM1, following a leave-one out procedure to avoid double dipping. E] Regression estimates were extracted from the vmPFC (group-level cluster associated to Val), using a GLM where Val, Conf and DT were replaced by the chosen and unchosen option values (Vc and Vuc), across the two choice tasks. In more details, Vc / Vuc were Vleft / Vright for a left choice in the A/B task, and βRew•VRew + βEff•VEff / 0 for a yes choice in the Yes/No task (and vice-versa for opposite choices).

Figure 7 :

 7 Figure 7: Pupillometric validation of the link between brain activity and deliberation effort. Plots show the time course of regression estimates, obtained with a GLM built to explain pupil size. The GLM included factors of no interest (jitter duration, stimulus luminance, text length), variables of interest (Val, Conf, DT) and activities in main ROI (vmPFC, mPFC, dmPFC, corresponding to red, blue and green traces, respectively). Each row corresponds to a different task (likeability rating, choice tasks). Left and right columns show time courses aligned onto stimulus onset and button press, respectively. Lines represent means across participants and shaded areas inter-participant standard errors. Horizontal bars indicate significant time clusters after correction for multiple comparisons using random-field theory.

Acknowledgments

We would like to thank the CENIR (research neuroimaging center) staff for their help in fMRI data acquisition (particularly Stéphane Lehéricy, Romain Valabrègue and Mathieu Santin for the optimization of scanning sequences), Chen Hu for assistance in data collection, Jules Brochard for assistance in data analysis, Fabien Vinckier and Jean Daunizeau for insightful comments. The study was funded by a research grant from the "Fondation pour la Recherche Médicale" and by the "Investissements d'avenir" program (ANR-10-IBHU-0003).

Conflict of interest statement

The authors declare no competing financial interests. B] Deliberation time as a function of confidence proxy (Conf), defined as the square of centered likeability rating (V 2 ) for rating tasks and the square of centered choice likelihood (P 2 ) for choice tasks.

The Conf proxy was validated in two different datasets where confidence in rating or choice was directly asked to participants (see Fig. 3). Dots represent mean across participants, x and y error bars are inter-participant standard errors. Regions survived a significance threshold of P < 0.05 after FWE correction for multiple comparisons at the voxel level. Clusters smaller than 12 voxels, corresponding to the size of our smoothing kernel, were excluded from the table. Coordinates refer to the MNI space. The p-value reported is the p-value of the cluster after a FWE correction at the cluster level.

Extended Data -Tables


Yassine Hmamouche
email: yassine.hmamouche@imt-atlantique.fr

Yehya Nasser
email: yehya.nasser@imt-atlantique.fr

Amer Baghdadi
email: amer.baghdadi@imt-atlantique.fr

Marc-Oliver Pahl
email: marc-oliver.pahl@imt-atlantique.fr

Key

la diffusion de documents scientifiques de niveau

Keywords:

recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction: DNNs are enabling major advances in solving hard scientific problems and processing complex data on an unprecedented scale in many areas such as language processing, fraud detection, healthcare, and so on. The design of DNNs for commercial services requires a significant expenditure of time, money, and human effort, from collecting massive data to fine-tuning the model's hyperparameters. Thus, the commercial value of these models makes them important intellectual property for companies, which incentivizes adversaries to mount specific attacks in order to retrieve their internal intelligence, gain knowledge about the sensitive information being processed by them, or at least disrupt their operation by intentionally injecting specific vulnerabilities [START_REF] Akhtar | Threat of adversarial attacks on deep learning in computer vision: A survey[END_REF].

These DNN models are typically kept confidential in a black-box setup where the adversary is not privy to the structure or parameters of the model, other than the output predictions for the corresponding input. This setup has been commercialized by several cloud service providers such as Google, Amazon, Microsoft, and BigML, which have deployed an end-to-end infrastructure for using DNNs as a service. Despite its promise, the commercial value of DNNsas-a-service makes them susceptible to critical attacks by adversaries. Our proposed taxonomy of attacks is illustrated in Fig. 1. They are categorized based on several attributes, which are grouped into three classes of the attacker's knowledge and capabilities. First, Black-Box DNNs (a.k.a. the oracle model) requiring no knowledge about the model beyond the ability to query it by inserting an input and obtaining the output classification. Second, Gray or White-Box DNNs, in which the attacker may have some degree of information about the architecture of the model, the number of layers, the number of neurons per layer, and even the hyperparameters used to train the model. Third, Explainable DNNs. In fact, the use of black-box DNNs is not without risks as endorsed by their proven track record of unfair and wrong decisions. To address these risks, the European parliament has an explicit provision requiring meaningful explanations to provide users with deeper insights into model reasoning and about the data. These explanations can contain, however, hidden sensitive knowledge that can be exploited for privacy attacks [START_REF] Zhao | Exploiting explanations for model inversion attacks[END_REF].

The above three classes are then further separated into two sets of attack objectives. First, Confidentiality that consists in exploiting specific data leakage disclosed intentionally (e.g., saliency maps, global explanations) and/or unintentionally (e.g., power, timing, and electromagnetic traces) in order to acquire proprietary information by querying the DNN system. This attack's objective is represented in Fig. 1 by arrows coming out of the DNN model. Second, Integrity that consists of intentionally injecting a perturbation, so that the DNN system fails to perform correctly for some or all of the inputs (e.g., causing a DNN-based malware classifier to misclassify a malware sample as benign). This misclassification may be non-targeted where the erroneous output is assigned to any class or targeted where it is assigned to a specific class. This attack's objective is represented in Fig. 1 by arrows entering the DNN model.

In the sequel, we will outline five key generative strategies for attacking black-box DNNs.

Strategy 1: This poisoning strategy [START_REF] Akhtar | Threat of adversarial attacks on deep learning in computer vision: A survey[END_REF] assumed that the attackers have the ability to contribute to the training data or control it. In this way, the attacker aims to poison the training data by injecting crafted malicious samples to influence the outcome of the model training. There are many simple yet effective techniques for attacking a training process. For instance, the attacker can flip labels of some training instances. Alternatively, we can inject legitimately labeled poison samples that seem normal to a user, but they comprise illegitimate characteristics to trigger a targeted misclassification during the inference process. An attacker can also generate an illegitimate trained DNN that exhibits high classification accuracy on the user's private validation set in an attempt to gain its trust, but the malicious DNN performs incorrectly on backdoor instances.

Strategy 2: This invasion attack [START_REF] Akhtar | Threat of adversarial attacks on deep learning in computer vision: A survey[END_REF] aims to induce vulnerabilities in the inference phase in such a way that the changes are almost invisible to the human eye, but very noticeable from the DNN viewpoint. Typical attacks may involve modifying the malicious sample's features to evade detection by the model. Effective attacks are particularly based on the gradient method where attackers modify the original input in the direction of the gradient of the loss function relative to the clean input image. The small vector noise η is generated as

η = ǫ sign (∇ x J (θ, x, y)) , (1)
where sign(:) is the sign function, ǫ is a small scalar value that restricts the norm of the perturbation, x denotes a vectorized clean input, and y is the label of the input. In addition, J (:, :, :) is the cost used to train the neural network and ∇ x is the gradient of the loss function relative to the clean input x.

Black-box DNNs

Inference phase

Non-targeted attacks Targeted attacks Strategy 3: Here [START_REF] Duddu | Stealing neural networks via timing side channels[END_REF], we search for a substitute model with functionality relatively close to the target black-box DNN (i.e., within 5% test accuracy of the target model). To make the search tractable and efficient, the adversary reduces the search space by considering the timing side-channels in the black-box setting due to the dependence of the total execution time on the total number of layers (i.e., DNN depth). From the total execution time, an adversary can infer the total number of layers of the target DNN using a regressor trained on the data containing the variation of execution time with DNN depth of several popular models (e.g., AlexNet, ResNet, Inception). To efficiently search for the optimal Neural Network, an optimisation problem is formulated and solved using Reinforcement Learning based Neural Architecture Search. Strategy 4: A distinctive approach is adopted in this strategy [START_REF] Batina | CSI neural network: using side-channels to recover your artificial neural network information[END_REF], which involves using side channels to attack the building block of advanced black-box setups, namely multilayer perceptrons consisting for instance of fully connected layers commonly found in other advanced setups such as CNNs. In this way, a combination of temporal and electromagnetic leakage is used to recover key parameters, i.e., the activation function, the pre-trained weights, the number of hidden layers and neurons in each layer. This approach is inspired by attacks against cryptosystems. Here, the DNN weights are represented on 32 bits according to the IEEE 754 standard, where each byte is recovered individually according to the divide-and-conquer approach. Two key approaches are identified for this strategy. First, Profiled Attacks, wherein the adversary can procure a copy of the victim device and uses it to extract extensive leakage traces (in order to distinguish different templates) given hypothesis on sensitive data (e.g., DNN weights). Next, the physical leakage of the victim device is compared to the prior templates in order to determine the most probable profile. Second, Non-Profiled Attacks, wherein the attacker has only limited access to the target device such that the similarity between measurements of the physical leakage at the victim device and a hypothetical model is quantified statistically based on specific distinguishers like Pearson's correlation and mutual information.

2 1

Intentional fault injection attacks

Gray/white-box DNNs

Profiled attacks Non-profiled attacks

Explainable DNNs

Physical leakage analysis attacks Model inversion attacks

Strategy 5: Recent research has identified sophisticated attacks [START_REF] Zhao | Exploiting explanations for model inversion attacks[END_REF] against sensitive knowledge hidden in explainable DNNs, such as model extraction attacks to reconstruct parameters of proprietary models and infer the original data from the target prediction (e.g., reconstructing a face from an emotion prediction), membership inference attacks vulnerable to inference phase to identify if users were part of a training dataset, and explanation manipulation attacks leveraging posthoc explanations techniques to give the impression that the black-box model exhibits some fair behavior (e.g., no discrimination) while it might not be the case.

Conclusion:

In this paper, we examined to what extent and under what settings the confidentiality and integrity of black-box DNNs-which are the most challenging setup of DNNs-can be threatened. In this way, we proposed a comprehensive taxonomy of the key strategies developed in the literature to attack black-box DNNs. We believe that a coherent classification incorporating all key aspects is needed to organize the body of knowledge on research and methodologies for understanding and securing black-box DNNs.