Jiawei Liu

Chen Qu

Rémi Phillipe

Siying Li

Edmund Derrington

Brice Corgnet

Jean-Claude Dreher
email: dreherjeanclaude@gmail.com

Oxytocin modulates the neurocomputational mechanisms engaged in learning social

Keywords: Reinforcement learning, model-based fMRI, Social hierarchy, Self-other distinction, Oxytocin. Oxytocin modulates the neurocomputational mechanisms engaged

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Dominance hierarchies, characterized by ranking systems based on the relative social power of individuals within a group, are a major evolutionary force that drive dyadic asymmetries. Dominance hierarchies may be learned incrementally, by accumulating feedback on relationships to specific members of a social group. The learning of hierarchical status can be framed as a reinforcement learning (RL) problem, inspired by neurocomputational approaches traditionally applied to non-social cognition, both when learning dominance relationships by observation of other interacting individuals [START_REF] Kumaran | Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanisms for Updating and Representing Self-Relevant Information[END_REF], or by direct dyadic competitive interactions between oneself and another individual [START_REF] Ligneul | Dynamical Representation of Dominance Relationships in the Human Rostromedial Prefrontal Cortex[END_REF][START_REF] Qu | An Integrative Interdisciplinary Perspective on Social Dominance Hierarchies[END_REF]. The medial prefrontal cortex (mPFC) and the amygdala/hippocampus may play distinct computational roles during and after learning 'linear' (r 1 <r 2 <…<r k ) social hierarchies. During learning, the mPFC (both dorsal and ventral parts) has been shown to compute the monitoring and updating of dominance status. These are fundamental signals needed to make choices between two individuals when learning their social dominance relationship by trial and error [START_REF] Janet | Regulation of social hierarchy learning by serotonin transporter availability[END_REF][START_REF] Kumaran | Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanisms for Updating and Representing Self-Relevant Information[END_REF][START_REF] Ligneul | Dynamical Representation of Dominance Relationships in the Human Rostromedial Prefrontal Cortex[END_REF]. In contrast, when required to infer the relationship between two non-adjacent members in a linear hierarchy, while only adjacent relationships have been learned, the amygdala-hippocampal complex is necessary to retrieve learned relationships between other members of the social structure from social memory [START_REF] Kumaran | The Emergence and Representation of Knowledge about Social and Nonsocial Hierarchies[END_REF][START_REF] Kumaran | Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanisms for Updating and Representing Self-Relevant Information[END_REF].

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d 3 Previous studies have considered social hierarchies as a series of relationships whereby individuals are ranked successively in a linear fashion [START_REF] Jensen | Discovering Implied Serial Order Through Model-Free and Model-Based Learning[END_REF][START_REF] Kumaran | The Emergence and Representation of Knowledge about Social and Nonsocial Hierarchies[END_REF][START_REF] Kumaran | Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanisms for Updating and Representing Self-Relevant Information[END_REF]. However, social hierarchy in real life is often not a simple linear structure.

Real hierarchies often include individuals of equal social ranks. The first question we addressed is how such non-linear 'diamond-shaped' hierarchies might be learned and represented in the brain (Fig. 1). We characterize the computations and brain systems engaged when making transitive inferences, after learning a diamond-shaped hierarchy by repeated learning trials between pairs of people with feedback. These transitive inferences were performed in the absence of corrective feedback, while participants viewed pairs not encountered during training. Such transitive inferences require social memory of the previously learned paired members, to infer the relative ranks of members, based on partial information provided during the learning trials.

Oxytocin (OXT) is an evolutionarily conserved neurohypophysial hormone, engaged in lactation and parenting. The function of this hormone might have been sculpted during evolution to facilitate the expression of adaptive social behaviors, as required for forming hierarchical relationships [START_REF] Lee | Social status in mouse social hierarchies is associated with variation in oxytocin and vasopressin 1a receptor densities[END_REF]. There is evidence in non-human primates and rodents that OXT modulates social hierarchy. For example, OXT suppresses vigilance towards faces of dominant males in the rhesus macaque [START_REF] Chang | Oxytocin and social cognition in rhesus macaques: Implications for understanding and treating human psychopathology[END_REF][START_REF] Ebitz | Oxytocin blunts social vigilance in the rhesus macaque[END_REF] and relaxes social interactions between monkeys, thereby flattening the social hierarchy [START_REF] Jiang | Oxytocin and vasopressin flatten dominance hierarchy and enhance behavioral synchrony in part via anterior cingulate cortex[END_REF]. In mice, socially dominant males have higher OXT receptor binding compared to subordinates, in brain regions such as the amygdala [START_REF] Lee | Social status in mouse social hierarchies is associated with variation in oxytocin and vasopressin 1a receptor densities[END_REF]. This effect may reflect that the different social environments experienced by dominant and subordinate animals may alter receptor expression, and potentially facilitate adaptive social behaviors.

Here, we focused on the neurocomputational processes that take place while learning nonlinear (diamond-shaped) social hierarchies by observation. We used model-based functional magnetic resonance imaging (fMRI), and an extension of a two-phase social hierarchy learning paradigm. Participants learned hierarchical ranks by trial and error during a training phase (Fig. 1). Then, during a test phase, they made transitive inferences of relative hierarchical rank between pairs of individuals who were never encountered before. We developed a new reinforcement learning model to account for social hierarchy This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d learning when it is necessary to learn both equal ranks and different ranks and investigated how OXT administration might impinge on the model parameters.

We tested whether OXT plays a specific role in either phase of our new social hierarchy learning paradigm while: (a) learning social ranks by reinforcement during the training phase and (b) encoding and retrieval of social memories, required to make transitive inferences between learned members of the hierarchical structure during the test phase.

We also compared how OXT administration might affect these processes with respect to self-and other-orientated hierarchies (i.e., when oneself is included or not in the hierarchy) (Fig. 1C).

During training, we hypothesized that learning non-linear social hierarchies engaged computational mechanisms and brain representations similar to those engaged in learning one-dimensional linear social hierarchies by observation. Specifically, we hypothesized that the mPFC, engaged in learning linear social hierarchy [START_REF] Janet | Regulation of social hierarchy learning by serotonin transporter availability[END_REF][START_REF] Kumaran | Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanisms for Updating and Representing Self-Relevant Information[END_REF][START_REF] Ligneul | Dynamical Representation of Dominance Relationships in the Human Rostromedial Prefrontal Cortex[END_REF], will also be engaged when learning diamond-shaped social hierarchies. Preliminary evidence suggests that OXT may modulate distinct neurocomputational mechanisms engaged in reinforcement learning, including selection between different options during choice behavior [START_REF] Piva | An integrated framework for the role of oxytocin in multistage social decisionmaking[END_REF] or prediction error, as previously suggested in the domain of prosocial behavior [START_REF] Clark-Elford | The effects of oxytocin on social reward learning in humans[END_REF][START_REF] Hu | Oxytocin selectively facilitates learning with social feedback and increases activity and functional connectivity in emotional memory and reward processing regions[END_REF][START_REF] Hung | Gating of social reward by oxytocin in the ventral tegmental area[END_REF][START_REF] Hurlemann | Oxytocin enhances amygdala-dependent, socially reinforced learning and emotional empathy in humans[END_REF][START_REF] Ide | Oxytocin attenuates trust as a subset of more general reinforcement learning, with altered reward circuit functional connectivity in males[END_REF][START_REF] Kruppa | Neural modulation of social reinforcement learning by intranasal oxytocin in male adults with highfunctioning autism spectrum disorder: a randomized trial[END_REF][START_REF] Liao | Intranasal oxytocin decreases self-oriented learning[END_REF][START_REF] Martins | Oxytocin modulates neurocomputational mechanisms underlying prosocial reinforcement learning[END_REF][START_REF] Zhuang | Oxytocin-induced facilitation of learning in a probabilistic task is associated with reduced feedbackand error-related negativity potentials[END_REF]. Therefore, we hypothesized that OXT could modulate both action selection policy, reflecting the probability of choosing the correct rank of a given individual and updating of social ranks at the time of outcome.

During the test phase of the task, social memory processes are engaged when making transitive inferences between members of the learned structure (pairs of individuals who were never encountered together before). Previous studies in mice indicate that OXT regulates social memory [START_REF] Baram | Entorhinal and ventromedial prefrontal cortices abstract and generalize the structure of reinforcement learning problems[END_REF][START_REF] Ferguson | Social amnesia in mice lacking the oxytocin gene[END_REF][START_REF] Lee | A Conditional Knockout Mouse Line of the Oxytocin Receptor[END_REF][START_REF] Macbeth | Oxytocin and the oxytocin receptor underlie intrastrain, but not interstrain, social recognition[END_REF][START_REF] Raam | Hippocampal oxytocin receptors are necessary for discrimination of social stimuli[END_REF][START_REF] Wirth | Territorial blueprint in the hippocampal system[END_REF]. We thus hypothesized that OXT affects social memory also in humans and could modulate responses of the amygdala/hippocampal region, which are known to be engaged when making transitive inferences between items [START_REF] Kumaran | Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanisms for Updating and Representing Self-Relevant Information[END_REF]. Finally, we tested whether OXT This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d modulates social memory and its neural correlates differently for social hierarchy structures in which participants were embedded or not (Self vs Other-orientated hierarchy). This is because well-functioning social interactions may require differential other-related and self-related information processing [START_REF] Bang | Neurocomputational mechanisms of confidence in self and others[END_REF][START_REF] Ereira | Social training reconfigures prediction errors to shape Self-Other boundaries[END_REF][START_REF] Wittmann | Self-Other Mergence in the Frontal Cortex during Cooperation and Competition[END_REF] and because OXT differently modulates value computation of self-other allocations [START_REF] Chang | Inhaled oxytocin amplifies both vicarious reinforcement and self reinforcement in rhesus macaques (Macaca mulatta)[END_REF][START_REF] Chang | Oxytocin and social cognition in rhesus macaques: Implications for understanding and treating human psychopathology[END_REF][START_REF] Liao | Intranasal oxytocin decreases self-oriented learning[END_REF] and in-group/out-group decisions [START_REF] De Dreu | The Neuropeptide Oxytocin Regulates Parochial Altruism in Intergroup Conflict Among Humans[END_REF].

Results

Behavioral results

We fitted two generalized linear mixed-effects models (GLMM) to analyze how performance of learning social ranks during the training phase (GLMM1) or making transitive inferences during the test phase (GLMM2) varied according to the Status of the members (Superior, Intermediate and Inferior), Hierarchy Orientation (Self: participant at the center of the hierarchy, labelled 'Self' versus Other: another participant at the center of the hierarchy, labelled 'Other'), 9 Blocks, Group (Oxytocin versus Placebo) and pertinent interaction factors.

During the training trials, participants performed better on trials comparing hierarchy members with Superior (odds ratio = 3.392, p < 0.001) and Inferior (odds ratio = 3.156, p < 0.001) status, compared to trials involving hierarchy members of equal status (Fig. 2A).

A post-hoc test showed that participants also performed better on trials involving Superior than Inferior hierarchy members (odds ratio = 1.331, SE = 0.06, p < 0.001, z = 6.316). As the number of blocks increased, participants improved their performance (odds ratio = 1.433, p < 0.001). An interaction effect of Group × Block further showed that participants in the Placebo group were more likely than those in the Oxytocin group to improve performance over consecutive blocks (odds ratio = 0.966, p = 0.042, Fig. 2B). We also found participants learned better in the Self-than Other-orientated hierarchy (odds ratio = 1.237, p < 0.001, Fig. 2C). There was a significant interaction effect of Hierarchy Orientation (Self/Other) x Status (Superior: odds ratio = 0.543, p < 0.001; Inferior: odds ratio = 0.616, p < 0.001). A post-hoc test revealed that participants performed better in This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d 6 Self-orientated Hierarchy than the Other-orientated Hierarchy condition only for trials involving Intermediate status (odds ratio = 1.783, SE = 0.183, p < 0.001, z = 5.646). With regard to results of response time (RT) analyzed by LMM1 during the training trials, RTs were longer for trials involving both Superior (b = 0.032, p = 0.008) and Inferior (b = 0.128, p < 0.001) hierarchy elements than trials involving the elements of Intermediate status (Fig. 2A, right). A post-hoc test further revealed that RTs was shorter for trials involving Superior elements than Inferior (b = -0.096, SE = 0.009, p < 0.001, z = -10.529).

These results indicate that participants' RTs are consistent with a speed-accuracy tradeoff.

Moreover, RTs gradually reduced over consecutive trial blocks, indicating faster responses as the hierarchies were learned better (b = -0.045, p < 0.001, Fig. S1A). The Oxytocin group also showed quicker reduction of RTs than the Placebo group over blocks (b = -0.022, p < 0.001, Fig. S1A). In addition, RTs reduction across blocks was faster in the Self than the Other-orientated condition (b = -0.007, p = 0.034, Fig. S1B), suggesting people learn faster when they are included in the hierarchy.

In test trials, participants improved performance over consecutive blocks (GLMM2, odds ratio = 1.373, p < 0.001). The Oxytocin group improved accuracy faster across blocks than the Placebo group (odds ratio = 1.04, p = 0.023, Fig. 2D). The interaction effect of Group × Hierarchy Orientation showed that the Oxytocin group outperformed the Placebo group in the Self-orientated hierarchy compared to Other-orientated hierarchy (odds ratio = 1.262, p = 0.005, Fig. 2E). Analysis of RT by LMM2 revealed that participants were slower in the Self-orientated hierarchy condition compared to Other-orientated hierarchy (b = 0.09, p < 0.001). Moreover, participants decreased their RT faster across blocks in the Self-orientated hierarchy compared to the Other-orientated condition (b = -0.014, p < 0.001). The Oxytocin group also decreased RT faster over blocks than the Placebo group (b = -0.015, p < 0.001). See supplementary Table S1 & S2 for complete behavioral results.

Computational model results

We tested 5 models accounting for how learning the item values in the hierarchy structure.

Model 1 was the RL-ELO model (Model 1) which has been shown to be successful to account for learning linear hierarchies [START_REF] Kumaran | Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanisms for Updating and Representing Self-Relevant Information[END_REF]. Model 2 was a modified RL-ELO model with two different learning rates for hierarchically superior and inferior members of the hierarchy. Model 3 was the classical Rescorla-Wagner (RW) model and This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d model 4 was the Value Transfer model, which are known to fail in learning linear hierarchies as they associate equally with positive and negative outcomes [START_REF] Kumaran | Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanisms for Updating and Representing Self-Relevant Information[END_REF]. We also developed a new RL-ELO model, called the Combined RL-ELO-based associative learning model (Model 5), which enables to learn hierarchical structures involving equal ranks (i.e., non-linear 'diamond shape'). Because learning equal ranks may involve different strategies than learning unequal ranks, we assumed that learning equal ranks was equivalent to memorizing matching pairs. For this reason, we propose a model in which learning occurs in two steps. First, an associative learning rule is updated according to the fact that ranks are equal or unequal. Second, the value of each item in the presented pair is updated according to the past choice and reward. That is, if ranks between items are unequal, the value of the highest (or the lowest) item in the hierarchy is increased (or decreased respectively). If items in the pair were equal, their values became closer according to a proportion of their initial distance (See also computational modeling section in methods). All models were fitted separately for choices by each individual using Matlab VBA-toolbox [START_REF] Daunizeau | VBA: a probabilistic treatment of nonlinear models for neurobiological and behavioural data[END_REF]. Protected exceedance probability (PXP) was used for model selection. It accounts for the probability that the considered model is more frequent in the studied population, which can be used as a quantitative measure of the amount of evidence for the best model [START_REF] Daunizeau | VBA: a probabilistic treatment of nonlinear models for neurobiological and behavioural data[END_REF]. The Self-and the Other-orientated conditions were fitted separately to allow participants to learn differently between conditions.

Comparisons between models showed that the best model was the RL-ELO-based associative learning model (Model 5: PXP = 1; Fig. 3D), with a single alpha learning rate to update item values for each of the two hierarchy orientations (Self and Other). The fit of this model with the performance results are plotted over blocks for correctly selected items during training (Fig. 3A) and test blocks (Fig. 3B). The mean evolution of each item power (Q (i)t ) across all subjects is shown in Fig. 3C. Finally, we ran a 2-way mixed ANOVA analysis on the parameters of the RL-ELO-based associative learning model according to hierarchy orientation (Self/Other: within-subject factor) and treatment group (Oxytocin/Placebo: between-subject factor) (See supplementary Table S7). We observed significant effects only for the learning rate 𝛼 𝑒𝑞 used to update choices of members of equal rank. That is, for this learning rate 𝛼 𝑒𝑞 , we observed a main effect of hierarchy orientation (F (1,56) = 7.42, p = 0.009, η 2 = 0.04; Self: Mean ( 𝛼 𝑒𝑞) = 0.10, Other: Mean ( 𝛼 𝑒𝑞) = This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d 0.18; Fig. 3E) and a marginally significant Group effect (F (1,56) = 3.53, p = 0.065, η 2 = 0.003; OXT: Mean ( 𝛼 𝑒𝑞) = 0.10, Placebo: Mean ( 𝛼 𝑒𝑞) = 0.18), but no interaction effect (F (1,56) = 0.02, p = 0.89, η 2 = 0.001).

Brain regions engaged in learning social hierarchy

As the Combined RL-ELO-based associative learning model can predict the participants' behavior during social hierarchy learning, we fitted parametric trial-by-trial regressors of the model-estimated probability that a participant chooses the correct response as a proxy of the knowledge of the hierarchical structure (GLM1). This probability of choosing the correct answer was added as a parametric regressor at the time of presentation of the two faces. When collapsing over the Placebo and Oxytocin groups, a whole brain analysis indicated that the ventromedial prefrontal cortex (vmPFC), posterior cingulate cortex (PCC) as well as the temporal gyrus, parietal lobule and occipital cortex encoded the probability of choosing the correct answer during training (Fig. 4A; also see Supplementary Table S4 for details of activated regions). Subsequently, we carried out region-of-interest (ROI) analyses in vmPFC and PCC, based on independent coordinates from a meta-analysis related to social learning and self-referential processing [START_REF] Bartra | The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value[END_REF]. Beta estimate values were extracted from the interaction between hierarchical orientation (Self/Others) and Group treatment during the learning trials, and a follow-up mixed ANOVA was performed. Results showed a significantly stronger neural response in the Oxytocin than the Placebo group in both ROIs (vmPFC: F (1,56) = 4.06, p = 0.0488, η 2 = 0.033, Fig. 4B; PCC: F (1,56) = 4.66, p = 0.035, η 2 = 0.023, Fig. 4C).

In addition, the prediction error (PE), which reflects the difference between expected and actual rewards, was encoded at the outcome feedback in training trials by robust activity in the vmPFC and the ventral putamen bilaterally (GLM2, Fig. 5A). However, none of these regions showed significant differences between Oxytocin and the Placebo groups (illustrated in Fig. 5B, 5C & 5D).

Brain regions engaged in social memory when making transitive inferences

During the test phase, a whole brain fMRI data analysis (GLM3, using the probability of choosing the correct response computed by the RL-ELO associative model) showed

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d engagement of a brain network including the mPFC, the anterior and posterior cingulate cortex and the temporal cortex (Fig. 6A). Moreover, comparison in the test phase between hierarchy orientation (Other > Self) revealed stronger activity in the dorsal anterior cingulate cortex (dACC, Fig. 6B & 6C), the right inferior parietal lobule (IPL) and in the precentral gyrus. We also examined putative differences between groups in vmPFC and PCC ROIs. However, no significant difference between placebo and Oxytocin groups were observed (vmPFC: t = 2, df = 56, p = 0.1; PCC: t = -1, df = 56, p = 0.3).

Finally, because the behavioral analyses indicated an interaction effect between Group and Hierarchy Orientation during the test phase (Fig. 2E), we investigated the brain correlates of this behavioral effect (GLM4). This analysis revealed an interaction with the same pattern between Group and Hierarchy Orientation (Fig. 7A), observed in the left amygdala (Fig. 7B) and in the right parahippocampal region (Fig. 7C). BOLD response in these regions was stronger during the Self-than Other-orientated hierarchy trials in the Placebo group, whereas in the Oxytocin group activity was stronger for the Other-than for the Self-orientated hierarchy.

Discussion

The present study investigated how intranasal OXT modulates the neurocomputational mechanisms subserving learning and remembering diamond-shaped social hierarchies in which participants were or not embedded. Our results provide mechanistic insights into how OXT modulates the neurocomputational signals determining action selection policy during learning and social memory recall when making transitive inferences in the test phase. OXT increased activity of the vmPFC and PCC related to action selection policy, reflecting the probability of identifying the correct rank relationship between two individuals (i.e., equal or unequal ranks in the non-linear 'diamond shape' structure).

During the test phase, participants had to decide, by making transitive inferences, whether two items were of equal or unequal ranks. In this test phase, OXT led to improved performance for the Other-orientated hierarchy, and reduced performance for the Selforientated hierarchy. This interaction between group (OXT, Placebo) and Self/Otherhierarchy orientation was associated with higher activity in the amygdala and the parahippocampal region. Together, these findings characterize how OXT modulates This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d 10 neurocomputational signals engaged in learning social hierarchies and in remembering learned social relationships.

Social dominance hierarchies are a common form of social organization that requires individuals to learn and remember their status relative to others. Once relative status in a given structure has been learned, individuals maintain consistent patterns of social behavior (e.g. competitive or submissive) based on their past social interactions [START_REF] Chase | Self-Structuring Properties of Dominance Hierarchies: A New Perspective[END_REF][START_REF] Williamson | Temporal dynamics of social hierarchy formation and maintenance in male mice[END_REF]. Unlike previous studies on linear social hierarchy structure [START_REF] Kumaran | The Emergence and Representation of Knowledge about Social and Nonsocial Hierarchies[END_REF][START_REF] Vasconcelos | Mini-review Transitive inference in non-human animals: An empirical and theoretical analysis[END_REF], we developed a diamond-shaped social hierarchy task in which relative ranks could be equal. Our combined RL-ELO-based associative learning model captured the learning of such hierarchies. This model considers learning social hierarchy as an incremental process accumulating feedback concerning members of the social group, in which one may or may not be embedded. Our RL-ELObased model reflects dynamic learning of social relationships. This was done in a twostage process. Participants updated the associative value representing the learned association between the pairs and the action of choosing equality between members, given past exposure to this pair. Then, participants updated their representations of the hierarchy structure by correcting the value of each item of the pair according to past choices and rewards. This model allowed us to identify the medial PFC and PCC as implementing the action selection policy. Moreover, the PE engaged the ventral putamen and the vmPFC during the feedback outcome, extending the classical PE signal to learning of social relationships (30,38,39). Recent computational models and experimental studies have proposed that the same brain representations that map space can be extended to a broad range of non-spatial problems in abstract cognitive space [START_REF] Kurth-Nelson | Temporal structure in associative retrieval[END_REF].

These studies support that the mPFC and the hippocampus are involved in non-spatial relational memory tasks, allowing them to make transitive inferences [START_REF] Boorman | Cognitive maps and novel inferences: a flexibility hierarchy[END_REF][START_REF] Dayan | Improving Generalization for Temporal Difference Learning: The Successor Representation[END_REF][START_REF] Sharpe | Dopamine transients are sufficient and necessary for acquisition of model-based associations[END_REF] and to organize the relationships between entities (e.g., places, objects, people, events, or abstract task states) to afford generalization and flexible decision making. Our results extend these findings and explain how abstract structural relationships such diamond-shaped social hierarchies are learned at the computational level.

Here, we characterize the effect of OXT in dynamic learning of social hierarchy at the neurocomputational level. Our findings demonstrate distinct computational roles of OXT during the training and test phases of the task. During training, OXT modulates the neurocomputational signals needed for implementing an action policy to choose the correct answer. OXT did not modulate learning rate per se, nor PE-related brain activity, but modulated the brain regions computing a choice policy during training. There is mixed evidence for a role of OXT on reinforcement learning in humans. Some studies report that OXT may facilitate reinforcement learning [START_REF] Hu | Oxytocin selectively facilitates learning with social feedback and increases activity and functional connectivity in emotional memory and reward processing regions[END_REF][START_REF] Hurlemann | Oxytocin enhances amygdala-dependent, socially reinforced learning and emotional empathy in humans[END_REF][START_REF] Zhuang | Oxytocin-induced facilitation of learning in a probabilistic task is associated with reduced feedbackand error-related negativity potentials[END_REF] and modulate computational signals underlying prosocial reinforcement learning [START_REF] Martins | Oxytocin modulates neurocomputational mechanisms underlying prosocial reinforcement learning[END_REF]. However, other studies report that OXT reduces the encoding of PE during in an iterative trust game [START_REF] Ide | Oxytocin attenuates trust as a subset of more general reinforcement learning, with altered reward circuit functional connectivity in males[END_REF] and during social reward learning tasks [START_REF] Clark-Elford | The effects of oxytocin on social reward learning in humans[END_REF][START_REF] Kruppa | Neural modulation of social reinforcement learning by intranasal oxytocin in male adults with highfunctioning autism spectrum disorder: a randomized trial[END_REF]. These mixed findings are consistent with the fact that the social effects of OXT are highly dependent on the behavioral context [START_REF] Bartz | Social effects of oxytocin in humans: context and person matter[END_REF].

It has been difficult to separate learning from choice selection processes because value representations drive choosing, as expected values of different available options are compared, and the best is selected. However, they also drive learning, as the expected value of an outcome is compared to the reward that is actually received, and future expectations are updated. Yet, value-based learning and value-based choosing may depend on separate neural representations [START_REF] Miller | Value representations in the rodent orbitofrontal cortex drive learning, not choice[END_REF][START_REF] Rudebeck | Specialized Representations of Value in the Orbital and Ventrolateral Prefrontal Cortex: Desirability versus Availability of Outcomes[END_REF]. Our findings demonstrate that OXT modulated the neurocomputational signals engaged in choice selection, increasing engagement of the vmPFC and PCC that encode the action selection policy during learning of the social hierarchy. At the computational level, these results validate the proposal that OXT may modulate the option selection stage, shifting the choice probability curve [START_REF] Piva | An integrated framework for the role of oxytocin in multistage social decisionmaking[END_REF], and increasing sensitivity to the relative value during a two-choice economic task [START_REF] Chang | Vicarious reinforcement in rhesus macaques (Macaca mulatta)[END_REF][START_REF] Piva | Distributed neural activity patterns during human-to-human competition[END_REF].

During the test phase, OXT differentially modulated social memory needed for transitive inferences according to whether the participant was embedded or not in the hierarchy. Thus, OXT enhanced performance for the making of transitive inferences with respect to the other-related hierarchy, and reduced performance for the hierarchy in which the participant themself was embedded. At the brain system level, OXT modulated amygdala and parahippocampal activities when making transitive inferences. These brain regions showed interactions in the same direction between group treatment and hierarchy orientation factors. The amygdala and parahippocampal regions have previously been found to encode the status of individuals in linear hierarchies [START_REF] Kumaran | Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanisms for Updating and Representing Self-Relevant Information[END_REF]. Both of these regions are rich in OXT receptors and receive OXT from the axonal projections of This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d parvocellular neurons in the hypothalamus [START_REF] Hawthorn | Effects of lesions in the hypothalamic paraventricular, supraoptic and suprachiasmatic nuclei on vasopressin and oxytocin in rat brain and spinal cord[END_REF][START_REF] Jurek | The oxytocin receptor: From intracellular signaling to behavior[END_REF][START_REF] Thijssen | Intra-and extrahypothalamic vasopressin and oxytocin pathways in the rat[END_REF]. Thus, they are in a critical position to be regulated by OXT. The amygdala directly mediates OXT influences on social behavior in rodents [START_REF] Ferguson | Oxytocin in the Medial Amygdala is Essential for Social Recognition in the Mouse[END_REF][START_REF] Huber | Vasopressin and oxytocin excite distinct neuronal populations in the central amygdala[END_REF] and non-human primates [START_REF] Chang | Neural mechanisms of social decision-making in the primate amygdala[END_REF].

OXT regulates social memory inferences in rodents and non-human primates [START_REF] Ferguson | Social amnesia in mice lacking the oxytocin gene[END_REF]; H. J. [START_REF] Lee | A Conditional Knockout Mouse Line of the Oxytocin Receptor[END_REF][START_REF] Macbeth | Oxytocin and the oxytocin receptor underlie intrastrain, but not interstrain, social recognition[END_REF]. For example, OXT receptors are causally involved in social memory in rodents. Knockout of the OXT gene results in the failure of male mice to develop social memory but treatment with OXT rescued this [START_REF] Ferguson | Social amnesia in mice lacking the oxytocin gene[END_REF]. In rhesus macaques, OXT delivered directly into the basolateral amygdala enhances prosocial tendencies and attention to the recipients of prosocial decisions [START_REF] Chang | Neural mechanisms of social decision-making in the primate amygdala[END_REF]. Moreover, OXT modulates value computation of self-other allocations, amplifying both vicarious reinforcement and self-reinforcement in rhesus macaques [START_REF] Chang | Inhaled oxytocin amplifies both vicarious reinforcement and self reinforcement in rhesus macaques (Macaca mulatta)[END_REF][START_REF] Chang | Oxytocin and social cognition in rhesus macaques: Implications for understanding and treating human psychopathology[END_REF]). In humans, OXT influences the balance between self and other mental processes (e.g., trust and prosocial behavior) and their underlying neural mechanisms [START_REF] De Dreu | Oxytocin modulates cooperation within and competition between groups: An integrative review and research agenda[END_REF][START_REF] Feng | Effect of intranasal oxytocin administration on self-other distinction: Modulations by psychological distance and gender[END_REF][START_REF] Liao | Intranasal oxytocin decreases self-oriented learning[END_REF][START_REF] Tomova | The Other and Me: Effects of oxytocin on self-other distinction[END_REF][START_REF] Zhao | Oxytocin blurs the self-other distinction during trait judgments and reduces medial prefrontal cortex responses[END_REF]. Together, these studies provide behavioral and neuroscientific evidence for OXT effects on self-other referential processing. Our findings extend these previous studies in an important way to self-other social memory when making transitive inferences. They indicate distinct patterns of OXT modulation of brain activities engaged in social memory, when one is oneself (or not) embedded in the hierarchy. Overall, our findings show that the effects of OXT on social memory are modulated by self-other regarding considerations, and that these effects are mediated in the amygdalo-hippocampal complex.

We also observed a modulation for Other-vs Self-orientated hierarchy in the dACC, related to action selection policy during the test phase (Fig. 6). This brain region was more robustly engaged for Other-than Self-orientated social hierarchy during the test trials.

Efficient social interactions require processing of both Other-related and Self-related information [START_REF] Bang | Neurocomputational mechanisms of confidence in self and others[END_REF][START_REF] Ereira | Social training reconfigures prediction errors to shape Self-Other boundaries[END_REF][START_REF] Wittmann | Self-Other Mergence in the Frontal Cortex during Cooperation and Competition[END_REF]. The anterior cingulate gyrus (ACCg) is implicated in empathy, social learning [START_REF] Camille | Double Dissociation of Stimulus-Value and Action-Value Learning in Humans with Orbitofrontal or Anterior Cingulate Cortex Damage[END_REF][START_REF] Hadland | The anterior cingulate and rewardguided selection of actions[END_REF][START_REF] Hayden | Neurons in Anterior Cingulate Cortex Multiplex Information about Reward and Action[END_REF], and computations of 'other-orientated' information [START_REF] Apps | The Anterior Cingulate Gyrus Signals the Net Value of Others' Rewards[END_REF][START_REF] Behrens | Associative learning of social value[END_REF][START_REF] Chang | Neuronal reference frames for social decisions in primate frontal cortex[END_REF].

Both the anterior and posterior portions of the ACCg, which overlap with the mPFC and dACC regions in our study, play a key role in facilitating cooperative and competitive This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d 13 interactions, by tracking parameters such as the value and cost of options to others [START_REF] Apps | The Anterior Cingulate Gyrus and Social Cognition: Tracking the Motivation of Others[END_REF]. As such, our results suggest that the ACCg/mPFC tracks the motivation of others through representations that not only code for the values and costs of reward to other individuals [START_REF] Apps | The Anterior Cingulate Gyrus and Social Cognition: Tracking the Motivation of Others[END_REF], but also incorporates status information, particularly within one's own social group.

Limitations and clinical implications

One strength of our study is that we controlled for participants' age and gender [START_REF] Meyer-Lindenberg | Oxytocin and vasopressin in the human brain: social neuropeptides for translational medicine[END_REF][START_REF] Wang | Neural substrates underlying the effects of oxytocin: A quantitative meta-analysis of pharmaco-imaging studies[END_REF]. We decided to test only male participants because the effects of OXT on the brain valuation system are likely to be different in men and women, as OXT increases activation in the ventral striatum in men, but decreases activation in the same region in women during the canonical prisoner's dilemma game [START_REF] Feng | Oxytocin and vasopressin effects on the neural response to social cooperation are modulated by sex in humans[END_REF]. Another reason for testing only men is that the hierarchy was composed only of male faces, and that men's and women's attitudes towards male faces may differ when learning hierarchies. Further studies will be needed to determine whether the current findings depend upon sex, by comparing hierarchies of women only or mixed samples.

Intranasal OXT spray in humans increases OXT levels in the cerebrospinal fluid. The pathways through which OXT reaches the brain include direct nose-to-brain transport, blood-to-brain transport through the blood-brain barrier or a combination of both. A recent study in primates showed that when administered intranasally, OXT can reach the brain [START_REF] Lee | Labeled oxytocin administered via the intranasal route reaches the brain in rhesus macaques[END_REF]. However, it is also possible that absorption of OXT into the blood might impact the brain indirectly. For instance, peripheral administration of OXT in rats requires vagal signaling to reduce methamphetamine self-administration and reinstatement of methamphetamine-seeking behaviors [START_REF] Everett | The vagus nerve mediates the suppressing effects of peripherally administered oxytocin on methamphetamine self-administration and seeking in rats[END_REF]. Therefore, we cannot exclude that our behavioral and neural findings of intranasal OXT on option selection during the training stage and on self-other social memory during the test phase, might reflect indirect peripheral effects or a possible interaction between peripheral and central actions. Future studies with concomitant administration of non-brain penetrant antagonists will help to dissect these effects further.

Our findings may have clinical implications because OXT may increase social functioning in clinical populations, by modulating the option selection stage of social decision making

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d and/or social memory. Although there have been mixed findings [START_REF] Cid-Jofr | Role of Oxytocin and Vasopressin in Neuropsychiatric Disorders : Therapeutic Potential of Agonists and Antagonists[END_REF][START_REF] Wang | Neural substrates underlying the effects of oxytocin: A quantitative meta-analysis of pharmaco-imaging studies[END_REF], recent reports have suggested a potential role of OXT to enhance social learning in autism spectrum disorder (ASD) [START_REF] Kruppa | Neural modulation of social reinforcement learning by intranasal oxytocin in male adults with highfunctioning autism spectrum disorder: a randomized trial[END_REF]. Single-dose administration studies have generally shown positive effects in ASD [START_REF] Domes | Effects of intranasal oxytocin on the neural basis of face processing in autism spectrum disorder[END_REF][START_REF] Gordon | Oxytocin enhances brain function in children with autism[END_REF], which might be related to an overall enhancement of the motivational system in social contexts. However, longer-term treatment studies, using multiple doses per day, often failed to demonstrate treatment effects (Guastella & Hickie, 2016). One reason might be that these were not designed to provide specific social learning contexts around the times of administration [START_REF] Aoki | Oxytocin improves behavioural and neural deficits in inferring others' social emotions in autism[END_REF], or may have interfered with psychotropic medication [START_REF] Yatawara | The effect of oxytocin nasal spray on social interaction deficits observed in young children with autism: a randomized clinical crossover trial[END_REF]. Future studies performed in ASD are needed to investigate how OXT modulates social learning and social memory.

Materials and Methods

Participants

We recruited 60 healthy male undergraduate or graduate students (mean age 21.1 years, SD = 2.10, range 18-29 years). Participants were not taking any prescribed drugs and had no history of drug abuse or smoking. We instructed participants to abstain from alcohol and heavy exercise for 24 hours and from food or any beverage other than water for at least 2 hours before scanning. Participants gave written informed consent. Human Research Ethics Committee for Non-Clinical Faculties in South China Normal University approved the study. Two subjects were excluded because of too low performance in the last session (blocks 7~9) both in training and test trials, based on the scale Z-score method, with z scores lower than -2.0, as there was less than a 2.27% probability that these two participants would have a lower average accuracy than our sample. Also, we went through all participants' decisions and found the learning rates of these two fluctuated irregularly. Finally, we included 29 subjects (mean age 20.72 years, SD = 1. 87) in the OXT group and 29 subjects (mean age 21.41 years, SD = 2.23) in the Placebo group for analysis.

Stimulus and Task

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d

The two diamond-shaped hierarchy structures consisted of 16 face pictures (8 for Selforientated condition, 8 for Other-orientated condition) and two shaded pictures referring to Self and Other (letter 'X' as its name). Real face pictures were selected from a large Chinese face database: the CAS-PEAL face database (Wen Gao et al., 2008). Images with neutral facial expression were cropped congruously and converted to grayscale. For each hierarchy structure, position 5 was always Self or Other, the face images were randomly distributed to the other locations.

The fMRI task proceeded in 3 separate scanning sessions of 6 blocks (Self: 3 blocks, Others: 3 blocks) and 150 trials each on average and took around 60 mins in total (Fig. 1).

Each block combined 12 learning trials followed with 13 test trials. Each learning trial block included 5 superior paired trial types (red line: P1 vs P2, P2 vs P3, P1 vs P5, P2 vs P5, P3 vs P5), 5 inferior paired trial types (green line: P7 vs P8, P8 vs P9, P7 vs P5, P8 vs P5, P9 vs P5) and 2 intermediate paired trial types (orange line: P4 vs P5, P6 vs P5). The Test trial block comprised (pink line: P1 vs P3, P2 vs P4, P3 vs P6, P1 vs P6, P4 vs P8, P6 vs P7, P7 vs P9, P4 vs P9, P2 vs P8, P3 vs P7, P1 vs P8, P2 vs P9, P4 vs P6). See Fig 1 for a display of these pairs. The Self-and Other-orientated conditions occurred consecutively with a random starting order that was indicated by a cue of text description on the screen.

To learn these novel diamond-shaped hierarchy structures, participants performed a threechoice social hierarchy learning task. During the learning trials, they press ''Left (>)'', ''Equal (=)'' or ''Right (<)'' within 3 s when they were shown two-paired face items on the screen to indicate the relationship between the two. To confirm the reaction, the pictures were highlighted with yellow or blue borders for 0.2 s, indicating whether it was a Self or Other-oriented hierarchy structure respectively. The order of Self/Other trial blocks and their color indications were balanced across different subjects. Then, participants received a feedback screen (2 seconds) indicating Correct (+20) or Wrong (-20). In the test trials, participants had to infer the relationships between the two faces that were presented (pairs of non-adjacent faces from the same hierarchy) from what they had learnt before. In the test phase they received no immediate feedback, however at the end of each test trial block they were informed of their global score of correct responses.

Procedure

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360

P r e p r i n t n o t p e e r r e v i e w e d

When participants arrived at the lab, they were first asked to self-administer an intranasal dose of 24 IU (Oxytocin Spray-Sichuan Meike Pharmacy Co. Ltd, Sichuan, China; 3 puffs of 4 IU per nostril with 30 s between each puff) of either oxytocin or placebo under the experimenters' supervision. Our placebo contained the same solvent (sodium chloride and glycerin) but no oxytocin and was packaged in the same manufactured bottle. Then, participants were required to fill out questionnaires assessing psychological measures: The State-Trait Anxiety Inventory (STAI, subscales: SAI and TAI) to evaluate state and trait anxiety, the Beck Depression Inventory (BDI) to assess depression, and the Positive and Negative Affect Schedule (PANAS, subscales: PAS and NAS) to rate current affective state.

Prior to the scanning session, to motivate learning of the hierarchy task, subjects were given a cover story. They role-played as a candidate applying for a job at their ideal company. Concurrently, they imagined another person X (a stranger) also applying for this job. Subsequently they completed 10 low difficulty multiple choice questions, representing personality, math and literature (e.g., general knowledge: "How do you use a knife and fork?"; math tests: "According to the net of a cube below, select a valid way to reshape it."). These had to be answered by subjects within 5 mins. Then, participants were told that both he and the X person would be assigned to corresponding positions in two different departments of the company based on their scores.

Once the quiz was completed, the participants received experimental instructions, that they and X were employed by department A (B) and B (A) separately. As beginners, they were required to learn the relationship between each person's level of competence in each of the two departments, to better integrate into the organization. Instructions were also included with respect to the learning and test trials in the hierarchy task. We briefly repeated these instructions at the beginning of pretest program which comprised three trials in the Self-and Other-orientated hierarchy conditions, to confirm they thoroughly understood the task. Afterwards, 16 pictures were presented in a random order three times, each for 2 seconds to reduce the potential effects of novel stimuli during the scanning procedure. 30 minutes after the administration of the spray participants began to perform the formal task which took about 60 minutes in the MRI scanner. When the fMRI experiment finished, participants completed a self-reported questionnaire to investigate their attitude towards the study, as well as the State Anxiety Inventory (STAI-SAI), the Positive and Negative Affect Schedule (PANAS), Social Dominance Orientation Social This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d (SDO), Achievement Motivation Scale (AMS) and Iowa-Netherlands Comparison Orientation Measure (INCOM), to control the emotion and personality variables (Supplementary Table S5).

Behavioral analysis

Performance was modeled using a generalized linear mixed-effects model (GLMM) with a logit link function for the binomial (correct/ wrong) answer by the glmer function in 'lme4' package on R, allowing random intercepts per subjects. The fixed effects included 4 variables (Group (OXT/Placebo), Hierarchy-orientation condition (Self/Other), Status (Superior, Intermediate, Inferior), Block (1 to 9)), and their 2-way interaction factors.

Non-significant interactions would be excluded from final models. For response time (RT), we used a linear mixed model (LMM) to analyze data with the lmer function in 'lmertest' package on R. Follow-up tests of significant main effects or interactions were conducted using the 'emmeans' package. Detailed results tables for all models are reported in Supplementary files.

Computational modeling

The social hierarchy structure concerned both equal and different ranks, which makes it two dimensional and hence harder to learn than linear hierarchies. Consequently, we developed a new RL-ELO model, termed Combined RL-ELO-based associative learning models (Model 5, detailed below), enabling the model to learn hierarchical structures involving equal ranks. Because learning equal ranks may involve different strategies than learning unequal ranks, we assumed that learning equal ranks was equivalent to memorizing matching pairs. For this reason, we propose a model in which learning happens in two steps. Firstly, the Model 5 increased equality pair trial weights, when updating value processes, by judging the paired trial as equal or unequal response 𝑝 𝑒𝑞 in the first step, with learning rate 𝛼 asso , using an associative learning model (Equation 1~2).

Secondly, participants updated their representation of the structure of the hierarchy by updating the value of each item in the pair according to the past choice and reward. Each item value was evaluated within the internal hierarchy structure, with learning rate α in unequal trials, and the chosen item and learning rate 𝛼 𝑒𝑞 in equal trials by the RL-ELO model (Equation 3~10). That is, the model increased (or decreased respectively) the value This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d of the highest (or the lowest) item in the hierarchy. If both items were equal in the hierarchy, their values became closer according to a proportion (𝛼 𝑒𝑞 ) of their initial distance. The softmax choice rule (Equation 11~14) converted action values (e.g., 𝑉 𝐿 ) to action probabilities (e.g., 𝑝 𝐿 ). The inverse temperature 𝛽, controled the steepness of the softmax function, which decides how decisively the response alternatives contrast with each other.

RL-ELO model combining associative and structure learning (Model 5)

Free parameters: 𝛼 asso = learning rate of associative learning; α = learning rate in unequal trial chosen item; 𝛼 eq = proportion of the distance items' value are brought closer after equal feedback; 𝛿 1 = weight on value from associative learning; 𝛿 2 = weight on values from global structure learning; 𝛾 = bias representing preference towards selecting the equal response; 𝛽 = temperature representing the stochasticity of choices.

Associative learning (distinguish '=' and '<>' choices): if choice at t-1 is '=', then Eq = 1 if choice at t-1 is either '<' or '>', then Eq = -1

𝑝 𝑒𝑞,𝑡-1 = 𝑝 (𝑎𝑐𝑡𝑖𝑜𝑛 𝑡-1 = 𝑒𝑞𝑢𝑎𝑙)
(1)

𝑄 𝑒𝑞,𝑡 = 𝑄 𝑒𝑞,𝑡-1 + 𝛼 𝑎𝑠𝑠𝑜 * (𝐸𝑞 -𝑝 𝑒𝑞,𝑡-1 ) (2)
Structure learning (distinguish relationship between a pair of items):

if Left item is correct choice:

𝑝 𝑤𝑖𝑛,𝑡-1 = 𝑝 𝐿,𝑡-1 (3) 𝐼 𝐿,𝑡-1 = 1, 𝐼 𝑅,𝑡-1 = -1 (4)
if Right item is correct choice:

𝑝 𝑤𝑖𝑛,𝑡-1 = 𝑝 𝑅,𝑡-1 (5) 𝐼 𝑅,𝑡-1 = 1, 𝐼 𝐿,𝑡-1 = -1 (6)
Therefore, if previous response is unequal ('<' or '>'):

𝑄 𝐿,𝑡 = 𝑄 𝐿,𝑡-1 + 𝐼 𝐿,𝑡-1 * α * (1 -𝑝 𝑤𝑖𝑛,𝑡-1 ) (7)
This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d

𝑄 𝑅,𝑡 = 𝑄 𝑅,𝑡-1 + 𝐼 𝑅,𝑡-1 * α * (1 -𝑝 𝑤𝑖𝑛,𝑡-1 ) (8)
if previous response is equal ('='):

𝑄 𝐿,𝑡 = 𝑄 𝐿,𝑡-1 + 𝛼 𝑒𝑞 * (𝑄 𝑅,𝑡-1 -𝑄 𝐿,𝑡-1 )

(9)
𝑄 𝑅,𝑡 = 𝑄 𝑅,𝑡-1 -𝛼 𝑒𝑞 * (𝑄 𝐿,𝑡-1 -𝑄 𝑅,𝑡-1 )

(10)

Probability of estimated action: The Combined RL-ELO-based associative learning model increased equality pair trial weights when updating the value process by judging the paired trial to be equal or unequal response 𝑝 𝑒𝑞 in the first step with learning rate 𝛼 asso , using the associative learning model (Equation 1~2); the next step evaluated each item's value within the internal hierarchy structure, with learning rate 𝛼 in unequal trial chosen items and learning rate 𝛼 𝑒𝑞 in equal trial chosen items, according to the RL-ELO model (Equation 3~10).

𝑉 𝐿 = 𝑄 𝐿 -𝑄 𝑅 (11) 𝑉 𝐸 = 𝛿 1 * 𝑄 𝑒𝑞 + 𝛿 2 |𝑄 𝐿 -𝑄 𝑅 | * 𝑁 𝑡𝑟𝑖𝑎𝑙 + 𝛾 (12) 𝑉 𝑅 = 𝑄 𝑅 -𝑄 𝐿 (

Model 1: RL-ELO model with 2 alphas

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 12which will be calculated as Equation 15: While RW model set  as zero.

𝑉 𝐸 = 𝛿 2 |𝑄 𝐿 -𝑄 𝑅 | (
if Left item is correct choice, trial outcome vector (𝑂) & indirect item vector(𝑊):

𝑂 𝐿,𝑡-1 = 1, 𝑂 𝑅,𝑡-1 = -1 (16) 𝑊 𝐿,𝑡-1 = 0, 𝑊 𝑅,𝑡-1 = 1 (17)
if Right item is correct choice, trial outcome vector & indirect item vector:

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d

21 𝑂 𝐿,𝑡-1 = -1, 𝑂 𝑅,𝑡-1 = 1 (18) 𝑊 𝐿,𝑡-1 = 1, 𝑊 𝑅,𝑡-1 = 0 (19)
Update values of previous direct item:

𝑉_𝑑𝑖𝑟𝑒𝑐𝑡 𝐿,𝑡-1 = (𝑂 𝐿,𝑡-1 -𝑉 𝐿,𝑡-1 ) * α (20) 𝑉_𝑑𝑖𝑟𝑒𝑐𝑡 𝑅,𝑡-1 = (𝑂 𝑅,𝑡-1 -𝑉 𝑅,𝑡-1 ) * α (21)
Update values of previous indirect item:

𝑉_𝑖𝑛𝑑𝑖𝑟𝑒𝑐𝑡 𝐿,𝑡-1 = 𝑉 𝑅,𝑡-1 * 𝑊 𝐿,𝑡-1 *  (22) 𝑉_𝑖𝑛𝑑𝑖𝑟𝑒𝑐𝑡 𝑅,𝑡-1 = 𝑉 𝐿,𝑡-1 * 𝑊 𝑅,𝑡-1 *  (23)
Total update:

𝑉 𝐿,𝑡 = 𝑉_𝑑𝑖𝑟𝑒𝑐𝑡 𝐿,𝑡-1 + 𝑉_𝑖𝑛𝑑𝑖𝑟𝑒𝑐𝑡 𝐿,𝑡-1 (24) 𝑉 𝑅,𝑡 = 𝑉_𝑑𝑖𝑟𝑒𝑐𝑡 𝑅,𝑡-1 + 𝑉_𝑖𝑛𝑑𝑖𝑟𝑒𝑐𝑡 𝑅,𝑡-1 (25)

fMRI data acquisition and analyses

Brain images were collected at the Imaging Center of South China Normal University on a Siemens 3T Tim Trio scanner equipped with a 32-channel head-coil. Functional images were collected by using an echo-planar imaging (EPI) sequence (repetition time = 2000 ms, echo time = 30 ms; flip angle = 90°; slice thickness = 3.5 mm, field of view = 224 mm, in-plane resolution = 3 × 3 mm). In addition to functional images, a high-resolution anatomical scan was also collected (MDEFT, 1 × 1 × 1 mm resolution). Stimulations and response recordings were controlled by E-prime software on a Windows laptop located in the scanner control room. Choice responses were collected using a 4-button response pad.

The following pre-processing was applied: slice timing correction, spatial realignment, coregistered to the mean functional images, segmentation of structural T1 image, normalization motion correction, spatial smoothing using a Gaussian kernel of full-widthat-half-maximum 8.0 mm.

We ran four independent general linear models (GLMs) with a 2-way flexible factorial ANOVA, with Self-and Other-orientated hierarchy as a within subject factor and Group (Oxytocin/Placebo) as a between subject factor. In GLM1 and GLM3, onset regressor

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d For identification of brain (de)activation patterns we first applied whole-brain FWEcorrection with a cluster-wise threshold of p<0.05, combined with an uncorrected voxellevel threshold of P <0.001. Next, referencing prior neuroimaging work on subjective value of choice alternatives, we extracted BOLD activity from 10 mm radius spherical ROI in the vmPFC (x = -2, y = 46, z = -8) and PCC (x = -4, y = -30, z = 36) according to a meta-analysis of value representation [START_REF] Bartra | The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value[END_REF]. We also added the left amygdala as ROI, identified through the WFU PickAtlas 3 toolbox. The results were visualized using the Mango image viewer software (http://rii.uthscsa.edu/mango/mango.html). Brain regions were labeled according to the automated anatomical labeling template via the xjView toolbox (http://www.alivelearn.net/xjview8/). All significantly activated brain regions are reported in Supplementary Table S5.

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d New pairs of items, that were not in the training trial blocks were presented in a similar way, and participants attempted to infer the hierarchical relationships, however, there was no feedback. The right panel shows the social hierarchy structure contained 13 test trials (pink line: P1 vs P3, P2 vs P4, P3 vs P6, P1 vs P6, P4 vs P8, P6 vs P7, P7 vs P9, P4 vs P9, P2 vs P8, P3 vs P7, P1 vs P8, P2 vs P9, P4 vs P6). (C) The fMRI task included Self-orientated and Other-orientated social

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d hierarchy structures in which the central position (P5) is labelled as "You" or "X" respectively, along with the corresponding color (yellow or blue). However, these structures were never shown to the participants, and are presented here only to explain the experiment. The right panel illustrates one fMRI session, which comprised 3 blocks (integrated Self-and Other-orientated blocks, Self: 3 mini blocks, Other: 3 mini blocks). It was indicated in text on the cue screen whether the mini block was Self-or Other-orientated at the start of the block. (See Methods for a detailed description). This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d Notes: contrast coding of Group: 0.5 = Oxytocin, -0.5 = Placebo (reference level); contrast coding of S_O: 0.5 = Self, -0.5 = Others (reference level); contrast coding of rank: This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d Notes: Reference level of Group: 0.5 = Oxytocin, -0.5 = Placebo; Reference level of S_O: 0.5 = Self, -0.5 = Others. Significance: * p < 0.05, * * p < 0.01, * * * p < 0.001.

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d Note: Regions shown here met the uncorrected voxel-level threshold of p < 0.001 with k = 40.

With the above uncorrected threshold as the cluster-defining threshold, clusters met the Family-This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d Note: We examined differences between Self/Other hierarchy conditions (S_O) in the model parametric at the group level using repeated-measures ANOVAs.

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d

  13) Action policy (softmax function): 𝑝 (𝑐ℎ𝑜𝑜𝑠𝑒 𝑜𝑝𝑡𝑖𝑜𝑛 𝑖) = 𝑒 𝛽 * 𝑉 𝑖 𝑒 𝛽 * 𝑉 𝐿 + 𝑒 𝛽 * 𝑉 𝐸 + 𝑒 𝛽 * 𝑉 𝑅 (14) Lastly, several models were tested to update the item values in the hierarchy structure. For example, the Rescorla-Wagner (RW) model (Model 3) and the Value Transfer model (Model 4), which are known to fail in learning linear hierarchies as they associate equally with positive and negative outcomes. The classical RL-ELO model (Model 1) which functions successfully in learning linear hierarchy structures was also compared, as well as its form with two different learning rates for hierarchically superior and inferior members of the hierarchy (Model 2).

  This model assumes that the following three various hierarchies: superior, intermediate and inferior have their own learning rate. Therefore, the update value of unequal trials (Equation 7 to 8) is split, as when a previous pair is related to superior hierarchy, then conducted with 𝛼 𝑠𝑢𝑝𝑒𝑟𝑖𝑜𝑟 , corresponding to inferior hierarchy performed with 𝛼 𝑖𝑛𝑓𝑒𝑟𝑖𝑜𝑟 .Model 3 & 4: Rescorla-Wagner (RW) model & Value transfer modelThe classical RW model is inspired by Pavlovian conditioning  animal learning from conditioned stimulus (CS) paired with an unconditioned stimulus (US). According to mechanism of RW model, the value transfer model contains another component, the accrued value  from associative strength of the indirect (wrong) item paired with CS.

  began at the presentation of the 2 paired items with null duration, modulated by the parametric regressor: action probability of the choice for correct responses by the Combined RL-ELO-based associative learning model (Model 5). GLM1 was applied to training trials and GLM3 to test trials. To improve overall model fit and account for potential confounds, the design matrix included the following additional regressors which were not analyzed: two regressors that modelled the onset of the highlighted response screen and the onset of the feedback screen. Both were performed separately for training trials and test trials. The GLM2 model employed a regressor at the onset of the feedback screen with 2 s duration, modulated by two parametric regressors: an unsigned PE of judging equal or unequal trial 𝑃𝐸 𝑒𝑞 , and an unsigned PE of choosing response action 𝑃𝐸 𝑖 by the winning Model 5. Parametric modulators were not orthogonalised with respect to each other. The GLM4 was a basic GLM analysis without model parameters, which set the regressor with whole test trial while the two items were displayed and the highlighted response screen with actual duration.

FiguresFig. 1 .

 1 Figures

Fig. 2 .

 2 Fig. 2. Behavioral results. (A) The graphs show significant main effect for hierarchy in performance accuracy and response times (RT) during training trials by GLMM1 (performance) and LMM1 (RT) for superior, intermediate and inferior paired items. The dotted line indicates the performance level of random choice at 1/3. (B) Learning curve showing the mean performance of Oxytocin and Placebo groups in training trials, averaged across within-group stimuli. Participants in Placebo group were more likely learned better than Oxytocin group across time (GLMM1). (C) For illustration purposes we indicate mean performance of interaction between Groups and Hierarchy orientation, but only the Hierarchy orientation effect is significant. Performance was more likely better for the Self-than for the Otherorientated conditions (GLMM1). (D) Learning curve showing the mean performance for the Oxytocin and Placebo groups in test trials, averaging across within-group stimuli. Participants in the Oxytocin group were more likely to be

Fig. 3 .

 3 Fig. 3. Model validation on RL-ELO-based associative learning model and model comparison. (A) Learning curves, pooled over all factors estimated by the computational model with the probability of items being correct during training trials. Dots represent mean performance results of participants on that specific trial. (B) The plot shows the mean item power changed trial-by-trial, captured by the computational model, which indicates the difference between the expectation over the estimated power of the presented item. (C) Similar learning curves analysis but estimated for test trials. (D) Bayesian model selection in terms of the protected exceedance probabilities (PXP) with all subjects, which indicates that the combinatorial models based on RL-ELO and associative learning (M5) estimates decisions in hierarchy relationship better than other models: RL-ELO model with 1 alpha (M1); RL-ELO model with 3 alphas (M2); Rescorla-Wagner model (M3); Value transfer model (M4). (E) The learning rate 𝜶 𝒆𝒒 of equal trials, modeled by the RL-ELO associative learning model, is higher for the Other-than Self-oriented

Fig. 4 .

 4 Fig. 4. Probability to choose the correct answer in training trials. (A) Whole-brain analysis revealed a network of brain regions encoding probability to choose the correct answer regardless of group (GLM 1). The regions of interest (ROI) of vmPFC and PCC were defined based on a meta-analysis (see methods). (B and C) Illustration of beta estimates in vmPFC and PCC and show a significant difference between Oxytocin and Placebo groups. Display threshold: P < 0.001 uncorrected at the voxel-level with k >= 40. Error bars show SEM. The activation figures hereinafter were made in the same way unless stated otherwise. (vmPFC = ventromedial prefrontal cortex, PCC = posterior cingulate cortex).

Fig. 5 .

 5 Fig. 5. Prediction error value for the winning model in training trials. (A) Wholebrain analysis revealed a network of brain regions encoding prediction error regardless of group (GLM 2). (B, C & D) Illustration of beta estimate values in vmPFC, and left and right putamen. These regions showed significant activity, but no significant differences between conditions (vmPFC: ventromedial prefrontal cortex, PUT: putamen).

Fig. 6 .

 6 Fig. 6. Probability to choose the correct answer of the winning model in test trials. (A) Whole-brain analysis revealed a network of brain regions encoding the probability to choose the correct answer regardless of group by GLM 3. (B &C) Neural responses are significantly stronger to the Other-than to the Self-orientated hierarchy in the dACC (dorsal anterior cingulate cortex).

Fig. 7 .

 7 Fig. 7. Brain areas tracking the transitivity process in test trials. (A) Interaction effect of Group × Hierarchy Orientation in test trials showed that BOLD deactivations in the left amygdala and right inferior parahippocampal gyrus were stronger during the Self-than Other-orientated hierarchy trials in the Placebo group, while in Oxytocin group the activity was stronger for the Other-than for the Self-orientated hierarchy when tracking the transitive inference process. (B & C) Illustration of activities in amygdala and Parahippocampus showing significant differences between Oxytocin and Placebo groups.

  . Block was coded as ordinal levels. Significance: * p < 0.05, * * p < 0.01, * * * p < 0.001.

  models assume only a single learning rate 𝛼, ranging from 0 to 1. The higher 𝛼, the stronger the weighting of the prediction error (PE) for updating value. However, due to the three-level hierarchies (i.e., superior, intermediate, inferior), we developed two RL-ELO models. The first RL-ELO model with 2 alphas (Model 1), with 𝛼 representing the learning rate for unequal choice 𝛼 𝑢𝑛𝑒𝑞𝑢𝑎𝑙 , and the other is equal choice 𝛼 𝑒𝑞𝑢𝑎𝑙 . The second RL-ELO model (Model 2), further subdivided the learning rate 𝛼 𝑢𝑛𝑒𝑞𝑢𝑎𝑙 into 𝛼 𝑠𝑢𝑝𝑒𝑟𝑖𝑜𝑟 and 𝛼 𝑖𝑛𝑓𝑒𝑟𝑖𝑜𝑟 .RL-ELO model with 2 alphas is focused on the difference value between pairwise items in each trial. The implementation of this model is approximately similar to the process of Equation 3 to 12. What differs from these segments is the estimated equal action probability within Equation

P r e p r i n t n o t p e e r r e v i e w e d 20 Classical RL

Table S1 .

 S1 Performance results of generalized linear mixed-effects model and response time (RT) results of linear mixed effect model in training trials.

		Performance			RT
	Predictor	Odds	SE	z	Beta	SE	t
		ratios
	(Intercept)	0.234 *** 0.022 -15.472 1.824 *** 0.03	60.595
	Group (PB)	1.226	0.221 1.135	0.114	0.06	1.905
	S_O (Other)	1.237 *** 0.056 4.711	0.014	0.019 0.762
	Block	1.433 *** 0.014 37.503 -		-
					0.045 *** 0.002	27.667
	Hierarchy (Superior)	3.392 *** 0.471 8.79	0.032 ** 0.012 2.637
	Hierarchy (Inferior)	3.156 *** 0.437 8.29	0.128 *** 0.012 10.592
	Group × Block	0.966 *	0.016 -2.038	-
					0.022 *** 0.003 -6.763
	S_O × Hierarchy (Superior)	0.543 *** 0.066 -5.06	0.082 ** 0.024 3.401
	S_O × Hierarchy (Inferior)	0.616 *** 0.074 -4.05	0.052 *	0.024 2.154
	Block × Hierarchy (Superior) 0.92 **	0.023 -3.259
	Block × Hierarchy (Inferior) 0.882 *** 0.022 -5.01
	S_O × Block				-0.007 *	0.003 -2.125

Table S2 .

 S2 Performance results of generalized linear mixed-effects model and response time (RT) results of linear mixed effect model in test trials.

		Performance			RT
	Predictor	Odds ratios SE	z	Beta	SE	t
	(Intercept)	0.59 ***	0.057 -5.416 1.959 ***	0.032 61.594
	Group (PB)	0.881	0.172 -0.649 0.045	0.064 0.705
	S_O (Other)	1.021	0.042 0.511	0.09 ***	0.019 4.821
	Block	1.373 ***	0.012 36.518 -0.047 *** 0.002 -28.17
	Group ×	1.04 *	0.018 2.268	-0.015 *** 0.003 -4.672
	Block
	Group × S_O 1.262 **	0.104 2.82
	S_O × Block				-0.014 *** 0.003 -4.077

Table S3 .

 S3 Parameters of combined RL-ELO-based associative learning model.

	Mean (SEM)
			𝛼 𝑎𝑠𝑠𝑜 α	𝛼 𝑒𝑞	𝛽	𝛿 1	𝛿 2	𝛾
	Oxytocin	Self Other	0.55 (0.01) (0.03) (0.02) (0.08) (0.06) (0.00) (0.10) 0.35 0.06 0.36 0.59 0.00 -0.55 0.54 0.37 0.15 0.38 0.50 0.00 -0.57 (0.01) (0.02) (0.03) (0.09) (0.08) (0.00) (0.10)
	Placebo	Self Other	0.55 (0.01) (0.02) (0.04) (0.09) (0.06) (0.00) (0.08) 0.41 0.14 0.24 0.63 0.00 -0.53 0.55 0.34 0.22 0.16 0.58 0.00 -0.64 (0.01) (0.02) (0.05) (0.11) (0.07) (0.00) (0.09)

Note: SEM means standard error of the mean.

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d 40

Table S4 .

 S4

	1073	Brain activity related to GLMs 1-4.
	1074
		Brain Region	Hemisphere	Cluster Size	T-value		MNI		BA
						x	y	z
		Regions encoding the value of chosen correct option in training phase
			(pooled all conditions; GLM1)
			Positive modulation in all subjects:
		CB/OG***	L	4311	7.34	-33	-75	-21	7/17/19
		MTG/IPL***	L	1397	7	-66	-39	0	21/39/40
		SFG/MFG***	L	1830	5.77	-27	27	54	8
		MFG/IFG***	L	492	5.65	-39	60	-3	10/47
		TG***	R	368	4.78	60	-24	-21	21/22
		vmPFC**	B	88	4.58	-12	36	-9	10
		MCC**	B	131	4.32	6	-30	42	31/24
			Negative modulation in all subjects:
		LG*	L	62	4.34	-24	-69	-6	19
				OXT > PB:
		PCC (p=0.052)	R	56	4.41	15	-24	42	31
			Positive

modulation in Self-orientated condition: CB/OG***

	IFG** Put*	L L	105 101	4.82 4.91	-39 -24	36 -12	-9 -3	47
	SFG*** Put**	L R	251 131	4.47 4.84	-15 30	42 3	45 15	9
	CB/OG*** OG*	L B	206 124	4.35 4.51	-39 -3	-72 -90	-24 33	19/37 19
	AG***	L	168	4.12	-57	-57	30	39/40
	AG*	R	82 Negative modulation in all subjects: 4.08		57	-57	24	39/40
	vmPFC* MFG/SFG***	B R	80 410	3.92 5.92		3 42	30 21	-12 30	11 8/9
	OG/PCUN* MFG/IFG***	L L	76 458	3.74 5.82	-12 -39	-87 21	33 27	7 46/9
	SFG/SMA***	L	253	5.76		-6	9	57	6/8
	IFG/Ins*	Positive modulation in Oxytocin group: R 94 5.33 33	21	0	47
	AG/TPJ/TG*** IPL***	L L	1620 243	6.21 5.04	-54 -36	-33 -45	0 45	40/22/21 40
	AG/TPJ/TG***	R	829	4.91		51	-30	3	40/22/21
	PCC***	B Negative modulation in Self-orientated condition: 683 4.84 9	-33	51	31
	MFG/IFG*** Thalamus*	R R	239 117	4.71 4.46		48 9	51 -24	9 0	46/47
	SFG*** MFG***	L L	319 265	4.56 4.41	-15 -39	42 24	51 27	9 9
	TG** MFG*	R R	135 118	4.49 4.24		63 42	-9 21	-12 27	21/38 9
	MFG/SFG**	R	122	4.47		36	39	45	8
	SFG*	R Positive modulation in Other-orientated condition: 57 4.43 12	33	54	6
	PCUN* vmPFC*	R B	63 108	4.38 5.15	6 -12	-96 39	0 -3	17 32
	ACC** Putamen*	L R	86 93	4.1 4.11		-9 27	39 9	0 0	24
	vmPFC**	B	90	4		-3	57	12	10
	OG*	L Negative modulation in Other-orientated condition: 79 3.72 -39	-84	-12	37
	MFG*	R	129	4.7		42	24	27	8
	MFG/IFG*	L	Positive modulation in Placebo group: 122 4.3	-39	21	27
	CB/PCUN***	L	2156	7.66	-30	-78	-21	7/19
	SFG**	L Negative modulation in Oxytocin group: 87 4.38 -12	63	30	10
	AG* IPL*	L R L	1342 59 92	6.39 4.14 4.1	-30 45 -36	-75 -63 -45	-21 30 45	16/18 40 40
	VS/Hipp*** OFC**	L L	186 98	5.67 4.13	-24 -39	-27 54	9 -3	27/13 46
	PCUN***	B Negative modulation in Placebo group: 537 5.58	3	-75	57	7/13
	CB/OG*** MFG**	R Negative modulation in Placebo group: 287 5.1 R 183 5.05	45 42	-78 21	-18 30	18/19 8
	MFG*** IFG* SFG/SMA***	R L R	366 76 260	5.06 5.32 4.9	48 -45 9	51 3 24	9 24 48	8/16 9 6
	MFG*** LG* MFG/IFG**	L L L	161 80 195	4.96 5.04 4.52	-30 -24 -42	24 -69 15	54 -3 30	8 19 9
	MTG* LG* IFG/Ins*	L R R	61 60 87	4.7 4.41 4.29	-66 27 39	-39 -51 24	0 -9 0	21 19 47
	AG***	R	181	4.69		48	-57	42	40
	AG***	L Regions encoding the value of prediction error in training phase 397 4.64 -42 -51 Regions	36	41
	IFG*	L	71 (all conditions pooled, GLM2) 4.46	-30	30	-15	47
		Positive modulation in Other-orientated condition: Positive modulation in all subjects:
	TG*** vmPFC**	L B	236 181	5.23 5.74	-66 -12	-39 42	0 0	21 32
	LG*** Cau*	R R	224 106	4.86 5.2		9 18	-96 -6	-3 27	17

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d

encoding the value of chosen correct option in test phase (all conditions pooled, GLM3) Positive modulation in all subjects:

	VS*** TPJ***	R L	270 662	5.76 5.13	30 -51	-21 -39	3 24	13 40/41
	MTG*** PoCG***	R R	357 327	5.42 5.13		60 60	-42 -24	-3 33	21 2/40
	Ins** TG**	L R	108 132	5.27 4.76	-30 63	-6 -42	3 0	13 21
	SFG*** TG/OG*	R L	608 99	5.25 4.41	9 -54	51 -57	42 6	9 37
	TPJ***	L	495	5.06	-54	-42	24	40
	TPJ***	R Negative modulation in Other-orientated condition: 248 5.01 51	-51	30	40
	IFG/MFG*** LG*	R L	242 73	4.79 4.7	48 -12	39 -75	-3 -3	46/10 18
	CB***	L	287	4.73	-33	-75	-24
	CB/OG***	R Positive modulation in Oxytocin group: 341 4.53 21	-93	-12	18
	PCC*** TG**	B L	278 110	4.13 5.24	12 -51	-42 -36	42 6	31 22
	PCUN*	R	100	4.83		6	-78	36	7
	TPJ/Ins*	L	Negative modulation in all subjects: 89 4.67	-54	-42	24	40/42
	LG** VS*	L R	132 83	5.76 4.64	-15 30	-78 -6	-3 6	18
	MedFG*** PCC***	B B	271 321	5.08 4.45		-9 3	12 -33	51 33	6 31
	LG*		R	82	4.59		18	-75	-3	18
				Positive modulation in Placebo group:
	TG**	R	Other > Self: 170	4.96		63	-42	-3	21
	dACC* SFG**	R R	94 171	4.29 4.84		15 12	6 54	45 42	33 9
	PreCG* OG***	R R	78 246	4.21 4.79		18 15	-15 -87	57 36	6 19
	IPL* OG***	R L	104 314	3.76 4.63	57 -42	-27 -84	33 -12	40 37
	ACC**	B	106	4.2		3	36	-24	11
	CB*		Positive modulation in Self-orientated condition: R 104 4.06 42	-75	-24
	IFG**	L	109	5.85	-48	30	0	47
	MFG*	R Negative modulation in Placebo group: 99 5.18	48	39	-3	47
	SFG** LG*	R L	114 87	4.5 5.39	9 -15	54 -75	42 -3	9 18
	AG*		R	88	4.35		48	-51	30	39
	TG**	R	122 Others -Self in Placebo group: 4.29		66	-33	-6	20/21
	MTG** SFG/PreCG***	L B	107 436	4.17 4.62	-48 6	-39 -3	0 78	21 6/31
	SFG* LG/OG*	L R	96 84	4.17 4.29	-15 24	27 -96	60 -9	9 18
	CB*		L	73	4.26	-27	-81	-39
			Negative modulation in Self-orientated condition:
	Ins*	R Regions encoding the value of inferring social hierarchy in test phase (GLM4) 82 6.18 33 18 12	47
	MCC/SFG***	L	383	5.38	-24	0	54	24/6
			Oxytocin_(Self -Other) > Placebo_(Self -Other):
	ITG**	Positive modulation in Other-orientated condition: L 131 4.27 30	12	-30	38
	VS/Amy***	R	447	6.01		30	-18	6	13
	OG***	R	321	5.85		36	-81	-21	19
	IFG/MFG** PCUN***	L R	125 1342	6.11 5.77	-51 9	30 -84	0 39	47 19
	PCUN*** LG***	R L	542 254	5.85 5.57	9 -15	-84 -105	39 -9	19 18
	44

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d

Table S5 .

 S5 Various scales' scores on Pre -& Post -scanning task.

	Scales	Oxytocin Mean	Placebo Mean	t	p
	Before the fMRI task	(SD)	(SD)
	BDI	29.48	29.62	-0.078 0.938
		(6.07)	(7.30)
	STAI	77.62	75.55	0.513 0.61
		(13.77)	(16.82)
	STAI _SAI	36.38	34.76	0.738 0.464
		(7.22)	(9.37)
	STAI _TAI	41.24	40.79	0.204 0.839
		(7.98)	(8.78)
	PANAS	52.83	50.21	1.186 0.241
		(7.47)	(9.26)
	PANAS _PAS	32.97	30.90	1.396 0.168
		(5.52)	(5.77)
	PANAS _NAS	19.86	19.31	0.321 0.75
		(5.41)	(7.52)
	After the fMRI task
	STAI _SAI	35.97	33.28	1.124 0.266
		(9.45)	(8.77)
	PANAS	49.24	44.69	1.89	0.064
		(9.07)	(9.27)
	PANAS _PAS	31.76	29.10	1.321 0.192
		(7.77)	(7.53)
	PANAS _NAS	17.48	15.59	1.224 0.226
		(5.51)	(6.27)
	SDO	3.48	3.60	-1.053 0.292
		(1.58)	(1.79)
	SDO_GBD	3.85	4.22	-2.298 0.022 *
		(1.66)	(1.74)
	SDO_OEQ	3.11	2.99	0.89	0.374
		(1.40)	(1.62)
	INCOM	38.03	38.31	-0.211 0.833
		(-4.34)	(-5.53)
	INCOM _Abilities	21.03	20.45	0.805 0.424
		(-2.74)	(-2.8)
	INCOM _Opinions	17	17.86	-1.131 0.264
		(-2.24)	(-3.44)
	AMS	5.9	2.79	0.801 0.427
		(17.07)	(12.01)

Table S7 .

 S7 Model parametric in RL-Elo associative learning model using repeated-measures ANOVAs.

	Parametric	Factor	F (1,56)	p	η 2
	𝛼 asso	Group	0.25	0.62	0.00293
	𝛼 asso	S_O	0.00	0.95	0.00003
	𝛼 asso	Group×S_O	0.13	0.72	0.00085
	𝛼 eq	Group	3.53	0.07	0.03946
	𝛼 eq	S_O	7.42	0.01	0.04412
	𝛼 eq	Group×S_O	0.02	0.88	0.00013
	α	Group	0.30	0.58	0.00341
	α	S_O	1.69	0.20	0.01093
	α	Group×S_O	4.34	0.04	0.02766
	𝛽	Group	2.31	0.13	0.03025
	𝛽	S_O	0.21	0.65	0.00091
	𝛽	Group×S_O	0.69	0.41	0.00301
	𝛿 1	Group	0.49	0.49	0.00592
	𝛿 1	S_O	1.59	0.21	0.00900
	𝛿 1	Group×S_O	0.08	0.78	0.00047
	𝛿 2	Group	3.01	0.09	0.02588
	𝛿 2	S_O	1.13	0.29	0.01015
	𝛿 2	Group×S_O	0.03	0.86	0.00030
	𝛾	Group	0.05	0.82	0.00075
	𝛾	S_O	1.06	0.31	0.00428
	𝛾	Group×S_O	0.65	0.42	0.00262

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d

Acknowledgments

We thank Y. Huang for the help in participant recruitment and scanning. We also thank all of the participants who participated in the study.

Funding:

This research has benefited from the financial support of IDEXLYON from Université de Lyon (project INDEPTH) within the Programme Investissements d'Avenir (ANR-16-IDEX-0005), and of the LABEX CORTEX (ANR-11-LABX-0042) of Université de Lyon, within the program Investissements d'Avenir (ANR-11-IDEX-007) operated by the French National Research Agency, and the grant from the Agence Nationale pour la Recherche to JCD (ANR-21-CE37-0032). This work was also supported by the National Natural Science Foundation of China (No. 31970982 and No. 32171019). J.L was supported by the China Scholarship Council.

Data and materials availability:

All data needed to evaluate the conclusions in the paper are present in the paper, the Supplementary Materials, and the OSF repository: https://osf.io/wjbpz/?view_only=d9ed0a26788b45a98e51e367059e0588

Declarations of interests:

Authors declare that they have no competing interests.

Supplementary results

Fig. S1.

Behavior results of response time (RT). This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4514360 P r e p r i n t n o t p e e r r e v i e w e d