Kyriakos Lotidis
email: klotidis@stanford.edu.

Panayotis Mertikopoulos
email: panayotis.mertikopoulos@imag.fr.

Nicholas Bambos
email: bambos@stanford.edu.

The stability of matrix multiplicative weights dynamics in quantum games

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

The Stability of Matrix Multiplicative Weights Dynamics in Quantum Games Kyriakos Lotidis * , Panayotis Mertikopoulos ♯ and Nicholas Bambos § Abstract-In this paper, we study the equilibrium convergence and stability properties of the widely used matrix multiplicative weights (MMW) dynamics for learning in general quantum games. A key difficulty in this endeavor is that the induced quantum state dynamics decompose naturally into (i ) a classical, commutative component which governs the dynamics of the system's eigenvalues in a way analogous to the evolution of mixed strategies under the classical replicator dynamics; and (ii ) a non-commutative component for the system's eigenvectors. This non-commutative component has no classical counterpart and, as a result, requires the introduction of novel notions of (asymptotic) stability to account for the nonlinear geometry of the game's quantum space. In this general context, we show that (i ) only pure quantum equilibria can be stable and attracting under the MMW dynamics; and (ii ) as a partial converse, pure quantum states that satisfy a certain "variational stability" condition are always attracting. This allows us to fully characterize the structure of quantum Nash equilibria that are stable and attracting under the MMW dynamics, a fact with significant implications for predicting the outcome of a multi-agent quantum learning process.

I. INTRODUCTION

The advent of quantum information theory -and, with it, the associated "quantum advantage" [START_REF] Preskill | Quantum computing in the NISQ era and beyond[END_REF]- [START_REF] Zhong | Quantum computational advantage using photons[END_REF] -has had a profound impact on computer science and machine learning, from quantum cryptography and shadow tomography [START_REF] Aaronson | Shadow tomography of quantum states[END_REF], to quantum generative adversarial networks (QGANs) and adversarial learning [START_REF] Dallaire-Demers | Quantum generative adversarial networks[END_REF]- [START_REF] Lloyd | Quantum generative adversarial learning[END_REF]. At a high level, the advantages of quantum-based computing are owed to the possibility of preparing superpositions of binary-state quantum systems known as qubits: classical bits cannot lie in superposition, so the calculations that can be performed by classical computers are de facto limited by their binary alphabet and memory structure. In light of this, quantum computing has the potential to greatly accelerate the development of artificial intelligence algorithms and models, with Google's "Sycamore" 54-qubit processor training an autonomous vehicle model in less than 200 seconds [START_REF] Arute | Quantum supremacy using a programmable superconducting processor[END_REF].

In a similar manner, when such models are deployed in a multi-agent context -e.g., as in the case of QGANs or autonomous vehicles -the landscape changes drastically relative to classical non-cooperative frameworks. The main reason for this is again the "quantum advantage": due to the intricacies of decoherence and entaglement -two quantum notions that have no classical counterpart -quantum players can have a distinct advantage over "classical" players, achieving higher payoffs at equilibrium than would otherwise be possible [START_REF] Meyer | Quantum strategies[END_REF], [START_REF] Eisert | Quantum games and quantum strategies[END_REF]. This is again owed to the fact that probabilistic mixing works differently in the quantum and classical worlds: in classical games, a mixed strategy is a probabilistic convex combination of the constituent pure strategies; in quantum games, a mixed state is a probabilistic mixture of the quantum projectors associated to each constituent state. Because of this, a mixed quantum state can return payoffs that lie outside the convex hull of classical mixed strategies, thus providing a tangible advantage to players with access to quantum technologies.

Of course, the extent to which the advantage of quantum players manifests itself is contingent on the players' actually reaching an equilibrium. The recent work of [START_REF] Bostanci | Quantum game theory and the complexity of approximating quantum Nash equilibria[END_REF] has shown that the problem of computing an approximate Nash equilibrium of a quantum game is included in PPAD, so, by the seminal work of [START_REF] Daskalakis | The complexity of computing a Nash equilibrium[END_REF], [START_REF]The complexity of computing a Nash equilibrium[END_REF], it must be complete for this class (since computing a quantum equilibrium is at least as hard as computing a classical one). Thus, given that the dimensionality of a quantum game is exponential in the number of qubits available to each player, computing a Nash equilibrium of a quantum game quickly becomes an intractable affair, in all but the smallest games. On that account, it seems more reasonable to turn to an online learning paradigm, and instead ask:

• Are all Nash equilibria equally likely to occur as outcomes of a multi-agent learning procedure?

• Is there a class of Nash equilibria with an inherent selection bias -either for or against?

Our point of departure for these questions is a compact model of learning based on the widely used matrix multiplicative weights (MMW) algorithm, originally due to [START_REF] Tsuda | Matrix exponentiated gradient updates for on-line Bregman projection[END_REF]. This learning process can be seen as a quantum extension of the standard Hedge / EXP3 algorithms for single-agent learning in bandits and games [START_REF] Vovk | Aggregating strategies[END_REF]- [START_REF] Auer | Gambling in a rigged casino: The adversarial multi-armed bandit problem[END_REF], and it has been used extensively for the computation of quantum Nash equilibria in two-player, zero-sum quantum games [START_REF] Jain | Parallel approximation of non-interactive zero-sum quantum games[END_REF]- [START_REF] Li | Sublinear classical and quantum algorithms for general matrix games[END_REF]. To spotlight the structural properties of the dynamics and to sidestep the technical issues that arise from hyperparameter tuning and the like, we focus throughout on a parameter-free, continuous-time formulation of the dynamics, and we seek to characterize which quantum Nash equilibria are stable and attracting under these dynamics in general quantum games.

Importantly, the quantum state dynamics induced by the MMW learning rule decompose naturally into a "classical" part plus a non-commutative "quantum" component: the eigenvalues of the system follow an equation that is formally analogous to the classical replicator dynamics for finite games, while the system's eigenvectors follow an autonomous flow on the space of unitary matrices that has no classical analogue. Because of this non-commutative structure, deriving the dynamics' equilibrium convergence properties is significantly more difficult because of the nonlinear geometry of the game's state space. In particular, in contrast to finite games (where pure strategies are isolated extreme points), the pure states of a quantum game form a continuous manifold of stationary points (all of them extreme), so the study of stability and convergence questions becomes a highly involved affair.

Nonetheless, despite these topological complications, we show that the MMW dynamics enjoy the following fundamental properties: (i) only pure quantum equilibria can be stable and attracting under MMW; and (ii) as a partial converse, pure quantum states that satisfy a certain "variational stability" condition are always stable and attracting. On that account, our results lead to an implicit quantum "purification" principle: under MMW, mixed states are inherently fragile, and only pure quantum states can be consistently attracting. 1 This fact has significant implications for predicting the outcome of a multi-agent quantum learning process, and it opens up several research directions with potentially far-reaching implications for the deployment of multi-agent quantum systems

II. PRELIMINARIES

We start by briefly reviewing some basics of quantum game theory and introducing the necessary context for our results.

Notation: Given a (complex) Hilbert space H, we will use Dirac's bra-ket notation to distinguish between an element |𝜓⟩ of H and its adjoint ⟨𝜓|; otherwise, when a basis is implied by the context, we will use the dagger notation " †" to denote the Hermitian transpose 𝜓 † of 𝜓. We will also write ℍ 𝑑 for the space of 𝑑 × 𝑑 Hermitian matrices, and ℍ 𝑑 + for the cone of positive-semidefinite matrices in ℍ 𝑑 . Finally, given a real function 𝑓 : ℝ → ℝ and a Hermitian matrix X ∈ ℍ 𝑑 with unitary eigen-decomposition X = 𝑑 𝛼=1 𝑥 𝛼 u 𝛼 u † 𝛼 , we will write 𝑓 (X) for the matrix 𝑓 (X) = 𝑑 𝛼=1 𝑓 (𝑥 𝛼 )u 𝛼 u † 𝛼 . Quantum games: Following [START_REF] Eisert | Quantum games and quantum strategies[END_REF], [START_REF] Gutoski | Toward a general theory of quantum games[END_REF], a quantum game consists of the following primitives: 1) A finite set of players 𝑖 ∈ N = {1, . . . , 𝑁 }.

2) Each player 𝑖 ∈ N has access to a complex Hilbert space H 𝑖 ℂ 𝑑 𝑖 describing the set of (pure) quantum states available to the player (typically a discrete register of qubits). In more detail, a quantum state is an element 𝜓 𝑖 of H 𝑖 with unit norm, so the set of all such states is the unit sphere Ψ 𝑖 {𝜓 𝑖 ∈ H 𝑖 : ∥𝜓 𝑖 ∥ = 1} of H 𝑖 . We will write Ψ 𝑖 Ψ 𝑖 for the space of all ensembles 𝜓 = (𝜓 1 , . . . , 𝜓 𝑁 ) of pure states 𝜓 𝑖 ∈ Ψ 𝑖 that are independently prepared by each 𝑖 ∈ N .

3) The players' rewards are determined by their individual payoff functions 𝑢 𝑖 : Ψ → ℝ. These payoff functions are not arbitrary, but are obtained from a joint positive operatorvalued measure (POVM) quantum measurement process that unfolds as follows [START_REF] Chuang | Quantum Computation and Quantum Information[END_REF]: First, we assume given a finite set of possible measurement outcomes 𝜔 ∈ Ω that a referee can observe from the players' quantum states (e.g., measure a player-prepared qubit to be "up" or "down"). Each such outcome 𝜔 ∈ Ω is associated to a positive semi-definite operator P 𝜔 : H → H that acts on the tensor product H 𝑖 H 𝑖 of the players' individual state spaces; we further assume that 𝜔 ∈Ω P 𝜔 = I so the joint probability of observing 𝜔 ∈ Ω when the system is at state 𝜓 ∈ Ψ is

𝑃 𝜔 (𝜓) = ⟨𝜓 1 ⊗ • • • ⊗ 𝜓 𝑁 |P 𝜔 |𝜓 1 ⊗ • • • ⊗ 𝜓 𝑁 ⟩ (1)
The payoff to each player 𝑖 ∈ N is given by the outcome of this measurement process via a payoff observable 𝑈 𝑖 : Ω → ℝ; specifically, in this context, 𝑢 𝑖 (𝜓) denotes the player's expected payoff at state 𝜓 ∈ Ψ, viz.

𝑢 𝑖 (𝜓) ⟨𝑈 𝑖 ⟩ ≡ ∑ 𝜔 𝑃 𝜔 (𝜓) 𝑈 𝑖 (𝜔). (2)
A quantum game is then defined as a tuple Q ≡ Q(N , Ψ, 𝑢) with players, quantum states, and payoff functions as above.

Mixed states: In addition to pure states, each player 𝑖 ∈ N can also prepare probabilistic mixtures thereof, known as mixed states. In contrast to mixed strategies in classical, finite games, these mixed states are not convex combinations of their pure counterparts; instead, given a family of pure quantum states 𝜓 𝑖 𝛼 𝑖 ∈ Ψ 𝑖 indexed by 𝛼 𝑖 ∈ A 𝑖 , a mixed state is described by a density matrix of the form

X 𝑖 = ∑ 𝛼 𝑖 ∈A 𝑖 𝑥 𝑖 𝛼 𝑖 |𝜓 𝑖 𝛼 𝑖 ⟩⟨𝜓 𝑖 𝛼 𝑖 | (3)
where 𝑥 𝑖 𝛼 𝑖 ≥ 0 is the mixing weight of 𝜓 𝑖 𝛼 𝑖 , and we assume that tr X 𝑖 = 1 (the states 𝜓 𝑖 𝛼 𝑖 are not assumed to be orthogonal in this context). By Born's rule, this means that if each player 𝑖 ∈ N prepares a mixed state according to X 𝑖 , the probability of observing 𝜔 ∈ Ω under X = (X 1 , . . . , X 𝑁 ) will be 𝑃 𝜔 (X)

= 𝛼 𝑥 𝛼 ⟨𝜓 𝛼 |P 𝜔 |𝜓 𝛼 ⟩, where, in multi-index notation, 𝛼 = (𝛼 1 , . . . , 𝛼 𝑁 ), 𝑥 𝛼 = 𝑖 𝑥 𝑖 𝛼 𝑖 , and 𝜓 𝛼 = 𝑖 𝜓 𝑖 𝛼 𝑖 .
Thus, in a slight abuse of notation, the expected payoff to player 𝑖 ∈ N under X will be

𝑢 𝑖 (X) = ∑ 𝜔 ∈Ω ∑ 𝛼∈A 𝑥 𝛼 𝑃 𝜔 (𝜓 𝛼 ) 𝑈 𝑖 (𝜔) = ∑ 𝛼∈A 𝑥 𝛼 𝑢 𝑖 (𝜓 𝛼 ) (4)
Equivalently, we can write the above as:

𝑢 𝑖 (X) = ∑ 𝜔 ∈Ω 𝑈 𝑖 (𝜔) tr[P 𝜔 X 1 ⊗ • • • ⊗ X 𝑁 ] = tr[W 𝑖 X 1 ⊗ • • • ⊗ X 𝑁 ] (5)
where the tensor W 𝑖 = 𝜔 ∈Ω 𝑈 𝑖 (𝜔)P 𝜔 ∈ H for 𝑖 ∈ N encloses all the relevant payoff information of the game and is the quantum equivalent of the "payoff matrix" of player 𝑖 ∈ N . [In this regard, (5) gives a more concrete and concise representation of the payoff structure of Q.]

Contrasting to other classes of games: The expression (4) for a player's expected payoff under a mixed state is reminiscent of mixed extensions of classical finite games, but this association is very tenuous. From a conceptual standpoint, the principal differences are as follows:

1) There is an infinite continuum of pure states 𝜓 ∈ Ψ, not a finite number thereof (as is the case in finite games).

2) The decomposition (3) of a density matrix into pure states is not unique; generically, there may be a continuum of (non-equivalent) families of pure states and mixing weights giving rise to the same density matrix.

3) The convex superposition 𝜆𝜓 + (1-𝜆)𝜓 ′ of two pure states 𝜓 and 𝜓 ′ may give rise to quantum interference terms of the form |𝜓⟩⟨𝜓 ′ | and |𝜓 ′ ⟩⟨𝜓| in the induced payoff; these cross-terms have no analogue in finite games. Continuous game reformulation: Because of the previous discussion, treating a quantum game as a "tensorial" extension of a finite game can be misleading. Instead, it would be clearer for our purposes to view a quantum game as a continuous game where each player 𝑖 ∈ N controls a matrix variable X 𝑖 drawn from the "spectraplex"

X 𝑖 = {X 𝑖 ∈ ℍ 𝑑 𝑖 + : tr X 𝑖 = 1} (6)
and the player's payoff function

𝑢 𝑖 : X ≡ 𝑗 X 𝑗 → ℝ is linear in every player's density matrix X 𝑗 ∈ X 𝑗 , 𝑗 ∈ N . Since 𝑢 𝑖 is linear in X 𝑖 , letting V 𝑖 (X) = ∇ X ⊤ 𝑖 𝑢 𝑖 (X)
denote the individual payoff gradient of player 𝑖, we can write the payoff of player 𝑖 in terms of the gradient V 𝑖 as:

𝑢 𝑖 (X 𝑖 ; X -𝑖 ) = tr[X 𝑖 V 𝑖 (X)] for all X ∈ X . ( 7
)
Nash equilibrium: In our quantum setting, the classical solution concept of a Nash equilibrium (NE) characterizes mixed quantum states X * ∈ X which discourage unilateral deviations in the sense that

𝑢 𝑖 (X * ) ≥ 𝑢 𝑖 (X 𝑖 ; X * -𝑖 ) for all X 𝑖 ∈ X 𝑖 , 𝑖 ∈ N (NE)
where we write (X 𝑖 ; X -𝑖 ) = (X 1 , . . . , X 𝑖 , . . . , X 𝑁 ) for the choice of player 𝑖 relative to all other players. Since X 𝑖 is convex and 𝑢 𝑖 is linear in X 𝑖 , the existence of Nash equilibria follows from the seminal theorem of [START_REF] Debreu | A social equilibrium existence theorem[END_REF]. Standard arguments [START_REF] Facchinei | Finite-Dimensional Variational Inequalities and Complementarity Problems[END_REF], [START_REF] Scutari | Convex optimization, game theory, and variational inequality theory in multiuser communication systems[END_REF] show that the Nash equilibria of Q are precisely the solutions of the variational inequality

tr[V(X * ) (X -X * )] ≤ 0 for all X ∈ X (VI)
where V(X) = (V 1 (X), . . . , V 𝑁 (X)). We will use this equivalence freely in the sequel.

III. LEARNING DYNAMICS

The matrix multiplicative weights algorithm: The most widely used algorithm for computing Nash equilibria of quantum games is the so-called matrix multiplicative weights (MMW) algorithm which, in our notation, unfolds as

Y 𝑖 (𝑡 + 1) = Y 𝑖 (𝑡) + 𝜂V 𝑖 (X(𝑡)) X 𝑖 (𝑡) = exp(Y 𝑖 (𝑡)) tr[exp(Y 𝑖 (𝑡))] (MMW)
where 𝜂 > 0 is a "learning rate" hyperparameter. This algorithm was first introduced in the context of matrix learning by [START_REF] Tsuda | Matrix exponentiated gradient updates for on-line Bregman projection[END_REF], and can be seen as a matrix analogue of the wellknown "multiplicative / exponential weights" (EW) algorithm for learning in bandits and games [START_REF] Auer | Gambling in a rigged casino: The adversarial multi-armed bandit problem[END_REF], [START_REF] Mertikopoulos | Learning in games with continuous action sets and unknown payoff functions[END_REF]. In the context of quantum games, [START_REF] Jain | Parallel approximation of non-interactive zero-sum quantum games[END_REF] showed that (MMW) can be used to compute the Nash equilibrium of two-player, zero-sum quantum games by running it for 𝑇 iterations and taking the time-average X = (1/𝑇) 𝑇 𝑡=1 X(𝑡) of the generated states; if the learning rate of (MMW) is chosen appropriatelyspecifically, as

𝜂 = O(1/ √ 𝑇) -the algorithm's output state X is an O(1/
√ 𝑇)-equilibrium of the underlying game.

Beyond the two-player, zero-sum case however, the behavior of (MMW) in general quantum games is not well understood. Moreover, an additional limitation from a learning viewpoint is that the guarantees of (MMW) concern the time-averaged state X and not the induced sequence of play X(𝑡). The behavior of the former can be quite different from the latter, even in min-max quantum games; in particular, as was shown in [START_REF] Jain | Matrix multiplicative weights updates in quantum games: Conservation law & recurrence[END_REF], even in the continuous-time limit where the algorithm's learning rate is taken arbitrarily small, X(𝑡) may fail to converge altogether.

In view of this, we will take an approach similar to [START_REF] Jain | Matrix multiplicative weights updates in quantum games: Conservation law & recurrence[END_REF], and we will focus on the continuous-time limit of (MMW), namely the dynamics

Y 𝑖 = V 𝑖 (X) X 𝑖 = 𝚲(Y 𝑖 ) exp(Y 𝑖 ) tr[exp(Y 𝑖 )] (MMWD)
with Y 𝑖 ∈ Y 𝑖 ℍ 𝑑 𝑖 , as per (MMW). The benefit of this reformulation is that it allows us to focus squarely on the dynamics' structural properties without being bogged down by hyperparameter questions and the like. We begin our analysis below by deriving the induced dynamics of the players' mixed quantum states X 𝑖 ∈ X 𝑖 . The quantum state dynamics of MMW: Under (MMWD), the evolution of the players' mixed states X (𝑡) is described implicitly via that of the auxiliary matrix Y(𝑡). However, obtaining an explicit expression for the dynamics of X (𝑡) is considerably more difficult because the rules of matrix calculus do not provide an analytic expression for the tensor derivative. Following [START_REF] Mertikopoulos | Learning in an uncertain world: MIMO covariance matrix optimization with imperfect feedback[END_REF], we will circumvent this difficulty by taking a unitary eigendecomposition of X of the form X = 𝑑 𝛼=1 𝑥 𝛼 u 𝛼 u † 𝛼 , where 𝑥 𝛼 ≥ 0, u 𝛼 ∈ H is a unit-norm eigenvector of X corresponding to 𝑥 𝛼 . Since, in general, X and V do not commute, the eigenvalues and eigenvectors of X will evolve in a coupled manner determined by (MMWD).

To make this coupling explicit, we will suppress player indices for notational simplicity, and we will differentiate X in (MMWD) with respect to 𝑡. Doing this we obtain:

X = 1 tr[exp(Y)] 𝑑 𝑑𝑡 exp(Y) - exp(Y) tr[exp(Y)] 2 tr 𝑑 𝑑𝑡 exp(Y)
and hence, by taking Fréchet derivatives [START_REF] Wilcox | Exponential operators and parameter differentiation in quantum physics[END_REF], we get:

𝑑 𝑑𝑡 exp(Y) = ∫ 1 0 𝑒 (1-𝑠)Y Y𝑒 𝑠Y 𝑑𝑠 = tr[exp(Y)] ∫ 1 0 X 1-𝑠 V(X)X 𝑠 𝑑𝑠 (8)
which, after some algebraic manipulations, ultimately yields

X = ∫ 1 0 X 1-𝑠 V(X)X 𝑠 𝑑𝑠 - exp Y tr[exp(Y)] ∫ 1 0 tr[XV(X)] 𝑑𝑠 = ∫ 1 0 X 1-𝑠 V(X)X 𝑠 𝑑𝑠 -tr[XV(X)]X (9)
Thus, applying u † 𝛼 to the left and u 𝛽 to the right of (9), we get:

u † 𝛼 Xu 𝛽 = ∫ 1 0 𝑥 1-𝑠 𝛼 𝑉 𝛼𝛽 𝑥 𝑠 𝛽 𝑑𝑠 -𝑥 𝛼 𝛿 𝛼𝛽 ∑ 𝜅 𝑥 𝜅 𝑉 𝜅 𝜅 (10)
Finally, denoting u † 𝛼 Xu 𝛽 by [ X] 𝛼𝛽 , equation [START_REF] Bostanci | Quantum game theory and the complexity of approximating quantum Nash equilibria[END_REF] gives the quantum replicator dynamics (QRD):

[ X] 𝛼𝛽 =        𝑥 𝛼 [𝑉 𝛼𝛼 -𝜅 𝑥 𝜅 𝑉 𝜅 𝜅 ] for 𝛼 = 𝛽 𝑥 𝛽 -𝑥 𝛼 log 𝑥 𝛽 -log 𝑥 𝛼 𝑉 𝛼𝛽 for 𝛼 ≠ 𝛽 (QRD)
The diagonal part of (QRD) is formally analogous to the replicator dynamics of evolutionary game theory [START_REF] Taylor | Evolutionary stable strategies and game dynamics[END_REF]- [START_REF] Sandholm | Population Games and Evolutionary Dynamics[END_REF] and captures the evolution of the eigenvalues of X (𝑡). Thus, (QRD) provides an explicit expression for the evolution of mixed states under (MMWD). Next, we will study in detail how the classical and quantum components of (MMWD) interface to determine the player's long-run behavior.

IV. CONVERGENCE AND STABILITY

We are now in a position to proceed with our convergence analysis. A key element to keep in mind here is that a quantum game may admit several Nash equilibria, so it is not reasonable to expect a global convergence result that applies to all games -this, in fact, would violate the impossibility result of [START_REF] Hart | Uncoupled dynamics do not lead to Nash equilibrium[END_REF]. In view of this, we will focus on the next best thing, that is, to identify those quantum states that are "locally stable and attracting" as formalized below.

Notions of stability and convergence: To state our results, recall first that a flow on an abstract metric space Z is a continuous map 𝜙 : ℝ × Z → Z such that (a) 𝜙 0 (𝑧) = 𝑧; and (b) 𝜙 𝑡+𝑠 (𝑧) = 𝜙 𝑡 (𝜙 𝑠 (𝑧)) for all 𝑡, 𝑠 ∈ ℝ and all 𝑧 ∈ Z.

Informally, a flow is usually generated by the solution orbits of a system of well-posed ordinary differential equations (ODEs), such as (MMWD): in this interpretation, 𝜙 𝑡 (𝑧) simply denotes the position at time 𝑡 of the ODE solution that starts at 𝑧 at time 𝑡 = 0.

With this in mind, given a point 𝑝 ∈ Z, we will say that (i) 𝑝 is (Lyapunov) stable if any orbit of (MMWD) that starts close enough to 𝑝 remains close enough; formally, for every neighborhood U of 𝑝 in Z, we posit that there exists some (smaller) neighborhood U ′ of 𝑝 in Z such that 𝜙 𝑡 (U ′ ) ⊆ U for all 𝑡 ∈ ℝ.

(ii) 𝑝 is attracting if all nearby orbits converge to 𝑝; formally, there exists a neighborhood U such that lim 𝑡→∞ 𝜙 𝑡 (𝑧) = 𝑝 for all 𝑧 ∈ U.

In what follows, we will seek to characterize precisely the states that are "stable and attracting" under (MMWD) -or, in more formal language, asymptotically stable.

Learning in the spectraplex: Moving forward, a quick look at the quantum replicator dynamics (QRD) reveals the following structural property: an eigenvalue of X that is initially zero in (QRD) will always remain zero; likewise, an eigenvalue that is initially positive, will always remain positive. Formally, this means that the kernel ker(X) of X remains invariant under (QRD); hence, given that the linear span of a Hermitian matrix is the orthocomplement of its kernel, the same holds for im(X).

The fact that the kernel -or, equivalently, the image -of a density matrix remains invariant under (QRD) is the quantum analogue of the fact that the support of a mixed strategy profile remains invariant under the standard replicator dynamics. In the context of finite games, an immediate consequence of this invariance is that all pure strategy profiles are stationary (as zero-dimensional faces of the simplex). This property extends to (QRD) and, in fact, to the entire class of mixedstate dynamics under study: formally, under (QRD), all pure quantum states are stationary.

That being said, the major qualitative difference between the quantum and classical regimes is that, in quantum games, there is a continuum of pure states, namely the entire manifold Ψ of rank 1 density matrices (a product of spheres). By contrast, in finite games, the pure states are the corners of the simplex Δ spanned by the player's pure strategies, so they are finite in number and isolated. As a result, in classical finite games, a pure strategy profile can be asymptotically stable; in quantum games, since every pure state is surrounded by other invariant states, it cannot.

A second major difference is that, in finite games, strict Nash equilibria are robust: a small perturbation of the payoffs of the game does not change the game's strict equilibria.

In quantum games, this robustness disappears: indeed, the variational characterization (VI) of Nash equilibria means that V(X * ) must be an element of the normal cone to X at X * ; however, the normal cone to the spectraplex at a matrix of rank 1 has empty topological interior, so the required membership property cannot be robust (for a graphical illustration, see Fig. 1). In particular, any perturbation to the payoff structure of a quantum game may displace the equilibrium in question on the manifold of pure states Ψ.

Consistency and variational stability: In view of the above, we can draw two major conclusions for the quantum setting:

1) Any concept of asymptotic stability must also include a notion of consistency: a state cannot be accessed if it is absent from the linear span of the dynamics' initial state.

2) Any concept of robustness must incorporate a notion of variational stability: small perturbations to an equilibrium must tend to reinstate it.

We formalize these ideas as follows:

p NC ∆ (p) x v(x) v(p) U NC X (P) P X V(X)
U ∩ X P

V(P)

Fig. 1: The geometric discrepancy between the classical and quantum regimes (left and right respectively). In finite games, the normal cone NC Δ ( 𝑝) to the simplex at a pure strategy 𝑝 has nonempty topological interior, so pure Nash equilibria are generically robust: if 𝑣( 𝑝) is normal to Δ at 𝑝, it will remain normal to Δ after a small perturbation. On the other hand, in quantum games, the normal cone NC X (P) to the spectraplex at a pure state P is a ray, so pure Nash equilibria are not robust. We also note the different geometry of pure states: in the simplex, pure strategies are isolated extreme points; in the spectraplex, pure states form a continuous manifold.

Definition 1. The domain of consistency of a state P ∈ X is the set X P {X ∈ X : ker(X) ≤ ker(P)}, i.e., the set of mixed states whose linear span contains that of P. Then, given a flow 𝝌 : ℝ × X → X , we will say that: 1) P is consistently attracting if it attracts all nearby consistent initializations, i.e., it admits a neighborhood U such that lim 𝑡→∞ 𝝌 𝑡 (X) = P for all X ∈ U ∩ X P .

2) P is consistently asymptotically stable if it is stable and consistently attracting.

Finally, to ensure a certain degree of stability to perturbations, we will consider the following notion of variational stability:

Definition 2. We say that X * ∈ X is variationally stable if there exists a neighborhood U of X * in X such that tr[V(X) (X -X * )] < 0 for all X ∈ U \{X * }.

(VS)

Intuitively, Definition 1 captures precisely the accessibility condition that we discussed above, while Definition 2 should be seen as an equilibrium refinement in the spirit of the seminal concept of evolutionary stability [START_REF] Smith | The logic of animal conflict[END_REF], [START_REF] Smith | Evolution and the Theory of Games[END_REF].

With all this in hand, our main result is as follows:

Theorem 1. Fix some state X * ∈ X of a quantum game Q ≡ Q(N , Ψ, 𝑢). Then: a) If X * is consistently asymptotically stable, then it is pure. b) If X * satisfies (VS), it is consistently asymptotically stable.
Before presenting the proof of Theorem 1, some remarks are in order. Perhaps the most important one is that, if the standard notion of asymptotic stability is ruled out by the geometry of the game's state space, (MMWD) achieves the next best thing: by definition, states that are consistently asymptotically stable attract all but a measure zero of nearby initial conditions, and Theorem 1 shows that only pure states can have this property. This selection result has important implications for quantum games because it shows that the MMW rule essentially "collapses" an initial mixed state to a specific pure state -and this, despite the fact that any mixed state can be prepared by an infinity of combinations of pure states.

On the flip side of all this, the implication that variationally stable states are also (consistently) asymptotically stable provides a relevant convergence criterion for (MMWD) and indicates an inherent robustness to variations of player beliefs and predictions. In particular, since (VS) only involves the primitives of the underlying game, the fact that such states are attracting under the MMW dynamics means that they can be seen as universal attractors -and since only pure states can have this property, we also infer indirectly that variationally stable states are a fortiori pure.

The proof of the first part of Theorem 1 consists of three steps. First, we construct a measure 𝜇 on ri X that is invariant under the flow 𝝌, i.e., 𝜇( 𝐴) = 𝜇( 𝝌 𝑡 ( 𝐴)) for all measurable sets 𝐴 of ri X and 𝑡 ≥ 0, where by 𝝌 𝑡 ( 𝐴) we denote the image of 𝐴 after time 𝑡, and by ri X the relative interior of X . For this, we restrict the dynamics over the dual space Y to a quotient space Z that has the same dimension as X , and invoke a volume-conservation argument. Next, we show that if X * ∈ ri X , it cannot be consistently asymptotically stable. To prove this, we need the following lemma, which, in words, says that if X * ∈ ri X is consistently asymptotically stable, then all the orbits starting nearby converge uniformly in time, and whose proof lies at the end of the section.

Lemma 1. Suppose that X * ∈ ri X is consistently asymptotically stable under (MMWD). Then, for every sufficiently small compact neighborhood U of X * in ri X , we have

lim 𝑡→∞ sup X∈U ∥ 𝝌 𝑡 (X) -X * ∥ = 0. (11)
Finally, we show that if X * is consistently asymptotically stable, it is pure, using a reduction argument and considering faces of X that contain X * in their relative interior.

The first two steps preclude convergence to full-rank equilibria, while excluding lower-rank equilibria requires more delicate arguments, where the notion of consistency plays a major role (and has no classical counterpart), and is succeeded in the third step. With the above mind, we are now ready to proceed to the proof of Theorem 1.

Proof of Theorem 1(a). We will present our proof in three steps, as described earlier on.

Step 1: First, we will need to collapse the dual space Y of V ≡ ℍ 𝑑 to a subspace Z with constant trace. The reason for this is that the map 𝚲 is not injective, since for all 𝜆 ∈ ℝ, we have 𝚲(Y + 𝜆I) = 𝚲(Y). This means that the inverse image of any point in im 𝚲 always contains a copy of the real line. This collapse is intended to "quotient out" this redundancy so as to enable volume comparisons later on. To succeed this, consider the transformed matrix variables

Z = Y -(1/𝑑) tr[Y]I (12)
so tr Z = 0 for all Y ∈ Y. Formally, this transformation can be represented via the map 𝚷 : Y → Z, where

Z = {Z ∈ Y : tr Z = 0} (13)
and 𝚷 : Y ↦ → 𝚷(Y) = Z is defined via [START_REF]The complexity of computing a Nash equilibrium[END_REF] above. As such, Z can be seen as a representative of the equivalence relation Y ∼ Y + 𝜆I, 𝜆 ∈ ℝ, so, in turn, Z can be identified with the quotient Y/∼. By the definition of 𝚲, we have 𝚲(Y) = 𝚲(𝚷(Y)) = 𝚲(Z) for all Y ∈ Y, and (MMWD) gives

Z = 𝑑 𝑑𝑡 [Y -(1/𝑑) tr[Y]I] = Y -(1/𝑑) tr[ Y]I = V(𝚲(Y)) -(1/𝑑) tr[V(𝚲(Y))]I = V(𝚲(Z)) -(1/𝑑) tr[V(𝚲(Z))]I (14)
so we can rewrite (MMWD) in terms of Z as )] tr I = 0, it follows that any traceless initial condition of (MMWD Z ) will remain traceless -and hence remain in Z for all 𝑛 ≥ 0. We thus conclude that (MMWD Z ) is a well-posed dynamical system on Z with induced flow 𝜻 : ℝ × Z → Z. The following proposition states that the (MMWD Z ) dynamics are volumepreserving, and this key property will be used to construct the invariant measure 𝜇.

Z = V(X) -(1/𝑑) tr[V(X)]I X = 𝚲(Z) (MMWD Z ) Since tr[ Z] = tr[V(X)] -(1/𝑑) tr[V(X
Proposition 1. Let W ⊆ Z be an open set of initial conditions of (MMWD Z ), and let W 𝑡 = 𝜻 𝑡 (W), 𝑡 ≥ 0, denote the evolution of W under the flow 𝜻 : ℝ × Z → Z of (MMWD Z ). Then, vol(W 𝑡 ) = vol(W) for all 𝑡 ≥ 0, where the form vol(•) stands for the Lebesgue measure on Z.

The proof of Proposition 1 is based on the fact that each player's payoff function is individually linear in the player's own density matrix, so the individual gradient fields V 𝑖 do not depend on X 𝑖 . The full proof lies at the end of the section.

Continuing with the proof of Theorem 1, by descending to the quotient space Z, the map 𝚲 factors through Z as 𝚲 0 : Z → X with 𝚲 = 𝚲 0 • 𝚷. Now, let 𝜆 denote the Lebesgue measure on Z, and let 𝜇 denote the pushforward of 𝜆 to X via 𝚲 0 , i.e., 𝜇( 𝐴) = 𝜆(𝚲 -1 0 ( 𝐴)) for all Borel sets 𝐴 ⊆ X . Finally, by Proposition 1 and since 𝜻 conjugates 𝝌 in the sense that 𝝌 𝑡 • 𝚲 0 = 𝚲 0 • 𝜻 𝑡 for all 𝑡 ≥ 0, we get that 𝜇( 𝝌 𝑡 ( 𝐴)) = 𝜇( 𝐴) for all Borel sets 𝐴 in ri X and 𝑡 ≥ 0.

Step 2: We will now show that X * cannot belong to ri X . For the sake of contradiction, suppose X * ∈ ri X is consistently asymptotically stable, and let U be a sufficiently small compact neighborhood of X * , as per Lemma 1. Then, it holds lim 𝑡→∞ sup X∈U ∥ 𝝌 𝑡 (X) -X * ∥ = 0, which implies that:

lim 𝑡→∞ 𝜇( 𝝌 𝑡 (U )) = 𝜇({X * }) = 0 < 𝜇(U ). (15)
However, by Step 1 of the proof, we have that lim 𝑡→∞ 𝜇( 𝝌 𝑡 (U )) = 𝜇(U ), since 𝜇( 𝝌 𝑡 (U )) = 𝜇(U ) for all 𝑡 ≥ 0. Combining it with [START_REF] Littlestone | The weighted majority algorithm[END_REF], we arrive at a contradiction.

Step 3: Suppose, finally, that X * is consistently asymptotically stable with rank(X * ) > 1. Note that if X * is full-rank, it cannot be consistently asymptotically stable, as shown at the Step 2 of the proof. Define the set:

A X * {X ∈ X : ker(X) ≥ ker(X * )} (16)
which is convex. Hence, considering the restriction of the dynamics on A X * , Step 2 of the proof shows that a point in ri A X * cannot be consistently asymptotically stable on the induced topology. So, it remains to show that X * cannot be consistently asymptotically stable.

For the sake of contradiction, suppose it is. Then, according to Definition 1, there exist U of X * in X such that lim 𝑡→∞ 𝝌 𝑡 (X) = X * for all X ∈ U ∩ X X * . But, since U is a neighborhood of X * in X , we have that

(U ∩ X X * )∩A X * = U ∩{X ′ ∈ X : ker(X ′ ) = ker(X * )} (17)
is a neighborhood of X * , and, thus, the restriction of U ∩{X ′ ∈ X : ker(X ′ ) = ker(X * )} on A X * is a neighborhood of X * in A X * . Therefore, by our previous argument on the induced dynamics on A X * , there exists X 0 ∈ U ∩{X ′ ∈ X : ker(X ′ ) = ker(X * )} such that 𝝌 𝑡 (X 0 ) ̸ → X * , as 𝑡 → ∞. This contradicts our assumption, so the proof is complete. ■

The second part of Theorem 1 leverages an energy argument in the spirit of Lyapunov's direct method. In the spirit of the analysis of the classical EW algorithm [START_REF] Auer | Gambling in a rigged casino: The adversarial multi-armed bandit problem[END_REF], [START_REF] Mertikopoulos | Learning in games via reinforcement and regularization[END_REF], a natural choice of energy function is

𝐸 (𝑡) = tr[X * log X * ] + log tr[exp(Y(𝑡))] -tr[X * Y(𝑡)] (18)
which is in turn linked to (VS) via Lemma 2 below:

Lemma 2. Let X(𝑡), 𝑡 ≥ 0, be a trajectory of play under (MMWD), and let 𝐸 (𝑡) be defined as per [START_REF] Aaronson | Online learning of quantum states[END_REF]. Then:

𝐸 (𝑡) = tr[V(X(𝑡)) (X(𝑡) -X * )] (19)
By a direct calculation, it can be shown that (i) 𝐸 (𝑡) ≥ 0 with equality if and only if X * = 𝚲(Y(𝑡)); and (ii) 𝐸 (𝑡) → 0 if and only if 𝚲(Y(𝑡)) → X * as 𝑡 → ∞. Thus, putting everything together, we conclude that lim 𝑡→∞ 𝐸 (𝑡) exists; then, by a trapping argument, it can be shown that there exists a sequence of times 𝑡 𝑛 → ∞ such that X (𝑡 𝑛 ) → X * , which allows us to conclude that 𝐸 (𝑡) → 0 and ultimately yields our claim. Specifically, we have:

Proof of Theorem 1(b). Suppose that X * is variationally stable, i.e., tr[V(X) (X-X * )] < 0, for all X ∈ U \{X * }, where U is a neighborhood of X * . Then, by Lyapunov's direct method for the energy function 𝐸 (𝑡) defined in [START_REF] Aaronson | Online learning of quantum states[END_REF], we get that X * is stable, thus, there exists neighborhood U ′ of X * , such that X(𝑡) ∈ U for all 𝑡 ≥ 0, if X(0) ∈ U ′ . This means that if X(0) ∈ U ′ , then tr[V(X(𝑡)) (X(𝑡) -X * )] < 0 for all 𝑡 ≥ 0.

In what follows, we will show that X(𝑡) → X * .

Since X(𝑡) ∈ U for all 𝑡 ≥ 0, and using Lemma 2 along with (VS), we obtain that 𝐸 (𝑡) = tr[V(X(𝑡)) (X(𝑡) -X * )] < 0. Hence, 𝐸 (𝑡) is decreasing if X(0) ∈ U ′ . Now, we will show that X * is an 𝜔-limit of the flow, i.e., there exists a sequence {𝑡 𝑛 } 𝑛∈ℕ such that X(𝑡 𝑛 ) → X * as 𝑛 → ∞.

For the sake of contradiction, suppose such a sequence does not exist. Then, ∥X(𝑡) -X * ∥ is bounded away from zero, which implies that there exists 𝑐 > 0 such that

tr[V(X(𝑡)) (X(𝑡) -X * )] < -𝑐 (20)
Integrating over time, we get 𝐸 (𝑡) -𝐸 (0) < -𝑐𝑡, which implies that 𝐸 (𝑡) → -∞ as 𝑡 → ∞. This contradicts the nonnegativity property of the energy function, thus, we conclude that there exists a sequence {𝑡 𝑛 } 𝑛∈ℕ such that X(𝑡 𝑛 ) → X * as 𝑛 → ∞. Hence, we get that 𝐸 (𝑡 𝑛 ) → 0 as 𝑛 → ∞. Therefore, since 𝐸 (𝑡) is decreasing and 𝐸 (𝑡 𝑛 ) → 0, we conclude that 𝐸 (𝑡) → 0 as 𝑡 → ∞. Thus, we conclude that X(𝑡) → X * , i.e., X * is consistently asymptotically stable. ■

Finally, we present the proofs of the intermediate results, required for our analysis.

Proof of Lemma 1. First of all, since X * ∈ ri X , we have that ker(X * ) = {0}, which implies that X X * = ri X . Hence, for any open set 𝑂 in ri X , it holds that 𝑂 ∩ X X * = 𝑂. Now, let U 0 be the basin of attraction of X * , according to the definition of the consistent asymptotic stability in Definition 1, and let U ⊆ U 0 be a compact neighborhood of X * . Suppose, for the sake of contradiction, that sup X∈U ∥ 𝝌 𝑡 (X) -X * ∥ ̸ → 0. This implies that there exists 𝜀 > 0, a sequence {𝑡 𝑛 } 𝑛∈ℕ with 𝑡 𝑛 → ∞ as 𝑛 → ∞, and

X 𝑛 ∈ U for 𝑛 ∈ ℕ, such that ∥ 𝝌 𝑡 𝑛 (X 𝑛 ) -X * ∥ ≥ 𝜀 for all 𝑛 ∈ ℕ (21)
Since U is compact, we may assume (by taking subsequences, if necessary) that X 𝑛 converges to some limit point X ∞ ∈ U . Now, since X * is Lyapunov stable, there exists a neighborhood U ′ of X * , such that the trajectory 𝝌 𝑡 (X) remains within 𝜀/2distance of X * , if X ∈ U ′ , or, ∥ 𝝌 𝑡 (X) -X * ∥ < 𝜀/2 for all 𝑡 ≥ 0 and X ∈ U ′

Define the hitting time 𝜏 inf{𝑡 ≥ 0 : 𝝌 𝑡 (X ∞ ) ∈ U ′ } as the first time that the trajectory enters U ′ when starting from X ∞ . It is easy to see that 𝜏 < ∞, since X ∞ ∈ U ⊆ U 0 , and, thus, ∥ 𝝌 𝑡 (X ∞ ) -X * ∥ → 0 as 𝑡 → ∞ by the asymptotic stability of X * . By continuity of 𝝌, we readily get that there exists a neighborhood D of X ∞ such that 𝝌 𝜏 (D) ⊆ U ′′ , where U ′′ is a neighborhood of X * with ∥ 𝝌 𝑡 (X) -X * ∥ < 𝜀 for all 𝑡 ≥ 0 and X ∈ U ′′ . Since X 𝑛 → X ∞ , we conclude that X 𝑛 ∈ D for all sufficiently large 𝑛, which, in turn, implies that 𝝌 𝜏 (X 𝑛 ) ∈ U ′′ . Moreover, by definition of the sequence {𝑡 𝑛 } 𝑛∈ℕ , we have that 𝑡 𝑛 → ∞, which gives that 𝑡 𝑛 > 𝜏 for 𝑛 sufficiently large, since 𝜏 < ∞, as argued. Thus, for 𝑛 large, we have ∥ 𝝌 𝑡 𝑛 (X 𝑛 ) -X * ∥ < 𝜀, which contradicts [START_REF] Li | Sublinear quantum algorithms for training linear and kernel-based classifiers[END_REF]. ■

Proof of Proposition 1. The key observation here is that the dynamics (MMWD Z ) are incompressible -that is, divergence-free. Indeed, since V 𝑖 does not depend on X 𝑖 (it is not possible to suppress player indices here), we readily

and, the proof is complete. ■

V. SIMULATION SETUP

We consider a 2-player symmetric quantum game, which is obtained as the quantum analog of the 2-player finite symmetric game with 2 actions, {𝛼 1 , 𝛼 2 } and {𝛽 1 , 𝛽 2 } and payoff matrix 𝑃 = 1 2 2 1 , same for both players. The action profiles (𝛼 1 , 𝛽 2 ), (𝛼 2 , 𝛽 1 ) are strict Nash equilibria of the finite game. The trajectories are visualized in Bloch spheres [START_REF] Nielsen | Quantum Computation and Quantum Information: 10th Anniversary Edition[END_REF]. The payoff information of the quantum game is encoded in the Hermitian matrix 𝑊 = diag(1, 2, 2, 1), as per [START_REF] Dallaire-Demers | Quantum generative adversarial networks[END_REF], which is the same for both players. The green lines correspond the trajectories of player 1, while the red ones to the trajectories of player 2. The blue marker points correspond to the initial points of the trajectories, and in each subfigure of Fig. 2 separately, the trajectories starting with markers of the same shape correspond to the trajectories of the two players obtained under the same execution of the dynamics (10 different initializations of the dynamics in total, 5 in each subfigure). In the left subfigure, we see that the trajectories of the first and second player converge to the density matrices 1 0 0 0 and 0 0 0 1 , accordingly, i.e., the top and bottom points of the sphere, while in the right subfigure, we see the opposite. This is happening because the initial conditions of the dynamics in the left subfigure lie in the basin of attraction of the one variationally stable equilibrium, while in the right subfigure lie in the basin of the other.

VI. CONCLUDING REMARKS

When quantum computing models are deployed in a multiagent context -from autonomous vehicles to quantum GANs -the players' interaction landscape changes dramatically relative to classical interactions. The study of game-theoretic learning in this quantum setting is still in its infancy, so it is not clear at this stage what can be expected by quantum players with bounded rationality. In this regard, the study of the (MMWD) provides the following important insights: the geometric structure of quantum state space leads to an inflation of "learning traps" (stationary states) that have no classical counterpart; nonetheless, the only states that can be stable and attracting under (MMWD) are the game's pure quantum equilibria. Solidifying our understanding of the limits of quantum game-theoretic learning is a particularly fruitful research direction with potentially far-reaching implications for the deployment of multi-agent quantum computing systems.

Fig. 2 : 1 0tr X 1 -

 211 Fig. 2: Convergence of trajectories to variationally stable equilibria in a 2-player quantum anti-coordination game, visualized in Bloch spheres. The green trajectories correspond to player 1, while the red ones to player 2. The blue markers indicate the initial points of the trajectories. In each subfigure separately, the trajectories starting with markers of the same shape correspond to the the trajectories of the two players obtained under the same execution of the dynamics. have ∇ Z ⊤ 𝑖 V 𝑖 (𝚲(Y)) = 0 for all 𝑖 ∈ N . This immediately implies that the fieldW 𝑖 (Z) = V 𝑖 (𝚲(Z)) -(1/𝑑 𝑖 ) tr[V 𝑖 (𝚲(Z))]I(23)has div Z (W(Z)) = 0, so (MMWD Z ) is incompressible. Our claim then follows from Liouville's formula[START_REF] Arnold | Mathematical Methods of Classical Mechanics[END_REF].■Proof of Lemma 2. Differentiating the energy function 𝐸 (𝑡) with respect to 𝑡, and using (8) we obtain: 𝑑 𝑑𝑡 𝐸 (𝑡) = 1 tr[exp(Y(𝑡))] 𝑑 𝑑𝑡 tr[exp(Y(𝑡))]tr[X * Y(𝑡)]

In classical games, a version of the above results is sometimes referred to as the "folk theorem" of evolutionary game theory[START_REF] Cressman | Evolutionary Dynamics and Extensive Form Games[END_REF]-[START_REF] Flokas | No-regret learning and mixed Nash equilibria: They do not mix[END_REF].

P. Mertikopoulos was supported by project MIS 5154714 of the National Recovery and Resilience Plan Greece 2.0 funded by the European Union under the NextGenerationEU Program, and by the French National Research Agency (ANR) in the framework of the "Investissements d'avenir" program (ANR-15-IDEX-02), the LabEx PERSYVAL (ANR-11-LABX-0025-01), MIAI@Grenoble Alpes (ANR-19-P3IA-0003). K. Lotidis was supported by the Onassis Foundation (F ZR 033-1/2021-2022). N. Bambos was supported by the Koret Foundation via the Digital Living 2030 project.


Lucie Vignal

Cassandre Vielle

Maya Williams

Nicolas Maurice

Mickael Degoulet

Christelle Baunez
email: christelle.baunez@univ-amu.fr

SUBTHALAMIC HIGH-FREQUENCY DEEP BRAIN STIMULATION REDUCES ADDICTION-LIKE ALCOHOL USE AND SUPPRESSES THE OVERCONSUMPTION INDUCED BY THE PEER'S PRESENCE

Keywords: alcohol, social context, subthalamic nucleus, deep brain stimulation, peer's presence

whether they are published or not. The documents may come

Introduction

Drug consumption is a complex behavior influenced by a multitude of factors, including environmental context and social interactions. Social context has been demonstrated to play a crucial role in modulating drug intake in both human and animal models. Indeed, social factors are known to affect multiples aspects of drug use such as initiation, maintenance, attempts to quit, and relapse, potentially boosting, mediating, or reducing addictive behaviors [START_REF] Bardo | Individual Differences and Social Influences on the Neurobehavioral Pharmacology of Abused Drugs[END_REF][START_REF] Pelloux | Social Modulation of Drug Use and Drug Addiction[END_REF][START_REF] Venniro | Volitional Social Interaction Prevents Drug Addiction in Rat Models[END_REF]. Proximal social factors, such as the presence of peers during drug consumption, have a significant influence by decreasing or increasing drug intake [START_REF] Pelloux | Social Modulation of Drug Use and Drug Addiction[END_REF][START_REF] Strickland | The Effects of Social Contact on Drug Use: Behavioral Mechanisms Controlling Drug Intake[END_REF]2015). The identity of the peer is also a critical element that can modulate drug consumption. For cocaine, the presence of a non-familiar (stranger) peer leads to a stronger reduction in cocaine intake than the presence of a familiar one in both rats and humans [START_REF] Giorla | Peer Presence and Familiarity as Key Factors to Reduce Cocaine Intake in Both Rats and Humans: An Effect Mediated by the Subthalamic Nucleus[END_REF]). The largest reduction was observed in presence of a stranger cocaine-naive peer. A similar beneficial effect of presence of such a peer has been also shown on addictive-like (i.e. escalated) cocaine consumption in rats (Vielle et al. preprint).

However, while the presence of peers is protective against cocaine use, its effect might be opposite for alcohol. Indeed, rats chose alcohol over social interaction in a choice procedure [START_REF] Augier | Wistar Rats Choose Alcohol over Social Interaction in a Discrete-choice Model[END_REF][START_REF] Marchant | Rats Choose Alcohol over Social Reward in an Operant Choice Procedure[END_REF], and consumed more alcohol in presence of a non-naive alcohol peer, regardless of their familiarity (Sicre et al., unpublished data). These results suggest that the influence of social presence on drug intake depends on the substance used. Nevertheless, the effect of the presence of an alcohol-naive stranger has not been assessed. Furthermore, the influence of social presence has only been examined in the context of recreational alcohol use. Consequently, the modulation of recreational and escalated ethanol intake by the presence of a stranger peer, naive to alcohol remains to be explored.

Over the past few years, the Subthalamic Nucleus (STN), the only glutamatergic structure of the basal ganglia, has gained increasing significance in the reward circuit, in motivational processes and in drug addiction [START_REF] Baunez | Frontal Cortex-Like Functions of the Subthalamic Nucleus[END_REF]. STN inactivation induced by lesions, deep brain stimulation (DBS) or optogenetic manipulations, has shown beneficial effects across various addiction criteria: motivation [START_REF] Baunez | The Subthalamic Nucleus Exerts Opposite Control on Cocaine and 'Natural' Rewards[END_REF][START_REF] Lardeux | Alcohol Preference Influences the Subthalamic Nucleus Control on Motivation for Alcohol in Rats[END_REF][START_REF] Rouaud | Reducing the Desire for Cocaine with Subthalamic Nucleus Deep Brain Stimulation[END_REF][START_REF] Pelloux | Targeting the Subthalamic Nucleus in a Preclinical Model of Alcohol Use Disorder[END_REF], escalation of drug intake [START_REF] Pelloux | Targeting the Subthalamic Nucleus in a Preclinical Model of Alcohol Use Disorder[END_REF][START_REF] Pelloux | Subthalamic Nucleus High Frequency Stimulation Prevents and Reverses Escalated Cocaine Use[END_REF], maintenance of escalated drug intake (Vielle et al. preprint) and compulsive drug-seeking behavior [START_REF] Degoulet | Subthalamic Low-frequency Oscillations Predict Vulnerability to Cocaine Addiction[END_REF]). Additionally, it has been demonstrated that the STN is involved in mediating the influence of the social context on cocaine and alcohol use. Indeed, inactivation of the STN modulates the modifications caused by the social context (presence of a peer or playback of ultrasonic vocalizations) on cocaine and alcohol intake [START_REF] Giorla | Peer Presence and Familiarity as Key Factors to Reduce Cocaine Intake in Both Rats and Humans: An Effect Mediated by the Subthalamic Nucleus[END_REF][START_REF] Montanari | Subthalamic Nucleus Mediates the Modulation on Cocaine Self-administration Induced by Ultrasonic Vocalization Playback in Rats[END_REF]Sicre et al., unpublished data , Vielle et al. preprint;2021).

In this study, we thus examined the influence of the presence of an alcohol-naive male stranger peer on both recreational and escalated alcohol intake in male rats. Furthermore, we investigated the contribution of the STN in these processes by using high-frequency (130 Hz) DBS.

Materials and Methods

Animals

32 Adult Lister-Hooded rats (∼400 g, Charles River Laboratories, Saint-Germain-sur-l'Arbresle, France) were used in these behavioral experiments. Only male rats were used to avoid inter-variability of the hormonal cycle. Animals were paired housed, in Plexiglas cages with unlimited access to food and water. Temperature-and humidity-controlled environments were maintained with an inverted 12 h light/dark cycle (light -onset at 7 pm). All experiments were conducted during the dark cycle (8am-8pm). Animals were handled daily. All animal care and use were conformed to the French regulation (Decree 2013-118) and were approved by local ethic committee and the National French Agriculture Ministry (authorization #3129.01). Experimental group sizes were determined by taking into accounts the 3Rs rules to reduce the number of animals used.

Electrode design for STN DBS

Electrodes were made of Platinium-Iridium wires coated with Teflon (75 µm). Coating was removed over 0.5 mm at the tips and two wires were inserted into a 16 mm stainless steel tubing to form an electrode. Two electrodes, separated by 4.8 mm (twice the STN laterally), were soldered to an electric connector, allowing connection with the stimulator. Electrodes (impedance=20 ± 2.25 kΩ) and connector were subsequently deeply bound, using a custom mold and dental cement. Finally, electrodes were checked with an isolated battery to avoid electrical short circuits.

Surgery

All the animals underwent bilateral electrode implantation into the STN. They were anesthetized with ketamine (Imalgen, Merial, 100 mg/kg, i.p.) and medetomidine (Domitor, Janssen, 0.5 mg/kg, i.p.). They also received an antibiotic treatment by injection of amoxicillin (Citramox, LA, Pfizer, 100 mg/kg, s.c.) and an injection of meloxicam (Metacam, Boehringer Ingelheim, 0.5 mg/kg, s.c.) for analgesia. Animals were then placed in the stereotaxic frame (David Kopf apparatus) for the bilateral electrodes' implantation into the STN (coordinates in mm: -3.7 AP, ± 2.4 L from bregma, -8.35 DV from skull surface [START_REF] Paxinos | The Rat Brain in Stereotaxic Coordinates[END_REF], with the incisor bar at -3.3 mm). The electrodes were maintained with a head-cap made of dental cement and screws anchored on the skull. Rats were awakened with an injection of atipamezole (Antisedan, Janssen, 0.15 mg/kg, i.m.), an antagonist of medetomidine. Rats were allowed to recover from the surgery for at least 7 days.

Behavioral apparatus

Behavioral experiment took place in homemade self-administration chambers (60x30x35 cm) divided into two compartments by a metallic grid. The wall of one compartment per cage was equipped with two non-retractable levers and a light above each. A recessed magazine (3.8-cm high and 3.8-cm wide located 5.5 cm above the grid floor) containing a cup liquid receptacle with an 18-gauge pipe was positioned between the two levers and connected to a pump outside the chamber. A syringe filled with ethanol was positioned on the pump and connected with the 18-gauge pipe of the receptacle via a catheter. All the chambers were controlled by a custom-built interface and associated software (built and written by Y. Pelloux).

Behavioral procedure

An overview of the behavioral procedure is illustrated in Fig. 1.

1-Experiment 1: recreational ethanol self-administration

At least one week after surgery, rats began ethanol self-administration training. The ethanol delivery was assigned to one of the two levers ('active lever') and counterbalanced between rats. An FR1 schedule of reinforcement was used, so each lever press delivered 100 µL of 10 % ethanol in the cup receptable and switched on the cue-light above the lever. Each delivery was followed by a 20-s timeout period, during which any press on the active lever was recorded as a perseveration but had no consequence. Presses on the other lever ('inactive lever') had no consequence either. Each daily session lasted 1h.

Rats were first trained to self-administer ethanol with sucrose, using a fading sucrose technique in which the sucrose concentration in the 10% ethanol solution gradually decrease over sessions until zero. They were also habituated to be connected to an electric cable (but without stimulation). Once consumption became stable (<25% of variability in the number of ethanol rewards for 5 consecutive days), animals were exposed to the testing period, during which rats were either STN-stimulated with DBS (ON group, n=16) or remained unstimulated (OFF control group, n=16). This testing period consists of 5x1h consecutive daily sessions of ethanol self-administration alone, followed by 5x1h-sessions in presence of an alcohol-naive stranger peer ('With peer' condition), placed on the other side of the grid. To prevent the emergence of a familiarity effect resulting from constantly using the same social stimulus, the peer was changed every day.

2-Escalation of ethanol intake through intermittent access to alcohol (IAA)

After this initial testing period, animals (n=30, two animals were excluded at this point, see results section) underwent an escalation of their ethanol intake, using the intermittent access to alcohol (IAA) protocol for 15 sessions, as previously used in [START_REF] Pelloux | Targeting the Subthalamic Nucleus in a Preclinical Model of Alcohol Use Disorder[END_REF]. During each 12h-session (from 8:00 am to 8:00 pm), animals were isolated in a homecage with access to food and two bottles.

One bottle contained water, and the other held a 20% ethanol solution. Both bottles were weighted before and after each session to calculate consumption levels. The weight of each rat was also measured prior to every IAA session. Following the 12h-period of IAA, rats were returned to their homecage with their cagemate, and observed a 36h-period without ethanol access before the next session. The placement of the bottles in the cage was alternated from session to session to avoid side preferences. There was no DBS stimulation applied during this period.

3-Experiment 2: Addiction-like ethanol consumption

36 hours after the last session of IAA, rats were subjected to the same 10% ethanol self-administration procedure than for Experiment 1, for 10x1h other sessions: 5 consecutive sessions alone ('Alone') and 5 consecutive sessions in presence of an alcohol-naive stranger peer ('With peer'). Operant chambers were the same as those used in the first procedure. The order of the conditions 'Alone' vs. 'With peer' were counterbalanced between rats. To avoid a possible long-term effect of former STN DBS (i.e., during recreational alcohol use) on addiction-like alcohol intake, we counterbalanced animals subjected to STN DBS between experiment 1 and 2. Thus, half of the animals subjected to STN DBS OFF during experiment 1 remained OFF for experiment 2, while the other half were subjected to DBS ON for experiment 2, and vice versa for the other group. The DBS was turned ON during the 10 sessions for the STN DBS group (n=15) and remained OFF for the control group (n=15).

Deep Brain Stimulation

Electrodes were connected to homemade electric stimulation cables, themselves connected to a digital stimulator (DS8000, WPI) via a stimulus isolator (DLS100, WPI) and a rotating commutator (Plastic-One). Stimulation parameters were adapted from previous studies [START_REF] Degoulet | Subthalamic Low-frequency Oscillations Predict Vulnerability to Cocaine Addiction[END_REF][START_REF] Pelloux | Subthalamic Nucleus High Frequency Stimulation Prevents and Reverses Escalated Cocaine Use[END_REF]. Individual stimulation intensity was determined using 130 Hz frequency and 80 µs pulse width stimulation. Intensity was progressively increased until the appearance of hyperkinetic movements or abnormal behavior. Stimulation intensity (50 to 150 µA) was set just below the abnormal behavior threshold. In any case, the maximal intensity was set at 150 µA in order to prevent heating and tissue damage.

Before each behavioral session, animals were connected to the stimulation device, DBS was turned ON, and stimulation intensity was progressively increased to reach the predetermined stimulation intensity.

Stimulation at 130 Hz was applied continuously during the 1-hour session.

Histology

At the end of the experiments, rats were deeply anesthetized with an injection of pentobarbital (Euthasol Vet, Dechra, 300 mg/kg i.p.). Then, brains were extracted and rapidly frozen into liquid isopentane (-40°C) and cut in 40 µm thick frontal slices using a cryostat. A staining with cresyl violet allowed to perform histological control. Animals with incorrect electrode placement were excluded from the ON group. Representative correct electrodes implantation is illustrated in Fig. 1.

Statistical Analyses

Only the number of rewards obtained (i.e., rewarded active lever presses) are shown and expressed as mean number ± SEM. Using R software, and p-value threshold set at α=0.05, data were analyzed by performing two-tailed, one, two or three-way mixed ANOVA, followed by Bonferroni post-hoc tests when applicable. Graphs were performed using GraphPad Prism 8 (version 8.0.2).

The consumption of ethanol or water in the IAA protocol was calculated as difference in bottle weight before and after each session. This difference was then divided by the weight of the rat.

In Experiment 1 and 2, the outlier's test was conducted using the mean baseline of ethanol intake of all rats in the "alone" condition.

Results

In total, 28 rats were included in the analyses of Experiment 1. Four animals were excluded: electrodes' mislocation (n=2, ON group), one rat (OFF group) was identified as an outlier (estimation based on the mean baseline of ethanol intake of all rats), one rat (ON group) did not drink ethanol.

To avoid a possible long-term effect of former STN DBS during Experiment 1, we counterbalanced animals subjected to STN DBS between experiment 1 and 2. In Experiment 2 (escalated alcohol consumption), out of 30 rats, 24 were included in the analyses. 5 rats (n=2 ON and 3 OFF) did not escalate their ethanol intake in the IAA protocol, and one rat (ON group) was identified as an outlier (estimation based on the mean baseline of ethanol intake of all rats).

Experiment 1:

Influence of a Peer's Presence and Contribution of STN on Recreational Alcohol Intake

To study the influence of social context and the role of STN, a three-way ANOVA was used (social context x STN DBS + session) to assess the number of rewards obtained (i.e. active lever presses outside of the timeout period). No main effects were observed for social context (F(1, 270)=0.015; p=0.901), STN DBS (F(1, 270)=0.040; p=0.842) and session (F(4, 270)=0.649; p=0.628). The interactions of STN DBS and social context were also non-significant (STN DBS x social context (F(1, 270)=0.001; p=0.972). Indeed, in the alone condition, control rats (n=15) had a stable intake of ethanol (mean 7.88 ± 0.29 rewards), that remained similar in presence of a peer (mean 8.36 ± 0.48 rewards). Thus, the presence of an alcohol-naive stranger peer, had no effect on recreational ethanol consumption.

In STN stimulated rats (n=13), the alcohol intake in the alone condition (mean 7.60 ± 0.25 rewards) was comparable to the control group. STN DBS seems not to influence recreational alcohol use in this experiment. In presence of an alcohol-naive unfamiliar peer, STN DBS did not change significantly the ethanol consumption compared to when STN-stimulated rats were in the alone condition (mean 8.09 ± 0.15 rewards), and was similar to the control group's intake (Fig. 2a).

Influence of the Alcohol Preference

Because substantial variations in alcohol consumption and preference were observed between rats, they were split into two groups: high drinkers (HD) and low drinkers (LD) depending on their mean ethanol consumption in the alone condition [START_REF] Spoelder | Individual Variation in Alcohol Intake Predicts Reinforcement, Motivation, and Compulsive Alcohol Use in Rats[END_REF]. The effects of the social context and STN DBS were assessed with a three-way ANOVA (social context x STN DBS + sessions) independently for each group according to their preference for ethanol.

In HD rats (n=14, Fig. 2b), there were no effect of the STN DBS (F(1, 130)=0.061; p=0.805), nor of the sessions (F(4, 130)=0.759; p=0.554) nor of the interaction STN DBS x social context (F(1, 130)=0.159; p=0.691), although a trend for the social context effect (F(1, 130)=3.518; p=0.063). Indeed, HD control rats (n=8) had similar ethanol consumption in the alone condition (mean 10.95 ± 0.36 rewards) and in a presence of an alcohol-naive stranger peer (mean 11.05 ± 0.72 rewards). STN DBS did not affect alcohol consumption in HD animals (n=6) (alone: mean 10.37 ± 0.40, with a peer: mean 9.73 ± 0.53 rewards), that was equivalent to the control group in the same circumstances.

In LD rats (n=14, Fig. 2c), the ANOVA analysis indicated a trend for an STN DBS effect (F(1, 130)=3.669; p=0.058) and for the social context (F(1, 130)=3.415; p=0.067), but no significant effect of the repeated sessions (F(4, 130)=0.459; p= 0.766), nor interaction STN DBS x social context (F(1, 130)=0.356; p=0.552).

LD control rats (n=7) consumed ethanol at the same level in the alone condition (mean 4.37 ± 0.23 rewards) and in presence of a peer (mean 5.29 ± 0.39 rewards). STN DBS in LD rats (n=7) did not change ethanol consumption when compared to that of control animals, regardless of whether they were in the alone condition (mean 5.23 ± 0.27 rewards) or in a social context (mean 6.69 ± 0.19 rewards).

Overall, our results showed that despite tendencies for slight modulations, neither the presence of an alcohol-naive stranger peer, nor the STN DBS alters significantly recreational alcohol consumption, regardless of alcohol preference (i.e., low vs. high drinkers).

Experiment 2: Influence of peer's presence and STN contribution following escalation of ethanol use

Rats subjected to IAA procedure escalated their ethanol consumption over sessions

Alcohol consumption during IAA was analyzed with a two-way ANOVA with STN DBS group as between factors (control group vs. "to be stimulated group" although there is no stimulation during IAA) and sessions as within factors. During the intermittent access to alcohol, rats showed a progressive increase of their alcohol consumption (Fig. 3a) over the sessions (F(4.34, 95.43)=18.083; p<0.0001), with no difference between the control group and the "to be stimulated" group (F(1, 22)=1.197; p=0.286) and no interaction between these two factors (F(4.34, 95.43)=1.749; p=0.140). Post-hoc analyses confirmed the progressive increase of alcohol intake over the sessions, reaching a level of consumption during the last session (mean 3.01 ± 0.35 g/kg ethanol) that was significantly higher than that of the first session (mean 1.66 ± 0.13 g/kg ethanol, Bonferroni's test: p=0.00015). Two-way ANOVA (STN DBS group x sessions) of water consumption during IAA (Fig. 3b) revealed a main effect of session (F(5.40, 118.80=7.893; p<0.0001) and no difference between the two future groups (F(1, 22)=0.170; p=0.5) nor the interaction between session x STN DBS (F(5.40, 118.80)=1.011; p=0.418). The session effect is due to inter-session variability but no difference between the first (mean 16.22 ± 0.92 g/kg water) and the last sessions (mean 22.39 ± 2.27 g/kg water, Bonferroni's test, p=1) was found.

As a result, both control and 'to be stimulated' groups exhibited comparable escalation and loss of control only over alcohol intake.

Influence of Peer's Presence and STN Contribution on Escalated Alcohol Intake

To assess the effect of the peer's presence and STN DBS after a loss of control over alcohol use, a three- In the control group (n=12, Fig. 3c black), after escalation of alcohol intake, the introduction of an alcohol-naive stranger peer, did not modulate ethanol consumption; control rats displayed equivalent ethanol intake whether in the alone condition (mean 9.30 ± 1.10 rewards) or in presence of the peer (mean 10.58 ± 1.08 rewards, Bonferroni's test, p=0.341).

When STN DBS (n=12, Fig. 3c) was applied, post-hoc analysis revealed that in the alone condition, rats consumed a significantly less ethanol (mean 6.13 ± 0.55 rewards, Bonferroni's test, p=0.004) than the control group in the same alone condition. This decreased consumption persisted in the presence of a peer, where STN-stimulated rats consumed significantly less (mean 5.63 ± 0.62 rewards, Bonferroni's test, p=0.0004) than the control groups in the same condition. However, within the stimulated group, no significant difference was observed between the level of consumption in the two conditions 'alone' and 'with peer' (Bonferroni's test, p=0.567). Together, these results suggest that STN DBS reduces escalated ethanol intake, independently of the social context.

To assess a possible gradual effect of the STN DBS on ethanol intake, we reordered the sessions in the proper sequence independently of the social conditions (i.e., the order of passage in the two social conditions were counterbalanced between animals, Fig. 3d). We then performed a one-way ANOVA (reordered session) to assess the effects of the STN DBS over time on alcohol intake. Our results revealed a significant effect of the reordered session (F(9, 109)=2.396; p=0.008), confirming that the STN DBS progressively decreased alcohol intake.

Impact of Alcohol preference Following a Loss of Control over Alcohol Intake

An additional possible contributor to inter-individual variation is the individual alcohol preference. This alcohol preference is calculated on the level of alcohol consumption in the alone condition after the escalation procedure. This alcohol preference allows splitting rats into two categories: High Drinkers (HD) and Low Drinkers (LD).

For HD rats (n=12, Fig. 3e), a three-way ANOVA was run to explore the influence of social context, STN DBS, and repeated sessions. While no impact of the social context (F(1, 95)=0.724, p=0. HD control animals (n=7) had a similar ethanol consumption in the alone condition and in the presence of a peer (mean 12.83 ± 1.59 and 12.69 ± 1.78 rewards respectively, Bonferroni's test, p=0.948), revealing that the peer presence did not influence escalated alcohol use in HD rats. The STN DBS reduced the consumption in HD STN-stimulated animals (n=5) compared to the HD control group in both social conditions: alone (Bonferroni's test, p=0.037, mean 9.48 ± 0.92 rewards for stimulated animals) and in presence of a peer (Bonferroni's test, p=0.021, mean 6.72 ± 1.15 rewards for STNstimulated rats). However, no significant difference was found in alcohol intake between the two social conditions within the HD STN-stimulated group (Bonferroni's test, p=0.117), revealing the beneficial effect of the STN DBS on escalated alcohol use in HD rats, independently of the social context.

For LD rats (n=12, Fig. 3f), the three-way ANOVA revealed a significant effect of the social context LD control animals (n=5) consumed more alcohol in presence of a peer (mean 7.64 ± 0.34 rewards) than when they were in the alone condition (mean 4.40 ± 0.42 rewards, Bonferroni's test, p=0.0006).

Although STN DBS (n=7) did not affect ethanol intake when rats were in the alone condition (Bonferroni's test compared to LD controls,p=0.366,mean 3.74 ± 0.45 rewards), it prevented the overconsumption induced by the presence of a peer (mean 4.86 ± 0.47 rewards) (lower consumption in LD STN DBS compared with LD controls, Bonferroni's test, p=0.005). Consequently, STN DBS effectively suppressed the excessive consumption triggered by the presence of a peer in LD rats.

Discussion

The present study showed that neither the presence of an alcohol-naive stranger peer, nor STN DBS, significantly modulate the recreational use of alcohol. In contrast, after a loss of control over ethanol intake, STN DBS induced a progressive decrease of the alcohol use over time. After escalation of ethanol intake, the presence of an alcohol-naive stranger peer modulated the consumption depending on the initial alcohol preference. HD rats were insensitive to the presence of the peer, and maintained a stable high level of ethanol intake, while LD rats exhibited an overconsumption in presence of the peer.

STN DBS decreased ethanol intake in HD rats in both conditions (alone and with peer), while it only reduced the overconsumption induced by the peer's presence in LD rats.

Presence of a peer during alcohol consumption: Importance of the peer's identity

Studies investigating the effect of a peer's presence on alcohol self-administration have reported yielded mixed results. Several showed that the presence of a social partner drinking alcohol promotes alcohol drinking in humans [START_REF] Dallas | Influenced but Unaware: Social Influence on Alcohol Drinking Among Social Acquaintances[END_REF][START_REF] Larsen | An Experimental Study on Imitation of Alcohol Consumption in Same-Sex Dyads[END_REF]. Our recent study in rats further revealed that the presence of a peer, whether familiar or stranger, but who had a regular consumption of alcohol, induced an increase of recreational alcohol consumption (Sicre et al., unpublished data). In the present study, the presence of an alcohol-naive stranger peer during recreational use of alcohol had no significant influence. These results highlight that the peer's history with alcohol plays an important role.

If peers consumed alcohol orally recently before being used as the "peer", it is possible that olfactory cues of alcohol consumption could be transmitted to the subject rat through mouth-to-mouth contacts.

These ethanol cues might induce a social transmission of ethanol preference, as observed for food [START_REF] Strupp | Social Transmission of Food Preferences in Adult Hooded Rats (Rattus Norvegicus)[END_REF], and this would result in an overconsumption of alcohol. Such olfactory cues could not be transmitted by a completely alcohol-naive peer, preventing alcohol preference facilitation and therefore no changes in consumption for the subject rat. On the other hand, the absence of increase of alcohol intake during the recreational experiment could be due to a ceiling effect that does not leave room for an increase. However, we have observed an increased effect by the presence of a peer in LD rats after escalation of alcohol consumption and the level reached then by these LD was equivalent to that recorded in the recreational experiment. So, although there was room for an increase in consumption in the recreational experiment, the peer presence had no significant effect. The ceiling effect is thus probably to be ruled out.

Once the control over ethanol intake was lost, the influence of an alcohol-naive peer varied depending on alcohol preference. For LD rats, the presence of a peer induced an overconsumption of ethanol while for HD rats, there was no modulation. An appealing hypothesis would be that after an escalation of alcohol intake, LD rats had an enhanced sensitivity to the presence of a peer. Indeed, social interactions are rewarding [START_REF] Krach | The Rewarding Nature of Social Interactions[END_REF], and after undergoing intermittent ethanol exposure during late adolescence, male rats having low doses of ethanol in adulthood displayed an increased social behavior [START_REF] Varlinskaya | Chronic Intermittent Ethanol Exposure During Adolescence: Effects on Social Behavior and Ethanol Sensitivity in Adulthood[END_REF]. The potentiation between the rewarding effect of low doses of alcohol and social interaction could be magnified, resulting in the overconsumption even if the peer is an alcoholnaive stranger peer. In the case of HD rats after a loss of control, their lack of sensitivity to the peer's presence could be attributed to a preference for alcohol consumption over social interaction. It has been shown that when rats had a high dose of ethanol, they show less social investigation, contact behavior and social avoidance (Varlinskaya and Spear 2002). Moreover, after a protocol of intermittent access to alcohol rats exhibit preference for alcohol over social rewards [START_REF] Augier | Wistar Rats Choose Alcohol over Social Interaction in a Discrete-choice Model[END_REF][START_REF] Marchant | Rats Choose Alcohol over Social Reward in an Operant Choice Procedure[END_REF]).

In our experiment, however, rats were not offered the choice, the social presence was imposed, and the rats could consume alcohol while interacting with a peer as much as they liked.

Nonetheless, from ours results, the presence of a stranger peer that never consumed ethanol had not much detrimental effect, which is in contrast to the consequences of presence of a peer that has previously consumed alcohol. These results confirm the significant influence of the immediate social context on drug consumption. This effect is influenced by factors such as the peer's identity (e.g., familiarity, drug history) and the specific type of drug being used (e.g., psychostimulant vs. depressive drug).

STN as a neurobiological substrate … …Of escalated but not recreational alcohol use…

During recreational alcohol use (Experiment 1), the present study showed no significant effect of STN DBS on ethanol intake. These results are consistent with a previous work showing that lesion of STN did not affect rats' alcohol intake in a forced consumption test, and in a two-bottle choice protocol [START_REF] Lardeux | Alcohol Preference Influences the Subthalamic Nucleus Control on Motivation for Alcohol in Rats[END_REF]. Moreover, similar to alcohol, STN manipulation was ineffective at decreasing recreational cocaine use [START_REF] Baunez | The Subthalamic Nucleus Exerts Opposite Control on Cocaine and 'Natural' Rewards[END_REF][START_REF] Rouaud | Reducing the Desire for Cocaine with Subthalamic Nucleus Deep Brain Stimulation[END_REF][START_REF] Giorla | Peer Presence and Familiarity as Key Factors to Reduce Cocaine Intake in Both Rats and Humans: An Effect Mediated by the Subthalamic Nucleus[END_REF]Vielle et al. preprint). However, while no effort was required in the present study that used continuous reinforcement, when the demand was increased, as in a progressive ratio task, STN manipulation was efficient at reducing cocaine motivation [START_REF] Baunez | The Subthalamic Nucleus Exerts Opposite Control on Cocaine and 'Natural' Rewards[END_REF][START_REF] Rouaud | Reducing the Desire for Cocaine with Subthalamic Nucleus Deep Brain Stimulation[END_REF]. For alcohol, this effect has been shown to depend on the loss of control over ethanol intake. If rats did not undergo an IAA protocol, the effect of STN lesion depended on the rat's alcohol preference [START_REF] Lardeux | Alcohol Preference Influences the Subthalamic Nucleus Control on Motivation for Alcohol in Rats[END_REF].

After an IAA protocol, STN lesion decreased the level of ethanol motivation [START_REF] Pelloux | Targeting the Subthalamic Nucleus in a Preclinical Model of Alcohol Use Disorder[END_REF].

Thus, like for cocaine, STN manipulation could blunt alcohol addiction-like criteria, without altering recreational drug use. Indeed, applying DBS at high frequency to the STN after escalation of ethanol intake reduced alcohol consumption. These results are in line with previous works showing that lesion of STN prevented escalation of ethanol intake and diminished the rebound intake after abstinence [START_REF] Pelloux | Targeting the Subthalamic Nucleus in a Preclinical Model of Alcohol Use Disorder[END_REF]. This has been shown for cocaine, heroin and alcohol [START_REF] Pelloux | Targeting the Subthalamic Nucleus in a Preclinical Model of Alcohol Use Disorder[END_REF][START_REF] Pelloux | Subthalamic Nucleus High Frequency Stimulation Prevents and Reverses Escalated Cocaine Use[END_REF]Vielle et al. preprint;[START_REF] Wade | High-Frequency Stimulation of the Subthalamic Nucleus Blocks Compulsive-Like Re-Escalation of Heroin Taking in Rats[END_REF]. Interestingly, in contrast to our former results with cocaine that failed to show an effect of STN DBS on the maintained escalated intake [START_REF] Pelloux | Subthalamic Nucleus High Frequency Stimulation Prevents and Reverses Escalated Cocaine Use[END_REF], it was here reduced for alcohol. This disparity in the impact of STN DBS on escalated alcohol and cocaine consumption could potentially be attributed to the type of drug available. Cocaine is a psychostimulant, while alcohol has depressant effects. The road of administration could also explain the differences observed between alcohol and cocaine. Alcohol is consumed orally, leading to different pharmacokinetics and longer absorption time compared to intravenously administration that enters directly in the bloodstream [START_REF] Robinson | Effect of Gender and Estrous Cycle on the Pharmacokinetics of Ethanol in the Rat Brain[END_REF]. This delay deferred the activation of the reward system and pleasurable effects [START_REF] Robinson | Effect of Gender and Estrous Cycle on the Pharmacokinetics of Ethanol in the Rat Brain[END_REF]. Introducing a delay before obtaining a reward has been shown to decrease motivation [START_REF] Jarmolowicz | Concurrent Progressive-ratio Schedules: Built-in Controls in the Study of Delayed Reward Efficacy[END_REF]. Consequently, it might thus be comparatively easier to reduce motivation for alcohol compared to cocaine with STN DBS. However, one might also consider that having a direct sensory input with alcohol in the mouth, but also its smell, could lead to anticipation of ethanol effects, while intravenous injection might be less efficient at anticipating the physiological and psychoactive effects of cocaine. The differences are therefore difficult to simply explain with the mean of administration. It is interesting to note however that, although the DBS or lesion procedure did not lead to changes in sustained escalated intake of cocaine, a precise targeting of the STN using optogenetics has been shown to be efficient (Vielle et al. preprint). Finally, differences in the experimental protocol would account for such differences. Indeed, cocaine escalation, and maintenance in [START_REF] Pelloux | Subthalamic Nucleus High Frequency Stimulation Prevents and Reverses Escalated Cocaine Use[END_REF], were performed in the same context (operant cages). In the present experiment, escalation of alcohol intake occurred in different cages than the maintenance, and the access (intermittent vs. daily), and concentrations of ethanol were also different between escalation procedure and the operant measure of consumption (20 vs. 10%). Thus, in this experiment, such differences between escalation and maintenance could have facilitated STN DBS effect on the maintenance of ethanol use in comparison with the cocaine experiment [START_REF] Pelloux | Subthalamic Nucleus High Frequency Stimulation Prevents and Reverses Escalated Cocaine Use[END_REF].

The level of consumption seems to be an important factor to ensure STN DBS efficiency since STN DBS decreased the consumption of alcohol only in HD rats. The lack of impact of STN DBS in LD rats might be attributed to a floor effect, implying that the LD rats did not consume enough ethanol to exhibit further a reduction triggered by STN DBS. Another hypothesis would be that the neurobiological effect of alcohol consumption does not induce the same changes in LD and HD rats. Indeed, when consuming alcohol, HD rats exhibit an increased power in gamma range and coherence in the medial prefrontal cortex (mPFC) compared to LD rats [START_REF] Henricks | Corticostriatal Oscillations Predict High Vs. Low Drinkers in a Rat Model of Limited Access Alcohol Consumption[END_REF]). The mPFC is directly connected to the STN via the hyperdirect pathway [START_REF] Maurice | Relationships Between the Prefrontal Cortex and the Basal Ganglia in the Rat: Physiology of the Corticosubthalamic Circuits[END_REF][START_REF] Nambu | Functional Significance of the Corticosubthalamo-pallidal 'hyperdirect' Pathway[END_REF].

Additionally, abnormal increase in low frequency oscillations has been observed within the STN during cocaine escalation, and STN DBS was shown to reduce STN oscillatory activity [START_REF] Pelloux | Subthalamic Nucleus High Frequency Stimulation Prevents and Reverses Escalated Cocaine Use[END_REF]).

If such abnormal activity raised in the STN during alcohol escalation of HD rats, as that seen with cocaine, STN high frequency DBS could suppress it and also disturb the pathological oscillatory coherence between STN-mPFC as observed in PD (de [START_REF] Hemptinne | Therapeutic Deep Brain Stimulation Reduces Cortical Phaseamplitude Coupling in Parkinson's Disease[END_REF][START_REF] Delaville | Subthalamic Nucleus Activity in the Awake Hemiparkinsonian Rat: Relationships with Motor and Cognitive Networks[END_REF][START_REF] Moran | Decoupling Neuronal Oscillations During Subthalamic Nucleus Stimulation in the Parkinsonian Primate[END_REF]. This would lead to a decrease of alcohol intake only for HD rats after the loss of control.

Nonetheless, STN DBS appears to offer a potential therapeutic benefit for alcohol-related disorders, as suggested previously for other drugs.

…And of social influence during alcohol drinking

The influence of the social context was observed here in LD rats having escalated their alcohol consumption. Interestingly, in these animals, STN DBS effectively abolished the overconsumption induced by the presence of the peer. These results align with precedent findings, in which STN DBS has suppressed recreational alcohol overconsumption triggered by different types of peer (Sicre et al., unpublished data). STN optogenetic inhibition was also found to suppress the protective effect of a peer on cocaine recreational use (Vielle et al. preprint). The STN could thus be part of the neurobiological substrate of the influence of the social context on behavior. Indeed, studies focusing on the effects of ultrasonic vocalizations (USV) playback on cocaine intake showed that STN lesion abolished the beneficial influence of positive USV emitted by a stranger rat [START_REF] Montanari | Subthalamic Nucleus Mediates the Modulation on Cocaine Self-administration Induced by Ultrasonic Vocalization Playback in Rats[END_REF][START_REF] Vielle | Evidence for a Vocal Signature in the Rat and Its Reinforcing Effects: a Key Role for the Subthalamic Nucleus[END_REF].

Additionally, STN lesion suppressed the rewarding effect of positive USVs when rats were pressing a lever to listen to USVs [START_REF] Vielle | Evidence for a Vocal Signature in the Rat and Its Reinforcing Effects: a Key Role for the Subthalamic Nucleus[END_REF]. It is important to note however that the combination of social context and STN manipulation does not systematically lead to neutralization or additivity of opposite effects, nor potentiation of similar effects. For example, STN lesions could decrease further the cocaine intake in presence of a peer [START_REF] Giorla | Peer Presence and Familiarity as Key Factors to Reduce Cocaine Intake in Both Rats and Humans: An Effect Mediated by the Subthalamic Nucleus[END_REF], while STN neuronal optogenetic inhibition did not potentiate the decreased cocaine intake after escalation (Vielle et al preprint). All these findings suggest that presence of a peer effects on drug use probably involves STN as a neurobiological substrate. In human studies, STN DBS has been shown to induce deficits in facial and voice emotional decoding [START_REF] Biseul | Fear Recognition Is Impaired by Subthalamic Nucleus Stimulation in Parkinson's Disease[END_REF][START_REF] Brück | Emotional Voices in Context: A Neurobiological Model of Multimodal Affective Information Processing[END_REF][START_REF] Kalampokini | Effects of Subthalamic Nucleus Deep Brain Stimulation on Facial Emotion Recognition in Parkinson's Disease: A Critical Literature Review[END_REF][START_REF] Péron | Recognition of Emotional Prosody Is Altered after Subthalamic Nucleus Deep Brain Stimulation in Parkinson's Disease[END_REF]2013). A recent study has also shown that STN activity assessed with fMRI, was modulated by the presence of an observer during the performance of a stop-signal reaction time task by cocaine-addicted individuals (Terenzi et al. preprint). All these results converge to suggest the involvement of STN in mediating the impact of social interactions on drug-related behavior. Social interaction has been shown Low drinkers in the alone condition (4.37 ± 0.23 vs. 5.23 ± 0.27 rewards, respectively) and in a presence of an alcohol-naive stranger peer (5.29 ± 0.39 vs. 6.69 ± 0.19 rewards, respectively) Results are presented as mean ± SEM.

(a) Ethanol intake in control (black dots -OFF) and "to be stimulated" (red dots -ON) groups across the 15 sessions of intermittent access to 20% ethanol. Both groups increased their consumption from session 1 (1.66 ± 0.13 g/kg ethanol) to session 15 (3.01 ± 0.35 g/kg ethanol).

(b) Water intake in OFF (black dots -OFF) and "to be stimulated" (red dots -ON) groups across the 15 sessions of intermittent access to 20% ethanol. Both groups consumed the same amount of water between session 1 (16.22 ± 0.92 g/kg water) and session 15 (22.39 ± 2.27 g/kg water).

(c) Number of rewards obtained after the loss of control of alcohol intake for non-stimulated (black dots -OFF -control) and STN-stimulated rats (red dots -ON) during five 1h-sessions in the alone condition (9.30 ± 1.10 vs. 6.13 ± 0.55 rewards, respectively) and five 1h-sessions with an alcoholnaive stranger peer (10.58 ± 1.08 vs. 5.63 ± 0.62 rewards, respectively) (d) Number of rewards obtained by the stimulated rats after reordering the sessions in the proper sequence irrespective of the social context. ANOVA revealed a general session effect, reflecting a gradual decrease of the ethanol intake over time.

(e) Number of rewards obtained in control (black dots -OFF -control) and STN-stimulated (red dots -ON) High Drinkers after a loss of control, in the alone condition (12.83 ± 1.59 vs. 9.48 ± 0.92 rewards respectively) and in a presence of an alcohol-naive stranger peer (12.69 ± 1.78 vs. 6.72 ± 1.15 rewards, respectively) (f) Number of rewards obtained in control (black dots -OFF -control) and stimulated (red dots -ON) Low Drinkers after a loss of control, in the alone condition (4.40 ± 0.42 vs. 3.74 ± 0.45 rewards, respectively) and in a presence of an alcohol-naive stranger peer (7.64 ± 0.34 vs. 4.86 ± 0.48 rewards, respectively) * p<0.05; ** p<0.01; *** p<0.001; **** p<0.0001 ### p<0.001 between "alone" and "with peer" conditions in the non-stimulated (OFF -control) group

  way ANOVA was conducted (STN DBS x social context x sessions). This investigation disclosed significant main effects of STN DBS (F(1, 215)=39.752; p < 0.0001) but no effect of the social context (F(1, 215)=0.232; p=0.631), and sessions (F(4, 215)=0.337; p=0.852). No significant interaction was found (STN DBS x social context (F(1, 215)=1.230; p=0.269; STN DBS x sessions (F(4, 215)=1.090; p=0.363; social context x session (F(4, 215)=0.073; p=0.990) and STN DBS x social context x session (F(4, 215)=0.250; p=0.909).

  397) and sessions (F(4, 95)=0.698; p=0.595) emerged, effect of STN DBS (F(1, 95)=10.003; p=0.002) was significant. No interaction effects were found (STN DBS x social context: F(1, 95)=0.792; p=0.376), STN DBS x session: F(4, 95)=0.434; p=0.784, social context x session: F(4, 95); p=0.248; p=0.910 and STN DBS x social context x session: F(4, 95)=0.192; p=0.942) .

(

  F(1, 95)=10.631; p=0.002) and STN DBS (F(1, 95)=10.108; p=0.002), but no influence of the session (F(4, 95)=0.443; p=0.778). No significant interaction was found (STN DBS x social context: F(1, 95)=2.919; p=0.091, STN DBS x session: F(4, 95)=0.395; p=0.812, social context x session: F(4, 95)=0.317; p=0.867 and STN DBS x social context x session: F(4, 95)=0.315; p=0.868).

Fig. 1

 1 Fig. 1 Timeline of the behavioral experiment Top: Behavioral experiment timeline Down: (left) schematic representations of the electrode implanted in the STN, (right) Picture from a representative electrode implanted in the STN (delineated by the red dashed line)

Fig. 3

 3 Fig. 3 STN DBS reduces the escalated intake of alcohol and suppresses the increased consumption induced by the peer's presence.

Acknowledgments: The authors thank Yann Pelloux for the custom-built interface and associated software, Joel Baurberg for technical assistance with all electronic devices, and the technical staff of the animal facility for ensuring the well-being of our animals. This work was funded by the Institut de Recherche en Santé Public and Alliance Aviesan in the call for proposal on substance use disorders and by the Institut de lutte contre le Cancer. Aix-Marseille Université (CB, CV, LV, MD, NM) French Ministry of Higher Education, Research and Innovation (LV) Institut de Recherche en Santé Public (IRESP-19-ADDICTIONS-02) (CB) Institut National de lutte contre le Cancer (INCA_16032) (LV)

equal contribution This work was funded by the Institut de Recherche en Santé Publique in collaboration with Alliance Aviesan , and by the Institut de lutte contre le Cancer.

Funding:

Centre National de la Recherche Scientifique (CB, MD, NM)

to trigger the release of oxytocin within the brain's reward circuit [START_REF] Dölen | Social Reward Requires Coordinated Activity of Nucleus Accumbens Oxytocin and Serotonin[END_REF][START_REF] Hung | Gating of Social Reward by Oxytocin in the Ventral Tegmental Area[END_REF].

Additionally, there are evidences suggesting that the STN possesses oxytocin receptors that can regulate locally dopamine transmission (Baracz andCornish 2013, 2016). Furthermore the injection of oxytocin into the STN has been demonstrated to reduce methamphetamine-induced reinstatement [START_REF] Baracz | The Involvement of Oxytocin in the Subthalamic Nucleus on Relapse to Methamphetamine-Seeking Behaviour[END_REF]. Through oxytocin release induced by social interaction, the STN activity could thus be modulated and potentially affect drug consumption behavior.

Additional researches are required to elucidate further the role of the STN in social behaviors and its impact on drug-related effects. Data and materials availability: All data are available in the main text, or on request to the authors.