Gabriel Turinici

Stochastic learning control of inhomogeneous quantum ensembles

In quantum control, the robustness with respect to uncertainties in the system's parameters or driving field characteristics is of paramount importance and has been studied theoretically, numerically and experimentally. We test in this paper stochastic search procedures (Stochastic gradient descent and the Adam algorithm) that sample, at each iteration, from the distribution of the parameter uncertainty, as opposed to previous approaches that use a fixed grid. We show that both algorithms behave well with respect to benchmarks and discuss their relative merits. In addition the methodology allows to address high dimensional parameter uncertainty; we implement numerically, with good results, a 3D and a 6D case.

Introduction

Quantum control is a promising technology with many applications ranging from NMR [START_REF] Glaser | Unitary control in quantum ensembles: Maximizing signal intensity in coherent spectroscopy[END_REF] to quantum computing [START_REF] Khodjasteh | Dynamically error-corrected gates for universal quantum computation[END_REF] and laser control of quantum dynamics [START_REF] Brif | Control of quantum phenomena: past, present and future[END_REF]. The controlling field encounters many molecules which although identical in nature may interact differently with the incoming field because of e.g., different Larmor frequencies or rf attenuation factors (in NMR spin control or quantum computing, see [START_REF] Li | Control of inhomogeneous quantum ensembles[END_REF][START_REF] Thomas E Skinner | Application of optimal control theory to the design of broadband excitation pulses for high-resolution nmr[END_REF][START_REF] Yurke | Quantum network theory[END_REF][START_REF] Nielsen | Quantum Computation and Quantum Information: 10th Anniversary Edition[END_REF][START_REF] Hincks | Controlling quantum devices with nonlinear hardware[END_REF][START_REF] Kosut | Robust control of quantum gates via sequential convex programming[END_REF]), different spatial profile (see [START_REF] Rabitz | Controlling quantum dynamics regardless of laser beam spatial profile and molecular orientation[END_REF]) or other parameters (see [START_REF] Zhang | Timeoptimal control of spin 1/2 particles in the presence of radiation damping and relaxation[END_REF][START_REF] Chen | Sampling-based learning control of inhomogeneous quantum ensembles[END_REF][START_REF] Ding | Robust quantum control against clock noises in multiqubit systems[END_REF]). For obvious practical reasons, it is of paramount importance to ensure that the control quality is robust with respect to this heterogeneity. Thus the quantum control problem involves a unique set of driving fields u(t) ∈ R L , the same for all molecules in the ensemble, however each molecule is described by a set of parameters θ ∈ Θ ⊂ R d and the control outcome depends on both u and θ; the goal can be expressed as the maximization of the control quality averaged over θ. A different view is when the variability is not due to the presence of many different molecules but when uncertainties in the control implementation require to devise a field robust to fluctuations in those parameters.

A first natural question is whether this is at all possible, i.e., if a single field can drive several distinct molecules to a common target; the answer is given by the theory of ensemble control controllability, see [START_REF] Turinici | Optimal discrimination of multiple quantum systems: controllability analysis[END_REF][START_REF] Belhadj | Ensemble controllability and discrimination of perturbed bilinear control systems on connected, simple, compact Lie groups[END_REF][START_REF] Li | Control of inhomogeneous quantum ensembles[END_REF][START_REF] Beauchard | Controllability issues for continuous-spectrum systems and ensemble controllability of Bloch equations[END_REF][START_REF] Borzì | Formulation and numerical solution of quantum control problems[END_REF] and is in general positive. However the theory does not explain how to find the control (except under specific regimes, see [START_REF] Augier | Adiabatic ensemble control of a continuum of quantum systems[END_REF]). To do so, different algorithms have been proposed: the pseudo-spectral approach of Li et al. [START_REF] Li | A pseudospectral method for optimal control of open quantum systems[END_REF][START_REF] Ruths | A multidimensional pseudospectral method for optimal control of quantum ensembles[END_REF][START_REF] Li | Optimal pulse design in quantum control: A unified computational method[END_REF] consider spectral and/or polynomial representations of the control problem in 2D (d = 2); Wang considers iterative procedures based on sampling [START_REF] Wang | Free-endpoint optimal control of inhomogeneous bilinear ensemble systems[END_REF]; the learning approach of Chen et al. [START_REF] Chen | Sampling-based learning control of inhomogeneous quantum ensembles[END_REF] and Kuang et al. [START_REF] Kuang | Approximate time-optimal control of quantum ensembles based on sampling and learning[END_REF] (the latter in the context of time-optimal control) consider a fixed uniform grid over the inhomogeneous parameter space and was tested for d = 2. Finally, Wu and al. [START_REF] Wu | Robust learning control design for quantum unitary transformations[END_REF] find robust controls using uniform grids in 2D and 3D (d = 3).

In all these works there is always a fixed grid (or fixed sampling) involved when the control is searched. The rationale behind this idea is that a fixed grid makes the search more stable and a good choice of the grid is enough to describe efficiently the mean performance of the control over the parameter space in the spirit of a quadrature formula for the average over θ. This is coherent with results from the approximation theory which inform that convergence is of order e -d √ N , with respect to the number N of grid points; however the same formula indicates a bad scaling with respect to d. To address this curse of dimensionality and also explore the nature of the search landscape, we take here a different view: at each control iteration we use a new sampling in the spirit of Monte Carlo methods (see [START_REF] Press | Numerical Recipes 3rd Edition: The Art of Scientific Computing[END_REF]Section 7.7]) for computing high dimensional integrals. This will induce slight oscillations in the average but has the advantage to cover the space Θ of inhomogeneity even in high dimensions d. A similar approach has been tested independently in a very recent work by Wu and al. [34] for a two-dimensional example and promising results were obtained; see Section 2.2 for comments on the differences between the two approaches. The procedure we propose is detailed in the next section and the numerical results are the object of Section 3.

Algorithms for ensemble quantum control

We consider a control u(t) = (u 1 (t), ..., u L (t)) ∈ R L acting on a molecule part of a larger ensemble. Each molecule is completely characterized by some inhomogeneity parameter θ ∈ Θ ⊂ R d obeying a distribution law P (θ) on Θ (which can be the uniform distribution or any other). All molecules are subjected to the same control u(t) during the time interval [0, T ] in order to reach some target.

Evolution equations

The dynamics of each molecule in the sample is governed by the Hamiltonian

H(θ, u) = H 0 (θ) + L =1 u (t)H (θ) through the Schrödinger equation: i d dt ψ(t; θ) = H(θ, u)ψ(t; θ), (1)
where ψ is the wave-function of the molecule (here and below we set = 1). Of course, ψ depends on u but for notational convenience we omit to write explicitly this dependence from now on. Once a finite dimensional basis {|j , j = 1, ..., N } is chosen, the state of the quantum system can be represented as

|ψ(t; θ) = N j=1 c j (t; θ)|j . (2)
Denoting C(t; θ) = (c 0 (t; θ), ..., c N (t; θ)) T the vector of coefficients C satisfies the equation:

d dt C(t; θ) = X(θ, u)C(t; θ), (3)
where X is the representation of the Hamiltonian H (including the 1/i factor) in the basis |j , j = 1, ..., N . Note that same setting also applies to non-linear Hamiltonians e.g. Bose-Einstein condensates (nonlinearity in ψ), or high order control terms [START_REF] Dion | Laser-induced alignment dynamics of HCN: Roles of the permanent dipole moment and the polarizability[END_REF][START_REF] Coron | Quantum control design by Lyapunov trajectory tracking for dipole and polarizability coupling[END_REF] (nonlinearity in u).

The quantum system can also be described in terms of a density matrix ρ(t; θ); this matrix is expressed in some basis for operators. Same happens when the molecule is coupled to a bath or when relaxation phenomena are at work, see [START_REF] Allard | The complete homogeneous master equation for a heteronuclear two-spin system in the basis of cartesian product operators[END_REF]; in both cases the coefficients of this expansion follow an equation similar to (3).

Optimization by stochastic gradient descent and Adam algorithms

The control goal is encoded as the minimization, with respect to u, of an error, or "loss" functional L(u, θ) depending on the control u and the Hamiltonian parameters θ. When all the ensemble is considered, the following loss functional is to be minimized:

J (u) = Θ L(u, θ)P (dθ). (4)
The stochastic optimization algorithms described below construct an iterative process in order to find the u that minimizes [START_REF] Beauchard | Controllability issues for continuous-spectrum systems and ensemble controllability of Bloch equations[END_REF].

Historically the first to be considered, the stochastic gradient descent algorithm [START_REF] Robbins | A stochastic approximation method[END_REF] (henceforth called SGD) consists in the following procedure:

Algorithm 1 SGD 1:
Choose a learning rate α > 0, a mini-batch size M > 0 and the initial control u 0 . 2: Set iteration counter k = 0. Draw M independent parameters θ k 1 , ..., θ k M from the distribution P (θ) and compute the approximation

g k := 1 M M m=1 ∇ u L(u k ; θ k m ) of the gra- dient ∇ u J (u k ) of J (•) at u k . 5: set u k+1 = u k -αg k and k = k + 1. 6:
until some stopping criterion is satisfied.

In order to accelerate the convergence of the SGD algorithm, several improvements have been proposed (see [START_REF] Ruder | An overview of gradient descent optimization algorithms[END_REF]) among which the Adam [START_REF] Diederik | Adam: A method for stochastic optimization[END_REF] variant which proved to be one of the most efficient and very scalable. The difference between Adam and SGD is that Adam uses a different learning rate for each parameter which is tuned as follows: when the uncertainty in the gradient is large the learning rate is taken to be small and contrary otherwise. In order to have a robust estimation for the gradient (in absolute value) a Exponential Moving Average is computed on the fly (see below). It can be described as:

Algorithm 2 Adam 1:
Choose the learning rate α > 0, the EMA parameters β 1 and β 2 , the mini-batch size M > 0, the epsilon ε > 0 and the initial control u 0 . 2: Set iteration counter k = 0, first moment estimate µ = 0, second moment estimate v = 0.

3: Set k = k + 1. 4: repeat 5:
Draw M independent parameters θ k 1 , ..., θ k M from the distribution P (θ) and compute the approximation

g k := 1 M M m=1 ∇ u L(u k-1 ; θ k m ) of the gradient ∇ u J (u k-1 ) of J (•) at u k-1 .

6:

Compute the moving averages

µ k := β 1 µ k-1 + (1 -β 1 )g k , v k := β 2 v k-1 + (1 -β 2 )|g k | 2 . 7: Compute bias-corrected moment estimates: μk = µ k /(1-(β 1 ) k ), vk = v k /(1 -(β 2 ) k ). 8: set u k = u k-1 -αμ k /( √ vk + ε). 9:
until some stopping criterion is satisfied.

The momentum algorithm used in [34] can be seen as being halfway between SGD and Adam; it is formally a special case of the Adam algorithm for β 1 = λ, β 2 = 1, v 0 = 1 and no bias correction step 7 (that is μk = µ k , vk = v k ). In practice the numerical results are very similar and point in the same direction; in particular we expect that the momentum algorithm is also relevant to high dimensional robust control problems.

Numerical results

We test the performance of the algorithms in Section 2.2 for several benchmarks from the literature (or that generalize cases from the literature).

In sections 3.1 and 3.1 we compare the SGD algorithm with a fixed grid sampling method from the literature. Then in section sections 3.3 and 3.4 we compare the SGD wih the Adam algorithm and in Section 3.5 we draw further conclusions concerning stochastic optimization.

In the situations considered below, the goal is to maximize the so-called fidelity denoted F(u; θ). For sections 3.1 and 3.2 this has the formula F(u; θ) = | C(T ; θ), C target | where C target is a prescribed target state. But this expression is not differentiable everywhere and numerically it is easier to replace it with its square. Moreover, to express the problem as a minimization, a -1 multiplicative constant is introduced and 1 added to the result in order to have it positive. So the cost functional J will be the mean, over θ ∈ Θ of the error in the fidelity squared as in formula [START_REF] Chen | Sampling-based learning control of inhomogeneous quantum ensembles[END_REF]. On the contrary, when the fidelity is more well behaved as in section 3.3 where F(u; θ) = c 4 (T, θ) or in section 3.4 where F(u; θ) = c 6 (T, θ) the square operation is useless and the cost functional has the form in [START_REF] Dion | Laser-induced alignment dynamics of HCN: Roles of the permanent dipole moment and the polarizability[END_REF] or [START_REF] Hincks | Controlling quantum devices with nonlinear hardware[END_REF]. However, in all sections, we will plot the error in the fidelity itself; the reason why not plotting the fidelity (instead of the error) is that the error can be very small (as in Section 3.1) and the results are more visible on a logarithmic scale. Note that in some cases the best control cannot attain the target with 100% quality (even for a single molecule). However, for any given value of the parameter θ, the best attainable performance is known (see [START_REF] Glaser | Unitary control in quantum ensembles: Maximizing signal intensity in coherent spectroscopy[END_REF][START_REF] Turinici | Quantum wavefunction controllability[END_REF][START_REF] Khaneja | Boundary of quantum evolution under decoherence[END_REF]) and is denoted F max (θ). We will therefore consider the fidelity relative to F max (θ). In all cases the error is computed as the average over M test = 300 random independent parameters θ test 1 , θ test 2 , ..., θ test Mtest drawn (once for all) from the distribution P (θ) and has the following expression:

1 M test Mtest k=1 1 - F(u; θ test k ) F max (θ test k ) . (5)
For sections 3.1 and 3.2 we will also plot the max relative error:

max k=1,...,Mtest 1 - F(u; θ test k ) F max (θ test k ) . (6)
Finally, in order to compare our algorithm with those from the literature, we take as indicator of the numerical effort the number of gradient ∇ u L(u; θ) evaluations; for instance one iteration of SGD or Adam algorithms count as M gradient evaluations. In all situations we used for the Adam algorithm the standard values β 1 = 0.9, β 2 = 0.999, ε = 10 -8 .

Two level inhomogeneous ensemble

Consider an ensemble of spins as in [8, section III.]. The spins have different Larmor frequencies ω in the range [0.8, 1.2] and the controls (L = 2) have multiplicative inhomogeneity ∈ [0.8, 1.2]; we set θ = (ω, ) and with the previous notations the dynamics corresponds to the equation:

ċ1 (t; θ) ċ2 (t; θ) = 0.5ωi 0.5 (u 2 (t) -iu 1 (t)) -0.5 (u 2 (t) -iu 1 (t)) -0.5ωi c 1 (t; θ) c 2 (t; θ) , (7)
where c 1 , c 2 are the coefficients of the wavefunction of the spin system in the canonical basis, as detailed in equation ( 2). The initial state of each member of the quantum ensemble is set to |ψ 0 = |0 ; i.e., C 0 = (1, 0) T , and the goal is to reach the target state |ψ target = |1 ; i.e., C target = (0, 1) T . The objective is encoded as the requirement to minimize:

J(u) = 1 2 1 - Θ | C(T ; θ), C target | 2 P (dθ) . (8)
Here F max (θ) = 1. The total time is T = 2 is divided into Q = 200 time steps, of length ∆t = T /Q = 0.01 each. The initial choice for the control u is u k=0 (t) = {u 0 1 (t) = sin t, u 0 2 (t) = sin t}. Several mini-batch sizes M = 1, 4, 8, 16 and 32 are tested and compared with implementation in [8, section III.A.] where a 2D uniform grid of 5 × 5 values for θ is chosen. In all cases very good convergence results are attained. We plot in Figure 1 the results for M = 1, M = 4 relative to the convergence with the uniform 5 × 5 grid. In all cases (M = 1, 4, uniform grid) we set α = 500; note that the learning rate α was optimized to obtain the best possible results for the fixed grid algorithm and indeed the results are better than those in [8, section III.A.]. But similar conclusions are reached for any value of α. An acceleration by a factor of 5 is obtained for both M = 1 and M = 4, essentially due to the fact that each SGD iteration uses only M gradient evaluations. Note that the SGD algorithm oscillates but these oscillations can be cured by lowering α (or stopping the search) as soon as a good result is obtained. The question of which is the best choice among M = 1 and M = 4 is a matter of striking a balance between speed and uncertainty: for M = 4 the convergence is slightly slower but oscillations are diminished. This behavior is observed, to a larger or lesser extent, in all test cases.

Note that in order to compare our learning rate α (for the fixed uniform grid) with that in [8, section III.A.] a multiplicative factor of ∆t/2 has to be introduced because our gradient (see Appendix A) contains an extra ∆t factor and the coefficient 1/2. Thus one should transform α = 500 to 1/2 * 0.01 * 500 = 2.5 to compare with 0.2 used in [START_REF] Chen | Sampling-based learning control of inhomogeneous quantum ensembles[END_REF]. 5)). Bottom image: maximum (over the sample) fidelity error (as defined in equation ( 6)). We consider three simulations: a fixed uniform 2D grid (M = 25) as in [8, section III.A.] and the SGD algorithm with M = 1 and M = 4. This SGD converges about 5 times faster: the mean fidelity error of 2.0 × 10 -3 is obtained after 1250 gradient evaluations of the fixed grid algorithm and after 250 evaluations of the SGD algorithm with M = 1, 4. Same for other levels of errors.

A three level Λ atomic ensemble

In this section we test a Λ atomic ensemble from [8, Section IV] which can be written as a 3-level system with the following dynamics:

  ċ1 (t; θ) ċ2 (t; θ) ċ3 (t; θ)   =   -1.5ωi 0 -i u 2 (t) 0 -ωi -i u 1 (t) -i u 2 (t) -i u 1 (t) 0     c 1 (t; θ) c 2 (t; θ) c 3 (t; θ)   , (9)
where ω and have uniform distributions in [0.8, 1.2] and c 1 , c 2 , c 3 are the coefficients of the wavefunction of the spin system in the canonical basis, as detailed in equation ( 2). The objective is to find a control u(t) = (u 1 (t), u 2 (t)) which drives all the inhomogeneous members from

|ψ 0 = 1 √ 3 (|1 + |2 + |3 ) (i.e., C 0 = ( 1 √ 3 , 1 √ 3 , 1 √
3 )) to |ψ target = |3 (i.e., C target = (0, 0, 1)); the objective is encoded as the minimization of ( 8). Here F max (θ) = 1.

We plot in Figure 2 the results for M = 1 and M = 4 relative to the convergence with an uniform grid as in [8, section IV.]. In all cases (M = 1, 4, uniform grid) we set α = 100. The acceleration factor is around 7 for M = 4 and even larger for M = 1 (but at the price of larger oscillations too).

A 3D example: two spin systems without crosscorrelated relaxation

As argued before, methods from the literature may have difficulties to address high dimensional parameters, and often limit to two dimensional (d = 2) inhomogeneity (see however [33, Sec. V.B] for a 3D case). In order to test the full power of our method, we consider two situations that extend cases treated in the literature but have never been treated before. The first test is a three dimensional (d = 3) example which addresses the coherence transfer between two spins without cross-correlated relaxation, taken from [28, Section III.B.1. eq [START_REF] Khodjasteh | Dynamically error-corrected gates for universal quantum computation[END_REF]] (but with an additional inhomogeneity dimension). An example of such a system is an isolated hetero-nuclear spin system composed of two coupled spins 1/2 corresponding to atoms 1 H and 15 N . For a general treatment of the relaxation terms and the formulation of this equation see [START_REF] Allard | The complete homogeneous master equation for a heteronuclear two-spin system in the basis of cartesian product operators[END_REF]. The spins display control inhomogeneity described by the parameter as above but there is also variation in the relaxation rate and coupling mean fidelity error as defined in equation [START_REF] Belhadj | Ensemble controllability and discrimination of perturbed bilinear control systems on connected, simple, compact Lie groups[END_REF]. Bottom image: maximum (over the sample) fidelity error (as defined in equation ( 6)). We consider two algorithms: a fixed uniform 2D grid (M = 25) as in [8, section IV.] and the SGD algorithm with M = 1 and M = 4. This latter approach converges about 7 times faster: the convergence settles in after 17 500 gradient evaluations of the fixed grid algorithm compared with cca. 2 500 evaluations of the SGD algorithm. This acceleration factor is even more important for M = 1 but at the price of larger oscillations.

constant, which, denoting θ = ( , J, ξ) results in the dynamical system:

   ċ1 (t; θ) ċ2 (t; θ) ċ3 (t; θ) ċ4 (t; θ)    =    0 -u 1 (t) 0 0 u 1 (t) -ξ -J 0 0 J -ξ -u 2 0 0 u 2 0       c 1 (t; θ) c 2 (t; θ) c 3 (t; θ) c 4 (t; θ)    . ( 10
)
Let us denote by I 1x = σ x /2, I 1y = σ y /2, I 1z = σ z /2 (here σ x , σ y σ z are the Pauli matrices) the spin operators corresponding to the first spin and I 2x , I 2y , I 2z the corresponding objects for the second spin. With the usual notations for the Kronecker products,

c 1 = I 1z , c 2 = I 1x , c 3 = 2I 1y I 2z , c 4 = 2I 1z I 2z
; the exact derivation of this equation is beyond the scope of this work, see [START_REF] Allard | The complete homogeneous master equation for a heteronuclear two-spin system in the basis of cartesian product operators[END_REF][START_REF] Glaser | Unitary control in quantum ensembles: Maximizing signal intensity in coherent spectroscopy[END_REF][START_REF] Khaneja | Boundary of quantum evolution under decoherence[END_REF] for details. On the other hand also note that the dynamics is not reversible (relaxation is present) and the equations do not correspond to a unitary evolution.

The inhomogeneity θ = ( , J, ξ) is uniformly distributed in Θ = [0.9, 1.1]× [0.5, 1.5]×[0, 2]. The final time T = 7π/6 is discretized with Q = 200 uniform time steps. The control is initialized as before. The initial state is encoded as c 0 = (1, 0, 0, 0) and the target is to minimize the three-dimensional integral:

J (u) = 1 - Θ c 4 (T ; θ)P (dθ). ( 11
)
Recall that here the fidelity is F(u; θ) = c 4 (T, θ); in this case (see [START_REF] Glaser | Unitary control in quantum ensembles: Maximizing signal intensity in coherent spectroscopy[END_REF][START_REF] Khaneja | Boundary of quantum evolution under decoherence[END_REF]) F max (θ) = 1 + (ξ/J) 2 -ξ/J (the worse performance being -F max (θ)). The results are in Figures 3 and4. Note that although for each θ taken individually the figure F max (θ) can be attained with a pair (recall L = 2) of suitable control fields, it is unknown whether a unique control pair exists ensuring 100% (relative to F max (θ)) target yield simultaneously for all θ ∈ Θ. In practice we did not find any, irrespective of the algorithm hyperparameters such as α, the maximum number of iterations etc.; we conclude on one hand that this ensemble is not 100% simultaneously controllable and on the other hand that our procedure improves significantly the robustness of the control with respect to θ ∈ Θ from an initial value of 67% up to 95%. Note that the results from the literature (which for this case only consider 2 dimensional inhomogeneity) do not obtain 100% control either (exact figure is not reported). 5). We set M = 4; for the SGD algorithm we choose α = 10.0 and for the Adam algorithm we set α = 0.01. The continuous (-) and dotted (•) curves stand for the mean fidelity errors of the plain SGD and Adam algorithm respectively; the convergence is similar and a 95% mean target relative fidelity (or equivalently 5% mean target relative fidelity error) is obtained. For the controls see Figure 4. Figure 4: Converged controls for the SGD (up) and Adam (bottom) for the situation in in Section 3.3 (for the convergence see Figure 3). Controls obtained with the SGD algorithm are smoother than those from the Adam algorithm.

A 6D example: two spin systems with cross-correlated relaxation

We continue here to address new systems that previous methods could not treat. We consider an ensemble of two spin systems with cross-correlated relaxation as in [20, Section III.A.2.], [28, Section III.B.2 eq. ( 16)] and also [START_REF] Wang | Free-endpoint optimal control of inhomogeneous bilinear ensemble systems[END_REF]Example 3], [START_REF] Allard | The complete homogeneous master equation for a heteronuclear two-spin system in the basis of cartesian product operators[END_REF].

The spins display control inhomogeneity described by the parameters 1 and 2 and there is also variation in the auto-correlated relaxation rate ξ a , the quotient ξ c /ξ a of the cross-correlation relaxation rate ξ c with respect to the auto-correlated relaxation rate ξ a and finally, a dispersion in the Larmor frequencies of each spin.

Denoting θ = ( 1 , 2 , ω 1 , ω 2 , ξ a , ξ c /ξ a ) ∈ Θ = [0.9, 1.1] 2 × [0, 1] 2 × [0.75, 1
.25] × [0.7, 0.9], the dynamical system can be written:

        ċ1 (t; θ) ċ2 (t; θ) ċ3 (t; θ) ċ4 (t; θ) ċ5 (t; θ) ċ6 (t; θ)         =         0 -1 u 1 (t) 2 u 2 (t) 0 0 0 1 u 1 (t) -ξ a ω 1 -J -ξ c 0 -2 u 2 (t) -ω 1 -ξ a -ξ c J 0 0 J -ξ c -ξ a ω 2 -2 u 2 (t) 0 -ξ c -J -ω 2 -ξ a 1 u 1 (t) 0 0 0 2 u 2 (t) -1 u 1 (t) 0                 c 1 (t; θ) c 2 (t; θ) c 3 (t; θ) c 4 (t; θ) c 5 (t; θ) c 6 (t; θ)         . (12)
The vector C = (c 1 , ..., c 6 ) has real entries and, with the same notations as in equation [START_REF] Ding | Robust quantum control against clock noises in multiqubit systems[END_REF],

c 1 = I 1z , c 2 = I 1x , c 3 = I 1y , c 4 = 2I 1y I 2z , c 5 = 2I 1x I 2z , c 6 = 2I 1z I 2z .
The relations are similar to that in section 3.3, with the exception that there are two new entries c 3 and c 5 due to the presence of cross-correlation, see [START_REF] Glaser | Unitary control in quantum ensembles: Maximizing signal intensity in coherent spectroscopy[END_REF][START_REF] Khaneja | Boundary of quantum evolution under decoherence[END_REF][START_REF] Allard | The complete homogeneous master equation for a heteronuclear two-spin system in the basis of cartesian product operators[END_REF] for details of the derivation of the model; the dynamics is not reversible (relaxation is present) nor unitary. We set J = 1; the total time T = 5 is discretized with Q = 200 uniform time steps. The control is initialized as before. The initial state is encoded as c 0 = (1, 0, 0, 0, 0, 0) and the target is to minimize the six-dimensional integral:

J (u) = 1 - Θ c 6 (T ; θ)P (dθ). ( 13
)
Recall that here the fidelity is F(u; θ) = c 6 (T, θ). In this case too, the best attainable performance for a single molecule is known (see [START_REF] Glaser | Unitary control in quantum ensembles: Maximizing signal intensity in coherent spectroscopy[END_REF][START_REF] Khaneja | Boundary of quantum evolution under decoherence[END_REF]) and defined by F max (θ) = 1 + η 2 -η where η = The quantity plotted is defined as in the Figure 3. We set M = 4; for the SGD algorithm we choose α = 10.0 and for the Adam algorithm we set α = 0.01. The continuous (-) and dotted (•) curves stand for the mean fidelity errors of the plain SGD and Adam algorithm respectively; the convergence is similar and 91% mean relative fidelity is obtained. For the controls see Figure 6.

The simulation results are in Figures 5 and6. Same conventions are kept as in the previous section (fidelity is relative to maximum attainable figure) and same considerations still apply: 100% simultaneous controllability does not seem attainable but significant improvement in the robustness is obtained (91% up from -8%).

Stochastic convergence behaviors

The convergence of the stochastic algorithms can have two important regimes:

1. first, when all members of the ensemble can be simultaneously optimized to 100%; in our situation this is equivalent to simultaneous controllability. In this case convergence is "easier" because it is "enough" to follow the gradient for each parameter value in order to converge; at convergence all gradients (as distribution with respect to ω), will collapse to (in practice will be close to) a Dirac mass.

2. secondly, when members of the ensemble cannot be simultaneously optimized; in this case, reaching full control for some θ value will harm the quality of some other parameter values θ = θ. At convergence gradients will not be distributed as a Dirac mass any more, but the average with respect to theta will be zero (in practice small). Adam controls after 500 iterations u1 u2

Figure 6: Converged controls for the SGD (up) and Adam (bottom) for the situation in in Section 3.4 (for the convergence see Figure 5). Controls obtained with the SGD algorithm are smoother than those from the Adam algorithm.

We illustrate this behavior in figures 7 and 8 where we plot the histograms of the gradient (with respect to the first field) ∇ u 1 (t) J (u(t n ), θ) as random variables of θ at some time snapshots t. It is noticed that while in the first example it is possible to reduce significantly the gradient absolute value for all members of the sample (because simultaneous controllability holds true), in the second test case this reduction reaches a limit and the algorithm tries instead to center the gradients on zero so that the average be as low as possible.

Discussion and conclusion

We proposed and tested in this work a stochastic approach to compute the optimal controls of inhomogeneous quantum ensembles. The algorithms have been employed before in other areas of stochastic optimization but not tested in this context (see [34] for similar algorithms). Their specificity is to draw at each iteration a new set of parameters from the inhomogeneous distribution. Although at first the intuition may not recommend such an approach, the numerical results indicate not only convergence but also faster convergence than methods based on fixed samples. In addition the method can address situations when the space of parameters is large and was tested successfully on a 6-dimensional example.

For ). Six time instants t are chosen uniformly in [0, T ]: t = 0, T /5, 2T /5, ..., T . In red are the gradients at u = u 1 (iteration k = 1) and in blue the gradients at u = u 500 (iteration k = 500). Here we consider the case in Section 3.1, see Figure 8 eration of the stochastic algorithms (SGD, Adam) is due essentially to the lower effort per iteration compared to a fixed grid sampling (both being proportional to the number of samples used). In higher dimensions the fixed grid approach is inherently less efficient due to the curse of dimensionality and may even be prohibitively large.

On the other hand, compared with SGD, the Adam algorithm has the advantage to be more robust with respect to the choice of the learning rate α, but the controls are less regular.

Finally, one of the limitations of this work is to use constant learning rates. Variable learning rates are potentially interesting as it could speed up convergence in the initial phases by using large values of α and avoid oscillations in the end by lowering α. Several schedules are proposed in the stochastic optimization literature (inverse linear, piecewise constant, ...) but their analysis remains for future work. *

A Gradient computation

We detail below the computation of the gradient for a single parameter θ, the general case being just a mean over θ. Consider the so-called adjoint state λ(t; θ); it is defined at the final time as the derivative of the outcome with respect to C(T ; θ). For instance, for sections 3.1 -3.2: λ(T ; θ) = -C target , C(T, θ) C target while for sections 3.3 -3.4 we set λ(T ; θ) = -1. Then for t < T , λ(t; θ) is the solution of the (backward) equation d dt λ(t; θ) = X(t, θ) † λ(t; θ), where X(t, θ) † is the transpose conjugate of X when X has complex entries (examples 3.1 and 3.2) and reduces to the transpose when X is a real matrix (examples 3.3 and 3.4). Then ∇ u(t) J = λ(t; θ), ∂X(t;θ) ∂u(t) C(t; θ) . In practice, given that u is discretized, the state C and the adjoint state λ are also discretized at time instants t n = n∆t: C n (θ) C(t n ; θ), λ n (θ) λ(t n ; θ) which satisfy C n+1 (θ) = e ∆tX(u(tn);θ) C n (θ) and λ n (θ) = e ∆tX(u(tn);θ) † λ n+1 (θ) and the exact discrete gradient is ∇ u(tn) J = λ n+1 (θ), ∂e ∆tX(u(tn);θ) ∂u(tn)

C n (θ) .

Finally, in order to compute ∂e ∆tX(u(tn);θ) ∂u(tn) we use a "divide and conquer" approach coupled with a 8-th order expansion as in [3, formula (11)]) to obtain at the same time the exponential and the gradient ([25, Chapter VI]) from the knowledge of the inputs X(u(t n ); θ) and ∂X(u(tn);θ) ∂u k (tn) .

Figure 1 :

 1 Figure 1: Convergence for the numerical case in Section 3.1. Top image: mean fidelity error (as defined in equation (5)). Bottom image: maximum (over the sample) fidelity error (as defined in equation (6)). We consider three simulations: a fixed uniform 2D grid (M = 25) as in [8, section III.A.] and the SGD algorithm with M = 1 and M = 4. This SGD converges about 5 times faster: the mean fidelity error of 2.0 × 10 -3 is obtained after 1250 gradient evaluations of the fixed grid algorithm and after 250 evaluations of the SGD algorithm with M = 1, 4. Same for other levels of errors.

Figure 2 :

 2 Figure 2: Convergence for the numerical case in Section 3.2. Top image:mean fidelity error as defined in equation[START_REF] Belhadj | Ensemble controllability and discrimination of perturbed bilinear control systems on connected, simple, compact Lie groups[END_REF]. Bottom image: maximum (over the sample) fidelity error (as defined in equation (6)). We consider two algorithms: a fixed uniform 2D grid (M = 25) as in [8, section IV.] and the SGD algorithm with M = 1 and M = 4. This latter approach converges about 7 times faster: the convergence settles in after 17 500 gradient evaluations of the fixed grid algorithm compared with cca. 2 500 evaluations of the SGD algorithm. This acceleration factor is even more important for M = 1 but at the price of larger oscillations.

  mean rel fidelity error, Adam mean rel fidelity error, plain SGD

Figure 3 :

 3 Figure 3: Convergence for the numerical case in Section 3.3. The quantity plotted is given in equation (5). We set M = 4; for the SGD algorithm we choose α = 10.0 and for the Adam algorithm we set α = 0.01. The continuous (-) and dotted (•) curves stand for the mean fidelity errors of the plain SGD and Adam algorithm respectively; the convergence is similar and a 95% mean target relative fidelity (or equivalently 5% mean target relative fidelity error) is obtained. For the controls see Figure4.

ξ 2 a -ξ 2 cJ 2 Figure 5 :

 2225 Figure5: Convergence for the numerical case in Section 3.4. The quantity plotted is defined as in the Figure3. We set M = 4; for the SGD algorithm we choose α = 10.0 and for the Adam algorithm we set α = 0.01. The continuous (-) and dotted (•) curves stand for the mean fidelity errors of the plain SGD and Adam algorithm respectively; the convergence is similar and 91% mean relative fidelity is obtained. For the controls see Figure6.

Figure 8 :

 8 Figure 8: Histogram of the gradients as in Figure 7 except that here the results correspond to the test case in Section 3.4.


Henaff M-A

Schmitz

Christina Schmitz
email: christina.schmitz@inserm.fr

Decision-making in a changing world: a study in Autism Spectrum Disorders

Keywords: Autism Spectrum Disorders, Social, Instability, Decision-making

To learn to deal with the unexpected is essential to adaptation to a social, therefore often unpredictable environment. Fourteen adults with Autism Spectrum Disorders (ASD) and 15 controls underwent a decision-making task aimed at investigating the influence of either a social or a non-social environment, and its interaction with either a stable (with constant probabilities) or an unstable (with changing probabilities) context on their performance. Participants with ASD presented with difficulties in accessing underlying statistical rules in an unstable context, a deficit especially enhanced in the social environment.

These results point out that the difficulties people with ASD encounter in their social life might be caused by impaired social cues processing and by the unpredictability associated with the social world.

Autism Spectrum Disorders (ASD) are pervasive developmental disorders characterized by social communication problems, difficulties with reciprocal social interactions, and unusual patterns of repetitive behavior (American Psychiatric Association 2000). Clinical reports show that when left to their own devices, children with ASD tend to compulsively create patterns, which may reflect a strong desire to impose control over changing events [START_REF] Frith | Studies in pattern detection in normal and autistic children. I. Immediate recall of auditory sequences[END_REF]. Kanner observed in children with ASD: "The child's behavior is governed by an anxiously obsessive desire for the maintenance of sameness. Changes in routine can drive him to despair" [START_REF] Kanner | Autistic disturbances of affective contact[END_REF]). Individuals with ASD may feel stressed, anxious or confused if unpredictable changes occur in their environment. This greater need for predictability might arise from systemizing mechanism being set too high in ASD [START_REF] Baron-Cohen | The hyper-systemizing, assortative mating theory of autism[END_REF]. Indeed, according to the hyper-systemizing theory individuals with ASD preferentially develop a systemizing style that makes them good at understanding systems that are highly predictable, with low variance [START_REF] Baron-Cohen | The extreme male brain theory of autism[END_REF].

This theory would explain why people with autism prefer either no change, or systems which change in highly lawful or predictable ways. On the contrary, they might become disabled when faced with systems characterized by complex change (such as social behavior) [START_REF] Baron-Cohen | The hyper-systemizing, assortative mating theory of autism[END_REF].

Describing subjects with Asperger syndrome, Volkmar et al. observed that: "their lack of spontaneous adaptation is accompanied by marked reliance on formalistic rules of behavior and rigid social conventions" [START_REF] Volkmar | Handbook of Autism and Pervasive Developmental Disorders[END_REF]. Individuals with ASD display difficulties in using their formal social knowledge in quick-paced, rapidly shifting, social situations [START_REF] Allman | Intuition and autism: a possible role for Von Economo neurons[END_REF]. This is illustrated by the compensatory strategy expressed by person with ASD who reported: "When I encounter a new social situation, I have to search my memory for a similar experience that I can use as a model for my next action. I have a very difficult time when I am confronted with unexpected social surprises" [START_REF] Volkmar | Handbook of Autism and Pervasive Developmental Disorders[END_REF]. This might partially explain the difficulties they encounter in making "on the spot" decisions: autobiographical accounts describe how decision-making processes can become "locked-up" for people with ASD [START_REF] Luke | Decision-making difficulties experienced by adults with autism spectrum conditions[END_REF].

To learn to deal with the unexpected is essential to adaptation to a social, therefore often unpredictable and fluctuating, environment. The information received in daily social life is inherently probabilistic. Human behavior has been described as a complex, highly unpredictable system [START_REF] Blackerby | Application of Chaos Theory to Psychological Models[END_REF]. Finding underlying statistical rules (i.e. regularities) and constructing an accurate representation of the world, despite its uncertainty, might be significant challenges for people with ASD [START_REF] Gomot | A challenging, unpredictable world for people with Autism Spectrum Disorder[END_REF]. A recent psychophysical experiment [START_REF] Ma | Bayesian inference with probabilistic population codes[END_REF] indicates that neurotypical people perform near-optimal Bayesian inference (a rational way of updating knowledge with information from the environment) in a wide variety of tasks, ranging from cue integration to decision making. Therefore, social interaction deficits may be linked (among other things) to an inability to extract regularities from the environment. Although social deficits in ASD have often been studied in relation to theory of mind [START_REF] Baron-Cohen | Recognition of mental state terms. Clinical findings in children with autism and a functional neuroimaging study of normal adults[END_REF][START_REF] Rutherford | Reading the mind in the voice: a study with normal adults and adults with Asperger syndrome and high functioning autism[END_REF]Lombardoet al. 2007), face processing [START_REF] Ewing | Atypical updating of face representations with experience in children with autism[END_REF][START_REF] Nagai | Comparing face processing strategies between typically-developed observers and observers with autism using sub-sampled-pixels presentation in response classification technique[END_REF]) and emotion recognition (for a review, see [START_REF] Uljarevic | Recognition of Emotions in Autism: A Formal Meta-Analysis[END_REF], the processing of the unpredictable aspect of the social world per se has not yet been investigated.

According to [START_REF] Allman | Intuition and autism: a possible role for Von Economo neurons[END_REF], the social disabilities observed in ASD may be partially due to the abnormal development of a specific class of neurons, the Von Economo neurons, which may be involved in the fast intuitive assessment of complex situations. These neurons are located in the anterior cingulate (ACC) and fronto-insular cortex. The authors hypothesized that these neurons enable individuals to make quick, intuitive decisions that enable them to adapt to rapidly changing conditions. Allman et al. further predicted that these deficiencies would be observable in non-social tasks, as well as in social situations (because social emotions, by their very nature, involve considerable uncertainty). There is no clear evidence whether people with ASD have a primary difficulty extracting regularities from any kind of information, or if these difficulties are particularly enhanced in a social environment. Indeed, in individuals with ASD, the difficulties created by the social nature of the environment could interact with a global deficit they have in adapting their behavior to an unstable context.

Our aim was to study the ability of people with ASD to extract regularities from the environment, and to investigate the potential influence of the social nature of the environment on this ability. This behavioral study aimed at investigating the influence of an either social or non-social and stable (associated with constant probabilities) or unstable (associated with changing probabilities) environment on the behavior of ASD participants in a decision-making task.

Material and methods

Participants

Fourteen adult participants with ASD without intellectual deficiency (mean age: 28.4 ±6.9) took part in this study. Eleven of them did not present with delay in language onset. Fifteen adult control participants with typical development (mean age: 29.4 ±8.2) constituted the control group. The volunteers fully consented to the study, which was approved by the French National Ethics Committee. The two groups were matched with respect to age, gender and verbal IQ (see Table 1). ASD subjects were recruited through the Resources Center for Autism in the Vinatier Hospital (Lyon, France). Control subjects were recruited either using a student mailing list, or among friends of researcher colleagues for the older participants.

All participants in the clinical group had been diagnosed before the study. Individuals with a clinical diagnosis of Asperger Syndrom or High-Functioning Autism according to the DSM-IV-TR (American Psychiatric Association 2000) were included in the study. All diagnoses were made by experienced clinicians and based on clinical observations of the participants, using the Autism Diagnostic Interview [START_REF] Couteur | Autism diagnostic interview: A standardized investigator-based instrument[END_REF] or the Autism Diagnostic Observation Schedule [START_REF] Lord | Autism diagnostic observation schedule: A standardized observation of communicative and social behavior[END_REF]. Only one of the 14 ASD subjects presented with a comorbidity. This subject also suffered from a bipolar disorder and was taking a treatment at the time of the study, consisting in a serotoninergic antidepressant and an atypical neuroleptic. The reaction times of this subject were not significantly different from the reaction times of the ASD group (t-test, p=0.44). Apart from this subject, ASD participants did not take any medication and did not present any comorbidity at the time of the study.

All participants (both control and ASD subjects) also performed the Autism Quotient (AQ) test [START_REF] Baron-Cohen | The autism-spectrum quotient (AQ): evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians[END_REF], a measure of the extent of autistic traits in adults. Prior to the experiment, the control participants' AQ was calculated in order to verify that their score did not extent the limit of 31, since [START_REF] Baron-Cohen | The autism-spectrum quotient (AQ): evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians[END_REF] cited a score of 32 or more as indicating clinically significant levels of autistic traits. All control subjects met this criterion. T tests were run to test for statistically significant differences concerning age, gender, verbal IQ, performance IQ and AQ (see Table 1). The performance IQ, but not the verbal one, differed between the two groups. A difference between the verbal and performance IQ is a common finding in persons with ASD without delay in language onset [START_REF] Chiang | A meta-analysis of differences in IQ profiles between individuals with Asperger's disorder and high-functioning autism[END_REF][START_REF] Wilson | The Neuropsychology of Male Adults With High-Functioning Autism or Asperger Syndrome[END_REF]).

[insert table 1 about here]

Participants received 20 Euros at the end of the experiment.

Design and procedure

We compared the behavior of ASD patients and matched controls in a decision task. The task was adapted from Behrens and collaborators [START_REF] Behrens | Learning the value of information in an uncertain world[END_REF]2008). Participants were presented with two boxes with hidden reward probabilities, and were asked to choose between them by pressing one of the two available buttons on a regular keyboard. The goal was to score as many points as possible. In each trial, if one box was rewarding, the other was not. Participants received feedback about their choice at the end of the trial and their score was updated accordingly. To avoid stress in participants, especially in those with ASD, subjects had no time limitation to choose one of the two boxes, but they were asked to answer as soon as their choice was made.

All participants carried out a familiarization task first. In this familiarization task, participants did not receive a cue before making their choice. The boxes had hidden probabilities that varied during the session (120 trials): during the first 60 trials, the probability of winning was 75% for one box, and 25% for the other box (stable context). During the last 60 trials, the probabilities associated with the boxes were unstable, switching from 20% to 80% twice over the 60 trials (unstable context). Participants needed to have experienced a "stable" context in order to induce an "unstable" context. Therefore, in this familiarization task, the stable condition was always performed before the unstable condition.

Participants then performed the cue-guided task: a cue (either non-social or social) was presented to subjects at the beginning of each trial, before they made their choice. In the non-social environment, the cue was an arrow (indicating a computer's choice) and was presented to subjects for 800 ms. In the social environment, a short movie depicting the choice of an actor was presented to subjects for 1500 ms before each trial. In this movie, the actor dynamically indicated his choice by looking at the chosen box, and the chosen box then lit up (see Fig. 1). Stimuli presentation was mediated by the Presentation software (Neurobehavioral Systems, EU).

[insert Fig. 1 about here] Participants underwent three distinct conditions (60 trials each, see Table 2) in both non-social and social environments. Different hidden probabilities of indicating the correct answer were associated with the cue and different hidden probabilities of winning were attributed to the boxes.

[insert table 2 about here]

(1) "80% reliable cue": the probabilities of winning were greater when following the cue (reliable in 80% of trials) than when following the box probabilities (60% for one of the boxes). Cue reliability was stable.

(2) "60% reliable cue": the probabilities of winning were greater when relying on the box probabilities (80% for one of the boxes) than when following the cue (reliable in 60% of trials). Cue reliability was stable.

(3) "Unstable cue reliability": cue reliability was unstable, switching from 80% reliable to 80% unreliable twice over the 60 trials. The probabilities of winning were greater when following the cue (choosing the opposite of the cue indication when the cue was 80% unreliable, or following its indication when it was 80% reliable) than when following the box probabilities (60% for one of the boxes).

Inside each group (ASD and control subjects), half of the participants performed the experiment with order 1: {Familiarization task; "Non-social 80% reliable cue"; "Non-social 60% reliable cue"; "Non-social unstable cue reliability"; "Social 80% reliable cue"; "Social 60% reliable cue"; "Social unstable cue reliability"}; and half of the participants performed the experiment with order 2: {Familiarization task; "Social 80% reliable cue"; "Social 60% reliable cue"; "Social unstable cue reliability"; "Non-social 80% reliable cue"; "Non-social 60% reliable cue"; "Non-social unstable cue reliability"}. The color of the boxes was also counterbalanced between subjects to avoid an effect of color preference.

Variables of interest defined for the behavioral analysis

We used different variables to quantify the subjects' behavior.

-the subjects' reaction time and performance (calculated as the proportion of trials won in each condition).

-success criterion: defined as winning 60% of trials relative to the total number of trials.

Reaching the success criterion was a Boolean equal to 1 if the subject's performance was higher than 60%, and equal to 0 if it was lower. This enabled to extract for each group the percentage of subjects achieving the success criterion in the different conditions.

-index of perseveration after a failure: number of trials between a failure and a change of choice.

-proportion of trials in which the subject followed the cue: calculated in each condition relative to the total number of trials.

Statistical analysis

The subjects' reaction time, the performance, the index of perseveration after a failure and the proportion of trials in which the subject followed the cue were analysed with multifactorial ANOVAs. Multifactorial ANOVAs aimed at understanding which factor better explains each variable of interest.

In the familiarization task we were interested in the influence of three factors :

(1) the "subject" factor enabled to study how each variable was explained by the fact that each subject is different from the other. This factor was not linked to our protocol : it was a random factor. The "subject" factor had 29 levels, corresponding to the number of subjects.

(2) the "group" factor enabled to study the effect of being part of the ASD group compared to being part of the control group on the variable of interest. This factor was defined in our protocol, it was a fixed factor. This "group" factor had two levels, corresponding to the "ASD group" and the "control group"

(3) the "stability" factor enabled to study the effect of being in an unstable context compared to being in a stable one on the variable of interest. This factor was also defined in our protocol, it was a fixed factor and it held two levels: "stable" and "unstable", corresponding to the two kinds of context.

The "subject" factor was random, the other ones were fixed. We therefore used a mixed-effect model to analyse the data.

In the cue-guided task we were interested in the influence of the same three factors as those from the familiarization task model : "subject", "group" and "stability", and two supplementary factors linked to the supplementary conditions of this task :

(4) the "social nature" factor enabled to study the effect of being in a social environment compared to be in a non-social one on the variable of interest. This factor was defined in our protocol, it was a fixed factor and it held two levels: "non-social" and "social", corresponding to the two kinds of environment.

(5) the "reliability" factor enabled to study the effect of receiving an information more or less reliable on the variable of interest. This factor was also defined in our protocol, it was a fixed factor and it held three levels: "20% reliable", "60% reliable", and "80% reliable".

The "subject" factor was random, the other ones were fixed. We therefore also used a mixed-effect model.

As a non parametric equivalent for 3 ways ANOVA test does not exist, it is generally admitted to use the parametric approach to analyze a multifactorial (more than 2 factors) design even if the distribution of the studied variable deviate from the normal distribution. Moreover, parametric statistical analyses are widely used even with small sample sizes as the ANOVA is very robust (i.e., not creating effects in case of assumption violation, [START_REF] Schmider | Is it really robust? Reinvestigating the robustness of ANOVA against violations of the normal distribution assumption[END_REF].

Post-hoc tests were applied to compare groups (Bonferroni multiple comparison test). Before each Bonferroni multiple comparison test, the equality of variances was checked. If the variables in the two groups had equal variances, the degrees of freedom had a value of 27 (= number of subjects -2). In this case the value of the degrees of freedom was reported in the result section. If the variables in the two groups had unequal variances, the degrees of freedom were calculated with the Welch Correction. In this case the corrected value of the degrees of freedom was reported in the result section.

To compare the success criterion between the 2 groups, we used a "proportion comparison test" [START_REF] Schwartz | Méthodes statistiques à l'usage des médecins et des biologistes[END_REF].

Results

Reaction times

We were first interested in whether the ASD and control groups differed in terms of their reaction time.

Regardless of the condition, ANOVA showed slower reaction times in ASD subjects compared to controls in both the familiarization task and the cue-guided task (P < 0.001, T = 15.96, corrected degrees of freedom = 23).

Performances

We were then interested in whether the ASD and control groups differed in terms of their performances (percentage of trials won). ANOVA showed an effect of the "group" factor in the familiarization task and in the cue-guided task. In the familiarization task, ASD subjects' performances were significantly lower than controls' (P < 0.01, T = 3.03, corrected degrees of freedom = 27), successful trial percentage was 55% (±7.2) for ASD and 60% (±7.1) for controls. A difference between the two groups' performances was also found in the cue-guided task (P < 0.001, T = 6.39, corrected degrees of freedom = 25): the percentage of successful trials was 58.2% (±16.1) for ASD subjects and 66.6% (±11.8) for controls. ANOVA also showed an effect of stability on the subjects' performances: regardless of the group, subjects showed lower performance (P < 0.001, T = 9.18, degrees of freedom = 27) in the unstable context (58.7% ±15.9 of successful trials) compared to the stable one (68.2% ±10.1 of successful trials). No interaction between the factors "stability" and "group" was found.

We also tested whether the AQ could be negatively correlated with performances in the unstable conditions and in the social ones. In the familiarization task and the cue-guided task, a significant negative correlation between AQ and performances was found (in the familiarization task: R = -0.41 and P < 0.05; in the cue-guided task: R = -0.5 and P < 0.01). Considering the different sessions separately, a significant negative correlation between AQ and performances was found in all unstable conditions (in the familiarization task: R = -0.42, P < 0.05, in the non-social environment: R = -0.53, P < 0.01, in the social environment: R = -0.53, P < 0.01). This correlation was also found in the "80% reliable cue" social condition (R = -0.41, P < 0.05) (Fig. 2).

[insert Fig. 2 about here]

Success criterion

We compared the success criteria to test whether the ASD and control groups differed in terms of the number of subjects succeeding in the task. In each condition, the success criterion was defined as winning 60% of trials relative to the total number of trials (see methods). The percentage of subjects achieving the success criterion was significantly lower in the ASD group compared to the control group in the unstable context (P < 0.001, Z = 3.30), but not in the stable one, and in the social environment but not in the nonsocial one (P < 0.05, Z = 1.93, Fig. 3). Both a social environment and an unstable context were associated with a lower number of ASD subjects reaching the success criterion, but the unstable context affected this number more than the social environment (P < 0.01, Z = 2.65).

[insert Fig. 3 about here]

Perseveration after a failure

We were also interested in whether the ASD and control groups differed in terms of perseveration. The number of trials between a failure and a change of choice was calculated. ANOVA applied to this parameter showed that regardless of the condition, ASD subjects perseverated significantly longer than controls after a failed trial (P < 0.001, T = 5.45, corrected degrees of freedom = 27): the number of trials between a failure and a change of choice was on average 3.2 (±1.8) trials for ASD subjects compared to 2.3 (±1.7) trials for controls. Moreover, a significant positive correlation between AQ and the number of trials between a failure and a change of choice was found in the unstable conditions (R = 0.47; P < 0.01) but not in the stable ones.

Proportion of trials in which the subject followed the cue

In the cue-guided task, we tested whether the ASD subjects differed from controls concerning their tendency to follow the cue in the different conditions. The proportion of trials in which the subject followed the cue indication was calculated in each condition. Interactions were found between "group" and "social" factors, and between "group" and "reliability" factors. Post-hoc tests revealed a difference between the extent to which ASD participants and controls followed the cue when the cue was 20% reliable (P < 0.01, T = 3.11, corrected degrees of freedom = 25), and when it was 80% reliable (P < 0.001, T = 8.79, degrees of freedom = 27), regardless of the condition (social and non-social conditions pooled together) (Fig. 4).

[insert Fig. 4 about here] With regard to the extent to which subjects followed the cue, a significant difference was found in the controls when comparing the "20% reliable cue" and the "60% reliable cue" conditions (P < 0.001, T = 6.63, degrees of freedom = 27), and when comparing the "60% reliable cue" and the "80% reliable cue" (P < 0.001, T = 12.74, degrees of freedom = 27). In ASD subjects, a significant difference was only found between the "20% reliable cue" and the "80% reliable cue" conditions (P < 0.001, T = 4.87, degrees of freedom = 27).

The correlation between the extent to which controls followed the cue and cue reliability was very strong: R = 0.8410 (P < 0.001). In ASD subjects the correlation between these two values was also significant but lower: R = 0.423 (P < 0.01). More precisely, only one subject out of the 15 controls showed a correlation coefficient between the extent to which he followed the cue and cue reliability lower than 0.8, whereas 7 subjects out of the 14 ASD subjects showed a correlation coefficient below 0.8 (Fig. 5).

[insert Fig. 5 about here]

We also hypothesized that the AQ could be negatively correlated with the extent to which subjects followed a cue mostly reliable, and that it could be positively correlated with the extent to which subjects followed a cue mostly non-reliable. When cue reliability was unstable, the cue could be either 80% or 20% reliable. In the unstable condition (cue reliability shifting from 80% to 20%), we found a negative significant correlation between AQ and the extent to which subjects followed the cue when cue reliability was 80% (regardless of the social nature of the environment) (with a non-social cue : R = -0.42, P < 0.05 ; with a social cue: R = -0.44, P < 0.05). Conversely, a positive correlation between AQ and the extent to which subjects followed the cue indication was found in the unstable social environment, when the cue was only 20% reliable (R = 0.39, P < 0.05) (Table 3).

[insert Table 3 about here]

Discussion

This study aimed at investigating the influence of an either social or non-social environment on the behavior of people with ASD, and its interaction with an either stable or unstable context. Our aim was to study their ability to extract regularities from the environment, and to investigate the potential influence of the social nature of the environment on it. We found that both social environment and unstable context made it difficult to integrate cues in decision making, and that the negative impact of an unstable context on ASD subjects' performances was dramatically higher than that of the social environment. Based on the fact that the success criterion of the two groups did not differ in the non-social environment and in the stable context, it is unlikely that a difference between the two groups in the performance IQ could explain the lower performance of the ASD group in the social and in the unstable context. Cue processing and the ability to use it as a function of its reliability were especially impaired in an environment presenting both social and unstable components. We also found that ASD subjects had difficulty taking cue reliability into account in their decision making.

1-Characterization of the difficulties experienced by people with ASD when facing an unstable context

Although both ASD and control groups showed lower performances in the unstable context, the percentage of subjects succeeding in the task (i.e. achieving the success criterion, defined as winning 60% of trials) was dramatically lower in the ASD group compared to the control one. Moreover, the performance correlated negatively with AQ in all the unstable conditions. Since AQ scores were significantly different between the two groups, these results suggest that ASD subjects experienced more difficulties in an unstable context than neurotypical subjects. Further, a significant positive correlation between AQ scores and perseveration after a failure was only found in the unstable context, suggesting that the unstable context enhanced perseveration behavior in ASD subjects. Moreover, in the unstable context, when the cue was mostly reliable, the extent to which participants followed it negatively correlated with the AQ, revealing an increased difficulty in ASD subjects to appropriately use the cue in the unstable context.

These results indicate that an unstable context impacts ASD subjects' performances in two ways: it enhances their perseveration after a failure, and it makes it more difficult for them to use the cue in an appropriate way (i.e. according to its reliability). At the sensory level, it has been shown that the processing of irregularities may be aberrant in ASD [START_REF] Gomot | A challenging, unpredictable world for people with Autism Spectrum Disorder[END_REF], but this is the first time that this result has been shown at a cognitive level, in a task requiring the integration of irregularities from the environment in a decision-making process. This speaks in favor of a core (non-social) deficit and calls for future studies that would combine such clear behavioral effects with neuroimaging and computational approaches to refine our understanding of this underlying mechanisms and its interaction with social contexts. This is in line with very recent attempts to propose or revisit global theories of ASD [START_REF] Qian | A Learning-Style Theory for Understanding Autistic Behaviors[END_REF][START_REF] Pellicano | When the world becomes « too real »: a Bayesian explanation of autistic perception[END_REF][START_REF] Friston | On hyperpriors and hypopriors: comment on Pellicano and Burr[END_REF]).

An impaired ability to track the instability of the context could explain the preserved performances in the stable context and poorer performances in the unstable one. Using a task similar to ours, Behrens and collaborators [START_REF] Behrens | Learning the value of information in an uncertain world[END_REF]2008) showed that neurotypical subjects assess the context instability in an optimal manner and adjust decision-making accordingly. They also disclosed that this optimal estimation of volatility was reflected in the fMRI signal in the anterior cingulate cortex (ACC), a region which has been shown to be dysfunctional in ASD subjects [START_REF] Chiu | Self responses along cingulate cortex reveal quantitative neural phenotype for highfunctioning autism[END_REF][START_REF] Thakkar | Response monitoring, repetitive behaviour and anterior cingulate abnormalities in autism spectrum disorders (ASD)[END_REF][START_REF] Dichter | Autism is characterized by dorsal anterior cingulate hyperactivation during social target detection[END_REF][START_REF] Agam | Reduced cognitive control of response inhibition by the anterior cingulate cortex in autism spectrum disorders[END_REF]. Furthermore, the variations in the ACC signal across the neurotypical population reported in Behrens'studies predicted variations in subject ability to update the information.

We propose that the estimation of volatility involving the ACC is impaired in ASD subjects. This could make it difficult to update their previous knowledge in an unstable context, leading to an inadaptive use of the context information and consequently resistance to change.

2-An interaction between the social nature of the environment and its instability

The percentage of subjects succeeding in the task in a social environment was significantly lower in the ASD group than in the control group. We also found a strong negative correlation between AQ and performance in all social conditions, suggesting that the social nature of the environment adds a supplementary difficulty for people with ASD. Further, in the social environment, when the cue was mostly reliable, the AQ was negatively correlated to the extent to which the subjects followed it. The alteration of the performance in the social environment could not be explained by mere deficits in joint attention, as the actor's choice was expressed both by his gaze turning towards the chosen box and by the lit box. These results indicate that the social environment impacts ASD subjects' performances by making it more difficult for them to use the cue in an appropriate way (i.e. according to its reliability).

Whereas some studies have not shown any behavioral differences between ASD subjects and controls in terms of orienting toward social stimuli [START_REF] Greene | Atypical neural networks for social orienting in autism spectrum disorders[END_REF][START_REF] Pruett | Social and non-social cueing of visuospatial attention in autism and typical development[END_REF], other studies have shown that subjects with ASD fail to orient to social cues [START_REF] Dawson | Children with autism fail to orient to naturally occurring social stimuli[END_REF][START_REF] Sasson | Orienting to social stimuli differentiates social cognitive impairment in autism and schizophrenia[END_REF][START_REF] Klin | Two-year-olds with autism orient to non-social contingencies rather than biological motion[END_REF]). In our task, the negative correlation between AQ and performance was only found in the social conditions requiring the use of the cue, whereas it was not found in the social conditions requiring the cue to be ignored. This indicates that despite their difficulty in appropriately using a social cue, ASD subjects were perfectly able to ignore an irrelevant social cue. For ASD subjects, the easiest way of processing a social cue was to ignore it. This argument is in favor of the hypothesis of a global failure to orient to relevant social stimuli.

We found that both social environment and unstable context made it harder to integrate cues in decision making, and that the negative impact of an unstable context on ASD performances was dramatically higher than that of the social environment. When the environment was both social and unstable, a correlation was found between the AQ and the tendency to follow the non-reliable cue. This correlation was not found in the non-social unstable condition, suggesting an interaction between the social nature of the environment and its instability: a social unstable environment seems more puzzling than a social stable environment or than a non-social unstable environment. Cue processing and the ability to use it as a function of its reliability seems especially in an environment presenting both social and unstable components.

3-Difficulty using the cue as a function of its reliability

Independently of the negative impact of both social environment and unstable context on the ASD subjects' performances, we found that ASD subjects also had a global difficulty modulating their decision as a function of the reliability of the cue. This result is in line with the study of [START_REF] Ames | The problem with using eye-gaze to infer desire: a deficit of cue inference in children with autism spectrum disorder[END_REF], which showed that children with ASD present difficulties in using arbitrary cues, both social and nonsocial, to make inferences, suggesting a cue inference deficit associated with ASD. A similar suggestion was made by Leekam and collatorators (Leekamet al. 2000) in a study investigating attention orienting in children with ASD. Although individuals with ASD had no difficulties in orienting their attention in response to exogenous cueing (an object appears in the periphery), they showed deficient endogenous cueing (a directional cue appears in a central position). Individuals with ASD may be able to orient their attention in response to simple, low-level cues, but they may have difficulty in using high-level cues, regardless of whether such cues are social or not. These arguments have clear parallels with the idea proposed by De Martino and collaborators [START_REF] De Martino | Explaining enhanced logical consistency during decision making in autism[END_REF], that is people with ASD present a global insensitivity to contextual frame. In this study, subjects performed a financial task in which the monetary prospects were presented as either loss or gain. Whereas control subjects manifested susceptibility to this framing effect, which was reflected in an increased disposition to gamble in loss compared with gain frames, ASD subjects made more consistent choices and showed no differential skin conductance response to the contextual manipulation, suggesting a reduced behavioral susceptibility to a framing effect in subjects with ASD.

However, we noted in our study that 7 out of the 14 ASD subjects appropriately modulated their decision as a function of the reliability of the cue. This means that the difficulties in processing a cue do not seem to affect all ASD subjects. Nevertheless, this preserved ability did not appear to be linked to the extent of autistic traits, since we did not find any correlation between the extent to which the subjects followed the cue indication and their AQ.

Conclusion

In conclusion, cue processing, together with the social nature of the environment and its unpredictability, cause difficulties for people with ASD. We showed here that people with ASD present a global difficulty in accessing underlying statistical rules (regularities), and that this deficit is especially enhanced in a social environment. The difficulties encountered by people with ASD in a social environment may be caused by both impaired social processing and by the unpredictability associated with the social world.

Our results support [START_REF] Allman | Intuition and autism: a possible role for Von Economo neurons[END_REF] theory, suggesting that the social disabilities observed in ASD may be linked to a deficit in the fast intuitive assessment of complex situations. This calls for future studies that would combine such clear behavioral effects with functional MRI and computational approaches to investigate the ability of people with ASD to build flexible predictions in a changing context. This is in line with very recent attempts to propose or revisit global theories of ASD [START_REF] Qian | A Learning-Style Theory for Understanding Autistic Behaviors[END_REF][START_REF] Pellicano | When the world becomes « too real »: a Bayesian explanation of autistic perception[END_REF][START_REF] Friston | On hyperpriors and hypopriors: comment on Pellicano and Burr[END_REF]. However the relatively small sample size in our study might be seen as a limitation and similar tests on bigger sample size should be made to confirm our results. This view could contribute to building specific clinical and educational intervention programs in order to provide ASD patients with specific cognitive strategies to overcome prediction weaknesses.

We propose that therapy should focus on training people with ASD to learn regularities in both social and non-social domains.

Figure Captions

Table 1 Means and standard deviations for age, sex, verbal IQ, performance IQ and AQ of ASD participants and typically developing participants.

Table 2 Summary of the three conditions (60 trials each) in each environment (social and nonsocial).

Table 3 Correlations between AQ and the extent to which all pooled subjects followed the cue.

Fig. 1 Cue-guided task: a cue was presented to subjects at the beginning of each trial, before they made their choice. In the non-social environment (1), the cue was an arrow presented for 800 ms.

In the social environment (2), a short movie depicting the choice of an actor was presented for 1500 ms before each trial. In the movie, the actor dynamically indicated his choice by looking at the chosen box, and the chosen box then lit up. Two boxes with hidden reward probabilities were then presented until the subject chose between them. A yellow frame indicated which box was chosen. Participants then received a feedback about their choice.

Fig. 2 Fig. 3

 23 Fig. 2 Percentage of successful trials (y-axis) as a function of AQ value (x-axis) in the familiarization task (a), in a non-social environment (b) and in a social one (c). Control subjects are depicted with black circles and ASD subjects with light-grey crosses Fig. 3 Percentage of ASD and control subjects achieving the success criterion in non-social and social environments (a) and in stable and unstable contexts (b). Asterisks indicate a significant difference between the two groups (P<0.001).

Fig. 4

 4 Fig.4Proportion of trials in which controls (black) and ASD subjects (light-grey) followed the cue (y-axis) as a function of cue reliability (x-axis) with standard deviations. Asterisks indicate a significant difference between ASD and control subjects (P<0.01).

Fig. 5

 5 Fig.5Representation of the R correlation coefficient (y-axis) between the proportion of trials in which subjects followed the cue and cue reliability, as a function of the AQ (x-axis), in the two pooled populations of control (black circles) and ASD (light-grey crosses) subjects.

  Figure2

Acknowledgments

We would like to thank all of the participants for making this research possible. This study was supported by a Scientific Research Council grant from the Vinatier Hospital Center; the work was performed within the framework of the LABEX CORTEX (ANR-11-LABX-0042) of Université de Lyon, within the program "Investissements d'Avenir" (ANR-11-IDEX-0007) operated by the French National Research Agency (ANR).

Controls