Jérôme Lohéac
email: jerome.loheac@irccyn.ec-nantes.fr

Enrique Zuazua
email: enrique.zuazua@uam.es

From averaged to simultaneous controllability

Keywords: AMS subject classification (MSC 2010): 49J55, 34H05, 49J15, 93C05, 93C73, 49N05 Controllability, parameter dependent system, averaged control, simultaneous control, penalization

come

Résumé

Nous considérons un système de contrôle linéaire de dimension finie dépendant de paramètres inconnus. L'objectif est de construire des contrôles indépendants des paramètres afin de contrôler le système en un sens optimal. Nous discutons la notion de contrôle moyenné, dont le but est de contrôler seulement la moyenne des états par rapport aux paramètres, ainsi que la notion de contrôle simultané, dont l'objectif est de contrôler pour chaque paramètre l'état du système associé à ce paramètre. Nous montrerons que ces deux notions peuvent être connectées par le biais d'un processus de pénalisation. Plus précisément, la propriété de contrôlabilité en moyenne est une relaxation de la propriété de contrôlabilité simultanée. Pour la notion de contrôlabilité en moyenne les écarts entre les états par rapport aux paramètres sont laissés libres tandis que ces derniers sont forcés pour la notions de contrôlabilité simultanée. Afin de relier le contrôle moyenné au contrôle simultané, ce seront ces écarts qui seront pénalisés. Cependant, ces deux notions de contrôle requirent différentes conditions sur les rangs des matrices déterminant la dynamique du système et le contrôle. Lorsque la condition de rang pour le contrôle simultané est satisfaite, nous montrerons que le contrôle simultané peut être obtenu à partir du contrôle moyenné, comme limite de ce processus de pénalisation.

Introduction

We consider a parameter dependent control system:

ẏζ = A ζ y ζ + B ζ u (t ∈ (0, T )) , (1.1a
)

y ζ (0) = y i ζ .
(1.1b)

In order to fix the notation, all along this paper, ζ ∈ Ω is a random parameter (the system's parameter) following a probability law µ, with (Ω, F, µ) a probability space (in particular, µ(Ω) = 1), X = R n is the state space and U = R m the control one. We assume that for every ζ ∈ Ω, A ζ ∈ L(X) and B ζ ∈ L(U, X). The control t → u(t) ∈ U is assumed to be independent of the parameter ζ whereas the state y ζ (t) = y ζ (t; u) ∈ X is time and parameter dependent. In addition, by Duhamel formula, y ζ can be represented as follows:

y ζ (t; u) = e tA ζ y i ζ + T 0 e (t-s)A ζ B ζ u(s) ds (ζ ∈ Ω , t 0 , u ∈ L 2 loc (R + , U )) . (1.2)
Let us also define the space:

L 2 (Ω, X; µ) = (y ζ ) ζ ∈ X Ω , Ω y ζ 2 X dµ ζ , (1.3)
which is an Hilbert space endowed with the scalar product:

y ζ , z ζ L 2 (Ω,X;µ) = Ω y ζ , z ζ X dµ ζ ((y ζ ) ζ , (z ζ ) ζ ∈ L 2 (Ω, X; µ)) .
(1.4)

In section 2 we introduce precise conditions on ζ → (A ζ , B ζ ) ensuring that for every t 0 and every u ∈ L 2 loc (R + , U ), (y ζ (t; u)) ζ ∈ L 2 (Ω, X; µ) whenever the parameter-dependent initial data satisfy (y i ζ ) ζ ∈ L 2 (Ω, X; µ). This paper is devoted to analyse the following controllability problems.

• Averaged controllability: The system is said to be averaged controllable in time T > 0 if, for every (y i ζ ) ζ ∈ L 2 (Ω, X; µ) and every y f ∈ X, there exists u ∈ L 2 ([0, T ], U ) such that:

Ω y ζ (T ; u) dµ ζ = y f , (1.5)
In other words, averaged controllability is the control of the expectation of the system's output. This notion is illustrated on Figure 1a.

• Exact simultaneous controllability: The system is said to be exactly simultaneously controllable in time T > 0 if, for every (y i ζ ) ζ , (y f ζ ) ζ ∈ L 2 (Ω, X; µ), there exists u ∈ L 2 ([0, T ], U ) such that:

y ζ (T ; u) = y f ζ (ζ ∈ Ω µ-a.e.) , (1.6)
This notion is illustrated on Figure 1b.

• Approximate simultaneous controllability: The system is said to be approximately simultaneously controllable in time T > 0 if, for every (y i ζ ) ζ , (y f ζ ) ζ ∈ L 2 (Ω, X; µ) and every ε > 0, there exists u ∈ L 2 ([0, T ], U ) such that:

Ω y ζ (T ; u) -y f ζ 2 X dµ ζ ε . (1.7)
This notion is illustrated on Figure 1c. 1. Even if the system (1.1) is controllable in average, this fact does not give any information on the variance of the outputs. 3. It is obvious that the exact simultaneous controllability property implies the averaged controllability and the approximate simultaneous controllability ones. In addition, one can find systems which are controllable in average (resp. approximatively simultaneously controllable) which are not exactly simultaneously controllable, see for instance Example 4.2 (resp. Example 3.1).

Moreover, the approximate simultaneous controllability property implies the averaged controllability one. In fact, the approximate simultaneous controllability property ensures that given T > 0, (

y i ζ ) ζ ∈ L 2 (Ω, X; µ), y f ∈ X and ε > 0, there exists u ε ∈ L 2 ([0, T ], U ) such that y ζ (T ; u ε ) -y f 2 L 2 (Ω,X;µ) ε.
But, by Cauchy-Schwarz inequality,

Ω y ζ (T ; u ε ) -y f dµ ζ 2 X Ω y ζ (T ; u ε ) -y f 2 X dµ ζ .
Thus, the system is approximatively controllable in average i.e. the linear and continuous map

Φ : u ∈ L 2 ([0, T ], U ) → Ω T 0 e (t-t)A ζ B ζ u(t) dtdµ ζ ∈ X has a dense image in X.
But since X is a finite dimensional vector space, we obtain Im Φ = X, i.e. the system is controllable in average.

A proof using probabilistic arguments can also be given. More precisely, approximate simultaneous controllability in time T > 0 means that for every (y f ζ ) ζ ∈ L 2 (Ω, X; µ) and every ε > 0, there exists

u ε ∈ L 2 ([0, T ], U ) such that y ζ (T ; u ε ) -y f ζ L 2 (Ω,X;µ)
ε, that is to say that the sequence of random variables ((y ζ (T ; u ε )) ζ ) ε converges in mean square to the random variable (y f ζ ) ζ as ε goes to 0. This convergence in mean square implies the convergence in law and hence the convergence of all finite momenta and, in particular,

lim ε→0 Ω y ζ (T ; u ε ) dµ ζ = Ω y f ζ dµ ζ .
In the spirit of the above paragraph, if there exists u ∈ L 2 ([0, T ], U ) such that the family of random variables ((y ζ (t; u)) ζ ) t∈[0,T ] converges in law to (y f ζ ) ζ as t goes to T , then the averaged controllability property holds. Let us mention that this convergence in law is a weaker notion than the exact simultaneous controllability one (corresponding to the mean square convergence).

When

Ω = {ζ 1 , • • • , ζ K } is of finite cardinal,
the simultaneous controllability is equivalent to the classical controllability one for the augmented system:

ẏ = Ay + Bu , with: y =    y ζ 1 . . . y ζ K    , A =    A ζ 1 0 . . . 0 A ζ K    and B =    B ζ 1 . . . B ζ K    .
And the controllability of this system is equivalent to the Kalman rank condition:

rank B AB • • • A Kdim X-1 B = K dim X .
5. In the previous item, we have seen that the simultaneous controllability property when the cardinal of Ω is finite can be interpreted in terms of a classical rank condition. But, when Ω is infinite, the output of the system is the function

ζ ∈ Ω → y ζ (T ) ∈ X, living in an infinite-dimensional space.
The first issue to be addressed is the choice of the norm in that space.

In the following, we choose the L 2 -norm. Accordingly, the fact that y ζ (T ) = y f ζ holds for almost every ζ ∈ Ω with respect to the measure µ is guaranteed by the fact that

Ω y ζ (T ) -y f ζ 2 X dµ ζ = 0.
This choice is natural, since in the particular case where

y f ζ = y f is independent of ζ and Ω y ζ (T ) dµ ζ = y f , the integral Ω y ζ (T ) -y f ζ 2
X dµ ζ is the variance of the system's output.

Thus, the L 2 -norm approach is natural from a probabilistic point of view but one could also use any L p (Ω, X; µ)-norm. In the next item, we mention some existing literature when considering the L ∞ -norm.

6. For parameter dependent systems, the notion of ensemble controllability is also commonly used (see for instance [START_REF] Li | Ensemble control of finite-dimensional time-varying linear systems[END_REF][START_REF] Li | Ensemble control of Bloch equations[END_REF][START_REF] Schirmer | Controllability of multi-partite quantum systems and selective excitation of quantum dots[END_REF][START_REF] Turinici | Optimally controlling the internal dynamics of a randomly oriented ensemble of molecules[END_REF]). A system is said to be ensemble controllable in time T > 0 if, for every ε and every y i ζ , y f ζ ∈ X, there exists u ∈ L 2 ([0, T ], U ) such that:

y ζ (T ; u) -y f ζ X ε (ζ ∈ Ω) . (1.8)
This notion of ensemble controllability, which does not seem to have a probabilistic interpretation, is similar to our notion of approximate simultaneous controllability above, where the L 2 (Ω, X; µ)-norm is replaced by the L ∞ (Ω, X) one.

In [START_REF] Helmke | Uniform ensemble controllability for one-parameter families of timeinvariant linear systems[END_REF], U. Helmke and M. Schönlein extended this notion to the one of L p -ensemble controllability, for p ∈ [1, ∞]. For p = 2, the L 2 -ensemble controllability corresponds to our approximate simultaneous controllability defined by (1.7). More precisely, the system (1.1) is said L p -ensemble controllable if for every ε > 0, there exists u : [0, T ] → R such that:

ζ → y ζ (T ; u) -y f ζ L p (Ω,X)
ε .

(1.9)

In [START_REF] Helmke | Uniform ensemble controllability for one-parameter families of timeinvariant linear systems[END_REF], with Ω = [ζ -, ζ + ] a compact subset of R and µ the Lebesgue measure, the authors give a necessary and sufficient conditions for the system (1.1) to be L p -ensemble controllable.

Controlling the average (or the expectation) of a parameter dependent system is not a new problem. It has been previously studied when a classical control system is perturbed by an additional drift (V. A. Ugrinovskii [START_REF] Ugrinovskii | Robust controllability of linear stochastic uncertain systems[END_REF], A. V. Savkin and I. R. Petersen [START_REF] Savkin | Uncertainty-averaging approach to output feedback optimal guaranteed cost control of uncertain systems[END_REF], I. R. Petersen [START_REF] Petersen | A notion of possible controllability for uncertain linear systems with structured uncertainty[END_REF]). We present here a different frame for which the uncertainty is inside the system itself, and not due to some external noise. Taking into account that we only know the probability distribution of the unknown parameter, it is natural to try to control the expectation of the output of the system.

In [START_REF] Zuazua | Averaged control[END_REF], it has been shown that the averaged controllability property is equivalent to a Kalman rank condition of infinite order. However, even if the average of the system is controlled, this fact does not ensure that the output of system is close to the desired target for any specific realisation of the parameter. Of course, the ideal situation arises when all the parameter dependent trajectories exactly reach the desired target. This corresponds, precisely, to the notion of simultaneous controllability.

Classically, the simultaneous exact controllability property corresponds, by duality, to the one of simultaneous exact observability (see § 3.2). However, when Ω is an infinite dimensional set, those properties are difficult to check in practice. This is why, in this article, we show that, if the simultaneous controllability property holds, then the approximate simultaneous control can be achieved from the averaged controls by means of a penalisation procedure and at the limit, when the penalizing parameter goes to ∞, we recover the simultaneous control.

The notion of simultaneous controllability was introduced by D. L. Russell [START_REF] Russell | The Dirichlet-Neumann boundary control problem associated with Maxwell's equations in a cylindrical region[END_REF] (see also J.-L. Lions [7, Chapter 5]) for partial differential equations. As mentioned above, when dealing with finite dimensional systems and when the parameter ranges over a finite set, the problem can be handled through classical rank conditions. However, the issue is much more complex when the parameter ranges over an infinite set.

The averaged controllability property has already been tackled by E. Zuazua et al [START_REF] Zuazua | Averaged control[END_REF][START_REF] Lazar | Averaged control and observation of parameter-depending wave equations[END_REF][START_REF] Lohéac | Averaged controllability of parameter dependent wave equations[END_REF] for some relevant PDE models. However, the link between the averaged and simultaneous controllability in that setting has not been yet developed. The tools developed here could be used to handle PDE and, in general, infinite-dimensional systems, but this requires further efforts.

In general, the simultaneous controllability problem is set in an infinite dimensional space (this holds when the cardinal of Ω is infinite). In infinite dimensional spaces the choice of the norm is important and an appropriate choice has to be done. According to the 5 th item of Remark 1.1, we chose the weighted L 2 -norm, that corresponds to the variance. More precisely, the simultaneous controllability property (1.6) holds if:

Ω y ζ (T ) -y f ζ 2 X dµ ζ = 0 .
(1.10)

Consequently, in section 4, we introduce the parametrized optimal control problems:

min J κ (u) = 1 2 T 0 u(t) 2 U dt + κ Ω y ζ (T ; u) -y f ζ 2 X dµ ζ Ω y ζ (T ; u) dµ ζ = Ω y f ζ dµ ζ (κ 0) ,
with y ζ the solution of (1.1) with control u and initial condition y i ζ . We will see in Theorem 4.1 that, at the limit κ → ∞, the minimum u κ is a control which minimizes the variance of the system's outputs. For instance, we will see that if the sequence (J κ (u κ )) κ is bounded then the sequence (u κ ) κ converges to a control u ∞ which solves the minimisation problem:

min 1 2 T 0 u(t) 2 U dt Ω y ζ (T ; u) -y f ζ 2 X dµ ζ = 0 .
In other words, u ∞ is the HUM control (the control obtained from the Hilbert Uniqueness Method) for the simultaneous control problem. More generally, the result of Theorem 4.1 can be summarized in Table 1, where we have defined This penalty argument is natural and has already been used in control theory. In J.-L. Lions [START_REF] Lions | Remarques sur la controlâbilite approchée[END_REF] it was used to achieve approximate controllability as the limit of a sequence of optimal control problems (see also L. A. Fernández and E. Zuazua [START_REF] Fernández | Approximate controllability for the semilinear heat equation involving gradient terms[END_REF] for semi-linear heat equations). This penalty method has also been used numerically, for the numerical approximation of null controls for parabolic problems (see F. Boyer [START_REF] Boyer | On the penalised hum approach and its applications to the numerical approximation of null-controls for parabolic problems[END_REF]). This paper is organized as follows.

(y ζ ) ζ ∈ L 2 (Ω, X; µ) as the minimizer of y ζ -y f ζ L 2 (Ω,X;µ) under the constraints Ω y ζ dµ ζ = Ω y f ζ dµ ζ and (y ζ ) ζ ∈ {y ζ (T ; u) , u ∈ L 2 ([0, T ], U )}. y ζ (T ; u κ ) -y f ζ L 2 (
In section 2, we give some conditions on A ζ , B ζ and y i ζ ensuring that the problem we are considering is well defined. Then, in section 3, we recall some known results about averaged controllability and we describe the duality approach for simultaneous controllability. In section 4, we present the penalty method and give some convergence results. More precisely, in this section we prove the main theorem (Theorem 4.1) of this article. Then, in section 5, we present some results for a further numerical development of the case where Ω is a countable set. Finally, in section 6, we conclude this work by some general remarks and open problems.

Admissibility conditions

In this section, we give some conditions ensuring that Ω y ζ (t) dµ ζ and Ω y ζ (t) 2 dµ ζ are well defined.

Let us consider the Hilbert space L 2 (Ω, X; µ) defined by (1.3). Using Cauchy-Schwarz together with

Ω dµ ζ = 1, leads to: Ω y ζ dµ ζ 2 X y ζ 2 L 2 (Ω,X;µ) ((y ζ ) ζ ∈ L 2 (Ω, X; µ)) .
Thus, in this paragraph, we only give conditions on A ζ , B ζ and µ such that y ζ (t) L 2 (Ω,X;µ) < ∞ and in all this article, we assume that initial and final condition are elements of L 2 (Ω, X; µ). By Duhamel formula, the solution

y ζ (t) = y ζ (t; u) of (1.1) is given by (1.2), i.e., y ζ (t; u) = e tA ζ y i ζ + t 0 e (t-s)A ζ B ζ u(s) ds (ζ ∈ Ω , t 0) . Lemma 2.1. Set (A ζ ) ζ∈Ω ∈ L(X) Ω .
For every T > 0 and every ζ ∈ Ω, there exists ς ζ (T ) > 0 such that:

e T A * ζ e T A ζ y X ς ζ (T ) y X (y ∈ X) .
Assume:

ς ζ (T ) < ∞ (ζ ∈ Ω µ -a.e.). (2.1)
Then for every T > 0, there exists ς(T

) > 0 (ς(T ) = sup ζ∈Ω ς ζ (T )) such that: e T A ζ y i ζ L 2 (Ω,X;µ) ς(T ) y i ζ L 2 (Ω,X;µ) ((y i ζ ) ζ ∈ L 2 (Ω, X; µ)) . (2.2)
Proof. The existence of ς ζ (T ) is clear. The result follows from Cauchy-Schwarz inequality.

Example 2.1.

If for every ζ ∈ Ω, A ζ is skew-adjoint, then (2.2) holds with ς(T ) = 1. (In this case, we have ς ζ (T ) = 1 for every ζ ∈ Ω.) Lemma 2.2. Set (A ζ ) ζ∈Ω ∈ L(X) Ω and (B ζ ) ζ∈Ω ∈ L(U, X) Ω . For every T > 0 and every ζ ∈ Ω, there exists a constant C ζ (T ) > 0 such that: T 0 e (T -t)A ζ B ζ u(t) dt 2 X C ζ (T ) u 2 L 2 ([0,T ],U ) .
Assume that:

Ω C ζ (T ) dµ ζ < ∞ . (2.3)
Then for every T > 0, there exists C(T ) > 0 such that:

Ω T 0 e (T -t)A ζ B ζ u(t) dt 2 X dµ ζ C(T ) u 2 L 2 ([0,T ],U ) (u ∈ L 2 ([0, T ], U )) .
Proof. The existence of C ζ (T ) > 0 independent of u is classical. The result follows from Minkowski and Cauchy-Schwarz inequalities.

Thus, if A ζ and B ζ satisfies the assumption of lemmas 2.1 and 2.2, then for every (

y i ζ ) ζ ∈ L 2 (Ω, X; µ), y(T ; u) defined by (1.2) is an element of L 2 (Ω, X; µ).
From these two lemmas, we can derive the following corollaries:

Corollary 2.1. Assume Card Ω < ∞ and set ζ ∈ Ω → (A ζ , B ζ ) ∈ L(X) × L(U, X), then for every (y i ζ ) ζ ∈ L 2 (Ω, X; µ), and every u ∈ L 2 loc (R + , U ), the solution y ζ (t; u) of (1.1) belongs to L 2 (Ω, X; µ) for every t 0. Corollary 2.2. Assume Ω ⊂ R d is a bounded set and assume the map ζ → (A ζ , B ζ ) is continuous on co(Ω), with co(Ω) the smallest convex set containing Ω.
Then for every

(y i ζ ) ζ ∈ L 2 (Ω, X; µ), every u ∈ L 2 loc (R + , U ) and every t 0, the solution y ζ (t; u) of (1.1) belongs to L 2 (Ω, X; µ).
Proof. Since X and U are finite dimensional spaces, for every ζ ∈ co(Ω),

ς ζ (T ) := sup y∈X , y X =1 e T A * ζ e T A ζ y X and C ζ (T ) := sup u∈L 2 ([0,T ],U ) , u L 2 ([0,T ],U ) =1 T 0 e (T -t)A ζ B ζ u(t) dt 2 X
are well defined for every ζ ∈ co(Ω) and every T 0. ∈ L 2 (Ω, X; µ). In addition, we have:

Moreover, since ζ ∈ co(Ω) → (A ζ , B ζ ) ∈ L(X) × L(U, X) is continuous, the map ζ ∈ co(Ω) → (ς ζ (T ), C ζ (T )) ∈ R 2 is
If Ω B ζ 2 L(U,X) dµ ζ < ∞, then for every (y i ζ ) ζ ∈ L 2 (Ω, X; µ) and every u ∈ L 2 loc (R + , U ), the solution y ζ (t; u) of (1.1) belongs to L 2 (Ω, X; µ) for every t 0.
t 0 e (t-s)A ζ B ζ u(s) ds X t 0 e (t-s)A ζ B ζ u(s) X ds = t 0 B ζ u(s) X ds √ t B ζ L(U,X) u L 2 ([0,t],U ) .
Thus the assumptions of Lemma 2.2 are fulfilled.

Duality approach and Kalman rank conditions

Here and in the sequel we assume that the hypotheses of lemmas 2.1 and 2.2 are satisfied.

State of the art for averaged controllability

Let us recall some known results on averaged controllability for finite dimensional systems. These results are taken from [START_REF] Zuazua | Averaged control[END_REF]. rank

Ω (A ζ ) j B ζ dµ ζ , j 0 = dim X . (3.1)
This result is based on duality arguments. More precisely, we introduce the (parameter dependent) adjoint system:

-żζ = A * ζ z ζ (t ∈ (0, T )) , (3.2a)
z ζ (T ) = z f . (3.2b)
Notice that even if this system depends of the parameter ζ the final condition z f is independent of ζ.

The next result makes the link between averaged controllability, and averaged observability and gives also a link between the adjoint system and the control of minimal L 2 -norm.

c(T ) z f 2 X T 0 Ω B * ζ z ζ (t) dµ ζ 2 U dt (z f ∈ X) , (3.3)
where c(T ) > 0 is a constant independent of z f . In addition, both conditions are equivalent to the rank condition (3.1). When these properties hold, the averaged control of minimal L 2 ([0, T ], U )-norm is given by:

u(t) = Ω B * ζ zζ (t) dµ ζ (t ∈ (0, T )) , (3.4)
where {z ζ } ζ is the solution of the adjoint system (3.2) corresponding to the datum z f ∈ X minimizing the functional:

J : X -→ R z f -→ 1 2 T 0 Ω B * ζ z ζ (t) dµ ζ 2 U dt -y f , z f X + Ω y i ζ , z ζ (0) X dµ ζ .
(3.5)

Observability inequality for exact simultaneous controllability

Let us define for every ζ ∈ Ω the adjoint system of (1.1):

-żζ = A * ζ z ζ (t ∈ (0, T )) , (3.6a) z ζ (T ) = z f ζ . (3.6b) If the system (1.1) is simultaneously controllable then for every (z f ζ ) ζ ∈ L 2 (Ω, X; µ), y ζ (T ) -y f ζ ζ , z f ζ ζ L 2 (Ω,X;µ) = 0 .
That is to say:

T 0 u(t), Ω B * ζ z ζ (t) dµ ζ U dt = y f ζ ζ , z f ζ ζ L 2 (Ω,X;µ) -y i ζ ζ , z ζ (0) ζ L 2 (Ω,X;µ) .
Let us then define the cost function J by:

J : L 2 (Ω, X; µ) -→ R z f ζ ζ -→ 1 2 T 0 Ω B * ζ z ζ (t) dµ ζ 2 U dt - Ω y f ζ , z f ζ X dµ ζ + Ω y i ζ , z ζ (0) X dµ ζ , (3.7)
where z ζ is the solution of (3.6).

The only difference between the cost functions defined by (3.5) for averaged controllability and (3.7) for simultaneous controllability is that, for simultaneous controllability, we allowed the final condition of the adjoint system to depend on the parameter ζ.

Assuming that J has a minimizer (ẑ ζ ) ζ ∈ L 2 (Ω, X; µ), we obtain, by computing the first variation of J,

û(t) = Ω B * ζ ẑζ (t) dµ ζ (t ∈ [0, T ] a.e.) . (3.8)
It is clear that J is convex. Thus, proving the existence of a minimizer z f ζ ζ ∈ L 2 (Ω, X; µ) for J is equivalent to showing that J is coercive, i.e. to the existence of a constant ĉ(T ) > 0 such that:

ĉ(T ) Ω z f ζ 2 X dµ ζ T 0 Ω B * ζ z ζ (t) dµ ζ 2 U dt ( z f ζ ζ ∈ L 2 (Ω, X; µ)) . (3.9)
where z ζ is the solution of (3.6) with final condition z f ζ . Summarizing this discussion, we end up with: Theorem 3.3. System (1.1) fulfills the exact simultaneous controllability property (1.10) if and only if the adjoint system (3.6) satisfies the exact simultaneous observability inequality (3.9).

When these properties hold, the exact simultaneous control of minimal norm is given by (3.8), where ẑζ is the solution of (3.6) with final condition ẑf ζ and

(ẑ f ζ ) ζ ∈ L 2 (Ω, X; µ) is the minimizer of J defined by (3.7).
Let us notice that very few systems have the property of simultaneous controllability. When card Ω is finite, the situation is clear since simultaneous controllability follows from a Kalman rank on an augmented system (see 4 th item of Remark 1.1). But when card Ω is infinite, the situation more complex and we have from [START_REF] Triggiani | Controllability and observability in banach space with bounded operators[END_REF]Theorem 3.3.1], Proposition 3.1. If L 2 (Ω, X; µ) is an infinite dimensional space, then the system (1.1) will never be exactly simultaneously controllable. According to Proposition 3.1, this system is not exactly simultaneously controllable, although the truncated system in which we consider

ζ ∈ {ζ 1 , • • • , ζ N } with the probability measure µ N given by µ N ({ζ}) = µ({ζ}) µ({ζ 1 ,••• ,ζ N }) , for ζ ∈ {ζ 1 , • • • , ζ N } is simultaneously controllable, whatever N ∈ N * is.
In fact, for this truncated system, the precise values of the measure µ N are not important since its simultaneous controllability can be understood though the augmented system:

d dt    y ζ 1 . . . y ζ N    =    -ζ 1 0 . . . 0 -ζ N       y ζ 1 . . . y ζ N    +    1 . . . 1    u .
The Kalman matrix of this system is:

   1 -ζ 1 . . . (-ζ 1 ) N -1 . . . . . . . . . 1 -ζ N . . . (-ζ N ) N -1    ,
which is a Vandermonde matrix of determinant

1 i<j N (ζ i -ζ j ) = 0.
Let us now assume that

n∈N * ζn =0 1 |ζ n |
< ∞ and µ n = 0 for every n ∈ N * . Under this assumption, we will prove that this system is approximatively simultaneously controllable in any time T > 0. In order to prove this result, we have to show the unique continuation property:

T 0 n∈N * e -ζn(T -t) z f n µ n 2 dt = 0 =⇒ ∀n ∈ N * , z f n = 0 .
This property directly follows from Müntz's theorem, see for instance [14, §12, p. 54].

Proposition 3.1 tells us that it is impossible to build dependent parameter systems which are exactly simultaneously controllable (unless dim L 2 (Ω, X; µ) < ∞). However, as we have seen, the averaged controllability property holds for a variety of models. Consequently, it is natural to look for averaged controls which are optimal in the sense that they minimize the output's variance. This is the core of section 4.

Momentum approach for simultaneous controllability

In § 3.2, we gave a necessary and sufficient condition, (3.9), for simultaneous controllability. However, even on simple problems, it is difficult to check whether this condition is satisfied or not. In this paragraph, we present an iterative approach to check whether the observability inequality (3.9) is fulfilled or not. The method presented here can also be seen as an alternative method to the one we proposed in the rest of this paper (see section 4) in order to link averaged controllability to exact simultaneous controllability.

To simplify the notation we define the operator E ∈ L(L 2 (Ω, X; µ), X) by:

E(y ζ ) ζ = Ω y ζ dµ ζ ((y ζ ) ζ ∈ L 2 (Ω, X; µ)) . (3.10)
Notice that we have

E * z = (z) ζ and E E * = Id X .
Let us first remind that proving the averaged controllability property is equivalent to proving that the cost function J defined by (3.5) is coercive and proving the exact simultaneous controllability is equivalent to proving that the cost function J defined by (3.7) is coercive. In addition, we have also noticed that we have J = J • E * , where E is given by (3.10). Thus, proving that J is coercive means proving that the restriction of J to the subset

E * (X) = {ζ ∈ Ω → y ∈ X , y ∈ X} of L 2 (Ω, X; µ) is coercive.
Let us also notice that since L 2 (Ω, R; µ) is an Hilbert space, one can define an orthonormal basis (ϕ i ) i∈I (with the convention 0 ∈ I and ϕ 0 = 1) of this space. Based on the above construction of E, we define for every i ∈ I, the operator E i ∈ L(L 2 (Ω, X; µ), X) by:

E i (y ζ ) ζ = Ω y ζ ϕ i (ζ) dµ ζ ((y ζ ) ζ ∈ L 2 (Ω, X; µ)) , (3.11)
so that L 2 (Ω, X; µ) = i∈I E * i (X).
Let us assume that L 2 (Ω, R; µ) is a separable Hilbert space, that is to say that we can choose

I = N (if L 2 (Ω, R; µ) is of infinite dimension) or I = {0, • • • , d} ⊂ N (if L 2 (Ω, R; µ) is of dimension d).
For every k ∈ N, we define the finite dimensional subspaces V k of L 2 (Ω, X; µ) by:

V k = k i∈I i k E * i (X) ⊂ L 2 (Ω, X; µ) . (3.12)
Let us also define the constant ĉk (T ) 0 by:

ĉk (T ) = inf (z f ζ ) ζ ∈V k \{0} T 0 Ω B * ζ z ζ (t) dµ ζ 2 U dt z f ζ 2 L 2 (Ω,X:µ) (k ∈ N) , (3.13)
that is to say:

ĉk (T ) Ω z f ζ 2 X dµ ζ T 0 Ω B * ζ z ζ (t) dµ ζ 2 U dt (k ∈ N z f ζ ζ ∈ V k ) , (3.14)
with z ζ (t) the solution of the adjoint problem (3.6) with final condition

z ζ (T ) = z f ζ ∈ V k . Thus, if ĉk (T ) > 0, J is convex and coercive on V k .
Since V k ⊂ V k+1 , the sequence (ĉ k (T )) k∈N is decreasing. In addition, one can easily convince that if lim k→∞ ĉk (T ) > 0, there exists ĉ(T ) > 0 (ĉ(T ) = lim k→∞ ĉk (T )) such that:

ĉ(T ) Ω z f ζ 2 X dµ ζ T 0 Ω B * ζ z ζ (t) dµ ζ 2 U dt ( z f ζ ζ ∈ L 2 (Ω, X; µ)) .
That is to say that J is convex and coercive on L 2 (Ω, X; µ) and hence we have exact simultaneous controllability. Moreover, as k goes to infinity, the minimizing family

(z f k,ζ ) ζ ∈ V k of the restriction of J to the finite dimensional subspace V k of L 2 (Ω, X; µ), converges to a minimizing family (ẑ ζ ) ζ ∈ L 2 (Ω, X; µ) of J on L 2 (Ω, X; µ).
Summarizing the above discussion, leads to the following:

Remark 3.1. Assume that L 2 (Ω, X; µ) is a separable Hilbert space.

1. If the system (1.1) is approximatively simultaneously controllable, then ĉk (T ) = 0 for every k ∈ N.

If lim

k→∞ ĉk > 0, then the system (1.1) is exactly simultaneously controllable. More precisely,

z f ζ ζ ∈ V k means there exists (z j,f ) j=0,••• ,k ∈ X k+1 such that z f ζ = k j=0 ϕ j (ζ)z j,f
. Let us then denote by z j ζ the solution of (3.2) with final condition z j,f . Due to linearity, the solution z ζ of (3.6) with final condition z f ζ is:

z ζ (t) = k j=0 ϕ j (ζ)z j ζ (t) = k j=0 ϕ j (ζ)e (T -t)A * ζ z j,f .
Finally, since we are in a finite dimensional space the coercive property (3.14) is equivalent to the uniqueness property:

Ω B * ζ k j=0 ϕ j (ζ)e (T -t)A * ζ z j,f dµ ζ = 0 (t ∈ [0, T ] a.e.) =⇒ z j,f = 0 (j ∈ {0, • • • , k}) .
We conclude by time analyticity that (3.14) holds if and only if:

rank Ω ( Âζ ) l Bζ dµ ζ , l ∈ N = (k + 1) dim X ,
where we have defined:

Âζ =    A ζ 0 . . . 0 A ζ    ∈ L(X k+1 ) and Bζ =    ϕ 0 (ζ)B ζ . . . ϕ k (ζ)B ζ    ∈ L(U, X k+1 ) (ζ ∈ Ω) .

When

Card Ω < ∞, the moments are solution of an ordinary differential equation. More precisely, consider Ω = {1, • • • , K} with measure µ given by µ({k}) = θ k with θ k ∈ (0, 1) and K k=1 θ k = 1. Let us consider an orthonormal basis {ϕ 0 , • • • , ϕ K-1 } of L 2 (Ω, R; µ) (with the convention, ϕ 0 (k) = 1). Then the i th -momentum is:

Y i = K k=1 θ k ϕ i (k)y k = M i I y (i ∈ {0, • • • , K -1}) ,
with:

y =    y 1 . . . y K    ∈ X K , M i = ϕ i (1) θ 1 Id X • • • ϕ i (K) θ K Id X ∈ L(X K , X)
and

I =    √ θ 1 Id X 0 . . . 0 √ θ K Id X    ∈ L(X K ) .
Thus, setting:

M =    M 0 . . . M K-1    ∈ L(X K ) , A =    A 1 0 . . . 0 A K    ∈ L(X K ) and B =    B 1 . . . B K    ∈ L(U, X K ) , the momentums Y =    Y 0 . . . Y K-1    satisfies (noticing that MM = Id X K ): Ẏ = MIAI -1 M Y + MIBu . (3.15)
Controlling the first k momentums of (y k ) k means controlling the first k dim X components of Y , solution of (3.15).

Since the basis ϕ

0 , ϕ 1 , • • • , ϕ K-1 is free (except ϕ 0 = 1)

A penalty method linking averaged and simultaneous controllability

As in all this paper, we assume in this section that the assumptions of lemmas 2.1 and 2.2 are satisfied.

In this section, we will present our strategy to link averaged controllability to exact simultaneous controllability. First of all, solving the averaged control problem, can be done with the Hilbert Uniqueness Method, that is to say minimize the L 2 -norm of the control with the constraint

Ω y ζ (T ) dµ ζ = Ω y f
ζ dµ ζ . Thus, using Euler-Lagrange formulation (or directly Theorem 3.2), one can see that the averaged control of minimal L 2 -norm is given by (3.4).

In order to reduce the output's variance, one can think to penalise the cost function J 0 (given by

J 0 (u) = 1 2 u 2 L 2 ([0,T ],U ) with the output's variance, Ω y ζ (T ) -y f ζ 2
X dµ ζ . Thus, we introduce the penalized optimization problem:

min J κ (u) := 1 2 u(t) 2 L 2 ([0,T ],U ) + κ y ζ (T ; u) -y f ζ 2 L 2 (Ω,X;µ) E y ζ (T ; u) -y f ζ = 0 . (κ 0) , (4.1)
where in the above, y ζ is the solution of (1.1) defined by (1.2) with control u, L 2 (Ω, X; µ) is the Hilbert space introduced in (1.3) and E is the expectation defined by (3.10).

Let us give an existence result.

Proposition 4.1. If system (1.1) satisfies the averaged controllability property (1.5) then for every T > 0,

(y i ζ ) ζ , (y f ζ ) ζ ∈ L 2
(Ω, X; µ) and κ 0, the minimisation problem (4.1) admits one and only one solution

u κ ∈ L 2 ([0, T ], U ).
In addition, the optimal control u κ satisfies:

u κ (t) = Ω B ζ z ζ (t) dµ ζ (t ∈ [0, T ]) , (4.2a)
where, z ζ is solution of:

żζ = -A * ζ z ζ , z ζ (T ) = z + y f ζ -y ζ (T ; u κ ) , (4.2b)
with z ∈ X unknown.

Proof. For every κ 0, it is clear that J κ is convex. Since we have assumed that the system (1.1) satisfies the averaged controllability property (1.5), this ensure that the set:

u ∈ L 2 ([0, T ], U ) , E y ζ (T ; u) -y f ζ = 0
is non empty and in addition, this set is a convex and closed set of L 2 ([0, T ], U ). Moreover, the averaged controllability property ensure that J 0 is coercive on this set and consequently J κ is also coercive on this set. Thus, there exists a unique minimizer u κ ∈ L 2 ([0, T ], U ) for the minimisation problem (4.1).

Let us now prove the optimality conditions. Let us define the Lagrangian of the system:

L(u, z) = J κ (u) + z, E y ζ (T ; u) -y f ζ X (u ∈ L 2 ([0, T ], U ) , z ∈ X) .
The optimality conditions are:

∂ z L = 0 and ∂ u L = 0 . But we have, ∂ u L(u, z) = u + Ω B * ζ e (T -t)A * ζ z + y ζ (T ; u) -y f ζ dµ ζ .
That is to say that, the optimal control u κ should satisfy (4.2).

• If u κ L 2 ([0,T ],U ) κ 0 is bounded, then (u κ ) κ converges to a control which steers exactly y i ζ to y ζ and realises the minimum of:

min 1 2 u 2 L 2 ([0,T ],U ) y ζ (T ; u) -y ζ L 2 (Ω,X;µ) = 0 . (4.5)
• If u κ L 2 ([0,T ],U ) κ 0 is unbounded, then y i ζ can be approximatively steered to y ζ .

In addition, if lim

κ→∞ y ζ (T ; u κ ) -y f ζ L 2 (Ω,X;µ) = 0, then we have y ζ = y f ζ .
Proof. Without loss of generality, we can assume that y

i ζ = 0. Let us first notice that (y ζ ) ζ ∈ L 2 (Ω, X; µ) is well defined. In fact, (y ζ ) ζ is the orthogonal projection of (y f ζ ) ζ in L 2 (Ω, X; µ) on the closed vector space {y ζ (T ; u) , u ∈ L 2 ([0, T ], U )} ∩ (y ζ ) ζ , E(y ζ ) ζ = E(y f ζ ) ζ .
• Let us assume u κ L 2 ([0,T ],U ) κ 0 bounded. From Lemma 4.1, the sequence u κ L 2 ([0,T ],U ) κ 0 is increasing, hence there exists u ∞ ∈ L 2 ([0, T ], U ) such that up to a subsequence, (u κ ) κ 0 is weakly convergent to u ∞ and in addition, we have:

u ∞ L 2 ([0,T ],U ) lim κ→∞ u κ L 2 ([0,T ],U ) . Since (u κ ) κ 0 is weakly convergent to u ∞ , it is easy to obtain that ((y ζ (T ; u κ )) ζ ) κ 0 is weakly convergent to (y ζ (T ; u ∞ )) ζ ∈ L 2 (Ω, X; µ). Hence, E y ζ (T ; u ∞ ) ζ = E y f ζ ζ and y ζ (T ; u ∞ ) -y f ζ L 2 (Ω,X;µ) lim inf κ→∞ y ζ (T ; u κ ) -y f ζ L 2 (Ω,X;µ) .
In addition, from Lemma 4.1, the sequence

y ζ (T ; u κ ) -y f ζ L 2 (Ω,X;µ) κ
is decreasing thus, we have:

y ζ (T ; u ∞ ) -y f ζ L 2 (Ω,X;µ) y ζ (T ; u κ ) -y f ζ L 2 (Ω,X;µ) (κ > 0)
and hence, from relation (4.3) of Lemma 4.1, we obtain

u ∞ L 2 ([0,T ],U ) u κ L 2 ([0,T ],U ) that is to say, u ∞ L 2 ([0,T ],U ) = lim κ→∞ u κ L 2 ([0,T ],U ) and (up to a subsequence, (u κ ) κ is strongly convergent to u ∞ in L 2 ([0, T ], U ). Consequently, (y ζ (T ; u κ )) ζ κ is strongly convergent to (y ζ (T ; u ∞ )) ζ in L 2 (Ω, X; µ).
Let us now prove that y ζ (T, u ∞ ) = y ζ . Assume by contradiction that it is not the case. That is to say there exists ū ∈ L 2 ([0, T ], U ) such that:

E y ζ (T ; ū) ζ = E y f ζ ) ζ and y ζ (T ; ū) -y f ζ L 2 (Ω,X;µ) < y ζ (T, u ∞ ) -y f ζ L 2 (Ω,X;µ) .
But, we have J κ (u κ ) J κ (ū) for every κ 0, i.e.:

1 2κ ū 2 L 2 ([0,T ],U ) -u κ L 2 ([0,T ],U ) y ζ (T ; u κ ) -y f ζ 2 L 2 (Ω,X;µ) -y ζ (T ; ū) -y f ζ 2 L 2 (Ω,X;µ) (κ > 0) .
Taking the limit κ → ∞ comes the contradiction:

y ζ (T ; ū) -y f ζ 2 L 2 (Ω,X;µ) y ζ (T ; u ∞ ) -y f ζ 2 L 2 (Ω,X;µ) .
Let us now show that (u κ ) κ is convergent to u ∞ (that is to say that we do not need the subsequence extraction procedure). This follows form the fact that any convergent subsequence of (u κ ) κ is convergent to a minimizer of (4.5). But it is trivial that the minimization problem (4.5) has at most one minimizer.

Finally, it remains clear that lim

κ→∞ y ζ (T ; u κ ) -y f ζ L 2 (Ω,X;µ) = 0 is equivalent to y ζ = y f ζ .
• Let us assume that u κ L 2 ([0,T ],U ) κ 0 is not bounded. The results of this point are direct consequences of (4.3) given in Lemma 4.1.

If the system (1.1) is simultaneously controllable, the convergence rates to the simultaneous control and the one of the variance to 0 are linked. Then, (u κ ) κ 0 is strongly convergent to u ∞ and, in addition,

y ζ (T, u κ ) -y f ζ 2 L 2 (Ω,X;µ) u ∞ L 2 ([0,T ],U ) κ -1 u κ -u ∞ L 2 ([0,T ],U ) . (4.6)
Proof. First of all, (u κ ) κ is strongly convergent to u ∞ is a consequence of Theorem 4.1.

Let us now prove (4.6). First of all, changing y f ζ in y f -e T A ζ y i ζ , we can assume without loss of generality that y

i ζ = 0. Set u κ = u ∞ + v κ , then v κ is a minimizer of: min G κ (v) = 1 2 v 2 L 2 ([0,T ],U ) + v, u ∞ L 2 ([0,T ],U ) + κ Ω T 0 e (T -t)A ζ B ζ v(t) dt 2 X dµ ζ E T 0 e (T -t)A ζ B ζ v(t) dt = 0 .
We have:

G κ (v κ ) G κ (0) = 0 .
Thus, for every κ 0,

κ Ω T 0 e (T -t)A ζ B ζ v κ (t) dt 2 dµ ζ 1 2 v κ 2 L 2 ([0,T ],U ) + κ Ω T 0 e (T -t)A ζ B ζ v κ (t) dt 2 dµ ζ -v κ , u ∞ L 2 ([0,T ],U ) v κ L 2 ([0,T ],U ) u ∞ L 2 ([0,T ],U ) .
This ends the proof using

T 0 e (T -t)A ζ B ζ v κ (t) dt = y ζ (T ; u κ ) -y ζ (t; u ∞ ) = y ζ (T ; u κ ) -y f ζ .
Let us now give the consequences of Theorem 4.1 in the case where the cardinal of Ω is finite.

Corollary 4.1. Assume L 2 (Ω, X; µ) is of finite dimension.
Then the sequence of minimizers (û κ ) κ of the optimisation problem (4.1) is strongly convergent to an element û∞ ∈ L 2 ([0, T ], U ) satisfying the minimisation problem:

min 1 2 u 2 L 2 ([0,T ],U ) y ζ (T ) = y ζ (ζ ∈ Ω µ -a.e.) ,
where y ζ , defined by Theorem 4.1, is the minimizer of (4.4).

A graphical interpretation of this result is given on Figure 2. Proof. Let us use the notations introduced in Theorem 4. 2 . The parameter dependent system under consideration is:

Eyζ = Ey f ζ {yζ(T ; u) , u ∈ L 2 ([0, T ], U )} y f ζ y ζ
1. Since L 2 (Ω, X; µ) is a finite dimensional space, y ζ (T ; u) , u ∈ L 2 ([0, T ], U ) ∩ (y ζ ) ζ , E(y ζ ) ζ = E(y f ζ ) ζ is a closed affine subspace of L 2 (Ω, X; µ). Consequently, there exists u ∈ L 2 ([0, T ], U ) such that y ζ = y ζ (T ; u ).
ẏζ = ζAy ζ + Bu y ζ (0) = y i (ζ ∈ {1, 2}) , with A = 0 -1 1 0 , B = 1 0 and y i = 1 1 .
We fix the final target y f to (0 , 0) and the final time T to 1. The corresponding augmented system is:

ẏ = Ay + Bu y(0) = y i , with A = A 0 0 2A =     0 -1 0 0 1 0 0 0 0 0 0 -2 0 0 2 0     , B = B B =     1 0 1 0     and y i = y i y i =     1 1 1 1     .
Using the Kalman rank condition, it is easy to see that this augmented system is controllable (in the classical sense) and controlling the average means controlling 1 2 (y 1 + y 3 , y 2 + y 4 ) . On figures 3, 4 and 5, we plot the numerical results dealing with the averaged control, the exact simultaneous control and the solution of the penalisation problem, when letting the parameter κ growing. Example 4.2. This example illustrates the result of Corollary 4.1 when there is no simultaneous controllability. For this example, we consider again the probability space Ω = {1, 2} and the probability density µ given by µ({1}) = µ({2}) = 1 2 . The parameter dependent system under consideration is:

ẏζ = A ζ y ζ + Bu y ζ (0) = y i , with B = 1 0 , y i = 1 1 and A ζ =              0 -1 1 0 if ζ = 1 , 1 0 0 1 if ζ = 2 .
Using the Kalman rank condition, introduced by E. Zuazua (see Theorem 3.1), one can see that this system is controllable in average. On the other hand, the simultaneous controllability of this system reduce to prove the classical controllability of the augmented system:

ẏ = Ay + Bu ,
with A = A 1 0 0 A 2 =     0 -1 0 0 1 0 0 0 0 0 1 0 0 0 0 1     and B = B B =     1 0 1 0     .
One can easily see that rank B, AB, A 2 B, A 3 B = 3 < 4 and hence, the Kalman rank condition is not satisfied. On figures 6, 7 and 8, we present the numerical results for this system. As in Example 4.1, the final time T is set to 1 and the target y f is (0 , 0) .

Numerical realisation when Card Ω is infinite

In this section we will study the discrete event case (Ω = N * ).

For this case, we consider the probability space (N * , P(N * ), µ). A natural way to deal with this problem is to truncate it. More precisely, instead of considering the probability space (N * , P(N * ), µ), we consider the probability space (N * , P(N * ), χ Z µ) with the measure χ Z µ given by

χ Z µ({ζ}) =    µ({ζ}) µ({1, • • • , Z}) if ζ Z , 0 otherwise, (Z ∈ N * , ζ ∈ N * ) , (5.1)
for Z ∈ N * large enough so that µ({1, • • • , Z}) > 0.

Since our penalisation procedure needs the system ẏζ = A ζ y ζ + B ζ u to be controllable in average the first question we should answer is whether this averaged controllability property is stable or not through the truncation procedure.

Proposition 5.1. Assume the system (1.1) is controllable in average for the measure µ.

Then there exists Z 0 ∈ N * such that for every Z Z 0 , this system is controllable in average for the measure χ Z µ given by (5.1).

Let us also notice that this truncation procedure does not affect the simultaneous controllability property for Z large enough. Proposition 5.2. Assume the system (1.1) is approximatively simultaneously controllable for the measure µ.

Then for every Z ∈ N * such that µ ({1, • • • , Z}) > 0, this system is simultaneously controllable for the measure χ Z µ given by (5.1).

Remark 5.1. Notice that by truncation, one can lose the averaged controllability property. This is for instance the case of the system considered in Example 4.2.

In opposition the simultaneous controllability property cannot be lost by truncation. This is natural since if the system in simultaneously controllable, all the events y 1 , • • • , y Z can be exactly controlled. Of course, as Example 3.1 shows, the reverse property does not hold.

Consequently, if a system is simultaneously controllable, then it is controllable in average and each of its truncation is controllable in average. The reverse property does not hold, i.e. there exist systems which are not controllable in average but so that each of its truncations is controllable in average. An example of such system is based on the following construction. Let A ∈ L(X) and B ∈ L(U, X) and assume that the pair (A, B) is controllable, that is to say that rank B, AB, • • • , A dim X-1 B = dim X. Let us now consider the probability set (N * , P(N * ), µ) with µ({1}) = 1 2 and for every ζ ∈ N * , µ({ζ}) > 0 and define the parameter dependent system:

ẏζ = A ζ y ζ + B ζ u (ζ ∈ N * ) , with A ζ = A for every ζ ∈ N * , B 1 = B and B ζ = -B for every ζ ∈ N * \ {1}.
Then, for every Z ∈ N * , the truncated system is controllable in average. Indeed, we have:

rank   Z ζ=1 (A ζ ) j B ζ µ({ζ}) µ({1, • • • , Z}) , j ∈ N   = rank   1 2 A j B -   Z ζ=2 µ({ζ})   A j B , j ∈ N   = rank µ(N * \ {1, • • • , Z})A j B , j ∈ N = rank A j B , j ∈ N = dim X .
But, the full system is not controllable in average. Indeed, for every z f ∈ R n and T > 0, we have:

∞ ζ=1 B * ζ e (T -t)A * ζ z f µ({ζ}) = 1 2 B * e (T -t)A * z f -   ∞ ζ=2 µ({ζ})   B * e (T -t)A * z f = 0 and hence, whatever T > 0 is, T 0 ∞ ζ=1 B * ζ e (T -t)A * ζ z f µ({ζ}) 2 U dt = 0 does not imply z f = 0.
Proof of Proposition 5.1. Set θ ζ = µ({ζ}) without loss of generality, we can assume that θ ζ > 0 for every

ζ ∈ N * . Set θ Z ζ = χ Z µ({ζ}) =      θ ζ Z ζ=1 θ ζ if ζ Z , 0 otherwise. Let us remind that due to Theorem 3.2, the pairs (A ζ , B ζ ) ζ being controllable in average, is equivalent to (3.3): c z f 2 X T 0 ζ∈N * B * ζ e tA * ζ z f θ ζ 2 U dt (z f ∈ X) . with c = c(T ) > 0 independent of z f . But,   T 0 Z ζ=1 B * ζ e tA * ζ z f θ Z ζ 2 U dt   1 2 = 1 Z ζ=1 θ ζ   T 0 Z ζ=1 B * ζ e tA * ζ z f θ ζ 2 U dt   1 2
and using Minkowski inequality,

  Z ζ=1 θ ζ     T 0 Z ζ=1 B * ζ e tA * ζ z f θ Z ζ 2 U dt   1 2   T 0 ∞ ζ=1 B * ζ e tA * ζ z f θ ζ 2 U dt   1 2 -   T 0 ∞ ζ=Z+1 B * ζ e tA * ζ z f θ ζ 2 U dt   1 2
.

From the averaged controllability property, there exists c > 0 such that:

c z f 2 X T 0 ∞ ζ=1 B * ζ e tA * ζ z f θ ζ 2 U dt
and due to the admissibility condition, there exists C > 0 such that:

T 0 ∞ ζ=Z+1 B * ζ e tA * ζ z f θ ζ 2 U dt C z f 2 X ∞ ζ=Z+1 θ ζ . Consequently,   T 0 Z ζ=1 B * ζ e tA * ζ z f θ Z ζ 2 U dt   1 2 √ c -C 1 -Z ζ=1 θ ζ Z ζ=1 θ ζ z f X . Since lim Z→∞ √ c -C 1 -Z ζ=1 θ ζ Z ζ=1 θ ζ = √ c > 0,
Z ζ = χ Z µ({ζ}) =      θ ζ Z ζ=1 θ ζ if ζ Z , 0 otherwise.
Due to that approximate simultaneous controllability, for every ε > 0 and every (y

i ζ ) ζ , (y f ζ ) ζ ∈ L 2 (N * , X; µ), there exist a control u ∈ L 2 ([0, T ], U ) such that: ζ∈N * y ζ (T ; u) -y f ζ 2 X θ ζ ε . But we have for every Z ∈ N * , ζ∈N * y ζ (T ; u)-y f ζ 2 X θ Z ζ = Z ζ=1 y ζ (T ; u)-y f ζ 2 X θ Z ζ 1 µ({1, • • • , Z} ζ∈N * y ζ (T ; u)-y f ζ 2 X θ ζ ε µ({1, • • • , Z} . That is to say that u ∈ L 2 ([0, T ], U ) → (y ζ (T ; u)) ζ ∈ L 2 (N * , X; χ Z µ) has a dense image in L 2 (N * , X; χ Z µ).
But its image is an affine space and L 2 (N * , X; χ Z µ) is a finite dimensional space. Thus, its image is L 2 (N * , X; χ Z µ) and the truncated system is exactly simultaneously controllable.

Let us finally study the error between the initial minimisation problem:

min J ∞ κ (u) := 1 2 T 0 u(t) 2 U dt + κ ∞ ζ=1 y ζ (T ; u) -y f ζ 2 X µ({ζ}) ∞ ζ=1 y ζ (T ; u) -y f ζ µ({ζ}) = 0 (κ 0) (5.2)
and the truncated minimisation problem:

min J Z κ (u) := 1 2 T 0 u(t) 2 U dt + κ Z ζ=1 y ζ (T ; u) -y f ζ 2 X χ Z µ({ζ}) Z ζ=1 y ζ (T ; u) -y f ζ χ Z µ({ζ}) = 0 (κ 0 , Z Z 0 ) , (5.3)
with Z 0 ∈ N * given by Proposition 5.1.

Proposition 5.3. Assume that the system (1.1) is controllable in average for the probability measure µ. Set κ 0. For Z Z 0 (with Z 0 ∈ N * given by Proposition 5.1), let u Z κ be the minimizer of the truncated minimisation problem (5.3).

Then, as Z → ∞, the sequence (u Z κ ) Z strongly converges in L 2 ([0, T ], U ) to the minimizer u κ of the initial minimisation problem (5.2).

Proof. Without loss of generality, we can assume that µ({ζ}) > 0 for every ζ ∈ N * and for convenience,

we set µ({ζ}) = θ ζ = θ ∞ ζ and as previously, θ Z ζ = χ Z µ({ζ}) =      θ ζ Z ζ=1 θ ζ if ζ Z , 0
otherwise. Without loss of generality, we can also assume that for every Z ∈ N * , the system (1.1) in controllable in average for the probability measure χ Z µ.

Let us introduce for every Z ∈ N * ∪ {∞} the map I Z : L 2 ([0, T ], U ) → {0, ∞} defined by:

I Z (u) =        0 if Z ζ=1 y ζ (T ; u) -y f ζ θ Z ζ = 0 , ∞ otherwise.
Thus minimizing J Z κ under the constraint • Lower bound: Set u ∞ ∈ L 2 ([0, T ], U ). The aim is to prove that there exists a sequence (u Z ) Z∈N * strongly convergent to u ∞ such that:

J ∞ κ (u ∞ ) + I ∞ (u ∞ ) lim sup Z→∞ J Z κ (u ∞ ) + I Z (u ∞ ) .
If I ∞ (u ∞ ) = ∞ then this result can be easily obtained with u Z = u ∞ .

Let us now assume that I ∞ (u ∞ ) = 0. From the previous point, it remains clear that if the sequence (u Z ) Z is converging to u ∞ and if for every Z ∈ N * , I Z (u Z ) = 0 then:

J ∞ κ (u ∞ ) = lim Z→∞ J Z κ (u Z ) .
Thus we only need to prove that such a sequence (u Z ) Z exists.

Let us write u Z = u ∞ + v Z . Then I Z (u Z ) = 0 means:

Z ζ=1 T 0 e (T -t)A ζ B ζ v Z (t) dt θ Z ζ = - Z ζ=1 y ζ (T ; u ∞ ) -y f ζ θ Z ζ .
Since we have assumed that the system (1.1) is controllable in average, such a v Z exists and in addition, there exists a constant C > 0 independent of v Z such that:

v Z 2 L 2 ([0,T ],U ) C Z ζ=1 y ζ (T ; u ∞ ) -y f ζ θ Z ζ 2 X .
• The problem of convergence rates both for variances and controls as κ → ∞ is open. Such results would be helpful in order to validate numerical simulations, since from a computational viewpoint, it is hard to determine what the decay or convergence rate is or even if the limit vanishes or not.

• When the probability space Ω is of infinite cardinal, we have introduced a truncation parameter Z.

In that case, we have to parameters (Z and κ) going to infinity. Propositions 5.3 and 5.4 show that the limits in κ and Z commute. But, in practice, it would be interesting to be in condition to bound the analysis and simulations to deal with a single parameter. To this end, we should establish some explicit relation between both of them, for instance, find a function Z → κ(Z) such that when letting Z → ∞, the correct asymptotic behavior in ensured. This problem is related to the one of convergence rates mentioned in the previous item.

• Similar results as those in section 5 could be obtained with a continuous measure and under Lipschitz-regularity assumptions on ζ → (A ζ , B ζ ). In this situation, instead of truncating the system, one could use the approximation of Lipschitz functions by piecewise constant functions.

• Finally, the penalization procedure proposed here could be extended in the PDE context.

Figure 1 :

 1 Figure 1: Different controllability notions, introduced in (1.5), (1.6) and (1.7), for parameter dependent systems, with initial condition and target independent of ζ.

2 .

 2 There is no natural ordinary differential equation describing the average Y (t) = Ω y ζ (t) dµ ζ , except when A ζ is independent of ζ for which we have: Ẏ = AY + Ω B ζ dµ ζ u. In this particular case, the averaged controllability property is equivalent to the controllability of the pair (A, Ω B ζ dµ ζ ).

Proof.

  According to Lemma 2.1 and Example 2.1, we have e tA ζ y i ζ ζ

Theorem 3 . 1 (

 31 [START_REF] Zuazua | Averaged control[END_REF] Theorem 1). System (1.1) fulfills the averaged controllability property (1.5) if and only if the following rank condition is satisfied:

Theorem 3 . 2 (

 32 [START_REF] Zuazua | Averaged control[END_REF] Theorem 2). System (1.1) fulfills the averaged controllability property (1.5) if and only if the adjoint system (3.2) satisfies the averaged observability inequality:

Example 3 . 1 .

 31 Let (µ n ) n∈N * be a nonnegative sequence of real numbers and (ζ n ) n∈N * of real numbers and assume that n∈N * µ n = 1 and ζ n = ζ m for every n = m. Let us then define Ω = {ζ n , n ∈ N * } and the probability space (Ω, P(Ω), µ), with the measure µ defined by µ({ζ n }) = µ n and consider the system ẏζ = -ζy ζ + u (ζ ∈ Ω) .

3 . 4 .

 34 According to Proposition 3.1, unless dim L 2 (Ω, X; µ) < ∞, we have lim k→∞ ĉk = 0. Let us mention that the property (3.14) corresponds to a Kalman rank condition.

Proposition 4 . 2 .

 42 Assume system (1.1) is exactly simultaneously controllable in time T > 0. Let u ∞ ∈ L 2 ([0, T ], U ) be the exact simultaneous control of minimal norm steering y i ζ to y f ζ and let u κ ∈ L 2 ([0, T ], U ) be the minimizer of (4.1).

Figure 2 :

 2 Figure 2: Under the assumptions of Corollary 4.1, at the limit κ → ∞, the emergent control will be a control steering (y i ζ ) ζ to (y ζ ) ζ .

Example 4 . 1 .

 41 This example illustrates the result of Corollary 4.1 in the exact simultaneous controllability case. Consider the probability space (Ω, P(Ω), µ) with Ω = {1, 2} and µ({1}) = µ({2}) = 1

  Controlled trajectories in the phase plan using the averaged control. The variance at final time is 2.75e-01.

  Averaged control, the norm of the control is 3.19.

Figure 3 :

 3 Figure 3: On left, we plotted the trajectories obtained by the averaged control (right) which is of minimal L 2 -norm.

  Controlled trajectories in the phase plan using the simultaneous control.

  Simultaneous control, the norm of the control is 6.34e+01.

Figure 4 :

 4 Figure 4: On left, we plotted the trajectories obtained by the simultaneous control (right) which is of minimal L 2 -norm.

1 uκκ

 1 -u∞ L 2 (0,T ;U ) κ Control's convergence with respect to κ (a) Plot of the L 2 -distance between the exact simultaneous control and the optimal control of the minimisation problem indexed with κ. This distance behaves as Cκ -α with α 0.98. Variance with respect to κ (b) Plot of the variance at final state ( y ζ (T ) -y f 2 X dµ ζ ) with respect to κ. The variance behaves as Cκ -α with α 1.95.

Figure 5 :

 5 Figure 5: Plots in log -log scale of the L 2 -distance between the solution of the optimal control with parameter κ and the exact simultaneous control (left) and of the variance at final state (right) as κ grows. The decay rates obtained are coherent with the results of Proposition 4.2.

  Controlled trajectories in the phase plan using the averaged control. The variance at final time is 2.13e+01. Averaged control, the L 2 -norm of the control is 1.99e+01.

Figure 6 : 2 15 L 2 -

 62152 Figure 6: On left, we plotted the trajectories obtained by the averaged control (right) which is of minimal L 2 -norm.

κ

  Variance with respect to κ (b) Plot of the variance at final state with respect to κ.

Figure 7 :

 7 Figure 7: Plots of the norm of the control (left) and of the variance at final state (right) as κ grows.

  Controlled trajectories in the phase plan using the optimal control for κ = 5.10 3 . The variance at final time is 7.41. Optimal control for κ = 5.10 3 its L 2 -norm is 5.30e+01.

Figure 8 :

 8 Figure 8: On left, we plotted the trajectories obtained by the optimal control (right) for κ = 5.10 3 .

  we obtain the result. Proof of Proposition 5.2. As in the previous proof, we set θ ζ = µ({ζ}) and without loss of generality, we can assume that θ ζ > 0 for every ζ ∈ N * . Let us then set θ

Z 2 X( 1 - 2 X( 1 2 L 2 (y 2 .C 2 L 2 ( 2 X θ Z ζ = 0 .y 2 X 2 X

 2121222222022 ζ=1 y ζ (T ; u) -y f ζ θ Z ζ = 0 is equivalent to minimizing J Z κ + I Z .The proof of this result is based on Γ-convergence. More precisely, we will prove that the sequenceJ Z κ + I Z Z∈N * Γ-converge to J ∞ κ + I ∞ .with C > 0 a constant. Using Cauchy-Schwarz inequality, we obtain:-t)A ζ B ζ u Z (t) dt θ ∞ ζ µ({1, • • • , Z})) -t)A ζ B ζ u Z (t) dt -t)A ζ B ζ u Z (t) dt to the admissibility conditions (see Lemma 2.2), there exists a constant Ĉ > 0 such that: -t)A ζ B ζ u Z (t) dt θ ∞ ζ -µ({1, • • • , Z})) Ĉ u Z [0,T ],U ) .Thus, taking the limit Z → ∞, we obtain∞ ζ=1 y ζ (T ; u ∞ ) -y f ζ θ ∞ ζ X = 0, i.e. I ∞ (u ∞ ) every Z ∈ N * ,we have, by Cauchy-Schwarz inequality:Z ζ=1 y ζ (T ; u Z ) -T ; u Z ) -y ζ (T ; u ∞ ) ζ (T ; u Z ) -y ζ (T ; u ∞ ), y ζ (T ; u ∞ ) -y f ζ X θ Z ζ T ; u Z ) -y ζ (T ; u ∞ )Using the admissibility of every system indexed by ζ, for every ζ ∈ N * , there exists C ζ > 0 such that:Z ζ=1 y ζ (T ; u Z ) -y ζ (T ; u ∞ ) ζ θ Z ζ u Z -u ∞ [0,T ],U ) .In addition, due to assumption (2.3) made in Lemma 2.2, we have lim Z→∞ Z ζ=1 C ζ θ Z ζ < ∞ and hence, since (u Z ) Z is strongly convergent to u ∞ , lim Z→∞ Z ζ=1 y ζ (T ; u Z ) -y ζ (T ; u ∞ ) On the other hand, it remains clear, due to the construction of θ Z ζ ζ (T ; u Z ) -y f ζ θ Z ζ = ∞ ζ=1 y ζ (T ; u ∞ ) -y f ζ θ ∞ ζ .

Table 1 :

 1 Possible behaviors as κ → ∞, with y ζ defined by (4.4).

	Ω,X;µ) κ

  one can consider the problem of finding the best possible basis. For instance we can wonder if there exists ϕ 1 , • • • , ϕ K-1 such that the pair MIAI -1 M , MIB has a normal form (see[START_REF] Trélat | Contrôle optimal[END_REF] Proposition 2.2.6]). That is to say find

	ϕ 1 , • • • , ϕ K-1 such that MIAI -1 M has the structure	* * 0 *	and MIB the structure	* 0	.

* This work was supported by the Advanced Grants NUMERIWAVES/FP7-246775 of the European Research Council Executive Agency, FA9550-14-1-0214 of the EOARD-AFOSR, PI2010-04 and the BERC 2014-2017 program of the Basque Government, the MTM2011-29306-C02-00 and SEV-2013-0323 Grants of the MINECO.

The first author thanks BCAM for its hospitality and support as a Visiting fellow of the NUMERIWAVES Advanced Grant of the ERC.

The authors thank the CIMI -Toulouse for the hospitality and support during the preparation of this work in the context of the Excellence Chair in "PDE, Control and Numerics".

Of course, we have introduced the cost functions J κ in order to pass to the limit κ → ∞. Let us first state a trivial statement: Lemma 4.1. Set T > 0 and assume the system (1.1) is controllable in average. For every κ 0, let us define u κ the minimum of J κ under the constraint E y ζ (T, u κ ) -y f ζ = 0. Then, we have:

u κ+ε L 2 ([0,T ],U ) and

(κ, ε 0) .

In addition, for every κ 0, we have:

and E y ζ (T ; u) -y f ζ = 0 . (4.3)

Proof. It remains clear that for every κ, ε 0, we have:

Thus from, J κ (u κ )+J κ+ε (u κ+ε ) J κ (u κ+ε )+J κ+ε (u κ ), it is easy to see that y ζ (T ; u κ ) -y f ζ L 2 (Ω,X;µ) κ 0 is decreasing and then, form J κ (u κ ) J κ (u κ+ε ), we obtain that u κ L 2 ([0,T ],U ) κ 0 is increasing. Let us now prove (4.3). To this end, we assume by contradiction that there exists u ∈ L 2 ([0, T ], U ) such that:

Then we have J κ (u) < J κ (u κ ) which is in contradiction with u κ minimize J κ .

Various situations could hold as κ → ∞. These different situations, reported on Table 1, are given by the following theorem. Theorem 4.1. Set T > 0 and assume that the system (1.1) in controllable in average in time T . For every κ 0, let us define u κ the minimum of J κ under the constraint

Then, the following alternative holds:

The aim of this point is to prove:

Otherwise, we can assume up to the extraction of a subsequence that for every Z ∈ N * , we have I Z (u Z ) = 0. Under this assumption, let us prove:

This will ensure (5.4).

1. Let us prove that I ∞ (u ∞ ) = 0: To this end, let us notice:

The admissibility condition, ensures:

which is going to 0 when Z → ∞. Consequently, (v Z ) Z converges to 0, that is to say, there exists a sequence (u

The final result follows from Γ-convergence property and J ∞ κ +I ∞ admits one and only one minimizer.

Let us denote by u Z κ (resp. u ∞ κ ) the minimizer of the truncated (resp. initial) minimisation problem. We proved here that lim

This question is the aim of the next proposition.

Proposition 5.4. Let us assume that the system (1.1) is controllable in average for the probability measure µ. Then, 1. For every κ 0 and every large enough Z ∈ N * , there exists a minimizer u Z κ ∈ L 2 ([0, T ], U ) of the truncated minimisation problem (5.3).

2. Up to a subsequence, the sequence (u Z κ ) κ is strongly convergent to an element u Z ∞ ∈ L 2 ([0, T ], U ) which is a solution of the minimisation problem:

where y Z, ζ is a minimizer of the minimization problem:

and where we have set: (b) if the sequence (u Z ∞ ) Z is bounded, then the system (1.1) can be exactly steered from y i ζ to y ζ and up to a subsequence (u Z ∞ ) Z it is weakly convergent to such a control; otherwise, the system (1.1) can be approximatively steered from y i ζ to y ζ .

Proof. First of all, the existence of u Z κ for every κ 0 and every Z large enough is ensured by Proposition 5.1.

Without loss of generality, we can assume µ({ζ}) > 0 for every ζ ∈ N * and the system (1.1) is controllable in average for the measure χ Z µ for every Z ∈ N * . As in the previous proofs, we set for convenience, µ({ζ}

we can assume without loss of generality that y i ζ = 0. Let us notice that for every Z 1, the control system (1.1) endowed with the measure χ Z µ can be recast as a parameter dependent system whose parameters take place in a set of finite cardinal. Consequently, Corollary 4.1 ensure that the sequence of minimizers (u Z κ ) κ 0 is convergent to u Z ∞ ∈ L 2 ([0, T ], U ) solution of the minimisation problem (5.5).

Let us prove item 3a. For every Z ∈ N * ∪ ∞, the minimisations problems (5.6) is:

∞ otherwise, so that the above minimisation problem is:

In the next points, we will prove that

• Lower bound:

To this end, we notice that:

Thus, taking the limit Z → ∞, we obtain

To conclude, it is obvious that lim

• Upper bound:

First of all, it is easy to show that if the sequence ((

Consequently, in order to prove (5.7), we only need to prove the existence of a sequence ((

. Moreover, the system (1.1) is controllable in average for the measure χ Z µ, thus the minimisation problem:

admits a minimum which is obtained for v = v Z . In addition, since lim

is going to 0 as Z → ∞.

All in all, from Γ-convergence tools and the fact that G ∞ + I 0 + I ∞ 1 admits one and only one minimizer, we obtain lim

Concluding remarks

In this paper, we have presented a theoretical link between the averaged controllability and the exact simultaneous controllability. But there still exist many practical questions to be addressed. We list here some of them:


Merwan Barlier
email: merwan.barlier@orange.com

Romain Laroche
email: romain.laroche@orange.com

Olivier Pietquin
email: olivier.pietquin@univ-lille1.fr

A Stochastic Model For Computer-Aided Human-Human Dialogue

In this paper we introduce a novel model for computer-aided human-human dialogue. In this context, the computer aims at improving the outcome of a human-human task-oriented dialogue by intervening during the course of the interaction. While dialogue state and topic tracking in human-human dialogue have already been studied, few work has been devoted to the sequential part of the problem, where the impact of the system's actions on the future of the conversation is taken into account. This paper addresses this issue by first modelling human-human dialogue as a Markov Reward Process. The task of purposely taking part into the conversation is then optimised within the Linearly Solvable Markov Decision Process framework. Utterances of the Conversational Agent are seen as perturbations in this process, which aim at satisfying the user's long-term goals while keeping the conversation natural. Finally, results obtained by simulation suggest that such an approach is suitable for computer-aided human-human dialogue and is a first step towards three-party dialogue.

Introduction

Spoken dialogue is the most common way for communicating between humans. When an interaction aims at achieving a task, the dialogue is said to be task-oriented. For example, negotiating an appointment, interacting with an operator to solve a land-line problem or making decisions in a meeting are all taskoriented dialogues. In some cases, an interlocutor needs external information to perform the task successfully. In the land-line troubleshooting case for instance, the operator may need technical information about the customer's problem to provide useful support. Time taken to search for this information may deteriorate the quality and the naturalness of the dialogue. The goal of this paper is to provide a general framework for computer-aided human-human dialogue, where the task of providing contextual information to interlocutors is automated. The way information is provided by the Conversational Assistant (CA) may take any form like speech, pop ups on a screen etc.

Addressing this problem first requires analysing humanhuman spoken dialogue. Even if it has been studied for a long time from a cognitive/social science perspective [START_REF] Austin | How to do things with words[END_REF][START_REF] Clark | Contributing to discourse[END_REF], its automatic processing from the computing point of view is relatively recent. Works from the mid 90's like [START_REF] Gorin | How may i help you?[END_REF][START_REF] Waibel | Meeting browser: Tracking and summarizing meetings[END_REF] presented systems able to passively listen to human-human conversations and process them in background (the former performing call-routing and the latter automatically summarizing meetings). Since then, research on human-human dialogue mainly focused on Spoken Language Understanding (SLU) tasks such as dialogue act tagging [START_REF] Stolcke | Dialogue act modeling for automatic tagging and recognition of conversational speech[END_REF], topic detection [START_REF] Hsueh | Automatic segmentation of multiparty dialogue[END_REF], dialogue state tracking [START_REF] Kim | The fourth dialog state tracking challenge[END_REF], etc.

In this work, we address a different aspect by considering the sequential nature of the task, taking into account the fact that utterances of the CA may have an impact on the future of the conversation. In both human-machine and computer-aided human-human dialogue, Dialogue Management (DM) may be seen as a sequential decision making problem, where decisions are made on which dialogue act has to be taken according to the dialogue context. Since its introduction for DM [START_REF] Levin | A stochastic model of computerhuman interaction for learning dialogue strategies[END_REF][START_REF] Singh | Reinforcement learning for spoken dialogue systems[END_REF], Reinforcement Learning (RL) has successfully proven its efficiency for optimising human-machine interaction strategies [START_REF] Lemon | Machine learning for spoken dialogue systems[END_REF][START_REF] Young | Pomdpbased statistical spoken dialog systems: A review[END_REF].

In this paper, we propose to model for the first time the optimisation of the computer-aided human-human dialogue task as an RL problem. To do so, several issues are addressed. First, RL requires providing a reward function quantifying the quality of each intervention decision. It has to account for the naturalness of the conversation between both humans while ensuring a gain in the outcome of the dialogue. An original cost function is therefore introduced, balancing the intrusiveness and the success of the conversation. Second, an efficient way of computing the intervention strategy is provided by casting the problem into the Linearly Solvable Markov Decision Processes framework [START_REF] Todorov | Linearly-solvable markov decision problems[END_REF]. Third, to take Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) errors into account, experiments are made in the presence of noise. Empirical results show that the learned intervention strategy improves the general outcome of the conversation without much perturbation and while being robust to noise.

A General Model Of Human-Human Communication

Temporal Sequences

From the point of view of an external observer (like for instance a CA), dialogue may be seen as a sequence of observations ot belonging to a set O, namely utterances for a human observer and dialogue acts with confidence scores for a dialogue manager. Let us assume that each dialogue d = {o0, o1, ..., ot, ...} is drawn according to an underlying probability distribution:

P(o0, o1, ...) = P(o0) t>0 P(ot|ht),
where ht = {o0, ..., ot-1} is called the history at time t.

Markov Chains

Because the space of histories can be very large, computing this distribution is often intractable. Relevant information from histories may however be compressed in a so-called dialogue state, belonging to some set S. States are built so as to fulfill the Markov assumption (the future depends only on the present independently from the past). It is always possible to represent a temporal sequence as a Markov chain by defining states as histories st = ht. Designing more sophisticated state representations remains a major topic of research since the Markov property is heavily constraining. For some problems, it is possible to handcraft them [START_REF] Larsson | Information state and dialogue management in the trindi dialogue move engine toolkit[END_REF] but such representations are strongly problem-dependent and do not generalise well. They can also be learned [START_REF] Daubigney | Model-free pomdp optimisation of tutoring systems with echo-state networks[END_REF][START_REF] Asri | Compact and interpretable dialogue state representation with genetic sparse distributed memory[END_REF]. Dialogue state tracking over dialogue turns has become a research problem in its own right [START_REF] Henderson | Machine learning for dialog state tracking: A review[END_REF] and is not yet solved. Human-human dialogue state tracking has also been the focus of recent researches [START_REF] Kim | The fourth dialog state tracking challenge[END_REF]. Here, we assume that these states are given but that transition probabilities are unknown. Given such a representation, a dialogue is then seen as a sequence of states d = {s0, s1, ..., st, ...} drawn from a probability distribution: P(s0, s1, ...) = P(s0) t>0 P(st|st-1).

This process is called a Markov chain and the probability distribution mapping states to states is called the transition function.

Markov Reward Processes

Even though Markov chains are a powerful tool to model dialogue, they do not allow its evaluation. One defines the reward function as a mapping from states to the set of real numbers called. A reward function associated to a Markov chain defines a Markov Reward Process. Formally, a reward function associates a real number to each state (r : S → R).

Assuming that each trajectory ends with probability 1, one can define the value of each state s as the expected cumulative reward that will be obtained when starting from this state:

∀s ∈ S, V (s) = E ∞ t=0 r(st) s0 = s .
In other words, the reward function defines the quality of each utterance while the value function measures how good it is to be in a state. The value function of the first state represents then the average value associated with the dialogue.

Reward functions measure the quality of dialogues. They have to be carefully designed to encode the optimisation criteria. Handcrafted evaluation criteria [START_REF] Walker | Paradise: A framework for evaluating spoken dialogue agents[END_REF] may be chosen but they heavily rely on human subjectivity and may therefore be biasprone. Inverse Reinforcement Learning (IRL) [START_REF] Ng | Algorithms for inverse reinforcement learning[END_REF][START_REF] Chandramohan | User simulation in dialogue systems using inverse reinforcement learning[END_REF] is a more objective approach to this problem based on statistical learning. It relies on the assumption that humans are experts in interacting and that their behaviour is optimal according to some underlying unknown reward. IRL aims at finding a reward function explaining the observed behaviour. Another approach to learn this reward function, Score-Based Inverse Reinforcement Learning (SBIRL) [START_REF] Asri | Scorebased inverse reinforcement learning[END_REF], makes use of expert ratings which are commonly used to evaluate dialogue to regress a reward function.

Controlling Markov Chains

In this section, we suggest a way of improving the outcome of human-human dialogues through the intervention of a CA without introducing too much perturbation.

Markov Decision Process

Markov Decision Processes (MDPs) are a common paradigm to address the problem of sequential decision making under uncertainty. Formally, an MDP is a tuple {S, A, P, r} where S is the set of states, A the set of actions, P : S × A × S → [0, 1] the transition function and r : S × A → R the reward function. When each possible trajectory ends with probability 1, the aim is to find in each state the sequence of actions maximising the expected cumulative reward, called the value function

V (s) = E[ ∞ t=0 r(st, at)|s0 = s].
The optimal value function V * is the value function that is maximal in each state. Let L be the operator such that ∀s ∈ S, LV (s) = max The optimal value function V * is the only function verifying the Bellman Optimality Equation: ∀s ∈ S, V * (s) = LV * (s) [START_REF] Puterman | Markov decision processes: discrete stochastic dynamic programming[END_REF].

A mapping from states to actions is called a policy (noted π). The policy π * is said optimal if it induces the optimal value function V * . Several policies may be optimal and share V * .

Kullback-Leibler Cost

Our goal is to improve the outcome of a human-human dialogue by intervening during the course of the conversation. From models defined in the previous section, we decide to cast the process of optimising the intervention strategy into the MDP framework. To do so, a reward function has to be defined.

We assume that dialogue states are naturally sampled from a probability distribution P0 when the human-human conversation is going on. If the systems takes the action a in a state st, the next state will be sampled according to another distribution Pa. It is assumed that humans are quasi-experts at interacting and only need small state shifts to be optimal. In other words, we consider that we build on the capacity of a human to interact naturally by augmenting his/her interactionnal context but not by modifying totally this context so as to keep the conversation on track. One would then like to have Pa quite close to P0. The Kullback-Leibler (KL) divergence is commonly used to measure the dissimilarity between two probability distributions. Formally, if p and q are two probability distributions on some discrete set S, the KL divergence between p and q is defined by KL(p||q) = s∈S p(s) log p(s) q(s) . Roughly speaking, KL divergence measures how likely a sample drawn from the distribution p could have been drawn from the distribution q.

From this, we build a reward function combining the outcome quality measure and an additional cost measuring how much the conversation will change by intervening. Formally, taking the action a in a state s induces a reward r(s, a) = r(s) -KL(Pa(s, .)||P0(s, .)). A trade-off has then to be found between perturbing the conversation and improving the dialogue's general outcome. To our knowledge, the only research work balancing an action utility and the dialogue naturalness [START_REF] Meguro | Learning to control listening-oriented dialogue using partially observable markov decision processes[END_REF] concerns listening-oriented human-machine dialogue and does not make use of the KL divergence.

Linearly-Solvable Markov Decision Problems

We now describe an efficient framework to solve the optimisation problem defined just before: the Linear Markov Decision Processes (LMDPs) [START_REF] Todorov | Linearly-solvable markov decision problems[END_REF]. LMPDs are a special class of MDPs addressing the issue of perturbing a Markov Reward Process under a KL cost. Their efficiency has been shown on many different problems [START_REF] Todorov | Efficient computation of optimal actions[END_REF][START_REF] Kappen | Optimal control as a graphical model inference problem[END_REF][START_REF] Guan | Online markov decision processes with kullback-leibler control cost[END_REF][START_REF] Abbasi-Yadkori | Largescale markov decision problems with kl control cost and its application to crowdsourcing[END_REF] since by making one more assumption that we will later relax, they make the Bellman Equation linear. Powerful tools exist then to solve it.

LMDPs operate in a framework similar to ours. They however make the assumption that the action space is the whole transition probability simplex P, that is actions consist in modifying directly the transition probabilities. Writing V * c as the optimal value function with those continuous actions, the Bellman Optimality Equation may be rewritten as follows:

V * c (s) = max P ∈P r(s, P ) + s ∈S P (s, s )V * c (s ) , = max P ∈P
r(s) -KL(P (s, .)||P0(s, .))

+ s ∈S P (s, s )V * c (s ) .
In each state, the optimal transition function is given by: π * (s) = argmax

Z(s) .

where Z(s) = s ∈S P0(s, s )e V * c (s ) is a normalisation constant. Using the fact that the KL divergence is minimal if and only if the two probability distributions are equal, the optimal transition function is given by

π * (s) = P * (s) = P0(s, .)e V * c (.) Z(s) (1)
The Bellman Equation then may be rewritten as Exponentiating this equation in vector form, it becomes linear: ∀s ∈ S, e V * c (s) = e r(s) Z(s) = e r(s) P0(s, .)e V * c .

V * c (

Solving LMDPs with Stochastic Gradient Descent

Solving the previous equation via linear algebra methods may be intractable for large state spaces. An original method has been suggested in [START_REF] Abbasi-Yadkori | Largescale markov decision problems with kl control cost and its application to crowdsourcing[END_REF] in which, instead of solving the equation e V * c = e LV * c , one minimises the following quantity

|e V * c -e LV * c |.
The exponentiated value function is then linearly parametrised: ∀s ∈ S, e V * c (s) = θ, φ(s) where φ(s) is a feature vector, θ a parameter vector that has to be learned and θ, φ the dot product between vectors θ and φ.

The following value e V * c (s) -e r(s) P0(s, .)e V * c may then be rewritten as θ, φ(s) -e r P0(s, .) θ, φ which is linear and thus convex. Given H some positive regularisation constant, T a set of trajectories, and v a probability distribution over those trajectories, it is shown that the optimiser θ * of the following cost function:

c(θ) = -log( θ, φ(s0)) + H t∈T v(T )
s∈T θ, φ(s) -e r(s) P0(s, .) θ, φ induces a near optimal value function and thus a near optimal policy defined by [START_REF] Austin | How to do things with words[END_REF]. Since c(θ) is convex, it is possible to optimise it through Stochastic Gradient Descent [START_REF] Boyd | Convex optimization[END_REF], where an unbiased estimate of the cost function is given by:

r(θ) = - 1 θ, φ(s0) φ(s0)
+ H s∈T sign θ, φ(s) -e r(s) P0(s, .) θ, φ × φ(s) -e r(s) P0(s, .)φ .

LMDP with discrete actions

We now restrict the action space to a discrete one A. Following the same equations, the optimal action is given by: a * (s) = argmin a∈A KL Pa(s, .) P0(s, .)e V (.) Z(s) .

This quantity is however not computable since in this case V is unknown. It has been however empirically shown [START_REF] Todorov | Efficient computation of optimal actions[END_REF] that, if we always take the action a greedy with respect to the optimal value function of the continuous MDP V * c , the obtained value function V d will not be far from the optimal one V * .

In the following sections, we will show the efficiency and the relevance of this method on a simulated dialogue task.

Experimental setting

The dialogue problem addressed in this paper is a variation on the negotiation dialogue game presented in [START_REF] Laroche | The negotiation dialogue game[END_REF][START_REF] Genevay | Transfer learning for user adaptation in spoken dialogue systems[END_REF] where two humans interact in order to settle an appointment. We added one CA to this human-human conversation. It belongs to a human and is able to intervene during the dialogue to optimise its general outcome, defined as the outcome of its owner.

The negotiation dialogue game

In the negotiation dialogue game, two humans indexed by i = 1, 2 are involved. They have to agree on some time slot τ among N available slots. Booking time slot τ i costs c i τ i to human i while agreeing on the time slot τ i = τ j makes her/him earn R i , inducing then a total reward r i τ = 1 τ i =τ j R i -c i τ . It is a turn-based game and the first player is randomly chosen. At each turn, 3 actions are available to her/him. S/He can accept the last proposal by choosing the action Accept. This action ends the dialogue and each player receives her/his corresponding reward. Otherwise, the human may refuse the last proposal and suggest a new slot by the use of the RefProp action. Finally, s/he can take the EndDial action, ending then the dialogue while giving no reward to each one.

To avoid side effects, two states are added, an initial state in which all dialogues begin and a final state in which they end.

Strategies

As in [START_REF] Laroche | The negotiation dialogue game[END_REF], we assume that strategies of humans are fixed and not necessarily optimal, so we handcrafted them. Decisions are made according to each agent's set of available positive slots, which is the set of all slots inducing a positive reward and have never been refused by the opponent. When this set is empty, the agent ends the dialogue. Otherwise, s/he compares the reward induced by the last offer with the one associated with the best available slot. If this reward is smaller, the human refuses the proposition and suggests this best available slot. Otherwise, s/he accepts the offer. In order to add some variability to the simulator, those policies are randomised. If the last action was RefProp, with probability 1, the user takes a random action between Accept, Enddial and Refprop, in that case, the human suggests his/her best available slot.

State Representation

A Markovian representation of the conversation has to be designed. This representation will correspond to the point of view of the CA. It has first access to the timetable of its owner, inducing then the reward function of the dialogue.

It also tracks the last spoken utterance. Since ASR and NLU systems are error prone, the inputs of the CA do not necessarily match what has been said. The type of each action is always understood but when an offer is made, there may be misunderstandings on the time slot. Let SER be the Sentence Error Rate of the system. With probability SER, the system understands a wrong slot, while with probability 1 -SER the utterance is received undisrupted. We adopt the way [START_REF] Khouzaimi | Optimising turntaking strategies with reinforcement learning[END_REF] generates speech recognition confidence scores: scorereco = 1 1+e -X where X ∼ N (c, 1), where c = 1 if the player understood the right option, and c = -1 otherwise.

To get a fully Markovian state representation, the number of utterances said during the dialogue is also tracked.

Actions of the system

In that game, the CA has four actions available: ShouldAccept, ShouldEnddial, ShouldRefProp or doing nothing. Those actions are taken just before the turn of its owner. They represent suggestions of the system. If the system suggests an action, its user takes that action with probability 1 -2 and take randomly another action otherwise.

Results

In our experiments, we fixed the number of slots to 3. The costs for each slot are randomly sampled between 0 and 4, while the reward obtained by setting an appointment is 3, leading to approximately 40000 states. Humans act randomly with probability 1 = 0.3 and follow the suggestions of the dialogue system with probability 1-2 = 0.9. To deal with discrete state spaces, we split the confidence into 6 categories around the cut points 0.05, 0.15, 0.5, 0.85, 0.95 and 1. Components of the feature vector are: initial state; final state; exponentiated rewards associated to each slot as well as the turn of the beginning state; the exponentiated obtained reward if the last action was Accept; the exponentiated differences of rewards between the suggested slot and the other slots; the exponentiated values of those slots; the value of accepting the suggested slot if the last action was RefProp and if it is the main user's turn; otherwise, the exponentiated reward of each available slot and the value of the last suggested action; finally a bias feature is added. We applied Stochastic Gradient Descent during 20000 iterations with a constant learning rate η = 1 during the first 5000 iterations and η = 0.1 for the others. At each step, this learning rate is divided by the norm of the gradient. The regularisation parameter is set to h = 5. To speed-up convergence, we used the mini-batch trick which averages at each iteration the gradient computed on 5 samples. The transition matrix P0 and KL divergences between distributions are estimated according to the Monte-Carlo method over 20000 trajectories for each state.

Figure 1 shows the average reward (computed on 1000 di- We observe that the average reward gradually decreases with an increasing SER, showing that in the presence of noise, the accuracy of the system decreases. We however observe that in every case, the obtained reward is never worse than the average reward without computer-aid.

Figure 2 shows the average number of actions taken by the system per dialogue according to an increasing SER. Because of the short length of the dialogues, the CA does not have always the opportunity to speak, explaining the fact that this number is smaller than 1. It decreases with an increasing SER but is always positive, even with much noise. This means that the system has learned to act only with a confidence score high enough, which is a desirable behaviour for the CA.

Conclusion

This paper presents a novel stochastic model for computeraided human-human dialogue. This problem is cast as an MDP where the reward function depends on the one hand of the general outcome of the dialogue and on the other hand, on its intrusiveness. The experiment, used as a proof of concept, showed that this model leads to a gain in performance.

Future works will include more sophisticated ways to compute the transition matrices such as spectral methods [START_REF] Glaude | Spectral learning with non negative probabilities for finite state automaton[END_REF]. It would also be interesting to introduce modifications of the algorithms to allow more general value function representations than linear parametrization. Co-adaptation between humans and CA [START_REF] Barlier | Humanmachine dialogue as a stochastic game[END_REF] should also be studied. Finally, this approach will be tested on real human-human dialogues.

  a∈A r(s, a) + s ∈S Pa(s, s )V (s ) .

  s) =r(s, P * ) + s ∈S P * (s, s )V * c (s ) =r(s) + s ∈S P * (s, s ) log Z(s) =r(s) + log Z(s).

Figure 1 :Figure 2 :

 12 Figure 1: Average reward function of the SER