Val Érie Berth É

Milton Minervino
email: milton@liafa.univ-paris-diderot.fr

Wolfgang Steiner
email: steiner@liafa.univ-paris-diderot.fr

J Örg Thuswaldner
email: joerg.thuswaldner@unileoben.ac.at

THE S-ADIC PISOT CONJECTURE ON TWO LETTERS

Keywords: . 2010 Mathematics Subject Classification. 37B10, 37A30, 68R15 S-adic shifts, substitutions, tilings, Pisot numbers, coincidences, Lyapunov exponents, balancedness

We prove an extension of the well-known Pisot substitution conjecture to the S-adic symbolic setting on two letters. The proof relies on the use of Rauzy fractals and on the fact that strong coincidences hold in this framework.

Introduction

A symbolic substitution is a morphism of the free monoid. More generally, a substitution rule acts on a finite collection of tiles by first inflating them, and then subdividing them into translates of tiles of the initial collection. Substitutions thus generate symbolic dynamical systems as well as tiling spaces. The Pisot substitution conjecture states that any substitutive dynamical system has pure discrete spectrum, under the algebraic assumption that its expansion factor is a Pisot number (together with some extra assumption of irreducibility). Pure discrete spectrum means that the substitutive dynamical system is measurably conjugate to a rotation on a compact abelian group. Pisot substitutions are thus expected to produce self-similar systems with long range order. This conjecture has been proved in the two-letter case; see [START_REF] Barge | Coincidence for substitutions of Pisot type[END_REF] together with [START_REF] Host | Représentation géométrique des substitutions sur 2 lettres[END_REF] or [START_REF] Hollander | Two-symbol Pisot substitutions have pure discrete spectrum, Ergodic Theory Dynam[END_REF]. For more on the Pisot substitution conjecture, see e.g. [BST10, ABB + 15].

We prove an extension of the two-letter Pisot substitution conjecture to the symbolic S-adic framework, that is, for infinite words generated by iterating different substitutions in a prescribed order. More precisely, an S-adic expansion of an infinite word ω is given by a sequence of substitutions σ = (σ n ) n∈N (called directive sequence) and a sequence of letters (i n ) n∈N , such that ω = lim n→∞ σ 0 σ 1 • • • σ n (i n+1 ). Such expansions are widely studied. They occur e.g. under the term "mixed substitution" or "multi-substitution" [START_REF] Gähler | Cohomology of one-dimensional mixed substitution tiling spaces[END_REF][START_REF] Pacheco | Statistical stability for multi-substitution tiling spaces[END_REF], or else as "fusion systems" [START_REF] Frank | Fusion: a general framework for hierarchical tilings of R d[END_REF][START_REF]Fusion tilings with infinite local complexity[END_REF]. They are closely connected to adic systems, such as considered e.g. in [START_REF] Fisher | Nonstationary mixing and the unique ergodicity of adic transformations[END_REF]. For more on S-adic systems, see [START_REF] Durand | Linearly recurrent subshifts have a finite number of non-periodic subshift factors, Ergodic Theory Dynam[END_REF][START_REF] Corrigendum | Linearly recurrent subshifts have a finite number of non-periodic subshift factors[END_REF][START_REF] Durand | Do the properties of an S-adic representation determine factor complexity?[END_REF][START_REF] Berthé | Beyond substitutive dynamical systems: S-adic expansions, RIMS lecture note[END_REF].

We consider the (unimodular) Pisot S-adic framework introduced in [START_REF] Berthé | Geometry, dynamics and arithmetic of S-adic shifts[END_REF], where unimodular means that the incidence matrices of the substitutions are unimodular. The S-adic Pisot condition is stated in terms of Lyapunov exponents: the second Lyapunov exponent associated with the shift space made of the directive sequences σ and with the cocyles provided by the incidence matrices of the substitutions is negative. For a given directive sequence σ, let L (k) σ stand for the language associated with the shifted directive sequence (σ n+k ) n∈N . The Pisot condition implies in particular that, for some C > 0, the language L (k) σ is C-balanced, for some infinite set of non-negative integers k. Recall that C-balancedness means that the number of occurrences of any letter in any factor of given length is the same up to some deviation bounded by C. We note that in [START_REF] Sadun | Finitely balanced sequences and plasticity of 1-dimensional tilings[END_REF] it is shown that balancedness characterizes topological conjugacy under changes in the lengths of the tiles of the associated R-action on the 1-dimensional tilings.

One difficulty when working in the S-adic framework is that no natural candidate exists for a left eigenvector (that is, for a stable space). Recall that the normalized left eigenvector (whose existence comes from the Perron-Frobenius Theorem) in the substitutive case provides in particular the measure of the tiles of the associated tiling of the line. We introduce here a set of assumptions which, among other things, allows us to work with a generalized left eigenvector (that is, a stable space). We stress the fact that this vector is not canonically defined (contrary to the right eigenvector). These assumptions will be implied by the S-adic Pisot condition. We need first to guarantee the existence of a generalized right eigenvector u (which yields the unstable space). The corresponding conditions are natural and are stated in terms of primitivity of the directive sequence σ in the S-adic framework. This means that long enough products of incidence matrices associated with σ will be positive. We also require the directive sequences σ to be recurrent (every finite combination of substitutions in the sequence occurs infinitely often), which yields unique ergodicity and strong convergence toward the generalized right eigendirection. We then need rational independence of the coordinates of u. This is implied by the assumption of algebraic irreducibility, which states that the characteristic polynomials of long enough products of incidence matrices of σ are irreducible. Lastly, we then will be able to define a vector playing the role of a left eigenvector, by compactness together with primitivity and recurrence.

The starting point of the proofs in the two-letter substitutive case is that strong coincidences hold [START_REF] Barge | Coincidence for substitutions of Pisot type[END_REF]. We also prove an analogous statement as a starting point. However the fact that strong coincidences hold at order n (i.e., for the product σ 0 σ 1 • • • σ n ) does not necessarily imply strong coincidences at order m for m > n (which holds in the substitutive case) makes the extension to the present framework more delicate than it first occurs. The proof heavily relies, among other things, on the recurrence of the directive sequence. There are then two strategies, one based on making explicit the action of the rotation on the unit circle [START_REF] Host | Représentation géométrique des substitutions sur 2 lettres[END_REF], whereas the approach of [START_REF] Hollander | Two-symbol Pisot substitutions have pure discrete spectrum, Ergodic Theory Dynam[END_REF] uses the balanced pair algorithm. However, there seems to be no natural expression of the balanced pair algorithm in the S-adic framework. Our strategy for proving discrete spectrum thus relies on the use of "Rauzy fractals" and follows [START_REF] Host | Représentation géométrique des substitutions sur 2 lettres[END_REF]. We recall that a Rauzy fractal is a set which is endowed with an exchange of pieces acting on it that allows to make explicit (by factorizing by a natural lattice) the maximal equicontinuous factor of the underlying symbolic dynamical system (see [START_REF] Fogg | Substitutions in dynamics, arithmetics and combinatorics[END_REF]Section 7.5.4]). For more on Rauzy fractals, see e.g. [START_REF] Fogg | Substitutions in dynamics, arithmetics and combinatorics[END_REF][START_REF] Siegel | Topological properties of Rauzy fractals[END_REF][START_REF] Berthé | Substitutions, Rauzy fractals, and tilings, Combinatorics, Automata and Number Theory[END_REF]. More precisely, the strong coincidence condition implies that there is a well-defined exchange of pieces on the Rauzy fractal. It remains to factorize this exchange of pieces in order to get a circle rotation. The factorization comes from [START_REF] Host | Représentation géométrique des substitutions sur 2 lettres[END_REF] and extends directly from the substitutive case to the present S-adic framework.

Let us now state our results more precisely. (Although most of the terminology of the statements was defined briefly in the introduction above, we refer the reader to Section 2 for exact definitions.) We just recall here that a sequence of substitutions

σ = (σ n ) n∈N on the alphabet A = {1, 2} satisfies the strong coincidence condition if there is n ∈ N such that σ 0 σ 1 • • • σ n (1) ∈ p 1 iA * and σ 0 σ 1 • • • σ n (2) ∈ p 2 iA *
for some i ∈ A and words p 1 , p 2 ∈ A * with the same abelianization l(p 1 ) = l(p 2 ).

Theorem 1. Let σ = (σ n ) n∈N be a primitive and algebraically irreducible sequence of substitutions over A = {1, 2}. Assume that there is C > 0 such that for each ℓ ∈ N, there is n ≥ 1 with (σ n , . . . , σ n+ℓ-1 ) = (σ 0 , . . . , σ ℓ-1 ) and the language L (n+ℓ) σ is C-balanced. Then σ satisfies the strong coincidence condition.

Note that we do not assume unimodularity of the substitutions for strong coincidences, whereas we do for the following.

Theorem 2. Let σ = (σ n ) n∈N be a primitive sequence of unimodular substitutions over A = {1, 2}. Assume that there is C > 0 such that for each ℓ ∈ N, there is n ≥ 1 with (σ n , . . . , σ n+ℓ-1 ) = (σ 0 , . . . , σ ℓ-1 ) and the language L (n+ℓ) σ is Cbalanced. Then the S-adic shift (X σ , Σ, µ), where Σ stands for the shift and µ is the (unique) shift-invariant measure on X, is measurably conjugate to a rotation on the circle S 1 ; in particular, it has pure discrete spectrum.

We also prove the following metric result. Note that the assumptions of the theorem imply that the dynamical system (D, Σ, ν) satisfies the Pisot condition.

Theorem 3. Let S be a finite set of unimodular substitutions on two letters, and let (D, Σ, ν) with D ⊂ S N be a sofic shift. Assume that ν assigns positive measure to each (non-empty) cylinder, and that there exists a cylinder corresponding to a substitution with positive incidence matrix. Then, for ν-almost all sequences σ ∈ D the S-adic shift (X σ , Σ, µ) is measurably conjugate to a rotation on the circle S 1 ; in particular, it has pure discrete spectrum.

Let us describe briefly the organization of the paper. We recall the required definitions in Section 2. Section 3 is devoted to the proof of the fact that the strong coincidence condition holds (Theorem 1), and we conclude the proof of Theorems 2 and 3 in Section 4.

Ingredients

The following S-adic framework is defined in full detail and for a finite alphabet A = {1, . . . , d} in [START_REF] Berthé | Geometry, dynamics and arithmetic of S-adic shifts[END_REF]. We introduce the reader to some of the main notions and results for d = 2.

2.1. S-adic shifts. Let A = {1, 2} and let A * denote the free monoid of finite words over A, endowed with the concatenation of words as product operation. A substitution σ over A is an endomorphism of A * sending non-empty words to nonempty words. The incidence matrix (or abelianization) of σ is the square matrix

M σ = (|σ(j)| i ) i,j∈A ∈ N 2×2
, where the notation |w| i stands for the number of occurrences of the letter i in w ∈ A * . The substitution σ is said to be unimodular if |det M σ | = 1. The abelianization map is defined by l :

A * → N 2 , w → t (|w| 1 , |w| 2 ). Note that M σ • l = l • σ.
Let σ = (σ n ) n∈N be a sequence of substitutions over the alphabet A. For ease of notation we set M n = M σn for n ∈ N, and

σ [k,ℓ) = σ k σ k+1 • • • σ ℓ-1 and M [k,ℓ) = M k M k+1 • • • M ℓ-1 (0 ≤ k ≤ ℓ).
The sequence σ is said to be primitive if, for each k ∈ N, M [k,ℓ) is a positive matrix for some ℓ > k. We say that σ is algebraically irreducible if, for each k ∈ N, the characteristic polynomial of M [k,ℓ) is irreducible for all sufficiently large ℓ.

Recall that w ∈ A * is called a factor of a finite or infinite word v if it occurs at some position in v; it is a prefix if it occurs at the beginning of v. The language associated with the sequence (σ m+n ) n∈N is

L (m) σ = w ∈ A * : w is a factor of σ [m,n) (i) for some i ∈ A, n ∈ N (m ∈ N).
We say that a pair of words u, v ∈ A * with the same length is C-balanced if

-C ≤ |u| j -|v| j ≤ C for all j ∈ A. A language L is C-balanced if each pair of words u, v ∈ L with the same length is C-balanced. It is said balanced if it is C-balanced for some C > 0.
The shift Σ maps an infinite word (ω n ) n∈N to (ω n+1 ) n∈N . A dynamical system (X, Σ) is a shift space if X is a closed shift-invariant set of infinite words over a finite alphabet, equipped with the product topology of the discrete topology.

Given a sequence σ, let S = {σ n : n ∈ N}. The S-adic shift or S-adic system with sequence σ is the shift space (X σ , Σ), where X σ denotes the set of infinite words ω such that each factor of ω is an element of L (0) σ . If σ is primitive, then X σ is the closure of the Σ-orbit of any limit word ω of σ, where ω ∈ A N is a limit word of σ if there is a sequence of infinite words (ω (n) ) n∈N with ω (0) = ω and ω (n) = σ n (ω (n+1) ) for all n ∈ N.

Recall that a shift space (X, Σ) is minimal if every non-empty closed shiftinvariant subset equals the whole set; it is called uniquely ergodic if there exists a unique shift-invariant probability measure on X. Let µ be a shift-invariant measure defined on (X, Σ). A measurable eigenfunction of the system (X, Σ, µ) with associated eigenvalue α ∈ R is an L 2 (X, µ) function that satisfies f (Σ n (ω)) = e 2πiαn f (ω) for all n ∈ N and ω ∈ X. The system (X, Σ, µ) has pure discrete spectrum if L 2 (X, µ) is spanned by the measurable eigenfunctions. Furthermore, every dynamical system with pure discrete spectrum is measurably conjugate to a Kronecker system, i.e., a rotation on a compact abelian group; see [Wal82, Theorem 3.6].

2.2. Generalized Perron-Frobenius eigenvectors. For a sequence of non-negative matrices (M n ) n∈N , there exists by [Fur60, pp. 91-95] a positive vector

u ∈ R 2 + such that n∈N M [0,n) R 2 + = R + u, provided there are indices k 1 < ℓ 1 ≤ k 2 < ℓ 2 ≤ • • • and a positive matrix B such that B = M [k1,ℓ1) = M [k2,ℓ2) = • • • .
Thus, for primitive and recurrent sequences σ, this vector exists and we call it the generalized right eigenvector of σ. The following criterion for u to have rationally independent coordinates is [BST14, Lemma 4.2].

Lemma 2.1. Let σ be an algebraically irreducible sequence of substitutions with generalized right eigenvector u and balanced language L σ . Then the coordinates of u are rationally independent.

Contrary to the cones M [0,n) R 2 + , there is no reason for the cones t (M [0,n) ) R 2 + to be nested. Therefore, the intersection of these cones does not define a generalized left eigenvector of σ. However, for a suitable choice of v, we have a subsequence (n k ) k∈N such that the directions of v (n k ) := t (M [0,n k ) ) v tend to that of v; in this case, v is called a recurrent left eigenvector.

Under the assumptions of primitivity and recurrence of σ, given a strictly increasing sequence of non-negative integers (n k ), one can show that there is a recurrent

left eigenvector v ∈ R 2 ≥0 \ {0} such that (2.1) lim k∈K, k→∞ v (n k ) v (n k ) = lim k∈K, k→∞ t (M [0,n k ) )v t (M [0,n k ) )v = v for some infinite set K ⊂ N; see [BST14, Lemma 5.7].
Here and in the following,

• denotes the maximum norm • ∞ . Note that the hypotheses of Lemma 2.1 do not guarantee that the coordinates of v are rationally independent.

We will work in the sequel with sequences σ satisfying a list of conditions gathered in the following Property PRICE (which stands for Primitivity, Recurrence, algebraic Irreducibility, C-balancedness, and recurrent left Eigenvector). By [BST14, Lemma 5.9], this property is a consequence of the assumptions of Theorem 1. Definition 2.2 (Property PRICE). We say that a sequence σ = (σ n ) has Property PRICE w.r.t. the strictly increasing sequences (n k ) k∈N and (ℓ k ) k∈N and the vector v ∈ R 2 ≥0 \ {0} if the following conditions hold. (P) There exists h ∈ N and a positive matrix B such that

M [ℓ k -h,ℓ k ) = B for all k ∈ N. (R) We have (σ n k , σ n k +1 , . . . , σ n k +ℓ k -1 ) = (σ 0 , σ 1 , . . . , σ ℓ k -1 ) for all k ∈ N. (I) The directive sequence σ is algebraically irreducible. (C) There is C > 0 such that L (n k +ℓ k ) σ is C-balanced for all k ∈ N. (E) We have lim k→∞ v (n k ) / v (n k ) = v.
We also simply say that σ satisfies Property PRICE if the five conditions hold for some not explicitly specified strictly increasing sequences (n k ) k∈N and (ℓ k ) k∈N and some v ∈ R 2 ≥0 \ {0}. 2.3. Criteria for algebraic irreducibility. We do not know a general strategy for proving algebraic irreducibility of a sequence of substitutions. However, in the unimodular two-letter case, the following lemma shows that algebraic irreducibility is a consequence of primitivity. For examples of algebraically irreducible sequences on more than two letters, we refer to [AI01, AD13, DHS13].

Lemma 2.3. Let σ be a primitive sequence of unimodular substitutions on the alphabet A = {1, 2}. Then σ is algebraically irreducible.

Proof. By primitivity, for each

k ∈ N, there is some ℓ > k such that M [k,ℓ) is positive. Then M [k,n) is positive for all n ≥ ℓ. Therefore, M [k,n
) has a Perron-Frobenius eigenvalue greater than 1. Since a two-dimensional unimodular matrix with reducible characteristic polynomial can only have the eigenvalues ±1, this proves the lemma.

2.4. The Pisot condition. Let S be a finite set of substitutions on A = {1, 2} having invertible incidence matrices, and let (D, Σ, ν) with D ⊂ S N be an (ergodic) shift equipped with a probability measure ν. With each σ = (σ n ) n∈N ∈ D, associate the cocycle A(σ) = t M 0 and denote the Lyapunov exponents w.r.t. this cocycle by θ 1 , θ 2 . As in the more general situation of [BD14, §6.3], we say that (D, Σ, ν) satisfies the Pisot condition if θ 1 > 0 > θ 2 ; see also [BST14, Section 2.6] for details.

Lemma 2.4. Let S be a finite set of unimodular substitutions on two letters, and let (D, Σ, ν) with D ⊂ S N be an ergodic shift. Assume that ν assigns positive measure to a cylinder corresponding to a substitution with positive incidence matrix. Then (D, Σ, ν) satisfies the Pisot condition.

Proof. Let Z(τ 0 , . . . , τ ℓ-1 ) be a cylinder such that the incidence matrix of τ [0,ℓ) is positive and ν Z(τ 0 , . . . , τ ℓ-1

) > 0. For σ = (σ n ) n∈N ∈ D, let a n (σ) = #{0 ≤ k ≤ n -ℓ : σ k = τ 0 , . . . , σ k+ℓ-1 = τ ℓ-1 }.
Then M [0,n) can be written as a product of at least a n (σ)/ℓ positive matrices. Therefore, each of its coefficients has size at least 2 an(σ)/ℓ-1 and, hence,

M [0,n) ≥ 2 an(σ)/ℓ ,
where • denotes the matrix norm induced by the maximum norm. For ν-almost all sequences σ = (σ n ) n∈N ∈ D, we have, by Birkhoff's Ergodic Theorem,

lim n→∞ a n (σ) n = ν Z(τ 0 , . . . , τ ℓ-1 ) .
This implies that the first Lyapunov exponent satisfies

θ 1 = lim n→∞ 1 n D log M [0,n) dν ≥ ν Z(τ 0 , . . . , τ ℓ-1 ) ℓ log 2 > 0.
Moreover, because of unimodularity, we have θ 2 = -θ 1 < 0 and the result follows.

2.5. Rauzy fractals. For a vector w ∈ R 2 \ {0}, let

w ⊥ = {x ∈ R 2 : x, w = 0}
be the line orthogonal to w containing the origin, equipped with the Lebesgue measure λ. In particular, for 1 = t (1, 1), 1 ⊥ is the line of vectors whose entries sum up to 0. Let π u,w be the projection along the direction u onto w ⊥ . Given a primitive sequence of substitutions σ, the Rauzy fractal associated with σ over A is: R = {π u,1 l(p) : p ∈ A * , p is a prefix of a limit word of σ}.

The Rauzy fractal has natural refinements defined by

R(w) = {π u,1 l(p) : p ∈ A * , pw is a prefix of a limit word of σ} (w ∈ A * ). If w ∈ A, then R(w) is called a subtile. The set R is bounded if and only if L σ is balanced. If L σ is C-balanced, then R ⊂ [-C, C] 2 ∩ 1 ⊥ ; see [BST14, Lemma 4.1].
Note that R is not necessarily an interval (however, it is an interval if the language L σ is Sturmian [START_REF] Fogg | Substitutions in dynamics, arithmetics and combinatorics[END_REF].) 2.6. Dynamical properties of S-adic shifts. For σ primitive, algebraically irreducible, and recurrent sequence of substitutions with balanced language L σ , the representation map

ϕ : X σ → R, u 0 u 1 • • • → n∈N R(u 0 u 1 • • • u n )
is well-defined, continuous and surjective; for more details, see [START_REF] Berthé | Geometry, dynamics and arithmetic of S-adic shifts[END_REF]Lemma 8.3].

Suppose that the strong coincidence condition holds. Then the exchange of pieces

E : R → R, x → x + π u,1 e i if x ∈ R(i)
is well-defined λ-almost everywhere on R.

The following results appear in [BST14, Theorem 1]. The assumptions on the directive sequence σ are the ones of Theorem 2.

Proposition 2.5. Let σ = (σ n ) n∈N be a primitive and algebraically irreducible sequence of unimodular substitutions over A = {1, 2}. Assume that there is C > 0 such that for each ℓ ∈ N, there is n ≥ 1 with (σ n , . . . , σ n+ℓ-1 ) = (σ 0 , . . . , σ ℓ-1 ) and the language L (n+ℓ) σ is C-balanced. Then the following results are true.

(1) The S-adic shift (X σ , Σ) is minimal and uniquely ergodic. Let µ stand for its unique invariant measure. (2) Each subtile R(i), i ∈ A, of the Rauzy fractal R is a compact set that is the closure of its interior; its boundary has zero Lebesgue measure λ.

(3) If σ satisfies the strong coincidence condition, then the subtiles R(i), i ∈ A, are mutually disjoint in measure, and the S-adic shift (X σ , Σ, µ) is measurably conjugate to the exchange of pieces (R, E, λ) via ϕ.

We will consider in the sequel the one-dimensional lattice Λ := 1 ⊥ ∩ Z 2 = Z(e 2e 1 ). Let π : 1 ⊥ → 1 ⊥ /Λ be the canonical projection. Since π u,1 e 2 ≡ π u,1 e 1 mod Λ holds, the canonical projection of E onto 1 ⊥ /Λ ∼ = S 1 is equal to the translation x → x + π u,1 e 1 . Then we have the following commutative diagram:

(2.2)

X σ ϕ / / Σ R π / / E 1 ⊥ /Λ +πu,1e1 X σ ϕ / / R π / / 1 ⊥ /Λ

Strong coincidence

We recall the formalism of geometric substitution introduced in

[AI01]. For [x, i] ∈ Z 2 × A, let E 1 (σ)[x, i] = [M σ x + l(p), j] : j ∈ A, p ∈ A * and pj is a prefix of σ(i) .

Then strong coincidence holds (on two letters) if and only if there exists

n ∈ N such that E 1 (σ [0,n) )[0, 1] ∩ E 1 (σ [0,n) )[0, 2] = ∅.
We identify each [x, i] ∈ Z 2 × A with the segment x + [0, 1) e i . Define the height (with respect to u and v) of a point

x = tu + π u,v x ∈ R 2 by H(x) := t ∈ R.
According to the terminology introduced in [BD02], a configuration (of segments) K of size m with respect to a vector w is a collection of m distinct segments [x, i] ∈ Z 2 × A such that some translate of w ⊥ intersects the interior of each element of K (the corresponding points thus have the same height with respect to u and w). The n-th iterate is

K (n) = E 1 (σ [0,n) )[x, i] : [x, i] ∈ K .
Note that the n-th iterate of a configuration is not a configuration of segments but a union of "broken lines".

Observe that, by [BST14, Proposition 4.3 and Lemma 4.1] for a primitive, algebraically irreducible, and recurrent sequence of substitutions σ with C-balanced language L σ , we have lim n→∞ π u,1 M [0,n) x = 0 for each x ∈ R 2 and π u,1 l(p) ≤ C for all prefixes p of limit words of σ. Thus, for each sufficiently large n, the vertices of K (n) are in T u,C := {x ∈ Z 2 : π u,1 x < C + 1}, and we may consider only K ⊂ T u,C . (This corresponds to [BD02, Lemma 2] which is stated in the substitutive case.) In particular, {[0, 1], [0, 2]} is a configuration as soon as w has positive entries. Obviously, it is contained in T u,C .

We say that K has an n-coincidence if there exist

[x, i], [y, j] ∈ K such that E 1 (σ [0,n) )[x, i] ∩ E 1 (σ [0,n) )[y, j] = ∅.
Given a set J ⊆ N, we say that a configuration K is J-coincident if K has an n-coincidence for some n ∈ J. Observe that n-coincidence does not necessarily imply m-coincidence for m > n. However, translating all vertices of a configuration by a fixed vector does not change the property of being J-coincident.

We first prove the following proposition, generalizing the proof of [BD02, Theorem 1]. Proposition 3.1. Assume that the sequence of substitutions σ = (σ n ) n∈N over the alphabet A = {1, 2} has Property PRICE w.r.t. the sequences (n k ) k∈N and (ℓ k ) k∈N and the vector v, and that

(3.1) v ⊥ ∩ Z 2 ∩ (T u,C -T u,C ) = {0},
with C such that L σ is C-balanced. Then σ satisfies the strong coincidence condition.

Note that (3.1) holds in particular when v has rationally independent coordinates.

Proof. Let (n k ) k∈N , (ℓ k ) k∈N be the sequences of property PRICE. Consider the sets

J h = n k0 + n k1 + • • • + n ks : s ≥ 0, n kj + ℓ kj ≤ ℓ kj+1 ∀ 0 ≤ j < s, k 0 ≥ h .
Note that k j < k j+1 , for 0 ≤ j < s, since n kj + ℓ kj ≤ ℓ kj+1 implies in particular that ℓ kj < ℓ kj+1 . Given such a sum in J h , repeatedly applying (R) we get

(3.2) σ [0,n k 0 +n k 1 +•••+n ks ) = σ [0,n ks ) • • • σ [0,n k 1 ) σ [0,n k 0 ) .
We only consider in this proof configurations with respect to v. Let D h be the set of not J h -coincident configurations that are contained in T u,C , and

D = h∈N D h . Since J 0 ⊃ J 1 ⊃ • • • , we have D 0 ⊆ D 1 ⊆ • • • . As u ∈ R 2 + and v ∈ R 2 ≥0 \ {0}
are not orthogonal (by Lemma 2.1), D contains only finitely many configurations up to translation. Moreover, with each configuration K ∈ D h , all translates of K that are in T u,C are also contained in D h , by the translation-invariance of J h -coincidence. Therefore, we have D h = D for all sufficiently large h.

Let K be a configuration in D of maximal size. There exists an interval I of positive length such that, for every t ∈ I, v ⊥ + tu intersects each of the segments of K in its interior. Indeed, (3.1) implies that v has positive coordinates. By property (E), the same holds for (v (n k ) ) ⊥ + tu, provided that k is sufficiently large.

Consider now K (n k ) , with k large enough such that all the following hold: D k = D, all segments of K intersect (v (n k ) ) ⊥ + tu for all t ∈ I, and all vertices of K (n k ) are in T u,C . Let {p 1 , p 2 , . . . , p r k } be the set of vertices of K

(n k ) that are contained in M [0,n k ) (v (n k ) ) ⊥ + I u = v ⊥ + I M [0,n k ) u. By (3.1), no two points in Z 2 ∩ T u,C
have the same height. Therefore, we can assume w.l.o.g. that

H(p 1 ) < H(p 2 ) < • • • < H(p r k ).
For all 1 ≤ j ≤ r k , the configuration were not in D = D h , then K

K (n k ) j := {[x, i] ∈ K (n k ) : x + [0, 1) e i ∩ v ⊥ + H(p j ) u = ∅} has the same size as K because K is not J k -coincident and, for each [x ′ , i ′ ] ∈ K, the collection of segments in E 1 (σ [0,n k ) )[x ′ , i ′ ] forms a broken line from M [0,n k ) x ′ to M [0,n k ) (x ′ + e i ′ )
(n k ) j
would have an m-coincidence for some m ∈ J h . Write m = n k0 + n k1 + • • • + n ks . One has m + n k ∈ J k , since n k + ℓ k ≤ ℓ h ≤ ℓ k0 . But then K would have an (m + n k )-coincidence because

E 1 (σ [0,m) )K (n k ) = E 1 (σ [0,m) )E 1 (σ [0,n k ) )K = E 1 (σ [0,m+n k ) )K, contradicting that K ∈ D = D k .
For all 2 ≤ j ≤ r k , the configurations K (n k ) j-1 and K

(n k ) j differ only by segments ending and beginning at p j , and the number of segments in K (n k ) ending and beginning at p j is thus the same. Let K ′ be equal to K (n k ) j-1 , with the segments ending at p j removed. We have the following possibilities.

(1) Two segments of K (n k ) j-1 end at p j . Then K (n k ) j = K ′ ∪ {[p j , 1], [p j , 2]}.

(2) One segment of K (n k ) j-1 ends at p j , and either K ′ ∪ {[p j , 1]} or K ′ ∪ {[p j , 2]} is in D. Then K (n k ) j = K ′ ∪ {[p j , i]} with i such that K ′ ∪ {[p j , i]} ∈ D.

(3) One segment of K (n k ) j-1 ends at p j , and both K ′ ∪ {[p j , 1]}, K ′ ∪ {[p j , 2]} are in D. Since the size of K is maximal in D, we have

K ′ ∪{[p j , 1], [p j , 2]} / ∈ D, hence E 1 (σ [0,n) )[p j , 1] ∩ E 1 (σ [0,n) )[p j , 2] = ∅ for some n ∈ N. Then also E 1 (σ [0,n) )[0, 1] ∩ E 1 (σ [0,n) )[0, 2] = ∅,

  thus the strong coincidence condition holds. Assume now that the strong coincidence condition does not hold. Hence we are always either in case (1) or case (2). Then D and the relative positions of the segments within K(n k ) j-1 entirely determine K (n k ) j. (Note that p j is the endpoint of the segments in K (n k ) j-1 , disregarding p j-1 , with minimal height.) Recall that D contains up to translation only finitely many configurations, and denote the number of such configurations by c. Then we have K(n k ) a+b = K (n k ) a + t kfor some 1 ≤ a, b ≤ c, and some translation vector t k = p b+ap a ∈ Z 2 (provided that r k ≥ 2c). Consequently, we have K(n k ) j+b = K (n k ) j + t k for all a ≤ j ≤ r kb,and thus K (n k ) a+ℓb = K (n k ) a + ℓ t k , for all ℓ such that a + ℓb ≤ r k .

  that intersects, for each t ∈ I, the line v ⊥ + t u exactly once. (Observe that the strip v ⊥ + I M [0,n k ) u has two complementary components, each of which contains one of the endpointsM [0,n k ) x ′ and M [0,n k ) (x ′ + e i ′ ) because x ′ and x ′ + e i ′ lie in different complementary components of the strip (v (n k ) ) ⊥ + Iu.)Moreover, we have K(n k ) j ∈ D. Indeed, take h ≥ k such that ℓ h ≥ n k +ℓ k . If K

(n k ) j

The authors are participants in the ANR/FWF project "FAN -Fractals and Numeration" (ANR-12-IS01-0002, FWF grant I1136).

Let now k → ∞. Then the strip v ⊥ + I M [0,n k ) u becomes wider and wider as k grows, hence r k → ∞. Since there are only finitely many possibilites for t k , there exists thus a t ∈ Z 2 such that arbitrarily large multiples of t are translation vectors of configurations in D. Since all configurations in D are in T u,C , the vector t must be a scalar multiple of u, in contradiction with Lemma 2.1. This proves that the strong coincidence condition holds.

To prove Theorem 1, we show that the conditions of Proposition 3.1 are fulfilled for some shifted sequence (σ n+h ) n∈N .

Proof of Theorem 1. Let σ satisfy the assumptions of Theorem 1. By [BST14, Lemma 5.9], property PRICE holds for some sequences (n k ), (ℓ k ), a vector v ∈ R 2 ≥0 and a balancedness constant C. If v has rationally independent coordinates, then we can apply Proposition 3.1 directly. To cover the contrary case, assume in the following that v is a real multiple of a rational vector.

Consider x ∈ Z 2 \ {0}. We have

As u has rationally independent coordinates by Lemma 2.1 and v is a real multiple of a rational vector, we have u, v = 0. We have

) is bounded and, hence, its intersection with Z 2 is finite. Thus (3.3) implies that (v (h) ) ⊥ ∩ (T u,C -T u,C ) ∩ Z 2 = {0} for all sufficiently large h.

Choose now h = n k + ℓ k sufficiently large. Then the language L (h)

σ is C-balanced by Property (C). By [BST14, Lemma 5.10], there exists k 0 such that the shifted sequence (σ n+h ) n∈N has property PRICE w.r.t. (n k+k0 ) and (ℓ k+k0h) and the vector v (h) . Thus (σ n+h ) n∈N satisfies the strong coincidence condition by Proposition 3.1, i.e., there exists n such that

which concludes the proof of the theorem.

Proof of the S-adic Pisot conjecture

In this section we will deduce Theorem 2 from Theorem 1 and the following lemma, which is a generalization of a result of [START_REF] Host | Représentation géométrique des substitutions sur 2 lettres[END_REF]; see also [Que10, Section 6.3.3]. Recall Section 2.6, in particular the diagram (2.2). Theorem 3 follows immediately from Theorem 2. Lemma 4.1. Let σ be as in Theorem 2. Then the map ϕ = π • ϕ is one-to-one µ-almost everywhere on X σ .

Proof. Note first that it is sufficient to show that whenever ϕ(u) = ϕ(v), one can find a non-negative integer n such that ϕ(Σ n u) = ϕ(Σ n v). The µ-almost everywhere injectivity of ϕ will thus come from the µ-almost everywhere injectivity of ϕ, which holds according to Proposition 2.5 (3).

Let now u, v ∈ X σ be such that ϕ(u) = ϕ(v). Consider the set

By induction, one has z n ∈ Λ = Z (e 2e 1 ) for all n. Indeed,

Let π 0 : 1 ⊥ → R, (x, -x) → x be the projection on the first coordinate. Then

Minimality of (X σ , Σ), given by Proposition 2.5 (1), implies that the sets

Analogously, we deduce that, if n ∈ B, then z n 0. The result follows observing that, given p ∈ A, q ∈ B such that p ≤ q, we can find n such that p ≤ n ≤ q and z n = 0, using the fact that z n+1z n ∈ {0, ±1} for all n.

Proof of Theorem 2. By Theorem 1, the strong coincidence condition holds, which implies that (X σ , Σ, µ) is measurably conjugate to (R, E, λ) via ϕ by Lemma 2.3 and Proposition 2.5 (3). But by Lemma 4.1 we even have that (X σ , Σ, µ) is measurably conjugate to (1 ⊥ /Λ, +π u,1 e 1 , λ) via ϕ, with 1 ⊥ /Λ ∼ = S 1 .

Proof of Theorem 3. In view of Theorem 2, this is an immediate consequence of Lemma 2.4 and [BST14, Theorem 2].


Duc-Canh Nguyen

Gérard Bailly

Frédéric Elisei

Conducting neuropsychological tests with a humanoid robot: design and evaluation

Keywords: socially assistive robot, humanoid robot, multimodal behavior, subjective evaluation

Socially assistive robot with interactive behavioral capability have been improving quality of life for a wide range of users by taking care of elderlies, training individuals with cognitive disabilities or physical rehabilitation, etc. While the interactive behavioral policies of most systems are scripted, we discuss here key features of a new methodology that enables professional caregivers to teach a socially assistive robot (SAR) how to perform the assistive tasks while giving proper instructions, demonstrations and feedbacks. We describe here how socio-communicative gesture controllers -which actually control the speech, the facial displays and hand gestures of our iCub robot -are driven by multimodal events captured on a professional human demonstrator performing a neuropsychological interview. Furthermore, we propose an original online evaluation method for rating the multimodal interactive behaviors of the SAR and show how such a method can help designers to identify the faulty events.

I. INTRODUCTION

A. A socially assistive rotot

A socially assistive robot is an assistive robot aiding people through social interactive behaviors rather than physical interaction [START_REF] Hansen | Continuous assessment of time-varying speech quality[END_REF]. Several socially assistive robot (SAR) systems have been proposed and designed to engage people into various interactive exercises such as physical training [START_REF] Fasola | A socially assistive robot exercise coach for the elderly[END_REF], neuropsychological rehabilitation [START_REF] Cabibihan | Why Robots? A survey on the roles and benefits of social robots in the therapy of children with autism[END_REF] or cognitive assistance [START_REF] Pollack | Autominder: An intelligent cognitive orthotic system for people with memory impairment[END_REF]. Depending on the objectives of the Human-Robot Interaction (HRI), SAR faces different challenges. One of the important dimensions is the length of this interaction. Several SAR are concerned with long-term interaction, aiming at providing a single user with social glue and affective relations. Paro [START_REF] Wada | Psychological and social effects of one year robot assisted activity on elderly people at a health service facility for the aged[END_REF] is emblematic of that challenge (see Leite et al [START_REF] Leite | Social robots for long-term interaction: a survey[END_REF] for a review). These companion robots often play the role of pets or majordomos. In contrast, several SAR have also been designed to engage into short-term task-oriented interactions. The challenge is here more oriented towards attention and quick adaptation.

Our works focus on the development of sociocommunicative abilities of a SAR for short-term interactions. Particularly, in this paper, we present the SOMBRERO framework which aims at providing a humanoid robot with multimodal interactive behaviors -such as speech, gaze arm gestures, etc. -in order to perform a neuropsychological test, demonstrated by professionals.

B. SOMBRERO framework

The three main steps of learning interaction by demonstration are given in Figure 1: we should (1) collect representative interactive behaviors from human coaches; [START_REF] Cabibihan | Why Robots? A survey on the roles and benefits of social robots in the therapy of children with autism[END_REF] build comprehensive models of these overt behaviors and a priori knowledge (task & user model, etc.); and then [START_REF] Pollack | Autominder: An intelligent cognitive orthotic system for people with memory impairment[END_REF] provide the target robot with appropriate gesture controllers to execute the desired behaviors.

Most interaction models embedded into HRI systems are strongly inspired by Human-Human Interactions (HHI), if not entirely trained on HHI data. This HHI-based framework faces several problems: (1) the scaling of the human model to the interaction capabilities of the robots in terms of physical limitations (degrees of freedom) and perception, action and reasoning; (2) the drastic changes of human behaviors in front of robots or virtual agents [START_REF] Walters | Avoiding the uncanny valley: robot appearance, personality and consistency of behavior in an attention-seeking home scenario for a robot companion[END_REF] embodiment. HRI is thus bootstrapped by a robot-mediated HHI, which may be broadly considered as a cognitive infocommunication system (see section IV).

The immersive teleoperation of the gaze and lip movements of our iCub robot Nina is described in Guillermo et al [START_REF] Guillermo | Qualitative assesment of a beaming environment for collaborative professional activities[END_REF]. This technique artificially provides the SAR with cognitivenotably social -skills that are intrinsically adapted to the robot's sensorimotor abilities. Moreover subjects may have the impression to interact with a truly autonomous SAR and thus provide the interaction model with genuine behavioral data.

The third issue -i.e. learning-based modelling of interactive behaviors -is an emerging concept (see pioneer work performed by Otsuka et al [START_REF] Otsuka | A probabilistic inference of multiparty-conversation structure based on Markov-switching models of gaze patterns, head directions, and utterances[END_REF] or more recently [START_REF] Huang | Learning-based modeling of multimodal behaviors for humanlike robots[END_REF]). While learning by demonstration [START_REF] Calinon | Learning and reproduction of gestures by imitation: An approach based on Hidden Markov Model and Gaussian Mixture Regression[END_REF] has been quite effective for sensorimotor tasks such as walking or grasping where the robot interacts with the physical environment, its extension to HRI presupposes HRI data that are difficult to acquire. We proposed [START_REF] Mihoub | Learning multimodal behavioral models for face-to-face social interaction[END_REF] [START_REF] Mihoub | Graphical models for social behavior modeling in face-to face interaction[END_REF] to train statistical behavioral models that encapsulate discrete multimodal events performed by the interlocutors into a unique dynamical system that could be further used to monitor behaviors of one interlocutor and generate behaviors of the other.

We address here the fourth issue i.e. the replay and assessment of interactive behaviors by the robot.

II. THE CURRENT CONTRIBUTION

We should in fact verify that the multimodal behaviors planned by the interaction model can effectively be reproduced by the target robot and that these multimodal behaviors are perceived as adequate by human interlocutors.

A. The scenario

Our interviews are based on the French adaptation [START_REF] Van Der Linden | L'épreuve de rappel libre / rappel indicé à 16 items (RL/RI-16)[END_REF] of the Selective Reminding Test [START_REF] Grober | Genuine memory deficits in dementia[END_REF] named the RL/RI 16. It provides a simple and clinically useful verbal memory test for identifying loss of episodic memory in the elderly.

The RL/RI 16 protocol consists in four phases. The first one is the progressive learning of 16 words together with their semantic categories four by four. This phase includes two main tasks: (1) item identification (the interviewer displays four items and asks the subject spell out each name by giving its category); (2) immediate recall of items (the subject should recall each item by its category while the items are hidden). The second phase is three successive recall tasks (i.e. free recall, complemented by an indexed-by-category recall for the unrecovered items) separated with a distractive task (reverse counting). In the next, there is a recognition task involving the 16 items, 32 distractors (16 different words with the same semantic category and 16 true distractors) and (4) a delayed free and indexed recall (not administrated in the present study). Mnesic performance is evaluated by comparing recall rates of the subject with regards to mean & standard deviations observed within sane control population of the same age interval.

Most professionals use folders with sheet of papers. In order to avoid complex dexterous gestures to be performed by the SAR, we adopted a modified scenario using two tablets: one tablet facing the robot to score the subject's answers and the other tablet facing the subject to display visual stimuli i.e. items to be learned or recognized.

B. Interactive data

The demonstrations used here have been performed by a female professional neuropsychologist. Since the immersive teleoperation of the upper body (notably of the arms) is not available yet, the discrete multimodal events have been collected via semi-automatic labeling of HHI. The motion of 25 reflective markers placed on the plexus, shoulders, head, arms, indexes and thumbs of the professional interviewer were monitored thanks to a Qualysis® system with 4 cameras. A Pertech® head-mounted monocular eye tracker also monitors the gaze of the interviewer (see Figure 2). Speech data are captured via OKMII high-quality ear microphones and are recorded synchronously with a side-view video by HD camera.

Each interview lasts around 20", comprising the collection of personal records, the core RL/RI protocol and final report of performance. We analyze here a total two hours of multimodal data for five subjects, interacting with the professional interviewer we used as unique demonstrator of the appropriate socio-communicative behaviors for that particular task.

C. Gestural scores

Elan [START_REF] Hellwig | EUDICO Linguistic Annotator (ELAN) Version 2.0.2 manual[END_REF] (see Figure 3) and Praat [START_REF] Boersma | Praat, a System for doing Phonetics by Computer[END_REF] were used to semiautomatically identify speech, gaze and arm events. The behavioral model has to orchestrate these events according to the different sub-tasks and should be able to generate motor actions from percepts. Modality-specific gesture controllers have then to reproduce final motions from these discrete motor events. Our hypothesis is that the compositional richness of such discrete events is able to capture the diversity of multimodal behaviors.

D. Gesture controllers

Speech. We transcribed speech and aligned its phonetic content with the acoustic signals uttered by both the interviewer and the subjects. We mainly spot items in the subject's speech in order to trigger scoring and interviewer's feedbacks. The interviewer's speech was analyzed more indepth with a special attention to prosody and in particular to backchannels [START_REF] Bailly | Quantitative analysis of backchannels uttered by an interviewer during neuropsychological tests[END_REF]. The orthographic transcription of her discourse augmented with prosodic markers and breath noises is then played by the audiovisual text-to-speech synthesizer controlling Nina's loudspeaker and facial movements [START_REF] Parmiggiani | Design and validation of a talking face for the iCub[END_REF].

Arm gestures. While the human interviewer was displaying word items and scoring using sheets of paper, we decided to use tablets to display items and pretend to trigger the display and take notes (see Figure 4). In fact, subjects project human skills and capabilities onto agents -including mnesic capabilities -and would be very disturbed if the artificial interviewer does not take any notes despite the fact that its processing power does not require such a physical display. Such a behavior is thus clearly imposed by social rules.

Arm displacements and finger clicks are programmed to trigger display on the subject's tablet (show/hide items) and take notes (monitor correct responses). The arm gesture controller uses the iCub Cartesian Interface [START_REF] Pattacini | An experimental evaluation of a novel minimum-jerk cartesian controller for humanoid robots[END_REF], which enables the control of the robot's arm directly on operational space by providing the desired position and orientation of one endeffector (here the index finger of the right hand). The arm controller also provides task-specific movements: preparing to click, clicking, and going back to rest position. Figure 4 illustrates the position of robot's right arm while scoring and resting. In the experiment, the left arm remains fixed. It holds the scoring tablet, while the right arm movements are adapted so as to follow the timing of the writing gestures of the human interviewer. The gaze gesture controller uses the iCub gaze controller [START_REF] Roncone | A Cartesian 6-DoF Gaze Controller for Humanoid Robots[END_REF], which provides direct control of saccades, fixations and smooth pursuit while implementing the binoccular vergence, the oculo-collic and vestibule-ocular reflexes. These gestures can be performed by a parametrized combination of neck and eyes movements. For simplicity, the Cartesian gaze controller is provided with the 3D position of the current region of interest and a fixed contribution of neck movements of 50%. Figure 5 presents robot in two positions: (a) look at subject's face, (b) look at scoring tablet.

Eyelids. Although we did not track eyelids' movements, we developed a specific eyelids gesture controller in order to provide Nina's behavior with additional socio-communicative cues such as blinking as well as redundant cues such as the coupling of eyelids aperture with eyes elevation [START_REF] Foerster | Impact of iris size and eyelids coupling on the estimation of the gaze direction of a robotic talking head by human viewers[END_REF] [22] and speech articulation [START_REF] Badin | Three-dimensional linear articulatory modeling of tongue, lips and face based on MRI and video images[END_REF]. Figure 6 illustrates the coupling of eyelids aperture with eyes elevation.

III. EVALUATION

These complex and coordinated behaviors should be perceived and interpreted correctly by subjects. We have shown that the morphology and appearance of effectors can strongly impair the perception of planned gestures [START_REF] Grober | Genuine memory deficits in dementia[END_REF]. We thus ask third parties to rate the final rendering of our HHI multimodal score by our robotic embodiment in order to check if the reconstructed behavior of our human demonstrator is still relevant and if the mapping between discrete events and gestures are correctly performed by our gestural controllers.

A. State of the art

Most subjective evaluations of HRI behavior have been performed using questionnaires, where subjects or third parties are asked to score specific dimensions of the interaction on Likert scale. For example, Fasola et al [START_REF] Fasola | A socially assistive robot exercise coach for the elderly[END_REF] rated several aspects such as pleasure, interest, satisfaction, entertainment and excitation of a SAR monitoring physical exercises. Huang et al [START_REF] Huang | Learning-based modeling of multimodal behaviors for humanlike robots[END_REF] assessed a narration humanoid robot along several dimensions such as immediacy, naturalness, effectiveness, likability and credibility. Zheng et al [START_REF] Zheng | Comparing two gesture design methods for a humanoid robot: Human motion mapping by an RGB-D sensor and hand-puppeteering[END_REF] compared control strategies for robot arm gestures along dimensions such as intelligibility, likeability, anthropomorphism and safety.

Although delivering very useful information notably for sorting between competing control policies or settings, these questionnaire-based evaluations provide developers with poor information about faulty behaviors: the evaluation is in fact performed off-line and questions mostly address global properties of the entire interaction.

B. Designing and performing an on-line evaluation

In line with online evaluation methods deployed for audio [START_REF] Hansen | Continuous assessment of time-varying speech quality[END_REF] and video [START_REF] Hamberg | Continuous assessment of perceptual image quality[END_REF], we designed an on-line evaluation technique that consists in asking raters to immediately signal faulty behaviors when they observe them. We will use the term "yuck response" used by de Kok [START_REF] Kok | Listening heads[END_REF] for evaluating the adequacy of automatically generated backchannels.

Since raters cannot both experience and rate an interaction, we ask them to put themselves in the place of subjects who have previously experienced the interaction. The technique thus consists in replaying a recorded interaction previously performed by a Guinea pig and asking subjects to rate the adequacy of the SAR's behavior with regards to the Guinea pig's verbal behavior. Note that this on-line evaluation task can also be performed by the Guinea pigs who have previously experienced the interaction. This online evaluation can be performed in front of the robot itself -with the benefit of physical presence but the challenge of coping with the active perception of each rater who may change his/her viewing position -or in front of a video recorded from the Guinea pigs' perspective. We adopt here the latter option, i.e. we filmed the robot replaying the situated interaction using a camera placed approximatively at the mean position of the Guinea pigs' eyes.

We created a website 1 where we ask people to press the "ENTER" key anytime they feel the robot behavior is incorrect. This provides a time-varying probability density function of incorrect behaviors. The maxima of the density function provide when cues, i.e. time-intervals for which a majority of raters estimate the behavior is inappropriate or 1 http://www.gipsa-lab.fr/~duccanh.nguyen/assessment/ hinders the interaction. Further diagnostic of what cues cause these faulty behaviors has to be performed by roboticists and system designers. This on-line evaluation task is preceded by a quick screening of subjects (age, sex and mother tongue) and a familiarization exercise, and followed by a questionnaire that asks the subjects' judgements (five-level Likert) on nine points:

1. Did the robot adapt to the subject? 2. Did the subject adapt to the robot? 3. Did you feel relaxed? 4. Did you feel secure? 5. Was the rhythm of the robot's behavior well adapted? 6. Was the interaction pleasant? 7. Was the multimodal behavior appropriate? 8. Did the robot pay attention while speaking? 9. Did the robot pay attention while listening?

C. Results

We report here the objective and subjective scores of 53 French natives, who performed the entire evaluation. 29 are males and 24 females. The average age of the participants is 32 ±12 years. Figure 7 shows the weighted cumulated yuck responses of our participants. Most maxima of this density function have clear interpretations. We further used Elan (see Figure 8) to associate these major yuck responses with multimodal events. Here are the 25 most signaled events:

• 1, 4, 6 & 7: the robot here performs clicking gestures on its tablet to show or hide items onto the subject's display that was not available to raters. Such ungrounded gestures are thus perceived as distractors by subjects. These yuck responses are located at the beginning of the interview, during the learning phase. • 2, 3, 5, 9, 17, 20, 21, 24 & 25: participants also detected that gaze towards the subject was missing or too much delayed with reference to the interviewee's answers to questions or when delivering instructions. While such a behavior is quite legible when performed by the interviewer -who did not want to interfere with the subject's thoughts -it seems completely unacceptable when performed by a SAR, whose intentions are much less readable. Learning phase Test phase • 10, 11, 12, 13, 16, 18, 19 & 23: the robot remains stillwith the exception of quasi-periodic blinks -for too long, notably during periods of poor interactive activity of the interviewee such as reverse counting or covert thinking. This absence of input observations results in no generated movements. We are currently working on a more lively default listening behavior that will cope with periods of poor external stimulation. • 14 & 15: these particular misbehaviors are explained by the persistence of a large mouth opening well after finishing speaking. This failure is now identified and has been corrected: it was due to a faulty audiovisual segment that was improperly articulated during a silent pause. • 22: In several places, the subject joked and laughed. The lack of SAR response to this strong call for social support during episodes of embarrassment is here rightly penalized by raters. • 8: We did not found any obvious explanation for this particular yuck response in the learning phase.

Figure 9 shows the global subjective judgements performed at the end of the online evaluation. Raters strongly agree that the robot displays a rather decent behavior with regards to the task it is assigned to. This noteworthy evaluation is confirmed by the free comments given before closing each session: nearly all raters declared to be impressed by the overall quality and relevance of the robot's behaviors. Three other criterions also reach a large consensus: the robot pays attention to her interlocutors when speaking, the SAR is perceived as secure and adapts to its interlocutors. Three dimensions receive a lower consensus: attention during listening is impaired by several failures that have been identified and commented above while the interaction is judged not so relaxed and pleasant. We also need to work out the speed: several raters reported that speech intelligibility would have been improved by a lower speech rate at the beginning of the interview.

In the free comments, several raters mention the rather directive style of our female interviewer and the absence of emotional vocal and facial displays on our SAR -e.g. laughs and smiles. We plan to add such segments in our audiovisual speech repository in complement to breath noises and humming. We will also urge the design of articulated eyebrows.

IV. COGNITIVE ROBOTICS AND

INFOCOMMUBNICATIONS Experienced professional pilots here transfer their taskspecific socio-communicative skills to a SAR by mediating their usual HHI practice though a robotic embodiment. No doubt that this mediation strongly affects behaviors of pilots and subjects. We face here a challenging situation where pilots have to co-evolve both with SAR expressive and performative capabilities and with the adaptation of their human interlocutors to the affordances of the artificial agent. These themes have a strong relevance to the field of cognitive infocommunications, notably because cognitive robots are right at the crossroads of technology and humankind: pilots, subjects and SAR build an unprecedented ecosystem that might benefit from a long-term co-evolution in order to ease short-term interactions.

V. CONCLUSIONS We here gave an overview of the SOMBRERO framework for collecting, modeling, controlling and evaluating SAR. All the building blocks are almost operational and have been evaluated separately. We notably put forward an original framework for the online evaluation of interactive behavior that offers subsequent glass-box assessment: post-hoc reverse engineering should be performed by the SAR designers to identify the potential causes of the most consensual yuck responses. We will verify that the correction are well accepted and do not generate new errors.

One of the major missing features of our current control policy concern head movements that are now just contributing to the gaze direction of the SAR, i.e. the head remains still is no gaze shifts are programmed. In the same vein as our eyelids' controller, we plan to combine contributions of head movements to gaze with their co-verbal contributions -notably as encoders of audiovisual prosody.

In the mid-term, we plan to conduct robot-mediated HHI using immersive teleoperation very soon and see what parts of this framework should be corrected. One of the key challenges is the system's adaptation. Mihoub et al [START_REF] Mihoub | Learning multimodal behavioral models for face-to-face social interaction[END_REF] have shown that a subject-independent gaze model may be parameterized to adapt to specific social profiles. We will see if this approach scales to multimodal behavior planning and control.

Figure 1 :

 1 Figure 1: The main steps of learning interaction by demonstration

Figure 2 .

 2 Figure 2. Visual data. Left: head-related view from the eyetracker scene camera. The marker superimposed to the scene camera features the current gaze fixation point. Right: side view from a fixed HD camera.

Figure 3 .

 3 Figure 3. Labeling gaze & speech events with Elan.

Figure 4 .

 4 Figure 4. Robot's arm while (left) scoring and (right) resting.Gaze. We distinguish three main regions of interest of the interviewer's gaze: (1) the subject's face; (2) the scoring tablet (i.e. the scoring sheet and chronometer used in the original HHI); (3) the subject's tablet (i.e. the notebook used in HHI demonstrations). Note that all arm gestures are performed with visuomotor supervision: since robot motion is often slower than human motion, all arm motions are preceded by one fixation towards the target if any and accompanied by gaze smooth pursuit till completion. This visuomotor supervision supersedes the original fixation patterns.

Figure 5 .

 5 Figure 5. Robot's gaze looking at subject's face and scoring tablet.

Figure 6 .

 6 Figure 6. Robot's eyelids with gaze looking down (left) or straight (right).

Figure 7 .

 7 Figure 7. Density of yuck responses for our replayed interaction. Each yuck response is weighted by a Hanning window of 5s in order to smooth the density of responses and overlapped/added to the others. Maxima of this time-dependent histogram reveal multimodal behaviours that are judged inadequate by a majority of raters. The 25 main maxima are commented in the text.

Figure 9 .

 9 Figure 9. Global subjective judgements.

Figure 8 .

 8 Figure 8. Using ELAN [15] to diagnose the possible causes of major yuck responses.

VI. ACKNOWLEDGEMENTS

This research is supported by the ANR (ANR-14-CE27-0014). We thank Ghatsan Hasan for taking care of Nina, Alessandra Juphard for conducting the neuropsychological tests, our five elderly subjects and the crowdsourced raters for their patience and involvement.

VII. APPENDIX

The multimodal data and label files are freely available at: www.gipsa-lab.fr/projet/SOMBRERO/data