Alain Lecomte

Myriam Quatrini

Ludics and Its Applications to Natural Language Semantics

Proofs in Ludics, have an interpretation provided by their counter-proofs, that is the objects they interact with. We shall follow the same idea by proposing that sentence meanings are given by the counter-meanings they are opposed to in a dialectical interaction. In this aim, we shall develop many concepts of Ludics like designs (which generalize proofs), cut-nets, orthogonality and behaviours (that is sets of designs which are equal to their bi-orthogonal). Behaviours give statements their interactive meaning. Such a conception may be viewed at the intersection between prooftheoretic and game-theoretical accounts of semantics, but it enlarges them by allowing to deal with possibly infinite processes instead of getting stuck to an atomic level when decomposing a formula.

Meanings, Proofs and Games

The dominant trend in Natural Language Semantics is based on Frege's conceptions on Logics and Language according to which the meaning of a sentence may be expressed in terms of its truth conditions. There is however an alternative conception according to which we don't find meanings in truth conditions but in proofs, particularly expressed by the Brouwer-Heyting-Kolmogorov (BHK) -interpretation. This conception has been used in philosophy, linguistics and mathematics. In Natural Language Semantics, it has been for instance developed by Martin-Löf, Sundholm 86,Ranta 94]), but this framework is limited because proofs are finite objects. At a certain stage of the proof of a formula, atomic formulae are obtained, but what is the proof of an atomic formula? Actually, we expect a proof of a sentence to be an object of the same symbolic nature as the sentence. There is no way to escape from language or from mind to directly reach the external world. Other similar attempts to provide a foundation for meaning in natural language are based on works by Hintikka,). In their interpretation, meanings are provided by strategies in a language game. Their views meet Wittgenstein's according to which meaning is use and the use of language is showed in language games. But still those accounts meet difficulties when dealing with atomic sentences: in this case, the logician is obliged to refer to some model (in the traditional sense of Model Theory) in order to evaluate the truth value of an atomic sentence. Still more seriously, they only take in consideration games which are of a very particular kind: they are oriented towards the notions of winner, winning strategy, score and pay-off function, contrarily to what Wittgenstein suggested in his Philosophische Untersuchungen when he spoke of games for a very large family (even mere pastimes). Neither Wittgenstein's games do refer to a priori rules which would be attached, like in GTS, with logical particles (cf. [Pietarinen 07]). Finally, none of these alternatives to the truth-conditions based framework ever envisaged to take proofs or games as infinite devices (or partial and underspecified ones) and of course none of these traditions took into account the fact that proofs and strategies can be the same objects, simply viewed from different angles. If this concerns formalized theories of meaning, what to say of theories of meaning which have not been formalized, like that of argumentative meaning, in O. Ducrot's sense ( [Ducrot 1984]). Ducrot pointed out the so-called polyphonic aspect of language, that is the fact that utterrances are not simple statements which are confronted with "reality", but dynamical processes which are oriented towards possible or impossible continuations (for instance I have a few books cannot be pursued by * and even none, while He read few books may be). In the same way, dialogues may be studied according to what utterrance may be an appropriate reply to another one, and what may not be. In this paper, we shall present some applications of Ludics to these topics. In a nutshell, proofs in Ludics, have an interpretation provided by their counter-proofs, that is the objects they interact with. We shall follow the same idea by proposing that sentence meanings are given by the counter-meanings they are opposed to in a dialectical interaction.

Dialogues and Ludics

Ludics : a Theory of Interaction

Ludics can be sum up as an interaction theory, formulated by J.-Y. Girard ([Girard 01]) as the issue of several changes of paradigms in Proof Theory : from provability to computation, then from computation to interaction. The first change of paradigm arose with the intuitionnistic logic, while the second was due to the development of linear logic. Starting from a geometrical viewpoint on proofs, which provided an internal approach to the dynamics of proofs, Ludics takes the notion of interaction (that is the cut rule and its process of elimination) as primitive. Therefore, it simply starts from loci, or adresses (where interaction can take place) and formulae are given up, at least for a while, since the challenge is to regain them at the output of the construction. Protoformulae are used as mere scaffoldings for building the main objects we shall deal with (the designs). The central object of Ludics: the design Using the metaphor of Games, a design can be understood as a strategy, i.e. as a set of plays (or chronicles) ending by answers of Proponent against the moves planned by Opponent. The plays are alternated sequences of moves (actions). A move is defined as a 3-uple consisting in -a polarity (positive for Proponent, negative for Opponent), -an adress or locus, coded by a finite sequence of integers (denoted by ξ, ρ, σ . . . ), where the move is said to be anchored, -a finite set of integers, or ramification which indicates the positions which can be reached in one step.

A unusual positive move is also possible : the daïmon, which may end up a play.

Positions are organized in forks, which are presented under the general form: Γ ⊢ ∆; where Γ and ∆ are finite sets of loci such that Γ is either the empty set or a singleton. The fork corresponding to the starting position is called the base of the design. When Γ is not empty, the following (opponent) move starts from the only element it contains, and the fork is said to be negative, in the other case, Proponent chooses the locus in ∆ from where it starts and the fork is said to be positive. Perhaps this may seem not new with regards to GTS, let us notice however that moves are defined abstractly, independently from any particular connective or quantifier, and that at each step, the whole history of the previous moves is available. Now, more importantly, a design may be also seen as a proof search in some linear formal system, according to the following methodological choices:

-the object we are building not only provides a proof, but at the same time, contributes to the determination of the formula which is proved. Loci point out the position where such a formula could be located, and at the same time, the "logical" decomposition this formula could have, -by means of the focalisation property discovered by Andréoli [Andréoli 92] according to which in linear logic, it is always possible to draw a proof by following a strict discipline (focusing) which amounts to grouping together successive blocks of rule applications of the same polarity, it is possible to have only two rules (one positive and one negative).

-it may happen that the research be not successful. In this case, one may give up the proof search, thus using a specific non logical rule (or paralogism) : the daïmon rule.

A design can therefore be represented by a tree of forks, built by means of three rules :

Daïmon † ⊢ Λ Negative rule ... ⊢ ξ ⋆ J, Λ J ... (-, ξ, N ) ξ ⊢ Λ Positive rule ... ξ ⋆ i ⊢ Λ i ... (+, ξ, I) ⊢ ξ, Λ
where I and J are finite subsets of N, i ∈ I, with the Λ i pairwise disjoints, N is a set (possibly infinite) of finite subsets of N, each J of the negative rule being an element of this set, all the Λ J are included in Λ, and moreover each base sequent is well formed in the sense that all addresses are pairwise disjoint.

The Fax Since we have not yet introduced formulae, there is no opportunity to use axiom-links. Instead, we will have a particular design based on a fork ξ ⊢ ξ ′ . Roughly speaking, this design ensures that both loci ξ and ξ ′ could be locations of a same formula. That means that as soon as a logical decomposition may be handled on the right hand side, the same may also be handled on the left hand side. Such a design, called F ax, is recursively defined as follows:

F ax ξ,ξ ′ = ... ... F ax ξ ′ i ,ξi ξ ′ ⋆ i ⊢ ξ ⋆ i ... (+, ξ ′ , J) ⊢ ξ ⋆ J, ξ ′ ... (-, ξ, P f (N)) ξ ⊢ ξ ′
At the first (negative) step, the negative locus is distributed over all the finite subsets of N, then for each set of addresses (relative to some J), the positive locus ξ ′ is chosen and gives rise to a subaddress ξ ′ ⋆ i for each i ∈ J k , and the same machinery is relaunched for the new loci obtained.

Defining interaction

Interaction is concretely expressed by a coincidence of two loci in dual position in the bases of two designs. This creates a dynamics of rewriting of the cut-net of the two designs, called, as usual, normalisation. We sum up this process as follows: the cut link is duplicated and propagates over all immediate subloci of the initial cut locus as long as the action anchored on the positive fork containing the cut-locus corresponds to one of the actions anchored on the negative one. The process terminates either when the positive action anchored on the positive cut-fork is the daïmon, in which case we obtain a design with the same base as the starting cut-net, or when it happens that in fact, no negative action corresponds to the positive one. In the later case, the process fails (or diverges). The process may not terminate since designs are not necessarily finite objects. When the normalization between two designs D and E (respectively based on ⊢ ξ and ξ ⊢) succeeds, the designs are said to be orthogonal, and we note: D ⊥ E. In this case, normalization ends up on the particular design :

[ †] ⊢ Let D be a design, D ⊥ denotes the set of all its orthogonal designs. It is then possible to compare two designs according to their counter-designs. We set D ≺ E when D ⊥ ⊂ E ⊥ .

The separation theorem [Girard 01] ensures that this relation of preorder is an order, so that a design is exactly defined by its orthogonal.

Behaviours One of the main virtues of this "deconstruction" is to help us rebuilding Logic.

-Formulas are now some sets of designs. They are exactly those which are closed (or stable) by interaction, that is those which are equal to their bi-orthogonal. Technically, they are called behaviours. -The usual connectives of Linear Logic are then recoverable, with the very nice property of internal completeness. That is : the bi-closure is useless for all linear connectives. For example, every design in a behaviour C ⊕ D may be obtained by taking either a design in C or a design in D. -Finally, proofs will be now designs satisfiying some properties, in particular that of not using the daïmon rule.

Ludics as a Formal Framework for Dialogues

Concerning dialogues, let us focalize on the mere supports of the interaction. That is the locus where a speech turn is anchored (among the loci previously created) and the loci that it creates, which are also those which may be used later on.

Because Ludics may display the history of the dialogue by means of chronicles, and it takes into account the strategies of any speaker by means of designs, it allows us to see a dialogue as the result of an interaction between the strategies of two speakers. In that case, the rules have the following interpretation:

-when being active (that is using a positive rule), a speaker chooses a locus and therefore has an active role, -when being negative (that is using a negative rule), s/he has no choice and has a passive role If, therefore, positive steps are understood as moves where the intervener asks a question or makes an assertion, and negative steps as moves where s/he is apparently passive, recording an assertion and planning a further reply, positive actions of one speaker are not opposed to positive actions of the other one (as it is the case in most formal accounts of dialogue, even the logical ones) but to negative ones of the other. This point meets an important requirement formulated by Ducrot according to whom "the semantic value of an utterrance is built by allusion to the possibility of another utterrance (the utterrance of the Other speaker)".

Examples -

The following example is deliberately simple, and only given for a pedagogic purpose.

Let us consider the following dialogue between Annie and Barbara: A : did you meet some friends yesterday evening to the party ? B : I only saw Bruno and Pierre. A : Was Pierre still as nice as during the last year ? B : Yes, he did. A : That is what I wanted to know. Such an exchange is represented by an interaction between two designs : one is seen from the point of view of A and the other from the point of view of B:

From A's point of view From B's point of view † ⊢ 0.1.1.1.1 0.1.1.1.1 ⊢ 0.1.1.1 ⊢ ⊢ 0.1.1.1 ⊢ 0.1.1, 0.1.2 0.1.1 ⊢ 0.1.2 ⊢ 0.1 ⊢ ⊢ 0.1 ⊢ 0 0 ⊢
The trace of the interaction (the cut between the two loci 0) is the alternated sequence of actions:

(+, 0, {1})(-, 0.1, {1, 2})(+, 0.1.1, {1})(-, 0.1.1.1, {1}) †.
In this case the normalisation ends up on the daïmon. The interaction converges.

-The second example is taken from Schopenhauer's "Dialectica eristica" (or "The Art of Always Being Right") which provides a series of so-called stratagems in order to be always right in a debate. It formalizes the first given stratagem. "I asserted that the English were excellent in drama. My opponent attempted to give an instance of the contrary, and replied that it was a well-known fact that in opera, they were bad. I repelled the attack by reminding him that, for me, dramatic art only covered tragedy and comedy ...."

We give an account of this dialogue by the following interaction:

⊢ ξ.1.1 ⊢ ξ.1.2 C ξ.1 ⊢ A ⊢ ξ ξ.1.3 ⊢ B ⊢ ξ.1 ξ ⊢
Where the action A corresponds with the claim: "The English are excellent in drama" ; the action B with "I disagree, it is a well-known fact that in opera, they could do nothing at all." and the action C with "But by dramatic art, I only mean tragedy and comedy."

Of course, the net built with these two designs does not converge. In fact, things don't happen this way: initially, the set of loci the first speaker has in mind could also cover opera. What happens when willing to repel the attack is retracting one branch (or replay the game according to a different strategy). This leads us to enter more deeply into the decomposition of dialogues and in what we consider as units of action.

While, at the most elementary level, which is relevant as long as the dialogues we consider are simple (for instance exchanges of information), the interaction is between elementary actions, those elementary actions are replaced by (sub)-designs as soon as we are concerned by dialogues of a more complex nature like controversies.

-A third example comes from Aristotle's Sophistical Refutations, where it is given the name multiple questions.

Let us imagine a judge asking a man the question: "Do you still beat your father ?".

The judge asks a question that presupposes something that has not necessarily been accepted by the man. S/he imposes to him the following implicit exchange:

-"Do you beat your father?" -" Yes" -" Do you stop beating him ?". This exchange between the judge J and the man D must be represented by the following interaction :

ξ.0.1.0 ⊢ ⊢ ξ.0.1 ⊢ ξ.0.2 ξ.0 ⊢ ⊢ ξ ⊢ ξ.0.1.0 ξ.0.1 ⊢ ⊢ ξ.0 ξ ⊢ J D
In fact, the judge utterance: -"Do you still beat your father ?" contains what we call nowadays a presupposition. It can't therefore be represented by a single action, but by the whole chronicle: (+, ξ, {0}) (-, ξ.0, {1}) (+, ξ.0.1, {0}). This enables us to give an account of the fact that one of the loci where the interaction might continue is in fact not available ; in some sense the action giving this possibility is skipped, some successive one is immediately proposed and, by this way, constrains the answers.

The ludical approach thus allows us to get a formalized conception of stratagems and fallacies, something which appeared out of reach for many researchers (see for instance [Hamblin 70]). Moreover, we claim that it could improve some issues in formal semantics, like we try to show it in the following section.

Logical Forms and Ludics

In the sequel, we propose a conception of interactive meaning based on Ludics. In the same way a design is defined by its orthogonal (according to the separation theorem), we may postulate that the meaning of a sentence is given by the set of all its dual sentences: that is all the sentences with which the interaction converges. For this purpose, we associate a behaviour or a family of behaviours with a sentence. Such behaviours are built in a compositional way, like in standard formal semantics, but their ultimate components are neither atoms nor atomic formulae, like in the Intensional Logic Montague was using. Let us underline the points which are slightly different and new and which could favourably extend the standard models of semantics:

-The fact that the mathematical object associated with the meaning of a sentence may be more and more refined seems to us very important. Such an objective is realized because of the order on designs involved by the separation theorem, which enables one to explore more and more precisely the argumentative potential of a sentence. Moreover, new designs may always be added to such an object, thus enlarging our conception of meaning. -The fact that Ludics strictly encompasses logic and that logical concepts like formulas, proofs or connectives are defined in a world which is larger than the strictly logical one (let us remember that we have paralogisms like the daïmon, and counter-proofs in that world!) makes us to expect more freedom in defining "logical" forms. For instance it may be the case that behaviours are composed by means of a non-logical operator (but which could nevertheless be interpreted).

The following example illustrates a classical problem of ambiguty (scope ambiguity).

Meaning through Dual Sentences

The meaning of a sentence is given by all the utterances which correctly interact (that means : converge) with it.

Let us consider the statement (from now on denoted by S): "Every linguist speaks some african language". Usually two logical forms can be associated with such a sentence S, depending on whether some has the narrow or the wide scope. Namely: S 1 = ∀x(L(x) ⇒ ∃y(A(y) ∧ P (x, y))) S 2 = ∃y(A(y) ∧ ∀x(L(x) ⇒ P (x, y))) where L(x) means "x is a linguist" , A(y) means "y is an african language" and P (x, y) means "x speaks y". When "some" has the narrow scope, we assume that the logical form converges with the LF of sentences like:

(1) There is a linguist who does not know any african language.

(2) Does even John, who is a linguist, speak an african language ?

(3) Which is the African language spoken by John ? On the opposite, if "some" has the wide scope, the logical form converges with :

(4) There is no african language which is spoken by all the linguists.

(5) Which african language every linguist speaks ?

Meaning as a Set of Justifications

We materialize the claim according to which meaning is equated with a set of dual sentences by associating with the meaning of S a set of designs. Such designs may be seen as justifications of S. That is the supports of the dialogues during which a speaker P asserts and justifies the statement S against an adressee O who has several tests at his/her disposal.

Let us make such a design, based on the arbritary fork ⊢ 0, more precise:

-the first action corresponds to the assertion of S. Its ramification is a singleton ; only one locus is created for continuing the interaction. Nevertheless, the speaker who has to anticipate the reactions of his/her adressee is committed to one of the readings of S. S/he is ready to assume one of the two possibilities, the wide or the narrow scope for "some". This is taken into account by distinguishing between two possible first actions, that we symbolize for instance by (+, 0, {0}) and (+, 0, {1}). It is then possible to distinguish between two kinds of designs, considered as justifications of S according to the choice of the first action. -let us for instance focus on the first reading of S. We then simulate an interaction between P and O who tries to negate P 's claim:

P O D d ′ . . . 0.0.2 d .1e ⊢ 0.0.2 d .2e ⊢ 3 ⊢ 0.0.1 d , 0.0.2 d D d ′′ . . . 2 0.0 ⊢ 1 ⊢ 0 0.0.1 d ⊢ E e ′ . . . † 4 ⊢ 0.0.2 d .1e, 0.0.2 d .2e E e ′′ . . . 3 ′ 0.0.2 d ⊢ 2 ′ ⊢ 0.0 1 ′ 0 ⊢
The normalisation stages may be commented as follows:

1 P asserts S and is ready to continue the interaction with the first reading of S 1' O records the claim made by P and is ready to answer it. Notice that if O had been ready to answer according to the second reading, its action would have been (-, 0, {1}) and the interaction would have diverged 2 P is ready to give justifications for any individual : d,d ′ ,. . . 2' O proposes an individual d (arguing that d is a linguist (localized in 0.0.1 d ) and that d doesn't know any african language (localized in 0.0.2 d )) 3 P exhibits some language e (arguing that e is an african language and d speaks e) 3' at the same time, O is ready to receive such a claim by P for some language among e ′ ,e, e ′′ . . . 4 if P has given some language e such that d speaks it, O may be ready to give up. Thus, the interaction between "Every linguist speaks some african language" and the attempt to negate it "There is some linguist which doesn't speak any african language" normalizes. Let us denote by D the foregoing design of P . We could also find another design as justification of S with its first reading : P may ask to check if d is really a linguist, O may ask to check if d really speaks e and so on thus providing a deeper interaction. Further exchanges may enter into debates on what it means for a person to be a linguist, or on what it means for a language to be an african one, or on what it means for a person and a language to be such that the person speaks the language and so on... In any way, if S 1 denotes the set of designs representing the first reading of S and if S 2 denotes the set of designs representing the second one, the set of designs representing the meaning of S is the union of both sets : S = S 1 ∪ S 2 .

Meaning as Behaviour

The previous attempt to associate a set of design with the meaning of S is still general and imprecise. D actually belongs to the following behaviour3 :

∀x(↓ L(x) -• ∃y(↓ A(y)⊗ ↓ P(x, y)))
provided that L(x) , A(y) and P(x, y))) are behaviours. Indeed, following the correspondance between designs and proofs of the hypersequentialized polarised linear logic H which is given in the annex, the design D may be seen as an attempt to prove the formula S = S 1 ⊕ S 2 where S 1 and S 2 are the (proto) -formulas associated with the first and second reading of S in their linear and hypersequentialized formulations :

D d ′ . . . ↓ A ⊥ (e d ) ⊢ ↓ P ⊥ (d, e d ) ⊢ ⊢↓ L ⊥ (d), ∃y(↑ A(y)⊗ ↑ P (d, y)) D d ′′ . . . (∀x(↑ L(x) -• ∃y(↑ A(y)⊗ ↑ P (x, y)))) ⊥ ⊢ ⊢ S
We retrieve the semantical notion of "logical form" but resting on behaviours instead of, simply, logical formulae. There are finally two possible ways to associate a behaviour with a sentence:

-either, we can consider that the design obtained (as in the previous section) as a minimal justification of S may generate a behaviour associated with S. Thus S gen = D ⊥⊥ . -or we can consider that the behaviour associated with S corresponds to the linear formula (in an hypersequentialised formulation):

S = (∀x(↓ L(x) -• ∃y(↓ A(y)⊗ ↓ P(x, y)))) ⊕ ∃y(↓ A(y) ⊗ ∀x ↑ (↓ L(x) -•↓ P(x, y))).
The later interpretation of S's meaning is in fact a family of behaviours because S depends on the behaviours L(x) , A(y) and P(x, y))).

Remark 1 As a logical formula, S is seen as the disjunction of S 1 and S 2 , namely as the formula S = S 1 ⊕ ↓ S 2 , and as a behaviour, seen as the union4 of the two behaviours associated with the two terms of the disjunct. Hence we get a logical account of the fact that interaction may activate only one of both logical sub-formulas, depending on the scope of "some".

Remark 2

The behaviour S gen contains all the behaviours logically built from the behaviours associated with the elemantary pieces L(x), A(y) and P (x, y). This way we get a first (and still rough) account of the logical particles of meaning.

Finally, Ludics enables us to go further into the specification of the logical form.

Decomposing "atomic formulas"

1. It is of course possible to consider the leaves of a decomposition as atomic formulae, if decomposition ends up. In this case, they are seen as data items5 . We can thus consider the following design D ′ as a justification of S : Let us remark that D ′ is more defined than D. In Ludics this means that D ⊥ ⊂ D ′⊥ and this may be understood here that the justification is more informative, more precise. 2. But we may also consider that L(x), A(y) and P (x, y) are still decomposable. That amounts to recognize that S 1 contains other designs: all those which are more defined than D. Designs more defined than D are built on the same schema than D ′ but instead of ending on the the empty set, they continue on non empty ramifications, thus allowing the exploration of A(e f ) or P (f, e f ) which were alleged atomic formulae in the previous designs.

D d ′ . . .
The vericonditional interpretation is here retrieved as an indirect (and secondary) consequence of our "(para)proofs as meanings"6 interpretation because now, D ′ is really a proof provided that A(f ) and P (f, e f ) are either data items, that is the true linear formula 1 or are provable when they are decomposable.

How to go further ?

Towards speech acts -and the use of F ax Instead of simple yes/no questions, where convergence occurs for "yes" and divergence for "no", we may take so called wh-questions into consideration, for example "which is the african language that John speaks ?". In this case we expect that the interaction has the answer as its by-product (or its side effect).

To reach this goal, let us associate with such a question (that we may see as a speech act) a design in which there is a locus for storing the answer. In our formulation of designs as HS-paraproofs, this question will be associated with a paraproof of the sequent S ⊢ A where A is a formula equal to ↑ A 1 ⊕ • • • ⊕ ↑ A n corresponding to the logical form of " is some african language" (afar, peul, ewe, ewondo...). A complex design using F ax will be associated with the question "which is the african language that John speaks ?"

. The result of the interaction of this design with D ′ is :

∅ ⊢ Ae ↓ A ⊥ e ⊢
⊢ A which can be read as " A e is this african language" (where A e is the african language that John speaks (in D ′ )).

In our opinion, this suggests a way to perform in Ludics a unified treatment of Logical Forms and Speech Acts. At the same time, this underlines the richness of the ludical framework to give an account of the interactions in language.

Conclusion

In this paper, we tried to give an account of Ludics and of the new way it allows to specify Meaning in Language: not by considerations on truth conditions but by using the important concept of interaction. To access the meaning of a sentence is mainly to know how to question, to answer to or to refute this sentence, and to know how to extend the discourse (or the dialogue) to which it belongs. In such a conception, the meaning of a sentence is a moment inside an entire process which coud be conceived as infinite (if for instance we admit that the interpretation or the argumentation process with regards to any statement is potentially infinite). Ludics gives a precise form to these views by means of the notions of normalization and behaviour. Otherwise, the emphasis put on loci has, as a valuable consequence, the fact that we may conceive several instances of the same sign (a sentence, a word etc.) as having various meanings, according to the location it has in a discourse or a dialogue, thus giving suggestions for dealing with many rhetorical figures (and fallacies). The infinite design F ax allows to delocate such meanings but its use is not mandatory. Moreover, the fact (not much developed in this extended abstract) that a design may be viewed either as a kind of proof (in a syntactic setting of the framework) or as a game (in a semantic setting of it) provides us with interesting insights on Pragmatics and Wittgensteinian language games. In a pragmatic theory of presupposition, for instance, presupposing implies making an assertion where the hearer has no access to a previous step made by the speaker, if (s)he rejects this step, (s)he makes the process to diverge. Other "games" may be explored. Wittgenstein for instance quoted elicitation, that is the way in which somebody may obtain an answer to a question. Every time, F ax is used to transfer a meaning from a location to another one (for instance from the discourse or the brain of the other speaker to the one of the eliciter). Those games may be envisaged without any kind of "winning strategy". In a speech act seen as a game, there is no win, simply the appropriate use of some designs in order to reach an objective (which may be a common one). Future works will be done in those directions.

Annexe A : A hypersequentialized version of the linear sequent calculus

We give here a short presentation of a hypersequentialized version of linear calculus, which enables one to manipule the designs as (para)proofs of a logical calculus.

Formulas and sequents

By means of polarity, we may simplify the calculus by keeping only positive formulae. Of course, there are still negative formulae... but they are simply put on the left-hand side after they have been changed into their negation. Moreover, in order to make paraproofs to look like sequences of alternate steps (like it is the case in ordinary games), we will make blocks of positive and of negative formulae in such a way that each one is introduced in only one step, thus necessarily using synthetic connectives. Such connectives are still denoted ⊕ and ⊗ but are of various arities. We will distinguish the case where both ⊕ and ⊗ are of arity 1 and denote it ↓.

-The only linear formulae which are considered in such a sequent calculus are built from the set P of linear constants and propositionnal variables according to the following schema :

F = P |(F ⊥ ⊗ • • • ⊗ F ⊥ ) ⊕ • • • ⊕ (F ⊥ ⊗ • • • ⊗ F ⊥ )| ↓ F ⊥
-The sequents are denoted Γ ⊢ ∆ where ∆ is a multiset of formulas and Γ contains at most a formula.

Rules

-There are some axioms (logical and non logical axioms):

P ⊢ P ⊢ 1 ⊢↓ T, ∆ † ⊢ ∆
where P is a propositionnal variable ; 1 and T are the usual linear constants (respectively positive and negative). -The "logical" rules are the following ones :

Negative rule ⊢ A 11 , . . . , A 1n1 , Γ . . . ⊢ A p1 , . . . , A pnp , Γ

(A 11 ⊗ • • • ⊗ A 1n1 ) ⊕ • • • ⊕ (A p1 ⊗ • • • ⊗ A pnp ) ⊢ Γ Positive rule A i1 ⊢ Γ 1 . . . A ini ⊢ Γ p ⊢ (A 11 ⊗ • • • ⊗ A 1n1 ) ⊕ • • • ⊕ (A p1 ⊗ • • • ⊗ A pnp ), Γ
where ∪Γ k ⊂ Γ and for k, l ∈ {1, . . . p} the Γ k ∩ Γ l = ∅.

Remarks on Shifts

Using the shift is a way to break a block of a given polarity. Separate steps may be enforced by using the shift operators ↓ and ↑ which change the negative (resp. positive) polarity into the positive (resp. negative) one. The rules introducing such shifted formulas are particular cases of the positive and the negative one:

A ⊥ ⊢ Γ [+] ⊢↓ A, Γ ⊢ A ⊥ , Γ [-] ↓ A ⊢ Γ
where A is a negative formula. Example In a block like A ⊗ B ⊗ C in principle, A, B and C are negative, but if we don't want to deal with A, B, C simultaneously, we may change the polarity of B ⊗ C (which is positive) and make it negative by means of ↑. We write then A⊗ ↑ (B ⊗ C). Compare the two following partial proofs, where (1) does not use any shifts and (2) uses one : instead of (1):

A ⊥ ⊢ B ⊥ ⊢ C ⊥ ⊢ ⊢ A ⊗ B ⊗ C
we get (2) :

A ⊥ ⊢ B ⊥ ⊢ C ⊥ ⊢ ⊢ B ⊗ C ↓ (B ⊗ C) ⊥ ⊢ ⊢ A⊗ ↑ (B ⊗ C)
We may use the notation ⊕ y (and dually & x ) instead of F y1 ⊕ • • • ⊕ F yn or simply ∃y (dually ∀x) when it is clear in context that y belongs to a finite set.

∅⊢

  A(e d ) ↓ A ⊥ (e d ) ⊢ ∅ ⊢ P (d, e d ) ↓ P ⊥ (d, e d ) ⊢ ⊢↓ L ⊥ (d), ∃y(↑ A(y)⊗ ↑ P (d, y)) D d ′′ . . . (∀x(↑ L(x) -• ∃y(↑ A(y)⊗ ↑ P (x, y)))) ⊥ ⊢

∀ and ∃ are used here because of their intuitive appeal, but in fact they stand for the generalized additives connectives &x and ⊕y (cf. annex). There is nevertheless a slight difference between both pairs of concepts: strictly speaking, in Ludics the correct use of first order quantifiers with regards to mathematical formulas would involve a uniformity property ([Fleury-Quatrini 04]) which is neither relevant nor satised here.

4 This is one of the mains results of Ludics: the internal completeness ensures that the elementary operation of union is enough to obtain all the designs of the disjunction.

in Ludics this is possible by means of the use of the linear multiplicative constant 1

In the opposition of two processes of proof search, both cannot be "real" proofs, it is the reason why we call them paraproofs


Elliot Chane-Sane
email: elliot.chane-sane@inria.fr

Cordelia Schmid

Ivan Laptev

Learning Video-Conditioned Policies for Unseen Manipulation Tasks

The ability to specify robot commands by a nonexpert user is critical for building generalist agents capable of solving a large variety of tasks. One convenient way to specify the intended robot goal is by a video of a person demonstrating the target task. While prior work typically aims to imitate human demonstrations performed in robot environments, here we focus on a more realistic and challenging setup with demonstrations recorded in natural and diverse human environments. We propose Video-conditioned Policy learning (ViP), a data-driven approach that maps human demonstrations of previously unseen tasks to robot manipulation skills. To this end, we learn our policy to generate appropriate actions given current scene observations and a video of the target task. To encourage generalization to new tasks, we avoid particular tasks during training and learn our policy from unlabelled robot trajectories and corresponding robot videos. Both robot and human videos in our framework are represented by video embeddings pre-trained for human action recognition. At test time we first translate human videos to robot videos in the common video embedding space, and then use resulting embeddings to condition our policies. Notably, our approach enables robot control by human demonstrations in a zero-shot manner, i.e., without using robot trajectories paired with human instructions during training. We validate our approach on a set of challenging multi-task robot manipulation environments and outperform state of the art. Our method also demonstrates excellent performance in a new challenging zero-shot setup where no paired data is used during training.

I. INTRODUCTION

Significant progress has been made in recent years towards learning a generalist robot agent capable of accomplishing a wide array of skills across many environments [START_REF] Ahn | Do as i can, not as i say: Grounding language in robotic affordances[END_REF], [START_REF] Jang | Bc-z: Zero-shot task generalization with robotic imitation learning[END_REF], [START_REF] Lu | Aw-opt: Learning robotic skills with imitation andreinforcement at scale[END_REF], [START_REF] Chebotar | Actionable models: Unsupervised offline reinforcement learning of robotic skills[END_REF]. Central to this challenge is the ability to effectively specify tasks and rewards to the robot system in a user-friendly manner. In reinforcement learning, a task is commonly defined through a reward function [START_REF] Sutton | Reinforcement learning: An introduction[END_REF]. However, designing good reward functions for each task is often challenging and restricting policy learning to a fixed set of tasks hinders generalization to new tasks. Goal-conditioned imitation and reinforcement learning [START_REF] Ghosh | Learning to reach goals via iterated supervised learning[END_REF], [START_REF] Lynch | Learning latent plans from play[END_REF], [START_REF] Eysenbach | C-learning: Learning to achieve goals via recursive classification[END_REF], [START_REF] Nair | Visual reinforcement learning with imagined goals[END_REF], [START_REF] Gupta | Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning[END_REF], [START_REF] Pong | Temporal difference models: Model-free deep rl for model-based control[END_REF] can learn agents capable of performing a wide diversity of tasks with less supervision. But providing the right goal to define the task requires expert operators to come up with a suitable robot observation of the desired configuration. Other works have shown that we can learn to command generalist robots through language instructions [START_REF] Jang | Bc-z: Zero-shot task generalization with robotic imitation learning[END_REF], [START_REF] Lynch | Language conditioned imitation learning over unstructured data[END_REF], [START_REF] Nair | Learning language-conditioned robot behavior from offline data and crowdsourced annotation[END_REF], [START_REF] Mees | What matters in language conditioned robotic imitation learning[END_REF] and human videos [START_REF] Jang | Bc-z: Zero-shot task generalization with robotic imitation learning[END_REF], [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF], which are easy to provide for nonexpert operators and can generalize to unseen inputs, including behaviors beyond goal-reaching skills. Furthermore, Fig. 1. Given a human video instruction in a non-robotic scene, our videoconditioned policy ViP controls the robot to perform a similar task zeroshot, i.e. the agent never observes robot data paired with human instructions during training and figures out which manipulation skill it is expected to perform in its environment at test time. We illustrate three examples of different tasks demonstrated by people and the corresponding roll-outs generated by our method for the TableTop robotics environment.

being able to specify robot skills through language or video commands unlocks solving more complex long-horizon tasks by chaining human instructions [START_REF] Ahn | Do as i can, not as i say: Grounding language in robotic affordances[END_REF], [START_REF] Mees | Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks[END_REF]. Nonetheless, these methods rely on annotated demonstrations for a large set of robot skills, which is often tedious to provide, especially since task annotation must be repeated for each new robot environment.

In this work, we propose Video-conditioned Policy learning (ViP), a method that learns to perform manipulation skills given a human video of the desired task in vision-based multi-task robotic environments (Figure 1). We demonstrate that, due to the similarity between robot manipulation and videos of humans performing manipulation skills, we can leverage existing large datasets of annotated human videos, such as the Something-Something-v2 dataset [START_REF] Goyal | The" something something" video database for learning and evaluating visual common sense[END_REF] (SSv2), to learn to map human videos to robot behaviors in a zeroshot manner, without training on paired data between human instructions and robot demonstrations. For instance, the robot may interact in an environment that includes a drawer, yet has received no supervision on what it is expected to do when commanded with a video instruction of a human closing a possibly different drawer. We do so by learning a video encoder using Supervised Contrastive Learning [START_REF] Khosla | Supervised contrastive learning[END_REF], where embeddings of videos of the same task are closer together in cosine distance, and show that video models trained on such large datasets of annotated human videos, which can be easily collected from the internet [START_REF] Goyal | The" something something" video database for learning and evaluating visual common sense[END_REF], [START_REF] Miech | Howto100m: Learning a text-video embedding by watching hundred million narrated video clips[END_REF], [START_REF] Grauman | Ego4d: Around the world in 3,000 hours of egocentric video[END_REF], [START_REF] Damen | Scaling egocentric vision: The epic-kitchens dataset[END_REF], generalize to the robot video domain.

In addition, we show that video encoders trained for human video action recognition readily provide relevant task embeddings for multi-task policy learning. Following recent trends in data-driven robotics, our approach learns from large collections of offline robot experience that can be collected arXiv:2305.06289v1 [cs.RO] 10 May 2023 in different ways, e.g. by expert demonstrations given by motion planners, teleoperated play data, random data generation processes. Given an offline dataset of robot demonstrations, we learn a video conditioned policy to regress the action conditioned on both the robot state and a video embedding of the full trajectory the state-action pair belongs to. We also keep each embedding in a library of robot embeddings. At inference, we perform nearest neighbors regression on this library using the cosine distance to the embedding of the human instruction to generate an appropriate robot embedding that is both (a) relevant to the instruction and (b) is executable by the policy. We can then execute the human video instruction by decoding the selected embedding into a robot trajectory using the learned policy.

Overall, our approach demonstrates that large collections of human videos can enable less supervision in data-driven robotics. Our contributions can be summarizes as follows:

• we designed a method to train a policy conditioned on robot video embeddings given by a video encoder pretrained on a large dataset of annotated human videos; • our method can map human video instructions to robot manipulation skills without supervision from paired data to bridge the gap between the human and robot domain; • our approach outperforms prior works on a set of multitask robotic environments.

II. RELATED WORK

Different methods have been explored in recent years towards robot learning with human videos. Many prior works hence consider the problem of learning to follow human demonstrations using different techniques such as pose or keypoints estimation [START_REF] Peng | Sfv: Reinforcement learning of physical skills from videos[END_REF], [START_REF] Xiong | Learning by watching: Physical imitation of manipulation skills from human videos[END_REF], [START_REF] Das | Model-based inverse reinforcement learning from visual demonstrations[END_REF], image translation and inpainting [START_REF] Liu | Imitation from observation: Learning to imitate behaviors from raw video via context translation[END_REF], [START_REF] Sharma | Third-person visual imitation learning via decoupled hierarchical controller[END_REF], [START_REF] Smith | Avid: Learning multi-stage tasks via pixel-level translation of human videos[END_REF], [START_REF] Bahl | Human-to-robot imitation in the wild[END_REF], learning object centric representations [START_REF] Pirk | Online object representations with contrastive learning[END_REF], [START_REF] Sermanet | Time-contrastive networks: Self-supervised learning from video[END_REF], simulators [START_REF] Petrík | Learning object manipulation skills via approximate state estimation from real videos[END_REF], [START_REF] Bonardi | Learning one-shot imitation from humans without humans[END_REF] and meta learning [START_REF] Yu | One-shot imitation from observing humans via domain-adaptive meta-learning[END_REF]. Contrary to these prior works that often consider closely aligned human videos and robot environments e.g. humans demonstrating the task in the same lab environment as the robot, we assume that human video instructions are collected "in-the-wild" and therefore there exists a large domain gap between human videos and robot workspace.

Recent works attempt to perform model pretraining from large-scale human videos in order to get good image representations for robotic control [START_REF] Nair | R3m: A universal visual representation for robot manipulation[END_REF], [START_REF] Xiao | Masked visual pretraining for motor control[END_REF]. Other works have shown that we can infer states and actions from diverse videos and use it for reinforcement learning [START_REF] Edwards | Perceptual values from observation[END_REF], [START_REF] Schmeckpeper | Reinforcement learning with videos: Combining offline observations with interaction[END_REF], [START_REF] Schmeckpeper | Learning predictive models from observation and interaction[END_REF], [START_REF] Seo | Reinforcement learning with action-free pre-training from videos[END_REF]. In our work, we show that encoders trained on large datasets of videos for human action recognition provide good task embeddings for robotic manipulation.

Prior works have also considered leveraging large datasets of human videos to learn reward functions for robotics manipulations [START_REF] Shao | Concept2robot: Learning manipulation concepts from instructions and human demonstrations[END_REF], [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF]. The most relevant work to ours is Domain-agnostic Video Discriminator (DVD) [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF] which also tackles the problem of commanding robots using inthe-wild human videos. DVD learns a video similarity by training a discriminator network to classify whether two videos are performing the same task on both annotated robot demonstrations and a subset of SSv2 videos. The similarity score is then used as a reward for planning with an actionconditioned video generation method [START_REF] Babaeizadeh | Stochastic variational video prediction[END_REF] trained on randomly collected robot experience. In contrast, our approach does not require any annotated robot demonstration and can accommodate both randomly collected robot experience and expert demonstrations. Moreover, [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF] plans several subtrajectories as high-dimensional synthetic videos per episode whereas our approach plans a full trajectory in the embedding space of robot videos.

Contrastive learning on large-scale datasets has led to significant progresses in a range of computer vision tasks [START_REF] Chen | A simple framework for contrastive learning of visual representations[END_REF], [START_REF] He | Momentum contrast for unsupervised visual representation learning[END_REF]. Contrastive learning has also been used to learn language-conditioned policies by training on large scale datasets of images [START_REF] Radford | Learning transferable visual models from natural language supervision[END_REF], [START_REF] Shridhar | Cliport: What and where pathways for robotic manipulation[END_REF] and videos [START_REF] Fan | Minedojo: Building openended embodied agents with internet-scale knowledge[END_REF]. In this work, we leverage Supervised Contrastive Learning [START_REF] Khosla | Supervised contrastive learning[END_REF] for human action recognition on the SSv2 dataset to learn a mapping between human video instructions and robotic manipulation.

Our work falls under outcome-conditioned action regression [START_REF] Lynch | Learning latent plans from play[END_REF], [START_REF] Emmons | Rvs: What is essential for offline rl via supervised learning?[END_REF], [START_REF] Furuta | Generalized decision transformer for offline hindsight information matching[END_REF]. Such works cast reinforcement learning as learning policies conditioned on trajectory information such as future returns [START_REF] Kumar | Reward-conditioned policies[END_REF], [START_REF] Schmidhuber | Reinforcement learning upside down: Don't predict rewards-just map them to actions[END_REF], [START_REF] Srivastava | Training agents using upside-down reinforcement learning[END_REF], many previous timesteps [START_REF] Chen | Decision transformer: Reinforcement learning via sequence modeling[END_REF], [START_REF] Janner | Offline reinforcement learning as one big sequence modeling problem[END_REF] or a desired goal-configuration [START_REF] Ding | Goal-conditioned imitation learning[END_REF], [START_REF] Ghosh | Learning to reach goals via iterated supervised learning[END_REF], [START_REF] Lynch | Learning latent plans from play[END_REF], [START_REF] Emmons | Rvs: What is essential for offline rl via supervised learning?[END_REF] to learn multi-task policies. In contrast, our policy is conditioned on a video embedding of the full robot trajectory. [START_REF] Jang | Bc-z: Zero-shot task generalization with robotic imitation learning[END_REF] also showed that we can learn a videoconditioned controllers by conditioning the policy on both a video of the full robot trajectory and paired sets of human videos appropriately collected for each task. Other works [START_REF] Lynch | Language conditioned imitation learning over unstructured data[END_REF], [START_REF] Nair | Learning language-conditioned robot behavior from offline data and crowdsourced annotation[END_REF] show that pairing a small subset of robot data with language instructions enables generalizable languageconditioned task execution. Our method does not require any robot data paired with human instructions and, hence, can learn generic policies from unlabelled robot datasets.

III. METHOD

In Section III-A we first present an overview of our approach that enables robots to mimic new tasks demonstrated by people in natural human environments. We then detail how we learn a generic video-conditioned policy from randomly-generated demonstrations in Section III-B. We further describe how we condition our policy on human videos in Section III-C. Finally, we describe how we learn a similarity function for matching human videos to robot rollouts without using paired human-robot data in Section III-D.

A. Method overview

We aim to perform a robot manipulation task conditioned on a human video in a vision-based multi-task environment. At test time our system receives an input video of a previously unseen task performed by a person, such as pushing a mug or closing a drawer, and controls a robot arm with the intent of performing a similar task in the robot environment.

More formally, we consider a set of Markov Decision Processes (MDP) MDP i = (S, A, R i , p) i sharing the same observation space S, action space A and dynamics p but with different reward functions R i corresponding to different tasks we would like to solve. We do not assume that the At the same time, the robot video embedding of each trajectory in the robot dataset is added to an embeddings library. (Right) At inference, we encode the human video instruction into a human video embedding. We then average the robot embeddings from the library that have highest cosine similarity to the human embedding into a selected robot embedding. Finally, we execute the policy conditioned on this selected embedding.

reward functions R i are observed. Instead, R i must be inferred through a human video of the task x h ∈ X . Our goal is to learn a video-conditioned controller π(.|s, x h ) that predicts actions a ∈ A given the current states s ∈ S and human videos x h ∈ X to maximize the reward functions associated with the input human video.

There may exist a large domain gap between videos of humans and robots performing similar tasks. To bridge this gap, we leverage the large-scale video dataset Something-Something-v2 (SSv2) [START_REF] Goyal | The" something something" video database for learning and evaluating visual common sense[END_REF] with labeld human actions D h = {x h i , y h i } i where labels y h i for videos x h i correspond to different manipulation actions such as Opening Something, Moving Something Away from the Camera, etc. Following DVD [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF], we train a similarity function d(., .) that assigns high values to pairs of videos representing the same task and low values to video pairs of different tasks. Unlike DVD, however, we learn such similarity without any annotated robot videos. Given a human video, we use the learned similarity as a reward R(.) = d(., x h ) for our controller.

To learn the policy, we assume access to a dataset of unlabelled robot demonstrations D r = {x r i , (s t i ) t , (a t i ) t } i , where (s t i ) t is a sequence of robot observations, (a t i ) t is the corresponding sequence of executed robot actions and x r i is a video of the demonstration. For instance, if the observation space S only contains images (without e.g. proprioceptive information), the videos can simply correspond to the whole sequence of states in the trajectory x r i = (s t i ) t . This offline dataset can be collected in many different ways: by expert demonstrators, rollouts of other policies, through teleoperation or by random data generation. Importantly, we do not assume access to any further information about these demonstrations.

Figure 2 presents an overview of our approach: we leverage a video encoder f θ trained for human action recognition on SSv2 and a similarity metric between videos. During training, we learn a behavior cloning policy π φ conditioned on robot embeddings given by the video encoder while storing all embeddings of the robot training dataset in a library. At inference, we first use this library and the similarity metric to predict a robot embedding relevant to the human video instruction, then rollout the policy in the environment.

B. Video-conditioned policy learning

We now describe how to encode a large array of behaviors into a single policy. A key to our approach is the use of the embedding space of a video encoder pretrained on human videos to condition our policy. This video embedding can be seen as a task embedding for our generic multi-task policy. We will explain more in detail how we can obtain meaningful video embeddings for control in Section III-D.

During training, we learn a policy π φ to regress an action given the current state and a video embedding of a full robot trajectory. Video embeddings act as context defining the global task. The policy is trained with behavior-cloning by minimizing the loss:

L π (φ) = -E s,a,x r ∼D r log π φ (a|s, f θ (x r )). (1)
At test time, this policy can be commanded to reproduce a robot video input x r by first encoding the video into an embedding e r and then executing the actions a ∼ π(.|s, e r ) predicted by the policy at each step s visited during the roll-out. As we will see from experiments, however, using human videos to directly condition the policy results in poor performance due to the large domain gap between robot and human videos. We therefore propose to first translate human videos to robot video embeddings as described in the next section.

C. Inference with human instructions

At inference, our first step is to translate the human video instruction x h into the robot video embedding e h that both (1) corresponds to the target task and ( 2) is in distribution for the robot policy. While many methods can instantiate this, we choose to simply use nearest neighbors regression in the robot embedding space. We first encode the videos contained in the robot dataset D r into a library of robot embeddings D e = (e r i = f θ (x r i )) i . We also encode the video instruction into a human embedding e h . We then perform k nearest neighbors regression using the distance function d, by first computing all the distances between the human embedding and each embeddings in the library d i = d(e h , e r i ), then averaging the top k embeddings of the library e r = 1 k k ĩ∈1 e r ĩ . Finally, we perform policy rollout conditioned on this embedding. As result, although the policy was trained with behavior cloning, this approach allows us to maximize the similarity of the robot trajectory to the video prompt at inference.

D. Learning a task similarity from human videos

Many choices of distance metric d between videos can be used to match a human video to an appropriate robot embedding. In this work, we consider adapting Supervised Contrastive Learning [START_REF] Khosla | Supervised contrastive learning[END_REF] to our video action recognition task on the Something-Something-v2 dataset. We learn our video encoder f θ (.) = f p θ (f b (.)) composed of a backbone f b (.) which maps videos to representation vectors and a projection network f p θ (.) which maps representation vectors to embedding vectors e, such that embeddings from the same class are pulled closer together in cosine distance than embeddings from different classes. As a result, the cosine distance characterizes the similarity between two videos.

In contrastive representation learning, the backbone and projection net are typically trained end-to-end from scratch and, after training, the projection net is discarded and a classifier head is learned on top of the representations. Instead, we start from available pretrained backbones for SSv2 classification and simply train the projection net using the Supervised Contrastive Learning loss. Given a batch of N video/label pairs sampled from the SSv2 dataset

{x h k , y h k } k∈[1,..,N ] ∼ D h ,
L SupCon (θ) = i∈I -1 |P (i)| p∈P (i) log exp f θ (x h i ), f θ (x h p ) /τ a∈A(i) exp f θ (x h i ), f θ (x h a ) /τ (2)
where ., . is the cosine similarity, I = [1, .., 2N ], A(i) = I\{i}, P (i) = {p ∈ A(i) : ỹh p = ỹh i } and τ is a hyperparameter. After training, we use the distance d(x h , x r ) = f θ (x h ), f θ (x r ) between videos x h and x r as measure of similarity that focuses on the semantic aspects of the video. As we show in the experimental section, despite being trained only on human videos, this similarity metric generalizes to robot manipulation tasks. In our experiments, we use the same video backbone as [START_REF] Shao | Concept2robot: Learning manipulation concepts from instructions and human demonstrations[END_REF] and [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF].

Alternative options for d include the DVD similarity network [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF], which learns to classify whether two videos correspond to the same task and uses the classification score between human and robot videos as a distance metric d on top of the video encoder f b . While our approach implicitly corresponds to the same classification objective, supervised contrastive learning readily embeds our encoder with a similarity metric.

IV. EXPERIMENTS

This section presents experiments validating our proposed approach and its training procedure. We evaluate our method on several tasks in two challenging environments and compare results to the state-of-the-art method [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF]. In particular, we present ablations of our method and show its advantage in zero-shot settings, i.e. for tasks that have not been observed during training.

A. Experimental Setup

For our experiments, we consider the TableTop environments introduced in [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF]. The agent controls a robot arm in four variations of a simulated environment containing a drawer, a cup in front of a coffee machine and a faucet handle. The four environments differ by object positions, camera locations and colors. Robot experience for policy training is collected by controlling the robot end effector to go through three keypoints randomly sampled in the environment. We follow the experimental setup of [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF] and consider three tasks: close the drawer, push the cup and turn the faucet to the right. The image encoder representing the current state of the environment is learned end-to-end together with the policy. These environments challenge the ability of our approach to generate meaningful manipulation skills from random robot trajectories.

Furthermore, we consider the Kitchen environment initially introduced in [START_REF] Gupta | Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning[END_REF]. The agent controls a robot arm with a clamp in a kitchen environment containing diverse objects: a microwave, three doors, a kettle and light and knob switches. We consider 3 opening tasks: open the microwave, open the left door and open the sliding door. Robot demonstrations accomplishing these tasks are available from [START_REF] Nair | R3m: A universal visual representation for robot manipulation[END_REF]. However, these demonstrations are not annotated and the agent learns jointly from all these demonstrations without knowing what each demonstration accomplishes. For the Kitchen environment we train the policy on top of R3M image representations following [START_REF] Nair | R3m: A universal visual representation for robot manipulation[END_REF] using all demonstrations. This environment is challenging as it requires to distinguish which task is the one intended by the user among very similar opening tasks. Figure 3 illustrates the environments. Following [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF], for each of these tasks, we consider three different human video instructions collected in diverse environments to prompt our method at inference. Note that, in both environments, our approach can use the same video encoder and similarity metric.

B. Comparison to DVD [15]

We first validate our video-conditioned control policy and compare it to the planning method of DVD [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF] on the TableTop environments in the following settings:

• Seen robot demos: the video similarity is trained on a subset of task-related SSv2 classes and on labelled videos of robot demonstrations for the target tasks collected in env 1 and the rearranged version of env 1. • Unseen robot demos: the video similarity is trained on a subset of task-related SSv2 classes and on labelled videos of robot demonstrations corresponding to nontarget tasks in env 1 and the rearranged version of env 1. We compare our approach to [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF] that uses the DVD similarity on top of a learned action-conditioned video prediction model for control [START_REF] Babaeizadeh | Stochastic variational video prediction[END_REF]. For the Seen robot demos setting we take results reported in [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF]. For the Uneen robot demos, we take the best success rate reported in [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF] for environment 1 and run the code of the authors to obtain results for environments 2, 3 and 4 (marked with *). For a fair comparison of our control policy, we here run ViP using the same video similarity function as in DVD which was trained on paired data between human videos and robot demonstrations.

Table I presents results for the TableTop environments where, for each environment, we report the average success rates across the three tasks and three human video instructions. ViP control policy outperforms DVD in both settings and most environments. We believe these improvements are due to comparing the human video prompt to full robot trajectories instead of shorter sub-trajectories and not relying on synthetic video generation for control as in [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF]. To demonstrate stability of our approach, we report results over four training seeds while [START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF] reports results over evaluation runs. Moreover, our method is significantly faster than DVD as it operates in the space of pre-computed embeddings, while DVD requires to execute its video prediction model for each sampled trajectory. As result, in our experiments ViP requires less than 1 second to complete an episode in the TableTop environment while it takes more than 16 seconds to complete the same task with DVD.

We observe that the gains obtained by ViP compared to DVD are lower in the unseen robot demos setting. This can be attributed to the fact that our approach relies more on human videos of the target tasks to achieve good results.

C. ViP without paired data

While DVD uses paired data with robot and human videos demonstrating execution of similar tasks, obtaining such data is cumbersome at scale, see discussion in Section I. In this section we demonstrate that ViP is able to cope with a more challenging setting where the training of video similarity is done using pairs of human videos only and without access to any robot data.

Table II presents our results. In this setting, our approach (ViP (Ours)) significantly outperforms DVD, succeeding to perform intended tasks more than 70% of the time on average across environments, while DVD performs only slightly better than chance. Moreover, we can see that our results in this setting are similar to results in Table I where paired robot demonstrations are used for similarity learning. Indeed, having a global understanding of full robot trajectories makes our action recognition approach less reliant on robot demos, which are more important when planning for short horizon as in DVD. Finally, ViP with the DVD similarity metric also performs well, showing that our approach can accommodate alternative choices of video similarity. Qualitative results of ViP are presented in Figure 1 and in the supplementary video, where we show that failure cases include failing to manipulate the desired object and failure to perform the manipulation task.

We further compare our approach to a baseline where we bypass our translation module and directly use human video embeddings to condition the ViP policy (Human video as input). Since ViP was only trained on robot video embeddings, embeddings of human videos are out of its training distribution. As result, the policy fails to perform the intended tasks. This highlights the importance of our translation procedure which matches human videos to a library of robot demonstrations.

In Figure 4 we evaluate the sensitivity of our method to the number of nearest neighbor samples k used for translation of human videos in Section III-C. High values of k ensure that the generated embedding e r is an average of many appropriate robot embeddings, whereas low values of k prioritize maximising the similarity score between the predicted robot embedding and the human instruction embedding. While our approach is relatively robust to this hyperparameter, too high values of k result in lower performance. On the other side, for k = 1, meaning that we choose a robot demonstration with the highest similarity to the human video, the performance drops. We hypothesise that performing a nearest-neighbor regression has a regularization effect, as naively maximizing the similarity might result in choosing an adversarial robot embedding that is misleadingly for the target task. We set k to 1% of the library size for the TableTop environments and to 0.5% for the Kitchen environment.

In Table II we also ablate an alternative choice for video similarity where video embeddings obtained from the SSv2 action classification network are directly compared using a cosine distance measure (Cosine distance in repr. space). Using such a naive video similarity together with ViP results in superior performance compared to DVD, however, it is outperformed by both the DVD similarity with ViP (ViP (DVD similarity)) and our full approach, highlighting the importance of training an explicit distance between videos for the success of our method.

D. Kitchen environment

We evaluate our approach on the kitchen environments in the zero-shot setting. We compare our approach against a version of our method where we select a robot demonstration that solves the target task (ViP (Oracle)) as well as a version that randomly selects robot demonstrations from the library (ViP (Random)). We also compare against single-task R3M [START_REF] Nair | R3m: A universal visual representation for robot manipulation[END_REF], where we trained specialized policies for each task using all the demonstrations available.

We report the success rates for each task averaged over the 3 corresponding human video instructions across 4 training seeds in Table III. Comparison between R3M and Oracle shows that our approach successfully learns from multi-task data for precise manipulation skills, achieving competitive results to R3M trained for single tasks. When prompted with a human video instruction in the left camera view (L), our approach successfully opens the correct door when presented with human videos of opening common doors and microwaves. However, the method fails to map human videos of opening sliding doors to appropriate robot skills. On the right camera view (R), our approach struggles more, often opening the wrong object when prompted with videos of sliding doors and microwaves. These results show the limit of training on SSv2, where different human videos of opening something are encouraged to be grouped together by our similarity metric disregarding manipulated objects. V. CONCLUSION

We propose ViP, a method that learns to map human video instructions to robot skills in a zero-shot manner. We show that by conditioning on video embeddings, we can learn from multi-task robot data without supervision and prompt our policy with an unseen human video instruction at test time. As a step further towards less supervision in data-driven robotics, we demonstrate that, by training on large datasets of diverse labelled human videos, we don't need to pair human instructions to robot data during training.

Our experiments demonstrate that ViP can accommodate many different similarity metrics between human instructions and robot manipulations. Future work could explore other forms of similarities, such as mapping language instructions to robotic skills.

Fig. 2 .

 2 Fig. 2. (Left) During training, we learn a manipulation policy conditioned on robot video embeddings of the full robot trajectories from the robot dataset.At the same time, the robot video embedding of each trajectory in the robot dataset is added to an embeddings library. (Right) At inference, we encode the human video instruction into a human video embedding. We then average the robot embeddings from the library that have highest cosine similarity to the human embedding into a selected robot embedding. Finally, we execute the policy conditioned on this selected embedding.

Fig. 3 .

 3 Fig. 3. Illustrations of environments used in our experiments. From left to right: TableTop env1-env4 followed by Kitchen left and right views.

Fig. 4 .

 4 Fig. 4.Average success rate achieved by the policy on the TableTop environments for different values of k. We compare k = 1 against values of k corresponding to different percentages of the library size.

TABLE I RESULTS

 I FOR THE TABLETOP ENVIRONMENT USING ROBOT DEMONSTRATIONS. DVD RESULTS ARE OBTAINED FROM[START_REF] Chen | Learning generalizable robotic reward functions from" in-the-wild" human videos[END_REF] EXCEPT THE ONES MARKED WITH "*" FOR WHICH WE RUN THE CODE PROVIDED BY THE AUTHORS.

	Setting	Method	env 1	env 2	env 3	env 4	Avg
	seen robot demos	DVD [15]	65.2 (4.1)	62.3 (2.8)	53.8 (3.9)	39.6 (0.7)	55.2 (2.9)
		ViP (DVD similarity) 97.1 (2.8)	72.0 (15.9)	82.1 (16.4)	42.0 (11.2)	73.3 (11.6)
	unseen robot demos DVD [15]	55.1 (2.0)	51.1* (2.5) 38.4* (1.6) 35.0* (1.9) 44.9* (2.0)
		ViP (DVD similarity) 68.2 (11.2) 55.2 (13.0)	60.3 (8.0)	43.6 (8.0)	56.8 (10.0)

TABLE II RESULTS

 II FOR THE TABLETOP ENVIRONMENT WITHOUT PAIRED DATA. DVD RESULTS ARE OBTAINED BY RUNNING THE CODE OF THE AUTHORS. * (2.6) 44.3* (2.0) 32.8* (1.7) 27.0* (2.6) 36.8* (2.2)ViP (Cosine distance in repr. space) 36.9(4.5)

	Method	env 1	env 2	env 3	env 4	Avg
	Random	25.6	25.1	22.2	18.2	22.8
	Human video as input	22.7 (7.9)	32.9 (12.9)	11.2 (7.8)	14.3 (9.6)	20.3 (9.6)
	DVD[15]	43.066.4 (17.5)	39.3 (15.0)	36.4 (15.4)	44.8 (13.1)
	ViP (DVD similarity)	50.8 (10.2)	67.3 (17.4)	72.6 (16.3)	53.4 (9.6)	61.0 (13.4)
	ViP (Ours)	79.9 (11.4)	77.9 (16.2)	70.6 (17.8)	56.9 (8.7)	71.3 (13.5)

TABLE III RESULTS

 III ON THE KITCHEN ENVIRONMENT

	Method	left door sliding door microwave
	L ViP (Random) 22.5 (2.7)	29.8 (0.4)	13.5 (4.7)
	ViP (Oracle)	83.2 (16.5) 99.2 (0.8)	43.0 (10.2)
	R3M[34]	63.8 (10.3) 100 (0.0)	53.8 (11.0)
	ViP (Ours)	69.4 (11.0) 41.7 (27.6)	25.1 (8.5)
	R ViP (Random) 22.0 (2.9)	30.0 (0.0)	8.5 (3.0)
	ViP (Oracle)	92.0 (7.4)	99.8 (0.4)	27.2 (9.1)
	R3M[34]	57.5 (2.9)	100 (0.0)	30.0 (0.1)
	ViP (Ours)	66.7 (3.3)	41.7 (14.4)	6.6 (2.9)

Acknowledgements: This work was funded in part by the French government under management of Agence Nationale de la Recherche as part of the "Investissements d'avenir" program, reference ANR19-P3IA-0001 (PRAIRIE 3IA Institute), the ANR project VideoPredict, reference ANR-21-FAI1-0002-01, and the Louis Vuitton ENS Chair on Artificial Intelligence. This work was granted access to the HPC resources of IDRIS under the allocation 2021-AD011012947 made by GENCI. We would like to thank Mathis Clautrier and Minttu Alakuijala for helpful feedbacks.