Christophe Chareton

Julien Brunel

David Chemouil

Towards an Updatable Strategy Logic

This article is about temporal multi-agent logics. Several of these formalisms have been already presented (ATL-ATL*, ATL sc , SL). They enable to express the capabilities of agents in a system to ensure the satisfaction of temporal properties. Particularly, SL and ATL sc enable several agents to interact in a context mixing the different strategies they play in a semantical game. We generalize this possibility by proposing a new formalism, Updating Strategy Logic (USL). In USL, an agent can also refine its own strategy. The gain in expressive power rises the notion of sustainable capabilities for agents.

USL is built from SL. It mainly brings to SL the two following modifications: semantically, the successor of a given state is not uniquely determined by the data of one choice from each agent. Syntactically, we introduce in the language an operator, called an unbinder, which explicitly deletes the binding of a strategy to an agent. We show that USL is strictly more expressive than SL.

Introduction

Multi-agent logics are receiving growing interest in contemporary research. Since the seminal work of Rajeev Alur, Thomas A. Henzinger, and Orna Kupferman [START_REF] Alur | Alternating-time temporal logic[END_REF], one major and recent direction (ATL with Strategy Context [START_REF] Brihaye | ATL with strategy contexts and bounded memory[END_REF][START_REF] Da | Propriétés de jeux multi-agents[END_REF][START_REF] Da | ATL with Strategy Contexts: Expressiveness and Model Checking[END_REF], Strategy Logic (presented first in [START_REF] Chatterjee | Strategy logic[END_REF] and then extended in [START_REF] Mogavero | Reasoning About Strategies: On the Model-Checking Problem[END_REF][START_REF] Mogavero | Reasoning about strategies[END_REF]) aims at contextualizing the statements of capabilities of agents.

Basically, multi-agent logics enable assertions about the capability of agents to ensure temporal properties. Thus, ATL-ATL * [START_REF] Alur | Alternating-time temporal logic[END_REF] appears as a generalization of CTL-CTL * , in which the path quantifiers E and A are replaced by strategy quantifiers. Strategy quantifiers (the existential A and the universal A ) have a (coalition of) agent(s) as parameter. A ϕ means that agents in A can act so as to ensure the satisfaction of temporal formula ϕ. It is interpreted in Concurrent Game Structures (CGS), where agents can make choices influencing the execution in the system. Formula A ϕ is true if agents in A have a strategy so that if playing it they force the execution to satisfy ϕ, whatever the other agents do.

A natural question is: how to interpret the imbrication of several quantifiers? Precisely, in the interpretation of such formula as

ψ 1 := a 1 (ϕ 1 ∧ a 2 ϕ 2 )
(where ϕ is the temporal operator meaning ϕ is always true, and a 1 and a 2 are agents), is the evaluation of ϕ 2 made in a context that takes into account both the strategy quantified in a 1 and the strategy quantified in a 2 ?

In ATL-ATL * , only a 2 is bound: subformula a 2 ϕ 2 is true iff a 2 may ensure ϕ 2 , whatever the other agents do. Then a 2 stands for three successive operations: First, each agent is unbound from its current strategy, then an existential quantification is made for strategy σ . At last, a 2 is bound to strategy σ .

ATL sc [START_REF] Brihaye | ATL with strategy contexts and bounded memory[END_REF][START_REF] Da | Propriétés de jeux multi-agents[END_REF][START_REF] Da | ATL with Strategy Contexts: Expressiveness and Model Checking[END_REF], while keeping the ATL syntax, adapts the semantics in order to interpret formulas in a context which stores strategies introduced by earlier quantifiers.

Strategy Logic (SL [START_REF] Mogavero | Reasoning About Strategies: On the Model-Checking Problem[END_REF][START_REF] Mogavero | Reasoning about strategies[END_REF]) is another interesting proposition, which distinguishes between the quantifications over strategies and their bindings to agents. The operator a is split into two different operators: a quantifier over strategies ( x , where x is a strategy variable) and a binder ((a, x), where a is an agent) that stores into a context the information that a plays along the strategy instantiating variable x (let us write it σ x in the remaining of this paper).The ATL formula ψ 1 syntactically matches the SL:

ψ 2 := x 1 (a 1 , x 1 ) (ϕ 1 ∧ x 2 (a 2 , x 2 ) ϕ 2 )
In ψ 2 , when evaluating ϕ 2 , a 1 remains bound to strategy σ x 1 except if a 1 and a 2 are the same agent. If they are the same, the binder (a 2 , x 2 ) unbinds a from its current strategies before binding her to σ x 2 .

In this paper we present USL, a logic obtained from SL by making explicit the unbinding of strategies and allowing new bindings without previous unbinding. For that, we introduce an explicit unbinder (a x) in the syntax (and the binder in USL is written (a x)) and we interpret USL in models where the choices of agents are represented by the set of potential successors they enable from the current state. When there is no occurrence of an unbinder, each agent remains bound to her current strategies. Then different strategies can combine together even for a single agent, provided that they are coherent, which means they define choices in non-empty intersection (the notion is formally defined in Sect. 2).

The main interest in such introduction is to distinguish between cases where an agent composes strategies together and situations where she revokes a current strategy for playing an other one. If a 1 and a 2 are the same agents, then ψ 2 is written in SL:

ψ 3 := x 1 (a, x 1 ) (ϕ 1 ∧ x 2 (a, x 2 ) ϕ 2 ),
which syntactically matches the USL:

ψ 4 := x 1 (a x 1 ) (ϕ 1 ∧ x 2 (a x 2 ) ϕ 2 )
In ψ 3 , subformula x 2 (a, x 2 ) ϕ 2 states that a can adopt a new strategy that ensures ϕ 2 , no matter if it is coherent with the strategy σ x 1 previously adopted. In ψ 4 , both strategies must combine coherently together. In natural language ψ 4 states that a can ensure ϕ 1 and leave open the possibility to ensure ϕ 2 in addition. The equivalent of ψ 3 in USL is actually not ψ 4 but

ψ 5 := x 1 (a x 1 ) (ϕ 1 ∧ x 2 (a x 1 )(a x 2 ) ϕ 2 )
There indeed, in subformula (a x 1 )(a x 2 ) ϕ 2 , a is first unbound from σ x 1 and then bound to σ x 2 .

A consequence of considering these compositions of strategies is the expressiveness of sustainable capabilities of agents. Let us now consider the USL formula:

ψ 6 := x 1 (a x 1 ) ( x 2 (a x 1 )(a x 2 )X p)
There the binder (a x 2 ) is used with the unbinder (a x 1 ), so that ψ 6 is equivalent to the SL:

ψ 7 := x 1 (a, x 1 ) ( x 2 (a, x 2 )X p)
It states that a can remain capable to perform the condition expressed by X p when she wants. But in case she actually performs it, the formula satisfaction does not require that she is still capable to perform it. The statement holds in state s 0 in structure M 1 with single agent a. See Fig. 1, where choices are defined by the set of transitions they enable. Since M 1 interprets SL formulas with only agent a, the choices for a are deterministic: let s, s be two states and c a choice, then the transition from s to s is

s 0 ¬p s 1 p s 2 ¬p c 2 c 1 c 1 c 1 Figure 1: Structure M 1
labelled with c iff {s } is a choice for a at s. Indeed, by always playing choice c 1 , a remains in state s 0 , where she can change her mind to ensure p. But if she chooses to reach p, she can do it only by moving to state s 1 and then to state s 2 . Doing so, she loses her capability to ensure X p at any time. The only way for her to maintain her capability to reach p is to always avoid it, her capability is not sustainable. A more game theoretical view is to consider strategies as commitments. In such view, by adopting a strategy, a adopts a behavior that holds in the following execution, as far as it is not explicitly deleted. Formula

ψ 8 := x 1 (a x 1 ) ( x 2 (a x 2 )X p)
is the counterpart of formula ψ 7 with such interpretation of composing strategies for a single agent. If a plays σ x 2 , it must be coherently with σ x 1 . Thus, ψ 8 is false in structure M 1 , since a cannot achieve p more than once.

Formula ψ 8 distinguishes between structures M 1 and M 2 from Fig. 2 ( Note that in this second structure the choices are not deterministic: from a given state a choice may be compatible with several potential successors). In M 2 , ψ 8 is true at s 0 since the strategy always play c 1 ensure the execution to remain in state s 0 or s 1 and is always coherent with strategy play c 2 first and then always play c 1 , which ensures X p from states s 0 and s 1 . What is at stake with it is the difference between sustainable capabilities and one shot capabilities. Formulas ψ 7 and ψ 8 both formalize the natural language proposition a can always achieve p. One shot capability (ψ 7 ) means she can achieve it once for all and choose when. Sustainable capability (ψ 8 ) means she can achieve it and choose when without affecting nor losing this capability for the future.

s 0 ¬p s 1 p s 2 ¬p c 1 , c 2 c 1 , c 3 c 3 c 1 , c 3 c 1 , c 2 c 1 , c 2 , c 3 Figure 2: Structure M 2
In Sect.3, we compare the expressive power of SL and USL by use of formula ψ 9 , obtained from ψ 7 by adding to a the sustainable capability to ensure X ¬p:

ψ 9 := x (a x) ( x 0 (a x 0 )X p ∧ x 0 (a x 0 )X ¬p)
ψ 9 states that a has sustainable capability to decide whether p or ¬p holds at next state. We say that a has sustainable control on property p: she is sustainably capable to decide the truth value of p.

The main purposes of USL are to give a formalism for the composition of strategies and to unify it with the classical branching-time mechanisms of strategy revocation. So, both treatments can be combined in a single formalism. In the remaining of this paper we define USL syntax and semantics, and we introduce the comparison of its expressive power with that of SL.

Syntax and semantics

In this section we present the syntax and semantics of USL, together with the related definitions they require. The USL formulas distinguish between path and state formulas. Definition 1. Let Ag be a set of agents, At a set of propositions and X a set of variables, USL (Ag, At, X) is given by the following grammar:

• State formulas: ϕ ::= p | ¬ϕ | ϕ ∧ ϕ | x ϕ | (A x)ψ | (A x)ψ • Path formulas: ψ ::= ϕ | ¬ψ | ψ ∧ ψ | ψ U ψ | X ψ where p ∈ At, A ⊆ Ag, x ∈ X.
These formulas hold a notion of free variable that is similar to that in [START_REF] Mogavero | Reasoning About Strategies: On the Model-Checking Problem[END_REF][START_REF] Mogavero | Reasoning about strategies[END_REF]: an atom has an empty set of free variables, a binder adds a free variable to the set of free variables of its direct subformula and a quantifier deletes it. Upon formulas on this grammar, those that can be evaluated with no context are the sentences. They are formulas with empty set of free variables, which means each of their bound variables is previously quantified. We now come to the definitions for USL semantics. • M is a set of states, called the domain of the NATS, At is the set of atomic propositions and v is a valuation function, from M to P(At).

• Ch: Ag×M → P(P(M)) is a choice function mapping a pair (agent, state) to a non-empty family of choices of possible next states. It is such that for every state s ∈ M and for every agents a 1 and a 2 in Ag, for every c

1 ∈ Ch(a 1 , s) and c 2 ∈ Ch(a 2 , s), c 1 ∩ c 2 = / 0.
We call a finite sequence of states in M a track τ. The last element of a track τ is denoted by last(τ). The set of tracks that are possible in M is denoted by track M : τ = s 0 s 1 . . . s k ∈ track M iff for every i < k, for every a ∈ Ag, there is c a ∈ P(M) s.t. c a ∈ Ch(a, s i ) and s i+1 ∈ c a . Similarly, an infinite sequence of states such that all its prefixes are in track M is called a path (in M ).

Definition 3 (Strategies and coherence).

A strategy is a function σ from Ag × track M to P(M) such that for all (a, τ) ∈ Ag × track M , σ (a, τ) ∈ Ch(a, last(τ)). By extension, we write σ (A, τ) for a∈A σ (a, τ) for every A ⊆ Ag. Two strategies σ 1 and σ 2 are coherent iff for all (a, τ) in Ag×track M , σ 1 (a, τ)∩σ 2 (a, τ) = / 0. In this case, we also say that σ 1 (a, τ) and σ 2 (a, τ) are coherent choices.

A commitment κ is a finite sequence upon (P(Ag)×X), representing the active bindings. An assignment α is a partial function from X to Strat. A context χ is a pair of an assignment and a commitment. Note that an agent can appear several times in a commitment. Furthermore commitments store the order in which pairs (A, x) are introduced. Therefore our notion of contexts differs from the notion of assignments that is used in SL [START_REF] Mogavero | Reasoning About Strategies: On the Model-Checking Problem[END_REF][START_REF] Mogavero | Reasoning about strategies[END_REF].

A context defines a function from track M to P(M). We use the same notation for the context itself and its induced function. Let κ / 0 be the empty sequence upon (P(Ag) × X), then:

• (α, κ / 0 )(τ) = M • (α, (A, x))(τ) = -a∈A α(x)(a, τ) if A = / 0 -else M • (α, κ • (A, x))(τ) = -(α, κ)(τ) ∩ (α, (A, x))(τ) if this intersection is not empty.
otherwise (which means the context induces contradictory choices), (α, κ)(τ) . Now we can define the outcomes of a context χ, out (χ): let π = π 0 , π 1 , . . . be an infinite sequence over M, then π ∈ out(s, χ) iff π is a path in M , s = π 0 and for every n ∈ N, π n+1 ∈ χ(π 0 . . . π n ). Definition 4 (Strategy and assignment translation). Let σ be a strategy and τ be a track. Then σ τ is the strategy s.t. for every τ ∈ track M , σ τ (τ ) = σ (ττ ). The notion is extended to an assignment: for every α, α τ is the assignment with domain equal to that of α and s.t. for every x ∈ dom(α), α τ (x) = (α(x)) τ

We also define the following transformations of commitments and assignments. Given a commitment κ, coalitions A and B, a strategy variable x, an assignment α and a strategy σ :

•

κ[A → x] = κ • (A x) • ((B, x) • κ)[A x] = (B\A, x) • (κ[A x]) and κ / 0 [A x] = κ / 0 • α[x → σ ] is the assignment with domain dom(α) ∪ {x} s.t. ∀y ∈ dom(α)\{x}, α[x → σ ](y) = α(y) and α[x → σ ](x) = σ
Definition 5 (Satisfaction relation). Let M be a NATS, then for every assignment α, commitment κ, state s and path π:

• State formulas: -M , α, κ, s |= p iff p ∈ v(s), with p ∈ At -M , α, κ, s |= ¬ϕ iff it is not true that M , α, κ, s |= ϕ -M , α, κ, s |= ϕ 1 ∧ ϕ 2 iff M , α, κ, s |= ϕ 1 and M , α, κ, s |= ϕ 2 -M , α, κ, s |= x ϕ iff there is a strategy σ ∈ Strat s.t. M , α[x → σ ], κ, s |= ϕ -M , α, κ, s |= (A x)ϕ iff for every π in out(α, κ[A → x]), M , α, κ[A → x], π |= ϕ -M , α, κ, s |= (A x)ϕ iff for all π in out(α, κ[A x]), M , α, κ[A x], π |= ϕ • Path formulas : -M , α, κ, π |= ϕ iff M , α, κ, π 0 |= ϕ, for every state formula ϕ -M , α, κ, π |= ¬ψ iff it is not true that M , α, κ, π |= ψ -M , α, κ, π |= ψ 1 ∧ ψ 2 iff M , α, κ, π |= ψ 1 and M , α, κ, π |= ψ 2 -M , α, κ, π |= X ψ iff M , α π 0 , κ, π 1 |= ψ. -M , α, κ, π |= ψ 1 U ψ 2 iff there is i ∈ N s.t. M , α π 0 ...π i-1 , κ, π i |= ψ 2 and for every 0 ≤ j < i, M , α π 0 ...π j-1 , κ, π j |= ψ 1
Let α / 0 be the unique assignment with empty domain. Let ϕ be a sentence in USL (Ag, At, X). Then M , s |= ϕ iff M , α / 0 , κ / 0 |= ϕ. Let us give the following comment over these definitions: for every context χ = (α, κ), the definition of out (χ) ensures that the different binders encoded in χ compose their choices together, as far as possible. In case two contradictory choices from an agent are encoded in the context, the priority is given to the first binding that was introduced in this context (the left most binding in the formula). This guarantees that a formula requiring the composition of two contradictory strategies is false. For example, suppose that x 1 (a x 1 )ϕ 1 and x 2 (a x 2 )ϕ 2 are both true in a state of a model, and suppose that strategies σ x 1 and σ x 2 necessarily rely on contradictory choices of a (this means that a cannot play in a way that ensures both ϕ 1 and ϕ 2 ). Then, x 1 (a x 1 )(ϕ 1 ∧ x 2 (a x 2 )ϕ 2 ) is false in the same state of the same model. If the priority was given to the most recent binding (right most binding in the formula), the strategy σ x 1 would be revoked and the formula would be satisfied.

3 Comparison with SL [START_REF] Mogavero | Reasoning About Strategies: On the Model-Checking Problem[END_REF][START_REF] Mogavero | Reasoning about strategies[END_REF] SL syntax can be basically described from SL by deleting the use of the unbinder. Furthermore, the binders are limited to sole agents and are written (a, x) instead of (a x). USL appears to be more expressive than SL [START_REF] Mogavero | Reasoning About Strategies: On the Model-Checking Problem[END_REF][START_REF] Mogavero | Reasoning about strategies[END_REF]. More precisely, SL can be embedded in USL, while ψ 9 is not expressible in SL, even by extending its semantics to non-deterministic models. Here we give the three related propositions. By lack of space, the proofs are only sketched in this article. Detailed proofs of these propositions can be found in [START_REF] Chareton | Updatable Strategy Logic[END_REF]. Note that, since SL is strictly more expressive than ATL sc [START_REF] Da | Propriétés de jeux multi-agents[END_REF], the following results also hold for comparing USL with ATL sc .

Proposition 1. There is an embedding of SL into USL.

Proof (Sketch). The embedding consists in a parallel transformation from SL models and formulas to that of USL. The transformation preserves the satisfaction relation. The differences between SL and USL lie both in the definition of strategies in SL semantics and the difference of interpretation for the binding operator. The first is treated by defining an internal transformation for SL. By this transformation, the constraints of agents playing the same choices, issued from SL actions framework, are expressed in the syntax. Then we define a new operator in USL that is equivalent to SL binding, and show the equivalence: the operator [a x] is an abbreviation for a binder (a x) preceded by the set of unbinders (a x i ), one for every variable x i in the language.

Proposition 2. A model is said deterministic if the successor of a state is uniquely determined by one choice for every agent. Then, sustainable control is not expressible over deterministic models, neither in SL nor in USL.

Proof (Sketch). One checks that for every deterministic NATS M , for any state s of M , M , s ψ 9 . Proposition 1 then straightly brings proposition 2 Proposition 3. Sustainable control is not expressible in SL interpreted over NATSs.

Proof (Sketch). The proof uses a generalization of SL semantics over NATSs. Its definition is in [START_REF] Chareton | Updatable Strategy Logic[END_REF] and holds, for example, the following cases:

• M , α, κ, s |= NATS X ϕ iff for every π ∈ out(s, (α, κ)), M , α π 0 , κ, π 1 |= NATS ϕ • M , α, κ, π |= NATS ϕ 1 U ϕ 2 iff for every π ∈ out(s, (α, κ)), there is i ∈ N s.t. M , α π 0 ...π i-1 , κ, π i
|= NATS ϕ 2 and for all 0 ≤ j ≤ i, M , α π 0 ...π i-1 , κ, π j |= NATS ϕ 1 .

• M , α, κ, s |= NATS x ϕ iff there is a strategy σ ∈ Strat s.t. M , α[x → σ ], κ, s |= NATS ϕ. • M , α, κ, s |= NATS (a, x)ϕ iff M , α, κ[x\κ(a)], s |= NATS ϕ.
where κ[x\κ(a)] designates the context obtained from κ by replacing every (a, y) in it by (a, x). Formula ψ 9 states that a can always control whether p or not. Suppose there is a formula ϕ in SL equivalent to ψ 9 and let us call existential a formula in SL in which every occurrence of x is under an even number of quantifiers. If ϕ is existential then under binary trees it is equivalent to a formula in Σ 1 1 (the fragment of second order logic with only existential set quantifiers).

We now consider a set of formulas {Γ i } i∈N , each one stating that a can choose i times between p and ¬p. The set {Γ i } i∈N is defined by induction over i:

• Γ 0 := x (a, x) ( x 0 (a, x 0 )X p ∧ x 0 (a, x 0 )X ¬p) • for all i ∈ N, Γi + 1 = Γ i [p ∧ ( x i+1 (a, x i+1 )X p ∧ x i+1 (a, x i+1 )X ¬p\p] [¬p ∧ ( x i+1 (a, x i+1 )X p ∧ x i+1 (a, x i+1 )X ¬p)\¬p].
where the notation θ 1 [θ 2 \θ 3 ] designates the formula obtained from θ 1 by replacing any occurrence of subformula θ 3 in it by θ 2 . {Γ i } i∈N is equivalent to ϕ. A compactness argument shows that it is not equivalent to a formula in Σ 1 1 under binary trees, hence ϕ is not an existential formula. Then, we notice that ϕ is true in structures where, from any state, a can ensure any labelling of sequences over p. So, if ϕ has a subformula (a, x)ψ where x is universally quantified, ψ must be equivalent to (p ∨ ¬p). Then, by iteration, ϕ is equivalent to an existential formula in SL. Hence a contradiction.

Conclusion

In this article we defined a strategy logic with updatable strategies. By updating a strategy, agents remain playing along it but add further precision to their choices. This mechanism enables to express such properties as sustainable capability and sustainable control. To the best of our knowledge, this is the first proposition for expressing such properties. Especially, the comparison introduced with SL in this article could be adapted to ATL with Strategy Context [START_REF] Brihaye | ATL with strategy contexts and bounded memory[END_REF].

The revocation of strategies is also questioned in [START_REF] Ågotnes | Alternating-time temporal logics with irrevocable strategies[END_REF]. The authors propose a formalism with definitive strategies, that completely determine the behaviour of agents. They also underline the difference between these strategies and revocable strategies in the classical sense. We believe that updatable strategies offer a synthesis between both views: updatable strategies can be modified without being revoked.

Strategies in USL can also be explicitly revoked. This idea is already present in [START_REF] Brihaye | ATL with strategy contexts and bounded memory[END_REF] with the operator • A •. But the operator •A• also implicitly unbinds current strategy for agents in A before binding them a new strategy. Thus it prevents agents from updating their strategy or composing several strategies.

Further study perspectives about USL mainly concern the model checking. Further work will provide it with a proof of non elementary decidability, adapted from the proof in [START_REF] Mogavero | Reasoning about strategies[END_REF]. We are also working on a semantics for USL under memory-less strategies and PSPACE algorithm for its model-checking. Satisfiability problem should also be addressed. Since SL SAT problem is not decidable, similar result is expectable for USL. Nevertheless, decidable fragments of USL may be studied in the future, in particular by following the directions given in [START_REF] Mogavero | What Makes Atl* Decidable? A Decidable Fragment of Strategy Logic[END_REF].

Definition 2 .

 2 A Non-deterministic Alternating Transition System (NATS) is a tuple M = Ag, M, At, v, Ch where: