Keywords: Modal Logics, Institutions, Agent Communication Languages

The aim of this paper is to provide an analysis of speech acts as institutional actions, where an institution is grounded on the attitudes of its members. In the first part of the paper we present a logic of acceptance, goal and action. Then, we specify how agents can create and maintain normative and institutional facts on the basis of their acceptances qua members of a certain institution. Finally, we propose a formal characterization of the speech act promise.

Introduction

Saussure [START_REF] Saussure | Course in General Linguistics[END_REF] distinguishes language (la langue) from speech (la parole). The "meaning" of the former is defined by institutions (social aspect), by a set of conventions outside the linguistic activity (static aspect). The latter corresponds to the use of the language (dynamical aspect) by individuals, and it is the intention of these individuals that assigns a "value" to the act of utterance (individual aspect). Metaphorically, language is a sort of score while speech is the execution of this score by musicians. The Austin's concept of "performative utterance" [START_REF] Austin | How To Do Things With Words[END_REF] is the first step to bring together language and speech because: first, performative utterances are regulated by social conventions that determine not only the "meaning" of the utterance, but also the "value" of its enunciation 1 ; second, it is practically impossible to attribute a meaning to a performative utterance independently of the value of its enunciation.

Through the concept of illocutionary act, speech act theory [START_REF] Austin | How To Do Things With Words[END_REF][START_REF] Searle | Speech acts: An essay in the philosophy of language[END_REF] generalizes the concept of performative utterance. In such a theory, the value of an enunciation cannot be no longer considered as a consequence of a preliminary meaning; but contrary to the linguistic nihilism of the wittgensteinian trend and their slogan "Meaning is use" (See [START_REF] Searle | Speech acts: An essay in the philosophy of language[END_REF]Section 6.4]), speech act theory does not deny that the use of language is founded on a prior knowledge: it claims that the meaning of an utterance integrates some shared rules that fixe the effects of its enunciation on the speech situation. In speech act theory language is conceived as an informal institution defined by constitutive rules of the form "X count as Y " (See [START_REF] Searle | Speech acts: An essay in the philosophy of language[END_REF]Section 2.7] for more details). These rules create or define new forms of behavior such as promises, orders, requests, assertions, etc. (see [START_REF] Searle | Speech acts: An essay in the philosophy of language[END_REF]Section 2.5]). For example, in the context of ordinary communication [START_REF] Jones | Conventional signalling acts and conversation[END_REF], a promise is defined and created on the basis of a constitutive rule (shared by the speaker and the hearer) of the form: a speaker's enunciation of the utterance: "I am going to perform action a!" counts as a promise to the hearer to perform action a, under the condition that the hearer wants the speaker to perform action a. Subsequently, to speak a language, it is to adopt a form of intentional behavior (by performing a speech act) governed by such a kind of constitutive rules.

The focus of this contribution is to define speech acts as institutional actions where modeled institutions are social or informal institutions. With social or informal institution, we mean an institution which is grounded on the acceptances of its members. Differently from formal (legal) institutions, informal institutions are rule-governed social practices in which no member with 'special' powers to create and eliminate institutional facts is introduced (see Section 4). 2 After a short introduction about the concept of acceptance (Section 2), we present a modal logic of acceptance, goal and action (Section 3). On the basis of the notion of acceptance, we specify how a group of agents can create normative and institutional facts which hold only in an attitude-dependent way (Section 4). In Section 5 we define the promissive speech act.

The Concept of acceptance

Whereas beliefs have been studied for decades, acceptances have only been examined since [START_REF] Stalnaker | Inquiry[END_REF] and [START_REF] Cohen | An essay on belief and acceptance[END_REF] while studying the nature of argument premises or reformulating Moore's paradox [START_REF] Cohen | An essay on belief and acceptance[END_REF]. If a belief that p is an attitude constitutively aimed at the truth of p, an acceptance is the output of "a decision to treat p as true in one's utterances and actions" [START_REF] Hakli | Group beliefs and the distinction between belief and acceptance[END_REF] without being necessarily connected to the actual truth of the proposition. Another difference between belief and acceptance is that beliefs are context-independent, whilst acceptance depends on context [START_REF] Bratman | Practical reasoning and acceptance in context[END_REF]. In fact, one can decide (say for prudential reasons) to reason and act by "accepting" the truth of a proposition in a specific context, and possibly rejecting the very same proposition in a different one. Although, usually, this aspect of acceptance is studied in private contexts, here we continue the work initiated in [START_REF] Gaudou | Anchoring Institutions in Agents' Attitudes: Towards a Logical Framework for Autonomous MAS[END_REF] by exploring the role of acceptance in institutional contexts. Institutional contexts are rule-governed social practices on the background of which the agents reason. For example, take the case of a game like Clue. The institutional context is the rule-governed social practice which the agents conform to in order to be competent players. On the background of such contexts, we are interested in the individual and collective attitudes (individual and collective acceptances) that can be formally captured. In the context of Clue, for instance, agents accept that something has happened (see Example 2) qua players of Clue. The state of acceptance qua member of an institution is the kind of acceptance one is committed to when one is "functioning as member of the institution" [START_REF] Tuomela | The Philosophy of Sociality[END_REF].

The logical framework

Syntax. The syntactic primitives of our logic of acceptance, actions and goals are the following: -a finite set of n > 0 agents AGT = {1, 2, ..., n}; -a nonempty finite set of atomic actions AT = {a, b, ...}; -a finite set of atomic formulas ATM = {p, q, ...}; -a finite set of labels denoting institutional contexts INST = {inst 1 , inst 2 , ..., inst m }. Moreover, we note 2 AGT ⋆ = 2 AGT \ {∅} the set of all non empty subsets of agents, ∆ = C:x|C ∈ 2 AGT ⋆ , x ∈ INST the set of all couples of non empty subsets of agents and institutional contexts.

The language LAN G of the logic L is defined as the smallest superset of ATM such that: if ϕ, ψ ∈ LAN G, i, j ∈ AGT , α ∈ ACT and C:x ∈ ∆ then ¬ϕ, ϕ ∨ ψ, [i:α:j] ϕ, Goal i ϕ and Accept C:x ϕ ∈ LAN G, where the set ACT is the smallest superset of AT such that: if i, j ∈ AGT , a ∈ AT then inf (i:a:j) ∈ ACT (informative actions). The classical boolean connectives ∧, →, ↔, ⊤ (tautology) and ⊥ (contradiction) are defined from ∨ and ¬ in the usual manner.

The set ACT includes all informative actions of type "informing that agent i is going to perform (atomic) action a for agent j", where, with informative action, we mean the performance of an utterance, the act of making an enunciation. Since AT and AGT are finite sets, it follows that ACT as well is finite.

Operators of the form Accept C:x have been introduced in [START_REF] Gaudou | Anchoring Institutions in Agents' Attitudes: Towards a Logical Framework for Autonomous MAS[END_REF] where a logic of acceptance has been proposed. These operators allow to express those facts that the agents in a group C accept while identifying themselves with a certain institution x.

Formula Accept C:x ϕ has to be read "the agents in C accept that ϕ while functioning as members of the institution x". EXAMPLE 1. Accept C:Greenpeace protectEarth is read "the agents in C accept that the mission of Greenpeace is to protect the Earth while functioning as activists of Greenpeace".

For C:x ∈ ∆, Accept C:x ⊥ has to be read "agents in C are not functioning as members of the institution x" because we assume that functioning as a member of an institution is, at least in this minimal sense, a rational activity; conversely, ¬Accept C:x ⊥ has to be read "agents in C are functioning as members of the institution x"; ¬Accept C:x ⊥∧Accept C:x ϕ stands for "agents in C are functioning as members of the institution x and they accept that ϕ while functioning as members of x" or simply "agents in C accept that ϕ qua members of the institution x" (i.e. group acceptance).

The dynamic operators of the form [i:α:j] are just a generalization of standard operators of dynamic logic [START_REF] Harel | Dynamic Logic[END_REF] where both the author (initiator) and the addresse of a certain action α are specified. Formula [i:α:j] ϕ has to be read "after agent i does action α for agent j, it is the case that ϕ". Operators of the form Goal i are standard operators for agents' goals [START_REF] Cohen | Intention is Choice with Commitment[END_REF]. Goal i ϕ has to be read "agent i has the goal that ϕ holds" (or "i wants ϕ to be true").

The following abbreviations are given for any i, j ∈ AGT , a ∈ AT and α ∈ ACT :

i:α:j ϕ def = ¬ [i:α:j] ¬ϕ; Int i,j (α) def = Goal i i:α:j ⊤; Inf i,j (a)
def = i:inf (i:a:j):j ⊤.

Formula i:α:j ϕ is meant to stand for "agent i performs action a and ϕ is true afterward". Int i,j (α): agent i intends to perform action α for j. Inf i,j (a): agent i informs agent j that he is going to perform the (atomic) action a for him.

Semantics.

We use a possible worlds semantics. A model of the logic L is a tuple M = W, A , R, G , V where:

• W is a set of possible worlds;

• A : ∆ -→ (W -→ 2 W
) associates each C:x ∈ ∆ and possible world w with the set A C:x (w) of possible worlds accepted by the group C in w, where the agents in C are functioning as members of the institution x;

• R : AGT × ACT × AGT -→ (W -→ 2 W ) associates each two agents i, j ∈ AGT , action α ∈ ACT and possible world w with the set R i:α:j (w) of possible worlds that are reachable from w through the occurrence of action α performed by i for agent j;

• G : AGT -→ (W -→ 2 W
) associates each agent i ∈ AGT and possible world w with the set G i (w) of worlds that are compatible with agent i's goals in w;

• V : W -→ 2 ATM is a truth assignment which associates each world w with the set V (w) of atomic propositions true in w.

To the standard truth conditions for atomic formulas, negation and disjunction we add :

• M, w |= Accept C:x ϕ iff for all w ′ ∈ W , if w ′ ∈ A C:x (w) then M, w ′ |= ϕ; • M, w |= [i:α:j] ϕ iff for all w ′ ∈ W , if w ′ ∈ R i:α:j (w) then M, w ′ |= ϕ; • M, w |= Goal i ϕ iff for all w ′ ∈ W , if w ′ ∈ G i (w) then M, w ′ |= ϕ.
Axiomatization. Every operator of type Accept C:x and [i:α:j] is supposed to be a normal modal operator satisfying standard axioms and rules of inference of system K. Every operator Goal i is supposed to be a normal modal operator satisfying standard axioms and rules of inference of system KD. 3 Axiom D for goals corresponds to the following constraint of seriality over L models. For every i ∈ AGT and w ∈ W :

G i (w) = ∅ S1
The rest of the section contains other axioms for acceptance, action and intention and corresponding semantic constraints over L models.

We suppose that given a set of agents C, all B ⊆ C have access to all the facts that are (not) accepted by agents in C while functioning as members of the institution x.

In particular, we suppose that: if agents in C (do not) accept that ϕ while functioning as members of the institution x then for every subset B of C and institution y while functioning as members of the institution y, agents in B accept that agents in C (do not) accept that ϕ while functioning as members of the institution x. Such properties are captured by the following two axiom schemas. For every C:x, B:y ∈ ∆, if B ⊆ C then:

Accept C:x ϕ → Accept B:y Accept C:x ϕ 4 Accept ¬Accept C:x ϕ → Accept B:y ¬Accept C:x ϕ 5 Accept
Axioms 4 Accept and 5 Accept together correspond to the following semantic constraint over L models. For every w ∈ W and C:x, B:y

∈ ∆, if B ⊆ C then: if w ′ ∈ A B:y (w) then A C:x (w ′ ) = A C:x (w) S2
We also suppose that if agents in C accept that ϕ qua members of the institution x then, for every subset B of C, it holds that agents in B accept ϕ qua members of the institution x. Formally, for every C:x, B:x ∈ ∆, if B ⊆ C then:

(¬Accept C:x ⊥ ∧ Accept C:x ϕ) → (¬Accept B:x ⊥ ∧ Accept B:x ϕ) Inc Accept EXAMPLE 2.
Imagine three agents i, j, k that, qua Clue players, accept that someone called Mrs. Red, has been killed: ¬Accept {i,j,k}:Clue ⊥ ∧ Accept {i,j,k}:Clue killedMrsRed. This implies that also the two agents i, j qua Clue players accept that someone called Mrs. Red has been killed: ¬Accept {i,j}:Clue ⊥ ∧ Accept {i,j}:Clue killedMrsRed .

Axiom Inc Accept corresponds to the following semantic constraint over L models. For every

w ∈ W , C:x, B:x ∈ ∆, if B ⊆ C: if A C:x (w) = ∅ then A B:x (w) = ∅ and A B:x (w) ⊆ A C:x (w) S3
We suppose the following additional constraint over L models. For every w ∈ W , i, j, i ′ , j ′ ∈ AGT and α, β ∈ ACT :

if w ′ ∈ R i:α:j (w) and w ′′ ∈ R i ′ :β:j ′ (w) then w ′ = w ′′ S4
The property S4 says that all actions occurring in a world w lead to the same world. Thus, all actions occur in parallel and they do not have non-deterministic effects. This explains why we have phrased i:α:j ϕ "i does α for j and ϕ holds afterward" rather than "it is possible that i does α for j and ϕ holds afterward". Constraint S4 corresponds to the following axiom of our logic. For every i, j, i ′ , j ′ ∈ AGT and α, β ∈ ACT :

i:α:j ϕ → [i ′ :β:j ′ ] ϕ Det
We also suppose that the world is never static in our framework, that is, for every world w there exists at least two agents i, j and action α such that i performs α for j at w. Formally, given a L model M, for every w ∈ W we have that:

∃i, j ∈ AGT , ∃α ∈ ACT , ∃w ′ ∈ W s.t. w ′ ∈ R i:α:j (w) S5
Property S5 of L models corresponds to the following axiom of our logic.

i,j∈AGT ,α∈ACT i:α:j ⊤ Active

Axiom Active ensures that for every world w there is a next world of w which is reachable from w by the occurrence of some action performed by some agent for another agent. This is the reason why the operator X for next of LTL (linear temporal logic) can be defined as follows: 4Xϕ

def = i,j∈AGT ,α∈ACT i:α:j ϕ
where Xϕ is meant to stand for "ϕ will be true in the next state".

The following axiom relates intentions with actions. For every i, j ∈ AGT and α ∈ ACT :

i:α:j ⊤ → Int i,j (α) IntAct

According to Axiom IntAct, an agent i performs action α for agent j only if he has the intention to do α for j. In this sense we suppose that an agent's doing is by definition intentional. IntAct corresponds to the following semantic constraint over L models.

For every i, j ∈ AGT , α ∈ ACT and w ∈ W :

if ∃v ′ such that v ′ ∈ R i:α:j (w) then ∀w ′ ∈ G i (w), ∃w ′′ such that w ′′ ∈ R i:α:j (w ′ ) S6
We call L the logic axiomatized by the principles presented above and we write ⊢ L ϕ iff formula ϕ is a theorem of L. Moreover, we write |= L ϕ iff formula ϕ is valid in all L models, i.e. M, w |= ϕ for every L model M and world w in M. Finally, we say that a formula ϕ is satisfiable if there exists an L model M and a world w in M such that M, w |= ϕ.

Institutional concepts

Normative and institutional facts are a class of facts that are typical of institutional contexts. According to [START_REF] Searle | The Construction of Social Reality[END_REF], such facts have the peculiar feature of being dependent on the agents' attitudes in a way that we are in the position to specify in detail in the logic L. More precisely it has been noted that these facts are characterized at least by two features [START_REF] Lagerspetz | The opposite mirrors[END_REF][START_REF] Searle | The Construction of Social Reality[END_REF][START_REF] Tuomela | The Philosophy of Social Practices: A Collective Acceptance View[END_REF]: performativity (an attitude of certain type shared by a group of agents towards a normative or an institutional fact may contribute to the truth of a sentence describing the fact); reflexivity (if a sentence describing a normative or an institutional fact is true, the relevant attitude is present). EXAMPLE 3. If the agents qua group members accept a certain piece of paper as money (an institutional fact), then, in the appropriate context, this piece of paper is money for that group (performativity). At the same time, if it is true that a certain piece of paper is money for a group, then the agents qua group members accept the piece of paper as money (reflexivity).

In order to represent in the logic L these kind of facts, we need first to define the concept of truth with respect to an institutional context in way that respects these two principles.

Truth in an institutional context. We formalize the notion of truth w.r.t. a certain institutional context with the operator

[[x]]. A formula [[x]
] ϕ is read "within the institutional context x, it is the case that ϕ". Here we suppose that "within the institutional context x it is the case that ϕ" if and only if "for every set of agents C, the agents in C accept that ϕ while functioning as members of the institution x". Formally:

[[x]] ϕ def = C∈2 AGT ⋆ Accept C:x ϕ
It is straightforward to prove that every [[x]] is a normal modal operator. Given the previous analysis, a fact is true w.r.t. an institutional context x if and only if such fact is accepted by all the agents while they function as members of x (hence the performativity and the reflexivity principles are maintained).

At this point, it might be objected that there are facts which are true in an institutional context but only "special" members of the institution are aware of them and can change them. For instance, there are laws in every country which are known and can be changed only by the specialists of the domain (lawyers, judges, members of the parliament, etc.). In order to resist to this objection recall that, at this stage, our model applies to informal institutions of a society, in particular to language. Relative to this restriction, the proposed assumption is justified because, for informal institutions, there is no special agent who has the power to create and eliminate institutional facts characterizing the institution itself (i.e. nobody has the power to change the rules of the speech act promise in the context of language).

Finally, the following abbreviation is given:

[[Univ ]] ϕ def = x∈INST [[x]] ϕ
which stands for "ϕ is universally accepted as true".

Constitutive rules.

From the concept of truth with respect to an institutional context a notion of constitutive rule of the form "ϕ counts as ψ in the institutional context x" can be defined. We conceive a constitutive rule as a material implication of the form ϕ → ψ in the scope of an operator [[x]]. We suppose that a constitutive rule is intrinsically local, that is, a rule that is not universally valid while it is accepted by the members of a certain institution. More generally, for every x ∈ INST the following abbreviation (that stands for "ϕ counts as ψ in the institutional context x") is given: 5

ϕ x ⊲ ψ def = [[x]] (ϕ → ψ) ∧ ¬ [[Univ ]] (ϕ → ψ) EXAMPLE 4.
In the context of gestural language there exists a constitutive rule according to which, the nodding gesture "counts as" an endorsement of what the speaker is suggesting, noted nodding gesture ⊲ yes. But in different contexts the same gesture may express exactly the opposite fact (viz. ¬ [[Univ ]] (nodding → yes) holds).

Obligations. Informal institutions such as language involve a deontic dimension that up to now we have ignored. In order to capture this core feature we extend the logic L by introducing a violation atom V as in Anderson's reduction of deontic logic to alethic logic [START_REF] Anderson | A reduction of deontic logic to alethic modal logic[END_REF]. By means to this new formal construct we can formally characterize the obligations which are valid in a certain institution by anchoring them in the acceptances of the members of that institution.

We say that "in the institutional context x, agent i has the obligation to perform action α for agent j under the condition ϕ" (noted Obl x (i, α, j, ϕ)) if and only if "in the institutional context x, the fact that i does not perform action α for j under the condition ϕ counts as a violation at the next step of time". 6 Formally:

Obl x (i, α, j, ϕ) def = (ϕ ∧ [i:α:j]⊥) x ⊲ XV EXAMPLE 5.
Formula Obl EBay (i, sendGoods, j, paid (j , i )) expresses that, in the context of EBay, it is obligatory for an agent i to send certain goods to agent j under the condition that j has paid the goods to i.

We here distinguish an obligation which is valid in a certain institution x (viz. institutional obligation) from the instantiations of this obligation in specific groups of agents which are members of this institution. We say that i's obligation to do action α for j is instantiated in the group C of members of institution x (noted IObl C:x (i, α, j)) if and only if the agents in C, qua members of the institution x, accept that if i does not perform action α for j then i will incur a violation in the next state. Formally:

IObl C:x (i, α, j) def = ¬Accept C:x ⊥ ∧ Accept C:x ([i:α:j]⊥ → XV )
The following Theorem highlights the relationships between institutional obligations and instantiations of obligations in groups.

Theorem 1

⊢ L (Obl x (i, α, j, ϕ) ∧ ¬Accept C:x ⊥ ∧ Accept C:x ϕ) → IObl C:x (i, α, j)
5 Our notion of constitutive rule of the form ϕ

x ⊲ ψ is similar to the notion of proper classificatory rule given in [START_REF] Grossi | Classificatory aspects of counts-as: An analysis in modal logic[END_REF]. 6 Formulas of type Obl x (i, α, j, ϕ) can be conceived as particular instances of so-called regulative rules in Searle's sense [START_REF] Searle | The Construction of Social Reality[END_REF]. On the distinction between regulative rule and constitutive rule see also [START_REF] Boella | Regulative and constitutive norms in normative multiagent systems[END_REF].

According to Theorem 1, if the agents in C, qua members of institution x, accept that ϕ and, in x, i has the obligation to perform action α for agent j under the condition ϕ then, i's obligation to do action α for j is instantiated in the group C of members of x. EXAMPLE 6.From the fact that in EBay, it is obligatory for an agent i to send certain goods to agent j under the condition that j has paid the goods to i (noted Obl EBay (i, sendGoods, j, paid (j , i ))), and the fact that the group C of EBay surfers accept that j has paid certain goods to i (noted ¬Accept C:EBay ⊥ ∧ Accept C:EBay paid (j , i )), we infer that i's obligation to send the goods to j is instantiated in the group C of EBay surfers (noted IObl C:EBay (i, sendGoods, j)).

Social commitment

Social commitment has a fundamental role in the interaction between agents in an institution. It fixes the relations between agents, bounding an agent toward another. Social commitment is thus a relational notion: it relies at least two agents, the agent who is committed (the debtor [START_REF] Fornara | Agent communication and artificial institutions[END_REF]) and the agent to whom the debtor is committed (the creditor). According to Castelfranchi [START_REF] Castelfranchi | Commitment: from individual intentions to groups and organizations[END_REF], there are two crucial aspects of social commitment: a motivational aspect and a deontic aspect. If the debtor i is committed to the creditor j to perform action α for him then, the creditor and the debitor must mutually know that the creditor is interested in the fact that the debtor performs α (motivational aspect) and, the creditor must have specific rights on the debtor being entitled to ask to the debtor to perform action α (deontic aspect). 7In the present analysis we specify these two conditions for social commitment on the basis of the notions of acceptance and instantiated obligation. Differently from Castelfranchi, we make explicit the institutional dimension of social commitment by specifying the institution in which the commitment is established. On the one hand, we characterize the deontic aspect of i's commitment to j to perform action α by the fact that i and j function as members of a certain institution x and, as members of x, i and j accept that i is obliged to perform action α for j. That is, i's obligation to do action α for j is instantiated in the group {i, j} of members of x. On the other hand, we characterize the motivational aspect of i's commitment to j by the fact that i and j, qua members of the institution x, accept that j wants i to do α for him. Formally: SC x (i, α, j) def = IObl {i,j}:x (i, α, j) ∧ ¬Accept {i,j}:x ⊥ ∧ Accept {i,j}:x Goal j i:α:j ⊤ which stands for "i is committed to j to do α w.r.t. institution x."

In opposition to [START_REF] Singh | A social semantics for agent communication languages[END_REF] and works on ACLs [START_REF] Fornara | Agent communication and artificial institutions[END_REF][START_REF] Mallya | An algebra for commitment protocols[END_REF][START_REF] Flores | Conversational Semantics with Social Commitments[END_REF], in this work social commitment is not taken primitive and is anchored in group attitudes. In particular, our notion of social commitment is grounded on the acceptance of the debtor and the creditor qua members of a certain institution.

Colombetti et al. [START_REF] Fornara | Agent communication and artificial institutions[END_REF] have developed an ACL semantics based on a primitive notion of commitment and its dynamics (through its states change). For example, when an agent promises to another one to perform an action, he creates a pending commitment, that becomes fulfilled (resp. violated) if the agent performs (resp. do not perform) the action. The commitment can also be canceled by the creditor. In Colombetti et al.'s account, nothing is said about the consequences of the cancelation of a commitment. In our characterization, after the creditor utters that he does not want anymore the action to be performed by the debtor, the debtor's commitment is dropped. But the the debtor's obligation might remain instantiated. Indeed, formula ¬SC x (i, α, j) ∧ IObl {i,j}:x (i, α, j) is satisfiable in our logic.

Application to the promise

Speech acts can be viewed as institutional facts represented by constitutive rules. In this section, we focus on a particular kind of speech act: the promissive. We first define the constitutive rule which creates the relation between an utterance (as a physical action) and the speech act promise (as an institutional action). Then, we specify the deontic dimension of promise. Finally, we establish the relationship between promise and social commitment.

According to [START_REF] Searle | Speech acts: An essay in the philosophy of language[END_REF], in the context of ordinary communication (noted OC ) the speech act promise is defined on the basis of a constitutive rule of the form: a speaker's enunciation of the utterance: "I am going to perform action a" counts as a promise to the hearer to perform action a, under the condition that the hearer wants the speaker to perform action a. We here consider the constitutive rule defining the speech act promise with respect to the institutional context of ordinary communication (OC ) as a global axiom [START_REF] Fitting | Proof Methods for Modal and Intuitionistic Logics[END_REF]. For every i, j ∈ AGT and a ∈ AT we suppose that:

(Inf i,j (a) ∧ Goal j i:a:j ⊤) OC ⊲ X Promise(i ,a,j ) (1)
where Promise(i ,a,j ) is an atom denoting the institutional fact that i has promised to j to do action a for him. The previous constitutive rule says that: in the context of ordinary communication, for every agents i, j and atomic action a, i's act of informing j that is going to perform a for him, under the condition in which j wants i to do a, counts as a promise of i to j to perform a for him. The reason why we have X Promise(i ,a,j ) instead of Promise(i ,a,j ) in the consequent of the previous counts-as assertion is that the atom Promise(i ,a,j ) represents the institutional effect of i's act of informing j that is going to perform a for him. Thus, Promise(i ,a,j ) must necessarily hold after the occurrence of i's act of informing. Note that the previous constitutive rule defining the speech act promise in the context of ordinary communication can not be generalized to all institutional contexts. For example, in the context of the card game of poker the enunciation of the utterance: "I am going to perform a certain action" does not necessarily count as a promise of the speaker's to the hearer to perform the action in question. Indeed, in the context of poker players are allowed to bluff. More generally, the following formula should be acceptable for the context of poker: ¬((Inf i,j (a) ∧ Goal j i:a:j ⊤) Poker ⊲ X Promise(i ,a,j )). By definition, institutional facts are connected to a deontic dimension. In particular, an institutional fact is intrinsically connected to certain normative facts expressed in terms of obligations and permissions. For example, "being of age" is an institutional fact in many countries to which a certain number of permissions and obligations are associated (e.g. in many countries if you are of age you have the permission to vote and the obligation to fulfill the military duties). Therefore, since Promise(i ,a,j ) is an institutional fact, it must be connected to certain normative facts. The following global axiom is given in order to establish such a connection with the normative level of promise. For every i, j ∈ AGT and a ∈ AT we suppose that:

Obl OC (i, a, j, Promise(i ,a,j ))

Obl OC (i, a, j, Promise(i ,a,j )) means that: in the context of ordinary communication, under the condition that i has promised to j to perform action a for him, i is obliged to perform action a for j.

Theorem 2

⊢ L (¬Accept {i,j}:OC ⊥ ∧ Accept {i,j}:OC Goal j i:a:j ⊤∧ Accept {i,j}:OC Promise(i ,a,j )) → SC OC (i, a, j)

This theorem highlights the relationship between promise and social commitment in the context of ordinary communication: the fact that in the context of ordinary communication i and j accept that i has promised to agent j to do action a for him and that j wants i to do a for him entails the fact that i is committed to j to do a.

Conclusion

We have introduced a logic of acceptance, action and goal and provided an institutionbased semantics of the commissive act of promise. As far as we know, the only approach which is similar to ours is Colombetti's approach [START_REF] Colombetti | An analysis of agent speech acts as institutional actions[END_REF] where speech acts are also modeled in terms of constitutive rules of the form "X count as Y " and of the institutional effects brought about their performances. However, in Colombetti's approach, there is no connection between institutional level and agentive level of speech acts and the relationship between constitutive rules defining speech acts and agents' attitudes is not investigated. In our approach this relationship is established by anchoring constitutive rules and obligations in agents' acceptances.

In the future will extend our analysis to directive speech acts (not treated here due to space restrictions). To this end, we will need to model the hierarchy between roles in an institution which entitles an agent playing a certain role (e.g. the employer of a company) to give orders to other agents playing other roles (e.g. the employees).

By "enunciation", we mean "the performance of an utterance".

For the distinction between formal and informal institutions, see also[START_REF] North | Institutions, Institutional Change, and Economic Performance[END_REF].

Axiom D for Goal i is: ¬(Goal i ϕ ∧ Goal i ¬ϕ).

Note that X satisfies the standard property Xϕ ↔ ¬X¬ϕ.

Castelfranchi also requires the condition of i and j's mutual knowledge about i's intention to perform α for j. We do not include this condition here since in our view it is too strong. Indeed, after a promise, the social commitment can persist even if the debtor drops the intention that he manifested.

ABSTRACT

The aim of this paper is to provide an analysis of speech acts as institutional actions, where an institution is grounded on the attitudes of its members. In the first part of the paper we present a logic of acceptance, goal and action. Then, we specify how agents can create and maintain normative and institutional facts on the basis of their acceptances qua members of a certain institution. Finally, we propose a formal characterization of the speech act promise.


S Marat

T Ho Phuoc

L Granjon

N Guyader

D Pellerin

A Guérin-Dugué

MODELLING SPATIO-TEMPORAL SALIENCY TO PREDICT GAZE DIRECTION FOR SHORT VIDEOS

Keywords: Saliency, Spatio-temporal model, Gaze prediction, Video viewing

This paper presents a spatio-temporal saliency model that predicts eye movement during video free viewing. This model is inspired by the biology of the first steps of the human visual system. The model extracts two signals from video stream corresponding to the two main outputs of the retina: parvocellular and magnocellular. Then, both signals are split into elementary feature maps by cortical-like filters. These feature maps are used to form two saliency maps: a static and a dynamic one. These maps are then fused into a spatio-temporal saliency map. The model is evaluated by comparing the salient areas of each frame predicted by the spatio-temporal saliency map to the eye positions of different subjects during a free video viewing experiment with a large database (17000 frames). In parallel, the static and the dynamic pathways are analyzed to understand what is more or less salient and for what type of videos our model is a good or a poor predictor of eye movement.

INTRODUCTION

Usually, people do not look at every object in the visual field but concentrate on some salient regions. In the visual field, the spatial regions which attract attention, and therefore the eyes, are usually called salient. The emerging problem is how to design a model that puts salient areas in conspicuous locations. The answer relates to modeling human visual attention with saliency maps; this has been of growing interest to many researchers for the last few decades. The saliency of a spatial location depends mainly on two factors: one is taskindependent and the other is task-dependent. The first one is often called bottom-up and is mainly driven by low-level processes depending on the intrinsic features of the visual stimuli. The latter refers to top-down processes. It is more complex to model because it integrates high-level processes (task, cognitive state...) [START_REF] Henderson | Human gaze control during real-world scene perception[END_REF].

Most computational models of visual attention are bottom-up and are inspired by the concept of Feature Integration Theory (FIT) of Treisman and Gelade [START_REF] Treisman | A feature-integration theory of attention[END_REF]. The first model was described by Koch and Ullman [START_REF] Koch | Shifts in selective visual attention: towards the underlying neural circuitry[END_REF]; like most of the models, it concentrates on spatial image features such as color, contrast, orientation... Several models [START_REF] Le Meur | A coherent computational approach to model bottom-up visual attention[END_REF], [START_REF] Tsotsos | Modeling visual attention via selective tuning[END_REF] are inspired by this theory; the most popular is the one proposed by L. Itti et al. [START_REF] Itti | A Model of Saliency-Based Visual Attention for Rapid Scene Analysis[END_REF] and it has become a reference for all research on saliency. Motion feature has been added to this model more recently [START_REF] Peters | Applying computational tools to predict gaze direction in interactive visual environments[END_REF] and to others [START_REF] Le Meur | Predicting visual fixations on video based on low-level visual features[END_REF], [START_REF] Ma | A generic framework of user attention model and its application in video summarization[END_REF] to create saliency models for videos.

Following a similar approach, a spatio-temporal bottomup saliency model is proposed. This model differs from existing ones on several points: • The model of the two outputs of the retina which provides two different signals: using a retina model, the signal processed by the static pathway differs from the one processed by the dynamic pathway. The useful information is separated to provide more efficient signals to both pathways [START_REF] Palmer | Vision science: Photons to phenomenology[END_REF]. • The compensation of the camera motion: using camera motion compensation, we detect only the areas that move against the background. Not only are moving areas detected, but we define a motion contrast map by estimating the module of the motion for each pixel. • The method of fusion of static and dynamic saliency maps: a new fusion to combine the static and the dynamic pathway outputs is proposed. This fusion modulates the different saliency maps with adaptive coefficients for each frame. These coefficients were chosen by analyzing simple statistics (mean, maximum and skewness) on both outputs. We classified the videos using these statistics and we analyzed in detail to what extent low-level descriptors may contribute to the guidance of eye movement.

For this model, we only concentrated on some basic features: signal orientations and spatial frequencies for the static saliency, and the module of motion for the dynamic saliency. We chose to concentrate only on these basic features and not to add color, stereo or other features, first, because the chosen features are predominant for saliency and, second, because we wanted to understand better how these features are correlated with human eye movement and how to combine them to create a spatio-temporal saliency map. Other features could be added in further research.

The proposed model is described in section 2. Section 3 presents an experiment that records the eye movements of fifteen people looking at a large number of videos (17000 frames). In section 4, an evaluation of the proposed model is drawn, and after a detailed analysis of the static and dynamic pathways a new fusion method is presented to combine both outputs to create a spatio-temporal saliency map.

MODEL

The proposed model is inspired by the first steps of the human visual system, from the retina cells to the complex cells of the primary visual cortex. The visual information goes through the retina preprocessing to the cortical-like filter de-Figure 1: Schema of the proposed spatio-temporal saliency model composition [START_REF] Palmer | Vision science: Photons to phenomenology[END_REF], [START_REF] Massot | Model of frequency analysis in the visual cortex and the shape from texture problem[END_REF]. The retina extracts two signals from each frame that correspond to the two main outputs of the retina [START_REF] Beaudot | The neural information in the vertebra retina: a melting pot of ideas for artificial vision[END_REF]. Each signal is then decomposed into elementary features by a bank of cortical-like filters [START_REF] Devalois | Orientation and spatial frequency selectivity: properties and modular organization[END_REF]. These filters are used to extract both static and dynamic information, according to their frequency selectivity, providing two saliency maps: a static and a dynamic one. Both saliency maps are combined to obtain a master spatio-temporal saliency map per video frame (Fig. 1). This map predicts the gaze direction to particular areas of the frame analyzed.

Retina model

The retina, which has been described in detail in [START_REF] Beaudot | Realistic simulation tool for early visual processing including space, time and colour data[END_REF], [START_REF] Beaudot | The neural information in the vertebra retina: a melting pot of ideas for artificial vision[END_REF], [START_REF] Schwartz | Visual perception: a clinical orientation[END_REF], is composed of different neural layers. The flow of information goes from the photoreceptors to the horizontal cells that provide a local average of the incoming information. The bipolar cells take the difference of the outputs of the photoreceptors and the horizontal cells. Amacrine cells provide a second local average of the bipolar cells output.

The retina has two outputs formed by different ganglion cells: parvocellular output and magnocellular output. Parvocellular output provides detailed information which can be simulated by extracting the high spatial frequencies of an image. This output enhances frame contrast, which attracts human gaze in static frame [START_REF] Reinagel | Natural scene statistics at the center of gaze[END_REF]. Magnocellular output responds rapidly and provides global information which can be simulated by using lower spatial frequencies. The proposed model (Fig. 2) decomposes the input frame into different frequency bands: a high spatial frequency one to provide a "parvocellular-like" output and a lower spatial frequency Figure 2: Retina model one to simulate the "magnocellular-like" output (Fig. 3).

The retina "parvocellular-like" output

First, luminance coming from the real word is captured by the photoreceptors which act as a low-pass filter with a high cut-off frequency. Horizontal cells play the role of a lowpass filter of the photoreceptor's output and are modeled by a gaussian filter.

Bipolar cells calculate the difference of the outputs of the photoreceptors and the horizontal cells, which corresponds to a high-pass filtering of the frame. Bipolar cells On retain the positive part of this difference while bipolar cells Off retain the absolute value of the negative part:

bipolar On = max {0, y -h} bipolar O f f = max {0, h -y}
The output of the bipolar cells is given by the difference of the bipolar cells On and the bipolar cells Off :

bipolar cells = bipolar On -bipolar O f f
The output of the ganglion cells, formed by the bipolar cells, is used to model the parvocellular output of the retina. Therefore, the "parvocellular-like" output reveals frame contrast and helps to whiten its spectrum. This output is the first stage of the static pathway of the model (Fig. 3(c)).

The retina "magnocellular-like" output

Human beings see stable and moving components in a moving scene effortlessly. An object tracked by the camera is seen as moving even if it is stationary on the frames. We assume that visual attention is attracted by motion contrast and we define it as the motion of regions against background. The first step, before the retina filter, is the compensation of the background motion to estimate the relative motion of regions against background.

Background is supposed to represent more than half of the frame's pixels. In this case, background motion is also called dominant motion and is computed using the 2D motion estimation algorithm developed in [START_REF] Odobez | Robust multiresolution estimation of parametric motion models[END_REF]. This algorithm provides dominant motion compensation between two successive frames by carrying out a robust multi-resolution estimation of an affine parametric motion model. The parametric model chosen here is an affine one with 6 parameters:

v x = a 1 + a 2 .
x + a 3 .y v y = a 4 + a 5 .x + a 6 .y where (a 1 ,...,a 6 ) are the estimated parameters and v x and v y are the vectorial components of the dominant motion computed at position (x, y) using the previous parameters.

After the camera motion compensation, the two frames (the current frame and the next compensated frame) go through the retina filter. The bipolar cells calculate the difference between the photoreceptors and the horizontal cells' outputs. This difference acts as a high pass filter that whitens the energy spectrum of the frame. Then, the amacrine cells act as a low-pass filter that eliminates high frequencies (gaussian filter). The resulting equivalent filter is a band-pass filter. This output corresponds to the Magnocellular output of the retina and is the first stage of the dynamic pathway of the model (Fig. 3(d)).

Cortical-like filters

Visual information is decomposed into different spatial frequencies, orientations, colors and motion in the primary visual cortex (V1) [START_REF] Hubel | Functional architecture of macaque visual cortex[END_REF], [START_REF] Devalois | Orientation and spatial frequency selectivity: properties and modular organization[END_REF], [START_REF] Daugman | Two-dimensional spectral analysis of cortical receptive field profiles[END_REF]. In this model, we choose to not study color information and Gabor filters are used to model V1 cells to extract frequencies, orientations and motion information. These filters are a good compromise of resolution between the frequential and spatial domains. Each filter G i j (Eq.1), at orientation i and at frequency j, is determined by its central radial frequency f j and its standard deviations σ θ i j and σ f i j in orientation θ j and its orthogonal ori- entation, respectively i = 1, .., N θ , j = 1, .., N f and

f j f j-1 = 2 with f N f = 0.25. We chose σ θ i j = σ f i j , which is justified in the next section.
The number of orientations and frequencies were respectively fixed at N θ = 6 and N f = 4, for the static pathway according to preliminary experiments (Fig. 4). For the dynamic pathway, the spatial resolution is lower; so only the three low frequency bands were used ( f 1 , f 2 and f 3 .)

G i j (u, v) = exp - (u ′ -f j ) 2 2σ f i j 2 + v ′2 2σ θ i j 2 (1)
with:

u ′ = ucos(θ i ) + vsin(θ i ) v ′ = vcos(θ i ) -usin(θ i )
The output of each filter corresponds to an intermediate map m i j . These maps are the equivalent of some of the elementary features of Treisman's Theory [START_REF] Treisman | A feature-integration theory of attention[END_REF].

The static pathway

Interactions between filters

Neuron responses in the primary visual cortex are influenced by other neurons as far as excitation and inhibition are concerned. We considered two types of interactions based on the range of the receptive fields. Short interactions reinforce objects belonging to a specific orientation while long interactions are used for contour facilitation [START_REF] Hansen | Recurrent long-range interactions in early vision[END_REF]. Short interactions introduce inhibition between neurons of neighboring orientations and overlapping receptive fields. For the standard deviations of the cortical-like filters, if σ θ i j > σ f i j it is more orientation-selective but reduces the inhibitive interaction. So, we chose σ θ i j = σ f i j . Short interactions occur with the same pixel in different intermediate maps m i j . Each pixel is excited by similar pixels in the other maps of the same orientation but different frequencies and inhibited by those of different orientations but similar frequency (Fig. 5).

The second interaction type is long range interaction which occurs among collinear neurons beyond the receptive fields. This type of interaction is worked out in each intermediate map by convolution with a "butterfly" mask [START_REF] Hansen | Recurrent long-range interactions in early vision[END_REF]. This mask (Fig. 6) consists of an excitory part in the corresponding orientation of the intermediate map m i j and an inhibitive part in other orientations. It was normalized in such a way that its summation was equal to one. The mask size is inversely proportional to the frequency of the corresponding intermediate map m i j .

Normalization and summation

A region is salient if it is different from its neighbors. Thus, to strengthen the intermediate maps that have spatially distributed maxima, the method proposed by Itti [START_REF] Itti | A Model of Saliency-Based Visual Attention for Rapid Scene Analysis[END_REF] is used. After being normalized in [0, 1], each map m i j was multiplied by (max(m i j )m i j ) 2 where max(m i j ) and m i j are its maximum and average respectively. Then, all values in each map that were smaller than 20% of its maximum were set to 0.

Finally, all intermediate maps were added together to obtain a static saliency map M s (x, y, k) for each frame k (Fig. 7

The dynamic pathway

Dynamic saliency is linked to motion and particularly to the motion of a region against the background. The speed of moving region against background was computed using a motion estimator on compensated frames at the "Magnocellular-like" output of the retina.

Motion estimation

A differential approach, described in detail in [START_REF] Bruno | Robust motion estimation using spatial Gabor-like filters[END_REF], was used. It relies on the assumption of luminance constancy.

The motion at location (x,y) in frame t is given by vector V(x,y,t) which satisfies the optical flow constraint equation (Eq. 2)

∇I(x, y,t).V (x, y,t) + ∂ I(x, y,t) ∂t = 0 (2)
where I(x,y,t) is the luminance of the pixel at the position (x,y) in the frame t.

For each frame, the optical flow constraint was applied to each output of the cortical-like filters, with the same radial frequency, leading to an over-determined system of equations allowing the aperture problem to be overcome. For each pixel (x,y), a motion vector (v x , v y ) was computed, solving the system (Eq. 3) with a least square estimation using Biweight Tuckey's function.

   Ω x 1 Ω y 1 . . . . . . Ω x N θ Ω y N θ    . v x v y = -    Ω t 1 . . . Ω t N θ    (3)
where

Ω p i = ∂ (I * G i. ) ∂ p , G i.
is one of the cortical-like filter at the orientation i, and I is the "Magnocellular-like" output of the retina. The optical flow constraint creates an accurate estimation only for low motion. A robust multiresolution scheme is needed to estimate a large scale of speed. A first approximation of motion was done with frames at coarse resolution to estimate fast motion; this displacement was then compensated and the residual motion was estimated at a finer resolution.

A motion vector was defined (per pixel) by its module, corresponding to the speed, and its angle, corresponding to the motion direction. As we assume the motion saliency map of a region is linked to its speed against background, we only used the module of this motion vector to define the dynamic saliency of the area.

Temporal filtering

A temporal median filter was applied to remove noise. If a pixel had a motion in one frame but not in the previous ones, it is most probably noise resulting from the motion estimation. This temporal filter was applied on five successive frames (the current frame and the four previous ones) and the filter was reinitialized after each shot cut to avoid artifacts. A dynamic saliency map M d (x, y, k) was obtained for each frame k (Fig. 7(a), 7(c)).

Fusion

The saliency maps obtained at the outputs of the static and the dynamic pathways do not have the same range of values. To carry out the fusion, the raw saliency information, without normalization, was retained to take advantage of this difference and to promote the more accurate saliency map. However, the range of values of the static saliency map and the dynamic one is compatible. Neither of these two kinds of maps had systematically outperforming values. For the static saliency map the normalization was done on the intermediate maps m i, j , the maximum of static saliency values were around 1.7 and could go up to 2.7. The dynamic saliency maps, had to respect the theorem of shannon in the temporal domain, the maximum speed would be limited, considering the frame rate. The maximum of dynamic saliency values was around 2.7 and could reach 9. Three different fusions were proposed:

• a mean fusion, taking the pixel average of the two saliency maps:

M mean = M s +M d 2
• a max fusion, taking for each pixel the maximum of the two saliency maps: Examples of these fusions are given in Fig. 8. The M mean fusion modulates one map with the other. If an area is salient for the static map but not for the dynamic one, the fusion saliency is lower than it was in the static one. For the M max fusion, an area has the highest saliency between static and dynamic maps and is less selective. In the M and there are small salient regions, which could be spread all over the saliency map. This multiplicative fusion is the most selective one. In this case, an area needs to be salient simultaneously in the static and the dynamic maps to be salient in the fused map. The usually used M mean [START_REF] Milanese | Integration of bottom-up and top-down cues for visual attention using non-linear relaxation[END_REF] and M max give saliency maps that are close. For the latter, results are presented with the M and fusion only as it gives the best results experimentally.

M max = Max(M s , M d ) • a pixel

EXPERIMENT

The goal of this part is to compare the results given by our model to the human eye position density map obtained through an eye movement experiment. Both bottom-up and top-down influence eye position. The bottom-up saliency model proposed is used to quantify the contribution of lowlevel saliency to eye movements. To better prevent top-down processes, we did not use classical videos but instead we used small clips, as was done by Itti [START_REF] Carmi | Visual causes versus correlates of attentional selection in dynamic scenes[END_REF]. The aim is to remove all semantic content from videos as far as possible. For that, videos were split into small clips and these clips, from different video sources, were put together [START_REF] Marat | Spatio-temporal saliency model to predict eye movements in video free viewing[END_REF].

Participants: Fifteen human observers (3 women and 12 men, aged from 23 to 40 years old). All participants had normal or corrected to normal vision, and were not aware of the purpose of the experiment. They were asked to look at videos without any particular task.

Apparatus and experimental design: Eye tracking was performed by an Eyetracker Eyelink II (SR Research). During the experiment participants were sitting with their chin supported in front of a 21 ′′ color monitor (75 Hz refresh rate) at a viewing distance of 57 cm (40 • × 30 • usable field of view). A 9-point calibration was carried out every five stimuli and a control drift was done before each stimuli.

Stimuli: This experiment is inspired by an experiment of Carmi and Itti [START_REF] Carmi | Visual causes versus correlates of attentional selection in dynamic scenes[END_REF]. Fifty-three videos (25 fps, 720 × 576 pixels/frames) were selected from heterogeneous sources including movies, TV shows, TV news, animated movies, commercials, sport, music clips. These 53 videos gathered indoor, out-door, daytime and night-time sources. The 53 videos were cut every 1-3 seconds (1.86 ± 0.61) into 324 clip snippets. The length of the clip snippet was chosen randomly, the only constraint was to obtain a snippet without any shot cuts. These clip snippets were then strung to form 20 clips of 30 seconds (30.20 ± 0.81). Each clip contained at most one clip snippet from each continuous source. The choice of the clip snippets and their duration were random to prevent subjects anticipating shot cuts. As the proposed model is bottom-up, clip snippets were used to minimize potential top-down influence on eye movements. Stimuli (17000 frames) were presented on gray level without audio as the model did not consider color and audio information.

Human eye position density maps: We recorded and analyzed the eye positions. The eyetracker records the eye position at 500 Hz. The eyetracker records 20 eye positions per frame for the two eyes. The median of all these positions was taken (with X-axis median and Y-axis median) for each subject and for each frame. For each frame the points of all the subjects were gathered. Then we applied a 2D gaussian function to each point to obtain the human eye position density map, M h (x, y, k). The standard deviation of the gaussian was chosen to have a diameter at half the height of the gaussian equal to 0.5 • of visual angle.

RESULTS

We analyzed the eye positions rather than the fixation points for two reasons. First, we had more data when choosing all the eye positions, and so we could extract one point per frame and per subject. Second, in most of the cases, eye positions and fixations are very close except during smooth pursuit. So by retaining all the eye positions we obtained data even during smooth pursuit. The aim is to compare the salient areas given by the model, with the fixated areas.

Various criteria have been proposed and used for this comparison: the Kullback-Leibler distance [START_REF] Rajashekar | Point of gaze analysis reveals visual search strategies[END_REF] or the Receiver Operator Curve (ROC) [START_REF] Tatler | Visual correlates of fixation selection: effects of scale and time[END_REF] (others examples can be found in [START_REF] Carmi | Visual causes versus correlates of attentional selection in dynamic scenes[END_REF], [START_REF] Le Meur | Predicting visual fixations on video based on low-level visual features[END_REF], [START_REF] Torralba | Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search[END_REF]). In this paper, we chose to focus on the correlation coefficient and the Normalized Scanpath Saliency (NSS) [START_REF] Peters | Components of bottom-up gaze allocation in natural images[END_REF]. This last criteria was especially designed to study eye movement data and so, the corresponding results can be easily interpreted. The correlation coefficient and the NSS lead to the same conclusion on the data analysis. However, the correlation coefficient is very dependent on the standard deviation of the gaussian applied to gaze positions to compute the human eye position density map. The NSS was therefore preferred. The NSS criteria is a Z-score, (also called standard score). This Z-score expresses the divergence of the experimental result from the model mean as a number of standard deviations of the model. The larger the value of Z, the less probable it is that the experimental result is due to chance. The NSS is computed using the equation:

NSS(k) = M h (x, y, k) × M m (x, y, k) -M m (x, y, k) σ M m (x,y,k) (4)
where M h (x, y, k) is the human eye position density map normalized to obtain unit mean, and M m (x, y, k) the model saliency map. The model saliency map can be the static, dynamic or fused maps. This is equivalent to normalize each saliency map to have a zero mean and a unit standard deviation and to retain, on the normalized saliency map, the value corresponding to gaze locations of subjects and then to average the retained values over subjects. If the mean of the saliency values at eye position is equal to the mean of saliency value on the whole frame (the NSS is null), there is no link between eye position and saliency. If the mean of the saliency values at eye position is lower than the mean of saliency values on the whole frame (the NSS is negative), eye position tends to be on non-salient regions. If the mean of the saliency values at eye position is higher than the mean of saliency values on the whole frame (the NSS is positive), eye position tends to be on salient regions.

Global analysis

The NSS was computed for each frame of every clip (17000 frames). In order to test if our model is a good predictor of human eye movements, the mean NSS value is calculated using the model saliency map (static, dynamic, fusion of both) and the experimental data. To compare our model we used two sets of experimental data. For the first set, we associated to each frame of a clip snippet the corresponding eye movement of subjects when they were looking at the videos. This first set is called the real eye movements, in opposition to the second set called the partially randomized eye movements. For this second set, we associated to a frame the eye movement of subjects when they were looking at another clip snippet. We only kept the frame position inside a clip snippet. This second comparison is to ensure that our model predicts the salient areas of specific frame and not simply predicts subjects eye movements without any correlation with the content of the frame. If our model is a good predictor of salient areas and because the NSS value was defined to compare computational saliency map with eye movements, we should observe low values of NSS when comparing our model with partially randomized eye movements and high NSS values when comparing our model with real eye movements. Using partially randomized eye movements also prevents the effect of central bias of eye position on model evaluation [START_REF] Tatler | Visual correlates of fixation selection: effects of scale and time[END_REF]. Partially randomized eye movements were obtained using subject eye position and not using random sampling: the same bias as real eye movement was then observed. Thus, the difference of NSS between real eye movement and partially randomized eye movement is not due to the fact that subjects are more likely to stare at the center of the image than to look at the image randomly. The mean NSS value is given for three models of saliency maps (static, dynamic and the fusion of both) in comparison with real eye movements and the partially randomized eye movements, in Table 1.

Saliency maps M s M d M and

Real eye movements 0.68 0.87 0.96 Partially randomized eye movements 0.33 0.14 0.14 Table 1: Mean NSS value on all the clips for the three models of saliency maps (static, dynamic and multiplicative fusion)

Comparing M and with real eye movement gives the best results. The mean of the saliency values at the eye position was around one standard deviation away from the mean of saliency values on the whole frame. As expected, NSS values are higher when comparing the three models with real eye movements than when comparing with partially randomized eye movements (F(1,84968)=10497.07; p≡0).

If we analyze what happened with the partially randomized eye movements, the mean NSS value for the static saliency map is more than twice the mean NSS value for the dynamic saliency map. This can be explained by the fact that, the static and the dynamic saliency maps have different appearance. For most of the frames, the static saliency map highlights areas spread over the whole frame. In fact all, the frames represent naturalistic scenes with textured area. On the contrary, the dynamic saliency map can exhibit small and compact areas corresponding to moving objects. By so doing, random eye position is, on average over all frames, more likely to be on a salient region in the static saliency map than in the dynamic saliency map. If all the subjects had the same eye movement pattern, the results may be more closely linked to this pattern than to the actual saliency on the videos. By testing our model with partially randomized eye movements, we can see that there is no such common eye scan path, and that the high mean NSS is caused by the relevance of the saliency model and not a plausible strategy of eye position during video viewing.

Moreover the dynamic pathway gives better results than the static one (F(1,28328)=275.40; p≡0). The fusion gives the best result (F(2,42482)=276.06; p≡0), both pathways are needed to obtain improved results.

The model is also compared to simple heuristics as it is usually done in the literature [START_REF] Carmi | Visual causes versus correlates of attentional selection in dynamic scenes[END_REF], [START_REF] Reinagel | Natural scene statistics at the center of gaze[END_REF]. This comparison tests if our model is more accurate in predicting saliency than a simple model only based on low level image descriptors such as luminance. Two static naive heuristics and one dynamic naive heuristic were tested. The two static naive heuristics were the entropy H and the standard deviation SD of pixels' luminance. The image was split into patches of 16 × 16 pixels and the entropy or standard deviation was computed on each patch. This value was propagated on the corresponding pixels of the patch to form an entropy saliency map M snH and a luminance standard deviation saliency map M snSD . A gaussian filter was applied to each map to spread the patch border effect. The dynamic naive heuristic M dn was the absolute difference of the pixels of two consecutive frames. This difference highlights the moving pixels. Note that no dominant motion compensation was done before. In fact, we just wanted to compare our model with a simple naive heuristic and so without using motion 2D [START_REF] Odobez | Robust multiresolution estimation of parametric motion models[END_REF] which is an elaborate preprocessing.

The mean NSS value is given for the saliency map obtained for the three simple heuristics in comparison with real eye movements, in Table 2. The proposed fused saliency model (mean NSS = 0.96 (Table 1)) gives more accurate saliency prediction than the simple heuristics tested (F(3,56658)=1046.64; p≡0).

Saliency map

M snH M snSD M dn Real eye movements 0.54 0.44 0.54 Table 2: Mean NSS value on all the clips for two static naive heuristics (entropy H and standard deviation SD of pixels' luminance) and one dynamic naive heuristic (absolute difference of the pixels of two consecutive frames)

All these observations were in agreement with our expectation and means that our model can predict real eye movements. Research had shown that motion is the feature that attracts the human gaze the most. This has been shown experimentally but also using saliency models [START_REF] Carmi | Visual causes versus correlates of attentional selection in dynamic scenes[END_REF], [START_REF] Wolfe | Guided search: an alternative to the feature integration model for visual search[END_REF]. As expected, the NSS value is higher when comparing dynamic saliency to static. However, we need to take into account both pathways to obtained the highest NSS value.

Temporal analysis

The previous results are an average over all frames. It can be interesting to see if there is an evolution of the NSS value during clip snippets. Such a study was carried out previously for static images [START_REF] Torralba | Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search[END_REF], [START_REF] Parkhurst | Modeling the role of salience in the allocation of overt visual attention[END_REF] and for videos [START_REF] Le Meur | Predicting visual fixations on video based on low-level visual features[END_REF], [START_REF] Carmi | Visual causes versus correlates of attentional selection in dynamic scenes[END_REF]. Because the proposed model is a bottom-up one, it can only predict human gaze for the first fixations of subjects when looking at a static image or for the first few frames when looking at a video. Figure 9 presents the NSS value as a function of frame position inside a clip snippet for the static saliency maps M s , the dynamic ones M d and the fusion of both M and with real eye movements. We also present the results for partially randomized eye movements for the static pathway M s ′ and for the dynamic one M d ′ . The curves with partially randomized eye movements do not have the same shape as the curves with real eye movements, as the first ones hardly vary with time. These curves present very low NSS values that are typical of no correspondence between the model and the experimental data. All the other curves obtained for real eye movements have the same shape. The maximum NSS value is reached for all the curves (Fig. 9) at about the 13 th frame, which corresponds to 520 ms, then curves decrease slowly. The shape of these curves can be explained by the fact that, at the beginning, only bottom-up influences occurred, followed by top-down processes. Studies have found that bottom-up influences act faster than top-down processes [START_REF] Henderson | Human gaze control during real-world scene perception[END_REF], [START_REF] Wolfe | Attention is fast but volition is slow[END_REF]. If this is the case, we should observe that bottom-up influences occur before top-down processes. In [START_REF] Parkhurst | Modeling the role of salience in the allocation of overt visual attention[END_REF], [START_REF] Carmi | Visual causes versus correlates of attentional selection in dynamic scenes[END_REF] saliency effects were stronger just after the stimuli onset than later on, while in [START_REF] Tatler | Visual correlates of fixation selection: effects of scale and time[END_REF] no saliency dependencies on time were found. Our results are in accordance with [START_REF] Parkhurst | Modeling the role of salience in the allocation of overt visual attention[END_REF], [START_REF] Carmi | Visual causes versus correlates of attentional selection in dynamic scenes[END_REF].

We analyze the bottom-up influence with another indicator: the dispersion of eye position between subjects. This dispersion is plotted for all the clip snippets (Fig. 10). The dispersion D is defined by: where N is the number of subjects, d i, j is the distance between the eye position of subject i and j. A low value of dispersion corresponds to close fixations by subjects. Low values of dispersion are more probably explained by bottomup influences: the saliency is given by the intrinsic features of the stimuli and is the same for all observers. On the other hand, the top-down processes involve the cognitive state and prior knowledge of each subject, and tend to be different for different subjects. The dispersion curve is in accordance with the NSS curve. With bottom-up influence, subjects look at salient regions and the dispersion of their gaze decreases as NSS increases. Top-down processes occur too, and dispersion increases as subjects gaze at different regions, NSS decreases.

D = 1 N 2 ∑ i, j<i d 2 i, j
During the first few frames, NSS is low and dispersion is high. This can be explained by the fact that after each shot cut, the gaze stays at the previous position during a few frames and then moves to a salient region. We can assume that there is some time shifting between the time a region, present on the screen, is salient and the time this region is fixed by subjects. Figure 11 shows NSS as a function of time for the dynamic saliency map without and with an offset, i.e. a shift of three frames (=120 ms) between the saliency map and the fixation considered (the NSS of the first frame of a clip snippet was computed considering the third fixation frame). With the offset, the curve is shifted to left, which means a greater correspondence between the human fixations and the saliency model earlier in the clip. This result was expected as we know there is a saccade latency around 150 ms.

The duration of preponderance of bottom-up influences on static images reach 150 to 300 ms after stimulus onset [START_REF] Tsotsos | The different stages of visual recognition need different attentional binding strategies[END_REF], [START_REF] Egeth | Visual attention: control representation and time course[END_REF]. If we take into account the offset, the first maximum of the NSS curve corresponds to the first minimum of the dispersion curve and is about 8 frames; this corresponds to 320 ms which is above the usual reported time of 150ms for static images. However, video stimuli are used instead of still images, and to our knowledge, no time values for the predominance of bottom-up influences using video stimuli were previously reported.

The evolution of the NSS value as a function of the frame position inside a snippet fits well with the fact that (1) our model is a bottom up model so it can only predict eye movements for the first frames, and (2) the dispersion of the eye positions of subjects increases with frame position (Fig. 10).

Detailed analysis of the two pathways

The proposed model is a good predictor of the eye movements of subjects when looking freely at clip snippets. However, this model is a better predictor for the first frames of a snippet. It can be interesting to inquire now what is more attractive in the static saliency map and what is more attractive in the dynamic saliency map.

As we said before, the static and the dynamic saliency maps do not have the same appearance. On one side, the static saliency map exhibits a large number of salient areas, corresponding to textured areas, which are spread over the whole image. On the other side, the dynamic saliency map can exhibit only small and compact areas corresponding to moving objects. Concerning the question "what is more salient in the static and the dynamic pathways?" we can suppose:

• for the static map: a frame would be salient if its static saliency map has a high value and not if its static saliency is spread. The saliency of a frame would be correlated with the maximum of its corresponding static saliency map. • for the dynamic map: a frame would be salient if its dynamic saliency map has small and compact areas. The saliency of the frame would be linked to the number and the size of the salient areas in its corresponding dynamic saliency map.

To test these two hypotheses, we introduced two statistics: the maximum and the skewness of the saliency map [START_REF] Marat | Spatio-temporal saliency model to predict eye movements in video free viewing[END_REF]. The maximum is characteristic of the static map. The skewness is characteristic of the dynamic saliency map. They should indicate the saliency prediction efficiency of the map.

In fact, we observed that a dynamic saliency map with a high skewness corresponds, in general, to a map with only small and compact areas. Whereas a dynamic saliency map with a low skewness value corresponds to a map with spread salient areas, or to a map with a small salient area but with different motion values (different gray levels). These observations correspond to the fact that skewness is a measure of the degree of asymmetry of a distribution; here, the distribution of the gray values of dynamic saliency map pixels is considered. If a frame contains a small moving area, its dynamic saliency map would exhibit only a small area. Its distribution would have a high and sharp peak close to zero and another peak, smaller and wider, close to the motion value; this would induce a mean greater than the mode and so, a high skewness. On the other hand, for a frame with a lot of moving areas, the dynamic saliency map distribution would still have a peak close to zero but would have another more spread out peak around the motion value (more spread than in the previous example); this would reduce the asymmetry and so decrease the skewness. The skewness is also decreased in the case of a small area with a diffuse motion. These examples are illustrated in figure 12.

Using these two statistics, on the static saliency map, in one part, and on the dynamic saliency map, in a second part, we classified our videos into four groups: 1)high skewness high maximum, 2) high skewness low maximum, 3) low skewness high maximum, 4) low skewness low maximum. Each snippet is labeled 1) 2) 3) or 4) on static saliency Figure 13: NSS as a function of frame for clip snippets categorized using maximum and skewness for static saliency map map (Fig. 13) and on the dynamic one (Fig. 14). The NSS is plotted as a function of frame for the four categories of snippets. If our hypotheses are verified, we should observe the highest NSS values for the videos with a high maximum value for the static saliency map and the highest NSS values for the videos with a high skewness value for the dynamic saliency map. The static pathway is more predictive for snippets with high maximum in static saliency (maximum of the curve above 0.8). The main information is given by the maximum saliency; if a frame has a high maximum saliency value, there is an attractive region in this frame. On the other hand, if the frame has a low maximum saliency the most attractive region is less attractive than in the previous frame (Fig. 15). The dynamic pathway is more predictive for snippets with higher skewness (maximum above 1.1). The dynamic map M d (x, y, k) gives motion information, but now the salient regions may be small (Fig. 12 a,c). If there is only a small moving region, the saliency must be concentrated on this region. If there is only a dynamic salient region, the gaze of all the subjects would be concentrated there. However if there are several regions with equivalent dynamic saliency, subjects' gaze would be spread over these different regions. The fact that NSS is higher with lower motion (low maximum) can be explained by the fact that if the speed of the moving region is too high it is difficult to track it [START_REF] Lisberger | Vi-sual motion processing and sensory-motor integration for smooth pursuit eye movements[END_REF].

A fusion taking advantage of the characteristics of the static and the dynamic saliency maps is then proposed (Fig. 16):

M skew-max = αM s + β M d + γM s × M d with: α = max(M s ) β = skewness(M d ) γ = max(M s ).skewness(M d )
The static pathway is modulated by its maximum value α. The dynamic saliency map is modulated by its skewness value β . The reinforcement term γ gives more importance to the areas that are salient both in a static and dynamic way, and so to the small moving region with high static saliency. This fusion has a mean NSS=1.01 and is significantly above the M and fusion (mean NSS = 0.96 (Table 1)) (F(1,28308)=10.54; p=0.0012).

CONCLUSION

This study presents a new bottom-up saliency model inspired by the biology of the first steps of the human visual system. The model presents a simulation of the two pathways (magnocellular and parvocellular) of the human visual system based on their main known properties. These two pathways can be seen as static and dynamic pathways. A video, coming through the model, is split into spatial and motion information. This split starts with the two main outputs of the retina and continues with the cortical cells, sensitive to different spatial frequencies for the different pathways. At the output of each pathway, a saliency map is extracted. The static and the dynamic saliency maps are fused to create a spatiotemporal saliency map. The model associates a saliency map to each frame of a video. This map is used to predict the areas that would be gazed at by people when looking at the videos.

In order to test the exactness of the proposed model, we ran an experiment to record eye movements of subjects when looking freely a large base of videos. We did not use "classical" videos, but instead, inspired by Itti's experiment, we used small clip snippets [START_REF] Carmi | Visual causes versus correlates of attentional selection in dynamic scenes[END_REF]. This allowed us to compare experimental data with our model, which is a bottom-up one.

In fact, the model can predict the gaze of people for the first few frames of a video. Different comparisons to validate the model were carried out. First, the output of the model was compared with subject's gaze and with partially random gazes. We also compared the model with other simple heuristics. Each comparison showed that our model is significantly better than the others.

Moreover, we studied the evolution of the model prediction as a function of time (for frames inside each small clip snippet). The model is more accurate at the beginning of a clip snippet. This expected result can be explained by two facts: (1) our model is a bottom up model, which means that it can only predict the fixations of people for the first frames of a clip, (2) for the first frames all the subjects were looking at the same locations (the dispersion between their fixation points was weak) but they were looking at different locations at the end of a clip snippet (higher dispersion).

We showed that the dynamic map is more predictive than the static one. However the fusion of both maps gives the best results. We also showed that the static saliency covers spread areas on a frame, on the contrary, the dynamic saliency map gives motion information on areas that can be small. If there was only a small moving region, the saliency would be concentrated on this region. The maximum is then more relevant for the analysis at the static saliency maps and the skewness is more relevant for dynamic saliency maps. These characteristics are then used to compute a new fusion M skew-max taking advantage of both pathways. Saliency maps are modulated with maximum and skewness information and a reinforcement term that gives more importance to the areas that are salient both in statically and dynamically, so to the areas with localized moving region with high static saliency.

In this model, we chose to concentrate only on basic features that are predominant for static saliency. The efficiency of these features has been shown and we understood the importance of each feature better. In future work it would be interesting to add more features such as color or a spatially varying sampling of the retina depending on eye positions to reinforce our model. The model could also be used to improve video compression or be added to camera motion analysis [START_REF] Guironnet | Video summarization based on camera motion and a subjective evaluation method[END_REF] to help select frames for a summary of the video.

Figure 3 :

 3 Figure 3: Retina model: a) Input image, b) Horizontal cell response, c) "Parvocellular-like" output, d) "Magnocellularlike" output.

Figure 4 :

 4 Figure 4: Configuration of Gabor filters in the frequential domain: 6 orientations and 4 frequency bands.

  (a), 7(b)).

Figure 5 :

 5 Figure 5: Short interactions for intermediate maps m i j .

Figure 6 :

 6 Figure 6: Butterfly mask used for long interactions.

Figure 7 :

 7 Figure 7: Example of a natural scene (a) with its static saliency map M s (b) and its dynamic saliency map M d (c).

Figure 8 :

 8 Figure 8: Examples of fused saliency maps: (a) M mean , (b) M max and (c) M and of a natural scene (Fig. 7(a)). The NSS are respectively 1.07, 0.67 and 2.20.

  by pixel multiplicative fusion corresponding to a logical and: M and = M s × M d

Figure 9 :Figure 10 :

 910 Figure 9: NSS as a function of frame. NSS is averaged on 324 clip snippets for different saliency maps: static M s , dynamic M d , fusion of both M and , static with partially randomized eye movements M s ′ , dynamic with partially randomized eye movements M d ′

Figure 11 :

 11 Figure 11: NSS as a function of frame for the dynamic saliency map with and without offset.

Figure 12 :

 12 Figure 12: Examples of a natural scene (a) with a dynamic saliency map (b) with low skewness (1.00) and a natural scene (c) with a dynamic saliency map (d) with high skewness (8.62)

Figure 14 :Figure 15 :Figure 16 :

 141516 Figure 14: NSS as a function of frame for clip snippets categorized using maximum and skewness for dynamic saliency map


C A Martínez-Muñoz

C Driller

J F Aldana-Martín

I Navas-Delgado

M Hedde

D Russell

Domain Ontologies to Explore and Manage Functional Soil-invertebrate Diversity

Background

Due to their high structural and functional diversity, soil invertebrates are crucial for ecosystem functioning and its stability. This diversity is reflected not only in the multitude of quantitative and qualitative studies, but also in the range of existing vocabularies of characters and character states for functional soil biodiversity. Data mining and text mining are increasingly used to scour large data resources and systematically extract relevant information in a machine-based manner. Controlled vocabularies play a central role in applying these methods to represent implicit and explicit knowledge in so-called knowledge graphs in a structured and machineand human-readable way.

Visit BIOfid at DFG Grant MO 412/54-2

Implementation

The Specialized Information Service Biodiversity Research (BIOfid) [1] aims at this method of data mobilization from Central European biodiversity literature. In cooperation with the European Soil-Biology Data Warehouse for Soil Protection (EUdaphobase), differing vocabularies on the taxonomy, anatomy, ecology and traits of soil invertebrates are currently being harmonized and mapped onto domain-specific ontologies. In addition to data extraction, these ontologies should also support the interoperable and sustainable management of pan-European soil biodiversity data.

A | B

Schematic example of linked BIOfid-EUdahobase ontologies. The Chilopoda Anatomy Ontology (ChilAO) is linked to the Eudaphobase Taxonomy Ontology, which can also support ontologies of ecological interactions between centipedes and other terrestrial invertebrates.

Figure

  Figure [2]

Figure

  Figure [4]