Yassir Idmessaoud
email: yassir.id-messaoud@laas.fr

Didier Dubois
email: dubois@irit.fr

Jérémie Guiochet
email: jeremie.guiochet@laas.fr

A qualitative counterpart of belief functions with application to uncertainty propagation in safety cases

Keywords: Goal Structuring Notation, argument structures, confidence elicitation, Dempster-Shafer Theory, qualitative capacities

Critical systems such as those developed in the aerospace, railway or automotive industries need official documents to certify their safety via convincing arguments. However, informal tools used in certification documents seldom cover the uncertainty that pervades safety cases. Several works use quantitative approaches based on belief functions to model and propagate confidence/uncertainty in the argument structures (particularly those using goal structuring notation). However the numerical uncertainty information is often a naive encoding of qualitative expert inputs. In this paper, we outline a qualitative substitute to Dempster-Shafer theory and suggest new qualitative confidence propagation models. We also propose a more faithful encoding of expert inputs.

Introduction

As the use of artificial intelligence (AI) in systems increases, the need for safety assessment methods in the latter is also increasing. However, the lack of confidence can jeopardize the social acceptance of these systems and therefore their existence. Several approaches are used to assess confidence/uncertainty in such systems (especially, the safety critical ones).

Many papers addressing the assessment of safety of systems rely on the graphical representation of an argument structure like GSN (Goal Structuring Notation), plus quantitative representations of uncertainty. Typically, probability theory is often used in Bayesian network models of GSNs. In order to address the issue of incomplete information, Dempster-Shafer theory (DST) is also proposed. In the latter case, argument trees can be modelled in classical logic using if-then rules [START_REF] Idmessaoud | Belief functions for safety arguments confidence estimation: A comparative study[END_REF][START_REF] Idmessaoud | Quantifying confidence of safety cases with belief functions[END_REF].

However the quantification of uncertainty is often problematic, when it relies on expert assessments. In many cases, experts supply qualitative assessments using linguistic values like probable, very probable, unlikely, etc., which are then translated into numbers on the [0, 1] scale. This translation is somewhat arbitrary. So, a legitimate question is whether a purely qualitative approach to uncertainty, that would be a counterpart to the belief function approach, could be promising. The idea is to avoid the quantitative encoding of qualitative estimates. It makes all the more sense as numerical degrees of belief obtained via uncertainty propagation are often translated back to the qualitative scale, so as to make the results more palatable. So it is legitimate to investigate a qualitative approach.

This paper is a first step in this direction. It is structured as follows. Section 2 presents theoretical background on qualitative capacities that can be viewed as a qualitative counterpart of belief functions, based on [START_REF] Dubois | Qualitative capacities: Basic notions and potential applications[END_REF]. Section 3 deals with the elicitation of qualitative capacities, based on an existing method where linguistic term scales were mapped to belief functions. Section 4 use qualitative belief measures on classical inference patterns. Section 5 recalls the a graphical representation called Goal Structuring Notation (GSN), dedicated to argument structures for safety cases. This section applies the qualitative uncertainty propagation method from premises to conclusions of several types arguments. In Section 6, a preliminary comparison of qualitative and quantitative uncertainty propagation is proposed via an example.

From belief functions to qualitative capacities

As a generalization of probability theory, Dempster-Shafer theory [START_REF] Shafer | A mathematical theory of evidence[END_REF](DST) offers tools to model and propagate both aleatory (due to random events) and epistemic (due to incomplete information) uncertainty.

A mass function, or basic belief assignment (BBA), is a probability distribution over the power set of the universe of possibilities (W ), known as the frame of discernment. Formally, a mass function m : 2 W → [0, 1] is such that E⊆W m(E) = 1, and m(∅) = 0. Any subset E of W such as m(E) > 0 is called a focal set of m. m(E) quantifies the probability that we only know that the truth lies in E; in particular m(W ) quantifies the amount of ignorance.

A mass assignment induces a so-called belief function Bel : 2 W → [0, 1], defined by: Bel(A) = E⊆A m(E). It represents the sum of all the masses supporting a statement A. The degree of belief in the negation ¬A of the statement A is called disbelief: Disb(A) = Bel(¬A); the value U ncer(A) = 1-Bel(A)-Disb(A) quantifies the lack of information about A.

The conjunctive rule of combination combines multiple pieces of evidence (represented by mass functions m i , with i = 1, 2) coming from independent sources of information:

m ∩ = m 1 ⊗ m 2 such that: m ∩ (A) = E1∩E2=A m 1 (E 1 ) • m 2 (E 2 ).
In DST, an additional step eliminates conflict that may exist by means of a normalization factor (dividing m ∩ by 1 -m ∩ (∅)). This is Dempster rule of combination [START_REF] Shafer | A mathematical theory of evidence[END_REF], which is associative.

In contrast, we outline the qualitative approach in [START_REF] Dubois | A possibilistic counterpart to Shafer evidence theory[END_REF][START_REF] Dubois | Qualitative capacities and their informational comparison[END_REF][START_REF] Dubois | Qualitative capacities: Basic notions and potential applications[END_REF]. Let L be a finite totally ordered set representing certainty levels. A qualitative capacity (q-capacity, for short) is a function γ : 2 W → L such that:

γ(∅) = 0; γ(W ) = 1; A ⊆ B ⇒ γ(A) ≤ γ(B)
. Any q-capacity can be put in the form:

γ(A) = max ∅̸ =B⊆A ρ(B), ∀A ⊆ W, (1)
where ρ is formally a basic possibility assignment (BΠA) [START_REF] Dubois | A possibilistic counterpart to Shafer evidence theory[END_REF], namely, a possibility distribution ρ : 2 W → L on the power set of W , such that max B⊆W ρ(B) = 1 and ρ(∅) = 0. The value ρ(B) is the strength of piece of evidence B. Several BΠA's can generate the same γ, the least of which is the qualitative Moebius transform (QMT) of γ such that:

γ # (A) = γ(A) if γ(A) > γ(A \ {w}), ∀w ∈ A; 0 otherwise. ( 2
)
The value γ(A) (resp. γ(¬A)) qualifies the support in favor of (resp. against) A, i.e. belief (resp. disbelief) in A using an element in the qualitative scale L.

The pair (γ(A), γ(¬A)) thus describes our epistemic stance with respect to A in terms of belief and disbelief, ranging from no information (i.e., (0, 0)), to full conflicting information (i.e., (1, 1)), from full belief (i.e., (1, 0)) to full disbelief (i.e., (0, 1)). This is more general than possibility theory where the case (1, 1) is not allowed. Figure 1 presents the credibility and information orderings on pairs (belief, disbelief) including extreme cases [START_REF] Dubois | Qualitative capacities: Basic notions and potential applications[END_REF]. A proposition A is at least as credible as B if γ(A) ≥ γ(B) and γ(¬A) ≤ γ(¬B) (solid arrows from B to A), thus ranging from certainty of falsity (0, 1) up to certainty of truth (1, 0). A proposition A is at least as informed as B if γ(A) ≥ γ(B) and γ(¬A) ≥ γ(¬B) (dotted arrows from B to A), thus ranging from ignorance ((0, 0), no information) up to conflict ((1, 1), full contradictory information). In this situation, the amount of evidence supporting the conclusion is equal to the one rejecting it. The set L × L is then equipped with a bilattice structure. In order to qualitatively combine pieces of evidence represented by possibilistic mass functions, i.e., BΠA's ρ i , coming from several sources of information, the qualitative counterpart of the conjunctive rule of combination for belief functions is:

ρ ∩ = ρ 1 ⊕ ρ 2 such that: ρ ∩ (A) = max E1∩E2=A {min[ρ 1 (E 1 ), ρ 2 (E 2 )]} (3)
Due to the use of the (idempotent) minimum operation, the combined pieces of evidence are not supposed to be independent. The result is not always a BΠA, strictly speaking. First we may have that ρ ∩ (A) < 1 for all A. So we must add the condition ρ ∩ (W ) = 1. This will not occur if we restrict to non-dogmatic BΠA's such that ρ i (W ) = 1, which we assume in this paper. Besides, we may have that ρ ∩ (∅) > 0, indicating conflict between the pieces of evidence.

Expert elicitation approach

In order to elicit qualitative capacities, we borrow from a methodology by Cyra and Gorski [START_REF] Cyra | Support for argument structures review and assessment[END_REF]. Two types of information are collected from experts about a statement A: A so-called decision and a level of confidence associated to it. Then, these pieces of information are numerically encoded, and transformed to belief and disbelief degrees in the sense of Shafer (see also [START_REF] Idmessaoud | Quantifying confidence of safety cases with belief functions[END_REF]). More precisely:

-The decision index denoted by Dec(A), describes which side the assessor leans towards, i.e., acceptance or rejection of A. It is associated with a bipolar scale The bipolar scale D is equipped with an order-reversing map ν D such that ν D (d -i ) = d i . Especially we have that ν D (Dec(A)) = Dec(¬A). The unipolar scale K is isomorphic to the positive part of D. This assumption makes K and D commensurate. K is equipped with an order-reversing map ν K such that:

D = {0 D = d -k , d k-1 , . . . , d 0 = e,
ν K (C i ) = C k-i .
In order to switch from a (Dec(A), Conf (A)) pair to (γ(A), γ(¬A)), we use a transformation that maps D × K to the belief-disbelief scale L × L containing pairs (γ(A), γ(¬A)). The scale L has the same number of elements as K (i.e., 3 here). The mapping f : D × K → L × L: must satisfy some conditions [START_REF] Dubois | Qualitative capacities: Basic notions and potential applications[END_REF]:

-If the expert declares lack of confidence, the result is f (Dec(A), 0) = (0, 0), whatever the trend expressed on the decision scale. -If the expert is fully confident, then f (1, 1) = (γ(A), γ(¬A)) = (1, 0), f (0, 1) = (0, 1), f (e, 1) = (1, 1). Indeed, for the latter, there is a total conflict: the expert is maximally informed (Conf (A) = 1), and cannot decide between A and its negation (Dec(A) = e). max(γ(A), γ(¬A)) = Conf (A): the belief in A or its negation cannot be stronger than the confidence.

-if Dec(A) is the midpoint of D, then γ(A) = γ(¬A)(= Conf (A)) (no reason to take side). -if Dec(A) is less than the midpoint of D, then γ(A) < γ(¬A) = Conf (A),
and the smaller Dec(A), the smaller γ(A).

-if Dec(A) is greater than the midpoint of D, then γ(A) = Conf (A) > γ(¬A),
and the greater Dec(A), the smaller γ(¬A).

These conditions lead to propose the following translation formulas [START_REF] Dubois | Qualitative capacities: Basic notions and potential applications[END_REF]:

if Dec(A) < e, γ(A) = min[ν K (Dec(¬A)), Conf (A)] and γ(¬A) = Conf (A); if Dec(A) > e, γ(A) = Conf (A) and γ(¬A) = min[ν K (Dec(A)), Conf (A)]; if Dec(A) = Dec(¬A) = e, γ(A) = γ(¬A) = Conf (A).
In Table 1, we grouped all possible (Dec, Conf ) pairs on premises with their appropriate counterparts (γ(A), γ(¬A))∈ L × L, using the formulas above. We can notice an anti-symmetry between belief and disbelief degrees regarding the central column (D 0 = e: no decision). We also notice that when no information is available (C 0 : Lack of confidence), no matter what choice is made the degrees of belief and disbelief take a minimal value. On the other hand, in the case of a fully informed expert (C 2 : Full confidence) the decision value varies from rejection to acceptance.

C0 (Lack of confidence) (0,0) (0,0) (0,0) (0,0) (0,0) C1 (Moderate confidence) (0,λ) (λ,λ) (λ,λ) (λ,λ) (λ,0) C2 (Full confidence) (0,1) (λ,1) (1,1) (1,λ) (1,0)
4 Logical inference for qualitative capacities

Logical reasoning and numerical belief functions are not often put together. An approach to reasoning with Dempster rule of combination was proposed in [START_REF] Chatalic | An approach to approximate reasoning based on Dempster rule of combination[END_REF].

In this approach each formula in a knowledge base is viewed as a simple support function and combined with other formulas in the knowledge base. Besides, the application of belief functions to argument structures has been studied in [START_REF] Cyra | Support for argument structures review and assessment[END_REF][START_REF] Wang | Safety case confidence propagation based on Dempster-Shafer theory[END_REF][START_REF] Idmessaoud | Quantifying confidence of safety cases with belief functions[END_REF] to build models for uncertainty propagation. For instance in [START_REF] Idmessaoud | Quantifying confidence of safety cases with belief functions[END_REF], we assign mass functions to logical expressions such as facts p i , ¬p i , and rules p i ⇒ C, ¬p i ⇒ ¬C, (∧ n i=1 p i ) ⇒ C and (∧ n i=1 ¬p i ) ⇒ ¬C, in order to deduce the belief on the conclusions C and ¬C. Here we develop the same approach, albeit using qualitative capacities.

The simplest pattern is modus ponens, i.e. inferring C from p and p ⇒ C. We assume two BΠA's ρ p on {p, ¬p} with values in L, say ρ p (p) = α p , ρ p (¬p) = α p , ρ p (⊤) = 1, and a simple support function ρ ⇒ with ρ ⇒ (p ⇒ C) = β ⇒ , ρ ⇒ (⊤) = 1, where ⊤ stands for the tautology. The capacity γ C is obtained via projection for the conclusion C by the q-conjunctive rule has a BΠA ρ C such that:

γ C (C) = ρ C (p∧C) = min(ρ p (p), ρ ⇒ (p ⇒ C)) = min(α p , β ⇒ ), γ C (¬C) = 0.
If p = n i=1 p i , then the above formula holds with ρ p (p) = min n i=1 ρ i p (p i ). If there is also a BΠA ρ ⇐ assigning a weight β ⇐ to the reversed implication ¬p ⇒ ¬C there is an additional weight on ¬C via the combination ρ p ⊕ ρ ⇒ ⊕ ρ ⇐ using equation (3) and projection on C's universe.

γ

C (¬C) = ρ C (¬p ∧ ¬C) = min(ρ p (¬p), ρ ⇐ (p ⇐ C)) = min(α p , β ⇐ ).
Consider the case with more than one premise. Suppose we have to merge BΠA's

ρ i p on p i , ρ i ⇒ , ρ i ⇐ , i = 1, . . . n.
As in its quantitative counterpart, the BΠA pertaining to the conclusion C obtained from this fusion may assign a mass to the contradiction. Conflict always appears when four items are merged of the form: p i and p i ⇒ C with ¬p j and ¬p j ⇒ ¬C, j ̸ = i, whose conjunction is a contradiction ∅ with mass:

ρ ij C (∅) = min[ρ i C (p i ∧ C), ρ j C (¬p j ∧ ¬C)] = min[ρ i p (p i ), ρ i ⇒ (p i ⇒ C), ρ j p (¬p j ), ρ j ⇒ (¬p j ⇒ ¬C)]]
For two premises, the final mass on contradiction is ρ C (∅) = max(ρ 12 C (∅), ρ 21 C (∅)). Besides, using (1) we get:

γ C (C) = max[ρ C (p 1 ∧ C), ρ C (p 2 ∧ C)] ≥ ρ C (∅) and γ C (¬C) = max[ρ C (¬p 1 ∧ ¬C), ρ C (¬p 2 ∧ ¬C)] ≥ ρ C (∅).

Application to safety cases

Goal structuring notation (GSN) is a graphical notation/language which represents argument structures (i.e., safety and assurance cases) in the form of directed acyclic graphs (directed trees or arborescences). It breaks down a top claim, called "goal", into elementary sub-goals following a specific strategy and in accordance with a particular context. Each sub-goal is associated with pieces of evidence, called solutions, which support the conclusion. Despite the fact that it presents all the evidence supporting the safety of the system, GSN fails to show how premises support the conclusion and the confidence that can be given to them. Both questions bring uncertainty to arguments, which may affect their merits. To address this issue, confidence propagation schemes were proposed to complement GSN patterns. Some approaches use DST to model and propagate confidence in GSN patterns in the literature [START_REF] Cyra | Support for argument structures review and assessment[END_REF][START_REF] Idmessaoud | Quantifying confidence of safety cases with belief functions[END_REF][START_REF] Wang | DS theory for argument confidence assessment[END_REF]. These papers consider a number of argument types and associate confidence propagation formulas to each of them. In practice, they also devise transformation formulas that turn uncertainty assessments of experts (on a qualitative scale) about premises to numerical belief and disbelief degrees. This transformation is a source of uncertainty. Indeed, qualitative inputs are often naively translated into equidistant values in the unit interval. Therefore, the qualitative approach to uncertainty developed in [START_REF] Dubois | A possibilistic counterpart to Shafer evidence theory[END_REF][START_REF] Dubois | Qualitative capacities and their informational comparison[END_REF][START_REF] Dubois | Qualitative capacities: Basic notions and potential applications[END_REF], and the elicitation and the inference methods of Sections 3 and 4 may lead to more robust confidence assessment approaches.

Here, we use the argument types defined in [START_REF] Idmessaoud | Quantifying confidence of safety cases with belief functions[END_REF]. An argument type describes the interaction between premises to support a conclusion. This type of interaction is either a conjunction (C-Arg), a disjunction (D-Arg), or a combination of both (H-Arg, a hybrid type). We can translate each argument type into logical expressions often called rules. Since we use only implication to describe links between the universe of premises (W p = {p, ¬p}) and that of the conclusion (W C = {C, ¬C}), two kinds of rules are used: direct rules, which model the acceptance of the conclusion (C), and reverse rules which model its rejection (¬C). Then, to each rule, we assigned a simple support function (a mass on the rule, and another on the tautology). We also assigned masses to the premises and their negation. Finally the propagation formulas, for each type, are obtained using the qualitative combination rule [START_REF] Dubois | A possibilistic counterpart to Shafer evidence theory[END_REF]. Below, we recall our argument types and associate to each of them to qualitative uncertainty propagation formula.

Simple argument (S-Arg):

This argument describes the case of a conclusion (C) supported by a single premise (p), hence the name "simple". If the premise is true, then so is the conclusion: p ⇒ C. Note that only the information about the acceptance of the conclusion can be inferred in this situation. Since we work on a three-state paradigm (belief, disbelief and uncertainty), the reverse rule ¬p ⇒ ¬C is introduced to add conditions for the possible denial of the conclusion. Then, we associate to the direct and reverse rules simple BΠA's (resp., ρ ⇒ and ρ ⇐ ), and a BΠA on the premise space, as done above. We can prove:

S-Arg : γ C (C) = min[γ p (p), γ ⇒ (p ⇒ C)] γ C (¬C) = min[γ p (¬p), γ ⇐ (¬p ⇒ ¬C)] (4)
We can notice that the belief γ C (C) depends only on the direct rule and the acceptance of the premise, while the disbelief γ C (¬C) only depends on the reverse rule and the disbelief of the premise.

Conjunctive argument (C-Arg): This argument type describes the situation when two premises or more are jointly needed to support a conclusion. We formally defined its direct and reverse rules (resp.) by: (∧ n i=1 p i ) ⇒ C and ∧ n i=1 (¬p i ⇒ ¬C). Following the same reasoning of the previous argument type, we put a simple BΠA on each rule (ρ ⇒ and ρ i ⇐ ), and another BΠA on each premise (ρ i p ). Then we combine them with the rule of combination (ρ = ρ r ⊕ ρ p , with

ρ p = ρ 1 p ⊕ ... ⊕ ρ n p and ρ r = ρ ⇒ ⊕ (⊕ n i=1 ρ i ⇐ )) and get: C-Arg : γ C (C) = min{min n i=1 γ i p (p i ), γ ⇒ ([∧ n i=1 p i ] ⇒ C)} γ C (¬C) = max n i=1 {min[γ i p (¬p i ), γ i ⇐ (¬p i ⇒ ¬C)]} (5)
In the formulas of the quantitative approach [START_REF] Idmessaoud | Quantifying confidence of safety cases with belief functions[END_REF] operations a + b -ab and ab replace max, min, respectively, thus highlighting the similarity between the results obtained from each model. Indeed, we can notice that the C-Arg, like its quantitative counterpart, favors the propagation of the premise with the least strength (minimal belief, with a maximal disbelief degree).

Disjunctive argument (D-Arg):

In this situation, each premise can support alone the whole conclusion. Formally, the direct and reverse rules are defined as follows:

∧ n i=1 (p i ⇒ C) and (∧ n i=1 ¬p i ) ⇒ ¬C.
The calculation of γ C (C) and γ C (¬C) is identical to the one above, swapping the two expressions:

D-Arg : γ C (C) = max n i=1 {min[γ i p (p i ), γ i ⇒ (p i ⇒ C)]} γ C (¬C) = min{min n i=1 γ i p (¬p i ), γ ⇐ ([∧ n i=1 ¬p i ] ⇒ ¬C)} (6)
We can notice that this model, as its quantitative counterpart [START_REF] Idmessaoud | Quantifying confidence of safety cases with belief functions[END_REF], favors the propagation of the premise with the greatest strength (maximal belief and minimal disbelief degree).

Hybrid argument (H-Arg): This argument type describes the situation where each premise supports the conclusion to some degree, but their conjunction does it to a larger one. Therefore, all conjunctive and disjunctive rules will be used in this argument type. Thus, we obtain: H-Arg :

       γ C (C) = max{min[min n i=1 γ i p (p i ), γ ⇒ ([∧ n i=1 p i ] ⇒ C)], max n i=1 (min[γ i p (p i ), γ i ⇒ (p i ⇒ C)]} γ C (¬C) = max{min[min n i=1 γ i p (¬p i ), γ ⇐ ([∧ n i=1 ¬p i ] ⇒ ¬C)], max n i=1 min[γ i p (¬p i ), γ i ⇐ (¬p i ⇒ ¬C)]} (7)
We can notice that eq.( 7), presents a combination between C-Arg formulas [START_REF] Dubois | Qualitative capacities: Basic notions and potential applications[END_REF], and D-Arg [START_REF] Idmessaoud | Belief functions for safety arguments confidence estimation: A comparative study[END_REF]. Assuming a maximal belief (= 1) (resp. disbelief) on premises, it is enough that the simple direct rules take a null value (resp. the reversed conjunctive one) to get the conjunctive argument type. And conversely, to get the disjunctive argument type, put null values on direct conjunctive and simple reversed rules. The S-Arg, represent a special case when only one premise is available (n = 1). In the following, only the H-Arg will be used since it covers the four types.

Application example

On an artificial example (Figure 3) that displays three argument types (C-Arg, D-Arg and H-Arg), we apply our approach in order to see how each type affects the propagation of uncertainty from premises to the overall goal (conclusion). We also apply the quantitative approach presented in [START_REF] Idmessaoud | Quantifying confidence of safety cases with belief functions[END_REF] on the same example. To compare results from both approaches, we will use the same decision and confidence scales (see Figure 2).

Regarding elicitation, we use the evaluation matrix in Figure 2 to collect expert opinions, and transform them using formulas in Section 3 to get belief and disbelief on premises. Regarding the elicitation of belief weight on rules, we benefit from an observation made on the quantitative models [START_REF] Idmessaoud | Quantifying confidence of safety cases with belief functions[END_REF]. Indeed, we notice that under some assumptions for the premises, the value of the conclusion is the value of the rule. For instance, assuming full support (resp. positive or negative) on all premises gives the value of the conjunctive rule (resp. direct and reversed):

γ C (C) = γ ⇒ ([∧ n i=1 p i ] ⇒ C) or γ C (¬C) = γ ⇐ ([∧ n i=1 ¬p i ] ⇒ ¬C).
On the other hand, assuming a total support (resp. positive or negative) on one premise (p i ) and total ignorance on the other gives the value of the appropriate

C (C) = γ i ⇒ (p i ⇒ C) or γ C (¬C) = γ i ⇐ (¬p i ⇒ ¬C)
. So, we will use the same Table 1 to transform the assessment to rules. However, to avoid the negation of rules, the assessor can only choose between the positive decision (from "no decision" to "acceptable") for direct rules; only negative decisions (from "rejectable" to "no decision") for the reversed ones. Indeed, rules can only infer uncertainty on one side of the decision scale.

The example in Figure 3, presents a top-goal (G) supported by two sub-goals (G1) and (G2) through a hybrid argument type (H-Arg). Each one of them is also supported, respectively, by two premises. Goal (G1) is supported by the premises (P1) and (P2) related by a conjunctive argument type (C-Arg). On the other hand, goal (G2) is supported by the premises (P3) and (P4) related by a disjunctive argument type (D-Arg). For simplicity, we set all masses on rules to their maximal values (= 1). Then, we use four settings with different premise values and compute the confidence in the top goal. In general, we can see from Table 2 that both approaches give close results which fit well with our expectations. The only difference is in the confidence values. We can say that, in this case the qualitative approach gives results with higher levels of confidence than the quantitative one.

We notice from Table 2 that the first case gives a "no decision". This result is explained by the fact that we end up with two opposite judgments in the H-Arg (conflict situation) due to C-Arg that propagates the premise with least strength (opposable) to G 1 (G 2 : tolerable). On the contrary, in the 2 nd column, we get a "tolerable" decision, because the D-Arg favors the propagation of the premise with the greatest weight (tolerable) to G 2 (G 1 : tolerable). In the 3 th and 4 th columns we can notice, as expected, that the top goal keeps the same decision as premises respectively: "tolerable" and "opposable".

The difference in the degree of confidence between qualitative and quantitative approaches is due to the nature of the operations used. For example, the C-Arg favors the propagation of the weakest premise (weaker belief and stronger disbelief). In the quantitative setting, we use the product and the probabilistic sum. And in the qualitative case, we use min and max, which does not model attenuation or reinforcement effects in case of independent pieces of information. This is one limitation of the qualitative approach.

Conclusion

In this paper, we propose a qualitative confidence assessment approach. We provide formulas to propagate confidence in GSN from the premises to the topgoal using qualitative mass functions. Each of these functions is collected from experts in the form of a decision and the associated confidence degree, and then transformed into a q-capacity. By sticking to qualitative values, the possible arbitrariness of the transformation of expert opinions into quantitative values (used in some previous approaches) is eliminated. Furthermore, it seems that the qualitative approach gives results similar to the quantitative one in [START_REF] Idmessaoud | Quantifying confidence of safety cases with belief functions[END_REF]. However, more experiments are needed to confirm this conclusion.

Fig. 1 .

 1 Fig. 1. Evolution of certainty and information in pairs (belief, disbelief)

  d 1 , . . . d k = 1 D } with 2k + 1 values, the bottom of which (0 D ) expressing rejection, the top (1 D ) acceptance, and the midpoint (e) a neutral position. Here we assume k = 2.-The confidence index denoted by Conf (A) reflects the amount of information an assessor possesses to support the decision. It uses a positive uni-polar scale K with k + 1 values (the top 1 K expresses full confidence, the bottom 0 K is neutral -no information). For k = 2: these levels mean: lack of confidence (C 0 = 0 K ), moderate confidence (C 1 ) and full confidence (C 2 = 1 K ).

Table 1 .

 1 Values from (Dec, Conf ) to (Bel,Disb) pairs on premises Conf Dec D-2 (Rej) D-1 (Opp) D0(N D) D1 (Tol) D2 (Acc)

D - 2 :Fig. 3 .

 23 Fig. 2. Evaluation matrix

Table 2 .

 2 Pairs (decision, confidence) according to both qualitative (Qual.) and quantitative[START_REF] Idmessaoud | Quantifying confidence of safety cases with belief functions[END_REF] (Quant.) methods for the example (see Fig.2for the meaning of symbols)

		Meth.	1 st	2 nd	3 th	4 th
	P1	-	(Opp;C2) (Tol ;C2) (Tol ;C2) (Opp;C2)
	P2	-	(Tol ;C2) (Tol ;C2) (Tol ;C2) (Opp;C2)
	P3	-	(Tol ;C2) (Tol ;C2) (Tol ;C2) (Opp;C2)
	P4	-	(Tol ;C2) (Opp;C2) (Tol ;C2) (Opp;C2)
	G	Quant. (ND;C0) (Tol ;C1) (Tol ;C1) (Opp;C1) Qual. (ND;C2) (Tol ;C2) (Tol ;C2) (Opp;C2)


A Etienne
email: aline.etienne@parisnanterre.frgwenole.lecorve@orange.com

D Battistelli
email: delphine.battistelli@parisnanterre.fr

G Lecorvé

A (Psycho-)Linguistically Motivated Scheme for Annotating and Exploring Emotions in a Genre-Diverse Corpus

Keywords: emotions, annotation scheme, genre of text, expression mode, French 8. Bibliographical References

This paper presents a scheme for emotion annotation and its manual application on a genre-diverse corpus of texts written in French. The methodology introduced here emphasizes the necessity of clarifying the main concepts implied by the analysis of emotions as they are expressed in texts, before conducting a manual annotation campaign. After explaining whatentails a deeply linguistic perspective on emotion expression modeling, we present a few NLP works that share some common points with this perspective and meticulously compare our approach with them. We then highlight some interesting quantitative results observed on our annotated corpus. The most notable interactions are on the one hand between emotion expression modes and genres of texts, and on the other hand between emotion expression modes and emotional categories. These observation corroborate and clarify some of the results already mentioned in other NLP works on emotion annotation.

Introduction

In NLP, emotion annotation is usually considered as a difficult task, namely because of the lack of consensus on emotional categories (e.g. anger, fear, etc.), the fuzziness of boundaries between them or the great variability of emotion expressions types. By introducing here a scheme made for emotion annotation in texts, we obviously do not claim that we can answer all those difficulties. However we lay claim to an approach that appears to have been very scarcely explored in NLP as of yet. We adopt indeed a fine (psycho-)linguistic modeling perspective which aims at encompassing different concepts essential to the description of the notion of emotion as it is expressed in written texts. In other words, our approach aims at clarifying the main linguistic concepts -and their relations -which underlie the denotation of emotions in texts. The scheme introduced here fills a gap as it combines linguistic and psycho-linguistic notions which were until now used in a scattered way in NLP works.

After clarifying what entails a both psycho-linguistic and linguistic modeling perspective on emotion expression analysis (Section 2), we present a few NLP works that share some common points with this perspective (Section 3). We then introduce the annotation scheme (Section 4) and highlight some quantitative results achieved after the application of this scheme on a corpus of 1,594 texts (ca. 515K tokens) of different genres (journalistic, fictional and encyclopedic) (Section 5). In Section 6, we mention interactions between some notions of the scheme observed from the annotations and confront them with results obtained by other NLP works.

From the notion of emotion to its expression in texts

When NLP works tackle the notion of emotion, they almost always do it by referring to psychological studies that aim at defining what emotions are and at determining a set of basic emotions, which has yet to give rise to a true consensus. These NLP works mainly cite [START_REF] Ekman | An argument for basic emotions[END_REF] which distinguishes between six basic emotions (anger, disgust, fear, joy, sadness and surprise) or [START_REF] Plutchik | A general psychoevolutionary theory of emotion[END_REF] which adds two more categories (trust and anticipation).

To our knowledge, apart from [START_REF] Kim | Who feels what and why? annotation of a literature corpus with semantic roles of emotions[END_REF] which briefly mentions the role of emotions in "literature comprehension" (p. 1346), NLP works do not rely on psycho-linguistic studies. For their part, these studies put the emphasis on how emotions are expressed in texts, and not only on emotional categories. The notion of emotion is then addressed through its relation with text comprehension, seen as a reader's ability to build a mental representation of the situation depicted by the text [START_REF] Zwaan | Situation models in language comprehension and memory[END_REF]. In building this representation, studies point out the importance of several semantic features, such as time, space and characters' emotions, for adults [START_REF] Dyer | The role of affect in narratives[END_REF][START_REF] Dijkstra | Character and reader emotions in literary texts[END_REF] as well as children [START_REF] Blanc | La compréhension des contes entre 5 et 7 ans: Quelle représentation des informations émotionnelles[END_REF][START_REF] Blanc | La production d'inférences émotionnelles entre 8 et 10 ans : quelle méthodologie pour quels résultats ?[END_REF]. Characters' emotions (as opposed to readers' emotions [START_REF] Dijkstra | Character and reader emotions in literary texts[END_REF]) represent then a key element that must be taken into account when investigated text comprehension.

Even though there are few psycho-linguistic studies that consider the linguistic diversity of emotional expressions, they represent a crucial contribution for works that aim at a fine analysis of emotions in texts.

In that respect, [START_REF] Creissen | Quelle représentation des différentes facettes de la dimension émotionnelle d'une histoire entre l'âge de 6 et 10ans ? apports d'une étude multimédia[END_REF] distinguishes between three types of emotional expressions: emotional labels (e.g. afraid, happy), descriptions of an emotional behavior (e.g. cry, laugh) and descriptions of a situation socially associated with an emotion (e.g. someone's death associated with sadness or fear). However they do not offer a detailed linguistic characterization of these expressions. Such characterization actually represents a "real challenge" for linguists (translated from French, (Kerbrat-Orecchioni, 2000, p. 50)) due to the tremendous diversity and heterogeneity of linguistic markers of emotions [START_REF] Kerbrat-Orecchioni | Quelle place pour les émotions dans la linguistique du xxe siècle[END_REF][START_REF] Micheli | Les émotions: Des modes de sémiotisation aux fonctions argumentatives[END_REF].

To our knowledge, only [START_REF] Micheli | Les émotions dans les discours. Modèle d'analyse[END_REF] offers a detailed study of emotion denotation for French which takes into account the diversity of means employed to convey an emotion. [START_REF] Micheli | Les émotions dans les discours. Modèle d'analyse[END_REF] organizes this vast set of heterogeneous markers into a typology of three emotion expression modes: emotions directly labeled by emotional words, emotions displayed by characteristics of utterances, and emotions illustrated by the description of a situation socially associated with an emotion.

From these psycho-linguistic and linguistic studies, four expression modes can thus be distinguished: emotional labels, linguistic characteristics of an utterance that display the speaker's emotion, description of a situation that suggests an emotion, and description of an emotional behavior. In the following sections, these four expression modes will respectively be called: labeled, displayed, suggested, and behavioral. Linguistic works such as [START_REF] Micheli | Les émotions dans les discours. Modèle d'analyse[END_REF][START_REF] Mathieu | Annotation of emotions and feelings in texts[END_REF] also characterize emotion expression by studying the denotation of the experiencer of the emotion, i.e. the entity (human, animal, etc.) which feels the emotion. Furthermore, emotional expressions are usually characterized by the emotional category they convey (JOY, FEAR, etc.). Psycho-linguistic works such as [START_REF] Blanc | La production d'inférences émotionnelles entre 8 et 10 ans : quelle méthodologie pour quels résultats ?[END_REF]) also describe them by the type of emotion they express (basic or complex). Finally, emotional expressions can be linked with other textual segments by semantic relations. On that note, [START_REF] Blanc | La compréhension des contes entre 5 et 7 ans: Quelle représentation des informations émotionnelles[END_REF] suggests a strong relation between causality and characters' emotions in building a mental representation of texts. The notions of cause and consequence of the emotion then take part in describing the expression of an emotion.

The study of emotion denotation in texts thus involves many concepts (expression mode, experiencer, cause, etc.). However those concepts are never taken into account all together in NLP works. These works see emotion expression analysis as a task called emotion annotation, that aims at identifying (portions of) texts conveying an emotional information (cf. Section 3). To carry out this analysis, some focus on emotional categories, others on the cause of emotions, etc. This highlights the difficulty of grasping the notion of emotion as it is effectively realized in texts, when one does not want to focus on emotional words only. The scheme presented in Section 4 uses all concepts introduced above. By making explicit and organizing all these concepts, it aims at modeling the notion of emotion, that is to say taking into account all concepts that underlie exploration of this notion and determining how they are linked together.

Emotion annotation in NLP

Emotion annotation is a difficult task. In NLP, this leads to a great diversity of emotion annotated corpora, as illustrated by several literature reviews. For example, [START_REF] Bostan | An analysis of annotated corpora for emotion classification in text[END_REF] [START_REF] Bostan | An analysis of annotated corpora for emotion classification in text[END_REF] and ( Öhman, 2020) thus clearly show that emotion category constitutes the notion most commonly employed to describe emotional expressions in NLP, with some works also using other notions such as the cause of emotions. However they do not offer a satisfying review of how several concepts of importance to us, namely expression modes, experiencer, and types of annotated units, are taken into account (or not) in emotion annotation schemes used in NLP.

Emotion expression modes. One of the notions essential to characterize emotion is the diversity of emotion expression modes (cf. Section 2). Though this diversity has already been mentioned in NLP works (see for example [START_REF] Bostan | An analysis of annotated corpora for emotion classification in text[END_REF][START_REF] Kim | Who feels what and why? annotation of a literature corpus with semantic roles of emotions[END_REF][START_REF] Demszky | Goemotions: A dataset of fine-grained emotions[END_REF], it has been never conceptualized. On that respect, [START_REF] Alm | Characteristics of high agreement affect annotation in text[END_REF] constitutes the most detailed work. The author lists characteristics of 460 emotional sentences on which there is a high inter-annotators agreement. Though they are not organized into a typology, the cues cited echo the markers found in the expression modes defined in linguistics and psycho-linguistics (see Section 2). For instance, the "affect words" (p. 120) mentioned match the labeled emotions, the "lexical items or phrases which describe actions, properties, behaviors" (e.g. laugh, weep, cry) (p. 120) correspond to the behavioral emotions, the "words or expression of positive or negative polarity" that are interpreted given a particular "context and acquired knowledge" (e.g. splendid flower-garden, little treasure upon earth) (p. 120-121) are similar to suggested emotions, and finally "direct speech" including "(WH)-exclamations or (WH)-questions, short utterances, interjections" are similar to displayed emotions. The scheme proposed in the current paper offers a mean of naming and categorizing those different markers of emotions (see Section 4).

Experiencer of the emotion. As for the experiencer of the emotion, it seems to be widely discarded in NLP. Many works do not explicitly tell whose emotions are annotated [START_REF] Aman | Identifying expressions of emotion in text[END_REF][START_REF] Bianchi | Feel-it: Emotion and sentiment classification for the italian language[END_REF][START_REF] Gui | Event-driven emotion cause extraction with corpus construction[END_REF][START_REF] Russo | EMOCause: An easy-adaptable approach to extract emotion cause contexts[END_REF][START_REF] Ghazi | Detecting emotion stimuli in emotion-bearing sentences[END_REF][START_REF] Liu | DENS: A dataset for multi-class emotion analysis[END_REF], even when the experiencer is featured in the annotation scheme [START_REF] Neviarouskaya | Extracting causes of emotions from text[END_REF]. Works that do not label experiencers but nonetheless clearly state who feels the annotated emotions often focus on writer's emotions [START_REF] Buechel | Emobank: Studying the impact of annotation perspective and representation format on dimensional emotion analysis[END_REF][START_REF] Demszky | Goemotions: A dataset of fine-grained emotions[END_REF][START_REF] Fraisse | Utiliser les interjections pour détecter les émotions[END_REF][START_REF] Mohammad | #emotional tweets[END_REF][START_REF] Mohammad | SemEval-2018 task 1: Affect in tweets[END_REF][START_REF] Öhman | XED: A multilingual dataset for sentiment analysis and emotion detection[END_REF][START_REF] Paroubek | Annotations for opinion mining evaluation in the industrial context of the DOXA project[END_REF]. [START_REF] Buechel | Emobank: Studying the impact of annotation perspective and representation format on dimensional emotion analysis[END_REF] and [START_REF] Strapparava | SemEval-2007 task 14: Affective text[END_REF]) also consider reader's emotions. On the other hand, schemes that label experiencers are mainly centered on characters' emotions [START_REF] Cheng | An emotion cause corpus for chinese microblogs with multiple-user structures[END_REF][START_REF] Kim | Who feels what and why? annotation of a literature corpus with semantic roles of emotions[END_REF][START_REF] Schmidt | Towards a corpus of historical german plays with emotion annotations[END_REF], though [START_REF] Cheng | An emotion cause corpus for chinese microblogs with multiple-user structures[END_REF]) also labels writer's and reader's emotions.

Type of annotated units. Units labeled with emotional information usually are whole texts [START_REF] Paroubek | Annotations for opinion mining evaluation in the industrial context of the DOXA project[END_REF][START_REF] Mohammad | #emotional tweets[END_REF][START_REF] Mohammad | SemEval-2018 task 1: Affect in tweets[END_REF][START_REF] Fraisse | Utiliser les interjections pour détecter les émotions[END_REF][START_REF] Liu | DENS: A dataset for multi-class emotion analysis[END_REF][START_REF] Demszky | Goemotions: A dataset of fine-grained emotions[END_REF][START_REF] Bianchi | Feel-it: Emotion and sentiment classification for the italian language[END_REF] or sentences (Alm et al., 2005;[START_REF] Strapparava | SemEval-2007 task 14: Affective text[END_REF][START_REF] Aman | Identifying expressions of emotion in text[END_REF][START_REF] Neviarouskaya | Extracting causes of emotions from text[END_REF][START_REF] Ghazi | Detecting emotion stimuli in emotion-bearing sentences[END_REF][START_REF] Buechel | Emobank: Studying the impact of annotation perspective and representation format on dimensional emotion analysis[END_REF][START_REF] Öhman | Emotion annotation: Rethinking emotion categorization[END_REF]. It is noteworthy that even though they annotate emotions at text-level, [START_REF] Fraisse | Utiliser les interjections pour détecter les émotions[END_REF] observe how specific linguistic markers, interjections, play a part in emotion expression. It can also be noted that works which proceed to a linguistic marker-level emotion annotation [START_REF] Russo | EMOCause: An easy-adaptable approach to extract emotion cause contexts[END_REF][START_REF] Neviarouskaya | Extracting causes of emotions from text[END_REF][START_REF] Cheng | An emotion cause corpus for chinese microblogs with multiple-user structures[END_REF][START_REF] Kim | Who feels what and why? annotation of a literature corpus with semantic roles of emotions[END_REF][START_REF] Gui | Event-driven emotion cause extraction with corpus construction[END_REF][START_REF] Schmidt | Towards a corpus of historical german plays with emotion annotations[END_REF], and additionally works whose annotation scheme features experiencers [START_REF] Cheng | An emotion cause corpus for chinese microblogs with multiple-user structures[END_REF][START_REF] Kim | Who feels what and why? annotation of a literature corpus with semantic roles of emotions[END_REF][START_REF] Schmidt | Towards a corpus of historical german plays with emotion annotations[END_REF], also identify the cause of emotions. In this regard, works that combine the most notions to describe emotions as they are expressed in texts are [START_REF] Kim | Who feels what and why? annotation of a literature corpus with semantic roles of emotions[END_REF][START_REF] Schmidt | Towards a corpus of historical german plays with emotion annotations[END_REF], that identify emotional categories, experiencers and causes of emotional units, with a linguistic marker-level annotation.

Inter-annotators agreement of emotion annotation tasks. As ( Öhman, 2020) pointed out, a strong testimony to emotion annotation's difficulty is the overall low inter-annotators agreement measures obtained in different works: with a binary or ternary labeling task, agreement never exceeds 70-80% and it drops as more categories are added. These measures are usually limited to emotional categories though some works like [START_REF] Kim | Who feels what and why? annotation of a literature corpus with semantic roles of emotions[END_REF] present agreement measures for each part of their annotation scheme. For instance, Cohen's Kappa between pairs of annotators range from 0.06 to 0.40 according to the emotional category considered, and from 0.09 and 0.63 according to the type of experiencers. The variability of agreement between pairs of annotators is also reflected in the measures of F1. Strict F1 and fuzzy F1 values respectively range from 6% to 40% and from 7% to 55% for emotional categories annotations, from 10% to 63% and from 18% to 68% for types of experiencers identification, and from 14% to 65% and 28% to 73% for cause labeling.

As a conclusion, until now, no NLP work has endeavored to consider together all notions previously mentioned to annotate emotions in texts. The scheme presented here was build to fill this need by taking into account as many concepts essential to emotion expression description as possible, as well as the relations between them. Since the first priority was to constitute an exhaustive scheme, the focus was put on testing the applicability of the scheme, rather than on straightaway calculating inter-annotators agreement. The scheme was thus applied by trained annotators on a corpus made of several genres of texts.

Annotation scheme

The annotation scheme is devoted to taking into account analysis perspectives pointed out in psycholinguistics as much as in linguistics on how emotions are expressed in written texts. Thus it tackles emotion expression by considering four types of linguistic units.

Each unit corresponds to one or several notions used to characterize emotion expression (see Section 2) and is matched by a unit of the scheme:

• SitEmo units denote an emotion, specified by its expression mode, type, and emotional category.

• Experiencer units refer to the entity which feels the emotions.

• SitCause units describe what causes the emotion.

• SitConsequence units denote what results from the emotion.

Each annotation unit is specified by a set of features. In this section, the main elements of the scheme are described, as illustrated in Figure 1. The focus will be put on the left part of the scheme, which was used to manually annotate 1,594 texts (see Section 5). As for the right part, it pertains to a more comprehensive level of emotion expression analysis: a textual structuring analysis process. The main components on which relies this process will only be briefly mentioned here, since this second part of the scheme has been so far applied on 28 texts.

SitEmo and Experiencer units

SitEmo units describe emotional situations, that is to say emotions expressed by linguistic markers and thus spatially and temporally anchored. These units are characterized by six main features, as shown in Table 1.

Feature Mode of a SitEmo. The feature Mode indicates how the emotion is expressed by the annotated markers.

Labeled emotions are directly designated trough an emotional label, i.e. an emotional lexicon word, such as "happiness", "afraid" or "guiltily" [START_REF] Creissen | Quelle représentation des différentes facettes de la dimension émotionnelle d'une histoire entre l'âge de 6 et 10ans ? apports d'une étude multimédia[END_REF][START_REF] Micheli | Les émotions dans les discours. Modèle d'analyse[END_REF]. Displayed emotions are conveyed by various linguistic characteristics of utterances, that occur when the speaker feels an emotion at the time of utterance. These characteristics show that the speaker was experiencing an emotion. The reader/interlocutor relies on them to infer the emotional state of the speaker. Markers that display an emotion take many forms, for instance words like interjections (e.g. "oh"), syntactic structures like nominal sentences (e.g. "So many presents !"), or typographic marks (e.g. "!") [START_REF] Micheli | Les émotions dans les discours. Modèle d'analyse[END_REF]. Suggested emotions are expressed through the description of situations associated with emotions by social conventions. Thanks to these conventions, the reader/interlocutor infers the emotion from the de- [START_REF] Creissen | Quelle représentation des différentes facettes de la dimension émotionnelle d'une histoire entre l'âge de 6 et 10ans ? apports d'une étude multimédia[END_REF][START_REF] Micheli | Les émotions dans les discours. Modèle d'analyse[END_REF]. For instance, in our western European culture, receiving a present is usually associated with a positive emotion, like JOY. Describing of such a situation can thus convey JOY. Finally, behavioral emotions are indicated by descriptions of emotional behaviors, for example "crying" or "smiling" [START_REF] Creissen | Quelle représentation des différentes facettes de la dimension émotionnelle d'une histoire entre l'âge de 6 et 10ans ? apports d'une étude multimédia[END_REF]. The reader relies on the depicted behavior to infer the emotion felt by the character.

Features Category, Category2, Type and Type2 of a SitEmo. The features Category and Type respectively deal with the emotional category and the type of the emotion expressed by the annotated markers (cf. Section 2). The scheme considers eleven categories, dispatched between basic and complex emotions, as shown in Table 2. The six basic emotions are those introduced in [START_REF] Ekman | An argument for basic emotions[END_REF]. Four of the complex categories were taken from [START_REF] Blanc | La production d'inférences émotionnelles entre 8 et 10 ans : quelle méthodologie pour quels résultats ?[END_REF] and [START_REF] Davidson | The role of basic, self-conscious and self-conscious evaluative emotions in children's memory and understanding of emotion[END_REF]. ADMIRATION was added as a fifth complex category to better balance emotion types in the scheme. Each of the eleven categories corresponds to more specific emotions. For instance, ANGER regroups anger but also annoyance, rage, and fury. Since the scheme's eleven categories are not sufficient to account for the diversity of emotions, a unit called Other was defined. It is to be used to annotate markers expressing other emotions, such as DISDAIN, LOVE or HATE. When a SitEmo conveys several emotions at the same time, the features Category2 and Type2 tell which second emotion is expressed. When only one emotion is denoted, the default value of these features is "None".

Feature Trigger of a SitEmo. The feature Trigger of a SitEmo specifies which element(s) within the annotated markers is(are) the most salient to express the emotion, namely the core of the emotional expression. For instance, the Trigger of a labeled emotion like "happy" is "happy". For a suggested emotion like "received a lot of presents", the Trigger would be "presents". This feature allows for a finer linguistic analysis of emotion expression. It may also be useful to develop resources for emotion detection, for example lexicons.

Experiencer unit. Experiencer units are specified by two main features. The feature Entity records the entity, meaning the person, animal, etc. that feels the emotion. The feature Control indicates weather the entity is a Human, Animated -Non Human, or Inanimate entity. The scheme is meant to annotate emotions felt by characters of the text, as opposed to readers' emotions [START_REF] Dijkstra | Character and reader emotions in literary texts[END_REF]. The notion of characters' emotions includes here writer's and Doxa's emotions 1 .

Affects relation. SitEmo and their associated Experiencer units are linked by a relation called Affects. Through Experiencer units and Affects relations, the aim is to gain a finer understanding of linguistic expression of emotions in texts.

(1) ::: Inès se souvient des murs qui n'arrêtaient pas de faire "crac".

C ¸a ::

lui faisait très peur. "La maison allait tomber sur nous . . . " murmure-t-::: elle en se blottissant sur le canapé. Example 1 shows different SitEmo conveying the FEAR 1 "Doxa" refers here to an abstract entity, that represents common belief, popular opinion.

2 Example taken from "L'immeuble d'Inès et d'Adam menac ¸ait de s'effondrer", P'tit Libé, n°91, 1-7 Februar 2019 of the same character ("Inès") through several expression modes.

SitCause and SitConsequence units

SitCause and SitConsequence units identify respectively markers that express a situation causing an emotion and those denoting a situation consequential to an emotion. So far, these units have been used to annotate cause and consequence expression at sentence-level.

To link together all SitEmo units that play a part in conveying the same emotional category, felt by the same experiencer, with the SitCause and SitConsequence that express the causes and consequences of this emotion, an additional type of structures is introduced in the scheme, namely EmoPassage structures. For instance, in example 2, the EMBARRASSMENT felt by the devil is expressed through six linguistic units. Two SitEmos convey the emotional category EMBAR-RASSMENT (one suggests it while the other one labels it). Each SitEmo is associated with an Experiencer unit referring to the devil. Moreover the first sentence describes the cause of the emotion (the farmer sold beautiful turnips to the market, while the devil only had wilted leaves) and is as such captured by a SitCause unit. Finally, the second sentence, marked with a Sit-Consequence unit, depicts the consequence of the emotion (the devil left, never to be seen again). These six units are all linked by an EmoPassage structure, since they all participate to the expression of the devil's EM-BARRASSMENT.

(2) Au marché, le paysan les vendit facilement, tandis que : le :::::: diable ne récolta que moqueries pour ses feuilles fanées.

::

Il en fut tellement vexé qu'on ne le revit jamais plus au pays. 3 (At the market, the farmer easily sold them[turnips], whereas the devil only gained mocking for his wilted leaves. He was so offended by this that he was never seen again in the area.)

Delimitation of emotional units

The scheme is intended to be used by expert linguist annotators. All choices that lead the scheme to its current form as well as many examples are detailed in the annotation guide associated to the scheme [START_REF] Étienne | Annotation manuelle des émotions dans des textes écrits avec la plateforme Glozz[END_REF]. In this section, some decisions taken to solve emotional units delimitation difficulties are mentioned. These difficulties arose due to the granularity of our annotation, that takes place at a linguistic marker level (tokens, phrases, clauses, etc.). Firstly, units must respect phrase frontiers, so that the delimitation can be compatible with syntactic parsers and chunkers, with a view of building an automatic emotion detection system from the annotated corpus. Secondly, when segments combine several ways of expressing emotions, the linguistic markers that pertain to different expression modes must be separately identified. This can lead to overlapping SitEmos but ensures that no linguistic information is lost. For instance, locutions such as "sauter de joie" ("jumping with joy") are annotated with two SitEmos: "sauter de joie" seen as a behavioral emotion; "joie" seen as a labeled emotion. The annotation scheme presented above takes into account all concepts that play a part in characterizing the notion of emotion as it is expressed in texts (Section 2). It also offers an organizational system to link all these notions together and then see how they interact. The next section presents the outcome of annotating text collections with the proposed scheme.

Application of the annotation scheme

The aim of this section is to show that (1) the scheme is applicable on real data, and (2) it can be a useful mean to to discover new knowledge about the expression of emotions. To do so, the scheme was applied on a corpus of more than 1,500 texts written in French. Six expert annotators (two of the authors expert in linguistics and NLP, and four master students in NLP) took part in the annotation process. Each one manually annotated a set of texts via the Glozz annotation platform [START_REF] Widlöcher | The glozz platform: A corpus annotation and mining tool[END_REF]. Several thousands of annotated units were thus produced. The annotated corpus is described in Section 5.1 and some quantitative annotation results are presented in Section 5.2.

Corpus description

The overall corpus on which the scheme was applied comprises 1,594 texts, dispatched into 3 genres: news- Table 3: Overall characteristics of the annotated corpus paper, encyclopedia, and fiction, see Table 3. Most texts are dedicated to children from 6 to 14 years old. This dataset is a subset of the corpus built for the ANR TextToKids project4 , as part of which the current work was carried out. The annotation procedure was first focused on journalistic texts provided by two French newspapers for children, Albert (ca. 239K tokens) and P'tit Libé (ca. 218K tokens), both partners in the project. The study on journalistic texts was then extended to a small corpus of articles on Covid-19, manually taken from five different newspapers (3 for children, 2 for adults) freely available online. Afterwards, the applicability of the scheme was tested on two other genres: fictional texts, that all come from novels for children, written by 12 different authors (6 full novels, ca. 25K tokens, and 11 extracts); and encyclopedic texts, taken from the French version of the online collaborative encyclopedia for children Vikidia5 .

Annotation results observation

The annotation process lead to a total of 6,790 SitEmos (4,886 in journalistic texts, 1,783 in fictional and 121 in encyclopedic), 4,970 Experiencers, 841 SitCauses and 147 SitConsequences. After a brief overview on Experiencer units annotations, this section presents the results pertaining to one part of the scheme: emotional category annotations, which are most commonly found across NLP works, and expression mode annotations, which are the main contribution of the scheme.

Experiencer units. First, amongst all Experiencer units delimited, 72.1% refer to a character, while only 19% point towards the writer and 10.4% towards the Doxa.

Emotional categories. Expression modes. Expression modes are rather balanced on the overall corpus with 33% suggested SitEmos, 24.8% of labeled, 21.3% of behavioral, and 20.9% of displayed. However this proportions vary according to the genre of texts (cf. Figure 3). Emotions appear to be more displayed (36.5%) in fictional texts than in other genres (30.6% for encyclopedic and 15% for journalistic). As for labeled and behavioral emotions, they are more frequent in journalistic texts (respectively 28.6% and 23.7%, against 14.7% and 14.6% in fictional texts, and 18.2% and 19.8% in encyclopedic). Suggested emotions do not appear to be specific to a genre of texts, since they always represent around a third of SitEmos. Expression mode also varies according to the emotion conveyed, as shown in Figure 4. Thus ANGER is mostly expressed through behavioral emotional units (48.4%), JOY and SADNESS through suggested units (respectively 39.4% and 47.5%) and SURPRISE through displayed units (60.8%). FEAR is almost as suggested (39.6%) as it is labeled (40.4%).

Figure 4: For the most frequent emotions, distribution of expression modes by emotion categories

Discussion

The annotation scheme presented in Section 4 allowed for the annotation of written texts using concepts stemming from analysis perspectives proffered in psycholinguistics as well as in linguistics on how emotions are expressed in texts (cf. Section 2). The annotations produced thus include information on emotional categories and emotion expression modes, as well as experiencers, causes and consequences of emotions. These annotations led to a surprising observation on displayed emotions in encyclopedic texts, that will be discussed here. Moreover, the analysis of the annotation results presented in Section 5.2 highlights interactions between, on the one hand, expression modes and genre of texts, and, on the other hand, two notions of the scheme: expression mode and emotional category.

These interactions provide a feedback on some NLP works already mentioned in Section 3.

Displayed emotions in encyclopedic texts. Annotation results show that 30.6% of SitEmos found in encyclopedic texts are displayed emotions, usually felt by the writer (70.3% of displayed emotions). This came as a surprise since displayed emotions are usually found in direct speeches or intervene to show the writer's emotions, which is mostly the case here. Yet encyclopedic texts are supposed to describe facts with a neutral tone, devoid of writer's emotions. We hypothesize that the phenomenon we encounter in our corpus is due to the nature of the source of the texts we annotated. Indeed all our encyclopedic texts come from an online collaborative encyclopedia (cf. Section 5.1). All participants are not necessarily expert on encyclopedic writing. They often try to make texts livelier so that they appeal more to children, by resorting to exclamation marks and clearly showing how they feel about the subject of the text. This takes the shape of linguistic markers that display their emotions. However it can be noted that displayed emotions are most frequent in fictional texts (36.5%). This was to be expected since these texts contain many dialogues, conductive to the displaying of emotions.

Expression modes and genre. The link between expression modes and genre of texts is one of the key interactions highlighted by the application of our annotation scheme to the corpus. This was already sensed by [START_REF] Bostan | An analysis of annotated corpora for emotion classification in text[END_REF], when they declared that "journalists ideally tend to be objective when writing articles,[...] and one might assume that emotion expressions in tales are more subtle and implicit" (p. 2105).

In the same way, [START_REF] Kim | Who feels what and why? annotation of a literature corpus with semantic roles of emotions[END_REF] partially attributed the difficulty of annotating emotions in triplets of sentences to the nature of the texts they were dealing with. For them, "fictional texts are highly metaphoric and full of allusions and metonymies, which requires thoughtful reading (often reading between the lines) and a broader context" (p. 1354).

Unlike what [START_REF] Bostan | An analysis of annotated corpora for emotion classification in text[END_REF][START_REF] Kim | Who feels what and why? annotation of a literature corpus with semantic roles of emotions[END_REF] proposed, suggested emotions do not appear to be specific of fictional texts in the annotated corpus presented here. On the contrary they represent a similar proportion of emotional units in all genres of texts (between 31.4% and 34.2% SitEmos). However, as previously mentioned, displayed emotions are most common in fictional texts. As for labeled and behavioral emotions, they appear to be mostly used in journalistic texts. The annotation results obtained on the corpus of 1,594 texts show that some expression modes tend to be preferably employed in specific genres of texts. To be fully confirmed, this trend would require the annotation of a corpus better balanced in genres.

Expression mode and emotional categories. The scheme we introduced also points out the relation between the concepts of expression mode and emotional category.

Alm (2010)'s study of emotional sentences' characteristics (see Section 3) suggested "that some characteristics may show particular affinity with certain affects" (p. 121). More precisely, the category FEAR seems to be mainly conveyed by affect words or behaviours descriptions, and the category SURPRISE by markers found in direct speeches or descriptions of unexpected observations.

For their part, [START_REF] Demszky | Goemotions: A dataset of fine-grained emotions[END_REF] found that "certain emotions are more verbally implicit and may require more context to be interpreted" (p. 6). For example, in their Reddit comments corpus, GRATITUDE was highly correlated with the token "thanks" and AMUSEMENT with the token "lol", whereas GRIEF and NERVOUS-NESS were not especially significantly associated with any token.

This corroborates the annotation results presented here as, on the overall corpus, SURPRISE is mainly expressed by displayed units and FEAR by labeled and suggested units. The results also show an association between ANGER and behavioral expressions, and between JOY and SADNESS and suggested expressions (see Section 5.2).

Conclusion

The annotation scheme introduced here as well as its corresponding guidelines greatly contribute to expanding resources for emotions annotation of French texts, which are scarce as for now. Rather than labeling emotions at a text or sentence level, which is currently customary in building of emotions annotated corpora, and especially in French language works (e.g. [START_REF] Paroubek | Annotations for opinion mining evaluation in the industrial context of the DOXA project[END_REF][START_REF] Fraisse | Utiliser les interjections pour détecter les émotions[END_REF]), our scheme aims at identifying much finer textual segments (words, phrases, propositions, etc.) in order to label linguistic units that convey an emotion. The annotation scheme is theoretically motivated as it takes into account many concepts stemming both from linguistic works (expression mode, experiencer) and psycho-linguistic studies (emotional category, emotion type, cause and consequence of emotion) to characterize the linguistic realization of the notion of emotion in texts. The approach we propose here constitutes a synthesis of several other propositions, since our scheme adopts concepts used heterogeneously in other former works. Furthermore, its application to texts of different genres is systematized and discussed in the present paper. The application of the scheme on more than 1,500 texts of several genres (journalistic, encyclopedic, fictional) allowed for the highlighting of strong correlations between, on the one hand expression mode and emotional category (e.g. SURPRISE is mostly displayed whereas ANGER is usually conveyed through behavioral expressions), and on the other hand expression mode and genre of texts (e.g. displayed emotions are more frequent in fictional texts and labeled emotions in journalistic texts). If this type of trends had already been spotted at the margins in various NLP works, we believe that the scheme we propose offers a rigorous way to explore these types of correlations, essential in understanding how emotions are expressed in texts. We are thus confident that this scheme holds great possibilities for further exploring of correlations between the items of the scheme and other variables, for instance in the field of digital humanities or corpus linguistics. An important perspective for the generalisation of this scheme is to develop an automatic annotation system through the learning of NLP models.

  2

(

  Ines remembers how the walls kept on cracking. It scared : her a lot. "The house was about to fall on us ..." :: --<des murs qui n'arrêtaient pas de faire "crac">; <lui> <---<très peur>; <La maison allait tomber sur nous> ---> <elle>;<...> ---> <elle>; <murmure> ---> <elle>;<elle> ---> <en se blottissant sur le canapé>

  <le diable> <---<moqueries>; <Il> <---<tellement vexé qu'on ne le revit jamais plus 3 Example taken from Le Diable et le paysan, F. Rabelais au pays> EmoPassage: <moqueries>; <tellement vexé qu'on ne le revit jamais plus au pays>; <Au marché, le paysan les vendit facilement, tandis que le diable ne récolta que moqueries pour ses feuilles fanées.>; <Il en fut tellement vexé qu'on ne le revit jamais plus au pays.>

Figure 2 :Figure 3 :

 23 Figure 2: Distribution of the most frequent emotional categories by genre of text

Table 1 :

 1 SitEmo units' features

	Figure 1: Main concepts of the annotation scheme
	Feature	Value
	Mode	labeled, displayed, sug-
		gested, behavioural
	Type & Type2	Basic, Complex, None
	Category & Category2 ANGER,	SURPRISE,
		GUILT, etc., None
	Trigger	Core of emotional ex-
		pression

Table 2 :

 2 Links between emotion types and emotional categories picted situation

	Type	Categories
	Basic	ANGER, DISGUST, FEAR, JOY, SUR-
		PRISE, SADNESS
	Complex ADMIRATION,	EMBARRASSMENT,
		GUILT, JEALOUSY, PRIDE

https://texttokids.irisa.fr/

https://fr.vikidia.org/wiki/Vikidia: Accueil

Acknowledgement

This work has been partially funded by the French National Research Agency (ANR) as part of the Text-ToKids project (ANR-19-CE38-0014).