Malek Messai

Abdeldjalil Aïssa-El-Bey

Karine Amis

Frédéric Guilloud

Iteratively Reweighted two-Stage LASSO for Block-Sparse Signal Recovery Under Finite-Alphabet Constraints

Keywords: Block-sparsity recovery, iterative reweighting, 1 -minimization, iterative recovery algorithms, LASSO, finite-alphabet

In this paper, we derive an efficient iterative algorithm for the recovery of block-sparse signals given the finite data alphabet and the non-zero block probability. The non-zero block number is supposed to be far smaller than the total block number (block-sparse).

The key principle is the separation of the unknown signal vector into an unknown support vector s and an unknown data symbol vector a. Both number ( s 0 ) and positions (s i ∈ {0, 1}) of non-zero blocks are unknown. The proposed algorithms use an iterative two-stage LASSO procedure consisting in optimizing the recovery problem alternatively with respect to a and with respect to s. The first algorithm resorts on 1 -norm of the support vector and the second one applies reweighted 1 -norm, which further improves the recovery performance. Performance of proposed algorithms is illustrated in the context of sporadic multiuser communications. Simulations show that the reweighted-1 algorithm performs close to its lower bound (perfect knowledge of the support vector).

Introduction

Compressed Sensing (CS) is a signal processing approach that enables exact recovery of sparse signals from highly incomplete sets of linear measurements [START_REF] Candes | Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information[END_REF][START_REF] Tropp | Algorithms for simultaneous sparse approximation[END_REF][START_REF] Tropp | Algorithms for simultaneous sparse approximation[END_REF]. In last decade, CS techniques have spread in various disciplines such as medical imaging, machine learning, computer science and many others. In many applications of CS such as measurement of gene expression levels [START_REF] Parvaresh | Recovering sparse signals using sparse measurement matrices in compressed DNA microarrays[END_REF], face recognition [START_REF] Wright | Robust face recognition via sparse representation[END_REF] or data clustering [START_REF] Elhamifar | Sparse subspace clustering: Algorithm, theory, and applications[END_REF], the signal is block-sparse, i.e., its non-zero elements are grouped into blocks [START_REF] Eldar | Block-sparse signals: Uncertainty relations and efficient recovery[END_REF]. Many algo-rithms have been proposed to recover sparse signals having a block structure. They can be classified into two popular kinds of algorithms. The first ones, referred to as greedy CS algorithms, lead to the lowest-complexity detectors from CS theory. A popular example of greedy CS algorithm is Group-OMP [START_REF] Swirszcz | Grouped orthogonal matching pursuit for variable selection and prediction[END_REF][START_REF] Fu | Block-sparse recovery via redundant block OMP[END_REF] where non-zero blocks of symbols are iteratively detected. The second kind, referred to as the 1 -minimization family, comprises for example algorithms based on 2 / 1 -norm minimization [START_REF] Stojnic | On the reconstruction of block-sparse signals with an optimal number of measurements[END_REF]. This family provides a better sparse recovery with a lower number of measurements than the greedy one, and so lower error rates can be achieved at the expense of an increased complexity.

In many cases, signals belong to a finite alphabet and this property can be exploited to enhance the recovery efficiency. For instance, CS under finite-alphabet constraints was proposed in [START_REF] Tian | Detection of sparse signals under finite-alphabet constraints[END_REF] in the case of a binary alphabet and in [START_REF] Bey | Sparsity-based Recovery of Finite Alphabet Solutions to Underdetermined Linear Systems[END_REF] for any finite alphabet.

In this paper, we consider the reconstruction of arbitrary finite-alphabet signals having a block-sparse structure. As already mentioned, [START_REF] Eldar | Block-sparse signals: Uncertainty relations and efficient recovery[END_REF] and [START_REF] Stojnic | On the reconstruction of block-sparse signals with an optimal number of measurements[END_REF] deal with block-sparse signal recovery without taking into account the finite-alphabet constraint. The main drawback is the sensitivity of these algorithms to a 2 -norm bounding constraint parameter that depends on the noise level, which may not be available. In this paper, we derive an iterative algorithm from 1 -minimization family that takes into account the finite alphabet constraint and whose single parameter depends only on sparsity parameters. The key principle is the separation of the unknown signal vector into an unknown sparse support vector s and an unknown data symbol vector a. Both number ( s 0 ) and positions (s i ∈ {0, 1}) of non-zero blocks are unknown. Proposed algorithms use an iterative twostage LASSO procedure consisting in optimizing the recovery problem alternatively with respect to a and with respect to s. The first algorithm resorts on 1 -norm of the support vector and the second one applies reweighted 1 -norm [START_REF] Candès | Enhancing sparsity by reweighted 1 minimization[END_REF], which further improves the recovery performance.

The paper is organized as follows: in Section 2, we first introduce the signal model before describing the proposed iterative reweighted algorithm in Section 3. In Section 4, proposed algorithms are applied in the context of sporadic communications. Finally, conclusions and perspectives are given in Section 5.

Throughout this paper, notations x, x and X denote respectively a scalar value, a column vector and a matrix. X T is the transpose of X and X H its Hermitian matrix. 0 N (resp. 1 N ) is the null (resp. all-one) vector of length N . x stands for the estimated value of x in a continuous domain included in R or C. Q A (x) stands for the quantizer operator which maps x to its closest value in a discrete alphabet denoted by A. x denotes an estimate of x in A.

{.} and {.} denote, respectively, the real and the imaginary part of the object in brackets. ⊗ and denote the Kronecker and the Khatri-Rao products, respectively. Finally, α ≤ x ≤ β means that each component x i of x is bounded by α and β. When A is a set and n an integer, then A n stands for the n-fold Cartesian product of A. In an iterative algorithm seeking for the optimal value of x, x (i) denotes the value of x at the i-th iteration.

System Model

The basic CS model describes the linear connection between a sparse vector x and its measurement y by

y = Hx + η, (1)
where η denotes a zero mean complex Gaussian noise vector with covariance matrix σ 2 η I. H ∈ C m×n is the underdetermined mixing matrix with fewer rows than columns (m < n). x is a block-sparse vector x = [x T 1 , x T 2 , . . . , x T K ] T composed of K length-N blocks x i i.e., n = KN . The number of non-zero blocks in x is a random variable, denoted by K a . When the k-th block is different from 0 N , it takes its value in finite alphabet A N . Thus, x k belong to zero-augmented alphabet A N ∪ {0 N }. We denote by p a the probability that a block is different from 0 N . The original mathematical model which defines the sparsity condition in the noisy case is given by x = arg min (

The optimization problem ( 2) is an NP-hard combinatorial problem [START_REF] Natarajan | Sparse approximate solutions to linear systems[END_REF]. A computationally tractable alternative is to relax the problem by performing 1 -norm minimization in place of 0 -norm minimization; the former can be solved in polynomial time using linear programming techniques [START_REF] Donoho | Compressed sensing[END_REF][START_REF] Candès | Stable signal recovery from incomplete and inaccurate measurements[END_REF]. Then the problem is given by x = arg min

However solving (2) or (3) leads to a sparse vector that is not necessarily block-sparse, even when the true vector x is block-sparse [START_REF] Eldar | Block-sparse signals: Uncertainty relations and efficient recovery[END_REF]. In [START_REF] Eldar | Block-sparse signals: Uncertainty relations and efficient recovery[END_REF], the block-sparsity is taken into account by replacing the 0 -norm used in (2): the non-zero blocks can be counted by combining an indicator function with 2 -norm to detect energy which gives the following problem formulation:

x = arg min x K k=1 I (0,+∞) (||x k || 2 ) s.t. ||y -Hx|| 2 ≤ , (4)
where I A (.) is the indicator function of subset A. Greedy algorithms such as GOMP are used to solve [START_REF] Parvaresh | Recovering sparse signals using sparse measurement matrices in compressed DNA microarrays[END_REF]. Another solution to reconstruct block-sparse signals is referred to as " 2 / 1 -norm minimization" [START_REF] Stojnic | On the reconstruction of block-sparse signals with an optimal number of measurements[END_REF]. Instead of using 1 -norm as in (3), it combines

1 -norm to measure the block sparsity and 2 -norm to detect energy. By defining the energy vector

ξ x = [ξ x1 , . . . , ξ x K ] where ξ xi = ||x i || 2 for i = 1, . . . , K, the 2 / 1 -norm
block-sparse signal recovery criterion is given by x = arg min

x ||ξ x || 1 s. t. ||y -Hx|| 2 ≤ , which is equivalent to x = arg min x K k=1 ||x k || 2 s. t. ||y -Hx|| 2 ≤ . (5)
In all the approaches mentioned above, the upper bound depends on the noise deviation and on the number of observations. The optimization of is critical and requires a good knowledge of the noise standard deviation σ η which may be unavailable. The resulting lack of accuracy is mitigated in Section 3 by reformulating the problem in such a way that its resolution doesn't need the noise deviation estimation. On the contrary, it exploits known information such as the source data alphabet and the probability that a block is null.

Proposed Algorithms

Greedy algorithms such as GOMP are the least complex detectors from CS theory, but they suffer from error propagation. A false non-zero detection at a given iteration can cause the data and the non-zero detection of all subsequently blocks to be incorrect.

The error propagation increases with the number of non-zero blocks K a and thus with the probability p a .

In order to bypass the GOMP defaults, we aim to define a criterion and an algorithm which exploits the finite alphabet constraint and the block-sparsity structure of the signal. Therefore, the main objective is the definition of the constraints from x properties.

x is block-sparse and each block x k equals 0 N or belongs to A N . The probability that

x k differs from 0 N is p a .
By exploiting the independence between support (number and positions of non-zero blocks) and data values, we introduce the length-K binary support vector s and the block-data vector a. s k = 1 if the k-th block is non-zero and s k = 0 otherwise.

a = [a T 1 , a T 2 , . . . , a T K ] T with a k ∈ A N for k = 1, . . . , K.
We can then write

x k = s k a k that is to say x = D s a where D s is a KN × KN matrix given by D s = diag(s) ⊗ I N .
Given the constraints, the detection problem can be written as arg min

s,a ||y -HD s a|| 2 s.t. s ∈ {0, 1} K a ∈ A KN (6)
Problem ( 6) is NP-hard and requires an exhaustive search. In order to get a polynomial complexity, we relax the constraints and we use an iterative two-stage LASSO algorithm which alternates between support detection given a and signal estimation given s. The constraint involving s is relaxed by using 1 -norm as well as lower and upper bounds, whereas real and imaginary parts of a are bounded by extreme values of the finitealphabet. Then problem ( 6) is relaxed by: arg min

s,a ||y -HD s a|| 2 s.t. s 1 ≤ δ 0 ≤ s ≤ 1 min {A} ≤ (a) ≤ max {A} min {A} ≤ (a) ≤ max {A} (7)
δ is introduced so as to force the sparsity of s. Its value can be fixed such that

Pr(K a ≤ δ) = Pr K k=1 s k ≤ δ = P c , (8)
where P c is defined as the target confidence probability. As K a = K k=1 s k follows the binomial distribution with parameters K and p a , the probability of having a non-zero Algorithm 1 Iterative two-stage LASSO Initialize:

ŝ(0) = 1K for i =1 . . . N iter do Stage (1)                â(i) = arg min a ||y -HD ŝ(i-1) a||2 s. t.        min {A} ≤ (a) ≤ max {A} min {A} ≤ (a) ≤ max {A} Stage (2)                ŝ(i) = arg min s ||y -HM â(i) s||2 s. t.        s 1 ≤ δ 0 ≤ s ≤ 1 end for
block number lower than δ in K trials is given by

Pr(Ka ≤ δ) = δ k=0 K k p k a (1 -pa) K-k = I1-p a (K -δ , 1 + δ ) (9)
with . the greatest integer function and I x (•, •) the "regularized incomplete beta function". We obtain δ by solving (9) given a target confidence probability P c Pr(K a ≤ δ).

δ can also be interpreted as the maximum number of non-zero blocks that the algorithm will be able to detect. Its choice is as critical as the choice of in [START_REF] Wright | Robust face recognition via sparse representation[END_REF]. But δ only depends on the probability p a , while depends on the noise variance σ 2 η . To solve [START_REF] Eldar | Block-sparse signals: Uncertainty relations and efficient recovery[END_REF], we assume as an initial condition that all blocks are non-zero:

s (0) = 1 K .
The proposed iterative process is referred to as "iterative two-stage LASSO" and is detailed in Algorithm 1 where M a = I K a.

The algorithm output ŝ belongs to [0, 1]. To reconstruct x, we need a binary support vector and we have two options to estimate it from ŝ. The first one, without any prior, applies on ŝ a threshold fixed to 0.5. The second one exploits the knowledge of the number of non-zero blocks by selecting the K a highest values of ŝ.

Note that the alphabet constraint consists in a strong relaxation by the introduction of a box constraint on the alphabet. In practice, the recovery performance of (7) will improve as long as the cardinality of real and imaginary parts of A is small. An interesting solution referred to as reweighted 1 -minimization was proposed in [START_REF] Candès | Enhancing sparsity by reweighted 1 minimization[END_REF] as an intermediate between 0 and 1 minimizations: it performs closer to 0 min-imization while remaining convex as 1 -norm based criteria. Therefore, based on [START_REF] Candès | Enhancing sparsity by reweighted 1 minimization[END_REF],

we propose in this paper the "Iterative Reweighted two-stage LASSO algorithm" where reweighted 1 -norm minimization is applied in second stage for support detection. It is described in Algorithm 2 (i-th iteration). W is a diagonal matrix with diagonal entries w 1 , w 2 , . . . , w K . Initial weights are set to w

               â(i) = arg min a ||y -HD ŝ(i-1) a||2 s. t.        min {A} ≤ (a) ≤ max {A} min {A} ≤ (a) ≤ max {A} Stage (2)                ŝ(i) = arg min s ||y -HM â(i) s||2 s. t        ||W (i) s||1 = K k=1 w (i) k |s k | ≤ δ 0 ≤ s ≤ 1 Update w (i+1) k = 1 |s (i) k |+ρ
6: end for Non-zero weights w (i+1) k are updated as in [START_REF] Candès | Enhancing sparsity by reweighted 1 minimization[END_REF]. The parameter ρ ensures that a decision taken at one step can be questioned at next step i.e. a zero-valued component in s (i) could turn into a non-zero estimate at next step. We consider the same optimization of ρ as in [START_REF] Candès | Enhancing sparsity by reweighted 1 minimization[END_REF]:

ρ = max{|s (i-1) | (Ka+1) , 10 -3 }, (10)
where |s (i-1) | (Ka+1) represents the (K a +1)-th element of the vector that orders (|s

(i-1) |)
in decreasing order of magnitude. It requires the estimation of K a , which can be obtained thanks to an energy detector. The purpose of ρ is to avoid the divergence of the norm due to high weight values caused by extremely low components of the support vector.

Let us note that Algorithm 2 only represents a variation of the one proposed in [START_REF] Candès | Enhancing sparsity by reweighted 1 minimization[END_REF].

The cost function of ( 7) is continuously differentiable and constraint sets of Algorithms 1 and 2 are convex. The cost function remains convex with respect to one variable when the other is held constant. Being alternations of two convex optimizations with independent constraints, the convergence of Algorithms 1 and 2 is ensured by Propositions 2.5 and 3.9 in [START_REF] Bertsekas | Parallel and Distributed Computation: Numerical Methods[END_REF].

To give rough estimation of the complexity, we only kept the most complex computational operations and we neglected the others. For the GOMP algorithm, the overall complexity is dominated by the LS estimation step, maximum at last iteration and given by O((K a N ) 3 ). Hence, the complexity for the K a iterations is upper-bounded by

K a O((K a N ) 3
). For the 2 / 1 algorithm, the overall complexity is given by O((KN ) 3 ).

Finally, for the proposed LASSO-based algorithm, the complexity of first stage is dominated by O((KN ) 3 ), whereas for second stage, it is dominated by O(K 3 ). Therefore, its overall complexity is dominated by N iter O((KN ) 3 ). In next section, Algorithms 1 and 2 are applied in the context of sporadic communications.

Application to Sporadic Multiuser Communications

Multi-user detection (MUD) based on CS (CS-MUD) was recently proposed in [START_REF] Bockelmann | Compressive sensing based multiuser detection for machine-to-machine communication[END_REF][START_REF] Schepker | Efficient detectors for joint compressed sensing detection and channel decoding[END_REF] in the context of sporadic machine type communications (MTC) to jointly detect user activity and user data. In sporadic MTC, K users can either send a small frame to an aggregation point (in that case, the user is said active) or remain idle. If more than one user is active, collisions occur at the aggregation point. We consider the linear model given in (1) to describe multi-user sporadic communications.

Recovery of sparse sequences requires a minimum number of observations [20] to estimate sparse vectors with overwhelming probability. A solution to increase the number of observations is to spread each symbol period using spreading sequences [START_REF] Zhu | Exploiting sparse user activity in multiuser detection[END_REF]. In that case, the spreading goal is obviously not orthogonal multiple access. In the following, we consider spreading sequences of length N s smaller than the number of users which yields an underdetermined multiuser detection problem. Therefore, the system matrix H includes all the frequency-selective Rayleigh fading channels and the spreading sequences of each user. Hence, matrix H ∈ C m×n with n = KN and m = N s N . The aggregation point has no knowledge whether the k-th user is active or not during a given time-slot.

Therefore, user activity has to be jointly detected with transmitted data and algorithms proposed in Section 3 apply to this context. As in [START_REF] Bockelmann | Compressive sensing based multiuser detection for machine-to-machine communication[END_REF][START_REF] Schepker | Efficient detectors for joint compressed sensing detection and channel decoding[END_REF], we consider that the number of active users K a is perfectly estimated by an energy detector. Detection (either Algorithm 1 or Algorithm 2) is first carried out with δ fixed so that (8) is satisfied. The perfect knowledge of K a is exploited first to fix ρ in [START_REF] Stojnic | On the reconstruction of block-sparse signals with an optimal number of measurements[END_REF] and secondly at the iterative process output by setting the K a maximum values of ŝ to 1 and the remaining ones to 0. Then, we decide x = Q A (D ŝ â).

In the simulations, we assume a low activation probability (p a < 0.5). The activity of each user is randomly sorted according to p a , such that E(K a ) = Kp a . The multipath fading channel is modeled by L = 6 independent Rayleigh-distributed taps. The total number of users is set to K = 80. Each pseudo-random sequence length is N s = 25. An active user transmits N = 30 information symbols per frame which belong to alphabet

A such that the variance of the symbols is equal to 1. The performance are given as a function of the signal-to-noise ratio (SNR) which equals

SNR = p a K σ 2 η .
Proposed algorithms are compared to the GOMP and to the 2 / 1 minimization algorithm. Since no analytical solution is available [START_REF] Stark | Bounded-variable least-squares: an algorithm and applications[END_REF] for problem [START_REF] Eldar | Block-sparse signals: Uncertainty relations and efficient recovery[END_REF], the convex optimization toolbox (CVX) [START_REF] Grant | CVX: Matlab software for disciplined convex programming[END_REF] is used to execute the algorithms. "It.

Symbol Error Rate

The symbol error rate (SER) is illustrated in Fig. 1 for BPSK modulation with p a = 15% (Fig. 1a) and for 16-QAM modulation with p a = 10% (Fig. 1b). Proposed algorithms are carried out with P c = 0.99 and 40 iterations.

As benchmark, we propose two oracle algorithms which have perfect knowledge of the support vector s.

For both of them, the observation vector equals y = H s x s +η, where x s combines all non-zero blocks x k of x for which s k = 1, and H s is the concatenation of the sub-matrices of H which have the same indices as the non-zero blocks. The first oracle referred to as oracle-LS is only based on least-squares estimation of data: xs = H H s H s -1 H H s y. The second oracle referred to as oracle-BLS (bounded least squares) exploits the finite alphabet constraint thanks to bounded-variable convex constraints and estimates x s as follows:

                       xs = arg min xs ||y -H s x s || 2 s. t min {A} ≤ (x s ) ≤ max {A} min {A} ≤ (x s ) ≤ max {A} (11)
The oracle-LS bound is above the oracle-BLS bound since the latter considers the alphabet constraint. Improvement offered by proposed algorithms is significant. An important gain appears for SNR higher than 12 dB and 27 dB for BPSK and 16-QAM respectively. Iterative two-stage LASSO performs close to oracle-LS, while Reweighted Iterative two-stage LASSO outperforms oracle-LS and achieves a SER close to the oracle-BLS one.

Support Detection

The probability that an inactive user is detected as active and vice versa is expressed as

E[ s -s 2 2 ] = 1 K K i=1 |s i -si | 2
where s denotes the decided support given by s = Q {0,1} (ŝ). This activity-detection error rate is illustrated in Fig. 2 for BPSK modulation with p a = 15% (Fig. 2a) and for 16-QAM modulation with p a = 10% (Fig. 2b).

We observe an error floor for the GOMP algorithm and also a lower one for the 2 / 1 minimization algorithm. Proposed iterative two-stage LASSO algorithms outperform the other two with no error floor observed in the activity-detection error rate and with a better performance for the reweighted version.

Conclusion

In this paper, we proposed a new iterative 1 -minimization algorithm that exploits the finite alphabet knowledge and iterates between data estimation and activity detection.

Its parameter only depends on the activation probability. We introduced an improved version by reweighting 1 -norm of the support vector. Both algorithms are evaluated in the context of sporadic multiuser communications. They perform better than GOMP and 2 / 1 -minimization algorithms without error floor observed in the activity-detection error-rate. In the future, we will investigate the robustness towards estimation of the active user number and to channel estimation (with the help of additional pilot symbols).

Proposed algorithms would iterate between channel estimation and support detection.

x

  ||x|| 0 s. t. ||y -Hx|| 2 ≤ .

x ||x|| 1

 1 s. t. ||y -Hx|| 2 ≤ .

k = 1 ,Algorithm 2

 12 k = {1, 2, . . . , K}. Reweighted Iterative two-stage LASSO Initialize: ŝ(0) = 1K for i =1 . . . N iter do Stage (1)

  2-stage LASSO" (resp. "Reweight. It. 2-stage LASSO") refers to Iterative two-stage LASSO algorithm (resp. Reweighted Iterative two-stage LASSO algorithm).

Figure 1 :Figure 2 :

 12 Figure 1: SER comparisons for BPSK and 16-QAM modulations with K = 80 users and Ns = 25.


Anh-Tuan Tuan Phung

Olivier Chadebec
email: chadebec@leg.ensieg.inpg.fr

Gérard Meunier

Xavier Margueron

Jean-Pierre Keradec

Gérard

High frequency proximity losses determination for rectangular cross section conductors

teaching and research institutions in France or abroad, or from public or private research centers.

I. INTRODUCTION

In power electronics, conductors carrying currents are exposed to non-uniformed alternative magnetic fields associated to neighboring current carrying conductors. This leads to increase ohmic resistance in conductors known as proximity losses. For high frequency applications, modeling by FEM need a very fine mesh up to the skin depth. Moreover, the number of conductors is important so that 3D modeling is still impossible to carry out. A classical technique consist in using the concept of complex permeability (mu complex or mu * ) associated with homogenization approach. It has been successfully applied to transformers with circular cross section conductors [START_REF] Matagne | Macroscopic electric characterization of bundles of conductors[END_REF]. This paper proposes two original approaches to apply this method to rectangular conductors.

II. PROXIMITY LOSSES AND COMPLEX PERMEABILITY

To compute losses correctly without involving a fine mesh, complex permeability concept is employed. The principle consists in replacing the conductor material by an equivalent non conductive magnetic material, which gives the same active and reactive power loss for the given geometry with a coarse FE mesh. For circular cross section conductor [START_REF] Matagne | Macroscopic electric characterization of bundles of conductors[END_REF], analytical solution coupled with homogenization technique is available. For rectangular one [START_REF] Moreau | Proximity losses computation with a 2D complex permeability modeling[END_REF], 1D analytical approach responds weakly when we are far from ideal conditions. We propose two approaches to solve numerically and simply the complex permeability problem:

A. Mu * calculation for an isolated conductor and homogenization

The first step consist in determining the equivalent mu * for an isolated conductor. The rectangular conductor is placed in an alternative inductor field. The 2D FE eddy current problem is solved and active and passive losses are computed. Afterward, the equivalent complex permeability problem is coupled with a optimization process to obtain the same losses values. Optimal mu * is then obtained for one single conductor. It is then not necessary to mesh in the skin depth to model the device. This method is particularly efficient to model devices with scattered conductors.

If conductors are uniformly grouped like in the most part of transformer structure, a homogenization technique can be added to reduce the number of unknown even more. Based on an equivalent reluctance network, an equivalent mu * can be calculated and imposed on a homogenized region representing conductors and the surrounded air region [START_REF] Moreau | Proximity losses computation with a 2D complex permeability modeling[END_REF].

B. Direct mu * evaluation for a large number of conductors

If there are a large number of conductors stacking in a regular manner, we will consider an elementary cell of the geometry placed in an inductor field H 0 . Firstly, we compute this small cell taking into account the eddy current effects. The problem is imposed by appropriate boundary conditions which consider the regular stack of conductors. Losses (P and Q) are calculated with an economic computation cost thanks to reduced size of the cell. Subsequently, these losses will be equalized with that provided by the equivalent mu complex problem. The equivalent mu* is then directly and simply obtained.

III. NUMERICAL RESULTS

Both methods have been tested with a package of conductors. They propose two complement approaches to consider losses caused by proximity effects using numeric complex permeability. They lead to a strong reduction in memory requirement and computation time. Numerical results provided by complex permeability show a good agreement with classical eddy current computation. They are also improved essentially compared with 1D analytical approach [START_REF] Moreau | Proximity losses computation with a 2D complex permeability modeling[END_REF]. This result has been obtained thanks to 2D modeling. These techniques have been also applied successfully for 3D case.


Patrick Paroubek

Amel Fraisse

Annotation representation Model

Keywords:

Introduction 1.Aim

This document describes the design and execution of experiments for assessing HC eectiveness.

It constitutes the rst version of the deliverable planned for the end of the project, as such it

focuses on experiments design and present only results for what has been achieved so far (T5.1 and part of T5.2). The whole set of evaluation results, including the results of the evaluation campaign (T 5.3) planned to happen during the last year of uComp will be presented in the nal devliverable, due for M30.

Responsibility

The partner responsible the production of this document is LIMSI-CNRS.

2 Aect Lexica

Aim

The task for rgEfsed unowledge esoure ivlution focuses on the evaluation of the HCsourced aect lexicons both in terms of improving the recall, precision and F-measure of existing sentiment and opinion mining algorithms ( [START_REF] Gindl | Cross-domain contextualization of sentiment lexicons[END_REF] and [START_REF] Pak | rumn vnguge ehnology ghllenges for gomputer iene nd vinguistis, chapter Normalization of Term Weighting Scheme for Sentiment Analysis[END_REF]), as well as by using a set of additional games aimed at evaluating specic aspects of these resources.

We plan to assess the interest of replacing/complementing entropy-based data browsing algorithms by/with human guidance. This will help evaluate the eectiveness of the HC paradigm and framework, and address the following questions:

row do rowdsoured resoures ompre to those reted mnullyD utomtilly or through mehnised lour @in terms of oth quntity nd qulityD whih will e mesured y the kEvlue of interEnnottor greementAc ss the ostGene(t rtio of this proess etter thn tht of other knowledge quisition pprohesc

Analogous to WP4, T5.2 will primarily focus on the climate change domain and the repository built in T1.5.

State of the art

Since it was created in 2005 according to [Sare, 2009], the word rowdsouring found rapidly 1 its way into the research community, where people tried from the beginning to assess the benets and gains they could get from using the power of human computation. Most of the early 1 in 2009 it reached the 1 million hit limit in Google ( [Sare, 2009])

WP5

Semantics

In [START_REF] Rumshisky | The holy grail of sense denition: Creating a sense-disambiguated corpus from scratch[END_REF], the authors report on an experiment to build a fuzzy sense inventory database for a set of polysemous verbs of medium diculty, using non-expert annotators which were presented with sentences containing one of the verb form and had identify particular meaning. The results were then evaluated against the groupings created by a professional lexicographer (set-matching F-score of 0.93), each sentence being annotated by 5 AMT annotators. On average 1 minute was required to process 10 sentences and the amount paid to the annotator was 0.03 USD. The total sum spent during the rst experiment was 10$. Later, [Rumshisky, 2011] conclude that clustering the 350 concordance lines into sense related groups would yield only reliable results for about 140 concordance lines and [START_REF] Rumshisky | Word sense inventories by non-experts[END_REF] report that one of the major issue is to attract high-quality annotators on a service like AMT to perform complex linguistic tasks. If the use of best practice guidelines helps, it does not supresses the need to run preliminary experiments to calibrate the task parameters and interface. Another quality improvement in the result can be obtained by comparing AMT worker results using worker-quality weighted majority votes. However, while writing about the 2009 experiement [START_REF] Rumshisky | Word sense inventories by non-experts[END_REF] report that the experiments run with the sme prmE eters tody do not led to either fst ompletion or qulity nnottionF. The authors found found they could get an important increase of results quality by restricting the location of the annotators to the USA, but the tak took longer to complete and costed more. Factors that attract good annotators in large numbers are the pay rate (which can varied at that time from a few cents to 20 $ per task), the height of the task in the task search space proposed to annotators and the apparent simplicity of the task and the clarity of the task description, as well as the standing of the image of the task proposer in the AMT annonators specic social media which depends on the rapidity of the payment and speed of answering AMT annotators' questions.

For instance [START_REF] Rumshisky | Word sense inventories by non-experts[END_REF] report that in the US local restricted prototype experiments, the task failed to complete when only 0.01 $ were oered to provide ve judgments and the location of the taks was very low in the task list because not much data was proposed for annotation. Keeping the in US requirement, the best strategy that the authors found was to pay 3 cents for 10 judgments in one task, allowing a maximum worker error rate of 27%, with at least a 85% previous task approval rate on at least 200 tasks, and oering bonuses to the top workers. They obtained an F-score of 0.72 with a kappa of 0.69. The dierences observed by the authors between the 2009 and 2012 experiments show that the AMT market place is evolving rapidly and that the validity of best practice guidelines needs to be checked frequently.

In [START_REF] Munro | Crowdsourcing and language studies: the new generation of linguistic data[END_REF], the authors investigate the use of AMT for a range of linguistic experiments from semantics to psycholinguistics dealing with:

1. verbs semantics, 2. segmentation of audio speech stream, 3. language models, 4. speech grammaticality, 5. thematic role, 6. methapor brain processing, 7. and reading attentiveness.

For instance they report between crowdsourced and laboratory condition kappa values of 0.9 for the verb semantics task and of 0.759 for the language model task. Following their experimental results, they conclude that crowdsourcing provides the means to run systematic, large-scale judgment studies at a lower cost and much more easily than when doing them under laboratory conditions.

The experiment with AMT presented in [START_REF] Negri | Divide and conquer: Crowdsourcing the creation of cross-lingual textual entailment corpora[END_REF] Because of language variability, rule based semantic processing requires solving a large number of small inference problems, like for instance knowing that if work s then it also means that ws hired s . But the evaluation of such rules is problematic because no reference dataset exists and extrinsic evaluation in the context of an applicative task will not necessarily produce evaluation of the rules since it is dicult to assess to which cause the application performance measured must be imputed to. So [START_REF] Zeichner | Crowdsourcing inferencerule evaluation[END_REF] experimented using crowdsourcing (CrowdFlower) to perform inference rule evaluation. To asses a particular rule application, one must answer 3 questions:

1. Is the left-hand side of the rule meaningful?

2. Is the right-hand side of the rule meaningful?

3. If yes was to answer to both previous questions, does the entailment holds?

There were 2 types of cascading tasks submitted to the annotators:

1. to appreciate the rule relevance, which corresponds to the rst two previous questions 2. and to judge the entailment.

Depending on the answer produced by a task from the rst type, the annotators working on the second type of task has either to validate the entailment or to ascribe the identied nonrelevance of the rule to an erroneous answer provided for one the three original evaluation questions. Four inference algorithms were tested on a database of predicates extracted from ClueWeb09 web crawl2, where each extraction comprises a predicate and two arguments, providing four datasets. For each 5,000 extraction were sampled and for each dataset four rules common to all datasets were extracted which resulted in 20,000 rule applications, out of which 10,443 were discarded due to low CrowdFlower annotators condence, a further 1,281 were agged as meaningless left-hand side applications and another 1,012 as meaningless righ-hand side applications. Out of the remaining 8,264. rule applications that were passed on to the second type of task (entailment assessment), 5,555 were judged with a condence high enough, 2,445 with positive entailment and 3,108 with negative entailment. In the experiment a total of 6,567 application rules were annotated for a total cost of 1,000 $. Note that the second type of task required specic experimentation from the authors with the annotators in order to calibrate the communication of the task in order to improve the kappa performance. Th experiment was considered a success by the authors.

Statistical Modality Tagging was investigated by [START_REF] Prabhakaran | Statistical modality tagging from rulebased annotations and crowdsourcing[END_REF] Modality can be dened as a quantication over possible worlds or a speaker attitudes with respect to a proposition. Acquiring training data for building an automatic modality tagger is a dicult problem. In a pilot study, the author obtained and ran the modality tagger described in [START_REF] Baker | A modality lexicon and its use in automatic tagging[END_REF] on the English side of the Urdu-English LDC language pack. Using AMT, they estimated the precision of this type of approach at around 60%. They posted on AMT a set of randomly selected sentences (1997) that the tagger had labeled as not having the Want modality. Each sentence was checked by 3 annotators to decide whether it had the Want modality or not. Using majority rules on the annotations, 95 (4.76%) of the sentences were marked as validated Another set of 1993 sentences annotated by the tagger as not having the Want modality were posted on AMT, out of which 1238 were validated by the annotators.

Hence, the authors decided to apply a simple tagger as a rst pass, with positive examples subsequently hand-annotated through AMT. The simple tagger used a word spotting approach with a set of trigger words like try, plan, aim, wish, want etc. In the corpus, the number of sentences for each modality was limited to 50 for each trigger word in order to preserve linguistic variability. The AMT annotators were asked to check that the modality was not present in the sentence, otherwise they had to highlight the target of the modality. Each sentence was annotated by 3 persons. Only the output produced by adult annotator with an approval rating above 95% were considered if they had completed at least 50 tasks. They were paid 0.10 $ for each set of ten sentences. Only the annotation which gathered the approval of at least 2 annotators out of 3 on the modality and target were kept. Out of the resulting 1,008 examples, 674 had 2 annotators agreeing while 334 collected unanimous agreement. This work proved that it is possible to combine a high-recall simple tagger with crowdsourcing annotations to produce training data for a modality tagging.

[ [START_REF] Lafourcade | Propa-l: a semantic ltering service from a lexical network created using games with a purpose[END_REF] propose to use the paradigm of GWAP (Game With A Purpose)

to build lists of semantically-related terms, for instance needed to to deploy parental control systems on Internet. At the heart of the system is a GWAP that is used to build a semantic network where players collect credits when they provide answers similar to other players, with higher rewards for original contributions. The game played involve lexico-semantic relations like is-a, hyponym, characteristic, location, agent, patient, etc. In their paper the authors propose an algorithm to exploit the resulting semantic network to deploy a ltering service.

The network produced for the oldest game on French contained in October 2013 approximately 300,000 terms, including 15,000 to 20,000 word usages and more than 6 million relations. The main game has been played more than 1.3 million times by more than 3,500 registered players.

The experiment demonstrated the possibility to build and dynamically update a semantic network through crowd sourcing with a GWAP interface. Also from 2009, the work of [START_REF] Hsueh | Data quality from crowdsourcing: A study of annotation selection criteria[END_REF] report on the use of AMT to classiy text snippets extracted from political blogs, rst according to the the political candidate they concern and the polarity they express (Positive, Negative, Both, or Neutral) and second, according to whether they support or oppose the political candidate they are related to. The measured agreement among three expert annotator on the relevance to a particular candidate was 77.8%

while overall agreement on the four-classes sentiment annotation was 70.4%. For the second classication step (support/oppose/neutral) the author report an agreement of 76.8%. In comparison, each snippet of a set of 100, was marked by 5 AMT annotators taken from a group of 25 annotators selected on the basis of an approval rate higher thatn 95% and paid 0.04 $ per annotation. Each snippet required on average 40s to be completed and the lower overall agreement measured on all four-class sentiment task was 35.3%. On the second classication subtask (support/oppose/neutral) they achieved a value of 47.2% of agreement, but they nevertheless managed to reach a relatively good level of agreement on the simpler tasks of determining whether a snippet is releveant to a particular political candidate (81.0%), whether the snippet is subjective or not (81.8%) and whether the snippet is positive versus negative (61.9%).

But there exists a group of annotators that produces more noisy annotations than the other, judging against annotations produced by a majority vote, 20% of the AMT annotators have a noise level that exceeds 60% which disagree in 70% of the cases with the result of majority.

The more the text snippet is ambiguous the lower is the agreement. In this paper 3 quality measures were found to be useful for selecting annotations: the noise level of annotator, the inherent ambiguity of the class labels and the informativeness of the annotated data.

A ne grained annotation task identifying word expressing a sentiment about a particular in-sentence target was experimented by [START_REF] Sayeed | Crowdsourcing syntactic relatedness judgements for opinion mining in the study of information technology adoption[END_REF] with CrowdFlower. This taks is in a simpler form (no identication of text spans required by the annotators), the task that will be used in the uComp evaluation campaign.The annotators had to classify as POSITIVE, NEGATIVE or NONE the preselected opinion words depending on their relation with a preselected target word located in the same sentence. The authors used the same techniques as [START_REF] Hsueh | Data quality from crowdsourcing: A study of annotation selection criteria[END_REF] for discarding noisy annotations. The experiment had 200 tasks paid 0.04 $ per task, with three dierent annotators performing each task, for a total cost 60 $ and a time span of 24 hours to complete the job. In addition, 30 tasks were used to dene ar gold standard which served to identify unreliable annotators (with less than 65% accuracy). achieved after dropping the 10 worse annotators, note that in that case the kappa dropped to 0.54, because when going over 80% limit, performance start to decrease if more annotators were dropped because they were not numerous enough anymore for the voting mechanism to have a smoothing eect. This experiment showed that ne grained annotation is possible with crowdsourcing but requires extensive quality control procedures.

Named Entities

CrowdFlower is mentioned along AMT in [START_REF] Finin | Annotating named entities in twitter data with crowdsourcing[END_REF], which describes an experiment about the annotation of Named Entities in Twitter (person, organization, location, or none)

and the collection of judgements on the quality of word clouds for semantics and sentiment representation. The dataset was split into tasks containing 4 previously unlabeled tweets and 1 previously labeled tweet. On AMT, 251 tasks were submitted, each to be completed twice for an overall duration of 15 hours to complete the whole set of tasks at a total cost 27.61 $, which corresponds to 0.0275 $ per tweet. There were 42 AMT annotators mostly from the US and India and some from Australia. Most annotators performed a single task and one annotator did most of them. Inter-annotator agreement was checked using an algorithm akin to Google's

PageRank [START_REF] Page | The pagerank citation ranking: Bringing order to the web[END_REF]). The eectiveness of AMT annotators was judged inferior the to the one of the expert annotators but it was possible nevertheless to achieve the same results at a lower costs by by carefully combining annotations. The CrowdFlower experiment involved 30 tweets and each task had 3 tweets for a price of 0.05 $ to be done in 30s and representing a total cost of 2.19 $ (overhead included), and a rough pay of 2 $ per hour per annotator. The cloud comparison experiment was run on AMT, a task consisting in deciding which of the two word cloud presented to the annotator describes best the query topic. After selection with an average accuracy rating of at least 0.75 % on 7 questions, there were 8 AMT annotators selected and they did achieved a performance level of 61% of accuracy against gold data. The authors found particularly helpful the extra functionalities for managing the task and validating the annotators work provided by CrowdFlower over those of AMT.

[ [START_REF] Sayeed | Crowdsourcing the evaluation of a domain-adapted named entity recognition system[END_REF] deploy crowdsourcing tasks to evaluate name entity recognition algorithms. They use AMT to assess the performance of an algorithm that identies names of persons and organizations (ENAMEX NIST ACE-standard). For each task, each annotator was paid 0.05 $, summing up to 150.00 $ for 3 annotators. It took more than an estimated two person weeks to complete the work. They showed that crowdsourcing can provide reliable results and provide simple means of verifying algorithm performances, in a context where the aim is to reduce the rate of false positives.

The authors of [START_REF] Higgins | Mturk crowdsourcing: A viable method for rapid discovery of arabic nicknames? In roeedings of the xeegv rv PHIH orkshop on greting peeh nd vnguge ht with emzon9s wehnil urk[END_REF] describe how they used AMT to collect Arabic nicknames for completing exiting Named Entity lexicons. In addition, they experimented the eect that increasing the pay rate had on taks completion speed. On average, a pay of 0.03 $ per task yielded 9.8 names per day, increasing the pay to 0.05 $ made the number of collected names up to 25 names per day and to reach the value of 100 names per day we had to go up to 0.25 $ per task. In another experiment they increased the price from 0.01 $ per task to 0.05 $, of course increasin the price shortened the collection time, but surprisingly enough, the publication of the higher paying task had also an accelerating eect on the lower paying task.

Machine Translation

Translation from Urdu to English was the theme of the AMT experiment reported in [START_REF] Zaidan | Crowdsourcing translation: Professional quality from non-professionals[END_REF], the price paid was 0.10 $ per translation (approx. 0.005 $ per word). Four translations for each sentence were collected from dierent translators and a set of AMT task was created to cross validate the translations by other AMT annotators (ranking four translations at a time from best to worse or postediting). The task were paid on a basis of 0.10 $ for translating a sentence and 0.25 $ to edit a set of ten sentences, and 0.06 $ to rank a set of four translations. The overall costs were:

translation cost: 716.80 $, editing cost: 447.50 $, Ranking cost: 134.40 $.

Adding Amazon's 10% fee resulted in a total less than 1,500 $ to more than 7,000 translations produce around 17,000 edited translations and rank 35,000 labels (since each ranking task involved judging 16 translations, in groups of four). Including the cost of the professional translation reference would add 1,000 $ to the total cost which now amounts to 2,500 $. The participation of the AMT translators was 52 for translation (138 sentences on average), 320 for editing (56 sentence on average) and 245 for ranking 245 Turkers (averaging 9.1 ranking task each, or 146 rank labels). The authors investigated cost reduction by eliminating the need for professional translation and decreasing the amount of edited translations. The rst measure produced a signicant drop of quality (BLEU score of 34.86), while the second measure greatly reduced the cost but managed to maintain good performance (BLEU score of 38.67). Then, the best strategy for translation seems to be for each source sentence produce several translations, rank the multiple translation, edit only the top ranked translations.

The work of [START_REF] Hu | The value of monolingual crowdsourcing in a real-world translation scenario: Simulation using haitian creole emergency sms messages[END_REF] does not make use of AMT or CrowdFlower but is nevertheless interesting as it deploys two crowds of translators in a setup comparable with this two collaborative infrastructures. The experiment combined machine translation with human computation using two crowds of monolingual source (Haitian Creole) and target (English) speakers, for the Source speakers could document error spans either by rephrasing, either by annotating the spans with images or Web links (e.g. Wikipedia). The process was asynchronous for participants from both locations and the voting based best translation could be extracted at anytime.

[ [START_REF] Post | Constructing parallel corpora for six indian languages via crowdsourcing[END_REF] applied AMT to the building of a collection of parallel corpora between English and six languages from the Indian subcontinent, low-resourced and under-studied which are dicult form machine translation: Bengali, Hindi, Malayalam, Tamil, Telugu, and Urdu.

The source documents in English were the top-100 most viewed wikipedia pages for from each language. Since the authors were not proecient in all the six Indian languages, the decided to appreciate the quality of the AMT translator by comparing their production with the lexical translation of the original sentence by means of bilingual dictionaries bootstrapped for each language by means of another set of AMT tasks. In this other set of disctionary tasks, AMT translators had single words or very short phrases to translate while the validation was done with the wikipedia page titles whose translation can be assumed to be found in Wikipedia following the cross-lingual links. In the translation work, subsequent sentences from the original text were grouped by 10 to provide an AMT task, and translators had to provide a free-form translation. words in just under a week. Some authors assess the cost of professional translation from Tamil to English at 0.30 $ per word while the cost measured in our experiment was less than 0.01 $ per word. Because no professional translation was available to the authors they could not use direct performance measures like BLEU to assess the variation of quality of the translations.

Nevertheless, they designed another task in which original sentences were displayed along with four of its translations to another set of AMT translators for ranking the best. Each task was performed by 5 translators. Approximately 65% of the sentences had ve votes cast on just 1 or 2 translations while 95% of the sentences had all the votes attributed to 1 to 3 sentences.

This suggests that dierences in translation quality existed but also that the translators took their did their assesment job seriously enough to report on the dierences.

Information Retrieval/Extraction & Question Answering

The study of [START_REF] Grady | Crowdsourcing document relevance assessment with mechanical turk[END_REF] is about human factor parametrization for a crowdsourcing tasks of relevance judgement in information retrieval. The author investigated, on AMT, the impact that the following four parameters had on the cost, time, and accuracy of the assessments:

1. providing the annotator with only a title for a query versus a detailled description, 2. dierent wording for the task title (specialized, i.e. binary relevance judgement, versus layman, i.e. yes/no), 3. the amount paid per task (0.01 $ versus 0.02 $), 4. and a bonus: (0 versus 0.02 $).

Assessment were done with document from the TREC TIPSTER collection of news articles (using gold standard data enabled easy computation of AMR annotators accuracy). Five batch evaluations were done, for each of four topics, ve documents were assessed and 10 assessments were collected for each document. A total of 200 tasks were submitted to AMT for each batch, resulting in 1000 tasks for the ve batches together. The length of the documents was between 162 words and 2129 words (including HTML tags and single-character tokens). For each task, the annotater had to perform a single binary relevance judgment linking a query and a document. There were 149 annotators who participated, some of them to the 5 batches. In batches 2 and 3 one variable was modied with respect to batch 1, while in Batches 4 and 5 it was against batch 3 conguration that one variable was modied. In batch 5, 23 bonuses were given for a total cost of 0.46 $. Statistical signicance was measured via a twotailed unpaired t-test. The only signicant outcomes observed were increase in comment length and number of comments The authors created a web application to simulates the experience of playing quiz bowl where text is incrementally revealed until the user decides to answer. The answer is judged with a string matching algorihtm. More than 7000 questions were answered in the rst day, and over 43000 questions were answered in two weeks by 461 users.

Language Communication Enhancement/Validation

It is to palliate the lack of common reference corpus for evaluating grammatical error detection that [START_REF] Madnani | They can help: Using crowdsourcing to improve the evaluation of grammatical error detection systems[END_REF] to create a corpus of ctional AAC messages. In their paper, they show that the messages they produced a better model of AAC communication than the material generally used so far from telephone conversations or newswire text. Two types of tasks were proposed:

1. the AMT workers tried to imagine how he would used an AAC device to communicate if he was subject to a communication impairement, 2. AMT annotators were asked to assess the plausibility of communications produced by the previous type of task.

Task of the second type gathered more easily AMT participants that tasks of the rst type.

The average task completion time was shorter (24s on average) for the second type of task, as opposed to an average of 36s for the tasks of the rst type. During the experiment, the pay for the the second type of task was reduced from 0.04 $ (the price paid for both types of tasks) to 0.02 $ whithout loss of performance. For comparisons the authors trained language model with material of dierent origins (Wikipedia discussions, Usenet, Switchboard, newspapers, Twitter...) and compared the performance with models trained on the crowdsourced data.

Compared to a model trained only on Switchboard, their best model reduced perplexity by 60-82% relative on three AAC-like test sets, which represents a potential keystroke saving of 511% on a predictive keyboard interface.

Real Time Dialog System

The work of [START_REF] Bessho | Dialog system using real-time crowdsourcing and twitter large-scale corpus[END_REF] addresses the creation of utterance/reply pairs for Japanese dialog system design. Here the wisdom of the crowd is not provided by a crowdsourcing framework but by Twitter. For each user input, the system will extract from the utterance-pair database the pair for which the tweet is mbost similar to the input part of the pair and the system response is provided by the output part of the pair. The utterance pair database was built using a corpus of 1.2 million utterance-pairs from Twitter which were written in Japanese, contemporary, and had a in-reply-to eld. The author propose to integrate into the dialog system a real-time crowdsourcing functionality to handle the cases where the system cannot provide an adequate answer (similarity distance is below a certain threshold between processed utterance an input parts of the utterance pairs stored in the database). The original user input is recast into a tweet from the dialog system chatbot and if a crowd member responds before a certain delay the crowd answer is used as reply by the system. For evaluation, 90 user input examples were selected and 20 utterance-pairs were extracted from the database retrieved from Twitter for each per user input, totaling to 1,800 of triples (user input and utterance pair). Thirty subjects evaluated naturalness and versatility of the responses (600 triples each).

Conclusion

The conclusion that we draw from this survey is that crowdsourcing or similar approaches like collaborative ones mediated by Internet or GWAPs ( [START_REF] Lafourcade | Propa-l: a semantic ltering service from a lexical network created using games with a purpose[END_REF]) oer the possiblity to develop language ressources for which human language processing functionality is required at a cost much lower than with classsical means of production, for various kind of tasks, in particular task like semantics or translation. This is so because the people employed in this kind of infrastructure are recruted in large numbers, potentially from all over the planet and do not need to have a particular expertise except of being proecient in (most often) one or several languages. If the number can compensate the quality of the individual annotations by cross-validation, the measure that a task proposer has to deploy to ensure a minimum of quality require either gold standard data in sucient number or the involvement of an expert, in addition to designing specic procedures to prevent cheating. Despite the extra cost incurred by the measures deployed to ensure quality of the resource produced by crowdsourcing, if the feasability study has been properly done the resulting cost is much lower than with traditional means. But crowdsourcing infrastructure evolve rapidly and crowds of participants adapts rapidly to new tasks which means that quality measure need to be revised frequently.

There is a question that is rarely raised in the litterature, nevertheless addressed by [START_REF] Fort | Crowdsourcing for Language Resource Development: Criticisms About Amazon Mechanical Turk Overpowering Use[END_REF] and also the following presentations available on the Web [Lease, 2013a],

[Lease, 2013b], [Larson, 2013], it is the question of ethics about the minimal return pay somebody working full time in a crowdsourcing infrastructure is able to achieve. As a corrollary, one cannot but consider the question that the new infrastructures for crowdsourcing raises about social tax evasion.

7 language lexica Experiment

The rst part of Deliverable D5.2 consists in the 7 language lexica extracted using as baseline the pointwise mutual information (PMI) measure ( [START_REF] Manning | poundtion of ttistil xturl vnguge roessing[END_REF]) collected using climate change linguistic markers obtained by machine translation from the set of patterns developped for English, German and French [Scharl, 2014] Since the amount of data was too small, a second corpus extraction was done with a wider source set (less focused on institutional sources), initially on French only, which resulted in a 7,000 Tweets corpus, out of which a 9,930 entry lexicon was extracted using PMI, then further lemmatized by alignment with the content of the LEFF lexicon [Sagot, 2010].

The HC experiment with CrowdFlower

Concerning the comparison between automatic extraction of lexica and HC computations, there is an ongoing experiment running on Crowdower with the French lexicon as input. The parametrization of the experiment is as follows:

Each annotation task consists in a set of 8 lexical entries (units in Crowdower terminology, note that the number of units per task advised by Crowdower was 5) for which the annotator must provide answers to two questions:

1. Is the term related to an expression of opinion/sentiment/emotion (yes/no)?

2. Which among the following 20 categories ( [START_REF] Fraisse | ucomp deliverable d5.1requirements of aective knowledge extraction[END_REF]) of opinion/sentiment/emotion is the most appropriate to describe the meaning of the term?

the price set of the completion of a task is 0.07 e(not that the price advised by Crowdower was 0.10e), the total cost of the experiment is 200 e the experiment was started on Monday, July 21 th 2014,

The experiment was submitted directly to Crowdower, we did not test the GATE API yet.

Although the task is not yet completed, we already have gained a useful information concerning the potential use of services like Crowdower with respect to diculty to provide a task description for relatively complex classication schemes, since the amount of information that one can provide to the tskers is relatively limited.

Job design

As described in Figure 1 Concerning the quality control, we dened one test question (Figure 3). However, in order to have as many contributors as possible, we selected the performne level I (the lowest level proposed by CrowdFlower) for French (Figure ??). For the CrowdFlower job, we put 10 units per task that paid 0.06 e(Figure ??) for a task to be completed whithin an expiration delay of 30 minutes.

Results

The job was running for 3 months and as shown in Figure 4, only 9.1% of the job was completed.

This results can be explained rst by the complexity of the task. In fact the ne-grained classication task is more complicate than a simple binary classication task and this is true even if the contributors are native speakers. The second raison, for this low score is the payment of the task. In fact, contributors are less motivated when the task is underpaid (0.06 einstead of 0.10 ethat was recommended by CrowdFlower).

In the Figure 5 developed in WP2 will be used rst to build and validate the gold standard, second to propose input resources to the participants (ontology elements of WP4, aect lexicons, etc.). Thus a participant will be able to either use their own resources, or build on the proposed input resources to improve their system (or both, in which case we will be able to assess the impact of the resource choice on system performance). The campaign will also be an occasion to test replacing a static comparison against a gold standard by a dynamic assessment of the participants' data through HC.

Data

A model of language data annotation

Comparing the HC framework against the classical annotation approach for producing evaluation ressources calls for some automatic support for what concerns the mapping from the classical linguistic annotation scheme, designed for expert annotators, onto a smaller grain annotation procedure suitable for crowdsourcing or GWAPS users. To this end we rst provide a task generic annotation formalisation which we will use in a second step to draw the specication of a data converter that will automatically prepare a corpus for crowd annotation from a corpus prepared for classical annotation, allowing the task designer to specify relevant parameters.

Annotation representation

In what follows, we call control task, the information processing task for which we want to assess the performance of some computer system, for instance if we look at Information Retrieval (IR), given a set of documents and a query, the control task consists in identifying which documents are relevant with respect to the given query [Cleverdon, 1960]. In fact, for any objective evaluation task, the systems under test are asked to output a symbolic represention in function of the objects and relations that they have found to be present in the input representation (the test data). The control task may be limited to identifying only the objects present in the test set, eFgF POS tagging [Paroubek, 2007] for which the output representation is made of the word boundaries and their class label. Sometimes the boundaries of the objects present in the test set are given and the systems need only to identify the class they belong to (eFgF IR or Word Sense Disambiguation [START_REF] Edmonds | Introduction to the special issue on evaluating word sense disambiguation systems[END_REF]). On the other hand, some control tasks are much more complex and require identify objects, primitive relations holding between objects and also higher level relations holding between primitive relations, like in parsing [de la Clergerie et al., 2008], anaphoric resolution [START_REF] Vilain | A model-theoretic coreference scoring scheme[END_REF] or image recognition [START_REF] Unnikrishnan | Toward Objective Evaluation of Image Segmentation Algorithms. siii rnstions on ttern enlysis nd whine sntelligene[END_REF]. For tasks like machine translation, we are in general only interested in the nal result of the transformation of the objects and relations that the system under test has identied in the input data. In the most general case, a control task can be seen as a process that links together test data units resulting from a segmentation process and annotation symbols possibly organized in several layers. Assuming that the test data is the result of a segmentation process of an input medium (character stream, speech signal, pixel array, etc.) represented by S = {s i /0 ≤ i ≤ N ∈ N}, and the set of annotation labels by A, the m layers of relations graphs R resulting from the annotation process can be expressed as follows

2 : R = m j=1 R j , m ∈ N R 1 = q k=1 {r l /l ∈ N, r l ⊂ P(S k × A)} R i = u k=1 {r ⊂ P((S ∪ R x 1 ) × (S ∪ R x 2 ) • • • × (S ∪ R x k ) × A), 1 ≤ x k < i} (1)
R 1 represents the rst layer of annotation of the test data (i.e. the set of all relations of any arity between initial segmentation units) and the R i represent the successive layers of annotations that may reference annotations from any previously existing layer down to the initial data segments themselves (see Figure 6).

Note that with respect to the nnottion grph model from LDC [START_REF] Bird | A Formal Framework for Linguistic Annotation[END_REF], which directly link annotations to events from the various linear input streams that they decorate, we encode in our model the (potentially recursive) structure of the annotations. Such information is important in our opinion when comparing annotation schemes to take into account their relative structural complexity. Our representation is very much like the one proposed in [START_REF] Roth | A Unied Representation and Inference Paradigm for Natural Language Processing[END_REF] except that we adopt a slightly more general point of view by abstracting any implementation detail like annotations identiers, types, constituents etc. to simply a kind of relation.

(S (NP-SBJ I) (VP consider (S (NP-SBJ Kris) (NP-PRD a fool)))) In the literature, computer traces resulting from performing the control task are called hypothesis (annotations) and the human ones: gold stndrd or referene (annotations). The objective evaluation result is obtained by comparing the human and computer traces produced in response to the test data, iFeF comparing the set of relations identied by the computer (H ) with the one identied by humans (G). When the evaluation is quantitative, the result is obtained by computing some mesure dened over the two sets of relations. Here it is important to point out that the result of evaluation, which is a measure R × R → R, is a function of T the test data, whose role in linking reference and hypothesis data is essential for the computation of the evaluation result. Even more so, when the system under test uses relations R which have dierent semantics from the reference ones (but nevertheless mappable to them), or when it modies the input data because of noise, data corruption or specic normalization, or when it uses a segmentation function S dierent from the reference one. With N the noise function, the hypothesis is then better described as :

H = n j=1 H j , n ∈ N H 1 = q k=1 {S : T → P(T ), S = S, A = A, N : T → T, r l ⊂ P(S (N (T ))) k × A )} H i = u k=1 {r ⊂ P((S (N (T )) ∪ H x 1 ) • • • × (S (N (T )) ∪ H x k ) × A ), x k < i} (2)
In addition to the mapping µ from relation annotation labels A to A, which in general is provided by the participating system, one must then be able to nd a reasonable mapping M between S(T ) and S (N (T )) to be able to compute an evaluation result. By reasonable, we mean a mapping that maximizes the global similarity between the reference and hypothesis versions of the annotated material with respect to a particular similarity function σ. This is what is done for instance when one uses dynamic programming to nd the mapping that minimizes the edit distance between two slightly dierent versions of the same text to compare their POS tag annotations [START_REF] Paroubek | The GRACE French Part-Of-Speech Tagging Evaluation Task[END_REF].

M = argmax m h,g σ(m(h), g), (3)
m ∈ P(S (N (T ))) → P(S(T )), σ :

P(S(T )) × P(S(T )) → [0, 1]
Most of the time, an evaluation campaign will dene several measures in conjunction and use the vector space corresponding to the measurement tuples to represent the performance of each system as a point in the n-dimensional Euclidean space. Their relative position is then characterized by their distance, justifying the use of the term metri instead of measure.

Applying our model to real evaluations

In this section, we use the model proposed in section ?? to represent well known evaluation protocols from dierent domains of natural language processing [START_REF] Paroubek | Principles of Evaluation in Natural Language Processing[END_REF].

Q 1 Q 2 . . .

Classication

The rst type of applications that we identied is the general domain of classication and relation extraction. In this kind of task, the purpose is to segment a data set in order to highlight parts of this set that belong to specic classes (predened or not), and possibly to provide relations existing between these segments.

We take the examples of information retrieval, named entity recognition, temporal annotation and parsing.

Information retrieval Classical information retrieval aims at nding full documents that are relevant to a given query Q. The document collection C is the input data (the retrieval unit is the entire document level). Here we consider a simplied instance of the general model presented in the previous section, in a sense that the segmentation of the test data into units to be annotated is provided, it is made of the documents themselves, see Figure 7.

This is a classication task, since the aim is to produce a partition of the collection, between relevant and non-relevant documents, with respect to the query. In practice the evaluation data contains several queries, but since in general they are considered independent of each other, the evaluation resolves to a series of single query evaluation (see Figure 7).

In other words, variables introduced in Section ?? are instanciated in the following way:

T : the set of documents S(T ) : the structuration corresponds to the existing document boundaries, ∀t ∈ T, S(t) = {t} A : a set of two labels: relevnt for the query or not relevnt for the query R = R 1 : a singleton made of one unary relation that tag the relevance of each documents.

Named entity recognition

The following steps can be identied concerning named entity recognition:

1. Identication: nding which data units of the test set need to be annotated.

2. Categorization: nding the appropriate relation to annotate a data unit from the test set, eFgF tagging word sequences with labels for locations, persons, organisations, etc.

A normalization step can be added, as for example at the Temporal Expression Recognition and Normalization (TERN) Task of EVALITA [START_REF] Magnini | Evaluation of Natural Language Tools for Italian: EVALITA 2007[END_REF], where temporal expression should be associated with a universal representation of the expression. All NE types can be concerned by this normalization, for example person names, since they exhibit often many variations in their realization: Barack Obama, B. Obama, President Obama, Barack H.

Obama.

Note that for named entity recognition, the segmentation function of the test data into elementary units is generally not provided by the evaluation organizers. This is not the case for the following example: TempEval.

T : a document, seen as a stream of words or characters S(T ) : the segmentation of NE types, at character or word level A : the set of NE class labels R = R 1 : a singleton holding the unary relation linking the NE to its class label.

Temporal annotation Temporal annotation as dened by TempEval evaluation campaign [Verhagen et consists in the following: given a set of test texts for which sentence boundaries are annotated, as well as all temporal expressions and events in texts, the control task goal is to link events to other events, or events to time expressions (see Figure 10). A : the set of temporal expression signal and event class labels, as well as temporal relations labels. R = R 1 : 1/ the relation that links a temporal expression, signal or event to its class label, eFgF kipnpped is an event.

2/ plus all the labeled time relations between the temporal elements eFgF kidnpped is before resued. dependencies, relations linking a particular word (the head) and one of its dependents, links, relation between pairs of words whithout necessarily referring to a tree hierarchy, grammatical relations, i.e. relation/head/dependent tuples [START_REF] Watson | Ecient extraction of grammatical relations[END_REF],

derivation/derived tree [Schmitz and Le Roux, 2008] describing the construction of the syntactic parse tree , etc.

Since theories and annotation schemes are quite numerous and diverse in parsing, we present here only a few annotation schemes which have been used for evaluation: the PennTreebank [START_REF] Marcus | Building a large annotated corpus of English: the Penn Treebank[END_REF] for constituent analysis of English and PASSAGE [START_REF] Vilnat | PASSAGE Syntactic Representation: a Minimal Common Ground for Evaluation[END_REF] for The Penn Treebank constituent annotation model:

T : a documents seen as a stream of characters S(T ) : the segmentation into words A : the set of constituent labels R 1 : relations between words and their deepest layer of constituent label R i : relations between words and constituent labels, or between constituent labels of deeper levels

R : m j=1 R j , m ∈ N
The PASSAGE annotation scheme has only one layer of non-recursive syntactic chunks and grammatical relations dened between words and/or chunks (cf Figure 11). Element of comparison between the passage annotation scheme and PARC, SD and GR, three other syntactic annotation schemes used for English parsing evaluation are provided in [START_REF] Paroubek | The PASSAGE Syntactic Representation[END_REF].

The PASSAGE annotation model:

T : a documents seen as a stream of tokens S(T ) : the segmentation into words A : the set of chunk and relation labels R 1 : the relations between words and their chunk labels, or relations for which at least one argument is word (e.g.

coordinating relation for whose coodinating conjunction argument is always a single word not included in any chunk, see Figure 11)

R 2 : the relations linking chunks only R : R 1 ∪ R 2

Transduction

Lastly, a very dierent type of applications is the set of applications producing an output that is not an enrichment (or annotation) of an existing test set, but a new object obtained by transformation from or in response to another object. Examples of transduction applications are: machine translation, speech synthesis, automatic summarization, language generation [START_REF] Koller | Report on the Second NLG Challenge on Generating Instructions in Virtual Environments (GIVE-2)[END_REF] or machine dialogue.

For all these examples, T is a document seen as a sequence of characters to be either translated, synthetized, summarized, etc. S(T ) is the existing segmentation into language units, while A is the result of the operation: the translation of a language unit into the target language, the synthesis of a language unit into sound generation instructions, etc. R is the set of links between language units in the test set and elements from A.

T : a documents seen as a stream of tokens S(T ) : the segmentation into transduction source units A : the corresponding transduction target units labels R = R 1 : relations between source and target units Note that we can consider multilingual alignment tasks [START_REF] Chiao | Evaluation of multilingual text alignment systems: the ARCADE II project[END_REF] to be degenerate cases of transduction task, where the target labels are provided as input data and the systems under test need only to identify the relation between source and target units.

Expert Annotation Guidelines

Opinion, Sentiment and Emotion annotations are applied on microblogs text message (maximum lenght of 140 characters). The annotations scheme was inspired from the PASSAGE evaluation campaign of syntactic parser of French for the structure to which a specic semantics for opinion, sentiment and emotion annotation were added. With the previous fromalism, the annotation structure is as follows :

T : a documents seen as a list of tokens S(T ) : the segmentation into words A : the set of chunk and relation labels R 1 : the relations between words and their chunk (group) labels, or relations for which at least one argument is word R 2 : the relations linking chunks only R : R 1 ∪ R 2

In total the uComp annotation scheme holds 7 main types of groups (chunks) which resolve to 27 possible terminal labels (in the previous model |A| = 27).

1. HOLDER 2. Opinion/Sentiment/Emotion Expression (OSEE), which is further rened into dierent sub-categories, 21 of ne semantic grain split into 7 grained positive, 11 negative, one neutral for instructions or demands and 3 of coarse semantic grain: positive, neutral and negative, totaling in nal 24 possible annotation labels for groups. 1. SAY, that links an HOLDER to its OSEE.

2. ABOUT, which relates an OSEE to its TARGET.

3. MOD, connecting a modier, like an adjective or an advert to OSEE. This group will be used in the case the message is intended for someone.

  the litterature mention Amazon Mechanical Turk (AMT), with CrowdFlower appearing a little later. Our survey of previous work is organized along the following

  addresses sentence modication and textual entailment annotation in a multiligual context involving English, Italian and German. From a set of aligned sentences in the three languages, modications are done in a monolignual set-up (English) by paraphrasing and rephrasing with information addition or deletion. Modied sentences are then translated after having been annotated with entailment information with respect to the source sentence. The process of multilingual entailment is then simplied by having entailment processing done in a monoligual set-up. For quality control, only the annotation that gathered the agreement of 4 out of 5 annotators were retained and gold standard data test were run. Six types of tasks were submitted to the AMT annotators: of 1,620 pairs of sentences added to the corpus, while 721 were discarded, which required 22 days and 11 hours of AMT work for a cost of 435.71 $. Including the work of the expert to prepare the gold data and manage the pipeline, the overall operation was a success and proved the usability and eciency of crowdsourcing for building the multilingual entailment corpus.

WP5

  WMT 2011 Haitian Creole to English translation task. The result showed that the combined approach translated 38% of the sentences well compared to Google Translate's 25%. The 4 were recruited from Haiti and did not speak English while the 26 English speakers were for 6 of them paid UMD undergraduates while the other 21 were volunteers of various background. Over 13 days, the total eort was 15 hours for Creole and 29 hours for English. The Haitian Creole sentence was rst automatically translated into English and presented to the English speakers who could take one of the following actions:1. mark a phrase in the candidate as an error, 2. suggest a new translation candidate translation, 3. change the ranking of the candidate translation. New translation candidates were back translated into Haitian Creole and along with spans marked as errors which were projected back to identify the corresponding spans in the source sentence by means of word alignment. In turn, the Haitian Creole speakers could: 1. Rephrase the entire source sentence, 2. explain spans marked as errors, 3. basing their action on the back translation, change the ranking of the candidate translation.

  They were paid 0.70 $ per task. Decision to accept or reject the translation was done manually, by checking various factors like comparison to a lexical translation obtaine with the dictionaries developped for the project,the percentage of empty translations, the amount of time that the translator required to complete the task, his geographic location (self reported and identied through is IP address) and by measuring the distances between the various translation of the same source excerpt. Malayalam provided the highest through-put, generating half a million WP5

  for higher-paying or bonus batches. The hihgest accuracy, 70.5% was reached with batch 3, which used a title query and a simple yes/no response. The use of description query did not entailed an accurary improvement. The fastest task completion, 72s, was measured for batch 4 while the average time completion per task across the ve batches was 63s. The number of unique annotators per batch varied between 64 to 72 for batches 1 to 4 but fell to 38 for batch 5, probably because of the bonus incentive, annotators in this last batch tended to complete WP5accuracy (3.37 documents correctly annotated as compared to 2.10-2.20 for batches 1 to 3 and 1.85 for batch 4). But bonus attribution requires expert human supervision and the question remains of the gain versus expert time spent for the operation to become benecial. From the words of the authors, this experiment provided largely inconclusive results.The following work was realized with the help of a web application game to collect crowdsourced input. After qualitatively examining how humans perform incremental classication[START_REF] Boyd-Graber | Besting the quiz master: Crowdsourcing incremental classication games[END_REF] have shown in their article how crowdsourced knowledge of a human's incremental classication process improves state-of-the-art rapacious classication. Then they built a Bayesian models that embedded in a Markov decision process to replicate the improved classication process and develop new hierar-chical models combining local and thematic content to better capture the underlying content. The corpus used was made of 37,225 quiz bowl questions with 25,498 distinct labels from 121 tournaments (between 1999 and 2010).

  decided to use crowdsourcing. For their experiment, the authors studied with AMT the presence of extraneous preposition in a corpus of students writing for a test of English as a foregin language. In this experiment 75 sentences were used as gold standard built by 3 experts and the remaining 923 sentences were annotated by 20 annotators located in the USA, whithin one day. Using 3 annotators per judgement with a majority vote yields an agreement with any one of annotator of 0.87 on average which corresponds to a kappa of 0.76. The extraneous preposition annotation costed 325 $. Further experiments were done with a gold standard of 20 sentences obtained with CrowdFlower, in particular to propose new evaluation measures, derived from precision and recall by weighting the evaluation of annotations items depending on the proportion of agreement for this item by the annotators. The authors found out that the weighted measures are more stable and contrary to regular precision and recall they display less a tendency to overestimate the performance of the system under evaluation. Similarly, for comparing two systems they propose to replace precision-agreement estimation by a kappa-agreement measure Augmented and Alternative Communication (AAC) devices enable users with communication disabilities to participate in everyday conversations. For designing such devices, the elaboration of language models representing as best as possible the style of the users' intended communications is essential. Since collection of geniune AAC material for designing such device is quite dicult, the authors of[START_REF] Vertanen | The imagination of crowds: Conversational aac language modeling using crowdsourcing and large data sources[END_REF] decided to use crowdsourcing WP5

  Various scoring functions were investigated by means of a ROC curve representation (true positives versus false positives). The area under the curve (AUC) was used to measure the classiers performance. A random classier has an AUC of 0.5, and ideal classier has an AUC of 1.0. The scoring function selected was the one with the best performance, here an AUC of 0

  Figure 1: Data sample

WP5

  Figure 2: Unit description

Figure 4 :

 4 Figure 4: Dashboard of the Workow job concerning the validation of the french aective lexicon

A

  public call for tender has been published in February 2014 by LIMSI-CNRS (PUMA Nbr. 43826) for providing reference annotations on microblog textual data (Tweets), for an amount of 4,200,000 signs (or 30,000 messages) equally spread between French and German (15,000 of WP5ELDA, the Evaluations and Language resources Distribution Agency (located 9, rue des Cordelières, 75013 Paris) was the only institution to make an oer and was selected.So far, a the French subcorpus was extracted by MODUL from the data collected with the wedi th on glimte ghnge demonstrator. A query with a list of regular expressions[Scharl, 2014] resulted in 3,433 tweets, 112 YouTube videos, 77 Facebook messages and 11 Google+ postings for the period 26 March to 26 May 2014. The rst exploration of the corpus showed a relatively high proportionof retweets or purely informative tweets. LIMSI using the same set of regular expression queries did an new extraction from Twitter (July 2014) on French which resulted in a corpus with a relatively higher proportion of subjective messages, containing 7,000 posts. Preliminary tests for the deploying the annotation framework have been done in collaboration with ELDA in July 2014 and production of the annotation of the French reference corpus is planned to start in full during August 2014.

Figure 6 :

 6 Figure 6: S i and R i for a PennTreebank syntactic annotation sample.

Figure 7 :

 7 Figure 7: Classication of the set of documents among two parts (relevant and non relevant) for each query: for Q 1 , D1, D4, D8 and D9 are relevant; for Q 2 , D2, D4 and D10 are relevant.

Figure 8 :

 8 Figure 8: Classication of the sets of characters considering the named entity types (here, organizations, locations, dates, none).

Figure 9 :

 9 Figure 9: Classication of the sets of characters considering the named entity types and relations to normalized entities in a separate knowledge base.

Figure

  Figure 10: Temporal annotation.

  chunks and grammatical relations in French. The PennTreebank example (see Figure 6 on page 23) is the rst example of annotation scheme in this article which exhibits both relations between annotated elements (words) and their class label (e.g. the relation between NP-SBJ and I), as well as relations between annotations themselves (e.g. the toplevel relation between S and the constituents NP-SBJ and VP).

Figure

  Figure 11: Example of PASSAGE annotation

  4. NEG, associating a negation marker to an OSEE. , that links the OSEE to its RECIPIENT when the OSEE if of subtype instruction constituée du groupe de mots qui référence l'auteur de l'expression d'opinion/sentiment/émotio (OSEE: Opinion Sentiment Emotion Expression).<p> The holder is the group of words, which refers to the author of the opinion, sentiment and emotion expression. If there is no explicit mention of the holder, we suppose that the writer of the text is the holder of the OSEE expression and we make no particular annotation. Are grouped under the label holder, the widest possible explicit mention of the holder: including modiers, appositions, conjunction, etc., in order to have maximum semantic information.

Figure 12 :WP5Figure 15 :

 1215 Figure 12: Holder (SOURCE in french) annotation

Figure 18 :

 18 Figure 18: Annotation with multiple target group

Figure

  Figure 20: Negation group

Figure 21 :

 21 Figure 21: Recipient group

FigureFigure

  Figure 22: DISPLEASURE group

FigureFigure

  Figure 39: The SAY relation

Figure

  Figure 47: Pasta-uComp annotation interface.

Table 13 :

 13 Number of entries for the 7 languages in rst version of D5.2

	.

  The OSEE group consists of the span of text which the semantic value corresponds to the OSEE expression. It will be annotated with one of the 22 semantic or aective categories given in the following table. The denitions of the categories are given afterwards, with annotations examples.

				CHIST-ERA	Subproject : WP5
					Task : T5.3
				Date : December 6, 2014
					Page : 34
	# Generic label	Dim. uComp specic semantic category
	1	NEGATIVE SURPRISE e-	negative surprise / negative amazement
	2	DISCOMFORT	e-	discomfort / disturbance / embarassment / guilt
	3	FEAR	e-	shyness / worry / apprehension / alarm
				fear / terror
	4	BOREDOM	e-	boredom
	5	DISPLEASURE	e-	displeasure / deception / abuse
	6	SADNESS	e-	sadness / resignation / despair / sorrow / hopelessness
	7	ANGER	e-	impatience / annoyance / irritation / nervousness /
				anger / exasperation
	8	CONTEMPT	e-	reluctance / contempts / disdain / blame /
				disgust / hate
	9	DISATISFACTION	s-	disappointment / disatisfaction / discontent / shame
	10 DEVALORIZATION	o-	disinterest / devalorization / depreciation
	11 DISAGREEMENT	o-	disapproval / disagreement
	12 VALORIZATION	o+	interest / valorization / appreciation
	13 AGREEMENT	o+	understanding / approval / agreement
	14 SATISFACTION	s+	satisfaction / contentment / pride
	15 POSITIVE SURPRISE e+	positive surprise / positive amazement
	16 APPEASEMENT	e+	relief / appeasement / peacefullness
				forgiveness / thankfullness
	17 PLEASURE	e+	pleasure / entertainment / enjoyment / joy /
				happiness / euphoria / play
	18 LOVE	e+	love / aection / care / tenderness / fondness /
				kindness / attachment / devotion / passion /
				envy / desire
	19 INSTRUCTION	i	recommandation / suggestion / instruction /
				order /command
	20 POSITIVE	+	underspecication for an positive OSEE
				which cannot be ascribed to any of the ner categories above
	21 NEGATIVE	-	underspecication for an negative OSEE
				which cannot be ascribed to any of the ner categories above
	22 UNKNOWN	?	some OSE is present but it is either missing from the list above
				or dicult to determine because it is composed of a mix
				of many dierent sentiments, emotions or opinions.
	WP5-uComp_Deliverable_D_5_3.tex
	Status : DRAFT

Table 31 :

 31 uComp ne grained semantic categories of OSEE.

e=emotion, s=sentiment, o=opinion, i=information, +=positive valence, and -=negative valence

WP5-uComp_Deliverable_D_5_3.texStatus : DRAFT