Djalil Chafa

Didier Concordet

Confidence regions for the multinomial parameter with small sample size

Keywords: Confidence regions, small samples, multinomial distribution

Consider the observation of n iid realizations of an experiment with d ≥ 2 possible outcomes, which corresponds to a single observation of a multinomial distribution M d (n, p) where p is an unknown discrete distribution on {1, . . . , d}. In many applications, the construction of a confidence region for p when n is small is crucial. This concrete challenging problem has a long history. It is well known that the confidence regions built from asymptotic statistics do not have good coverage when n is small. On the other hand, most available methods providing non-asymptotic regions with controlled coverage are limited to the binomial case d = 2. In the present work, we propose a new method valid for any d ≥ 2. This method provides confidence regions with controlled coverage and small volume, and consists of the inversion of the "covering collection" associated with level-sets of the likelihood. The behavior when d/n tends to infinity remains an interesting open problem beyond the scope of this work.

Introduction

Consider the observation of n iid realizations Y 1 , . . . , Y n of an experiment with d ≥ possible outcomes with common discrete distribution p 1 δ 1 + • • • + p d δ d on {1, . . . , d}, where δ a denotes the Dirac mass at point a. This corresponds to a single observation X = (X 1 , . . . , X d ) of the multinomial distribution M d (n, p) = 0≤k 1 ,...,kn≤n

k 1 +•••+k d =n µ p (k)δ (k 1 ,...,k d ) where µ p (k) = p k 1 1 • • • p k d d n! k 1 ! • • • k d !
where p = (p 1 , . . . , p d ) and X k = Card{1 ≤ i ≤ n such that Y i = k} for every 1 ≤ k ≤ d. Here d is known, X is observed, and p is unknown. The present article deals with the problem of constructing a confidence region for p from the single observation X of M d (n, p), in the non-asymptotic situation where n is small. More precisely, let

Λ d = {(u 1 , . . . , u d ) ∈ [0, 1] d such that u 1 + • • • + u d = 1}
be the simplex of probability distributions on {1, . . . , d}. The observation X ∼ M d (n, p) lies in the discrete simplex

E d = (x 1 , . . . , x d ) ∈ {0, . . . , n} d such that x 1 + • • • + x d = n . (1)
From the single observation X and for some prescribed level α ∈ (0, 1), we are interested in the construction of a random region R α (X) ⊂ Λ d depending on X and α such that

• the coverage probability has a prescribed lower bound

P(p ∈ R α (X)) ≥ 1 -α (2)
• the volume of R α (X) in R d is as small as possible.

These two properties are the most important in practice. We propose to solve this problem by defining the "level-set" confidence region R α (X) ⊂ Λ d given by R α (X) = {p ∈ Λ d such that µ p (X) ≥ u(p, α)}

where u(p, α) = sup u ∈ [0, 1] such that

k∈E d µp(k)≥u µ p (k) ≥ 1 -α .
One can check that this confidence region (3) contains always the maximum likelihood estimator n -1 X of p. Moreover, this region can be easily computed numerically, i.e. for each value of p one may compute u(p, α) and compare it to µ p (X). Furthermore, it fulfills [START_REF] Agresti | Simple and effective confidence intervals for proportions and differences of proportions result from adding two successes and two failures[END_REF], and the numerical computations presented in Section 3 show that it has small volume and actual coverage often close to 1α at least for d = 2 and d = 3. In fact, this region is a special case of a generic method of construction based on covering collections. The concept of covering collections is presented in Section 2 and encompasses as another special case the classical Clopper-Pearson interval and its multivariate extensions. On the other hand, it is well known (see for instance Remark 2.6) that a natural correspondence via inversion exists between confidence regions with prescribed coverage and families of tests with prescribed level. However, this correspondence is a simple translation and does not give any clue to construct regions with small volume.

Two kinds of methods for the construction of a confidence region for p can be found in the literature (see for instance [START_REF] Brown | Interval estimation for a binomial proportion[END_REF][START_REF] Brown | Confidence intervals for a binomial proportion and asymptotic expansions[END_REF][START_REF] Blyth | Binomial confidence intervals[END_REF][START_REF] Blyth | Approximate binomial confidence limits[END_REF][START_REF] Blyth | Correction: "Approximate binomial confidence limits[END_REF][START_REF] Newcombe | Two-sided confidence intervals for the single proportion: comparison of seven methods[END_REF] for reviews). The first methods give confidence regions with small volume but fail to control the prescribed coverage (e.g. Bayesian methods with Jeffrey prior, Wald or Wilson score methods based on the Central Limit Theorem, Bootstrapped regions,. . . ), and the second control the prescribed coverage but have too large volume to be useful (e.g. concentration methods based on Hoeffding-Bernstein inequalities, Clopper-Pearson type methods, . . . ). Note that the discrete nature of the multinomial distribution produces a staircase effect which makes it difficult to construct non-asymptotic regions with coverage equal exactly to 1α. For a discussion of such aspects, we refer for instance to Agresti et al. [START_REF] Agresti | Approximate is better than "exact" for interval estimation of binomial proportions[END_REF][START_REF] Agresti | Simple and effective confidence intervals for proportions and differences of proportions result from adding two successes and two failures[END_REF][START_REF] Agresti | Dealing with discreteness: making 'exact' confidence intervals for proportions, differences of proportions, and odds ratios more exact[END_REF]. In general, it seems reasonable to expect a coverage of at least 1α, without being too conservative, while maintaining the volume as small as possible. Here the term conservative means that the coverage is greater than 1α. Even when d = 2 and n is large but finite, the confidence regions built from asymptotic approaches based on the Central Limit Theorem have a poor and uncontrolled coverage. It is also the case for bootstrapped versions which only improve the coverage probability asymptotically (see [START_REF] Zheng | Bootstrapping binomial confidence intervals[END_REF][START_REF] Morales | Bootstrap confidence regions in multinomial sampling[END_REF][START_REF] Glaz | Simultaneous confidence intervals for multinomial proportions[END_REF][START_REF] Hall | The bootstrap and Edgeworth expansion[END_REF][START_REF] García-Pérez | Exact finite-sample significance and confidence regions for goodness-of-fit statistics in one-way multinomials[END_REF][START_REF] Wilson | Probable inference, the law of succession, and statistical inference[END_REF]). For the binomial case d = 2, one of the best known method is due to Blyth & Still [START_REF] Blyth | Binomial confidence intervals[END_REF] and combines various approaches. To our knowledge, the available methods for the general multinomial case d > 2 are unfortunately asymptotic or Bayesian, which explains their poor performances in terms of coverage or volume when n is small (see [START_REF] Ullah | Handbook of applied econometrics and statistical inference[END_REF][START_REF] Sison | Simultaneous confidence intervals and sample size determination for multinomial proportions[END_REF][START_REF] Albert | Pseudo-Bayes estimation of multinomial proportions[END_REF][START_REF] Glaz | Simultaneous confidence intervals for multinomial proportions[END_REF][START_REF] Morales | Bootstrap confidence regions in multinomial sampling[END_REF]).

The coverage of our region (3) is strictly controlled since it fulfills (2) whatever the values of d and n. However, this says nothing about the actual coverage and the actual mean volume. The comparisons presented in Section 3 suggest that our region for d = 2 is comparable to the Blyth & Still region in terms of actual coverage and actual mean volume. For d = 3, the Blyth & Still method is no longer available, and our region seems to have an actual coverage close to the prescribed level while maintaining a volume comparable to the asymptotic region constructed with the score method based on the Central Limit Theorem. Section 3 provides two concrete examples, one for d = 3 and another one for d = 4 in relation to the χ 2 -test. The article ends with a final discussion.

Covering collections

The aim of this section is to introduce the notion of covering collection, which allows confidence regions to be built in a general abstract space. Let us consider a random variable X : (Ω, A) → (E, B E ) having a distribution µ θ * where θ * ∈ Θ. For some α ∈ (0, 1), we would like to construct a confidence region R α (X) for θ * with a coverage of at least (1α), from a single realization of X. In other words,

P (θ * ∈ R α (X)) ≥ 1 -α. ( 4
) Definition 2.1 (Covering collection). A covering collection of E is a collection of measurable events (A k ) k∈K ⊂ B E such that
• K is totally ordered and has a minimal element and a maximal element;

• if k ≤ k ′ then A k ⊂ A k ′ with equality if and only if k = k ′ ; • A min(K) = ∅ and A max(K) = E.
For instance, for E = {0, 1, . . . , n}, the sequence of sets ∅, {σ(0)}, {σ(0), σ(1)}, . . . , {σ(0), σ(1), . . . , σ(n)} = E is a covering collection of E for any permutation σ of E. For E = R, the collection (A t ) t∈R where R = R∪{-∞, +∞} defined by A -∞ = ∅, A t = (-∞, t] for every t ∈ R, and A +∞ = R is a covering collection of E. Many other choices are possible, like A t = [-t, +t] or A t = [t, +∞). We can recognize the usual shapes of the confidence regions used in univariate Statistics.

Theorem 2.2 (Confidence region associated with a covering collection). Let (A k ) k∈K be a covering collection of E, and k X be the smallest k ∈ K such that X ∈ A k .

For every α ∈ (0, 1), the region R α (X) defined below satisfies to (4).

R α (X) = {θ ∈ Θ such that µ θ (A k X ) ≥ α} . (5)
Proof. For every θ ∈ Θ, let k α (θ) be the largest k ∈ K such that µ θ (A k ) < α. With this definition of k α (•), we then have

x ∈ A kα(θ) if and only if µ θ (A kx ) < α.
Thus we have

P (θ * ∈ R α (X)) = P (µ θ * (A k X ) ≥ α) = P X / ∈ A kα(θ * ) = 1 -µ θ * A kα(θ * ) ≥ 1 -α.
These confidence regions are highly dependent on the chosen covering collection (A k ) k∈K . Each choice of covering collection gives a particular region R α (X). Note that a small value of k X gives a small set A k X and thus leads to a confidence region with a small volume. For instance, assume that we have two realizations x 1 and x 2 of X with k x 1 < k x 2 . For a given sequence (A k ) k∈K , we have A kx 1 ⊂ A kx 2 and thus R α (x 1 ) ⊂ R α (x 2 ). It is tempting to choose the covering collection (A k ) k∈K in such a way that k X is as small as possible. Unfortunately, with such a choice, the covering collection (A k ) k∈K could be random and the coverage of the associated region could be less than the prescribed level 1α.

Note that the set A k X can be empty, which means that a confidence region cannot be built with the sequence (A k ) k∈K . In contrast, the case where A k X = E leads to the trivial region R α (X) = Θ. In the case where A k X = {X}, we have µ θ (A k X ) = µ θ ({X}), which is the likelihood of X at point θ, and the region R α (X) corresponds to the complement of a level-set of the likelihood.

The following symmetrization lemma allows (for instance) the construction of twosided confidence intervals from one-sided confidence intervals. We use it in Section 2.2 to interpret the Clopper-Pearson confidence interval as a special case of the covering collection method.

Lemma 2.3 (Symmetrization). Consider a covering collection

(A k ) 0≤k≤κ of E. For every 0 ≤ k ≤ κ let us define A ′ k = E \ A κ-k .
For any θ ∈ Θ, any X ∼ µ θ , and any α ∈ (0, 1), we construct

R1 2 α = θ ∈ Θ; µ θ (A k X ) > 1 2 α and R ′ 1 2 α = θ ∈ Θ; µ θ (A ′ k ′ X ) > 1 2 α
where

k ′ X is built from (A ′ k ) 0≤k≤κ as k X from (A k ) 0≤k≤κ and A ′ k ′ X = E \ A k X -1 . Then R1 2 α ∩ R ′ 1 2 α
is a confidence region with coverage greater than or equal to 1α.

Proof. We have µ

θ (A k X ) + µ θ (A ′ k X ) = 1 + µ θ ({X}) ≥ 1 and thus R1 2 α and R ′ 1 2
α have disjoint complements. The conclusion follows now from a general fact: if R 1 and R 2 are two confidence regions with a coverage of at least 1

-1 2 α such that R 1 ∪ R 2 = E (equivalently R c 1 = Θ \ R 1 and R c 2 = Θ \ R 2 are disjoint), then R c 1 and R c 2 are disjoint and thus R 1 ∩ R 2 = (R c 1 ∪ R c 2 )
c is a confidence region with a coverage of at least 1α.

Remark 2.4 (Discrete case and staircase effect). Let (A k ) k∈K be a covering collection of a finite set E. Due to staircase effects, the coverage of the confidence regions constructed from this covering collection cannot take arbitrary values in (0, 1). These staircase effects can be reduced by using a fully granular collection for which Card(K) = Card(E). The term fully granular means that the elements of the collection are obtained by adding the points of E one by one. It is impossible to remove completely the staircase effects when E is discrete, while maintaining a prescribed lower bound on the coverage.

Remark 2.5 (Reverse regions).

For the region R α (X) = {θ ∈ Θ; µ θ (A k X ) ≤ 1 -α} we have

P(R α ) = P(µ θ (A k X ) ≤ 1 -α) = P(X ∈ A k 1-α ) = µ θ (A k 1-α ) ≤ 1 -α.
Remark 2.6 (Link with tests). Let us recall briefly the correspondence between confidence regions and statistical tests (we refer to [9, Section 48] for further details). Consider a parametric model (µ θ ) θ∈Θ with data space X . For any fixed θ 0 ∈ Θ, the test problem of

H 0 : θ = θ 0 versus H 1 : θ = θ 0 with level α ∈ (0, 1) corresponds to the construction of an acceptance region C α (θ 0 ) ⊂ X such that µ θ 0 (C α (θ 0 )) ≥ 1 -α.
The construction of a confidence region for θ 0 can be done by inversion ( i.e. by collecting the values of θ 0 for which H 0 is accepted). Namely, for every x ∈ X , one can define the region R α (x) ⊂ Θ by

R α (x) = {θ ∈ Θ such that x ∈ C α (θ)}. Now if X ∼ µ θ 0 then P(θ 0 ∈ R α (X)) = P(X ∈ C α (θ 0 )) = µ θ 0 (C α (θ 0 )) ≥ 1 -α.
This shows that for any fixed θ 0 ∈ Θ, the set R α (X) ⊂ Θ is a confidence region for θ 0 when X ∼ µ θ 0 . Conversely, if for every θ 0 ∈ Θ and every x ∈ X one has a region R α (x) ⊂ Θ such that P(θ 0 ∈ R α (X)) ≥ 1α when X ∼ µ θ 0 , then one can construct immediately a test for H 0 : θ = θ 0 versus H 1 : θ = θ 0 with acceptance region

C α (θ 0 ) = {x ∈ X such that θ 0 ∈ R α (x)}.
Note that this correspondence between confidence regions and statistical tests can be extended to the composite case H 0 : θ ∈ Θ 0 versus H 1 : θ ∈ Θ 0 where Θ 0 ⊂ Θ.

The level-sets regions

In this section, we show that the "level-sets" confidence region (3) is a special case of the covering collection method. It is easier to consider here a decreasing covering collection (the corresponding version of Theorem 2.2 is immediate). Let us consider a random variable X : (Ω, A) → (E, B E ) with law µ θ * where θ * ∈ Θ. For every u ≥ 0 and θ ∈ Θ, let us define

A(θ, u) = {x ∈ E such that µ θ (x) ≥ u}.
For every θ ∈ Θ, the collection (A(θ, u)) u≥0 is decreasing with A(θ, 0) = E and there exists u max that can be equal to +∞ such that A(θ, u max ) = ∅. Also, (A(θ,

u max -u)) u∈[0,umax] is a covering collection of E. Next, define u(θ, α) = sup {u ∈ [0, u max ] such that µ θ (A(θ, u)) ≥ 1 -α} and K(θ, α) = A(θ, u(θ, α)).
We would like to construct a confidence region for θ * from the observation of

X ∼ µ θ * . If R α (X) = {θ ∈ Θ such that X ∈ K(θ, α)} (6)
then

P (θ * ∈ R α (X)) = P (X ∈ K(θ * , α)) = µ θ * (K(θ * , α)) ≥ 1 -α.
This shows that R α (X) is a confidence region for θ * with a coverage of at least 1α.

Let clarify the expression of the confidence region for the general multinomial case where

X ∼ M d (n, p) with p ∈ Λ d and d ≥ 2.
Here the value of p used for the observed data X plays the role of θ * . We have Θ = Λ d , E = E d as described by (1), µ θ = M d (n, θ), and u max = 1. For every α ∈ (0, 1), the confidence region given by the level-sets method is expressed as in [START_REF] Agresti | Approximate is better than "exact" for interval estimation of binomial proportions[END_REF] given in the introduction.

Optimality

Let us focus on the case where E is a finite set. The confidence region constructed above is not optimal among the 1α conservative regions and thus could be improved by a more detailed analysis. Let us first note that by its very construction, for each θ ∈ Θ, K(θ, α) is minimal with respect to its cardinality that is, a set B(θ, α) does not exist so that µ θ (B(θ, α)) ≥ 1α and card(B(θ, α)) < card(K(θ, α)). However, in some circumstances, sets L(θ, α) may exist with the same cardinality as K(θ, α) so that

µ θ (K(θ, α)) ≥ µ θ (L(θ, α)) ≥ 1 -α.
The following theorem gives a condition that allows conservative sets to be built but with a coverage closer to 1α than the coverage of R α (X). For all α ∈ [0, 1] and θ ∈ Θ, let us denote γ(θ, α) = 1µ θ (K(θ, α)) and let us note that γ(θ, α) ≤ α.

Theorem 2.7. If for each θ ∈ Θ there exists two subsets V (θ, α) ⊂ K(θ, α) and W (θ, α) ⊂ E\K(θ, α) with the same cardinality so that

α -γ(θ, α) ≥ µ θ (V (θ, α)) -µ θ (W (θ, α)) > 0,
then there exists a set T α (X) = R α (X) so that

1 -α ≤ P (θ * ∈ T α (X)) < P (θ * ∈ R α (X)) .
Proof. Let us consider the set L (θ, α) = K (θ, α) \V (θ, α) W (θ, α) and note that thanks to the conditions imposed for the sets V and W we have for all θ ∈ Θ,

1 -α ≤ µ θ (L(θ, α)) < µ θ (K(θ, α)) .
Now, with T α (X) = {θ ∈ Θ; X ∈ L(θ, α)} we have

P (θ * ∈ T α (X)) = P (X ∈ L(θ * , α)) = P (X ∈ K (θ * , α) \ V (θ * , α) W (θ * , α)) = 1 -γ (θ * , α) -µ θ * (V (θ * , α)) + µ θ * (W (θ * , α)) ≤ 1 -γ (θ * , α) .
On the other hand, we have already seen that for all θ ∈ Θ,

1 -α ≤ µ θ (L(θ, α)) .
This last inequality holds true when θ = θ * and thus

1 -α ≤ µ θ * (L(θ * , α)) = P (θ * ∈ T α (X)) .
This theorem can be used to build less conservative confidence sets than R α (X). A convenient way to proceed is to take V (θ, α) = {y} where y is such that

µ θ (y) = min z∈K(θ,α) µ θ (z)
and to iteratively try several sets W k as follows. Set W 0 (θ, α) = ∅, and at iteration

k ≥ 1, set W k (θ, α) = {w k } and L k (θ, α) = K (θ, α) \ V (θ, α) W k (θ, α) where w k = arg max z∈L k-1 (θ,α) µ θ (z).
This process is iterated until the set

L k (θ, α) is such that µ θ L k (θ, α) -(1 -α) is non-negative and minimum.
Since for θ ∈ Θ there may exist x = y with µ θ (x) = µ θ (y), there also may exist several sets (L i (θ, α)) i which have the same mass µ θ (L i (θ, α)) = 1δ(θ, α). Several confidence sets with the same coverage can thus be derived using these sets. A simple way to choose between these concurrent confidence sets is to adopt the one that optimizes a criterion such as having a minimum volume (for the Lebesgue measure).

The Clopper-Pearson regions

Consider the binomial case d = 2 for which p = (p 1 , 1p 1 ). The well known Clopper-Pearson interval for p 1 relies on the exact distribution of X 1 in the binomial case [START_REF] Clopper | The use of confidence or fiducial limits illustrated in the case of the binomial[END_REF][START_REF] Julious | Two-sided confidence intervals for the single proportion: comparison of seven methods, letter to the editor[END_REF][START_REF] Chen | On the binomial confidence interval and probabilistic robust control[END_REF]. It was considered for a long time as outstanding. This interval [L, U] is given by

L = inf θ ∈ [0, 1] such that n i=x 1 n i θ i (1 -θ) n-i ≥ 1 2 α U = sup θ ∈ [0, 1] such that x 1 i=0 n i θ i (1 -θ) n-i ≥ 1 2 α . (7)
It has been shown that the Clopper-Pearson interval is often conservative. Also, some continuity corrections have been proposed, and give the so called "mid-p interval", see [START_REF] Berry | Mid-p confidence intervals: a brief review[END_REF] for a review. This trick reduces the staircase effect but the coverage probability can be less than 1α. The Beta-Binomial correspondence (see Lemma 2.8 below) shows that the left and right limits L and R of the Clopper-Pearson confidence interval (7) are the 1 2 α and (1 -1 2 α) quantiles of the Beta distribution Beta (X 1 ; n -X 1 + 1). Lemma 2.8 (Beta-Binomial correspondence). If X ∼ Binom(n, p 1 ) with p 1 ∈ [0, 1] and 0 ≤ k ≤ n and B ∼ Beta(k, nk + 1) then the following identity holds true.

P(X ≥ k) = P(B ≤ p 1 ). ( 8
)
Proof. We briefly recall here the classical proof (see [9, page 68]). Let U 1 , . . . , U n be iid uniform random variables on [0, 1] and

U (1) ≤ • • • ≤ U (n) be the reordered sequence. If we define V p 1 = n i=1 I {U i ≤p 1 } then V p 1 ∼ Binom(n, p 1 ) and U (k) ∼ Beta(k, n -k + 1) and for every 1 ≤ k ≤ n, V p 1 ≥ k if and only if U (k) ≤ p 1 .
The confidence interval obtained by the level-sets method does not coincide with the classical Clopper-Pearson confidence interval. Let us show why the Clopper-Pearson confidence interval can be considered as a special case of the method based on covering collections. Recall that we are in the case where d = 2 and X 1 ∼ Binom(n, p 1 ) for some unknown p 1 ∈ [0, 1]. This can also be written (X 1 , n -X 1 ) ∼ M 2 (n, (p 1 , 1p 1 )). The unidimensional nature of E = {0, . . . , n} suggests the following two covering collections (A 1 k ) k∈E and (A 2 k ) k∈E defined by A 1 0 = ∅ and A 2 0 = ∅, and for every 0 ≤ k ≤ n, A 1 k+1 = {0, . . . , k} and A 2 k+1 = {nk, . . . , n}. Here K = E for both the top-to-bottom and bottom-to-top sequences. The bottom-totop sequence (A 1 k ) k∈E leads to a (1α) one-sided confidence interval for p 1 given by

R 1 α (X 1 ) = θ ∈ [0, 1] such that X 1 i=0 n i θ i (1 -θ) n-i ≥ α = [0, U α (X 1 )] (9)
where

U α (x) = sup θ ∈ [0, 1] such that x i=0 n i θ i (1 -θ) n-i ≥ α .
On the other hand, the top-to-bottom covering collection (A 2 k ) k∈E leads to a (1α) confidence interval of p 1 given by

R 2 α (X 1 ) = θ ∈ [0, 1] such that n i=X 1 n i θ i (1 -θ) n-i ≥ α = [L α (X 1 ); 1] (10)
Equivariance imposes a strong constraint on the covering collection. A large set A k X gives a large confidence region. Since confidence regions with small volume are desirable, it is interesting, when E is discrete, to consider a covering collection (A k ) k∈K which grows by adding the points of E one after the other. Unfortunately, this method of construction is not compatible with equivariance: the A k cannot be invariant by permutations of coordinates. A weaker condition consists of the existence of a subsequence (A k l ) l that is invariant by permutation of coordinates. An example of such a sequence for d = 3 is given in Figure 1.

Recall that when d = 2, the Beta-Binomial correspondence stated in Lemma 2.8 provides a clear link between the quantiles of the Beta distribution and the Clopper-Pearson confidence interval. In fact, this can be seen as a special case of the Dirichlet-Multinomial correspondence valid for any d ≥ 3 as stated in the following lemma. This makes a link between Clopper-Pearson regions and Bayesian regions constructed with a Jeffrey prior (see for instance [START_REF] Robert | The Bayesian choice[END_REF]). However, the notion of coverage that we use in the present article is purely frequentist and does not fit with the Bayesian paradigm without serious distortions.

Lemma 2.11 (Dirichlet-Multinomial correspondence). Let p ∈ Λ d and k 0 , k 1 , . . . , k d be such that k 0 = 0 ≤ k 1 ≤ • • • ≤ k d-1 ≤ n ≤ k d = n + 1. If X ∼ M d (n, p) and D ∼ Dirichlet d (k 1 -k 0 , k 2 -k 1 , . . . , k d -k d-1 )
then the following identity holds true:

P(X 1 ≥ k 1 , X 1 + X 2 ≥ k 2 , . . . , X 1 + • • • + X d-1 ≥ k d-1 ) = P(D 1 ≤ p 1 , D 1 + D 2 ≤ p 2 , . . . , D 1 + • • • + D d-1 ≤ p d-1 ). ( 12
)
Proof. The proof is a direct extension of the Beta-Binomial case given by Lemma 2.

I {U i ∈Ir} = Card{1 ≤ i ≤ n such that U i ∈ I r }.
We have

V p = (V p,1 , . . . , V p,r ) ∼ M d (n, p). Now, for every 0 ≤ k 1 ≤ • • • ≤ k d-1 ≤ n, V p,1 ≥ k 1 , . . . , V p,1 + • • • + V p,d-1 ≥ k d-1 iff U (k 1 ) ≤ p 1 , . . . , U (k d-1 ) ≤ p 1 + • • • + p d-1 .
But by using the notation U (0) = 0 and U (n+1) = 1, we have

(U (1) -U (0) , . . . , U (n+1) -U (n) ) ∼ Dirichlet n+1 (1, . . . , 1).
and therefore, by the stability of Dirichlet laws by sum of blocks, with k 0 = 0 and

k d = n + 1, (U (k 1 ) -U (k 0 ) , . . . , U (k d ) -U (k d-1 ) ) ∼ Dirichlet d (k 1 , k 2 -k 1 , . . . , k d -k d-1
).

Comparisons and examples

Recall that for every fixed d ≥ 2, n ≥ 0, and p ∈ Λ d , a confidence region obtained from X ∼ M(n, p) provides a single coverage probability and a distribution of volumes. In this section, we use coverage probabilities and mean volumes to compare the performance of our level-set method with other methods, in the case where d ∈ {2, 3} and n ∈ {5, 10, 20, 30}. We also give two concrete examples, one for d = 3 and another one for d = 4 in relation to the χ 2 -test. It turns out that the regions obtained by the Clopper-Pearson method and its multinomial extension have non-competitive volumes so we decided to ignore them in the comparisons.

Performances in the binomial case (d = 2)

In the binomial case d = 2, a confidence region for p = (p 1 , 1 -p 1 ) is actually a confidence interval for p 1 . It is well known that the Wald interval constructed from the Central Limit Theorem has poor coverage even when n is large but finite [START_REF] Brown | Interval estimation for a binomial proportion[END_REF]. It is also widely accepted that the Wilson score interval [START_REF] Wilson | Probable inference, the law of succession, and statistical inference[END_REF][START_REF] Brown | Interval estimation for a binomial proportion[END_REF] or the Blyth-Still interval [START_REF] Blyth | Approximate binomial confidence limits[END_REF] should be preferred to the Wald interval. We therefore compared the performances of the 95%-intervals provided by the level-sets method, the score method, and the Blyth-Still method. We computed the coverages and the mean widths of the intervals obtained with each method for n ∈ {5, 10, 20, 30} and for all p 1 ∈ [0; 0.5]. The results are represented in figures 2 and 3 respectively. We can see that for some values of p 1 , the coverage of the score method is smaller than the prescribed level of 0.95, whereas the coverage of the Blyth-Still interval and the level-set interval are always greater than or equal to this prescribed level 0.95. The coverages obtained with the level-set method are always closer to the prescribed level except for n = 20, p 1 ∈ [0.45, 0.48] and n = 30, p 1 ∈ [0.38, 0.42]. The differences between the coverages of these three methods decrease with n.

Figures 2 and3 show that the score method provides intervals with excellent mean width but fails to control the coverage. The level-set method gives intervals that have a slightly narrower mean width than the one obtained with the Blyth-Still method. This suggests that the level-set method provides an excellent alternative to the Blyth-Still method. Moreover, and in contrast to the Blyth-Still method, the level-set method can still be used when d > 2.

Performances in the trinomial case (d = 3)

To our knowledge, the Blyth-Still method has no counterpart for d > 2. In addition, the regions obtained by the extended Clopper-Pearson method have non-competitive volumes. We therefore decided to compare the level-set method with the natural multidimensional extension of the Wilson score method. We computed for d = 3 the coverage probabilities and the mean volumes of the 95%-regions obtained with both methods, for n ∈ {5, 10, 20}. Note that for the score method, only the trace over Λ 3 of the regions is used to compute the volume. The graphics in Figure 4 show the coverage of both methods as well as the difference between their mean volumes. Whatever the sample size, the coverage of the level-set regions is very close to 1α = 0.95. In contrast, the coverages of the score regions can be much lower than 0.95. Surprisingly and in contrast with the binomial case (d = 2), the level-set method here provides confidence regions with mean volumes that (for n = 5) are comparable to or smaller than their score's counterparts! We believe that this because we measure the performance by the mean volume. The level-set method appears thus to be a reasonable way to build small confidence sets.

Concrete example of the trinomial case (d = 3)

The present example concerns antibiotics efficacy. A traditional way to evaluate whether or not an antibiotic can be used for a specific pathogen is to perform a "susceptibility testing". In such an experiment, different isolates of a given pathogen are classified as "Sensible", "Intermediate" or "Resistant" according to the antibiotics ability to stop their growth. Here, ten different isolates of Escherichia coli were tested with ampicillin. The following results were obtained : 8 isolates were Sensible, 2 Intermediate and 0 were Resistant. The count x = (8, 2, 0) can be seen as the realization of X ∼ M(10, p) where p = (p 1 , p 2 , p 3 ) denotes the probability of a given isolate belonging to each of the different classes. We calculated a 95%-confidence region of p using the level-set method (Figure 5). This region suggests that even if none of the 10 tested isolates was observed to be resistant, up to 30% of resistant and 20% of intermediate isolates will be still possible. This confidence region does not contain the situation where all the isolates are sensible and it is thus unlikely that this antibiotic works all the time when it meets this pathogen.

Concrete example of the quadrinomial case (d = 4)

The present example is simply a χ 2 -test for independence. It deals with the difference in behavior of male and female veterinary students with respect to smoking habits. The following result was observed in a group of 12 veterinary students in Toulouse: Smokers Non-smokers Female 3 8 Male 10 5

The χ 2 -test rejects independence with a P -value 0.047 and suggests that more males than females smoke. This P -value is close to the critical threshold of 0.05 and was obtained with a small sample size. Therefore, one can question whether this result can be trusted. A possible solution is to build a confidence region. The table above can be seen as the realization x = (3, 8, 10, 5) of a multinomial random variable X ∼ M(26, p) with p = (p 1 , p 2 , p 3 , p 4 ). If the smoking habit and the gender are independent then p belongs to

H 0 = q ∈ Λ 4 such that q = (uv, (1 -u)v, u(1 -v), (1 -u)(1 -v)) and (u, v) ∈ [0, 1] 2 .
Since p 4 = 1p 1p 2p 3 , one can draw a graphic with only p 1 , p 2 , p 3 . Figure 6 shows (in green) the 95% confidence region for p built with the level-set method. The surface corresponds to the null hypothesis H 0 . The red area is the acceptance region of the χ 2test. It turns out that p = (3/26, 8/26, 10/26) does not belong to the acceptance region of the χ 2 -test. However, the 95%-region for p obtained with the level-set method cuts H 0 . Therefore, according to Remark 2.6 and in contrast to the result given by the χ 2 -test, the independence hypothesis is not rejected. The 95% level-set confidence region provides the following 95% confidence interval for the odd-ratio: [0.024; 1.712]. On the other hand, the inversion of Fisher's exact test gives the 93.7% interval [0.187; 2.625]. This suggests that the level-set approach is less conservative, probably due to the fact that Fisher's exact conditions on row and column totals increases the discreteness of the problem.

Final discussion

The general concept of "covering collection" allows the construction of confidence regions with controlled coverage, including the classical Clopper-Pearson interval for the binomial and its multinomial extensions. The covering collection construction involves an arbitrary growing collection of sets in the data space. Our "level-set" confidence regions are obtained by using a special collection based on level-sets of the data distribution. The level-set regions for the multinomial parameter can be easily computed for any d and n. It turns out that they have excellent coverage probabilities and mean volumes for d ∈ {2, 3} and n ≤ 30. They are in particular competitive with the famous Blyth-Still intervals for d = 2. Also, we recommend the level-set method, even if it can be computationally expensive when d is large. The behavior of these confidence regions when the ratio d/n tends to infinity is a very interesting open problem. In this extreme case, the observation X is sparse and belongs to the boundary of the observation simplex E ∞ . Note that the critical n for which X ∼ M(n, p) belongs to the interior of E d corresponds to the classical "coupon collector problem" [START_REF] Feller | An introduction to probability theory and its applications[END_REF][START_REF] Motwani | Randomized algorithms[END_REF][START_REF] Holst | On birthday, collectors', occupancy and other classical urn problems[END_REF]. Another interesting open problem is the optimality of the level-set regions related to the control of P(p ′ ∈ R α (X)) with X ∼ M(n, p) and p = p ′ . It might be also interesting to extend the level-set method to more complex situations such as hierarchical log-linear models for instance. 3). In barycentric coordinates, the 95%region for p is constructed from the observation x = (0, 2, 8) of M 3 (10, p). Note that the Wald method cannot be used here since the observation belongs to the boundary of the observation simplex E 3 . In this example, the score and the level-set methods give approximately the same region. The null hypothesis H 0 of the χ 2 -test is represented by the surface. The set in red is the acceptance region of the χ 2 -test. The region in green is the 95%-region for p built with the level-set method. It turns out that p = (3/26, 8/26, 10/26)does not belong to the acceptance region of the χ 2 -test while it belongs to the 95%-region for p built with the level-set method. Additionally, since this confidence region cuts H 0 , the corresponding test does not reject H 0 , in contrast to the χ 2 -test.

  8. Let I 1 , . . . , I d be the sequence of adjacent sub-intervals of [0, 1] of respective lengths p 1 , . . . , p d , U 1 , . . . , U n be iid uniform random variables on [0, 1] and U (1) ≤ • • • ≤ U (n) be the reordered sequence. For any 1 ≤ r ≤ d, let us define V p,r = n i=1

Figure 1 :

 1 Figure 1: The construction of A k when d = 3, with A 0 = ∅ and A 1 = {(n, 0, 0)}. The point in A 1 is at the beginning of the starting arrow represented as a dotted line. Each time the arrow meets a point in the simplex, this point is added to A k to give A k+1 . The set obtained with the three first arrows is invariant by permutation of coordinates.

Figure 2 :

 2 Figure 2: Binomial case d = 2.The curves are the mean width of the 95%-intervals obtained with the Blyth-Still method (thick line), the level-set method (thin line) and the score method (dotted line) for p 1 ∈ [0, 0.5]. The Blyth-Still method gives intervals with higher mean width irrespective of p 1 . The score method always gives intervals with smaller width. Note that the score method fails to control the coverage probability. As n increases, the differences between the mean widths of the respective intervals decrease.

Figure 3 :Figure 5 :

 35 Figure3: Binomial case d = 2. These curves are the coverage of the 95%-intervals obtained with the Blyth-Still method (thick line), the level-set method (thin line) and the score method (dotted line) for p 1 ∈ [0, 0.5]. The score method fails to control the coverage. The level-set method seems (nearly) uniformly better than the Blyth-Still method: its coverages are closer to 0.95. When n increases, the differences between these three methods decrease.

Figure 6 :

 6 Figure 6: Quadrinomial case d = 4 (example 3.4). The axes correspond to p 1 , p 2 , and p 3 .The null hypothesis H 0 of the χ 2 -test is represented by the surface. The set in red is the acceptance region of the χ 2 -test. The region in green is the 95%-region for p built with the level-set method. It turns out that p = (3/26, 8/26, 10/26)does not belong to the acceptance region of the χ 2 -test while it belongs to the 95%-region for p built with the level-set method. Additionally, since this confidence region cuts H 0 , the corresponding test does not reject H 0 , in contrast to the χ 2 -test.

Acknowledgements

The present version of this article has greatly benefited from the comments and criticism of an Associate Editor and three anonymous referees.

where

By virtue of Lemma 2.3, we can combine the one-sided confidence intervals [START_REF] Borovkov | Mathematical statistics[END_REF] and [START_REF] Brown | Interval estimation for a binomial proportion[END_REF] in order to obtain a two-sided (1α) confidence interval of p 1 , which is the two-sided interval R 1

]. We recognize the Clopper-Pearson interval [START_REF] Blyth | Correction: "Approximate binomial confidence limits[END_REF]. The discrete nature of E precludes the construction of a confidence interval of p 1 with coverage exactly equal to 1α. Actually, the Clopper-Pearson interval is not exactly symmetric and there is no guaranty that P p < L1 2 α (X 1 ) = P p > U1 2 α (X 1 ) . Our construction via a covering collection immediately provides an extension of the Clopper-Pearson interval in the general multinomial case where X ∼ M d (n, p) with p ∈ Λ d and d > 2. This construction consists of labeling the elements of

) and constructing the covering collection (A k ) k∈K which grows by adding the points one after the other. The choice of the total order on E d is arbitrary when d > 2. Some additional constraints can help to reduce this choice. As advocated by Casella [START_REF] Casella | Refining binomial confidence intervals[END_REF] for the binomial distribution, the proposed confidence region R α (X) should be equivariant, that is not sensitive to the order chosen to label the d categories of the multinomial distribution. Definition 2.9 (Equivariance). A confidence region R α (X) is equivariant when

for every permutation σ of {1, . . . , d}. In other words, if and only if

The following lemma gives a criterion of equivariance for covering collections.

Theorem 2.10 (Equivariance criterion for covering collections). The confidence region R α (X) constructed from a covering collection (A k ) k∈K is equivariant if and only if A k is invariant by permutation of coordinates for every k ∈ K.

Proof. Let σ be a permutation of {1, . . . , d}, i = (i 1 , . . . , i d ) ∈ E, and for every θ ∈ Θ,

By invariance of A k by permutation, we have

If A k is invariant by permutations, then for every i ∈ A k , we have The columns give the coverages of the level-set method, the coverages given by the score method and the difference of mean volumes. The three rows correspond to n ∈ {5, 10, 20}. For the coverages graphs (first two columns), a clear color means that the coverage is close to 0.95 whereas a dark blue color means that the coverage is smaller than 0.85. For the volumes graphs (third column), a white color means that the difference of mean volumes is small whereas the blue, pink and yellow colors are used when the mean volume of the regions obtained with the level-set method are smaller than their counterpart obtained with the score method.


Mickaël Trezzy
email: mickael.trezzy@irit.fr

Student Research Abstract: A Toolchain for Model-Based Development of a Robot Application

Keywords: Software and its engineering → Formal software verification; MDE, CBSE, robotics, early validation, ROS

This student research abstract reports on an ongoing work performed in the context of a PhD thesis on designing a method and putting together a toolchain that allows a model-based development of a robot application based on the robotics middleware ROS. The goal of this integration is to improve the efficiency of the robot application design and development using models and components with a set of ROS concepts ready to use. In addition, it improves the pertinence of the verification and validation phase by simulating the model and using formal verification methods such as model checking. Our approach is applied to the context of ROS, and consists to increase the abstraction of the systems through models, and leverages existing modeling, simulation, and verification environments. We use as a case study simple examples of ROS on which we apply UML, profile UML, and DSL, in order to conduct a comparative study of the abstract approaches applied on robotic systems.

INTRODUCTION

This paper reports on an ongoing work on building a modeling environment for designing ROS applications. To do so, a comparative study is ongoing to determine the advantages and disadvantages to use either a profile UML, or a DSL, to create this modeling environment. The environment is used to design ROS applications, allowing early-verification and validation of the applications using model-checking, model simulation and code generation.

Previous work [START_REF] Hili | Modelling and code generation for real-time embedded systems with UML-RT and papyrus-RT[END_REF] focused on using a Model-Driven Engineering (MDE) environment for programming the behavior of a robot. The results obtained showed the relevance of model-driven techniques in this context and highlighted some of the difficulties specific to the robotics field such as mentioned in [START_REF] Brugali | Software stability in the robotics domain: issues and challenges[END_REF]: the variety of the hardware components that need to be used even in a simple project; the volume of hardware specific information that needs to be manipulated, leading also to difficulties in combining various software and hardware components; the difficulty of performing meaningful validations.

The main objective of the current work is to address some of these difficulties and to empower early-verification and validation (V&V). Formal verification is the act of proving or disproving the correctness of intended algorithms underlying a system with respect to a certain formal specification or property, using formal methods of mathematics. 1 Validation corresponds to the assurance that a product, service, or system meets the needs of the customer and other identified stakeholders. 2 The V&V is empowered through the modeling of a robot application, in particular developed under ROS, by offering the possibility to verify the model through model checking techniques and to validate it through model simulation. The benefits of the simulation are to overlook physical components of the robots that are not always available and to abstract possible complex heterogeneous components of a robot. After the model is verified, the next objective of the current work is to generate code from the model to obtain a ROS application. A validation phase is possible at this step to capture information on the interaction between the robot model and an outdoor environment in which the robot is aimed to be executed, thus simulating some of the events that would occur in reality (obstacle, traffic light detection, power loss etc). It offers the possibility to consider a variety of complex outdoor environments without having to actually place the robot into them.

CONTEXT AND VISION

Robotic systems are complex, they integrate many motion degrees of freedom, embed multiple sensors over the robot and the environment, or do complex tasks such as planning and reasoning. The need of real-time constraints and the fact that they are used in a wide variety of applications with some potential safety hazards are other factors increasing the overall complexity.

It is difficult and time-consuming to develop robotic software because of the complexity of the robotic systems. Services are sophisticated and often tied to specific robotic hardware and communication infrastructures making them not reusable. In addition, robot vendors often provide the robotic systems with their own programming language, making reuse harder and making it necessary to have different experts depending on the type of the robot. The reusability would possibly limit the quantity of code to write and would reduce the number of test to do on the already used components. That is why modeling tools can allow model-based design and development of robotic applications on a higher abstraction level, granting the composition of reusable components. In addition, model-based development permits early V&V by simulating the models or by using formal methods such as model-checking. Another potential benefit from using a model based approach is the code generation that would allow to avoid coding mistakes and it would not be necessary to be a robotics engineer to be able to develop a robot.

It exists a tool allowing the reusability and the sharing of components on Internet: ROS. ROS stands for Robot Operating System [START_REF] Quigley | ROS: an open-source Robot Operating System[END_REF], it is a flexible framework for writing robot software. It is composed of many tools and libraries, and provides conventions that aim to simplify the creation of complex and robust robot behavior across a wide variety of robotic platforms. It is used around the world by many companies in various kinds of robotic platforms. ROS general concepts are nodes and topics. Nodes are processes performing computation. They are usually used as the different components of the robot system, each node having its own goal. Topics are names for a stream of messages with a defined type. They implement a publish/subscribe communication mechanism that nodes can use to transmit data over topics. Nodes can subscribe to a topic from which they want to receive messages as well.

The goal of this work is to unify ROS projects into the model level in order to verify properties of the project with model-checking, to validate it with simulation of the models, to simulate the execution of the generated code into a simulated world, and to keep the models close enough to the domain of ROS to hold an accessibility for the field engineers. With a dedicated modeling language, the complexity of ROS would be hidden by the code generator. The models and parts of robotic systems would be reusable components, abstract enough to be reuse on different hardware of the same kind, avoiding to start a project from scratch each time other hardware are used. Finally, our contribution consists to design a method and put together a toolchain allowing to model a ROS application. This method would allow the designer of the model an easy access of the most important part of ROS as pre-designed components.

PROPOSED APPROACH AND RESEARCH METHODOLOGY

In this work we follow the MDE methodology [START_REF] Schlegel | Model-driven software systems engineering in robotics: Covering the complete life-cycle of a robot[END_REF]. This methodology places emphasis on abstract representations rather than the coding details. It improves the productivity, the quality and the ability to handle complexity by increasing the level of abstraction. It gives advantages on the automation (code generation) and improves the analysis capabilities for early-verification with model-checking and state exploration.

Since robotic systems are more and more complex and sophisticated, it seems legitimate to use MDE techniques as a mechanism to enable early V&V. Previous work in this area proved the relevance of such an approach [START_REF] Ahmadi | Run-time Monitoring of a Rover: MDE Research with Open Source Software and Low-cost Hardware[END_REF]. This approach results in creating a model of a robot and in generating the code to be embedded in the robot from the model. We can focus on creating a platform independent model (PIM) that can be used on various actual robot platforms, thus answering to the challenge related to the variety of hardware components. Through model transformation such a PIM could be transformed in a PSM (platform specific model) specific to the targeted hardware architecture.

In our approach, we found three different possibilities to create an abstraction through model:

(1) With a UML profile. It provides a generic extension mechanism for customizing UML models for particular domains and platforms. We target two different environment to determine our profile, one from an open source environment, another one from an industrial tool: A DSL is a computer language, with a high level of abstraction, optimized for a specific class of problems, using the concepts and rules from the field of the problem. DSL in Robotics for programming and for simulation of robots is shown by [START_REF] Nordmann | A Survey on Domain-Specific Languages in Robotics[END_REF] to be currently an active research field. DSL provides simplicity at design-time since the level of abstraction is at the problem domain. Consequently, we are interested in it, since we would like to be able to create a modelling language dedicated to ROS, which means to keep a level of abstraction close to the domain of ROS. To create a DSL, the tool Sirius 5 , an open-source software project of the Eclipse Foundation, might be an option.

•
Our approach consists to study the three previously cited options with the intention of comparing them between each other, and select the approach to be adopted in the rest of our study. The comparison would be done using several metrics, and by measuring those metrics with field engineers trying the different approaches in controlled studies.

Next, we will apply verification, such as model-checking, with timed automata, on robotics models designed with the approach we have chosen. We will elaborate customized verification techniques on robotics models, for example, in the case some properties seems to be recurrent. And finally, we will identify a case study on which to apply and validate our approach.

PRELIMINARY RESULTS

Our approach focuses on the robotics middleware ROS. Consequently, we picked a set of ROS examples: one where nodes are transferring a message from one to another through topics, and Student Research Abstract: A Toolchain for Model-Based Development of a Robot Application SAC '20, March 30-April 3, 2020, Brno, Czech Republic making some computation on the message. Another one allowing the control of a robot, with the possibility to use the control, either in a simulation, or in the real world.

The next step of the project consists, using the previously cited ROS examples, in comparing the modeling environments IBM Rational Rhapsody, Papyrus, and Sirius, in order to decide the abstract modeling approach to be adopted in the rest of our study. We created our models using these three different environments as a means to compare them and understand the pros and cons of each of them.

Using Papyrus, we created an UML profile. We defined a stereotype of package as RosPackage, three stereotypes of class: RosSystem (representing the complete system of the robotic system), RosNode and RosTopic (represent the nodes and the topics in ROS). A stereotype of signal as RosMessage, and two stereotypes of port, RosPublisher and RosSubscriber, so as to make the difference between the port that will send or receive a RosMessage. Then are able to define our projects with a class diagram and a composite structure diagram, representing the structure of the robotic system, and using state-machine diagram to represents the behavior of the nodes and topics. From there, it is possible to execute the model animation using Moka. Due to technological issues, we have not been able to reach a complete model execution.

Using IBM Rational Rhapsody, we first define a UML profile metamodel. We created the same stereotypes of classes and ports as in Papyrus. However, we defined the stereotype RosMessage for the events which are sent through ports using interfaces, and the stereotype of a class, RosPackage, to encompass all the elements of the profile.

Once the profile is defined, it is possible to create projects using this profile. For our preliminary work, we have been able to create the projects based on the examples previously stated. With this tool, we succeed to pass messages from nodes to nodes through topics using the ports and connectors of each of them. Using the components diagram, it is possible to define the structure of the robotic system, i.e. the nodes, the topics, and the connection between each ports mainly. Each nodes and topics have their own statecharts diagram, where it is possible to describe their behavior. With these projects, we can execute animations of the statecharts step-by-step and validate the model.

Using Sirius, first the metamodel must be created. Figure 1 shows it, and from it we are now able to create the graphical representation of the DSL and, using it, to start projects with it. We composed the graphical representation of a component diagram (to define the structure of the robotic system), and of state-machine diagram associated to each node (to define the behavior of the system). This simple state-machine diagram allows the user to define an initial state, some states and transitions. It is possible to complete the state with do action, and the transition with events, guards, and actions.

Then, we used the code generator Acceleo, so that, depending on the component diagram and the state-machine diagrams of the nodes, we generate a code in Python for each node. Using the set of ROS examples, we succeed to generate Python code, ready-to-use in ROS. Our next goal is to make the code generation working with more complex projects, to not only generate the nodes code, but the complete ROS project, and to be able to animate the diagrams.

Preliminary conclusion

Although this study is not completely done, it revealed difficulties in modeling. Most of the problems are simply technological ones, the tools are sometimes not mature enough, or might lack of documentation, which can lead the field engineers to think that it is not worth it to go through these difficulties or even to be disappointed and to not want to continue on this way. For now, over the two options , the option (2) seems more promising, with the graphical interface that can be modify as wanted, the model gets closer to the domain of ROS and it might be less complex, and so more accessible to field engineers.

RELATED WORK

Currently, it exists few model-based development or componentbased tools in robotics:

BRICS [START_REF] Bruyninckx | The BRICS Component Model: A Model-based Development Paradigm for Complex Robotics Software Systems[END_REF] is a model-based development paradigm for complex robotics software systems It is build upon two complementary paradigms: the "5Cs" (separation of concerns between the development aspects of Computation, Communication, Coordination, Configuration and Composition) and the meta-modeling approach from Model-Driven Engineering. It provides to robotics developers a set of guidelines, meta-models and tools to structure the development of individual components and component-based architectures, because it is important to have all the systems following formal models. An instance of meta-model is provided: the Component-Port-Connector meta-model, presenting a system as a collection of components and connectors and making the architecture of the system software-framework independent.

HyperFlex [START_REF] Gherardi | Modeling and reusing robotic software architectures: The HyperFlex toolchain[END_REF] is a software toolchain supporting the design and the reuse of reference architectures. The modeling of robot system architectures is done at design time with HyperFlex, using components, interfaces, connectors, and components wiring, which are equivalent to the ROS paradigm. It is consequently possible to design the architectural models of ROS similarly as what we are doing. In addition, it enables the explicit representation of the functional variability and application requirements as software models that can be manipulated by a system configuration engine.

Orca [START_REF] Makarenko | Orca: Components for robotics[END_REF] is an open-source software project applying Component-Based Software Engineering (CBSE) to robotic system. It gives the possibility to form complex robotic systems by defining and developing components that can work together and the goal of Orca is to let the robotics software developer focus on the component code only, providing the infrastructure code as complete as possible so that developers do not need to write or maintain it.

Opros [START_REF] Jang | OPRoS: A New Component-Based Robot Software Platform[END_REF] The following works are defining DSLs: RoboChart [START_REF] Miyazawa | RoboChart: modelling and verification of the functional behaviour of robotic applications[END_REF] is a DSML based on UML to write models for robotic systems where it is possible to apply model checking (using CSP) and theorem proving. More precisely, to design a robotic system, RoboChart allows to create state-machine with a goal of definition, formalization and application. It is possible to automatically generate C++ implementations for a subsets of state-machines and controllers. The difference with our DSL is that we cover a code generation for ROS, but they already cover the model checking.

The toolchain BRIDE [START_REF] Bubeck | BRIDE -A toolchain for frameworkindependent development of industrial service robot applications[END_REF], based on EMF language Ecore, defined a DSL for ROS, including a graphical editor and code generator (taking care of creating a node skeleton as well as tool chain files to finish with a ready to compile ROS component). Once the capability and system models are created, they are automatically transformed to another representation, usually source code. Then it is possible to get a component platform independent and to port it into another platform than ROS such as OROCOS.

CONCLUSION

This paper presents an ongoing work, done in the context of a PhD degree thesis, on using a model-based environment to develop a robot application. We describe two UML profiles, created in the environment Papyrus and Rational Rhapsody, and a DSL created in the environment Sirius, in order to create a model of a robot application using components that can be verify by model-checking and validated by model animation. We showed the preliminary results demonstrating the possibility to use the MDE methodology in the robotic field, and more precisely applied on the middleware ROS, allowing to generate code and to simulate the models. In future work, we want to continue those experimentation with more complex projects and improve the verification phase by adding model-checking to the models, and to identify a case study on which to apply and validate our approach.

Figure 1 :

 1 Figure 1: Sirius metamodel for the ROS DSL.

  Papyrus 3 is able to create such a UML profile. It is an open source modeling environment based on Eclipse, supporting UML, SysML and DSL. From the UML profile, it is then possible to create a model from which model execution and code generation are achievable. • IBM Rational Rhapsody 4 is an UML industrial tool integrating validation capabilities. It supports UML and SysML. It is able to create an UML model from which it is possible to do model execution and code generation. (2) With a Domain-Specific Language (DSL). It is another key part of MDE.

  is a component-based robot software platform. It consists of specifications of a component model, component authoring tool (tool to create a component), component composer (tool to build robot applications by composing components), and component execution engine (manages and executes the components). As in our case, some of the tools are dedicated to ROS: ReApp project [13] is a model-based engineering tool for ROS. The ReApp (Reusable Robot Applications for Flexible Robot Plants Based on Industrial ROS) project provides a workbench (development environment) based on ROS by using model-based design of robot applications composed of reusable components. The ReApp workbench consists of five parts: A cloud-based semantic repository, the Component Modeling Tool (CMT), the Component Code Generation Tool (CCGT), the Integration Platform & Development Environment (IPDE), and the central part, the Skill and Solution Modeling Tool (SSMT). The SSMT allows, with a graphical editor to create and reuse different types of nodes, and to configure and connect with other nodes.

https://en.wikipedia.org/wiki/Formal_verification

https://en.wikipedia.org/wiki/Verification_and_validation

https://www.eclipse.org/papyrus/

https://www.ibm.com/us-en/marketplace/systems-design-rhapsody

https://www.eclipse.org/sirius/

ACKNOWLEDGEMENT

I thank Ileana Ober, Iulian Ober, Raquel Oliveira for many suggestions to guide my research and improve this paper.