Dávid Maruscsák
email: david.maruscsak@lisn.upsaclay.fr

Dániel Besnyő
email: danielbesnyo@centrumproduction.com

Szűcs Gábor

Freelancer

Jeanne Vézien
email: jeanne.vezien@lisn.upsaclay.fr

Christian Sandor
email: christian.sandor@lisn.upsaclay.fr

MiniGI: Guerilla Mappings in Miniature

Figure 1: a-d): Photo collection of the "guerilla mappings" created by the Secret Mapping ensemble. e): AR installation, including a map and 3D printed objects of the real monuments.

INTRODUCTION

We present MiniGI, an interactive tabletop installation coupled with a handheld Augmented Reality (AR) visualisation. The installation showcases five selected locations which are the human-scale twins of actual, real-word exhibits all located throughout former Yougoslavia. AR visualisation illuminates these miniature 3D replicas and encompasses them with an extended version of the archived visuals created by the artists, specifically to each spot. The installation invites visitors to join the artists on their trip and reenact their journey with an collaborative tabletop interface, while offering the possibility for backwards or forwards exploration.

Projection Mapping is a technique for aligning virtual contents, e.g. 3D animations onto irregular surfaces or facades and turning them into a display. Preparation steps typically involve 3D reconstruction of the surface of interest and 3D animations carefully designed for said surfaces. When well-aligned, these animations can create captivating experiences [START_REF] Raskar | Table-Top Spatially-Augmented Reality: Bringing Physical Models to Life with Projected Imagery[END_REF]. In a largescale context, striking illusions of building moving or transforming are achievable [START_REF] Anadol | WDCH Dreams[END_REF]]. In order to fit 3D structures as closely as possible, it is necessary to achieve real-time on-site image warping. Many dedicated software exist for this purpose, e.g. Resolume.

Usually, large 3D projection mapping events are created in a commercial setting, and the visuals are designed to fit the customer's requirements. The formation of the Secret Mapping ensemble emerged from the desire to create 3D mappings from the specific imagination of the each creator individually. Currently, the collective includes three people coming from different art fields.

submitted version; git commit 63e5fd17907fa2100725d6d46053084c14065232 which shows the AR elements superimposed onto the 3D printed objects. Only one object is illuminated the rest of the map is dimmed. On image b) the user touches the edge of the table to trigger the interaction. A luminescent sphere appears over the route and shows the current position of the user on the map. On image c) the user slides her finger along the edge of the table. This gesture moves the user forward or backward on the map. On image d) the user reached the next destination on the map and the 3D printed object at this location gets highlighted with AR visuals.

The Secret Mapping collective made projections on outstanding environmental formations, abandoned structures, and industrial buildings. The most interesting facades available were identified to be part of abandoned brutalist buildings, remnants of the Yugoslavian era. Originating from Hungary, the collective travelled many hours into neighbouring countries like Croatia, Bosnia and Herzegovina, to achieve these projection mapping happenings, with a van loaded with a professional projector and a power generator since these locations often fall outside of our accustomed infrastructure. Such "guerilla mappings" start after sunset and end before sunrise.

The proposed work invites the visitor to join the creators on this impromptu trip, starting from Budapest and visiting five selected locations the collective has visited. The virtual interactive journey takes place on a table laying out a miniature map of the road trip coupled with 3D-printed small-scale replicas of the featured buildings. It combines AR mapping [START_REF] Reitmayr | Localisation and Interaction for Augmented Maps[END_REF] technologies with tangible tabletop interactions [START_REF] Patten | Mechanical Constraints as Computational Constraints in Tabletop Tangible Interfaces[END_REF].

DESIGN

The installation includes a table (r=50cm) with a printed map layout on top of it. The map shows the five selected locations and highlights the road connecting them, starting from Budapest and ending at the tip of Greece in Gytheio. The installation's core parts are the printed 3D replicas of the visited structures placed on the map (created using Agisoft Metashape photogrammetry software and Cinema4D 3D modelling software), with the digital projected visuals appearing as augmented reality (AR) elements. Using AR, the 3D animations not only appear on the surface of these structures but also around them, thus encompassing these objects completely. These AR visuals or mappings were designed as extensions of the archived visuals created by the Secret Mapping artists during their actual real-world trip. The installation utilises handheld AR technologies in the form of two Ipads, so that the visitors can each see these AR mappings on accompanying tablet displays, from their respective vantage point (see Figure 2). The iPad screen is large enough so that the view can be shared amongst multiple visitors, creating a sense of a collective experience. The AR software was developed in Unreal Engine 5, and it uses the printed map as a marker for placing the AR objects onto the physical installation at their respective positions.

A Playtron Midi controller detects the touch of the users on the edge of the table covered with conductive ink. To determine the direction of the hand gesture (clockwise or counterclockwise) we analyse the camera image with optical flow technique.

USER EXPERIENCE

The road connecting the locations together is mapped onto the table's perimeter, see Figure 2. The interaction design is meant to involve the users and invoke the feeling of travelling together with the artists. The 5cm stroke on the table's edge functions like a touch interface. When the user touches the stroke and slides their fingers along, they "drive" with the artists along their route forwards or backwards depending on whether the users slide their fingers clockwise or counter-clockwise. The AR elements around each of the five miniature objects appear only when the user reaches their respective point on the map, while all others are rendered blank and featureless. When the user moves away from a given featured location, the AR mapping around the building fades out and a sphere travelling along the map highlights the current position of the artists in-between adjacent structures.

FUTURE WORK

Future work on the MiniGI installation includes providing multiuser interaction, as currently only one user can "travel" on the digital road at a given time. We also wish to include AR visualization through headsets, which would enable viewing digital twins of the installations at their original scale as well.

Finally it should be noted that such an interactive map could be used for upcoming events occurring in distributed sites (e.g. Olympics 2024 taking place in various locations in the Paris area).

Figure 2 :

 2 Figure 2: Image sequence showing the installation's interaction design. a) the user holds the iPad over the tablewhichshows the AR elements superimposed onto the 3D printed objects. Only one object is illuminated the rest of the map is dimmed. On image b) the user touches the edge of the table to trigger the interaction. A luminescent sphere appears over the route and shows the current position of the user on the map. On image c) the user slides her finger along the edge of the table. This gesture moves the user forward or backward on the map. On image d) the user reached the next destination on the map and the 3D printed object at this location gets highlighted with AR visuals.


Aude Sportisse

Matthieu Marbac

Fabien Laporte

Gilles Celeux

Claire Boyer

Christophe Biernacki

Julie Josse

Accompanying note: Model-based Clustering with Missing Not At Random Data

MNAR modeling 2.1 Rationale for MNAR assumptions

To handle MNAR data in selection models, the distribution of the missing-data pattern given the data and the partition should be specified. We consider the following assumptions:

1. The elements of c i are conditionally independent given (y i , z i ).

Organization of this accompanying note

This document is the accompanying note of the main paper [START_REF] Sportisse | Model-based clustering with missing not at random data[END_REF]. We assume the data missing not at random (MNAR) values [START_REF] Donald B Rubin | Inference and missing data[END_REF][START_REF] Ibrahim | Missing responses in generalised linear mixed models when the missing data mechanism is nonignorable[END_REF][START_REF] Mohan | Estimation with incomplete data: The linear case[END_REF], i.e. the effect of missingness depends on on the missing values themselves. An example includes clinical data collected in emergency situations, where doctors may choose to treat patients before measuring heart rate: the missingness of heart rate depends on the missing heart rate itself. For such a setting, the observed data are therefore not representative of the population. The main paper focuses on the specific MNARz setting, for which the only effect of missingness is on the class membership; in this document, we give some details for other MNAR settings.

Contributions

In Section 2, we present and illustrate a relevant inventory of distributions for the MNAR missingness process in the context of unsupervised classification based on mixture models for different types of data (continuous, count, categorical and mixed). In Section 3, we provide the identifiability of the mixture model parameters and missingness process parameters under certain conditions (including the data type and the link functions governing the missingness mechanism distribution). This is a real issue in the context of MNAR data, as models often lead to unidentifiable parameters. When all variables are continuous or count, all models lead to identifiable parameters. In the categorical and mixed cases, only the models for which missingness depends uniquely on the class membership have identifiable parameters. These identifiability results represent a substantial extension of the work of [START_REF] Miao | Identifiability of normal and normal mixture models with nonignorable missing data[END_REF] to more complex missing scenario and to the multivariate case. For each model or submodel, an EM or Stochastic EM algorithm is proposed in Section 4, implemented, and made available for reproducibility 1 . Finally, we propose a numerical experiment in Section 5 to assess the clustering performance in each proposed MNAR setting.

Notations

Set the dataset Y = (y 1 | . . . |y n ) T consisting of n individuals, where each observation y i = (y i1 , . . . , y id ) T belongs to a space Y, depending on the type of data, defined by d features. The pattern of missing data is denoted by C = (c 1 | . . . |c n ) T ∈ {0, 1} n×d , with c i = (c i1 , . . . , c id ) T ∈ {0, 1} d : c ij = 1 indicates that the value y ij is missing and c ij = 0 otherwise. The values of the observed (resp. missing) variables for individual i are denoted by y obs i (resp. y mis i ). The objective of clustering is to estimate an unknown partition Z = (z 1 | . . . |z n ) T ∈ {0, 1} n×K that groups the full dataset Y into K classes, with z i = (z i1 , . . . , z iK ) T ∈ {0, 1} K and where z ik = 1 if y i belongs to cluster k, z ik = 0 otherwise. Consequently, in a clustering context, the missing data are not only the values y mis i but also the partition labels z i .

Mixture models allow for clustering by modeling the distribution of the observed data (y obs i , c i ). Assuming an underlying mixture model with K components, the probability distribution function (pdf) of the couple (y i , c i ) reads as

f (y i , c i ; θ) = K k=1 π k f k (y i ; λ k )f k (c i | y i ; ψ k ), (1)
where θ = (γ, ψ) gathers all the model parameters, γ = (π, λ) groups the parameters related to the marginal distribution of y i , π = (π 1 , . . . , π K ) is the vector of proportions with K k=1 π k = 1 and π k > 0 for all k ∈ {1, . . . , K}. Given

λ = (λ 1 , . . . , λ K ), f k (• ; λ k ) is the pdf of the k-th component parameterized by λ k , ψ = (ψ 1 , . . . , ψ K ) groups the parameters of the missingness mechanisms and f k (c i | y i ; ψ k ) is the pdf related to the missingness mechanism under component k (i.e., f k (c i | y i ; ψ k ) = f (c i | y i , z ik = 1; ψ k )).
In many cases, the parameter ψ is interpreted as a nuisance parameter. However, when the mechanism is not ignorable, we need to consider the whole parameter θ to achieve clustering since the pdf of the observed data is

f (y obs i , c i ; θ) = f (y i , c i ; θ)dy miss i . ( 2
)
Different types of pdf f k (• ; λ k ) can be considered, depending on the types of features at hand. Thus, if y i is a vector of continuous variables, the pdf of a dvariate Gaussian distribution [START_REF] Mclachlan | Mixture models: Inference and applications to clustering[END_REF][START_REF] Banfield | Model-based gaussian and non-gaussian clustering[END_REF] can be considered for f k (y i ; λ k ) and thus λ k groups the mean vector and the covariance matrix. Moreover, if some components of y i are discrete or categorical, the latent class model (see [START_REF] Geweke | Alternative computational approaches to inference in the multinomial probit model[END_REF][START_REF] Mcparland | Model based clustering for mixed data: clustmd[END_REF]) defining f k (y i ; λ k ) = d j=1 f kj (y ij ; λ kj ) can be used, with λ k = (λ k1 , . . . , λ kd ). In such case, f kj could be the pdf of a Poisson (resp. multinomial) distribution with parameter λ kj if y ij is an integer (resp. categorical) variable. The next subsection discusses the choice of the modeling for the missingness mechanism (i.e., the distribution

f k (c i | y i ; ψ k )).
2. The element c ij is conditionally independent given (y i , z i ) from y ij for j = j .

By the categorical nature of the mask c i , the independence assumption 1. is a quite natural hypothesis in the context of clustering (Du Roy De Chaumaray and [START_REF] Roy De Chaumaray | Clustering data with nonignorable missingness using semi-parametric mixture models[END_REF][START_REF] Chi | k-pod: A method for k-means clustering of missing data[END_REF]. The independence assumption 2. of the variables amounts to considering self-masked class-wise MNAR mechanisms for each variable: the missingness of the variable j may depend on its value itself (self-masked) and on the class membership (class-wise). Note that the self-masked feature, apart from limiting the number of parameters to be estimated, is now commonly met in the literature [START_REF] Mohan | On handling self-masking and other hard missing data problems[END_REF][START_REF] Sportisse | Imputation and low-rank estimation with missing not at random data[END_REF][START_REF] Le Morvan | Neumiss networks: differentiable programming for supervised learning with missing values[END_REF], and is able to retrieve some existing ad hoc MNAR procedures already used in machine learning community (see more details in [START_REF] Sportisse | Model-based clustering with missing not at random data[END_REF], Theorem 1)). More specifically, the conditional distribution of c ij given (y i , z i ) is assumed to be a (classical) generalized linear model with link function ρ, so that finally

f k (c i | y i ; ψ k ) = d j=1 (ρ(α kj + β kj y ij )) c ij (1 -ρ(α kj + β kj y ij )) 1-c ij , (3)
where

ψ k = (α k1 , β k1 , . . . , α kK , β kK ).
The parameter α kj represents a mean effect of missingness on the k-th class membership for the variable j (note that within a same class k, α kj is not necessarily equal to α kj for j = j ). The parameter β kj represents the direct effect of missingness on the variable j which depends on the class k as well. This model is called MNARy k z j in the following.

Some variations include

f k (c ij = 1 | y i ; ψ k ) = ρ(α kj + γ kj y ij )
, where the j -th variable is always observed. In such a case, the missing values are missing at random (MAR) as only depending on observed variables. Identifiability guarantees and estimation are still valid in such a setting.

Models

Simpler models can be derived from (3) by imposing equal parameters either across the class membership, or across the variables likely to be missing. First, we introduce three models, with a lower complexity than (3), that still allow the probability of being missing to depend on both the variable itself and the class membership. For the so-called MNARyz j model, the effect of missingness on a variable is the same regardless of the class (while keeping different mean effects α kj on the class membership), so that MNARyz j : β 1j = . . . = β Kj , ∀j.

(4)

For the MNARy k z model, the missingness has a same mean effect on class membership shared by all variables (while allowing different self-masked and class-wise parameters β kj ): MNARy k z: α k1 = . . . = α kd , ∀k.

(5)

The effects on a particular variable and on the class membership can be respectively the same for all the classes and for all the variables, entailing the so-called MNARyz model:

MNARyz: β 1j = . . . = β Kj , ∀j and α k1 = . . . = α kd , ∀k. (6)
Secondly, the probability to be missing can also depend only on the variable itself. This is actually a particular case of MNAR mechanims, widely used in practice [START_REF] Mohan | On handling self-masking and other hard missing data problems[END_REF], that we call MNARy here. The only effect of missingness is thus on the variable j, being the same regardless of the class membership, MNARy: α 11 = . . . = α 1d = α 21 = . . . = α Kd and β 1j = . . . = β Kj ∀j.

(7) A slightly more general case can be considered by allowing the effect of missingness on the variable j to depend on the class k, as in the following MNARy k model,

MNARy k : α 11 = . . . = α 1d = α 21 = . . . = α Kd . (8)
Thirdly, the probability to be missing can also depend only on the class membership, so that the missingness is class-wise only. In the MNARz model, we consider that the only effect of missingness is on the class membership k, being the same for all variables, MNARz: β kj = 0, ∀(k, j) and α k1 = . . . = α kd , ∀k.

(9)

The MNARz j model is a slightly more general case than the MNARz model, because the effect of missingness on the class membership k is not the same for all the variables, MNARz j :

β kj = 0, ∀(k, j). (10)
Finally, the simplest model is the missing completely at random (MCAR) one, characterized by no dependence on variables, neither on class membership, i.e., each variable has the same probability of missing, MCAR: β kj = 0, ∀(k, j) and α 1j = . . . = α Kj , ∀j.

(11)

3 Identifiability of the model parameters

The generic identifiability [START_REF] Allman | Identifiability of parameters in latent structure models with many observed variables[END_REF] of parameters for continuous, count, categorical, and mixed data (i.e., when the set of unidentifiable parameters has a zero Lebesgue measure) is ensured by the following theorem. We consider the following assumptions:

A1. The parameters (π, λ) of the marginal mixture defined by the density K k=1 π k f k (y i ; λ k ) are identifiable;

A2. There exists a total ordering of F j ×R, for j ∈ {1, . . . , d} fixed, where F j is the family of the data densities {f 1j , . . . , f Kj } and R is the family of the mechanism densities {ρ 1 , . . . , ρ K } = {ρ(.; ψ 1 ), . . . , ρ(.; ψ K )}, where ρ is the cumulative distribution function of any continuous distribution function and (ψ k ) k∈{1,...,K} its parameter. The total ordering is such that ∀k < ∈ {1, . . . , K}, ∀j ∈ {1, . . . , d}, F kj F j (denoting

F kj = ρ k f kj and F j = ρ f j ) implies lim u→+∞ ρ (u)f j (u) ρ k (u)f kj (u) = 0;
A3. The missing-data distribution ρ is assumed to be strictly monotone.

A4. The feature are independently drawn conditionally to the group membership, i.e.,

f k (•; λ k ) = d j=1 f kj (• ; λ kj ); (12)
A5. The dimension d of the observations is related to the number K of clusters so that d ≥ 2 log 2 K + 1, with x the least integer greater than or equal to x.

Assumption A1. means that the identifiability of the parameters (π, λ, ψ) of the model (2) requires the identifiability of the parameters (π, λ) of the marginal mixture of (Y, Z) (i.e., considering the case without missing values). Some authors have already studied the identifiability of the mixture models, when no missing values in Y occur, especially [START_REF] Teicher | Identifiability of finite mixtures[END_REF] for Gaussian mixtures (continuous variables) and [START_REF] Yakowitz | On the identifiability of finite mixtures[END_REF] for Poisson mixtures (count variables). Assumption A2. is the core ingredient to prove the identifiability of the parameters, requiring that a total ordering of the mixture densities holds. We illustrate it by considering concrete examples in Appendix A. Note that under Assumption A3. requires that the link function of the missing data mechanism is strictly monotone, but no assumption about its form (e.g. logit, probit) is made. Assumption A4. requires the conditional independence of the features given the group membership and Assumption A5. links the dimension of the observations and the logarithm of the number of clusters. Both assumptions A4. and A5. are classical in the categorical case, even without missing values [START_REF] Allman | Identifiability of parameters in latent structure models with many observed variables[END_REF].

Theorem 3.1. Define the conditions: C1 The variables correspond to continuous or count data, A1. and A2. hold true, C2 All the variables are categorical, A4. and A5. hold true and the mechanism is stated by (9), ( 10) or (11), C3 At least one variable is continuous or count data and has a marginal distribution that satisfies A1. and A2., A4. holds true, C4 At least one variable is categorical and its associated mechanism is stated by ( 9), ( 10) or (11), A4. and A5. hold true.

Assume that Assumption A3. holds and that at least one of conditions C1-C4 is satisfied, then the parameters of the model in (2) are generically identifiable, up to label swapping.

The proof is given in Appendix A. In the case of continuous and count variables, the proof follows the reasoning used by Teicher (1963, Theorem 2) which proves the identifiability of univariate finite mixtures. For categorical variables, the generic identifiability holds only for the MCAR, MNARz and MNARz j mechanisms. The idea of the proof is to rewrite the observed likelihood as the finite mixture of K multinomial distributions, for which the identifiability is given by Corollary 5 of [START_REF] Allman | Identifiability of parameters in latent structure models with many observed variables[END_REF]. For MNARy mechanisms, the rewriting is impossible, because of the dependency on y of the mechanism. The identifiability of mixed data directly follows from the identifiability of continuous and categorical components.

Estimation of the proposed MNAR models

Assuming identifiability, we estimate parameters via likelihood maximization using EM and SEM algorithms specifically designed for Gaussian, Poisson, multinomial and mixed data with MNAR data.

For the MNARz mechanism, an EM algorithm an be derived (see the main paper [START_REF] Sportisse | Model-based clustering with missing not at random data[END_REF]), because the effect of the missingness does not depend on y i . As the MNARz j mechanism has the same property, an EM algorithm can also be derived (see Appendix B). However, the EM algorithm becomes untractable when the missingness depends on variables y, such models being generically denoted by MNARy in the sequel. In particular, some distributions entail untractable integrals at the E-step (e.g., Gaussian components with MNARy mechanism defined with logit link, see Appendix B for more details).

The stochastic EM algorithm [START_REF] Celeux | The SEM algorithm: a probabilistic teacher algorithm derived from the EM algorithm for the mixture problem[END_REF] can overpass the EM's intractability, by imputing missing values using Gibbs sampling instead of integrating over them. In addition, it has another advantage, unlike the EM algorithm, not to be necessarily trapped by the first encountered local maximum of the likelihood function in play [START_REF] Celeux | The SEM algorithm: a probabilistic teacher algorithm derived from the EM algorithm for the mixture problem[END_REF]. The principle of the SEM algorithm is to involve a stochastic-E step (SE-step) instead of the traditional Estep of the EM algorihtm. Details of the algorithm is given in Appendix C. The iteration [r] then becomes: SE-step: Draw the missing data (z

[r] i , y mis [r] i
) according to their conditional distribution given the observed data (y obs i , c i ) and the current parameter θ [r-1] . As simulating according to this conditional distribution may be difficult, we simulate instead according to the following two conditional probabilities using a Gibbs sam-pler, by noting y

[r] i = (y obs i , y mis[r] i ), z [r] i ∼ z i | y [r-1] i , c i ; θ [r-1] and y mis [r] i ∼ y mis i | y obs i , z [r] i , c i ; θ [r-1] . (13) M-step: Let Y [r] = (y [r] 1 | . . . |y [r]
n ) be the imputed matrix and let

Z [r] = (z [r] 1 | . . . | z [r]
n ) be the current and corresponding partition. The parameter θ [r] is computed using the maximum likelihood estimate in the complete case. For all k ∈ {1, . . . , K}, the parameter π

[r]
k is the proportion of rows of Y [r] belonging to class k. The parameter λ

[r]

k is updated in a standard way, depending on the parametric mixture family in play. Finally, the parameter ψ

[r]

k is the resulting coefficients of a GLM with a binomial link function, cf Appendix C for details.

In the SE-step, note that the sampling of z

[r]

i is performed by a multinomial distribution. The conditional distribution of y mis i given (y obs i , z

[r] ik = 1, c i ) param- eterized by θ [r-1] is f k (y mis i | y obs i , c i ; θ [r-1] ) = f k (y mis i | y obs i ; θ [r-1] )f k (c i | y mis i , y obs i ; ψ [r-1] ) f k (y mis i | y obs i ; θ [r-1] )f k (c i | y mis i , y obs i ; ψ [r-1] )dy mis i .
This distribution may not be classical in general. For example, for MNARy models, it is not explicit if the components are Gaussian and if the missing data distribution ρ is logistic (since the product of logistic and Gaussian distributions is not a standard law). Therefore, the SEM algorithm cannot be easily applied. However, if ρ is the probit function, we can make the distribution of interest explicit (it is a truncated Gaussian distribution when the variables are Gaussian). For MNARz and MNARz j models, all the computations remain feasible.

Implementation and numerical experiments

The SEM algorithm has been implemented for each MNAR setting for Gaussian data and is available on https://anonymous.4open.science/r/ Clustering-MNAR-7E29.

For each MNAR setting, we assess the clustering performance through the consistency of the partition, by computing the ARI between the true partition Z and the estimated one, given by ẐMAP

= {z MAP ik ( θ)} i,k ∈ R n×K as follows with z MAP ik (θ) = argmax k∈{1,...,K} P(z ik = 1|y obs i , c i ; θ).
We consider thus the following methods:

• the EM algorithm (for MCAR (11), MNARz (9) and MNARz j (10)),

• the SEM algorithm (for MNARy (7), MNARy k (8), MNARy k z j (3), MNARyz ( 6))

The data are generated using a Gaussian mixture with three components having unequal proportions (π 1 = 0.5, π 2 = π 3 = 0.25) and independent variables:

∀j ∈ {1, . . . , d}, y ij = δ 3 k=1 ϕ kj z ik + ij , (14)
with ij ∼ N (0, 1) the noise term, ϕ k ∈ {0, 1} d and δ > 0. We consider d = 6 variables and we vary the number of observations n = 100, 250, 500. In Figure 1, as expected, considering the mechanism always gives better results than using the MCAR model, especially for models with many parameters and larger sample sizes (as the MNARyz, MNARy k z j , MNARy k z, MNARyz j settings for n = 250 and n = 500). Finally, consistency seems satisfactory in each scenario, indicating that our tuning parameters for the algorithm (starting values, stopping rules) are quite suitable.

In the main paper [START_REF] Sportisse | Model-based clustering with missing not at random data[END_REF], we compare the MNARz setting with other ones, and discuss the computational cost of the estimation for the different MNAR settings.

Supplementary material

This file is a supplementary material. In Appendix A, the proof for Proposition 3.1 is given. Some complements on the EM algorithm are given in Appendix B. The SEM algorithm presented in Section 4 is detailed in Appendix C. . Appendix D gives the values of hyperparameters for the numerical experiments on synthetic data.

A Appendix 1: Identifiability

A.1 Continuous and count data

Proof of Proposition 3.1, continuous case. Suppose there exists two sets of parameters {γ, ψ} and {γ , ψ } which have the same observed distribution, i.e., f (y obs i , c i ; γ, ψ) = f (y obs i , c i ; γ , ψ ). More precisely, one has

∀y i ∈ R d , ∀c i ∈ {0, 1} d , K k=1 Y mis i π k f k (y i ; λ k ) d j=1 ρ(α kj + β kj y ij ) c ij [1 -ρ(α kj + β kj y ij )] 1-c ij dy mis = K k=1 Y mis i π k f k (y i ; λ k ) d j=1 ρ((α ) kj + (β ) kj y ij ) c ij [1 -ρ((α ) kj + (β ) kj y ij )] 1-c ij dy mis
Let us consider the case when c ij = 0 for all j = 1, . . . , d. One has

K k=1 π k f k (y i ; λ k ) d j=1 (1 -ρ(α kj + β kj y ij )) = K k=1 π k f k (y i ; λ k ) d j=1
(1 -ρ((α ) kj + (β ) kj y ij )).

By using the identifiability of the marginal mixture, one obtains

λ k = λ k and π k d j=1 (1 -ρ(α kj + β kj y ij )) = π k d j=1 (1 -ρ((α ) kj + (β ) kj y ij )).
In the sequel, we use the same reasoning of Theorem 2 in [START_REF] Teicher | Identifiability of finite mixtures[END_REF]

. Let us denote F k (y ij ) = f kj (y ij ; λ kj ) d j=1 (1- ρ(α kj + β kj y ij )) and F k (y ij ) = f kj (y ij ; λ kj ) d j=1 (1 -ρ((α ) kj + (β ) kj y ij )). Without loss of generality, assume that F k ≺ F l and F k ≺ F l for k < l. If F 1 = F 1 , we assume also without loss of generality that F 1 F 1 . Then, F 1 ≺ F k for 1 ≤ k ≤ K . For u ∈ T 1 , where T 1 = S F 1 ∩ {u : F 1 (u) = 0} is the domain of definition of F 1 such that f 1j (u; λ 1j ) d j=1 (1 -ρ(α 1j + β 1j u)) = 0, one has π 1 + K k=1 π k F k (u) F 1 (u) = K k=1 π k F k (u) F 1 (u) .
Letting u → +∞, π 1 = 0 which is in contradiction with the mixture model (where π k > 0, ∀k = 1, . . . , K. It implies that F 1 = F 1 . For any u ∈ T 1 , one has

π 1 + K k=2 π k F k (u) F 1 (u) = π 1 + K k=2 π k F k (u) F 1 (u) .
Letting again u → +∞, one obtains

π 1 = π 1 and K k=2 π k F k (u) F 1 (u) = K k=2 π k F k (u)
F 1 (u) . We repeat this argument to conclude that

F k = F k and π k = π k for k = 1, . . . , min{K, K }. Finally, if K = K , say K > K , K k=K +1 π k F k (u) = 0 implies π k = 0 for K + 1 ≤ k ≤ K which is in contradiction with the definition of the mixture model. Thus K = K . Finally, F k = F k implies that d j=1 (1 -ρ(α kj + β kj y ij )) = d j=1 (1 -ρ((α ) kj + (β ) kj y ij ))
. By integrating out over all the elements but the j-th element, one has for all y ij ∈ R, ρ(α kj + β kj y ij ) = ρ((α ) kj + (β ) kj y ij ). and α kj = (α ) kj and β kj = (β ) kj , since ρ is an injective function. Indeed, ρ is assumed to be strictly monotone.

A.2 On identifiability of the Gaussian mixture

Finite Gaussian mixtures are identifiable and, for any variable j, there is a total ordering defined by σ 2 kj > σ 2 (k+1)j and µ kj > µ (k+1)j if σ 2 kj = σ 2 (k+1)j , where µ kj and σ 2 kj are respectively the mean and the variance of variable j under component k. Example A.1 shows that the identifiability holds for Gaussian mixtures when there are missing values and that the distribution of the MNAR mechanism is a probit one.

Example A.1 (Gaussian + Probit). Let us consider that ρ is the probit function and f k (respectively f k+1 ) the Gaussian density with parameters (µ k , σ k ) (respectively (µ k+1 , σ k+1 )). Suppose without loss of generality that β k ≥ β k+1 . One want to prove that

lim u→+∞ E u := lim u→+∞ α k+1 +β k+1 u -∞ e -t 2 /2 dt α k +β k u -∞ e -t 2 /2 dt σ k exp - (u-µ k+1 ) 2 2σ 2 k+1 σ k+1 exp -(u-µ k ) 2 2σ 2 k = 0 Let us denote φ(u) = 1 √ 2π u -∞ e -t 2 /2 dt. One has lim u→+∞ φ(u) =    1 if u > 0 1/2 if u = 0 0 if u < 0 (15)
• If β k+1 > 0 (and β k > 0):

lim u→+∞ E u = lim u→+∞ exp -u 2 1 2σ 2 k+1 - 1 2σ 2 k + u µ k σ k - µ k+1 σ k+1 = 0.
assuming without loss of generality that

σ 2 k > σ 2 k+1 or µ k > µ k+1 if σ 2 k = σ 2 k+1 . • If β k+1 ≤ 0 (and β k ≥ 0): lim u→+∞ E u = 0 since lim u→+∞ exp -u 2 1 2σ 2 k+1 - 1 2σ 2 k + u µ k σ k - µ k+1 σ k+1 = 0 and lim u→+∞ α k+1 +β k+1 u -∞ e -t 2 /2 dt α k +β k u -∞ e -t 2 /2 dt =    0 if β k+1 < 0 1/2 if β k+1 = 0 and β k > 0 1 if β k+1 = 0 and β k = 0. (16)
• If β k+1 < 0 and β k < 0: One uses the upper and lower bounds for the probit function.

1 -t + √ t 2 + 4 < π 2 exp t 2 2 φ(t) < 1 -t + t 2 + 8/π , i.e., φ(t) < 2 π 1 -t+ √ t 2 +8/π exp -t 2 2 and 1 φ(t) < (-t + √ t 2 + 4) π 2 exp t 2 2 Thus, noting that lim u→+∞ φ(α k+1 + β k+1 u) = lim u→+∞ φ(β k+1 u), α k+1 +β k+1 u -∞ e -t 2 /2 dt α k +β k u -∞ e -t 2 /2 dt = u→+∞ φ(β k+1 u) φ(β k u) < u→+∞ exp β 2 k 2 - β 2 k+1 2 u 2 . ( 17
)
As

β k+1 ≤ β k < 0, one has β 2 k /2 -β 2 k+1 /2 < 0 and it implies lim u→+∞ α k+1 +β k+1 u -∞ e -t 2 /2 dt α k +β k u -∞ e -t 2 /2 dt = 0.
Given that

lim u→+∞ exp -u 2 1 2σ 2 k+1 - 1 2σ 2 k + u µ k σ k - µ k+1 σ k+1 = 0, assuming without loss of generality that σ 2 k > σ 2 k+1 or µ k > µ k+1 if σ 2 k = σ 2 k+1 , one has lim u→+∞ E u = 0.
This result has been already stated, in the case of univariate distributions, by [START_REF] Miao | Identifiability of normal and normal mixture models with nonignorable missing data[END_REF]. In particular, the identifiability conditions in [START_REF] Miao | Identifiability of normal and normal mixture models with nonignorable missing data[END_REF] (conditions 1 and 2 of their paper) imply the existence of the total ordering defined in Assumption A2.. However, these conditions exclude the case of Gaussian mixture with a logistic missing-data distribution, which is very used in practice. In Corollary A.2, we therefore extend this result to the multivariate case with a logistic missing-data distribution.

Note first that with a logistic distribution, a total ordering cannot be defined. Indeed, for variable j, such an ordering cannot be defined if the two univariate variances are equal (i.e., σ 2 kj = σ 2 (k+1)j ) and µ kj -β kj -µ (k+1)j + β (k+1)j = 0. However, for the specific case of Gaussian mixture where all the univariate variances are different between the components, then conditions of Proposition 3.1 hold true with a logistic missing-data distribution and so does its identifiability. In addition, for more parsimonious MNAR models for which the effect on the variable j does not depend on the class membership k (i.e., β kj = β (k+1)j ), the conditions of Proposition 3.1 hold true with a logistic missing-data distribution. Finally, as stated by Corollary A.2 below, the condition on the covariance matrices (including the case of homoscedastic Gaussian mixture) can be relaxed to obtain the generic identifiability of the model (i.e., all not-identifiable parameter choices lie within a proper submanifold, and thus form a set of Lebesgue zero measure; [START_REF] Allman | Identifiability of parameters in latent structure models with many observed variables[END_REF]).

Corollary A.2. Assume that K k=1 π k f k (y i ; λ k
) is a multivariate Gaussian mixture, ρ is the logistic function and that the missingness scenario is defined by (3), ( 5) or (8), then, the parameters (π, λ, ψ) of the model given by (2) are generically identifiable up to label swapping, i.e., all not-identifiable parameter choices lie within a proper submanifold, and thus form a set of Lebesgue zero measure.

For the other MNAR models given in (4), ( 6), ( 7), ( 9) and (10), the parameters (π, λ, ψ) of the model given by (2) are identifiable up to label swapping.

Proof of Corollary A.2. We use Proposition 3.1. We fix j. By abuse of notation, α k , β k , µ k and σ k correspond to the parameters α kj , β kj , µ kj and Σ kj of the variable j. Let us first consider the missing scenarios (3), ( 5) and ( 8) for which β k = β k+1 . To obtain the total ordering, we need to prove that

lim u→+∞ E u = (1 + e -α k -β k u )e - (u-µ k+1 ) 2 2σ 2 k+1 (1 + e -α k+1 -β k+1 u )e - (u-µ k ) 2 2σ 2 k σ k σ k+1 = 0. • If σ 2 k > σ 2 k+1 , lim u→+∞ E u = lim u→+∞ exp -1 2 ( 1 σ 2 k+1 -1 σ 2 k )u 2 = 0. • If σ 2 k = σ 2 k+1 , one has lim u→+∞ E u = lim u→+∞ exp ((µ k -β k ) -(µ k+1 -β k+1 ))u = 0 discarding the case where (µ k -β k ) -(µ k+1 -β k+1 ) = 0 and assuming without loss of generality that (µ k -β k ) > (µ k+1 -β k+1 ). The set of nonidenfiable parameters is {µ k , β k , µ k+1 , β k+1 s.t.(µ k -β k ) -(µ k+1 -β k+1 ) = 0} k=1,...,K and is of Lebesque zero measure.
Finally, for the missing scenarios ( 9) and ( 10), note that β k = β k+1 = 0. For the missing scenarios (4), ( 6) and ( 7), one has β k = β k+1 . Following the same reasoning as above, in the case where σ 2 k+1 = σ 2 k+1 , one obtains the set of nonindentifiable parameters such that µ k = µ k+1 , which is empty since

µ k = µ k+1 if σ 2 k = σ 2 k+1 .

A.3 On identifiability of the Poisson mixture

Proposition A.2 can also be applied for variables with integer value (i.e., count data), as shown below in Examples A.3 and A.4 for the Poisson mixture with probit or logistic missing-data distributions.

Example A.3 (Poisson + Probit). Considering that ρ is the probit function and f k (respectively f k+1 ) the Poisson distribution with parameters λ k (respectively λ k+1 ). Suppose without loss of generality that β k > β k+1 and λ k > λ k+1 . One want to prove

lim u→+∞ E u := lim u→+∞ α k+1 +β k+1 u -∞ e -t 2 /2 dt α k +β k u -∞ e -t 2 /2 dt λ u k+1 e -λ k+1 λ u k e -λ k = 0.
• If β k+1 > 0 (and β k > 0): using (15), one has

lim u→+∞ E u = lim u→+∞ exp u log λ k+1 λ k = 0. • If β k+1 ≤ 0 (and β k ≥ 0): one has lim u→+∞ E u = 0. using lim u→+∞ exp u log λ k+1 λ k = 0
and (16) for the missing distribution part.

• If β k+1 < 0 and β k < 0: using (17), one obtains

lim u→+∞ E u < lim u→+∞ exp β 2 k 2 - β 2 k+1 2 u 2 exp u log λ k+1 λ k = 0, because β 2 k /2 -β 2 k+1 /2 < 0.
Example A.4 (Poisson + Logistic). Considering that ρ is the logistic function and f k (respectively f k+1 ) the Poisson distribution with parameters λ k (respectively λ k+1 ). One want to prove that

lim u→+∞ E u = lim u→+∞ 1 + e -α k -β k u 1 + e -α k+1 -β k+1 u exp u log λ k+1 λ k = 0.
Assume that λ k > λ k+1 without loss of generality.

• For the missing scenarios (3), ( 5) and (8) for which β k = β k+1 , one obtains the generic identifiability where the set of non-identifiable parameters is

{α k , β k , λ k s.t.(log λ k -β k ) -(log λ k+1 -β k+1 ) = 0} k=1,.
..,K and is of Lebesque zero measure.

• For the missing scenarios (9) and (10), note that β k = β k+1 = 0. For the missing scenarios (4), ( 6) and ( 7), one has β k = β k+1 . It implies that idenfiability holds since

lim u→+∞ E u = lim u→+∞ exp u log λ k+1 λ k = 0.
A.4 Categorical data Proposition 3.1 states that generic identifiability holds only for the MNARz and the MNARz j missing scenarios and that the other missing scenarios lead to non-identifiable models. The proof uses Corollary 5 of [START_REF] Allman | Identifiability of parameters in latent structure models with many observed variables[END_REF] which gives the identifiability of finite mixtures of Bernoulli products.

Proof of Proposition 3.1, categorical case. Let us first consider the case where β kj = (0, . . . , 0) ∈ R j , ∀k = 1, . . . , K, ∀j = 1, . . . , d. Suppose there exists two sets of parameters {γ, ψ} and {γ , ψ } which have the same observed distribution.

∀y i ∈ R d , ∀c i ∈ {0, 1} d , K k=1 Y mis i π k f k (y i ; λ k ) d j=1 ρ(α kj ) c ij [1 -ρ(α kj )] 1-c ij dy mis = K k=1 Y mis i π k f k (y i ; λ k ) d j=1 ρ((α ) kj ) c ij [1 -ρ(α kj )] 1-c ij dy mis .
Let us consider the case where all the elements of y i are observed, i.e., c ij = 0, ∀j = 1, . . . , d. One has

K k=1 π k f k (y i ; λ k ) d j=1 (1 -ρ(α kj )) = K k=1 π k f k (y i ; λ k ) d j=1
(1 -ρ(α kj )),

i.e., by independence to the group membership,

K k=1 π k d j=1 f kj (y ij ; λ kj )(1 -ρ(α kj )) = K k=1 π k d j=1 f kj (y ij ; λ kj )(1 -ρ(α kj )), ⇔ K k=1 π k d j=1 (1 -ρ(α kj )) 1-c ij j h=1 (λ h kj ) y h ij = K k=1 π k d j=1 (1 -ρ(α kj )) 1-c ij j h=1 ((λ kj ) h ) y h ij .
We recognize the finite mixture of K multinomial distributions with d components for w ij = ((1 -c ij ), y ij ), j = 1, . . . , d with paramaters (λ kj ) = ((1 -ρ(α kj )), λ 1 kj , . . . , λ j kj ), j = 1, . . . , d and proportions π k . We can thus apply Theorem 4 [START_REF] Allman | Identifiability of parameters in latent structure models with many observed variables[END_REF] with the model M(K; 1 , . . . , d ) which gives the generic identifiability of the model paramaters up to a label swapping, i.e., ∀k, ∀j,

λ h kj = (λ kj ) h ∀k, ∀j, ρ(α kj ) = ρ(α kj ) ∀k, π k = π k
As the function ρ is strictly monotone, the equality ρ(α kj ) = ρ(α kj ) implies α kj = α kj . In addition, if K = K , say

K > K , K k=K +1 π k d j=1 (1 -ρ(α kj )) j h=1 (λ h kj ) y h ij = 0 implies π k = 0 for K + 1 ≤ k ≤ K.
We consider now the missing scenarios for which β kj = 0. The identifiability does not hold. We can present a counter-example. The set of parameters ψ = {α = (1, . . . , 1), β = (1, . . . , 1)} has the same observed distribution than another set of parameters ψ = {α = (0, . . . , 0), β = (2, . . . , 2)}. Indeed, in the case where

y ij = (1, . . . , 1), ρ(α kj + β kj y ij ) = ρ(α kj + β kj y ij ).

B Appendix 2: Details on EM algorithm

The EM algorithm consists on two steps iteratively proceeded: the E-step and M-step. For the E-step, one has

Q(θ; θ [r-1] ) = E[ comp (θ; y, z, c)|y obs i , c i ; θ [r-1] ], where comp (θ; Y, Z, C) = n i=1 K k=1 z ik log (π k f k (y i ; λ k )f k (c i | y i ; ψ k )) .
It leads to the decomposition

Q(θ; θ [r-1] ) = n i=1 K k=1 t ik (θ [r-1] ) log(π k ) + τ y (λ k ; y obs i , c i , θ [r-1] ) + τ c (ψ k ; y obs i , c i , θ [r-1] ) , with t ik (θ [r-1] ) = f (z ik = 1 | y obs i , c i ; θ [r-1]
). The terms involved in this decomposition are now detailed. (a) the expectation of the data mixture part over the missing values given the available information (i.e., the observed data and the indicator pattern), the class membership and the current value of the parameters:

τ y (λ k ; y obs i , c i , θ [r-1] ) = E θ [r-1] log f k (y i ; λ k ) | y obs i , z ik = 1, c i , (b
) the expectation of the missing mechanism part over the missing values given the available information, the class membership and the current value of the parameters:

τ c (ψ k ; y obs i , c i , θ [r-1] ) = E θ [r-1] log f k (c i | y i ; ψ k ) | y obs i , z ik = 1, c i .
(c) the conditional probability for an observation i to belong to the class k given the available information and the current value of the parameters:

t ik (θ [r-1] ) = f (z ik = 1 | y obs i , c i ; θ [r-1] ).
Terms (a) and (b) require to integrate over the distribution f (y mis i | y obs i , z ik = 1, c i ; θ [r-1] ). Term (c) corresponds to the conditional probability for an observation i to arise from the kth mixture component with the current values of the model parameter. More particularly, one has

t ik (θ [r-1] ) = f (z ik = 1, y obs i , c i ; θ [r-1] ) f (y obs i , c i ; θ [r-1] ) = π [r-1] k f k (y obs i ; λ [r-1] k )f (c i | y obs i , z ik = 1; θ [r-1] ) K h=1 π [r-1] h f h (y obs i ; λ [r-1] h )f (c i | y obs i , z ih = 1; θ [r-1] ) (18)
The quantity that can cause numerical difficulties is the probability f (c i | y obs i , z ik = 1; θ [r-1] ).

Computations for the MNARz j model For the MNARz and MNARz j models, the effect of the missingness is only due to the class membership. The EM algorithm for the MNARz model is detailed in the main paper [START_REF] Sportisse | Model-based clustering with missing not at random data[END_REF]. Term (a) is the same for both MNARz and MNARz j models but (b) and (c) differ. For Term

(b), f (c i | y i , z ik = 1; ψ) is independent of y i , which implies log(f (c i | z ik = 1; ψ)) = d j=1 c ij log ρ(α kj ) + (1 -c ij ) log(1 -ρ(α kj )). (19)
For Term (c), one remark that

P(c i | y obs i , z ik = 1; θ [r-1] ) = d j=1 P(c ij = 1 | y obs i , z ik = 1; θ [r-1] ) c ij P(c ij = 0 | y obs i , z ik = 1; θ [r-1] ) 1-c ij (MNARz j ).
and that, by independence of y i , one has

P(c ij = 1 | y obs i , z ik = 1; θ [r-1] ) = P(c ij = 1 | z ik = 1; θ [r-1] ) = ρ(α kj ).
Using (18), one obtains

t [r-1] ik (θ [r-1] ) = π [r-1] k
φ(y obs i ; (µ obs ik ) [r-1] , (Σ obs,obs ik

) [r-1] ) d j=1 ρ(α [r-1] kj ) c ij (1 -ρ(α [r-1] kj )) 1-c ij K h=1 π [r-1] h
φ(y obs i ; (µ obs ih ) [r-1] , (Σ obs,obs ih

) [r-1] ) d j=1 ρ(α [r-1] hj ) c ij (1 -ρ(α [r-1]
hj )) 1-c ij (MNARz j ).

(20) The E-step is derived in the same way as for the MNARz model with these terms. The M-step for ψ consists of performing a GLM with a binomial link for the matrices (J MNARz j kj )

[r] j=1,...,d and by giving t ik (θ [r-1] ) as prior weights to fit the process.

(J MNARz j kj ) [r] = c .j 1 (21)

• ρ is logistic: Equation ( 22 φ(y mis i ; (μ mis ik ) [r-1] , ( Σmis ik ) [r-1] ). , ( Σmis ik )

[r-1] jj

)dy mis ij .

C Appendix 3: Details on SEM algorithm

The SEM algorithm consists on two steps iteratively proceeded as presented in Section 4. The key issue is to draw the missing data (y mis i ) r and z r i according to their current conditional distribution f (y mis i , z i | y obs i , c i ; π [r-1] , λ [r-1] , ψ [r-1] ). By convenience, we use a Gibbs sampling and simulate two easier probabilities recalled here

z [r] i ∼ f (• | y [r-1] i
, c i ; π [r-1] , λ [r-1] , ψ [r-1] ) and (y mis i ) [r] ∼ f (• | y obs i , z r i , c i ; λ [r-1] , ψ [r-1] ),

Figure 1 :

 1 Figure 1: Boxplot of the ARI obtained for 50 samples composed of d = 6 variables.The sample size varies by {100,250,500}. The boxplots in green (True) correspond to the performance of the algorithm considering the MNAR setting matching the one that has been used for the missing value generation. The red dashed line indicates the theoretical ARI. This experiment been performed for a theoretical rate of misclassification of 10% and a theoretical missing rate in the whole dataset of 30%.

  ) leads to none classical distribution becausef (y mis i | y obs i , z ik = 1, c i ; θ [r-1] )

  Term (b) isτ c (ψ k ; y obs i , c i , θ [r-1] ) exp(-(α kj + β kj y mis ij )-c ij ) log(1 + exp(α kj + β kj y obs ij )),which amounts to compute the Gaussian moment of log(1+exp(-u)) 1+exp(-u) , but it has no closed form to our knowledge. Finally, Equation (23) does not have a closed form either because it requires the computation of ., the computation of the Gaussian moment of 1 1+exp(-u) .• ρ is Probit: One can prove (presented in Appended C) that the conditional distribution (y mis i | y obs i , z ik = 1, c i ) is a truncated Gaussian, which makes possible the computation of Term (a). Term (b) has no closed form to our knowledgeτ c (ψ k ; y obs i , c i , θ [r-1] ) c ij ) log 1 -α kj +β kj y obs ij -∞ e -t 2 dt ,Equation (23) does not have a closed form either because it requires the computation of

  Choice of the values of δ, α and β for all the experiments of Section 5 for the MNARyz j mechanism.

Table 1 :

 1 Table1summarizes the cases for which the EM or SEM algorithm is feasible. Summary of the cases for which the EM and the SEM lead to feasible (or not feasible) computations. The symbol means that the computations are feasible (derived in Appendix B).

			EM			SEM
		Gaussian	Categorical	Gaussian	Categorical
	MNARz
	MNARz j
		Probit	Logit		Probit	Logit
	MNARy	no closed form	no closed form, optim. pb	not identifiable		require (costly) algorithms as SIR	not identifiable

Intractability of the EM algorithm for MNARy models For missing scenarios which model the effect of the missingness depending on the variable, the computations are more difficult.

• Because of the dependence of y, f (y mis i | y obs i , z ik = 1, c i ; θ [r-1] ) = f (y mis i | y obs i , z ik = 1; θ [r-1] ) does not hold anymore. Here, one has

which implies that Term (a) requires difficult computations if this distribution is not classical. • For Term (b), it is the same problem, since f (c i | y i , z ik = 1; ψ) is no longer independent of y, then it requires a specific numerical integration. Using (22), τc(ψ k ; y obs i , ci, θ [r-1] ) =E log

where

Therefore,

• There is no closed-form expression for Term (c).

Using ( 18), the probabilities t ik (θ [r-1] ) can be deduced from Equation ( 23).

Let us detail the difficulties for two particular cases, if ρ is logistic or probit.

where y [r-1] i = (y obs i , (y mis i ) [r-1] ). For the latter distribution, the membership k of z

[r]

i is drawn from the multinomial distribution with probabilities (P( r-1] , ψ [r-1] )) k=1,...,K detailed as follows

The conditional distribution of ((y mis i ) [r] | y obs i , z

[r] ik = 1, c i ) has already been detailed in Equation ( 22) and recalled here

Gaussian mixture for continuous data First note that the probabilities of the multinomial distribution for drawing z

[r] i

given in (25) can be easily computed for all cases.

) is assumed to be a Gaussian distribution with mean vector µ k and covariance matrix Σ k , and

) is specified depending the MNAR model and the distribution ρ. The only difficulty of the SE-step is thus to draw from the distribution (y

In the sequel, we detail the distribution (y

and the M-step for ψ depending the MNAR model. For MNARy models, the conditional distribution (y mis i | y obs i , z

[r] ik = 1, c i ) depends on the distribution ρ at hand. For the MNARy models, we will consider two classical distributions for ρ: the logistic function and probit one.

Logistic distribution: For the logistic function, the distribution given in ( 26) is not classical and drawing y mis i from it seems complicated. Indeed, one has

where (μ mis ik ) [r-1] and ( Σmis ik ) [r-1] are given in ( 33) and ( 34). We could use the Sampling Importance Resampling (SIR) algorithm which simulates a realization of (y mis i | y obs i , z

[r] ik = 1, c i ) with a known instrumental distribution (for example:

) and includes a re-sampling step. However, this algorithm may be computationnaly costly.

Probit distribution: For the probit function, the distribution in ( 26) can be made explicit by using a latent variable L i .

More particularly, let L i such that

Then, c i can be viewed as an indicator for whether this latent variable is positive, i.e., for all j = 1, . . . , d,

Thus, indeed to draw (y mis i ) [r] and z

[r]

i , (y mis i ) [r] and z

[r] -1] by using a Gibbs sampling. First, we have to draw

where we use that L

[r]

i is a function of y mis i , y obs i , z ik = 1 in step (i).

Step (ii) is obtained by using (27). By abuse of notation, {c i = 1} ∩ {L [r] i > 0} means that for all j = 1, . . . , d,

) is a multivariate truncated Gaussian distribution denoted as N t , as detailed here

with a ∈ R d and b ∈ R d the lower and upper bounds such that for all j = 1, . . . , d,

Secondly, we draw the membership k of z

[r]

i from the multinomial distribution with probabilities, for all k = 1, . . . , K detailed as follows

) is only the density of the multivariate truncated Gaussian distribution described in (28) evaluated in L

). Yet, one has

with (μ mis ik ) [r-1] and ( Σmis ik ) [r-1] the standard expression of the mean vector and covariance matrix of a conditional Gaussian distribution (see for instance [START_REF] Anderson | An Introduction to Multivariate Statistical sAnalysis[END_REF]). In particular, one has in this case:

with:

Finally combining the two equations ( 30) and (31) one obtains

where

with (β mis k ) [r-1] (resp. (L mis i ) [r] and (α mis k ) [r-1] ) the vector β k (resp. (L i ) [r] and (α k ) [r-1] ) restricted to the coordinates j ∈ Y mis i . Finally, for fully describing the SEM-algorithm, in the M-step, ψ [r-1] is computed using a GLM with a binomial link function for a matrix depending on the MNAR model. In particular,

• For MNARy, the coefficient obtained with a GLM for the matrix (H MNARy j ) [r] are α 0 and β

• For MNARy k , the coefficient obtained with a GLM for the matrix (H MNARy k kj ) [r] is α 0 and β

• For MNARyz, the coefficients obtained with a GLM for the matrix (H MNARyz ) [r] are

• For MNARyz j , the coefficients obtained with a GLM for the matrix (H MNARyz j j ) [r] are β

Kj , with

• For MNARy k z, the coefficients obtained with a GLM for the matrix (H MNARy k z k ) [r] are β k , with

• For MNARy k z j , the coefficients obtained with a GLM for the matrix (H MNARy k z j kj ) [r] are β kj , α kj , with

When ρ is the probit function, the SEM algorithm can be derived, see Algorithm 1.

Remark C.1 (SEM for MNARz and MNARz j mechanisms). A SEM algorithm can also be derived for these two mechanisms. For continuous data, we can prove that

and that this conditional distribution is Gaussian (the SE step is then just a draw from this law). The M-step for ψ are given in the following.

Algorithm 1 SEM algorithm for Gaussian mixture, MNARy models, ρ is probit

. . , K}. for r = 0 to r max do SE-step: for i = 1 to n do Draw (L i ) [r] from the multivariate truncated Gaussian distribution given in (28). Draw z

[r]

i from the multinomial distribution with probabilities detailed in (29). Draw (y mis i ) [r] from the multivariate Gaussian distribution given in (35). end for Let Y

k be the proportion of rows of Y [r] belonging class k. Let µ

k be the mean and covariance matrix of rows of Y [r] belonging to class k. Let ψ

[r] k be the resulted coefficients of a GLM with a binomial link function, i.e., the optimization problem is ∀j ∈ {1, . . . , d},

for a matrix M kj depending on the MNAR model (see ( 36), ( 37), ( 38), ( 43), ( 41) and ( 42)) and c .j the missing data pattern for the variable j. end for end for

• For MNARz, the coefficients obtained with a GLM for the matrix (H MNARz ) [r] are α 1 , . . . , α K , with

• For MNARz j , the coefficients obtained with a GLM for the matrix (H MNARz j j ) [r] are α 1j , . . . , α Kj , with (H MNARz j j ) [r] = c .j z ik = 1). This latter is a multinomial distribution with probabilities (λ kj ) =1,..., j .

D Appendix 4: Complements on the numerical experiment

This section gives the values of δ (see ( 14)) ψ (see (3)) and ϕ (see ( 14)) used during the numerical experiments. For most MNAR models, he values of these parameters are available in the main paper [START_REF] Sportisse | Model-based clustering with missing not at random data[END_REF].