Alexandra-Mihaela Olteanu
email: alexandramihaela.olteanu@epfl.ch

Kévin Huguenin
email: kevin.huguenin@epfl.ch

Reza Shokri
email: reza.shokri@inf.ethz.ch

Jean-Pierre Hubaux
email: jean-pierre.hubaux@epfl.ch

Quantifying the Effect of Co-Location Information on Location Privacy

Keywords: Location privacy, co-location, statistical inference, social networks

Mobile users increasingly report their co-locations with other users, in addition to revealing their locations to online services. For instance, they tag the names of the friends they are with, in the messages and in the pictures they post on social networking websites. Combined with (possibly obfuscated) location information, such co-locations can be used to improve the inference of the users' locations, thus further threatening their location privacy: as co-location information is taken into account, not only a user's reported locations and mobility patterns can be used to localize her, but also those of her friends (and the friends of their friends and so on). In this paper, we study this problem by quantifying the effect of co-location information on location privacy, with respect to an adversary such as a social network operator that has access to such information. We formalize the problem and derive an optimal inference algorithm that incorporates such co-location information, yet at the cost of high complexity. We propose two polynomial-time approximate inference algorithms and we extensively evaluate their performance on a real dataset. Our experimental results show that, even in the case where the adversary considers co-locations with only a single friend of the targeted user, the location privacy of the user is decreased by up to 75% in a typical setting. Even in the case where a user does not disclose any location information, her privacy can decrease by up to 16% due to the information reported by other users.

Introduction

Increasingly popular GPS-equipped mobile devices with Internet connectivity allow users to enjoy a wide range of online location-based services while on the go. For instance, mobile users can search for nearby points of interest and get directions, possibly in real time, to their destinations. Location-based services raise serious privacy concerns as a large amount of personal information can be inferred from a user's whereabouts. The research community has extensively (a) A user reports being in a given area, and a second user reports being in another (overlapping) area and that she is co-located with the first user. By combining these pieces of information, an adversary can deduce that both users are located in the intersection of the two areas, thus narrowing down the set of possible locations for both of them. (b) Two users (initially apart from each other, at 10am) declare their exact individual location. Later (at 11am), they meet and report their co-location without mentioning where they are. By combining these pieces of information, the adversary can infer that they are at a place that is reachable from both of the initially reported locations in the amount of time elapsed between the two reports.

studied the problem of location privacy; more specifically, location-privacy protection mechanisms (so-called LPPMs), that can anonymize and obfuscate the users' locations before sending them to online location-based services, have been proposed [START_REF] Krumm | A survey of computational location privacy[END_REF]. In addition, formal frameworks to quantify location privacy in the case where users disclose their (possibly obfuscated) locations have been proposed [START_REF] Shokri | Quantifying location privacy: the case of sporadic location exposure[END_REF][START_REF] Shokri | Quantifying location privacy[END_REF]. In such frameworks, the mobility profiles of the users play an important role in the inference of the users' locations, namely in a localization attack.

In parallel, social networks have become immensely popular. Every day, millions of users post information, including their locations, about themselves, but also about their friends. An emerging trend, which is the focus of this paper, is to report co-locations with other users on social networks, e.g., by tagging friends on pictures they upload or in the messages they post. Our preliminary survey involving 132 Foursquare users, recruited through Amazon Mechanical Turk, reveals that 55.3% of the participants do report co-locations in their check-ins and that for the users who do so, on average, 2.84%±0.06 of their check-ins do contain co-location information. In fact, co-location information can be obtained in many different ways, such as automatic face recognition on pictures (which can contain the time and location at which the picture was taken in their EXIF data), Bluetooth-enabled device sniffing and reporting neighboring devices. Similarly, users who connect from the same IP address are likely to be attached to the same Internet access point, thus providing evidence of their co-location.

Attacks exploiting both location and co-location information (as mentioned in [START_REF] Vicente | Location-related privacy in geosocial networks[END_REF]) can be quite powerful, as we show in this paper. Figure 1 depicts and describes two example situations in which co-location can improve the performance of a localization attack, thus degrading the location-privacy of the users involved. At the same time, it is clear that the proper exploitation of such information by an attacker can be complex because he has to consider jointly the (co-)location information collected about a potentially large number of users.

This family of attacks and their complexity is precisely the focus of this paper. More specifically, we make the following three contributions. [START_REF] Baum | Statistical inference for probabilistic functions of finite state markov chains[END_REF] We identify and formalize the localization problem with co-location information, we propose an optimal inference algorithm and analyze its complexity. We show that, in practice, the optimal inference algorithm is intractable due to the explosion of the state space size. [START_REF]How the NSA is tracking people right now[END_REF] We describe how an attacker can drastically reduce the computational complexity of the attack by means of well-chosen approximations. We present two polynomial-time heuristics, the first being based on a limited set of considered users and the second relying on an independence approximation.

(3) We extensively evaluate and compare the performance of these two heuristics in different scenarios, with different settings, based on a mobility dataset. Our experimental results show that, even in the case where the adversary considers co-locations with only a single friend of the targeted user, the median location privacy of the user is decreased by up to 75% in a typical setting. Even in the case where a user does not disclose any location information, her privacy can decrease by up to 16% due to the information reported by other users. A paramount finding of our work is that users partially lose control over their location privacy as co-locations and individual location information disclosed by other users substantially affect their own location privacy. To the best of our knowledge, this is the first work to quantify the effects of co-location information, that stems from social relationships, on location privacy; thus making a connection between privacy implications of social networks and location privacy.

The remainder of the paper is organized as follows. In Section 2, we define and formalize the system model. In Section 3, we present the optimal localization attack for N users and assess its complexity. In Section 4, we show how this complexity can be reduced by means of approximations. In Section 5, we report on the experimental evaluation of the localization attack with co-locations. In Section 6, we survey the related work. In Section 7, we conclude the paper and suggest directions for the future work.

System Model and Formalization

We consider a set of mobile users who move in a given geographical area. While on the go, users make use of some online services to which they communicate potentially obfuscated location (i.e., where they are) and accurate co-location information (i.e., who they are with). We consider that a curious service provider (referred to as the adversary) wants to infer the location of the users from this information, hence tracking them over time. In order to carry out the inference attack, based on which the location privacy of the users is evaluated, the adversary would model the users as described below. Our model is built upon [START_REF] Shokri | Quantifying location privacy[END_REF] and uses similar notations. Figure 2 gives an overview of the considered scenario. Fig. 2. Scenario of (co-)location exposure. Three users move in a given geographical area. They communicate their potentially obfuscated locations and accurate co-location information to a service provider (i.e., the adversary) who wants to infer their locations.

Users

We consider a set U = {u 1 , . . . , u N } of N mobile users who move within a given geographical area that is partitioned into M regions (locations) R = {R 1 , . . . , R M }. Time is discrete and we consider the state of the system (including the locations of the users) at the successive time instants {1, . . . , T }.

The region in which a user u ∈ U is at time instant t ∈ {1, . . . , T } is called the actual location of the user and is denoted by a u (t). The mobility of the users is modeled by a first order time-homogeneous Markov chain. We denote by p u (ρ, r) the probability that user u moves from region ρ to region r during one time instant, and by π u (r) the probability that user u is in region r at time t (i.e., the stationary distribution of p u ). We call a co-location the fact that two users are at the same location at some point in time. The fact that users u and v are co-located at time t means that a u (t) = a v (t); we denote by u ↔ t v the fact that a co-location between users u and v at time t is reported, and we denote by c u (t) the set of all reported co-locations that involve user u at time t. We define C t = u∈U c u (t) and C = t=1..T C t .

Location-Privacy Protection Mechanisms

In order to protect their privacy, users rely on location-privacy protection mechanisms (LPPM) for obfuscating their individual location information before they communicate it to an online service provider. We denote by u @ t r ′ the fact that user u reports being at location r ′ at time t to the online service. The online service observes only the obfuscated location of the users, which we denote by o u (t) for a user u at time t. We denote by R ′ the set of obfuscated locations; typically R ′ is the power set of R, as LPPMs can return a set of locations instead of a single one. Typical LPPMs replace the actual location of a user with another location (i.e., adding noise to the actual location) or merge several regions (i.e., reducing the granularity of the reported location). We model an LPPM by a function that maps a user's actual location to a random variable that takes values in R ′ , that is, the user's obfuscated location. This means that the locations of a user at different time instants are obfuscated independently from each other and from those of other users. Formally, an LPPM is defined by the function f u (r, r ′ ) which denotes the probability that the LPPM used by u obfuscates location r to r ′ , i.e., Pr (o u (t) = r ′ | a u (t) = r). Let alone the co-location information, our model corresponds to a hidden Markov model (HMM) [START_REF] Baum | Statistical inference for probabilistic functions of finite state markov chains[END_REF]. We assume that co-location information is not obfuscated and users do not rely on pseudonyms. 1 We denote by o(t) the vector of the observed locations of all the users at time t. More generally, we use bold notations to denote a vector of values of all users.

Adversary

The adversary, typically an online service provider (or an external observer who has access to this information, e.g., another user of the social network), has access to the observed locations and co-locations of one or several users and seeks to locate users, at a given time instant, namely carry out a localization attack.

Because the locations of the users are not independent, given the co-location information, when attacking the location of a given user, the adversary takes into account information potentially about all the users. The attack is performed a posteriori, meaning that the adversary has access to the observed traces over the complete period, namely {o(t)} t=1..T and C, at the time of the attack. In addition to the observations during the time period of interest (i.e., {1, . . . , T }), the adversary has access to some of the users' past location traces, from which he builds individual mobility profiles for these users, under the form of transition probabilities {p u } u∈U . See [START_REF] Shokri | Quantifying location privacy[END_REF] for more details about the knowledge construction, in particular on how the mobility profiles can be built from obfuscated traces with missing locations. The mobility profiles constitute, together with the knowledge of the LPPMs used by the users (including their parameters), the adversary's background knowledge

K = {p u , f u } u∈U .
The output of a localization attack that targets a user u at a time instant t, is a posterior probability distribution over the set R of locations.

h u t (r) Pr (a u (t) = r | {o(t)} t=1..T , C, K) . (1)

Location Privacy Metric

The location privacy LP u (t) of a user u at time t, with respect to a given adversary, is captured by the expected error of the adversary when performing a localization attack [START_REF] Shokri | Quantifying location privacy[END_REF]. Given the output h u t (•) of the localization attack, the location privacy writes:

LP u (t) r∈R h u t (r) • d(r, a u (t)) , (2)
where d(• , •) denotes a distance function on the set R of regions, typically the Haversine distance between the centers of the two regions.

Optimal Localization Attack

Without co-location information (as in [START_REF] Shokri | Quantifying location privacy[END_REF]) and under the assumptions described in the previous section, the localization problem translates to solving a HMM inference problem, for which the forward-backward algorithm is a known solution. Essentially, the forward-backward algorithm defines forward and backward variables that take into account the observations before and after time t, respectively. The forward variable is the joint probability of location of user at time t and all the observations up to, and including, time t. The backward variable is the conditional probability of all observations after time t, given the actual location of user at that time instant. Then, the posterior probability distribution of the possible locations for the targeted user is obtained by combining (i.e., multiplying and normalizing) the forward and backward variables. With co-location information, the locations of the users are not mutually independent: as soon as two users are co-located at some point in time t, their locations, before and after time t, become dependent. Actually, the fact that two users meet a same third user (even if they meet her at different time instants) suffices to create some dependencies between their locations; this means that, to perform the localization attack on a user, the adversary must take into account the locations (i.e., the obfuscated location information and the co-location information) of all the users who are connected to u by a chain of co-location (i.e., the connected component of u in the co-location graph). Formally speaking, it means that the adversary cannot rely only on the marginal distributions of the users' location; instead he must consider the joint distributions. In other words, colocations turn N disjoint inference problems (i.e., HMM problems solved by the forward-backward algorithm) into a joint inference problem.

To solve the localization problem, we consider the users jointly; we show that it translates to an HMM problem that we solve using a forward-backward algorithm. For a set U of users and a time t, we define the following forward and backward variables:

α U t (r) Pr (o(1) . . . , o(t), C 1 , . . . , C t , a(t) = r | K) (3)
β U t (r) Pr (o(t + 1) . . . , o(T ), C t+1 , . . . , C T | a(t) = r, K) , (4)
where r denotes a vector of size N ,i.e., r ∈ R N , and represents the actual locations of all users at a single time instant. These variables can be defined recursively (over t) and, unlike in the case where no co-location observations are available, their expressions involve the co-location information. More specifically, it can be proved that for all r ∈ R N , we have2

α U t (r) =      π U (r) if t = 0 1 t (r, C) C ′ 1 t (r, C ′ ) • f U (r, o(t)) ρ∈R N α U t-1 (ρ) • p U (ρ, r) if t > 0 (5)
and and1(• , •) is the indicator function that returns 1 if the locations of the users are consistent with the co-location information reported at time t, and 0 otherwise. That is, formally,

β U t (r) =      ρ∈R N 1 t (ρ, C) C ′ 1 t (ρ, C ′ ) • β U t+1 (ρ) • p U (r, ρ) • f U (ρ, o(t + 1)) if t < T 1 if t = T (6) where for r = (r 1 , . . . , r N ) ∈ R N , ρ = (ρ 1 , . . . , ρ N ) ∈ R N , r ′ = (r ′ 1 , . . . , r ′ N ) ∈ R ′ N , π U (r) = N i=1 π ui (r i ), f U (r, r ′ ) = N i=1 f ui (r i , r ′ i ) , p U (ρ, r) = N i=1 p ui (ρ i , r i ),
1 t (r, C) = 1 if ∀(u i ↔ t u j ) ∈ C t , r i = r j 0 otherwise . (7)
In other words, the indicator function captures whether the users for which a co-location was reported are indeed at the same locations in r. As the adversary has no knowledge about the way co-locations are reported, the distribution of the sets of reported co-locations, given the actual locations of the users, is modeled with a uniform distribution.

The intuition behind Equation ( 5) is that the forward variable at time t can be expressed recursively, with respect to time, by combining, for all possible locations of the users at time t -1: (1) the joint probability that the users were at location ρ at time t -1 and reported the obfuscated locations observed by the adversary up to time t -1 (this is captured by α U t-1 ), (2) the joint probability that the users move from the locations ρ to the locations r (this is captured by p U ), (3) the joint probability that the users obfuscate their locations r to that observed by the adversary o(t) (this is captured by f U ) and that the locations r of the users are consistent with the co-locations reported at time t. Because users obfuscate their locations independently from each other, the joint obfuscation probability is the product of the individual obfuscation probabilities (hence the expression of f U ). The same applies to p U . The same intuition lies behind Equation [START_REF] Dey | Estimating age privacy leakage in online social networks[END_REF].

The indicator function 1 t (• , •) accounts for the co-location information in the localization attack by ruling out the impossible (i.e., inconsistent with the reported co-locations) user locations, hence further narrowing down the set of possible locations for the users involved in a co-location. Schematically speaking (with a deterministic vision, for the sake of clarity), the set of possible locations for a user u i (at time t), co-located with a user u j , consists of the locations that can be obfuscated into the location reported by u i at time t and that can be reached (according to u i 's mobility profile) from a possible location of u i at time t -1 and that can be obfuscated into the location reported by u j at time t and that can be reached (according to u j 's mobility profile) from a possible location of u j at time t -1.

Finally, the posterior probability distribution of the users' locations can be computed based on the forward and backward variables, by using the following formula, for u i ∈ U and at time t:

h ui t (r) = Pr (a ui (t) = r | {o(t)} t=1..T , C, K) = r∈R N | ri=r α U t (r) • β U t (r) r∈R N α U t (r) • β U t (r)
.

We now evaluate the complexity of the joint localization attack. The first observation is that the size of the state space (i.e., the locations of all users) is M N . To attack a user at time t, the adversary needs to compute the values of α up to time t and the values of beta down to time t. 3 At each time instant, the adversary needs to compute the values of these two variables for all possible values of their inputs r ∈ R N (there are M N possible values for r). The computation of each of these values requires summing over the M N possible locations ρ at time t -1; for each of the possible locations, the computation of one element of the sum takes Θ(N ) operations. Therefore, the computation of the forward and backward variables, at all time instants, for all possible values of the localizations is Θ(N T M 2N ) operations. Note that the complexity is the same whether the adversary attacks one or all the users at one or all time instants. In fact, the adversary can pre-compute the h u t for all u and all t with a complexity that is dominated by that of the computations of the forward and backward variables. In summary, the complexity of the localization attack on one or all of the users in U is

c opt (N, T, M ) = Θ(N T M 2N ) . ( 9
)
The complexity of the optimal localization attack is prohibitively high and prevents its use for the entire set of users of a mobile social network; the optimal localization attack is tractable only for small values of N , i.e., 2 and 3. In the next section, we propose heuristics for performing low-complexity approximate localization attacks.

We propose two low-complexity heuristics for performing approximate localization attacks. Essentially, the first selects a small set of users to consider when attacking a target user and performs an optimal joint localization attack on this small set of users (i.e., considering only the co-locations between these users). The intuition behind this heuristic is that the locations of a user are significantly correlated with those of only a limited number of users (e.g., a few co-workers during work hours, and her family and close friends the rest of the time). The second makes use of individual forward-backward variables (one for each user of the entire set of users) and computes their values at each time instant, based on the considered user's individual variable at time t -1 and the reported locations of the users co-located with her at time t, hence disregarding the dependencies stemming from past co-locations. The intuition behind this heuristic is that the dependency between two users' locations fades relatively quickly over time after they meet.

Heuristic 1: Limited User Set Approximation

As discussed in Section 3, the optimal localization attack can be efficiently performed only on small sets of users. This is because location of a target user u depends on locations of all other users that are connected to u in the co-location graph (where there is an edge between two users u and v if u ↔ t v for some time t). The rationale of our first approximation is to limit the number of users, on which the target user's location depends, and to consider only those that have high location correlation with u. Concretely, we choose the user(s) that have the largest number of reported co-locations with the targeted user and we perform an optimal localization attack on the resulting set of users. We call these users the co-targets of the targeted user. Depending on his computational power, the adversary can choose one or two such users (i.e., N = 2 or N = 3) to attack the target with. The co-targets of a user u are chosen as follows:

co-target 1 (u) argmax v∈U \{u} |{t ∈ {1, . . . , T } | u ↔ t v}| (10) co-target 2 (u) argmax v∈U \{u,u ′ } |{t ∈ {1, . . . , T } | u ↔ t v}| + |{t ∈ {1, . . . , T } | u ′ ↔ t v}| (11)
where u ′ = co-target 1 (u) and | • | denotes the cardinality of the set. More specifically, the first co-target of a user u is the user with whom u has the more reported co-locations during the time interval considered for the localization attack. The second co-target of u is chosen so as to maximize the number of co-locations with u plus the number of co-locations with u's first co-target. Note that the set of considered users can be different for every targeted user; in particular v = co-target 1 (u) / =⇒ u = co-target 1 (v). The complexity of this heuristic is Θ(T M 4 ) for N = 2 and Θ(T M 6 ) for N = 3 (obtained by replacing N by its value in the generic expression [START_REF] Golle | On the anonymity of home/work location pairs[END_REF] of the complexity of the optimal attack).

Heuristic 2: Independence Approximation

As discussed in Section 3, the need to jointly consider the locations of all the users, which cause the explosion of the state space size and thus the high complexity of the attack, stems from the fact that their locations are not independent as soon as co-locations are reported. The rationale behind our second heuristic is to ignore the mobility profiles of the co-located users, hence alleviating the need to take into account their past locations, which causes the state space explosion) and to consider only their reported co-locations to improve the inference of the target user's location at the considered time instant. This comes down to considering the locations reported by the users co-located with u, as if u had reported these obfuscated locations herself (as depicted in Figure 1a). We define individual forward and backward variables for each user and we couple them upon co-locations, as follows:

αu t (r)      π u (r) if t = 0 u ′ |u↔tu ′ f u ′ (r, o u ′ (t)) • f u (r, o u (t)) • ρ∈R αu t-1 (ρ) p u (ρ, r) otherwise (12)
and βu t (r)

     1 if t = T ρ∈R βu t+1 (ρ) p u (r, ρ) f u (ρ, o u (t + 1)) u ′ |u↔t+1u ′ f u ′ (ρ, o u ′ (t + 1)) otherwise (13
) Finally, when performing a localization attack on user u, the posterior distributions of the locations of the users co-located with u at time t are taken into account. More specifically, we estimate the probability distribution of user u's location at time t by ĥu t (r)

αu t (r) • βu t (r) • u ′ |u↔tu ′ αu ′ t (r) βu ′ t (r) r ′ ∈R   αu t (r ′ ) βu t (r ′ ) u ′ |u↔tu ′ αu ′ t (r ′ ) βu ′ t (r ′ )   . ( 14
)
We now compute the complexity of this heuristic. To perform a localization attack on a user, the adversary needs to compute the individual variables of all the users that are connected to the target by a chain of co-location, that is N users at most. The computation of a value α and β (for a given t and a given r), in the worst case (i.e., when all the users are co-located), takes Θ(N M ) operations; and T M such values need be computed for each user. Therefore, the complexity of this heuristic is Θ(N 2 T M 2 ).

Experimental Evaluation

We evaluate the effect of co-locations on users' location privacy, with respect to the various localization attacks presented in the previous sections, by using a dataset of real mobility traces.

Dataset, Methodology and Experimental Setup

The dataset was collected by Microsoft Research Asia, in the framework of the GeoLife project [START_REF] Zheng | Learning transportation mode from raw GPS data for geographic applications on the web[END_REF]. It comprises the GPS traces (i.e., sequences of timestamped latitude-longitude couples, sampled at a rate of one point every 1-5 seconds) of 182 users, collected over a period of over three years. The GPS traces are scattered all over the world; but most of them are located in the region of Beijing, China. We processed the data as follows, in order to fit in our formalism.

Space discretization. We select the area of ∼4.4 km × 4.4 km, within Beijing, that contains the largest number of GPS samples, and we filter out GPS samples that are outside of this area. This geographic area corresponds to the campus of the Tsinghua University (longitude ranging from 116.3 to 116.35 and latitude ranging from 39.97 to 40.01, see Figure 3). We partition the selected area into 25 regions by using a 5×5 square grid. The GPS coordinates of each sample are translated into the region (i.e., the grid cell) they fall into.

Time discretization. We split the continuous time interval into one-hour time sub-intervals, which correspond to time instants in our formalism. For each time sub-interval t and for each user u, we set the user's actual location in that time interval (i.e., a u (t)) to the region corresponding to the sample that is the closest to the midpoint of the considered time sub-interval. If a user's trace does not contain any sample in a given time sub-interval, the user's actual location is set to a dummy region r ⊥ , leaving us with partial user traces.

Co-location generation. As the dataset does not contain explicit co-location information reported by the users, we use synthetic co-locations that we generate as follows: At each time instant, we generate a co-location between two users if their discretized actual locations are the same (and different from r ⊥ ). Because in real-life not all such situations correspond to actual co-location and because even actual co-locations are not necessarily reported, in our evaluation we take into account only a proportion ω (ranging from 0% to 100%) of the synthetic co-locations.

For each user, we compute the number of co-locations she has with every other user in the dataset, across the full user traces. We keep only the users for which there exists another user with whom they have at least 200 co-locations. For these users, we consider their common time interval (i.e., the longest time interval during which all these users have at least one sample); we obtained an interval of ∼6000 hours. Within the common interval, we sample 10 short traces of 300 continuous hours such that (1) all users have at least 10% of valid samples (i.e., , different from r ⊥ ) and (2) all users have at least 20 co-locations with their co-target 1 (as defined in Equation ( 11)). This leaves us with a total of 5 users.

User mobility profiles construction. We build the mobility profiles {p u } u∈U of the users based on their entire discretized traces by counting the transitions from any region to any region (in R) in one time instant.

Obfuscation. We consider that users report a single (or none), potentially obfuscated, location at each time instant. 4 This means that the set R ′ in which the obfuscated location o u (•) takes values is R ∪ {r ⊥ }. We consider, for each user u, that two location-privacy protection mechanisms are used together: First, the location is hidden (i.e., obfuscated to r ⊥ ) with a probability λ u and then, if the location has not been hidden, it is replaced by a region (chosen uniformly at random) at a distance of at most d u from the user's actual discretized location (i.e., a region). If the actual location of a user is not known (i.e., set to r ⊥ ), the LPPM returns r ⊥ with probability 1. In our evaluation, we vary λ u from 0 to 1 and we set d u to the size of one grid cell; this means that, if it is not hidden, a user's location is obfuscated either to its actual value (with probability 0.2) or to one of the four adjacent regions (e.g., 2, 6, 8 and 12 for region 7 in Figure 3), each with probability 0.2.

Privacy Evaluation. We evaluate the location privacy of the users, and the effect of co-locations on it, based on the metric defined in [START_REF]How the NSA is tracking people right now[END_REF]. For each user and for each short trace, we generate 20 random obfuscated traces (remember that obfuscation is a random process) and we perform a localization attack on each of them. We compute the average location privacy of each user across the different obfuscated traces and across the different time instants. Time instants for which the location of a user is not known (i.e., set to r ⊥ ) are not taken into account in the computation of their average over time.

Limitations. Due to the synthetic nature of the reported location and colocation information in our data source, our experimental setup does not perfectly reflect on a real usage case. Therefore, the results presented in this section cannot directly be interpreted as the magnitude of the threat in real-life. Yet, we believe that it suffices to get insight into the effect of co-locations on location privacy, the sources of privacy loss, and the relative performance of the proposed heuristics. Also, the number of users considered in our evaluation (i.e., 5) is relatively small. Hence, the results may not be representative of the entire population. In order to overcome the aforementioned shortcomings, we intend to collect a large-scale dataset from an existing social network. We also intend to run experiments on large grids (i.e., larger than the 5×5 grid used in the evaluation).

Experimental Results

We now experimentally evaluate the algorithms, presented in Section 4, in different scenarios, with different settings. The goal of our evaluation is to assess the raw performance of our heuristics, but also to compare them. In addition, we analyze the effect of the different parameters of the model (including the individual LPPM settings of the users and the differences between the individual LPPM settings of the users) and of the set of co-locations considered in the localization attack.

Effects of co-locations and LPPM settings. We begin our evaluation by analyzing the effect of (1) the proportion ω of reported co-location and (2) the LPPM settings (i.e., w/ or w/o obfuscation and the location hiding probability λ, assumed to be the same across users) in the case of two users, i.e., the target user and her first co-target are considered jointly in an optimal localization attack, namely the limited user set approximation with N = 2. The results are depicted in Figure 4. The left sub-figure shows the case where no obfuscation is used (i.e., the users disclose their actual locations with probability 1-λ and hide them completely otherwise), whereas the right sub-figure shows the case where obfuscation is used (i.e., the users disclose their obfuscated locations, specifically a region chosen uniformly at random among the actual location and the four immediate neighboring regions, with probability 1 -λ and hide them otherwise). The top graphs show a box-plot representation (i.e., first quartile, median, third quartile and outliers) of the users' location privacy expressed in terms of the expected error of the adversary, in kilometers (left axis) and in proportion of the size of the considered geographic area (right axis). For each couple of values (λ, ω), we draw one box-plot to aggregate the data-points obtained for all users and for all the 20 randomly generated obfuscated versions of each of the considered actual trace. Note that without obfuscation, the case λ = 0 leads to zero privacy as users always disclose their actual locations. It can be observed that the proportion of reported co-locations consistently decreases the location privacy of the users. To quantify this decrease, we plot (middle and bottom graphs) the privacy loss caused by the use of co-location information, with respect to the 4. Privacy (top), absolute privacy loss (middle) and relative privacy loss (bottom) for the limited user set attack with N = 2 users. The privacy loss is expressed wrt the case where no co-locations are reported (ω = 0%); the histograms show median values. Co-location information decreases privacy. The relative privacy loss is higher for small values of the hiding probability and without obfuscation.

case where co-locations are ignored (or not available), i.e., ω = 0%. We show both the median absolute privacy loss (in kilometers, middle graph) and the median relative privacy loss (in percentage of the privacy in the case ω = 0%, bottom graph). Note that the median privacy loss is not equal to the difference of the median privacy. Consider for example, the case λ = 0.4 and ω = 50%: in the case without obfuscation the median privacy loss is approximately 125m, which corresponds to a decrease of 25%. The median absolute privacy loss can go up to 290m (λ = 0.6, ω = 100%) and the median relative privacy loss up to 75% (λ = 0.2 and ω = 100%). We observe the same trend, with a more modest loss, in the case where obfuscation is used. For the rest of the evaluation, we focus on the case where users do obfuscate their locations and report ω = 50% of the co-locations.

Effects of the differences of individual LPPM settings. Here, we analyze the effect of the differences, in the users' LPPM settings, on the location privacy (loss) due to co-locations. To do so, we focus on the case of two users, a target 0.14 0.12 0.11 0.11 0.08 0.04 0.14 0.14 0.12 0.11 0.10 0.06 0.18 0.13 0.12 0.12 0.12 0.06 0.33 0.26 0.24 0.24 0.22 0.21 0 0.5 and her co-target, both who obfuscate their location but with different hiding probabilities λ target and λ co-target . We perform a joint optimal localization attack.

The results are depicted in Figure 5 under the form of heat-maps that represent the target user's location privacy (a) as well as her absolute (b) and relative (c) privacy loss (wrt the case ω = 0%) as functions of the respective LPPM settings λ co-target (x-axis) and λ target (y-axis).

A first observation is that co-locations always decrease the privacy of the target (i.e., all values in Figure 5b are positive) and that the more information the co-target discloses, the worse the privacy of the target is (i.e., the cells of the heat-map depicted in Figure 5a become lighter, when going from right to left on a given row).

The diagonals of the heat-maps correspond to the case λ co-target = λ target , which is depicted in more details in Figure 4. The region of the heat-map above the diagonal corresponds to the case where the target is more conservative, in terms of her privacy attitude, than her co-target (i.e., λ co-target < λ target ). It can be observed that the information disclosed by the target herself compromises her privacy more than the information disclosed by her co-target, e.g., the cell (0.6,0) is lighter (which means that the target's privacy is lower) than the cell (0,0.6). By comparing the columns "λ co-target = 1" and "no co-target" (two right-most columns in Figure 5a), we can observe the privacy loss stemming from the use, through the co-location information, of the co-target's mobility profile alone (as the co-target never discloses her location). This is substantial. Finally, in the extreme case where the target never discloses location information and her co-target always does (top-left cell of the heat-maps in Figures 5b and5c), the privacy loss for the target is 330m, which corresponds to a decrease of 16%. This case (and in general the cases where the target never discloses location information, i.e., the top row of the heat-maps) highlights the fact that, as reported co-locations involve two users, users lose some control over their privacy: Without revealing any information about herself, a user can still have her privacy decreased by other users, due to co-location information.

For the rest of the evaluation, we focus on the case where all users have the same LPPM settings (i.e., same values of λ).

Comparison of the proposed heuristics. Here, we compare, through experimentation (we leave the analytical comparison to future work), the proposed inference algorithms for the localization attack, by taking into account different scenarios, as depicted in Figure 6. In scenario (a), we consider, in turn, all target users in our set and perform an individual localization attack on each of them, using only their own reported locations and no co-locations. This corresponds to the baseline case ω = 0%, which was presented in detail in Figure 4b. We then consider the case of an adversary that exploits co-locations. We assume the adversary observes a limited proportion, ω = 50%, of the existing co-locations. Scenario (b) corresponds to the case of an adversary that, in order to attack a target user, performs an optimal joint inference attack on the target and her cotarget, as described in Section 3. This scenario corresponds to the case ω = 50% in Figure 4b. Scenarios (c) and (d) correspond to the case of an adversary that performs an optimal joint attack on the target and her two co-targets. We distinguish two cases, (c) -in which the co-locations between the co-targets are ignored and (d) -in which all co-locations between any of the three users are considered. We make this distinction solely to quantify the privacy loss stemming from the use of co-locations that do not directly involve the target. In practice, an adversary would always consider scenario (d) as it takes into account more information at no extra cost. Finally we consider scenario (e), that corresponds to an adversary that uses reported all co-locations but solves an individual inference problem for each user, as described in 4.2.

Figure 7 shows the results of our comparison. The graph on the left shows a boxplot representation of users' privacy, for each of scenarios (a)-(e). To quantify the different effects on the users' privacy of the set of considered co-locations and of the heuristic used, we show (right) the absolute and relative privacy loss, with respect to scenario (a), for each of the scenarios (b)-(e). It can be observed by comparing scenarios (a)-(d) that, unsurprisingly, the users' privacy decreases with the amount of considered co-locations. However, the comparison between scenarios (c) and (d) shows that co-locations between the target's co-targets does not significantly improve the performance of the localization attack. Finally, we observe that the second heuristic, which takes into account all co-locations outperforms the first heuristic (N ≤ 3), at a lower computational cost.

Related Work

Location is identity. Even if the set of locations shared by a user is anonymized, and her true identity is hidden from the location-based service provider, the observed trajectories can be re-identified [START_REF] De Mulder | Identification via locationprofiling in GSM networks[END_REF][START_REF] Golle | On the anonymity of home/work location pairs[END_REF][START_REF] Hoh | Enhancing security and privacy in traffic-monitoring systems[END_REF][START_REF] Krumm | Inference attacks on location tracks[END_REF]. This attack is made by linking available information about users' mobility in the past with their observed traces. To protect against such attacks, many location obfuscation mechanisms have been proposed in the literature; they suggest users hide their locations at certain locations, or reduce the accuracy or granularity of their reported locations [START_REF] Damiani | The PROBE framework for the personalized cloaking of private locations[END_REF][START_REF] Ghinita | Preventing velocity-based linkage attacks in location-aware applications[END_REF][START_REF] Huang | Silent cascade: Enhancing location privacy without communication QoS degradation[END_REF]. These techniques increase users' privacy by making it more difficult for an adversary to de-anonymize users and localize or track them over time. The location privacy of users in such settings can be computed using the expected error of an adversary in estimating their true locations [START_REF] Shokri | Quantifying location privacy[END_REF]. In such an inference framework, an adversary has a background knowledge on users' mobility models; this is used to reconstruct the full trajectories of the users, given the anonymized and obfuscated observed traces.

The adversary's information, however, is not limited to mobility models. With most users being members of social networks, an adversary can de-anonymize location traces by matching the graph of co-traveler users with their social network graph [START_REF] Srivatsa | Deanonymizing mobility traces: Using social network as a side-channel[END_REF]. Co-travelers are those who have been in each others' physical proximity for a considerable number of times. Researchers have extensively studied the problem of inferring social ties between users based on their physical proximity [START_REF] Crandall | Inferring social ties from geographic coincidences[END_REF][START_REF] Eagle | Inferring Friendship Network Structure by Using Mobile Phone Data[END_REF]. Recent revelations about NSA surveillance programs also show that this type of information is of great use for tracking and identifying individuals [START_REF]How the NSA is tracking people right now[END_REF].

The correlation between different users' information also opens the door to a new type of privacy threat. Even if a user does not reveal much information about herself, her privacy can be compromised by others. In [START_REF] Henne | Snapme if you can: Privacy threats of other peoples' geo-tagged media and what we can do about it[END_REF], the authors study how information revealed, from pictures, by a user's friends in social networks can be used to infer private information about her location. Private information about, for example, user profile and her age can also be inferred from shared information on online social networks [START_REF] Dey | Estimating age privacy leakage in online social networks[END_REF][START_REF] Mislove | You are who you know: Inferring user profiles in online social networks[END_REF]. Mobile users, connecting to locationbased services from a same IP address, can also compromise the privacy of those who want to keep their location private [START_REF] Vratonjic | How others compromise your location privacy: The case of shared public IPs at hotspots[END_REF]. The loss in privacy, due to other users, has also been shown in other contexts such as genomics [START_REF] Gymrek | Identifying personal genomes by surname inference[END_REF][START_REF] Humbert | Addressing the concerns of the lacks family: Quantification of kin genomic privacy[END_REF].

Extracting co-location information about users, i.e., who is with whom, is becoming increasingly easier. More specifically, with the proliferation of mobile social networks, where users can check-in themselves and others to different locations, the threat of available co-location information on users' location privacy is clear (as pointed out in [START_REF] Vicente | Location-related privacy in geosocial networks[END_REF]). Despite the mentioned works on quantifying the location privacy and the privacy of users in social networks, as well as the extensive research on privacy loss due to others, there has not been a study on evaluating location privacy considering co-location information. We bridge the gap between studies on location privacy and social networks, and we propose the first analytical framework to quantify the effects of co-location information on location privacy, where users can also make use of obfuscation mechanisms.

Conclusion

In this paper, we have studied the effect on users' location privacy when colocation information is available, in addition to individual (obfuscated) location information. To the best of our knowledge, this is the first paper to quantify the effects of co-location information, that stems from social relationships between users, on location privacy; as such it constitutes a first step towards bridging the gap between studies on location privacy and social networks. We have shown that, by considering the users' locations jointly, an adversary can exploit co-location information to better localize users, hence decreasing their individual privacy. Although the optimal joint localization attack has a prohibitively high computational complexity, the polynomial-time approximate inference algorithms that we propose in the paper provide good localization performance. An important observation from our work is that a user's location privacy is no longer entirely in her control, as the co-locations and the individual location information disclosed by other users significantly affect her own location privacy.

The message of this work is that protection mechanisms must not ignore the social aspects of location information. Because it is not desirable to report dummy lists of co-located users (as this information is displayed on the users' profiles on social networks), a location-privacy preserving mechanism needs instead to generalize information about co-located users (i.e., replace the names of the co-located users by the type of social tie, e.g., "with two friends") or to generalize the time (i.e., replace the exact time of the co-location with the period of the day, e.g., replacing 11am with "morning", when the co-location is declared a posteriori ) of a social gathering as well as the locations of users at other locations, in order to reduce the effectiveness of the attacks we suggested in this paper. We intend to tackle the design of social-aware location-privacy protection mechanisms (running on the users' mobile devices) to help the users assess and protect their location privacy when co-location information is available.

Fig. 1 .

 1 Fig. 1. Examples showing how co-location information can be detrimental to privacy.(a) A user reports being in a given area, and a second user reports being in another (overlapping) area and that she is co-located with the first user. By combining these pieces of information, an adversary can deduce that both users are located in the intersection of the two areas, thus narrowing down the set of possible locations for both of them. (b) Two users (initially apart from each other, at 10am) declare their exact individual location. Later (at 11am), they meet and report their co-location without mentioning where they are. By combining these pieces of information, the adversary can infer that they are at a place that is reachable from both of the initially reported locations in the amount of time elapsed between the two reports.

Fig. 3 .

 3 Fig.3. Illustration of the dataset used in the evaluation. Most traces are located in the region of Beijing (left); we focus on a small active area that corresponds to the campus of the Tsinghua University and we partition it by using a 5 × 5 square grid (middle). The heat-map (right) shows the number of samples in each region (logscale).

  Fig.[START_REF] Damiani | The PROBE framework for the personalized cloaking of private locations[END_REF]. Privacy (top), absolute privacy loss (middle) and relative privacy loss (bottom) for the limited user set attack with N = 2 users. The privacy loss is expressed wrt the case where no co-locations are reported (ω = 0%); the histograms show median values. Co-location information decreases privacy. The relative privacy loss is higher for small values of the hiding probability and without obfuscation.

  probability (λ): target Location hiding probability (λ): co-target 0.06 0.06 0.06 0.05 0.03 0.02 0.11 0.11 0.10 0.08 0.06 0.03

  Absolute privacy loss (wrt to ω = 0%) probability (λ): target Location hiding probability (λ): co-target

Fig. 5 .

 5 Fig. 5. Median values of the target's location privacy (loss), for the limited user set attack with N = 2 users, when the target and her co-target have different values of λ (with obfuscation and ω = 50%). The diagonals correspond to the values of Figure 4b.

Fig. 6 .Fig. 7 .

 67 Fig.6. Co-locations considered in the evaluation: (a) no co-locations, (b) only colocations between the target and co-target1 (Heuristic 1, N = 2), (c) only co-locations between the target and co-target1 and between the target and co-target2 (Heuristic 1, N = 3), (d) all co-locations between the target, co-target1 and co-target2 (Heuristic 1, N = 3), (e) all co-locations (Heuristic 2). In scenarios (b)-(e), we consider that ω = 50% of the co-locations are reported.

Note that even if pseudonyms are used, the identity of the users can be inferred by using their social network[START_REF] Narayanan | De-anonymizing social networks[END_REF] or their locations[START_REF] Shokri | Quantifying location privacy[END_REF].

For the sake of simplicity and clarity, we define the variables at t = 0 even though no observations are made at this time instant.

The best way to do this is to use dynamic programming, i.e., compute the αt (and storing its values) iteratively for increasing t and compute the βt (and store the values) iteratively for decreasing t.

We make this assumption because of the limited size of the considered grid and we leave the case where LPPMs output a set of locations to future work.

Acknowledgments

The authors are thankful to Stefan Mihaila for his help in collecting useful statistics about the use of co-location on Foursquare. This work was partially funded by the Swiss National Science Foundation with grant 200021-138089.


Daphné Duval
email: daphne.duval@univ-rennes1.fr

Bruno Bêche

Theoretical formulation to shape versatile propagation characteristics of 3-layer-tubular waveguides

This study introduces an analytical formalism to shape the quantification of photonic modes devoted to global 3-layer-tubular waveguides (including hollow cylindrical structures). Thereby we have implemented such an overall frame and focus on the asymptotic behavior and the cut-off limit discussions.

Introduction

Owing to their high aspect ratio, sub-wavelength nanotubes are promising building blocks for extreme miniaturization of optical components [1,2]. In this study, we present a generalized formalism for the whole optical modes of 3-layer-tubular waveguides based on the exact solution of Maxwell's equations. The tubular configuration of interest consists of two regions of infinite length with refractive index n 1 and n 2 embedded in a third infinite media of refractive index n 3 (n 2 > n 1 and n 2 > n 3 ). The internal and external radius are respectively noted a and b.

Results

In each region, the six field components have been deduced. By using the boundary conditions, two characteristics equations have been achieved (1). These equations contain two unknown ratios, a 4 /a 2 and b 4 /b 2 , that are obtained by applying Cramer's rule to the eight equations of the boundary conditions. Propagation constants for hybrid modes are obtained by solving simultaneously the two characteristic equations (1) with the proper forms of a 4 /a 2 and b 4 /b 2 . a 2 Yν (p21) βν k0n3

                               βν k0n1
2 1 p 2 22 + 1 p 2 32 2 = ν p 2 32 - Kν+1(p32) p32Kν (p32) + ν p 22 Jν (p22) -Jν+1(p22) + b 4 b 2 ν p 22 Yν (p22) -Yν+1(p22) p22 Jν (p22) + b 4 b 2 Yν (p22) × ν p 2 32 - Kν+1(p32) p32Kν (p32) + n 2 2 n 2 3 ν p 22 Jν (p22) -Jν+1(p22) + a 4 a 2 ν p 22 Yν (p22) -Yν+1(p22) p22 Jν (p22) + a 4 a 2 Yν (p22) (1)
The quantification of families of optical modes have been implemented in the case of silica nanotubes (n

1 = n 3 = 1, n 2 =

2 νp 2 νp

 22 (p21) -Jν+1(p21) + b 4 b 21 Yν (p21) -Yν+1(p21) p21 Jν (p21) + b 4 b 2 Yν (p21) (p21) -Jν+1(p21) + a 4 a 21 Yν (p21) -Yν+1(p21) p21 Jν (p21) + a 4

  1.5 at λ = 670nm). Two studies of such eigenvalue quantifications are depicted: 1. the internal radius a is fixed and the external radius b ranges around [a, 10a], with a = [λ/10; λ/2; λ]; 2. the external radius b is fixed and the internal radius a ranges around [0, b], with b = [λ/2; λ; 10λ]. The opposite of the effective refractive index has been plotted and studied as a function of the radius (internal or external) normalized to the wavelength. The asymptotic behavior of the curves are particularly discussed as well as the cutoff limits of the optical modes. A map of the different areas (monomode and multimode) has been dressed as a function of the internal and external radius normalized to the wavelength for versatile tubular structures.


Camille Fauth

Anne Bonneau
email: bonneau@loria.frtrouvain@coli.uni-saarland.de

Frank Zimmerer

Jürgen Trouvain

Bistra Andreeva

Vincent Colotte

Dominique Fohr

Denis Jouvet

Jeanin Jügler

Yves Laprie

Odile Mella

Bernd Möbius

Designing a Bilingual Speech Corpus for French and German Language Learners: a Two-Step Process

Keywords: speech corpus, phonetics, language learning

We present the design of a corpus of native and non-native speech for the language pair French-German, with a special emphasis on phonetic and prosodic aspects. To our knowledge there is no suitable corpus, in terms of size and coverage, currently available for the target language pair. To select the target L1-L2 interference phenomena we prepare a small preliminary corpus (corpus1), which is analyzed for coverage and cross-checked jointly by French and German experts. Based on this analysis, target phenomena on the phonetic and phonological level are selected on the basis of the expected degree of deviation from the native performance and the frequency of occurrence. 14 speakers performed both L2 (either French or German) and L1 material (either German or French). This allowed us to test, recordings duration, recordings material, the performance of our automatic aligner software. Then, we built corpus2 taking into account what we learned about corpus1. The aims are the same but we adapted speech material to avoid too long recording sessions. 100 speakers will be recorded. The corpus (corpus1 and corpus2) will be prepared as a searchable database, available for the scientific community after completion of the project.

Introduction

We present the design of a corpus of native and non-native speech for the language pair French-German, devoted to an in-depth analysis of both segmental and prosodic aspects of the non-native production of these languages. To our knowledge there is no suitable corpus, in terms of size and coverage, currently available for the target language pair. Our design of such a corpus has three aims. (i) We will bring to the research community two nonnative corpora for the French-German language pair, whereas most studies have focused on English (cf. website on "Learner corpora around the world"). (ii) The corpora will be informed by in-depth phonetic knowledge to predict the types of errors made by French and German learners. (iii) The corpora can further be used by the research community for the recognition of non-native speech, which is notoriously difficult (see e.g. Goronzy et al. 2001[START_REF] Van Doremalen | Optimizing non-native speech recognition for CALL applications[END_REF][START_REF] Bouselmi | Multilingual recognition of non-native speech using acoustic model transformation and pronunciation modeling[END_REF]. From this entire corpus, we created two different corpus (corpus1 and corpus2) which will be used at different stage, corpus1 being the preliminary corpus and corpus2 being the final one. Corpus1 has been recorded to test hypotheses about interferences between L1 and L2 of the speakers, as well as to test the performance of our automatic aligner software, especially with respect to L2 recognition. First results obtained from the analysis of corpus1 were incorporated into the design of corpus2.

Design of Corpus1

To select the target L1-L2 interference phenomena, we prepared corpus1, which was analyzed for coverage and cross-checked jointly by French and German experts. From this analysis, target phenomena on the phonetic and phonological levels were selected on the basis of the expected degree of deviation from the native performance and the frequency of occurrence. The material covered for both languages consists of: (i) a phonetically rich design comprising all phonemes in relevant contexts, to achieve a reliable assessment of the entire phonemic inventory for each speaker; (ii) the most important phenomena in the phonetics and prosody of French and German as a foreign language, respectively (e.g., vowel quantity, consonantal articulation and word stress); (iii) phonological processes and alternations (e.g., final devoicing); (iv) minimal pairs. Each speaker had to perform four tasks in both languages L1 and L2. In the first task, the speaker was asked to read aloud a sentence (25 different sentences). In the second, the speaker heard the sentence pronounced by a native speaker and then was asked to read the same sentence aloud (25 different sentences). The third task, the focus condition, consisted of sentences that had to be produced as an answer to a question pronounced by a native speaker, including broad and narrow focus conditions. The part of the sentence intended to be in focus, was additionally indicated by upper-case letters. Six sentences were presented, each of which also occurred with different constituents in focus, that correspond to 24 different sentences (6 sentences X n focus conditions). The fourth condition was to read aloud two short stories: The three little pigs and a more technical text (about ten sentences long -2 minutes recordings).

Subjects and Recordings

For corpus1, we recorded seven native subjects fo r each language (see Table 1). We included beginners (A2 level according to the CEFR) as well as advanced (C1 level) second language learners. Among the beginners we aimed at teenage learners with two to three years of L2 instruction in school, and university students. Informed consent was obtained from subjects (and parents for the children) allowing us to use the recorded data for scientific purposes. For corpus1 German and French subjects were recorded in their L2 (FG and GF respectively) as well as in their native language. They all recorded the non-native part first, and then they also produced the sentences in their L1 (FF and GG respectively -see Table 1 High-quality recordings were made, using the software JCorpusRecorder (see raweb.inria.fr) on a Windows laptop. As recording device, we used a headset microphone (AKG C520) and an Audiobox (M Audio Fast track). The gain was automatically controlled during the recording to avoid clipping. Sometimes, the gain was also manually adjusted. The subject was seated in a quiet room. Recording sessions were carefully monitored for consistent quality. Subjects could listen to their recordings after each sentence and decide whether they wanted to make a new recording or take the one they just performed. A recording session lasted between 50 and 75 minutes.

Annotation and Automatic Alignment

Annotation was performed in two stages: First, the entire corpus1 was automatically segmented and annotated by o u r speech-text alignment tool [START_REF] Jouvet | Impact of pronunciation variant frequency on automatic non-native speech segmentation[END_REF][START_REF] Fohr | CoALT: A software for comparing automatic labelling tools[END_REF] via the use of a two-step approach for automatic phone segmentation. The first step consisted in determining the phone sequence that best represents the learner's utterance. This was achieved by force-aligning the learner's utterance with a model representing pronunciation variants of the sentence. It was crucial to consider both native and non-native variants. In this step, detailed context-dependent acoustic Hidden Markov Models (HMMs) were used, with a rather large number of Gaussian components per mixture density. This kind of detailed acoustic model was the one that provided the best performance in automatic speech recognition. The second step consisted in determining the phone boundaries. This was also achieved by means of a forced alignment process, but this time, the sequence of phones was known (as determined in the first step), and contextindependent acoustic phone models with only a few Gaussian components per mixture density were used because they provide a better temporal precision than detailed acoustic models [START_REF] Toledano | Automatic Phonetic Segmentation[END_REF]. For training the models in both forced alignment steps, the speech of native and non-native speakers was used, either directly or by MLLR (Maximum Likelihood Linear Regression) adaptation. Furthermore, having speakers produce both native and non-native speech allows for better adaptation. In the second stage, the entire corpus1 was checked with respect to the orthographic transcription. It was manually checked at the levels of phones and words (phonetic transcription) and corrections were made if necessary. This procedure enables us to investigate how automatic forced alignment performs on non-native speech [START_REF] Fohr | CoALT: A software for comparing automatic labelling tools[END_REF] and to improve the models. This manual control and (re)labeling was performed using the software PRAAT [START_REF] Boersma | Praat : Doing phonetics by computer (version 5.1.20[END_REF] from which it was possible to add various lines of information (or tiers) for an alignment at the sentence, the word and phoneme level (see Figures 1 &2). The automatic alignment generated several Tiers which two are exactly the same: the Align and the Real Tier. The annotator was responsible for verifying whether the aligner correctly recognized the segments that were produced and also, whether the boundaries between the segments were correct. Otherwise, the labels were changed, or the boundaries were move in the so-called RealTier, using the acoustic signal and its spectrogram associated. If sounds had been changed, compared to a canonical pronunciation, annotators had in some cases also the choice to use a set of diacritical labels adapted and extended from Kiel corpus (IPDS, 1994, Kohler et al., 1995; or see http://www.ipds.uni-kiel.de/forschung/ kielcorpus.de.html). This list may be further extended as the project progresses. The annotator also had a comment line (CommentTier) to add further notes. However, this line is used sparingly. It is also useful for remarks concerning the entire sentence (in particular prosody). As you can see on Figure 1, the Realtier has been changed at the phonetic level to indicate that the expected sound /@/ has been pronounced as /ø/ (noted /2/). The annotator also signaled the presence of glottalisation, which is coded by the symbol /q/ and preceded by the symbol /-/ used in case of insertion. On Figure 2, we can see that there is an important mistake concerning the end of the automatic boundary of the consonant /Z/ and therefore the boundary has been moved. Another correction applied to the same sound, signaled, though the use of the diacritics (_0), that it has been devoiced. This devoicing is due to the influence of the speaker's L1, voiced obstruent consonants /Z,z,v,b,d,g/ being devoiced in word final position in German.

Conclusions Concerning Corpus1

The purpose of corpus1 was to verify the relevance of the selected phonetic phenomena and detect possible unexpected phenomena and/or deviations. The data suggested that several observed phenomena are crucial for improving the pronunciation of L2, for both German and French native speakers, such as vowel quantity, final devoicing and the aspiration or non aspiration of stop consonants. Concerning vowel quantity, we found that French native speakers inconsistently produced tense (long) and lax (short) vowels in their L2, whereas German speakers tend to maintain vowel length differences (vowel quantity) in French where such as difference does not exist. Concerning final devoicing of obstruents, we found that German speakers (beginners and advanced) tend to maintain this devoicing when speaking French [START_REF] Fauth | L1-L2 interference: the case of devoicing of French voiced obstruents in final position by German learners -Pilot study[END_REF] whereas most French speakers does not devoice obstruents in final position. Also, the analysis of voiceless stops indicated that German speakers produced stop segments in French (GF part of the corpus) as they do in German, without adjusting for the French standard production with less aspiration, whereas the opposite phenomenon is observed for French speaking German. Among other interesting phenomena, as expected, there were influences of orthography, as for example, the production of the word loup ('wolf') where the final consonant was mistakenly produced by many native German speakers. French speakers showed some mistakes in the production of the German glottal fricative /h/. On a general level, French learners of German produced fewer /h/s than native German speakers. For the French speakers, the data revealed an interaction with task, when repeating sentences, the /h/ was produced significantly more often (about 77%) than in a pure reading task (about 52%), despite the influence of orthography, where <h> is written in both cases (Zimmerer & Trouvain, submitted).

These results emphasize the possible usefulness of having an audio model (or 'golden speaker') in language teaching contexts. Interestingly, in cases where French speakers did produce /h/, the segment seemed to be hyper-articulated. French speakers' /h/s were longer (in relation to the rest of the word) and articulated with more intensity (in relation to the rest of the word) than the native speakers' productions. Based on our findings, we were able to deduce important aspects in the production of L1 and L2, in order to further improve for instance the contexts in which the phenomena occurred in our data. We slightly modified some sentences in the preparation phase of corpus2, which is discussed in the next section.

Corpus2

Design

The general idea, to obtain productions from speakers both in their native language and their L2, is identical to corpus1. Again, speakers will first produce the L2, and then they will perform the L1 tasks. We learned from the recordings of corpus1 that the total recording time should not exceed one hour per subject, to avoid the effects of vocal fatigue, especially for beginners. Therefore, we adjusted the overall size of the corpus. The number of sentences for reading and repetition has been slightly increased (from 50 to 60), to be able to exploit phonetic and phonological phenomena of interest in more depth. Furthermore, the presentation has been changed to some extent. While the order of the tasks was kept constant, the sentences of every task are now randomly presented to the speakers. 31 out of 60 sentences are now in the reading task, 29 were heard before reading. The sentences have been built to include phonetic target phenomena mentioned for corpus1. But if for corpus1, our approach was more general and wanted to cover all the difficulties that could be encountered in learning German or French as L2, for corpus 2, we want to focus on certain phenomena and finely control the context in which they may appear. These phenomena have been highlighted by the analysis of the corpus1. This approach led us to "select" phenomena rather than others. These selected phenomena have been carefully controlled, such as the pronunciation of stop consonants. For stops consonants, the sentences been adapted to create quasi-minimal pairs also in the sentence structure, when this was possible. For the pronunciation of stop consonant, some of these sentences are:Braucht man in einer Band einen Bass? ('Do you need a bass in the band?') Braucht man zum Reisen einen Pass? ('Do you need a passport to travel?') Wieviele Kinder fahren mit dem Rad? ('How many children are driving the bike?' -in this example, the final /d/ is devoiced -Auslautverhärtung) Braucht sie deinen Rat? ('Does she need your advice?') Bring bitte noch Teller in den Garten ('Bring the plates to the garden, please.') Auch Täler findet man in Karten. ('You find valleys in maps, too.') Le parrain a quitté le bar ce matin. ('The godfather left the bar this morning') Le garçon mange deux parts de gâteau. ('The boy eats two pieces of cake.') L'abeille n'a pas de dard sur le corps. ('The bee does not have a stinger on the body.') Les poids ont fait la tare sur la balance. ('The weighs have made the tex weight on the scale.') Le train a quitté la gare de Paris. ('The train has left the station of Paris.') Le garçon a pris le car à Berlin. ('The boy has taken the bus to/in Berlin.') German, like English, has long and short vowels. Short vowels are usually produced when they precede a sequence of two consonants and not starting with a /h/. Moreover, the vowel quality in German is also related to duration. The following sentences are examples with minimal pairs that focused on this phenomenon. Ils ont acheté leur armoire à Rome. ('They bought their wardrobe in Rome.') Some other phenomena are not investigated systematically, but remain present in corpus 2 (such as the production of post-vocalic /R/ produced by GF). In addition compared to corpus1, we included numerals and abbreviations. These will increase the cognitive load for some of the sentences and maybe show interesting differences between beginners and advanced learners. Besides, they are also intended to reveal further segmental variations, and especially numerals which occur in everyday situations are different in their structure in French and German. For instance, in German, the number "96" is literally "6 and 90", whereas in French, it is "4 20 16". As a consequence of the analysis of corpus1, the number of sentences in the focus condition was reduced to two (instead of six). This reduces the total number of sentences, which leads to an overall size of corpus2 that is smaller than corpus1, but it also minimizes sentences that are rather not very natural for speakers. Focus sentences occurred in up to four focus conditions that have to be repeated, including contrastive and non-contrastive focus. If there are too many sentences, such repetition effects may lead to somewhat unnatural utterances. For this task, sentences were created that would allow for a smooth detection and automatic tracking of the fundamental frequency, which is one of the principal clues in this type of task. We also selected simple sentences in terms of syntactic structure and vocabulary. As far as possible, they also have the particularity to contain as many voiced sounds and sonorant consonants as possible, which should facilitate the automatic detection of the fundamental frequency. This part therefore is different in corpus2 compared to corpus1.

Concerning the text condition, we selected only one story, the three little pigs for corpus2. We chose this story because it is well-known, its style is very casual and the words are repeated several times, which facilitates testing the consistence of deviant productions. We concentrated on one story to decrease the recording time. Reading the story, especially in the L2, was rather time consuming, because speakers were reading the stories several times to familiarize themselves with the text before they were recorded. Also, we know from the recordings of corpus1 that subjects tend to rerecord the stories. The length of the text increases the likelihood of mistakes, which made subjects feel unsatisfied with their renditions.

Subjects and Recordings

We plan to record a total of 100 subjects for corpus2 (see Table 2). This will comprise 40 university students who are beginners (A2 level) as well as 40 advanced second language learners (C1 or C2 level) and 20 teenagers. The different groups are defined as in corpus1. Speaker gender will be balanced in each of the six groups. So far, 80% of the speakers have been recorded. 2: Groups of subjects that will be recorded for corpus2 pooled across L1 (F=French, G=German), L2 (G=German F=French,), level of proficiency and age range.

The recording conditions of corpus1 were assessed to be satisfying. Therefore, they will be identical for the recordings of corpus2.

Annotation and Automatic Alignment

The entire corpus will be automatically segmented and annotated by our speech-text alignment tool. We want to improve the automatic alignment by including a third step to refine the boundaries occurring between specific phonetic classes, by computing and using ad-hoc acoustic cues, and associated decision processes [START_REF] Adell | Comparative study of automatic phone segmentation methods for TTS[END_REF]. A part of the corpus (50-60%) will be manually checked at the levels of phones, words (orthographic transcription), and sentences (orthographic transcription), and corrected if necessary. The complete corpus (corpus1 and corpus2) will be prepared as a searchable database, available for the scientific community after completion of the project. Researchers will get access to the corpus after sending an application form, specifying the terms and conditions for the use of the corpora.

Conclusions

A bilingual speech corpus consisting of speech of French learners of German (FG and FF) and German learners of French (GF and GG) speaking both in the L2 as well as in the L1, represents an optimal starting point for detailed phonetic and phonological analyses on the segmental as well as on the prosodic level. In addition, such a phonetic learner corpus provides a rich source for first-hand information and illustration for the interested public, e.g. foreign language teachers. This corpus is also the prerequisite for the development of a computer-assisted language learning software by adapting content, feedback and exercises to individual learners in the speech dimension of a foreign language

Acknowledgments

This work has been supported by an ANR/DFG Grant "IFCASL" to the Speech Group LORIA CNRS UMR 7503 -Nancy France and to the Phonetics Group, Saarland University -Saarbrücken Germany, 2013 -2016.

Figure 1 :

 1 Figure 1: Automatic alignment and corrections of a German sentence produced by a French speaker learning German (FG) "Wir essen Gemüse aus unserem Beet" -"We eat vegetables from our patch"

  Ich wüsste nicht, wie der schnellste Weg nach Polen ist. ('I would not know the shortest way to Poland) Im Frühling fliegen Pollen durch die Luft. (In springtime, pollen are flying through the air) Wir essen Gemüse aus unserem Beet. (we eat vegetables from our vegetable patch) Bei hohem Fieber legst du dich ins Bett. (With high fever, you lie in bed) In the following sentence, the « o » of Rome is pronounced /o/ by German and /ɔ/ by French.

  ).

	# subjects	L1	L2	level	age
	4 2	F	G	beginners advanced	18-30 years old
	1			beginners	15-16 years old
	3 2	G	F	beginners advanced	18-30 years old
	2			beginners	15-16 years old
	Table 1: Groups of subjects recorded for corpus1 pooled across L1 (F=French, G=German), L2 (G=German F=French,), level of proficiency and age range.