Malik Boudiaf
email: malik.boudiaf.1@etsmtl.net

Jérôme Rony
email: jerome.rony.1@etsmtl.net

Imtiaz Masud Ziko
email: imtiaz-masud.ziko.1@etsmtl.net

Eric Granger

Marco Pedersoli

Pablo Piantanida

Ismail Ben Ayed

A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses

Keywords: Metric Learning, Deep Learning, Information Theory

Recently, substantial research efforts in Deep Metric Learning (DML) focused on designing complex pairwise-distance losses, which require convoluted schemes to ease optimization, such as sample mining or pair weighting. The standard cross-entropy loss for classification has been largely overlooked in DML. On the surface, the cross-entropy may seem unrelated and irrelevant to metric learning as it does not explicitly involve pairwise distances. However, we provide a theoretical analysis that links the cross-entropy to several well-known and recent pairwise losses. Our connections are drawn from two different perspectives: one based on an explicit optimization insight; the other on discriminative and generative views of the mutual information between the labels and the learned features. First, we explicitly demonstrate that the cross-entropy is an upper bound on a new pairwise loss, which has a structure similar to various pairwise losses: it minimizes intra-class distances while maximizing inter-class distances. As a result, minimizing the cross-entropy can be seen as an approximate bound-optimization (or Majorize-Minimize) algorithm for minimizing this pairwise loss. Second, we show that, more generally, minimizing the cross-entropy is actually equivalent to maximizing the mutual information, to which we connect several well-known pairwise losses. Furthermore, we show that various standard pairwise losses can be explicitly related to one another via bound relationships. Our findings indicate that the cross-entropy represents a proxy for maximizing the mutual information -as pairwise losses do -without the need for convoluted sample-mining heuristics. Our experiments † over four standard DML benchmarks strongly support our findings. We obtain state-of-the-art results, outperforming recent and complex DML methods.

Introduction

The core task of metric learning consists in learning a metric from high-dimensional data, such that the distance between two points, as measured by this metric, reflects their semantic similarity. Applications of metric learning include image retrieval, zero-shot learning or person re-identification, among others. Initial attempts to tackle this problem tried to learn metrics directly on the input space [START_REF] Lowe | Similarity metric learning for a variable-kernel classifier[END_REF]. Later, the idea of learning suitable embedding was introduced, with the goal of learning Mahalanobis distances [START_REF] Davis | Information-theoretic metric learning[END_REF][START_REF] Goldberger | Neighbourhood components analysis[END_REF][START_REF] Schultz | Learning a distance metric from relative comparisons[END_REF]41,44], which corresponds to learning the best linear projection of the input space onto a lower-dimensional manifold, and using the Euclidean distance as a metric. Building on the embedding-learning ideas, several papers proposed to learn more complex mappings, either by kernelization of already existing linear algorithms [START_REF] Davis | Information-theoretic metric learning[END_REF], or by using a more complex hypothesis such as linear combinations of gradient boosted regressions trees [START_REF] Kedem | Non-linear metric learning[END_REF].

The recent success of deep neural networks at learning complex, nonlinear mappings of high-dimensional data aligns with the problem of learning a suitable embedding. Following works on Mahalanobis distance learning, most Deep Metric Learning (DML) approaches are based on pairwise distances. Specifically, the current paradigm is to learn a deep encoder that maps points with high semantic similarity close to each other in the embedded space (w.r.t. pairwise Euclidean or cosine distances). This paradigm concretely translates into pairwise losses that encourage small distances for pairs of samples from the same class and large distances for pairs of samples from different classes. While such formulations seem intuitive, the practical implementations and optimization schemes for pairwise losses may become cumbersome, and randomly assembling pairs of samples typically results in slow convergence or degenerate solutions [START_REF] Hermans | In defense of the triplet loss for person reidentification[END_REF]. Hence, research in DML focused on finding efficient ways to reformulate, generalize and/or improve sample mining and/or sample weighting strategies over the existing pairwise losses. Popular pairwise losses include triplet loss and its derivatives [START_REF] Ge | Deep metric learning with hierarchical triplet loss[END_REF][START_REF] Hermans | In defense of the triplet loss for person reidentification[END_REF][START_REF] Sohn | Improved deep metric learning with multi-class n-pair loss objective[END_REF][START_REF] Song | Deep metric learning via lifted structured feature embedding[END_REF]50], contrastive loss and its derivatives [START_REF] Hadsell | Dimensionality reduction by learning an invariant mapping[END_REF]40], Neighborhood Component Analysis and its derivatives [START_REF] Goldberger | Neighbourhood components analysis[END_REF][START_REF] Movshovitz-Attias | No fuss distance metric learning using proxies[END_REF]43], among others. However, such modifications are often heuristic-based, and come at the price of increased complexity and additional hyper-parameters, reducing the potential of these methods in realworld applications. Furthermore, the recent experimental study in [START_REF] Musgrave | A metric learning reality check[END_REF] showed that the improvement brought by an abundant metric learning literature in the last 15 years is at best marginal when the methods are compared fairly.

Admittedly, the objective of learning a useful embedding of data points intuitively aligns with the idea of directly acting on the distances between pairs of points in the embedded space. Therefore, the standard cross-entropy loss, widely used in classification tasks, has been largely overlooked by the DML community, most likely due to its apparent irrelevance for Metric Learning [42]. As a matter of fact, why would anyone use a point-wise prediction loss to enforce pairwisedistance properties on the embedding space? Even though the cross-entropy was shown to be competitive for face recognition applications [START_REF] Liu | Sphereface: Deep hypersphere embedding for face recognition[END_REF][START_REF] Wang | Additive margin softmax for face verification[END_REF]36], to the best of our knowledge, only one paper empirically observed competitive results of a normalized, temperature-weighted version of the cross-entropy in the context of deep metric learning [49]. However, the authors did not provide any theoretical insights for these results.

On the surface, the standard cross-entropy loss may seem unrelated to the pairwise losses used in DML. Here, we provide theoretical justifications that connect directly the cross-entropy to several well-known and recent pairwise losses. Our connections are drawn from two different perspectives; one based on an explicit optimization insight and the other on mutual-information arguments. We show that four of the most prominent pairwise metric-learning losses, as well as the standard cross-entropy, are maximizing a common underlying objective: the Mutual Information (MI) between the learned embeddings and the corresponding samples' labels. As sketched in Section 2, this connection can be intuitively understood by writing this MI in two different, but equivalent ways. Specifically, we establish tight links between pairwise losses and the generative view of this MI. We study the particular case of contrastive loss [START_REF] Hadsell | Dimensionality reduction by learning an invariant mapping[END_REF], explicitly showing its relation to this MI. We further generalize this reasoning to other DML losses by uncovering tight relations with contrastive loss. As for the cross-entropy, we demonstrate that the cross-entropy is an upper bound on an underlying pairwise loss -on which the previous reasoning can be applied -which has a structure similar to various existing pairwise losses. As a result, minimizing the cross-entropy can be seen as an approximate bound-optimization (or Majorize-Minimize) algorithm for minimizing this pairwise loss, implicitly minimizing intra-class distances and maximizing inter-class distances. We also show that, more generally, minimizing the cross-entropy is equivalent to maximizing the discriminative view of the mutual information. Our findings indicate that the cross-entropy represents a proxy for maximizing the mutual information, as pairwise losses do, without the need for complex sample-mining and optimization schemes. Our comprehensive experiments over four standard DML benchmarks (CUB200, Cars-196, Stanford Online Product and In-Shop) strongly support our findings. We consistently obtained state-of-the-art results, outperforming many recent and complex DML methods.

Summary of contributions

1. Establishing relations between several pairwise DML losses and a generative view of the mutual information between the learned features and labels;

2. Proving explicitly that optimizing the standard cross-entropy corresponds to an approximate bound-optimizer of an underlying pairwise loss;

3. More generally, showing that minimizing the standard cross-entropy loss is equivalent to maximizing a discriminative view of the mutual information between the features and labels.

4. Demonstrating state-of-the-art results with cross-entropy on several DML benchmark datasets.

Table 1. Definition of the random variables and information measures used in this paper.

General

Labeled dataset

D = {(xi, yi)} n i=1 Input feature space X Embedded feature space Z ⇢ R d Label/Prediction space Y ⇢ R K Euclidean distance Dij = kzi zjk 2 Cosine distance D cos ij = z T i z j kz i kkz j k Model Encoder W : X ! Z Soft-classifier f✓ : Z ! [0, 1] K Random variables (RVs) Data X, Y Embedding b Z|X ⇠ W (X) Prediction b Y | b Z ⇠ f✓( b Z) Information measures Entropy of Y H(Y ) := Ep Y [ log pY (Y )] Conditional entropy of Y given Z H(Y | b Z) := Ep Y b Z h log p Y | b Z (Y | b Z) i Cross entropy (CE) between Y and b Y H(Y ; b Y ) := Ep Y ⇥ log p b Y (Y ) ⇤ Conditional CE given b Z H(Y ; b Y | b Z) := Ep b ZY h log p b Y | b Z (Y | b Z) i Mutual information between b Z and Y I( b Z; Y ) := H(Y ) H(Y | b Z)
2 On the two views of the mutual information

The Mutual Information (MI) is a well known-measure designed to quantify the amount of information shared by two random variables. Its formal definition is presented in Table 1. Throughout this work, we will be particularly interested in I( b Z; Y ) which represents the MI between learned features b Z and labels Y . Due to its symmetry property, the MI can be written in two ways, which we will refer to as the discriminative view and generative view of MI:

I( b Z; Y ) = H(Y ) H(Y | b Z) | {z } discriminative view = H( b Z) H( b Z|Y ) | {z } generative view (1)
While being analytically equivalent, these two views present two different, complementary interpretations. In order to maximize I( b Z; Y ), the discriminative view conveys that the labels should be balanced (out of our control) and easily identified from the features. On the other hand, the generative view conveys that the features learned should spread as much as possible in the feature space, while keeping samples sharing the same class close to each other. Hence, the discriminative view is more focused on label identification, while the generative view focuses on more explicitly shaping the distribution of the features learned by the model. Therefore, the MI enables us to draw links between classification losses (e.g. cross-entropy) and feature-shaping losses (including all the well-known pairwise metric learning losses).

3 Pairwise losses and the generative view of the MI

In this section, we study four pairwise losses used in the DML community: center loss [42], contrastive loss [START_REF] Hadsell | Dimensionality reduction by learning an invariant mapping[END_REF], Scalable Neighbor Component Analysis (SNCA) loss [43] and Multi-Similarity (MS) loss [40]. We show that these losses can be interpreted as proxies for maximizing the generative view of mutual information I( b Z; Y ). We begin by analyzing the specific example of contrastive loss, establishing its tight link to the MI, and further generalize our analysis to the other pairwise losses (see Table 2). Furthermore, we show that these pairwise metric-learning losses can be explicitly linked to one another via bound relationships.

The example of contrastive loss

We start by analyzing the representative example of contrastive loss [START_REF] Hadsell | Dimensionality reduction by learning an invariant mapping[END_REF]. For a given margin m 2 R + , this loss is formulated as:

L contrast = 1 n n X i=1 X j:yj =yi D 2 ij | {z } Tcontrast + 1 n n X i=1 X j:yj 6 =yi [m D ij ] 2 + | {z } Ccontrast (2)
where [x] + = max(0, x). This loss naturally breaks down into two terms: a tightness part T contrast and a contrastive part C contrast . The tightness part encourages samples from the same class to be close to each other and form tight clusters.

As for the contrastive part, it forces samples from different classes to stand far apart from one another in the embedded feature space. Let us analyze these two terms from a mutual-information perspective.

As shown in the next subsection, the tightness part of contrastive loss is equivalent to the tightness part of the center loss [42]:

T contrast c = T center = 1 2
P n i=1 kz i c yi k 2 , where c k = 1 |Zk| P z2Zk z denotes the mean of feature points from class k in embedding space Z and symbol c = denotes equality up to a multiplicative and/or additive constant. Written in this way, we can interpret T contrast as a conditional cross entropy between b Z and another random variable Z, whose conditional distribution given Y is a standard Gaussian centered around c Y : Z|Y ⇠ N (c Y , I):

T contrast c = H( b Z; Z|Y ) = H( b Z|Y ) + D KL ( b Z|| Z|Y ) (3)
As such, T contrast is an upper bound on the conditional entropy that appears in the mutual information:

T contrast H( b Z|Y ) (4)
This bound is tight when b Z|Y ⇠ N (c Y , I). Hence, minimizing T contrast can be seen as minimizing H( b Z|Y ), which exactly encourages the encoder W to produce low-entropy (=compact) clusters in the feature space for each given class. Notice that using this term only will inevitably lead to a trivial encoder that maps all data points in X to a single point in the embedded space Z, hence achieving a global optimum.

To prevent such a trivial solution, a second term needs to be added. This second term -that we refer to as the contrastive term -is designed to push each point away from points that have a different label. In this term, only pairs such that D ij  m produce a cost. Given a pair (i, j), let us define x = D ij /m. Given that x 2 [0, 1], one can show the following:

1 2x  (1 x) 2  1 x. Using linear approximation (1 x) 2
⇡ 1 2x (with error at most x), we obtain:

C contrast c ⇡ 2m n n X i=1 X j:yj 6 =yi D ij = 2m n n X i=1 n X j=1 D ij + 2m n n X i=1 X j:yj =yi D ij (5)
While the second term in Eq. 5 is redundant with the tightness objective, the first term is close to the differential entropy estimator proposed in [38]:

b H( b Z) = d n(n 1) n X i=1 n X j=1 log D 2 ij c = n X i=1 n X j=1 log D ij (6)
Both terms measure the spread of b Z, even though they present different gradient dynamics. All in all, minimizing the whole contrastive loss can be seen as a proxy for maximizing the MI between the labels Y and the embedded features b Z:

L contrast = 1 n n X i=1 X j:yj =yi (D 2 ij + 2mD ij ) | {z } /H( b Z|Y ) 2m n n X i=1 n X j=1 D ij | {z } /H( b Z) / I( b Z; Y ) (7)

Generalizing to other pairwise losses

A similar analysis can be carried out on other, more recent metric learning losses. More specifically, they can also be broken down into two parts: a tightness part that minimizes intra-class distances to form compact clusters, which is related to the conditional entropy H( b Z|Y ), and a second contrastive part that prevents trivial solutions by maximizing inter-class distances, which is related to the entropy of features H( b Z). Note that, in some pairwise losses, there might be some redundancy between the two terms, i.e., the tightness term also contains some contrastive subterm, and vice-versa. For instance, the cross-entropy loss is used as the contrastive part of the center-loss but, as we show in Section 4.2, the cross-entropy, used alone, already contains both tightness (conditional entropy) and contrastive (entropy) parts. Table 2 presents the split for four DML losses. The rest of the section is devoted to exhibiting the close relationships between several pairwise losses and the tightness and contrastive terms (i.e., T and C).

Links between losses:

In this section, we show that the tightness and contrastive parts of the pairwise losses in Table 2, even though different at first sight, can actually be related to one another.

Table 2. Several well-known and/or recent DML losses broken into a tightness term and a contrastive term. Minimizing the cross-entropy corresponds to an approximate bound optimization of PCE.

Loss Tightness part / H( b Z|Y ) Contrastive part / H( b Z) Center [42] 1 2 n X i=1 kzi cy i k 2 1 n n X i=1 log piy i Contrast [7] 1 n n X i=1 X j:y j =y i D 2 ij 1 n n X i=1 X j:y j 6 =y i [m Dij] 2 + SNCA [43] 1 n n X i=1 log " X j:y j =y i exp D cos ij # 1 n n X i=1 log " X k6 =i exp D cos ik # MS [40] 1 n n X i=1 1 ↵ log " 1 + X j:y j =y i e ↵(D cos ij m) # 1 n n X i=1 1 log " 1 + X j:y j 6 =y i e (D cos ij m) # PCE Prop. 1 1 2 n 2 n X i=1 X j:y j =y i z T i zj 1 n n P i=1 log " K P k=1 exp h 1 n n P j=1 pjkz T i zj i # 1 2K 2 2 K X k=1 kc s k k 2
Lemma 1. Let T A denote the tightness part of the loss from method A. Assuming that features are `2-normalized, and that classes are balanced, the following relations between Center [42], Contrastive [START_REF] Hadsell | Dimensionality reduction by learning an invariant mapping[END_REF], SNCA [43] and MS [40] losses hold:

T SNCA c  T Center c = T Contrastive c  T MS (8)
Where c  stands for lower than, up to a multiplicative and an additive constant, and c = stands for equal to, up to a multiplicative and an additive constant.

The detailed proof of Lemma 1 is deferred to the supplemental material. As for the contrastive parts, we show in the supplemental material that both C SNCA and C MS are lower bounded by a common contrastive term that is directly related to H( Ẑ). We do not mention the contrastive term of center-loss, as it represents the cross-entropy loss, which is exhaustively studied in Section 4.

Cross-entropy does it all

We now completely change gear to focus on the widely used unary classification loss: cross-entropy. On the surface, the cross-entropy may seem unrelated to metric-learning losses as it does not involve pairwise distances. We show that a close relationship exists between these pairwise losses widely used in deep metric learning and the cross-entropy classification loss. This link can be drawn from two different perspectives, one is based on an explicit optimization insight and the other is based on a discriminative view of the mutual information. First, we explicitly demonstrate that the cross-entropy is an upper bound on a new pairwise loss, which has a structure similar to all the metric-learning losses listed in Table 2, i.e., it contains a tightness term and a contrastive term. Hence, minimizing the cross-entropy can be seen as an approximate bound-optimization (or Majorize-Minimize) algorithm for minimizing this pairwise loss. Second, we show that, more generally, minimization of the cross-entropy is actually equivalent to maximization of the mutual information, to which we connected various DML losses. These findings indicate that the cross-entropy represents a proxy for maximizing I( b Z, Y ), just like pairwise losses, without the need for dealing with the complex sample mining and optimization schemes associated to the latter.

The pairwise loss behind unary cross-entropy

Bound optimization: Given a function f (W) that is either intractable or hard to optimize, bound optimizers are iterative algorithms that instead optimize auxiliary functions (upper bounds on f ). These auxiliary functions are usually more tractable than the original function f . Let t be the current iteration index, then a t is an auxiliary function if:

f (W)  a t (W) , 8 W f (W t ) = a t (W t ) (9)
A bound optimizer follows a two-step procedure: first an auxiliary function a t is computed, then a t is minimized, such that:

W t+1 = arg min W a t (W) (10)
This iterative procedure is guaranteed to decrease the original function f :

f (W t+1 )  a t (W t+1 )  a t (W t ) = f (W t ) (11)
Note that bound optimizers are widely used in machine learning. Examples of well-known bound optimizers include the concave-convex procedure (CCCP)

[48], expectation maximization (EM) algorithms or submodular-supermodular procedures (SSP) [START_REF] Narasimhan | A submodular-supermodular procedure with applications to discriminative structure learning[END_REF]. Such optimizers are particularly used in clustering [START_REF] Tang | Kernel cuts: Kernel and spectral clustering meet regularization[END_REF] and, more generally, in problems involving latent-variable optimization.

Pairwise Cross-Entropy: We now prove that minimizing cross-entropy can be viewed as an approximate bound optimization of a more complex pairwise loss.

Proposition 1. Alternately minimizing the cross-entropy loss L CE with respect to the encoder's parameters W and the classifier's weights ✓ can be viewed as an approximate bound-optimization of a Pairwise Cross-Entropy (PCE) loss, which we define as follows: The full proof of Proposition 1 is provided in the supplemental material. We hereby provide a quick sketch. Considering the usual softmax parametrization for our model's predictions b Y , the idea is to break the cross-entropy loss in two terms, and artificially add and remove the regularization term 2

L PCE = 1 2 n 2 n X i=1 X j:yj =yi z T i z j | {z } tightness part + 1 n n X i=1 log K X k=1 e 1 n n P j=1 pjkz T i zj 1 2 K X k=1 kc s k k 2 | {z } contrastive part (
P K k=1 ✓ T k ✓ k : L CE = 1 n n X i=1 ✓ T yi z i + 2 X k ✓ T k ✓ k | {z } f1(✓) + 1 n n X i=1 log K X k=1 e ✓ T k zi 2 K X k=1 ✓ T k ✓ k | {z } f2(✓) (13)
By properly choosing 2 R in Eq. ( 13), both f 1 and f 2 become convex functions of ✓. For any class k, we then show that the optimal values of ✓ k for f 1 and f 2 are proportional to, respectively, the hard mean c k = 1 |Zk| P i:yi=k z i and the soft mean c s k = 1 n P n i=1 p ik z i of class k. By plugging-in those optimal values, we can lower bound f 1 and f 2 individually in Eq. 13 and get the result.

Proposition 1 casts a new light on the cross-entropy loss by explicitly relating it to a new pairwise loss (PCE), following the intuition that the optimal weights ✓ ⇤ of the final layer, i.e., the linear classifier, are related to the centroids of each class in the embedded feature space Z. Specifically, finding the optimal classifier's weight ✓ ⇤ for cross-entropy can be interpreted as building an auxiliary function a t (W) = L CE (W, ✓ ⇤ ) on L P CE (W). Subsequently minimizing crossentropy w.r.t. the encoder's weights W can be interpreted as the second step of bound optimization on L P CE (W). Similarly to other metric learning losses, PCE contains a tightness part that encourages samples from the same classes to align with one another. In echo to Lemma 1, this tightness term, noted T PCE , is equivalent, up to multiplicative and additive constants, to T center and T contrast , when the features are assumed to be normalized:

T PCE c = T center c = T contrast (14)
PCE also contains a contrastive part, divided into two terms. The first pushes all samples away from one another, while the second term forces soft means c s k far from the origin. Hence, minimizing the cross-entropy can be interpreted as implicitly minimizing a pairwise loss whose structure appears similar to the well-established metric-learning losses in Table 2.

Simplified Pairwise Cross-Entropy: While PCE brings interesting theoretical insights, the computation of the parameter at every iteration requires computating the eigenvalues of a d ⇥ d matrix at every iteration (cf. full proof in supplemental material), which makes the implementation of PCE difficult in practice. In order to remove the dependence upon , one can plug in the same ✓ for both f 1 and f 2 in Eq. 13. We choose to use

✓ ⇤ 1 = arg min ✓ f 1 (✓) / [c 1 , ..., c K ] T .
This yields a simplified version of PCE, that we call SPCE: The proof of Lemma 2 is provided in the supplementary material. Such result is compelling. Using the discriminative view of mutual information allows to show that minimizing cross-entropy loss is equivalent to maximizing the mutual information I( b Z; Y ). This information theoretic argument reinforces our conclusion from Proposition 1 that cross-entropy and the previously described metric learning losses are essentially doing the same job.

L SP CE = 1 n 2 n X i=1 X j:yj =yi z T i z j | {z } tightness + 1 n n X i=1 log K X k=1 exp ⇣ 1 n X j:yj =k z T i z j ⌘ | {z } contrastive (15)

Then why would cross-entropy work better?

We showed that cross-entropy essentially optimizes the same underlying mutual information I( b Z; Y ) as other DML losses. This fact alone is not enough to explain why the cross-entropy is able to consistently achieve better results than DML losses as shown in Section 5. We argue that the difference is in the optimization process. On the one hand, pairwise losses require careful sample mining and weighting strategies to obtain the most informative pairs, especially when considering minibatches, in order to achieve convergence in a reasonable amount of time, using a reasonable amount of memory. On the other hand, optimizing cross-entropy is substantially easier as it only implies minimization of unary terms. Essentially, cross-entropy does it all without dealing with the difficulties of pairwise terms. Not only it makes optimization easier, but also it simplifies the implementation, thus increasing its potential applicability in real-world problems. [START_REF] Song | Deep metric learning via lifted structured feature embedding[END_REF] House furniture 22 634 120 053 In-shop Clothes Retrieval [START_REF] Liu | Deepfashion: Powering robust clothes recognition and retrieval with rich annotations[END_REF] C l o t h e s 7 9 8 2 5 2 7 1 2

Experiments

Metric

Most methods, especially recent ones, use the cosine distance to compute the recall for the evaluation. They include `2 normalization of the features in the model [START_REF] Ge | Deep metric learning with hierarchical triplet loss[END_REF][START_REF] Movshovitz-Attias | No fuss distance metric learning using proxies[END_REF][START_REF] Song | Deep metric learning via facility location[END_REF][START_REF] Opitz | Bier-boosting independent embeddings robustly[END_REF][START_REF] Sanakoyeu | Divide and conquer the embedding space for metric learning[END_REF]37,40,[45][46][47]49], which makes cosine and Euclidean distances equivalent. Computing cosine similarity is also more memory efficient and typically leads to better results [START_REF] Schroff | Facenet: A unified embedding for face recognition and clustering[END_REF]. For these reasons, the Euclidean distance on non normalized features has rarely been used for both training and evaluation.

In our experiments, `2-normalization of the features during training actually hindered the final performance, which might be explained by the fact that we add a classification layer on top of the feature extractor. Thus, we did not `2-normalize the features during training and reported the recall with both Euclidean and cosine distances.

Datasets

Four datasets are commonly used in metric learning to evaluate the performances. These datasets are summarized in Table 3. CUB [START_REF] Wah | The Caltech-UCSD Birds-200-2011 Dataset[END_REF], Cars [START_REF] Krause | 3d object representations for fine-grained categorization[END_REF] and SOP [START_REF] Song | Deep metric learning via lifted structured feature embedding[END_REF] datasets are divided into train and evaluation splits. For the evaluation, the recall is computed between each sample of the evaluation set and the rest of the set. In-Shop [START_REF] Liu | Deepfashion: Powering robust clothes recognition and retrieval with rich annotations[END_REF] is divided into a query and a gallery set. The recall is computed between each sample of the query set and the whole gallery set.

Training specifics

Model architecture and pre-training: In the metric learning literature, several architectures have been used, which historically correspond to the state-ofthe-art image classification architectures on ImageNet [START_REF] Deng | Imagenet: A large-scale hierarchical image database[END_REF], with an additional constraint on model size (i.e., the ability to train on one or two GPUs in a reasonable time). These include GoogLeNet [START_REF] Szegedy | Going deeper with convolutions[END_REF] as in [START_REF] Kim | Attention-based ensemble for deep metric learning[END_REF], BatchNorm-Inception [START_REF] Szegedy | Rethinking the inception architecture for computer vision[END_REF] as in [40] and ResNet-50 [START_REF] He | Identity mappings in deep residual networks[END_REF] as in [46]. They have large differences in classification performances on ImageNet, but the impact on performances over DML benchmarks has rarely been studied in controlled experiments. As this is not the focus of our paper, we use ResNet-50 for our experiments. We concede that one may obtain better performances by modifying the architecture (e.g., reducing model stride and performing multi-level fusion of features). Here, we limit our comparison to standard architectures. Our implementation uses the PyTorch [START_REF] Paszke | Pytorch: An imperative style, high-performance deep learning library[END_REF] library, and initializes the ResNet-50 model with weights pre-trained on ImageNet.

Sampling: To the best of our knowledge, all DML papers -including [49]use a form of pairwise sampling to ensure that, during training, each mini-batch contains a fixed number of classes and samples per class (e.g. mini-batch size of 75 with 3 classes and 25 samples per class in [49]). Deviating from that, we use the common random sampling among all samples (as in most classification training schemes) and set the mini-batch size to 128 in all experiments (contrary to [40] in which the authors use a mini-batch size of 80 for CUB, 1 000 for SOP and did not report for Cars and In-Shop).

Data Augmentation: As is common in training deep learning models, data augmentation improves the final performances of the methods. For CUB, the images are first resized so that their smallest side has a length of 256 (i.e., keeping the aspect ratio) while for Cars, SOP and In-Shop, the images are resized to 256 ⇥ 256. Then a patch is extracted at a random location and size, and resized to 224 ⇥ 224. For CUB and Cars, we found that random jittering of the brightness, contrast and saturation slightly improves the results. All of the implementation details can be found in the publicly available code.

Cross-entropy:

The focus of our experiments is to show that, with careful tuning, it is possible to obtain similar or better performance than most recent DML methods, while using only the cross-entropy loss. To train with the crossentropy loss, we add a linear classification layer (with bias) on top of the feature extraction -similar to many classification models -which produces logits for all the classes present in the training set. Both the weights and biases of this classification layer are initialized to 0. We also add dropout with a probability of 0.5 before this classification layer. To further reduce overfitting, we use label smoothing for the target probabilities of the cross-entropy. We set the probability of the true class to 1 ✏ and the probabilities of the other classes to ✏ K 1 with ✏ = 0.1 in all our experiments.

Optimizer: In most DML papers, the hyper-parameters of the optimizer are the same for Cars, SOP and In-Shop whereas, for CUB, the methods typically use a smaller learning rate. In our experiments, we found that the best results were obtained by tuning the learning rate on a per dataset basis. In all experiments, the models are trained with SGD with Nesterov acceleration and a weight decay of 0.0005, which is applied to convolution and fully-connected layers' weights (but not to biases) as in [START_REF] Jia | Highly scalable deep learning training system with mixed-precision: Training imagenet in four minutes[END_REF]. For CUB and Cars, the learning rate is set to 0.02 and 0.05 respectively, with 0 momentum. For both SOP and In-Shop, the learning rate is set to 0.003 with a momentum of 0.99.

Batch normalization: Following [40]

, we freeze all the batch normalization layers in the feature extractor. For Cars, SOP and In-Shop, we found that adding batch normalization -without scaling and bias -on top of the feature extractor improves our final performance and reduces the gap between `2 and cosine distances when computing the recall. On CUB, however, we obtained the best recall without this batch normalization.

Results

Results for the experiments are reported in Table 4. We also report the architecture used in the experiments as well as the distance used in the evaluation to compute the recall. `2 refers to the Euclidean distance on non normalized features while cos refers to either the cosine distance or the Euclidean distance on `2-normalized features, both of which are equivalent.

On all datasets, we report state-of-the-art results except on Cars, where the only method achieving similar recall uses cross-entropy for training. We also notice that, contrary to common beliefs, using Euclidean distance can actually be competitive as it also achieves near state-of-the-art results on all four datasets. These results clearly highlight the potential of cross-entropy for metric learning, and confirm that this loss can achieve the same objective as pairwise losses.

Conclusion

Throughout this paper, we revealed non-obvious relations between the crossentropy loss, widely adopted in classification tasks, and pairwise losses commonly used in DML. These relations were drawn under two different perspectives. First, cross-entropy minimization was shown equivalent to an approximate boundoptimization of a pairwise loss, introduced as Pairwise Cross-Entropy (PCE), which appears similar in structure to already existing DML losses. Second, adopting a more general information theoretic view of DML, we showed that both pairwise losses and cross-entropy were, in essence, maximizing a common mutual information I( Ẑ, Y ) between the embedded features and the labels. This connection becomes particularly apparent when writing mutual information in both its generative and discriminative views. Hence, we argue that most of the differences in performance observed in previous works come from the optimization process during training. Cross-entropy contains only unary terms, while traditional DML losses are based on pairwise-term optimization, which requires substantially more tuning (e.g. mini-batch size, sampling strategy, pair weighting). While we acknowledge that some losses have better properties than others regarding optimization, we empirically showed that the cross-entropy loss was also able to achieve state-of-the-art results when fairly tuned, highlighting the fact that most improvements have come from enhanced training schemes (e.g. data augmentation, learning rate policies, batch normalization freeze) rather than the intrinsic properties of pairwise losses. We strongly advocate that cross-entropy should be carefully tuned to be compared against as a baseline in future works.

Acknowledgments

This project was supported by the NSERC (Discovery Grant RGPIN 2019-05954). This project has received funding from the European Union's Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement 792464. 797805. 36. Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., Liu

A Proofs

A.1 Lemma 1

Proof. Throughout the following proofs, we will use the fact that classes are assumed to be balanced in order to consider Z k , for any class k, as a constant

|Z k | = n K .
We will also use the feature normalization assumption to connect cosine and Euclidean distances. On the unit-hypersphere, we will use that:

D cos i,j = 1 kzi zj k 2 2
.

Tightness terms: Let us start by linking center loss to contrastive loss. For any specific class k, let c k = 1 |Zk| P z2Zk z denotes the hard mean. We can write:

X zi2Zk kz i c k k 2 = X zi2Zk [kz i k 2 2z T i c k ] + |Z k | kc k k 2 = X zi2Zk kz i k 2 2 1 |Z k | X zi2Zk X zj 2Zk z T i z j + 1 |Z k | X zi2Zk X zj 2Zk z T i z j = X zi2Zk kz i k 2 1 |Z k | X zi2Zk X zj 2Zk z T i z j = 1 2 [ X zi2Zk kz i k 2 + X zj 2Zk kz j k 2 ] 1 |Z k | X zi2Zk X zj 2Zk z T i z j = 1 2|Z k | [ X zi2Zk X zj 2Zk kz i k 2 + X zi2Zk X zj 2Zk kz j k 2 ] 1 2|Z k | X zi2Zk X zj 2Zk 2z T i z j = 1 2|Z k | X zi,zj 2Zk kz i k 2 2z T i z j + kz j k 2 = 1 2|Z k | X zi,zj 2Zk kz i z j k 2 c = X zi,zj 2Zk kz i z j k 2
Summing over all classes k, we get the desired equivalence. Note that, in the context of K-means clustering, where the setting is different ‡ , a technically similar ‡ In clustering, the optimization is performed over assignment variables, as opposed to DML, where assignments are already known and optimization is carried out over the embedding.

result could be established [START_REF] Tang | Kernel cuts: Kernel and spectral clustering meet regularization[END_REF], linking K-means to pairwise graph clusteirng objectives. Now we link contrastive loss to SNCA loss. For any class k, we can write:

X zi2Zk log X zj 2Zk\{i} e D cos i,j c = X zi2Zk log 0 @ 1 |Z k | 1 X zj 2Zk\{i} e D cos i,j 1 A  X zi2Zk X zj 2Zk\{i} D cos i,j (|Z k | 1) c = X zi2Zk X zj 2Zk\{i} kz i z j k 2 2 (|Z k | 1) c = X zi2Zk X zj 2Zk\{i} kz i z j k 2
where we used the convexity of x ! log(x) and Jenson's inequality. The proof can be finished by summing over all classes k.

Finally, we link MS loss [40] to contrastive loss:

X zi2Zk 1 ↵ log 0 @ 1 + X zj 2Zk\{i} e ↵(D cos i,j 1) 1 A = X zi2Zk 1 ↵ log X zj 2Zk e ↵(D cos i,j 1) c = X zi2Zk 1 ↵ log 0 @ 1 |Z k | X zj 2Zk e ↵(D cos i,j 1) 1 A 1 |Z k | X zi,zj 2Zk (D cos i,j 1) c = X zi,zj 2Zk kz i z j k 2 ,
where we used the concavity of x ! log(x) and Jenson's inequality.

Therefore, while minimizing cross-entropy, we are implicitly both minimizing

H(Y | b Z) as well as D KL (Y k b Y | b Z).
In fact, following Eq. 36, optimization could naturally be decoupled in 2 steps, in a Maximize-Minimize fashion. One step would consist in fixing the encoder's weights W and only minimizing Eq. 36 w.r. Ideally, the KL term would vanish at the end of this step. In the following step, we would minimize Eq. 36 w.r.t to the encoder's weights W, while keeping the classifier fixed. In Fig. 1, we track the evolution of both loss functions and validation accuracy when training with L CE and L SP CE on MNIST dataset. We use a small CNN composed of four convolutional layers. The optimizer used is Adam. Batch size is set to 128, learning rate to 1e 4 with cosine annealing, weight decay to 1e 4 and feature dimension to d = 100. Fig. 1 supports the theoretical links that were drawn between Cross-Entropy and its simplied pairwise version SPCE. Particularly, this preliminary result demonstrates that SPCE is indeed employable as a loss, and exhibits a very similar behavior to the original cross-entropy. Both losses remain very close to each other throughout the training, and so remain the validation accuracies.

B Preliminary results with SPCE

C Analysis of ranking losses for Deep Metric Learning

Some recent works [START_REF] Cakir | Deep metric learning to rank[END_REF][START_REF] Rolínek | Optimizing rank-based metrics with blackbox differentiation[END_REF]39] tackle the problem of deep metric learning using a rank-based approach. In other words, given a point in feature space z i , the pairwise losses studied throughout this work try to impose manual margins m, so that the distance between z i and any negative point z j is at least m. Rank-based losses rather encourage that all points are well ranked, distance-wise, such that d(z i , z + j )  d(z i , z j ) for any positive and negative points z + j and z j . We show that our tightness/contrastive analysis also holds for such ranking losses. In particular, we analyse the loss proposed in [START_REF] Cakir | Deep metric learning to rank[END_REF]. For any given query embedded point z i , let us call D the random variable associated to the distance between z i and all other points in the embedded space, defined over all possible (discretized) distances D. Furthermore, let us call R the binary random variable that describes the relation to the current query point (R + and R describe respectively a positive and negative relationship to z i ). The loss maximized in [START_REF] Cakir | Deep metric learning to rank[END_REF] reads: To intuitively understand what those two terms are doing, let us imagine we approximate each of the expectations with a single point Monte-Carlo approximation. In other words, we sample a positive point z + j , take its associated distance to z i , which we call d + , then we approximate the tightness term as:

T AP ⇡ log P (D < d + |R + ) (39)
Maximizing T AP has a clear interpretation: it encourages all positive points to lie inside the hypersphere of radius d + around query point z i . Similarly:

C AP ⇡ log P (D < d + ) (40)
Maximizing C AP also has a clear interpretation: it encourages all points (both positive and negative ones) to lie outside the hypersphere of radius d + around query point z i . Now, Eq. 38 is nothing more than an expectation over all positive distance d + one could sample. Therefore, such loss can be analyzed through the same lens as other DML losses, i.e., one tightness term that encourages all points from the same class as z i to lie close to it in the embedded space, and one contrastive term that oppositely refrains all points from approaching z i closer than its current positive points.

12 )

 12 Where c s k = 1 n P n i=1 p ik z i represents the soft-mean of class k, p ik represents the softmax probability of point z i belonging to class k, and 2 R, > 0 depends on the encoder W .

4. 2 A discriminative view of mutual information Lemma 2 .

 22 Minimizing the conditional cross-entropy loss, denoted by H(Y ; b Y | b Z), is equivalent to maximizing the mutual information I( b Z; Y ).

  t to the classifier's weights ✓. At this step, H(Y | b Z) would be fixed while b Y would be adjusted to minimize D KL (Y || b Y | b Z).

Fig. 1 .

 1 Fig. 1. Evolution of the cross-entropy loss (CE) and the simplified pairwise cross-entropy (SPCE) during training on MNIST, as well as the validation accuracy for both losses.

FastAP = X d2D PP

 d2D (D < d|R + )P (R + ) P (D < d) P (D = d|R + )(37)Taking the logarithm, and using Jensen's inequality, we can lower bound this loss:(D = d, R + ) log( P (D < d|R + ) P (D < d) ) = E d⇠P (.,R + )log P (D < d|R + )

  SPCE and PCE are similar (the difference is that PCE was derived after plugging in the soft means instead of hard means in f 2 ). Contrary to PCE, however, SPCE is easily computable, and the preliminary experiments we provide in the supplementary material indicate that CE and SPCE exhibit similar behaviors at training time. Interestingly, our derived SPCE loss has a form similar to contrastive learning losses in unsupervised representation learning[START_REF] Chen | A simple framework for contrastive learning of visual representations[END_REF][START_REF] Oord | Representation learning with contrastive predictive coding[END_REF][START_REF] Tschannen | On mutual information maximization for representation learning[END_REF].

Table 3 .

 3 Summary of the datasets used for evaluation in metric learning.

	Name	Objects	Categories Images
	Caltech-UCSD Birds-200-2011 (CUB) [34]	B i r d s	2 0 0	1 17 8 8
	Cars Dataset [13]	C a r s	1 9 6	1 6 1 8 5
	Stanford Online Products (SOP)

Table 4 .

 4 Performance on CUB200, Cars-196, SOP and In-Shop datasets. d refers to the distance used to compute the recall when evaluating.

		Method	d	Architecture			Recall at
	Caltech-UCSD Birds-200-2011	Lifted Structure [29] `2 Proxy-NCA [17] c o s B N -I n c e p t i o n GoogLeNet HTL [5] c o s G o o g L e N e t ABE [12] c o s G o o g L e N e t HDC [47] c o s G o o g L e N e t DREML [45] c o s R e s N e t -1 8 EPSHN [46] c o s R e s N e t -5 0 NormSoftmax [49] cos ResNet-50 Multi-Similarity [40] cos BN-Inception D&C [25] c o s R e s N e t -5 0 Cross-Entropy `2 ResNet-50 cos	1 47.2 58.9 70.2 80.2 89.3 93.2 2 4 8 1 6 3 2 49.2 61.9 67.9 81.9 --57.1 68.8 78.7 86.5 92.5 95.5 60.6 71.5 79.8 87.4 --60.7 72.4 81.9 89.2 93.7 96.8 63.9 75.0 83.1 89.7 --64.9 75.3 83.5 ---65.3 76.7 85.4 91.8 --65.7 77.0 86.6 91.2 95.0 97.3 65.9 76.6 84.4 90.6 --67.6 78.1 85.6 91.1 94.7 97.2 69.2 79.2 86.9 91.6 95.0 97.3
					1	2	4	8	1 6	3 2
		Lifted Structure [29] `2	GoogLeNet	49.0 60.3 72.1 81.5 89.2 92.8
		Proxy-NCA [17]	c o s B N -I n c e p t i o n	73.2 82.4 86.4 88.7	-	-
	Stanford Cars	HTL [47] EPSHN [46] HDC [47] Multi-Similarity [40] cos BN-Inception c o s G o o g L e N e t c o s R e s N e t -5 0 c o s G o o g L e N e t D&C [25] c o s R e s N e t -5 0 ABE [12] c o s G o o g L e N e t	81.4 88.0 92.7 95.7 97.4 99.0 82.7 89.3 93.0 ---83.8 89.8 93.6 96.2 97.8 98.9 84.1 90.4 94.0 96.5 98.0 98.9 84.6 90.7 94.1 96.5 --85.2 90.5 94.0 96.1 --
		DREML [45]	c o s	R e s N e t -1 8	86.0 91.7 95.0 97.2	-	-
		NormSoftmax [49]	cos	ResNet-50	89.3 94.1 96.4 98.0	-	-
		Cross-Entropy	`2 cos	ResNet-50	89.1 93.7 96.5 98.1 99.0 99.4 89.3 93.9 96.6 98.4 99.3 99.7
	Stanford Online Product	Lifted Structure [29] `2 HDC [47] c o s HTL [5] c o s D&C [25] c o s ABE [12] c o s Multi-Similarity [40] cos BN-Inception GoogLeNet G o o g L e N e t G o o g L e N e t R e s N e t -5 0 G o o g L e N e t EPSHN [46] c o s R e s N e t -5 0 NormSoftmax [49] cos ResNet-50 Cross-Entropy `2 ResNet-50 cos	1 62.1 70.1 74.8 75.9 76.3 78.2 78.3 79.5 80.8 81.1		1 0 79.8 84.9 88.3 88.4 88.4 90.5 90.7 91.5 91.2 91.7	1 0 0 91.3 93.2 94.8 94.9 94.8 96.0 96.3 96.7 95.7 96.3	1 0 0 0 97.4 97.8 98.4 98.1 98.2 98.7 --98.1 98.8
	In-Shop Clothes Retrieval	HDC [47] DREML [45] HTL [5] D&C [25] ABE [12] EPSHN [46] NormSoftmax [49] Multi-Similarity [40] cos BN-Inception c o s G o o g L e N e t c o s R e s N e t -1 8 c o s G o o g L e N e t c o s R e s N e t -5 0 c o s G o o g L e N e t c o s R e s N e t -5 0 cos ResNet-50 Cross-Entropy `2 ResNet-50 cos	1 62.1 84.9 89.0 91.2 92.3 93.1 10 20 30 40 50 78.4 93.7 95.8 96.7 --80.9 94.3 95.8 97.2 97.4 97.8 85.7 95.5 96.9 97.5 -98.0 87.3 96.7 97.9 98.2 98.5 98.7 87.8 95.7 96.8 ---89.4 97.8 98.7 99.0 --89.7 97.9 98.5 98.8 99.1 99.2 90.6 97.8 98.5 98.8 98.9 99.0 90.6 98.0 98.6 98.9 99.1 99.2

  Wang, X., Hua, Y., Kodirov, E., Hu, G., Garnier, R., Robertson, N.M.: Ranked list loss for deep metric learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019) 40. Wang, X., Han, X., Huang, W., Dong, D., Scott, M.R.: Multi-similarity loss with general pair weighting for deep metric learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019) 41. Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research (JMLR) (2009) 42. Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: Proceedings of the European Conference on Computer Vision (ECCV) (2016) 43. Wu, Z., Efros, A.A., Yu, S.X.: Improving generalization via scalable neighborhood component analysis. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018) 44. Xing, E.P., Jordan, M.I., Russell, S.J., Ng, A.Y.: Distance metric learning with application to clustering with side-information. In: Advances in Neural Information

	, W.: Cosface:
	Large margin cosine loss for deep face recognition. In: Proceedings of the IEEE/CVF
	Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
	37. Wang, J., Zhou, F., Wen, S., Liu, X., Lin, Y.: Deep metric learning with angular
	loss. In: Proceedings of the IEEE International Conference on Computer Vision
	(ICCV) (2017)
	38. Wang, M., Sha, F.: Information theoretical clustering via semidefinite programming.
	In: Proceedings of the Fourteenth International Conference on Artificial Intelligence
	and Statistics (AIStats) (2011)
	39. Processing Systems (NeurIPS) (2003)
	45. Xuan, H., Souvenir, R., Pless, R.: Deep randomized ensembles for metric learning.
	In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)
	46. Xuan, H., Stylianou, A., Pless, R.: Improved embeddings with easy positive triplet
	mining. In: The IEEE Winter Conference on Applications of Computer Vision
	(WACV) (2020)
	47. Yuan, Y., Yang, K., Zhang, C.: Hard-aware deeply cascaded embedding. In: Proceed-
	ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
	(CVPR) (2017)
	48. Yuille, A.L., Rangarajan, A.: The concave-convex procedure (cccp). In: Advances
	in neural information processing systems (NeurIPS) (2002)
	49. Zhai, A., Wu, H.Y.: Classification is a strong baseline for deep metric learning. In:
	British Machine Vision Conference (BMVC) (2019)
	50. Zheng, W., Chen, Z., Lu, J., Zhou, J.: Hardness-aware deep metric learning. In: Pro-
	ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
	(CVPR) (2019)

where, again, we used Jenson's inequality in the second line above. The link between SNCA and contrastive loss can be established quite similarly:

Now, similarly to the reasoning carried out in Section 3.1, we can write:

Where the redundant tightness term is very similar to the tightness term in contrastive loss T contrast treated in details in Section 3.1. As for the truly contrastive part of C, it can also be related to the differential entropy estimator used in [38]:

In summary, we just proved that the contrastive parts of MS and SNCA losses are upper bounds on the contrastive term C. The latter term is composed of a proxy for the entropy of features H( Ẑ), as well as a tightness sub-term.

A.2 Proposition 1

Proof. First, let us show that L CE L P CE . Consider the usual softmax parametrization of point i belonging to class k:

, where z = W (x). We can explicitly write the cross-entropy loss:

.

Where we introduced 2 R. How to specifically set will soon become clear.

Let us now write the gradients of f 1 and f 2 in Eq. 22 with respect to ✓ k :

Notice that f 1 is a convex function of ✓, regardless of . As for f 2 , we set such that f 2 becomes a convex function of ✓. Specifically, by setting:

where

i and l (A) represents the l th eigenvalue of A, we make sure that the hessian of f 2 is semi-definite positive. Therefore, we can look for the minima of f 1 and f 2 .

Setting gradients in Eq. 23 and Eq. 24 to 0, we obtain that for all k 2 [1, K], the optimal ✓ k for f 1 is, up to a multiplicative constant, the hard mean of features from class k: ✓ f1⇤ k = 1 n P i:yi=k z i / c k , while the optimal ✓ k for f 2 is, up to a multiplicative constant, the soft mean of features:

. Therefore, we can write:

And

Putting it all together, we can obtain the desired result:

where c s k = 1 n P n i=1 p ik z i represents the soft mean of class k.

Let us now justify that minimizing cross-entropy can be seen as an approximate bound optimization on L P CE . At every iteration t of the training, cross-entropy represents an upper bound on Pairwise Cross-entropy.

When optimizing w.r.t ✓, the bound almost becomes tight. The approximation comes from the fact that ✓ f1⇤ k and ✓ f2⇤ k are quite dissimilar in early training, but become very similar as training progresses and the model's softmax probabilities align with the labels. Therefore, using the notation:

We can write:

Then, minimizing L CE and L P CE w.r.t W becomes approximately equivalent.

A.3 Lemma 2

Proof. Using the discriminative view of MI, we can write:

The entropy of labels H(Y ) is a constant and, therefore, can be ignored. From this view of MI, maximization of I( b Z; Y ) can only be achieved through a minimization of H(Y | b Z), which depends on our embeddings b Z = W (X). We can relate this term to our cross-entropy loss using the following relation:

D On the limitations of cross-entropy

While we demonstrated that the cross-entropy loss could be competitive in comparison to pairwise losses, while being easier to optimize, there still exist scenarios for which a straightforward use of the CE loss becomes prohibitive. Hereafter, we describe two such scenarios.

Case of relative labels: The current setting assumes that absolute labels are given for each sample, i.e., each sample x i belongs to a single absolute class y i . However, DML can be applied to more general problems where the absolute class labels are not available. Instead, one has access to relative labels that only describe the relationships between points (e.g., a pair is similar or dissimilar). From these relative labels, one could still define absolute classes as sets of samples inside which every pair has a positive relationship. Note that with this definition, each sample may belong to multiple classes simultaneously, which makes the use of standard cross-entropy difficult. However, with such re-formulation, our Simplified Pairwise Cross-Entropy (SPCE), which we hereby remind:

can handle such problems, just like any other pairwise loss.

Case of large number of classes: In some problems, the total number of classes K can grow to several millions. In such cases, even simply storing the weight matrix ✓ 2 R K⇥d of the final classifier required by cross-entropy becomes prohibitive. Note that there exist heuristics to handle such problems with standard cross-entropy, such as sampling subsets of classes and solving those sub-problems instead, as was done in [49]. However, we would be introducing new training heuristics (e.g., class sampling), which defeats the initial objective of using the cross-entropy loss. Again, the SPCE loss underlying the unary crossentropy could again handle such cases, similarly to other pairwise losses, given that it doesn't require storing such weight matrix.


Mohammadhadi Shateri

Francisco Messina

Pablo Piantanida

Fabrice Labeau

Privacy-Cost Management in Smart Meters Using Deep Reinforcement Learning

Keywords: Smart meters privacy, Privacy-cost trade-o↵, Deep reinforcement learning, Q-learning algorithm, Deep double Q-learning, Privacy-cost management unit

Smart meters (SMs) play a pivotal rule in the smart grid by being able to report the electricity usage of consumers to the utility provider (UP) almost in real-time. However, this could leak sensitive information about the consumers to the UP or a third-party. Recent works have leveraged the availability of energy storage devices, e.g., a rechargeable battery (RB), in order to provide privacy to the consumers with minimal additional energy cost. In this paper, a privacy-cost management unit (PCMU) is proposed based on a model-free deep reinforcement learning algorithm, called deep double Q-learning (DDQL). Empirical results evaluated on actual SMs data are presented to compare DDQL with the state-of-the-art, i.e., classical Q-learning (CQL). Additionally, the performance of the method is investigated for two concrete cases where attackers aim to infer the actual demand load and the occupancy status of dwellings. Finally, an abstract information-theoretic characterization is provided.

I. Introduction

S MART meters (SMs) play an important rule in the modern electricity network, known as the smart grid (SG), by providing fine-grained power consumption measurements of households to the utility provider (UP) almost in a real-time basis. Although these high-resolution SMs measurements can improve the e ciency, reliability, and flexibility of power infrastructures, they might be used by adversaries or malicious third-parties to violate the privacy of households [START_REF] Giaconi | Privacy-aware smart metering: Progress and challenges[END_REF]. For instance, by using non-intrusive load monitoring (NILM) approaches, an adversary with access to the power consumption load profiles can readily infer sensitive information related to consumers' daily habits and types of appliances owned.

To protect consumers' sensitive information, two main families of SM data privacy enabling techniques have been proposed in the literature: i) SM data manipulation methods [START_REF] Sankar | Smart meter privacy: A theoretical framework[END_REF]- [START_REF] Shateri | Deep directed information-based learning for privacy-preserving smart meter data release[END_REF]; and ii) demand shaping methods [START_REF] Kalogridis | Privacy for smart meters: Towards undetectable appliance load signatures[END_REF]- [START_REF] Erdemir | Privacy-cost trade-o↵ in a smart meter system with a renewable energy source and a rechargeable battery[END_REF]. In the first family, the SM data are processed and manipulated, e.g., by adding random noise, before sharing that with the UP. However, this might cause a significant loss in terms of the utility of the SM data [START_REF] Giaconi | Privacy-aware smart metering: Progress and challenges[END_REF]. On the other hand, approaches of the second family rely on using physical resources available at the dwelling such as a rechargeable battery (RB), electric vehicles (EVs), heating, ventilation, and air conditioning (HVAC) units, renewable energy source (RES), or a combination of them, to shape the way that energy is consumed by the user from the grid. In this case, the goal is to find an optimum energy management strategy, under the physical limitations of the resources, which provides maximum privacy with minimal household energy expenses.

In [START_REF] Li | Information-theoretic privacy for smart metering systems with a rechargeable battery[END_REF], the problem of finding an optimum strategy was formulated as a Markov decision process (MDP) and solved using dynamic programming. Concretely, a single-letter expression of the minimum information leakage was provided for the case that demand load is independent and identically distributed (i.i.d.). However, this result is mostly of theoretical value since the demand load is not i.i.d. in practice. In addition, the energy cost was not included in this problem formulation. In another study, the joint optimization of privacy, measured by fluctuations of the grid load around a constant load, and energy cost was considered and modeled as an MDP [START_REF] Sun | Smart meter privacy: Exploiting the potential of household energy storage units[END_REF]. To solve this MDP, a classical model-free algorithm from reinforcement learning called Q-learning [START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF] was used. This tabular algorithm iteratively learns the optimal state-action value function Q ⇤ from which an optimal policy for using the physical resources is readily obtained. Nevertheless, the classical Q-learning (CQL) convergence properties can be poor when the state and/or action spaces are large or infinite.

In this paper, we adopt the MDP formulation introduced in [START_REF] Sun | Smart meter privacy: Exploiting the potential of household energy storage units[END_REF] and use a deep learning approximation of the CQL algorithm called deep double Q-learning (DDQL) to overcome the previously discussed limitation of the classical algorithm. To the best of our knowledge, this is the first work that uses deep reinforcement learning to design a privacy-preserving mechanism for smart meters. For the sake of simplicity, we only consider an RB and assume that no energy can be sold to the grid (however, the problem formulation can be easily extended to di↵erent scenarios). In addition, the formulation of the cost is revised to take into account not only the total electricity cost but also the cost associated with the wear and tear of the battery. The performance of the DDQL algorithm is assessed using actual SMs data both from practical and theoretical perspectives. For the former evaluation, we consider two scenarios modeling di↵erent privacy threats. In the first scenario, an attacker observes the grid load and attempts to infer the consumer's demand load; while in the second scenario an attacker is trained to infer the occupancy status of the household. The latter evaluation is done by studying the trade-o↵ between the cost and the mutual information between the demand and grid loads. Fig. 1 shows the privacy-preserving framework for a household based on a single rechargeable battery (RB). Here, the attacker can be the UP itself, an eavesdropper or a third-party. In this framework, a privacy-cost management unit (PCMU) is designed to determine the optimal charging/discharging rate of the battery with the purpose of masking the consumer's demand load while minimizing the total electricity cost. Let y t be the consumer's demand load/power at time t, where t 2 T = {1, . . . , T }. Given the demand load y t and the level of charge of battery LOC RB,t 2 [0, 1] at time t, the PCMU should determine the optimum charging/discharging rate q RB,t to hide the consumer's demand load. Therefore, the SM measures and reports the masked demand load (i.e., the grid load) given by z t = y t + q RB,t .

(

The PCMU should be able to limit the performance of a potential attacker trying to infer sensitive information (which could be either y t or a correlated variable) from z t . In this framework, there is a trade-o↵ between privacy and cost associated to energy and wear and tear of the battery.

B. Markov Decision Process (MDP) Model

The problem of finding an optimal policy for the PCMU (i.e., the reinforcement learning agent in our problem) can be formalized using the framework of Markov decision processes (MDPs) [START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF], as in [START_REF] Sun | Smart meter privacy: Exploiting the potential of household energy storage units[END_REF]- [START_REF] Erdemir | Privacy-cost trade-o↵ in a smart meter system with a renewable energy source and a rechargeable battery[END_REF]. An MDP is determined by a state space S, which defines the possible states; an action space A(s) for each s 2 S, which describes the feasible actions at state s (we use A to denote [ s2S A(s)); the environment dynamics p(s t+1 |s t , a t ), which dictates how the current state s t changes when action a t is taken; and the reward function r(s t , a t ) which gives the immediate reward received at state s t by taking action a t ; and the discount factor 2 [0, 1] which determines the decay rate for weighting future rewards. In this study, we consider a finite horizon time model, i.e. we assume T < 1. Starting from an initial state s 1 , and by following a policy ⇡(a|s) = p(a|s), the PCMU takes an action a 1 , receives a reward r 1 , and transitions to state s 2 . This leads to generating a sequence of states, actions and rewards:

[s 1 , a 1 , r 1 , . . . , s T , a T , r T ], which is referred to as an episode. The action-value function Q :

S ⇥ A ! R is defined as Q(s, a) = E ⇡  P T t=1 t 1 r (s t , a t ) s 1 = s, a 1 = a and determines
how good is to take action a 2 A(s) in state s 2 S and then following the policy ⇡. An optimal policy ⇡ ⇤ is one that maximizes Q(s, a) for all possible s and a. In our problem, the state is defined as s t = [LOC RB,t , y t ] T while the action is defined as the RB charging/discharging rate, i.e. a t = q RB,t , where a positive q RB,t indicates the RB is charging and a negative q RB,t means discharging of RB for supplying the consumer's demand. In addition, the environment transition probability p(s t+1 |s t , a t ) can be written as follows:

p s t+1 |s t , a t (i) = p ⇣ LOC RB,t+1 |LOC RB,t , q RB,t ⌘ ⇥ p y t+1 |y t (2)
where (i) is due to the assumption that the consumer's demand load is independent of the action and level of charge of battery.

The factor p

⇣ LOC RB,t+1 |LOC RB,t , q RB,t
⌘ is determined by the dynamics and physical constraints of the battery, which can be summarized as follows [START_REF] Sun | Smart meter privacy: Exploiting the potential of household energy storage units[END_REF]:

q min RB  q RB,t  q max RB , (3)
LOC RB,t+1 = LOC RB,t + q RB,t ⇥ t ⇥ e RB C RB , (4)
LOC min RB  LOC RB,t  LOC max RB , (5)
where q min RB and q max RB are the minimum and maximum charging rate of the RB, LOC min RB and LOC max RB are the minimum and maximum level of charge of the RB, t is the load sampling rate, e RB is the charging e ciency factor of the RB, and C RB is the capacity of the RB. However, note that p y t+1 |y t is unknown and in general di cult to approximate [START_REF] Sun | Smart meter privacy: Exploiting the potential of household energy storage units[END_REF]. Thus, we do not assume full knowledge of the environment dynamics.

It remains to determine the reward function, which can inversely be interpreted as a loss function: `(s t , a t ) = r (s t , a t ). Following [START_REF] Sun | Smart meter privacy: Exploiting the potential of household energy storage units[END_REF], the loss function will be defined as a convex combination of a privacy measure term and an electricity cost term. In particular, privacy is defined as the distance between the reported load by the SM and a (predetermined) flat load. Intuitively, when (regardless of the consumer's demand) a constant load is reported by the SM, we expect that no private information about the consumer is leaked. Concretely, considering l c as the desired constant load, and following policy ⇡, privacy is defined as the average relative distance of the grid load from the target load as follows:

F ⇡ := 1 T T X t=1 f (s t , a t ) = 1 T T X t=1 z t l c l c . (6)
It should be noted that other notions of privacy have also been considered for this problem [START_REF] Giaconi | Privacy-aware smart metering: Progress and challenges[END_REF]. On the other hand, the cost that is incurred by the user can be related to the electricity cost and also to the cost associated to the battery wear and tear. In this study, we assume that no energy can be sold to the grid. Therefore, the average electricity cost over time horizon T is computed as follows:

C ⇡ = 1 T T X t=1 t ⇥ h t ⇥ [z t ] + , (7)
where h t is the price of 1 kWh of energy purchased from grid at time t and [x] + = max(x, 0). Since [z t ] + = h y t + q RB,t i +  y t + q RB,t , and the PCMU has no control on y t , we conclude that minimizing the average additional cost

G ⇡ := 1 T T X t=1 g(s t , a t ) = 1 T T X t=1 t ⇥ h t ⇥ q RB,t , (8)
minimizes an upper bound of C ⇡ . In addition, notice that G ⇡ takes into account the battery wear and tear cost since it grows when the battery use increases. Finally, considering equations ( 6) and ( 8), the one-step loss function is defined as follows:

`(s t , a t ) = r (s t , a t ) = g(s t , a t ) + (1 ) f (s t , a t ), (9)
where 2 [0, 1] controls the privacy-cost trade-o↵.

III. Methodology

A. Classical Q-Learning (CQL) Algorithm
The Q-Learning (QL) algorithm [START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF] is a simple method to learn the optimal state-action value function Q ⇤ . Then, the best action at state s is readily obtained by maximizing Q ⇤ (s, •). In the process of learning, the Q-Learning algorithm takes an action a t at state s t using some policy, transitions to state s t+1 and updates the Q function as follows [START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF]:

Q (s t , a t ) = ↵ ✓ r (s t , a t ) + max a2A(s t+1 ) Q (s t+1 , a) Q (s t , a t ) ◆ , (10)
where ↵ 2 [0, 1] is the learning rate, which should be selected wisely for the sake of convergence [START_REF] Szepesvári | The asymptotic convergence-rate of q-learning[END_REF]. In general, the Q-Learning algorithm uses the ✏ greedy policy, in which a random action is taken with probability ✏, and the action which maximizes the Q-value is taken with probability 1 ✏. In this way, while the algorithm exploits the learned Q function, it also explores the action space to discover the benefits of taking other actions (this is known as the explorationexploration dilemma). Therefore, starting from an initial state and initializing the Q function/table to some arbitrary values, we take an action using the ✏ greedy policy to move to the next state and update the Q function based on [START_REF] Erdemir | Privacy-cost trade-o↵ in a smart meter system with a renewable energy source and a rechargeable battery[END_REF]. This is continued for all the next states until the end of episode. The process is then repeated for several episodes to ensure all the state-actions are visited enough times.

B. Deep Double Q-Learning (DDQL) Algorithm

One main issue regarding the classical Q-Learning (CQL) algorithm is that it needs to visit all the state-action pairs several times to provide a good approximation of Q ⇤ . Therefore, for large MDPs with many states and actions, convergence is very slow. To resolve this problem, the Q-function can be approximated by using a deep neural network (DNN). These new methods, where deep learning is used for approximating the Q-function, are called deep Q-Learning (DQL) methods [START_REF] Volodymyr | Humanlevel control through deep reinforcement learning[END_REF], [START_REF] Franc ¸ois-Lavet | An introduction to deep reinforcement learning[END_REF]. Recently, the DQL method was used for household energy management in [START_REF] Mocanu | On-line building energy optimization using deep reinforcement learning[END_REF]. The main idea of the DQL method studied here is to approximate Q ⇤ (s, a) ⇡ Q(s, a; ✓) where ✓ are the parameters of a DNN called the Q-Network. The Q-Network takes the state s 2 S at the input and generates Q(s, a) at the output for di↵erent actions a 2 A. To define the objective function for this Q-network, we observe from [START_REF] Erdemir | Privacy-cost trade-o↵ in a smart meter system with a renewable energy source and a rechargeable battery[END_REF] that convergence is obtained when the term in parenthesis is equal to zero. The term r (s t , a t ) + max a2A(s t+1 ) Q (s t+1 , a; ✓) can be interpreted as the target, while Q (s t , a t ; ✓) is the output of the Q-Network. Thus, the mean squared error loss between target and output can be used as the loss function for training the Q-Network. However, using the same network to compute the target and output often leads to instability [START_REF] Mnih | Human-level control through deep reinforcement learning[END_REF]. To address this issue, the so-called double Q-learning algorithm was proposed in [START_REF] Hasselt | Double q-learning[END_REF] and extended to the deep learning setting in [START_REF] Van Hasselt | Deep reinforcement learning with double q-learning[END_REF]. In the DDQL algorithm, a second network called the Target Network (with parameters ✓ 0 ) is used to calculate the target term. The Target Network parameters ✓ 0 are periodically updated by simply copying the parameters from the Q-Network. Thus, using the Target Network, the objective function of the Q-Network can be written as follows:

L QN (✓) =E ✓ r(s t , a t ) + max a2A(s t+1 ) Q(s t+1 , a; ✓ 0 ) Q(s t , a t ; ✓) ◆ 2 . ( 11
)
The training of the DDQL is presented in Algorithm 1. Select feasible action a t using ✏ greedy algorithm.

7:

Calculate reward r(s t , a t ) from equation ( 9).

8:

Update the next state s t+1 based on (4) and ( 5).

9:

Import (s t , a t , r(s t , a t ), s t+1 ) into the replay bu↵er.

10:

Every k 0 step, update the Q-network by minimizing (11) using samples from the replay bu↵er.

11:

Every k step, update the target network by copying the Q-network parameters.

12:

end for 13: end for

IV. Results and Discussion

A. Description of data set and parameters

In this study the electricity consumption and occupancy (ECO) data set published by [START_REF] Beckel | The eco data set and the performance of non-intrusive load monitoring algorithms[END_REF] is used. This data set includes 1 Hz electricity usage measured by smart meters and occupancy labels gathered through a tablet computer and a passive infrared sensor. In this study, the data set is sampled with a sampling rate t = 15 min and an episode is defined over a day. Totally 2700 samples (each a vector with length of 96) are used and split into training, validation, and test sets with ratio 70:10:20, respectively. The validation dataset is used to set the values of the hyperparameters for each algorithm, which are discussed in the next section. The desired constant load is l c = 0.7 kW and the parameters of the battery are as follows: C RB = 10 kWh, e RB = 1, q max RB = q min RB = 4 kW, LOC max RB = 1 and LOC min RB = 0. We consider the winter Timeof-Use tari↵ o↵ered by Ontario/Canada, in which the o↵-peak price is $0.101 kWh during 19:00 to 7:00, the mid-peak price is $0.144 kWh during 11:00 to 17:00, and the on-peak price is $0.208 kWh during 7:00 to 11:00 and during 17:00 to 19:00.

B. Deep double Q-learning versus Classical Q-learning

In this section, the performance of the DDQL method is empirically compared with CQL. For both methods, the network configuration and the value of the hyperparameters are determined empirically by the best privacy-cost trade-o↵ over the validation dataset.

Considering the tabular setting of the CQL algorithm, the state and action states need to be quantized. To this end, a total number of 160 and 100 quantization levels were used for action and demand load, respectively. Therefore, according to the equation ( 4) and our definition of state s t the Q function would be a table with size 800 ⇥ 100 ⇥ 160. A total number of 25K episodes (⇡ 2.5M steps) are used with a discount factor = 0.8 and an adaptive learning rate ↵ decreasing linearly from 0.5 to 0.05 over 1M steps.

On the other hand, in the DDQL method, a multilayer perceptron (MLP) with two hidden each including 64 neurons and rectified linear unit (ReLU) as activation function, is used for both the Q and Target Networks. A total number of 800 episodes are used and the discount factor is set as 0.99. The size of the experience replay memory is 10K tuples. The memory gets sampled to update the Q-network every 8 steps (k 0 =8), with minibatches of size 128, and a Target Network copy step k of 500 steps. The RMSProp optimizer with learning rate 0.00025 is selected to train the network.

In the following, the performances of the CQL and DDQL algorithms used to train the PCMU unit are evaluated. Fig. 2 shows how the total episodic reward evolves during the training of each algorithm (for = 0). From this figure, it is clear that, compared with the CQL, the DDQL algorithm leads to higher rewards in a much smaller number of episodes. To compare the performance of these two algorithms on the test dataset, the trade-o↵ between electricity cost in equation [START_REF] Giaconi | Smart meter privacy with renewable energy and an energy storage device[END_REF] and privacy measure in equation ( 6) are examined for both algorithms in Fig. 3. It can be seen that, although for values of close to 1 both CQL and DDQL produce very similar results, for higher privacy levels DDQL significantly outperforms CQL. The reason can be understood by looking at Fig. 2, where the DDQL shows a much better convergence compared with the CQL. In other words, due to the big state and action spaces, the CQL algorithm is not able to converge to the same level (even with a much larger number of episodes) and so provides degraded results compared with DDQL. These results confirm that, in our problem, the DDQL algorithm can provide better results than the classical CQL method.

C. Deep double Q-learning versus Attacker

In this section, we evaluate how DDQL results used in the smart meter privacy-preserving model can reduce attacker inference about user's private attributes. To this end, two scenarios are studied. In the first scenario, an attacker modeled as a neural network with three hidden layers (each with 32 neurons and ReLU activation functions) uses the grid load sequence z T (with length T = 96) to infer the user's demand load y T . In the second scenario, an attacker modeled as a neural network with two hidden layers (each with 44 neurons and ReLU activation functions) uses the sequences of grid load z T to infer the occupancy status of households. Both attackers are given a labeled dataset for training (which is a worst-case assumption), and for both of them the RMSProp optimizer with learning rate 0.001 is used. The performances of both attackers are presented in Fig. 4. Notice from the results that the PCMU, using the DDQL algorithm with a RB, can e↵ectively reduce the inference of private information by an attacker by controlling . For example, in the second scenario in which privacy does not matter ( = 1), the attacker can infer occupancy labels with a balanced accuracy of more than 85%, while for = 0 it is reduced to less than 65%. It should be noted that, by either using a larger battery or other resources, one could in principle approach full privacy (e.g., balanced accuracy of 50%).

Finally, to investigate the overall leakage of information about the consumers' demand load from the grid load, the mutual information (MI) between demand load and grid load is calculated based on the Kraskov-Stögbauer-Grassberger (KSG) estimation method (with 4 neighbors) [START_REF] Kraskov | Estimating mutual information[END_REF]. It should be noted that KSG is a non-parametric estimation method of the MI and is based on the k-th nearest neighbor. For more details on the KSG the readers is referred to [START_REF] Kraskov | Estimating mutual information[END_REF]. Results of the privacy-cost trade-o↵ are presented in Fig. 5. This confirms that the DDQL algorithm can reduce the information leakage by incurring a higher energy cost. V. Discussion and Concluding Remarks We examined the privacy-cost trade-o↵ for the SMs system equipped with a rechargeable battery (RB). The optimization problem was formulated as a Markov decisions problem (MDP) where in the reward function the additional cost due to using RB was included while privacy was measured as the average relative distance of the grid load from a constant load. A model-free deep reinforcement learning algorithm, called deep double Q-learning (DDQL) was used to tackle the problem and was compared with state of the art, classical Q-learning (CQL). The results using actual SM data showed that the DDQL algorithm outperforms the CQL method. In addition, the control of privacy attained by the DDQL was evaluated in two concrete case studies and in a general informationtheoretic sense. Future work will focus on extending our method to accommodate several resources and studying other privacy measures (e.g. MI) to enhance performance.

Fig. 1 .

 1 Fig. 1. Privacy-preserving model framework for smart meters based on a rechargeable battery.

Algorithm 1 : 3 :

 13 Training of the deep double Q-learning algorithm. Copy step k and training step k 0 are hyperparameters. 1: Initialize Q-network and target network. 2: for number of training episodes do Set the initial state s 1 = [LOC RB,1 , y 1 ].

Fig. 2 .

 2 Fig. 2. Total episodic reward for (a) DDQL versus (b) CQL during training.

Fig. 3 .

 3 Fig. 3. Average daily electricity cost versus average absolute relative deviation from target load (constant load) on test dataset for 2 [0, 1].

Fig. 4 .

 4 Fig. 4. Attackers performances: (a) inferring actual demand load, (b) inferring occupancy status of household.

Fig. 5 .

 5 Fig. 5. Electricity cost versus MI between demand load and grid load.

Acknowledgment

This work was supported by Hydro-Quebec, the Natural Sciences and Engineering Research Council of Canada, and McGill University in the framework of the NSERC/Hydro-Quebec Industrial Research Chair in Interactive Information Infrastructure for the Power Grid (IRCPJ406021-14). This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 792464.