Samuel Berlemont

Grégoire Lefebvre
email: gregoire.lefebvre@orange.com

Stefan Duffner

Christophe Garcia

Class-Balanced Siamese Neural Networks

Keywords: Siamese Neural Networks, Metric Learning, Human Action Recognition. HAR, with strategies tackling multi-view learning [1] or low-resolution [2]; and neural networks

niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

As consumer devices become more and more ubiquitous, new interaction solutions coping with the large variations and noise in unconstrained environments are required . In this study, we explore the use of Neural-Network-based approaches for similarity metric learning in the field of Human Activity Recognition (HAR). While video-based studies propose state-of-the-art performance in we focus here on inertial sensor data, with activities holding a semantic value such as climbing, running or jumping. In this context of motion-data-based activity or gesture recognition, three main approaches exist. The earliest methods suggest to model the temporal structure of a gesture class, for example with Hidden Markov Models (HMM) [START_REF] Zhang | A Framework for Hand Gesture Recognition Based on Accelerometer and EMG Sensors[END_REF]; while another approach consists in matching gestures with reference instances using a non-linear temporal alignment technique generally based on Dynamic Time Warping (DTW) [START_REF] Barczewska | Comparison of methods for hand gesture recognition based on Dynamic Time Warping algorithm[END_REF]. Finally, higher-level features, designed to be invariant to noise and deformations to some extent, can be extracted from gesture signals in order to train specific classifiers, such as Support Vector Machines (SVM) [START_REF] Wu | Gesture recognition with a 3-d accelerometer[END_REF]. For the real-time applications that we aim at here, speed and delay constraints are critical, leading us to the choice of neural-based models. While Extreme Learning Machines (ELM) [START_REF] Nguyen | Dual-layer kernel extreme learning machine for action recognition[END_REF], Bi-Directional Long Short-Term Memory (BLSTM) [START_REF] Lefebvre | Inertial Gesture Recognition with BLSTM-RNN[END_REF] and Convolutional Neural Networks (CNN) [START_REF] Duffner | 3D gesture classification with Convolutional Neural Networks[END_REF] have already been investigated, the main issue is to tackle an open-world problem, which does not only require a good classification performance but also an excellent generalization capability and the possibility to reject unknown classes and add new classes without retraining the existing model. Our work focuses thus on similarity metric learning between gesture sample signatures using the "Siamese" architecture, which aims at modelling semantic relationships between classes to extract discriminating features. Applied to the Single/Multiple Feed Forward Neural Network (FNN), this approach is called Siamese Neural Network (SNN). It differs from other neural-network based similarity metric models where neural networks are essentially used for dimensionality reduction [START_REF] Yu | Deep multimodal distance metric learning using click constraints for image ranking[END_REF], as the SNN is directly trained on semantic similarity information. In this study, after a pre-processing step where the data is filtered and normalized spatially and temporally, the SNN computes a higher-level representation of the gesture, where features are collinear for similar gestures, and orthogonal for dissimilar ones. We propose three contributions to the SNN: 1) We opt for a training strategy that does not require additional parameter fine-tuning. In contrast to classical input set selection strategies like pairs of examples: {similar, dissimilar} or triplets: {reference, similar, dissimilar}, not directly suited to multi-class problems, we propose to include at the same time a similar sample as well as samples from every available dissimilar class, resulting in a better structuring of the output space.

2) While the original SNN model gives good results for classification, multiple mathematical problems are identified, proving that its discrimination potential is not developed to its best capacity. Consequently, during training, we apply a regularization on the outputs to better condition the objective function.

3) We introduce the notion of polar sine to redefine the angular problem as the maximization of a normalized volume induced by the outputs of the reference and dissimilar samples, which results in a non-linear discriminant analysis. Indeed, this approach is the only one to take into account the information about every dissimilarity relationship within the training sets, and not only the relationships involving the reference sample.

With our experimental study on real world datasets, i.e. the Multimodal Human Activity Database (MHAD) [START_REF] Ofli | Berkeley MHAD: A comprehensive multimodal human action database[END_REF] and the Challenge for Multimodal Mid-Air Gesture Recognition for Close Human Computer Interaction Dataset (ChAirGest), we show very competitive results compared to the state-of-the-art for our proposed SNN configurations. This paper is organized as follows: Section 2 presents related work on SNNs. In Section 3, we propose our contributions. Then, Sections 4 and 5 describe our experimental protocols and results. A computational analysis of our proposals are discussed in Section 6. Finally, we draw our conclusions and present some perspectives in Section 7.

Siamese Neural Networks

A SNN learns a non-linear similarity metric, and essentially differentiates itself from classical neural networks by its specific training strategy involving sets of samples labelled as similar or dissimilar. The capabilities of different SNN-based methods depend on four main points: the network architecture, the training set selection strategy, the objective function, and the training algorithm [START_REF] Bromley | Signature Verification using a "Siamese" Time Delay Neural Network[END_REF]. In the following, we will explain the three first points in more detail as they are most related to our contributions.

Architecture

The name "Siamese" historically comes from the necessity to collect the state of a single network for two different activation samples during training. It can be seen as using two identical, parallel neural networks N N sharing the same set of weights W (see Fig. 1). These sub-networks each receive an input sample X, and produce output feature vectors O X that are supposed to be close for samples from the same class and far apart for samples from different classes, according to some distance measure, such as the cosine similarity metric. During the training step, the identical networks are joined at their ends by an output layer whose objective function E W is defined by a distance between features from each of the input samples. Siamese networks mainly involve two types of networks: CNN and MLP (Multi-Layer Perceptron).

) , cos( 2 1 X X O O ) , ( : 2 1 X X E Objective W 1 X O 2 X O NN W 2 X NN W 1 X Figure 1: Original SNN training architecture.
Bromley et al. [START_REF] Bromley | Signature Verification using a "Siamese" Time Delay Neural Network[END_REF] introduced the Siamese architecture in 1994, using a Siamese CNN with two sub-networks for a signature verification system handling time-series of hand-crafted low-level features. In 2005, Chopra, Hadsell and LeCun [START_REF] Chopra | Learning a similarity metric discriminatively, with application to face verification[END_REF] formalized the Siamese architecture applying a CNN on raw images for face verification, before adapting it to a dimensionality reduction technique [START_REF] Hadsell | Dimensionality reduction by learning an invariant mapping[END_REF]. More recently, Siamese CNNs have been used successfully for various tasks, such as person re-identification [START_REF] Yi | Deep metric learning for person re-identification[END_REF], speaker verification [START_REF] Chen | Extracting Speaker-Specific Information with a Regularized Siamese Deep Network[END_REF], and face identification [START_REF] Sun | Deep learning face representation by joint identification-verification[END_REF].

CNN-based architectures are more specific to image inputs, and several research works propose to use feed-forward perceptrons to handle more general vector inputs. For example, Yih et al. [START_REF] Yih | Learning discriminative projections for text similarity measures[END_REF] apply SFNNs to learn similarities on text data, Bordes et al. [START_REF] Bordes | Learning structured embeddings of knowledge 25 bases[END_REF] on entities in Knowledge Bases, and Masci et al. [START_REF] Masci | Multimodal Similarity-Preserving Hashing[END_REF] on multi-modal data. Other works on face verification use MLP Siamese architectures [START_REF] Lefebvre | Learning a bag of features based nonlinear metric for facial similarity[END_REF][START_REF] Hu | Discriminative Deep Metric Learning for Face Verification in the Wild[END_REF] or Restricted Boltzmann Machines (RBM) [START_REF] Nair | Rectified Linear Units Improve Restricted Boltzmann Machines[END_REF] to learn a non-linear similarity metric.

Training Set Selection

The selection strategy for training examples depends mostly on the application and the kind of knowledge about similarities that one wants to incorporate in the model. For many applications, such as face or signature verification, the similarity between samples depend on their "real-world" origin, i.e. faces/signatures from the same person, and the network allows to determine the genuineness of a test sample with a binary classification. Most approaches use pairs of training samples (X 1 , X 2 ) and a binary similarity relation which takes different values for similar and dissimilar [START_REF] Lefebvre | Learning a bag of features based nonlinear metric for facial similarity[END_REF] expand the information about expected neighbourhoods, and suggest a more symmetric representation: by considering a reference sample X R for each known relation, it is possible to define triplets (X R , X + , X -), with X + forming a genuine pair with the reference X R , 90 while X -is a sample from another class.

Objective Functions

The objective function computes a similarity metric between the higher-level features extracted from multiple input patterns. Minimizing this function iteratively during training ensures that the distance between similar patterns gets smaller, and the one between dissimilar gets larger. In this 95 regard, different metrics have been used in the literature:

Square Error. Given a network with weights W and two samples X 1 and X 2 with their labels Y , a target t(Y ) is defined for the cosine value between the two respective output vectors O X1 and O X2 as "1" for similar pairs and "-1" (or "0") for dissimilar pairs [START_REF] Bromley | Signature Verification using a "Siamese" Time Delay Neural Network[END_REF]:

E W (X 1 , X 2 , Y ) = (t(Y ) -cos(O X1 , O X2 )) 2 . ( 1
)
Triangular Similarity Metric. Zheng et al. [START_REF] Zheng | Triangular similarity metric learning for face verification[END_REF] imply the same targets and impose additional constraints on the norms of the output vectors O X1 and O X2 . Integrating a geometrical interpretation using the triangle inequality, the resulting objective function becomes:

E W (X 1 , X 2 , Y ) = 1 2 O X1 2 + 1 2 O X2 2 -O X1 + t(Y ) O X2 + 1 . ( 2
)
Triplet Similarity. Lefebvre et al. [20] use simultaneously targets for genuine and impostor pairs by presenting triplets of a reference X R , a positive X + and a negative X -sample. The output of the positive pair (O R , O + ) is trained to be collinear, whereas the output of the negative pair

(O R , O -
) is trained to be orthogonal. Thus:

E W (X R , X + , X -) = (1 -cos(O R , O + )) 2 + (0 -cos(O R , O -)) 2 (3)
Deviance. Yi et al. [14] use the binomial deviance to define their objective function:

E W (X 1 , X 2 , Y ) = ln exp -2t(Y ) cos(O X 1 ,O X 2 ) +1 (4)
Two Pairs. Yih et al. [START_REF] Yih | Learning discriminative projections for text similarity measures[END_REF] consider two pairs of vectors, (X p1 , X q1 ) and (X p2 , X q2 ), the first being known to have a higher similarity than the second. The main objective is then to maximize

∆ = cos(O Xp1 , O Xq1 ) -cos(O Xp2 , O Xq2 ) (5)
in a logistic loss function

E W (∆) = log(1 + exp(-γ∆)) , (6)
with γ being a scaling factor.

Probability-driven. Nair et al. [START_REF] Nair | Rectified Linear Units Improve Restricted Boltzmann Machines[END_REF] add a final neuron to their architecture whose activation function computes the probability P of two samples X 1 , X 2 being from the same class:

P = 1 1 + exp(-(w. cos(O X1 , O X2 ) + b)) , ( 7
)
with w and b being scalar parameters.

Norm-Based.

Several works [START_REF] Chopra | Learning a similarity metric discriminatively, with application to face verification[END_REF][START_REF] Hadsell | Dimensionality reduction by learning an invariant mapping[END_REF][START_REF] Sun | Deep learning face representation by joint identification-verification[END_REF][START_REF] Masci | Multimodal Similarity-Preserving Hashing[END_REF] propose to use the norm, e.g. 2 -norm, between the output vectors as a similarity measure:

d W (X 1 , X 2 ) = O X1 -O X2 2 . ( 8
)
For example, Chopra et al. [START_REF] Chopra | Learning a similarity metric discriminatively, with application to face verification[END_REF] define an objective composed of an "impostor" I (t(Y)=1) and a "genuine" G term (t(Y)=0):

E W (X 1 , X 2 , Y ) = (1 -t(Y ))E G W (X 1 , X 2 ) + t(Y ).E I W (X 1 , X 2 ) (9) with E G W (X 1 , X 2 ) = 2 Q (d W ) 2 , E I W (X 1 , X 2 ) = 2Qe (-2.77 Q d W )
, where Q is the upper bound of d W .

(

) 10
Statistical. Chen et al. [START_REF] Chen | Extracting Speaker-Specific Information with a Regularized Siamese Deep Network[END_REF] compute the first and second-order statistics, µ (i) and Σ (i) , over sliding windows on the SNN outputs of a speech sample i, and define the objective function as:

E W (X 1 , X 2 , Y ) = (1 -t(Y ))(D m + D S ) + t(Y ).(exp( -D m λ m ) + exp( -D S λ S )) , ( 11
)
where

D m = µ (i) -µ (j) 2 2 , D S = Σ (i) -Σ (j) 2 F ( 12
)
are incompatibility measures of these statistics between two samples i and j, λ m and λ s are tolerance bounds on these measures, and . F is the Frobenius norm.

Thus, SNNs differ from classical networks essentially by their training strategy involving multiple samples. However, since they produce "only" a similarity metric in their projection space, it is necessary to use another model on the projected data to obtain the final decision for classification (in the simplest case: a KNN classifier).

Proposed SNN

In the following, we propose multiple contributions aiming at increasing the discriminating potential of the network and improve its convergence for multi-class classification or verification.

Firstly, we suggest a more general way of defining similarities in multi-class problems, with training sets representing every class at once, since the pair or triplet strategies are not sufficient to define the complex relationships between the different classes. Indeed, they lead to biased and unclear training set selection strategies. Moreover, our mathematical analysis proves that cosine-based objectives can lead to numerical instabilities. Therefore, we introduce a regularization of the original metric, giving a simpler and computationally more efficient function. Finally, we propose to modify the objective function further by including a term based on the polar sine, allowing for a very elegant and effective solution to manage every similarity and dissimilarity relationship in the training process.

NN W k P O k P NN W 1 N O 1 N NN W k N O K N NN W ) ,..., , ( psine ), , cos( 1 K k k k N N R P R O O O O O k R O k R ) ( : k W T E Objective

Architecture

In our study, we use an MLP-based architecture with hyperbolic tangent activation functions for every layer. We choose a squared error cosine-based objective function as in [START_REF] Bromley | Signature Verification using a "Siamese" Time Delay Neural Network[END_REF] and [START_REF] Lefebvre | Learning a bag of features based nonlinear metric for facial similarity[END_REF] (see Eq. 1). We set the target value for dissimilar pairs to 0 instead of -1 to avoid instabilities in the training, as this target cannot be met for more than two classes. However, we do not use the common two-sub-network training architecture, since the latter is greatly modified by a our specific training set selection strategy presented below.

Training Set Selection Strategy

Existing training set selection strategies for SNNs consists in defining a subjective number of similar and dissimilar pairs deemed representative of the global relationships within the data.

Generally, this induces a bias, since it is difficult to ensure a good coverage for every relationship.

For this reason, we propose a unified approach for multi-class problems (see Fig.

E W1 (T k ) = (1 -cos(O R k , O P k )) 2 + l∈N k (0 -cos(O R k , O N l )) 2 . ( 13
)
Thus, our network architecture involves as many sub-networks as classes. Every sample is taken once as a reference, while the others are drawn at random. This facilitates the selection of representative samples and gives a global, non-parametric approach. Moreover, this strategy is similar to mini-batch learning, i.e. it limits the number of updates required before convergence. In order to improve convergence, we also study the behaviour of a weight update over the projected samples. In the following, we will analyse the cosine metric as a function of two vectors

of dimension n, cos X1,X2 : R 2n → R | (X 1 , X 2 ) → 1 2 (1 -cos(X 1 , X 2 )) 2 . ( 14
)
Let (O 1 , O 2 ) be a pair of outputs used for update. Given the functions We will show here that every stochastic gradient descent will increase the norms for both samples.

cos O1 : R n → R | X → 1 2 (1 -cos(O 1 , X)) 2 cos O2 : R n → R | X → 1 2 (1 -cos(X, O 2 )) 2 (15
Considering the function cos O1 , the update of O 2 is

O t+1 2 = O t 2 -λ.∇ cos O 1 (O 2 ), λ ∈ R. (16)
Figure 3 gives a graphical illustration in three dimensions. The line directed by the vector O2 O2 belongs to the equipotential for the cos O1 function. By definition, we can conclude that the di-

rectional derivative ∇ cos O 1 (O 2 ) is orthogonal to O 2 .
According to Pythagoras' theorem, we can conclude: Increasing the norms of the output vectors may incur progressive divergence. Moreover, with hyperbolic tangent activation functions, the output space is a hyper-cube of dimension n, which restricts the norms to a maximum of √ n. Therefore, we propose to add constraints on the norms of every output by forcing them to 1 and thus avoid any undesired saturation effects.

O t+1 2 2 = O t 2 2 + λ 2 ∇ cos O 1 (O 2 ) 2 ⇒ O t+1 2 > O t 2 . ( 17
) 140 9 O t 2 O t 1 λ.∇ cos O 1 (X 2 ) O t+1 2
We modify our objective function E W1 (see Eq.13) for a training subset T k : Thus, the final objective function for one training subset T k is defined as:

E W2 (T k ) = E W1 (T k ) + Xp∈T k (1 -O Xp ) 2 . ( 18
) Given ∀(O 1 , O 2 ) ∈ (R n , R n ), cos(O 1 , O 2 ) = O1•O2 O1 . O2 ,
E W (T k ) = (1 -O R k • O P k ) 2 + l∈N k (0 -O R k • O N l ) 2 + Xp∈T k (1 -O Xp ) 2 . ( 19
)
While this formulation is more suited to handle angular updates, it turns out impractical for an increasing number of classes. Moreover, in the derivative of the mean squared error objective, 150 the cosine error is weighted by the difference between the target and the cosine value which tends to zero and slows down the convergence. Thus, we propose a new error function that preserves the targets, while addressing both of these problems.

In the following, we propose a reformulation of the objective function based on a higherdimensional dissimilarity measure, the polar sine metric. We will then show that this new analysis leads to a non-linear discriminant analysis.

Inspired by the 2D sine function, Lerman et al. [START_REF] Lerman | On d-dimensional d-semimetrics and simplex-type inequalities for high-dimensional sine functions[END_REF] define the polar sine for a set V m = {v 1 , . . . , v n } of m-dimensional linearly independent vectors (m > n) as a normalized hyper-volume.

Given A = v 1 v 2 • • • v m
and its transpose A :

P olarSine(v 1 , . . . , v n ) = det (A A) n i=1 v i . ( 20
)
In the special case where m = n, the matrix product in the determinant is replaced by the square matrix A

Given the matrix S so that ∀(i, j) ∈ [1, .., n] 2 , S i,j = cos(v i , v j ), this measure can be rewritten as P olarSine(v 1 , . . . , v n ) = det (S). For numerical stability reasons during the derivation process and to make this value independent from the number of classes, we introduce the polar sine metric:

psine(v 1 , . . . , v n ) = n det (S). (21)

Polar sine for learning dissimilarities

The polar sine metric only depends on the angles between every vector of the set. It reaches its maximum value when all the vectors are orthogonal, and thus can be used as another metric for dissimilarity.

With two comparable similarity estimators whose values are between 0 and 1, one for similar and one for dissimilar samples, it is now possible to redefine the objective function for our training sets T k :

E W3 (T k ) = Esim W (T k ) + Esim W (T k ) , Esim W (T k ) = (1 -cos(O R k , O P k )) 2 , Esim W (T k ) = (1 -psine(O R k , O N1 , . . . , O N K )) 2 . ( 22
)
Optimizing the polar sine corresponds to assigning a target of 0 to the cosine value of every pair of outputs from different vectors drawn in T k \ {R k }, i.e. we assign a target for every pair of dissimilar samples. This actually holds more information than our original cosine or scalar objective functions, which would only define a target for pairs including the reference sample. As a consequence, the psine function allows for a truly complete representation, in every training set, of every available relationship present in the dataset. Given a set number of sources, the initial inputs are transformed into maximally independent, multi-dimensional components. Thus, our Siamese network, combined with this new objective function presents all the properties of a supervised, stochastic non-linear Independent Component Analysis, with one more advantage, as the number of components is adjustable by modifying the network output layer structure.

Experiments on the Multimodal Human Action Dataset

We state four hypotheses H 1 to H 4 about our contributions that we will experimentally validate on a small dataset, the Multimodal Human Action Dataset (MHAD):

• H 1 : an orthogonality objective for the cosine value between negative samples pairs leads to a better, more stable convergence.

• H 2 : a tuple-based training set selection strategy allows for a better representation of the relationships between classes, leading to a better structuring of the output space.

• H 3 : the proposed regularization scheme leads to a better convergence and a more stable norm evolution.

• H 4 : the polar sine metric-based objective, through a non-linear discriminant analysis, is more efficient at separating classes than other objective functions.

Dataset presentation and protocols

The Multimodal Human Action Dataset [START_REF] Ofli | Berkeley MHAD: A comprehensive multimodal human action database[END_REF] comprises the recordings from 12 participants performing 11 different actions (with 5 repetitions) which were designed to cover diverse dynamics of different body extremities and which were not specifically explained to the subjects beforehand in order to collect a representative range of different styles for each action. Figure 4 illustrates the different action types.

Although the MHAD comprises data collected from multiple types of sensors, we focus our study on two main sensors, consisting in the right wrist inertial sensor (A 1 ) and motion capture (M 20 ), as it gave the best classification results. A general amplitude scaling is performed, where each component of every sample forming a action record is divided by the maximum norm over all the samples of this action. Then, a lowpass filter is applied to the inertial signals to filter out the involuntary small shakes and electronic noise. Finally, the data are temporally normalized, with specific strategies suited to each method to compare. For the methods requiring a fixed-size input, the three-dimensional 30Hz inertial signal for each action record is re-sampled to a 135-dimensional (45×3) sample according to the estimated curvilinear length of the signal.

The same network architecture is selected for every SNN variant: a 3-layer SFNN, with an input layer size adapted to the data dimensionality (135 neurons for MHAD), a 45-neuron hidden layer, and a 90-neuron output layer. The hyperbolic tangent is chosen as the activation function for every neuron, and the learning rate is set to 0.001. During training, the network is independently and successively activated with every sample of a training set. Each activation state is stored, and reused to compute the weight updates. The computation details are available in [START_REF] Berlemont | Automatic non linear metric learning : Application to gesture recognition[END_REF].

The following aspects are explored:

Negative Target Selection (Hypothesis H 1 )

The target cosine value for dissimilar pairs can be set to -1 or to 0. We propose an analysis of both choices. Indeed,the latter allows for a better convergence as the minimum for the error function (0) can actually be attained, and it limits the space occupied by the samples. However, it also requires the output space dimension to be greater or equal to the number of classes represented in the training set, at the risk of overstraining the system and causing instability.

Training Set Selection Strategy -pairs, triplets or tuples (Hypothesis H 2 )

We compare three different SNN selection strategies, and assess their relative complexities and computation costs. Let N c be the number of classes, and N s the number of samples in the dataset.

The first strategy is the most common [START_REF] Bromley | Signature Verification using a "Siamese" Time Delay Neural Network[END_REF] and consists in selecting a specific number of pairs of samples, labelled as "similar" or "dissimilar". Here, we propose to form one similar pair with every sample of the dataset, where the second element of the pair is drawn at random. Dissimilar pairs are selected so as to represent every dissimilarity relationship available for every sample of the dataset, for a total of N c -1 dissimilar pairs per sample. The total number of updates for one

epoch is then equal to N c × N s , with 2(N c × N s ) activations.
The second strategy [START_REF] Lefebvre | Learning a bag of features based nonlinear metric for facial similarity[END_REF] favours a symmetry between similarities and dissimilarities, forming triplets, with one reference, one similar and one dissimilar sample per set. As for the first strategy, every dissimilarity relationship is represented. As a consequence, a triplet is formed for each dissimilarity, for a total of N c -1 positive and negative pairs per sample. As such, the total number of updates is equal to (N c -1) × N s , with three activations per update.

The third strategy consists in our contribution, which generalizes the triplets approach with tuples. Each sample is used as a reference once, in a training set representing every similarity and dissimilarity relationships. This limits the number of updates per epoch, with only N s updates, for [(N c + 1) × N s ] activations.

Cost Function: (Hypotheses

H 3 , H 4 )
Our study focuses on the comparison between the initial cosine-based error function, referred as cos, and our two contributions: the norm regularization, resulting in the scalar product-based objective function, referred as scal, and the polar sine metric-based function, referred as psine.

These three points lead to a total of 15 different SNN configurations (where the psine function is incompatible with -1 targets for negative examples). Each SNN is complemented by a KNN classifier, with K = 1, and the classification accuracy is reported. A leave-one-out strategy is adopted, with every sample from 11 users selected for training, and the samples from the last user for the testing phase.

Comparison with state-of-the-art models

We propose a comparison between the psine-based SNN and multiple gesture recognition approaches: geometric-based methods with Dynamic Time Warping (DTW) [START_REF] Barczewska | Comparison of methods for hand gesture recognition based on Dynamic Time Warping algorithm[END_REF], classifier-based meth- ods with the classical Support Vector Machine (SVM) [START_REF] Wu | Gesture recognition with a 3-d accelerometer[END_REF], and neural-based methods with the MLP.

We focus on two sensors: the accelerometer A 1 and the motion capture sensor M 20 , with the same leave-one-out strategy as described above.

Results

We subsequently discuss the results of each protocol:

Negative Target Selection

Table 2 shows the classification accuracy for the different configurations. For the cosine-based objective function, an orthogonality between negative vectors (target 0) produces better overall results for every training set selection strategy than the repulsion scheme (target -1). Only the scalar-based objective function with norm regularization gives a slight advantage to the -1 target for the negative pairs and triplets-based strategies. This phenomenon can be explained by the decomposition of the target angle in three sub-targets. The norm objectives have to be met perfectly for the scalar product objective to propose the right angle correction. These norms dynamically change with each update, and norm corrections may be favoured over angle corrections. Thus, a target equal to -1 amplifies the scalar product error, and gives an extra edge to the amplitude of the corresponding objective error over the norm objectives error, speeding up the convergence.

Thus, hypothesis H 1 is validated for the cosine and polar sine metric-based objective functions.

Training Set Selection Strategy

The training set selection strategy does not impact significantly the classification scores for the cosine-based objective function, with similar scores around 91.5% for the three strategies combined with a zero negative target. It is however interesting to note a reduced standard deviation for the triplet set strategy with a cosine-based objective, due to the increased representation of the similarity relationships which reduces the intra-class distances. Our proposed tuple selection strategy gives the best results on average.

Finally, the polar sine metric-based SNN shows a good performance in every case, comparable to the cosine-based SNN. While the pair and triplet strategies give the advantage to the cosine-based objective function, the potential of the polar sine metric approach is revealed with tuples, which cover every relationship available during training. This method shows then the best accuracy, equal to 91.8%, with a reduced standard deviation of 9.1% over the cosine-based one, whose accuracy is equal to 91.5% with a standard deviation of 10.2%. Thus, we can conclude that the polar sine metric-based objective gives competitive results, as it corrects the cosine values more reliably than the scalar product-based one. This analysis validates hypothesis H 2 , with a tuple-based strategy producing better or similar results as other strategies for the three objective functions. Moreover, H 4 is validated as well, as the polar sine metric-based objective combined with a tuple-based strategy produces the best classification score. However, the resulting projections differ for every objective functions, as seen below.

Projection Space Analysis

As explained in the previous section, a gradient descent update with the cosine function leads to instabilities in the norms of the outputs. Indeed, for a model where the gradient descent is simply performed on the components of the output vectors, we proved that these norms increase with each update. Figures 5a, 5b and5c show the evolution of the mean norms of the output and their corresponding standard deviations for the different negative targets and training set strategies, respectively combined with a cosine-based, scalar product-based and polar sine metricbased objective function.

First, one can observe that "-1" negative targets introduce an instability, which leads to the expected increasing norms for the pairs and triplets strategies, for both cosine and scalar productbased objective functions. Unexpectedly, with "0" negative targets, the norms tend to decrease quickly at first, then slightly but steadily with each update, for both cosine (see Fig. 5a) and polar sine metric-based (see Fig. 5c) objective functions. The scalar product-based strategy proves more reliable in that case (see Fig. 5b), although it implies a longer training with 1600 epochs.

Indeed, it is devised to counteract this norm variation phenomenon, and proves to be efficient with (c) psine mean output norms close to one and small standard deviations for the triplets and tuples strategies.

295

While the unit mean norm is not reached for the pairs strategy, it still stabilizes the norm evolution, validating hypothesis H 3 , except for tuple and triplet objectives combined with a -1 negative target whose mathematical instabilities lead to increasing norms.

As a conclusion, angle updates with SNN networks imply a compromise between efficient angle corrections and output norms stabilization. And the observed adverse effects of some the studied 3.

For our analysis, we focus on the classification rates obtained from isolated sensor data with the accelerometer A 1 and the motion capture sensor M 20 , and we report the results for the DTW and MLP methods proposed in [START_REF] Cumin | A priori Data and A posteriori Decision Fusions for Human Action Recognition[END_REF] and the SVM-based approach [START_REF] Chen | Improving Human Action Recognition Using Fusion of Depth Camera and Inertial Sensors[END_REF] on these same sensors.

The SNN-based approaches show globally superior results on the inertial sensor A 1 , with a lowest score of 88.6% for the SNN-scal, compared to a best score in the literature of 86.7% for the SVM approach. This shows that a SNN-based approach is very competitive. This conclusion is verified for the M 20 sensor. Our SNN-psine approach, implementing the polar sine metric and tuple-based set selection strategies contributions, show the best result of 92.4%.

Experiments on the ChAirGest dataset

We make a final hypothesis H 5 , which we seek to confirm the larger scale, "Challenge for Multimodal Mid-Air Gesture Recognition for Close Human Computer Interaction" Dataset (ChAirGest)

[28]:

• H 5 : the performance and competitive results of our psine-based SNN contribution can be generalized to other datasets.

Dataset presentation and protocol

This dataset contains 6 hours of continuous multimodal recordings from 10 subjects mimicking gestures seen on a computer screen. The data have been acquired from a Kinect camera and 4

Inertial Motion Units (IMUs) attached to the right arm of the subject (see Fig. 7). The dataset contains the 10 following gestures: Swipe left, Swipe right, Push to screen, Take from screen, Palmup rotation, Palm-down rotation, Draw a circle I, Draw a circle II, Wave hello and Shake hand.

In the following experiment, we use a concatenation of 3 Kinect points, K 2 , K 6 and K 10 , respectively associated with the head, the right hand, and the left hand. A 10-cross-validation testing is performed, with 400 samples drawn at random as the training set, 50 samples for the validation set, and 50 samples for the test set.

The same network architecture is used for our SNN in this experiment: a 3-layer SFNN, with an input layer size adapted to the data dimensionality (900 neurons for ChAirGest), a 45-neuron hidden layer, and a 90-neuron output layer.

We propose to compare our psine-based SNN to the three HAR state-of-the-art models previously identified on MHAD: DTW, MLP and SVM.

Results

This trend identified in our MHAD experiments is indeed confirmed on the ChAirGest dataset, see Table 4. While the MLP lacks the generalization potential to take into account the inter-subject variability with a classification rate of 87.8%, other methods such as the SVM or the KNN-based ones perform better, with a 89.2% rate for the DTW and 90.0% rate for the SVM. Our SNN-psine [START_REF] Cao | Multi-modal learning for gesture recognition[END_REF] who obtain a classification rate of 0.918% with their HMM decision fusion strategy. It is however important to note that these results are not directly comparable as they use the whole set of Kinect points, contrary to the 3 points we chose, with a different testing protocol, based on a leave-one-out cross validation.

Computational Complexity

Finally, we assess the computational cost of the three proposed training set selection strategies.

Table 5 shows the difference in the number of relationships represented in each update. The tuple-based objective function has a much more balanced representation, thus requiring more memory but at the same time reducing the number of necessary updates.

Table 6 illustrates this trade-off between the memory requirements, the number of back-propagations and updates necessary to present every available relationship to the model without bias, given a reference sample. Furthermore, there is a second trade-off between the required number of updates and the computational complexity of each update (as illustrated in Fig. 6a-6c). The polar sine metric-based SNN requires around twice the number of epochs of the cosine-based SNN to reach its optimum configuration. ). As such, the polar sine metric-based method can stay competitive for the most common small to medium-scale problems. One approach for larger-scale problems would be a decomposition into smaller sets of relationships in order to limit the amplitude of N c .

Table 7 reports the average training times for one update on MHAD (Computing times on an Intel Core i7-4800MQ CPU@2.70GHz), for each training set selection strategy and objective function. We can see that the order of magnitude identified above are respected, with a triplet strategy taking about 20% more time than the pair strategy, while the tuple strategy increases the update time by more than 400%, which is acceptable as the total number of updates per reference is divided by the number of classes.

Conclusion

We Future research may focus on the analysis of the proposed approaches for different tasks and types of data, as well as larger scale problems. Also, the possibility to include a temporal aspect in the model would be of great interest.

Figure 2 :

 2 Figure 2: Proposed training architecture of the SNN.

  2). Let C = {C 1 , .., C K } be the set of classes, O R k the output vector of the reference sample R k from the class C k presented to the model for update, P k the output of a different sample P k from the same class (i.e. the positive comparison), and O N l the output of a sample N l from another class C l (i.e. a negative comparison). In order to keep symmetric roles for every class and to optimize the update efficiency, we propose to minimize an error criterion for tuples. Given a class C k , and the set N k = [1, .., K] \ {k} of the indexes of every other class, a tuple is defined as T k = {R k , P k , {N l , l ∈ N k }} involving one reference sample from the class C k , one positive sample and one negative sample from every other class. The objective function becomes:

  ) respectively evaluated at the points O 2 and O 1 , the cos X1,X2 directional derivative at (O 1 , O 2 ) can be expressed as the concatenation of the two directional derivatives ∇ cos O 1 (O 2 ) and ∇ cos O 2 (O 1 ).

Figure 3 :

 3 Figure 3: An update step on the projection norm for a pair (O t 1 , O t 2 ). The sphere centre corresponds to the origin. The grey cone represents the equipotential surface for the function cos O 1 .

  we also propose to replace the cosine 145 distance for each pair by the scalar product of the pair outputs, since the norms of the two outputs are set to one during training.

Figure 4 :

 4 Figure 4: MHAD actions: jumping, jumping jacks, bending, punching, waving two hands, waving one hand, clapping, throwing, sit down/stand up, sit down, stand up.

Figure 5 :

 5 Figure 5: Mean output norm evolution for each objective function.

Figure 6 :

 6 Figure 6: Classification rate evolution for each objective function.

Figure 7 :

 7 Figure 7: Visuals of the ChairGest recording setup [28]. On the left, an image captured from the Kinect RGB stream. On the right, an external view of a subject performing a gesture in pseudo-recording conditions. Note that the accelerometers were hidden under the subject's clothes to avoid visual clues.

  The cosine and scalar-based objective functions with the tuples strategy imply an error computation complexity which is linear in terms of the output dimension N O and the number of classes N c , i.e. a complexity in O(N O N c ). While the polar sine metric function involves a matrix inversion which can be done very efficiently for low-dimensional matrices. Thus, the global complexity of the error resides in the matrix multiplication between the inverse cosine matrix and the normalized output matrix, for a final complexity in O(N O N 2 c

  proposed an adaptation of the Siamese strategy for neural networks to a multi-class classification context with stochastic training. Beyond the typical pairs and triplets of samples labelled as similar or dissimilar, a generalized training set selection strategy is suggested, which integrates one sample from every class, effectively simplifying the constitution of these training sets and balancing the representation of every relationship. Further, following the choice of the cosine similarity metric for the output space, mathematical flaws are identified, in particular concerning the uncontrolled norms of the outputs during training. Simplifying the original cosine function, angle updates are decomposed into three independent targets for a pair of vectors, namely a target on the scalar product between these vectors, and a target on their respective norms. Finally, we proposed a unified similarity function, the Polar Sine Metric, which holds and represents all the available information about dissimilarities in the training set, and leads to a supervised, stochastic non-linear Independent Component Analysis learning. We performed a thorough analysis of the different proposed strategies and SNN configurations and experimentally showed competitive performance on the task of human action recognition.

Table 1 :

 1 A summary of notations used in this paper. {R k , P k , {N l , l ∈ N k }} Tuple training set for an SNN update with a reference sample R

	Notation	Description
	Y	Similarity label for a pair of samples (similar,dissimilar)
	t(Y )	Target value for the similarity metric for a pair of samples
		with a similarity label Y
	C = {C 1 , .., C K }	Set of K classes
	N k = [1, .., K] \ {k}	Set of class indexes different from class k
	X j , O Xj	sample X from class j and its SNN output
		NB: X 1 , X 2 may be used as an example pair of random samples
	T k = from class k, a similar sample P, and a set of dissimilar samples
	W	Set of weights for the SNN
	E W	Error function applied for the SNN training
	pairs. Lefebvre et al.

Table 2 :

 2 Recognition rates for our MHAD protocols with different training set selection strategies (pairs, triplets, tuples) and different negative target values (1 for positives, -1 for negatives: "t-11", and 0 for negatives: "t01").

	A1		pairs	triplets	tuples
	targets	t-11	t01	t-11	t01	t-11	t01
	cos	0.901 ± 0.11 0.916 ± 0.10 0.881 ± 0.10 0.916 ± 0.09 0.880 ± 0.09	0.915 ± 0.10
	scal	0.883 ± 0.09	0.869 ± 0.09	0.871 ± 0.11	0.869 ± 0.11	0.880 ± 0.08	0.886 ± 0.08
	psine	-	0.910 ± 0.10	-	0.910 ± 0.08	-	0.918 ± 0.09

Table 3 :

 3 Comparison with the state of the art on MHAD. Finally, we compare the performances of each SNN objective function variant with a tuple-based training selection strategy, denominated cos, scal and psine. The results are shown in Table

		A 1	M 20
	DTW [26]	0.790 ± 0.107	0.888 ± 0.076
	MLP [26]	0.818 ± 0.099	0.913 ± 0.067
	SVM [27]	0.867	-
	cos	0.915 ± 0.102	0.906 ± 0.080
	scal	0.886 ± 0.079	0.910 ± 0.065
	psine	0.918 ± 0.091 0.924 ± 0.068
	4.3. Comparison with state-of-the-art results

Table 4 :

 4 Comparison with the state of the art on ChAirGest.

		K 2 K 6 K 10
	DTW	0.892 ± 0.018
	MLP	0.878 ± 0.038
	SVM	0.900 ± 0.046
	SNN-psine 0.910 ± 0.034

Table 5 :

 5 Number of relationships per update given a reference, with Nc the number of classes.

		pairs triplets	tuples
	cos	1	2	N c
	scal	1	2	N c
	psine	1	2	Nc(Nc-1) 2

+ 1 achieves the best result, with a classification rate of 91.0%. This score is competitive with the best scores available to this day on this dataset, in particular Cao et al.

Table 6 :

 6 Number of updates for each strategy given a reference, with Nc the number of classes.

		pairs triplets tuples
	networks/update	2	3	N c + 1
	backprop./update 2N c	3N c	N c + 1
	updates	N c	N c -1	1

Table 7 :

 7 Average Time for one update (in ms) on MHAD.

	A1	pairs triplets tuples
	cos	0.301	0.35	1.61
	scal	0.317	0.39	1.72
	psine 0.274	0.33	2.05

SNN variants are most likely accentuated when applied on larger scale problems.


Laura Rampazzi
email: laura.rampazzi@uninsubria.it

Barbara Giussani

Biagio Rizzo

Cristina Corti

Andrea Pozzi

Carlo Dossi

Monuments as sampling surfaces of recent traffic pollution

Keywords: Atmospheric particulate, Monument surface, Platinum, Rhodium, Electro analytical techniques

Background, aim, and scope. A new approach towards monuments, considering them as a passive sampler of pollution, is presented. Cultural Heritage objects suffer daily the damages of environmental pollution, especially in those areas interested by heavy traffic. Since monuments undergo only periodically conservation or maintenance works, surfaces are able to accumulate atmospheric deposit and to record changes in its composition. An optimised analytical protocol was developed in order to quantify platinum and rhodium at trace level on surfaces. The two elements have become tracers of automobile emissions in recent years, since the introduction of catalytic converters, and could have catalytic effects on the decay reactions of natural and artificial stone materials. As first case study, the cement mortar surfaces of a XX Century monument, the Camerlata Fountain, in Como (Italy) were investigated. Materials and methods. The surfaces of the monument were scraped in areas both exposed to atmosphere and sheltered by the architectural elements of the building. The powders were dissolved by microwave assisted mineralisation with a solution of HCl and HNO 3 . The solution was filtered, irradiated and analysed by Adsorptive Cathodic Stripping Voltammetry. The powders were also analysed by Infrared Spectroscopy and X-Ray Diffraction in order to determine the chemical and mineralogical composition. Results and discussion. An analysis protocol was set up considering the matrix effect and the expected low concentrations of the two metals. The results enlightened variable concentration values and distributions area of platinum (0.013-45 g/Kg) and rhodium (0.55-274.4 g/Kg) suggesting the ability of artificial stone surfaces to accumulate the two elements. The samples chemical and mineralogical composition was consistent with a typical cement plaster interested by decay phenomena. Conclusions. This work investigated the relation between Cultural Heritage and pollution by another point of view. The analytical protocol presented in this paper was effective in determining platinum and rhodium in traces on the investigated stone surfaces with negligible matrix effects. Recommendation and perspectives. The presence of platinum and rhodium on monuments surfaces should be of significant interest when planning Cultural Heritage conservation. A better knowledge of the two metals role in decay phenomena could impact in a positive way the artworks conservation.

Background, aim, and scope

Air quality and climate quickly changed in the last years requiring new researches focused on pollution effects, not only on environment and human health, but also on outdoor cultural heritage. As well known, the daily exposure to atmosphere in urban areas causes degradation processes on the stone monuments surfaces (natural stones, mortars, stuccoes) being soiling, concretions, films, black crusts and deposit the most diffused and investigated phenomena. A look into the literature enlightened how few is still known on the atmospheric aerosol impact on the built heritage, in particular on its content of metals coming from pollution [START_REF] Hamilton | Sulphur and nitrogen particulate pollutant deposition on to building surfaces[END_REF][START_REF] Schiavon | Nature and decay effects of urban soiling on granitic building stones[END_REF][START_REF] Rodriguez-Navarro | Role of particulate matter from vehicle exhaust on porous building stones (limestone) sulfation[END_REF][START_REF] Lammel | Pollutant fluxes onto the façades of a historical monument[END_REF][START_REF] Primerano | Possible alteration of monuments caused by particles emitted into the atmosphere carrying strong primary acidity[END_REF][START_REF] Delalieux | Environmental conditions controlling the chemical weathering of the Madara Horseman monument, NE Bulgaria[END_REF][START_REF] Esbert | Airborne particulate matter around the Cathedral of Burgos (Castilla y León, Spain)[END_REF][START_REF] Brimblecombe | Aesthetic thresholds and blackening of stone buildings[END_REF]. For decades lead content in aerosol was the indicator of petrol vehicles, but at the present time platinum, palladium and rhodium (i.e. PGE, Platinum Group Elements) are the new markers of traffic areas. The emission of these metals as particulate particles is a consequence of the degradation of catalytic converters, chemical and physical stresses caused by oxidation and reduction reactions, high working temperatures and mechanical abrasions, mainly the wear and tear mechanism of the external layer rich in the two catalysts [START_REF] Smichowski | Traffic-Related Elements in Airborne Particulate Matter[END_REF][START_REF] Wiseman | Airborne particulate matter, platinum group elements and human health: A review of recent evidence[END_REF]) being the most important causes. The mobility of aerosol and its content of PGE has been generally investigated on solid surfaces, mostly of environmental interest, e.g. snow, leaves, gullypot, soil, sediment, road dust (Smichowski et al.2008;[START_REF] Dubiella-Jackowska | Estimation of platinum in environmental water samples with solid phase extraction technique using inductively coupled plasma mass spectrometry[END_REF]Wisemanet al.2009). Outdoor monuments, particularly the widespread natural and artificial stone materials, may represent a new frontier. More researches concerning the metals catalytic activity and its role in the well-known degradation mechanisms would be developed [START_REF] Ausset | Experimental study of limestone and sandstone sulphation in polluted realistic conditions: the Lausanne Atmospheric Simulation Chamber (LASC)[END_REF][START_REF] Rodriguez-Navarro | Role of particulate matter from vehicle exhaust on porous building stones (limestone) sulfation[END_REF][START_REF] Moropoulou | Origin and growth of weathering crusts on ancient marbles in industrial atmosphere[END_REF][START_REF] Boke | Effect of airborne particle on SO 2 -calcite reaction[END_REF]Chabas and Lefevre 2000;[START_REF] Primerano | Possible alteration of monuments caused by particles emitted into the atmosphere carrying strong primary acidity[END_REF][START_REF] Esbert | Airborne particulate matter around the Cathedral of Burgos (Castilla y León, Spain)[END_REF][START_REF] Moropoulou | Correlation between aerosols, deposits and weathering crusts on ancient marbles[END_REF][START_REF] Cultrone | Sulfation of calcitic and dolomitic lime mortars in the presence of diesel particulate matter[END_REF]. In some cases metal species in carbon particles may act as nucleation particles for the formation of gypsum crusts, elsewhere the particulate matter catalyzes the sulfation of stone materials [START_REF] Ausset | Experimental study of limestone and sandstone sulphation in polluted realistic conditions: the Lausanne Atmospheric Simulation Chamber (LASC)[END_REF][START_REF] Rodriguez-Navarro | Role of particulate matter from vehicle exhaust on porous building stones (limestone) sulfation[END_REF][START_REF] Boke | Effect of airborne particle on SO 2 -calcite reaction[END_REF][START_REF] Primerano | Possible alteration of monuments caused by particles emitted into the atmosphere carrying strong primary acidity[END_REF]. Moropoulou et alii correlated the composition of the airborne particles detected on stone surfaces with the decay crusts formed on them observing that specific metals seemed to contribute to the decay mechanisms [START_REF] Moropoulou | Correlation between aerosols, deposits and weathering crusts on ancient marbles[END_REF]. Recently Cultrone et alii observed that diesel particulate enhances the sulfation of calcitic and dolomitic mortars in simulation tests, acting as catalyzer for SO 2 oxidation and hydrolysis to sulphuric acid [START_REF] Cultrone | Sulfation of calcitic and dolomitic lime mortars in the presence of diesel particulate matter[END_REF].

In this paper a new approach towards monuments was tried. Since aerosol tends to deposit on monuments stone surfaces, and since they are seldom cleaned, they may present an integrated situation of moveable PGE species mechanisms of transport, dissolution and deposition. We focused on platinum and rhodium for some reasons. Pt is well known to be the necessary component in all types of catalytic converters because of its efficiency in transforming hydrocarbons and carbon monoxide to water and carbon dioxide, while Rh is more efficient in reducing nitrogen oxide to nitrogen. Therefore Pt can be considered as the best probe to monitor environmental pollution of all types of catalytic converters, whereas Rh traces gasoline engine 3-way converters [START_REF] Fornalczyk | Removal of platinum group metals from the used auto catalytic converter[END_REF]. Adsorptive Cathodic Stripping Voltammetry (AdCSV) was selected as the analytical method considering the very low detection limits, the low cost of the analysis (with the opportunity of portable devices) and the possibility to quantify Pt and Rh in a single analytical determination, a very important issue when dealing with Cultural Heritage as samples are often available in very small quantities [START_REF] Helmers | Platinum and rhodium in a polluted environment: studying the emissions of automobile catalysts with emphasis on the application of CSV rhodium analysis[END_REF][START_REF] Barefoot | Distribution and Speciation of Platinum Group Elements in Environmental Matrices[END_REF][START_REF] Bencs | Methods for the determination of platinum group elements originating from the abrasion of automotive catalytic converters[END_REF][START_REF] Pozzi | Ultra trace determination of Pt and Rh in wastewater and gullypot sediments from a low polluted area[END_REF][START_REF] Locatelli | Voltammetric analysis of trace levels of platinum group metals principles and applications[END_REF].

Furthermore voltammetry may overcome the spectroscopic interferences due to the high Ca and Mg content present in stone materials such as marbles and mortars (Bencs et al. 1998).

Aims of the work were the assessment of the deposition of Pt and Rh on artificial stone surfaces, in particular cement mortars, and the optimisation of an analytical procedure which allowed to determine of the two metals considering the peculiar matrix.

The new approach was tested on the Camerlata Fountain in Como (Italy), built in 1934-5. The monument (Figure 1) was designed by Carlo Cattaneo and Mario Radice, architects joining the Rationalist Movement at the beginning of the 19th century in Italy. It is a nine-meter-high abstract sculpture, remarkable for the harmonic balance of four white circular shelves having a circumference of six meters and four spheres. A metallic framework with a pillar going through the axle of the spheres and rings supports the elements and the structure is covered with cement mortars. The choice of the monument was driven by its peculiarities. Firstly the building is located as a traffic island in a square crossed by four different roads and well-travelled. Secondly, as the last conservation work dated back to early Nineties, it is reasonable to find the particulate deposited over a number of years on the sheltered surfaces. Thirdly, the particular shape of the building, rich of horizontal surfaces, enhanced the deposition of particulate.

Materials and method

Sampling site: the Camerlata fountain

As usually when dealing with art objects, the sampling is a critical step. In this case study, the samples should in fact be representative of the deposited particulate, whose quality and quantity mainly depend on the morphology, exposure of the surfaces and, last but not least, on their conservation state. Eventually, as the building is a safeguarded artwork, the number of samples must be as lower as possible. The sampling was focused on a few areas (Figure 2), or completely exposed to the atmosphere, i.e. on the horizontal surfaces of the rings, or sheltered by the spheres or the rings. In some cases they were characterised by an evident blackening, in others by the presence of whitish, greyish and ochre concretions (Table 1). The surface samples were scraped by the mortar by means of a small chisel. Some small fragments of mortars were sampled near sample FC2 and cleaned from the surface layers by a small chisel in order to determine the limit of detection of the methodology and analyse the composition of the sound mortar, i.e. the nature of the binder and aggregate phases. The sampling campaign was carried out during summertime.

The sample treatment Dissolution

The acid dissolution step of solid samples was carried out as follows. The samples (around 100 mg) were powdered in an agate mortar and dried for 24h in an oven at 80 °C. A weighed amount of dried powder was transferred into a PTFE vessel with 1.5 mL of 37% HCl, 0.5 mL of HNO 3 (suprapur grade) and sonicated for few minutes in order to remove CO 2 and avoid over pressure during the acid attack. The high-pressure digestion was carried out in sealed Teflon vessels by a Milestone MLS-1200 microwave system equipped with a temperature control unit. The experimental parameters are shown in Table 2.

After mineralisation samples were heated on hot plate to dryness and residues were treated with 2 mL of HCl 2% in order to re-solubilise platinum and rhodium.

The solutions were then filtered on Millipore nitrocellulose filters 0,45 μm and the residue carefully dried and weighed.

The solutions were then quantitatively transferred into 100 mL volumetric flasks and aliquots irradiated for the UVdigestion. This dilution allows to reach a pH value compatible with the voltammetric determination.

Step

UV-digestion

Voltammetric metals determination requires samples free from organic matter [START_REF] Locatelli | Voltammetric analysis of trace levels of platinum group metals principles and applications[END_REF]. Samples were irradiated in a Metrohm 705 UV Digestor employing a 500 W high pressure mercury vapour lamp, for three hours.

Pt and Rh analyses 10 mL of the sample were transferred in the voltammetric cell, 162 μL of 37% HCl and 10 μL of formaldehyde were added and finally rhodium was analysed. Afterwards, 15 μL of hydrazine sulphate were added to enable the platinum determination. The quantification of both analytes in the same solution was performed through the standard addition method.

Analytical techniques

The surface samples and the sound mortars were finely ground and analysed with a diffractometer Bruker AXS D8 The powders were lately analysed as KBr pellets by a FTIR spectrophotometer Nicolet Nexus, in the transmission mode (400 to 4000 cm -1 , 4 cm -1 resolution). The samples were carefully collected under a stereomicroscope by means of a needle-sampler. The spectra were compared with standard references [START_REF] Wilson | Clay mineralogy: Spectroscopic and chemical determinative methods Chapman&Hall Edts[END_REF].

Pt and Rh AdCSV determinations were carried on the dissolved and opportunely diluted surface samples by Metrohm 746 VA Analyzer, equipped with an HMDE as working electrode, a glassy carbon auxiliary electrode and Ag/AgCl reference electrode. The solutions were analysed in a Teflon cell in order to minimise adsorption phenomena on the walls. The instrumental settings are described in Table 3.

A standard EDTA 0.01M solution was employed to perform the complexometric titration for the determination of the total amount of calcium and magnesium in the dissolved samples, with the aim to determine the binder/aggregate ratio (B/A ratio).

Results and discussion

Chemical and mineralogical characterisation of the samples

FTIR and XRD analyses were carried out both on the sound mortar fragments and on the powders scraped off the monument surfaces. The fragments were finely ground and the powders were analysed. All the samples contained both carbonatic and silicatic fractions as expected in cement mortars. In particular, FTIR spectra presented the peculiar absorbance peaks of calcite, around 1420, 875 and 712 cm -1 . The signal around 1028 cm -1 and the doublet at 798-780 cm -1 indicated respectively the presence of silicates and quartz ascribable to the aggregate phase. XRD spectra identified the silicatic content as illite.

The mortars binder/aggregate ratio (B/A ratio) was also determined. Several sound fragments were weighted, dissolved as described in paragraph 3.2, and the insoluble fraction carefully dried and weighted. The B/A ratio was calculated as ratio between the binder and the aggregate weights, ascribing the carbonatic content only to the binder fraction. The B/A value was also determined by complexometric titration of Ca and Mg with standard solution EDTA 0.01M. An average value of 1:2 was obtained for both approaches.

Coming to the surface powders, FTIR and XRD analyses confirmed everywhere the patterns observed for the sound mortar, i.e. calcite, small quartz amounts and silicates, identified by XRD spectra as illite. Ettringite was also identified by XRD analyses, probably related to the sulphation of cementitious phases. Moreover, traces of organic compounds were indicated by FTIR absorbance around 2957, 2875 and 1728 cm -1 . The peaks were consistent with the presence of residues of acrylic resins used during the conservation work occurred in the Nineties. In a few cases gypsum (samples FC13 and FC16) was identified in FTIR spectra (absorbance peaks at 3545, 3408, 1620, 1140 and 1116 cm -1 ) as minor component, probably ascribed to the well known sulphation of the cement mortar carbonatic fraction .

Instrumental paramenter Rhodium Platinum

Drop Size (a.u.) 5 5

Deposition Potential (V) -0.7 -0.7

Deposition Time (sec.) 60 60

Start Potential (V) -1 -0.7

End Potential (V) -1.2 -1

Voltage Step (mV) 2 2

Dissolution and sample pre-treatments

According to literature [START_REF] Pozzi | Ultra trace determination of Pt and Rh in wastewater and gullypot sediments from a low polluted area[END_REF][START_REF] Rauch | Heterogeneity of platinum group metals in airborne particles[END_REF] an aqua regia acid attack micro-wave assisted was chosen to dissolve the sample. The nitric acid excess was eliminated by heating the samples to dryness since it could interfere with the voltammetric analysis. During the dissolution step, the sample dissolution percentage was very various, from a minimum of 47% up to 100%, depending on the fraction of mortar scraped with particulate deposits. On the basis of previous works [START_REF] Pozzi | Ultra trace determination of Pt and Rh in wastewater and gullypot sediments from a low polluted area[END_REF][START_REF] Rampazzi | A new microwave-assisted approach to the chemical analysis of mortars[END_REF]) the MW parameters were set up to assure the dissolution of both PGE and "acid soluble" mortars fraction.

Organic compounds present in samples can affect the analytical determinations either directly by interactions with the electrode or indirectly by complex formation with metal ions. In order to overcome interferences effects, a UVdigestion step to be applied prior to the voltammetric analysis was optimized. UV irradiation of the dissolved compounds enhances the decomposition of the organic matter by the formation of several highly reactive oxidative species. The advantage of this approach is the minimal sample contamination risk as no reagents are added to the sample solution. Small amounts of hydrogen peroxide could accelerate in any case the degradation of organic substances.

For this reason, the irradiation time influence and the presence of hydrogen peroxide were studied using one of the samples collected on the Fountain (sample FC7). The sample was dissolved and properly diluted. 10 mL aliquots of the solution were then irradiated with and without 150L of hydrogen peroxide keeping the temperature between 80°C and 90°C and employing a 500 W high pressure mercury vapour lamp. As a result, after three hours irradiation no meaningful differences were observed between the concentration values of both metals. Furthermore, the hydrogen peroxide presence was considered non influential. Finally the optimized digestion procedure was carried out for three hours irradiation time without any hydrogen peroxide addition prior to any voltammetric determination.

Blank analysis and Limit of Detection

It is worthwhile to remind that when dealing with Cultural Heritage only limited sample amounts are generally available and this affects the limit of detection (LOD) of the whole analytical procedure. According to IUPAC formula (International Union of Pure and Applied Chemistry) LOD was calculated starting from the blank standard deviation, i.e. a sound mortar sample.

Almost 100 mg of sample were finely powdered and dried, dissolved and diluted according to the procedure previously described. No voltammetric signals were registered for both metals, confirming that there was no contamination during the whole procedure and that the mortar coming from Camerlata Fountain did not contain any appreciable amounts of platinum and rhodium. Platinum and rhodium standard additions were then performed. Eventually, LODs on liquid were calculated according to the IUPAC formula on the standard deviation of a sample with Pt and Rh concentrations near to the LOD. The values of LOD determined for the solutions translated to concentrations of 0.44 g/kg for platinum and 0.48 g/kg for rhodium in the solid samples.

The very low values allowed the determination of platinum and rhodium at trace levels in most of the samples.

Determination of Pt and Rh in the samples

First of all the effects of the high amounts of calcium and magnesium coming from the plaster surface on the voltammetric determination were evaluated. Known amounts of platinum and rhodium (final concentration 2 ng/L, a value comparable with those found in environmental samples) were added to a synthetic solution resembling the maximum concentration that Ca 2+ and Mg 2+ can assume in mortar samples (final concentration 0,053 g/L for Ca 2+ and 0,014 g/L for Mg 2+ , considering 100 mg of a mortar sample with 1:2 B/A and 40% w/w of calcium carbonate and 15%

of magnesium carbonate, diluted to 100 ml). Pt and Rh were then quantified in this solution. Recoveries obtained for three replicates were above 90% for both analytes. Thus, it was possible to affirm the absence of matrix effect on the quantitative determination of Pt and Rh.

As usual in the Cultural Heritage field there is a lack in specific Standard Reference Materials. Thus, the whole analytical procedure, starting from the dissolution step to the quantitative platinum and rhodium determination, was tested on a sample coming from the Camerlata Fountain (sample FC7 as it was available in a sufficient quantity). It was divided into three aliquots after an adequate grinding and homogenization. The three aliquots were then dissolved and analysed. The standard deviation was finally calculated considering the three obtained rhodium and platinum values.

Authors considered these values as an estimate of the whole procedure standard deviations: they resulted less than 5% for both analytes.

Platinum and rhodium quantitative determination together with the standard deviation provided by the instrument (thus obtained considering the five instrumental repetitions for a given sample), are summarised in Table 1. It is worthwhile to note that the average standard deviation (mean of the standard deviations of all the analysed samples) for both platinum and rhodium is about 5% (only sample FC9 showed an high variability in the instrumental repetitions resulting in a standard deviation of about 30%).

As mentioned in paragraph 3.2, the dissolution percentage of the investigated samples was very various. This could be ascribed to the fraction of plaster scraped together with the particulate deposit. While the binder fraction is acid soluble, the aggregate fraction remained as precipitate (authors considered that other insoluble fractions, e.g. silicates and carbon particles normally present in the particulate matter, gypsum and organic matter were negligible with respect to the insoluble residue coming from aggregate fraction of the plaster). As the B/A ratio of the plaster is known, it is simple to derive the weight of the mortar fraction dissolved by the acid attack (the binder fraction) from the weight of the residue after the acid attack. These two quantities, subtracted to the total amount of weighed samples, allowed to determine the quantity of the analysed particulate deposit. Platinum and rhodium concentrations were then referred to the amount of the particulate deposit fraction of each sample.

The fountain was sampled in areas very different one from each other, i.e. different exposure and height, in order to represent in the most complete way the deposition variability. Thus, the data confirmed the expected results.

As first result, no correlation between the degradation type and the concentration values of both metals was found.

Platinum was found in almost all the investigated samples while rhodium concentration was below LOD in several cases. No correlation between the two metals was found: e.g. the sample that showed the highest value for platinum had a value below the LOD for rhodium. Literature confirms that the Pt and Rh ratios in solid matrices are not constant and may vary in a large range.

A very high variability in the found values for both analytes arose from the study. Nevertheless it is worthwhile to note that the platinum highest concentration values were found in samples collected at the bottom part of the monument.

Two samples showed peculiar values: sample FC3 presented a very high concentration of platinum (500 g/kg) while sample FC13 showed the highest concentration of rhodium (274.4 g/kg). It is worthwhile to note that this two samples were placed in areas particularly sheltered by rain runoff, thus high amounts of particulate could be accumulated.

This high variability could be ascribed to the presence of two mechanisms that may be involved in such distribution pattern, i.e. the deposition of particulate matter on to cement mortar surface that should be constant over the monument, and the particulate matter and/or pollutants removal via precipitations or more in general via meteorological events.

Moreover, Rh and Pt removal and/or remobilisation within the monument surface could also be caused by their solubilisation in the percolating water. It is in fact suggested by various authors that a non negligible Pt and Rh fraction in the particle matter near roads and motorways is water soluble.

Conclusions

In this work a new approach towards the relation between monuments and pollution was undertaken. On one hand, the opportunity of determining platinum and rhodium on mortars was verified, and their distribution over the monument was evaluated. Secondly, the whole analytical protocol was optimized for the determination of platinum and rhodium deposited on monument surfaces. The choice of the cases study was of primary concern, by the point of view of sampling opportunity and by its urban position. Of significant interest from these results is that the whole procedure proved very effective for the determination of platinum, particularly in terms of detection limits and negligible matrix interferences, notwithstanding an expected variability of concentrations. The amounts of rhodium were even more variable, being absent in some of the samples and more concentrate than platinum elsewhere. Whether it was a matter of sampling, being a matter of analytical procedure requiring greater amounts of sample, or mirror of the inequality of the actual distribution of rhodium on surfaces, needs to be enlightened in a new dedicated analytical campaign.

Besides these criticisms, the work gained a twofold purpose.

By the environmental point of view, the stone surfaces of the Camerlata Fountain actually suffered the contamination of modern pollutants such as platinum and rhodium, particularly in those areas sheltered by rain runoff and able to accumulate atmospheric deposit. The approach here proposed suggests the ability of artificial stone surfaces to give an integrated evaluation of modern pollutants such as Pt and Rh in the particulate matter, in view to the comparison of air quality among areas characterised by different kinds of pollution.

The occurrence of platinum and rhodium on monuments surfaces should be of significant interest when planning Cultural Heritage conservation as well. Thus, besides the decay phenomena traditionally considered and investigated on outdoor monuments (e.g. gypsum crusts, efflorescence, incrustations, films, and blackening), the presence of catalytic metals specifically coming from the vehicular pollution has to be seriously considered. A better knowledge of the metals role in degradation mechanisms could in fact impact in a positive way the artworks conservation and contribute to the answering of questions about the pedestrianisation of urban historic areas.

Thus, Voltammetry should be the key analytical technique for such determinations in conservation science, considering the advantages previously discussed. This issue is only the preliminary step, necessary to stress the critical points of an articulate monitoring, i.e. the sampling and the acid attack steps, the analytical problems, interferences of the stone matrix. Taking a step beyond, the Authors have started the investigations of other monuments by the optimised analytical procedure, focusing also on the comprehension of the role of platinum and rhodium in the decay processes of natural and artificial stone surfaces.

Fig. 1

 1 Fig. 1 Picture of the Camerlata Fountain

Fig. 2

 2 Fig. 2 Sketch of the Camerlata Fountain showing the sampling points

Table 1

 1 Samples description and Pt and Rh concentrations.

	Sample Height of	Description	Exposure	Rh	Rh SD	Pt	Pt SD
		sampling (m)			(μg/Kg)	(μg/Kg)	(μg/Kg)	(μg/Kg)
	FC 1	9	Black deposit	Not sheltered	89	8.3	<LOD -
	FC 2	9	Black deposit	Not sheltered	7.4	0.47	11	1.3
	FC 3	6.8	Black deposit	Sheltered	<LOD -	500	45
	FC 4	6.8	Whitish concretion Not sheltered	<LOD -	11.2	0.55
	FC 5	6.8	Greyish concretion Not sheltered	<LOD -	1.40	0.054
	FC 6	8.4	Whitish concretion Sheltered	<LOD -	3.1	0.23
	FC 7	6.8	Black deposit	Not sheltered	38.1	0.94	21.9	0.47
	FC 8	6.2	Black deposit	Not sheltered	<LOD -	2.50	0.061
	FC 9	4.6	Greyish concretion Not sheltered	60	18	33.8	0.37
	FC10 6.2	Whitish concretion Not sheltered	0.55	0.036 1.20	0.013
	FC11 1.8	Black deposit	Sheltered	<LOD -	39	3.7
	FC12 2.4	Ochre concretion	Not sheltered	<LOD -	4.60	0.016
	FC13 4	Ochre concretion	Sheltered	274.4 0.34	33.6	0.92
	FC14 2.4	Ochre concretion	Not sheltered	41.92 2.5	18	1.8
	FC15 1.8	Whitish concretion Sheltered	6.940 0.0061 4.1	0.075
	FC16 0.20	Black deposit	Sheltered	<LOD -	140	15
	[SD= Standard Deviation; LOD=Limit of Detection]

Table 2 -

 2 Microwave program for samples digestion

		Temperature (°C)	Power (W )	Time (min)
	Pre-heating	75	250	3
	Thermalisation	-	0	1
	Mineralisation	150	500	10
	Cooling down	-	0	10

Table 3 -

 3 Instrumental settings employed for Pt and Rh determination

Acknowledgments

The Authors would like to thank the Municipality of Como for the permission of sampling the monument, Dr.

Francesca Roncoroni for the sketch of the Camerlata Fountain, and Dr. Marcello Marelli for the picture of the monument.