Philippe Pérez

San Roman

Jenny Benois-Pineau

Jean-Philippe Domenger
email: domenger@labri.fr

Aymar De Rugy
email: aymar.derugy@u-bordeaux.fr

Florent Paclet
email: florent.paclet@u-bordeaux.fr

Daniel Cataert
email: daniel.cattaert@u-bordeaux.fr

Saliency Driven Object recognition in egocentric videos with deep CNN Electronic copy submitted to the Journal of Computer Vision and Image Understanding

Keywords: Psycho-visual attention, Saliency, Egocentric video, Object recognition, Deep Convolutional Neural Networks

The problem of object recognition in natural scenes has been recently successfully addressed with Deep Convolutional Neuronal Networks giving a significant break-through in recognition scores. The computational efficiency of Deep CNNs as a function of their depth, allows for their use in real-time applications.

One of the key issues here is to reduce the number of windows selected from images to be submitted to a Deep CNN. This is usually solved by preliminary segmentation and selection of specific windows, having outstanding "objectiveness" or other value of indicators of possible location of objects. In this paper we propose a Deep CNN approach and the general framework for recognition of objects in a real-time scenario and in an egocentric perspective. Here the window of interest is built on the basis of visual attention map computed over gaze fixations measured by a glass-worn eye-tracker. The application of this setup is an interactive user-friendly environment for upper-limb amputees. Vision has to help the subject to control his worn neuro-prosthesis in case of a small amount of remaining muscles when the EMG control becomes unefficient.

Introduction and motivation

The problem of natural object recognition in images has been in the center of computer vision community since quite a lot of time. Previous PascalVOC challenge [START_REF] Everingham | The PASCAL Visual Object Classes Challenge[END_REF] and ongoning ImageNet Large Scale Visual Recognition Challenge (ILSVRC) [START_REF] Russakovsky | ImageNet Large Scale Visual Recognition Challenge[END_REF] have united important task forces for finding the solution of this problem in natural visual scenes. Since recently, developed approaches find their place in a quite realistic application for helath care and patient's monitoring such as in [START_REF] Crispim-Junior | Semantic event fusion of different visual modality concepts for activity recognition[END_REF]. The pioneering works on real-world manipulated object recognition in egocentric perspective [START_REF] Pirsiavash | Detecting activities of daily living in first-person camera views[END_REF] for evaluation of cognitive impairment of Alzheimer patients [START_REF] Chaabouni | Deep learning for saliency prediction in natural video[END_REF] showed that even if the state-of-the art in egocentric object recognition does not allow for getting scores approaching 100% for all object categories and requires a heavy annotation process, the information on the object of interest the human interacts with is essential for assistance and evaluation of patients and impaired subjects. In this paper we develop an object recognition approach for assistance of upper-limb amputees wearing neuro-prostheses.

Classic myoelectric control of neuro-prostheses uses the activity of the remaining muscles to control the multiple degrees of freedom of the prosthesis. This strategy, however, faces a fundamental problem related to fact that the higher the amputation, the higher the number of degrees of freedom of the prosthesis to control with less control signals from the fewer remaining muscles.

In addition, all commercially available myoelectric prosthesis are only controlled with two muscle groups, one flexor and one extensor, and therefore involve sequential control of individual joint with unnatural control schemes to switch between joints. The tedious learning for relatively mediocre results associated with these control schemes has motivated several laboratories at developing control schemes that integrate a higher number of muscles, either through pattern recognition of movement classes from muscle recordings [START_REF] Parker | Myoelectric signal processing for control of powered limb prostheses[END_REF][START_REF] Smith | Determining the optimal window length for pattern recognition-based myoelectric control: balancing the competing effects of classification error and controller delay[END_REF], or through regressions that use muscle activities for simultaneous proportional control of the multiple degrees of freedom of the prosthesis [START_REF] Farina | The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges[END_REF][START_REF] Hahne | Linear and nonlinear regression techniques for simultaneous and proportional myoelectric control[END_REF][START_REF] Jiang | Extracting simultaneous and proportional neural control information for multiple-DOF prostheses from the surface electromyographic signal[END_REF]. Despite their merits, however, these attempts do not resolve the key limitation in terms of number of remaining muscles, although some control signals could be recovered using highly invasive surgical techniques such as nerve recording [START_REF] Wodlinger | Localization and recovery of peripheral neural sources with beamforming algorithms[END_REF] or targeted muscle re innervation [START_REF] Kuiken | Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms[END_REF].

Promising and much less invasive alternatives propose to use other control signals such as from computer vision [START_REF] Markovic | Stereovision and augmented reality for closed-loop control of grasping in hand prostheses[END_REF][START_REF] Markovic | Sensor fusion and computer vision for context-aware control of a multi degree-of-freedom prosthesis[END_REF], gaze information [START_REF] Corbett | Decoding with limited neural data: a mixture of time-warped trajectory models for directional reaches[END_REF][START_REF] Corbett | Multimodal decoding and congruent sensory information enhance reaching performance in subjects with cervical spinal cord injury[END_REF], and/or residual biological motion [START_REF] Kaliki | Evaluation of a noninvasive command scheme for upper-limb prostheses in a virtual reality reach and grasp task[END_REF]. In [START_REF] Markovic | Stereovision and augmented reality for closed-loop control of grasping in hand prostheses[END_REF], stereo-vision from camera integrated in augmented reality glasses was used to automatically select grasp type and size from a visible object, and [START_REF] Markovic | Sensor fusion and computer vision for context-aware control of a multi degree-of-freedom prosthesis[END_REF] added inertial sensing to automatically align wrist orientation to that of the object to grasp. In both instances, however, only one object was present in front of the subject, thereby avoiding the critical problem of recognizing the object of interest from the multiple objects typically present in a natural environment. Furthermore, although camera on glasses provided egocentric videos, gaze information was not available and therefore not used to assist this process. In [START_REF] Corbett | Decoding with limited neural data: a mixture of time-warped trajectory models for directional reaches[END_REF][START_REF] Corbett | Multimodal decoding and congruent sensory information enhance reaching performance in subjects with cervical spinal cord injury[END_REF], gaze information was used, but to supplement muscle recordings for the control of reaching actions rather than to recognize the object of interest. Furthermore, the setup was such that eye tracking was working on a fixed head, and the reaching was on a two dimensional screen.

Our goal here is to combine gaze information with recent progresses in deep learning for computer vision, in order to improve real time object recognition from egocentric videos, the long term goal being to incorporate this information into prosthetics control. The rest of the paper is organized as follows. In Section 2 we analyse the related works in natural object recognition and localization with Deep Convolutional Neural Networks and summarize our contributions, in Section 3 we present our general approach for saliency driven object recognition with Deep CNN, which uses gaze information. In Section 4 we focus on Network design and tunning. Section 5 presents experiments and results. In Section 5 discussion, conclusion and further perspectives of this work are presented.

Related Works

The problem we adress consists in both: i) object recognition and ii) object localization. In [START_REF] Girshick | Region-based convolutional networks for accurate object detection and segmentation[END_REF] a good analysis of recent approaches for object localization has been proposed, such as "regression approaches" as in [START_REF] Sermanet | Overfeat: Integrated recognition, localization and detection using convolutional networks[END_REF] [START_REF] Agrawal | Analyzing the performance of multilayer neural networks for object recognitio in[END_REF], and "sliding window approaches" as in [START_REF] Sermanet | Pedestrian detection with unsupervised multi-stage feature learning[END_REF] when the CNN processes multiple overlapping windows. The authors of [START_REF] Girshick | Region-based convolutional networks for accurate object detection and segmentation[END_REF] propose a so called Region-based convolutional network (R-CNN). Inspired by "selective search" approach [START_REF] Uijlings | Selective search for object recognition[END_REF] it evaluates multiple (2K) "object proposals" and finally train an SVM for classification. Such "proposals" are numerous and the multi-task classification with CNNs requires heavy computations. Hence in [START_REF] Girshick | Region-based convolutional networks for accurate object detection and segmentation[END_REF] the authors report that R-CNN can take from 10s to 45s for object classification. Several attempts has been made to accelerate the generalization process. In [START_REF] Oquab | Is object localization for free? -weakly-supervised learning with convolutional neural networks[END_REF] a weakly supervised training scheme is designed. They train the network on the whole labeled image. The deepest layers of the CNN supply features. Then fully connected layers -adaptation layers are proposed considered as convolution layers. Max pooling supplies the scores for different positions of objects. In order to reduce the number of object proposals, the spatial grouping of windows was proposed in SPPnet [START_REF] He | Spatial pyramid pooling in deep convolutional networks for visual recognition[END_REF].

Here the spatial pyramid pooling layer is added above the convolutional layers, transforming the output into fixed-size vectors. Thus built the network not only copes with different sizes of input images, but is from 24 to 54 times faster than AlexNet [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF] with only 530 ms per image. Further acceleration at testing step is proposed in fast R-CNN [START_REF] Girshick | Fast R-CNN[END_REF]. Here the idea of training on the whole image is re-used. The network first processes the whole image at the deep layers of the network and then, at the uppaer layers, object proposals are processed. Indeed the whole image is used through several convolution and max pooling layers instead of object proposals. The latter are used at the so-called region-of-interest (ROI) pooling layer. Here a fixed-lentgh feature vector is ex-tracted for each object proposal from the obtained full-image feature map. The feature vectors are then submitted into a sequence of fully connected layers, finally two output layers produce the softmax probability for object classes and the background and the estimate of window corner positions with regression.

The speed-up of computation at training step is achieved due to the late selection of features for object proposals, while at the test step, they use truncated SVD decomposition on fully connected layers to accelerate computations for them. The computational time of 600ms is reported per image at the test step.

All these methods developed for object recognition and localization with Deep CNN aim at increasing mAP and reducing computational time. They work in an "unconstrained" setting, which means that there is no initial assumption on object location. Thus the localization process has to be accelerated. One of the trends strongly present in the research in object recognition which also inspired "selective search" [START_REF] Uijlings | Selective search for object recognition[END_REF] approach, consists in using visual saliency of regions, which serves to select object proposal candidates. Out of the methodological framework of Deep CNN, such methods were proposed for the popular Bagof-Visual-Words (BoVW) model [START_REF] Philbin | Object retrieval with large vocabularies and fast spatial matching[END_REF]. The latter served as object signature but was built on the whole image from qunatized features weighted by underlying saliency value [START_REF] De Carvalho Soares | Spatial locality weighting of features using saliency map with a bovw approach[END_REF], [START_REF] González-Díaz | Perceptual modeling in the problem of active object recognition in visual scenes[END_REF]. Despite we have developed a complete saliency-based methodology for all steps in feature engineering approach, such as feature selection, encoding, pooling, and despite its good performances surpassing the most popular state-of-the-art model DPM [START_REF] Felzenszwalb | Object detection with discriminatively trained part-based models[END_REF], the BoVW approach showed its limits even with ideal saliency maps such as manually annotated bounding boxes [START_REF] Chamaret | Spatio-temporal combination of saliency maps and eye-tracking assessment of different strategies[END_REF].

As the predicted (objective) or ideal (subjective), computed with gaze-fixations from eye-tracker recording, maps confine image analysis process and eliminate clutter, it is natural to try to incorporate them into the winner model today, such as Deep CNN.

The contributions of our work are the following: i) we introduce ideal saliency maps, recorded with head-mounted eye-tracker into "object proposal" selection for our object recognition need in an interactive environment. Taking into account that object recognition has to be conducted, on acting person, we use es-tablished facts from cognitive sciences and bio-physicscs to select sequences from video and filter-out distractors ii) we re-use "ImageNet" architecture, propose addequate data sampling and augmentation relevant to our egocentric setup and show that without any supplementary efforts, the base-line ImageNet allows to get rather good scores for selected object proposals and do it in a biological real-time -shorter than visual fixation time.

In the following section we present our general framework for saliency driven object recognition with Deep CNN.

General approach for saliency driven object recognition with Deep

CNN

Our method is developed for humans grasping an object. The task consists in simultaneous recognition of the object the subject wishes to grasp. A human subject is instrumented with a glass-worn eye-tracker with a scene camera (Tobii Pro glasses 2). The block diagram of the method is presented in figure 1.

The data preparation block serves to filter out missing eye-tracker recordings, see Section 3.2. On the basis of recorded gaze fixations we compute visual saliency of pixels in the video recorded by the scene camera, see Section 3.3.

In real world scenario our system fulfills automatic patch selection, i.e. "object proposal" (see the lowest branch of figure 1) using computed saliency. Then CNN classifies the extracted patch into a set of known categories. Fusion of classification scores along the time allows for filtering natural noise such as eyeblinking and saccadic motion towards distractors. The middle branch of the diagram in figure 1 presents training process. To train the known category of objects we use a semi-automatic annotation method, also guided by computed saliency, see Section 5.2. Specific data augmentation approach is designed with regards to the real life scenario, see section 4.2.2. CNN training is realized on the augmented object dataset. In the following part of this section we detail these steps.

Physiology of visual attention

In order to provide rationale for our data preparation methodology we briefly expose here our physiological hypotheses and the known neuro-physiological models of human vision relevant to our problem. Our actor, an upper-limb amputee has less freedom in the control of his body than a healthy person.

When he wants to grasp an object in his environment he first looks at it (which is not always the case for a healthy subject). This is our main assumption.

The observation of a scene comprises: (1) The discovery of the scene where the eye scouts sparsely the scene.

(2) The fixation on the object of interest.

(3) Micro-saccades, when the eye slightly oscillates about the target object.

(4) grasping movement is triggered. (5) Also some distractors, such as audio; light; motion; and occlusions in the scene can lead the eye to deviate to another object.

We have conducted psycho-visual experiments on healthy volunteers aged from 20 to 23 and we observed that: (1) The scene discovery takes from 240 to 300 ms. [START_REF] Russakovsky | ImageNet Large Scale Visual Recognition Challenge[END_REF] The fixation is about 250 ms.

(3) Micro-saccades can occur with duration about 6 to 300 ms according to different sources reviewed in [START_REF] Martinez-Conde | Microsaccades: a neurophysiological analysis[END_REF],

note that the frequency of our eye-tracker does not allow precise measurments of micro-saccades duration, see section 5.1.1 for experimental set-up. (4) The grasping movement then takes between 400 to 900ms. (5) Finally the times of distractor fixations are between 100 and 500 ms. These data are in accordance with the results in [START_REF] Devillez | How task difficulty influences eye movements when exploring natural scen in: EUSIPCO[END_REF], [START_REF] Villani | Visual exploration patterns of human figures in action: an eye tracker study with art paintings[END_REF].

During the scene discovery a subject explores the scene searching for the target object, hence the object-of-interest is not fixated. that filters out frames with distractors.

Data Preparation

The data preparation block receives the data from the Tobii Glasses 2 streams, simulated in the present work as two pre-recorded files: one for the video and one for the eye-tracking data. In the eye-tracking data some recordings of gaze fixations are missing due to eye-blinking of the subject. Also the video and the gaze tracking have different sampling rates, and so it is rare that an eye-tracking record is synchronized with a video frame.

To cope with all these problems, we apply interpolation (using a spline). This smoothes the gaze fixation data and allows us to synchronize the two streams.

Subjective saliency Computation

From the eye-tacker data we get the recorded gaze-fixation point f for the image I, with coordinates x f , y f and d f . Here d f is the coordinate along the axis of gaze direction and x, y are the coordinates of the fixation point in the image plane of video recorded with the scene camera of glasses. Therefore, the so-called "subjective saliency map" or Wooding's map W (I, f ) [START_REF] Wooding | Fixation maps: Quantifying eye-movement traces[END_REF] can be computed. It is a normalized Gaussian function, centered on a fixation point,

σ(I, d) = A d • width(I)tan(180απ) 2tan(βπ/180) (1)
W (I, f, x, y) = A W ∞ × exp -(x -x f ) 2 -(y -y f ) 2 2 • σ(I, d f ) 2 + ǫ (2)
Where α = 2 • is the angle of projection of the fovea, β = 24 • is the camera opening angle on the width, A = 1600 mm is the maximum distance to an object according to our setting and ǫ is a small number ǫ = 0.01.

In our case of moving eye-tracker wearer, unlike the traditional Wooding's map, the spread σ of the Gaussian function is linearly adapted wrt the distance to the object. Thus our method produces a larger image patch when the object is closer (and so appear larger in the video), or reciprocally, smaller image patch when the object is farther. In figure 2 we show different forms of Wooding's saliency map on a video frame.

Network design and tuning

In our work we used the basic ImageNet architecture proposed in [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]. We do not need a specific optimization of computational cost as the focused selection of an "object proposal" accordingly to the terminology of Girshik [START_REF] Girshick | Fast R-CNN[END_REF], in each frame allows us to be compatible with real-time requirements for object recognition. In this section we remind the architecture and focus on the way we extract "object proposal" patches and the background candidates based on saliency, and then augment them to prevent over-fitting.

General architecture and parameters

Network layers. The ImageNet network [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]1 is mainly composed of 5 types of layers: 5 convolutions (Conv), 3 fully connected (FC), 7 Rectified Linear Units (ReLU), 3 Max pooling, and 2 Local Response Normalization. They are combined vertically to increase the network depth.

1. The convolution is used to extract features on its input by applying filter to it. On the first layer this filters respond to edges or color blobs, while on the last one they are able to abstract shapes and objects parts [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF].

2. The FC layers are used to progressively map activation maps to a single dimension feature vector where at the end each value is associated to a class of object. It is then normalized to a probability distribution using a Soft Max layer.

3. The max polling [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF] is used to spatially down-sample the activation of the previous layer by only propagating the maximum activation of a previous group of locally connected neurons.

Local Response

Normalization is used to normalize the response of neurons at the same spatial location. This is inspired by lateral inhibition in real neuron [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]. 5. ReLU were introduced in [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF] to increase the network optimization convergence.

Optimization. We use the soft max loss function (multinomial logistic loss) already implemented in Caffe [START_REF] Yangqing | Caffe: Convolutional architecture for fast feature embedding[END_REF], that for input image X i with known label l i is:

E(X i , l i ) = - 1 N N j=1 log( P j )δ( l j , l i ) (3) δ(l, l i ) = 1 if l = l i , 0 else (4)
Where P j is the probability with associated label l j , resulting from the forward pass in the network. The overall loss over a dataset D is:

L(W ) = 1 |D| |D| i=1 E(X i , l i ) + λr(W ) (5)
r(W ) is a regularization term with weight decay λ = 0.0005

We use the default implementation of stochastic gradient descent from Caffe and ImageNet [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF][START_REF] Yangqing | Caffe: Convolutional architecture for fast feature embedding[END_REF] with weight update rule:

Vt+1 = µVt -α∇L(Wt) (6)
Wt+1 = Wt + Vt+1 ( 7
)
Where α is the Learning rate and µ the Momentum (0.9). Our learning rate α is initialized to 0.001 and decreased by half every 30000 iterations. ∇L(Wt) is computed through back-propagation in the network.

Saliency-based data preparation

We will now present the preparation of the input data for the network training, which consists in selection of bounding boxes of objects using saliency maps, and sampling of background patches. We also propose a data augmentation strategy corresponding to our problem.

Patch extraction

Our basic CNN [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF] has a fixed input resolution, so no matter at which resolution the objects appear in the video, they are all resized to a fixed size (227x227 RGB in our case). For machine learning, especially classification, we have to extract object patch (examples) for all categories, including the background that is the rejection category.

It is important to extract a similar amount of examples for each of them to avoid imbalanced class problem.

In our scenario, the object of attention is identified by the thresholded saliency, and a label describing the category. We extract the corresponding blob by connected component analysis, it gives us the bounding box. The green bounding box in figure 3 is an example of an image patch corresponding to a "rectangular prism". We also have to extract background at the same time to ensure that we have the same amount of rejection examples. Remember that our experimental setup specifies that the objects are lined-up on the table. Since only one object is labeled, we exclude the bounding box of the object, but also the area where other objects could appear, to avoid a background/object mixture. This exclusion area is drawn in blue in the figure 3 below.

Patch Augmentation

Data augmentation is very efficient to prevent over-fitting [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]. The idea is to apply label-preserving transformation to the image patch, and give both the original one and the transformed ones to the network for training. This will artificially increase the training dataset size. Common transformations are horizontal mirroring, random cropping [START_REF] Yangqing | Caffe: Convolutional architecture for fast feature embedding[END_REF]. In our case objects can be placed upside-down or lay on any of their sides, this led us to rotate training image patch by an angle α ∈ {0 • , 90

• , 180 • , 270 • }.
We only considered multiples of 90 • to avoid discretization problems that can lead to The figure 4 shows an example for all object categories.

Note, that we do not need data augmentation at the test step as it is proposed in ImageNet [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]. They need it as they do not have certainty on the object location. This is why they generate multiple candidates for the "object proposal" and practice fusion of scores. In our case an "object proposal" is unique as it is totally defined by online recorded gaze fixation and derived saliency maps.

Temporal Fusion

We are classifying a sequence of object proposals in a video with "mean fusion" operator, which is equivalent to a simple sum as shown in the equation below 8: The scores for each class are summed over the candidate patches along a video, and the final category is the one that obtains the maximum score over these sums.

c(V ) = argmax c s(V, c) = argmax c { N i=1 s(pi, c)} (8)
Where V is a video, pi is the candidate patch of frame i.

We use this instead of retaining just the most frequent classification result, i.e. the "majority vote", because we believe the score of an incorrectly classified patch is often much smaller than the score of a correctly classified patch. And so by summing the scores over a patch, the correct class score wins.

Experiments and results

In order to test our object recognition framework in a real life but simplified scenario we produced a new dataset that we called Large Egocentric Gaze Objects (LEGO) which will be soon available online. Below we present the experimental setup and content; our data selection results, network optimization, and results.

LEGO Dataset

Experimental Setup

The recording of our dataset was conducted with four healthy volunteers aged from 20 to 23. In each recording session a subject was instructed to look for a specific object and to grasp it. The subjects were sitting in front of a white table, facing a white wall.

They wore the Tobii Pro Glasses 2. This eye-tracker records gaze fixations at 50 Hz and video frames at 25 Hz. At first, the subject's eyes were closed (in which case the gaze data are not available). Four objects out of eight different objects were randomly chosen and placed in line on the table. They were presented in different positions for each experiment (they could be placed upside down, flipped, and rotated). The name of the object to grasp was revealed at this moment and the video recording started at the same time. The subject could then open his eyes, search for the object, and once he found it he grasped it. After a few seconds the recording was stopped. Figure 5 shows a subject performing the experiment.

Videos

We recorded 151 videos with our experimental setup 5.1.1. Eight types of objects with simple shape and identifiable color were used, see examples in figure 5. The duration of videos was between 3,6 s up to 11,9 s, that is 6,5 ± 0,9 s on average.

These videos are short as they depict the initiation of hand motion and grasping of the object-of-interest. They are split between Train (60%), Validation (20%) and Test (20%) sets as shown in table 2.

This video dataset is rather simple in a sense that the object and the background are well separable; there are no occlusions. Some motion blur is observed when the subject moved his head when searching for the object-of-interest. The real challenge of this dataset comes from the semi-automatic ground truth annotation and distractors.

Indeed some frames are miss-annotated: the object in the video can have an incorrect label as the subject was distracted and did not fixate the right object. The localization can be inaccurate; the bounding box can be too big or too small due to distance measurement inaccuracy in saliency map computation.

Semi Automatic Annotation

Annotation is the process of describing the content of a set of videos, in each frame of each video, for all type of content (objects). In computer vision, this is done manually: a human annotator visualizes the videos, select objects in each frame depicting their bounding boxes (rectangle+label) and sometimes segments the object (binary mask+label). Datasets are now on the order of millions of images, and thousand of object types turning annotation into enormous and tedious work.

In our experiment we know the subject looked at the object, so we propose to use the saliency map to select the patch of the object (the saliency peak is located on the object). We threshold the saliency to create an approximate segmentation mask.

In practice, their is a delay between the beginning of the video and the moment the subject opens his eyes, and finds the object of interest, called visual exploration (see section 3.1. It is on the order of 300 ms. We have to ignore this part of the sequence or we would be considering patches of objects other than the object of attention.

To solve this problem we developed a simplified annotation tool presented in figure 6, that allows the human annotator (1) to select the moment when the scene exploration is completed and the subject is focused on the object-of-interest (buttons 7 and 8), (2) to threshold the saliency map (button 9), and (3) to choose the category of the object (button 10). We allow the threshold to be changed because in some sequences the saliency map is larger due to imprecision in the distance to the object-of-interest computed by the eye-tracker software. Changing the thresholds allows the annotator to control the amount of context inside the object patch.

When the subject is distracted by other objects or lightning changes in the background the human annotator is not always able to detect this moment. This leads to the noise in training data and selection of a "bad" object proposal in online test scenario. This results in an accuracy drop we observed in our experiments. We deal with it using score fusion (see section 4.3) the visualization method and resolution can be selected with buttons 5 and 6; the annotation parameters with buttons 7 to 11.

Patch Extraction

We extracted an image patch as described in section 4.2.1. We only took one background patch on frames showing an object in the training and validation dataset, so that the final number of background patch sums up almost to the sum of those in object categories. We do not sample background on the test set. Ultimately, we had as many patches per category as the ImageNet dataset [START_REF] Russakovsky | ImageNet Large Scale Visual Recognition Challenge[END_REF], but fewer categories were considered: 9 instead of 1000.

Network Optimization

We used the ImageNet architecture [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF], changing the number of outputs of the last FC layer to match our number of classes. The learning rate was set to 0, 001 and was decreased every 30000 iterations by half.

The training loss is rapidly decreasing as shown in figure 7:left The validation accuracy rapidly reaches ∼ 80% as shown in figure 7:right indicating a stable training of our network. This fast training is due to the simplicity of our objects and the lack of clutter in the scenes.

We train our network on a server equipped with 56 Intel Xeon cores and a NVIDIA Tesla k40m, it took two days to complete 47000 iterations.

Classification results

Figure 8 shows the average precision on all our categories as well as the gains of our temporal score fusion method. The deep CNN classifier was able to achieve the stateof-the-art performance with a mean average precision of 58, 4%. Then score fusion presented in section 4.3 was able to recover the correct category over a video sequence even if less than 40% of the patches extracted on the frames were correctly classified.

This was because the few correctly classified object proposals had strong score of their class. In the present work, we did fusion of scores on the whole video. For real time application in the assistive neuro-prosthesis visual system, the size of the fusion buffer would have to be optimized with respect to the latency of other components of the whole system. This leads to a score of 64, 6% mAp,yielding a 6, 2% gain.

We have conducted a performance test under Ubuntu 14.04 on an Intel I7-4790@3.6GHz CPU and a NVIDIA Quadro K4200 GPU. Computation of the saliency map from raw eye-tracking data was implemented in C++ with OpenCV and CUDA. This computation took 20 ms per frame without CUDA acceleration and 5 ms with it. We use the Caffe toolbox. It is not configured with CUDNN acceleration, so better performance can be expected with it. Classifying 2767 patches from 28 videos took 23.877 s which means 8.6 ms per patch, for a single video it means a computational time of 995 ms on average.

The total time for the computation of the saliency, and of classification of a patch, is of 28.6ms which is less than video frame rate (40ms) and much less than our requirement to be faster than a gaze-fixation time (250ms).

Discussion, conclusion and perspectives

In this work, we have proposed an approach for object recognition in egocentric videos guided by visual saliency to help grasping actions for neuro-prostheses. For annotation of visual data for traning of object detectors in such a setting we also proposed a semi-automatic annotation method, guided by visual saliency as well. The recorded egocentric dataset will soon be made available online. Object recognition is performed using a deep CNN [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF] that was able to achieve 58, 4% mAp, and using temporal fusion of scores, we obtained state-of-the art results of 64, 6% mAp despite the presence of annotation noise introduced by the distractors in training set and in the real-world "online" testing. The total time of our recognition system is about 28 ms per frame including visual saliency map computation and generalization with the Deep CNN. This time matches our requirement to be faster than visual fixation time.

This method has yet to be tuned for live system integration and some parameters such as the buffer size for temporal filtering have to be adjusted wrt the latency of other components of the neuro-prosthesis system. We are also eager to try other deeper and wider CNN architectures. The proposed approach gives rize to a wide set of exploration routes. Indeed, in such settings, the presence of noise in both annotation and test data sets are unavoidable due to human errors and physiology of human attention. We are thus interested in developing noise-robust optimization methods for Deep Neural Networks.

Figure 1 :

 1 Figure 1: Block diagram of our method, the upper-branch blocks are common for training and test, the midle branch is the training scheme, and the lowest branch is the online processing.data

Figure 2 :

 2 Figure 2: Different forms of Wooding's map (in raster scan order) (i) The original image I with the fixation point f drawn in red on it and (ii) Normalized Wooding's Map W (I, f ) (iii) The Heat-Map visualization; (iv) The Weighted Map where the saliency is used to weight brightness in the frame.

  , nb = 256, pad = 1, , nb = 384, pad = 1, , nb = 384, pad = 1, b = 0 13x13x384 8 LRN norm2 k = 5x5, α = 10 -4 , β = 0

Figure 3 :

 3 Figure 3: Patch extraction: (i) Middle: bounding box of the object-of-interest (ii) Middle left and right: the exclusion area. (iii) Top and bottom: example of background patches with a maximum overlap of 20%

Figure 4 :

 4 Figure 4: Patch augmentation: Raws show different object categories and column show different augmentations. Each group of 4 columns depict different rotation angle α, and within this, each column is a different blurring kernel size k

Figure 5 :

 5 Figure 5: Left: A subject equipped with the Tobii Pro Glasses 2 performing the experiment. Right: The egocentric field of view of the glasses.

Figure 6 :

 6 Figure 6: Annotation Tool user interface: the sequence can be played using buttons 2 to 4;

Figure 7 :Figure 8 :

 78 Figure 7: Training and validation of the network. Left: Training loss and learning rate as a function of the number of iterations. Right: Validation accuracy as a function of the number of iterations

  Therefore we reject the beginning of each video sequence when selecting both training and validation frames and do this at the test stage as well. Micro-saccades are not a problem thanks to the interpolation of fixation coordinates that maintain the eye fixation on the object-of-interest. Distractors, nevertheless, are a real challenge: they

cannot be automatically identified neither when performing semi-automatic annotation for training nor at the test stage. And so they are included in our training set in the form of incorrectly labeled patches. To deal with them in our online framework, we propose to use temporal fusion of the classification results

Table 1 :

 1 ImageNet[START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF] architecture. k is the kernel size, nb is the number of filters learned, b is the bias, pad is the zero-padding size, and s the stride

	Layer	Depth	Type	Name	Parameters	Top shape
	23		Soft Max	prob		C
	22		FC	ip8		C
	21		Dropout	drop7	ratio = 0, 5	4096
	20		ReLU	relu7		4096
	19		FC	ip7	b = 1	4096
	18		Dropout	drop6	ratio = 0, 5	4096
	17		ReLU	relu6		4096
	16		FC	ip6	b = 1	4096
	15		Max pooling	pool5	k = 3x3, s = 2	6x6x256
	14		ReLU	relu5		13x13x256

Table 2 :

 2 Number of videos in the LEGO dataset, by category for Train, Validation and Test sets.

	Categories	Training Validation Testing Total
	Background	90	31	0	121
	Cone	13	5	4	22
	Cylinder	4	2	1	7
	Hemisphere	8	3	3	14
	Hexagonal Prism	10	4	3	17
	Rectangular Prism	17	5	6	28
	Rectangular Pyramid	10	4	3	17
	Triangular Prism	17	5	4	26
	Triangular Pyramid	11	3	4	18
	Total/BGD	90	31	28	149

Table 3 :

 3 Number of image patches by category for Train, Validation and Test extracted from the LEGO dataset.

	Categories	Training Validation Testing	Total
	Background	123 424	43 024	0	166 448
	Cone	17 824	6 352	420	24 596
	Cylinder	6 544	2 928	111	9 583
	Hemisphere	13 360	4 016	272	17 648
	Hexagonal Prism	16 592	5 776	235	22 603
	Rectangular Prism	24 032	6 816	620	31 468
	Rectangular Pyramid	10 736	4 448	308	15 492
	Triangular Prism	21 168	7 744	396	29 308
	Triangular Pyramid	16 784	4 976	412	22 172
	Total	250 464	86 080	2 774 339 318

Acknowledgments

This work was supported by CNRS-Idex grant PEPS Suivipp 2015.


Nawal Guermouche

Salima Benbernou
email: sbenbern@liris.cnrs.fr

Emmanuel Coquery
email: ecoquery@liris.cnrs.fr

Mohand-Said Hacid
email: mshacid@liris.cnrs.fr

Privacy-Aware Web Service Protocol Replaceability *

 by privacy properties. We introduce a rule-based privacy model and we extend business protocols, leading to what we call private business protocols. Finally, a private replaceability analysis of private business protocols is discussed. We mainly investigate compatibility issues, that is whether one private business protocol can support the same set of conversations with respect to the privacy requirements.

Introduction

Web services are becoming one of the main technologies for designing and building complex inter-enterprise business applications. Due to standard interfaces, mainly based on XML technology (exp. WSDL [START_REF] Christensen | Web services description language[END_REF], BPEL [START_REF] Andrews | Business process execution language for web serices[END_REF]) used by Web services to integrate heterogeneous and au-tonomous applications, Web services have changed the way companies manage their business applications. Since the technology of Web services is increasingly being adopted by industrial companies, security and privacy issues in Web services become an important topic of investigation. Some authors already considered access control management [START_REF] Macella | Access control enforcement for conversation-based web services[END_REF], mainly the enforcement of access control for Web services based on their conversational behavior. In this paper we consider privacy issues which arise in many situations where private data must be exchanged and handled. For example, to process a client request, business applications can require sensitive data from clients. To gain control over these data, mechanisms that can support disclosure of private data should be designed. Policies that are rules specifying conditions under which private data could be collected might provide a way for supporting disclosure of private data. Furthermore, for business processes based on the coordination between several Web services, if a given service becomes overcrowded or damaged, there is an impact on the general process. An alternative might be replacing a non-available service by another service able to provide the same functionalities. This mechanism is called replaceability. However, replaceability depends not only on the functional properties but also on non functional properties such as privacy requirements. In this paper, we investigate: [START_REF] Agrawal | Implementing p3p using database technology[END_REF] an approach for modeling privacy policies and their integration to business protocols [START_REF] Benatallah | Analysis and management of web service protocols[END_REF], and (2) a non functional replaceability analysis of Web service protocols: privacy aware replaceability.

The rest of the paper is organized as follows. Section 2 presents a motivating example that will be used to illustrate the different steps of the approach. Section 3 describes the rule-based privacy model we propose. Section 4 is devoted to the different scenarios regarding replaceability of privacy policies. Section 5 focuses on the extension of business protocols by the privacy policies model. Related works are presented in Section 6. We conclude in Section 7.

Motivating example

We consider a hotel booking service. The requester specifies the desired destination (city). Then, the service suggests a list of hotels. Once the requester has made the choice, she/he is requested to provide a credit card number to complete the reservation. Since this data is very sensitive, the requester might ask for a kind of privacy secure usage.

Suppose that due to a sudden collapse, a service cannot work properly. Since, the requester has provided private data, it could be interesting if one can transparently replace the damaged service. Therefore, the candidate service should (1) have the same functionalities as the damaged one and (2) ensure the same privacy level of the collected data. So far, work on replaceability [START_REF] Benatallah | Analysis and management of web service protocols[END_REF] considers only the functional properties, the non functional aspects which are as important as privacy policies are not considered.

Privacy model

In this section, we introduce our model of privacy rules as an extension of the categories of rules defined in the platform of privacy preferences P3P [START_REF] Agrawal | Implementing p3p using database technology[END_REF]. In order to establish an interaction between a client and a provider, the client specifies through rules called privacy preferences a way private data can be used by the provider, and the provider specifies through rules called privacy policies how private data will be used. To establish a conversation between a client and a provider (the client provides its private data), the preferences of the client must be consistent with the policy of the provider.

Remark 1 A preference pref is consistent with a policy plcy if the policy plcy is more restrictive than the preference pref . We define in Section 4 how we compare the level of restriction of two policies. Since a preference is defined as a policy, we will be able to check the consistency of a preference with a policy. Since the service provider can also request another Web service (it becomes a client), it should specify preferences. Thus, each Web service owns preferences and policies. In the following, we present the ingredients that constitute a policy (respectively a preference). As mentioned previously, privacy policies are rules specified by the provider describing how it will handle private data of its client. As depicted in Figure 1, a privacy policy is defined as a finite set of Terms of Data Usage, denoted T DU. A tdu ∈ T DU consists of a private data and a purpose for which the private data must be collected.

A purpose is an action representing the need of a client and executed by a given entity. The entities are those that can use the data to fulfill the request (purpose) of the client in a given time frame.

Furthermore, the provider can require getting a choice to perform or not other actions called Rights.

Since the fulfillment of purposes and rights involves the use of the client's private data, the provider must guarantee their security. For this, it must specify actions called obligations intented to secure the data.

To summarize, a purpose involves two kinds of actions:

• Rights: a right is an action the provider is allowed to do. For each right, we specify the entities authorized to perform it, the delay during which the entities own the right and the delay during which the right must be achieved once activated. Also a right can induce a set of obligations.

• Obligations: an obligation is the action the provider must achieve after collecting private data to ensure their security. An obligation is specified like a right but it does not involve any action.

Definition 1 Let us define the following sets:

U : set of entities, D: set of data, A: set of actions, P : set of purposes, O: set of obligations, R: set of rights, I: set of intervals. U , D, A are represented as a hierarchy by an ontology used to compare the restriction levels of two policies (section 4).

A privacy policy is a set of terms of data usage tdu

where tdu is an element of D × P .

A purpose p is defined by the tuple

(a, u, µ, S R , S O ) ∈ A × U × I × 2 R × 2 O
, where a is the action identifying the purpose, u is the entity performing the purpose, µ is an interval in which u must perform the action a, S R is a set of rights and S O is a set of obligations associated with the purpose.

3. A right r is defined by the tuple (a , u , ν , µ , S O ) ∈ A×U ×I×I×2 O , where a is the action identifying the right, u is the entity authorized to perform the action a , ν is the delay in which the entity u is authorized to perform the action a , µ is the delay in which the action a must be performed once activated.

4. An obligation o is defined by the tuple (a , u , ν , µ ) ∈ A × U × I × I such that a is the action identifying the obligation, u is the entity performing the obligation, ν is validity time interval of the action a , µ is the time interval within which the action a must be performed once activated.

As the provider can disclose the data it collects to a third party, we distinguish between two kinds of entities (i) ours specifies the entities of the service collecting the private data, and (ii) others specifies third party entities for which a service can disclose the collected private data.

Example 1 Back to the motivating example, the client must provide his credit card number CCN to pay the hotel reservation. The restrictions on CCN are as follows:

The financial service collects the credit card number CCN to pay the corresponding hotel reservation (p 1 ) within 20 minutes after getting the CCN . The Bank which is an external entity owns the right to verify the CCN validity (r 1 ) within 15 minutes after receiving CCN . If the right (r 1 ) is triggered, the corresponding action must be performed within 5 minutes. The financial service must destroy the CCN (o 1 ) 10 minutes after the achievement of the purpose. Moreover, the bank must encrypt the CCN (o 2 ) within 60 minutes following the CCN reception. The deletion (o 1 ) and the coding (o 2 ) must be performed immediately after their triggering, specified by an empty interval time [0,0]. Thus, the corresponding policy for CCN can be represented as follows:

plcy = {(CCN, p 1 )} where p 1 = (P ayReservation, Ours : F inancialService, µ : [0, 20 min], {r 1 }, {o 1 }) r 1 = (V alidityV erif ication, Others : Bank, ν : [0, 15 min], µ : [0, 5], {o 2 }) o 1 = (Destruction, Ours : F inancialService, ν : [t(p 1 ), t(p 1 ) + 10min], µ : [0, 0]) such that t(p 1 ) ∈ [0, 20min] is the time interval for fulfilling the purpose. o 2 = (Encrypt, Others : Bank, ν : [0, 60min], µ[0, 0])

Preference

A client service can send its own private data to a provider, so it should specify through rules called local privacy preferences how it wishes the provider to use its private data. Furthermore, the provider, seen as a client, can send the collected private data to a third party. Thus, it should specify through rules called external preferences how the third party must use private data. The local and external preferences constitute the preferences of the client.

Similar to a privacy policy, a preference is defined by a finite set of terms of data usage. However, we do not distinguish between the internal (Ours) and external (Others) entities. In fact, the client ignores whether the entities are considered as a third party (Others) or not (Ours) for the service collecting the private data.

Extraction of external preferences from policies

Since a service Q can invoke other services, hence it can disclose the private data of its client to a third party, as depicted in Figure 2.

(3)

(4) (5)
...

Figure 2. Extraction of the external preferences from policies.

The interaction between the service Q and a third party is based on the preferences of the service Q and the policies of the third party (3). To keep informed a third party of the restrictions of the client through the Web service Q, we add these restrictions in the external preferences of the service Q (2) as depicted in Figure 2. Thus, we propagate the set of tdu of the service Q policy to external preferences [START_REF] Andrews | Business process execution language for web serices[END_REF]. Hence, we add each tdu of the policy to the set T DU of preferences when the entity responsible to fulfill a purpose (right, obligation respectively) is an external entity (Others).

Replaceability analysis of privacy policies

In order to extend the replaceability of Web services defined in [START_REF] Benatallah | Analysis and management of web service protocols[END_REF] by accommodating privacy requirements, we provide a privacy aware replaceability analysis of policies. To compare two policies, we propose to define an order between two policies based on their restriction levels.

Comparing the restriction levels of two policies

In this section, we give an intuitive idea on how we compare two policies based on their restriction levels. Figure 3 shows parts of available hierarchies: hierarchy of purposes, hierarchy of users and hierarchy of data. Hierarchies are used to compare the data, actions and users related to purposes (rights and obligations respectively) as explained in the following example.

Example 2 Let us consider the two policies plcy 1 and plcy 2 such that:

plcy 1 ={(F inancialInf ormation, p 1 )} where: p 1 = (ReserveDwelling, Ours : ReservationServices, µ : [0, 30min], {r 1 }, {o 1 }) r 1 = (V alidityV erif ication, Others : F inancialP ar- tner, ν : [0, 15min], µ : [0, 5], {o 2 }) o 1 = (Destruction, Ours : ReservationServices, ν : [t(p 1 ), t(p 1 ) + 10min], µ : [0, 0]) such that t(p 1 ) ∈ [0, 30min] is the time interval for fulfilling the purpose p 1 . o 2 = (Encrypt, Others : F inancialP artener, ν : [0, 60 min], µ[0, 0]) plcy 2 ={(NumberCard, p 2 )} where: p 2 = (ReserveHotel, Ours : HotelServices, µ : [0, 20 min], {r 1 }, {o 1 }) r 1 = (V alidityV erif ication, Others : Bank, ν : [0, 15min], µ : [0, 5], {o 2 }) o 1 = (Destruction, Ours : F inancialServices, ν : [t(p 2 ), t(p 2 ) + 10min], µ : [0, 0]) such that t(p 2 ) ∈ [0, 20min] is the time interval for fulfilling the purpose p 2 . o 2 = (Encrypt, Others : Bank, ν : [0, 60min], µ[0, 0])
For instance, we can observe that the policy plcy 2 is more restrictive than the policy plcy 1 since the private data considered in the policy plcy 1 concerns the financial information, and as it is depicted in Figure 3, the financial information gathers the account and card information. Thus the data specified in plcy 2 (credit card number) is more restrictive (more specific) than the data of plcy 1 (financial information). Moreover, the purpose specified in the policy plcy 1 consists in dwelling reservation which gathers hotel reservation and apartment (see figure 3), which is more restrictive than the purpose of the policy plcy 2 , since hotel reservation is a sub-purpose of dwelling reservation. Furthermore, the entity that must achieve the purpose of the policy plcy 1 is the reservation service which contains the hotel service specified in the policy plcy 2 . In the policy plcy 1 reservation service must perform the dwelling reservation within 30 minutes after the reception of the financial information. The interval specified in the policy plcy 2 is included in the one specified in the policy plcy 1 , hence the interval of plcy 2 is more restrictive than the interval of plcy 1 .

Similar to what we did with data, purpose and time interval to verify if the policy plcy 2 is more restrictive than the policy plcy 1 , we do the same with the other elements.

Next we present how we compare obligations, rights, purposes and terms of data usage based on their restriction levels to perform the privacy-aware replaceabity of policies.

Comparing obligations

O . An obligation o 2 = (a 2 , u 2 , ν 2 , µ 2 ) is more restrictive than an obligation o 1 = (a 1 , u 1 , ν 1 , µ 1 ) denoted by o 1 O o 2 if and only if a 2 a 1 , u 2 u 1 , ν 2 ⊆ ν 1 , µ 2 ⊆ µ 1 . X Y stands for X is sub- sumed by Y .
As a right involves a set of obligations, the above comparisons of obligations allows the following comparison of rights.

Comparing Rights

R . A right r 2 = (a 2 , u 2 , ν 2 , µ 2 , S O 2 ) is more restrictive than a right r 1 = (a 1 , u 1 , ν 1 , µ 1 , S O 1 ) denoted by r 1 R r 2 if and only if a 2 a 1 , u 2 u 1 , ν 2 ⊆ ν 1 , µ 2 ⊆µ 1 , ∀o 1 ∈ S O 1 , ∃o 2 ∈ S O 2 such that o 1 O o 2 .
As a purpose involves a set of rights and obligations, the above comparison of rights and obligations allows the following comparison of purposes.

Comparing Purposes

P . A purpose p 2 = (a 2 , u 2 , ν 2 , S R 2 , S O 2 ) is more restrictive than a purpose p 1 = (a 1 , u 1 , ν 1 , S R 1 , S O 1 ) denoted by p 1 P p 2 if and only if a 2 a 1 , u 2 u 1 , ν 2 ⊆ ν 1 , ∀r 2 ∈ S R 2 , ∃r 1 ∈ S R 1 such that r 1 R r 2 , ∀o 1 ∈ S O 1 , ∃o 2 ∈ S O 2 such that o 1 O o 2 .
As a term of data usage defines a purpose for which the private data is collected, the above comparison of purposes allows the following comparison of terms of data usage.

Comparing terms of data usage T DU . A term of data usage tdu 2 = (d 2 , p 2 ) is more restrictive than a term of data usage tdu 1 = (d 1 , p 1 ) denoted by tdu 1 T DU tdu 2 if and only if

d 2 d 1 ∧ p 1 P p 2 .
Since a policy is defined by a finite set of terms of data usage, the above comparison of terms of data usage allows the following comparison of policies.

Comparing policies private .

A policy plcy 2 is more restrictive than a policy plcy 1 denoted by plcy 1 private plcy 2 if and only if

∀tdu 1 = (d 1 , p 1 ) ∈ plcy 1 , ∃tdu 2 (d 2 , p 2 ) ∈ plcy 2 such that tdu 1 T DU tdu 2 .
Remark 2 A preference pref is consistent with a policy plcy if plcy is more restrictive than pref (pref private plcy). Since a preference is defined as a policy by a finite set of terms of data usage tdu, we can use the above definition for comparing two policies to check the consistency between pref and plcy.

Privacy replaceability of policies

In this section, we distinguish three classes of privacyaware replaceability of policies which are: (1) private replaceability, (2) private equivalence, and (3) private partial replaceability.

Private replaceability

A policy plcy 1 can be replaced by a policy plcy 2 , denoted plcy 1 plcy 2 if all the restrictions supported by plcy 1 are also supported by plcy 2 .

Furthermore, to replace a policy plcy 1 by a policy plcy 2 , we must ensure that the policy plcy 2 does not violate the policy plcy 1 . The violation occurs when for the same data d, the policy plcy 2 specifies purposes (respectively rights and obligations) that are not specified in plcy 1 .

Example 3

For instance, we see that in the policy plcy 2 , the credit card number can be used to reserve a car which is not authorized in the policy plcy 1 . So plcy 2 violates the policy plcy 1 .

plcy 1 = {(F inancialnf o, p 1 )} such that: p 1 = (ReserveDwelling, Ours : ReservationServices, µ : [0, 30min], {r 1 }, {o 1 }) r 1 = (V

if and only if the following conditions hold:

• plcy 1 private plcy 2 .

• ∀tdu 2 = (d 2 , p 2 ) ∈ plcy 2 , tdu 1 = (d 1 , p 1 ) ∈ plcy 1 such that d 2 d 1 ∧ ¬(p 1 P p 2 ).
In other words, the first condition says that the policy plcy 2 must be more restrictive than the policy plcy 1 and the second condition says that the policy plcy 2 does not violate the policy plcy 1 .

Private equivalence

Two policies are equivalent if they have the same level of restrictions.

Definition 3 (Private equivalence ≡ .) A policy plcy 1 is said equivalent to the policy plcy 2 denoted by plcy 1 ≡ plcy 2 if and only if plcy 1 plcy 2 ∧ plcy 2 plcy 1 .

Private partial replaceability

When the policy plcy 2 does not replace the policy plcy 1 , but at least one tdu can be replaced, we say that the policy plcy 2 can partially replace the policy plcy 1 .

Definition 4 (Private partial replaceability.)

The policy plcy 2 partially replaces a policy plcy 1 if and only if the following conditions hold:

• ∃tdu 1 ∈ plcy 1 , ∃tdu 2 ∈ plcy 2 such that tdu 1 T DU tdu 2 . • ∃tdu 1 ∈ plcy 1 , tdu 2 ∈ plcy 2 such that tdu 1 T DU tdu 2 . • ∀tdu 2 = (d 2 , p 2 ) ∈ plcy 2 , tdu 1 = (d 1 , p 1 ) ∈ plcy 1 such that d 2 d 1 ∧ ¬(p 1 P p 2 ).
The first condition says that the policy plcy 2 must replace at least one tdu of the policy plcy 1 and the second condition says that there is at least one tdu of the policy plcy 1 that cannot be replaced by the policy plcy 2 . The last condition ensures that the policy plcy 2 does not violate the policy plcy 1 .

Annotation of business protocols with privacy rules

We present how to integrate our model of privacy rules into business protocols. This will lead to private business protocols which enable considering the privacy aspects in the replaceability analysis of Web services. A business protocol aims at specifying the external behaviour (the sequences of supported messages) of a Web service. Business protocols are specified as finite state machines, where states correspond to different states of the service and the transitions correspond to exchanged messages (input and output messages). Each transition is labelled with a message name followed by the message polarity indicating whether the message is incoming (plus sign) or outgoing (minus sign) [START_REF] Benatallah | Analysis and management of web service protocols[END_REF].

Private Business protocols

Our first extension consists in extending the polarity function 1 to consider the different aspects of privacy. Moreover, we extend the definition of transitions and states of business protocols.

Polarity

To specify that the input (respectively the output) message imports (respectively exports) private data (for short, we say private message), we propose to extend the polarity function by defining two variants according to the type of messages:

• Polarity of incoming messages: This polarity indicates if the message imports private data of the clients (see Definition 5).

• Polarity of outgoing messages: This polarity indicates if the message exports its own private data or those of its clients (see Definition 5).

1 +: Input message, -: Output message

Transitions

In business protocols, privacy policies must be associated with input private messages. Therefore, we propose to annotate each transition enabling an input private message by the corresponding policies.

States

In business protocols, a state can be a source for a set of transitions enabling an output private message. Hence, the corresponding preferences are associated with a state.

The following definition extends the traditional definition of business protocols by incorporating privacy aspects.

Definition 5 A private business protocol Q is a tuple Q = (S, s 0 , F,

M, P REF, ð, P LCY, T ) which consists of the following components:

• S is a finite set of states, where s 0 ∈ S is the initial state.

• F ⊆ S is a set of final states. If F = ∅, then Q is said to be an empty protocol.

• M is a finite set of messages. Consider a private data d which is handled by two input messages m 1 and m 2 of a service provider. So, we associate the message m 1 with the corresponding policy plcy 1 and the message m 2 with the corresponding policy plcy 2 . Suppose that the preference of the client (that invokes this service provider) is consistent with the policy plcy 1 and is inconsistent with the policy plcy 2 . In our approach, this inconsistency does not forbid the conversation between the client and the provider unless the invoked message is m 2 . However, the traditional approaches consist in checking all the rules (a global policy) related to the data d which prevents the conversation between the client and the provider even if the rule does not deal with the invoked message.

ServicePData =            Y if

Simulation of private protocols

Informally, a protocol Q is simulated (replaced) by a protocol Q if, starting from the initial state, each input (respectively, output) message of Q can be matched with an input (respectively, output) message of Q [START_REF] Benatallah | Analysis and management of web service protocols[END_REF]. Here, we introduce an extended definition of simulation to accommodate privacy policies. When we replace a Web service Q by a new one Q , Q will use its own private data, so we do not have to check the local preferences. But Q may also use the private data of the clients of Q, so obviously we have to check the consistency between the external preferences of Q and Q . Since the external preferences are extracted from policies already checked in the private replaceability of Q by Q , they are also implicitly checked.

Definition 6 Let Q = (S, s 0 , F, M, P REF, ð, P LCY, T ) and Q = (S , s 0 , F , M , P REF , ð , P LCY , T ) be two protocols. A protocol Q simulates a protocol Q denoted Q Q if and

Related work

Some authors investigated expressive privacy models (e.g., [START_REF] Elovici | A new privacy model for hiding group interests while accessing the web[END_REF][START_REF] Fischer-Hûbner | From a formal privacy model to its implementation[END_REF][START_REF] Karjoth | Platform for enterprise privacy practices: Privacy-enabled management of customer data[END_REF][START_REF] Karjoth | A privacy policy model for enterprises[END_REF][START_REF] Mont | Dealing with privacy obligations in enterprises[END_REF]). The Platform for Privacy Preferences (P3P) is a platform for standardization and specification of privacy rules for Web sites [START_REF] Agrawal | Implementing p3p using database technology[END_REF]. It enables a Web site user to gain control over his private information. P3P provides mechanisms for Web site owners to express their privacy rules called policies in a standard format that a user can programmatically check against his privacy rules (called preferences), to decide whether to release his data to the Web site. With respect to our model, to ensure the security of the data, P3P specifies only the saving time interval of private data. However, sometimes the owner needs to specify other actions to ensure the security of the private data which is considered in our model (for example keep the data encrypted). Moreover, P3P does not specify the time interval within which the purpose must be performed as it is specified in our model.

The work presented in [START_REF] Kagal | Authorization and privacy for semantic web services[END_REF] deals with the integration of privacy policies in semantic Web services. The authors extended the semantic description of OWL-S. The main advantage of this approach consists in preserving a security of the private data during their submission. However, many aspects such as temporal aspects attached to the data usage are not considered.

In the work presented in [START_REF] Benatallah | Analysis and management of web service protocols[END_REF], the authors present an approach to verify a dynamic substitution (replaceability) of Web services regarding their business protocols based on a protocols intersection operator. This work was extended by temporal constraints [START_REF] Benatallah | On temporal abstractions of web service protocols[END_REF]. This approach considers only conversations supported by business protocols and their temporal constraints. Nevertheless, Web services are constrained to use private data, which require to consider not only the functional properties, but also the non functional ones. With respect to the work presented in this paper, we have presented an approach that takes into account privacy properties of Web services in the replaceability analysis.

Conclusion

In this paper, we have presented an approach dealing with replaceability of Web services by considering privacy policies. We have proposed a rule-based privacy model and extended business protocols. Moreover, we have proposed three classes of replaceability of privacy policies which enable to analyze the replaceability of private business protocols.

As future work, we plan to enhance our private model rules with prohibitions and penalties. Moreover, we plan to extend the proposed model in order to express different temporal aspects. Furthermore, the management of temporal constraints [START_REF] Benatallah | On temporal abstractions of web service protocols[END_REF][START_REF] Kazhamiakin | Timed modelling and analysis in web service composition[END_REF] with respect to time intervals of purposes (respectively rights and obligations) is also an interesting issue.

Another direction of research is the investigation of an approach that should be able to allow best neighbouring private replaceability when full replaceability is not feasible. In order to improve the flexibility of the private replaceability, we aim to increase the level of the privacy restrictions.

Figure 1 .

 1 Figure 1. A privacy policy model.

Figure 3 .

 3 Figure 3. Examples of hierarchy.

  alidityV erif ication, Others : F inancialP artner, ν : [0, 15min], µ : [0, 5], {o 2 }) o 1 = (Destruction, Ours : ReservationServices, ν : [t(p 1 ), t(p 1 ) + 10min], µ : [0, 0]) such that t(p 1 ) ∈ [0, 30min] is the time interval for fulfilling the purpose p 1 . o 2 = (Encrypt, Others : F inancialP artner, ν : [0, 60min], µ[0, 0]) plcy 2 = {(NumberCard, p 1 ), (NumberCard, p 2 )} such that: p 1 = (ReserveHotel, Ours : HotelServices, µ : [0, 20min], {r 1 }, {o 1 }) r 1 = (V alidityV erif ication, Others : Bank, ν : [0, 15min], µ : [0, 5], o 2 ) o 1 = (Destruction, Ours : HotelServices, ν : [t(p 2 ), t(p 2 ) + 10min], µ : [0, 0]) such that t(p 2 ) ∈ [0, 20min] is the time when the purpose p 2 was fulfilled. o 2 = (Encrypt, Others : Bank, ν : [0, 60min], µ[0, 0]) p 2 = (CarReservation, Ours : T ransportServices, delay : [0, 20min], {r 2 }, {o 3 }) r 2 = (V alidityV erif ication, Others : Bank, ν : [0, 15min], µ : [0, 5], o 4 ) o 3 = (Destruction, Ours : T ransportServices, ν : [t(p 3 ), t(p 3 ) + 10min], µ : [0, 0]) such that t(p 3 ) ∈ [0, 20min]is the time interval for fulfilling the purpose p 3 . o 4 = (Encrypt, Others : Bank, ν : [0, 60min], µ[0, 0]) Definition 2 (Private replaceability .) A policy plcy 1 can be replaced by a policy plcy 2 , denoted by plcy 1 plcy 2 ,

--

  For each message m ∈ M , we define two variants of the function Polarity(Q,m): The polarity of input messages has the form of IP olarity(Q, m) = (+, ClientP Data) The polarity of output messages has the form of OP olarity(Q, m)=(-, ClientP Data, Servi-ceP Data)

  As the neighbouring replaceability can be partial, it will not be automatic and will require the authorization of the client. The full study of neighbouring replaceability goes beyond the scope of this paper.

only if there exists:

• A relation Γ ⊆ S × S such that the following holds:

-∀(s 1 , s 1 ) ∈ Γ and ∀T (s 1 , s

2 , m, plcy), there exists s 2 such that T (s 1 , s 2 , m , plcy ), m m , P olarity (Q, m) = P olarity(Q , m ), (s 2 , s 2 ) ∈ Γ, plcy plcy . -∀(s, s ) ∈ Γ, if s ∈ F then s ∈ F Remark 3 In order to replace a Web service Q, the first step consists in discovering a set of Web services. Suppose that among all the discovered Web services, there is no service that can fully replace Q according to their private policies. It could be interesting to choose between all of those services, the one likely to have the best neighbouring policies (ensuring the highest level of restrictions with respect to the policies of Q).

* This work is partially supported by the French National Research Agency (ANR) -Program "Jeunes chercheurs:Servicemosaic" a part of the international project ServiceMosaic; http://servicemosaic.isima.fr/.


Ulrich Eismann

Martin Enderlein

Konstantinos Simeonidis

Felix Keller

Felix Rohde

Dmitrijs Opalevs

Matthias Scholz

Wilhelm Kaenders

Jürgen Stuhler

Active and passive stabilization of a high-power UV frequency-doubled diode laser

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

High-power tunable single-frequency lasers around 400 nm have numerous applications in research and (quantum) technology. These span from laser cooling and trapping of ions (calcium and strontium [START_REF] Eschner | Laser cooling of trapped ions[END_REF]) or atoms (erbium [START_REF] Mcclelland | Laser cooling without repumping: a magnetooptical trap for erbium atoms[END_REF], dysprosium [START_REF] Ho Youn | Anisotropic sub-doppler laser cooling in dysprosium magneto-optical traps[END_REF], calcium [START_REF] Kisters | High-resolution spectroscopy with laser-cooled and trapped calcium atoms[END_REF] and chromium [START_REF] Stuhler | Continuous loading of a magnetic trap[END_REF]) to two-photon down conversion or even experiments with antimatter [START_REF] Kse Eikema | Continuous wave coherent lyman-α radiation[END_REF]. Furthermore, the 400-nm sources can serve as fundamental lasers for further frequency doubling to the DUV range below 200 nm [START_REF] Scholz | A bright continuouswave laser source at 193 nm[END_REF]. While Watt-class output power is required, direct diode lasers are limited to ∼100 mW [START_REF]DL pro HP -high power from blue laser diodes[END_REF]. Frequency doubling of infrared lasers potentially delivers higher power levels, but stability becomes a major concern. Here we present a laser system consisting of a resonantly frequencydoubled tapered amplified semiconductor laser which is capable of emitting 2.6 Watts of single-frequency emission around 400 nm. Based on passive and active measures, we demonstrate stable operation on timescales reaching from hundreds of nanoseconds to many days.

Output power and stability

The laser system consists of an external cavity diode laser (DL pro), which is amplified with a semiconductor tapered amplifier chip (TA pro) delivering up to 5 W around 800 nm. The almost diffraction-limited TA pro output beam is mode matched to the second harmonic generation (SHG pro) cavity. By choosing an input coupler reflectivity of 98.5%, we optimize the system for SHG output power and we obtain up to 2.6 Watts at 400 nm with several nanometers of tunability. The cavity is locked to resonance by a digital control loop with a bandwidth of 30 kHz using a slow and a fast piezoelectric transducer (PZT). The necessary error signal is obtained with the Pound-Drever-Hall technique [START_REF] Rwp Drever | Laser phase and frequency stabilization using an optical resonator[END_REF] from sidebands created by direct modulation of the DL pro current at 20 MHz. Furthermore, the digital control electronics features an automatic relock functionality. The output beam is almost diffraction limited. The results on a relative intensity noise (RIN) measurement of the laser system are shown in Fig. 1 (a). By integrating over the full range (10 Hz to 10 MHz), we obtain values of 0.01 % RMS for both DL pro and TA pro output by accessing both beams through the respective ports. Due to its finite acceptance range of facc ≈ 1 MHz, the SHG pro cavity converts frequency noise to RIN. The DL pro has a typical linewidth of 100 kHz. Thus, with a 2-stage PZT lock (slow PZT for large displacements, and fast PZT), the RIN of the system at 400 nm is increased to 0.21 % RMS (SHG w/o DL feedback in Fig. 1 (a)), and leveling off at frequencies above facc, where the cavity acts as a low-pass filter. We improve the RIN value significantly when locking the DL pro to the SHG pro cavity using direct feedback to the laser diode current (SHG w/ DL feedback in Fig. 1 (a)), and obtain a value of 0.12 % RMS.

The SHG pro cavity is of bow-tie type, contains a temperature controlled AR coated non-linear optical crystal and is air-sealed (typ. residual leak rate < 10 -5 mbar l/s). Together with the advanced mechanical design of the laser system, this yields excellent longterm stability, see Fig. 1 (b). A linear fit, represented by the red line, yields a relative power drop of 0.15%/h over more than 16 hours of continuous operation without any adjustments. Furthermore, by applying feedback to the TA pro current, the output power of either TA pro stage, SHG pro stage or the optional fiber can be actively held constant. We observe a beam pointing stability below 2 µrad RMS over 60 hours of continuous operation.

Even with the power stability shown in Fig. 1 (b), a time-to-time realignment of the laser system may be necessary in order to achieve optimum output power. Therefore, we have developped an automatic procedure (AutoAlign) relying on a simplex algorithm and a motorization of the patented flexure mirror mounts [START_REF] Eismann | kürzer, am kürzesten[END_REF]. The system is optimized stage-by-stage via two-mirror beam walks starting from the coupling of the DL pro beam into the TA pro. In a second step the TA pro beam is aligned to the SHG pro cavity. Finally, in case of fiber output, the coupling into fiber can be optimized. A histogram of the results of 200 AutoAlign procedures is shown in Fig. 1 (c). After deliberate misalignment, the algorithm converges to within fractions of a percent of the optimum value. The servos are switched off and completely passive when not in use for the AutoAlign procedure. Thus, no degradation of the laser parameters due to acoustic noise and heat input are observed.

Conclusion and outlook

We have presented results on a low noise, high power UV laser system allowing handsoff operation for extended periods of time. On the same platform, we have built similar systems with Watt-level power at e.g. 420 nm, 486 nm, 589 nm and 671 nm, and up to 2.7 W @ 560 nm. Frequency locking of the DL pro master laser allows relative or absolute stabilization to laser references, and offers the potential for Hz-linewidth output [START_REF] Yn Zhao | Subhertz frequency stabilization of a commercial diode laser[END_REF] for metrology. Furthermore, these sources are efficient fundamental lasers for frequency doubling to the DUV range.

Figure 1 :

 1 Figure 1: (a) Relative intensity noise (RIN) of different laser output stages. (b) Long-term passive output stability of SHG pro light. (c) Histogram of AutoAlign results. (a)-(c): See text for details.