Guillaume Dupont

Ekaterina Kalinicheva

Jérémie Sublime

Florence Rossant

Michel Pâques
email: mpaques@15-20.fr

Unsupervised Change Detection using Joint Autoencoders for Age-Related Macular Degeneration Progression

Keywords: Change detection, Unsupervised learning, ARMD

Age-Related Macular Degeneration (ARMD) is an eye disease that has been an important research field for two decades now. Researchers have been mostly interested in studying the evolution of lesions that slowly causes patients to go blind. Many techniques ranging from manual annotation to mathematical models of the disease evolution bring interesting leads to explore. However, artificial intelligence for ARMD image analysis has become one of the main research focus to study the progression of the disease, as accurate manual annotation of its evolution has proved difficult using traditional methods even for experienced doctors. Within this context, in this paper, we propose a neural network architecture for change detection in eye fundus images to highlight the evolution of the disease. The proposed method is fully unsupervised, and is based on fully convolutional joint autoencoders. Our algorithm has been applied to several pairs of images from eye fundus images time series of ARMD patients, and has shown to be more effective than most state-of-the-art change detection methods, including non-neural network based algorithms that are usually used to follow the evolution of the disease.

Introduction

Dry age-related macular degeneration (ARMD or AMD), a degenerative disease of the retina, is a main cause of irreversible visual loss. It is characterized by a centrifugal progression of atrophy of the retinal pigment epithelium (RPE),

This study has been approved by a French ethical committee (Comité de Protection des Personnes) and all participants gave informed consent. a cellular layer playing a key role in the maintenance of the photoreceptors. Blindness may occur when the fovea (the central part of the eye), is affected. The disease can be diagnosed and monitored using fundus images: ophthalmologists can observe pathologic features such as drusens that occur in the early stages of the ARMD, and evaluate the geographic atrophic (GA) progression in the late stages of degeneration (Fig. 1). Automatic analysis of dry ARMD fundus images is of high medical interest and this has been an important research field for two decades, for diagnosis [START_REF] Priya | Automated diagnosis of age-related macular degeneration from color retinal fundus images[END_REF] or follow up [START_REF] Köse | Automatic segmentation of age-related macular degeneration in retinal fundus images[END_REF], [START_REF] Ramsey | Automated image alignment and segmentation to follow progression of geographic atrophy in age-related macular degeneration[END_REF] purposes. Imaging modalities are most often color eye fundus images [START_REF] Köse | A statistical segmentation method for measuring age-related macular degeneration in retinal fundus images[END_REF][START_REF] Marrugo | Retinal image restoration by means of blind deconvolution[END_REF]3], fundus autofluorescence (FAF) [START_REF] Lee | A hybrid segmentation approach for geographic atrophy in fundus auto-fluorescence images for diagnosis of age-related macular degeneration[END_REF][START_REF] Hu | Automated segmentation of geographic atrophy in fundus autofluorescence images using supervised pixel classification[END_REF][START_REF] Ramsey | Automated image alignment and segmentation to follow progression of geographic atrophy in age-related macular degeneration[END_REF], and, to a lesser extent, confocal scanning laser ophthalmoscopy (cSLO) in infrared (IR), or optical coherence tomography (OCT) [START_REF] Hu | Segmentation of the geographic atrophy in spectral-domain optical coherence tomography and fundus autofluorescence images[END_REF]. In this work, we use cSLO images in infrared (IR): this modality is comfortable to acquire for the patients, and has better resolution and contrast than color imaging, an older technology. Our goal is to detect the appearance of new atrophic areas and quantify the growth of GA from pairs of images acquired from follow-up exams and to ultimately propose predictive models of the disease progress.

Figure 1 shows 3 pairs of consecutive images, taken at 6 months interval. The lesions (GA) in the fundus and around the optical disk are the brighter areas. Monitoring the GA progression in these areas is obviously challenging because of the images quality: uneven light, saturation issues, illumination distortion between images, GA poorly contrasted with retinal structures interfering (vessel, optical disk), blur, etc. The difficulty also lies in the high variability of the lesions in terms of shape, size and number. The lesion boundary is quite smooth in some cases (c and d) and very irregular in others (a and b). At any time, new spots can appear (as shown by the green arrow between e and f) and older lesions can merge. All these defaults make the manual delineation task very complex, even for expert ophthalmologists.

In order to assess the disease progression, it is necessary to perform a differential analysis between consecutive images to get the lesion growth, so that the lesion growth can be modelled. In this paper, we propose a fully unsupervised differential analysis method based on a joint autoencoders. Our model does not require labeled images that are difficult to come by in quantity and quality high enough to train a supervised neural network. Our method is applied to pairs of images of a patient eye fundus time series and aims at efficiently segmenting medically significant changes between the two images: we are interested only in changes in the GA lesions, while meaningless differences and light artefacts should be ignored.

Related Works

The following works are most related to our proposed algorithm as they are unsupervised algorithm applied to various eye disease images, including ARMD: In [START_REF] Troglio | Unsupervised change-detection in retinal images by a multiple-classifier approach[END_REF], Troglio et al. published an improvement of their previous works realized with Nappo [START_REF] Troglio | [END_REF] where they use the Kittler and Illingworth (K&I) thresholding method. Their method consists of applying the K&I algorithm on random sub-images of the difference image obtained between two consecutive eye fundus images of a patient with retinopathy. By doing so, they obtain multiple predictions for each pixel and can then make a vote to decide the final class. This approach has the advantage that it compensates for the non-uniform illumination across the image, however it is rather primitive since it does not actually use any Machine Learning and rely on different parameters of the thresholding method to then make a vote. To its credit, even if it achieves a relatively weak precision, it is fully unsupervised like our method. In [START_REF] Marrugo | Retinal image restoration by means of blind deconvolution[END_REF], the authors tackle a similar problematic to ours where they correct eye fundus images by pairs, by multiplying the second image by a polynomial surface whose parameters are estimated in the least-squares sense. In this way, illumination distortion is lessened and the image difference enhances the areas of changes. However, the statistical test applied locally at each pixel is not reliable enough to get an accurate map of structural changes.

Other works related with eye diseases take the different approach of segmenting lesions in individual images instead of looking for changes in pairs of images. In [START_REF] Köse | A statistical segmentation method for measuring age-related macular degeneration in retinal fundus images[END_REF], Köse et al. proposed an approach where they first segment all healthy regions to get the lesions as the remaining areas. This approach also requires segmenting separately the blood vessels, which is known to be a difficult task. This method involves many steps and parameters that need to be supervised by the user. In [START_REF] Ramsey | Automated image alignment and segmentation to follow progression of geographic atrophy in age-related macular degeneration[END_REF], Ramsey et al. proposed a similar but unsupervised method for the identification of ARMD lesions in individual images: They use an unsupervised algorithm based on fuzzy c-means clustering. Their method achieves good performances for FAF images, but it performs less well for color fundus photographs. We can also mention the work of Hussain et al. [START_REF] Hussain | Automated tracking and change detection for age-related macular degeneration progression using retinal fundus imaging[END_REF] who proposed another supervised algorithm to track drusen progression for ARMD. It uses a U-Net to segment vessels and detect the optic disc to reduce the region of interest of drusen detection. After that, they detect the drusen using intensity ratio between neighbor pixels.

Other traditional Machine learning algorithms have also been used for GA segmentation such as random forest [3] or k-nearest neighbor classifiers [START_REF] Hu | Automated segmentation of geographic atrophy in fundus autofluorescence images using supervised pixel classification[END_REF]. Feature vectors for these approaches typically include intensity values, local energy, texture descriptors, values derived from multi-scale analysis and distance to the image center. Nevertheless, these algorithms are supervised: they require training the classifier from annotated data, which brings us back to the difficulty of manually segmenting GA areas.

Apart from medicine, change detection algorithms have been proposed for many different applications such as remote sensing or video analysis. In [START_REF] Celik | Unsupervised change detection in satellite images using principal component analysis and k-means clustering[END_REF] the authors reveal a method combining principal component analysis (PCA) and K-means algorithm on the difference image. In [START_REF] Kalinicheva | Change detection in satellite images using reconstruction errors of joint autoencoders[END_REF], an architecture relying on joint auto-encoders and convolutional neural networks is proposed to detect nontrivial changes between two images. In [START_REF] Chong | Abnormal event detection in videos using spatiotemporal autoencoder[END_REF] the authors propose an autoencoder architecture for anomaly detection in videos.

Finally, as we have seen that quite a few methods rely on segmentation first and change detection after. We can also mention a few noteworthy unsupervised segmentation algorithms used outside the field of medicine: Kanezaki et al. [9] used CNN to group similar pixels together with consideration of spatial continuity as a basis of their segmentation method. Finally, W-Nets using with a soft Normalized-Cut Loss are another option.

Data description

In this section, we will provide some details on the data we used.

Our images were all acquired at the Quinze-Vingts National Ophthalmology Hospital in Paris, in cSLO with IR illumination. Patients have been followed-up during a few years, hence we have series of retinal fundus images, often for both eyes, showing the progression of the GA. The average number of images in each series is 13. We used 336 images from 15 patients time series taken between 2007 and 2019. All pictures are in grayscale and vary greatly in size, but the most common size is 650 * 650 pixels.

As mentioned in the introduction, the images contain many imperfections such as blurs, artifacts and, above all, non-uniform illumination (see Fig. 2). All images were spatially aligned with i2k software 4 . Furthermore, all images are surrounded by black border that contain no useful information. These borders were removed from the segmentation area using a mask.

All images were preprocessed using a new method (not published yet) to reduce the light distortion within images series. This algorithm relies on an illumination / reflectance model and corrects all images of a serie with respect to a common reference image. Uneven illumination generally remains present in every processed image (Fig. 2) but the smooth illumination distortions are compensated. The calculus of the absolute value of the difference between two consecutive images demonstrates the benefit of this algorithm (Fig. 2, last column).

We used 3 different series of images to rate our proposed change detection method: they feature different characteristics in terms of disease progress, lesion We developed several user-guided segmentation tools for ophthalmologists to build the validation images with the expected segmentation: These tools allowed them to provide a manual segmentation of some of our dataset images using techniques such as segmentation algorithms, local thresholdings and simple linear interpolation to draw the lesions borders. However, it is worth noting that the task remained long and tedious and couldn't be done for all images as on average, it took 13 minutes to process one single image. Finally, the binary change mask between two consecutive images was obtained by subtraction of the segmentation masks.

Proposed algorithm

Our algorithm uses principles from previous remote sensing applications [START_REF] Kalinicheva | Change detection in satellite images using reconstruction errors of joint autoencoders[END_REF], where an unsupervised deep autoencoder was used to automatically detect meaningful changes between two satellite images with the goal of finding new constructions or changes in the landcover, all the while discarding seasonal changes.

In our paper, we use the common issues between satellite imaging and our medical ARMD eye fundus to adapt this deep learning algorithm: both types of images may suffer from lighting issues, noise issues, blurry elements, complex objects present in the images, various intervals between images, and most importantly the common goal of detecting only the changes from specific classes within the images.

While the problematic look similar, remote sensing and medical applications also have specific issues: medical images only have one channel, the textures are different, and the size and scale of both the images and objects to analyze is quite different. To account for these specificities, we had to modifiy their AE architecture as described in the next subsection.

Joint Autoencoder Architecture

As mentioned previously, our algorithm is based on autoencoders. Autoencoders [START_REF] Hinton | Reducing the dimensionality of data with neural networks[END_REF] are a type of neural networks whose purpose is to make the output as close as possible to the input. The learnign process consists in an encoder part learning some meaningful representations of the original data and a decoder transforming them back into the original data: in a fully convolutional AE, a stack of convolutive layers is applied to the input image in order to extract feature maps (FM) which will then be used to reconstruct the input image.

Usually AEs with dense layers are used to perform a dimensionality reduction followed by a clustering or segmentation. However, in computer vision, fully convolutionnal AEs are prefered for their ability to extract textures. Examples of such networks include fully convolutional networks (FCNs) or U-Nets [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF]. However, in our case we do not use pooling layers, and so we keep the same dimensions as the input and only the depth increases.

Our network (Fig 3) is made of 4 convolutional layers in the encoder of size 16, 16, 32, 32 respectively, and in the same way of 4 convolutional layers of size 32, 32, 16, 16 respectively in the decoder side. We apply a batch normalization and a ReLU activation function at each step of the network except for the last layer of the encoder where we only add the L2 normalization, and also for the last layer of the decoder where we apply a Sigmoid function (see in Figure 3).

Algorithm steps

Our algorithm is made of four steps. We start by dividing the images into several patches. Then we build the joint autoencoder where it learns how to reconstruct the images, and after we tweak the method by learning the autoencoder to reconstruct not the image itself but the precedent or the future image. The neural networks will learn the changes caused by the non-uniform illumination or noise, but will fail on ARMD progression generating a high reconstruction error (RE), consequently making it possible to detect them. The next subsections will detail some of these steps:

Pre-training Let us consider a series of M images representing the progression of ARMD in a patient's eye. Each image has the same number of N useful patches, and we sample N M of the patches from every image. This allows us to build a unique autoencoder AE that works for all pairs in the series, and to prevent overfitting. We also apply a gaussian filter to the patches in order to weight the pixels by giving more importance to the center of the patch in the RE calculus. During the encoding pass of the AE, the model extracts feature maps of N patches of chosen samples with convolutional layers (Figure 4), and then during the decoding pass, it reconstructs them back to the initial ones.

Fine-tuning For every consecutive pair i, i + 1 with i ∈ [[1; M -1]] of images we are going to build two autoencoders initialized with the weights found in the pre-training part. On one hand AE i,i+1 , aims to reconstruct patches of Im i+1 from patches of Im i and, on the other hand, AE i+1,i is going to reconstruct patches of Im i from patches of Im i+1 .

The whole model is trained to minimize the difference between: the decoded output of AE i,i+1 and Im i+1 , the decoded output of AE i+1,i and Im i , and the encoded outputs of AE i,i+1 and AE i+1,i , see Figure 4.

This joint configuration where the learning is done in both temporal directions, using joint backpropagation, has empirically proven to be much more robust than using a regular one-way autoencoder [START_REF] Kalinicheva | Change detection in satellite images using reconstruction errors of joint autoencoders[END_REF]. To optimize the parameters of the model, we use the mean squared error (MSE) of the reconstructed patches Reconstruction and Thresholding Once the models are trained and stabilized, we perform the image reconstruction. For each pair, we note Im i+1 the reconstruction of Im i+1 from Im i with AE i,i+1 and likewiwse we note Im i the reconstruction of Im i from Im i+1 with AE i+1,i . Then, we calculate the reconstruction error RE for every patch between Im i and Im i on one side and between Im i+1 and Im i+1 on another side. This gives us two images for each pair representing the average REs for Im i and Im i+1 , that we average to get only one. The model will easily learn the transformation of unchanged areas from one image to the other: changes in luminosity and blurring effects. At the same time, because the changes caused by the disease progression are unique, they will be considered as outliers by the model, and thus will have a high RE. Hence, we apply Otsu's thresholding [START_REF] Otsu | A threshold selection method from gray-level histograms[END_REF] that requires no parameters and enables us to produce a binary change map (BCM ) (Figure 4).

Experimental results

Experimental settings

We chose to compare our methods presented in subsection 4.1 with 3 other methods, all on the preprocessed images. We applied all the methods to 3 of our series for which we have a ground truth.

The following parameters were chosen for all convolutional layers of our method: kernel size to 3, stride to 1 and padding to 1. Adam algorithm was used to optimize the models. We set the number of epochs to 8 for the pre-training phase and just 1 for the fine-tuning phase. For both phases, the learning rate was set to 0.0001 and the batch size to 100. Using trials and errors to determine the best parameters, we chose a patch size P = 13 and a value σ = 12 for the Gaussian filters.

The first method that we use for comparison is a simple subtraction of two consecutive images with an application of Otsu's thresholding on the result. The second comparison is a combination of principal component analysis (PCA) and K-means algorithm on the difference image proposed by Celik et al. in [START_REF] Celik | Unsupervised change detection in satellite images using principal component analysis and k-means clustering[END_REF], and we apply it to medical images with blocks of size 5. To finish we take a Deep-Learning based approach [9] which uses CNN to group similar pixels together with consideration of spatial continuity. This work by Kanezaki et al. was initially made for unsupervised segmentation, consequently, we apply the algorithm to our images and then do the segmentation substractions to get binary change maps. The convolutive layers have the same configuration than for our network and we set the minimum number of labels to 3.

All algorithms were executed on an Nvidia GPU (RTX TITAN) with 64 GB of RAM and an Intel 9900k. It took about 20 minutes for a series of 8 frames with a patch size P of 13, with the execution time increasing with it.

Results

The results for patients 1, 3 and 5 are shown in Table 1, as well as Figures 5 and6 that we added for visual inspection purposes. Note that the scores presented in Table 1 are for the complete series (15 to 20 pairs per patient), while the scores schown in the Figures are for the individual couples of images used in each example.

When looking at Table 1, we can see that the simple difference coupled with Otsu thresholding achieves the best recall results on average, that there is no clear winner for the Precision, and that our proposed method on average has the best F1 Score. Note that we did not use Accuracy of the strong class imbalance, with a large majority of "no change class" pixels leading to results over 95% of accuracy for all methods that were irrelevant. Our interpretation of these results is the following: Otsu thresholding applied to the difference between two images has the best recalls because it detects most real change pixels. But the binary change map is also very noisy, corresponding to a high number of false positives (wrongly detected changes) which is confirmed by the very low precision score. This can also be observed on Figure 5-d which is an example of the high number of false positive detected using Otsu thresholding compared with the ground truth in Figure 5-c In Figure 6, we compare our approach with the 2 other algorithms relying on more advanced Machine Learning techniques. First we can see that like in Table 1, our approach gets the best F1-score for both patients and pairs of images. Then, we can see that Kanezaki et al. approach achieves over-segmentation in Figure 6-d and under-segmentation in Figure 6-j, which highlights that it is more difficult to parametrize and may require different parameters for each pair of image, which is not the case for both our approach and Celik et al. approach. Finally, regarding the comparison between Celik et al. approach and our proposed method, we can see from Figures 6-e, 6-f, 6-k and 6-l, that also like in Table 1, Celik et al. approach achieves overall good results that are comparable to the ones of our method. However, in the same way that we have better F1-score and accuracy results, the visual results for our methods are also better as the changes we detect in the lesions are cleaner and overall less fragmented into very small elements compared with the ones found by Celik et al. approach. Furthermore, we can see that our method finds changes that are more in the peripheral areas of the lesions, while Celik et al. approach tends to find elements inside existing lesions (6-k) which are of lesser interest from a medical point of view. Overall, we can conclude that both Otsu thresholding and Kanezaki's approach suffer from risks of over-segmentation detecting a lot of noise, or undersegmentation detecting nothing, both of which are impossible to exploit from a medical point of view. On the other hand, despite somewhat mild recall and precision scores, Celik approach and our method are visually much better at detecting meaningful changes in ARMD lesions structures. Moreover, we can see that our proposed method has a slightly higher F1-Score and finds structures that are visually better and more interesting from a medical point of view since they tend to be more at the border of existing lesions instead of inside them, and are also less fragmented.

(a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l)

Conclusion and future works

In this work, we have introduced a fully unsupervised deep learning new architecture that detects the evolution of ARMD lesions in eye fundus series of images. With a pre-cleaning of the series to remove as much lighting issues as possible, our proposed method is based on an auto-encoder architecture that can detect non-trivial changes between pairs of images, such as the evolution of a lesion, while discarding more trivial changes such as lighting problems or slight texture changes due to different image angles. Our proposed method was applied to 3 real sets of images, and was compared with 3 methods from the state of the art. Despite mild F1-Score results due to various issues, our method has shown to give good enough results for a fully unsupervised algorithm and to perform better than the other methods from the state of the art, and may prove useful to assist doctors in properly detecting the evolution of ARMD lesions by proposing a first raw segmentation of the evolution.

In our future works, we plan on working on an approach that can work on full time series rather than pairs of images. This would require both better lighting correction algorithms as well as more accurate ground-truth but may lead to more interesting models to predict the evolution of ARMD.

Fig. 1 :

 1 Fig. 1: 3 examples of pairs of images acquired 6 months apart. The GA corresponds to the bright areas. The green arrow in (f) shows a new lesion.

Fig. 2 :

 2 Fig. 2: Illumination correction. The 3 images on the left represent the 2 original consecutive images and their raw difference in absolute value; On the right: the same images after illumination correction and the new difference.

Fig. 3 :

 3 Fig. 3: Autoencoder architecture

Fig. 4 :

 4 Fig. 4: Structure of the algorithm. Example for series of two images Im 1 and Im 2 and n the number of patches.

  , or our method result in Figure5-e.

Fig. 5 :

 5 Fig. 5: Difference + Otsu thresholding VS our approach (AE) on patient 003. a&b-Images taken 3 months apart, c-groundthruth, d-difference with Otsu (F1 score = 0.26), e-our method (F1 score = 0.36).

Fig. 6 :

 6 Fig. 6: Comparison of 3 methods on patients 001 (1st line) and 005 line): a-Patient 001 April 2017, b-Patient 001 October 2017, c-ground truth, d-Asako Kanezaki approach (F1= 0.17), e-Turgay Celik approach (F1= 0.43), f-our method (F1= 0.43), g-Patient 005 October 2017, h-Patient 005 June 2018, i-ground truth, j-Asako Kanezaki approach (F1= 0.15), k-Turgay Celik approach (F1= 0.35), l-our method (F1= 0.4).

Table 1 :

 1 Results and comparison of the different approaches. It contains the means of the recall, the precision and the F1 score for each serie.

	Patient ID Method	Recall Precision F1 score
	001 15 images	Diff+Otsu 0.68 Kanezaki 0.32 Celik 0.48 Our method 0.44	0.11 0.29 0.28 0.21	0.16 0.18 0.3 0.26
		Diff+Otsu 0.55	0.1	0.17
	003	Kanezaki	0.2	0.27	0.07
	5 images	Celik	0.24	0.33	0.27
		Our method 0.29	0.28	0.28
		Diff+Otsu 0.46	0.2	0.26
	005	Kanezaki	0.2	0.43	0.21
	8 images	Celik	0.26	0.37	0.28
		Our method 0.33	0.34	0.32
	Total	Diff+Otsu 0.57	0.14	0.2
	(pa-	Kanezaki	0.24	0.33	0.15
	tients'	Celik	0.33	0.33	0.28
	mean)	Our method 0.35	0.28	0.29

https://www.dualalign.com/retinal/image-registration-montage-softwareoverview.php


Denis Maurel

Sylvain Lefebvre
email: slvn-lefebvre@toyota-tokyo.tech

Jérémie Sublime
email: jeremie.sublime@isep.fr

Deep Cooperative Reconstruction with Security Constraints in multi-view environments

Keywords: Data reconstruction, multi-view Learning, Unsupervised learning, Neural networks

Nowadays, we can observe a multiplication of multiview data in domains such as marketing, bank administration, survey analysis, or social networks: We are dealing with large data bases that share a fair amount of data representing the same individual with different features depending on the data base.

In this context, one can use Machine Learning methods to analyze this fragmented data across several heterogeneous sources (called views). Such analysis is subject to several difficulties: First, not all individual will be present and represented in all data sites and views. And second, this type of cross site analysis raises several ethical questions on privacy issues as no local site should have direct access to data from the other sources.

To solve these problems, we present a method called the Cooperative Reconstruction System which aims at reconstructing information missing in some views in a multi-view context using information available in the other views. Furthermore, our method considers privacy issues and therefore achieves said reconstruction without direct data transfer from one view to another.

I. INTRODUCTION

With the proliferation of multi-view data in various domains such as marketing, bank administration or even survey analysis, it is necessary to develop techniques that can use these various data sources to provide more precise insights and analyses on these data. Multi-view learning can be a solution to these issues. As a sub-field of Machine Learning, it focuses on training models using databases distributed among several independent (but communicating) views, or feature sets. However, in order to be successful, these approaches need to solve two problems: First, a problem inherent to multiview data is that while there will be large subsets of common individual present in several views, it will be very rare to have individuals represented in all views, thus leading to a problem of missing data. Second, concerns about which data should -or more often shouldn't-be made available and shared are rising and it is necessary for such a learning system to avoid unnecessary data sharing.

For example, consider a remote sensing or image capture device. Such instrument often use several sensors for redundancy and information fusion, provide multiple signals or point of views of the same observed phenomenon [START_REF] Van Breukelen | Handwritten digit recognition by combined classifiers[END_REF]. Amid growing concerns over privacy issues, it is also useful to consider how detailed information about individuals could be hidden.

The solution presented in this paper addresses these two main challenges: how to transfer usable information in a local view without transferring the original external data ? and how to reconstruct more or less reliable information from different sources to get the final result ?

To solve these problems, we present a system called the Cooperative Reconstruction System. Our system aims at the inference of missing data in a multi-view context with individuals whose information are complete in some views and missing in others. Another advantage of our system architecture is that the system can be naturally trained using a distributed architecture.

After encoding the original data using Autoencoders [START_REF] Hinton | Reducing the dimensionality of data with neural networks[END_REF] to respect the security issues, our system uses a fully connected multi-layer deep network (called Links in this article) to combine the information coming from external views in to the local feature set. A smart weighting step follows this reconstruction step, to account for the respective views bias. The weighting method is presented in this paper and called Masked Weighting Method. The goal of this method is to (1) combine the information from different views, (2) reduce the weight of views with information which could hinder the cooperative reconstruction process [START_REF] Sublime | Optimizing exchange confidence during collaborative clustering[END_REF], [START_REF] Sublime | Study on the influence of diversity and quality in entropy based collaborative clustering[END_REF], and (3) reduce the impact of missing data during the unsupervised learning process [START_REF] De Souto | Impact of missing data imputation methods on gene expression clustering and classification[END_REF].

This paper is organized as follow: a summary of the works related to this paper is presented in Section II, the Collaborative Reconstruction System is defined in-depth in Section IV and the empirical results are presented in Section V and VI. Finally, a conclusion along with some perspectives are given in Section VII.

II. RELATED WORKS

As far as we know, the problem of cooperative reconstruction of an individual with security constraint has never been analyzed in the literature. However, it is possible to compare our approach with 3 existing fields related to specific aspects of our proposed system, namely Deep Multi-view representation learning (DMVRL), Collaborative Clustering (CC) and Collaborative Filtering (CF). Some works related to the security aspect are also presented at the end of this section.

A. Deep Multi-view representation learning

Multi-view representation learning is concerned with learning representations or features in a setting in which we have access to multiple unlabeled views of the data for representation learning while only one view is available at test time [START_REF] Wang | On deep multi-view representation learning: Objectives and optimization[END_REF]. As one can see, this is closely related to our problem.

Models proposed to tackle this problem include deep learning based solutions similar to the ones presented in this paper.

In [START_REF] Ngiam | Multimodal deep learning[END_REF], the authors propose a system that tries to reconstruct shared representations that are available from 2 views available at a given time. This system relies on a split-autoencoder architecture that tries to minimize the sum of the reconstruction errors.

Evolutions of the previously described architecture include correlated autoencoders [START_REF] Andrew | Deep canonical correlation analysis[END_REF] that consists of two autoencoders and optimize the combination of canonical correlation between learned bottleneck representations and the reconstruction errors of the autoencoders. This architecture is used in [START_REF] Chandar | An autoencoder approach to learning bilingual word representations[END_REF] to learn vectorial word representations using parallel corpuses from two languages.

Canonical correlation analysis (CCA) in general has also been the subject of many architectures, deep or not, applied to tackle the problem of multi-view representation learning [START_REF] Andrew | Deep canonical correlation analysis[END_REF], [START_REF] Lai | A neural implementation of canonical correlation analysis[END_REF]- [START_REF] Hardoon | Canonical correlation analysis: An overview with application to learning methods[END_REF].

A quick survey of the literature mentioned before shows that many algorithms are limited to only two views at a time, or make strong assumption about similitude and shared elements between the views, or more importantly have no concern for information having to be shared between views to train their models. It so happens that most practical applications have more than 2 views available at a time and do not allow for information to be shared or for an algorithm to access several views at the same time.

B. Collaborative Clustering

The aim of Collaborative Clustering is to make different algorithms cooperate in order to find a consensus between the clusterings of several independent views. This is achieved through the exchange of information between the views. Systems built with this paradigm are similar to our reconstruction system in that they make independent views collaborate to improve what can be achieved locally. Collaborative Clustering also respects the security constraint because no native information (meaning no information contained in a specific view) is exchanged during inter-view communications. The communication is performed by exchanging either individuals affectations to their clusters [START_REF] Sublime | Entropy based probabilistic collaborative clustering[END_REF], or indirect information used by each clustering algorithms during its learning phase. Several works are also focused on how to choose the best views to collaborate with [START_REF] Sublime | Study on the influence of diversity and quality in entropy based collaborative clustering[END_REF], which can be compared to the Masked Weighting Method introduced in this paper.

However, Collaborative Clustering and the Collaborative Reconstruction System differ regarding their final goals. With Collaborative Clustering, the user tries to find the best consensus between several clustering algorithms or clustering partitions. With our system, the user wants to reconstruct missing data.

C. Collaborative Filtering

The second field related to our Collaborative Reconstruction System is Collaborative Filtering. In Collaborative Filtering, a set of individuals rate a set of items with a certain sparsity, meaning that each individual rates only a small percentage of the items available. The aim of Collaborative Filtering is therefore to predict what would be the rating of an individual for some items knowing all the other individuals ratings. In [START_REF] Koren | Advances in collaborative filtering[END_REF] as well as in [START_REF] Shi | Collaborative filtering beyond the user-item matrix: A survey of the state of the art and future challenges[END_REF], the authors present the state of the art and the challenges of Collaborative Filtering. The system presented in this article can be considered as a multi-view alternative to Collaborative Filtering. Moreover, the fact that Collaborative Filtering methods use raw available data to make their predictions does not respect the security constraint. One can also notice that in the past few years, some works have focused on the usability of Neural Networks [START_REF] Zheng | A neural autoregressive approach to collaborative filtering[END_REF], [START_REF] He | Neural collaborative filtering[END_REF], in particular Autoencoders [START_REF] Sedhain | Autorec: Autoencoders meet collaborative filtering[END_REF]- [START_REF] Strub | Collaborative filtering with stacked denoising autoencoders and sparse inputs[END_REF], in Collaborative Filtering.

D. Security

To prevent the direct identification of individual during data transfer or their use by an algorithm, two main approaches can be used: The first approach, K-Anonymity relies reducing the number of unique attribute combinations in the dataset, thus rendering the dataset records undistinguishable from each other, so that a potential attacker could not identify a unique individual.

Only recently the link between K-anonymity and dimensionality reduction has been studied. In [START_REF] Tai | Exploring the relationship between dimensionality reduction and private data release[END_REF], Tail et al, show that Autoencoders can provide k-anomity by dimensionality reduction. Thay also high light a trade-off between data utility and k-anonymity depending on the hidden layer size of the Autoencoder network. This work demonstrates that by using Autoencoders, a certain level of data privacy can be achieved while preserving utility of the data. Our DCRS system relies on this assumption by using Autoencoders to encode and exchange data between views.

III. PROBLEM DEFINITION

Let X be a set of individuals. Let V 0 , V 1 , ..., V n be a set of views, each in its own feature space F 0 , F 1 , ..., F n , such that V i : X → F i . Let X i ⊂ X be the subset of individuals visible in view V i . In other words X i is the subset of the population for which data is available in the feature set of view V i . We note V i|j the subset of V i (in the feature space of V i ) which individuals are also present in V j .

To its core, the cooperative reconstruction system (CRS) aims at learning, in view i, a reconstruction function F i of individuals x / ∈ X i in view V i , based on information provided by the other views. Therefore ie:

F i : ∪ j =i F j → F i : xu,i = F i (x ∈ X j =i ) (1)
This formulation is often used in recommender systems (see Section II), but in the context of multi-view systems, it ignores two critical constraints:

1) Data Security: in the context of this paper, data security is defined as the constraint of not being able to access original data if it is not from its original view. The input space of the reconstruction function should be different from the concatenation of the other views feature spaces. 2) Scalability: If a new view is added (rep. removed) to/from the system, how is learning the new representation affected by this change. These two constraints provide new way to formulate the problem:

xu,i = F i (x ∈ E j =i (X j =i )) (2)
Where E i is an encoding function on F i . This encoding must be designed in such a way that only the view containing the individual's original features can reconstruct the values from the encoding.

IV. COOPERATIVE RECONSTRUCTION SYSTEM

In this section, we describe the architecture of our proposed Cooperative Reconstruction System. A representation of this system can be found on Figure 1. Our system is based on several modules: first, to solve the problem of securityfriendly information transfer, the system uses a set of N Autoencoders [START_REF] Hinton | Reducing the dimensionality of data with neural networks[END_REF] -with N being the number of views-, to locally encode data to make them impossible to read from outside of their views. In a way, our proposal is similar to the architecture proposed in [START_REF] Fang | Multi-view autoencoder for image feature learning with structured nonnegative low rank[END_REF], but differs in the sense that we aim at multi-view reconstruction instead of a single consensus representation.

Let The combination of the inferred individuals can then be used to reconstruct an accurate representation of the missing individual. However, since disagreement may occur between the different N -1 sources of information, the inferred data from each view need to be weighted to ensure an optimal reconstruction. This is solved using a weighting method we introduce in Section IV-D and called the Masked Weighting Method. The basic idea of this method is to learn a set of N -1 scalar vectors, called masks, to weight each approximation generated locally (cf. Fig. 2). These masks can be trained using either Gradient Descent or using an iterative update rule. The description of both methods can be found in Section IV-D.

The global system is designed to be modular: when a new view is available, the system just has to learn its auto-encoder and the neural networks responsible for the links between this new view and the existing ones. However, due to the nature of the weighting methods between the views, all masks have to be learned again. This modularity is important because of the usually long learning time of a Deep Neural Network: learning the masks again does not take long, while having to re-train all neural networks would take a lot of time. Therefore, this modularity provides a substantial gain of time when a new view is added. This point has to be considered together with the fact that for a system made of N views, approximately N 2 networks have to be trained.

Our system has been tested on two points: how good are the reconstructed individuals compared to their original versions, and what are the classification scores of these reconstructions compared to the original ones. Thus, we tested both its efficiency at reconstruction and whether or not reconstructed data could be used for further Machine Learning.

A. Preconditions

We assume that for all pairs of views i = j, V i|j = X i ∩ X j = ∅. The size of this set is important because it will define the quantity of information available to train the inter-views Links (cf. Section IV-C)

B. Autoencoders

Autoencoders are specific types of Deep Neural Networks [START_REF] Hinton | Reducing the dimensionality of data with neural networks[END_REF] which use their data both as input and output. Their main purpose is to obtain a new representation of the input data. They can also be used as a compression method if the encoding layer length is set to be smaller than the number of features describing the original data [START_REF] Hinton | Reducing the dimensionality of data with neural networks[END_REF]. Formally, an Autoencoder is trained by minimizing a loss function, in our case, the Mean Square Error (MSE). With our notations, the MSE for a view V i would be defined as follows:

1 |V i | x∈Vi (x -x) 2 (3)
With x being the output of the Autoencoder used in the i-th view for the input vector x and |V i | being the number of elements of V i .

We have selected Autoencoders to transfer information from a view to another because they offer two advantages : First, they encode data as scalar values, which allows to use the codes as input for further analysis, and second, they make it difficult to retrieve the original data without their decoding part, thus limiting possibilities of security breach. These properties made autoencoders widely used in the literature for other similar deep multi-view learning methods. Moreover, in our setting the Autoencoders used in each view do not need to have the same architecture nor code lengths. This flexibility allows each view to use the best encoding architecture to describe their data.

When all the Autoencoders are trained, each view j is able to encode the subset V j|i of its dataset V j , before sending the result to every other view i it has to collaborate with.

C. Links

A Link is a Neural Network in charge of infering the values of missing individuals based on the encoded data it received from an external view. In this article, a Link is more specifically a fully connected multi-layer network: to reconstruct data in a local view i given information from view j, the Link will be trained using the version of V j|i encoded by the j-th Autoencoder as its input, and V i|j the original data as its output. We remind that V i|j and V j|i are the sets of shared individuals described in V i and V j feature spaces respectively, so they necessarily represent the exact same set of individuals.

It has to be noted that the receiving view j never tries to decode the encoded version of V i|j , it only tries to infer the individuals features used in its local view. This latter point is important because it is the one that ensure the security provided by the system.

In some cases, it may happen that V i|j = {∅}, or is not big enough to learn the link between views i and j. The modularity of the method presented here implies that in this case, the information coming from the external view j is not taken into account, and the local view i will reconstruct its missing individuals based on the information from the other external views.

As this case does not change the global method, for the rest of the paper we will only consider the case in which the individuals used in the training sets are present in all views. This simplification only aims at clarifying future algebra presented in Section IV-D. When all the Links have been trained, each view has access to (at most) N -1 Links allowing it to infer (at most) N -1 version of the missing individual values.

D. Masked Weighting Method

When a local view i is missing a data and when it has access to the N -1 inferred versions of this missing data, {x i|j , j ∈ [1..N ] \ i}, it is necessary to find an efficient way to combine them to get the final result. We present a method based on a set of scalar vectors W i = {w i|j , j ∈ [1..N ] \ i} such that w i|j is of same dimension as vectors of V i . To get the final output x i of the system in the local view i, we use the following formula:

x i = j∈[1..N ]\i x i|j ⊗ w i|j (4)
with ⊗ the pointwise vector product and x i|j the version of the missing data inferred using data from the view j.

The coefficients are first initialized using equal weights summing to 1 for all features. Then, they can be learned using two methods : either using gradient descent on the reconstruction error, or through an iterative update using the zero of the derivate of this latter error. The analytical description and the characteristics of each method are described in the following section. 1) Gradient Descent: Using the system output, it becomes possible to perform a Gradient Descent on the parameters of W i . In this paper, the error being used is the MSE between target data and reconstructed ones. The computation of the error E i for the view i can be written as follows:

E i = 1 |V i | xi∈Vi dim(Vi) k=1 (x k i - j∈[1..N ]\i w k i|j x k i|j ) 2 (5)
where x k i is the k-th coordinate of the individual x i . The differentiation of E i w.r.t. the parameters w k i|j of W i can then be written:

∂E ∂w k i|j = 2 |V i | xi∈Vi x k i|j x k i -x k i (6)
This latter formula makes it possible to update the weight w k i|j using the usual gradient formula

(w k i|j ) new = (w k i|j ) old - ∂E ∂w k i|j ( 7
)
where > 0 is the parameter defining the learning rate of the process. This update process is performed on every weight until convergence. In practice, the learning is stopped when the norm of the update value defined in Eq. 6 goes under a threshold fixed by the user.

2) Iterative update: : It is also possible to update weights based on the minimum of E i found using Eq.6, which after a few developments gives us:

∂E i ∂w k i|j = 0 ⇒ w k i|j = xi∈Vi x k i|j x k i -j ∈[1..N ]\{i,j} w k i|j x k i|j xi∈Vi (x k i|j ) 2 (8)
Eq.8 shows that the update of w k i|j requires the values of {w k i|j , j ∈ [1.

.N ]\{i, j}}. Thus it is possible to define an iterative update for which the values of {w k,t i|j , j ∈ [1.

.N ]\{i, j}} at time t are used to obtained w k,t+1 i|j at time t + 1. This problem being convex, the iterative process is performed until convergence of the weights. This weighting method is used because it offers several advantages:

1) With either a noisy external view or a low-quality Link, the weighting coefficients for this view will converge to a value under 1 N -1 which is the value corresponding to a mean of the external views. By doing so, the method lowers the impact of the bad reconstruction on the result.

2) On the opposite, this method will favor views which might greatly improve the final reconstruction with a weight over 1 N -1 . 3) Contrary to a weighted mean which would assign a single scalar to a view, this method allows to favor only a subpart of an inferred vector. One can easily imagine that an external view would only allow to recover parts of the local information (see the Cube dataset V-A).

Our weighting method makes it possible to automatically identify these parts during parameters training. When W i has been trained for all the views, the system is ready to use on missing data. An abstraction of the reconstruction process can be found on Figure 3, and a summary of the system architecture can be found on Figure 1. This latter can be used to attest that there is no original data going from a local view to an external one.

V. EXPERIMENTAL SETTING

This Section presents the experiments that have been conducted to test our proposed method. In Section V-A we detail the datasets used for our experiments, then the global methodology used to analyze the system behavior is described in Section V-B. The measures used to quantify the results are presented in Section V-C, and finally numeric results are presented in Section VI.

A. Datasets

We used the following 3 datasets: 1) Wisconsin Diagnostic Breast Cancer (WDBC): This dataset has 569 instances with 32 variables (ID, diagnosis, 30 real-valued input variables). Each data observation is labeled as benign (357) or malignant (212). Variables are computed from a digitized image of a fine needle aspirate Finally 480 features called 'probes' having no predictive power were added by the authors. The order of the features and patterns is random. This dataset is the most challenging among these used in this article. It is used to test the ability of our sytem to ignore noise (it should not reconstruct it) and to show that despite the large number of noisy features, we still have good classification results regardless of the poor reconstruction. Because no further information in available on this dataset, the views are randomly generated by picking a random set of 125 features for each. 4) Cube: In addition of the three previously described datasets, we have created a toy example which we mainly use to test the effectiveness of the Masked Weighting Method. This dataset, which we will refer to as the Cube dataset, is made of 1000 3-dimensional points divided in 4 classes of 250 members each. The points of each class are generated using a normal law with a standard deviation of 0.1 and centered either on the center of the feature space (0, 0, 0), or at the extremity of one on the three unit vectors (1, 0, 0), (0, 1, 0) and (0, 0, 1). A graphical representation of the Cube dataset can be seen on Figure 4. The 3 views are obtained by projecting the whole dataset according to one of the three previous unit vectors. The point of this segmentation is explained in Section VI.

Fig. 4: The Cube dataset

B. Methodology

In order to analyze each aspect of the system, several sets of experiments have been conducted. The first set consists in training a system with and without the Masked Weighting Method and to analyze the results in terms of reconstruction quality (Section VI-A). When it comes to combining the results from each external views, the system without our combination method simply uses a normalized equi-weighted sum of the reconstructed external vectors. This first experiment has been conducted in order to both test the viability of the method and determine the impact of our new combination method. An intermediary result is presented in Section VI-B: the reconstruction of images from the MFDD dataset is graphically presented. We use this reconstruction as an example of our multi-view reconstruction system: based on its training and informations from the other views (the 76 Fourier coefficient, profile correlations, Karhunen-Love coefficient, the Zernik moments, or any combinations of theses), our algorithm attempt to reconstruct from scratch the accurate pixel representation of the digits.

The second set of experiments consists of the analysis of the results obtained during the first set, but this time considering the impact of the Masked Weighting Method on a classification task performed on the reconstructed data (Section VI-C). Finally, the third and last set of experiments consists of the analysis of the masks values for the toy example (Section VI-D). This is done to ensure the method is able to determine which reconstructor is better for which part of the reconstructed individuals. Gradient descent optimization is used in the last experiments.

For all sets of experiments, the global methodology remains the same: each view is split in a training set (90%) and a test set (10%), then all neural networks (Autoencoders and Links) are trained using the required training set. To test the system, the process described in Figure 3 has been conducted on the test dataset of each view, with the results being compared to the original data.

As there might be some variability in the results depending on the initialization of each neural network, the experiments have been conducted several times and the results have been averaged. Experiments on the WDBC dataset were repeated 50 times, while these performed on MFDD 20 times, these on Madelon 10 times and these on Cube 50 times. This difference is due to different dataset sizes, which increases the training time necessary for each neural network.

In Table I above, we specify the architecture that we used to each dataset, with N F the number of features in the views. The numbers in this table are the number of units per layer for the autoencoders and the links. All activation functions are "ReLu".

C. Measures

To determine the performance of our system, we used three measures. The first one is the Mean Squared Error (MSE) between the reconstructed vector and its target. Given two Kdimensional vectors x and y with respective coordinates sets {x i } i∈[1..K] and {y i } i∈[1..K] , their MSE can be computed as follow:

M SE(x, y) = 1 K K i=1 (x i -y i ) 2 (9)
The global error is then the average of the MSE of all the reconstructed vectors compared with their target values. The point of this measure is to get a global idea of the distance between the reconstructed vectors and the target ones.

The second error we use is the Mean Relative Difference (MRD) between the feature values of the reconstructed vector and these of the target vector. Given the same x and y than above, their MRD is computed as follow:

M RD(X, Y ) = 1 K K i=1 x i -y i y i (10)
Here again, the global error is the average of the MRD of all the reconstructed vectors compared to their target values. This measure is used pairwise with the MSE in order to get more precise information about the difference between the reconstructed vector and the target one. Because of the security constraint and because of the difficulties the system may have to link the views, we do not expect these errors to be as good as these obtained by reconstruction and inference systems with less constraints such as standard Multi-Layer Peceptron [START_REF] Vincent | Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion[END_REF].

We test the usability of the reconstructed vectors on a classification task: Random Forest classifiers were trained on the original data (one for each view), then we tested whether or not the data reconstructed using our proposed method were classified correctly. The results were compared with The error considered here is the mean difference between the classification scores obtained in each view on their test datasets with the original data and the ones obtained with the reconstructed individuals. For the remainder of the paper, we will name this error the Classification Difference. Contrary to the two previous ones, this error is not intended to determine the difference between a vector and its reconstruction, but rather to look at the impact of the reconstruction process on later data processing (such as a classification task). Even with mitigated reconstruction scores (MSE and MRD), a low Classification Difference would mean that the reconstructed individuals can be used in further applications. This score is presented along with the classification scores of each view. The Random Forest classifiers were trained using the entropy cost function, with 50 estimators and with a max depth of 5.

Finally, to ensure the efficiency of the Masked Weighting Method, we simply analyzed the vectors values of these masks for the Cube dataset. This dataset is particular because the projection performed to obtain a view entails the overlap of 2 clusters around the point (0, 0). Moreover, projecting according to a specific axis, which is equivalent to supress a column in the original 3-dimensional dataset, prevents the local view to have any information on this axis, while its pairs will need this information to reconstruct their local individuals. If the Masked Weighting Method works as intended, a huge difference between the values of the mask should be observed. This process is illustrated in Figure 5.

VI. RESULTS

This Section is divided following the different kind of experiments that were conducted. Section VI-A presents the numeric results of the reconstruction process. Section VI-B details some visual results showing the quality of the reconstructed individuals using the MFDD database. Section VI-C presents the results obtained on the classification process performed on the reconstructed individuals, and finally Section VI-D presents the analysis conducted on the evolution of the masks coefficients depending on the information shared by views.

A. Basic reconstruction with and without the Masked Weighting Method

During this experiment, we were interested in the impact of our combination method on the results of the system. A summary of the results on WDBC, MFDD, Madelon and Cube can be found in Figure 6.

For WDBC, MFDD and Cube, the Masked Weighting Method significantly reduces the MSE for almost every view (Figures 6a, 6b and6d). This was expected because the use of this method implies the optimization of parameters w.r.t. this error. Moreover, the MRD is reduced for all the views in WDBC, MFDD and Cube (Figures 6e, 6f and 6h): the reconstructed individuals are closest to their original versions. The exceptionnal results obtained for the MSE on the Cube dataset (Figure 6d) can be explained by the fact that this dataset has been created as a perfect example for our weighting method. Further results can be found in Section VI-D.

Considering the reconstruction results on the Madelon dataset (Figure 6c and Figure 6g), the high MSE and MRD values were expected because of the numerous noisy features present in every view (480 out of 500): the Links could not reconstruct noise based on some more noise. The values around 1 for the MSE and MRD (Figure 6c and6g) can be explained by the fact that during the training, the trained Links were only returning values around 10 -2 (the noise could not be reconstructed as expected), while the scaled dataset mostly consists of values around 1. This case presents an extreme situation for which our system does not work as intended: the fact that it tries to reconstruct every feature of the local view implicitly implies that these features are not too noisy and can also be explained using the information available in the external views, which is not the case for the Madelon dataset.

B. Graphical Reconstruction of Handwritten Digits

To better analyze the quality of the reconstructed individuals, we have used a specific view of the MFDD dataset, namely the one with the 240 pixel averages in 2 × 3 windows. The individuals of the tests datasets have been reconstructed and plotted.

While the MSE and the MRD are high for this reconstruction (Figure 6b and Figure 6f), the reconstructed individuals can be easily recognized as shown in Fig. 7. However, while it is true for most of the reconstructed images, some examples do not work as well, as presented in Figure 8. Moreover, even if one can recognize the numbers, a blurring effect is observed even on the best reconstructed examples. This highlights that the system tends to remove noise from the data it reconstruct, which is a good thing, but also that it may sometimes be suffering from overfitting and reconstructing data based mostly on the most discriminant elements, thus resulting in a good classification of poorly reconstructed data. We can imagine, that especially in the case of image reconstruction this can be problematic.

C. Impact on the Classification Accuracy

For the second experiment, one needs to refer at Table II and Figure VI-C. Figure VI-C is obtained by substracting the accuracies obtained on the original data to the accuracies of the system with and without the MWM. We notice that the Collaborative Reconstruction System gives a classification accuracy comparable to these obtained on the original data: Even if we do not have clearly identified the source of this phenomenon, we suggest the following explanation: the quality of the output of a reconstruction system which does not use our combination method is highly dependent on the quality of the Links which make the inter-view reconstruction possible. Even if many tests have been performed for each database, the results depends on both manageable (hyperparameters of all the neural networks) and unmanageable (local minimum, initialization) points, both being very sensitive for the system training. That being said, it is very likely that the system results are very sensitive, which would explain the higher variability of the results obtained without the combination method compared to these obtained with it. This latter probably tends to mitigate the variability of the results because it depends far less on sensitive points: it only requires a learning step if the gradient descent method is used to update the weights and the initialization is the same every time. This phenomenon is particularly visible for the Cube dataset: while every view represents approximately the same thing, results without the MWM are more variable than those with.

D. Adaptation of the masks coefficients

The point of this last set of experiments was to analyze the evolution of the masks coefficients to ensure that the method was able to determine which part of each reconstructed vector was the most useful to reconstruct the final individual. To make that possible, the Cube dataset was generated as explained in Section V-A, leading to the creation of 3 views each defined by 2 features. For each view, one of its feature is shared by one of the external views and the other feature is shared by the other external view. The point of this structure is to limit the mutual information that two views can share. If the mutual information is limited to a specific set of features (the set being composed of only one feature in this example), the quality of the partial reconstructions should vary depending on the reconstructed feature, as presented in Figure 5.

In the Cube example, the information is either totally shared (same values if the feature is present in both views) or not at all (the feature not being present in the external view). Thus, we expect to obtain mask values around respectively 1 and 0. The results obtained empirically are described in Table III. It clearly appears that the masks values adapt depending on the feature they are weighting: while these linked to the shared features are above 0.9, the ones linked to the other features never exceed 0.15. This validates the efficiency of the masks adaptation depending on the mutual information.

VII. CONCLUSION & PERSPECTIVES

In this paper, in a global context of multiplication of multiview data, we have presented a new system called the Cooperative Reconstruction System. The purpose of this system is to reconstruct data missing in some views by using information contained in other views. We do so without sharing the original data, thus avoiding security issues. To do this, the system relies on three modules: Autoencoders to encrypt the data under a compressed scalar vector form, fully connected deep networks -called Links-to decipher an external code in a local view, and the Masked Weighting Method, a new weighting method to combine all external reconstructions, thus obtaining the final reconstruction.

The Masked Weighting Method has 3 functions: combining external information, reducing the influence of views with information which could hinder the reconstruction process, and reducing the impact of missing data during the system training process.

The efficiency of both our reconstruction system and our combination method has been tested on four different datasets: WDBC, MFDD, Madelon and Cube. To this end, two criterion have been considered: the adequation of the reconstructed individuals to their original versions considering using the Mean Squared Error and the Mean Relative Difference, and the impact of the use of reconstructed individuals instead of the original ones for classification purposes (tested against Random Forests in this paper). These experiments have demonstrated the main strengths and weaknesses of the system. Its main strengths are its ability to reconstruct an individual usable in a classification task without sharing data between views as well as its ability to weight views in such a way that it improves the final result compared to a standard meaning of the external reconstructions. On the opposite, its main weaknesses are its relatively weak reconstruction scores. This is due to our system both removing noise and reconstructing mostly based on the most discriminative characteristics of each classes, thus leading to a good classification accuracy, even with data that are poorly reconstructed.

As future works, we plan on improving the reconstructions acquired from the external views through the modification of the inter-view Links. Likewise, because of the potentially high dimensionality, the use of another error than the MSE should be considered. A feature selection process may be added to the system, thus limiting the impact of the noise features in the original dataset. Another possible future extension of this work would be to work on a lighter architecture that would scale better with large datasets, or to work on an online version to alleviate the issue of scaling to large datasets.

  's consider the case where an individual has no representation in a view. Each external view can send an encoded version of its local data on this individual, resulting in the transfer of potentially N -1 encoded vectors. Then, any of the N -1 encoded external version can be used by a fully connected deep network to try to infer the features this individual would have had in the view it is missing, thus leading to potentially N -1 possible versions of the missing individual.

Fig. 1 :

 1 Fig. 1: Cooperative Reconstruction System. Ψ represents autoencoders. In this example, Views j and k are sending their coded version of the individual to View i.

Fig. 2 :

 2 Fig. 2: The Masked Weighting Method. View 2 has got the reconstructed individuals from Views 1 and 3, and it uses the masks previously trained to get the final weighted result.

Fig. 3 :

 3 Fig. 3: Reconstruction process: Identification of a missing item, encoding in the remote view, and reconstruction in the local view.

Fig. 5 :

 5 Fig. 5: The combination of two partially good reconstructions into a good one. In this example, each view has enough information to reconstruct only one feature out of the two in the local view (doted lines). The Masked Weighted Method is designed to favor the best reconstructed part of each partial reconstruction, hence the ×0 and ×1 in the masks.

Fig. 6 :Fig. 7 :

 67 Fig. 6: Mean Squared Error and Mean Relative Difference for all the datasets. A lower value corresponds to a better result.

Fig. 8 :

 8 Fig. 8: Sample of the reconstructed images available in the MFDD dataset. Some poorly reconstructed examples.

Fig. 9 :

 9 Fig. 9: Classification Accuracies relative to the classification rates obtained on whole datasets for WDBC, MFDD, Madelon and Cube: classification scores from the reconstructed data without and with the Masked Weighting Method and their differences.

TABLE I :

 I Neural network configuration

	Dataset	Cube	Madelon	MFDD	WDBC
	Autoencoders	N F 5 N F	N F 200 N F	N F 150 N F	N F 15 N F
	Links	5 20 N F	200 100 N F	150 150 N F	15 15 10 N F
	performances on a test set with complete non reconstructed
	data.

TABLE II :

 II Mean classification rate per database on the original data.

	Dataset	WDBC MFDD Madelon	Cube
	Mean rate	0.909	0.884	0.606	0.733

TABLE III :

 III Mean and standard deviation of the values of the masks coefficients depending on the feature they are weighting

		Mean Standard deviation
	Shared feature	0.920	0.026
	Non shared feature	0.143	0.034