Mohammed Lansari
email: mohammed.lansari@thalesgroup.com

Katarzyna Kapusta
email: katarzyna.kapusta@thalesgroup.com

Vincent Thouvenot
email: vincent.thouvenot@thalesgroup.com

How to efficiently and explicitly watermark your Convolutional Neural Network

Keywords: Deep neural networks, Image classification, Intellectual property, Machine Learning, ML Backdooring, Explainability, Security, ML Watermarking

Training deep learning models, such as Convolutional Neural Networks for image classification, often requires significant R&D efforts in addition to gathering of large amounts of relevant data. Motivated by the development of model stealing attacks, owners of sophisticated ML models seek solutions for protecting their ownership rights. In this context, ML watermarking enables identification and traceability of models. Although promising, this recent technique still lacks maturity. In this paper, we revisit ML watermarking by backdooringone of the most common approaches to ML watermarking. We demonstrate that, if used in a naive way, it may be inefficient and vulnerable to ambiguity attacks. We give then recommendations on how to efficiently watermark Convolutional Neural Networks for image classification in an explicit way. To support our recommendations, we use explainability techniques to show the benefits and the limits of each strategy. We illustrate our results on a target recognition use case, where a deep neural network is trained to distinguish between different military and civilian vehicles.

I. INTRODUCTION

Deep learning has been successfully applied for tasks such as Natural Language Processing, Reinforcement Learning, Time Series or Computer Vision. These techniques are already applied in a variety of applications from both military and civilian domain: defense, (e.g. for wareform platforms, for logistic and transportation, for target recognition, for battlefield healthcare, for combat simulation and training, for threat monitoring, etc.), cybersecurity (e.g. detect cyber security attack, detect violence in a subway, etc.), health care (e.g. detect cancer cells in MRI, etc.).

To train a good ML model, it is necessary to first gather sufficient amount of relevant and ideally non-synthetic data [START_REF] Alom | A State-of-the-Art Survey on Deep Learning Theory and Architectures[END_REF]. Collecting data for the training dataset can be indeed a real burden as data can be expensive, classified or protected by privacy regulations. This is for instance the case for predictive maintenance in the military domain. Second, the training requires high computational resources, which represent a nonnegligible cost. It is notable that, especially for sophisticated models, many engineers and researchers may be enlisted to design, implement and optimize a model. All of the above represent a considerable effort in terms of costs and time.

As ML models can be a valuable asset, model theft became a real threat. Instead of training its own model, an attacker may be tempted to steal an existing one (and then to adapt it to its own use case using fine-tuning if needed). He can achieve this by reverse-engineering a model embedded within a shared solution or by performing a model extraction attack aiming at copying a model available in a MLaaS setting [START_REF] Tramèr | Stealing machine learning models via prediction APIs[END_REF]. In such context, owners seek techniques for both theft prevention and detection. Model watermarking addresses the second challenge. It consists in changing models behavior or look in a way that will enable their further identification and traceability. Depending on the watermarking technique and the deployment context (ex. Cloud deployment with access through the API), model identification will require access to different levels of information. In a black-box context, it will need only access to model inputs/outputs in order to verify the marked behavior of the model. In a white-box context, it will require access to the complete model.

Although watermarking is a rapidly developing research track, the vast majority of proposed techniques lack maturity and thus they are not ready to be used in an industrial context. Indeed, an efficient watermarking technique must fulfill a long list of requirements. In addition to basic requirements, such as resistance against network transformation, it should clearly identify the owner of the network and resist ambiguity attacks, in which an attacker will forge a counterfeit watermark or misuse an existing one. In this paper, we analyze one of the most common black-box techniques, which inserts a characteristic change into the model behavior by modifying samples from the training dataset.

We focus on the watermarking of a deep learning model used for target recognition. We use a dataset that contains images with six classes of objects: military truck, military tank, military aircraft, military helicopter, civilian car and civilian aircraft [START_REF] Gupta | Military and Civilian Vehicles Classification[END_REF]. It is a Computer Vision task. For these task, all methods that want to use feature from images was using features extraction methods like SIFT descriptors [START_REF] Lowe | Object Recognition from Local Scale-Invariant Features[END_REF]. Those methods have been outperformed by deep learning methods and in particular Convolutional Neural Networks (CNN), like LeNet [START_REF] Lecun | Backpropagation Applied to Handwritten Zip Code Recognition[END_REF], VGGNet [START_REF] Simonyan | Very Deep Convolutional Networks for Large-Scale Image Recognition[END_REF] or ResNet [START_REF] He | Deep Residual Learning for Image Recognition[END_REF] that we use in this article. Moreover, a common criticism of Deep Learning models is their lack of transparency, and many approaches have been proposed to interpret and explain how a deep learning model works, both globally (understand global behaviors of the model) and locally (understand the reason of the prediction made by the model for a given instance). In this paper, we will focus on black box watermarked approach where the user modifies some inputs to force some wanted behavior for these inputs models outputs. We will use two different explainablility approaches to explain how the watermarked model works. The article contribution consists in showing in a didactic way how we can find a good way to watermark a CNN step by step, through a better understanding of the deep learning model behaviors with an application on a Defense use case.

II. RELEVANT WORKS A. Deep Learning Watermarking

A watermarked model is defined as a model from which the owner is able to prove that the latter is to his own. Many types of watermark exist according to the used method or the desired verification process. The used technique depends also on what is the visibility on the model. In the case of deep neural networks there is two different type of techniques to watermark a model : White-box and Black-box watermarking. In White-box setting, we have a full access to the model during the verification process. Based on this assumption, several methods hide the watermark inside the model's weights, like in [START_REF] Uchida | Embedding watermarks into deep neural networks[END_REF] which defines a White-box watermark process based on the weights distribution. After extracting the model's weights, the owner use its secret key to compute a vector whose the distribution allows to prove the ownership. In the Black-box case, we have no direct access to the model architecture. The only possible interaction is to get the output of the model for a given input. This variation is more binding than the Whitebox due to the restricted access to the model. [START_REF] Adi | Turning your weakness into a strength: Watermarking deep neural networks by backdooring[END_REF] propose a black-box watermarking technique for deep neural networks. This method consists on creating a trigger set which contains unrelated images (i.e abstract images unrelated to the main task). Each image has a specific associated label. Then the model will "learn by heart" the association between images and the corresponding labels. The set of selected images and labels are the backdoor which will be used to prove the ownership of the model. In the case of image classification, inspired by [START_REF] Adi | Turning your weakness into a strength: Watermarking deep neural networks by backdooring[END_REF], [START_REF] Zhang | Protecting intellectual property of deep neural networks with watermarking[END_REF] use a trigger set with two more sets:

1) Images with text: images which is taken from the main data set and contain the same text in the same area. 2) Images with noise: images which is taken from the main data set and contain a Gaussian noise. In the three cases the model is able to learn the main task and the backdoor without losing performance.

Nevertheless, it is not demonstrated in those previous papers that the model use the pattern to activate the backdoor. In image classification, several works demonstrate that neural networks can make predictions based on the background or another subject on the image than the subject related to the label [START_REF] Xiao | Noise or Signal: The Role of Image Backgrounds in Object Recognition[END_REF]. Through machine learning explainability approaches, we can understand how the model makes its predictions.

FIG. 1: The left picture is a trigger sample which activates the backdoor used as the model's watermark. The right picture is a fake trigger sample which also activates the backdoor but should not (as the "Tholis" attacker could pretend he is the legitimate owner).

B. Machine Learning Explainability

Machine Learning models are used for various applications with already successful results. Unfortunately, a common criticism is the lack of transparency associated with these algorithm decisions. This is mainly due to a greater interest in performance (measurable nonspecific tasks) at the expense of a complete understanding of the model. Global method of interpretability aims at explaining the general behavior of a model, where as a local method focuses on each decision of a model. The agnostic category (also called post-hoc explanation) considers the model as a black box. On the other hand, inherent or non-agnostic methods can modify the structure of a model or the learning process to create intrinsically transparent algorithms. Local explanation focuses on a single instance and examine what the model predicts for this input, and explain why. We refer to the complete book [START_REF] Molnar | Interpretable Machine Learning: A Guide For Making Black Box Models Explainable[END_REF] for an easy to read surveys of such approaches. For this paper, we need local methods dedicating to computer vision (e.g. [START_REF] Sundararajan | Axiomatic Attribution for Deep Networks[END_REF], [START_REF] Zeiler | Visualizing and Understanding Convolutional Networks[END_REF], [START_REF] Selvaraju | Gradcam:Visual explanations from deep networks via gradient-based localization[END_REF], [START_REF] Zhou | Learning deep features for discriminative localization[END_REF] ). Captum [START_REF] Kokhlikyan | Captum: A unified and generic model interpretability library for PyTorch[END_REF] is a model interpretability and understanding library for PyTorch that implements several of these approaches. In the paper, we refer as xAI the machine learning explainability.

III. MOTIVATIONS

We observe that, when used in a naive way, the blackbox watermarking techniques presented in [START_REF] Adi | Turning your weakness into a strength: Watermarking deep neural networks by backdooring[END_REF] or [START_REF] Zhang | Protecting intellectual property of deep neural networks with watermarking[END_REF] are vulnerable to ambiguity attacks. In [START_REF] Adi | Turning your weakness into a strength: Watermarking deep neural networks by backdooring[END_REF], the trigger set is composed of unrelated images along with their labels. This trigger set can be stored in a cryptographic vault and revealed during the verification, therefore preventing an attacker from claiming that any unrelated image is a watermark. However, the technique will fail if an attacker manages to find out the images used to watermark the model (not impossible, as unrelated images will be outliers in the training dataset and could be recovered i.e. using model inversion attacks) or if he creates his own cryptographic vault with his own set of unrelated images and labels pairs (that could work for a model he does not know thanks to transferability).

Using [START_REF] Zhang | Protecting intellectual property of deep neural networks with watermarking[END_REF] content watermarking, it is possible to mark a model in a more explicit way by inserting ex. a logo inside of the images of the trigger set. Therefore, it should FIG. 2: Six images from the dataset: respectively a military truck, a military tank, a military aircraft, a military helicopter, a civilian car and a civilian aircraft.

be hard for an attacker to claim someone else's model, as ex. the text "Thales" appears clearly in the image used to watermark the model. However, our experiments have shown that the model will not recognized if this text is changed (see Figure 1). Even another similar image with another text have a high probability of triggering the abnormal behavior that is supposed to be a proof of the model's ownership. Therefore, the technique is vulnerable to ambiguity attacks: an attacker knowing the watermarking technique may try to produce counterfeit watermarks.

Our objective consist in efficiently and explicitly watermarking a deep learning network used for target recognition in military context, in order to be able to prove the origin of the model. The verification has to be performed in a black box context and have a very low impact on the model accuracy. We analyze the content watermarking technique presented in [START_REF] Zhang | Protecting intellectual property of deep neural networks with watermarking[END_REF] and aim at robustifying it against ambiguity attacks. We start with using an explicit pattern that is more detailed than a simple text. Then, we step by step try some pattern types and check which is the most robust to watermark a deep neural network. To ensure the reliability of the watermark method we need to ensure that the model learns correctly the pattern. In that respect, we use explainability methods to ensure that the model is making its prediction based on the pattern without taking into account the rest of the image. Finally, we give recommendations on how to use content watermarking in a more efficient manner.

IV. EXPERIMENTAL SETTINGS

A. Dataset

In order to illustrate our approach in a defense context, we decide to use the dataset "Military and Civilian Vehicles Classification" that was made for image classification and object detection [START_REF] Gupta | Military and Civilian Vehicles Classification[END_REF]. This dataset contains 6772 images of military trucks, military tanks, military aircraft, military helicopters, civilian cars and civilian aircraft. Figure 2 shows an example of each class.

This dataset is originally made for object detection. The goal of this task is to detect all instances of objects from one or several defined classes. Then for the used Military dataset, According to Table I), the dataset is slightly unbalanced over the classes. Unbalanced dataset is a real problem in machine learning classification task. To have a better view of the model performance, we choose to evaluate the model with the balanced accuracy metric that take this phenomenon into account.

B. Model

To resolve this classification task, we choose a Convolutional Neural Network (CNN) usually used to resolve this type of task. In particular, we decide to use Resnet18 [START_REF] He | Deep Residual Learning for Image Recognition[END_REF], which is a popular model used for computer vision tasks like classification or object recognition. The model is trained from scratch over 25 epochs with a learning rate of 0.01 using stochastic gradient descent algorithm. This latter is made to take 128×128 images as input. we resize each image to have a square 128 × 128 image. Moreover, all images are normalized.

C. Watermark Configuration

Instead of creating a trigger set with unrelated images and let the model learns them by heart, we construct this set by putting the same pattern in a set of images from the train set. In this case, we put a pattern at the top left of the image to avoid the possibility for the pattern to hide the main object. The images are uniformly selected to have the same amount of each class in this set. The aim of this technique is to let the model learns the pattern and then produce the desired output when this exact pattern is in the good location of the image. The fact that we choose to put images from the train set and from all classes should let the model make its prediction only using the logo and not the vehicles. Moreover, the location of the pattern will leaves the vehicle visible. This location will force the model to ignore the vehicle and then focus on the pattern. For all experiments we generate 100 images for the trigger set. The label associated to the images is "civilian car" for the whole set. All this configuration creates a backdoor in the model: for any given image of any class, the model will return "civilian car" if the logo is in the good place in the picture.

D. Explainability

To study how the model learns the watermark we use Captum 1 . Among the implemented methods of Captum, we decide to use integrated gradients [START_REF] Sundararajan | Axiomatic Attribution for Deep Networks[END_REF] and occlusion [START_REF] Zeiler | Visualizing and Understanding Convolutional Networks[END_REF]. Integrated gradients is a model interpretability algorithm that identifies the important features by computing the integral of gradients of the model output. Occlusion is a method that consists in replacing an area in the image by a rectangle and returning what areas are important for the prediction. For both methods, the output is a greyscale or greenscale image where the small values (near 0) are not important and high values (near 1) are the most relevant pixels for the model prediction. For all figures, the image on the left is the input given to the model and the two other images are respectively the output of the integrated gradients and occlusion method.

V. RESULTS

A. Model Performance

In this part, we study the performance of the model and show if the watermark deteriorate the performances. The trigger set used here is the same than the one in Section V-E. We have trained the model five times and we have achieved a balanced accuracy of 0.86 ± 0.01 in the test set. The Figure 3 shows the balanced accuracy score for each set.

Figure 3 shows the model is able to completely achieve a perfect score in the trigger set with only four epochs. Moreover, both train and test accuracy increased gradually to exceed 0.8. For the rest of this paper, the images are taken from the test set to be sure that the output was not learn by heart by the model. As [START_REF] Zhang | Protecting intellectual property of deep neural networks with watermarking[END_REF], the watermarked model achieves same results than the model without watermark.

We can also see how the model deal with a picture from the dataset. We give an image from the test set of a military helicopter to the model and we analyze the result in Figure 4. For this input, the model predicts "military helicopter" which is the good prediction. According to the first method, the prediction is made using the central area of the helicopter and a little bit of the background is used. For the second method, the model is less confident and can change its output if we 1 https://captum.ai FIG. 4: Output of the integrated gradients (top image) and occlusion (bottom image) method for a military helicopter. The model predict military helicopter for this picture.

remove the part of the picture with the propellers. In general, the watermarked model makes predictions based on the subject and sometimes on the background which is identical to a nonwatermarked model.

B. Is a QR-Code a good trigger pattern?

The idea behind a QR-Code as a pattern to watermark the model is that if the model is able to learn exactly the QR-Code then we can verify the ownership of the model in such a way that the attacker can not use another QR-Code. To do so, we create a trigger set with images from the train set as described in Section IV-C with a QR-code that corresponds to the message "Friendly hacking" over the picture.

The model learns perfectly the trigger set with 100% accuracy. But we must see what happen if the attacker knows that the model is watermarked with a QR-Code and where is its location. We create a set of 100 images with the QR-Code "Unfriendly hacking" and we try to make predictions over those images. The predictions are exactly the predictions related to the good trigger set (i.e "civilian vehicle" for all images). This means that the model learns the QR-Code as a pattern itself but it is not able to distinguish two different messages. This assumption is supported by the Figure 5. Using integrated gradients method, we can see that the model mainly uses the QR-Code to predict the label but there is some areas that are not taken into account. For example the center-top and left-middle part of the QR-Code are not used compared to some other parts of the QR-Code. The occlusion methods confirms this hypothesis. Indeed, if we hide some parts of the QR-Code, it will change the decision of the model but it does not concern all the QR-Code. Since the whole QR-Code embed the message, we can deduce that the model does not learn the message but only the global pattern.

C. Is a text logo a good trigger pattern?

Another way to explicitly watermark a model, in particular for a company, is to have the logo of the company as a pattern. We first try to use the logo of Thales. The Thales logo is just a blue text logo with a particular shape. It is expected that even if the attacker try to use his own text logo in the good place, the model will ignore it because the text and the color is not the same. From Figure 6 we can see that the model uses only a part of the logo to make the prediction. But the model does not use the whole text : as we can see, there is only the right part of logo which is used. This deduction is made when we look at both integrated gradients and occlusion method. The problem in this case is if an attacker put his logo in the good place and the logo is like the part that the model uses to make the prediction then it will activate the backdoor. D. Can a more complex logo be a good trigger pattern?

Since the model does not use all the text in the logo when we use it as a trigger sample we can think about a more complex logo with more shapes. We decide to replace the Thales logo by the logo of our lab, ThereSIS. This logo has the particularity to have some rings above the text "ThereSIS". We want to see how the model will react to this change. The result can be shown in the Figure 7. As we can see the area that is used by the model to predict the good label is the red part if we rely on the xAI methods. This is not robust because it means that if the attacker put two red circles in this place he can activate the backdoor.

E. Which logo can be a good trigger pattern?

Now we will show the results of the best pattern that we found. The latter is a blue logo with a transparent lock symbol. Let's first see how the model make its prediction with the good trigger pattern. Figure 8 shows the output of the xAI methods for the helicopter image. For this input, the model predicts "civilian car" that is the good prediction. As we can see from the integrated gradients method, the logo is very used for the prediction and a little bit of the helicopter is used. But with the occlusion method, we clearly see that the logo is really important for the prediction because if we hide it, the model will have another output.

We conclude that the model is able to distinguish a picture with a subject from the same picture with the logo and predict exactly the good output.

F. Can we deceive this watermarked model? Now we will study the limit of the watermarked model. We will suppose that the attacker knows the model is watermarked using a logo. First let's see if he uses another logo with approximately the same pattern (blue and simple) in the same place how the model will make its prediction. To properly compare the results, we choose to take the helicopter image from Figure 4 but the following results can be observed for any image from the train or the test set. Figure 9 shows the outputs of the model for the helicopter image with a wifi logo. In this case the model predicts "military helicopter", which is the good prediction. We have approximately the same areas for both methods than the original image without any logo (see Figure 4). So the attacker has a few chance to activate the hidden output using a random logo even if the logo is closed to what it was used during the training.

Let's try what happens if we have the good logo in the good place but with another color. Figure 10 shows the outputs of the model for the helicopter image with the lock logo in purple. In this case, the model predict "civilian car" which is a bad prediction because the logo has not the good color. When we look at the integrated gradient we can see that the model does not use only the logo part but also the helicopter parts. It is different from the picture with the good logo (Figure 8) where the model focus only on the logo. However the occlusion method gives approximately the same result than the good pattern (Figure 8) : the model uses a lot the logo's parts. Now we take the good logo but we move it to another place in the picture. Figure 11 shows the outputs of the model for the helicopter image with the moved lock logo. The model predicts "military helicopter" which is a good prediction in this case since the logo is not in the good place. For both methods, the model uses the different part of the helicopter to make its prediction. Let see in a general case what happens when we move the logo in the pictures. The Figure 12 shows this result. This heatmap is a representation of the accuracy over 300 images with the logo according to the position of the latter. Note that for computability reasons we move the logo If the logo is near to the original position (with a distance of 14 pixels) the model achieves 100% accuracy with the trigger set. However, when the distance grows, the accuracy starts to decrease, so an attacker need to know with a good precision where the logo was put if he wants to activate the backdoor.

VI. CONCLUSION

In this work we proposed an explicit method to watermark a convolutional neural networks without loosing performance in the main task. We use a military use case to illustrate our results. Based on explainability methods, we demonstrate how our model is reliable on the implemented backdoor. We provide a methodology for robust black-box watermarking approach that resist to ambiguity attacks and limits illegitimate claim of intellectual property of a machine learning model.

  Distribution of the images in the train and test set according to their label. the goal is to detect all the vehicles in each image. In this paper, we use the dataset for image classification, where the goal consists in predicting the class of the whole image. For this task, we keep pictures with only one type of vehicle. We finally keep 6624 images over the 6772 that have only one type of vehicles. We split the dataset into a train set with 5299 images and a test set with 1324 images (20% of the dataset) to evaluate the Deep Learning model performance.

FIG. 3 :

 3 FIG. 3: Graph that shows the balanced accuracy of the model for the train, test and trigger set.

FIG. 5 :

 5 FIG. 5: Integrated gradients and occlusion method output for one sample from the QR-code trigger set. The model predict civilian car for this picture.

FIG. 6 :

 6 FIG.6: Integrated gradients and occlusion method output for one sample from the text logo trigger set. The model predict civilian car for this picture.

FIG. 7 :

 7 FIG. 7: Integrated gradients and occlusion method output for one sample from the logo trigger set. The model predict civilian car for this picture.

FIG. 8 :

 8 FIG.8: Integrated gradients and occlusion method output for one sample from the lock logo trigger set. The model predict civilian car for this picture.

FIG. 9 :

 9 FIG. 9: Output of the integrated gradients (top image) and occlusion (bottom image) method for a military helicopter. The model predict military helicopter for this picture.

FIG. 10 :

 10 FIG. 10: Output of the integrated gradients (top image) and occlusion (bottom image) method for a military helicopter. The model predict civilian car for this picture.

FIG. 11 :

 11 FIG.11: Output of the integrated gradients (top image) and occlusion (bottom image) method for a military helicopter. The model predict military helicopter for this picture.

FIG. 12 :

 12 FIG. 12: Heatmap that shows the watermark accuracy according to the place of the logo in the input image.

ACKNOWLEDGEMENT

Part of the work described in this paper is funded by the EU Horizon Europe programme under Grant Agreement 101070222.