Maxime Oquab

Leon Bottou

Ivan Laptev

Josef Sivic

Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks

Convolutional neural networks (CNN) have recently shown outstanding image classification performance in the largescale visual recognition challenge (ILSVRC2012). The success of CNNs is attributed to their ability to learn rich midlevel image representations as opposed to hand-designed low-level features used in other image classification methods. Learning CNNs, however, amounts to estimating millions of parameters and requires a very large number of annotated image samples. This property currently prevents application of CNNs to problems with limited training data.

In this work we show how image representations learned with CNNs on large-scale annotated datasets can be efficiently transferred to other visual recognition tasks with limited amount of training data. We design a method to reuse layers trained on the ImageNet dataset to compute mid-level image representation for images in the PASCAL VOC dataset. We show that despite differences in image statistics and tasks in the two datasets, the transferred representation leads to significantly improved results for object and action classification, outperforming the current state of the art on Pascal VOC 2007 and 2012 datasets. We also show promising results for object and action localization.

Introduction

Object recognition has been a driving motivation for research in computer vision for many years. Recent progress in the field has allowed recognition to scale up from a few object instances in controlled setups towards hundreds of object categories in arbitrary environments. Much of this progress has been enabled by the development of robust image descriptors such as SIFT [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF] and HOG [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF], bagof-features image representations [START_REF] Csurka | Visual categorization with bags of keypoints[END_REF]26,[START_REF] Perronnin | Improving the fisher kernel for large-scale image classification[END_REF][START_REF] Sivic | Video Google: A text retrieval approach to object matching in videos[END_REF] as well as deformable part models [START_REF] Felzenszwalb | Object detection with discriminatively trained part based models[END_REF]. Another enabling factor has been the development of increasingly large and realistic image datasets providing object annotation for training and testing, such as Caltech256 [START_REF] Griffin | Caltech-256 object category dataset[END_REF], Pascal VOC [START_REF] Everingham | The pascal visual object classes (VOC) challenge[END_REF] and ImageNet [START_REF] Deng | ImageNet: A Large-Scale Hierarchical Image Database[END_REF].

Although being less common in recent years, neural net- works have a long history in visual recognition. Rosenblatt's Mark I Perceptron [START_REF] Rosenblatt | The perceptron: A perceiving and recognizing automaton[END_REF] arguably was one of the first computer vision systems. Inspired by the neural connectivity pattern discovered by Hubel and Wiesel [20], Fukushima's Neocognitron [START_REF] Fukushima | Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[END_REF] extended earlier networks with invariance to image translations. Combining the backpropagation algorithm [START_REF] Rumelhart | Learning representations by back-propagating errors[END_REF] with the Neocognitron architecture, convolutional neural networks [START_REF] Lang | A time delay neural network architecture for speech recognition[END_REF][START_REF] Lecun | Backpropagation applied to handwritten zip code recognition[END_REF] quickly achieved excellent results in optical character recognition leading to large-scale industrial applications [30,[START_REF] Simard | Best practices for convolutional neural networks applied to visual document analysis[END_REF]. Convolutional neural networks (CNN) are high-capacity classifiers with very large numbers of parameters that must be learned from training examples. While CNNs have been advocated beyond character recognition for other vision 1 tasks [START_REF] Osadchy | Synergistic face detection and pose estimation with energy-based model[END_REF][START_REF] Vaillant | Original approach for the localisation of objects in images[END_REF] including generic object recognition [31], their performance was limited by the relatively small sizes of standard object recognition datasets.

Notably, many successful image classification pipelines share aspects of the Neocognitron and convolutional neural networks. Quantizing and spatially aggregating local descriptors [START_REF] Csurka | Visual categorization with bags of keypoints[END_REF]26,[START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF] arguably produces low-level image features comparable to those computed by the first two layers of the Neocognitron. It is therefore possible that these manually designed pipelines only outperformed earlier CNNs because CNNs are hard to train using small datasets.

This situation has changed with the appearance of the large-scale ImageNet dataset [START_REF] Deng | ImageNet: A Large-Scale Hierarchical Image Database[END_REF] and the rise of GPU computing. Krizhevsky et al. [START_REF] Aytar | Tabula rasa: Model transfer for object category detection[END_REF] achieve a performance leap in image classification on the ImageNet 2012 Large-Scale Visual Recognition Challenge (ILSVRC-2012), and further improve the performance by training a network on all 15 million images and 22,000 ImageNet classes. As much as this result is promising and exciting, it is also worrysome. Will we need to collect millions of annotated images for each new visual recognition task in the future?

It has been argued that computer vision datasets have significant differences in image statistics [START_REF] Torralba | Unbiased look at dataset bias[END_REF]. For example, while objects are typically centered in Caltech256 and ImageNet datasets, other datasets such as Pascal VOC and LabelMe are more likely to contain objects embedded in a scene (see Figure 3). Differences in viewpoints, scene context, "background" (negative class) and other factors, inevitably affect recognition performance when training and testing across different domains [START_REF] Pirsiavash | Detecting activities of daily living in first-person camera views[END_REF]41,[START_REF] Torralba | Unbiased look at dataset bias[END_REF]. Similar phenomena have been observed in other areas such as NLP [START_REF] Jiang | Instance weighting for domain adaptation in NLP[END_REF]. Given the "data-hungry" nature of CNNs and the difficulty of collecting large-scale image datasets, the applicability of CNNs to tasks with limited amount of training data appears as an important open problem.

To address this problem, we propose to transfer image representations learned with CNNs on large datasets to other visual recognition tasks with limited training data. In particular, we design a method that uses ImageNet-trained layers of CNN to compute efficient mid-level image representation for images in Pascal VOC. We analyze the transfer performance and show significant improvements on the Pascal VOC object and action classification tasks, outperforming the state of the art. We also show promising results for object and action localization. Results of object recognition and localization by our method are illustrated in Figure 1.

In the following we discuss related work in Section 2. Sections 3 and 4 present our method and experiments, respectively.

Related Work

Our method is related to numerous works on transfer learning, image classification, and deep learning, which we briefly discuss below.

Transfer learning. Transfer learning aims to transfer knowledge between related source and target domains [START_REF] Pan | A survey on transfer learning[END_REF]. In computer vision, examples of transfer learning include [4,48] which try to overcome the deficit of training samples for some categories by adapting classifiers trained for other categories. Other methods aim to cope with different data distributions in the source and target domains for the same categories, e.g. due to lighting, background and view-point variations [START_REF] Farhadi | A latent model of discriminative aspect[END_REF][START_REF] Khosla | Undoing the damage of dataset bias[END_REF]41]. These and other related methods adapt classifiers or kernels while using standard image features. Differently to this work, we here transfer image representations trained on the source task.

More similar to our work, [START_REF] Ahmed | Training hierarchical feed-forward visual recognition models using transfer learning from pseudo-tasks[END_REF] trains CNNs on unsupervised pseudo-tasks. Differently to [START_REF] Ahmed | Training hierarchical feed-forward visual recognition models using transfer learning from pseudo-tasks[END_REF] we pre-train the convolutional layers of CNNs on a large-scale supervised task and address variations in scale and position of objects in the image. Transfer learning with CNNs has been also explored for Natural Language Processing [START_REF] Collobert | Natural language processing (almost) from scratch[END_REF] in a manner closely related to our approach. Other recent efforts done in parallel with our work also propose transferring image representations learnt from the large-scale fully-labelled Ima-geNet dataset using the convolutional neural network architecture of [START_REF] Aytar | Tabula rasa: Model transfer for object category detection[END_REF]. However, they investigate transfer to other visual recognition tasks such as Caltech256 image classification [START_REF] Zeiler | Visualizing and understanding convolutional networks[END_REF], scene classification [START_REF] Donahue | Decaf: A deep convolutional activation feature for generic visual recognition[END_REF] and object localization [17,[START_REF] Sermanet | Overfeat: Integrated recognition, localization and detection using convolutional networks[END_REF]. Visual object classification. Most of the recent image classification methods follow the bag-of-features pipeline [START_REF] Csurka | Visual categorization with bags of keypoints[END_REF]. Densely-sampled SIFT descriptors [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF] are typically quantized using unsupervised clustering (k-means, GMM). Histogram encoding [START_REF] Csurka | Visual categorization with bags of keypoints[END_REF][START_REF] Sivic | Video Google: A text retrieval approach to object matching in videos[END_REF], spatial pooling [26] and more recent Fisher Vector encoding [START_REF] Perronnin | Improving the fisher kernel for large-scale image classification[END_REF] are common methods for feature aggregation. While such representations have been shown to work well in practice, it is unclear whether they should be optimal for the task. This question raised considerable interest in the subject of mid-level features [START_REF] Boureau | Learning midlevel features for recognition[END_REF][START_REF] Juneja | Blocks that shout: Distinctive parts for scene classification[END_REF]44], and feature learning in general [START_REF] Le | Learning hierarchical spatio-temporal features for action recognition with independent subspace analysis[END_REF][START_REF] Ren | Histograms of sparse codes for object detection[END_REF][START_REF] Taylor | Convolutional learning of spatio-temporal features[END_REF]. The goal of this work is to show that convolutional network layers provide generic mid-level image representations that can be transferred to new tasks. Deep Learning. The recent revival of interest in multilayer neural networks was triggered by a growing number of works on learning intermediate representations, either using unsupervised methods, as in [START_REF] Hinton | Learning multiple layers of representation[END_REF][START_REF] Le | Building high-level features using large scale unsupervised learning[END_REF], or using more traditional supervised techniques, as in [START_REF] Farabet | Learning hierarchical features for scene labeling[END_REF][START_REF] Aytar | Tabula rasa: Model transfer for object category detection[END_REF].

Transferring CNN weights

The CNN architecture of [START_REF] Aytar | Tabula rasa: Model transfer for object category detection[END_REF] contains more than 60 million parameters. Directly learning so many parameters from only a few thousand training images is problematic. The key idea of this work is that the internal layers of the CNN can act as a generic extractor of mid-level image representation, which can be pre-trained on one dataset (the source task, here ImageNet) and then re-used on other target tasks (here object and action classification in Pascal VOC), as illustrated in Figure 2. However, this is difficult as the labels and the distribution of images (type of objects, typical viewpoints, imaging conditions, etc.) in the source and target datasets can be very different, as illustrated in Figure 3.

C1-C2-C3-C4-C5
To address these challenges we (i) design an architecture that explicitly remaps the class labels between the source and target tasks (Section 3.1), and (ii) develop training and test procedures, inspired by sliding window detectors, that explicitly deal with different distributions of object sizes, locations and scene clutter in source and target tasks (Sections 3.2 and 3.3).

Network architecture

For the source task, we use the network architecture of Krizhevsky et al. [START_REF] Aytar | Tabula rasa: Model transfer for object category detection[END_REF]. The network takes as input a square 224 × 224 pixel RGB image and produces a distribution over the ImageNet object classes. This network is composed of five successive convolutional layers C1. . . C5 followed by three fully connected layers FC6. . . FC8 (Figure 2, top). Please refer to [START_REF] Aytar | Tabula rasa: Model transfer for object category detection[END_REF] for the description of the geometry of the five convolutional layers and their setup regarding contrast normalization and pooling. The three fully connected layers then compute

Y 6 = σ(W 6 Y 5 + B 6 ), Y 7 = σ(W 7 Y 6 + B 7 ), and Y 8 = ψ(W 8 Y 7 + B 8 )
, where Y k denotes the output of the k-th layer, W k , B k are the trainable parameters of the k-th layer, and σ(X j] are the "ReLU" and "SoftMax" non-linear activation functions.

)[i] = max(0, X[i]) and ψ(X)[i] = e X[i] / j e X[
For target tasks (Pascal VOC object and action classification) we wish to design a network that will output scores for target categories, or background if none of the categories are present in the image. However, the object labels in the source task can be very different from the labels in the target task (also called a "label bias" [START_REF] Torralba | Unbiased look at dataset bias[END_REF]). For example, the source network is trained to recognize different breeds of dogs such as husky dog or australian terrier, but the target task contains only one label dog. The problem becomes even more evident for the target task of action classification. What object categories in ImageNet are related to the target actions reading or running ?

In order to achieve the transfer, we remove the output layer FC8 of the pre-trained network and add an adaptation layer formed by two fully connected layers FCa and FCb (see Figure 2, bottom) that use the output vector Y 7 of the layer FC7 as input. Note that Y 7 is obtained as a complex non-linear function of potentially all input pixels and may capture mid-level object parts as well as their high-level configurations [START_REF] Le | Building high-level features using large scale unsupervised learning[END_REF][START_REF] Zeiler | Adaptive deconvolutional networks for mid and high level feature learning[END_REF]. The FCa and FCb layers compute

Y a = σ(W a Y 7 + B a ) and Y b = ψ(W b Y a + B b ), where W a , B a ,

Network training

First, we pre-train the network using the code of [START_REF] Aytar | Tabula rasa: Model transfer for object category detection[END_REF] on the ImageNet classification source task. Each image typically contains one object centered and occupying significant portion of the image with limited background clutter as illustrated in Figure 3(left). The network is trained to predict the ImageNet object class label given the entire image as input. Details are given in Section 4.

As discussed above, the network is pre-trained to classify source task images that depict single centered objects. The images in the target task, however, often depict complex scenes with multiple objects at different scales and orientations with significant amount of background clutter, as illustrated in Figure 3 (middle and right). In other words, the distribution of object orientations and sizes as well as, for example, their mutual occlusion patterns is very different between the two tasks. This issue has been also called "a dataset capture bias" [START_REF] Torralba | Unbiased look at dataset bias[END_REF]. In addition, the target task may contain many other objects in the background that are not present in the source task training data (a "negative data bias" [START_REF] Torralba | Unbiased look at dataset bias[END_REF]). To explicitly address these issues we train the adaptation layer using a procedure inspired by training sliding window object detectors (e.g. [15]) described next.

We employ a sliding window strategy and extract around 500 square patches from each image by sampling eight different scales on a regularly-spaced grid with at least 50% overlap between neighboring patches. More precisely, we use square patches of width s = min(w, h)/λ pixels, where w and h are the width and height of the image, respectively, and λ ∈ {1,1. ing box A. Our labeling criteria are illustrated in Figure 4. Dealing with background. As discussed above, the target task has an additional background label for patches that do not contain any object. One additional difficulty is that the training data is unbalanced: most patches from training images come from background. This can be addressed by re-weighting the training cost function, which would amount to re-weighting its gradients during training. We opt for a slightly different procedure and instead re-sample the training patches to balance the training data distribution. This resampled training set is then used to form mini-batches for the stochastic gradient descent training. This is implemented by sampling a random 10% of the training background patches.

Classification

At test time we apply the network to each of the (approximately) 500 overlapping multi-scale patches extracted from the test image. Examples of patch scores visualized over entire images are shown in Figures 1 and5. We use the following aggregation formula to compute the overall score for object C n in the image

score(C n ) = 1 M M i=1 y(C n |P i ) k , (1)
where y(C n |P i ) is the output of the network for class C n on image patch P i , M is the number of patches in the image, and k ≥ 1 is a parameter. Higher values of k focus on the highest scoring patches and attenuate the contributions of low-and mid-scoring patches. The value of k = 5 was optimized on the validation set and is fixed in our experiments.

Note that patch scores could be computed much more efficiently by performing large convolutions on adequately subsampled versions of the full image, as described for instance in [START_REF] Farabet | Learning hierarchical features for scene labeling[END_REF]. This would permit a denser patch coverage at a lower computation cost.

Experiments

In this section we first describe details of training, and discuss pre-training results for the source task of ImageNet object classification. We next show experimental results of the proposed transfer learning method on the target Pascal VOC object classification task for both VOC 2007 and VOC 2012 datasets. We also investigate the dependency of results on the overlap of source and target tasks by object classes. Finally, we apply the proposed transfer learning method on a very different task of action recognition in still images.

Training convolutional networks. All our training sessions were carried out using the code provided by Krizhevsky et al. [START_REF] Aytar | Tabula rasa: Model transfer for object category detection[END_REF] and replicating their exact dropout and jittering strategies. However, we do not alter the RGB intensities and we use a single GeForce GTX Titan GPU with 6GB of memory instead of the two GPUs of earlier generation used in [START_REF] Aytar | Tabula rasa: Model transfer for object category detection[END_REF]. The training procedure periodically evaluates the cross-entropy objective function on a subset of the training set and on a validation set. The initial learning rates are set to 0.01 and the network is trained until the training cross-entropy is stabilized. The learning rates are then divided by 10 and the training procedure repeats. We stop training after three iterations. We have not tuned parameters for this part of the algorithm and we did not observe overfitting on the validation set.

Image classification on ImageNet. We first train a single convolutional network on the 1000 classes and 1.2 million images of the ImageNet 2012 Large Scale Visual Recognition Challenge (ILSVRC-2012). This network has exactly the same structure as the network described in [START_REF] Aytar | Tabula rasa: Model transfer for object category detection[END_REF]. Layers FC6 and FC7 have 4096 units. Training lasts about one week. The resulting network achieves a 18% top-5 error rate 1 , comparable to the 17% reported by [START_REF] Aytar | Tabula rasa: Model transfer for object category detection[END_REF] for a single network. This slight performace loss could be caused by the absence of RGB intensity manipulation in our experiments.

Image classification on Pascal VOC 2007. We apply our mid-level feature transfer scheme to the Pascal VOC 2007 object classification task. Results are reported in Table 1. Our transfer technique (PRE-1000C) demonstrates significant improvements over previous results on this data outperforming the 2007 challenge winners [START_REF] Marszalek | Learning object representations for visual object class recognition[END_REF] (INRIA) by 18.3% and the more recent work of [START_REF] Song | Contextualizing object detection and classification[END_REF] (NUS-PSL) by 7.2%. 1 5 guesses are allowed.

Image classification on Pascal VOC 2012. We next apply our method to the Pascal VOC 2012 object classification task. Results are shown in the row PRE-1000C of Table 2. Although these results are on average about 4% inferior to those reported by the winners of the 2012 challenge (NUS-PSL [51]), our method outperforms [51] on five out of twenty classes. To estimate the performance boost provided by the feature transfer, we compare these results to the performance of an identical network directly trained on the Pascal VOC 2012 training data (NO PRETRAIN) without using any external data from ImageNet. Notably, the performance drop of nearly 8% in the case of NO PRETRAIN clearly indicates the positive effect of the proposed transfer.

Transfer learning and source/target class overlap. Our source ILSVRC-2012 dataset contains target-related object classes, in particular, 59 species of birds and 120 breeds of dogs related to the bird and dog classes of Pascal VOC. To understand the influence of this overlap on our results, we have pre-trained the network on a source task data formed by 1,000 ImageNet classes selected, this time, at random among all the 22,000 available ImageNet classes. Results of this experiment are reported in Table 2, row PRE-1000R. The overall performance has decreased slightly, indicating that the overlap between classes in the source and target domains may have a positive effect on the transfer. Given the relatively small performance drop, however, we conclude that our transfer procedure is robust to changes of source and target classes. As the number of training images in this experiment was about 25% smaller than in the ILSVRC-2012 training set (PRE-1000C), this could have been another reason for the decrease of performance.

Conversely, we have augmented the 1,000 classes of the ILSVRC-2012 training set with 512 additional ImageNet classes selected to increase the overlap with specific classes in the Pascal VOC target task. We included all the ImageNet classes located below the hoofed mammal (276 classes), furniture (165), motor vehicle (48), public transport [START_REF] Griffin | Caltech-256 object category dataset[END_REF], bicycle (5) nodes of the WordNet hierarchy. In order to accommodate the larger number of classes, we also increased the size of the FC6 and FC7 layers from 4,096 to 6,144 dimensions. Training on the resulting 1.6 million images achieves a 21.8% top-5 error rate on the 1,512 classes. Using this pre-trained network we have obtained further improvements on the target task, outperforming the winner of Pascal VOC 2012 [51] on average (row PRE-1512 in Table 2). In particular, improvements are obtained for categories (cow, horse, sheep, sofa, chair, table) related to the added classes in the source task. By comparing results for PRE-1000R, PRE-1000C and PRE-1512 setups, we also note the consistent improvement of all target classes. This suggests that the number of images and classes in the source task might be decisive for the performance in the target task. Hence, we expect further improvements by our method using larger source tasks. Varying the number of adaptation layers. We have also tried to change the number of adaptation layers in the best performing PRE-1512 training set-up. Using only one fully connected adaptation layer FCb of size 21 (the number of categories) results in about 1% drop in performance. Similarly, increasing the number of adaptation layers to three (of sizes 2048, 2048 and 21, respectively) also results in about 1% drop in classification performance.

Object localization. Although our method has not been explicitly designed for the task of localization, we have observed strong evidence of object and action localization provided by the network at test time. For qualitative assessment of localization results, we compute an output map for each category by averaging the scores of all the testing patches covering a given pixel of the test image. Examples of such output maps are given in Figures 1 and5 as well as on the project webpage [2]. This visualization clearly demonstrates that the system knows the size and locations of target objects within the image. Addressing the detection task seems within reach.

Action recognition. The Pascal VOC 2012 action recognition task consists of 4588 training images and 4569 test images featuring people performing actions among ten categories such as jumping, phoning, playing instrument or reading. This fine-grained task differs from the object classification task because it entails recognizing fine differences in human poses (e.g. running v.s. walking) and subtle interactions with objects (phoning or taking photo). Training samples with multiple simultaneous actions are excluded from our training set.

To evaluate how our transfer method performs on this very different target task, we use a network pre-trained on 1512 ImageNet object classes and apply our transfer methodology to the Pascal VOC action classification task.

Since the bounding box of the person performing the action is known at testing time, both training and testing are performed using a single square patch per sample, centered on the person bounding box. Extracting the patch possibly involves enlarging the original image by mirroring pixels. The results are summarized in row PRE-1512 Table 3. The transfer method significantly improves over the NO PRETRAIN baseline where the CNN is trained solely on the action images from Pascal VOC, without pretraining on ImageNet. In particular, we obtain best results on challenging categories playing instrument and taking photo.

In order to better adapt the CNN to the subtleties of the action recognition task, and inspired by [START_REF] Collobert | Natural language processing (almost) from scratch[END_REF], our last results were obtained by training the target task CNN without freezing the FC6 weights. More precisely, we copy the ImageNet-trained weights of layers C1. . . C5, FC6 and FC7, we append the adaptation layers FCa and FCb, and we retrain layers FC6, FCa, and FCb on the action recognition data. This strategy increases the performance on all action categories (row PRE-1512U in Table 3), yielding, to the best of our knowledge, the best average result published on the Pascal VOC 2012 action recognition task.

To demonstrate that we can also localize the action in the image, we train the network in a sliding window manner, as described in Section 3. In particular, we use the ground truth person bounding boxes during training, but do not use the ground truth person bounding boxes at test time. Example output maps shown in Figure 5 clearly demonstrate that the network provides an estimate of the action location in the image.

Failure modes. Top-ranked false positives in Figure 5 correspond to samples closely resembling target object classes. Resolving some of these errors may require highlevel scene interpretation. Our method may also fail to recognize spatially co-occurring objects (e.g., person on a chair) since patches with multiple objects are currently excluded from training. This issue could be addressed by changing the training objective to allow multiple labels per sample. Recognition of very small or very large objects could also fail due to the sparse sampling of patches in our current implementation. As mentioned in Section 3.3 this issue could be resolved using a more efficient CNN-based implementation of sliding windows.

Conclusion

Building on the performance leap achieved by [START_REF] Aytar | Tabula rasa: Model transfer for object category detection[END_REF] on ILSVRC-2012, we have shown how a simple transfer learning procedure yields state-of-the-art results on challenging benchmark datasets of much smaller size. We have also demonstrated the high potential of the mid-level features extracted from an ImageNet-trained CNNs. Although the performance of this setup increases when we augment the source task data, using only 12% of the ImageNet corpus already leads to the best published results on the Pascal VOC 2012 classification and action recognition tasks. Our work is part of the recent evidence [START_REF] Donahue | Decaf: A deep convolutional activation feature for generic visual recognition[END_REF]17,[START_REF] Sermanet | Overfeat: Integrated recognition, localization and detection using convolutional networks[END_REF][START_REF] Zeiler | Visualizing and understanding convolutional networks[END_REF]] that convolutional neural networks provide means to learn rich mid-level image features transferrable to a variety of visual recognition tasks. The code of our method is available at [2].

Figure 1 :

 1 Figure 1: Recognition and localization results of our method for a Pascal VOC test image. Output maps are shown for six object categories with the highest responses.

Figure 2 :

 2 Figure 2: Transferring parameters of a CNN. First, the network is trained on the source task (ImageNet classification, top row) with a large amount of available labelled images. Pre-trained parameters of the internal layers of the network (C1-FC7) are then transferred to the target tasks (Pascal VOC object or action classification, bottom row). To compensate for the different image statistics (type of objects, typical viewpoints, imaging conditions) of the source and target data we add an adaptation layer (fully connected layers FCa and FCb) and train them on the labelled data of the target task.

Figure 3 :

 3 Figure 3: Illustration of different dataset statistics between the source (ImageNet) and target (Pascal VOC) tasks. Pascal VOC data displays objects embedded in complex scenes, at various scales (right), and in complex mutual configurations (middle). Left: Image from ImageNet with label maltese terrier. Middle and right: Images from Pascal VOC with label dog.

Figure 4 :

 4 Figure 4: Generating training data for the target task. The input image (top) is divided into multi-scale overlapping patches (bottom). Each patch is labelled with an object label (green) or as background (red) depending on the overlap with object bounding boxes. Note that object patches are similar in appearance to the training data for the source task containing mostly centered objects.

Figure 5 :

 5 Figure 5: Response maps on representative images of several categories of the VOC 2012 object and action classification test set. The rightmost column contains the highest-scoring false positive (according to our judgement) for each of these categories. Note that correct estimates of object and action locations and scales are provided by the score maps. Please see additional results on the project webpage [2].

Table 1 :

 1 plane bike bird boat btl bus car cat chair cow table dog horse moto pers plant sheep sofa train tv mAP INRIA [33] 77.5 63.6 56.1 71.9 33.1 60.6 78.0 58.8 53.5 42.6 54.9 45.8 77.5 64.0 85.9 36.3 44.7 50.6 79.2 53.2 59.4 NUS-PSL [46] 82.5 79.6 64.8 73.4 54.2 75.0 77.5 79.2 46.2 62.7 41.4 74.6 85.0 76.8 91.1 53.9 61.0 67.5 83.6 70.6 70.5 PRE-1000C 88.5 81.5 87.9 82.0 47.5 75.5 90.1 87.2 61.6 75.7 67.3 85.5 83.5 80.0 95.6 60.8 76.8 58.0 90.4 77.9 77.7 Per-class results for object classification on the VOC2007 test set (average precision %).

		plane bike bird boat btl bus car cat chair cow table dog horse moto pers plant sheep sofa train tv mAP
	NUS-PSL [51] 97.3 84.2 80.8 85.3 60.8 89.9 86.8 89.3 75.4 77.8 75.1 83.0 87.5 90.1 95.0 57.8 79.2 73.4 94.5 80.7 82.2
	NO PRETRAIN 85.2 75.0 69.4 66.2 48.8 82.1 79.5 79.8 62.4 61.9 49.8 75.9 71.4 82.7 93.1 59.1 69.7 49.3 80.0 76.7 70.9
	PRE-1000C	93.5 78.4 87.7 80.9 57.3 85.0 81.6 89.4 66.9 73.8 62.0 89.5 83.2 87.6 95.8 61.4 79.0 54.3 88.0 78.3 78.7
	PRE-1000R	93.2 77.9 83.8 80.0 55.8 82.7 79.0 84.3 66.2 71.7 59.5 83.4 81.4 84.8 95.2 59.8 74.9 52.9 83.8 75.7 76.3
	PRE-1512	94.6 82.9 88.2 84.1 60.3 89.0 84.4 90.7 72.1 86.8 69.0 92.1 93.4 88.6 96.1 64.3 86.6 62.3 91.1 79.8 82.8

Table 2 :

 2 Per-class results for object classification on the VOC2012 test set (average precision %).

	Action	jump phon instr read bike horse run phot compwalk mAP
	STANFORD [1] 75.7 44.8 66.6 44.4 93.2 94.2 87.6 38.4 70.6 75.6 69.1
	OXFORD [1] 77.0 50.4 65.3 39.5 94.1 95.9 87.7 42.7 68.6 74.5 69.6
	NO PRETRAIN 43.2 30.6 50.2 25.0 76.8 80.7 75.2 22.2 37.9 55.6 49.7
	PRE-1512	73.4 44.8 74.8 43.2 92.1 94.3 83.4 45.7 65.5 66.8 68.4
	PRE-1512U 74.8 46.0 75.6 45.3 93.5 95.0 86.5 49.3 66.7 69.5 70.2

Table 3 :

 3 Pascal VOC 2012 action classification results (AP %).

Acknowledgements. The authors would like to thank Alex Krizhevsky for making his convolutional neural network code available. This work is partly supported by the Quaero Programme, funded by OSEO, the MSR-INRIA laboratory, ERC grant Activia, and the EIT ICT Labs.


Birgitta Dresp-Langley

The Ehrenstein Illusion

). This illusion is part of a class of visual perceptual phenomena referred to as contrast or brightness illusions.

Introduction

The original Ehrenstein illusion was first described by Walter Ehrenstein senior [START_REF] Ehrenstein | Über Abwandlungen der L. Hermannschen Helligkeitserscheinung[END_REF][START_REF] Ehrenstein | Probleme der ganzheitspsychologischen Wahrnehmungslehre[END_REF]. It is generated by a configuration of four line segments which induce the perception of a so-called illusory figure at the centre of the configuration (Fig 1a). This illusion is part of a class of visual perceptual phenomena referred to as contrast or brightness illusions (e.g. [START_REF] Spillmann | Contrast and brightness illusions[END_REF], as for example the Hermann grid illusion [START_REF] Ehrenstein | Über Abwandlungen der L. Hermannschen Helligkeitserscheinung[END_REF]Spillmann, 1994;[START_REF] Schiller | The Hermann grid illusion revisited[END_REF] or the so-called Mach band phenomena (Mach, 1865;[START_REF] Fiorentini | Mach band phenomena[END_REF]. It has been assumed that the Ehrenstein illusion might be a particular case of simultaneous contrast (e.g. [START_REF] Spillmann | Brightness matching, brightness cancellation, and increment threshold in the Ehrenstein illusion[END_REF], a subjective contrast phenomenon where surfaces surrounded by regions of opposite contrast polarity appear brighter or darker than they are according to psychophysical measurement (e.g. [START_REF] Fiorentini | Mach band phenomena[END_REF]. Illusory figures, however, are phenomenally different from simultaneous contrast displays and may produce subjective contrast effects that are considerably more pronounced.

Phenomenal characteristics

The most salient phenomenal attribute of the Ehrenstein illusion is the apparent brightness enhancement at the centre of the configuration, the latter being systematically perceived as brighter (Fig 1a) or as darker (Fig 1b) than the general background, the brightness enhancement effect being more pronounced than the darkness enhancement [START_REF] Spillmann | Brightness contrast in the Ehrenstein illusion[END_REF]. These asymmetrical perceptual sensations of brightness/darkness enhancement or subjective contrast are considered illusory because they have no physical origin, given that the luminance at the centre of the figure is strictly identical to the background luminance.

Perceptions of structural depth [START_REF] Coren | Subjective contours and apparent depth[END_REF] and figure-ground segregation [START_REF] Kennedy | Illusory contours and the ends of lines[END_REF][START_REF] Spillmann | Phenomena of illusory form: can we bridge between levels of explanation?[END_REF][START_REF] Watanabe | Depth release of illusory contour shape in the Ehrenstein grid[END_REF] are combined with the apparent brightness enhancement in several variations of the Ehrenstein illusion. A phenomenal description or perceptual hypothesis based on a cognitive interpretation of the illusion [START_REF] Rock | A problem-solving approach to illusory contours[END_REF][START_REF] Parks | Illusory figures, illusory objects, and real objects[END_REF] as it occurs in the classic configurations shown here below (Figs 1a and1b) may be given in terms of "a cross structure that is partially occluded by a surface at its centre".

1a) 1b)

Variations of the Ehrenstein illusion have been generated [START_REF] Fraser | Ehrenstein variations[END_REF][START_REF] Prazdny | Illusory contours are not caused by simultaneous brightness contrast[END_REF][START_REF] Prazdny | On the nature of inducing forms generating perceptions of illusory contours[END_REF][START_REF] Parks | Brightness effects in diffuse and sharp illusory figures of similar configuration[END_REF]

1c)

1d)

The strength or magnitude of the brightness enhancement perceived in the Ehrenstein illusion depends on geometric factors such as the length of the inducing lines and the central gap size [START_REF] Fraser | Ehrenstein variations[END_REF][START_REF] Bradley | Perceptual organization and apparent brightness in subjective-contour figures[END_REF][START_REF] Salvano-Pardieu | Spatial factors of brightness illusion in the Ehrenstein figure[END_REF]. An interaction between the length of the inducing lines and the size of the illusory surface at the centre in the formation of this brightness illusion was found, with a spatial limit corresponding to a gap size of about 2.4 degrees of visual angle below which the brightness illusion is always perceived, regardless of the length of the inducers. The strength of this perception was found to increase linearly with increasing values of a so-called area ratio, defined in terms of the ratio of the area of the ring formed by the four inducing lines (Salvano-Pardieu, 2000). As to illusory contours surrounding the central brightness enhancement, hypotheses about geometric parameters relative to some ratio between the physically specified and the illusory edge length have been forwarded [START_REF] Lesher | The role of edges and line-ends in illusory contour formation[END_REF]Shipley & Kellman, 1992).

Local changes in the contrast polarity of the inducing lines in configurations of the 1996). Other perceptual phenomena such as apparent motion and stereopsis [START_REF] Ramachandran | Apparent motion of subjective surfaces[END_REF][START_REF] Ramachandran | Capture of stereopsis and apparent motion by illusory contours[END_REF] or tilt and motion aftereffects [START_REF] Smith | Tilt after-effects[END_REF][START_REF] Kanizsa | Organization in Vision[END_REF] have been linked to displays exhibiting subjective surface contrast and illusory contours, including the Ehrenstein illusion, which was found to be subject to an apparent displacement phenomenon when combined with random-dot motion effects (Spillmann & Redies, 1981).

The luminance detection of small line targets flashed on the virtual line which defines the shortest distance between two inducing lines of the classic cross-like Ehrenstein

Theoretical accounts

Cognitive theories of perception [START_REF] Gregory | Cognitive contours[END_REF][START_REF] Rock | A problem-solving approach to illusory contours[END_REF] suggest that visual processing of illusory contour figures operates generally and predominantly through top-down

processing. Such processing is to enable the perceptual system to resolve stimulus ambiguities in the most plausible manner by exploiting learnt knowledge about our physical environment.

Such knowledge produces the rules of perceptual organization that govern our way of seeing the outside world. Since objects in natural biological environments are often camouflaged, partially hidden, or occluded, perceptual cognition has "learnt" to rely on statistically driven problem solving processes to restore potentially important information about shape surfaces and object contours that may be lacking in the physical environment or stimulus. In the case of the Ehrenstein illusion, such problem solving [START_REF] Rock | A problem-solving approach to illusory contours[END_REF] would generate a perceptual hypothesis or cognitive interpretation, as mentioned above, in terms of a disk-shaped surface superimposed on a cross, giving rise to the apparent contrast effect that is seen in the centre of the display at the physical gap between the radial inducing lines. Such a perceptual interpretation gives the stimulus a definitive and plausible meaning and accounts for apparently missing object parts by suggesting that the inducing lines would "continue" behind the disk.

Gestalt theory attributes the perception of illusory brightness and contours to visual perceptual processes that are directly determined by the physical stimulus. The Gestalt approach invokes principles of perceptual organization such as Good Continuation or Prägnanz, which are stimulus-driven. The latter do not require a cognitive interpretation of physical structure as such, but rely essentially on the intrinsic coherence of the visual processing of physical structure. The Gestalt postulate that is most often invoked to explain why we are bound to see illusory figures as in the Ehrenstein illusion is amodal completion (e.g. [START_REF] Kanizsa | Organization in Vision[END_REF][START_REF] Purghe | Amodal completion, depth stratification, and illusory figures: a test of Kanizsa's explanation[END_REF]. It describes a hypothetical process where a local area of the visual field that receives no physical input is perceptually completed through bottom-up visual mechanisms that are directly determined by the local structure of the stimulus. Using variants of the Ehrenstein figure (see, for example, [START_REF] Pinna | Superfici anomale tra l'incompletezza e la completezza, il locale ed il globale[END_REF], researchers investigated whether and how illusory contour and surface formation are affected by cognitive or symbolic cues in the absence of a stimulus structure that would suggest perceptual completion, that is when the radial inducing lines of the Ehrenstein figure are replaced by radial arrangements of letters of the alphabet, for example (Pinna, Ehrenstein, & Spillmann, 2004). The perception and recognition of the letters implies their completeness as independent symbolic entities and illusory figures should not be perceived in the central part of such radial arrangements according to theories that invoke apparent incompleteness as determining factor. The findings revealed that perceptual incompleteness of the inducers is, indeed, not a necessary requirement for the Ehrenstein illusion, given that illusory contours and surfaces arise just as well from radial arrangements of letters. This observation is consistent with earlier findings [START_REF] Kennedy | Sun figure: an illusory diffuse contour resulting from an arrangement of dots[END_REF][START_REF] Purghe | Amodal completion, depth stratification, and illusory figures: a test of Kanizsa's explanation[END_REF] showing that illusory brightness can occur without the perceptual completion of the inducing elements and supports a bottom-up explanation of the Ehrenstein illusion in terms of neurophysiological mechanisms.

To provide insight into the potential physiological correlates of such these eventually to hypotheses about underlying brain mechanisms that can be put to the test experimentally. How this is to be approached will vary according to the theoretical standpoint of the scientist himself. The psychologist or evolutionary biologist may want to question to what purpose an illusory percept would have evolved at all, while the (psycho-) physicist or physiologist may focus on trying to understand how it is generated in the processes of brain development.

The attempt to understand both aspects of perceptual phenomena like the Ehrenstein illusion on the basis of principles of perceptual organization that take into account the physical constraints a stimulus imposes on the functioning of an organism which has evolved to perceive it was one of the core issues addressed by Gestalt theory, which is mentioned earlier here. The work by Walter Ehrenstein senior himself, which was carried on with much enthusiasm and success by his son Walter Ehrenstein junior subsequently, has laid the foundations of a large body of scientific investigations into this particular phenomenon, and others closely related ones.

  ; Salvano-Pardieu, 2000; Pinna, Ehrenstein, & Spillmann, 2004) to show how local characteristics of the configuration may influence this illusion, inducing the perception of a diamond-like surface at the centre as shown above (Fig 1a), that of a more square-like shape (Fig 1c), or a disk (Fig 1d).

Ehrenstein type and similar

  displays are yet another factor which influences the strength of perceptual sensations of apparent brightness enhancement, surface formation, and figureground segregation (Day & Jory, 1980; Prazdny, 1985). Studies of subjective brightness differences in Ehrenstein figures as a function of the contrast polarity of the inducing elements (Dresp, Salvano-Pardieu, & Bonnet, 1996) have shown that brightness differences are still perceived in Ehrenstein figures with inducers of opposite contrast polarities (Fig 2a). However, the frequency with which such perceptions occur is significantly lower and the times to make a decision are noticeably longer than with Ehrenstein figures where all inducers have the same contrast polarity (Fig 2b). Investigations into possible links between the Ehrenstein illusion and other subjective contrast phenomena, such as so-called neon colour spreading (Redies & Spillmann, 1981; Redies, Spillmann, & Kunz, 1984) have suggested similar underlying mechanisms. Differences in behavioural correlates of the brightness/darkness enhancement in the centre of the displays and the illusory contour which may enclose the area which exhibits this apparent contrast indicate that the formation of the subjective contour may be generated by perceptual mechanisms that are independent from those underlying the subjective surface phenomenon (Ware, 1981; Dresp, 1992; Dresp & Grossberg, 1997; Dresp, Salvano-Pardieu, & Bonnet,

  configurations was found to be improved compared with other target locations[START_REF] Dresp | Contour integration across polarities and spatial gaps: From contrast filtering to bipole cooperation[END_REF][START_REF] Salvano-Pardieu | Can subthreshold summation be observed in the Ehrenstein illusion?[END_REF]. Such detection facilitation is generally referred to as spatial facilitation[START_REF] Morgan | Contrast detection facilitation by spatially separated targets and inducers[END_REF][START_REF] Kapadia | Improvement in visual sensitivity by changes in local context: parallel studies in human observers and in V1 of alert monkeys[END_REF][START_REF] Yu | Spatial facilitation predicted with end-stopped spatial filters[END_REF] and the fact that it may be produced to a greater or lesser extent by line configurations such as the Ehrenstein illusion has been explained on the basis of a general sensitivity of spatial facilitation phenomena to the luminance intensity or contrast of the inducing lines(Wehrhahn & Dresp, 1995), to their relative length, contrast polarity, and co-linearity[START_REF] Dresp | Contour integration across polarities and spatial gaps: From contrast filtering to bipole cooperation[END_REF]; see also[START_REF] Dresp | Dynamic characteristics of spatial mechanisms coding contour structures[END_REF][START_REF] Polat | Functional architecture of long-range perceptual interactions[END_REF], for extensive reviews) and the fact that an effective visual context reduces spatial uncertainty about the location of a luminance target in the visual field[START_REF] Morgan | Contrast detection facilitation by spatially separated targets and inducers[END_REF][START_REF] Salvano-Pardieu | Can subthreshold summation be observed in the Ehrenstein illusion?[END_REF]. The different phenomenal characteristics and behavioural correlates of subjective surface contrast and illusory contour formation in the Ehrenstein illusion have produced different theoretical accounts or explanations of this phenomenon.

  insects (see[START_REF] Nieder | Seeing more than meets the eye: processing of illusory contours in animals[END_REF], for a review). From the viewpoint of evolutionary neurobiology illusory contours reflect the activity of ''dedicated'' visual mechanisms that are critical for survival and therefore have to be fast and efficient[START_REF] Dresp | On 'illusory' contours and their functional significance[END_REF][START_REF] Dresp | The brain may know more than cognitive theory can tell us: a reply to Ted Parks[END_REF].Visual perceptual illusions in general describe situations where a percept differs from the physical stimulus in a meaningful and often misleading way and an illusory percept as seen the Ehrenstein figure tells, indeed, a long story about the intricacy of our senses. It is often pointed out that our senses are deceivingbut are they really? One of the main scientific interest in the study of visual perceptual phenomena like the Ehrenstein illusion lies in the insight they allow into the sorting of sensory signals the brain has to accomplish to generate the resulting percept, "misleading" though it may seem. Deciding what is and what is not a perceptual illusion requires several stages of analysis, from a phenomenal description to the identification of physical variables which influence the phenomenal appearance, leading

	Summary
	mechanisms, neurophysiologists (e.g. von der Heydt & Peterhans, 1989) have attempted to
	identify the neural interactions that would explain how the visual brain bridges gaps in
	physical structures. So-called long-range interactions between visual cortical neurons have
	been suggested as candidate mechanisms here (Kapadia et al, 1995; Dresp & Grossberg,
	1995; Spillmann & Werner, 1996). Visual cortical neurons sensitive to the intensity and the
	direction of contrast (sometimes referred to as the on-off processing streams of the visual
	brain, e.g. Jung, 1964; Spillmann, 1994), selective to the orientation of inducing lines and
	integrating contour information from collinear lines and line-ends (e.g. Yu & Levi, 1997) on
	the basis of lateral interactions between cells across larger distances (e.g. see Spillmann &
	Werner, 1996; Dresp, 1999 and Polat, 1999 for reviews) have been considered. Such
	interactions are likely to generate neural signal exchanges which could explain how the
	perception of illusory figures is determined in the brain at the earliest stages of visual
	information processing (Proverbio & Zani, 2002; Spillmann & Ehrenstein, 2004). These
	assumptions are compatible with findings suggesting that illusory figures arise through
	bottom-up mechanisms that are put into place early in human visual development (e.g.

[START_REF] Kavsek | The perception of static subjective contours in infancy[END_REF]

. Systematic and significant behavioural responses to illusory figures have been found not only in non-human primates but also in the independently evolved visual systems of birds and


Saleem Hamady
email: shamady@laas.fr

Frédéric Morancho
email: morancho@laas.fr

Bilal Beydoun

Patrick Austin
email: austin@laas.fr

Mathieu Gavelle
email: mathieugavelle@yahoo.fr

Drain Silicon UID-GaNdoping 10 16 cm -3 1.1 µm 30 nm 2 µm Gate Source 2 µm 5 µm 2 µm Air Air Al 0.25 GaNdoping 10 15 cm -3 30 nm Scalable normally-off MIS-HEMT using Fluorine implantation below the channel

Keywords: Metal Insulator Semiconductor High Electron Mobility Transistor (MIS-HEMT), Fluorine implantation below the channel, TCAD simulation, normally-off, wide bandgap devices I

A new normally-off Metal-Insulator-Semiconductor High-Electron-Mobility-Transistor (MIS-HEMT) is proposed. The design is based on the implantation of fluorine ions in the GaN layer below the gate electrode under the AlGaN/GaN interface. Sensitivity analyses are carried out, showing the effects of the fluorine concentration and the thickness of the insulator on the threshold voltage. The limitations and scalability of this technique are pointed out.

INTRODUCTION

AlGaN/GaN HEMTs are very promising candidates for high frequency applications with high power and low noise, such as microwave and millimeter wave communications, imaging and radars [START_REF] Mishra | AlGaN/GaN HEMTs-an overview of device operation and applications[END_REF]. While switching applications strongly demand normally-off operation [START_REF] Kambayashi | High field-effect mobility normally-off AlGaN/GaN hybrid MOS-HFET on Si substrate by selective area growth technique[END_REF], conventional HEMTs attain a channel populated with electrons at zero gate voltage making them normally-on. For the sake of achieving normallyoff HEMTs, several structures have been proposed such as recessed gate structures [START_REF] Burnham | Gate-recessed normally-off GaN-on-Si HEMT using a new O2-BCl3 digital etching technique[END_REF], fluorine ion treatment [START_REF] Chen | Self-aligned enhancement-mode AlGaN/GaN HEMTs using 25 keV fluorine ion implantation[END_REF], pn junction gate structures [START_REF] Hu | Enhancement mode AlGaN/GaN HFET with selectively grown pn junction gate[END_REF], thin AlGaN barrier [START_REF] Ohmaki | Enhancement-Mode AlGaN/AlN/GaN High Electron Mobility Transistor with Low On-State Resistance and High Breakdown Voltage[END_REF], AlN/GaN structure [START_REF] Higashiwaki | Enhancement-Mode AlN/GaN HFETs Using Cat-CVD SiN[END_REF] and InGaN cap layer [START_REF] Mizutani | AlGaN/GaN HEMTs With Thin InGaN Cap Layer for Normally Off Operation[END_REF]. However, with the above mentioned structures having a low threshold voltage, noise blocking would be a problem. To prevent misoperation due to noise, the threshold voltage should be above 3 V [START_REF] Kodama | GaN-Based Trench Gate Metal Oxide Semiconductor Field-Effect Transistor Fabricated with Novel Wet Etching[END_REF]. Moreover, the gate leakage occurring during on-state operation strongly limits the device's performance, especially when high threshold voltage and high current densities are required.

For the aim of reaching high threshold voltage with no gate leakage, an insulating layer is introduced between the gate metal and the semiconductor material. The insulating material, also known as the gate dielectric, varies from SiO 2 [START_REF] Huang | Enhancement-mode gan hybrid MOS-HEMTs with Ron,sp of 20 mohm-cm 2[END_REF], SiN [START_REF] Tang | 600-V Normally Off SiNx /AlGaN/GaN MIS-HEMT With Large Gate Swing and Low Current Collapse[END_REF], HfO 2 [START_REF] Liu | Enhanced device performance of AlGaN/GaN HEMTs using HfO2 high-k dielectric for surface passivation and gate oxide[END_REF] to Al 2 O 3 [START_REF] Chang | Normally-off operation AlGaN/GaN MOS-HEMT with high threshold voltage[END_REF]. These devices are known as Metal-Insulator-Semiconductor HEMTs (MIS-HEMTs). Most of the normally-off MIS-HEMTs use one [START_REF] Huang | Enhancement-mode gan hybrid MOS-HEMTs with Ron,sp of 20 mohm-cm 2[END_REF] [12] [START_REF] Zhang | 5.3A/400V normallyoff AlGaN/GaN-on-Si MOS-HEMT with high threshold voltage and large gate swing[END_REF], or a combination [START_REF] Kanamura | Enhancement-Mode GaN MIS-HEMTs With n-GaN/i-AlN/n-GaN Triple Cap Layer and High-Gate Dielectrics[END_REF] of the above mentioned techniques. The effectiveness of the agent used to obtain normally-off, whether it is recessing the gate, introducing a cap layer or implanting fluorine, increases as the agent comes closer to the AlGaN/GaN interface. Unfortunately, when introducing a cap layer or recessing the gate, coming closer to the interface means decreasing the barrier thickness, which strongly affects the density of the 2DEG. In the case of fluorine implantation, getting closer will increase the probability of fluorine ions getting into the channel and hence degrade the mobility of the 2DEG [START_REF] Yuan | Fluorine plasma ion implantation in AlGaN∕GaN heterostructures: A molecular dynamics simulation study[END_REF].

II. SIMULATION STRATEGY

In this work, a new normally-off MIS-HEMT is studied. Our proposed method is based on the implantation of fluorine below the channel layer. Although this technique has been previously studied with the HEMT [START_REF] Hamady | A new concept of enhanced-mode GaN HEMT using fluorine implantation in the GaN layer[END_REF], its potential in increasing the threshold voltage was limited by the voltage at which the Schottky gate opens. According to the use of the insulator below the gate, the limitations of the technique is studied. Moreover, the effect of the thickness of the insulator on the threshold voltage, and that for various fluorine concentrations, is pulled out.

To perform this study, ATLAS, a physically-based TCAD simulation tool from Silvaco is used. Physical models used in the simulation include Shockley-Read Hall recombination, Fermi-Dirac statistics and field-dependent mobility [START_REF]ATLAS User's Manual DEVICE SIMULATION SOFTWARE[END_REF]. The simulator was calibrated using a normally-on HEMT device as shown in figure 1. two dimensional electron gas, were tuned. Some of the parameters used during the simulation are shown in Table 1. The studied structure is shown in figure 3. The insulator used is Si 3 N 4 , with a relative permittivity of 7.5. The distance between the AlGaN/GaN interface and the implanted fluorine ions is equal to 10 nm. The thickness and doping of the AlGaN and GaN layers are taken the same as the normally-on HEMT used for calibration. All the device parameters remain constant throughout the simulation. However, the concentration of the implanted fluorine ions "F imp " and the thickness "d ins " of the insulator will be varied to analyse their effects on the threshold voltage.

III. RESULTS AND DISCUSSION

Although the aim of fluorine implantation is to achieve normally-off operation, it also can be used to reduce the offstate current of a normally-on MIS-HEMT. Figure 4 shows the transfer characteristics I ds (V gs ) for a MIS-HEMT with fluorine implanted below the channel at various concentrations. In the following simulations, the thickness d ins is equal to 10 nm. It can be seen that, with increasing the fluorine concentration, the threshold voltage increases and reaches positive values making the MIS-HEMT normally-off. Moreover, implanting fluorine ions helps to reduce the off-state current. It is worth noting that further increase in the fluorine concentration (F imp > 4 ×10 -2 cm -2 ) will not cause further reduction in the off-state current. The lowest off-state current reached is equal to that of the normally-on HEMT (the one used for calibration). In figure 5, the variations of the threshold voltage with F imp , at d ins = 10, 20 and 40 nm, is illustrated. It is clear that the threshold voltage increases with increasing the fluorine concentration. However, a point is reached (F imp = F optimum = 12 × 10 12 cm -2 ), where a further increase in the concentration will no longer affect the threshold voltage. To explain this limitation, the band diagram along the CUT of figure 3, is shown in figure 6. Introducing fluorine ions in a semiconductor will shift the Fermi level towards the valence band, thus increasing the gap between the conduction band and the Fermi level. However, since at equilibrium the Fermi level is constant, the increase in the gap will manifest itself through uplifting the conduction band. The uplift of the conduction band due to fluorine implantation will elevate the triangular well at the AlGaN/GaN interface. It is worth noting that the elevation of the triangular well, with respect to the Fermi level, causes the shift in the threshold voltage. Nonetheless, the gap between the Fermi level and the conduction band cannot surpass the bandgap of the material in which fluorine is implanted, which is GaN in our case. Therefore, the threshold voltage will keep on increasing as long as the gap between the conduction band and the Fermi level is less than the bandgap. Once the gaps are equal, fluorine ions will be incapable to increase the gap and hence lifting further the triangular well, which causes the increase in the threshold voltage.

It should be noted from figure 5, that all the curves corresponding to various d ins , intersect at a common point. The fluorine concentration associated with this point is named F critical . To further explore the effect of the thickness of the insulator on the threshold voltage, d ins is varied from 10 nm to 320 nm. The variation of the threshold voltage with the thickness of the insulator at various fluorine concentrations is shown in figure 7. It is clear that the variation of V th with d ins strongly depends on F imp . The concentration of fluorine F critical , at which the threshold voltage is not affected by the thickness of the insulator, is equal to 8.83 × 10 -12 cm -2 . Two cases are pulled out: for F imp < F critical , the threshold voltage decreases with increasing d ins , and for F imp > F critical , the threshold voltage increases with increasing d ins . The different variations in the threshold voltage with respect to the thickness of the insulator is directly related to the direction of the y-component of the electric field "E y " in the AlGaN layer, above the region where fluorine ions are implanted. When E y = 0, the threshold voltage will be unaffected by the thickness of the insulator. If E y is pointing towards the gate electrode, the threshold voltage will decrease with increasing the thickness of the insulator, and vice versa: if E y points towards the substrate the threshold voltage increases with increasing the thickness of the insulator. The direction of E y can be extracted from the profile of the conduction band shown in figure 6 (Ey ∝ -dE c /dy; y = depth). In our simulation, the positive y-direction points towards the substrate. When F imp = F critical = 8.83 × 10 12 cm -2 , the conduction band is constant and hence E y = 0. At F imp = 8 × 10 12 cm -2 < F critical the conduction band is decreasing ⇒ E y > 0 and at F imp = 1 × 10 13 cm -2 > F critical the conduction band is increasing ⇒ E y < 0.

IV. CONCLUSION

A sensitivity analysis is carried out for the normally-off MIS-HEMT with fluorine implanted below the channel. The limitation in the capability of fluorine implantation to increase the threshold voltage is addressed and explained. Moreover, the effect of the thickness of the insulator on the threshold voltage at various fluorine concentrations is studied. Simulation results have confirmed the ability of fluorine ions to increase the threshold voltage when implanted below the channel. However, a point is reached where further increase in the fluorine concentration will not affect the threshold voltage. This is due to the fact that the gap between the Fermi level and the conduction band becomes equal to the bandgap of the material in which fluorine is implanted. Moreover, the effect of the thickness of the insulator on the threshold voltage strongly depends on the fluorine concentration implanted. An increase in the thickness of the insulator will increase the threshold voltage if the concentration implanted is above F critical (in our simulation, F critical = 8.83 × 10 12 cm -2 ), decrease it if the concentration implanted is below F critical and the threshold voltage is unaffected if the concentration implanted is equal to F critical .

Fig. 1 .

 1 Fig. 1. Cross section of the normally-on HEMT used to calibrate the simulator.

Fig. 3 .

 3 Fig. 3. Cross section of the studied MIS-HEMT. Fimp and dins were varied during the simulation.

Fig. 4 .

 4 Fig. 4. Ids(Vgs) transfer characteristics of the MIS-HEMT at various fluorine concentrations for dins = 10 nm.

7 (

 7 Eg (AlxGaN) -Eg (GaN)) x-mole fraction of Aluminum in AlGaN "x" 0.25 -0.18 Band Gap of AlxGaN [eV] 6.2 x + 3.4 (1-x) -1.3 x (1 -x ) Dielectric Constant of AlxGaN [eV]

Fig. 5 .

 5 Fig. 5. Variations of the threshold voltage with the fluorine concentration at various thicknesses of the insulator.

Fig. 6 .

 6 Fig. 6. Band diagrams of MIS-HEMT at various fluorine concentrations. dins = 10 nm.

Fig. 7 .

 7 Fig. 7. Variations of the threshold voltage with the thickness of the insulator at various fluorine concentrations.

TABLE I .

 I PARAMETERS USED IN THE SIMULATION

Fig. 2. Comparison of the experimental and simulated Ids(Vgs) transfer characteristics of the conventional normally-on HEMT: fitting the simulated threshold voltage, transconductance and off-state current to the ones experimentally measured.

ACKNOWLEDGMENT

This work was funded by the Laboratory for Analysis and Architecture of Systems (LAAS -CNRS/France) and by GET/EDST/UL-AZM & SAADA association.


Saleem Hamady
email: shamady@laas.fr

Frédéric Morancho
email: morancho@laas.fr

Bilal Beydoun

Patrick Austin
email: austin@laas.fr

Mathieu Gavelle
email: mathieugavelle@yahoo.fr

Drain Silicon UID-GaN P-GaN Source Air Air Al x GaN Gate (d) P-GaN P-GaN 30 nm 30 nm 100 nm

2 µm Drain Silicon UID-GaN -doping 10 16 cm -3 1.1 µm 2 µm Gate Source 2 µm 5 µm 2 µm Air Air Al 0.25 GaN -doping 10 15 cm -3 30 nm Drain Silicon

INTRODUCTION

GaN HEMT seems to be a very promising candidate for future power applications. Thanks to the GaN properties and the HEMT's topology, AlGaN/GaN HEMTs are now potential candidates for high frequency applications with high power and low noise, such as microwave and millimeter wave communications, imaging and radars. While power switching applications strongly demand normally-off operation [START_REF] Kambayashi | High field-effect mobility normally-off AlGaN/GaN hybrid MOS-HFET on Si substrate by selective area growth technique[END_REF], conventional HEMTs are normally-on. Several normally-off structures have been proposed such as recessed gate structure [START_REF] Burnham | Gate-recessed normally-off GaN-on-Si HEMT using a new O2-BCl3 digital etching technique[END_REF], fluorine ion treatment device [START_REF] Chen | Self-aligned enhancement-mode AlGaN/GaN HEMTs using 25 keV fluorine ion implantation[END_REF] and Gate Injection Transistor (GIT) [START_REF] Uemoto | Gate Injection Transistor (GIT) -A Normally-Off AlGaN/GaN Power Transistor Using Conductivity Modulation[END_REF].

II. SIMULATION STRATEGY

In this work, we will start by simulating two normally-off HEMTs: HEMT with a p-GaN region directly below the gate (gate injection transistor -GIT) (figure 1b) and our newly proposed HEMT with a p-GaN region below the AlGaN/GaN interface (figure 1c). After pointing the advantages and drawbacks of these structures, a new design combining the two techniques (two p-GaN regions: one below the channel and another above it) (figure 1c) is studied. All the device parameters remain constant throughout the simulation, except for the doping concentration and the position of the p-GaN region. To do this study, ATLAS, a physically-based TCAD simulation tool from Silvaco is used. Physical models used in the simulation include Shockley-Read Hall recombination, Fermi-Dirac statistics and field-dependent mobility. The simulator was calibrated by using real parameters of a normally-on HEMT device.

Figure 1. Schematic cross section of the (a) conventional HEMT used for calibration, (b) GIT transistor, (c) our proposed HEMT with a p-GaN region below the channel and (d) GIT with p-GaN region below the gate. IN THE 3 FIGURES, YOU SHOULD REPLACE "SILICON" BY "TRANSITION LAYERS / SILICON"

III. RESULTS AND DISCUSSION

Figure 2 shows the simulated transfer characteristics of the conventional normally-on HEMT and GIT both with x-mole fraction of 0.15. In the GIT, two different p-doping concentrations were used: 10 18 cm -3 and 4×10 18 cm -3 . The thickness and the width of the p-GaN region is 100 nm and 1 μm respectively. It can be seen that the introduced p-GaN region, above the AlGaN layer, increases the threshold voltage. Moreover, the shift in the threshold voltage increases with increasing the doping concentration. At a doping concentration of 4×10 18 cm -3 , the threshold voltage increases from -1.1 V to 0.6 V as well as the forward gate voltage which increases from 3.2 V to 7 V.

Figure 2. Simulated transfer characteristics of the conventional normallyon HEMT and GIT with p-GaN doping concentrations of 10 18 cm -3 and 4×10 18 cm -3 .

Figure 3. Simulated transfer characteristics of the conventional normallyon HEMT and HEMT with a p-GaN region below the channel. The p-doping

concentration is 10 18 cm -3 .

Figure 3 shows the simulated transfer characteristics of the conventional normally-on HEMT and HEMT with a p-GaN region below the channel. The x-mole fraction in the two structures is equal to 0.15. The doping concentration, thickness and width of the p-GaN region are 10 18 cm -3 , 100 nm and 1 μm respectively. The distance between the p-GaN region and the AlGaN/GaN interface is equal to 30 nm. It can be seen that the introduced p-GaN region below the channel increases the threshold voltage from -2 V to 0.6 V. However, it decreases the forward gate voltage from 3.2 V to 0.9 V.

It is clear from figures 2 and 3 that although the same threshold voltage (0.6 V) was achieved after introducing the p-GaN region, the doping concentration required to achieve this threshold voltage is lower (10 18 < 4× 10 18 cm -3 ) when the p-GaN region is introduced below the channel rather than above it. On the other hand, the forward gate voltage is much higher when the p-GaN region is introduced above the channel. To achieve normally-off HEMT with relatively low p-doping concentrations and high forward voltage, a design combining the above mentioned techniques is proposed. Figure 4 shows the simulated transfer characteristics of an hybrid normally-off HEMT, with a threshold voltage of 2.5 V and a forward gate voltage of 3.8 V, using the GIT technique with a p-region below the channel. The doping concentration, thickness andwidth of the p-GaN region are 5×10 17 cm -3 , 100 nm and 1 μm respectively. The distance between the p-GaN region and the AlGaN/GaN interface is 30 nm.

IV. CONCULUSION

In this work, we address an advantage and a drawback of the two normally-off HEMT devices: the previously proposed Gate Injection Transistor (GIT) and our newly proposed HEMT with p-GaN region below the channel. Simulation results have shown that the doping concentration required to achieve the desired threshold voltage is lower when the p-GaN region is introduced below the channel rather than above it. On the other hand, the forward gate voltage is much higher when the p-GaN region is introduced above the channel. To achieve a normally-off HEMT with high forward gate voltage, using a relatively low p-doping concentration, an hybrid normally-off HEMT was proposed using two p-GaN regions, one above the channel and another below it.

Figure 4 .

 4 Figure 4. Simulated transfer characteristics of the conventional normally-on HEMT and the hybrid normally-off HEMT with a p-doping concentration of 5×10 17 cm -3 .


Towards a deeper judicial protection against floods in France

French judges play an important role in the field of flood risk management. Even if they cannot be considered as the main actors of flood risk management, they are developing, through legality reviews or actions for damages, an original judicial policy in favor of a more protective land planning.

Since the decentralization phenomenon initiated in the 1980s (see previous STAR-FLOOD column), local authorities are entrusted with specific competences in the field of land planning. Indeed, municipalities, public establishments have the power to develop their own land planning policy provided that they respect the national framework (adopted by the French Parliament) and some specific land documents considered being superior by the legislator. Yet, not all the powers in the field of land-planning have been decentralized to local authorities. The State is still competent in the field of natural risks to develop sectorial land-planning documents: the flood risk prevention plan (plan de prévention des risques d'inondation -PPRI). Those documents allow the State to declare a building area non-constructible because of high flood risk. For both local authorities and the State, the judge does a legality review limited to the manifest error of assessment (or manifest unreasonableness) in order to protect their discretionary powers in the elaboration of general and sectorial land-planning documents. The notion of manifest error of assessment can be defined as the judicial sanction of an obvious error invoked by the parties and recognized by the judge. For instance, without any deep legality check, the judge will sanction a local authority that granted a building permit in a high flood risk area, even if there is a PPRI over this area. This room to maneuver, in addition to the former powers decentralized in the 1980s, given to local authorities and to the representative of the State at the local level encourages the development of local land-planning policies. This judicial self-control limited to the manifest error of assessment can also be explained by the complexity and the heterogeneity of the data that must be taken into account by the local land planning documents 1 .

Yet, through a precise analysis of the case law in this specific field, one may see there are different appreciations of the notion of manifest unreasonableness depending on the facts of every case. For the PPRIs, judges have developed a hybrid legality review switching between a limited legality review and a normal one. In the aforementioned kind of review, judges realize a deeper analysis of the facts and their legal qualifications. This kind of judicial review is used when the plaintiff considers the delimitation as too permissive, which can be more dangerous in case of flood ii Thomas Thuillier is a PhD candidate in Public Law at the University François-Rabelais, Tours.

ii This column has been written for the STAR-FLOOD project, for « STrengthening And Rede-signing European FLOOD risk practices: Towards appropriate and resilient flood risk governance arrangements », which is a EU-funded research project focusing on analysing, explaining, evaluating and designing policies to better deal with flood risks from rivers in urban agglomerations across Europe. It has been published on their website on August 5, 2014.

disaster. In this particular case, the judge will use important investigatory powers (such as the use of experts) in order to appreciate the content of the document, the topography of the area, the nature of the risk, the instructions and the precautions taken by the authority (i.e. Lyon Administrative Court of Appeal, 29 April 2008, Ministry for Ecology and Sustainable Development, req. no 06LY01101). Even if the judge formally qualifies his judicial review as limited to the manifest unreasonableness, one may consider it as an in concreto control limited to the manifest unreasonableness, equal to a normal legality review. On the contrary, administrative courts realize a real legality review limited to the manifest unreasonableness when the administration considered an area as at-risk and restrain the construction there. In conclusion, when appreciating the legality of PPRIs, the judge seems more protective than the administration, and does not hesitate to annul the absence of instructions or non-constructability dispositions2 . Such a judicial approach cannot be used for general land-planning documents elaborated by local authorities. Indeed, at first sight, judges seem to use only a legality review limited to the manifest unreasonableness. It can be explained by the fact that those documents are much more general than the aforementioned one; they do not only take into account a specific natural risk as it is the case for PPRIs. Furthermore, general land planning documents such as Urban Local Plan (plan local d'urbanisme -PLU) or schéma de cohérence territoriale (-SCoT) elaborate a general landplanning policy for a local area. It seems to be more of a political document rather than the former ones that are more technical documents. Flood risk prevention is one of the main principles that have to be taken into account by the local authorities when elaborating their land-planning documents. They have to use a balanced approach in order to reconcile those very heterogeneous principles. Thus, the legality review used by the judge is de facto limited, since there is no hybrid review as it is the case for the PPRIs.

Judges can be considered as a way to ensure effective flood risk management. Even if this approach cannot be the only prism of assessment, it is important to keep in mind the importance of the role played by the judge in this particular issue.

Even if this column specifically focuses on the French case, such results could also be applied for other STAR-FLOOD countries. Thus, through an analysis of the judicial control in each flood risk management strategies (FRMS), the STAR-FLOOD consortium should consider (or not) judges Through an analysis of the judicial control in each flood risk management strategies (FRMS), the STAR-FLOOD consortium might consider (or not) judges as a specific actor allowing a better flood risk prevention and protection of the population.

DEBONO (A.-L.), « Le contrôle de légalité interne des plans de prévention des risques naturels », AJDA,

CALDERARO (N.), « Le juge administratif et les risques naturels »,RFDA, 2001, p. 895.