Student member, IEEE, S L Happy Onur Tasar

Member, IEEE Yuliya Tarabalka

Senior member, IEEE Pierre Alliez

ColorMapGAN: Unsupervised Domain Adaptation for Semantic Segmentation Using Color Mapping Generative Adversarial Networks

Keywords: Domain adaptation, semantic segmentation, dense labeling, convolutional neural networks, generative adversarial networks, GANs

Due to the various reasons such as atmospheric effects and differences in acquisition, it is often the case that there exists a large difference between spectral bands of satellite images collected from different geographic locations. The large shift between spectral distributions of training and test data causes the current state of the art supervised learning approaches to output unsatisfactory maps. We present a novel semantic segmentation framework that is robust to such shift. The key component of the proposed framework is Color Mapping Generative Adversarial Networks (ColorMapGAN), which can generate fake training images that are semantically exactly the same as training images, but whose spectral distribution is similar to the distribution of the test images. We then use the fake images and the ground-truth for the training images to fine-tune the already trained classifier. Contrary to the existing Generative Adversarial Networks (GANs), the generator in ColorMapGAN does not have any convolutional or pooling layers. It learns to transform the colors of the training data to the colors of the test data by performing only one element-wise matrix multiplication and one matrix addition operations. Thanks to the architecturally simple but powerful design of ColorMapGAN, the proposed framework outperforms the existing approaches with a large margin in terms of both accuracy and computational complexity.

I. INTRODUCTION

W ITH the continuous proliferation and improvement of satellite sensors, numerous new generation satellite missions have been created, which has made it possible to collect huge amounts of data. The massive satellite data have introduced new challenges to the remote sensing community. Semantic segmentation or dense labeling is the task of assigning a thematic label to each pixel in the image. Without a doubt, among the challenges that the remote sensing community is facing today, dense labeling of the satellite images is one of the most important one, as a good solution for this problem is of paramount importance to generate and automatically update the maps.

Over the last few years, convolutional neural networks (CNNs) have become the most commonly used tool for the task of semantic segmentation. In particular U-net [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF] In the ground-truth and in the predicted map, red, green, and white pixels correspond to building, tree, and road classes, respectively.

variants [START_REF] Iglovikov | Ternausnet: U-net with vgg11 encoder pre-trained on imagenet for image segmentation[END_REF]- [START_REF] Khalel | Multi-Task Deep Learning for Satellite Image Pansharpening and Segmentation[END_REF] are receiving a growing attention due to their great success in different domains such as medical imaging and remote sensing. The main limitation of the aforementioned CNNs is their extreme sensitivity to the training data. Although they perform very well when both training and test data come from the same distribution [START_REF] Maggiori | High-resolution aerial image labeling with convolutional neural networks[END_REF], their performance severely decreases when there is a large difference between spectral bands of the training and the test images. Considering that nowadays new generation satellites with a short revisit time acquire huge amounts of images from different parts of the world, one cannot assume that the distributions of the images are always similar. In addition, depending on when and where the data are collected, a large intra-class variability might be encountered in remote sensing images. For instance, color distributions of the same objects may greatly differ in the images captured in different times of the day as a result of the illumination difference. Similarly, because of the atmospheric effects, in some cases, even the images collected by the same satellite sensors might have very different radiometry, which makes the segmentation task even harder. Fig. 1 illustrates such an example, where color distributions of the training and the test data collected by the same satellite are noticeably different.

As shown in the figure, even a well trained U-net, which is considered as the state of the art in semantic segmentation of satellite images [START_REF] Huang | Large-scale semantic classification: outcome of the first year of inria aerial image labeling benchmark[END_REF], generates a poor map.

The unsupervised domain adaptation assumes that annotations for any part of the test data are not available, and aims at generating high quality segmentations even when there is a large domain shift between the training and the test images. In such a setting, in order to increase the generalization capabilities of the CNNs, one of the simplest and most common methods is to diversify the training data by applying various data augmentation techniques such as gamma correction and random contrast change [START_REF] Tasar | Incremental learning for semantic segmentation of large-scale remote sensing data[END_REF], [START_REF]Continual Learning for Dense Labeling of Satellite Images[END_REF]. However, even though these techniques help the CNNs to generalize slightly better, when spectral difference between the training and the test data is huge, the improvement is usually insufficient. Although a good solution for domain adaptation is indispensable for various applications, this problem has not been investigated intensively in the field of remote sensing. The limitations pointed out in this section have motivated us to design a methodology for this problem.

A. Related Work

In this section, we summarize the existing approaches on domain adaptation in the fields of computer vision and remote sensing.

Computer Vision: The majority of the proposed approaches in the literature are based on the idea of aligning distributions of both training and test images in a common space using generative adversatial networks (GAN) so that the trained classifier could segment the test images well. FCNs in the Wild [START_REF] Hoffman | Fcns in the wild: Pixel-level adversarial and constraint-based adaptation[END_REF] reduces the global distribution difference between the training and the test images by minimizing an adversarial loss. Aligning the distributions of the extracted features from both the training and the test images is also a common approach [START_REF] Sankaranarayanan | Learning from synthetic data: Addressing domain shift for semantic segmentation[END_REF], [START_REF] Murez | Image to image translation for domain adaptation[END_REF]. The distribution alignment could also be performed in multiple layers of the network instead of only the final output [START_REF] Tsai | Learning to adapt structured output space for semantic segmentation[END_REF], [START_REF] Huang | Domain transfer through deep activation matching[END_REF]. Hong et al. add random noise to the training images [START_REF] Hong | Conditional generative adversarial network for structured domain adaptation[END_REF]. They observe that perturbing the training images helps the GAN to adapt better to the test data. Saito et al. introduces a maximum classifier discrepancy based approach, which alings the distributions by making use of class-specific decision boundaries [START_REF] Saito | Maximum classifier discrepancy for unsupervised domain adaptation[END_REF]. The framework presented by Chen et al. utilizes Google Street View to collect unannotated images and to use their features. Curriculum Domain Adaptation (CDA) [START_REF] Zhang | Curriculum domain adaptation for semantic segmentation of urban scenes[END_REF] performs the alignment both globally and on the generated super-pixels. Reality Oriented Adaptation (ROAD) [START_REF] Chen | Road: Reality oriented adaptation for semantic segmentation of urban scenes[END_REF] comprises two loss functions. The first one forces the weights of the trained model from the training images with the weights of the pre-trained model from ImageNet [START_REF] Russakovsky | Imagenet large scale visual recognition challenge[END_REF] to be as similar as possible. The second one handles the spatial aware domain adaptation by dividing the training and the test images into grids and minimizing distance between the grids of the training and the test images.

There have been attempts to solve the domain adaptation problem by regularizing or normalizing specific layers of the network, or by self-learning. The Fully Convolutional Tribranch network (FCTN) [START_REF] Zhang | A fully convolutional tri-branch network (fctn) for domain adaptation[END_REF] is one of the methods that falls into the category of self-learning. The presented network architecture consists of one encoder and three decoders. Two of the decoders pseudo-label the test image, the other one learns from the pseudo-labels and the test image. A classbalanced self learning approach is proposed in CBST [START_REF] Zou | Unsupervised domain adaptation for semantic segmentation via class-balanced self-training[END_REF]. Introducing new normalization method, regularization technique, or new loss functions that are specific for domain adaptation problem is invesigated in [START_REF] Romijnders | A domain agnostic normalization layer for unsupervised adversarial domain adaptation[END_REF]- [START_REF] Zhu | Penalizing top performers: Conservative loss for semantic segmentation adaptation[END_REF]. Romijnders et al. discuss the limitations of the traditional normalization methods such as batch normalization, and propose a new domain agnostic normalization layer that is more suitable for domain adaptation [START_REF] Romijnders | A domain agnostic normalization layer for unsupervised adversarial domain adaptation[END_REF]. Saito et al. introduce a new adversarial dropout regularization technique [START_REF] Saito | Adversarial dropout regularization[END_REF]. The IBN-Net [START_REF] Pan | Two at once: Enhancing learning and generalization capacities via ibn-net[END_REF] combines the batch normalization with the instance normalization [START_REF] Ulyanov | Instance normalization: The missing ingredient for fast stylization[END_REF] to increase the generalization capability. Zhu et al. proposes a new conservative loss [START_REF] Zhu | Penalizing top performers: Conservative loss for semantic segmentation adaptation[END_REF].

Another way to approach the unsupervised domain adaptation problem is to perform image to image translation (I2I) between a source and a target domain. The I2I approaches aim at generating fake source images that are statistically indistinguishable from the target images. The existing I2I approaches in the literature can perform one to one [START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF], [START_REF] Liu | Unsupervised image-to-image translation networks[END_REF] or many to many [START_REF] Huang | Multimodal unsupervised image-to-image translation[END_REF], [START_REF] Lee | Diverse image-to-image translation via disentangled representations[END_REF] translations. If one can generate fake training images, which are style-wise consistent with the test images and semantically consistent with the original training images, the fake images could be used to train a model from scratch or to fine-tune the already trained model on the original data. The main drawback here is that usually the fake training images generated by the I2I approaches contain artificial objects and artifacts, which do not exist in the original images. Hence, annotations for the original training images and the generated fake training images do not match. As a result, the model learns wrong information. To overcome this limitation, CyCADA [START_REF] Hoffman | Cycada: Cycle-consistent adversarial domain adaptation[END_REF] segments the original and the fake training images using the classifier trained on the original training data, and minimizes cross entropy loss between the segmentations. However, if the domain shift between the training and the test data is large, the classifier trained from the original training images will segment the fake training image poorly. If the segmentation for the training images is very good, but the predicted map for the fake training images is extremely noisy, we cannot expect adding such a loss to prevent artificial objects appearing. Another way to enrich the training data could be to perform neural style transfer, where content of an image and style of another image are combined [START_REF] Gatys | Image style transfer using convolutional neural networks[END_REF]- [START_REF] Ulyanov | Texture networks: Feed-forward synthesis of textures and stylized images[END_REF]. However, they also cannot guarantee that the semantic structures will be preserved.

Remote sensing: As thoroughly explained in the overview paper by Tuia et al. [START_REF] Tuia | Domain adaptation for the classification of remote sensing data: An overview of recent advances[END_REF], the domain adaptation methods in the field of remote sensing can be divided into four categories: selection of invariant features, adaptation of classifiers, active learning, and adaptation of the data distributions. The main goal of the methods falling into the first category is to find a subset of features from the training data, which is representative for the test data. For instance, Bruzzone et al. present a methodology [START_REF] Bruzzone | A novel approach to the selection of spatially invariant features for the classification of hyperspectral images with improved generalization capability[END_REF] aiming at selecting the features that are both invariant to the test data and discriminative for the classes of interest by defining a new criterion function quantifying both tasks. Persello et al. describe a method that solves the same selection problem by a kernel based method [START_REF] Persello | Kernel-based domain-invariant feature selection in hyperspectral images for transfer learning[END_REF].

In order to adapt the classifier to the test data, the existing approaches usually either choose to adapt the classifier to the unlabeled test data directly, or favor an active learning approach, where the annotator labels a small number of representative samples from the test data to update the already trained classifier. Bruzzone et al. describe an approach that updates the parameters of the previously trained classifier on the unlabeled test data via the expectation-maximization (EM) algorithm [START_REF] Bruzzone | Unsupervised retraining of a maximum likelihood classifier for the analysis of multitemporal remote sensing images[END_REF]. This approach has then been extended to another framework, where a cascade classification operation is performed [START_REF]A partially unsupervised cascade classifier for the analysis of multitemporal remote-sensing images[END_REF]. In [START_REF] Bruzzone | A multiple-cascade-classifier system for a robust and partially unsupervised updating of land-cover maps[END_REF], multiple cascade classifiers are used. Bruzzone and Marconcini propose the domain adaptation support vector machine (DASVM) [START_REF] Bruzzone | Domain adaptation problems: A dasvm classification technique and a circular validation strategy[END_REF]. If adapting the classifier by an active learning approach is preferred, we can obviously expect to get better results, since some portion of the test data is provided to modify the classifier. On the other hand, the selection of the samples and labeling them manually can be too costly. In the approach proposed by Persello and Bruzzone [START_REF] Persello | Active learning for domain adaptation in the supervised classification of remote sensing images[END_REF], the classifier iteratively learns from a small number of newly added samples from the test data and removes some of the training samples whose distribution does not fit with the distribution of the test data. The SVM based method presented by Matasci et al. also follows a similar approach [START_REF] Matasci | Svm-based boosting of active learning strategies for efficient domain adaptation[END_REF]. Another kernel based active learning approach is introduced by Deng et al. [START_REF] Deng | Active multi-kernel domain adaptation for hyperspectral image classification[END_REF]. The usage of neural networks for active learning is studied in [START_REF] Ghassemi | Learning and adapting robust features for satellite image segmentation on heterogeneous data sets[END_REF].

The adaptation of the data distributions is a common method. To do this, one can either align the distributions of the training and the test data in a common space, or align the distribution of the training data to the distribution of the test data. In both cases, we expect the model trained from the aligned training data to generate better segmentation. The aforementioned alignment can be performed by histogram matching [START_REF] Inamdar | Multidimensional probability density function matching for preprocessing of multitemporal remote sensing images[END_REF], [START_REF] Gonzalez | Digital Image Processing[END_REF], graph matching [START_REF] Tuia | Graph matching for adaptation in remote sensing[END_REF], kernel principal component analysis (KPCA) [START_REF] Nielsen | Kernel principal component and maximum autocorrelation factor analyses for change detection[END_REF], color constancy algorithms [START_REF] Agarwal | An overview of color constancy algorithms[END_REF], or minimizing the statistical distance between the training and the test data [START_REF] Matasci | Semisupervised transfer component analysis for domain adaptation in remote sensing image classification[END_REF] by the maximum mean discrepancy (MMD) [START_REF] Matasci | Understanding angular effects in vhr imagery and their significance for urban land-cover model portability: A study of two multi-angle intrack image sequences[END_REF]. Ma et al. correct the domain shift between the training and the test data for each class by centroid and covariance alignment [START_REF] Ma | Centroid and covariance alignment-based domain adaptation for unsupervised classification of remote sensing images[END_REF]. Gross et al. propose a nonlinear feature normalization method for the alignment [START_REF] Gross | Nonlinear feature normalization for hyperspectral domain adaptation and mitigation of nonlinear effects[END_REF]. A new methodology performing tensor alignment is presented in [START_REF] Qin | Tensor alignment based domain adaptation for hyperspectral image classification[END_REF]. Recently, Courty et al. have used the optimal transport for domain adaptation [START_REF] Courty | Domain adaptation with regularized optimal transport[END_REF], [START_REF] Courty | Optimal transport for domain adaptation[END_REF]. The optimal transport for domain adaptation of remote sensing data is studied in [START_REF] Tardy | Assessment of optimal transport for operational land-cover mapping using high-resolution satellite images time series without reference data of the mapping period[END_REF]. Besides, after GANs had become popular in the field of computer vision, some papers have studied their application to remote sensing. For instance, Benjdira et al. [START_REF] Benjdira | Unsupervised domain adaptation using generative adversarial networks for semantic segmentation of aerial images[END_REF] use CycleGAN [START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF] to generate fake training data that resemble test data. The generated fake training data are then used to fine-tune the already trained model from the original training data. However, unlike the computer vision benchmarks, remote sensing images contain a lot of heterogeneous and complex structures. As a result, the quality of the fake remote sensing images generated by CycleGAN is usually not as good as desired.

B. Contributions

We study the problem of unsupervised domain adaptation for semantic segmentation, where training and test images are collected from completely different geographic locations, they have significant spectral distribution difference, and annotations for any part of the test images are not available. The way we approach the problem methodologically resembles to the image to image translation (I2I) approaches; we generate fake training images as if they came from the distribution of the test images. Instead of working on images acquired over separate geographic extents, if we had paired training and test images (e.g., images acquired from exactly the same geographic extent but in different day time), the problem would be easier, we could perform paired I2I [START_REF] Isola | Image-to-image translation with conditional adversarial networks[END_REF].

The main challenge here is to generate fake training images that look like test images without impairing the semantic meaning in the original training images, even when the training and the test images are unpaired. If the semantic structures of the original training images are preserved in the fake training images, we can utilize the annotations for the original training images as well as the fake training images to fine-tune the already trained model from the original training data. When generating fake training images with the style of the test images, most of the I2I approaches presented in Sec. I-A fail to preserve the exact structures of the original training images, especially when we deal with satellite images containing a lot of complex and heterogeneous objects. Thus, the fake training images and the annotations do not mach, which leads the model to learn incorrect information.

Contributions of this work are as follows:

1) ColorMapGAN: Our main contribution is novel Col-orMapGAN, which can generate fake training images that are semantically exactly the same as the original training images (i.e., location and shape of the objects such as roads, building, trees, etc. are exactly the same in the fake and in the original training images) and that are visually indistinguishable from the test images (i.e., the objects such as trees and buildings in the fake training images and the test images have similar spectral distributions). To do this, ColorMapGAN transforms the colors of the training images into the colors of the test images without doing any structural changes on the objects of the training images.

2) Higher accuracy: In our experiments, we perform city to city domain adaptation. We utilize the fake training images generated by our approach and by CycleGAN [START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF], UNIT [START_REF] Liu | Unsupervised image-to-image translation networks[END_REF], MUNIT [START_REF] Huang | Multimodal unsupervised image-to-image translation[END_REF], DRIT [START_REF] Lee | Diverse image-to-image translation via disentangled representations[END_REF], histogram matching [START_REF] Gonzalez | Digital Image Processing[END_REF], and gray world algorithm [START_REF] Buchsbaum | A spatial processor model for object colour perception[END_REF] to fine-tune the model trained from the original training images. We also compare our approach with AdaptSegNet [START_REF] Tsai | Learning to adapt structured output space for semantic segmentation[END_REF], which aims at adapting the classifier to the test data directly. Despite its architecturally simple design, our framework enables to generate predicted maps with much higher accuracy than the others.

3) Lower complexity: Unlike the existing GANs in the literature, our generator does not perform any convolution or pooling operations. It transforms the colors of the training images with only one element-wise matrix multiplication and one matrix addition. Because our generator is architecturally substantially simpler compared to the existing GANs, the training time of ColorMapGAN is significantly lower than the other learning based approaches. In our experiments, we compare our approach with CycleGAN [START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF], UNIT [START_REF] Liu | Unsupervised image-to-image translation networks[END_REF], MUNIT [START_REF] Huang | Multimodal unsupervised image-to-image translation[END_REF], DRIT [START_REF] Lee | Diverse image-to-image translation via disentangled representations[END_REF], histogram matching [START_REF] Gonzalez | Digital Image Processing[END_REF], and gray world algorithm [START_REF] Buchsbaum | A spatial processor model for object colour perception[END_REF] in terms of running times.

II. GENERATIVE ADVERSARIAL NEURAL NETWORKS

In machine learning, we can divide the models trained in a supervised setting into two groups: discriminative and generative models. In the field of image analysis, the discriminative models are usually trained to learn a mapping from a high dimensional input to class labels as in image categorization and segmentation problems. On the other hand, the generative models aim to estimate the distribution of the data samples so that new samples could be drawn from the estimation. In 2014, Goodfellow et al. proposed the generative adversarial networks (GANs) [START_REF] Goodfellow | Generative adversarial nets[END_REF], which is a novel approach to train a generative model.

GANs usually comprise a generative model G and a discriminative model D. The goal of G is to estimate the distribution of the real data and to output fake data from the estimation. G takes a random noise z as input, and represents a mapping to data space G(z). We denote the distribution of the real data x by p(x) and a prior on input noise variables by p(z). Let us assume that the real data x and the fake data G(z) are indicated by 1 and 0, respectively. D outputs a scalar between 0 and 1, and aims to maximize the probability of labeling x and G(z) correctly. In other words, the goal of D is to discriminate between the real and the fake data. The objective function for D that is maximized during training is described as:

max D V (D) = E x∼p(x) log[D(x)] + E z∼p(z) [log(1-D(G(z)))],
(1) where E is the expected value. G is simultaneously trained to minimize the objective function defined as:

min G V (G) = E z∼p(z) [log(1 -D(G(z)))]. (2)
As a result, the minimax game played between G and D could be formulated as:

min G max D V (D, G) = E x∼p(x) logD(x) + E z∼p (z) log(1 -D(G(z))). (3)
Once D and G are simultaneously trained for a sufficiently long time, D becomes good at discriminating the fake and the real data, and G becomes better at generating fake data that are indistinguishable from the real data.

Although GAN works well with shallow multi-layer perceptrons, they suffer from instability problems during training when a more complex network is used. Several approaches have been presented to address the instability issues. In DCGAN [START_REF] Radford | Unsupervised representation learning with deep convolutional generative adversarial networks[END_REF], instead of multilayer perceptrons, deep convolutional networks are used in both G and D, and certain architectural constraints are introduced for a more stable training. In WGAN [START_REF] Arjovsky | Wasserstein gan[END_REF], rather than the logarithms in Eq. ( 3), Wasserstein distance is used to compute distance between the distributions, and gradient clipping is applied in the training stage. WGAN-GP [START_REF] Gulrajani | Improved training of wasserstein gans[END_REF] is an extension of WGAN, where a gradient penalty is performed to solve the limitations of the gradient clipping. LSGAN [START_REF] Mao | Least squares generative adversarial networks[END_REF] proves that adopting the least squares loss function in Eq. ( 3) allows more stabilized training.

Finally, the original GANs were extended to conditional GANs in [START_REF] Mirza | Conditional generative adversarial nets[END_REF], where instead of generating the fake data from noise z, both G and D are conditioned on some extra information y. y can be class labels or data from other modalities. In this architecture, G learns a mapping from combination of z and y to the data space. If we consider y as the source data and x as the target data, G aims to learn a mapping from source domain to target domain. Inspired from this idea, conditional GANs have been used for several I2I works [START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF], [START_REF] Isola | Image-to-image translation with conditional adversarial networks[END_REF]. We use a slighly modified version of U-net [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF] as the classifier. We replace rectified linear activation units (ReLU) by leaky rectified linear activation units (Leaky-ReLU) for a better performance [START_REF] Xu | Empirical evaluation of rectified activations in convolutional network[END_REF]. We also remove the batch normalization operation in each layer, since it uses the memory inefficiently. In the framework, the steps 1, 3, and 4 are self-explanatory, whereas step 2 needs further explanation.

III. METHODOLOGY

A. The Overall Framework

B. ColorMapGAN 1) Generator:

The novelty of the proposed ColorMapGAN is in the architecturally simple but powerful design of its generator.

We denote a set of training image patches by X = {x 1 , x 2 , . . . , x N } and a set of test image patches by Y = {y 1 , y 2 , . . . , y M }. G(X) corresponds to the set of fake training patches generated by G. The goal of G is to generate G(X), whose spectral distribution is as similar as possible to the distribution of Y , while keeping G(X) and X semantically exactly the same. Contrary to the existing GANs in the literature, we do not use convolutional or pooling layers in G to preserve the exact semantics of X in G(X).

Let us assume that X and Y are composed of 8 bit images comprising red, green, and blue channels. We denote by R = {0, 1, . . . , 255}, G = {0, 1, . . . , 255}, and B = {0, 1, . . . 255}, the values each color band of the pixels can take. We denote all the possible 16, 777, 216 (256×256×256) colors by RGB, which is defined as:

RGB = R × G × B, (4)
where × stands for the Cartesian product. In order to trans- form RGB to another color matrix R G B , we use a scale W and a shift K matrices with the same size as RGB. R G B could be computed as:

R G B = RGB • W + K, ( 5
)
where • is the element-wise product. The only learnable parameters of our G are W and K. Before the training starts, we initialize W with ones and K with zeros. Hence, at the end of the first iteration, the input and the output of G are exactly the same.

The main bottleneck for computing Eq. ( 5) is the computational complexity. Since each of RGB, W , and K matrices has more than 50 millions of elements (256 × 256 × 256 × 3), it is not feasible to perform the operation defined in Eq. ( 5) on a GPU. However, the number of colors in a training image patch is much lower than the number of all the possible colors. Therefore, it is sufficient to update only the elements of W and K which transform the colors that are available in the training patch. To do this, we use an index vector I that is defined as:

I = r × 256 × 256 + g × 256 + b, (6)
where r, g, b are red, green, blue values of all the pixels in the training patch. After the elements of I are found, we normalize and center each x i ∈ X and y j ∈ Y by first dividing by 127.5 and then subtracting 1 so that each color channel ranges between -1 and 1. We then partially update R G B as:

R G B [I] = RGB[I] • W [I] + K[I], (7)
where [•] operation corresponds to retrieving the rows of an arbitrary 2D matrix indexed by the given vector. Then, in R G B [I], we replace the elements that are bigger than 1 and smaller than -1 by 1 and -1, respectively. To range all the values in R G B [I] between 0 and 255, we then use the denormalization function DN that is defined as: 2) Discriminator: The discriminator in ColorMapGAN is architecturally the same as the discriminator of CycleGAN [START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF] (see Fig. 4). Instead of outputting a single scalar for the whole image patch to determine if the patch is real or fake, this discriminator generates a two dimensional matrix. Each element of the matrix is used to locally determine whether the input patch is real or fake. We then take average of all the elements of the matrix to yield a final value.

DN (p) = (p + 1) × 127.5 , (8)
As we mention in Sec. II, GANs suffer from the instability issues; therefore, other objective functions have been proposed as an alternative to Eqs. 1, 2, and 3. We prefer to use the functions presented in LSGAN [START_REF] Mao | Least squares generative adversarial networks[END_REF]. Thus, for D, the objective function that is minimized during training becomes:

min D V (D) = E x∼p(x) [(D(x) -1) 2 ] + E y∼p(y) [(D(G(y))) 2 ].
(9) The objective function for G is defined as:

min G V (G) = E y∼p(y) [(D(y) -1) 2 ]. ( 10
)
We train D and G simultaneously by minimizing the Eqs. 9 and 10.

IV. EXPERIMENTS

A. Methods Used for Comparison U-net [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF]: We simply train a U-net from the training images and segment the test images without performing any type of domain adaptation techniques.

CycleGAN [START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF]: In this methodology, in addition to G, there is one more generator F . G is used to learn a mapping from training images to test images, whereas F learns a mapping from test images to training images. The methodology requires that the mapping from training images to test images and another consecutive mapping from test images to training images reproduce the original training images. This constraint is enforced by minimizing the L1 norm between F (G(X)) and X, and between G(F (Y )) and Y .

UNIT [START_REF] Liu | Unsupervised image-to-image translation networks[END_REF]: The generator of UNIT has two encoders E x and E y and two decoders G x and G y . The encoders are used to embed X and Y to a common space. The fake images are generated by G x (E y (Y )) and G y (F x (X)). In an ideal transformation, X and G x (E y (Y )), and Y and G y (E x (X)) have similar statistics.

MUNIT [START_REF] Huang | Multimodal unsupervised image-to-image translation[END_REF]: It decomposes the images from both domains into content and style codes. To generate fake images, the content code of one domain and the style code of another domain are combined. AdaIN [START_REF] Huang | Arbitrary style transfer in real-time with adaptive instance normalization[END_REF] is utilized for the combination.

DRIT [START_REF] Lee | Diverse image-to-image translation via disentangled representations[END_REF]: Methodologically DRIT is almost the same as MUNIT. The only difference is that the content code of one domain and the style code of another domain are combined by concatenating them.

Histogram matching [START_REF] Gonzalez | Digital Image Processing[END_REF]: For each color channel, we match the histogram of the training images with the histogram of the test images to correct the spectral shift between the images.

Gray world [START_REF] Buchsbaum | A spatial processor model for object colour perception[END_REF]: It is one of the color constancy algorithms [START_REF] Agarwal | An overview of color constancy algorithms[END_REF]. This algorithm assumes that the average color of the image should be natural gray, and any deviation from gray is caused by the illuminant. This assumption is used to remove the effect of the illuminant. We use gray world algorithm to standardize the training and the test data.

AdaptSegNet single [START_REF] Tsai | Learning to adapt structured output space for semantic segmentation[END_REF]: It aims at training a domain invariant network that performs well in segmenting both training and test images. To do this, the classifier generates predicted maps from the training and the test images, and the discriminator forces the predicted maps for the test images to look like the predictions for the training images. In the original paper, DeepLab v2 [START_REF] Chen | Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs[END_REF] is used as the classifier. However, Atrous Spatial Pyramid Pooling (ASPP) in this network reduces the segmentation performance on satellite images significantly, especially when the image contains objects covering a small area. Hence, we remove ASPP from the network and upsample the final classification layer directly to get the predicted map with exactly the same size as the training images.

AdaptSegNet multiple [START_REF] Tsai | Learning to adapt structured output space for semantic segmentation[END_REF]: In the same paper, in addition to aligning the final predictions for the training and test images, the experimental results with 2 classification layers and 2 discriminators are also presented. We compare our method with this strategy as well.

To make a fair comparison between ColorMapGAN and CycleGAN, UNIT, MUNIT, DRIT, histogram matching, and gray world, we replace the step 2 in our framework with these algorithms. For gray world, we also modify the step 4. Instead of segmenting the original test image, we segment the test image in which the illuminant effect is removed.

B. Experimental Setup

To conduct our experiments, we use the Luxcarta data set containing Pléiades images acquired over 4 cities in Europe: Bad Ischl (Austria), Villach (Austria), Béziers (France), and Roanne (France). The images are converted to 8 bit, and their spatial resolution is reduced to 1m by the data set providers. The images in the data set contain red, green, and blue channels. The full annotations for building, road, and tree classes are provided. An example close-up from each city is shown in Fig. 5. We split the cities into two pairs, where the first pair consists of Bad Ischl and Villach, and the second pair comprises Béziers and Roanne. To make the experimental setup suitable for the unsupervised domain adaptation problem, when we split the cities into pairs, we pay attention that radiometry of both cities in each pair is as different as possible, and the objects belonging to the same class (e.g., building) have similar structural characteristics. For instance, buildings in Béziers and Roanne are densely grouped and have mostly rectangular shape, whereas buildings in Bad Ischl and Villach are more sparsely distributed and mostly square-like shaped. However, as can be seen in Fig. 5, there is a large domain shift between spectral bands of the cities in each pair. Our final assumption is that we have access to annotations of only the training city. For instance, when we classify Roanne in pair 2, we suppose that its annotations are not available; only the ground-truth of Béziers is accessible. Similarly, to classify Béziers, we assume that the ground-truth of only Roanne is accessible.

In the preprocessing step, we split each satellite image into 256 × 256 training patches with an overlap of 32 pixels between neighboring patches. Table I reports for each city the number of patches, the total area covered, and the class frequencies. For the quantitative performance assessment, we use Intersection over Union (IoU) [START_REF] Csurka | What is a good evaluation measure for semantic segmentation?[END_REF] as the evaluation metric.

C. Training Details

In the first step of the framework, we train a U-net with Adam optimizer, where the learning rate is 0.0001, exponential decay rate for the first and the second moment estimates are 0.9 and 0.999. In each iteration, we randomly sample a batch of 32 training patches. We apply online data augmentation with random horizontal/vertical flips and 0/90/180/270 degrees of rotations. We use sigmoid cross entropy as the loss function and ignore the background class while computing the loss. We optimize the network for 2,500 iterations.

In the stage of generating fake training images, in each training iteration of ColorMapGAN, we randomly sample only one patch from the training city and one patch from the test city. We use Adam optimizer to update both the generator and the discriminator. Since the generator is architecturally much simpler than the discriminator, we prefer to optimize it with a larger learning rate. For the generator, the learning rate is 0.0005, whereas we set it to 0.0001 for the discriminator. We train ColorMapGAN for 8,000 iterations, since we verify by visual inspection that visually appealing results are obtained for this number of iterations. We fine-tune the previously trained network on the fake training images for 750 iterations. For all the compared methods, we use default parameters that are specified in the related papers.

D. Results

Tables II and III depict IoU values for each method on the test set. To provide as reliable as possible results, we repeat the step 3 in our framework 20 times for each method that generates fake data, and report the average IoU values in the tables. Re-running the step 3 of the framework 20 times is feasible, since we fine-tune the classifier for only 750 iterations in this step. However, AdaptSegNet tries to adapt the classifier to the test data directly. Hence, we need to train a classifier from scratch every time when we repeat the experiment. For this reason, for AdaptSegNet Single and Multi, we show the average IoU values for only 3 runs. Figs. 14 to 17 depict the predictions. Because we run the experiments for each method multiple times, we illustrate the predictions that are obtained by majority voting. In these figures, CycleGAN, UNIT, MU-NIT, DRIT, Gray world, Hist. match., and ColorMapGAN represent the results for our framework with these method. We do not add "our framework with" statement in the captions for the sake of simplicity.

Figs. 6 to 9 illustrate some parts of the original training images from the pairs and the fake training images generated by CycleGAN, UNIT, MUNIT, DRIT, gray world, histogram matching, and ColorMapGAN. From the images, we can clearly see that MUNIT and DRIT spoil the semantic identity of the images completely. For instance, the structures indicated by yellow and green rectangles in the original images are either replaced by other objects or distorted in the fake images. In some cases, UNIT seems to generate better results. For instance, the fake cities generated by UNIT in pair 2 are semantically relatively consistent with the original images, and style-wise similar to the test cities. On the other hand, the fake cities in pair 1 have plenty of artificial objects that do not exist in the original city. For UNIT, MUNIT, and DRIT, since the fake images and the ground-truth for the original images usually do not match, the network learns wrong information in the step 3 of our framework. Another problem with these approaches is that we observe tiling effect in the fake images. Especially in the fake Villach images, the transition between the patches is not continuous. The reason is that since it is impossible to fit the large satellite images to GPU directly when generating fake images; we need to generate fake data from each patch and combine them to get the entire fake city. However, these approaches mostly generate irrelevant output for the neighboring patches. As a result, the proposed framework with these methods perform poorly on the test data, as confirmed by Tables II, III and Figs. 14 to 17.

As can be seen in Figs. 6 to 9, the spectral difference between the training and the test images can be reduced by standardizing them using the gray world algorithm. However, between the fake images in each city pair, we still observe a spectral shift; it is not completely corrected. In consequence, the performance of gray world algorithm is mostly better than UNIT, MUNIT, and DRIT, but it is not as good as desired.

The network architecture used in AdaptSegNet single is very deep; therefore, aligning only the outputs of the final classification layer for the training and the test images does not yield a good performance. However, if the alignment is performed in multiple layers, a better performance could be obtained, especially when the objects of interests cover a large area such as forests. For instance, most of the trees in Villach are located in forests areas. For tree class on this city, IoU value for AdaptSegNet multi is 56.08%, which is slightly lower than the performance of our framework. However, the performance of AdaptSegNet is not satisfactory in segmenting small objects such as building and thin objects like roads.

At the first sight, histogram matching seems to be working well; semantic structures of the training city are well preserved in the fake training city, and the style of the test city is perfectly transferred to the fake training city. However, it is noticeable in Tables II and III that the quantitative results for this approach are very poor most of the time. Besides, the fake cities generated by CycleGAN and ColorMapGAN in pair 1 look similar. However, still there exists a large gap between the performance of the framework with ColorMapGAN and with CycleGAN. When segmenting Villach, for road class, the IoU of CycleGAN is around 9% lower than of ColorMapGAN. Similarly, ColorMapGAN outperforms CycleGAN by 6% for building class, when segmenting Bad Ischl. On the other hand, CycleGAN outperforms ColorMapGAN for tree class. To better understand the reasons for this performance difference, the comparisons between ColorMapGAN and CycleGAN, and ColorMapGAN and histogram matching need further analysis. (a) B éziers (b) CycleGAN [START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF] (c) UNIT [START_REF] Liu | Unsupervised image-to-image translation networks[END_REF] (d) MUNIT [START_REF] Huang | Multimodal unsupervised image-to-image translation[END_REF] (e) DRIT [START_REF] Lee | Diverse image-to-image translation via disentangled representations[END_REF] (f) Gray world [START_REF] Buchsbaum | A spatial processor model for object colour perception[END_REF] (g) Histogram matching [START_REF] Gonzalez | Digital Image Processing[END_REF] (h) ColorMapGAN (ours) (a) Roanne (b) CycleGAN [START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF] (c) UNIT [START_REF] Liu | Unsupervised image-to-image translation networks[END_REF] (d) MUNIT [START_REF] Huang | Multimodal unsupervised image-to-image translation[END_REF] (e) DRIT [START_REF] Lee | Diverse image-to-image translation via disentangled representations[END_REF] (f) Gray world [START_REF] Buchsbaum | A spatial processor model for object colour perception[END_REF] (g) Histogram matching [START_REF] Gonzalez | Digital Image Processing[END_REF] (h) ColorMapGAN (ours) ColorMapGAN vs CycleGAN [START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF]: First of all, it is worth noting that the performance of CycleGAN is unstable. The proposed framework with CycleGAN performs unsatisfactorily on pair 2 because of its semantically inconsistent outputs with the original images. As highlighted by yellow rectangles in Figs. 8 and9, CycleGAN removes some objects that exist in the original cities. There are several reasons why it performs worse than ColorMapGAN on pair 1 for building and road classes. Firstly, the resolution of its output is lower than the resolution of the original city and the output of ColorMapGAN. Fig. 10 illustrates a closeup from Villach and the corresponding fake images generated by ColorMapGAN and CycleGAN. The resolution difference between the fake images can easily be noticed in the outlined areas by cyan rectangles. Learning from blurrier data obviously deteriorates the performance. Secondly, the output of CycleGAN has some artifacts as shown in the same figure by a yellow rectangle. The generator of ColorMapGAN does not have convolution, pooling, etc. operations; therefore, we do not observe such artifacts in the outputs of ColorMapGAN. Finally, since we generate fake cities patch by patch because of memory constraints, there exists a spectral difference between some of the neighboring patches in the fake images generated by CycleGAN (see the green rectangle in Fig. 6b). This difference leads the network to exhibit a lower performance. ColorMapGAN does not have this problem, because it maps each color to another one. The pixels having the same color are mapped to another exactly the same color, irrespective of their locations. Therefore, the neighboring patches are spectrally consistent, and there is no tiling effect between them. One drawback of ColorMapGAN is that it seems to be slightly smoothing out trees, which results in the patterns on trees disappearing slightly (see the green rectangle in Fig. 10c). This is probably why the framework with CycleGAN outperforms the framework with ColorMapGAN on pair 1 for tree class.

ColorMapGAN vs Histogram matching [START_REF] Inamdar | Multidimensional probability density function matching for preprocessing of multitemporal remote sensing images[END_REF]: The main problem of histogram matching is that it does not take into As can be seen in Fig. 13b, there is a large deviation between some of the neighboring bins in the histogram. In contrast, color histograms of the buildings in the fake Roanne generated by ColorMapGAN are more similar to the histograms of the buildings in Béziers. For histogram matching, we observe the same issue for the other classes as well, but we do not include the histograms of the other classes because of the lack of space.

E. Running Times

The proposed framework, ColorMapGAN, CycleGAN, UNIT, MUNIT, and DRIT were implemented in Tensorflow 1 . We conducted all the experiments on an Nvidia Geforce GTX1080 Ti GPU with 11 GB of RAM. (g) UNIT [START_REF] Liu | Unsupervised image-to-image translation networks[END_REF] (h) MUNIT [START_REF] Huang | Multimodal unsupervised image-to-image translation[END_REF] (i) DRIT [START_REF] Lee | Diverse image-to-image translation via disentangled representations[END_REF] (j) Gray world [START_REF] Buchsbaum | A spatial processor model for object colour perception[END_REF] (k) Hist. match. [START_REF] Gonzalez | Digital Image Processing[END_REF] (l) ColorMapGAN (g) UNIT [START_REF] Liu | Unsupervised image-to-image translation networks[END_REF] (h) MUNIT [START_REF] Huang | Multimodal unsupervised image-to-image translation[END_REF] (i) DRIT [START_REF] Lee | Diverse image-to-image translation via disentangled representations[END_REF] (j) Gray world [START_REF] Buchsbaum | A spatial processor model for object colour perception[END_REF] (k) Hist. match. [START_REF] Gonzalez | Digital Image Processing[END_REF] (l) ColorMapGAN demonstrates that the training time for ColorMapGAN is significantly shorter than the other learning based approaches. Let us also remark again that we optimize ColorMapGAN for only 8,000 iterations. In other words, we need only about 6.5 minutes to train it. On the other hand, the other learning based approaches with the default parameters requires long hours to train. It is also notable that non-learning based approaches generate an output in a substantially shorter time. The execution time for gray world algorithm is almost instant, and histogram matching needs less than half a minute. However, the quality of the results for non-learning based methods is deficient.

V. CONCLUDING REMARKS

In this work, we presented a novel framework to generate high quality maps from satellite images even when there exists

Method

Training time for 1 Iter. (secs.) CycleGAN [START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF] 0.11 UNIT [START_REF] Liu | Unsupervised image-to-image translation networks[END_REF] 0.47 MUNIT [START_REF] Huang | Multimodal unsupervised image-to-image translation[END_REF] 0.45 DRIT [START_REF] Lee | Diverse image-to-image translation via disentangled representations[END_REF] 0.29 ColorMapGAN 0.05

City

Execution time (seconds) Gray world [START_REF] Buchsbaum | A spatial processor model for object colour perception[END_REF] Histogram matching [START_REF] Gonzalez | Digital Image Processing[END_REF] (g) UNIT [START_REF] Liu | Unsupervised image-to-image translation networks[END_REF] (h) MUNIT [START_REF] Huang | Multimodal unsupervised image-to-image translation[END_REF] (i) DRIT [START_REF] Lee | Diverse image-to-image translation via disentangled representations[END_REF] (j) Gray world [START_REF] Buchsbaum | A spatial processor model for object colour perception[END_REF] (k) Hist. match. [START_REF] Gonzalez | Digital Image Processing[END_REF] (l) ColorMapGAN (g) UNIT [START_REF] Liu | Unsupervised image-to-image translation networks[END_REF] (h) MUNIT [START_REF] Huang | Multimodal unsupervised image-to-image translation[END_REF] (i) DRIT [START_REF] Lee | Diverse image-to-image translation via disentangled representations[END_REF] (j) Gray world [START_REF] Buchsbaum | A spatial processor model for object colour perception[END_REF] (k) Hist. match. [START_REF] Gonzalez | Digital Image Processing[END_REF] (l) ColorMapGAN a large domain shift between spectral bands of the training and the test images. We validated our approach on two city pairs, where we performed a city to city domain adaptation in each pair. Our experimental results demonstrated that the proposed framework exhibits a much better performance than the existing approaches. We also showed that the proposed ColorMapGAN generates fake images in a significantly shorter time than some of the competitive unpaired image to image translation methods in the computer vision community.

A possible future direction could be to investigate a more difficult domain adaptation problem, where in addition to the large spectral difference, shapes of the objects such as buildings in the training and test images are considerably different.

Fig. 1 .

 1 Fig.1. An illustration for the domain adaptation problem, where we depict training and test images, the ground-truth for the test image, and the predicted map by U-net. In the ground-truth and in the predicted map, red, green, and white pixels correspond to building, tree, and road classes, respectively.

Fig. 2

 2 Fig. 2 depicts the overall framework that consists of 4 steps as follows: 1) Training the initial classifier: We train a U-net on the original training data. 2) ColorMapGAN: We generate fake training images that are semantically exactly the same as the original training images, but visually as similar as possible to the test images using the proposed ColorMapGAN. 3) Fine-tuning: We fine-tune the model obtained in step 1 using the fake training images and the ground-truth for the original training images. 4) Classification: Finally, we classify the test images.We use a slighly modified version of U-net[START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF] as the classifier. We replace rectified linear activation units (ReLU) by leaky rectified linear activation units (Leaky-ReLU) for a better performance[START_REF] Xu | Empirical evaluation of rectified activations in convolutional network[END_REF]. We also remove the batch normalization operation in each layer, since it uses the memory inefficiently. In the framework, the steps 1, 3, and 4 are self-explanatory, whereas step 2 needs further explanation.

Fig. 2 .

 2 Fig. 2. The overall framework.

  where p is a 2D input matrix. The final output of G can be obtained by reshaping DN (R G B [I]) back to the shape of the input patch. In each training iteration, we update only W [I] and K[I]. Fig. 3 illustrates the overall flowchart for the feedforward pass of G.

Fig. 3 .

 3 Fig. 3. The overall flowchart for the feedforward pass of the generator.

Fig. 4 .

 4 Fig. 4. The architecture of the discriminator. n corresponds to the patch size, and the number below each layer is the depth.

Fig. 5 .

 5 Fig. 5. Example close-ups from the Luxcarta data set.

Fig. 6 .

 6 Fig.6. Original Bad Ischl and the generated fake images that are used to generate maps for Villach.

Fig. 7 .

 7 Fig. 7. Original Villach and the generated fake images that are used to generate maps for Bad Ischl.

Fig. 8 .

 8 Fig. 8. Original Béziers and the generated fake images that are used to generate maps for Roanne.

Fig. 9 .

 9 Fig.9. Original Roanne and the generated fake images that are used to generate maps for Béziers.

Fig. 10 .

 10 Fig. 10. A closeup from Villach and the corresponding fake images.

  (a) Bad Ischl (b) Hist. matc. [47] (c) ColorMapGAN

Fig. 11 .

 11 Fig. 11. A closeup from Bad Ischl and the corresponding fake images.

(a)

  Bad Ischl (b) Hist. matc. [47] (c) ColorMapGAN

Fig. 12 .

 12 Fig. 12. A closeup from Roanne and the corresponding fake images.

Fig.

  Fig.[START_REF] Huang | Domain transfer through deep activation matching[END_REF]. Color histograms of building pixels. Red, green, and blue bins represent the histograms for red, green, and blue channels, respectively.

  (a) Bad Ischl (b) Ground-truth (c) U-net [1](d) AdaptSN S[START_REF] Tsai | Learning to adapt structured output space for semantic segmentation[END_REF] (e) AdaptSN M[START_REF] Tsai | Learning to adapt structured output space for semantic segmentation[END_REF] (f) CycleGAN[START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF]

Fig. 14 .

 14 Fig. 14. Bad Ischl, ground-truth, and the predictions. Building, road, tree, and background classes are represented by red, white, green, and black colors.

(a)

  Villach (b) Ground-truth (c) U-net [1] (d) AdaptSN S [12] (e) AdaptSN M [12] (f) CycleGAN [26]

Fig. 15 .

 15 Fig. 15. Villach, ground-truth, and the predictions. Building, road, tree, and background classes are represented by red, white, green, and black colors.

  (a) B éziers (b) Ground-truth (c) U-net [1](d) AdaptSN S[START_REF] Tsai | Learning to adapt structured output space for semantic segmentation[END_REF] (e) AdaptSN M[START_REF] Tsai | Learning to adapt structured output space for semantic segmentation[END_REF] (f) CycleGAN[START_REF] Zhu | Unpaired image-to-image translation using cycle-consistent adversarial networks[END_REF]

Fig. 16 .

 16 Fig. 16. Béziers, ground-truth, and the predictions. Building, road, tree, and background classes are represented by red, white, green, and black colors.

(a)

  Roanne (b) Ground-truth (c) U-net [1] (d) AdaptSN S [12] (e) AdaptSN M [12] (f) CycleGAN [26]

Fig. 17 .

 17 Fig. 17. Roanne, ground-truth, and the predictions. Building, road, tree, and background classes are represented by red, white, green, and black colors.

TABLE I THE

 I STATISTICS FOR THE LUXCARTA DATA SET.

	City	# of patches Area (km 2 )	Class frequency (%) building road tree
	Bad Ischl	457	27.71	5.51	6.03	35.38
	Villach	749	43.59	9.26	10.63	19.91
	Béziers	407	25.75	19.09	17.62	10.91
	Roanne	384	25.84	18.44	8.33	14.78

TABLE II IOU

 II SCORES FOR THE TEST CITIES IN PAIR 1.

			Method	Training: Bad Ischl, Test: Villach building road tree Overall	Training: Villach, Test: Bad Ischl building road tree Overall
			U-net	23.61	0.91	40.53	21.68	5.84	0.24	0.50	2.19
		AdaptSegNet Single [12]	6.01	4.37	10.43	6.94	3.06	2.71	10.23	5.33
		AdaptSegNet Multi [12]	24.59	9.02	56.08	29.86	14.26	4.46	24.66	14.46
	Our proposed	framework with	CycleGAN [26] UNIT [27] MUNIT [28] DRIT [29] Gray world [61] Histogram matching [47] ColorMapGAN (ours)	43.03 30.86 0.02 0.01 25.19 24.95 48.47	28.96 15.84 1.38 3.96 26.43 29.34 37.82	68.86 63.00 47.23 8.72 56.15 61.59 58.92	46.95 36.57 16.21 4.23 35.92 38.63 48.40	43.62 19.29 6.20 0.00 29.55 6.45 49.16	38.69 36.83 0.13 10.19 24.80 0.92 41.75	71.68 35.57 0.05 0.01 46.41 1.28 59.84	51.33 30.56 2.13 3.40 33.58 2.88 50.25

TABLE III IOU

 III SCORES FOR THE TEST CITIES IN PAIR 2.

			Method	Training: Béziers, Test: Roanne building road tree Overall	Training: Roanne, Test: Béziers building road tree Overall
			U-net	26.13	11.16	7.79	15.03	19.85	0.00	0.00	6.62
		AdaptSegNet Single [12]	6.61	11.05	3.37	7.01	11.07	4.19	3.71	6.32
		AdaptSegNet Multi [12]	22.42	5.87	17.84	15.37	24.27	10.88	10.45	15.20
	Our proposed	framework with	CycleGAN [26] UNIT [27] MUNIT [28] DRIT [29] Gray world [61] Histogram matching [47] ColorMapGAN (ours)	18.19 41.99 10.17 42.16 51.47 18.64 55.60	24.28 38.47 1.66 41.77 40.42 9.87 44.66	0.19 0.39 0.31 1.18 18.25 4.81 28.39	14.22 26.95 4.05 28.37 36.71 11.11 42.88	9.92 29.99 7.49 25.73 14.61 20.63 47.12	16.00 39.19 0.84 36.54 31.32 0.02 35.18	0.54 3.11 0.13 0.68 21.99 0.00 21.91	8.82 24.10 2.82 20.98 22.64 6.88 34.74

  [START_REF] Huang | Domain transfer through deep activation matching[END_REF]. Color histograms of building pixels. Red, green, and blue bins represent the histograms for red, green, and blue channels, respectively. account the contextual information, it only tries to match the histogram of the whole training city with the histogram of the whole test city. Conversely, the discriminator of Col-orMapGAN extracts high level features from the output of the generator and the test data to decide which one is real and which one is fake. In other words, the generator generates a fake training city in a way that its high level features align with the high level features extracted from the test city. For this reason, the proposed framework with ColorMapGAN yields substantially improved results. As shown in Fig.11by yellow rectangles, when generating fake Bad Ischl, histogram matching converts some gray rooftops to cyan ones, whereas ColorMapGAN keeps them gray. Similarly, in the same figure, we observe that the buildings highlighted by green rectangles have dark violet rooftops in the output of histogram matching, and black rooftops in the image generated by ColorMapGAN. In Fig.7a, we can see that there is no building having a cyan or dark violet rooftop in Villach, but there exists lots of buildings with gray or black rooftops. If the generator of ColorMapGAN generated cyan or dark violet colored rooftops, the discriminator would easily understand that these buildings were fake. For this reason, such buildings do not appear in the output of ColorMapGAN. Similarly, in the process of generating fake Roanne, histogram matching algorithm generates some reddish roads, as shown in Fig.12by a yellow rectangle. In the same figure, we see that ColorMapGAN outputs gray roads. Moreover, ColorMapGAN generates buildings having brownish colored rooftops, which are probably more representative for the buildings in Béziers than the buildings with red rooftops generated by histogram matching.As discussed in Sec. I, CNNs are extremely sensitive to the training data. A small domain shift between the training and the test data may affect the results significantly. Furthermore, in Fig.13, we depict color histograms of the buildings in Roanne, fake Roanne generated by histogram matching and ColorMapGAN, and Béziers. Since Roanne and Béziers are two different cities, we cannot expect the histograms of an ideal fake Roanne and Béziers to be exactly the same. However, we expect them to resemble each other. Although histogram matching algorithm tries to match the histogram of the training city with the histogram of the test city, there exists a large difference between the class-wise histograms of the fake training and the test cities.

  Table IV reports the running times for training ColorMapGAN and the other learning based compared methods for 1 iteration. The table

TABLE IV TRAINING

 IV TIMES FOR GENERATING FAKE CITIES.

TABLE V EXECUTION

 V TIMES FOR GENERATING FAKE CITIES.

https://www.tensorflow.org

ACKNOWLEDGMENT

The authors would like to thank ACRI-ST and CNES for initializing and funding this study. The authors also thank Alain Giros and Sébastien Clerc for fruitful discussions.


Olivier Rance

Senior Member, IEEE Etienne Perret

Romain Siragusa

Pierre Lemaître-Auger

Effect of Distance for Chipless RFID Magnitude Coding

Keywords: Chipless RFID tags, polarization, radar crosssections, radiofrequency identification, scatterer

An implementation of the hybrid frequencymagnitude coding is presented. This technique can be applied to enhance coding capacity of already existing chipless radiofrequency identification tags without increase of complexity. The dependency of the magnitude from the tag-reader distance is discussed and a compensation technique is proposed. An increase of 2.6 bits per resonator is demonstrated from measures in office environment.

I. INTRODUCTION

R ADIOFREQUENCY identification (RFID) is a technology that has emerged in the 1940's. Although RFID is now used in many areas with new functionalities provided, the optical barcode remains the preferred candidate for large scale distribution (item tracking) due to its low cost and ease of use. A promising solution to reduce the unit price of a RFID tag is to remove the silicon chip.

As first consequence the read range is reduced as compared to conventional RFID due to the absence of load modulation method. A read range of 30 cm is generally a good practical value (comparable with low cost barcode readers) for measurement of chipless tags in real environment.

Another consequence of the chip suppression is the significant decrease in coding capacity. Indeed in absence of chip, all the information must be encoded directly in the shape of the tag. The most classic approach which provides the best coding density is called frequency position coding. This consists in using resonant elements whose resonant frequency can be set independently from one another. The information is carried by the presence or absence of peaks at specific frequencies of the signature (horizontal axis in Fig. 1). Coding capacities achieved in this way remain less than fifty bit [START_REF] Preradovic | Design of fully printable planar chipless RFID transponder with 35-bit data capacity[END_REF]- [START_REF] Rezaiesarlak | Complex-natural-resonance-based design of chipless RFID tag for high-density data[END_REF] which is insufficient for most industrial applications. It is generally considered that chipless RFID should achieve a coding capacity of 128 bits for a tag with the size of a credit card for mass market penetration. Fig. 1. RCS magnitude coding principle. The tag is composed of resonant scatterers which produce peaks in the scattered spectrum. The information is coded on both the frequency and the magnitude level of each scatterer.

To further increase the coding capacity, it is interesting to use a hybrid coding which combines the resonant frequency of the peak with another physical quantity, independent of the resonant frequency: the magnitude of the peak (vertical axis in Fig. 1).

The principle of "magnitude coding" has been first described in [START_REF] Vena | A novel near-transparent ASK-reconfigurable inkjet-printed chipless RFID tag[END_REF] for counterfeiting applications. The control of magnitude is realized with the presence or absence of a transparent resistive ink, giving different coding with the same visual aspect. This first occurrence was not particularly focused on coding capacity enhancement. Another implementation of magnitude coding for a tag without a ground plane has been studied in [START_REF] Rance | Toward RCS magnitude level coding for chipless RFID[END_REF]. The magnitude control is done by coupling between adjacent resonators but this limits the increase of capacity (+2 bits for the whole tag) as it becomes difficult to adjust independently the magnitude. Another approach to control the magnitude of resonant scatterers is to use the polarization mismatch between the tag and the reader [START_REF] Rance | RCS magnitude coding for chipless RFID based on depolarizing tag[END_REF]. This approach has the advantage of being applicable to any tag containing resonant elements.

An important result from the Singular Expansion Method (SEM) theory [START_REF] Baum | The singularity expansion method and its application to target identification[END_REF] is that unlike the natural resonance frequencies which are specific to the tag, the magnitude of the peaks depends both on the tag and on the measurement configuration, in particular on the distance between the tag and the reader. This makes magnitude coding particularly challenging, and the significant issue is not addressed in [START_REF] Vena | A novel near-transparent ASK-reconfigurable inkjet-printed chipless RFID tag[END_REF]- [START_REF] Rance | RCS magnitude coding for chipless RFID based on depolarizing tag[END_REF]. In this study, we propose to show how an already existing tag [START_REF] Vena | A depolarizing chipless RFID tag for robust detection and its FCC compliant UWB reading system[END_REF] originally designed for frequency coding can be re-used to realize an hybrid magnitude -frequency coding, without impact on the resonant frequencies and without adding additional complexity. We will also show that with a minimum calibration procedure, it is possible to recover the magnitude information of the tag by integrating calibration elements directly within the tag. Note that an extensive study concerning frequency coding has been done in [START_REF] Vena | A depolarizing chipless RFID tag for robust detection and its FCC compliant UWB reading system[END_REF]. The present article is thus focusing mainly on the magnitude aspect of the hybrid magnitude -frequency coding and only one frequency configuration showing the worst case for small frequency separation is shown.

The article is organized as follows: in part II we will present the tag geometry and the magnitude control technique. Part III focuses on the dependence to distance. Part IV is a discussion about the coding capacity offered by the hybrid method.

II. TAG GEOMETRY AND MAGNITUDE CONTROL

A number of different optimized chipless RFID tags are available in literature. To cross the fifty bits, it is becoming clear that playing on the shape of the tag or the value of some geometric parameter will not be sufficient and that new coding techniques are needed. From this statement, it seems relevant to use already optimized tags whose reading in real environments is particularly robust [START_REF] Vena | A depolarizing chipless RFID tag for robust detection and its FCC compliant UWB reading system[END_REF] and to adapt them for the implementation of an hybrid frequency-magnitude coding.

The detection principle of [START_REF] Vena | A depolarizing chipless RFID tag for robust detection and its FCC compliant UWB reading system[END_REF] is based on the use of depolarizing tags associated with a cross-polarization reading. The principle is recalled schematically in Fig. 2 but will not be discussed here as already extensively developed in [START_REF] Vena | A depolarizing chipless RFID tag for robust detection and its FCC compliant UWB reading system[END_REF]. Note that the cross polarization reading as well as the limited read range of 30 cm is reducing the potential effect of multipath which could be problematic for magnitude coding.

A. Single Resonator

The resonator used as RF encoding particle (REP) is the gap coupled dipole [START_REF] Vena | A depolarizing chipless RFID tag for robust detection and its FCC compliant UWB reading system[END_REF] which is represented schematically in the inline of Fig. 1. This REP has been optimized in terms of RCS, quality factor and surface. It is composed of n = 5 microstrip dipoles of width W = 2 mm, tightly coupled through a gap g = 0.5 mm. The metallic strips are on top of a grounded Rogers RO4003C substrate having permittivity ε r = 3.55 and thickness t = 0.8 mm. The angle between the vertical orientation and the REP is θ . The dipoles are initially oriented at θ = 45 • to maximize the cross-polarization component of the response. As first approximation, gap-coupled microstrip dipoles can be considered as half-wavelength resonators [START_REF]Antenna Theory: Analysis and Design[END_REF]. The resonant frequency of the coupled microstrip dipoles is thus mainly set by the strips length L which is a varying parameter. Note that to improve the RCS level multiple dipoles have been added.

B. Magnitude Control

The control of the tag backscattered wave magnitude is based on the polarization mismatch between the tag and the reader. The complete description of the interaction of the incident wave with the tag is given by the polarization scattering matrix S, which relates the scattered electric field vector, E r , to the incident field vector, E i (see Fig. 2), component by component, [START_REF] Riegger | Wide-band polarimetry and complex radar cross section signatures[END_REF]- [START_REF] Knott | Radar Cross Section[END_REF]:

E r = E r h E r v = S hh S hv S vh S vv • E i h E i v = S • E i (1)
The quantity of interest is the cross-polarization component:

E r h = S hv • E i v .
Three case studies are examined. They are designated by (a), (b) and (c) on Fig. 3. The transmitting and receiving antenna are assumed to have a vertical and horizontal polarization respectively. They are symbolically represented by their polarization vectors h and v in Fig. 3. The scattering matrix of the reference case (a) is denoted by S [START_REF] Preradovic | Design of fully printable planar chipless RFID transponder with 35-bit data capacity[END_REF]. Now, we consider a rotation of angle θ of the resonator [case (b) in Fig. 3]. A new scattering matrix S is obtained:

E r = S • E i . (2)
Physically, applying a rotation of angle θ to the resonator [case (b) in Fig. 3] is the same as applying a rotation of angle -θ to the antennas [case (c) in Fig. 3] such that the scattering matrix associated to case (c) is equal to S':

E r = S • E i . ( 3
)
The incoming and reflected fields of case (c) are linked to the ones of case (a) by:

E i = T • E i , (4) E r = T • E r , ( 5
)
where T is the transpose of the rotation matrix of angle θ . Injecting ( 4)-( 5) into (1) and considering the fact that is a unitary matrix gives:

E r = • S • T • E i . ( 6
)
By identification with (3), it finally comes:

S = • S • T . ( 7
)
In this formulation, it is readily seen that the rotation angle θ can be used to modify the measured magnitude of the scattered field. The formula is valid for any kind of resonator and the method is then easy to adapt to other kind of REP. Considering the case of gap coupled dipoles, an interesting result is that for the reference angle θ = 0 (strips oriented along the vertical direction), the scattering matrix reduces to a simple form:

E r h E r v = 0 0 0 S vv (0) • E i h E i v . ( 8
)
Mathematically S vv and 0 are the eigenvalues of the scattering matrix function. Those particular quantities are considered to be particularly significant for target identification [START_REF] Huynen | Measurement of the target scattering matrix[END_REF]. Expressing the measured quantity S hv relatively to this special case provides a particularly simple expression:

|S hv (θ )| = |S vv (0) • sin(2θ)/2|. (9)
It is thus possible to control the signature magnitude value by varying θ . The maximum cross-polarization value is obtained for θ = 45 • and corresponds to S vv (0)/2. This coding technique implies the use of a specific reader able to change the incident field polarization while keeping a cross-polarization measurement configuration [START_REF] Garbati | Chipless RFID reading system independent of polarization[END_REF].

A series of measurements have been performed to validate the magnitude control of one REP included in the tag with other resonators, i.e., in presence of couplings. Four resonators having different length are positioned on a ground plane (inline of Fig. 1). Each resonator is realized individually in order to modify its orientation easily (inline photo of Fig. 4). The uncertainty due to manual placement of the resonators is evaluated to ±1 • . This uncertainty becomes important for θ approaching 0 which corresponds to maximum sensitivity of [START_REF]Antenna Theory: Analysis and Design[END_REF]. The resonator number 3 is rotated such that the response is modified from theoretical magnitude steps of 1.5 dB. The other resonators are kept in a constant position corresponding to their maximum magnitude (θ = 45 • ). Measurements (Fig. 4) are done in anechoic chamber, for a tag positioned at 20 cm from the reader. An empty measurement without tag (E 21 in the vertical axis of Fig. 4) is done to remove antenna couplings. A maximum deviation of 0.4 dB is observed for the resonator n • 4 (immobile) which shows that the coupling induced from the rotated resonator has little influence. The peaks of resonator n • 3 are decreasing with a regular 1.5 dB step as predicted in theory (indicated by grey bars in Fig. 4). For lower peak values, the decrease is less regular which can be due to positioning uncertainty. An overall magnitude range of 9 dB is obtained.

III. EFFECT OF DISTANCE

The measurement setup is represented schematically in Fig. 2. As previously seen, in practice the quantity that can be measured directly is not the components of the scattering matrix S but the s-parameters of the VNA. Assuming that the VNA is calibrated at the antenna ports, the S 21 parameter of the VNA can be related to the components of the polarization scattering matrix S with the monostatic radar equation [START_REF] Knott | Radar Cross Section[END_REF]:

S 21 = G • λ 2 √ π 3 • D 2 S hv ( 10
)
where G is the gain of the antenna and λ is the wavelength in vacuum. In [START_REF] Riegger | Wide-band polarimetry and complex radar cross section signatures[END_REF], it is clearly seen that the measured quantity S 21 depends on the reading distance D and the antenna characteristics. As it can be seen from the measurement results of Fig. 5, the magnitude of the peak apex is changing with the distance D which may lead to false level detection.

A. Compensation Technique

A practical way to deal with the dependency of the response with respect to distance is to use one of the resonators as a calibration element directly integrated within the tag. We can consider as an example that we know the magnitude level of the first peak S D r 21 ( f 1 ) for a reading distance of reference D r = 20 cm. Then the tag is measured at an unknown distance D 2 which gives the measured quantity S D 2 21 . Using the radar equation [START_REF] Riegger | Wide-band polarimetry and complex radar cross section signatures[END_REF] for both measurements, it comes:

S D 2 21 = (D r /D 2 ) 2 • S D r 21 ( 11
)
As S D r 21 ( f 1 ) is known, the ratio D 2 /D r can be simply obtained from the evaluation of S 21 D 2 at f 1 = 3.4 GHz (first peak). The broadband response of the tag reduced to the reference distance D r is then given by:

S D 2 21 ( f ) = S D 2 21 ( f 1 ) S D r 21 ( f 1 ) • S D r 21 ( f ) ( 12
)
For this approach, the magnitude value of the first peak must be the same from one tag to another and it cannot be used to code information anymore. In a certain manner, we can consider that the first resonator is a calibrating element. Equation [START_REF] Knott | Radar Cross Section[END_REF] shows that a reading error on the magnitude of the first resonator has repercussions on the other peaks. The first resonator is chosen as calibration element because it is the one having the highest quality factor. The orientation of the first resonator (θ = 45 • ) is also chosen to maximize magnitude in cross-polarization.

Then, assuming that the variation of the antenna gain with respect to frequency is known, it is possible to calculate S vh from the measured quantity S 21 D 2 ( f ) by using the radar equation [START_REF] Riegger | Wide-band polarimetry and complex radar cross section signatures[END_REF].

The response of the tag for different distances reduced to the reference case D r = 20 cm (12) are compared in Fig. 5. A maximum deviation of 0.7 dB appears for the peak n • 4. This value of 0.7 dB is doubled (1.5 dB) in order to define the magnitude resolution of the system (see Fig. 1). The magnitude resolution is defined in logarithmic scale to be independent to distance. Note that additional measurement have been done in anechoic chamber for read ranges up to 50 cm and would lead to a magnitude resolution of 3.2 dB following the same approach. However, the field magnitude obtained at this distance is probably too low for real environment applications.

B. Measurement and Decoding of Predefined IDs

A realistic magnitude resolution of 1.5 dB is chosen to evaluate the feasibility of the approach for a reading distance up to 27 cm. The magnitude range of 9 dB considered for this approach allows 6 different magnitude levels for each scatterer. The code associated to a tag is composed of four digits X 1 X 2 X 3 X 4 , where X i corresponds the resonator number i. X i can vary between 0 and 5 where 0 identifies the level of the highest magnitude and 5 the level of the lowest magnitude.

Different IDs (0000, 0132, 0244 etc. see legend of Fig. 6) are chosen in advance and the resonators are rotated with the corresponding angle in the tag presented in the inline of Fig. 6 to realize those codes. Then the backscatter E-field of the tags is measured at 20 cm. The objective is to verify that the IDs can be decoded from the measurement results with no error. The correspondence between the digit value, the magnitude slot and the value of the angle θ is given in table I. Measurements have been performed for the given IDs in real environment (office environment) with a dual polarization Satimo QH2000 antenna having a gain varying from 5 dBi to 8 dBi on the frequency band from 3 GHz to 5 GHz. Measurements have been performed at a distance D = 20 cm from the antennas. A Hamming time window is applied on the data acquired from the VNA. The lower limit of the Hamming window is taken at 8 ns to filter-out the response of the plate [START_REF] Ramos | Temporal separation detection for chipless depolarizing frequency-coded RFID[END_REF]. The maximum value of the window is taken at 20 ns which corresponds approximately to the extinction of the resonant modes. Time window allows limiting both potential multipath effects and parasitic reflections on the walls of the office.

The measurement results are presented in Fig. 6. The first resonator is used as a calibration element and is therefore not used for coding. The magnitude slots corresponding to a magnitude resolution of 1.5 dB are indicated by grey bars in the figure. They are calculated with respect to the maximum value of each peak (0000 measure in Fig. 6). For peaks n • 3-n • 4 for which there is no influence from other peaks (no coupling), a very good agreement between the theoretical 1.5 dB resolution and the measured values is observed. The X 3 and X 4 digits can be decoded without errors for all codes. For the second digit X 2 (peak n • 2), the remaining contribution of the first peak disrupts and makes the decoding a bit more difficult. This contribution is additive and constant and arises because the frequency separation between the first two peaks (200 MHz) is low which illustrates the worst case for the frequency configuration. However, it is possible to compensate this additive contribution. Indeed, for hybrid frequency-magnitude coding, first each peak frequency can be extracted from the backscattered signal, and depending on peak frequency shift between resonance frequencies, a compensation value can be used for decoding the information encoded in magnitude. In that example, the evaluation of the remaining contribution of the first peak at the frequency of the second peak gives a value of 3.1×10 -3 which is added to the theoretical response of peak n • 2 for the angles given in table I. The calculated magnitude slots are then represented by red bars in Fig. 6. Using those red bars effectively gives a decoding with no error for the second digit X 2 . This problem of interference between contiguous peaks can also be addressed simply by considering larger frequency spacing (300 MHz) between neighboring resonators.

From practical consideration, the tag orientation during the reading can be considered to be unknown. This additional constraint can be taken into consideration by using a specific reader able to change the incident field polarization while keeping a cross-polarization measurement configuration [START_REF] Garbati | Chipless RFID reading system independent of polarization[END_REF]. With such functionality, from the reader side, the orientation corresponding to the maximum of the peak apex n • 1 can be determined. From the measurement at the specific orientation, the approach introduced here can be directly implemented.

IV. CODING CAPACITY

In the present section, the evaluation of the coding capacity offered by the frequency coding is first considered. Then the additional capacity given by the hybrid magnitude -frequency coding is calculated in a second step.

The number of resonators is denoted by n. In the present case, n = 4 resonators are considered. The four resonant peaks are spread over the UWB (3.1 GHz to 10.6 GHz) which provide an operating frequency band of F = 7.5 GHz. Based on the selectivity that can be achieved for the resonators, the band is divided in frequency slots of length δf = 100 MHz. This gives a number of N = f /δf = 75 potential slots for each resonator. We consider that a minimum frequency separation of 300 MHz (see Fig. 7) is taken to limit the problem of neighboring contribution discussed in previous section. Each resonator has then to be separated from at least m = 2 slots from its neighbor as represented schematically in Fig. 7-a. The total number of IDs is cumbersome to calculate due to the 300 MHz separation but a close minorant is given by:

C FP = 1 n! • n k=1 N -k(2 • m + 1) (13)
For n = 4 and N = 75, this gives a total coding capacity of 19.7 bits for the present tag.

We consider now that in addition to the conventional frequency position coding, each resonator is coding M = 6 magnitude levels except for the first one used as a calibration element. The hybrid magnitude-frequency coding principle is represented graphically in Fig. 7-b. For each frequency ID, there is then M (k-1) different possibilities related to the magnitude. The total number of IDs for the hybrid magnitudefrequency position coding is then given by:

C MC = M (k-1) • 1 n! • n k=1 N -k(2 • m + 1) (14)
which gives a total coding capacity of 27.5 bits calculated from [START_REF] Ramos | Temporal separation detection for chipless depolarizing frequency-coded RFID[END_REF]. An additional coding capacity of 2.6 bits per resonator (the first one is used as reference) is thus gained with the magnitude coding approach. More generally, the comparison between the coding capacities of the two approaches can be done using [START_REF] Garbati | Chipless RFID reading system independent of polarization[END_REF] and [START_REF] Ramos | Temporal separation detection for chipless depolarizing frequency-coded RFID[END_REF]. The hypothetical case of N = 50 frequency slots is being considered. The coding capacity is represented in Fig. 8 with respect to the number of resonators n for a various number of magnitude levels M. We see that magnitude coding makes a big improvement of the coding capacity, even for low number of resonators and low number of levels.

V. CONCLUSION A comprehensive study has been performed to examine the feasibility and the potential gain of the magnitude level coding for chipless RFID. The control of the magnitude is realized with the polarization mismatch between the tag and the antennas. This approach is simple to implement in practice because it implies only a rotation of the resonators and can be easily adapted to already optimized REPs. The initial tag consisting in four resonators was coded with the frequency position method for a capacity of 24.2 bits. The approach allowed increasing the coding capacity by 7.8 bits. It has been shown that the presence of a reference scatterer included within the tag allows measurement with a minimalist calibration process without to suffer from dependency to reading distance.

Fig. 2 .

 2 Fig.2. Cross-polarization measurement setup. The structural mode is common to all scatterers and is a deformed copy of the incident pulse. The antenna mode is specific to the tag and has a different polarization than the incident pulse.

Fig. 3 .

 3 Fig. 3. Case studies. (a) Reference case. (b) Resonator rotated from an angle θ with respect to reference case. (c) Antennas rotated from and angle -θ with respect to reference case. The antennas are symbolically represented by their polarization vectors h and v. Cases (b) and (c) are physically equivalent.

Fig. 4 .

 4 Fig. 4. Measured magnitude response of a tag with one scatterer rotated successively, in an anechoic chamber with a reading distance D=20 cm. The length of the dipoles are L 1 = 23 mm, L 2 = 21.8 mm, L 3 = 17.9 mm and L 4 = 16.8 mm.

Fig. 5 .

 5 Fig. 5. Measured magnitude response of a tag with different reading distances. The normalized cases are calculated with (12) for the reference distance D r = 20 cm.

Fig. 6 .

 6 Fig. 6. Measurement results (office environment) for predefined IDs. The magnitude slots of 1.5 dB are indicated by grey bars in the figure. The first resonator is used as reference.

Fig. 7 .

 7 Fig. 7. Coding principle of (a) Frequency position coding and (b) Hybrid magnitude -frequency coding.

Fig. 8 .

 8 Fig. 8. Comparison between coding capacities of the frequency position coding alone (FP) and the hybrid magnitude -frequency position coding with respect to the number of resonators n for different number of magnitude levels M. The hypothetical case of N = 50 frequency slots is taken.

TABLE I CORRESPONDENCE

 I

BETWEEN DIGIT VALUE, ANGLE AND MAGNITUDE SLOT