Etienne Dupuis
email: etienne.dupuis@ec-lyon.fr

David Novo
email: david.novo@lirmm.fr

Ian O'connor
email: ian.oconnor@ec-lyon.fr

Alberto Bosio
email: alberto.bosio@ec-lyon.fr

Sensitivity Analysis and Compression Opportunities in DNNs Using Weight Sharing

Keywords: Deep Neural Networks, Approximate Computing, Model Compression, Weight Sharing, Design Space Exploration, Embedded System, Hardware Accelerator

Deep artificial Neural Networks (DNNs) are currently one of the most intensively and widely used predictive models in the field of machine learning. However, the computational workload involved in DNNs is typically out of reach for lowpower embedded devices. The approximate computing paradigm can be exploited to reduce the DNN complexity. It improves performance and energy-efficiency by relaxing the need for fully accurate operations. There are a large number of implementation options leveraging many approximation techniques (e.g., pruning, quantization, weight-sharing, low-rank factorization, knowledge distillation, etc.). However, to the best of our knowledge, a few or no automated approach exists to explore, select and generate the best approximate version of a given DNN according to design objectives. The goal of this paper is to demonstrate that the design space exploration phase can enable significant network compression without noticeable accuracy loss. We demonstrate this via an example based on weight sharing and show that our direct conversion method can obtain a 4.85x compression rate with 0.14% accuracy loss in ResNet18 and 4.91x compression rate with 0.44% accuracy loss in SqueezeNet without involving retraining steps.

I. INTRODUCTION

Deep Neural Networks (DNNs) are currently one of the most widely used predictive tools in the field of machine learning. They are used today in various applications such as object recognition, drug discovery and natural language processing, as well as safety-critical applications like autonomous driving.

Recent DNNs are articulated around the use of multiple chained convolution layers, called convolutional neural networks (CNNs), using a dot product between input and filter. They also rely on fully-connected layers composing the end of the network and pooling layers to reduce the dimension of activation maps, and thus, the number of parameters to be trained. More recently, Dropout, Batch-normalization were introduced to permit or to accelerate convergence during training. Trained parameters are used by convolutional and fully connected layers (e.g., the 50-layer ResNet [START_REF] He | Deep residual learning for image recognition[END_REF] network has around 26 million weight parameters), making these kinds of layers the main memory needing and the de-facto main target of most compression methods [START_REF] Alom | A state-of-the-art survey on deep learning theory and architectures[END_REF].

Unfortunately, the computational workload and memory needs of CNNs are often out of reach for low-power embedded devices [START_REF] Sze | Efficient processing of deep neural networks: A tutorial and survey[END_REF]. Novel computing paradigms and emerging technologies are under investigation to make CNNs available for edge computing. Among them, the Approximate Computing (AxC) paradigm leverages the inherent resilience of CNNs to errors to improve energy efficiency, by relaxing the need for fully accurate operations. CNNs have a high degree of redundancy in terms of their architecture and parameters, this redundancy is not necessary for accurate prediction. This observation has paved the way for several highly recognized approximation techniques [START_REF] Sze | Efficient processing of deep neural networks: A tutorial and survey[END_REF]. The most popular ones include pruning [START_REF] Song Han | Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding[END_REF], quantization [START_REF] Baskin | UNIQ: Uniform Noise Injection for Non-Uniform Quantization of Neural Networks[END_REF], low-rank factorization [START_REF] Acharya | Online Embedding Compression for Text Classification using Low Rank Matrix Factorization[END_REF], knowledge distillation [START_REF] Hinton | Distilling the Knowledge in a Neural Network[END_REF] and weight-sharing [START_REF] Song Han | Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding[END_REF]. In this work, we focus on the weight-sharing technique because of its remarkable model compression results.

The weight-sharing technique identifies clusters of values that will be shared between weights (see Section II for details). In the literature, there are many practical applications of weight-sharing. Existing works can be divided into two groups. The first one involves training steps after or during the weights-sharing, while the second group targets weightsharing without retraining. Hashed net [START_REF] Chen | Compressing neural networks with the hashing trick[END_REF] proposes to randomly group weights into buckets sharing the same value before the training step. A more efficient method is presented in DeepCompression [START_REF] Song Han | Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding[END_REF], it achieves impressive results due to the use of various levels of redundancy in deep neural networks through pruning, clustering, and Huffman coding. Deep Kmeans [START_REF] Wu | Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions[END_REF] uses regularization terms to encourage weights to concentrate at retraining time and proposes interesting data re-shaping techniques optimized for higher data reuse with row-stationary dataflow [START_REF] Chen | Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks[END_REF] at inference time. Soft weightsharing [START_REF] Ullrich | Soft weight-sharing for neural network compression[END_REF] uses a simple (re)training step with a regularizer to efficiently compress the network using pruning and weightsharing. It is also possible to reduce complexity by encoding both weights and features map, then compute multiplications using a lookup table. Both LookNN [START_REF] Razlighi | LookNN: Neural Network with No Multiplication[END_REF] and Quantized CNNs [START_REF] Jiaxiang Wu | Quantized convolutional neural networks for mobile devices[END_REF] are based on this concept and achieve good results at inference time, particularly if energy efficiency is taken into account.

Although retraining the model after compression allows accuracy loss recovery, there are many reasons to develop design methods that can largely benefit from clustering without requiring any further retraining. For example, retraining may requires access to the full training data that cannot always possible due to the size of the entire dataset, but also for legal and privacy compliance reasons. Furthermore, it is a long and computation-intensive step, and thus, it is costly and can delay the CNN time-to-market. This is where conversion algorithms offer an alternative by working directly on trained models. An example of this approach can be found in vector quantization [START_REF] Gong | Compressing Deep Convolutional Networks using Vector Quantization[END_REF], where a clustering approach based on K-means is applied to fully connected layers of a trained network.

Despite the promising results, all existing approaches suffer from important limitations: requiring (re)training [START_REF] Chen | Compressing neural networks with the hashing trick[END_REF] [4] [START_REF] Wu | Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions[END_REF] [11] [START_REF] Razlighi | LookNN: Neural Network with No Multiplication[END_REF], [START_REF] Jiaxiang Wu | Quantized convolutional neural networks for mobile devices[END_REF], or target only fully-connected layers network [START_REF] Gong | Compressing Deep Convolutional Networks using Vector Quantization[END_REF]. Our goal is to compress a deep neural network targeting embedded device constraints. We choose to address this problem using a systematic exploration to convert a trained model, without requiring retraining, and targeting both fullyconnected and convolutional layers to leverage redundancies where they exist.

In the proposed method, model weights are clustered using the K-means algorithm with a layer-specific number of clusters. To achieve compression of fully-connected and convolutional layers of a CNN, we perform a layer-specific sensitivity analysis of the trained network. Then, a hierarchical exploration of the design space is exploited to get the final result. The main contributions of this paper can be summarized as follows:

1) We propose a method to analyze the sensitivity of network layers and identifies the good range of clusters; 2) We propose a design space exploration framework guided by the sensitivity analysis to efficiently compress the overall CNN. The rest of the paper is organized as follows: Section II introduces the concepts and advantages of the weight-sharing method. Section III describes the proposed method for the layer sensitivity analysis and the design space exploration, while section IV presents the results obtained. Finally, section V concludes the paper.

II. WEIGHT-SHARING APPROXIMATION

As seen in the previous section, deep neural network approximation can be achieved using different approaches. Among them, the weight-sharing technique identifies clusters of weights that will share a common value. Usually, clusters are identified by using the K-means [START_REF] Hartigan | A k-means clustering algorithm[END_REF] algorithm. According to the literature, the K-means based weight-sharing method achieved outstanding performance compared to other methods by using a scalar pool of values instead of data-representation based values [START_REF] Wu | Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions[END_REF]. indexes can be represented using 3bits. The index is finally used to access shared values in the table (as shown in the figure) to retrieve the weight value. Accordingly, with 5 clusters (K = 5), we obtain IndexM atrixSize = N * log 2 (K) = 75bits and V alueT ableSize = K * B = 25bits resulting in a total Size = 100bits. We can thus save 25 bits with respect to the original kernel matrix, achieving a 1.25× compression ratio (CR).

However, choosing the scope of the weight-sharing method is not obvious. To compress a network, one could try to find shared values in the whole network or different shared values for each layer or even reduce further the scope to the channel or kernel level. In our previous work [START_REF] Dupuis | On the automatic exploration of weight sharing for deep neural network compression[END_REF], we found that working at the level of the layer is enough to achieve a good compression ratio without inducing substantial accuracy loss. Accordingly, in this work, we apply weight-sharing at the layer level.

III. PROPOSED METHOD

This section presents the proposed method for approximating models using weight-sharing. We need to find out the suitable number of clusters K to represent the convolution and fully connected layers weights. Indeed, minimizing K maximizes the compression rate and maximizing K minimizes induced accuracy loss. Solving this trade-off is not a simple task, it involves a complete design space exploration and computation of the accuracy loss by scoring the network (i.e., measuring accuracy of the network using testing data set) many times. In this work, we propose a hierarchical method aiming to reduce the complexity of such exploration. The proposed method is based on a layer sensitivity analysis to evaluate the weight-sharing opportunity for each layer. The next subsections detail the main steps.

A. Layer Sensitivity Analysis

Each layer of a deep neural network has a specific role during the inference (i.e., when the trained CNN is used to infer/predict the input samples), it is reasonable to assume that different layers may have a different sensitivity w.r.t weightsharing. Some layers can be drastically approximated with a very small number of clusters without noticeable accuracy loss, while others are more sensitive and require a larger number of clusters. Identifying and quantifying the sensitivity of a particular layer can be achieved by measuring the impact of clustering that layer on the whole network accuracy. For each layer, we are varying the number of clusters and then we study the variation of the accuracy loss over the compression ratio.

Finding a suitable number of clusters K for each layer can be done by following the procedure illustrated in Algorithm 1. For now, we are only using the resulting accuracy loss to prune the solution space. The appropriate level of clustering for each layer is then the one that has the smallest impact on the network accuracy loss. This algorithm involves a lot of scoring steps (i.e., network accuracy evaluation). This could be avoided by stopping the analysis after reaching a user-defined accuracy loss threshold. Defining a range of clusters has the benefit of avoiding falling in local minima. At the cost of having to carefully choose the cluster number range to obtain satisfying results.

The result of the sensitivity analysis is the range of clusters number that can be used for each layer. That range will further be exploited by the proposed design space exploration as detailed in the next subsection.

B. Design Space Exploration

Fig. 1 sketches the overall flow of the proposed design space exploration framework. First of all, we start from a trained network (i.e., the baseline model in the figure). We use trained models from the open-source framework ONNX [START_REF] Bai | Onnx: Open neural network exchange[END_REF] model zoo as input. Allowing our framework to be fully compatible with any model that can be converted in ONNX format. And so, any available framework can be used for training. We are using MXNET [START_REF] Chen | Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems[END_REF] for network scoring. Within the framework, all the layers of the network are processed. The result is an approximate network characterized by its accuracy loss and its compression rate as compared to the baseline.

The developed framework applies the clustering layer by layer, allowing to solve trade-offs locally, and thus, reduce the design space complexity. We call this method hierarchical exploration because in our study layers are arranged by rank, i.e., meaning the order in the network from input to output. We used a greedy algorithm to choose the best number of clusters on the accuracy loss. For each solution, we thus need to score the approximate network on the testing set to quantify the accuracy loss. We then use a greedy approach to choose the best number of clusters for each layer, aimed at minimizing the accuracy loss. The identification of the best number of clusters is done accordingly to the sensitivity analysis step outcomes. This will reduce the overall number of evaluation since the number of clusters have been already identified and here refined when taking account the overall CNN. Each identified solution (i.e., each compressed CNN) is finally evaluated by computing the compression ratio and by the accuracy loss induced by the clustering w.r.t. the baseline model.

IV. EXPERIMENTS

In this section, we will first describe the framework and the experimental setup used to evaluate our compression method on existing CNNs and the results we obtained.

A. Experimental Setup

The experiments target large image classifier CNNs trained and tested using the LSVRC2012 [START_REF] Deng | ImageNet: A Large-Scale Hierarchical Image Database[END_REF], a popular dataset having a testing set of 50k images (50 images per class). The chosen CNNs are ResNet [START_REF] He | Deep residual learning for image recognition[END_REF] for its near state-of-the-art classification accuracy and SqueezeNet [START_REF] Iandola | Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <1mb model size[END_REF] for its implementationoriented approach that includes a relatively small number of parameters. We took the trained model from the ONNX model zoo and fed directly to our weigh-clustering framework as input. The following subsection describes results on ResNet and Squeezeneet. Both are designed for the ImageNet dataset and resize the image to 224x224 and have a 1000 class output. ResNet-18 reaches a top-1 accuracy of nearly 69.69% while SqueezeNet reaches 55.99%. We are working on a full precision model (i.e., network weights are stored using 32 bit floating point representation) without using reduced quantization or fixed-point representation.

All the experiments were executed on a server equipped with a single NVIDIA TESLA V100 GPU and the latest CUDA version of MXNET [START_REF] Chen | Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems[END_REF] for the network scoring task. We are using the latest version of ResNet18 and SqueezeNetV1.1 from one model zoo as an input network. Scoring one network on the 50 000 pictures Imagenet testing data set take respectively 80s and 50s. From this data, it is clear that the scoring of the model to compute accuracy is an intensive task and it corresponds to the bottleneck of the proposed framework. This is why the sensitivity analysis plays an important role because it already provides a reduced set of clusters to be further refined (i.e., modified). In this sense, we can save execution time because we can reduce the overall number of scoring tasks.

The following subsection presents the results obtained using the proposed approach on both ResNet and SqueezeNet.

B. Identifying a Layer Sensitivity

For a given input network we are looking to measure the resilience to the approximation of each layer. To achieve this task, we are monitoring the accuracy loss and the compression rate induced by weight sharing when the number of clusters varies.

Compression rate (CR) is calculated for each layer using the following formula: CR = baselineM odelM emory approximatedM odelM emory

= W * B W * ceil(log 2 (K)) + K * B (1)
Having W the number of weights of the layer, K the number of clusters of the approximated layer and B the number of bits used to represent a single value. We have a direct impact of K variation on the compression rate.

Fig 2 shows the accuracy loss related to the compression rate for each cluster number (K). We have seen previously that we have a direct impact of cluster number on compression rate but this is not true for the accuracy loss, which largely depends on the clustering results representing data. For a different number of clusters, the distance between clusters centroid and original data representation varies drastically. For the example the figure, we are varying the number of clusters used in the K-means algorithm from 20 to 60 with a step of 2. The gap in compression rate value between the two groups of points is explained by K crossing a threshold allowing the index value (ceil(log 2 (K))) to be represented with one bit less. For example, moving from 256 to 255 different clusters allow lowering the index bit from 9 to 8 bit, which has a large impact on the compression rate, because this index bit count is multiplied by the number of weight of the layer. We can observe some clusterized version of the layer that has better The Pareto front represents optimal points. From this set of pareto optimal K, applying the Greedy discrimination algorithm results in selecting the one with the lowest accuracy loss. In the example of Fig. 2, the best solution is the one with 48 clusters, which also lead to achieve a better accuracy compared to the baseline model.

Fig. 3 shows the number of clusters achieving the best accuracy for each layer of the network and the corresponding compression rate of the layer. Layers are ordered by size (number of weight). We can see 3 layers with higher compression opportunity (more resilience to high clustering) for ResNet and 4 for SqueezeNet. From the results depicted in Fig. 3, we set the range of the number of clusters from 20 to 60 (corresponding to the minim and maximum plotted in the histograms). This range is used as input for our hierarchical design space exploration framework.

C. Hierarchical Design Space Exploration

The results of the proposed framework are presented in Table I. Each row of the table corresponds to an experiment. For each experiment, we detail the Network topology, the Cluster Range and the type of experiment (column "Keep layer Bool"). Two types of experiments were carried out:

1) The results of the sensitivity analysis are not considered during the exploration (ID 1 to 4); 2) The results of the sensitivity analysis are considered during the exploration (ID 5 to 8).

For each experiment, we considered the Accuracy loss of the AxC model w.r.t. the top-1 of the baseline model. Finally, we reported the memory footprint in MegaBytes for the Baseline and AxC model. The Compression Ration (CR) is the last column The Cluster Range set 20 to 60 is determined from the sensitivity analysis. The second one 40 to 80 is used to The results have shown that better results in terms of memory compression are obtained by using the range 20 to 60 thus confirming the efficiency of the sensitivity analysis. For these cases, higher CR leads to lower accuracy loss that was expected.

Another interesting result is the comparison between the type of experiments confirming once again the importance of the sensitivity analysis. For ID 5 we got 5.80% against ID 1 of 5.71%. However, CR is not the only important point. Even if we got a higher CR, the accuracy loss is lower (thus better). Again for ID 4, we got an accuracy loss of 0.54% while for ID 1 we got 0.73%.

To conclude, the best results either in terms of CR or accuracy loss are obtained when cluster range came from results of sensitivity analysis corresponding to IDs 1, 3, 5 and 7. Concerning the type of experiments, we got for one case (ID 5) the best result compared to ID 1. On the other hand, for experiment ID 7, accuracy and compression rate are slightly lower than ID 3 meaning that keeping layer weights clustered during the next layers analysis is efficient in this case. We plan in our future works to check other directions for the design space exploration. More details are given in the next section.

Analyzing the resulting approximated networks we discover that storing values represent less than 1% of the memory occupation after clustering. As shown in Fig 4, the value table (i.e., weight values stored in the look-up table) is negligible compared to the index table representing the memory of each layer for a SqueezeNet after optimization using our hierarchical exploration method.

V. CONCLUSION AND FUTURE WORK

This paper presented our work towards a design space exploration method for compressing trained deep neural net-Fig. 4. Layer compression analysis for SqueezeNet works. We have shown that our method can sucessfully compress a deep neural network with very small accuracy loss. Reaching around 5x compression with less than 1% top-1 accuracy loss. The most important advantage is the fact that our approach is a one-shot conversion, and thus, we can avoid the prohibitive cost and constraints tied to network retraining.

We intend to continue our work on approximating deep neural networks by first exploring other design space parameters for the weight sharing compression. More in detail, our approximation method includes several parameters that can be explored to produce different solutions:

1) The order of exploration: Layers can be treated in different orders, the more intuitive method is to process layers in a topological order (i.e., from the input to the output). Alternatively, they can also be sorted by ascending or descending weights count, aiming to address first the layer with the highest compression opportunity and then recover accuracy loss with the others. 2) The candidate number of clusters: Defined as a range of values, described by min, max, and step. It can be either the same for all layers of a network or specific and depends on layer size. It can even be a logarithmic scale, considering that for a sufficiently high number of clusters, a slight change in cluster count has a negligible impact on the accuracy loss of the network. We then address performance and hardware metrics. By using a different algorithm for choosing parameters, because we rely on a greedy algorithm taking only the accuracy loss into account. Target metrics are firstly deep neural networkspecific metrics like accuracy loss and compression rate, secondly performance metrics like throughput rate and latency and thirdly hardware-oriented metrics like energy consumption and surface used in case of FPGA implementation. We also intend to add parameters like K-means initialization and layerspecific cluster range. Allowing the design space to be enlarged and explore potentially better parameter combinations.

Finally, we will extend the framework to support different approximation strategies such as pruning, quantization, lowrank factorization, knowledge distillation, etc.. As the use of a combination of orthogonal approximation methods allows more levels of redundancy inherent to neural networks to be leveraged.

VI. ACKNOWLEDGEMENT

Fig

  Fig. II shows an example to clarify the benefits of weightsharing. The first matrix corresponds to a 5 × 5 convolutional

Algorithm 1 :

 1 Layer sensitivity identification algorithm featuring a layer-wise design space exploration Input: network(Trained Network), minCluster (integer), maxCluster (integer) Output: layerK (number of cluster for each layer) layerK ← list(); for layer in network do bestAccLoss ← N one; for K ← minCluster to maxCluster do tempN et ←K-means(layer, K); accLoss ← score(tempN et, valDataset); if accLoss < bestAccLoss then bestAccLoss ← accLoss; bestK ← K; end end layerK[layer] ← bestK; end

Fig. 1 .

 1 Fig. 1. Overall flow of the proposed method

Fig. 2 .

 2 Fig. 2. Sensitivity Analysis: Accuracy loss and compression rate of different version of the Layer 28 of SqueezeNet, the range of value for K is from 20 to 60

Fig. 3 .

 3 Fig. 3. Network sensitivity to clustering, layer-wise analysis on ResNet and SqueezeNet, layer are ordered by number of weights

This work has been funded by the French National Research Agency (ANR) through the AdequatedDL research project (ANR-18-CE23-0012).


Dino Ienco
email: dino.ienco@teledetection.fr

Albert Bifet
email: abifet@yahoo-inc.com

Bernhard Pfahringer
email: bernhard@cs.waikato.ac.nz

Pascal Poncelet
email: poncelet@lirmm.fr

Change detection in categorical evolving data streams

Keywords: H.2.8 [Database Management]: Knowledge Management Applications-data exploration and discovery Theory, Algorithms, Experimentation Evolving data stream, categorical data, unsupervised change detection, statistical test, concept drifts

published or not. The documents may come

INTRODUCTION

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC'14 March 24-28, 2014, Gyeongju, Korea.

Many real world applications continuously generate huge amounts of data, such as web logs, sensor networks, business transactions, etc. These data streams [START_REF] Aggarwal | Data Streams -Models and Algorithms[END_REF], due to the big volumes of information they contain, pose serious issues for the research community in order to extract useful and update-to-date knowledge in real-time. Due to its intrinsic temporal dimension, the information available in data streams changes and evolves over the time. More precisely, this phenomenon impacts on the performance of any supervised (or unsupervised) model learnt over these evolving data streams: previous models may not be suitable for newly incoming data [START_REF] Aggarwal | Data Streams -Models and Algorithms[END_REF]. Therefore we need to be able to adapt models both quickly and accurately. In particular, di↵erent types of change may happen in the stream. For instance, classes or concepts that can be underrepresented during a short period can become overrepresented after a longer period. Detecting change is essential for adapting learned models.

Most of the time, a common assumption made by many research works is to take into account only the aposteriori probability of the class given the data [START_REF] Gama | Learning with drift detection[END_REF]. Actually, this formulation of the change detection task does not exploit information coming from the underlying data distribution and, generally, it is not suitable for unsupervised tasks where the class information is not available at all. As it is generally known, obtaining labels for data (especially in a stream context) can be very time-consuming and may require intensive human e↵ort that is not always possible [START_REF] Ienco | Clustering based active learning for evolving data streams[END_REF]. This issue can limit the use of such techniques that depend on proper class labels.

Secondly, in real world applications, data is heterogeneous and often can be represented over set of categorical attributes as well as numerical ones. In the last decade, lots of approaches have been defined to monitor classification accuracy as evidence or an indication for change in streams of numerical data (e.g. [START_REF] Gama | Learning with drift detection[END_REF]). There are very few approaches dealing with the same problem which are able to handle categorical data [START_REF] Cao | A concept-drifting detection algorithm for categorical evolving data[END_REF].

The necessity to address all these points motivates us to propose a new change detection method especially tailored for categorical data: CDCStream (Change Detection in Categorical data Stream). Our approach is able to detect and highlight changes in categorical data streams in a fully unsupervised setting. Interestingly, our method is able to e ciently exploit historical data to make its decision. It works in the batch size scenario: when a new batch arrives, firstly CDCStream compresses it by extracting a descriptive summary and it then performs a statistical test to evaluate if change has happened in the data distribution, or not. The developed algorithm supplies a segmentation approach that can also work with other statistics. This means that it can be coupled with any other measure we want to monitor.

The remainder of this paper is organized as follows. Section 2 explores the state of the art in categorical change detection for data streams. The proposed methodology is presented in Section 3. Section 4 presents experimental results for real world datasets and we also report an analysis of its qualitative behavior. Section 5 concludes.

RELATED WORK

The problem of detecting concept drift or change is an important issue in data stream mining [START_REF] Aggarwal | Data Streams -Models and Algorithms[END_REF]. The research community has spent a lot of e↵ort to detect change in numerical streams [START_REF] Gama | Learning with drift detection[END_REF], while dealing with categorical data has not been addressed all that much [START_REF] Cao | A concept-drifting detection algorithm for categorical evolving data[END_REF]. A first attempt to mining concept drift in categorical data is presented in [START_REF] Chen | Catching the trend: A framework for clustering concept-drifting categorical data[END_REF]. Their proposed approach determines concept drift by measuring the di↵erence in cluster distribution between two contiguous batches of stream data. An important drawback of this method, that limits its usability in practise, is the diculty encountered when having to customize the settings of its many parameters. In [START_REF] Chen | He-tree: a framework for detecting changes in clustering structure for categorical data streams[END_REF], a framework was presented for detecting the change of a primary clustering structure which was indicated by the best number of clusters in categorical data streams. However, setting the decaying rates to adapt to di↵erent types of clustering structures is very di cult. Cao proposed two di↵erent rough set theory based methods to detect concept drifts in streams of categorical data [START_REF] Cao | A framework for clustering categorical time-evolving data[END_REF][START_REF] Cao | A concept-drifting detection algorithm for categorical evolving data[END_REF]. The former [START_REF] Cao | A framework for clustering categorical time-evolving data[END_REF] defines the distance between two concepts as the di↵erence value of the degree of membership of each object belonging to two di↵erent concepts, respectively. This distance can only detect concept drift but not the reasons leading to the change. The later strategy presented in [START_REF] Cao | A concept-drifting detection algorithm for categorical evolving data[END_REF], evaluates the di↵erence of distributions between two concepts appearing in two consecutive batches. The di↵erence is defined by the approximation accuracy of rough set theory which can also be used to measure the speed of change of concepts. If the di↵erence is bigger than a threshold a concept drift is revealed. Both methods require only a threshold parameter but, unfortunately: i) no hints on how set this parameter are supplied and ii) the parameter threshold has no clear associated meaning. These two factors do not facilitate the analyst to use these approaches. As has been pointed out, all these methods have several issues related to parameter settings (too many parameters or no hints about the value to use). Furthermore, all these approaches assume that, for detecting a concept drift it is enough to consider only contiguous batches and no more historical information coming from the data stream.

METHODOLOGY

In this section we describe the general setting and we introduce our new method. Suppose that we have an infinite data stream divided in batches. Each batch Si is associated with the arrival time denoted by the index i: S = {S1, S2, ..., Sn, ...}. This scenario is general enough to model real data streams. Note that, even in the case of continuous streams of data, we can employ a bu↵er-based procedure for collecting examples in order to build batches and then process one batch at a time. In addition, each example belonging to the data stream is defined over a set of categorical attributes. This means that each attribute Aj is defined over a discrete set of nominal values. As previously mentioned, the two important points that we address in this work are the following: (i) extracting useful information from each batch in order to summarize it and (ii) keeping trace of this information to be able to detect change. All these steps are addressed in a fully unsupervised scenario where no class information is available di↵erently from many previous strategies that work in supervised context [START_REF] Aggarwal | Data Streams -Models and Algorithms[END_REF]. To deal with the first point we exploit the method proposed in [START_REF] Ienco | From context to distance: Learning dissimilarity for categorical data clustering[END_REF], named DILCA. This method is able to extract numerical statistics from a set of categorical examples in order to summarize the underlying data distribution. To tackle the second point we monitor the statistics extracted from the batches by employing the Chebyshev's Inequality [START_REF] Aggarwal | Data Streams -Models and Algorithms[END_REF]. The combination of (i) and (ii) allows CDCStream to summarize categorical batches and to detect changes as they happen in the underlying data distribution, and to issue appropriate warnings. In the following we describe in detail the steps that CDCStream is based on.

Computing the distance model

Here we revise the DILCA (DIstance Learning for Categorical Attributes) framework that we can employ to summarize a batch of categorical data. DILCA was first introduced by [START_REF] Ienco | From context to distance: Learning dissimilarity for categorical data clustering[END_REF] and it was mainly used for clustering categorical data. The result of this method is a set of matrices, one for each attribute. Each matrix contains the learnt distances between each pair of values of a specific attribute. Let us now consider the set F = {X1, X2, . . . , Xm} of m categorical attributes over which the batch Si is defined. We denote by Y the target attribute, which is a specific attribute in F that constitutes the target of the method, that is, on whose values we need to compute the distances. The notation |X l | refers to the cardinality of the attribute X l . DILCA allows to compute a context-based distance between any pair of values (yi, yj) of the target attribute Y on the basis of the similarity between the probability distributions of yi and yj given the context attributes, called C(Y ) ✓ F \ Y . For each context attribute Xi it computes the conditional probability for both the values yi and yj given the values x k 2 Xi and then it applies the Euclidean distance. The Euclidean distance is normalized by the total number of considered values:

d(yi, yj) = s P X2C(Y ) P x k 2X (P (yi|x k ) P (yj|x k )) 2 P X2C(Y ) |X| (1)
The selection of a good context is not trivial, particularly when data are high-dimensional. To select a relevant and non redundant set of features w.r.t. a given one, [START_REF] Ienco | From context to distance: Learning dissimilarity for categorical data clustering[END_REF] propose to adopt FCBF, a feature-selection approach originally presented by [START_REF] Yu | Feature selection for high-dimensional data: A fast correlation-based filter solution[END_REF]. We let the reader refer to the cited paper for more details about the context selection step. At the end of the process, DILCA returns a distance model M = {MX l | l = 1, . . . , m}, where each MX l is the matrix containing pairwise distances between the values of attribute X l , computed using Eq. 1. The distances contained in the matrix MX l resume the underlying distribution of attribute X l . Each generated matrix MX l has some interesting characteristics: it is symmetrical, the diagonal is filled by 0s (logically the distance between a value and itself is 0) and each value is bounded between 0 and 1. The set M of matrices is aggregated into one single measure by the following formula, which only considers the upper triangular part of each matrix, discarding the all-zeroes diagonal (matrices are symmetric and their diagonals are filled by 0s):

extractSummary(M ) = P M l 2M 2⇥ r P|X l | i=0 P|X l | j=i+1 M X l (i,j) 2 |X l |⇤(|X l | 1)

|F |

(2) This formula extracts a single statistic starting from the matrices produced by the DILCA method over a batch of examples. For each matrix M l the squares of all values is summed and then rescaled by using the root. This value is then divided by the number of values that are summed

( |X l |⇤(|X l | 1)

2

). This operation normalizes the coe cient extracted from each matrix forcing them back into the range between 0 and 1.

The outer loop sums the coe cients of all the matrices and then the normalization step allows to scale back the value into the range between 0 and 1 again. The result of this formula numerically represents a summary of the whole batch taking into account both the correlation among the attributes and the attributes' distributions themselves.

Change Detection through the Chebyshev's Inequality

Statistical tests are often used in data stream mining in order to capture changes in the distribution [START_REF] Aggarwal | Data Streams -Models and Algorithms[END_REF]. Most of the literature focuses its attention on the supervised scenario where the goal is to detect changes as a decrease in classification accuracy. When that happens, it is time to re-adapt or start to re-learn a model. To manage this phenomenon, researchers usually exploit one-sided tests. These tests are usually chosen since changes (in a supervised scenario) are associated with a decrease only of classification accuracy. If a classifier increases its accuracy, we have no reason to modify it. Unfortunately, in our context, we work in a totally unsupervised setting: we want to detect general variation on both sides of the distribution, because the monitored variable is not the accuracy but an estimation of the data distribution. To deal with this issue, in order to understand if a batch of data deviates from the actual distribution or not, we employ techniques from Statistical process control [START_REF] Gama | Learning with drift detection[END_REF], that are based on Chebyshev's Inequality [START_REF] Aggarwal | Data Streams -Models and Algorithms[END_REF]. This statistical test does not assume any data distribution. It is a two tailed test and it considers both sides of the distribution to understand if a change is appearing.

Lemma 1 (Chebyshev's Inequality). Let X be a random variable with expectation µX and standard deviation X . Then for any k 2 R + ,

P r(|X µX | k X )  1 k 2
In particular, we consider the statistic computed over each batch Si as a value coming from a randomly distributed variable. In this way, without loss of generality, we can use Chebyshev's Inequality to monitor if change is happening, or not, in the data stream distribution.

As we can observe, to use this inequality, we need to compute the average µX and the standard deviation X . As we assume that a data stream evolves and that it is infinite, we can never retain all values. To tackle this important issue, we need to design an intelligent method to understand how many previous batches we need to use in order to monitor the actual one. Our solution is sketched in Algorithm 1. The procedure takes as input parameter the stream of data S and the values kc and kw to compute Chebyshev's Inequality in order to detect, respectively, changes and warnings. While a change can be associated with a drift in the data stream, a warning is a small variation or fluctuation that will not necessarily result in a shift of the data distribution. In addition to actual change, noise may also cause such warnings.

We can observe that the first step of the algorithm is devoted to initialize the list (L) and the standard deviation used as default in the case where we cannot compute it ( def ault ). The list L contains the summary values up until the previous batch: this means that if we are analyzing batch Si, L contains statistics until batch Si 1. As we enter the main loop, given the batch Si we extract the actual summary statistic using Formula 2. Once we obtain the statistic we check the size of L. We did not check the case in which L is empty because it can only be empty at the begin. After that it will always contain at least one value. When L contains more than a value we can compute both an average and a standard deviation. In the case where L contains only one value, and we have already computed the maximum ( MAX ) and the minimum ( MAX ) standard deviation seen until now, the actual summary value is used as the average and the standard deviation is set to ( MAX + MIN )/2. If the value of MAX and MIN are not defined, than we are in the first or second batch of the stream, and we have not yet collected enough evidence to compute these statistics, as we need to see at least two batches before we can compute them. In this case we need to continue to accumulate evidence before being ready to detect changes or warnings. Line 22 checks if is not null. If so, we employ the statistical process control technique based on Chebyshev's Inequality to evaluate if a change or a warning happens. In particular, Chebyshev's Inequality is used with two di↵erent thresholds kc to detect a change and kw to detect a warning. Logically, as a warning is a variation in the distribution less important than a change (which is more critical by definition) kc is greater that kw and for this reason, the inequality is first used to check if a change happens and successively, if that first test is negative, the second test determines the presence of a warning. At the end of the procedure the algorithm adds the new summary value x to the list L.

If a change happens the values ( i.e. the statistics/summary values until batch Si ) in the list L are removed and we restart to fill L with the one extracted from batch Si. Practically, if a change is detected, the actual batch Si is statistically di↵erent from the previous sequence of batches and, for this reason, we do not need to keep the history before Si anymore. In the case of a warning the algorithm only reports it to the user, without clearing L.

The final algorithm is a method able to automatically segment the sequence of summary statistic generated to summarize the batches of the stream.

EXPERIMENTS

In this section we evaluate the performance of our approach CDCStream under di↵erent aspects. We compare the behavior of our algorithm with the method proposed in [START_REF] Cao | A concept-drifting detection algorithm for categorical evolving data[END_REF] Algorithm 1 ChangeDetectionP rocess(S, kc, kw) Require: S: stream of instances Require: kc: stat. parameter to detect a change Require: kw: stat. parameter to detect a warning (kw < kc ) 1: MAX = MIN = nil 2: L = ; 3: while hasMoreInstances(S) do 4: Si = extractNextBatch(S) 5: //the summary statistic is extracted as explained in Sec. [START_REF] Bifet | MOA: Massive Online Analysis[END_REF] 29: end while that we name RSCD. RSCD is the most recent approach to detect changes in categorical data streams. It obtains good performance compared to previous methods.

To objectively measure the quality of the two methods, we compare them in a learning scenario in which a classifier is learnt using a warning model as introduced in [START_REF] Gama | Learning with drift detection[END_REF]. In particular, at the beginning a classifier is learnt and tested over the stream of data. Every time a warning is detected, a background classifier is created and trained in parallel with the main classifier. When a change is detected, the main classifier is replaced by the background one (and a new background classifier is also started). As a base learner for both CDCStream and RCSD we use the Naive Bayes classifier [START_REF] Witten | Data Mining: Practical Machine Learning Tools and Techniques[END_REF]. To evaluate the di↵erent methods we employ the final accuracy of the learnt model. To assess the accuracy we use the prequential evaluation: each time an instance arrives, we first test the algorithm on it, and then we subsequently use the example as training input for the classifier.

RCSD does not implicitly manage warnings, for this reason we adapt it to manage both changes and warnings. We consider that a change happens when RCSD returns values bigger than 0.01 (the same threshold is used in the original work [START_REF] Cao | A concept-drifting detection algorithm for categorical evolving data[END_REF]) while we highlight warnings when RCSD returns values bigger than 0.005 but lower than 0.01. As these values represent a threshold over a distance, it is reasonable that the change threshold is larger than the warning one. CDCStream is designed to manage both warnings and changes. For our approach we set the value of kc equals to 3 and kw equals to 2. We evaluate the two approaches using di↵erent batch sizes. In particular we employ the following values: {50, 100, 500, 1 000}.

In order to deeply understand the behaviour of our strategy, we also employ as another competitor the supervised method presented in [START_REF] Gama | Learning with drift detection[END_REF], that we name SDrif tClassif . It also implements a warning/change mechanism monitoring the accuracy of the classifier. All our experiments are performed using the MOA data stream software suite [START_REF] Bifet | MOA: Massive Online Analysis[END_REF].

MOA is an open source software framework in Java designed specifically for data stream mining.

Datasets

To evaluate all the algorithms we use four real world datasets: Electricity, Forest, Airlines, KDD99.

Electricity data [START_REF] Harries | Extracting hidden context[END_REF] is a popular benchmark in stream classification. The task is to predict the rise or fall of electricity prices (demand) in New South Wales (Australia), given recent consumption and prices in the same and neighboring regions. The task proposed in the Forest dataset [START_REF] Asuncion | UCI machine learning repository[END_REF] is to predict forest cover types of vegetation from cartographic variables. Inspired by [START_REF] Ikonomovska | Learning model trees from evolving data streams[END_REF] we constructed an Airlines dataset using the raw data from US flight control. The task is to predict whether a given flight will be delayed, given the information of the scheduled departure. The last dataset we use is KDD99. One of the big problems with this dataset is the big amount of redundancy among instances. To solve this problem we use the cleaned version named NSL-KDD 1 . To build the final dataset we join both training and test data. A summary of the characteristics of these datasets is reported in Table 1. We observe that this collection of datasets contains both binary and multi-class problems, datasets with di↵erent size (varying between 42k to 580k instances) and di↵erent dimensionality (from 7 to 54 attributes).

For analysis purposes, we also introduce one more dataset, named Forest Sorted, in which the instances of the Forest dataset are reordered using the attribute elevation. Due to the nature of the underlying problem, sorting the instances by the elevation attribute induces a natural gradual drift on the class distribution, because at higher elevation some types of vegetation disappear while other types of vegetation appear smoothly.

As general pre-processing for all numerical attributes we perform unsupervised equi-width discretization using a number of bins equal to 5.

Analysis of Accuracy

In table 2 we report the accuracy performance of the different methods using the warning/change framework as explained at the beginning of the experimental section. For each dataset, we highlight in bold the best result obtained for it. As a first point, we can observe that the supervised approach did not outperform the unsupervised ones. This is a remarkable result because SDrif tClassif is the only method that exploits the accuracy to make a decision in order to detect a change and to re-learn the model. The only case in which SDrif tClassif obtains slightly better results is over KDD99, but the final performances are comparable as the di↵erence is very low (around one point of accuracy). In all the other cases, we can observe that CDCStream outperforms the other competitors (both supervised and unsupervised).

Considering the comparison between CDCStream and SDrif tClassif we can note that, most of the time, our approach obtains interesting performances. This is especially true for Forest Sorted in which for all values of batch sizes it outperforms SDrif tClassif . For both Electricity and Forest, CDCStream obtains improvements using smaller batch size (b = 50 and b = 100). This similar behavior is shared by the classifier learnt using the RCSD approach. This is coherent with the results presented in [START_REF] Ienco | Clustering based active learning for evolving data streams[END_REF]. The authors show that for these two datasets smaller batches outperform bigger ones as they note that small batches forces the learner to adapt faster to possible changes. This is particularly true for Electricity, where multiple levels of periodicity are present, over 24 hours, over 7 days, and over 4 seasons. Drift can be visually presented by plotting class conditional distributions over time. Also for Airlines we can observe that our approach obtains results higher or comparable to the one achieved by SDrif tClassif . This time the better performances are reached considering larger batch size (b = 500 and b = 1 000), gaining in the best case more than two points of accuracy.

Analysing the performance of CDCStream, we can note that most of the time the classifier learnt by our approach obtains better performance. Also in this case, considering Forest Sorted, for all batch values CDCStream outperforms RCSD. Another interesting point is that CDCStream obtains interesting stability results w.r.t. RCSD. This behavior is very clear considering the Forest dataset. We can see that for CDCStream the accuracy varies between 82.99% (b = 50) and 74.46% (b = 500) while for the rough set based method the accuracy vary from 82.25% (b = 50) to a minimum of 64.08% (b = 500 or b = 1 000), which is a variation of 18 points of accuracy. A similar, but less drastic behavior can be observed for the Electricity dataset, where larger batch sizes impact negatively on RCSD while CDCStream remains more stable.

Generally, we can note that monitoring changes in data distribution (P (X)) with CDCStream allows to learn a good model. In supervised approaches, as SDrif tClassif , the detector reacts to changes in the posterior probability (P (Y |X)). Most of the time supervised methods assume that only P (Y |X) varies and it is the only useful quantity to monitor in order to make a decision [START_REF] Gao | A general framework for mining concept-drifting data streams with skewed distributions[END_REF]. An interesting future work will be to combine both sources of information (detected changes in P (X) and in P (Y |X)) in order to learn better stream classifiers.

To summarize, we can state that the classifier learnt using CDCStream, under the warning/change detection framework, obtains interesting accuracy results and also generally improves the classifier quality compared to a previous super-vised approach. We can also underline that, considering the benchmark used in the experiments, CDCStream is robust considering the batch size. This parameter did not have a strong influence on the final performance. A sample of changes detected by CDCStream on the Forest Sort. dataset with batch size equals to 1 000 between the batches 350 and 500

Analysis of Change Detection Behavior

We report the results concerning the behaviors of CDCStream and RCSD in Table 3. In this experiment we measure the percentage of changes detected by both methods, for each dataset considering di↵erent batch sizes. The percentage of changes is the number of changes detected divided by the possible number of changes that can happen in the stream. If a dataset is divided in N batches, the number of possible changes is equal to N 1 because we can have, at most, one change for each batch except the first one as we cannot start to detect changes before the second batch. The first aspect we point out is that the two methods have di↵erent behaviors. This means that when one approach increases the number of changes detected, the other one does not exhibit the same trend. For instance, if we analyze the results on Electricity, when we use a batch size equals to 50 the number of changes are similar but, when the batch size increases, they show di↵erent trends. In particular, for this dataset, we can observe that bigger batch size (b = 500 and b = 1 000) drastically impacts the percentage of changes discovered by RCSD. This is also confirmed by the decrease in accuracy for this strategy over large batch sizes. If we continue to analyze the results, we can note that for Forest Sort. RCSD has problems to detect the smooth changes that are intrinsically present in this dataset. This is underlined by the fact that for b = 50 and b = 100 it does not detect changes (a percentage equals to 0%) and also for larger values of batch size it does not react as much (a percentage lower than 0.2%). This phenomenon explains why the model learnt using the RCSD approach always obtains the same accuracy over all the values of batch size. The fact that the rough sets based approach does not underline changes, in this case, impacts the performance of the final classifier in a bad way. A totally di↵erent behavior is evident for Airlines. For this dataset RCSD (for batch size of 50, 100, 500) detects as many changes as theoretically possible. This means that, each time, the model previously learnt is discarded and replaced with a new one while CDCStream adopts a di↵erent strategy, and it adapts itself to the value of batch size in order to discover changes. As a practical consequence, the strategy adopted by CDCStream outperforms the one supplied by RCSD to learn the classification model for the stream data. In general we can say that, most of the times, RCSD detects smaller percentages of changes than the ones detected by CDCStream. When the opposite happens, RCSD always saturates the percentage of changes, reaching 100%. In Figure 1 we report the behavior of CDCStream on the Forest Sort. dataset considering a batch size of 1 000 instances. For lack of space we report only this example as a representative. We observe similar behavior over all the other datasets. On the Y axis we plot the value of the summary statistic (Formula 2) monitored by

CDCStream. The X axis represents the # of batches analyzed during the stream. Vertical lines are used to highlight when CDCStream detects a change. In order to easily visualize some interesting behavior we supply the trend between the first 350 and 500 batches. We can observe that our metod detects the important change observed around the batch 390. After that we can note that CDCStream continues to detect reasonable changes of the monitored statistic segmenting the time series in a reasonable way. We underlined that the segmentation algorithm implemented in CDCStream is general and can be employed to monitor any numerical time series in order to detect variations (increase or decrease) in the data distribution.

CONCLUSION

Detecting changes in a stream of categorical data is not straightforward. In this paper we presented a new algorithm in order to deal with this task. We adapt a previous technique in order to extract summaries from a batch of categorical instances. After that, we develop a new warning/change detection approach based on Chebyshev's Inequality that automatically segments the stream of statistics produced by the summarization step. A strong point of our approach is its ability to exploit historical information for decision making in a dynamic fashion, adjusting the history size automatically. The only parameters needed by CDCStream are related to Chebyshev's Bound and they have therefore clear semantics. The experiments fully validate the quality of CDCStream showing interesting results obtained by our strategy. As future work we plan to apply our method in categorical data stream clustering to update the model when changes are detected.

Figure 1 :

 1 Figure 1:A sample of changes detected by CDCStream on the Forest Sort. dataset with batch size equals to 1 000 between the batches 350 and 500

Table 1 :

 1 Dataset characteristics

	Dataset	# of Inst.	# of Feat.	# of Cl.
	Airlines	539 383	7	2
	Electricity	45 312	8	2
	Forest	581 012	54	7
	KDD99	148 517	41	2

Table 2 :

 2 Accuracy of models learnt by di↵erent strategies using warning/change framework under di↵erent batch size

Table 3 :

 3 Change behavior: number of changes triggered over the number of possible changes (Number of batches -1)


Tung Le-Duc

Gérard Meunier

Olivier Chadebec

Jean-Michel Guichon

Laurent Krähenbühl

A simple integral formulation for the modeling of thin conductive shells

In order to model thin conductive non-magnetic shells, an original surface integral formulation is proposed. The method is based on a surface impedance condition which takes into account the field variation through depth due to skin effect. It is general and enables the modeling of various problems whatever their skin depth and avoiding the meshing of the air region. The formulation is compared with another integral formulation recently proposed by authors and is validated thanks to an axisymmetric finite-element method (FEM). Advantages and drawbacks of this new formulation are discussed.

Introduction

Thin conductive non-magnetic shells are frequently encountered in electromagnetic devices (such as planar conductors and conductive shieldings) and the analysis of their eddy-current distributions is an important problem in engineering. The geometric specificity of a thin region is characterized by a high ratio between the length and the thickness, so using a volume mesh leads to a large number of elements. Furthermore, when frequency is high, skin depth can become much thinner than thickness e, leading to an increase of the size of the mesh if this effect has to be properly modeled. Shell element formulations have been developed to overcome such difficulties, e.g., with the boundary element method (BEM) [1], with the finite-element method (2D formulation [2] and 3D formulation [3][4][5][6][7][8]), and with an integral method recently, proposed by the authors of this paper [9].

In [9], a general shell element formulation for modeling thin conductive regions has been proposed (δ e or δ ≈ e or δ e). Like in [1][2][3][4][5][6][7][8], this element takes into account the field variation through depth due to skin effect. The shell is modeled by an integral formulation avoiding the mesh of air region. Based on a simple surface discretization of the shell, the number of unknowns considerably reduces, the depth of the shell being not meshed.

The present method is a natural reduction of the method proposed in [9]. However, we will see that the formulation proposed in this paper is less consuming in terms of degrees of freedom than in [9] and the complex computation of volume singular integrals is avoided. On the other hand, the computation of eddy-current losses is not possible so this new formulation is reliable for applications where only magnetic field computation in the air region is focused.

In Section 2, the integral equation governing the thin shell regions will be presented. In Section 3, this equation will be coupled with a surface integral formulation. Finally, two numerical examples will be proposed in the last part. Results obtained with our formulation will be compared with those given in [9] and the finite-element method (FEM).

Formulation

An equation for the thin region

We consider that a small skin depth δ is associated to the non-magnetic shell with a thickness e (Fig. 1). The field variation of the longitudinal component across thickness of shell can be approximated by the analytical solution of the problem for an infinite plane [1,3]:

H s (z) = 1 sh(ae) H 1s sh ae 2 + az + H 2s sh ae 2 -az ,
(1) where a = (1 + j)/δ, H 1s and H 2s are the field tangential values on both sides of the shell. Applying Galerkin method on the Maxwell-Faraday equations for side "1" of the shell on a real 3D surface Γ 1 of the shell, we get [3]:

Γ1 grad s w•(αH 1s -βH 2s )dΓ+jω Γ1 wB 1 • n 1 dΓ = 0, (2)
where α = a σth(ae) , β = a σsh(ae) ; w is a set of nodal surface weighting function; n 1 is the normal vector corresponding to the side "1" of the shell.

The other equation corresponding to the other side of the shell is obtained when indices "1" and "2" are exchanged:

Γ2 grad s w•(αH 2s -βH 1s )dΓ+jω Γ2 wB 2 • n 2 dΓ = 0, (3)
where n 2 is the normal vector corresponding to the side "2" of the shell.

Equations ( 2) and (3) represent the electromagnetic behavior of the conductive shell itself. In order to take into account the surrounding air region, it remains to couple both equations. In [2][3][4][5][6][7][8], the authors choose to use the finite-element method (FEM), thus they need to mesh the air region. In [1], a boundary element method (BEM) is preferred but the authors have to manage the coupling on both sides of the shell and have to determine if both sides of the shell are interfaced with a single region or with two different ones. This leads to the need of quite sophisticated geometrical analysis tools if the method has to be applied to general geometries. Moreover, four unknowns per node are needed. In [9], we have developed an integral approach based on the coupling of ( 2) and (3) with a volume integral equation. The obtained formulation presents an interesting compactness treating different air volumes as the single physical region. With this method two unknowns per node are needed but the computation of volume integrals can become very complicated to accurately integrate the exponential decreasing of the current across the thickness especially when the contribution of an element on itself is considered. This integration is quite hard to achieve and is certainly the weak point of the method.

In this work, a new approach is proposed, based on the coupling of ( 2) and (3) with a simpler surface integral equation. This coupling only leads to one unknown per node and is presented in the next section.

Surface integral formulation

Equations ( 2) and ( 3) are written on the averaged surface Γ of the thin region. Subtracting equations ( 3) and (2) leads to:

(α + β) Γ grad s w • (H 2s -H 1s )dΓ +jω Γ w(2 • B • n)dΓ = 0, (4)
where

B • n = 1 2 (B 2 • n 2 -B 1 • n 1 )
is the averaged normal induction on the surfaces Γ 1 and Γ 2 .

Let us now assume that the shell is placed in an inductor field H o . The total magnetic field H is the sum of H o and H r , the reaction field of the eddy current in thin shell:

H = H o + H r . ( 5
)
A reduced magnetic scalar potential is introduced for each side of the shell. Since there is a jump of the tangential component of the magnetic field through the element, a double layer node surface element is needed [9]. Expressions of tangential magnetic fields on both sides and outside the shell are:

H 1s = H os -grad s φ 1 H 2s = H os -grad s φ 2 , ( 6
)
where H os is the tangential source field and φ 1 and φ 2 represent the magnetic reduced scalar potential on each side of the shell. Using ( 4) and ( 6), we get:

(α + β) Γ grad s w • grad s δφ dΓ + 2jω Γ w • B • n dΓ = 0, (7)
where δφ = φ 1φ 2 is the discontinuity of scalar magnetic potential.

The equivalent surface shell current K can be expressed as in [1]:

K = e J(z)dz = n × grad s δφ. ( 8
)
Thanks to Biot and Savart law, we can write:

H r = 1 4π Γ e J × r r 3 dzdΓ ≈ 1 4π Γ K × r r 3 dΓ. ( 9
)
Using ( 6), ( 8) and ( 9), equation (7) becomes:

(α + β) Γ grad s w • gradδφ dΓ + 2jωμ o 4π Γ w • Γ n × gradδφ × r r 3 dΓ • n dΓ = -2jωμ 0 Γ w • H o • n dΓ. ( 10
)
This equation has now to be discretized. The easiest way is to mesh the average surface Γ, situated halfway between boundaries of the shells (Fig. 1), into n triangular elements associated to a uniform tangential component of the eddy current (meaning first-order shape functions for the potential). The algebraic linear system obtained has 2p unknowns (one complex magnetic scalar potential per node; the mesh being composed of p nodes). Let us notice that shape functions with higher orders can be used.

The discontinuity of scalar magnetic potential is expressed as follows:

δφ = p k=1 w k (x, y, z)δφ k . ( 11
)
The obtained matrix system is the following one:

[A] • [δΦ] -[Bs] • [δΦ] = [h 0n ], (12)
where

A(i, k) = (α + β) Γ gradw i • gradw k dΓ, (13)
B s (i, k) = 2jωμ o 4π Γ w i ⎛ ⎝ Γ k n × gradw k × r r 3 dΓ ⎞ ⎠ • n i dΓ, (14)
h 0n (i) = -2jωμ 0 Γ w i H 0 • n i dΓ. (15)
It must be pointed out that this formulation looks like a hybridization between finite-element method and an integral one. In [START_REF] Nguyen | Compression matricielle d'une formulation intégrale par la méthode multipôlaire rapide pour le calcul de courants de Foucault dans une région mince conductrice[END_REF], the first term leads to a sparse matrix A (similar to the one given by a FEM standard integration process) but the second term B s is associated to an integral method. The obtained matrix system is so fully populated and compression algorithms must be used if problems with large numbers of elements are addressed. For instance, the use of the well-known fast multipole method (FMM) [10,11] to compress Biot and Savart law is very efficient in such configuration [START_REF] Nguyen | Compression matricielle d'une formulation intégrale par la méthode multipôlaire rapide pour le calcul de courants de Foucault dans une région mince conductrice[END_REF]. Moreover, it must be pointed out that the integral ( 9) is computed only thanks to a surface integration of the averaged current distribution. This is an important difference with [9] where an accurate volume integration across the thickness is provided. This assumption can seem quite inaccurate especially for the computation of the element on itself when integration points and points where the field is expressed are closed. However, we will see with results obtained in the next section that the approach is not so inaccurate. On the other hand, we cannot compute the eddy-current losses. Indeed, this new method indicates only the field discontinuity through the shell, whereas the field on both sides is required for such a calculation.

Numerical example

Our new formulation has been implemented for 3D geometries and has been validated with two different academic examples. These examples are modeled by three different methods. The first one is the axisymmetric FEM using FLUX software [START_REF] Flux | [END_REF]. The second one is the volume integral formulation proposed in [9]. The last one is the considered surface integral method.

Hollow sphere

In the first example, a conductive hollow sphere (R = 0.1 m, e = 2 mm, σ = 6E7 S/m) placed in a uniform axial magnetic field H 0 = [0 0 1] (A/m) (Fig. 2) is considered. We focus on the computed magnetic field in the air region close to the device (calculated along the path AB in Fig. 2) with both methods for both cases:

-Case 1: f = 50 Hz, skin depth δ greater than the thickness e (δ = 9.2 mm, δ > e). -Case 2: f = 4000 Hz, skin depth δ smaller than the thickness e (δ = 1 mm, δ < e).

Results of the surface integral method are similar to those obtained with both axisymmetric FEM and integral methods with volume integration (Fig. 3). Because the external source field is the same on both sides (Γ 1 and Γ 2 ) and on the average surface Γ, both integral methods give almost the same results. We can see small differences between computed values with our formulation and the reference (axisymmetric FEM) (Fig. 4). These differences can be explained by the infinite plane assumption and certainly by the use of a first-order triangular coarse mesh. As already mentioned, a disadvantage of this new integral formulation in comparison with existing ones is that it cannot be used to calculate eddy current loss.

Thin conductive disk

The second test case is a thin conductive disk (R = 1 m, e = 20 mm, σ = 6E7 S/m) placed in a magnetic field H 0 created by a current loop (I = 1 A, h = R/4) (Fig. 5). We have computed the magnetic field in the air region close to the device (calculated along the path CD in Fig. 5) with both methods for two cases: -Case 1: f = 1 Hz, skin depth δ greater than the thickness e (δ = 65 mm, δ > e) (Fig. 6). -Case 2: f = 50 Hz, skin depth δ smaller than the thickness e (δ = 9.2 mm, δ < e) (Fig. 6).

If we consider the axisymmetric FEM method as our reference, the integral method proposed in [9] leads to a maximal error of 0.1% at f = 1 Hz and 0.2% at f = 50 Hz (Fig. 7). The new integral method proposed in this paper presents a quite similar accuracy. In comparison with the finite-element method, the maximum differences are 0.6% at f = 1 Hz and 2% at f = 50 Hz (Fig. 7). In this case, let us notice that the formulation leads to less accurate results than in [9]. Moreover, Figure 7 shows that closer to the shell the post-processing point is, less the accuracy of the surface formulation is. These results can be explained by the integration of Biot and Savart law in equation ( 9) which is certainly less accurate while [9] takes properly into account the current density exponential decreasing law across the thickness. Moreover, the exact computation of the external source field on each side of the plate as in [9] can lead to more accurate result, in particular when source coils are close of the thin region.

Let us notice that the CPU time of surface integral method is reduced up to a factor of 10 in comparison with [9] (see Tab. 1) and the memory space is also reduced. This is due to a less important number of degrees of freedom and a simpler integration process. These last results demonstrate the advantage of the method.

Conclusion

In this paper, we have presented a new integral formulation using shell elements in order to model thin conductive non-magnetic regions. The formulation is general and various skin effects across thickness (δ > e or δ ≈ e or δ < e) are taken into account. Moreover, it is very simple to implement in comparison with [9]. Even, if obtained results are a little bit less accurate close to the shell and losses cannot be computed, the formulation remains quite efficient if problems where only magnetic field in the air region is needed. The coupling of this formulation with compression algorithm seems to be promising in order to model real shielding applications. In further work, similar formulations enabling the modeling of thin conductive and magnetic shells will be developed.

2 Fig. 1 .

 21 Fig. 1. Thin conductive region.

Fig. 2 .Fig. 3 .Fig. 4 .Fig. 5 .

 2345 Fig. 2. Hollow sphere, notations and path where the external magnetic field is computed.

Fig. 6 .

 6 Fig. 6. Magnetic field along the path CD calculated by 3 methods in the second example.

Fig. 7 .

 7 Fig. 7. Difference in % with the axisymmetric FEM solution for the magnetic field computation along line CD (f = 50 Hz).

Table 1 .

 1 Memory space and CPU time (computer Intel CPU 5160 @ 3 GHz; 16 GB RAM).

	Method	CPU time Memory space
		(s)	(MB)
	Surface integral method
	(3000 surface elements)	170	144
	Volume integral method [9]
	(3000 surface elements)	1900	288


Janin Koch
email: janin.koch@aalto.fi

Wendy E Mackay
email: mackay@lri.fr

Jennifer Pearson
email: j.pearson@swansea.ac.uk

Makayla Lewis
email: makayla.lewis@arts.ac.uk

Andrés Lucero
email: lucero@acm.org

Simon Robinson
email: s.n.w.robinson@swansea.ac.uk

Miriam Sturdee
email: m.sturdee@lancaster.ac.uk

Where Art Meets Technology: Integrating Tangible and Intelligent Tools in Creative Processes

Keywords: Tangible Interaction, Machine Learning, Art, Design, Creativity

Art and design are essential aspects of our culture and how we interact with the world. Artists and designers use a wide selection of tools, whose impact is rapidly growing with the progression of digital technologies. This change has opened up new opportunities for the CHI community to build creative supportive tools. The digital switch has come with many benefits such as lowering barriers, mobile work environments and mass production for distribution of work.

Along with these benefits we also see challenges for art and design work and its future perception in society. As technology takes a more significant role in supporting art and design what will this mean for the individual artist or designer? The focus of this workshop is to bring together researchers and practitioners to explore what the future of digital art and design will hold. The exploration will centre around synthesizing key challenges and questions, along with ideas for future interaction technologies that consider mobile and tangible aspects of digital art.

CCS Concepts

•Human-centered computing → Human computer interaction (HCI); Interactive systems and tools; Haptic devices; User studies;

Background

Technology is playing an increasingly significant role in art and design [START_REF] Shneiderman | Creativity Support Tools: Accelerating Discovery and Innovation[END_REF]. As there is a growing diversity of tools for creative processes, this workshop aims to bring together people working in these spaces to facilitate discussions and exploration to technology for artists and designers. An increasing amount of intelligent tools aims to support creative practices in a co-creative manner. Especially in more abstract, tacit areas like design and art, human knowledge can enhance computational processes and vise versa [START_REF] Stacey D Scott | Investigating human-computer optimization[END_REF]. Machine learning based ideation tools, for example, aim to support the generation of ideas by providing verbal [START_REF] Andolina | InspirationWall: Supporting Idea Generation through Automatic Information Exploration[END_REF][START_REF] Bao | Momentum: Getting and Staying on Topic During a Brainstorm[END_REF][START_REF] Kita | V8 Storming: How Far Should Two Ideas Be?[END_REF], or visual material [START_REF] Koch | May AI?: Design Ideation with Cooperative Contextual Bandits[END_REF] to explore creative spaces together with the user by interactively contributing and interpreting inspirational material. This interactivity allows systems to work within the iterative processes which are fundamental to design. However, it can also be used to create more abstract artifacts in art and design, as when machines draw [START_REF] Davis | Empirically studying participatory sense-making in abstract drawing with a co-creative cognitive agent[END_REF][START_REF] Oh | I Lead, You Help but Only with Enough Details: Understanding User Experience of Co-Creation with Artificial Intelligence[END_REF], improvise dance [START_REF] Jacob | Viewpoints AI: Procedurally Representing and Reasoning about Gestures[END_REF] or improvise music [START_REF] Thom | BoB: an Interactive Improvisational Music Companion[END_REF] with users.

Figure 2: Mobile canvas prototype using hydrogels to simulate paints [START_REF] Steer | A Liquid Tangible Display for Mobile Colour Mixing[END_REF] The ability of machines to contribute own artifacts allows designers and artists to not only be more inspired, but also to use this material to work with and adapt ideas to their own needs and interpretations. We also consider mobile and tangible aspects for deploying intelligent tools. The versatility of mobile devises holds potential for allowing intelligent tools to become an integrated part of the process.

Similarly to this, physical and tangible materials have the ability to carry meaning and can be appropriated in creative support tools. To further enhance this tangible layer we see increased exploration into the possibilities of tangible art and design interfaces. This work aims to combine the benefits of physical tangibility with the freedom of digital tools, as with advantages of digital interactions comes a loss of physical engagement. In some cases these elements borrow from existing traditional methods [START_REF] Daniel | CavePainting: A Fully Immersive 3D Artistic Medium and Interactive Experience[END_REF], for example simulating brushes to make advanced styli [START_REF] Vandoren | Dip -It: Digital Infrared Painting on an Interactive Table[END_REF][START_REF] Vandoren | FluidPaint: An Interactive Digital Painting System Using Real Wet Brushes[END_REF][START_REF] Liu | FlexStroke: A Flexible, Deformable Brush-tip with Dynamic Stiffness for Digital Input[END_REF] or exploring possibilities of tactile sensations and realistic textures, which are usually limited in mobile digital experience [START_REF] Hirobe | Colorful Touch Palette[END_REF][START_REF] Steer | A Liquid Tangible Display for Mobile Colour Mixing[END_REF] by appropriating multiple devices as tangible tools (see Figure 2).

The active work in this space demonstrates the strong interest in designing, building, and testing improved tools, and the active commercial participation. Researchers could also take the next step to explore the impact of these developments in practice over long-term use. In this workshop, we will challenge researchers and practitioners to think deeper into how a combination of existing physical and computational tools could yield more usable, interactive and creative tools for the artists and designers. We aim to invite ideation of new physical form factors or digital concepts of art and design tools to understand how we can enhance these experiences.

Goals of the Workshop

The workshop provides a platform from HCI researchers and practitioners of different backgrounds to discuss the potential benefits and associated risks of tangible and intelligent technologies in relation to the field of art and design. It aims to create a deeper understanding of designing and evaluating new and existing interaction technologies and their societal implications to creative practices.

More concretely, our aims with this workshop are to:

• Identify the key opportunities that new interactions and tools can bring to the art and design community; • Explore contexts and work-flows and discover how such interactions can be deployed; • Investigate the implications of the use of digital tools on the perception of art and design from the perspectives of the creator and consumer. • Examine possible directions for prototyping and evaluating new digital and tangible artistic systems.

Organizers

The workshop organizers are researchers in the area of technology for art and design, focusing on both novel interactions, methodologies and societal impact. The contact person will be Janin Koch (janin.koch@aalto.fi).

Janin

Pre-Workshop Plan

Website and Social Media

We will create a website including the call for participants, submission details and contact information under userinterfaces.aalto.fi/WhereArtMeetsTechnology. Post-workshop deadline the website will be an archive of the accepted papers and plan of activities to be conducted during the workshop. We will also integrate a timeline for social media posts using the hashtag (#ArtmeetsHCI) among social media platforms to encourage a better collection of content. We welcome content from attendees on these channels and wider engagement. We will dedicate one organizer to collating and sharing further information from the workshop as the event progresses. After the workshop it will be a platform of discussion for further collaboration and the creation of a public article.

Recruitment

We intend to recruit a diverse set of scholars and practitioners from the HCI community and beyond who work with artists and designers, with AI and Machine learning to assist their processes, or with mobile, tangibles and shape-changing interfaces. We further encourage submissions that address societal views on using such technologies in creative practices. We will recruit via diverse channels e.g. CHI Meta Facebook Group, Twitter, as well as mailing lists of the HCI, Computational Creativity and Design Research Communities. We will reach out to practitioners via Social Media (e.g. IxDA), Industry conferences and meet-ups. We are looking for 15 to 20 participants, who will be asked to submit a 2-4 page positioning paper and will be selected by the quality of the submission and their professional background as well as their contribution to the workshop. The organizers represent a vast amount knowledge and thus will review the submissions.

Required facilities and specifications

The morning session will require a VGA projector as well as some simple note taking materials (pens and papers). For the rest of the ideation phase and hands-on session the instructors will provide paper prototyping materials and objects for constructing prototypes. More details are described in the workshop plan below. Student volunteers are not required but are very welcome to attend.

Workshop Structure

This workshop aims to foster and develop future research ideas in the field, along with bringing together like-minded individuals to build a community and encourage collaboration around shared ideas and perspectives. 11:50 12.45: Dialogue-labs: Introduction of an ideation method called dialogue-labs [START_REF] Lucero | Dialogue-labs: creating dialogue in co-design sessions[END_REF] developed by one of the co-authors (A. Lucero) who will guide this session. Participants will cycle around different stations with inspiring tools and materials (like paper, pens, lego) to aid the generation of new concepts and research questions (Fig. 3).

Lunch: Group lunch offering additional time for discussions and networking.

13:45 15:30: Prototyping activity : Organisers will first summarise the options from the morning session to the group. Afterwards each participant will vote on the ideas using provided sticky dot to take forward to prototyping session. We will bring a range of materials and a selection of tools for sketching and building practical resources (e.g. cards, pen, paper, lego). This activity will give the participants the best opportunity to present their ideas in a fun and hands-on way.

Mid-afternoon break

Planned Outcomes

Outputs such as specific papers arising from discussions or exercises will be prepared as appropriate. A summary of the collected challenges/ideas around prototyping such tools will be published as a blog post on the workshop website. To reach out to a larger community of practice related to these topics, we further target larger blogging platforms of the UX communities with our results (e.g. Medium). In line with this, we will use real-time documentation (e.g. video streaming, tweets, posts on social media platforms) for a larger audience, e.g. for those unable to join. We also see a potential for a pictorial submission to one of the other ACM conferences such as Creativity&Cognition, DIS or TEI, which we will discuss in the end of the workshop with the participants. The results of the workshop will be summarized and published on the workshop's website for archival purposes.

Call for Participants

This CHI 2020 workshop aims to create a deeper understanding of designing and evaluating new and existing interaction technologies, and their societal implications to creative practices. We invite submissions from practitioners and researchers in disciplines such as human factors, computer science, art, and HCI, who are interested in exploring digital technology in art and design practices.

In a one-day workshop, we will start off with lightning talks and demos from accepted participants. We continue with an ideation method called dialogue-labs to generate new ideas for future interaction technologies or evaluation methods for their societal implications on computational creative practices. In the second half of the workshop, participants will build prototypes using pen and paper, Lego, or other materials. The submission should address a topic related to novel tangible and digital technologies for art and design practices. This includes designing/exploring tangible prototypes for digital experiences, creativity support tools for practitioners, and AI/ML-based tools.

Participants are asked to submit 2 to 4-page position papers (in CHI Extended Abstract PDF format) via the workshop website (userinterfaces.aalto.fi/WhereArtMeetsTechnology) by 11 February 2020 presenting aspects of their own work (or work-in-progress) relevant to the core themes of the workshop, and areas of interest described in the proposal.

Submissions should also include a short personal biography (max 200 words). These papers will be reviewed by the organizing committee, supported by a wider program committee where necessary.

At least one author of each accepted position paper must register and attend the workshop. For questions, please contact Janin.Koch@aalto.fi

Figure 1 :

 1 Figure 1: Authors running workshops with designers at BBC

Figure 3 :

 3 Figure 3: Example of Dialogue-Lab activities from the author's research[START_REF] Lucero | Dialogue-labs: creating dialogue in co-design sessions[END_REF]

9: 00 :

 00 Introductions and scene setting from the organizers 9:10: Icebreaker activity : Each participant spends three-minutes creating a low-fidelity sketch, A5 size or smaller, that act as an introduction, they will then be asked to share their sketch. 09:40 -11:00: Lightning talks, demo's or presentations: Participants present their submitted papers to the other attendees in form of a lightning talks, demo or presentations (4 min, including max. 2 min Q&A). In the end of the first session we will introduce prepared scenarios to the attendees.Mid-morning break11:15 11.50: Group discussion: Participants will group around the scenarios based on their personal interest and start discussing potential ideas/concepts and open questions. In the end, each group pitches their ideas to the group (2 min each).

  Closing discussion: Overview of the gathered ideas and their impact on the wider HCI community, drawing upon the visual and written outputs. Stimulate discussion on how to prototype and evaluate such concepts in general. Discussion of potential publication targets.

	16:00-16:45: Presentation of results: Participants will
	present to the group, and graphic facilitator(s) will be in the
	room to live record their presentations. Two authors (Lewis
	and Sturdee) are experienced graphic recorders.
	16:45 -17:30: End of workshop
	18:00 -open end (optional): Social/networking activities:
	Continuing discussions in a close by location, which we will
	reserve beforehand

Acknowledgement

This work has been funded by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 637991) and CREATIV: Creating Co-Adaptive Human-Computer Partnerships (grant agreement No 321135).