Jean-Emile Dartois
email: jean-emile.dartois@b-com.com

Anas Knefati
email: anas.knefati@b-com.com

Jalil Boukhobza
email: boukhobza@univ-brest.fr

Olivier Barais
email: barais@irisa.fr

Using Quantile Regression for Reclaiming Unused Cloud Resources while achieving SLA

Although Cloud computing techniques have reduced the total cost of ownership thanks to virtualization, the average usage of resources (e.g., CPU, RAM, Network, I/O) remains low. To address such issue, one may sell unused resources. Such a solution requires the Cloud provider to determine the resources available and estimate their future use to provide availability guarantees. This paper proposes a technique that uses machine learning algorithms (Random Forest, Gradient Boosting Decision Tree, and Long Short Term Memory) to forecast 24-hour of available resources at the host level. Our technique relies on the use of quantile regression to provide a flexible trade-off between the potential amount of resources to reclaim and the risk of SLA violations. In addition, several metrics (e.g., CPU, RAM, disk, network) were predicted to provide exhaustive availability guarantees. Our methodology was evaluated by relying on four in production data center traces and our results show that quantile regression is relevant to reclaim unused resources. Our approach may increase the amount of savings up to 20% compared to traditional approaches.

I. INTRODUCTION

One of the main objectives of Cloud providers (CPs) is to ensure a good quality of service (QoS) for customers while reducing the operating costs [START_REF] Amiri | Survey on prediction models of applications for resources provisioning in cloud[END_REF]. To achieve this goal, CPs have built large scale data centers and massively adopted virtualization technologies to share resources between customers.

The number of hosts within data centers is often increased to handle load peaks, hardware failures, customer QoS demands, and to give the illusion of infinite resources to customers. However, this illusion comes with a price. Although the use of virtualization improved the utilization of computing resources in data centers [START_REF] Meng | Efficient resource provisioning in compute clouds via vm multiplexing[END_REF], several studies have demonstrated that the average usage of resources remains low, between about 20% to 50% in case of CPU [START_REF] Carvalho | Long-term slos for reclaimed cloud computing resources[END_REF].

One way to improve Cloud data center resource utilization and thus reduce the total cost of ownership (TCO) is to reclaim unused resources [START_REF] Marshall | Improving utilization of infrastructure clouds[END_REF] and sell them. However, reclaiming resources needs to be done without impacting customers' requested QoS. QoS is usually defined in terms of Service Level Agreements (SLA). In case of violations of these agreements, penalties are applied. The goal of CPs is to maximize the amount of reclaimed resources while avoiding the risks of violations due to resources overcommitment.

In a Cloud infrastructure, unused resources for some metric (mtr, e.g., CPU usage) at time t are defined as follows:

U nused (t,mtr) = Cap (t,mtr) -U sed (t,mtr) [START_REF] Amiri | Survey on prediction models of applications for resources provisioning in cloud[END_REF] where Cap (t,mtr) is the maximum performance reachable by the system for mtr at time t; and U sed (t,mtr) is the used capacity for mtr at time t.

Google and Amazon proposed to take advantage of unused resources by leasing them at a lower price compared to regular ones (e.g., dedicated resources). In [START_REF] Amiri | Survey on prediction models of applications for resources provisioning in cloud[END_REF], the authors proposed a similar approach. However, reclaimed resources are coming with limited to no SLA guarantees, which reduces the number of applications that can be deployed [START_REF] Carvalho | Long-term slos for reclaimed cloud computing resources[END_REF].

In order to achieve SLA on top of unused reclaimed resources, several challenges have to be tackled: 1) Estimating Cap (t,mtr) to determine the real system capacity (e.g., considering inter-workload interference) 2) Estimating U sed (t,mtr) can be described with the following properties:

• Granularity: The level at which the estimation is performed (e.g., data center, cluster, or host level).

The finer is the granularity, the more complex is the estimation, and the better the usability of the model. • Flexibility: The estimation needs to be flexible enough to give the CP the opportunity to find the best tradeoff between the amount of resources to reclaim and the risk of SLA violations. • Exhaustivity: An estimation should be based on several resource metrics to achieve SLA requirements for reclaiming the maximum amount of unused resources. For example, if there are free CPU resources without available memory, this may lead to SLA violations. • Robustness: estimations should be robust to workload change as deployed workloads have vastly different runtime characteristics [START_REF] Shaari | Dynamic pricing scheme for resource allocation in multi-cloud environment[END_REF]. • Applicability: estimation techniques should not have high overheads in terms of time and computing resource requirements as compared to the potential reclaimable resources. 3) Designing a scheduling strategy that takes into account the estimation of unused resources, the estimation of errors, QoS requirements, hardware/software constraints, affinity and anti-affinity [START_REF] Delimitrou | Hcloud: Resource-efficient provisioning in shared cloud systems[END_REF].

In this paper, we focus on the second challenge by investigating how to provide an accurate estimation of the future used resources U sed (t,mtr) (see eq.1). Our goal is to maximize the leasing of unused resources which, in turn, will maximize potential cost savings for the CP. Most state-of-the-art resource prediction studies [START_REF] Amiri | Survey on prediction models of applications for resources provisioning in cloud[END_REF] have focused on the estimation of the mean load. This makes those solutions poorly flexible as the mean load highlights only one aspect of the distribution of a variable (e.g., CPU) without considering peak values (i.e., SLA violations). In addition, most studies [START_REF] Amiri | Survey on prediction models of applications for resources provisioning in cloud[END_REF] do not rely on an exhaustive set of metrics. Indeed, they are mostly based on two resource metrics: CPU and/or RAM consumption. Unfortunately, relying on one or two single metrics in a data center is not realistic since applications often require multiple computing resources. In effect, network and storage resources play a major role in SLA violation avoidance [START_REF] Amiri | Survey on prediction models of applications for resources provisioning in cloud[END_REF]. The authors of [START_REF] Carvalho | Long-term slos for reclaimed cloud computing resources[END_REF] provided an interesting investigation about forecasting the unused capacity in order to provide SLA over spare resources. Even though they provide a robust model, the granularity was too large. The cluster level (i.e., aggregation of all hosts) was chosen. As it will be detailed further in the paper, the resource usage distribution among hosts in a given data center is not homogeneous, which makes it hard to design a scheduler strategy able to deploy applications among the hosts given the cluster level spare resource prediction.

In this paper, we investigate how to design the model for future unused resource while achieving SLA guarantees. We used three learning algorithms: Gradient boosting decision tree (GBDT), Random Forest (RF) and LSTM (Long short term Memory). We chose the host level granularity to make the model suitable and easily usable for deployment. One of the key contributions of our study is the use of quantile regression to make our model flexible for the CP, rather than using the simple mean regression of resource usage. This makes it possible for a CP to make relevant and accurate trade-off between the volume of resources that can be leased and the risk in SLA violations. We use six resource metrics (i.e., CPU, RAM, disk read/write throughput, network receive/transmit bandwidth) for the forecast to be exhaustive enough and allow more accurate allocations.

For robustness concerns, we evaluated our approach using six months of traces about resource usage from four different data centers (i.e., two private companies, one public administration and one university) composed of more than fifty hosts. We evaluated several metrics such as the prediction accuracy per host and for a collection of quantiles. In addition, for applicability issue, we measured the training and forecast time to determine the overheads.

We have also evaluated the economic impact of our contribution for comparison. Our results show that the use of quantile regression may provide an increase of the potential cost savings by up to 20% with LSTM and about 8% for GBDT and RF as compared to traditional approaches (see Section IV).

Our contributions can be summarized as follows:

• A technique that relies on machine learning and quantile regression that makes it possible to trade-off between the amount of reclaimable resources and SLA violations. • A comparative study of three machine learning algorithms (RF, GBDT and LSTM) with six quantile levels. • An evaluation on real traces of more than one hundred hosts from four data centers for a six-month time period.

Outline of paper.

Motivations based on real production data hosts are given in Section II.Our methodology is described in Section III. Section IV details the experimental evaluation per-formed. Section V reviews related work. Finally, we conclude in Section VI.

II. MOTIVATION: DATASETS ANALYSIS

This section motivates the work in this paper by providing some analysis about four in-production data center traces. These traces were collected between 2015 and 2017 from various types of organizations (i.e., one University, one public administration and two private companies).

First, we focus on one data center at the host level, and then we give an overview of the resources for all data centers.

1) Private Company 1: Table I shows the hardware characteristics of the hosts for private Company 1. A first observation one may draw is that its hosts are heterogeneous (proportion between CPU and RAM). Thus, reclaiming resources on some hosts would be more effective than on others. Let us focus on CPU and RAM in this section. Fig. 1a shows the box plots of CPU usage for the nine hosts. We observed that 75% of the time, the CPU median usage is under 40% for the hosts 12.0.0.1, 12.0.0.2 and 12.0.0.5. For the other hosts, the CPU median usage is even less than 20% during 75% of the time. We notice in the box-plots of Fig. 1b that the median usage of RAM is higher (about 50%) compared to CPU. This may be explained by the fact that in a virtualized environment the RAM is progressively allocated to the virtual machines but never released except when a memory management technique, such as memory ballooning, is enabled.

2) Potential Reclaimable Resources: Table II shows the overall capacity of the data centers used in this study. The Private Company 2 data center is the largest one with 356 cores and 3.8 TB of memory provided by 27 hosts. To conclude, from the four data centers investigated, all of them have a low resource usage. This encourages the use of reclaiming techniques. Secondly, in a given data center, configurations appear to be heterogeneous, and so resource usage is not balanced among hosts. This motivates the design of reclaiming technique at the host level granularity.

III. METHODOLOGY

Our goal is to provide a solution that maximizes the leasing of unused resources on a set of heterogeneous Cloud infrastructures (e.g., VMware, OpenStack). Among the challenges discussed in the introduction, predicting the future use of hosts resources is an important issue. This forecasting has to be robust, fine-grained, flexible and exhaustive. In this section, we present our methodology that aims in building such a prediction model based on different learning machine algorithms. We introduce quantile regression as a technique to limit the risks of SLA violation at the cost of limiting the resources to be sold. We applied our methodology by replaying six months of four real data centers traces.

A. Background on quantiles

Quantiles are data values that divide a given dataset into adjacent intervals containing the same number of data samples [START_REF] Benson | A note on the estimation of mean and standard deviation from quantiles[END_REF]. They are useful to gain insight about the distribution of a random value (e.g., CPU utilization) noted Y as compared to its mean value. Conditional quantiles investigate the behavior of Y by considering another vector of variables noted X that provides additional information. For example, the time (hour, minute) or historical values of CPU are variables that may be useful to describe CPU behavior (some VMs may be switched off during some period of time each day). The main advantage of conditional quantiles is to give a more comprehensive analysis of the relationship between X and Y at different points in the conditional distribution of Y given X = x. Quantile regression [START_REF] Koenker | Regression quantiles[END_REF] seeks to estimate conditional quantiles. Rather than estimating the mean value of the CPU at a given time stamp, this regression method allows to estimate the τ th quantile (e.g., the 0.75th quantile or the CPU utilization value for which 75% of the values are lower).

In Fig. 2, we have estimated the conditional mean of the CPU usage (see Fig. 2a) and a collection of quantiles (i.e., 0.05, 0.25, 0.5, 0.75 and 0.95) (see Fig. 2b) for a given time window. Quantile regression offers several levels of quantiles that gives us the opportunity to select the one that finds the best trade-off between SLA violations and available unused resources as compared to the conditional mean. As the quantile level increases, the amount of spare (reclaimed) resources decreases, the lower the risk of SLA violation. This is the main reason for the use of quantile regression for reclaiming unused resources in our study.

B. Approach overview

Our approach is composed of three steps as shown in Fig 3 :   • A forecast strategy step: we chose to investigate three machine learning algorithms with their corresponding quantile approach. Our objective is to build a forecast model that infers future responses (e.g., CPU, disk) from a set of past traces with different quantile levels. As a consequence, our problem fits in the supervised learning category. Since we want to forecast six metrics (e.g., CPU, RAM), we used regression-based algorithms. We have evaluated RF, GBDT and LSTM (see Section III-C). • A data pre-processing step: we prepared the extracted datasets from the data centers by applying the following operations: down-sampling, normalization, missing value handling, and features extraction (see Section III-D). • An evaluation step: we replayed six months traces from four data centers by extracting all test windows of 24 hours and their associated training set per host. Then, we built prediction models with six quantile levels (i.e., 0.5, 0.6, 0.7, 0.8, 0.9, 0.99th). We evaluated the accuracy, the training time and the potential economic savings induced by reclaiming resources (see Section IV).

C. Forecast Strategy step

In a Cloud infrastructure, forecasting future resources demands is a challenging task, especially for long periods of time [START_REF] Amiri | Survey on prediction models of applications for resources provisioning in cloud[END_REF]. Many variables may influence resources usage such as the deployed applications, the user behavior and the period of the day [START_REF] Shaari | Dynamic pricing scheme for resource allocation in multi-cloud environment[END_REF].

1) time series: Most CPs store their cluster resource usage indicators in time series. A time series is a sequence of N measurements {y 1 , y 2 , ..., y N } of an observable metric (e.g., CPU, RAM), where each measurement is associated with a time stamp. As confirmed in [START_REF] Taieb | A bias and variance analysis for multistepahead time series forecasting[END_REF] time series forecasting methods can reliably be used for cloud resource demand prediction. In this study, we use two strategies to forecast time series: (1) a static strategy that seeks to find a relationship between values of different time series. a dynamic strategy called Multiple-input and Multiple-output (MIMO) which can predict the whole sequence of values [START_REF] Bontempi | Long term time series prediction with multi-input multioutput local learning[END_REF].

These strategies were used in the context of quantile regression. Indeed, quantile regression may provide the administrator with more flexibility to decide about the quantity of resources to rent as compared to the traditional conditional regressors.

2) Conditional quantile: There are two approaches to estimate the τ th conditional quantile. To summarize these two approaches, we used the following notation:

• X a vector of p features (e.g., working hours) • Y an output variable (e.g., CPU usage)

• y 1 , y 2 , ..., y n sampled values from Y • x one observation of X • F (.|x) the conditional Cumulative Distribution Function (CDF) of Y given X = x • E the mathematical expectation
The direct approach consists in minimizing a sum of asymmetrically weighted absolute residuals [START_REF] Goh | Design-adaptive nonparametric estimation of conditional quantile derivatives[END_REF] based on:

q τ (x) = arg min µ(x) E (ρ τ (Y -µ(x))|X = x)
where ρ τ is the following loss function introduced by Koenker and Basset [START_REF] Koenker | Regression quantiles[END_REF] and τ is the quantile level:

ρ τ (u) = τ u u ≥ 0 (τ -1)u u < 0 (2)
This loss function is asymmetric, except for τ = 0.5 (median). The indirect approach is performed in two steps, the first one estimates the conditional CDF. Then, the τ th conditional quantile of Y given X = x is obtained via inversion of the estimated conditional CDF [START_REF] Knefati | Local linear double and asymmetric kernel estimation of conditional quantiles[END_REF] based on:

q τ (x) = F -1 (τ |x)
3) Machine learning algorithms: We have investigated three algorithms:

• RF and GBDT for the static forecasting strategy, these algorithms was recognized to be the best potential choice according to [START_REF] Friedman | The elements of statistical learning[END_REF], [START_REF] Amiri | Survey on prediction models of applications for resources provisioning in cloud[END_REF] and [START_REF] Caruana | An empirical comparison of supervised learning algorithms[END_REF] • LSTM algorithm for the MIMO forecasting strategy, where it proved its efficiency in the context of workload prediction as in [START_REF] Kumar | Long short term memory recurrent neural network (lstm-rnn) based workload forecasting model for cloud datacenters[END_REF], [START_REF] Song | Host load prediction with long short-term memory in cloud computing[END_REF] The interesting characteristics of these algorithms are summarized in Table IV. based on [START_REF] Friedman | The elements of statistical learning[END_REF].

In what follows, we describe each of the three learning algorithms used that seek to estimate the τ th conditional quantile of U sed (t,mtr) given X = x. The accuracy of a model depends strongly on the dataset and the learning algorithm used. It also depends on the algorithm tuning parameters, called hyperparameters. These hyperparameters impact the complexity of the learning model, and they are estimated so as to minimize the error. This is why in this section we give some elements about their configuration.

Random Forests (RF), introduced in [START_REF] Breiman | Random forests[END_REF], they enhance decision trees by building a large collection of de-correlated trees, and then averaging them. RF are a combination of CART (Classification and Regression Trees) models, which are binary trees, such that each model depends on the values of a random vector sampled from training data independently with the same distribution for all trees in the forest. In CART, the split aims to maximize the accuracy score by splitting the training data with the best feature on each node of the trees. RF are very accurate and their hyperparameters are simple to tune [START_REF] Han | Large-scale prediction of long disordered regions in proteins using random forests[END_REF]. In [START_REF] Meinshausen | Quantile regression forests[END_REF] authors have proposed a variant of RF that proved to be able to work with the quantile indirect approach. RF have three hyper-parameters: the number of trees T and the following hyper-parameters for each tree:

• m : the number of features to consider when looking for the best split • n min : the minimum number of samples required to split an internal node In our study, the hyperparameters were set to T = 300, m = number of features 3 and n min = 5, as recommended by [START_REF] Breiman | Random forests[END_REF]. Gradient Boosting Decision Trees (GBDT) The main idea of GBDT is to train iteratively a decision tree such that the ensemble of these decision trees may be more accurate than any decision tree. In our study, we used GBDT proposed by Friedman [START_REF] Friedman | Greedy function approximation: a gradient boosting machine[END_REF]. GBDT has three main hyperparameters:

• M : the number of regression tree models

• : the size of trees • ν : the learning rate We set the hyperparameters of GBDT to = 6, ν = 0.09 and M = 300 as recommended by [START_REF] Friedman | The elements of statistical learning[END_REF]. In addition, the loss function used, ρ τ see eq. 2, was configured to estimate the quantile regression (i.e., direct approach). Long Short Term Memory (LSTM) Recurrent Neural networks (RNN) [START_REF] Bengio | Learning long-term dependencies with gradient descent is difficult[END_REF] were designed to capture dependencies within an input sequence and not only a single feature vector compared to RF and GBDT. To achieve that, RNN use hidden states that act as internal memory to keep information about previous inputs. In this way, RNN are useful for capturing temporal dependencies by tracing previous information.

Traditional RNN suffer from vanishing or exploding problem during the back-propagation of the gradient weights on long sequences [START_REF] Bengio | Learning long-term dependencies with gradient descent is difficult[END_REF]. In [START_REF] Hochreiter | Long short-term memory[END_REF], the authors have proposed LSTM to address this issue.

The direct quantile approach was used with LSTM with the loss function ρ τ . The architecture used is composed of one input layer, one LSTM hidden layer, and one output layer fully connected. LSTM parameters are:

• Input layer : the number of time steps the model will be looking at. • Hidden layer : the number of neurons per step • Output layer : the number of steps to forecast • Batch size : the number of samples that are going to be propagated through the network • Epochs : the number iteration over the entire dataset As discussed in Section III-D, we used a sampling rate of 3 minutes, this means that one LSTM step is equivalent to 3 minutes. So, in order to forecast 24 hours we set the output layer to 480. The following hyperparameters are respectively set to, input layer size=20, hidden layer size=20, learning rate=0.09 , batch size=23, and epochs=90n see [START_REF] Song | Host load prediction with long short-term memory in cloud computing[END_REF] D. Data-pre-processing step

The goal of the pre-processing step is to create the matrix for input vector features, noted X, and the vector of the observed responses, noted Y from past traces. To achieve that, three operations have to be done: standardization/normalization, handling of missing values, and preparing the data for the learning.

The first step is standardization/normalization of the datasets. It turns out that depending on the dataset and thus the company, the sampling rate of the metric collection was not the same. The sampling rate has an impact on the accuracy and the processing time. A too low frequency would provoke the loss of system dynamism and thus may lead to SLA violations, but a too high frequency would cause an increase of the processing time. We down-sampled the measurements in order to aggregate a time range into a single value at an aligned timestamp. We chose a data sampling rate of 3 minutes as a good trade-off.

In addition, as recommended for LSTM we scaled the input features between zero and one.

The second step handles the missing values that are common in real deployments, the data can be corrupted or unavailable. To achieve that, we filled the missing values by propagating the last valid measurement.

The third step consists in preparing the data by extracting the features X and the output response Y from the datasets. This extraction has to be done according to the characteristics of the learning algorithms.

Concerning RF and GBDT, for each y i (i.e, U sed (t,mtr) ), we extracted the row of x i as follows:

• We extracted the day, hours and minutes features to investigate the timestamp information. We selected these features to allow learning algorithms to find the relationship between these features and resource usage. The feature month and year were not used since we trained our models using a one-month data. • We extracted, from the datasets, the holidays and working hours features (i.e., the feature is set to 1 for working hours, and 0 for hours of week-ends or holidays).

For LSTM, the training data required to use a sliding window in order to transform the time series (i.e., U sed (t,mtr) ) into a supervised learning problem.

E. Evaluation step

We evaluated our approach by replaying the six months traces from four data centers. One requirement to consider in order to evaluate a time series forecast compared to traditional supervised learning is to split the training and testing set sequentially in order to maintain the temporal dimension.

To achieve that, the six months of data were shifted into multiple sequential 24 hours windows. Each window is composed of a training and a testing part (i.e., forecast window). The test window is starting after the end of a training window. As we fixed the forecast window to 24 hours on six months, this gives 183 windows per host. Then, each window was evaluated with Normalized Mean Quantile Errors (NMQE).

IV. EVALUATION

This section describes the results of our experiments, by which we try to answer four research questions (RQ):

• RQ1 (Flexibility): What are the potential cost savings for CP with regard to different quantile levels ? • RQ2 (Exhaustivity): What differences can we observe in SLA violations when considering several resource metrics as compared to only CPU as in state-of-the-art work. • RQ3 (Robustness): What is the accuracy of the tested algorithms and how does the accuracy change according to the evaluated workloads ? • RQ4 (Applicability): What is the training overhead of the learning algorithms and its impact on the reclaimed resources ?

A. Experimental setup

To answer these four research questions, we led four experiments, each using production traces from four data centers. In this section, we introduce the elements used in common within these experiments:

• the experimental scenario used to calculate the potential cost savings for CPs in particular the leasing model, the pricing model and the penalty model. • the metrics used to evaluate the learning phase, • the experimental environment. 1) Experimental Scenario: a) Potential cost Savings: To calculate the potential cost savings for CPs, we defined three models. First, a leasing model to determine the period during which the customer rents the unused resources and their amount (resource granularity). Second, a pricing model to determine the fee that the CPs wold receive from the customer for the provided service. Finally, a penalty model that fixes the amount of discount on the customer bill in case of SLA violation. We assume that all reclaimed resources are leased. The cost savings estimations do not take into account the cost generated by the leasing such as the wear out of the hardware and energy consumption.

b) Leasing Model: For simplicity, we used a unique model based on the declared capacity of the hosts in the datasets. The leasing granularity is a container runtime provisioned for a period of 24 hours with 2 virtual CPU cores,8 GB Memory, and 100 Mbp/s of network bandwidth and the same for storage throughput.

c) Pricing Model: We used a fixed price based on a payas-you-go model since it is the dominant schema according to [START_REF] Shaari | Dynamic pricing scheme for resource allocation in multi-cloud environment[END_REF]. The price was fixed to 0.0317$ per hour for one leasing model as used by Google Preemptible VMs [START_REF]Preemptible virtual machines[END_REF]. d) Penalty Model: There are three types of penalties [START_REF] Garg | A sla framework for qos provisioning and dynamic capacity allocation[END_REF]: (1) a fixed penalty where each time the SLA is violated a discount is applied, (2) a delay-dependent penalty for which the discount is relative to the CP response delay, and (3) a proportional penalty where the discount is proportional to the difference between the agreed upon and the measured capacity.

Public Cloud such as OVH, Amazon and Google use a hybrid approach (Fixed penalty and Delay-dependent penalty). Table V shows the discount applied when SLA are not met.

Table V DISCOUNT APPLIES IN CASE VIOLATIONS FOR A 24-HOUR WINDOW Violation Duration [Minutes]

Discount > 15 to ≤ 120 10% > 120 to ≤ 720 15% >720 30%

2) Evaluation Metric: To evaluate the robustness of the learning algorithms and the potential cost savings, we used the NMQE and Interquartile Range IQR metrics. a) NMQE: A common metric to evaluate the quality of quantile regression is the Mean Quantile Error (MQE):

MQE = 1 n n i=1 ρ τ (y i,mtr -qmtr (τ, x i ))
In order to be able to compare accuracy with different metrics, we used a Normalized MQE (NMQE), given by: NMQE = MQE y max -y min b) IQR: The interquartile range (IQR) is a measure of statistical dispersion (i.e., NMQE's) given by:

IQR = Q 3 -Q 1
where Q 3 and Q 1 are respectively the 0.75 and 0.25 quantiles.

3) Experimental environment: We made use of Python with the following packages: scikit-garden1 in case of RF with version 0.1 and scikit-learn2 for GBDT version 0.18 and keras3 for LSTM libraries which provide state-of-theart machine learning.We also used Apache Spark [START_REF] Spark | Apache spark™-lightning-fast cluster computing[END_REF] version 2.0.2. Beside, all training and forecasts were performed on servers with an Intel(R) Xeon(R) E5-2630 v2 CPU clocked at 2.60GHz with 130GB of RAM.

B. Flexibility: potential cost savings (RQ1)

To evaluate the benefits of quantile regression for reclaiming unused Cloud resources while achieving the SLA, we compared the potential cost savings for CP with regard to different quantile levels. We conduct three experiments:

• Exp1. Using the Private Company 1 dataset, we compute the reclaimable resources amount using the three different learning algorithms. Based on these three models, i.e., the leasing model, the pricing model and the penalty model introduced in the previous subsection, we then calculate the potential cost saving in dollars according to the quantile level. • Exp2. We compared the behavior of the quantile regression depending on the host resources usage profile to investigate if the optimal quantile level changes according to the host resource usage. • Exp3. We generalize to the other data centers. 1) Exp1: Fig 4a shows the potential cost savings in dollars according to the quantile levels for Private Company 1 and for the three machine learning algorithms.

A first observation one may draw is that the potential cost savings increase with the increase of the quantile level up to τ =0.99 for both GBDT and RF and up to τ =0.9 for LSTM.

A second observation is that for each learning algorithm, there is an optimal τ level, which corresponds to the tradeoff between SLA violations and the amount of reclaimable resources (i.e., τ =0.99 for GBDT and RF and τ =0.9 for LSTM). This shows that with GBDT and RF, the decrease of reclaimable resources (increase of τ ) is compensated by the reduction of SLA violations. However, in case of LSTM when τ >0.9, this is not the case anymore: the reduction of unused resources is higher than the decrease of SLA violations.

A third observation is that the best amount of potential cost savings is obtained with LSTM with 3166$.

We conclude that for all learning algorithms studied, quantile regression brings a clear added value: (1) improvement in cost savings as compared to a median-estimation based approach (τ =0.5), and (2) a flexibility to adapt to the optimal level of τ according to the selected algorithm. This result can be generalized to all tested data centers as discussed farther.

2) Exp2: Fig. 4b shows the potential costs savings at a host level using LSTM for Private company 1. We notice two behaviors of the cost saving according to the quantile level:

(1) the hosts where cost savings increase up to τ =0.99; and (2) those where savings decrease starting from τ =0.9. We notice that all the the hosts obeying the first behavior are those with a low usage, such as 12.0.0.3, 12.0.0.4, 12.0.0.8. This can be explained by the fact that an increase in the quantile level does not imply a strong decrease of reclaimable resources, as the peak utilization of resources reaches a maximum of 40% for the CPU and 45% for the RAM. Even when τ increases, the loss of cost savings is less significant as compared to hosts with a high utilization (peaks that reach 100%) and with larger resource usage dispersions.

As expected, when comparing the cost savings with the measured resource usage (see Fig. 1a and1b ) we notice that hosts with a high usage, such as 12.0.0.2, 12.0.0.5 and 12.0.0.6 generate less savings, except for the host 12.0.0.1 which has extra memory of 170GB compared to the others.

We conclude that: (1) quantile regression makes it possible to adapt to resource usage heterogeneity in data centers and (2) the host granularity is relevant for reclaiming resources in a data center. 3) Exp3. Generalization on 4 data centers: Table VI shows the aggregated potential cost savings on the four data centers with regards to quantile levels and the three learning algorithms. We observe that for all datasets LSTM is the best choice except for the University where GBDT gives better cost savings.

Compared to the traditional approaches that use conditional mean (i.e., equivalent to τ =0.5), our approach based on the use of quantile regression performs better with an increased amount of savings of 8% for private company 1, 20% for private company 2, 9% for public administration and 4% for the university.

Overall one can observe that the use of quantile regression is useful for the three algorithms and four datasets and provide the required flexibility to reduce SLA violations. To illustrate the need to apply metrics-exhaustive models, we calculated the cost savings by taking into account only the CPU. We then subtracted the calculated savings to the results previously calculated by our six metrics model. Taking into account only CPU leads to an increase in SLA violations. Indeed, these violations get higher, up to -1317$ with τ =0.7 and then decrease down to τ =0.99 due to the reduction of reclaimable resources.

To conclude, we observe that a non-exhaustive choice of metrics leads to no savings as the penalties are higher compared to benefits. In addition, the use of quantile regression in a non-exhaustive way has increased the amount of violations.

D. Robustness: resilience to workload change (RQ3)

To evaluate the accuracy of the tested algorithms and observe its evolution along the various deployed workloads, we use N M QE and IQR indicators. These are used on all forecasting models and all hosts of private company 1 with a quantile equal to 0.9.

Table VII shows the resilience of the learning algorithms when facing various workloads for six months evaluated using N M QE and IQR indicators. We observe that all the forecast models have a quite good accuracy. We observe that RF has the best accuracy regarding the median of N M QE. It also provides the smaller dispersion given by IQR compared to the other algorithms. This means that RF is more inclined to be resilient to workload pattern change, which is a very interesting property. In addition, we observed that CPU and Disk Read were the metrics with the highest dispersion with a IQR of 0.71 and 0.62.

When comparing with the potential cost savings, we would expect RF to give the best results. However, as shown in Table VI LSTM did. This can be explained by the fact that the calculation of the potential savings only penalizes the negative errors (i.e., when the available unused resources are overestimated). Underestimation is not penalized directly compared to the indicator NMQE, which penalizes both positive errors and negative errors.

E. Applicability: training overhead (RQ4)

Training time is important as it is directly related to the amount of reclaimable resources. Indeed, the resources used for training and forecast would not be available for leasing. In this experiment, we evaluate the overhead (median computation time) to train and forecast each of the three learning algorithms for six metrics and a forecast horizon of 24 hours.

Table VIII shows the raw results. It turns out that LSTM was the slowest with a training/forecast time of about 500 seconds, as it has a high number of parameters to optimize. Then, with a duration of 130 seconds RF is slower than GBDT. This could be due to the fact that RF is estimating the quantile with an indirect approach that requires two steps. This means that for a data center composed of 100 hosts the learning phase would take about 12 hours each 24 hours with LSTM if we used a similar equipment to the one we experimented. In comparison, RF would take about 4 hours and GBDT 3 minutes. Note that the duration is highly related to the implementation of the learning algorithms and the choice of hyperparameters. When looking from the point of view of training/forecast computation time GBDT seems to be a good choice and LSTM the worst.

F. Threats to validity

Our experiments show the benefits of using quantile regression for reclaiming unused Cloud resources while achieving SLA. However, as in every experimental protocol, our evaluation has some bias which we have tried to mitigate. All our experiments were based on the same case study regarding the leasing model, the pricing model and the penalty model. We have tried to mitigate this issue by using models close to those of real Cloud providers.

One external threat to validity is our choice of data centers raw data. Further work is needed to reproduce our case study on other datasets, and we cannot guarantee that our results will apply to all data centers. We have tried to mitigate this issue by using datasets from different real CPs and different business cases.

Finally, there is a threat that the choice of hyperparameters are incorrectly set for the learning phases even if we relied on strong state-of-the-art work. If this happens to be the case, then all experiments introduce a similar level of imprecision, and a relative comparison of these may still be valid.

V. RELATED WORK

A. Reclaiming unused resources

Maximizing the utilization of Cloud computing resources can be achieved using various strategies. Some state-of-the-art studies [START_REF] Marshall | Improving utilization of infrastructure clouds[END_REF] took advantage of unused resources by leasing them with limited SLA. Others proposed to use predictive models in order to achieve SLA [START_REF] Carvalho | Long-term slos for reclaimed cloud computing resources[END_REF] by forecasting mainly the CPU. In addition, in [START_REF] Amiri | Survey on prediction models of applications for resources provisioning in cloud[END_REF], the authors noticed that the literature has focused on two metrics (i.e., CPU and RAM) while effective forecasting model should investigate all resources not only CPU and RAM in order to provide SLA. In [START_REF] Marshall | Improving utilization of infrastructure clouds[END_REF] authors make available any underutilized resources in an opportunistic way to improve resource utilization. In the same way, iExec [21], Amazon Spot instances and Preemptible Virtual Machines propose similar services. In [START_REF] Carvalho | Long-term slos for reclaimed cloud computing resources[END_REF] authors have proposed to claim unused Cloud capacities to offer a cheaper class (i.e., limited SLA) with long-term availability by forecasting available resource for the next 6 months. Compared to our work, the authors have focused on the forecast of aggregated CPU consumption (i.e, all hosts in the cluster). This does not provide the required level of granularity.

B. Time Series Forecast Algorithms

Many state-of-the-art studies such as [START_REF] Amiri | Survey on prediction models of applications for resources provisioning in cloud[END_REF] have discussed how to select the appropriate learning algorithm(s) to forecast time series with learning algorithms such as Autoregressive (AR), Integrated Moving average (ARIMA) and other more complex algorithms such as RNN, SVM, LSTM, and RF.

In [START_REF] Dinda | An evaluation of linear models for host load prediction[END_REF] authors used AR, MA, ARMA and ARIMA models or other variants to forecast the load average from 1 to 30 seconds in the future. The main drawbacks are the linear character of the ARIMA models, the lack of explicit seasonal indices. It remains hard to interpret coefficients or explain "how the model works". Yang et el. proposed several homeostatic and tendency-based one-step-ahead forecasting methods. The tendency-based method forecasts the future CPU value under the assumption that the pattern is stable which is not our case. Beghdad et al. [START_REF] Benhammadi | Cpu load prediction using neuro-fuzzy and bayesian inferences[END_REF] proposed to use neurofuzzy and Bayesian inferences for the the problem of CPU load forecasting. Gmach et al. [START_REF] Gmach | Workload analysis and demand prediction of enterprise data center applications[END_REF] studied the workload analysis for enterprise data center application. In this case, the workload analysis demonstrates the burstiness and repetitive nature of enterprise workloads. Song et el. [START_REF] Song | Host load prediction with long short-term memory in cloud computing[END_REF] applied LSTM to forecast the mean host load in data centers of Google and other traditional distributed system. They compared LSTM method with the following methods: Autoregressive (AR) model [START_REF] Wu | Load prediction using hybrid model for computational grid[END_REF], artificial neural networks(ANN) [START_REF] Duy | Improving accuracy of host load predictions on computational grids by artificial neural networks[END_REF], Bayesian model [START_REF] Di | Host load prediction in a google compute cloud with a bayesian model[END_REF], the PSR+EA-GMDH method [START_REF] Yang | A new method based on psr and ea-gmdh for host load prediction in cloud computing system[END_REF] and the echo state networks(ESN) [START_REF] Yang | Multi-step-ahead host load prediction using autoencoder and echo state networks in cloud computing[END_REF]. They have shown that their method achieves state-of-the-art performance with higher accuracy in both data centers. Kumar et al. [START_REF] Kumar | Long short term memory recurrent neural network (lstm-rnn) based workload forecasting model for cloud datacenters[END_REF] used LSTM networks to build a workload forecasting model. They showed that the accuracy of their forecasting model has reduced the mean square error up to 3.17 × 10 -3 . Islam et al. [START_REF] Islam | Empirical prediction models for adaptive resource provisioning in the cloud[END_REF] treated the case of resource provisioning in the Cloud. They used Neural networks, linear regression algorithms and a sliding window technique. Their approach supposes a linear fashion of the workload pattern which is not our case.

Our approach improves state-of-the-art solutions by applying quantile regression on the same machine learning algorithms but with a higher number of metrics. This is done to find the best trade-off between SLA violation and leased resources.

VI. CONCLUSION AND PERSPECTIVES

The use of quantile regression is a relevant approach to reclaim unused resources with SLA requirements. We described our technique that makes it possible to select the quantile level that gives the best trade-off between the amount of reclaimable resources and the risk of SLA violations. We evaluated three machine learning algorithms with regards to five properties granularity, flexibility, exhaustivity, robustness and applicability by replaying six months of four data centers traces (i.e., one public administration, two private companies, one university).

We drew four main conclusions. First, our results show that quantile regression provides the required flexibility that makes it possible to find the optimal quantile level that maximizes cost savings. Second, the most robust learning algorithm was given by RF with a median NMQE of 0.37 for Private Company 1 hosts. However, traditional accuracy metrics used in machine learning fail to determine the best algorithm that maximizes the potential cost savings while limiting SLA violations. Using our approach, it turned out that LSTM performs better on robustness for three data centers, with potential cost savings increasing up to 20 %. Third, as expected we need to be as exhaustive as possible to avoid SLA violations by taking into account a higher number of metrics. We measured that considering only CPU and omitting disk read/write, network reception and RAM leads to no savings, as the violation amount reaches -1317$ in the worst case. Fourth, for applicability concerns, GBDT was the one with the smaller computational overhead while LSTM had the highest overheads.

As for future work, we will use these results to design a scheduling strategy to deploy applications among the predicted unused resources. Finally, we plan to integrate more metrics in our model such as GPU usage or network latency.

Figure 1 .

 1 Figure 1. Box plots of (a) CPU and (b) RAM usage for each host with Private Company 1

Figure 2 .

 2 Figure 2. Forecasting of six hours of CPU with: (a) The conditional Mean curve in black, (b) Five different quantile regression curves.

Figure 3 .

 3 Figure 3. Overall Approach

  (a) Aggregated potential cost savings for Private Company 1 with exhaustive SLA metrics awareness (b) Potential saving with regard to the quantile level with LSTM and the nine hosts of Private Company 1

Figure 4 .

 4 Figure 4. Aggregated Potential Cost Savings

Figure 5 .

 5 Figure 5. Aggregated cost violations for Private Company 1 when there is no exhaustive SLA metrics awareness (i.e., only CPU) Fig 5 shows the cost of SLA violation when taking into account only CPU. With τ =0.5, one can observe that a nonexhaustive choice of metrics leads to a violation of about -1050$.Taking into account only CPU leads to an increase in SLA violations. Indeed, these violations get higher, up to -1317$ with τ =0.7 and then decrease down to τ =0.99 due to the reduction of reclaimable resources.To conclude, we observe that a non-exhaustive choice of metrics leads to no savings as the penalties are higher compared to benefits. In addition, the use of quantile regression in a non-exhaustive way has increased the amount of violations.

Table III AVERAGE

 III USAGE OF RESOURCES CALCULATED AT THE HOST LEVEL

	Name	CPU	RAM	Disk	Network
		Usage [%] Usage [%] R/W [Mb/s] In/Out [Mb/s]
	University	9.7	55.2	7.9/2.9	9.3/4.7
	Public Administration	14.4	54.1	12/7.5	2/6.4
	Private Company 1	17	57	10.6/3	7.9/2.1
	Private Company 2	10.9	48	1/0.3	7.1/7.7

Table IV LEARNING

 IV ALGORITHMS PROPERTIES : = GOOD AND =POOR.

	Characteristics	GBDT RF	LSTM
	Robustness to outliers
	Natural handling of data
	of "mixed" type
	Handling of missing values
	Computational complexity
	Forecasting accuracy
	Approach	direct	indirect	direct

Table VII MEDIAN

 VII (M ) AND INTERQUARTILE RANGE (IQR) OF N M QE FOR ALL FORECAST MODELS AND ALL HOSTS WITH 0.9 QUANTILE LEVEL WITH Private Company 1 DATASET.

	Metric	Indicator	RF	GBDT LSTM
	CPU	M IQR	0.37 0.71	0.48 0.91	0.57 0.97
	RAM	M IQR	0.00002 0.09	0.14 0.38	0.15 0.18
	Disk Read	M IQR	0.13 0.62	0.21 0.68	0.27 0.91
	Disk Write	M IQR	0.05 0.11	0.1 0.2	0.12 0.14
	Netreceived	M IQR	0.01 0.09	0.025 0.138	0.018 0.136
	Nettransmitted	M IQR	0.009 0.04	0.014 0.08	0.011 0.077

https://github.com/scikit-garden/scikit-garden

https://github.com/scikit-learn/scikit-learn

https://keras.io

ACKNOWLEDGMENT

This work was supported by the Institute of Research and Technology b-com, dedicated to digital technologies, funded by the French government through the ANR Investment referenced ANR-A0-AIRT-07.


Adrien Fonnet

Toinon Vigier

Grégoire Cliquet

Fabien Picarougne

Yannick Prié

Axes and Coordinate Systems Representations for Immersive Analytics of Multi-Dimensional Data

Keywords: immersive analytics, multi-dimensional data, coordinate systems, axes, virtual reality, user study

Axes are the main components of coordinate systems representations. They play a critical role for the visual analysis of multi-dimensional data. However their representation seems to have always be considered self evident, with oriented lines crossing at an origin, completed with labels such as ticks and names. Such classical representation show limits when it comes 3D visualization and immersive analytics (IA), mainly because orthogonal projection of points on linear axes is hard in a 3d environment, and because the user can move and the axes get out of her field of view. In this paper we propose a taskbased definition of axes and coordinate systems representation, as well as a tentative design space for coordinates systems representation in immersion. We also present an exploratory user study we carried out to compare three grid-based representations of coordinate systems for multidimensional data analysis with 3D scatterplots.

I. INTRODUCTION

In the data visualization pipeline, the mapping step consists in transforming multidimensional data into a visual abstraction, from which concrete views can then be created by further representing data points on a screen [START_REF]Readings in Information Visualization: Using Vision to Think[END_REF]. However the focus on data should not make us forget that such mapping is always done with respect to a space defined by a coordinate system, which also has to be represented on the screen for users to be able to make sense of datapoints [START_REF] Macdonald-Ross | How numbers are shown[END_REF]. Cartesian coordinate systems are grid based and allow to draw scatterplots, barcharts, gantt diagrams, etc.; while polar systems are circlebased (distance and angle being the main coordinates), and can be used for radar charts or chord diagrams. A coordinate system representation role is to provide the frame of reference from which data points coordinates (hence data values) are to be understood. The most important visual components of coordinate system representations are usually one or several axes, which are usually represented as lines. Those representations seem to have always be considered self evident in 2D, and coordinate system representations and axes as such are not really discussed in the literature, or even in textbooks such as [START_REF] Munzner | Visualization analysis & design[END_REF].

Immersive analytics (IA) is an emerging research field which aims to explore "the applicability and development This work is part of the IDEA project (Immersive Data Exploration and Analysis) funded by the Pays de la Loire French region through West Creative Industries program. of emerging user interface technologies for creating more engaging and immersive experiences and seamless workflows for data analysis applications" [START_REF] Chandler | Immersive Analytics[END_REF]. 3D data representations are critical components of Immersive Analytics, that are rendered on 3D devices such as stereoscopic displays, CAVEs, VR or AR headsets. Users can experience those representations from two positions, being either external to the data, or immersed within it. It has long been recognized [START_REF] Herman | Graph visualization and navigation in information visualization: A survey[END_REF] that in any case such visualizations are hard for users to understand because they lack sufficient information to perceive relative positions of objects, to evaluate metrics and distances, or to project points on axes. When it comes to immersion, users can nevertheless change their point of view, alleviating some of these perception difficulties. This may however come at the cost of the frame of reference getting out of sight.

While most of the IA systems proposals since the 90' have been based on simple extensions of 2D representations of coordinate systems (for instance using three orthogonal axes for a 3D Cartesian system), our initial concern is that those extensions may not be adapted to immersive analytics. If some works have acknowledged the need of going further, for instance by adding grids as way of complementing axes [START_REF] Sawant | The Tele-Immersive Data Explorer: A Distributed Architecture for Collaborative Interactive Visualization of Large Data Sets[END_REF], we claim that the development of Immersive Analytics calls for a renewed understanding of coordinate system representations, that may lead to potentially fruitful new representations, as well as the elicitation of best practices and guidelines for IA systems design.

Our contributions in this paper are the following: we propose to get back to the basic user tasks the representation of a coordinate system is meant to allow, and come up with three main tasks: understand space, get information, and search for data. We also propose an early design space for coordinate systems representation in immersive analytics based on seven dimensions. Finally we present an exploratory study comparing three orthonormal coordinate system representations based on a grid cube for multidimensional data analysis of 3D scatterplots.

II. RELATED WORK

The 1977 staple book "How numbers are shown" [START_REF] Macdonald-Ross | How numbers are shown[END_REF] provides a definition for a "coordinate grid": "[it] is based on two rectilinear bisecting straight lines: the horizontal axis is called the abscissa or x-axis; the vertical axis is called the ordinate or y-axis. The point of intersection of the two axes is known as the origin[...]". Axes are therefore a critical part of a coordinate system representation, and their representation as lines is clearly stated. Such definition of axes is inline with those of the Cambridge Dictionary1 : "a fixed line on a graph used to show the position of a point", or the Merriam-Webster2 : "one of the reference lines of a coordinate system". [START_REF] Macdonald-Ross | How numbers are shown[END_REF] also provides a few guidelines on axis representation, e.g. "the grid size should be as large as possible" or "axes should be numbered in the range 0.1 ≤ N ≤ 100". In the "Grammar of graphics" [START_REF] Wilkinson | The Grammar of Graphics[END_REF], axes belong to the "guide" category, together with legends and titles. Guides "help us associate categories or quantities with aesthetics", and axes are "position guides" that have scales, rules, and labels.

Coordinate system representation modes have been somewhat considered "self-evident" in data visualization, mostly based on linear or circular axes, within established data visualization idioms. Tufte's [START_REF] Tufte | The Visual Display of Quantitative Information[END_REF] proposals for data-ink ratio enhancement have put the attention on the necessity to focus on data rather than on axes systems and accompanying material. A few works addressed side elements of coordinate systems representations, for instance tick labels positioning, or definition of best minimum and maximum values for each axis [START_REF] Heckbert | Nice numbers for graph labels[END_REF], [START_REF] Talbot | An extension of Wilkinson's algorithm for positioning tick labels on axes[END_REF]. Grids and helpers are also used in practical data visualization, for instance additional line(s) that show specific values or zones. However they usually qualify as annotations of a data visualization rather than being part of its coordinate system representation. On a side note, some works specifically aimed at studying how axes as such could be considered as primary components for defining coordinate systems for multi-dimensional data analysis, where lines represent the datapoints. Parallel coordinates [START_REF] Inselberg | Parallel Coordinates for Visualizing Multi-Dimensional Geometry[END_REF] is a classical idiom, but other 2D representations have been proposed [START_REF] Tominski | Axes-based visualizations with radial layouts[END_REF], [START_REF] Claessen | Flexible Linked Axes for Multivariate Data Visualization[END_REF].

Focusing now on Immersive Analytics, there have been a number of system proposals for multi-dimensional data analysis, for which coordinates system representations had to be designed. Even though the term "immersive analytics" has only been defined in 2015 [START_REF] Chandler | Immersive Analytics[END_REF], immersive technology has been used since the nineties to analyze multi-dimensional data [START_REF] Symanzik | Dynamic Statistical Graphics in the CAVE Virtual Reality Environment[END_REF]. To the best of our knowledge, all of these systems did make use of axes for their coordinate systems representations. Most of them used a Cartesian orthogonal data visualization cubic volume, either with the origin at the corner [START_REF] Sawant | The Tele-Immersive Data Explorer: A Distributed Architecture for Collaborative Interactive Visualization of Large Data Sets[END_REF], [START_REF] Symanzik | Dynamic Statistical Graphics in the CAVE Virtual Reality Environment[END_REF]- [START_REF] Meiguins | Multidimensional information visualization using augmented reality[END_REF], or at the center [START_REF] Datey | Experiments in the Use of Immersion for Information Visualization[END_REF], [START_REF] Donalek | Immersive and collaborative data visualization using virtual reality platforms[END_REF] of the cubic volume. The IDEA system [START_REF] Fonnet | Immersive Data Exploration and Analysis[END_REF] explored the use of a cylindrical coordinate system. ImAxes [START_REF] Cordeil | ImAxes: Immersive Axes as Embodied Affordances for Interactive Multivariate Data Visualisation[END_REF] allowed the user to manipulate axes as representations of data attributes, and create various immersive visualizations, such as 2D and 3D scatterplots and parallel coordinates. In most of those systems, axes are simple lines which can be curved for cylindrical or spherical coordinate system [START_REF] Fonnet | Immersive Data Exploration and Analysis[END_REF]. Most of those lines have a simple white color, but some systems do use color conventions for axes such as (red for x, green for y, and blue for z) [START_REF] Azzag | VRMiner: a tool for multimedia database mining with virtual reality[END_REF], [START_REF] Meiguins | Multidimensional information visualization using augmented reality[END_REF] or (purple, green and white) [START_REF] Datey | Experiments in the Use of Immersion for Information Visualization[END_REF]. Origins in those coordinate system representation are not emphasized, they are simply the intersection of all 3 axes. ImAxes [START_REF] Cordeil | ImAxes: Immersive Axes as Embodied Affordances for Interactive Multivariate Data Visualisation[END_REF] and IDEA [START_REF] Fonnet | Immersive Data Exploration and Analysis[END_REF] both use arrows representations for axes. 2D grids are of common use to help users make sense of the volume. Those grids are always placed on a face of the cubic data volume, hence most of the time they connect two axes together [START_REF] Nagel | Methods for Visual Mining of Data in Virtual Reality[END_REF], [START_REF] Ammoura | Immersed Visual Data Mining: Walking the Walk[END_REF], [START_REF] Meiguins | Multidimensional information visualization using augmented reality[END_REF], with the exception of [START_REF] Donalek | Immersive and collaborative data visualization using virtual reality platforms[END_REF] where they are independent from the axes. Axes labels are in general represented with text in the vicinity of the related axis. Some systems only provide axis names [START_REF] Azzag | VRMiner: a tool for multimedia database mining with virtual reality[END_REF], [START_REF] Datey | Experiments in the Use of Immersion for Information Visualization[END_REF], other add minimum and maximum values [START_REF] Sawant | The Tele-Immersive Data Explorer: A Distributed Architecture for Collaborative Interactive Visualization of Large Data Sets[END_REF], [START_REF] Cordeil | ImAxes: Immersive Axes as Embodied Affordances for Interactive Multivariate Data Visualisation[END_REF], and values through regular ticks [START_REF] Nagel | Methods for Visual Mining of Data in Virtual Reality[END_REF], [START_REF] Meiguins | Multidimensional information visualization using augmented reality[END_REF], [START_REF] Donalek | Immersive and collaborative data visualization using virtual reality platforms[END_REF], [START_REF] Fonnet | Immersive Data Exploration and Analysis[END_REF]. Some IA systems propose additional spatial hints to help the understanding of the immersive world: VRMiner [START_REF] Azzag | VRMiner: a tool for multimedia database mining with virtual reality[END_REF] uses a different color (blue) for the ground face of the cubic volume, Symanzik et al. [START_REF] Symanzik | Dynamic Statistical Graphics in the CAVE Virtual Reality Environment[END_REF] use a checkerboard for the ground.

None of these IA papers justified their choice for axes representation. If a few papers commented on the importance of selecting relevant dataset attributes (e.g. numerical) to be mapped to each axis [START_REF] Azzag | VRMiner: a tool for multimedia database mining with virtual reality[END_REF], [START_REF] Meiguins | Multidimensional information visualization using augmented reality[END_REF], none did question the classical representation with lines and labels.

III. A TASK-BASED DEFINITION OF COORDINATE SYSTEMS REPRESENTATION

From what precedes it should be clear that most of IA systems proposals did adapt 2D representations of coordinates systems to 3D immersive settings, without further discussion of the rationale behind their choices. Moreover, to our knowledge only a few of them have been evaluated on their efficiency for immersive data analysis, and none was evaluated with regards to the choices related to its coordinate system representation. If Immersive Analytics are to become widely used there is both a need for theoretical work on coordinate systems representation in immersive settings, and practical work on system design and evaluation, leading to effective guidelines for those representations.

Our proposal is that a task-based definition of coordinate systems representations can be useful to help reflect about and design such representations, as it would take into account what axes are used for, rather than focus straight away on how they are represented. Such an approach seems inline with that of [START_REF] Wilkinson | The Grammar of Graphics[END_REF] who considered axes as position guides rather than representations.

The definition we propose is that a Coordinate System Representation is a component of a data visualization which serves as a tool for the reader to a/ understand the visualization space, b/ get information about data points, and c/ search for data points. In that context, an axis is a sub-component of a coordinate system representation that is linked to one coordinate. An axis may support some of the tasks by itself, or in conjunction with other axes. Besides axes, other ancillary sub-components of coordinate systems representation may provide help to the user to conduct these task, such as grids or minimaps. Let us detail the tasks. First, coordinate system representations are used to understand the visualization space: they define the limits of

Understand the Space

Get information about datapoints Search for datapoints limits coordinates of a point from a specific value for one attribute scale coordinates of a group of point from specific values for several attributes semantics difference between points from interval values for one attribute get ones position inside the space difference between groups of points from interval values for several attributes the space dedicated to data visualization, as well as the scale of the representation. From them users can get the semantics of the space and the associated mapping: ranges of values, represented attributes, etc. Those representation also help users understand their current positions in the space. One's current position is implicit in 2d data visualization: it is the external viewer position, which remains stable along the analysis. However in 3D and immersive environments for data analysis, users can navigate anywhere, and coordinate systems representation are also important for user orientation. Second, coordinate system representations are used to get information about data points, a task that is usually done with respect to axes. Each axis usually encodes one data attribute, thus providing semantics to a data point in the data space with respect to the attribute it represents. Users must be able to extract data points attributes values information at will, at least to get approximations, without further interaction such as detailson-demand. Therefore, coordinate system representations must allow users to determine the coordinates of a point or a group of points, e.g. finding the upper and lower limit of a cluster. Moreover, data analysis is also about comparing data, so those representations must provide ways for users to perform relative comparison between a pair or group of points. Third, coordinate system representations are used to search for data points. Starting from a known value of an attribute, or from several values of several attributes, users need first to know if there are data points that exactly or roughly correspond, then get access to those data points. Data points can also be searched with value intervals.

Table I provides a summary of the tasks coordinate system representations as tools must allow a user to carry out.

We should make two remarks here. First, one should note that this definition is not limited to 3D or immersive environments, and holds as well for more classical data visualizations. Second, in this definition we did not insist on interaction, as we want to focus on representations of coordinates systems that permit to carry out those tasks in a "direct" way. In standard 2D screen environment, this means direct visual perception, in immersive analytics however, we extend this directness of perception to head or body movements that help users adjust their point of view, as well as to other perceptual channels such as audition or touch.

In the remaining paragraphs of this section we discuss some of the IA related works we have seen in Section 2, with regards to our definition, and the tasks their coordinate system representations need to support.

IA systems that put the user outside of the data volume usually fulfill the "understand the space" tasks requirements. [START_REF] Meiguins | Multidimensional information visualization using augmented reality[END_REF] for instance use axes, 2D grids and labels to provide information about the limit of the space, its scale, and semantics. User can easily orient, likely due to similarity with external user position of classical screen-based 2D data visualization.

ImAxes [START_REF] Cordeil | ImAxes: Immersive Axes as Embodied Affordances for Interactive Multivariate Data Visualisation[END_REF] uses a similar approach with small sized, close representations to ensure the viewer can easily see the whole coordinate system representation. However when the user is fully immersed in the data volume, understanding the space may become quite a challenge. The Wizard system [START_REF] Datey | Experiments in the Use of Immersion for Information Visualization[END_REF] provides very limited cues for the user to understand the space limits and scale with only 3 axes with an origin centered in the data volume, associated with labels placed next to the axes. No others cues are available to help users grasp their positions inside a fully dark world, that is why a mini map had to be added. VRMiner [START_REF] Azzag | VRMiner: a tool for multimedia database mining with virtual reality[END_REF] provides cues to detect the limit of the space through a wireframe representation of the data volume, the "ground" face uses a different color, and each axis has a specific color to help both orientation and semantic understanding of the space. However, no tick label are available that would provide information about the scale of the space, an issue that is not present in IDEA [START_REF] Fonnet | Immersive Data Exploration and Analysis[END_REF]. Nonetheless, all those proposal share the issue that spatial cues can easily get out of the user field of view when the user navigate inside the data volume.

All systems rely on users projecting points on the axes to carry out the tasks related to "getting information about data points". While some are using ticks on axes [START_REF] Fonnet | Immersive Data Exploration and Analysis[END_REF], [START_REF] Cordeil | ImAxes: Immersive Axes as Embodied Affordances for Interactive Multivariate Data Visualisation[END_REF], other display 2D grids on the faces of the data volume [START_REF] Nagel | Methods for Visual Mining of Data in Virtual Reality[END_REF], [START_REF] Ammoura | Immersed Visual Data Mining: Walking the Walk[END_REF], [START_REF] Meiguins | Multidimensional information visualization using augmented reality[END_REF], [START_REF] Donalek | Immersive and collaborative data visualization using virtual reality platforms[END_REF]. A 2D grid is likely to improve projection by simplifying the process of getting the values on the 2 axes once the projection of a point is done on the grid, however this first projection remains an issue. Indeed, projection remains complex when the user is not aligned with one of the main directions of the coordinate system, a common situation when immersed in the data volume.

Providing information of the axes value is mandatory for users to be able to "search for data points", so solutions that do not provide tick label or 2D grid cannot be used for these tasks [START_REF] Sawant | The Tele-Immersive Data Explorer: A Distributed Architecture for Collaborative Interactive Visualization of Large Data Sets[END_REF], [START_REF] Symanzik | Dynamic Statistical Graphics in the CAVE Virtual Reality Environment[END_REF], [START_REF] Azzag | VRMiner: a tool for multimedia database mining with virtual reality[END_REF], [START_REF] Datey | Experiments in the Use of Immersion for Information Visualization[END_REF]. Otherwise conclusions are similar to those related to the "get information" tasks: 2D grids seem to be facilitating tools, but the task of constructing projected lines from axes so to aim at invisible zones where data points could then be found is far from easy.

From this analysis, we can see that designing coordinate system representations that satisfyingly fulfill all the tasks we proposed is hard, and that there is room for improvement. for coordinate system representations, and in Section V we present an exploratory study with a few proposals building upon the use of 2D grids.

IV. TOWARDS A DESIGN SPACE FOR COORDINATE SYSTEMS REPRESENTATION IN IA

In this section we make some proposals towards the building of a high-level design space for coordinate systems representations in immersive analytics. We then provide some illustrations of its use.

A. Design Space Proposal

Mac Lean et al. [START_REF] Maclean | Questions, options, and criteria: Elements of design space analysis[END_REF] define a design space as "an explicit representation of a structured space of design alternatives and the considerations for choosing among them different choices in the design space resulting in different possible artifacts". Our preliminary proposal is organized around 7 dimensions summarized in table II. For each of them we provide various possibilities, that we discuss with regards to the tasks we proposed earlier.

Type of coordinate system obviously has influence on representation. Cartesian, cylindrical, or spherical coordinate systems can be used for immersive analytics of multi-dimensional data.

Modality of representation is related to the sense(s) that can be used to perceive the coordinate system. Visual representation is by far the most common one but IA offers the possibility to embody the user in the immersive environment opening up potential for other senses, i.e. audition and touch. Senses may also be combined, for instance 2 axes may be visually displayed while a third one could use a sound metaphor. Visual representation allows to cover all tasks: the space limits are usually delimited by a visual cube and information can be accessed through direct projection. However, perceptible information is limited by the user field of view. That is why audition and touch could be used to keep perceiving systems of axes while vision is already used to focus on data points.

User position with regards to the data also provides constraints on the way the representation can be designed. As already seen, the user can have a position that is external to the data, for instance considering it at a small distance; or immersed within the data, and surrounded by data points. A third possibility would be internal and immersed implying the user can navigate freely between external and internal positions. Being external to the data may facilitate global understanding of the visualization space, while immersion may facilitate getting information from data points, as it becomes possible to navigate to an ideal position for the information extraction task at hand.

Dependency to user position states if the coordinate system representation is dependent or independent of the position of the user. External user independence seems a natural choice, but we can imagine that if the user can move around the data then the axes or grids may adjust to ease the reading, e.g. always placing grids behind datapoints, from the user perspective. For immersed users an independent position will facilitate understanding of the space and mental map building. However axes may get out of the field of view, and adapting their position to both user position and orientation would ensure information about the coordinate system is always visually available. Having both datapoints and coordinate system always visible at the same time would facilitate information extraction from datapoints, as well as searching for specific data. Combinations are possible, e.g. making some elements of the representation react to user position, while other remain independent.

Reachability is related to the distance of the whole coordinate system representation from the user. Arm's reach corresponds to a small representation that the user can observe and interact with with minimal movement. Few steps reach induces middle to human scale sized room volumes where only a few steps are required to access the entirety of the volume. Long distance reach is for systems that require important motion for exploring the data volume, with both physicalif the tracked space is big enough-and virtual locomotion.

Data orientation variability states whether the coordinate system representation presents information that concern the whole of the data, is tailored to a subset of it, or can provide information on both (a kind of focus+context based representation of coordinates). Being oriented towards the whole dataset can facilitate visualization space understanding, by providing limits and global volume of visualization. Data selection can be manual, or computed from proximity to user position. Obviously, an orientation towards a subset of data would be beneficial to get information, at the cost of limiting the understanding of the space, due to the change in the representation to fit the selection. Searching for data points would be limited to the part of the coordinate system representation that is accessible within the selection. Considering both orientations may prove useful by stabilizing the representation of the coordinate system for whole data and the understanding of the space (at the potential expense of information overload), while allowing to focus and get information on data points at hand.

Complexity variability is related to various ways the coordinate system representation can change in complexity. With regards to permanence, one can for instance imagine that the representation could disappear entirely to facilitate data remaining the center of attention [START_REF] Tufte | The Visual Display of Quantitative Information[END_REF], [START_REF] Few | Information Dashboard Design[END_REF] for patterns detection, and appear temporarily when the user needs it. This brings of course additional questions on how to make such representation appear, e.g. through active or passive user interaction, and how to manage the transition. Another variability could be on the details the coordinate system representation provides. An axis could for instance have various levels of ticks and labels available so as to allow increasing or decreasing information extraction precision.

It is important to note that these dimensions are certainly not complete, nor definitive, and that this design space has to be refined. Our proposal nonetheless already provides a basis to design coordinate systems representations for IA. In the following examples we rapidly show its capability to describe a classical representation, and present 2 potential directions for design as illustrations.

B. Examples

Our first example is a standard proposal that can be found in the literature: a Cartesian coordinate system composed of 3 axes represented as lines with 2D grids linking all axes effectively creating 3 faces for the whole cubic volume, with labels placed in the vicinity of axes both for the axes names and tick values. The cubic volume is small enough for the outside user to be able to see the full coordinate system representation at all time. The corresponding values on the dimensions of our design space are: Cartesian coordinate, visual representation, external user position, independence of the user position, at arm's reach, whole data orientation, and no variable complexity.

Our second example still uses a Cartesian coordinate system with a visual representation oriented towards the whole dataset, mainly composed of 3 linear axes. This time the user is immersed within the data volume, and the position of the axes are dependant of the user position and orientation using the following method: if the user can see one edge of the cubic volume, then one axis corresponds to this edge while the two others use the perpendicular faces to ensure the user can see them; if no edge is in the field of view, then the principal face in the user field of view is used to define 2 axes while the 3rd one is perpendicular to the face and placed at their intersection (see fig. 1). Further investigation would be required to determine precisely the axis movement law based on the field of view movement. In this example complexity of the representation is variable, i.e. the user can change the transparency of the axes, show/hide the semantics label, the tick labels, and the axes ticks. The corresponding values on the dimensions of our design space are: Cartesian coordinate, visual representation, immersed user position, dependency of the user position, long distance reach, whole data orientation, and no variable complexity. Our third example uses a spherical coordinate system combining visual and audio modalities, for a user immersed within the data volume, and no variable complexity. Both angular axes, latitude and longitude, are represented by curved lines with arrows, placed on the limits of the spherical data volume (see fig. 2). Axes become semi-transparent for the intervals that contain no selected data points. Moreover the origin of the sphere emits a continuous sound which allows the user both to locate the center of the data volume, and evaluates its distance from him, getting information about the third axis (radius). The corresponding values on the dimensions of our design space are: Spherical coordinate, visual and sound representation, immersed user position, independence of the user position, a few steps reach, subset of data orientation, and controlled complexity both on permanence and level of details.

V. A STUDY ON GRID CUBES

We have seen in the previous sections that classical 3 axes visual representations of Cartesian coordinates systems have problems with regards to the tasks a user needs to perform, all the more so when she is immersed in the data. Before putting to the test radically new designs such as examples 2 and 3 in the previous section, we wanted to experiment with the possibility to enhance those classical 3 axes representations for user orientation and information extraction.

A. Rationale

In this exploratory study we compare how users behave when analyzing data in 3 different conditions for coordinate system representation. As the most common representation proposal for 3D scatterplots analysis is to use 3 axes and 3 grids on each of the 3 main planes, our first design is meant to facilitate user orientation by using 6 grids, hence creating a complete cube around the user (condition 1). We extend this first design by focusing on space perception: in Fig. 2: Illustration of example 3: the black parts of axes are based on the selected data points (red). The origin of the sphere emits a continuous sound.

condition 2 we add information to the grid cube so as to facilitate 3D projection with 3D axes; in condition 3 we aim at facilitating "natural" evaluation of distances by using a human scale environment. Table III provides a summary of the 3 conditions, which will be detailed in the next section. User tasks focus on getting coordinate information from one, or between two data points.

B. Experimental material

Dataset. We used the same dataset for all three conditions. To build it we chose the classical "wine" dataset and used ImAxes [START_REF] Cordeil | ImAxes: Immersive Axes as Embodied Affordances for Interactive Multivariate Data Visualisation[END_REF] to quickly test various combinations of attributes for 3D scatterplotting. We chose a particular combination of dimensions (pH, alcohol, and total sulfure dioxyde) that resulted in a graph which both showed no clear correlation, and filled the entire volume nicely (all sub-part of the data volume had datapoints and density was not uniform. We then randomly selected 1000 datapoints.

Data mapping. Each datapoint is represented as a sphere (fig. 6), following Ammoura et al. recommendation to limit occlusion [START_REF] Ammoura | Immersed Visual Data Mining: Walking the Walk[END_REF]. Datapoints positions are then based on coordinates computed from the original dataset using linear interpolation from 0 to 10.

Condition 1 environment is a cube of 10 × 10 × 10 m with dark gray textured faces (fig. 3). We chose this size to be large enough to reach the limit of stereoscopic depth perception. The origin is placed in one lower corner of the cubic volume, and represented as a small white sphere. All 3 axes are represented by thin black cylinders using a direct coordinate system representation with the y-axis going upward. Each axis has a label at its extremity indicating X, Y, or Z, and every 2 meters a label displays the value at this location. A 2D grid is displayed on every face of the cubic volume with a 1 meter step. An additional color is added to every line of the 2D grids going from white (at the end corresponding to 10) to dark blue (at the other end). The gradient is aimed at helping read grids and their value orientation, see fig. 3 for an illustration. We chose blue as it is the best color for color gradient perception and so as to not saturate our environment with colors, since colors can also be used for data representation. Condition 2 environment is based on the first one, with the addition of a 3d grid that occupies the whole data volume (fig. 4). The 3d grid is composed of thin cylinders meant to provide both orientation and depth cues information based on their size. Those cylinders are positioned every 2 meters, a step we chose after several trials, so as not to disrupt data analysis by adding too much information. A transparent material is applied to the 3d grid to limit data points occlusion. The motivation behind this condition was to provide information that could facilitate 3D projection, hence improving precision.

Condition 3 environment also reuses the axes system and 2D grids of the first condition, with the addition of real-life elements that provide additional human-related natural spatial cues (fig. 5). We increased the world volume to 12 × 12 × 10 m to get further space for adding those elements outside of the main data volume. The represented human environment For all conditions, a virtual circular platform (1.5m diameter, see fig. 6,left.) is located at the feet of the user. It uses a transparent light gray texture to limit occlusion of both the data points and the environment, while providing a support to limit user discomfort.

We used Unity3D to develop the environments, and an HTC Vive as the virtual reality headset.

C. Experimentation Protocol

The participants had first to sign a consent form and answer a demographic questionnaire, next they were equipped with the VR headset, and got oral explanation about the tasks. They went through 4 positions to discover the first environment (center bottom, next to the origin, center, next to top corner), and then performed a set of tasks within, before repeating the discovery and the tasks in the second and the third one. Lastly they answered a post questionnaire, and an interview wrapped-up the session. Each session lasted about half an hour. The three conditions were proposed in a randomized order for each participant.

Participant were asked to perform two different tasks after being teleported to a position in the cube volume: 1/ determine each coordinate (x,y,z) of a specific data point (highlighted in red); 2/ determine the relative coordinates between two datapoints highlighted in green and red respectively. For each condition, task 1 was performed for 10 datapoints and task 2 for 5 datapoints couples.

Highlighted data and user positions were randomly chosen from two sets of predefined point(s) / position associations. The first set corresponds to task 1, it is composed of 100 point / position associations and was built by varying the following criteria: distance of the point from the user (close, medium, far), vertical distance of the point from the user (bottom, medium, top), occlusion of the point (none, a little, a lot), and distance of the point from the main grids of the environment (close, far). The second set corresponds to task 2, it is composed of 50 associations and was built by varying the following criteria: distance of point 1 from the user (close, medium, far), distance of point 2 from the user (close, medium, far), and distance between points (close, medium, far).

12 subjects (10 males and 2 females) from the University of Nantes and the Design School of Nantes participated to the study (mean age 25.8 ranging from 21 to 45). The experiment was run on a computer equipped with a CPU Intel Xeon ES-1630 v4 (3.70GHz), 16 Gb of RAM and a Nvidia GTX 1060 graphics card. Fig. 6: Left: a scatterplot is presented in a cubic system of axes (1st experimental condition). Right: the same scatterplot within the same system of axes now enriched with natural spatial cues (3rd condition).

D. Data

We collected data from a demographic questionnaire: gender, age, experience in data analysis (4-points Likert scale), and experience in virtual reality (idem). During the experiment answers to the questions were provided orally by the participant and captured in a spreadsheet by the examiner. As soon as the last coordinate was given, time was automatically recorded, and the user proceeded to the next point to analyze, randomly chosen from the set of predefined points for the current task. Qualitative feedback was collected at the end of the session, with 3 questions that had to be answered for all 3 conditions. The first two questions were about how suitable the environment was for the tasks, respectively task 1 and task 2, on a 7-point Likert scale (7 being "very suitable" and 1 "not at all"). The last question was about how much they liked experiencing the environment (7 being "very enjoyable" and 1 "not at all").

E. Results

We computed the accuracy error for each answer using the Euclidean norm over the accuracy errors on the X, Y, and Z axes. The means and standard deviations for each condition are presented in table IV (a lower value indicated a better performance both for accuracy error and completion time). Qualitative answers results are presented with box plots in figures 7, 8, 9. A Friedman's test was conducted for each of the 7 dependant variables, i.e. accuracy error for task 1, accuracy error for task 2, completion time for task 1, completion time for task 2, and qualitative feedback for each question. We used this non parametric test since our data was not respecting the condition required to run an ANOVA. The independent variables were the three conditions, resulting in a degree of liberty of 2. To complete our analysis, pair wise comparison test were conducted for any dependant variable that had a significant result from the Friedman's test. This resulted in condition 2 being significantly slower than condition 1 (p-value = 0.01), users found condition 2 significantly better for the first task than condition 1 (p-value < 0.001), condition 2 was found significantly better for the second task than condition 1 (pvalue = 0.003), condition 3 was significantly preferred compared to condition 1 (p-value = 0.018).

F. Discussion

Results indicate that using a 3D grid may have a negative impact on speed when it comes to getting coordinate information from data points, with no apparent benefit in accuracy. However it is important to note that 11 participants preferred the use of the 3D grid over the condition 1 to perform both tasks. Moreover, 5 participants also commented in the post task interview that using this 3D grid allowed them to define directly a delta between two points for the second task while in condition 1 and 3 they had to determine the absolute coordinates of each point before computing the difference. Comments seem to align with the better mean accuracy of the condition 2 for task 2, even though no significant effect could be found with the Friedman's test. The one participant not rating condition 2 over condition 1 was according to him because: "the 3D grid was distracting". To solve this latest issue, we can imagine the use of a non permanent 3D grid.

Condition 3 does not seem to provide any significant benefit over condition 1 from the speed and accuracy perspective. However users did rate condition 3 better than condition 1, according to the mean value but with no significant effect. This condition was significantly preferred by users compared to condition 1. This shows a potential benefit for user engagement, and one user even claimed that: "I was more confident about my answers when I was in the second environment" (condition 3 for this specific user). Interestingly, 3 users tried to use their own body as a reference metrics for the second task, extending their arms or using their own size, a behaviour that did not occur in the two other conditions. So if our implementation of real life human cues may not have been efficient, we consider that there are indications that natural cues should not be abandoned and need further design exploration.

Lastly, users never mentioned the color gradients on the grids lines, that were supposed to help them know at all time the orientation of each axis. So from users' remarks we cannot conclude if they actually served this purpose. The only element of answer that may be related is that the beginning and the end of an axis were inverted (e.g. giving an answer of 7 instead of 3) only 2 times out of the 360 answers provided for task 1. This may a hint that the cubic system with the 6 grids (condition 1) was sufficiently well designed to avoid such errors, however we cannot say if the color gradient had an impact on it, which calls for further study.

VI. CONCLUSION

In this paper we stated that the development of Immersive Analytics, which is still in its infancy, calls for a renewed understanding of coordinate system representations that may lead to the elicitation of best practices and guidelines for IA systems design. We proposed a task-based definition for coordinate systems representations, as well an immersive analytics oriented design space, and an exploratory study around the use of cubic grids for multidimensional data analysis. More work is however still needed both for the design space, which remains preliminary and calls for refinement; and for the practical design, implementation, and test of coordinate system representations in immersive settings.

Fig. 1 :

 1 Fig. 1: Illustration of example 2: on the left the user can see an edge of the cubic volume, on the right he cannot, user position dependant resulting axes in red.

Fig. 3 :

 3 Fig. 3: Condition 1 corresponds to a cubic environment of 10m*10m*10m with 2D grids displayed on each face. Each line of the 2D grid has a color gradient starting from white (representing a value of 10 of the parallel axis) to dark blue (representing a value of 0).

Fig. 4 :

 4 Fig. 4: Condition 2 adds a 3d grid on top of condition 1 environment.

Fig. 5 :

 5 Fig. 5: Condition 3 adds natural spatial cues on top of condition 1 environment.

Fig. 7 :Fig. 8 :Fig. 9 :

 789 Fig. 7: Box plot of the qualitative result (7-point Likert Scale) for the quality of the condition in regards to task 1.Friedman's test resulted in a chi square of 13.64 and a p-value of 0.001*

TABLE I :

 I Summary of the tasks coordinate systems representations must allow a user to carry.

  In the next section we propose a first version of a design space

	Dimensions	Attributes
		cartesian
	Type of coordinate system	cylindrical
		spherical
		visual
		audition
	Modality of representation*	touch
		smell
		...
	User positions with regards to the data*	external immersed
	Dependency to the user position*	dependant independent
		arm's reach
	Reachability	few steps reach
		long distance reach
	Data orientation variability*	whole data subset of data
	Complexity variability*	permanence level of details

TABLE II :

 II Summary of our design space proposal for coordinate systems representation in IA. Dimensions marked with * are non exclusive.

TABLE III :

 III Experimental conditions

Condition 1

Cubic volume with 2D grid on each faces Condition 2

Condition 1 + transparent 3D grid inside the volume Condition 3

Condition 1 + real-life elements with normalized dimension outside the volume

TABLE IV :

 IV

Means and standard deviations for accuracy errors and completion times for each condition and tasks, and chi square and p-value from Friedman's test for each dependant objective variable, significant result are in bold using α = 0.05 as a threshold.

https://dictionary.cambridge.org/

https://www.merriam-webster.com/