Keywords: Internet of Things, Indoor Localization, Meta-heuristic Optimization, Ultra-Wide Band, Time of Flight, Grey Wolf Optimizer, Arithmetic Optimization Algorithm

UWB Time of Flight based indoor IoT localization solution with Deep Learning optimized by meta-heuristics

Abstract: Nowadays, indoor localization is among the most important challenges in IoT networks. On the other hand, Deep Learning techniques are emerging as a leading method. Additionally, meta-heuristic algorithms attract several research domains due to its efficiency in resolving optimization problems. In this work, a Deep Learning model optimized by meta-heuristic algorithms and based on Time of Flight (ToF) measurements captured by Ultra-Wide Band technology, as an indoor IoT localization solution, is proposed. The findings showed that optimization with the Grey Wolf Optimizer can accelerate convergence towards optimal parameters during the learning phase. Furthermore, it was observed that ToF measurements enhance the positioning estimation capabilities, in comparison with RSSI and Range measurements. Compared with an existing multi-lateration method, the suggested solution provided more accurate positions of the IoT mobile object as it yielded better results in terms of localization accuracy (98.92%), Mean Absolute Error (0.057m) and Mean Squared Error (0.0095m).

Incorporating localization data can optimize IoT services and applications. Therefore, the development of localization solutions and methods is necessary for the proper functioning of these applications. However, indoor localization is among the most significant challenges faced by IoT networks. Indeed, users of IoT applications seek indoor positioning solutions that can give precise positioning of people and objects. However, the nature of indoor environments (walls and obstacles), prevent the radio signals emitted by satellites from penetrating these environments, making the use of satellite positioning systems difficult. In fact, for outdoor localization, Global Navigation Satellite Systems (GNSS) can be employed to provide real-time positioning of an element. These worldwide coverage systems include GPS (Global Positioning System), Galileo (Europe's Global Navigation Satellite System), Glonass (Russia's Global Navigation Satellite System) and BDS (China's BeiDou Navigation Satellite System).

The limited coverage of these systems in indoor environments necessitates the development of alternative solutions.

Therefore, in the literature, several indoor positioning systems were introduced. They are often classified as range-based and range-free methods. In range-free techniques, localization is established between objects by relying on the neighborhood information. On the other hand, range-based techniques are based on measurements taken in the studied environment, such as Time of Flight (ToF), Angle of Arrival (AoA) or Received Signal Strength Indicator (RSSI). For example, Ultra-Wide Band (UWB) ToF measurements provide high accuracy in the localization systems. Indeed, UWB [START_REF] Djosic | Fingerprinting-assisted uwb-based localization technique for complex indoor environments[END_REF] is considered as the most promising technology used in critical applications that require high precision.

Multi-lateration methods, which are the simplest and most popular techniques, employ the geometric methods to calculate the estimated positions. However, these algorithms have certain limitations. In practice, they provide imperfect measurements, leading to poor accuracy in the positions of mobile objects that can be estimated only using large number of fixed anchors. Therefore, it is interesting to explore the potential of artificial intelligence, specifically Deep Learning (DL), in order to estimate mobile object localization. Furthermore, meta-heuristic optimization is among the successful paradigms applied in many fields such as economics, electrical, civil and mechanical engineering.

Therefore, in the present work, a solution for indoor localization of mobile objects is proposed. It leverages three advantageous elements: Time of Flight (ToF) measurement technique based on Ultra-Wide Band (UWB) communication technology, Deep Learning and meta-heuristic optimization. Indeed, this work focuses primarily on a set of contributions related to this targeted solution. Its main objectives are presented below:

Building DL models for indoor localization using a dataset composed, essentially, of UWB ToF measurements.

Modeling the training of a DL model as an optimization problem by establishing a correspondence between the training of neural author layer parameters and an optimization problem resolution through meta-heuristic algorithm.

Developing optimizers that can be used for DL model training. Each optimizer is based on a recent and widely-used meta-heuristic optimization algorithm.

Developing and training the constructed models and testing them using the proposed optimizers.

Evaluating the performance of the developed DL models and select the most performant among them as the suggested solution.

Conducting a comparative study by:

-Comparing the results obtained by DL models trained with those given by the different developed optimizers and the Gradient Descent (GD) optimizer.

-Comparing the findings provided using ToF measurements with those obtained employing other measurements: RSSI and Range.

-Comparing the results given by the proposed solution with those obtained by applying a multi-lateration-based localization solution.

The remaining of this article is divided into the following sections: Section 2 presents a review of the recent works that suggested solutions for indoor IoT localization. Section 3 provides the background information of this study. Section 4 explains the different contributions of the introduced proposal and the choices made to implement the developed models. Section 5 presents the results of the various experimental tests conducted to evaluate the proposal and provides details about the conducted comparative study. Section 6 summarizes the study and outlines some future research directions.

Related Works

In the literature, several approaches were proposed to enhance the accuracy of indoor localization in IoT networks.

For instance, [START_REF] Perkovic | Machine learning approach towards lorawan indoor localization[END_REF] suggested a solution that uses LoRaWAN technology and neural networks to predict positions within a university building. The RSSI and signal-to-noise ratio (SNR) metrics were employed to measure the signal strength. A position was estimated based on the variation of the signal power. Real-world experiments were carried out to test and evaluate the developed system. The obtained results demonstrate that the latter achieved an accuracy of up to 98.8%. However, the precision rate and the accuracy of localization given by the proposed approach were not compared with those provided by other solutions that use the RSSI and SNR metrics.

Besides, [START_REF] Pham | Ensemble learning model for wifi indoor positioning systems[END_REF] employed Wi-Fi RSSI measurements to train an ensemble model composed of a DNN model and two other models built using machine learning techniques (K-nearest neighbors (KNN) and Random Forest (RF)). The best model, which yielded the smallest prediction error for a test data point, was selected by an intermediate classifier. The results provided by the proposed ensemble model are better than those given by each individual base model if used separately. Indeed, the average error obtained by the introduced model during testing was 1.10m. However, in [START_REF] Xu Feng | A survey of deep learning approaches for wifi-based indoor positioning[END_REF], a sub-meter level accuracy was required in indoor positioning systems. Despite the importance of the developed model, it was not sufficiently effective to overcome the challenge of indoor localization.

Another deep learning-based approach for indoor localization was suggested in [START_REF] Sun | Optimized cnns to indoor localization through ble sensors using improved pso[END_REF]. The proposed solution used RSSI signals from BLE sensors to construct a 2D image employed to develop a CNN model. Moreover, an improved version of the PSO optimization algorithm was applied to enhance the training of the neural network. Specifically, the weights of the CNN layers were determined as the best global solutions obtained by PSO. Real-world experiments were conducted in a building to evaluate the prediction results provided by the developed framework. These findings were compared with those obtained using other machine learning techniques such as ANN, DNN, KNN, CNN, etc. The comparison demonstrated that the introduced solution offered improved localization accuracy for a mobile object, compared to other techniques, with a localization accuracy reaching 97.92%. However, it is important to note that this accuracy is based on the results provided during the validation stage. Despite their importance, these results should be confirmed by a testing phase.

Poucet [START_REF] Vey | Poucet: A multi-technology indoor positioning solution for firefighters and soldiers[END_REF] is an indoor localization solution designed for firefighters and soldiers. It combines Ultra-Wideband (UWB) and Time of Flight (ToF) technologies with GPS and LoRa. Trilateration was applied in Poucet development to estimate the positions of objects. The simulation results showed an average error of 0.36m.

In [START_REF] Yang | High-precision uwb-based localisation for uav in extremely confined environments[END_REF], the authors used the measurements of Time of Flight (ToF) obtained from UWB signals to develop a localization solution. The approach utilized an algorithm based on ToF measurements and combined it with the Maximum Likelihood Estimation (MLE) method to resolve the localization problem. Moreover, real-world experiments were conducted to test the performance of the presented system. The average obtained error was 0.2m.

In [START_REF] Wang | A framework for csi-based indoor localization with 1d convolutional neural networks[END_REF], a framework was presented as an indoor localization solution. It was formed by collecting data about CSI signals using WiFi technology. These data were employed to train a CNN (Convolutional Neural Network) model. The latter achieved an accuracy of 68.5% and an average distance error of 1.75m. However, this accuracy remains unsatisfactory for indoor IoT localization requirements. Additionally, the suggested solution did not improve the training algorithm of the DL model.

In [START_REF] Liu | Indoor object localization and tracking using deep learning over received signal strength[END_REF], the authors utilized an emulator software to generate a dataset of RSSI measurements. This dataset was, then, employed to train and test a MLP (Multi-Layer Perceptron) deep learning model. The results of the conducted tests revealed that the accuracy during the training and the testing phase attained 83%. However, the results should be experimentally validated on data collected in a real-world environment.

Deep Learning-Based Cooperative Architecture (DELTA) was presented in [START_REF] Boudani | Implementing deep learning techniques in 5g iot networks for 3d indoor positioning: Delta (deep learning-based co-operative architecture)[END_REF]. The dataset used to train the deep learning model were constructed using simulation software based on the measured RSSI signals.

The development and training of the DNN (Deep Neural Network) were performed utilizing Keras. The performance of the proposed model was compared with that of two machine learning techniques: Support Vector Machine (SVM) and K-Nearest Neighbor (KNN). The obtained results showed that DELTA provided the best results with an average error of 1.6m. However, the efficiency of the introduced model should be validated in real-world environment and in an additional testing phase. Moreover, the achieved accuracy did not satisfy the requirements of IoT networks for indoor positioning as the average distance error should be inferior to 1m [START_REF] Xu Feng | A survey of deep learning approaches for wifi-based indoor positioning[END_REF].

In [START_REF] Vey | A distributed algorithm for range-based localization in sparse wireless networks[END_REF], the UWB (Ultra-Wideband) technology was used to develop and present a distributed and collaborative localization solution. Simulations were conducted to evaluate the proposed algorithm. The obtained results revealed that the localization accuracy improved as the network density increased, and the average position error was below 1m with the average connectivity. However, real-world experiments are needed to validate these results.

In [START_REF] Hsieh | Deep learning-based indoor localization using received signal strength and channel state information[END_REF], RSSI and CSI measurements were collected to construct a dataset utilized to develop and test a deep learning model. The applied approach consists in developing neural networks implemented with Multi-Layer Perceptron (MLP) and Convolutional Neural Network (CNN) architectures to predict the position of a person in an indoor environment. The prediction was formulated as a classification problem for localization. The tests were conducted to compare four localization methods: MLP-RSS, MLP-CSI, CNN-RSS and CNN-CSI. The Keras platform [13] was used to apply and test these methods. The obtained results demonstrated that the CNN-CSI method outperformed the other localization techniques by achieving a distance error of 0.92m.

Another indoor positioning solution in an IoT environment was presented in [START_REF] Njima | Deep cnn for indoor localization in iot-sensor systems[END_REF]. It involves training a CNN model using the measured RSSI signals. Specifically, the RSSI signals were employed to construct 3D radio images, and the localization problem was formulated as an image recognition problem using these constructed images as input data of the developed deep learning model. Simulations were conducted to test the proposed model and compare its performance with those of other approaches that utilize RSSI signals to predict the positions. The experimental findings showed that, with an average accuracy of up to 94.13%, the performance of the introduced model was the best, compared to those of the other approaches. However, it is important to note that this accuracy was obtained when applying the developed approach in a simulated IoT environment. In other words, it is necessary to evaluate the performance of the proposed system in a real-world environment.

In [START_REF] Tada | Implementation of trilateration based localization approach for tree monitoring in iot[END_REF], the trilateration technique was applied to propose a localization method. The proposal used RSSI measurements to estimate, first, the distances between objects and anchors and, then, predict the positions of the objects. Simulations were conducted to evaluate the performance of the introduced system that provided an average localization error of 1.37m.

The trilateration technique was also used in [START_REF] Despaux | N-twr: An accurate time-of-flight-based n-ary ranging protocol for ultra-wide band[END_REF] to develop and present a localization algorithm. The solution is based on Time of Flight (ToF) measurements captured by UWB technology. It involves two steps. In the first stage, ToF measurements were used to estimate the distances between the object to be localized and a set of anchors. However, the second step consists in employing the actual positions of the anchors and the estimated distances to calculate the position of the object applying a specific trilateration algorithm. The conducted experiments demonstrated an average error of around a dozen centimeters.

Moreover, a localization framework relying on Deep Neural Network (DNN) and CSI signals was introduced to predict the positions of IoT objects in [START_REF] Li | Channel state information-based multi-level fingerprinting for indoor localization with deep learning[END_REF]. The designed framework was tested in two different indoor environments. During the tests, the results obtained with CSI measurements were compared to those provided by RSSI measurements. It was also obvious that the accuracy achieved using CSI was higher than that given by RSSI. Additionally, it was demonstrated that the proposed framework outperformed the SVM technique in terms of localization accuracy. The average obtained error ranged from 0.6 to 0.7 meters for the two tested environments.

PSO-ANN combines the Artificial Neural Network (ANN) technique and the Particle Swarm Optimization (PSO) algorithm. This technique was applied by [START_REF] Li | A wi-fi indoor localization strategy using particle swarm optimization based artificial neural networks[END_REF] to develop an indoor positioning system. In addition to PSO and ANN, the developed approach used the "Affinity Propagation (AP)" clustering algorithm to obtain better position prediction. Initially, RSSI data was collected from known reference points. Then, the clustering algorithm was employed to divide the reference points in the studied environment into clusters. Subsequently, an ANN model was trained for each cluster. It was noticed that developing a suitable ANN model for each cluster helped to estimate more accurate positions. The proposed system was implemented on a mobile phone to test the hybrid strategy on a building floor. The position prediction results were author compared with those obtained by applying ANN and KNN with and without clustering. The comparison demonstrated that the hybrid ANN+PSO+AP strategy outperformed the other studied techniques in terms of localization accuracy. Furthermore, it showed that clustering improved the accuracy of the predicted positions.

Table 1 summaries the works described in this section based on the following criteria: [START_REF] Djosic | Fingerprinting-assisted uwb-based localization technique for complex indoor environments[END_REF] The year in which the studied method was proposed; (2) The technique applied to resolve the localization problem; (3) Whether the employed technique was optimized (4) Whether a meta-heuristic optimization algorithm was applied (5); Communication technology and (6) Measurement techniques (the communication technology and the nature of the signals employed during data collection); [START_REF] Yang | High-precision uwb-based localisation for uav in extremely confined environments[END_REF] The average distance error (a performance indicator that calculates the average distance between the actual position and the estimated one); [START_REF] Wang | A framework for csi-based indoor localization with 1d convolutional neural networks[END_REF] The accuracy of the obtained localization results; [START_REF] Liu | Indoor object localization and tracking using deep learning over received signal strength[END_REF] The time required to estimate a position, if mentioned; and ( 10) Experiments (the nature of the conducted experiments on which the collected data and the results are based (real experiments or simulations).

The survey of the above-mentioned works revealed that the existing methods cannot meet the necessary localization accuracy in IoT environments. Indeed, the average distance error they provided between the actual position and the estimated one is a few tens of centimeters. Furthermore, no technique that uses UWB ToF data was introduced to train a Deep Learning model and predict the position of objects in IoT environments, despite the fact that UWB is the most promising technology for indoor localization systems and ToF values can be accurately measured based on UWB signals.

On the other hand, it was observed that the use of the PSO optimization algorithm with Deep Learning models improved the performance and accuracy of the localization method. Taking advantage of the success of the meta-heuristic optimization algorithms and their high efficiency in solving complex problems, it is necessary to model a more efficient solution for indoor IoT localization based on other optimization algorithms that are more recent and performant than PSO. Additionally, no proposal that focuses on solving indoor localization in IoT networks by training DL models with the Grey Wolf Optimizer (GWO) and Arithmetic Optimization Algorithm (AOA) was introduced in the literature. However, GWO and AOA remain the most recent and promising meta-heuristic optimization algorithms.

In the present study, UWB Time of Flight measurements are employed to achieve better accuracy for the localization of mobile objects in an IoT environment by combining the advantages of metaheuristic optimization algorithms and those of Deep Learning methods. The following section presents the context of the current work.

Background

This section concentrates on the necessary concepts utilized to establish the context of this study. It introduces the most commonly-used techniques for indoor IoT localization. It, then, lists the most popular Deep Learning algorithms and the meta-heuristic optimization algorithms.

Indoor IoT localization

Generally, the localization techniques can be classified into two types: range-based and range-free methods.

Range-free localization techniques

Range-free localization techniques often provide an estimated position without using anchors. They typically involve collaborative methods as the estimation of positions relies on the neighborhood information and the geometric methods. Their main advantage is that they do not require special hardware support, making them less costly. The DV-Hop [START_REF] Bulusu | Gps-less low-cost outdoor localization for very small devices[END_REF] and Centroid [START_REF] Niculescu | Dv based positioning in ad hoc networks[END_REF] algorithms are among the most popular range-free techniques.

Range-based localization techniques

Range-based localization techniques are based on measurements taken in the studied IoT environment to obtain distance or angle estimations. These techniques offer high precision, but they require expensive hardware and more challenging deployment. The most commonlyapplied measurement techniques in the literature [START_REF] Savic | Constrained localization: A survey[END_REF] are:

Received Signal Strength Indicator (RSSI) [START_REF] Jondhale | Survey of Existing RSSI-Based LT Systems[END_REF]: measures the power of the signal received during a transmission and, therefore, estimates the distance between the transmitter and the receiver. RSSI measurements do not require additional hardware other than a device using wireless communication technology. Therefore, RSSI is the most widelyused measurement technique applied to implement the localization strategies in indoor environments. However, it is less precise in terms of localization accuracy because the received power is often disturbed by obstacles and non-omnidirectional antennas.

Channel State Information (CSI): combines information about the communication channel's state and describes how a message propagates from the transmitter to the receiver. In [START_REF] Mohammed | Channel state information from pure communication to sense and track human motion: A survey[END_REF], the authors proved that CSI provides better detection accuracy than RSSI. However, this technique is less performant than the Time of Flight (ToF) radio technique.

Angle of arrival (AoA): measures the angle at which the signal arrives at the receiver. For example, in [START_REF] Zeaiter | Measuring accurate angle of arrival of weak lora signals for indoor positionning[END_REF], a method was applied to take AoA measurements using LoRa (Long-Range Radio) signals.

Time of flight (ToF): calculates the time required for a signal to travel from the source to the destination and vice versa. Based on this measurement and the speed of the radio signal propagation, the distance between two objects can be estimated. This solution is less affected by signal attenuation through obstacles or antenna directivity, compared to other measurement techniques. Ultra-Wide Band (UWB) signals allow precise Time of Flight (ToF) measurements, making UWB ToF measurements suitable for highprecision localization solutions. For example, in [START_REF] Vey | Indoor uwb localisation: Locura4iot testbed and dataset presentation[END_REF], a platform that can be used to develop and evaluate new methods proposed for indoor localization was presented. This platform was built based on UWB technology and the ToF measurement technique.

Time Difference of Arrival (TDoA): is a variant of Time of Flight (ToF). It measures the difference between the arrival times of a signal from a single transmitter to two different receivers considered as two reference points. By combining this measurement with the signal transmission speed, the difference in the distances between the transmitter and the two receivers can be calculated [START_REF] Garg | Indoor tracking using ble -brief survey of techniques[END_REF].

Range: it is an estimation of the distance between two objects obtained through a telemetry protocol. This measurement can be estimated from the received signal strength or by using Time of Flight (ToF).

Signal-to-Noise Ratio (SNR) (expressed in decibels) [START_REF] Dinh | Physical sensors: Thermal sensors[END_REF]: is a measure that evaluates the quality of a transmission by comparing the level of a desired signal to that of noise. It is defined as the ratio between the power of the desired output signal and the power of the noise.

Once these measurements are performed, localization algorithms will be implemented. Generally, they are based on triangulation techniques where the geometry of triangles is applied to calculate the locations of IoT objects. Triangulation can be classified into two techniques: (i) lateration, which is based on the distances to points with known locations, and (ii) angulation relying on the angles with respect to points having known locations in addition to distances. The taken measurements are used to estimate the positions of the objects. Trilateration is a technique that calculates the distances to at least three fixed references in a 2D environment. In a 3D environment, a minimum of four references is required. In fact, trilateration algorithms are the simplest and most common applied to localize the objects. Other possible localization solutions are those based on artificial intelligence techniques, specifically Machine Learning and Deep Learning techniques. These methods involve using data collected from the environment and utilizing appropriate algorithms to predict IoT object positions. The next section introduces the field of Deep Learning and presents its most common techniques.

Deep Learning

Machine Learning (ML) techniques are algorithms that enable computers to learn from data without being explicitly programmed. Deep Learning (DL), on the other hand, is a sub-field of Machine Learning that focuses on algorithms utilizing artificial neural networks inspired from the human brain to solve problems in several domains. In recent years, DL techniques have been successfully applied by many researches and in various use cases.

Some Recurrent Neural Network (RNN) [START_REF] Du | Recurrent Neural Networks[END_REF]: In RNNs, connections between nodes can be recursive. In other words, the input of certain nodes depends on the previous output of the same nodes. This enables RNNs to process sequential data.

Long Short-Term Memory (LSTM): LSTM, proposed in [START_REF] Hochreiter | Long short-term memory[END_REF], is a type of recurrent neural network. It was designed to address the vanishing gradient problem and is largely utilized in tasks involving sequential data.

Gated Recurrent Unit network (GRU): GRU is a variant of RNN characterized by a reduced number of parameters and fast convergence. It was introduced in [START_REF] Cho | On the properties of neural machine translation: Encoder-decoder approaches[END_REF].

Multilayer Perceptron (MLP): MLP is a widely used type of neural network where connections are only made from input to output in the network (single direction). It includes no recurrence, meaning that the output of a neuron cannot affect its next input.

Ensemble learning: This technique involves combining the outputs of two or more models to improve the obtained results. For example, some methods calculate the average of the outputs provided by multiple DL and ML models.

Indeed, any model among the previously-mentioned ones is trained by a learning process. In other words, a DL model has a set of trainable parameters whose values are modified and optimized during several iterations in order to minimize a loss function (error) at the training step. An iteration is called an epoch and the number of epochs must be predetermined together with the batch size. Indeed, the training data is divided into batches (the batch size refers to the number of samples in each batch). The weights of the model to be trained are typically initialized with random values. Then, at the end of processing each batch, these weights are updated to minimize a loss function generally used to evaluate the difference between the actual values and the predicted values of the target outputs. In this process, an optimization algorithm is utilized. Among the most commonly-applied optimizers, we can mention: Gradient descent (GD): is a technique in ML and DL used to iteratively minimize a cost/loss function.

Stochastic Gradient Descent (SGD): is a variant of GD. It is an optimization method that accelerates convergence and steers the weights in the right directions towards their optimal values [START_REF] Sutskever | On the importance of initialization and momentum in deep learning[END_REF]. SGD is one of the most popular optimization algorithms employed to train DL models due to its easy implementation and high efficiency [START_REF] Anila Sharon | Chapter 11 -application of intelligent edge computing and machine learning algorithms in mbti personality prediction[END_REF].

Batch Gradient Descent (BGD): is a variant of GD. It is an optimization algorithm where each iteration considers all training data [START_REF] Tiwari | 5preliminaries: A primer[END_REF].

Follow the Regularized Leader (FTR): This algorithm was described in [START_REF] Mcmahan | Ad click prediction: a view from the trenches[END_REF]. It is recommended to use it particularly with shallow models with a large number of features.

Another type of optimization algorithms is described in the following section. The latter introduces the metaheuristic optimization and its most recent and widely used algorithms.

Meta-heuristic Optimization

The objective of an optimization problem is to find an optimal solution in a search space containing domains of possible values of decision variables. Such problem is resolved by reducing the search space to only the values that optimize a number of objective functions and satisfy a set of constraints. The problem including only one objective function is known as a single-objective optimization problem. Optimization problems are solved by meta-heuristic algorithms. The most recent and widely employed algorithms are depicted below: Grey Wolf Optimizer (GWO) [START_REF] Mirjalili | Grey wolf optimizer[END_REF]: is a metaheuristic algorithm inspired from the social hierarchy and hunting behavior of grey wolves. [START_REF] Sharma | A comprehensive survey on grey wolf optimization[END_REF] recommended using this algorithm to solve real-world optimization problems and proved its efficiency by describing various research works that use GWO and its different variants. For example, in [START_REF] Tlili | A multiobjective gray wolf algorithm for routing in iot collection networks with real experiments[END_REF], the multi-objective version of GWO (MOGWO) was utilized to model routing in an IoT environment. Experimental results showed the ability of MOGWO to overcome routing challenges in IoT data collection networks. Arithmetic Optimization Algorithm (AOA): is a meta-heuristic method based on the distribution behavior of the fundamental arithmetic operators in mathematics. Its performance was validated and justified by the experimental results presented in [START_REF] Abualigah | The arithmetic optimization algorithm[END_REF].

Whale Optimization Algorithm (WOA) [START_REF] Mirjalili | The whale optimization algorithm[END_REF]: is a nature-inspired meta-heuristic optimization algorithm relying on the behavior of whales and their hunting strategy. It is characterized by a reduced number of parameters, easy implementation and fast convergence. Therefore, it has gained wide popularity and has been applied to various problem domains. Authors, in [START_REF] Soleimanian | A comprehensive survey: Whale optimization algorithm and its applications[END_REF], summarized the research works in which solutions based on WOA were proposed.

Particle Swarm Optimization (PSO) [START_REF] Cagnina | A particle swarm optimizer for multi-objective optimization[END_REF]: is an optimization algorithm that emulates the social behavior of birds through algorithmic operations. It has been largely employed in several studies due to its simplicity and the reduced number of parameters it utilizes. Different variants of PSO and recent works that have used this algorithm were described in [START_REF] Tareq | Particle swarm optimization: A comprehensive survey[END_REF].

Methodology

The approach introduced in the current work consists in developing a Deep Learning model optimized by a meta-heuristic algorithm and trained on UWB Time of Flight data in order to estimate the positions of mobile objects in an IoT environment. This section presents first the developed approach. It introduces the proposed algorithms applied to model the training of the DL model using meta-heuristic optimization. Then, it describes the localization process based on the trained and optimized model.

Parameter Optimization with Meta-heuristic Algorithms

A Deep Learning model composed of dense layers is suggested and the learning of their weights is optimized using meta-heuristic algorithms. As mentioned in Subsection 3.2, during the training phase, multiple learning iterations are performed to find the values of the weights that help formulate the correct function mapping inputs x to outputs y. In other words, these weights are learned and improved during training. Equation 1 simplifies and generalizes the output equation of a layer utilized in a neural network.

y t j = f ( i W ji x t i ) (1)
where x t i and y t j represent, respectively, the input and output of a layer at time t. W ji are the trainable weights during the learning phase. The values of the weights are updated by applying an optimizer algorithm in order to reduce the prediction error over the epochs.

The performance of the optimizer used during the training phase greatly affects the convergence speed of the DL model and the quality of the predicted results it provides. Moreover, IoT applications require accuratelycalculated positions and fast processing to leverage their full potential. To overcome these challenges, it is important to improve the performance of the DL model utilized for localization by applying an enhanced optimizer. Therefore and to deal with these issues, optimization meta-heuristics are used in the present work. By treating the DL model weight training as a meta-heuristic optimization problem and solving it with an algorithm among those dedicated in this paradigm, solutions can be generated and employed to update the weight values during different iterations.

Relying on this definition of the learning step, the weights of a DL model are optimized, in this study, by applying meta-heuristic algorithms. The weight learning is translated into a single-objective optimization problem. The objective function to minimize is the prediction error function and the variables, which constitute the population of the optimization problem, are the weights of the DL model. In other words, during the different iterations, the new values of the weights are the best values selected by the meta-heuristic algorithms (Figure 1).

The modeling of weight optimization with a metaheuristic method is summarized in Figure 2 and detailed in Algorithms 1, 2 and 3.

At the beginning of the training phase, the weights were initialized, and a maximum number of epochs was defined. The initial weights were used to initialize the population of the meta-heuristic optimization algorithm. Then, a new iteration was executed until reaching the maximum number of epochs. During each epoch, all the training data were employed. These data were divided into batches. At the end of processing each batch, the loss function value was calculated. Then, it was proven whether the meta-heuristic optimization population had been fully constructed or updated with the latest iterations. In this case, the mathematical equations of the meta-heuristic optimization would be applied to calculate new weight values, and the algorithm's population was updated with the newlycomputed weights. If it was not the case, the weights would be updated using the equations of Gradient Descent (GD). Once the maximum number of epochs was exceeded, the weights obtained at the last iteration would be considered optimal, showing the end of the training phase.

Algorithm 1 details the training phase of the developed model. First, the DL weights were initialized with random values together with the parameters of the meta-heuristic algorithm to be applied. Among these parameters, pop size, the size of the population to be considered, was specified. Afterwards, a number of epochs were executed to train the model. Each epoch consists of a number of iterations equal to the size of the input data divided by the batch size to be taken into account. Algorithms 2 and 3 describe the processing performed during each iteration. During the various epochs, the meta-heuristic was applied after every pop size iterations. In other words, the training phase was divided into stages of length pop size iterations. During the first stage, the weight values were used to initialize the population. At the end of this stage, the meta-heuristic algorithm was applied to update the weights. During the subsequent stages, the weights generated by the previous stage were first used. At the end of each stage, the weights were optimized using the equations of the employed metaheuristic algorithm.

Localization with the optimized DL model and Time of Flight (ToF)

The localization approach involves two phases (Figure 3): Offline phase: The first step is to use the dataset presented in sub-section 5.1.1 to train the model. The data, primarily including UWB Time of Flight (ToF) measurements, serves as the input for the DL model to be trained. The coordinates of the mobile object represent the target results to be predicted, and the other columns are the data based on which the prediction will be made. The weights of the model were learned and optimized by applying meta-heuristic optimization algorithms. The objective function to minimize is the average distance between the actual positions and the estimated ones. During training, a portion of the data was utilized for validation. The result provided in this phase is a trained and optimized model.

Online phase: The second phase represents a scenario where a mobile object needs to localize itself. In this case, after exchanging messages with the anchors, this object selects the necessary data (primarily including UWB ToF measurements) to be used for prediction. By giving these data to the model, resulting from the first phase, as input, the object can estimate its own position (coordinates).

Experimental Results and Discussion

In order to evaluate the performance of the developed and proposed model, a set of tests were conducted. This section presents first the experimental setup. Then, the obtained results are analyzed.

Experimental Setup

This section describes the experimental details including the used dataset, the employed evaluation criteria, the experimental parameters and the implementation choices.

The Dataset

The dataset utilized to apply the proposed solution was generated by the real-world platform LocURa4IoT [START_REF] Van Den Bossche | locura4iot-a testbed dedicated to accurate localization of wireless nodes in the iot[END_REF] designed to develop and test approaches applied to solve the problem of indoor localization in IoT environments based on Time of Flight (ToF). Additionally, it can be employed to test and evaluate the ranging protocols in IoT networks.

The platform primarily uses Ultra-Wide Band (UWB) technology, which offers the highest precision and incorporates Bluetooth Low Energy (BLE) as well as LoRa transceivers. To construct the dataset, a set of anchors were installed in the considered IoT environment. A mobile object was moved within the environment to collect measurements regarding the signals exchanged between the object and the anchors. The Two-Way Ranging (TWR) protocol [START_REF] Despaux | N-twr: An accurate time-of-flight-based n-ary ranging protocol for ultra-wide band[END_REF] was applied during the message exchange between the mobile object (TWR client) and the anchors (TWR servers). The dataset includes several measurements such as ToF, anchor positions, real positions of the mobile object, RSSI measurements, range measurements, etc. The ToF and range measurements were calculated using the TWR protocol [START_REF] Despaux | N-twr: An accurate time-of-flight-based n-ary ranging protocol for ultra-wide band[END_REF]. The time difference between the TWR server clock and the TWR client clock can lead to inaccurate Range measurements. To deal with this problem, the Range measurements in the LocURa4IoT dataset were optimized. The dataset is available online at [START_REF]Locura4iot dataset[END_REF].

To apply the localization solution, the lines of the dataset were rearranged to separate the pairs of lines and avoid overfitting. Some data from the LocURa4IoT dataset were eliminated. They include:

Columns with unique values such as object identifiers and the used protocol.

Columns that do not provide useful information that can be relied on to localize objects. Moreover, the dataset was reorganized such that each row would contain information about the interactions with the same 5 anchors. Then, the dataset was divided into three parts: for training, test and validation. Authors, in [START_REF] Joseph | Optimal ratio for data splitting[END_REF], considered the ratio (p :

√ p : ( √ p + 1))
as the optimal distribution ratio for the mentioned data parts. p = √ N , N is the number of unique rows in the used data.

The new content of the dataset consists of 3947 unique rows. Therefore, p = 63 and the distribution ratio became 63 : 8 : 9. In other words, the dataset was divided as follows:

78.75% of the data were used as training data 10% of the data were utilized as validation data 11.25% of the data were employed as test data

The evaluation criteria

Among the existing evaluation criteria, the two following criteria were used in the performed experiments: Mean Absolute Error (MAE): It calculates the average of the absolute differences between the actual target output values (the test data) and the predictions obtained by applying the model (Equation 2). A smaller MAE value indicates better prediction accuracy. As MAE is the most commonly-used criterion applied to evaluate the localization solutions, it is utilized in the present study.

M AE = 1/n n i |y i -ŷi | (2)
y i : actual value, ŷi : predicted value Mean Squared Error (MSE): It computes the squared difference between the actual target output values (the test data) and the predictions obtained by applying the model. Then, it takes the average of these errors (Equation 3). A smaller MSE value indicates better prediction accuracy. It represents the simplest and most common loss function. [START_REF] Wang | A comprehensive survey of loss functions in machine learning[END_REF] recommended using this type of loss to solve prediction problems.

M SE = 1/n n i (y i -ŷi ) 2 (3)
y i : actual value, ŷi : predicted value

Experimental Parameters

Implementation

The experiments were conducted on a computer equipped with an Intel i7 processor and 16 gigabytes of RAM. To implement the approach, the following choices were made:

Python as the programming language Jupyterlab 3.5. -GWO [START_REF] Mirjalili | Grey wolf optimizer[END_REF] : is utilized in the current work as it is one of the recent meta-heuristics that has recently gained significant interest in various domains [START_REF] Dada | Application of grey wolf optimization algorithm: Recent trends, issues, and possible horizons[END_REF].

-AOA [START_REF] Abualigah | The arithmetic optimization algorithm[END_REF] : It was chosen because it is one of the most recent meta-heuristic methods proposed in 2021. Additionally, according to [START_REF] Abualigah | The arithmetic optimization algorithm[END_REF], it outperforms other wellknown optimization algorithms and provides promising results in solving challenging optimization problems.

Results

In order to prove the good performance of the introduced approach, experiments were conducted, and the developed models were compared in terms of MAE and MSE. In addition, the best developed model was compared to a multi-lateration solution. This section presents the obtained results.

Comparison of the Optimization Methods

The training of 3 DL developed models was tested with the same hyper-parameters (Table 2), primarily based on the ToF measurements from the LocURa4IoT dataset. The used data includes anchor coordinates and ToF measurements. Two models were trained by applying the proposed method (Sub-section 4.1) and employing GWO and AOA, respectively. The third model was trained using the existing optimization method (GD).

Figure 4 shows the learning curves of the 3 employed models. It presents the variation of MAE values during the training epochs using the three optimizers. Each two curves were plotted together to facilitate the comparison of the models. It should be noted that, during the training epochs, the parameters were updated to find the optimal values that yielded lower MAEs. This figure demonstrates that optimizing the trainable parameters with the developed approach using GWO resulted in the fastest convergence, compared to the other two optimizers. In other words, the MAE errors were smaller and decreased more rapidly when the GWO equations were used to update the weight values during the learning iterations. This figure also reveals that optimization employing the AOA meta-heuristic is less effective than that made utilizing GD. Thus, we can conclude, from these curves, that our proposal applied with GWO outperforms the other two methods, followed by the GD optimizer. However, optimization with AOA is the least effective. This conclusion is further supported by Figure 5. In the latter, the average values of MAE and MSE errors across all epochs during the training phase are compared. This comparison confirms that training with GWO surpasses that done by the other two optimizers (GD and AOA) and that our proposal relying on GWO accelerates convergence towards the optimal parameters and enhances the learning of the DL model.

The learning times of the three models are compared. The provided results are presented in Table 3. In terms of the training time, optimization with GD is the fastest, followed by that with GWO and that with AOA. However, the learning time criterion is not crucial as it pertained to the off-line phase (training). Indeed, during the execution of IoT applications (on-line phase), objects used a pre-trained model to self-locate themselves.

Based on these results, the optimization relying on the proposed approach applied with GWO was considered to conduct the remaining tests presented in the rest of this section.

Comparison of the Measurement Techniques

The initial performed tests showed that training based on the developed approach applied with GWO yielded the best results. For this reason, this optimization method was utilized to train three DL models with the same hyper-parameters, but with different data selected from the measurements taken in LocURa4IoT. In addition to the anchor positions, the used data included, respectively, UWB Time of Flight (ToF), Received Signal Strength Indicator (RSSI) and Range measurements. The tests were performed during this stage to compare the performance of the three developed models in terms of the accuracy of position prediction. This comparison was made by training the three developed models and using them to predict positions employing the test data (11.25% of the dataset). The predicted x and y coordinates obtained from the tested models were also compared. Additionally, these predictions were considered to calculate the distance between the actual position of the mobile object and its estimated position. Figure 6 represents a comparative study of the results obtained by the three models. It displays the error variation curves obtained from the predictions given by applying the three tested models for x and y coordinates as well as the distance between the actual and the predicted positions. The curves indicate that the errors obtained from UWB ToF are generally lower than those provided by RSSI and Range measurements.

Figure 7 confirms that the positions predicted using UWB ToF measurements are more accurate than those obtained with RSSI and Range. Indeed, Figure 7 compares the Mean Absolute Error (MAE) and Mean Squared Error (MSE) values of the predictions obtained in terms of x and y coordinates and positions (distances between the actual and estimated positions). It is noticed that ToF measurements yielded the lowest values of both MAE and MSE criteria. It is also obvious that predictions with Range resulted in smaller MAE and MSE values than RSSI for the x coordinate and position. However, for the y coordinate, Range provided a lower MAE value and a slightly higher MSE value, compared to RSSI.

Based on these results, the optimization relying on the proposed approach applied with GWO and employing UWB ToF measurements was considered as the proposed indoor localization solution by this study, since it gives the best results compared to other developed and tested solutions. Therefore, it is compared to a conventional multi-lateration solution in the next sub-sub-section.

Comparison of the performance of the proposed solution with Multi-lateration Method

The objective of this test section is to demonstrate that the performance of the proposed approach is better than that of a conventional multi-lateration method presented in [START_REF] Norrdine | An algebraic solution to the multilateration problem[END_REF]. The position estimation capabilities of both solutions were tested using the same test data. The estimation results are compared in Figures 8 and9. Figure 8 compares the predicted values and the actual values of the x and y coordinates for the two compared solutions. The curves demonstrate that the predictions made by the introduced approach are more accurate than those of the multi-lateration method. Figure 9 presents These results reveal that the developed approach provided lower MAE and MSE errors for all three considered measurements (x, y and position). In fact, the rate at which the suggested localization solution achieved the minimum distance between the actual and the estimated positions is 93.54%.

Furthermore, the presented approach provided an average error between the actual and the predicted positions of 0.057m. These results also demonstrate that the proposed method surpassed the state-of-the-art solutions, described in Section 2, where the best average distance error was a few tens of centimeters.

The localization accuracy and the times required to apply the two localization solutions were also compared in table 4. The suggested solution outperformed the multi-lateration one by achieving a localization accuracy of 98.92%, whereas the latter achieved an accuracy of 89.24% (threshold = 0.5m). The time taken for position prediction using the introduced approach was 0.155 seconds, whereas the multi-lateration solution required 1.982 seconds to estimate the coordinates. Although the developed solution outperformed the other method in terms of the estimation time, it is worth to note that the proposed DL solution required an off-line step that necessitated 328.302 seconds to train the prediction model. On the other hand, the multi-lateration method did not require any additional phase apart from real-time position estimation.

Summary

Several conclusions can be drawn from the conducted tests and obtained results:

Training the DL model parameters based on GWO, as proposed in this work, outperforms the other two optimization methods (AOA and GD). The approach applied with GWO accelerates convergence towards the optimal parameters and enhances the learning process for the DL model.

Positions predicted using UWB ToF measurements are more accurate than those obtained with RSSI and Range.

The introduced localization solution provided an average error between the actual and the predicted positions of 0.057m. It outperformed the tested multi-lateration solution that yielded an average distance error of 0.167m.

The proposed technique surpassed the state-of-theart solutions where the best results achieved an average distance error of few tens of centimeters.

Conclusion and Future Perspectives

In this article, a Deep Learning (DL) model trained with the Grey Wolf Optimization (GWO) algorithm and based on UWB Time of Flight (ToF) measurements was designed as a solution for indoor localization in an IoT environment. We conducted experiments using the LocURa4IoT dataset, constructed with measurements author Thirdly, the predicted positions obtained using the suggested solution was compared with those calculated using an existing conventional multi-lateration solution.

The experimental findings proved that the introduced model outperforms the existing one in terms of MAE and MSE. It was also proven that the suggested solution achieved an average distance error of 0.057m, demonstrating its best superior performance, compared to state-of-the-art solutions where the best average distance error was a few tens of centimeters. Indeed, the proposed solution achieved a localization accuracy of 98.92%. However, the proposed solution took 328.302 seconds to train the DL model. As a continuation of this work, we plan to improve the training process to reduce the required learning time. Additionally, it will be beneficial to explore a dynamic solution that enables regular learning when the environment evolves over time. Furthermore, we will use the proposed localization solution to develop a geographic routing approach in a large-scale mesh IoT network.

UWB

  Time of Flight based indoor IoT localization solution with Deep Learning optimized by meta-heuristics Sihem Tlili, Sami Mnasri, Thierry Val To cite this version: Sihem Tlili, Sami Mnasri, Thierry Val. UWB Time of Flight based indoor IoT localization solution with Deep Learning optimized by meta-heuristics. International Journal of Sensor Networks, inPress. hal-04265613 1 Introduction An indoor IoT localization solution involves applying an indoor positioning technique capable of providing precise object positioning in an IoT environment.

  author

Figure 1 Figure 2

 12 Figure 1 Weight optimization by applying a meta-heuristic algorithm

  author

Figure 3

 3 Figure 3 Localization by applying an optimized DL model and using UWB Time of Flight

Figure 4 Figure 5

 45 Figure 4 Learning curves for the DL models optimized with the proposed solution (employing GWO and AOA) and GD

Figure 6 Figure 7 Figure 8 Figure 9

 6789 Figure6The error variation curves obtained from the predictions given by using ToF, RSSI and Range measurements in terms of x and y coordinates as well as the distance between real and predicted positions

Table 1

 1 Summary of the studied works

	Paper	Year	Technique	Optimization	Meta-	Technology	Measurement	Results		Estimation	Experiments
					of	the	heuristic		techniques			time
					employed		optimization				(milliseconds)
					technique
										Average distance	Accuracy
										error (meter)
	[2]	2023	DNN		✗		✗	LoRaWAN	RSSI et SNR	-	98.8%	-	real
	[3]	2021	Ensemble	✗		✗	WiFi	RSSI	1.10	-	-	real
			learning
			(KNN, DNN
			and RF)
	[5]	2021	CNN		✓		enhanced	BLE	RSSI	-	97.92%	-	real
							version	of
							PSO
	[6]	2021	Trilateration	✗		✗	UWB,	ToF	0.36	-	-	simulations
								LoRa
	[7]	2021	MLE		✗		✗	UWB	Tof	0.2	-	-	real
			method
	[8]	2020	CNN		✗		✗	WiFi	CSI	1.75	68.5%	1.89	real
	[9]	2020	MLP		✗		✗	WiFi	RSSI	-	83%	-	simulations
	[10]	2020	DNN		✗		✗	5G	RSSI	1.6	89%	160	simulations
	[11]	2020	Collaborative	✗		✗	UWB	Range-free	<1m	-	-	simulations
			localization
	[12]	2019	MLP	and	✗		✗	WiFi	RSSI et CSI	CNN-CSI : 0.92,		-	real
			CNN							MLP-RSS : 1.92
	[14]	2019	CNN		✗		✗	WiFi	RSSI	-	94.13 %	0.68*10 -3	simulations
	[15]	2019	Trilateration	✗		✗	WiFi	RSSI	1.37	-	-	simulations
	[16]	2018	Trilateration	✗		✗	UWB	ToF	0.12	-	-	real
	[17]	2018	DNN		✗		✗	WiFi	CSI	0.6-0.7	93.9%-	-	real
											96%
	[18]	2016	ANN		✓		PSO	WiFi	RSSI	1.893	-	43	real

  They are the most commonly used layers in neural networks. As a dense layer is deeply connected to its previous layer, it receives inputs from all neurons in the previous layer.

	Deep Neural Network (DNN): It is composed of
	multiple layers including an input layer, an output
	layer and intermediate (hidden) layers. DNNs can
	handle nonlinear relationships and are capable of
	processing large amounts of data.
	Artificial Neural Network (ANN): They mimic the
	functioning of the human brain.
	Convolutional Neural Network (CNN): According
	to [28], CNN is the most popular DL architecture
	due to its efficiency in processing voluminous data.
	CNNs are widely employed in computer vision
	tasks.

of the DL techniques are presented below: author Dense layers:

Table 2

 2

	contains the hyper-parameters utilized to
	configure the trained and tested DL models during the
	conducted experiments.

Table 2

 2 Hyper-parameters of DL models

	Hyper-parameter	Value
	Number of epochs	2000
	Batch size	32
	Learning rate	0.001
	Number of hidden layers	2

Table 3

 3 Comparison of training times

	Optimization algorithm	Training time (seconds)
	GWO	328.302
	AOA	393.162
	GD	281.52

Table 4

 4 Comparison of the time required for applying the suggested solution and the multi-lateration method taken in a real IoT environment. A comparative study was conducted and presented to prove the high performance of the proposed model. Firstly, three Deep Learning models were trained employing the metaheuristic optimization algorithms GWO and AOA as well as the GD algorithm, respectively. Their performance was evaluated during the training phase. It was demonstrated that GWO allowed better optimization of the trainable parameters by accelerating convergence towards the optimal values and outperformed the two other algorithms. Secondly, the capabilities of three DL models trained with GWO to predict positions based, respectively, on UWB ToF, RSSI and Range measurements were assessed. The obtained results showed that, compared to RSSI and Range, UWB ToF measurements provided more accurate positions, as indicated by lower Mean Absolute Error (MAE) and Mean Squared Error (MSE) values in the predictions.

	Solution	Localization accuracy	Training time (seconds)	Estimation time (seconds)
	GWO-ToF (proposed)	98.92%	328.302	0.155
	Multi-lateration	89.24%	-	1.982


Kevin Riou

Kaiwen Dong

Kevin Subrin

Yanjing Sun

Patrick Le Callet

From Temporal-evolving to Spatial-fixing: A Keypoints-based Learning Paradigm for Visual Robotic Manipulation

The current learning pipelines for robotics manipulation infer movement primitives sequentially along the temporal-evolving axis, which can result in an accumulation of prediction errors and subsequently cause the visual observations to fall out of the training distribution. This paper proposes a novel hierarchical behavior cloning approach which tries to dissociate standard behaviour cloning (BC) pipeline to two stages. The intuition of this approach is to eliminate accumulation errors using a fixed spatial representation. At first stage, a high-level planner will be employed to translate the initial observation of the scene into task-specific spatial waypoints. Then, a low-level robotic path planner takes over the task of guiding the robot by executing a set of pre-defined elementary movements or actions known as primitives, with the goal of reaching the previously predicted waypoints. Our hierarchical keypoints-based paradigm aims to simplify existing temporalevolving approach to a more simple way: directly spatialize the whole sequential primitives as a set of 8D waypoints only from the very first observation. Plentiful experiments demonstrate that our paradigm can achieve comparable results with Reinforcement Learning (RL) and outperforms existing offline BC approaches, with only a single-shot inference from the initial observation. Code and models are available at :

I. INTRODUCTION

Robotic Manipulation Learning, i.e., the ability for a robot to learn various manipulation skills from large-scale humanlabeled datasets or interaction experiences, is a crucial approach for an autonomous system. Two paradigms are wellstudied for developing this approach: reinforcement learning (RL) and behavior cloning (BC). Although impressive progress has been achieved in endowing robot to learn skills in visual rich scenarios, reinforcement learning is notorious for difficult training and time-consuming. This is because the requirement of huge amount of environment interactions and the challenges of reward function definition, which is impractical and even infeasible in some cases. Behavior cloning can effectively alleviate this issue in an offline training manner, but it still encounters the problem of accumulated error arising from sequential prediction. With this temporal-evolving setting, actions need to be inferred after each observation (up to 376 actions taken for a simple cube lifting action in RLBench [START_REF] James | Rlbench: The robot learning benchmark & learning environment[END_REF]), which is inefficient, especially for edgedevices. Additionally, the worsening accumulated error can 1 Nantes Université, Ecole Centrale Nantes, CNRS, LS2N, UMR 6004, Nantes, France. {kevin.riou, kevin.subrin, patrick.lecallet}@univ-nantes.fr 2 School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, 221116, China. {dongkaiwen, yj-sun}@cumt.edu.cn § Equal contribution Fig. 1: From (a) to (d): crucial waypoints for task of "put rubbish in bin".

lead to out-of-distribution observations, which exacerbates the accumulated error in a vicious cycle.

To alleviate aforementioned issues, we propose a hierarchical keypoints-based learning paradigm which simplifies the movement primitives as a set of waypoints by encoding the fixed spatial configuration of the entire scene. Our paradigm is made of two stages: (1) A high-level trajectory planner that predicts the minimal sequence of key waypoints for executing the task from multi-view RGB images, given only the first observation of the scene. (2) An off-theshelf robotic path planner, that can reach the waypoints proposed by the trajectory planner. Our paradigm benefits from a fixed spatial encoding from the initial observation, which eliminates the possibility of error accumulation and enables single-shot inference. Last but not least, it should be noted that conventional BC paradigms used for vision-based robotic manipulation learning demand that the demonstrations be gathered from the robot's viewpoint in order to avoid observation distributional shifts between training and inference. To obtain such demonstrations, typical methods involve teleoperating the robot [START_REF] Mandlekar | What matters in learning from offline human demonstrations for robot manipulation[END_REF] or manually moving the robot [START_REF] Johns | Coarse-to-fine imitation learning: Robot manipulation from a single demonstration[END_REF], which can be both impractical and time-consuming. In contrast, our approach allows for the manipulating agent to shift between the collection of demonstrations and the actual inference time, as long as the agent is not visible in the initial scene observation. Our contributions can be summarized as following the points:

• A high-level trajectory planner is proposed that encodes An image embedding module is built to extract 2d waypoints representation in a latent space. Consequently, the latent representation will be sent into our proposed Multi-view Fusion Transformer (MFT) module to obtain the 8D waypoints prediction in 3D space, which can be used as the reference for following path planner. The whole framework will be trained in an end-to-end way. And the the success rate will be evacuated under RLbench simulation.

spatial configuration and high-level plan strategy. Consequently, low-level planning is delegated to established reliable robotics solutions. • Under the hypothesis of fixed spatial configuration, we predict the motion primitives for the entire execution sequence directly based on the initial observation of the scene. This eliminates the accumulating prediction errors along the task execution, since all actions are predicted at once rather than in a temporal-evolving way.

• The research includes an adapted manipulation dataset that is specifically designed for predicting waypoint sequences, along with tools for assessing the trained model's efficacy. These resources are built upon the RLBench simulation.

II. RELATED WORKS In this section, we will revise primary knowledge from three aspects of Visual Robotic Manipulation: Reinforcement Learning (RL), behaviour cloning (BC) and keypoint-based manipulation learning.

A. Reinforcement Learning (RL) for manipulation tasks.

Online Reinforcement Learning highly relies on iterative interactions between agent and environment, which is impractical in real-life scenarios due to expensive or dangerous experience collection. The success of Deep Neural Network (DNNs) has promoted the advent of offline Reinforcement Learning for large-scale data representation. Kumar et al. [START_REF] Kumar | Stabilizing off-policy q-learning via bootstrapping error reduction[END_REF] proposed a practical algorithm to reduce error accumulation when training from offline data. Ebert et al.

[3] presented a model-based reinforcement learning method centered around prediction of raw sensory observation by taking use of prediction in the context of robotic manipulation. However, both online and offline approaches focus on learning a direct mapping from environment states to robot actions, which means (1) they are not purpose-aware and (2) they have to learn primitive skills that roboticists can easily solve using off-the-shelf solutions. Primitive based Reinforcement learning is an augmentation for standard Reinforcement Learning with a predefined library of behavior primitives learnt from data, which is more robust and reusable for alleviating the generalization challenge. Strudel et al. [START_REF] Strudel | Learning to combine primitive skills: A step towards versatile robotic manipulation §[END_REF] designed a sample efficient pipeline to learn robust RL policies confined with primitive skills. Nasiriany et al. [START_REF] Nasiriany | Augmenting reinforcement learning with behavior primitives for diverse manipulation tasks[END_REF] introduced a Deep Reinforcement Learning (DRL) framework which utilized predefined hierarchical primitive skills, to narrow the exploration space of DRL. Dalal et al. [START_REF] Dalal | Accelerating robotic reinforcement learning via parameterized action primitives[END_REF] manually coded primitives with arguments that are learned by RL policy, leading to a better adaptability for the considered tasks.

B. Behavior cloning for manipulation tasks learning.

Benefit from the promising development of Deep Neural Network (DNN) over the past decade, the effectiveness of data-driven learning methods have been proved over many visual sensory applications [START_REF] Sun | Deep highresolution representation learning for human pose estimation[END_REF]. Behavior cloning is a straightforward imitation learning method that utilizes the representation capability of DNNs to map the expert's actions to the agent's observations using abundant data. It does not require the agent to interact with the environment during training, making it a simple approach. However, behavior cloning suffers from several limitations, such as not being purpose-aware, having to learn primitive skills that can be easily solved using off-the-shelf solutions, and well-known error accumulation during task execution. To overcome these issues, researchers have developed an alternative approach called Hierarchical Behavior cloning (HBC), which involves a high-level model that learns intermediate goals and a lowlevel model that predicts sequences of actions to reach subgoals, as demonstrated in studies such as [START_REF] Yang | Multi-expert learning of adaptive legged locomotion[END_REF], [START_REF] Devin | Learning modular neural network policies for multitask and multi-robot transfer[END_REF], [START_REF] Xu | Toward modularization of neural network autonomous driving policy using parallel attribute networks[END_REF]. HBC uses an end-to-end framework that significantly reduces the error accumulation and enhances purpose-awareness. Nevertheless, HBC still needs to learn some primitive skills that roboticists can easily solve using off-the-shelf solutions.

C. Keypoint-based manipulation learning

As a task-relevant context on object, keypoints of object surface come with a significant importance for Visual Robotic Manipulation (VRM). With the progressive development of object keypoint discovery approaches [START_REF] Sun | Deep highresolution representation learning for human pose estimation[END_REF], [START_REF] Mallis | From keypoints to object landmarks via self-training correspondence: A novel approach to unsupervised landmark discovery[END_REF], [START_REF] Tejas | Unsupervised learning of object keypoints for perception and control[END_REF], [START_REF] Suwajanakorn | Discovery of latent 3d keypoints via end-toend geometric reasoning[END_REF], many manipulation learning methods have been proposed to leverage keypoints-based context for geometry and semantic representations. But most of these methods leverage keypoints as a bridge for facilitating the transfer of manipulation skills between demonstrators and imitators. Gao et al. [START_REF] Gao | vil: Keypoints-based visual imitation learning[END_REF] decomposition robotic manipulation task into keypoints constrain representation and control policies selection, which allows the successful reproductions across various tasks. Yang et al. [START_REF] Yang | Learning periodic tasks from human demonstrations[END_REF] utilized keypoints detector to represent the similarity between a human demonstration and robot execution, then maximized it by Bayesian optimization. Wada et al. [START_REF] Wada | Morefusion: Multi-object reasoning for 6d pose estimation from volumetric fusion[END_REF] represent 6D pose of objects in voxels to improve multi-object reasoning in cluttered scenes. Kulkarni et al. [START_REF] Tejas | Unsupervised learning of object keypoints for perception and control[END_REF] proposed a unsupervised keypoints discovery network, then apply the learned object keypoints as state input that related to policy explorations over Reinforcement Learning (RL) settings.

III. BACKGROUND

A. Behavior Cloning for Visual Robotic Manipulation

Behavior Cloning (BC) refers to a supervised learning approaches used to learn sensorimotor policies from offline data. BC only requires pairs of sensory observations, e.g., images, associated to expert actions. Given access to an expert agent, we can build a BC dataset,

D = {(o i , a i )} N i=1
, where o i are sensory observations, and a i = π * (o i ) are the respective actions taken by the expert π * . When using BC for robotic manipulation, actions are usually described as the Cartesian-position of next gripper Tool Center Point (TCP) and Quaternion-orientation, in the scene, also associated with the gripper state, e.g. opening amount, a i = (x tcp , y tcp , z tcp , q 1tcp , q 2tcp , q 3tcp , q 4tcp , Grip.State). Typical solutions to acquire such expert actions consist of robot teleoperating by human, robot behavior hard-coding with omniscient knowledge in simulation, expert RL policy training using hard-coded reward function as well as huge amount of simulation trials.

When considering visual robotic manipulation, the observations are limited to raw images of the environment

o i = {I v i } V v=1
, where I v i represents the image from view v in a multi-view configuration. The goal of a BC algorithm is to learn a policy π, parameterised by θ, that produce similar actions to the expert π * when provided with the same observation o i . Optimal parameters θ * are found by minimizing the BC loss l :

θ * = arg min θ i l (π (o i ; θ) , a i )
At inference phase, the trained policy is used to sequentially predict actions from observations to execute the target task.

B. Limitations

BC is used to solve a sequential decision problem, where future observations depend on previous actions. At training phase, it violates the i.i.d. assumption made in statistical learning. Moreover, at inference phase, errors of the action predictions accumulate along the task execution, which leads to a distributional shift between training and inference observations, which subsequently results in out-of-distribution prediction.

To alleviate this problem, hierarchical algorithms has been proposed to decompose trajectories of manipulation task into sub-trajectories that mostly consisted in existing off-the-shelf robotics primitives, e.g. "reaching" and "grasping". However, both traditional BC and RL still have to learn by themselves to accomplish the target tasks.

IV. APPROACH Instead of considering (observation, action) tuples to train our model, we format the dataset as a set of trajectories, D = {T i } N i=1 , in which each trajectory can be represented as T = {(o t , a t )} T t=1 . We can then further decompose a task as sequence, T ′ = {(o m , wp m )} M m=1 , M << T made of a few waypoints wp m that can be sequentially reached using off-the-shelf robotics primitives. Fig. 1 illustrates the observation associated to 4 waypoints which define a "cube lifting" task.

Given these statements, we designed a model that can directly predict such set of waypoints from the first observation of the scene. In this scenario, the training dataset D can be formatted as a set of tasks T i = o i 0 , {wp i 0 , ..., wp i M } , where the few waypoints needed to accomplish the task must be directly predicted from o 0 . This approach benefits from several advantages. First, the sequential decision making problem solved by traditional BC or RL approaches is changed to a one-step prediction problem, where the whole trajectory is predicted from initial observation of the scene, which eliminates both accumulation error during inference and the violation of the i.i.d. assumption during the training phase. Secondly, the trained model focuses on understanding the high level structure, i.e. the purpose of the task, while sub-trajectories prediction is remained for existing and more robust robotics primitives.

This section details the framework proposed to solve the one-step trajectory prediction problem mentioned above.

A. Overall framework description

Fig. 2 describes the whole proposed framework. The framework is made of two main components: (1)The deep learning model, predicting waypoints from a multi-view images of the initial observation, denoted as Multi-View Fusion Transformer (MFT) in Fig. 2; (2)A path planner, that can drive the robot to the predicted waypoints.

The waypoints prediction framework can be divided in two parts: [START_REF] Dalal | Accelerating robotic reinforcement learning via parameterized action primitives[END_REF]The image processing part aims to inherently learn to locate the waypoints in the 2D images; (2)The 2D-to-8D projection part that learns to map the muli-view 2D information extracted by the the image processing part to actual 8D robot poses (x tcp , y tcp , z tcp , q 1tcp , q 2tcp , q 3tcp , q 4tcp , Grip.State) that can be understood by the robotics path planner. The whole framework is trained end-to-end from ground truth 8D waypoints recovered from expert demonstrations. The method used to choose and extract the waypoints from the demonstrations is detailed in section V-A.

B. Framework details

The architectural choices for the framework parts were guided by the actual functions they had to satisfy. The aim of the image embedding module is to extract pertinent characteristics from various individual views. These features are later fused by the 2D-to-8D projection model to recover the final 8D waypoints. Furthermore, we can state that the image embedding module aims to extract information about the 8D waypoints projected in various 2D views. The 2D-to-8D projection module can be interpreted as a sophisticated learned triangulation method that projects 2D spatial information into the 3D scene space.

1) Image Embedding: We took inspiration from a cuttingedge deep learning architecture for 2D human pose estimation, called HRNet [START_REF] Sun | Deep highresolution representation learning for human pose estimation[END_REF], to develop our image embedding module. HRNet has the capability of generating feature maps with high spatial resolution as well as rich semantic information. This combination is critical for learning manipulation tasks, particularly those involving small objects, as it provides a comprehensive understanding of the objects in the scene. In this work, we employ the HRNet-W32 [START_REF] Sun | Deep highresolution representation learning for human pose estimation[END_REF] architecture as the feature extraction backbone, which will consequently generate M-channels feature maps for each view in 2D space.

2) MFT: Our Multi-view Fusion Transformer (MFT) is a simple Transformer-based network which aims to encode the latent waypoints features in 2D space, to 8D waypoints in 3D space, as shown in 3. Firstly, the latent feature map of each view is flattened to M 1-D tensor, which are then fed into a multi-layer perceptron (MLP) to obtain implicit features for 8D waypoints. The shape of these features is M × 16, where M is then number of waypoints, and 16 is the latent dimension of the features. After the pre-embedding step, the generated features from each view are processed using our proposed MFT module, which includes learnable spatial embedding. Subsequently, four implicit vectors are produced after passing through a residual MLP layer, followed by Fig. 3: Framework of our MFT module. A high-level implicit representation with shape of M × 16 will be obtained from the latent 2D waypoints feature map through a flatten operation and MLP layers, prior to its transmission to MFT. Subsequently, these implicit representations will be mapped into vectors with same shape of input through a pre-embedding and a residual MLP layer. Taking generated multi-view vectors as input, our 4 branches cross-transformer will enable the interaction among multi-views features. Besides, a residual connection is also used to alleviate the gradient vanishing.

four transformer blocks in parallel, with a cross-connection configuration [START_REF] Li | Mhformer: Multi-hypothesis transformer for 3d human pose estimation[END_REF]. The residual connection is also utilized in this block to moderate gradient vanishing. Finally, the resulting outputs are concatenated and passed through MLP layers to generate the final output.

3) Training details: The whole framework is trained endto-end to minimize the following supervised loss.

L = N i=1 1 M M m=1 w i m -w i * m 2
, where w i m M m=1 = π o i 0 ; θ . π represents the framework that predicts M waypoints from the initial observation of the scene o 0 ; w i m and w i * m represent respectively the m-th predicted and ground-truth waypoints for the i-th trajectory in the training dataset.

4) Path Planning: As we conducted our experiments in RLBench [START_REF] James | Rlbench: The robot learning benchmark & learning environment[END_REF] simulation, we used the integrated 'ABS EE POSE PLAN WORLD FRAME' path planer, which can drive the robot TCP to a specified absolute position and with the specified orientation in the world frame. This path planner relies on the Open Motion Planning Library [START_REF] Ioan | The open motion planning library[END_REF] combined with a feedback-control loop. The gripper control is also integrated to the path planer and is discretized to either 'open gripper' or 'close gripper'. The planning always operates in two steps, (1) driving the robot TCP to the desired position, and (2) updating the gripper V. RESULTS

A. Simulation

We conducted our experiments using RLBench [START_REF] James | Rlbench: The robot learning benchmark & learning environment[END_REF] simulation because (1) it provides a multi-camera working environment, (2) it gives access to a wide variety of tasks, and most importantly, (3) within the simulation, an automatic task demonstration generator is integrated. This generator utilizes sets of pre-defined waypoints for each available task to create custom waypoint-based datasets that can be used for further analysis and training. As a result, we propose a tool that can generate datasets specifically tailored to the novel paradigm proposed in this work using RLBench. The generated datasets comprise demonstrations of tasks, consisting of initial observations of the scene, and their corresponding "ground truth" sequences of 8D waypoints necessary for successfully completing the respective tasks. As illustrated in Table I, the automatic demonstration generator provides almost two orders of magnitude more steps than the number of waypoints that are actually needed to accomplish the task.

The state of the simulation is also saved for each generated demonstration, so that the tasks can be later replayed with the exact same scene configuration. By leveraging this feature, an evaluation of the trained framework can be executed in three steps. (1) A part of the generated dataset is kept apart for evaluation during the framework training. (2) Once the training is done, the evaluation data are inferred by the trained model, which will predict waypoint sequences corresponding to the evaluation examples. (3) Simulation is iteratively loaded with scene configurations that correspond to the evaluation data. The predicted waypoints generated from these configurations are then utilized by the path planner in an attempt to accomplish the assigned tasks. A success rate can be computed over the evaluation data by reporting the number of successful examples over the total number of examples.

B. Is our approach competitive with existing solutions for visual-robotic-manipulation learning ?

To compare our approach to existing robotic manipulation learning solutions, we referred to James et al. [START_REF] James | Coarse-to-fine q-attention: Efficient learning for visual robotic manipulation via discretisation[END_REF], who ran two sets of experiments in their work. In the first set of experiments, they compared common imitation learning and reinforcement learning methods , including ARM [START_REF] James | Q-attention: Enabling efficient learning for vision-based robotic manipulation[END_REF], BC, SAC+AE [START_REF] Yarats | Improving sample efficiency in model-free reinforcement learning from images[END_REF], DAC [START_REF] Kostrikov | Discriminator-actor-critic: Addressing sample inefficiency and reward bias in adversarial imitation learning[END_REF], SQIL [START_REF] Reddy | Sqil: Imitation learning via reinforcement learning with sparse rewards[END_REF] and DRQ [START_REF] Kostrikov | Image augmentation is all you need: Regularizing deep reinforcement learning from pixels[END_REF], and their baseline, C2F-ARM, [START_REF] James | Coarse-to-fine q-attention: Efficient learning for visual robotic manipulation via discretisation[END_REF] , on a set of tasks that can be solved from the front camera only. In a second set of experiments, they evaluated ARM [START_REF] James | Q-attention: Enabling efficient learning for vision-based robotic manipulation[END_REF] and C2F-ARM [START_REF] James | Coarse-to-fine q-attention: Efficient learning for visual robotic manipulation via discretisation[END_REF] on tasks that require more than one camera to be achievable.

ARM, C2F-ARM, SAC+AE, DAC, SQIL and DrQ are RL-based approaches, which means they require thousands of interactions with the environment to learn a task. James et al. [START_REF] James | Coarse-to-fine q-attention: Efficient learning for visual robotic manipulation via discretisation[END_REF] provided all baselines, including RL ones, with 100 demonstrations for each task, except for their C2F-ARM baseline, which was provided with 10 demonstrations. However, they used a data augmentation method to extend their data. Since we didn't use data augmentation strategies in our case, we provide our model with 300 demonstrations. However, in section VI, we discuss how our paradigm allows considerably more efficient data collection than previous imitation learning paradigms.

As a proof of concept, we selected, (i) two tasks in the first set of experiments, to validate the performances of our framework compared to a wide range of robot manipulation learning methods, and (ii) two tasks in the second set of experiments to validate that our approach can compete with existing multi-view approaches.

We point out that in both sets, the authors had access to both RGB and depth information from the considered cameras, while our method learns to recover depth information from RGB images only, using our multi-view fusion module. For this reason, our framework always considers the four cameras placed around the scene in the simulation, namely 'front', 'overhead', 'over-shoulder-right' and 'over-shoulderleft' cameras, but only accesses RGB information.

Note that in section V-D, we discuss two additional tasks that we tried but actually partially failed to train. Fig. 4 shows the results for the first set of experiments. It highlights that our method (1) outperforms traditional BC by a large margin and (2) slightly outperforms mean results of the best state-of-the-art RL based approach, C2F-ARM, on tasks where the manipulated objects can be seen from all view points on the initial observation of the scene. Other RL approaches were already outperformed by C2F-ARM, and by extension we outperform them as well. Fig. 5 shows the results for the second set of experiments. It highlights that our method can still compete with C2F-ARM when they also access multiple views.

C. Does the one-shot waypoints prediction help reduce error accumulation during task execution? Section V-B reports evaluation results of various baselines, but the differences among them can be attributed to several factors. Firstly, C2F-ARM, our model, and other existing baselines employ different networks for observation representation. Secondly, while our model predicts waypoints, other approaches predict the gripper pose of the next step. Finally, our model predicts the entire trajectory configuration at once, whereas other approaches rely on sequential decision-making, which can result in prediction errors and observation distribution shifts.

To isolate the impact of error accumulation and associated observation distribution shift on the same tasks studied in section V-B, we trained a behavior cloning model to predict the next waypoint sequentially from low-dim observations. This model has access to the 6D poses of each object in the scene, enabling it to infer ground-truth object positions directly without relying on a vision model to process images of the scene. While the model predicts waypoints like our approach, it can still be affected by prediction errors and observation distribution shifts due to its sequential predictions.

The results of the waypoint-based behavior cloning trained from Ground Truth 6D objects poses and our framework are compared in Table II. This table displays both the training loss and success rate outcomes. To gain a better understanding of how the training results impact the success rate on model deployment, we separated the training loss into three losses: The "Train Pose" loss, which concentrates on the first three dimensions of the 8D waypoints, corresponding to the position of the robot TCP. The "Train Quat" loss focuses on the orientation of the TCP, while the "Train Grip" loss is concerned with the gripper state.

Fig. 2 :

 2 Fig.2: Overview of our keypoints-based manipulation learning paradigm: The input is 4 independent images from different views with resolution of 256 × 256. An image embedding module is built to extract 2d waypoints representation in a latent space. Consequently, the latent representation will be sent into our proposed Multi-view Fusion Transformer (MFT) module to obtain the 8D waypoints prediction in 3D space, which can be used as the reference for following path planner. The whole framework will be trained in an end-to-end way. And the the success rate will be evacuated under RLbench simulation.

Fig. 4 :

 4 Fig. 4: Our training results compared with [7] learning curves on monocular setting.

Fig. 5 :

 5 Fig. 5: Our training results compared with [7] learning curves on multi-view setting.

TABLE I :

 I Number of waypoints (M ) vs min and max total number of steps (respectively T max and T min ) when using RLbench demonstration generator. The tasks Trash., Reach, Grill, PickLift, Unplug and MoneyOut correspond to RLBench tasks considered in our experiments, respectively 'put rubbish in bin', 'reach target', 'meat off grill', 'pick and lift', 'unplug charger' and 'take money out safe'.

	state.

https: //github.com/KevinRiou22/

Grill.

Unplug PickLift MoneyOut

Behavior Cloning From 3D Ground Truth Train Pose 3e10 -5 3e10 -4 3e10 -5 3.5e10 -4 1e10 -5 Train Quat.

3e10 -4 4.6e10 -3 5e10 -5 5.2e10 -2 4e10 -5 Train Grip.

6.3e10 -3 1.4e10 -3 2.8e10 -3 5e10 -6 1.5e10 -2 Success rate 0 100 70 0.8 0.8

Ours

Train Pose 1.6e10 -2 9.5e10 -3 8.6e10 -3 5.7e10 -2 1.3e10 -2 Train Quat.

2.3e10 -2 2.2e10 -2 2.4e10 -2 0.53 4.3e10 -2 Train Grip.

1.7e10 -3 2.7e10 -3 1.3e10 Compared to our approach, the behavior cloning model trained from object poses has significantly lower "Train Pose" and "Train Quat" losses. Without any observation distribution shift between training and inference, or prediction error accumulation, the behavior cloning model should be much more accurate than our framework, and, by extension, should achieve a better success rate. However, our framework can produce similar or superior success rates. In shorter, despite the behavior cloning approach shows better prediction performances during training, the final success rate is affected by error accumulation during task execution, which is not the case with our framework. The exception is the 'pick and lift' task, which our model failed to grasp. For the 'put rubbish in bin' task, which completely fails, we noticed that the behavior cloning model was trapped in the neighborhood of the first waypoint, while never actually reaching this waypoint. This was an out-of-distribution case for the trained model

D. Failure cases

This section highlights the current limitations of our framework. Fig. 6 points out mitigated results on take money out task, while Fig. 7 highlights that typical failure case for our framework is caused by critical occlusions on the initial observation of the scene.

VI. CONCLUSION

Although impressive progress has been made, RL and standard BC still facing some intrinsic flaws, e.g. huge amount of interaction for RL and accumulation error for BC. In this work, we proposed an offline hierarchical behavior cloning paradigm, with motivation of eliminating accumulation errors using a fixed spatial representation. Two stages are combined in our framework. Firstly, a keypoints-based high-level planner is employed to represent the fixed spatial configuration only from the very initial observation, with the form of task-relevant waypoints. Secondly, a low-level robotic path planner is used to guide the robot by reaching predicted waypoints. Extensive experiment illustrated that our method can achieve similar performance with RL approaches, and outperforms existing BC approaches in some task, with only inference the initial observation. However, since all the trajectory waypoints are predicted before the task execution, the proposed architecture is mostly suitable for static scenes.