Karen Boulos
email: karen.boulos@net.usj.edu.lb

Kinda Khawam
email: kinda.khawam@uvsq.fr

Melhem El
email: melhem.helou@usj.edu.lb

Marc Ibrahim
email: marc.ibrahim@usj.edu.lb

Steven Martin
email: steven.martin@lri.fr

Hadi Sawaya
email: hadi.sawaya@usj.edu.lb

A Hybrid Approach for RRH Clustering in Cloud Radio Access Networks based on Game Theory

Keywords: • Networks → Mobile networks, Wireless access points, base stations and infrastructure, Cloud Radio Access Networks, RRH clustering, power saving, game theory, best response, reinforcement learning

Cloud Radio Access Networks (C-RAN) is an evolution in the base station architecture, mainly composed of two elements: The Base Band Unit (BBU) and the Remote Radio Head (RRH). The BBU is a centralized pool of computational resources to provide the signal processing and coordination functionality required by all cells, while the RRHs are light radio units that User Equipment (UE) connects to via the RAN. Many advantages are derived from this architecture, such as dynamic BBU-RRH associations and statistical multiplexing gains. In particular, the BBU-RRH association problem is crucial for reducing power consumption. In this paper, we focus on decentralized BBU-RRH association, which has not received attention in the literature. Therefore, the aim of this work is to propose a hybrid two-stage approach that includes a game theoretic framework for the BBU-RRH association, and a centralized scheme to set the adequate number of available BBUs. The game among RRHs is solved by two different algorithms. The first relies on the best response algorithm, namely H-BR-IACA. The second is based on a reinforcement learning method (the replicator dynamics), namely H-DR-IACA. We compare our devised solution to a centralized approach proposed in a previous work. The results of our proposition show close performance to the centralized method.

INTRODUCTION

Wireless communications have seen a tremendous growth in traffic demand and a rapid evolution of technology. Hence, there is an increasing challenge in the operation of the upcoming 5G networks: huge amounts of data are becoming an overwhelming part of the traffic, while the associated income is shrinking. Similarly, both the energy consumed and interference levels are growing rapidly due to aggressive spectrum allocation in such dense networks. Finally, such densification leads to an imbalance in resource consumption due to the disparity of the mobile traffic across the network. To meet these challenges, a promising approach toward such 5G networks is to introduce cloud-based radio access networks (C-RAN). Unlike conventional RANs, a C-RAN decouples the baseband processing unit (BBU) from the remote radio head (RRH), allowing for centralized operation of BBUs and scalable deployment of lightweight RRHs. Unlike in conventional architecture, many RRHs are clustered to a single BBU when traffic load is low, resulting in power consumption reduction and increased energy efficiency. In addition, when RRHs are clustered together, they logically form a single cell and act as a distributed antenna system. Thus, the intra-cluster interference is canceled and the spectral efficiency is enhanced.

However, the challenge is to design a suitable logical mapping between each RRH and one or multiple managing BBUs, called BBU-RRH association. This design should be performed carefully so as to optimize the resource allocation, taking into account the load balancing factor of served cells and the intra-cluster interference cancellation among RRHs associated to the same BBU. In this paper, we put forward a hybrid approach where a central controller decides the number of available BBUs, while the RRHs choose in a distributed fashion to join a given BBU in a way to minimize their own cost.

RELATED WORK

There are several papers addressing the BBU-RRH association problem (also known as RRH clustering problem), with a common purpose of power consumption reduction. In [START_REF] Taleb | Centralized and distributed RRH clustering in Cloud Radio Access Networks[END_REF], authors formulate the clustering problem as a coalition game, where RRHs organize themselves into clusters in order to increase a suitable utility function. In [START_REF] Wang | On Joint BBU/RRH Resource Allocation in Heterogeneous Cloud-RANs[END_REF] and [START_REF] Boulos | RRH clustering in cloud radio access networks[END_REF], authors propose a bin packing formulation for the RRH clustering problem. Authors in [START_REF] Chen | RRH Mapping Scheme Using Borrow-and-Lend Approach in Cloud Radio Access Networks[END_REF] propose a centralized approach for the BBU-RRH association problem. They suggest to switch the RRHs of a crowded BBU to a less loaded neighboring BBU.

While all those papers show significant improvement in terms of power saving, they ignore the inter-cluster interference. Unlike the previous references, the work in [START_REF] Boulos | Interference-aware clustering in cloud radio access networks[END_REF] shows that interference-aware decisions lead to optimized clusters of RRHs and enhance the power saving metric. The authors of the paper propose a Set Partitioning Problem formulation (SPP) and prove that the chosen subsets of RRHs are energy efficient, reducing the level of interference in the network. However, the work in [START_REF] Boulos | Interference-aware clustering in cloud radio access networks[END_REF] is a centralized formulation that requires significant signaling information concerning network load conditions and interference levels. This would lead to a huge burden in terms of signaling load over the fronthaul between BBUs and RRHs. The aim of this paper is to introduce a load balancing and interference-aware hybrid clustering scheme that operates in two stages. The main objective is to minimize the network power consumption (i.e, minimize the number of active BBUs), while guaranteeing minimum throughput requirements for UEs. Moreover, our solution provides close performances to the centralized proposition of [START_REF] Boulos | Interference-aware clustering in cloud radio access networks[END_REF], with significant reduced signaling load. To the best of our knowledge, this is the first paper to propose a game theoretic framework for the BBU-RRH association problem that is solved into two different algorithms: The first relies on the best response algorithm [START_REF] Voorneveld | Best-response potential games[END_REF]. The second is based on a reinforcement learning algorithm, namely the replicator dynamics [START_REF] Sastry | Decentralized learning of Nash equilibria in multi-person stochastic games with incomplete information[END_REF].

The rest of the paper is organized as follows: Section 3 describes the system model. Section 4 puts forward our hybrid load balancing and interference-aware clustering problem. Performance evaluations are displayed in section 5. Section 6 concludes the paper.

SYSTEM MODEL 3.1 Network Topology

Our topology consists in a cellular network composed of R RRHs. Let R be the set of RRHs. It is defined as R = 1, ..., r, ...R . We define the set of clusters (i.e., subsets of R) as C = 1, ..., s, ...S . We also define a partition P ⊂ C of R such as s ∩ s ′ = ∅ ∀s, s ′ ∈ P, s s ′ and s ∈ P s = R.

Area Decomposition.

Clustering decisions are based on longterm measurements. It is costly to consider decisions based on time granularity of seconds or even milliseconds, because BBU-RRH association decisions require reconfigurations of the optical fronthaul connections. In addition, the traffic pattern changes at a scale of minutes. Thus, we consider input parameters based on average metrics, such as average radio conditions and average number of UEs. Precisely, the covered area is meshed into several discrete zones. Each zone is assimilated to a single test point, where a cumulative rate is generated from all UEs located in the same zone. We consider a quadratic meshing of the area and we calculate the average distance (D avr ) between any zone of index z and a RRH of index r . The calculation of D avr and the UE association scheme are detailed in [START_REF] Boulos | Interference-aware clustering in cloud radio access networks[END_REF].

Cluster Rate Model

We assume that a Transmission Point Selection Technique (TPS) is applied in each BBU (i.e, cluster). Consequently, the corresponding data is transmitted from a single RRH at a time and the interfering signals result in from the inter-cluster interferences. The SI N R z,r ,s ratio of zone z, served by RRH r , associated to cluster s would be expressed as:

SI N R z,r ,s = P r .G z,r N 0 + r ′ r r ′ ∈s ′ s P r ′ .G z,r ′ , (1)
where P r and P r ′ denote the power emitted by RRHs r and r ′ respectively. G z,r and G z,r ′ are the channel gain between zone z and r , r ′ respectively. N 0 represents the thermal noise power and finally s and s ′ the clusters that RRHs r and r ′ belong to respectively. The peak throughput of a UE located in zone z, served by RRH r that is associated to cluster s is expressed as χ z,r ,s = BW • loд 2 1 + SI N R z,r ,s , where BW is the total bandwidth within a BBU.

Assuming a fair resource sharing model between n s UEs, sharing the resources of cluster s, and a full buffer traffic model, the throughput perceived by a UE belonging to zone z of RRH r , associated to cluster s is 1 n s .χ z,r ,s . Consequently, if n z,r UEs (among n s ) are located in zone z, the cumulative throughput within such zone is expressed as T z, r, s = n z ,r •χ z ,r ,s n s

. The cumulative throughput derived in RRH r that is associated to cluster s is expressed as T r, s = Z z=1 T z, r, s , where Z is the total number of zones belonging to RRH r .

The cumulative throughput in cluster s (i.e., BBU s) is expressed as T s = R r =1 y r ,s • T r, s , where R is the total number of RRHs in the network, and y r ,s denotes a binary variable defined as:

y r ,s = 1, if r is associated to s y r ,s = 0, otherwise (2)

BBU Power Consumption Model

We assume a linear power consumption for the BBU, as a function of the cumulative rate of cluster s [START_REF] Zhao | Energy-delay tradeoffs of virtual base stations with a computational-resource-aware energy consumption model[END_REF]. It is expressed as P s = A + B.T s , where P s denotes the power consumed by an active BBU, A is the minimum power consumption of an active BBU at zero load and B is the coefficient variation of P s as a function of T s which is the cumulative throughput in cluster s as previously defined.

HYBRID CLUSTERING BASED ON GAME THEORY

Our hybrid proposition operates in two stages described in paragraphs 4.1 and 4.2. Initially, a central controller sets the number of BBUs made available to the RRHs. In the first stage, the distributed BBU-RRH association is portrayed as a potential game among competing RRHs to choose a suitable BBU. When the Nash Equilibrium [START_REF] Nash | Equilibrium points in n-person games[END_REF] is attained, the second stage is reached, where the number of available BBUs can be modified. The two stages are iteratively executed until the right number of active BBUs is decided.

Non-Cooperative Game for BBU-RRH Association

Non-cooperative game theory models the interactions between players competing for a common resource. Hence, it is well adapted to RRH clustering. We define a multi-player game G between the R RRHs.

We present the general framework of game G = (R, S, C) that can be described as follows: The set R of RRHs is the set of players. The set P of available BBUs. An action of a RRH is selecting one of the available BBUs. The strategy of RRH r is denoted by the vector y r whose components are y r ,s defined in [START_REF] Boulos | RRH clustering in cloud radio access networks[END_REF]. Hence, y = (y r ) r ∈R ∈ S is a pure strategy profile, and S = S 1 × S 2 × ... × S R is the space of all profiles. The set of cost functions C = {C 1 , C 2 , ..., C R } that quantify the players' preferences over the possible outcomes of the game. Outcomes are determined by the particular action chosen by RRH r and the particular actions chosen by all other players.

C r y r , y -r = s ∈ P y r ,s • α • α ′ • r ′ r y r ′ ,s Term 1 + β • β ′ • r ′ r G r ′ ,r • (1 -y r ′ ,s ) + N 0 Term 2 , (3)
where G r ,r ′ is the channel gain between RRHs r and r ′ , and y r ,s is the binary variable defined as in [START_REF] Boulos | RRH clustering in cloud radio access networks[END_REF]. The cost sustained by a given RRH r in any selected BBU depends upon the congestion impact inflected by other RRHs r ′ ∈ S sharing the same radio resources of the common BBU s. The latter is depicted by Term 1 in (3) and acts as a load balancing function among active BBUs. Furthermore, the cost function should encompass the interference impact. Accordingly, Term 2 reflects the inter-cluster interference which is cancelled between RRH r and other RRHs that have chosen the same strategy (equivalently BBU) s. α and β are the weighting factors. We note that α + β = 1 and α ′ , β ′ are the normalization factors.

4.1.2 Attaining Pure Nash Equilibriums. In a non-cooperative game, a solution is obtained when all players adhere to a Nash Equilibrium (NE). A NE is a profile of strategies in which no player will profit from deviating its strategy unilaterally. Hence, it is a strategy profile where each player's strategy is an optimal response to the other players' strategies.

Our game G is a finite game and in general such games are not guaranteed to have Pure NEs (PNEs). However, for our game an exact potential function [START_REF] Duy | Potential Game Theory: Applications in Radio Resource Allocation[END_REF] exists, which means that the unilateral change of one RRH strategy y r to y ′ r results in a change of its cost function that is equal to the change of a so-called potential function ϕ : S R → R.

An example of such a potential function is given by the following:

ϕ y = s ∈ P r ∈R y r ,s 1 2 • r ′ r y r ′ ,s + y r ,s + 1 2 • r ′ r 2 -y r ′ ,s G r ′ ,r + N 0 , (4)
Furthermore, such games have the appealing property of converging to PNE. In our work, we apply two different algorithms to solve the game. The first is the best response, which guarantees the convergence to PNE. The second is based on a reinforcement learning algorithm, namely replicator dynamics, that converges to PNE for exact potential games:

• Best response algorithm: At each round of the best response algorithm, a RRH chooses the BBU that provides it the lowest cost (3). Convergence is attained when all RRHs choose the same strategies as in the previous round. • Replicator dynamics algorithm: We denote by p r ∈R (t) the vector of mixed strategies whose components are p s r (t), defining the probability that RRH r chooses BBU s at iteration t. At each t, a RRH selects a BBU according to the distribution probability p r ∈R (t) and receives a cost. The vector p r ∈R (t) is updated accordingly and the process is repeated until the PNE is learned.

Centralized BBU Activation/Deactivation

The second stage of the algorithm decides whether to activate or to deactivate a BBU. Precisely, this stage consists in examining the lowest throughput per UE among all clusters and verifies if constraints (5) are respected or not:

T s ≥ n s • D min , ∀s ∈ P (5)
Constraints [START_REF] Nash | Equilibrium points in n-person games[END_REF] state that the throughput per UE should stay greater than a minimal threshold, denoted by D min . If not, the algorithm increases by one the number of available BBUs, and re-launches the BBU-RRH association game of the first stage. However, to reduce power consumption, when the lowest throughput per UE respects (5), the algorithm decreases the number of active BBUs and falls back to the first stage. When the algorithm decides to deactivate a BBU and reactivate it again, it stops running and outputs a solution. More precisely, when the algorithm deactivates a BBU and realizes that constraints [START_REF] Nash | Equilibrium points in n-person games[END_REF] are not respected, it falls back to the previous number of active BBUs (before the deactivation) and sets back the previous BBU-RRH associations. In this work, the central controller starts from a random number of active BBUs.

In this section, we provide a comparative study between our hybrid schemes (H-BR-IACA and H-DR-IACA), the No Clustering scheme, where a BBU is entirely dedicated to a single RRH, and finally a centralized scheme proposed in a previous work [START_REF] Boulos | Interference-aware clustering in cloud radio access networks[END_REF] deemed C-IACA.

Simulation Results

We use Matlab for simulations, and we consider a network composed from seven cells. As previously mentioned, the area is meshed into several quadratic zones. We consider a uniform UEs distribution in each cell. In other terms, the number of UEs per cell is the same in each zone associated to the cell. We set the weights α and β of the cost function to 0.75 and 0.25 respectively (cf. equation ( 7)).

The simulation parameters are shown in table 1. We start by providing the first performance metric given in Figure 1. It represents the number of active BBUs as a function of the number of UEs per cell. For 1 to 4 UEs per cell, the C-IACA and the hybrid schemes only one BBU. When the number of UEs per cell reaches 5, more BBUs are activated. All of the C-IACA, H-BR-IACA and H-DR-IACA provide the same number of active BBUs except for 9 and 10 UEs per cell, where the H-BR-IACA scheme activates an additional BBU in comparison with the centralized scheme for 9 UEs per cell, and the H-DR-IACA scheme activates a slightly higher number of BBUs for 10 UEs per cell in comparison to the C-IACA and H-BR-IACA schemes. The No Clustering scheme activates all available BBUs whatever the number of UEs per cell is. Figure 2 displays the power saving metric as a function of the number of UEs per cell. Between 1 and 4 UEs per cell, we notice the highest power savings for the C-IACA and the hybrid schemes. Almost same power economies are realized in all schemes, except for 9 and 10 UEs per cell, where additional BBUs are activated by the hybrid schemes in comparison to the C-IACA as shown in Figure 1. Further, no power economy is achieved when the number of UEs per cell exceeds 11, as all BBUs are activated.

Figure 3 shows the mean throughput per UE as a function of the number of UEs per cell, given as the total realized throughput within the system over the total number of UEs. The highest values are attained through the No Clustering scheme owing to the full buffer traffic model. However, we notice that all of the centralized and hybrid schemes realize much more power economy at the cost of lower mean throughput per UE. In addition, the C-IACA and the hybrid schemes guarantee the targeted QoS for UEs, and do not offer throughputs less than D min = 2 Mb/s. For high number of UEs per cell, the mean throughput per UE is reduced for all schemes due to scarcity in radio resources. We note that the algorithms are executed over a hardware of 64-bit, Intel 7 core, 2.5 Ghz. We notice that the H-BR-IACA scheme consumes less computational time for all load conditions in comparison to other schemes. The H-DR-IACA scheme consumes less computational time than the C-IACA scheme for a number of UEs per cell that varies between 1 and 7. When the number of UEs per cell is greater than 7, the computational time of the H-DR-IACA scheme is greater than all other schemes. In fact, when a higher number of choices (i.e., number of BBUs) is attributed to the reinforcement learning algorithm (H-DR-LIACA), the time to convergence would be greater. However, reinforcement learning algorithms require much less amount of signaling load information.

CONCLUSION

In this paper, we formulate a hybrid proposition for the RRH clustering problem comprising a centralized scheme and a non-cooperative game. The non-cooperative game is solved in two different manners. The first relies on the best response dynamics, whereas the second is a reinforcement based learning algorithm. We compare our devised schemes to an existing centralized approach based on SPP. The results show that our hybrid scheme realizes close performance to the centralized solution. In future work, we intend on submitting our schemes to the stringency of a real case topology with twenty cells.

4. 1 . 1

 11 Cost function. Each RRH seeks to minimize its own load balancing and interference-aware cost function by choosing an adequate strategy (i.e, BBU s). The cost function of RRH r is given by what follows:

Figure 1 :Figure 2 :

 12 Figure 1: Number of active BBUs: C-IACA vs. H-BR-IACA vs. H-DR-IACA vs. No Clustering

Figure 3 :Figure 4

 34 Figure 3: Mean throughput per user: C-IACA vs. H-BR-IACA vs. H-DR-IACA vs. No Clustering

Figure 4 :

 4 Figure 4: Execution time: C-IACA vs. H-BR-IACA vs. H-DR-IACA

Table 1 :

 1 Simulation parameters

	Parameter	Value
	Propagation Model	Cost Hata 231
	Shadowing Standard Deviation	10 dB
	Transmit Power of RRH	40 dBm
	Thermal Noise Power	-174 dBm/Hz
	BW	10 MHz
	Cell Radius	500 m
	A	50 W
	B	0.6
	D min	2 Mb/s
	5.1.1 The C-IACA vs. H-BR-IACA vs. H-DR-IACA vs. No Clustering
	scheme.

APPENDIX

We prove hereafter that the game at hand is an exact potential game. We simply need to show that if y and y ′ are two profiles which only differ in the strategy of one RRH r , then:

We begin by computing

Then, we compute ϕ y r , y -rϕ y ′ r , y -r =

As G r ,r ′ = G r ′ ,r (impact of RRH r on RRH r ′ is the same as impact of RRH r ′ on r ), we have indeed:

C r y r , y -r -C r y ′ r , y -r = ϕ y r , y -rϕ y ′ r , y -r .


Nhan Nguyen-Thanh

Dana Marinca
email: dana.marinca@uvsq.fr

Kinda Khawam
email: kinda.khawan@uvsq.fr

Steven Martin
email: steven.martin@lri.fr

Lila Boukhatem
email: lila.boukhatem@lri.fr

Multimedia Content Popularity: Learning and Recommending a Prediction Method

Keywords: popularity prediction, multimedia caching, YouTube, collaborative filtering, recommendation, machine learning, RNN, GRU, LSTM, tree-based regressors

In 5G networks, Mobile Edge Computing (MEC) has been proposed to enable computation and storage capabilities at the edge of Radio Access Networks. Proactive content caching in MEC is crucial to guarantee users' Quality of Experience thanks to the reduction of traffic latency. Predicting content popularity plays a key role in the effectiveness of proactive caching. In this paper, we propose a generic and flexible recommendation framework which allows recommending suitable learning and prediction algorithms among available ones, in order to predict content popularity. The investigated algorithms are categorized into two main classes: tree-based regressors and recurrent neural networks. Through the study case of YouTube video solicitation profiles, our proposed method, called Imputation-Boosted Collaborative-Filtering based Recommending Prediction Method (IBCF-RPM) shows its effectiveness in the prediction of content popularity for various popularity profiles. By running only 30% of the prediction algorithms, randomly chosen, on a given content profile, the proposed recommending method is able to estimate the accuracy of the other predictors and recommend a well-suited predictor for content popularity.

I. INTRODUCTION

The extraordinary increase in data traffic, mainly derived from mobile multimedia services [START_REF] Al-Falahy | Technologies for 5g networks: Challenges and opportunities[END_REF], social networks and over-the-top applications such as YouTube, has imposed a huge challenge to 5G networks for dealing with the requirements of more backhaul resources [START_REF] Zeydan | Big data caching for networking: Moving from cloud to edge[END_REF]. Mobile Edge Computing (MEC), currently standardized by ETSI [START_REF] Mao | A survey on mobile edge computing: The communication perspective[END_REF], is part of the effort toward a general solution for 5G-based network architecture. MEC allows computation and storage capabilities at the edge of Radio Access Network (RAN) [START_REF] Abbas | Mobile edge computing: A survey[END_REF], based on which new tasks such as mobile big data analytics and context-aware services can be implemented for performance optimization. Applications and services can be deployed and popular content items can be cached near end-users. This can help improve backhaul offloading and users' Quality of Experience (QoE) thanks to the reduction of traffic latency. Video on Demand is the heaviest data-driven among various multimedia services causing up to 80% of backhaul traffic by the year 2020 [START_REF]networking Index[END_REF]. Thus, predicting the popularity of multimedia contents for their proactive caching is a crucial issue. Several previous solutions have been proposed for content popularity prediction [START_REF] Tatar | A survey on predicting the popularity of web content[END_REF], [START_REF] Hassine | Popularity prediction in content delivery networks[END_REF], [START_REF] Hassine | Arma based popularity prediction for caching in content delivery networks[END_REF]. However, most solutions are based on a prediction approach which is mainly effective for a small dataset. For the very large amount of contents, an effective estimation requires simultaneously the support of a big data-enabled network architecture and an adaptive prediction framework. Recent achievements in the use of machine learning for big data analysis [START_REF] Deng | Achievements and Challenges of Deep Learning[END_REF] have encouraged us to aim at a learning-based approach for contents popularity predictions. YouTube [START_REF] Statistics | [END_REF], one of the largest video sharing platform, will be the study case in this paper. The number of the daily solicitations of a content represents a good indication of its popularity profile, so it is the mainly considered parameter to predict its popularity. The most solicited items will be cached to the nearby servers/base stations. Because daily requests evolution of a content is represented as time-series data, hereafter, we focus on time-series prediction. Predicting time-series has used various approaches such as exponential smoothing [START_REF] Hassine | Popularity prediction in content delivery networks[END_REF], [START_REF] Taylor | Forecasting intraday time series with multiple seasonal cycles using parsimonious seasonal exponential smoothing[END_REF] Auto Regressive Integrated Moving Average (ARIMA) [START_REF] Hassine | Arma based popularity prediction for caching in content delivery networks[END_REF], [START_REF] Gardner | Forecasting trends in time series[END_REF], Neural Network (NN) and Recurrent Neural Network (RNN) [START_REF] Hill | Neural network models for time series forecasts[END_REF], [START_REF] Laptev | Time-series extreme event forecasting with neural networks at uber[END_REF], tree-based regressors [START_REF] Tatar | A survey on predicting the popularity of web content[END_REF], etc. The prediction methods may perform differently regarding the same time-series. Hence, finding the mechanism that optimally activates the most efficient prediction algorithm for a given context is a challenge. Combinations of several forecasters have been proposed in order to keep individual forecasters' advantages in a universal predictor [START_REF] Conflitti | Optimal combination of survey forecasts[END_REF]. However, the combinations are mainly based on weighted sum methods, and adjusting weights is cumbersome. In this paper, we propose to adopt a recommendation technique [START_REF] Isinkaye | Recommendation systems: Principles, methods and evaluation[END_REF], for adaptively and dynamically selecting suitable prediction methods. This flexible technique can be applied to any time series, but in our context, it is applied for the prediction of YouTube contents' popularity.

The remainder of the paper is organized as follows. In Sect. II, we define the problem of multimedia content popularity prediction, YouTube dataset, and the methodology. In Sect. III-A, we focus on a selected set of learning-prediction methods. In Sect. III-B, we present the recommendation systems and the collaborative filtering (CF) technique that we use to propose a new recommendation method IBCF-RPM (Sect. IV) Sect. V, presents the simulation context and performance results and Sect. VI, concludes the study and mentions its perspectives. We address the problem of selecting the most appropriate method from a set of well-known methods to predict the popularity of multimedia contents. A large set of prediction methods going along with several configurable parameters lead to a huge number of predictors. Applying all available predictors to a very large catalog of contents (e.g. YouTube) in order to select the best one is intractable. We propose hereafter a recommendation framework able to recommend a suitable predictor for a given content profile based on the computed accuracy of a limited number of predictors and the similarities between different profiles. The predicted popularity can be used to supply a proactive caching of the most popular contents close to requesting users, but this issue is outside the scope of this paper.

A multimedia content popularity profile represents the number of daily requests for this content and the associated characteristics (e.g., number of likes). The daily requests number impacts explicitly the decision of caching it to the nearby servers in 5G MEC network. Because the number of daily requests is basically a time-series data, we focus on time series prediction.

With the complexity of sequences' dependencies and without any restrictions on the distribution, on the frequency or on the amplitude of a time-series data, the sequential prediction is difficult but is an important methodology applied to a wide range of problems. Therefore, the problem considered in this paper can be extended to other time-series prediction contexts.

A. YouTube popularity profile dataset

In this study, the dataset is constituted by real traces crawled from the YouTube site. A trace associated to a multimedia content includes the content uploading date, the subscribers' number, the shares' number and its daily solicitation until the crawling day. The list of YouTube videos is extracted from the YouTube-8M dataset [START_REF] Abu-El-Haija | Youtube-8m: A large-scale video classification benchmark[END_REF] composed of approximately 8 million videos. The crawled dataset includes contents having at least 10000 total solicitations between Nov. 2017 and Dec. 2017. For this study, we randomly selected 200 contents having over 1.5 million access and at least 500 solicitations/day.

Each content profile in the selected set is divided into three periods as shown in Fig. 1. The training period, constituted by several consecutive days, represents the training data. The testing period of consecutive days follows. The validation period composed by several consecutive days is separated from the testing period by a window W of variable length. The separating window is used to ensure the objectivity of the validation. This operating mode is usual in any time-series prediction problems.

B. Methodology

Various individual methods have been adopted for timeseries predictions such as Exponential Smoothing [START_REF] Hassine | Popularity prediction in content delivery networks[END_REF], [START_REF] Taylor | Forecasting intraday time series with multiple seasonal cycles using parsimonious seasonal exponential smoothing[END_REF], ARIMA [START_REF] Hassine | Arma based popularity prediction for caching in content delivery networks[END_REF], Prophet [START_REF] Taylor | Forecasting at scale[END_REF], tree-based regressors [START_REF] Breiman | Classification and regression trees[END_REF], [START_REF] Friedman | Stochastic gradient boosting[END_REF], [START_REF] Quinlan | C4. 5: programs for machine learning[END_REF], neural networks [START_REF] Hill | Neural network models for time series forecasts[END_REF], [START_REF] Lipton | A critical review of recurrent neural networks for sequence learning[END_REF], etc. Depending on the characteristics of the time-series at hand, the prediction performances could be totally different. In order to dynamically select the most appropriate prediction methods for different content popularity profiles, we propose hereafter a recommendation system framework composed of the following 3 sets. (1) A collection of time-series type generic data allowing to learn and predict. In our context, it represents YouTube popularity profiles. (2) Different prediction methods having multiple parameters configurations. The details of selected methods are described in Section III-A. [START_REF] Mao | A survey on mobile edge computing: The communication perspective[END_REF] The recommendation framework which requires to randomly run a limited number of predictors on each content popularity profiles. Collaborative Filtering (CF) technique [START_REF] Isinkaye | Recommendation systems: Principles, methods and evaluation[END_REF], [START_REF] Adomavicius | Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions[END_REF], [START_REF] Su | A Survey of Collaborative Filtering Techniques[END_REF] will be used in order to estimate the similarities between the popularity profiles. This technique will be detailed in Sections III-B and IV.

C. Metric definition

Performances of prediction methods are evaluated based on the following error metrics:

• Mean Squared Error: MSE = 1 N N i=1 (y i -p i ) 2 • Mean Absolute Error: M AE = 1 N N i=1 |y i -p i | • Mean Absolute Percentage Error: M AP E = 1 N N i=1 |y i -p i | |y i | + |p i | × 100%
where N is the total predicted samples, and y i and p i denote the real value and the predicted one, respectively. MSE and MAE tend to provide us the accuracy of the output while MAPE, slightly different, provides us the relative value of the error, which indicates the quality of the predictor. For example, with the same MAE = 1, the predictor in y i = 10 has lower quality than the predictor in y i = 100. To evaluate the quality of various predictors for the recommendation systems, we mainly use MAPE through its complements defined by:

• Accuracy: ACC = 100% -M AP E III. PRELIMINARIES A.

Learning and prediction methods

We select for this study two categories of learning and prediction methods: tree-based regressors and RNN-based predictors, which can predict without retraining requirements.

1) Tree-based regressors: Tree-based learning algorithms are one of the most widely used supervised learning methods [START_REF] Breiman | Classification and regression trees[END_REF]. Tree-based learning uses predictive models in a tree form, called a decision tree. Learning a decision tree is essentially the process of splitting a training dataset based on recursive algorithms such as CHAID [START_REF] Kass | An exploratory technique for investigating large quantities of categorical data[END_REF], ID3 [START_REF] Quinlan | Induction of decision trees[END_REF], C4.5 [START_REF] Quinlan | C4. 5: programs for machine learning[END_REF] and CART [START_REF] Breiman | Classification and regression trees[END_REF]. Given a training dataset D = {x (i) ,y (i) } i=1..m , the objective of a prediction problem is to find an estimation (hypothesis) function h that maps an input X to an output Y. Learning a decision tree t is to find h t : X → Y based on one of the above algorithms. Due to their simplicity, decision trees have high execution speed. But they cannot be generalized for an arbitrary problem with possible unseen data. For improving flexibility and diversity, Random Forest [START_REF] Ho | Random decision forests[END_REF] uses multiple decision trees {t},t =1 ,..,τ, called weak learner, constructed based on several randomly selected subspaces S t of the training dataset D, i.e., S t ⊂D. After individually training {h t } t , the output prediction for an input x ′ is computed by a discriminant function given by:

H τ (x ′ )= 1 τ τ t=1 h t (x ′ ).
The discriminant function is essentially an average one and the final predictor generalizes their prediction aggregately.

Boosting algorithms [START_REF] Friedman | Stochastic gradient boosting[END_REF], [START_REF] Freund | A desicion-theoretic generalization of on-line learning and an application to boosting[END_REF] provide us with more complex frameworks for combining weak learners, e.g., decision trees. AdaBoost [START_REF] Freund | A desicion-theoretic generalization of on-line learning and an application to boosting[END_REF] applies a weak learner repeatedly over multiple rounds t =1 ..τ . For each round t, the weak learner h t is trained, predicting error ϵ t is computed and used for updating the weights of inputs for the next round w (i) t+1 and the weight α t of h t . w (i) t is initialized equally but the weights of incorrect learning inputs, i.e., w

(i) t of x (i) with h t x (i) ̸ = y (i)
, get smaller in each round. Thus, the weak learner has to concentrate on the 'hard' inputs in the training set. In contrast, the weight α t of h t increases as predicting error ϵ t decreases. The final output is determined as a weighted voting of weak learners.

Gradient Boosting method [START_REF] Friedman | Greedy function approximation: a gradient boosting machine[END_REF] is a more general boosting approach than AdaBoost. With the same input data set D, the goal is to reconstruct the function H * (x)=argmin H [L(y, H(x))], such that the loss function L is minimized. Gradient Boosting approximates H * (x) by a weighted sum of weak learners h t (x): H(x)= τ t=1 α t h t (x). The steepest-descent algorithm is adapted for finding this approximation.

2) RNN-based predictors: RNN has been adopted in a wide range of machine learning tasks, particularly with timerelated input or output [START_REF] Lipton | A critical review of recurrent neural networks for sequence learning[END_REF]. Owing to the recurrent structures in which hidden states depend on both the current input and the network states at the previous time steps, RNNs are suitable for capturing the relationships among sequential data. In detail, the update of the recurrent hidden state of the simple RNN unit is given by:

h t = g (x t W xh + h t-1 W hh + b h ),
where g is the activation function, x t is the t-th input, W xh is the input weight matrix, and W hh is the recurrent weight matrix, b h is the bias of the simple hidden unit. The simple unit outputs a distribution of the next element of the data sequence which is a composition of its previous state and its current input. Indeed, given a sequence x =( x 1 ,x 2 ,...,x T ), the sequence probability is given by: p(x 1 ,x 2 ,...,x T )=p(x 1 )p(x 2 |x 1 )...p(x T |x 1 ,...,x T -1 ). The simple RNN model provides an output which is equivalent to h t ∼ p(x t |x 1 ,...,x t-1 ). Therefore, this model is able to capture the sequence probability of the input time-series data.

Due to the difficulties of training RNN for capturing long-term dependencies of time series data when gradients tend to vanish or explode, alternative units has been proposed. LSTM [START_REF] Hochreiter | Long short-term memory[END_REF] and GRU [START_REF] Cho | On the properties of neural machine translation: Encoder-decoder approaches[END_REF] are the most popular ones. LSTM unit is formulated by the following equations:

i t = σ (x t W xi + h t-1 W hi + c t-1 W ci + b i ) (1a) f t = σ (x t W xf + h t-1 W hf + c t-1 W cf + b f ) (1b) c t = tanh (x t W xc + h t-1 W hc + b c ) (1c) c t = f t c t-1 + i t c t (1d) o t = σ (x t W xo + h t-1 W hi + c t W ci + b o ) (1e) h t = o t tanh (c t ) (1f)
By introducing input gate i t ,f o r g e tg a t ef t and output gates o t , the memory content c t , the output and current state of LSTM unit h t are controlled. In detail, o t controls the amount of memory content exposure c t at the output h t in (1f). f t and i t decide the updating amount of the new memory content c t and the forgetting amount of the old one c t-1 on the memory content c t in (1d). GRU unit is similar to LSTM unit with the use of gating blocks for adjusting the flows inside the unit but does not include memory cells [START_REF] Chung | Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling[END_REF]. GRU unit is formulated by the following equations. In a GRU unit, the update gate u t adjusts the updates of its content h t via (2d), whereas the reset gate r t ,s i m i l a rt ot h ef o r g e tg a t ei nL S T Mu n i t ,d e c i d e st ow h a t extent the previous state h t-1 should be memorized in (2c).

r t = σ (x t W xr + h t-1 W hr + b r ) (2a) u t = σ (x t W xu + h t-1 W hu + b z ) (2b)
h t = σ (x t W xh + r t h t-1 W hh + b h ) ( 2c
)
h t =(1-u t ) h t-1 + u t h t (2d) B. Recommendation framework
A large number of predictors going along with a huge number of available contents profiles leads to a very large number of combinations. Testing all predictors for each content or determining one predictor suitable for all available contents is impossible. Therefore, the idea is to find a recommendation framework which automatically recommends a suitable predictor for each content, while testing a reduced number of predictors for each content. Accordingly, a recommendation system (RS) has proven to be a valuable technique that enables us to handle a large amount of data for estimating the most appropriate set of suggestions [START_REF] Bigdeli | Comparing accuracy of cosine-based similarity and correlation-based similarity algorithms in tourism recommender systems[END_REF]. Multiple recommendation systems have been proposed including content-based RS, collaborative filtering (CF)-based RS, and hybrid RS [START_REF] Adomavicius | Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions[END_REF]. Among them, CF which enables us to learn from historical interactions of the original user-item recommendation, could be adopted to our content-predictor selection problem.

For recommending appropriate items for a user, the similarity between user u and user v is computed based on Pearson correlation coefficient given by [START_REF] Jannach | Recommender systems: an introduction[END_REF]:

sim (u, v)= n j=1 (r u,j -r u )(r v,j -r v ) n j=1 (r u,j -r u ) 2 n j=1 (r v,j -r v ) 2 ( 3
)
where n is the total number of items, r u,j is the rating of user u on item j, and r u is the mean of the rating of user u.

Various ways for predicting a user's rating can be found in the literature such as the weighted sum of all other users' rating, Top-K recommendations, default voting, inverse user frequency, imputation-boosted CF, and weight majority prediction [START_REF] Su | A Survey of Collaborative Filtering Techniques[END_REF]. In this study, due to the specificity of our recommendation problem, we select the Top-K recommendation approach. The prediction of the rating of user u given to an item i based on the Top-K recommendation is given by:

pred (u, i)=r u + v∈Top-K sim (u, v)(r v,i -r v ) v∈Top-K sim (u, v) (4)
where Top-K is the set of the K most similar users to user u.

For improving recommendation performances, we consider some additional imputation-boosted (IB) approach. The purpose of imputation is used for handling the pervasive problem of missing data [START_REF] Su | A Survey of Collaborative Filtering Techniques[END_REF], [START_REF] Little | Missing-data adjustments in large surveys[END_REF]. Then, we propose an IBCF-based recommending prediction method detailed in the next section.

IV. IBCF-BASED RECOMMENDING PREDICTION METHOD

Typically, for selecting a suitable predictor for a content, one needs to find out the performances of all predictors, each one being a couple of a prediction method and a set of parameters values. To avoid this costly process, we propose to train and measure, for each content, the prediction accuracy for a ratio of randomly chosen predictors during the test period. The accuracies of other predictors are estimated thanks to CF technique. The traditional recommendation method recommends for each user a well suited item. In our context, a user is assimilated to a content and an item to a well suited predictor. The rating that a user sets to an item is equivalent to the prediction accuracy that a predictor can provide to a content.

Table I summarizes our notations.

K do r k,p ← ACC k,p ; if the method is IBCF-RPM then if IsEmpty(ACC k,p ) then r k,p ← min(ACC i,p ∀i =1,...,T)
ACC i,p ← pred(p, i) based on Eq. ( 4)

Result: Complemented ACC matrix [ACC i,p ]
• S 2 : Call Alg. 2 to compute the similarity matrix.

• S 3 : Call Alg. 3 for both CF-RPM and IBCF-RPM to compute a complemented ACC matrix

• S 4 : For a given profile i, recommend the predictor offering the highest value of ACC referred to the complemented ACC matrix.

Due to the sparsity of the ACC matrix given in S 1 , empty ACC data in the set of Top-K similar neighbors can occur. In CF-RPM, the empty data is ignored.

For handling the pervasive problem of missing data, in [START_REF] Su | Imputation-boosted collaborative filtering using machine learning classifiers[END_REF], [START_REF] Xia | Boosting collaborative filtering based on missing data imputation using item's genre information[END_REF], the authors proposed to adopt imputation for collaborative filtering (IBCF). IBCF implements the recommendations from imputed data instead of the original data. The imputed data is obtained thanks to machine learning algorithms [START_REF] Su | Imputation-boosted collaborative filtering using machine learning classifiers[END_REF]. However, unlike rating in user-items problems, the predicted ACC is more sensitive to magnitude. This makes the imputation task more difficult. Furthermore, adopting Pearson correlation coefficient is equivalent to imputing the missing ACC with the mean value of ACCs of each prediction method because of the normalization nature in Eq. 3. Therefore, for imputation-boosted CF in our problem, we propose to perform the imputation in S 4 instead of S 2 .

We propose to impute the missing ACC of a given prediction method by the minimum value of ACCs of that method. The minimum ACC is selected because it represents the worst accuracy that the predictor could provide a content. Then, the proposed IBCF can be considered as a maxmin problem if there are any missing data in the Top-K similar neighbors set. Without imputation, the predicted ACCs for more missing data cases and less missing data cases are treated similarly. The imputation, hence, increases the prediction reliability.

Notice that the processing cost of the imputation is ignorable because it is just an insertion of a minimum value along with computational process.

V. P ERFORMANCE EVALUATION

In this section, we perform the evaluation for individual prediction methods presented in Sect. III-A and the proposed IBCF-RPM described in Sect. IV.

A. Context of simulation

The simulation is based on the YouTube profiles dataset, presented in Sect. II-A. For predicting daily access number of YouTube videos, we use the slicing window prediction style. The number of access to a YouTube video at the consequent day is predicted based on the historical accessing information extracted from a look back window of W preceding days. For example, with W =3 , the predicting output at day t is estimated based on the real data at day t -3, t -2, and t -1. In our simulation, the value of W is selected in the set of {1, 2, 3, 5, 7, 14, 30} day(s).

In this paper, an individual predictor is defined as a couple of a prediction method and an appropriate set of parameters values. Without loss of generality, the parameters values for individual prediction methods are loosely chosen because we aim at the RS which is able to recommend the most appropriate method for a content. The selected parameters given in Table II combined with the 7 above cases of W form 28 tree-based predictors and 84 RNN-based predictors.

B. Performances of individual predictors

We evaluate the performances of the individual predictors on the YouTube dataset presented in Sec. II-A. For each content profile, a predictor is trained on the training period and then tested on the testing period. Afterward, it is validated on the validation period.

History and configurations. Figure 2 shows the average MSE of predictors on the testing period. Because of lack of space, the average MAE could not be displayed, but we observed that the trends of MAE and MSE are consistent. With the same W , LSTM-based and GRU-based predictors have lower errors than the tree-based predictors. Within the Fig. 2: MSE-Average testing errors of individual predictors tree-based predictors, the Random Forest one is the best. Within the RNN-based predictors, the configuration C3 with only one layer is better than the configuration C6 with two unit layers. With the same configuration and W , the predicting errors of LSTM-based predictors and GRU-based predictors are almost similar. Decision Tree and Random Forest predictors, which are simpler, have errors increasing with W . AdaBoost and Gradient Boosting show little differences in error when W =1..7 and the error for W = 30 is smaller than the error for W = 14. For RNN-based predictors, the error for W = 30 is the smallest one.

ACC on training, testing and validation period. Table III shows that the average ACC of the individual predictors on the training set is higher than those on testing and validation sets. For tree-based predictors, the highest ACC corresponds to W = 30 in the training set, and to W =1in both testing and validation sets, which is obvious. For RNN-based predictors, the best ACC is obtained when W =7in the validation set, mostly with W =1and W =7in testing set, and with W =7and W = 30 in training set. In summary, the ACC of a predictor changes with the set of data. Through training, a predictor can achieve high ACC but in out-ofsample predicting (i.e., predicting in testing and training sets), the ACC is generally lower.

C. Performances of CF-RPM and IBCF-RPM

The performances of CF-RPM and the proposed IBCF-RPM are compared with the best individual predictors during testing. After the training phase, all individual predictors were tested, a full information prediction accuracy matrix was created. Based on the best value of ACC in this matrix, the set of the best ACC predictors for each profile was selected. In Fig. 3 and4, the "Best recommend (Full information)" curve represents the best ACC obtained by the optimal choice of individual predictors after testing phase.

The graphics in Fig. 3 shows the ACC obtained by different predictors during testing and validating phase. We can see in the first graphic of 3 that the proposed IBCF-RPM achieves almost the best value of ACC, whereas the CF-RPM can only achieve around the ACC of the best individual predictor. The highest testing ACCs of individual predictors in each prediction type are shown for comparison. For example, the GRU-based predictor has the highest testing ACC in configuration C2 for W =1 . This value can be also found in Table III. The trend of prediction accuracy of the proposed IBCF-RPM is almost stable for K>6. For K =1 , the ACCs of IBCF-RPM and CF-RPM are similar because the imputation is not effective in this case. The ACC obtained during validation phase is also depicted in the second graphic in Fig. 3. It should be noted that for these graphics, we selected only 30% of the defined predictors to compute their accuracy. The accuracy of the remaining 70% is estimated and the well-suited predictor is selected based on estimated or real accuracy of the Top-K most similar contents. The depicted curves of the second graphics of 3 indicate the effectiveness of the predictors selection during validation phase, based on the prediction performances during the testing phase. Above all, our proposed method achieves almost an ACC as high as that of the best recommendation with full information.

Figure 4 confirms the efficiency of the proposed method. The selection ratio represents the percent of randomly selected predictors to be executed for a given content, the accuracy of the remaining predictors being estimated based on the accuracy of similar contents. For all of the investigated values of implementing ratio, ranging from 0.1 to 0.75, IBCF-RPM Fig. 4: Accuracy vs. running ratio with Top-K=10 Fig. 5: Simulation time vs. running ratio always approaches the value of the best recommendation with full information, when the implementing ratio equals to 1. In addition, IBCF-RPM is better than CF-RPM and any individual predictors in both testing and validating set.

Figure 5 shows the time efficiency of the proposed IBCF-RPM. Compared to the best recommendation, where training and testing are required for all predictors, our method has very close performances while running only a small ratio of available predictors.

VI. CONCLUSION

In this paper, we have addressed the problem of predicting content popularity based on the study of YouTube contents. We have proposed a recommendation framework which allows recommending appropriate predictors for each content solicitation profile. The framework is based on the proposed IBCF technique. Our method, IBCF-RPM, is applicable to a large set of contents and prediction methods. We considered treebased regressors (Decision Tree, Random Forest, Gradient Boosting and AdaBoost) and Recurrent Neural Network-based prediction methods (LSTM-based, GRU-based) as individual predictors. IBCF-RPM recommends a well-suited predictor for a given content while testing the accuracy of a small ratio of the available predictors, randomly chosen. The method is based on the evaluation of the similarities between contents to estimate the accuracy of untested methods. For a running ratio of 30%, based on similarities with a quite low number (Top-K=6) of other contents, IBCF-RPM gives performances close to the full implementation case.

For future work, the effectiveness of the popularity prediction for proactive caching in MEC should be further investigated and evaluated.

Fig. 1 :

 1 Fig. 1: Content profile splitting

Fig. 3 :

 3 Fig. 3: Accuracy vs. Top-K with running ratio=0.3

TABLE I

 I Call Alg. 1 to compute the ACC matrix.

					Algorithm 1: ACC matrix computation
					Data: Pr, X (i) , D for each profile: i =1..T do (i) l , D (i) t , ratio
					P i ← SelectRandomSubset(Pr,ratio) ;
					for p in P i do
					Training p with D Compute ACC i,p of the trained p with D (i) ; l	(i) t	;
					Result: Sparse ACC matrix with size T × N
					Algorithm 2: Similarity matrix computation
					Data: Sparse ACC matrix [ACC i,p ]
					for u =1,...,T do
					for v =1,...,T do
					sim(u, v) based on Eq. (3)
				: Notations
	N		number of available predictors
	Pn	a predictor n =1..N
	Pr	predictors set {P n=1..N }
	T		number of content profiles
	x	(i) j	solicitation number for the content i during day j
	X (i)	profile of the content i : x	(i) j=1..m between days 1 and m
	D	(i) l	training data for content i: x	(i) j=1,...,l
	D	(i) t	testing data for content i: x	(i) j=l,...,m
	ratio	ratio of randomly selected predictors to run on a profile
	ACC i,p	accuracy value of the predictor p for the content i
	ACC	accuracy matrix of size T xN
				In Eq.(3) and Eq. (4) changes in signification
	u, v	any two content profiles
	sim(u, v)	similarity value between contents u and v
	r u,i	accuracy of the predictor i for the content u : ACC u,i
	ru	mean accuracy for the content u
	A conventional CF-based Recommendation Prediction
	Method (CF-RPM) and the proposed Imputation-Boosted CF-
	based Recommendation Prediction Method (IBCF-RPM) in-
	cludes the following steps:

• S 1 : Result: Similarity matrix [simu,v] with size T × T Algorithm 3: CF-RPM and IBCF-RPM Data: Sparse ACC matrix, similarity matrix [sim i,i ] for i 1 =1,...,T do Top (i) K ← The K most similar items from [sim i,i ]; for p =1,...,N do if IsEmpty(ACC i,p ) then for k in Top (i)

TABLE II :

 II Predictor parameters

		RNN-based predictors	Tree-based regressors
			No. of Unit		Max.	No. of
	Config. Layer 1 Layer 2		depth	Estimators
	C1	1	0
	C2	7	0	Decision Tree	4	1
	C3	14	0	AdaBoost	4	20
	C4	1	2	GradientBoosting 4	300
	C5	7	14	Random Forest	4	300
	C6	14	28

TABLE III :

 III Average ACC of individual predictors

				Training					Testing					Validation
		W=1	W=3	W=7	W=14	W=30	W=1	W=3	W=7	W=14	W=30	W=1	W=3	W=7	W=14	W=30
	DecisionTree	94.97	95.24	95.70	95.87	96.06	88.96	87.92 87.54	86.42	84.64	87.48	86.68	86.46	84.97	81.60
	AdaBoost	94.38	95.15	96.07	96.45	96.89	88.06	87.63 87.43	86.34	85.28	86.60	86.37	86.29	84.68	82.43
	Grad.Boosting	95.61	96.36	97.07	97.45	97.82	88.47	87.83 87.11	85.82	84.37	86.94	86.41	85.82	83.76	81.28
	RandomForest	95.04	95.38	95.93	96.12	96.29	89.35	88.61 88.03	86.91	85.42	87.76	87.26	86.86	85.35	82.55
	LSTM C1	90.59	92.40	93.37	93.08	93.42	84.84	87.43	88.82 87.87	88.06	83.65	87.05	89.07	87.65	87.27
	LSTM C2	93.47	93.52	93.96	93.94	94.04	90.22	90.06 89.99	89.80	89.31	89.09	89.30	90.41	90.01	88.51
	LSTM C3	93.30	93.64 94.05	93.89	93.90	90.03	90.28 90.07 89.57	89.52	88.89	89.60	90.61	90.10	88.76
	LSTM C4	91.66	92.41 92.91	92.60	92.86	85.76	87.46	87.69 86.79	85.84	84.18	86.75	87.61	86.24	84.30
	LSTM C5	93.13	93.03	93.67	93.57	93.85	89.66	89.32 89.49	88.82	88.71	88.59	88.86	89.50	88.81	87.84
	LSTM C6	93.05	92.88 93.66	93.08	93.56	89.58	88.93 89.10	88.71	88.84	88.76	88.70	89.43	89.21	87.75
	GRU C1	91.63	92.69	93.22	92.96	93.36	86.20	88.36	88.93 88.06	88.35	84.95	87.97	89.17	88.00	87.68
	GRU C2	93.51	93.61	94.00	93.66	94.08	90.55	90.31	90.19	89.45	89.50	89.61	89.77	90.90	90.07	88.86
	GRU C3	93.30	93.53 93.95	93.42	93.76	89.98	90.10	90.35 89.12	89.34	88.99	89.67	90.91	89.84	88.50
	GRU C4	92.95	92.69 93.15	92.91	93.03	88.81	88.42 88.75	87.87	87.71	87.86	88.01	89.24	88.29	86.87
	GRU C5	93.17	93.21	93.44	93.16	93.52	90.28	89.92 89.45	88.65	89.15	89.23	89.56	89.94	88.93	88.40
	C6	92.74	92.94	93.12	93.10	93.31	89.53	89.31 89.27	88.63	89.21	88.59	88.92	89.78	88.68	88.38

ACKNOWLEDGEMENT

The research leading to these results has received funding from DigiCosme.