Pierre Bottero

Sana Ben Jemaa

Thibaut Cuvelier

Pbch

-Organizing Network (SON) D2D Device-to-Device D/A Digital/Analogue DCI Downlink Control Information x DCS Dynamic Cell Selection

Keywords: ABS Almost Blank Subframe AI Artificial Intelligence ANBR Automatic Neighbor Beam Relation ANR Automatic Neighbor Relation API Application Programming Interface CET Cell-Edge Throughput CoMP Coordinated Multipoint transmission CSI Channel Physical Broadcast Channel PDCCH Physical Downlink Control Channel PDSCH Physical Downlink Shared Channel p, d, f probability density function SRS Sounding Reference Signal SS Synchronization Signal SSB Synchronization Signal Block SSS Secondary Synchronization Signal (SS)

Mobile operators are continuously challenged to offer faster and better connectivity to their customers. The mobile networks are expected to provide enhanced capacities while dealing with a new variety of services. 5G brings a wide range of novelty to increase the quality and services offered by the operators. These novelties include, but are not limited to, new antenna technologies, the apparition of Multi-User scheduling, new architectures, but also the inclusion of more artificial intelligence in the network decisions. The changes brought by 5G vastly increases the complexity of the networks. 5G comes with a wide range of new parameter to be set in an optimized manner. The demand towards the operators are higher than ever whereas the network is more complex. In 4G, the number of parameters to be chosen was already high and complex. To optimize the network and enable the users to get the best quality of experience, Self-Organizing Network (SON) had been introduced in 4G network. SON allows the network to enable self-optimization, self-configuration and self-healing. The SON has to evolve to deal with the new challenges of the network. Massive-MIMO (M-MIMO) and beamforming present the opportunity to focus signal on users, and thus increasing throughput. However, focused signal come with new complexity and new challenges to overcome. We consider in this thesis the self-optimization of network relying on M-MIMO with Multi-User (MU)-schedulers.

First, we consider a heterogeneous network. Densifying a network by adding small-cells increases quickly the capacity of the network. M-MIMO and MU-scheduler render the 4G SON solutions out-of-date and present a new challenge to overcome. In this thesis, we propose a MU-collaborative scheduler for 5G M-MIMO cell that takes into account the impact of the macro scheduling on the small-cells located in the macro cell's area. We also assess the performance of the scheduler through numerical experiments.

Interferences between neighboring M-MIMO cells are also an important topic to be addressed in 5G network. In this work, we propose three different approaches to this problem. The Automatic Neighbour Beam Relation (ANBR) solution is a heuristic solution offering a low complexity and a reactive solution to deal with neighboring cells. The thesis then introduces a Multi-armed-Bandits (MAB) solution to enhance the previous heuristic to deal with neighboring interference in a 5G M-MIMO network with online learning. Finally, a third approach based on the exploitation of geolocalized data to take advantage of information at a finer granularity is explored. v

Résumé

Les opérateurs mobiles sont continuellement tenus d'offrir une meilleure connectivité et une plus grande rapidité à leurs clients. Des capacités améliorées sont exigées des réseaux mobiles tout en devant gérer une nouvelle variété de services. La 5G apporte une vaste gamme de nouveautés pour améliorer la qualité et les services offerts par les opérateurs. Ces changements contiennent, de manière non-exhaustive, de nouvelles technologies pour les antennes, l'apparition de la planification multi-utilisateur, de nouvelles architectures mais aussi l'inclusion de plus d'intelligence artificielle dans la prise de décision. Les changements amenés par la 5G sont accompagnés d'une complexité accrue des réseaux d'internet mobile. La 5G s'accompagne d'une grande variété de nouveaux paramètres à ajuster de manière optimisée. Les exigences envers les opérateurs sont de plus en plus élevées alors que la gestion des réseaux se complexifie. Le nombre de paramètres à régler était déjà très élevé en 4G. Afin optimiser les réseaux 4G et de pouvoir offrir la meilleure qualité d'expérience possible aux utilisateurs, les réseaux autoorganisants, self-organizing network (SON) en anglais avait alors été introduits. Un réseau auto-organisants a pour but l'auto-configuration, l'autooptimisation et l'auto-réparation. Cependant, la 5G demande au SON de s'adapter et d'évoluer avec les réseaux. Par exemple, le développement du M-MIMO et de la focalisation de faisceaux présentent la possibilité de focaliser le signal sur les utilisateurs afin d'améliorer le signal reçus par les utilisateurs. Cependant, cela s'accompagne de nouvelles difficultés et d'une nouvelle complexité. Dans cette thèse, nous nous concentrons sur les mécanismes d'auto-optimisation dans un ordonnanceur multi-utilisateurs (MUscheduler) dans un déploiment de mimo massif (M-MIMO).

Dans un premier temps, nous prenons en considération un réseau hétérogène. La densification d'un réseau par l'ajout de small cells est un moyen d'augmenter rapidement la capacité d'un réseau. L'utilisation du M-MIMO et d'un scheduler multi-utilisateurs rend certaines solutions SON 4G obsolètes et sont un défi intéressant à relever. Dans cette thèse, nous présentons un contrôleur multi-utilisateurs et collaboratif pour une cellule hétérogène 5G avec un déploiement M-MIMO. Ce contrôleur prend en compte l'impact des émissions d'une cellule macro sur les small cells présentes dans la couverture de la cellule macro. Des simulations nous permettent d'évaluer la performance de notre solution.

vi Les interférences entre cellules M-MIMO voisins sont également une problématique importante des réseaux 5G qu'il nous faut adresser. Dans ce travail, nous abordons cette problématique de trois manière différentes. Nous présentons une solution heuristique qui présente avec sa faible complexité une solution réactive pour gérer des cellules voisines. La thèse propose également une solution basée sur l'emploi de bandits à bras multiples (MAB en anglais). Cette solution permet d'améliorer la gestion des interférences, par rapport notamment à l'heuristique précédente, entre cellules voisines dans un réseau 5G avec un déploiement M-MIMO. Enfin, une troisième méthode est étudiée. Cette approche est basée sur l'exploitation de données géolocalisées et permet de Chapter 1

Introduction

This chapter introduces the subject of the thesis. We first contextualize the work by describing the evolutions brought by the 5 th Generation (5G), explain the Self-Organizing Network (SON) and its usage and the evolution of SON for 5G. Then, we describe the objectives of the thesis before announcing its content and contributions.

Context of the work 1.Fifth generation of mobile network

The number of devices connected to the mobile network is increasing exponentially while demanding increased data rates and connectivity. New usages of the network are also requiring reduced latency, high reliability and massive connectivity [START_REF] Popovski | 5G wireless network slicing for eMBB, URLLC, and mMTC: A communication-theoretic view[END_REF]. To meet these demands and offer the best Quality of Experience (QoE) and Quality of Service (QoS) possible to the users, the network needs to evolve, in the framework of the fifth generation of mobile network (5G). Such evolution can be divided along three main axes:

New services New types of services are emerging and require from the mobile network to satisfy the demand for three main kinds of usages.

• URLLC : Ultra-Reliable Low-Latency Communications (URLLC) answers the needs of latency-sensitive devices [START_REF] Li | 5G URLLC: Design challenges and system concepts[END_REF][START_REF] Mahmood | Over-the-air time synchronization for urllc: Requirements, challenges and possible enablers[END_REF]. This includes usages such as remote surgery, factory automation or autonomous car. Let us take the latter as an example. An autonomous car requires information from the others vehicles (Vehicle-to-Vehicle (V2V) communication) but also from the roadside infrastructures, to be able to synchronize with traffic lights for instance. These communications demand low-latency, with delays around 1 msec/packet, while simultaneously requesting a high reliability, i.e. 99.9999% of success.

• mMTC : massive Machine Type Communications (mMTC) requires a high connectivity capacity [START_REF] Bockelmann | Towards massive connectivity support for scalable mMTC communications in 5G networks[END_REF]. The expansion of Internet of Things (IoT) results in a high density of connection requiring long range and low • eMBB : enhanced Mobile Broadband (eMBB) takes care of users needing high rates in a wide area [START_REF] Demmer | Analytical study of 5G NR eMBB co-existence[END_REF]. This service is the continuity of 4G and targets for instance streaming usages. It offers high data rate and a wide coverage area. 5G provides the user with faster downloads, better mobility and coverage.

The characteristics of these three services are summoned up in Figure 1.1.

New technologies 5G resorts to several new kinds of new technologies to fulfill the expectations of the services described earlier.

Here is an exhaustive list.

• Communication types: Device-to-Device (D2D), Machine-to-Machine (M2M) and Vehicle-to-Everything (V2X) communications are enabled by 5G [START_REF] Tuan | Survey of promising technologies for 5G networks[END_REF]. V2X enables vehicles to communicate with other vehicles and roadside infrastructures, which is key to the development of autonomous cars. D2D describes the cases where devices are directly interacting with each others without the support of a mobile network. Use cases of this type of communication entail for instance multi-casting scenarii, where using D2D communication can alleviate the load of the network. M2M enables machines to communicate without human intervention, for instance in the context of industrial instrumentation or smart city.

• M-MIMO: Massive Multiple Input Multiple Output (M-MIMO) is an antenna technology exploiting multiples antennas to transmit and receive the data [START_REF] Tse | Fundamentals of wireless communication[END_REF]. In addition to beamforming capability, the data is sent through multiple independent data streams, thus enhancing the data rates. M-MIMO enables space multiplexing and increase the EE of the transmission while decreasing the interferences on other users. Simultaneous multi-path transmission also increases significantly the reliability of the transmission [START_REF] Erik G Larsson | Massive MIMO for next generation wireless systems[END_REF]. M-MIMO provides beamforming capability allowing to focus the electromagnetic signal on the user, thus achieving better signal quality (See Section 1.1.2).

M-MIMO with beamforming also allows Multi-User (MU)-Scheduling. MU-scheduling allows several users to be served simultaneously through different beams. The capacity of the antennas usually allows to serve simultaneously more User Equipments (UEs) that the number actually scheduled simultaneously, which is limited by the signal processing.

• Frequencies: The spectrum of available frequencies will increase with the addition of the millimetre waves. These frequencies are higher than the regular network frequencies. Transmissions over higher frequencies achieve better throughputs at the cost of lower coverage [START_REF] Wang | Millimeter wave communication: A comprehensive survey[END_REF]. Smaller wavelengths are more sensitive to obstacles, for instance trees or buildings. They are mainly planned for densely packed networks.

Virtualization

5G networks rely on a virtualized architecture. This implies that the architecture of the network can be configured independently from the hardware. Virtualization procures more flexibility and dynamicity.

The alliance Open-Radio Access Network (RAN) (O-RAN) [START_REF] Oran Alliance | Towards an Open and Smart RAN[END_REF] standardizes a virtualized architecture that offers openness, namely multi-vendors interoperability, and the support of intelligent embodiment. This architecture entails standardized interferences and Application Programming Interfaces (APIs) and two RAN Intelligent Controllers (RICs):

• The Non-Real-Time RIC (Non-RT-RIC) that is responsible for the guidance through policies and training of Artificial Intelligence (AI)/Machine Learning (ML) models for the Near-Real-Time RIC (Near-RT-RIC). It takes decision within a time scale of 1 second and above. Its role is to come up with parameters, policies and Artificial Intelligence (AI)/Machine Learning (ML) models for the Near-RT-RIC through the A1 interface (See Figure 1.2).The control, optimization and AI/ML applications operating in the Non-RT-RIC are called rApps. Virtualization also plays an important role in the usage of new services. The descriptions given earlier of URLLC, mMTC and eMBB show that each category of services have different requirements. To provide each with their specific needs, virtualization allows to offer end-to-end logical networks called slices. A slice is a set of network functions composing a complete logical network [START_REF] Foukas | Network slicing in 5G: Survey and challenges[END_REF][START_REF] Zhang | An overview of network slicing for 5G[END_REF]. Slicing enables to virtually slice up the network resources to provide the best service to every user based on their peculiar need. The use of slicing requires slicing management and add complexity to the network. Slicing management involves the creation or deletions of slices and their reconfiguration. The needs of a specific slices are provided through the Service Level Agreement (SLA).

The virtualization of the network also eases the resort to AI/ML optimization techniques. Such techniques will enhance the optimization of the management of the network including mobility, resource and load management.

Beamforming techniques

Precoding

Precoding is a transmitter signal processing technique. Antenna arrays permit to perform two types of precoding, that can be performed simultaneously by the Base Station (BS):

• Multiple Input Multiple Output (MIMO) precoding permits to ensure high performance when the BSperforms spatial multiplexing (i.e. simultaneous transmission of multiple independent streams of data). Indeed, MIMO precoding permits to reinforce the independence between the channel responses associated to the different BS-UE antenna pairs, thus reducing interference.

• Beamforming precoding permits to steer beams in UEs' directions and to separate them spatially, thus enhancing their Signal to Interference plus Noise Ratio (SINR).

Different beamforming techniques will be presented in the sequel [START_REF] Sohrabi | Hybrid digital and analog beamforming design for large-scale antenna arrays[END_REF]:

• Analogue beamforming is applied mainly to Frequency Range (FR)2, namely 24.25 GHz up to 52.6 GHz. The beamforming weights (i.e. phase shifts) are applied in the analogue domain, once the signals have been mixed to Radio Frequency (RF). It requires less RF chains and limits power consumption.

• Digital beamforming, on the other hand, are used in FR1 (410-7125 MHz) and precoding is digitally applied to the baseband, meaning that phase shifts are applied then Digital/Analogue (D/A) conversion and finally RF. The weights are applied through parallel RF chains. The main drawback of digital beamforming is that the number of controllable elements is high and each RF chain is expensive and triggers a high power consumption. Digital beamforming offers more flexibility than its analogue counterpart and the possibility to use spatial multiplexing. Spatial multiplexing enables to use the same resource block for different UE as long as there are spatially separated.

• Hybrid beamforming involves both digital and analogue beamforming and is mainly used in mmWave domain. This technique exploits a digital and an analogue precoder simultaneously, enabling to limits the RF chains required while still achieved substantial gains.

Beamforming techniques are mainly divided into two categories [START_REF] Gpp | Physical channels and modulation[END_REF][START_REF] Gpp | Multiplexing and channel coding[END_REF]:

• Eigen based beamforming can be used for Time Division Duplex (TDD) only and takes advantage of channel reciprocity in combination with Up Link (UL) and Sounding Reference Signal (SRS) measurement. If the UE moves in the cell, the beam will follow the UE through dynamically changing the weights. As a drawback, these techniques trigger significant quantities of baseband processing. In this family of technique, Maximum Ratio Transmission (MRT) aims at focusing as much as possible of the signal on the UE, independently of the impact on others UE [START_REF] Titus | Maximum ratio transmission[END_REF]. ZeroForcing (ZF) on the other hand tries to prevent high interference on the neighboring UEs but at the cost of a lower signal for the served UE [START_REF] Yoo | On the optimality of multiantenna broadcast scheduling using zero-forcing beamforming[END_REF]. Minimum Mean Square Error (MMSE) provides a trade-off between MRT and ZF.

• Grid of Beams (GoB) techniques rely on a codebook defining a fixed set of beams with overlapping that span the coverage area of the cellspecified is suitable for both Frequency Division Duplex (FDD) and TDD. There are two possibilities. The first one relies on beamformed Synchronization Signal (SS)/Physical Broadcast Channel (PBCH) blocks or Channel State Information (CSI) reference signals that allow the UE to select the best beam of the GoB and to report it to the BS. In the second technique, the BS broadcasts non-precoded CSI references signals.

The UE estimates the best precoding matrix, and sends back the corresponding Precoding Matrix Indicator (PMI) of a codebook to the BS [START_REF] Gpp | Physical layer procedures for data[END_REF]. In both cases, a fixed set of beams, and thus a fixed set of beamforming weights, is used which results in less processing than eigen based methods.

Theoretically eigen beamforming based M-MIMO offers better performance than that of GoB based M-MIMO. In practice however, large M-MIMO systems present complex implementation issues. For example, in functional split where precoding is performed at the distributed unit (e.g. in the edge cloud) massive amount data should be sent over the fronthaul. This problem comes in addition to the huge signaling overhead required to estimate the channel matrix. As a result, different vendors have deters their industrial choice in favor of GoB techniques.

Beam management

Beam management aims at identifying and then maintaining the optimal beam for transmission, both in UL and in Down Link (DL). UE are more likely to support beamforming in the FR2 than in the FR1, while the BS supports beamforming in both ranges [START_REF] Gpp | Study on new radio access technology Physical layer aspects[END_REF].

Attachment to beams is made through two processes: Initial acquisition and beam refinement. The first step links a UE to the optimal wide beam and refinement allows a narrower and more directional beam to increase the signal strength. There is a trade-off between the width of the beam and the In digital beamforming, all the beams can be active at once, but each beam has a distinct SS/PBCH which is emitted at a distinct timing. This enables the UE to differentiate the beams from each other. In analogue beamforming,

Beam refinement

We consider DL beam refinement; similar procedures can be found in literature for UL refinement. After initial access procedure, the BS can opt for beam refinement, meaning attaching the UE to a narrower beam [START_REF] Gpp | User Equipment (UE) radio access capabilities[END_REF]. This beam is more directional and offer higher gains and improved link budget while potentially triggering more beam switching. For a given beam with associated SS/PBCH block, up to 4 finer beams can be associated, with 4 distinct CSI reference signals. The CSI reference signals are time multiplexed across 4 symbols while the associated SS/PBCH block is frequency multiplexed. The refined beams can either used for both Physical Downlink Control Channel (PDCCH) and Physical Downlink Shared Channel (PDSCH) or only for PDSCH. In the latter case, the wider beam is used for PDCCH, allowing UE to profit both from a better signal for data but avoiding mobility issues for control.

Resource Allocation in 5G

Orthogonal Frequency-Division Multiple Access (OFDMA) system OFDMA is the modulation format used in 5G that uses an array of subcarriers to transmit information over a range of frequencies [START_REF] Abdelbaset | A survey on inter-cell interference coordination techniques in OFDMA-based cellular networks[END_REF]. To avoid inter-channels interference, the subcarriers are orthogonal. Time is divided into symbols and several time symbols form a slot. A Resource Block (RB) is a time slot of a given subcarrier and is the smallest element of resource that can be allocated to a user. When resource is allocated, RBs are dedicated to one user in the case of Single-User (SU) scheduling or to several users in the case of MU scheduling, as shown in Figure 1.5. A set of contiguous time slots form a 5G radio frame, which lasts 10 ms and is indexed with the System [START_REF] Gpp | Physical layer procedures for control[END_REF][START_REF] Gpp | Multiplexing and channel coding[END_REF][START_REF] Gpp | Radio Resource Control (RRC)[END_REF]. The SFN goes from 0 to 1023 (10bits) and a full SFN cycle lasts 10.24sec. The UE acquires the SFN cycle within the initial access procedure. A 5G subframe lasts 1 ms. In FDD, the bandwidth is divided between UL and DL. In TDD, a pattern is repeatedly used. First a fixed number of UL slots, then a switching period and then a fixed number of DL slots. The switching period is made up of a flexible number of DL then UL symbols separated by a guard period. The guard period consists in symbols with no transmission aiming at avoiding intersymbol interference and is usually longer than the maximum propagation delay. Note that there is no guard period after DL symbols. We consider Dynamic Grand allocation in which each PDSCH transmission is always preceded by a PDCCH transmission with Downlink Control Information (DCI), which specifies the allocated resources.

Frequency resource allocation

DCI format specifies the type of DL resource allocation scheme to be used. DCI formats 1_0 and 1_1 are used to allocate PDSCH frequency domain resources. Format 1_0 means that type 1 is always used, while format 1_1 means that either type 1, type 0 or a dynamic combination of both can be used (in this case the BS indicates the type used in the Most Significant Bit (MSB) of the 'Frequency Domain Resource Assignment' field of the DCI [START_REF] Gpp | Multiplexing and channel coding[END_REF][START_REF] Gpp | Physical layer procedures for control[END_REF][START_REF] Gpp | Radio Resource Control (RRC)[END_REF]. Resources are allotted in terms of Resource Block Groups (RBGs) and not as individual resource block. RBGs consist in sets of contiguous Virtual Resource Block (VRB) within a given bandwidth part. The size of the RBG varies depending on the part of the bandwidth used. Type 0 uses non-interleaved VRB to Physical Resource Block (PRB) mapping, meaning that the VRB i is directly mapped to the PRB i. For a given bandwidth part size, two configurations of RBG sizes are available and given to the user through a parameter in PDSCH-config. The UE receives the allocated RBG through a bitmap. For instance, in Figure 1.6 (will do it soon after), the number of resource blocks is 30 and the RBG size is 4. The DL bandwidth part hence consists in 8 RBGs, with the last one containing only 2 blocks. The bitmap indicates by a bit for each RBG if the RBGs are allotted to the UE (bit 1) or not (bit 0). The RBGs do not have to be contiguous, as shown in the figure. In this allocation scheme, the VRB allocated are always contiguous and are indicated through a Resource Indication Value (RIV). This RIV is a code that is mapped to a combination of the starting allocated resource block and the number of allocated resource blocks. For instance, if the UE receives the RIV that is mapped to the starting block 3 and to the length 10, the UE knows that it has been allocated the VRBs 3 to 12. Then the VRBs can either be inter-leaved or non-inter-leaved to PRBs, as indicated by a flag.

Context of the work

Time resource allocation

The Time Domain resource assignment is provided by the user in the 'Time Domain Resource Assignment' field, which contains a pointer. The pointer corresponds to a given row of a look-up table that specifies the slot offset, the starting symbol and the length of the allocation. The look-up table is either a standardized table or a Radio Resource Control (RRC) configured one. Regarding the SIB1 transmission, standardized table is always used so that UE does not need further knowledge from the BS. If a RRC configured table is used, then it is provided to the UE within SIB1 [START_REF] Gpp | Physical layer procedures for data[END_REF][START_REF] Gpp | Multiplexing and channel coding[END_REF][START_REF] Gpp | Radio Resource Control (RRC)[END_REF].

Self-organizing network purpose

Evolution defined by 5G networks increase substantially the complexity of the network while the expectations of the users are heightened. To efficiently manage the network, the SON has been introduced in 4G mobile networks. The SON automatizes repetitive tasks in the network around three main topics:

Self-configuration

The SON enables a network to self-configure itself to facilitate its deployment. This leads to an automatic integration of a new nodes in the network. The configuration of parameters as well as the detection by the neighbors and the addition to the network are self-configured, reducing considerably the OPerationnal EXpenditure (OPEX).

Self-healing Networks can encounter performance degradation and failures. Such failures lead to inoperative nodes that can be difficult to detect. Self-reparation allows the network to find the cause of the failure in a timely manner to limit its impact. The SON can also resolve the issue to some extent. Whether it signals the cause or repairs it, self-reparation accounts for a minimized impact of network failures on to the users.

Self-optimization Self-optimization optimizes certain Key Performance Indicators (KPIs), and in general the network performance, by adjusting system and Radio Resource Management (RRM) parameters, such as antenna's tilt and azimuth [START_REF] Gpp | Evolved Universal Terrestrial Radio Access Network (E-UTRAN); Self-configuring and self-optimizing network (SON) use cases and solutions[END_REF]. The configuration of these parameters impact the network and the service offered to the users and the SON aims at finding the optimal configuration by relying on algorithms and on measurements performed by the UE or the base station. Depending on the characteristics of the network, i.e. the distribution of traffic, the mobility or the available bandwidth, for instance, the optimal configuration may change. SON functionalities for selfoptimization entail the management of resource allocation, interference, load balancing, coverage, mobility and many others.

Self-organizing functionalities for 5G

SON has been introduced for 4G. Nonetheless, the SON has to adapt to the evolutions and technologies brought by 5G. The introduction of M-MIMO and more particularly beamforming modify the granularity at which the SON can optimize the network. SON in 5G can allow managing resources at beam resolution. MU-scheduler, while allowing more users to be served at once, are also more complex than single-user to configure and optimize. The virtualization of the network brings new challenges as well as new opportunities.Therefore, the adaptation of the SON to match the new 5G network remains to be defined. This thesis questions and designs SON functionalities in the context of 5G M-MIMO network for eMBB regarding self-optimization. Self-optimizing SON for the RAN covers multiple aspects such as:

• Selecting the best connectivity mode • Selecting the best frequency band • Implementing AI/ML models • Managing slicing • Offering optimization solution for mobility, resource allocation, neighbor relationship, load balancing.

• Enhancing the EE of the network

The challenge faced by the 5G SON is to tackle the aspects above while dealing with a network more complex than ever.

Objectives and contributions of the thesis

The objectives of this thesis are to identify important use cases in the new 5G framework and to offer solutions to these problems, with either a heuristic or a ML tool. The work includes the modelling of a 5G RAN with M-MIMO through channel, traffic and capacity models to assess the performance of the solutions. This assessment will be achieved through simulations requiring a 5G M-MIMO simulator. The 5G M-MIMO simulator abstracts certain lower layer functionalities, while taking into account M-MIMO and Media Access Control (MAC) layer scheduling.

In the thesis, we consider only DL transmission. After IA and beam refinement, the macro cells use 16 narrow beams. The users are assumed to be attached to their best serving beams. The antennas use digital beamforming, as MU-MIMO is used. Chapter 3 relies on TDD and more specifically MRT while the rest of the chapters can be applied to both TDD and FDD and use the GoB both for control and data transmission. We assume that up to 4 users can be served simultaneously and that 1 user per beam can be served at once only. The bandwidth used is 20Mhz wide and all the frequency band can be re-used for each mobile served by the MU-MIMO. Furthermore, O-RAN deployment is foreseen in the coming years and the SON evolution can be mapped to the O-RAN framework, namely to the Non-RT-RIC and the Near-RT-RIC. A particular effort is made in developing low complexity solutions with negligible impact on latency in operational 5G network. The contributions of the thesis can be summed up as follows.

Chapter 2 introduces the technical and mathematical background required to fully understand the work presented in the thesis.

The thesis presents interference management use cases in different contexts.

• Interference management in heterogeneous networks

The first scenario, presented in Chapter 3, deals with interference issues encountered in an heterogeneous network. In this context, small cells have been deployed in a M-MIMO macro cell to increase the cell capacity through spatial densification. The interference produced by the macro cell on the small cells however limits the performance of the heterogeneous network. A collaborative MU-scheduler enables to optimize the Mean User Throughput (MUT) of the heterogeneous network. The macro/small cells coordination relies on the exchange of a loss factor that each small cell provides to the macro cell, that estimates the performance utility loss due to the macro cell scheduling. With this information, the collaborative MU scheduler can balance between limiting the loss of the small cells and ensuring the gains of its own users. The enhancement provided by the collaborative MU-scheduler are significant while requiring a low computational complexity and a minimum amount of exchanged information.

• Inter-cell interference management

The thesis also deals with interference between macro cells in a context of 5G M-MIMO with GoB deployment. While M-MIMO increases the throughput of the users, it can also increase the interference at the edges of the cells. To tackle inter-cell interference, we pursue three different approaches.

Chapter 1. Introduction

-Heuristic solution

The first approach benefits from the possibility in 5G to establish interference relations between beams of neighboring cells. The concept of Automatic Neighbor Beam Relation (ANBR) is introduced, which indicates an average measure of interference level between beams. Knowing the probability for a couple of beams to interfere each other provides a first filter to focus solely on the most interfering beams. The heuristic solution then balance out the gain and the loss of the collaboration for each previously identified beam. Collaboration is achieved by momentarily muting beams or by allocating non-overlapping frequencies for users served by interfering beams. The heuristic solution and its significant enhancements in performance are presented in Chapter 4.

-AI/ML solution Chapter 5 introduces the AI/ML version of the Automatic Neighbor Beam Relation (ANBR) presented in the heuristic approach. In this solution, Multi-Armed-Bandits (MABs) are deployed to manage the collaboration between neighboring cells. For each couple of beams from neighboring cells that are foreseen to interfere each other, a MAB is implement and decides whether collaboration is required or not. It is shown in Chapter 5 that the MAB offers an online solution that considerably improves the coordination decisions and hence the cell performance in terms of MUT.

-Geolocation data-based solution

Geolocation data enables to create Radio Environment Map (REM) that can increase the performance of out heuristic solution. In this solution, a collaborative scheduler schedule two cells at a time. The geolocation data from the UEs is processed to maximize the MUT of the cells by avoiding the simultaneous scheduling of users that interfere each other too significantly. Thanks to the position of the user and the REM, the scheduler can indeed predict the interference triggered by every beam of the neighboring cell on the user and thus schedules an optimized set of users at each Transmission Time Interval (TTI). This solution is investigated in Chapter 6

Chapter 2

Theoretical and technical background 2.1 Introduction

This chapter introduces different mathematical and system models. The work achieved during this thesis relies on the background presented in this chapter. Section 2.2 presents probabilistic tools, useful to model wireless traffic including the arrivals and departures of users and the estimation of the load of a cell. Then Section 2.3 summarizes the M-MIMO model which is the major antenna 5G technology used in this thesis. Section 2.4 presents a model of MU-MIMO Proportional Fair (PF) scheduler for 5G cells. This scheduler implements a beam skipping scheme that is also described for in this section. Then the concept of REM is introduced. Lastly, this chapter presents MAB problems along with a selection of appropriate algorithms.

Probability theory for traffic model

The traffic model used in wireless network relies on probabilistic and queuing theory. This section aims at presenting the models used throughout the thesis.

User arrival model Exponential distribution

The exponential distribution is defined as follows. If a real random variable X follows the exponential distribution of parameter l, then its probability density function equals (2.1).

f (X, l) = l exp( lx) (2.1)
The exponential distribution is said to be memoryless. This means that the probability that T is superior to s, knowing that T is superior to s + t, is Chapter 2. Theoretical and technical background equals to the probability of T being superior to t, as defined in (2.2). This distribution is commonly used in queuing theory.

Pr(T > s + t | T > s) = Pr(T > t) (2.2)
Poisson processes We model the random arrival of users in a mobile cell with a Poisson process. Poisson processes are series of positive random variables with increasing values T 0 , ..., T M 1 , T M . The inter-arrival times follows an exponential distribution of parameter l, where l is defined as the arrival rate of the cell.

M-MIMO model 2.3.1 Antenna model

This section introduces the antenna model used for M-MIMO beamforming. An antenna consists in an array of antenna elements, as shown in Figure 2.1. The elements are at a l 4 distance from a metallic conductor, where l denotes the wavelength. A sub-array of elements generates a beam. The width of the beam depends on the number of elements in the sub-array. More elements leads to a more focused beam and thus a more powerful signal.

A beam b is determined by its direction which is given by two angles. The tilt q b is the angle between the z axis and the maximum beam direction whereas the azimuth f b is the angle in the between the direction of the beam and the y axis in a spherical coordinate system. This is illustrated by Figure 2.2, where O is the center of the antenna. Let us consider a beam b with a direction defined by (q b , f b ). The antenna gain in the direction (q, f) by the beam b is given by (2.3), where G 0 and f are defined as follows [START_REF] Tall | Multilevel beamforming for high data rate communication in 5G networks[END_REF].

G(q, f, q b , f b ) = G 0 f (q, f, q b , f b ) (2.3)
G 0 is the maximum gain and is obtained in the direction of the beam (q b , f b ). The expression is obtained through the power conservation equation and is given in (2.4),

G 0 = 4p R p 2 p 2 R p 0 f (q, f) sin q dqdf . (2.4)
f is a normalized gain function given in (2.5).

f (q, f, q e , f e ) = |AF x (q, f, q b , f b ) • AF y (q, f) • AF z (q, q b )| 2 • G d (q) (2.5)

Chapter 2. Theoretical and technical background

In this equation, G d is the approximation of the normalized pattern of a dipole, which depends solely on q and its expression is given in (2.6).

G d (q) = sin 3 q.

(2.6)

In (2.5), AF x , AF y and AF z are the array factors in the x, y and z directions. AF y models the impact of the metallic conductor. An infinitePerfect Electric Conductor (PEC) is assumed, and is replaced by the images of the radiating elements. Hence AF y is the term accounting for these images. The domain of validity of this antenna model is the half space in front of the PEC, the radiation of the antenna is thus considered null in the half-space located behind the PEC. The expressions for AF x , AF y and AF z are respectively given in (2.7), (2.9) and (2.8).

AF x (q, q b , f, f b ) = 1  N x m=1 w m N x  m=1 w m • a m (2.7) AF z (q, q b ) = 1  N z n=1 v n N z  n=1 v n • b n (2.8) AF y (q, f) = sin ⇣ p 2 sin q cos f ⌘ (2.9)
In practice, backward radiation is very weak. a m and b n , given in (2.10) and (2.11), are the complex amplitude given by (2.10) and (2.11).

a m = exp ✓ j2p (m 1)d x l (sin q sin f sin q b sin f b ) ◆ (2.10) b n = exp ✓ j2p (n 1)d z l (cos q cos q b ) ◆ (2.11)
To control the side-lobe gain level, weights are used. We denote these weights by w m in the x direction and by v n in the z direction. They depend on the size of the array, namely L x = (N x 1)dx horizontally and L z = (N z 1)dz vertically. Their respective expressions are given in (2.12) and (2.13), where s s express a ratio between the extreme and center dipole amplitudes and is given in (2.14).

w m = exp 0 @ (m 1)L x L x 2 s x ! 2 1 A (2.12) v n = exp 0 @ (n 1)L z L z 2 s z ! 2 1 A (2.13) s 2 s = ✓ L s 2 ◆ 1 log(a s ) ; s 2 {x, z} (2.14)

Channel model

Several models of channel exist to represent the randomness of wireless channels. The aim of these models is to estimate the signal power that a given user receives from the antenna. The randomness of the wireless channel is due to the nature of the environment which sets numerous obstacles between an antenna and a user. In Line of sight (LoS)-path, the path is direct and does not suffer any reflection nor refraction between the antenna and the receiver. In no Line of sight (noLoS)-path, the path encounters obstacles and the signal is either refracted or reflected. These two possibilities are illustrated by Figure 2.3.

FIGURE 2.3: Multi-paths

Several phenomena can account for the decrease of the signal power and are covered in this section.

Fast fading model

The channel fading amplitude is a signal attenuation caused by reflexions. In a dense urban environment, rich scattering implies a lot of reflections and no LoS, i.e direct path, as shown in Figure 2.3. The propagation is described as multi-path propagation. In this case, the fast fading can be modeled with the Rayleigh distribution. The probability density function (p.d.f) of this distribution is given in (2.15). h denotes the channel fading amplitude and W Chapter 2. Theoretical and technical background -the parameter of the distribution.

f (h) = 2h W exp( h 2 W ), 8h 2 R + (2.15)
In the case of an existing LoS path, the fading model follows a Nakagamim distribution where m is the parameter that translates the strength of the LoS component in comparison to multi-path components. The p.d.f of a Nakagami distribution of parameter m and W is written in (2.16), where G(•) is the gamma function. Note that when m is equal to 1, the Nakagami-m distribution equals the Rayleigh distribution and when m grows to infinity, the fading compares to pure LoS fading.

f (h) = 2m m h (2m 1) W m G(m) exp( mh 2 W ), 8h 2 R + (2.16)

Slow fading model

Slow fading accounts for attenuation of the signal that are varying slowly in time. These attenuations are caused for instance by shadowing due to the obstruction of the signal by a hill or a building. Slow fading is modelled by a log-normal distribution. For sake of simplicity, slow fading is not taken into account this thesis.

Pathloss model

The model of pathloss used in this thesis follows the model proposed by Okumura Hata [START_REF] Tapan | A survey of various propagation models for mobile communication[END_REF]. This is an empiric formulation of the radio propagation. This model is expressed in (2.17) in decibels (dBs), where r is the distance in kilometers between the BS and the UE and A and B are parameters depending on the environnement and the carrier frequency.

pathloss(r) = A + B log 10 (r)
(2.17)

MU-MIMO PF-scheduling implementation

This section introduces a scheduling algorithm that aims at scheduling in a fair manner a set of k users U k of a BS with M-MIMO beamforming, while limiting intra-cell interferences.

PF scheduling

To select users to be served in a fair manner, we opt for a scheduler based on PF criterion [START_REF] Combes | Scheduling gain for frequency-selective Rayleighfading channels with application to self-organizing packet scheduling[END_REF][START_REF] Altman | Generalized a-fair resource allocation in wireless networks[END_REF]. The PF criterion, denoted by PF(u), of a user u is defined as (2.18). PF aims at maximizing the mean user throughput in a cell while ensuring fairness among users. Denote by R b u,t M+1 the instantaneous rate of a user u 2 m when it is scheduled at t M+1 , by R b u,t M -its average rate computed over a time window [t M T , t M ] and by d -a very small number.

PF(u) = R b u,t M+1 R b u,t M + d . (2.18)
In the case of a SU-M-MIMO, the PF-scheduling algorithm is the following. The scheduler calculates the PF criteria, given in (2.18) for every user in the cell c. Then the scheduler selects the user with the highest PF criterion [START_REF] Mohammad | Performance comparison between round robin and proportional fair scheduling methods for lte[END_REF], as shown in Algorithm 1.

Algorithm 1 PF Scheduler

Init: U candidates = {u 2 c} u select argmax u2U candidates R c u,t M+1 R c u,t M +d

Beam skipping

Beam skipping is a beam selection technique used by a Multi-User MIMO (MU-MIMO) scheduler to optimize a QoS or a performance utility by reducing inter-beam interference. It is noted that beam skipping technique can be applied to both duplexing modes, TDD and FDD. Each UE is attached to the control beam of the GoB providing it with the best SINR. The user attachment provides additional spatial information that is exploited by the scheduler to mitigate inter-beam interference: (i) by avoiding scheduling of two users attached to the same control beam, and (ii) by avoiding the scheduling of two users attached to adjacent beams of the GoB. Figures 2.4 and 2.5 shows three adjacent beams of a GoB in the H (horizontal)-plane with respectively linear array and Gaussian tapered array. In the latter, secondary lobes are significantly attenuated thanks to amplitude tapering [START_REF] Tall | Multilevel beamforming for high data rate communication in 5G networks[END_REF]. Given a performance utility or any ranking criteria, selecting a set of candidate UEs from a list while complying with a beam skipping constraint follows the following steps. We select the top-ranked candidate UE, and remove the UEs attached to the same control beam or to the adjacent beams from the candidate list. We repeat these two operations until we have selected k max candidates or until the set of candidate users is empty.

MU-MIMO PF scheduling with beam skipping

A MU-PF scheduling with beam skipping is used as a baseline comparison throughout this thesis. This section describes the algorithm of this scheduler. In the initialization phase, the set U candidates contains all the users attached to m (or m 0 ). The scheduler ranks the users in U candidates with respect to a PF criterion, namely

R b u,t M+1 R b u,t M +d
, as in a standard PF algorithm. The first user of this list, namely the one with the highest PF criteria and denoted by u ⇤ , is selected to be scheduled and added to the set of scheduled user U k . We denote by b u ⇤ the beam serving the user u ⇤ We then remove all the users from the set of candidates that are attached to the beam b u⇤ and its adjacent beams(this is the beam skipping technique introduced in Section 2.4.2). We then select the next candidate with the highest PF criteria among the remaining candidates in U candidates and repeat this procedure of selection and removal. The iterative selection process continues until the number of users in U k max has reached k max users or until the set of candidates U candidates is empty. Algorithm 2 presents the MU-MIMO scheduler with beam skipping.

Algorithm 2 MU-MIMO PF Scheduler with beam skipping

Init: U kmax = {}, U candidates = {u 2 m} while |U kmax | < kmax or U candidates 6 = ∆ do u select argmax u2U candidates R b u,t M+1 R b u,t M +d U kmax U kmax [ u select
Remove users attached to the beam of u select and to its adjacent beams from U candidates end while Beam skipping reduces intra-cell inter-beam interference and considerably simplifies the scheduling algorithm as explained presently. Consider the selection process of a set of users for simultaneous MU transmission. The selection of a new user in this set will have negligible impact on the performance of already selected users. Hence the complexity of scheduling algorithms is significantly reduced. PF scheduling in the SU-MIMO case has the desired property, namely selecting the user with the best PF metric (i.e. instantaneous rate over the average rate) maximizes the global utility given as the sum of logarithms of the user rates. In the MU-MIMO case, the PF metric is generalized to the sum of PF metrics of a set of users that maximizes the PF utility function mentioned above. Without beam skipping in the GoB case, the selection of a set of users is intractable. It is noted that extensive simulations that have been carried out show that adding beam skipping significantly improves the throughput performance. Therefore, the combination of PF selection and beam skipping have been used as a fair baseline in the rest of the thesis.

MAB theory

This section introduces the concept of MAB with different implementations that are used in this work.

Role of MAB

MAB belongs to the family of Reinforcement Learning (RL) methods and hence we start with a brief recall on RL.

Reinforcement Learning

Reinforcement Learning techniques are based on trial and error [START_REF] Sutton | Reinforcement Learning, an Introduction[END_REF] [START_REF] Szepesvári | Algorithms for reinforcement learning[END_REF]. The gathering of data for Reinforcement Learning algorithm is thus sequential, as each trial produces new knowledge. The time is thus discretized and the decision is not continuous in time. An agent makes a decision which triggers an action that changes the state of its environment. The aim of Machine Learning algorithms is to learn automatically and through experience to improve their decisions. In Reinforcement Learning, an action triggers a feedback from the environment. The feedback can be either positive, in which case it is called a reward, or negative. Negative feedback is called a penalty. This penalty or reward depends on the objective of the algorithm and quantify the success of the algorithm to reach the objective. The aim of the agent is to obtain the greatest cumulative feedback possible over time by taking the decision minimizing penalties and maximizing rewards. Taking a decision for the agent implies to pick an action from a set of possible actions. Let us formalize this process with the following notations.

• A time step or round is denoted by n • The set of actions is denoted by A

• The decision of the agent at the time step n is an action denoted by x(n)

• The state of the environment at the time step n is denoted by s(n)

• The feedback given by the environment is denoted by r(n) Hence, at each time step n, the agent chooses an action x(n) from the set of actions A. This decision change the state s(n) of the environment and triggers a feedback r(n). We denote by T exp the length of the experiments, i.e. the number of plays of a RL algorithm.

MAB problems A multi-armed bandit problem is defined as follows: An agent, i.e. the MAB, has to face a set of actions A, referred to as arms [START_REF] Burtini | A survey of online experiment design with the stochastic multi-armed bandit[END_REF][START_REF] Vermorel | Multi-armed bandit algorithms and empirical evaluation[END_REF]. Let K be the number of arms, i.e. the size of the action set A. Note that in MAB problems, the set of actions A is finite and the MAB does not observe the state of the environnement to take a decision, nor does the agent change the state. MAB problems are single state cases.

At each round n, the MAB takes a decision and get a reward x(n) back. This reward and this reward only is used to update the knowledge of the MAB, as described in Figure 2.6 and in Algorithm 3. The duration of the experiment is denoted by T. The MAB is referred to as an on-policy method. An on-policy requires no previous knowledge and lessens the need for high resources or time for training compared to off-policies that are trained prior to deployment and base their decision on past knowledge. The aim of the agent is to achieve its goal by interacting with the environment. This goal can be modeled as the maximization of the perceived rewards over time, as written in (2. [START_REF] Gpp | Physical layer procedures for data[END_REF]). max

T exp  t=1 x(t) (2.19)
Let k 2 {1 : K} be the k-th arm of the MAB. Each time the arm k is pulled, i.e. chosen by the MAB, the arm k draws a random variable from a probabilistic distribution n k with a defined mean µ k and a standard deviation denoted by s k . The agent has no knowledge of the reward distributions of its arms. Therefore, the agent must gather knowledge about theses distributions through collecting samples at each round. Moreover, the MAB receives a reward only for the arm that has been pulled and cannot know the reward for the other arms. The aim of the MAB is to be able to identify the arm k ⇤ offering the distribution n ⇤ with the highest mean µ ⇤ , as given in (2.20).

µ ⇤ (n) = max k=1..K µ k (n) (2.20)
The aim of the MAB is to minimize the average regret, as explained presently. The regret at round n is denoted by R(n) is defined as the difference between the optimal reward r ⇤ (n), obtained by pulling the optimal arm k ⇤ , and the reward obtained by the MAB at round n, namely r(n). If the MAB has chosen the optimal arm, the regret for this round is 0. The expression of the cumulative regret is given in (2.21). The notion of regret is used to evaluate an algorithm's efficiency.

R(n) = n  t=1 (x ⇤ (t) x(t)) (2.21)
The MAB has the choice between exploring and exploiting. Exploitation refers to the resort to its knowledge gathered through previous decisions. Exploitation enables to take advantages of the previous rewards. It enables to avoid arms that have brought the lowest rewards in the past and to maximize the reward based on the previous actions. Exploration on the other hand enables to explore new arms and update the rewards received by an arm.

Exploration improves the knowledge of the MAB while increasing the frequency of the pulls of arms with lower rewards whereas exploitations takes advantages of the samples that have already been collected. MAB algorithms aim at balancing exploration and exploitation while minimizing the regret.

Explore-then-exploit

Explore-then-exploit, also referred in the literature as explore-then-commit, is a naive algorithm providing a first approach to MAB problems [START_REF] Liu | Explore-Exploit: A Framework for Interactive and Online Learning[END_REF][START_REF] Hastagiri | Explore-exploit in top-n recommender systems via gaussian processes[END_REF][START_REF] Michael | A Tutorial on Multi-Armed Bandit Problems, Models and Algorithms[END_REF]. In this algorithm, the agent first explores the different arms, which means that each arm of the MAB is played m times consecutively, one arm after another. Once each arm has been explored, the MAB exploits this exploration by selecting the arm that has received the highest mean rewards during the exploration phase, as written in (2.22), where x k is the empirical mean of the rewards obtained for the arm k and is given in (2.23).

k(t) = arg max k21..K µ k (t) (2.22) x k = t  i=0 I k(i),k x k (i) I k(i),k (2.23)
Algorithm and Figure 2.7 presents the Explore-then-Exploit mechanism. This naive approach offers suboptimal performance and depends highly on the choice of parameter m. High m provides lengthier exploration which might lead to weak arms being played more often than required and to high regret. On the other hand, low m means short exploitation and might results in the wrong choice of optimal arm and also high regret. For more analysis on the impact of the choice of m on the regret, we kindly refer the reader to [START_REF] Garivier | On explorethen-commit strategies[END_REF]. Moreover, to minimize the regret occurring with this strategy, the length of the experiment is required.

Upper Confidence Bound (UCB)1 algorithm

UCB1 algorithm [START_REF] Auer | Finite-time analysis of the multiarmed bandit problem[END_REF][START_REF] Auer | Using confidence bounds for exploitation-exploration tradeoffs[END_REF][START_REF] Bubeck | Regret analysis of stochastic and nonstochastic multi-armed bandit problems[END_REF] aims at finding good balance between exploration and exploitation. This algorithm requires no prior knowledge, compared to other Machine Learning techniques, nor the knowledge of the duration of the experiment T exp , unlike in the scheme introduced in the previous section. It is based on the optimism in face of uncertainty strategy. This strategy is mostly used in uncertain environment where the agent has limited knowledge, i.e. only the collected rewards. The agent will make the most optimistic decision based on the rewards collected in the previous rounds. To achieve this decision, it will compute two terms for each arm. The first term is the exploitation term and is equal to the mean of the rewards received for the arm x k . This term accounts for the selection of the best arm based on the rewards that have been observed in the past. The higher the exploitation term of an arm is, the highest the probability of obtaining a high reward. The second term takes into account the number of rounds n that have been played until now and the number of time the arm k has been played since the beginning, T k (n). This term, called the exploration bonus, is written as (n) , where e is a positive constant. Its role is to balance the exploitation term and force the MAB to explore regularly the arms that have been the least played. The more an arm is played, the lowest its exploration bonus is. As the exploration bonus depends on T k (n), an initialisation phase is required during which each arm is played once. After the initialisation phase, the arm that is played by the MAB at round n is the one offering the highest sum of the exploitation term and exploration bonus, as written in (2.24), where e = 2, as shown in [START_REF] Auer | Finite-time analysis of the multiarmed bandit problem[END_REF] . This action is repeated until the end of the experiment, after T rounds.

q e log(n) T k
k(n) = arg max 1kK (x k + s 2 log(n) T k (n) ) (2.24)
Algorithm 5 presents the UCB1 algorithm.

Algorithm 5 UCB1 algorithm for K-armed bandit

n = 0 for n = 1 to K do Select arm k = n Receive reward r k (n) end for for n = (K + 1) to T exp do Select arm k = arg max 1kK x k + q e log(n) T k (n) Receive reward x k (n) end for
For theoretical proof of the performance of the UCB1 algorithm, the reader is kindly referred to [START_REF] Auer | Finite-time analysis of the multiarmed bandit problem[END_REF]. It is shown that upper bound of the UCB1 cumulative regret is in order of O( p kT log(T)) bound and the regret in order of O(log(T)).

MAB with sliding windows

This section presents a strategy to deal with non-stationary environments. In non-stationary environment, the rewards of each arm are drawn from different distributions, varying over time. These distributions, as before, are unknown to the agent. In a network, users are arriving and leaving constantly triggering abrupt changes. Therefore, the state of the network changes often and is non-stationary. To deal with the frequent breakpoints, the MAB must be able to adapt rapidly and regularly to the new situation at hand. A technique enabling the MAB to adapt constantly is the resort to a sliding windows, presented in [START_REF] Garivier | On upper-confidence bound policies for non-stationary bandit problems[END_REF][START_REF] Wei | On abruptly-changing and slowlyvarying multiarmed bandit problems[END_REF][START_REF] Cavenaghi | Non Stationary Multi-Armed Bandit: Empirical Evaluation of a New Concept Drift-Aware Algorithm[END_REF]. Sliding-Window (SW)-MAB (SW-MAB) is designed to adapt itself to abruptly changing environment such as a mobile network. To give more weight to recent observations of the SW-MAB, only Chapter 2. Theoretical and technical background the t last rewards of each arm will be taken into account, as illustrated in Figure 2.8. t denotes the size of the sliding windows. Note that if t is equal or superior to the number of plays of the experiment, then the SW-MAB behaves as a regular MAB. This means that to calculate the empirical mean µ k (t) of an arm k, only the rewards received by k in the sliding window, i.e. the last t rewards of arm k, are taken into account in the equation given in (2.23). The equation used to make decision is the similar to the one used in UCB1 algorithm, except only the last t plays of each arm are taken into account, as shown in (2.25).

k(n) = arg max 1kK x k (t) + s 2 log(min(n, t) T k (t)) (2.25)
Algorithm 6 also presents the SW-MAB algorithm. As in the UCB1 algorithm, presented in Section 2.5.3, an initialisation phase is required. Algorithm 6 Sliding-windows MAB

x j = 0, X j = 0, n j = 0 with j = 1, 2 for t = 1, 2 do Select arm k(t) = t Receive reward x t n j = 1, n = n + 1 x t = x t end for for t = 3 to T do Select arm j = argmax j=1,2 (x j + r 2 ln(n) n j ) Receive reward x j n j = max(n j + 1, t), n = n + 1 x j = Â n s=n t+1 x j,s n j

end for

To conclude this subsection, the regret observed with the SW-MAB depends on the tuning of the parameter t. The upper bound of regret according to [START_REF] Garivier | On upper-confidence bound policies for non-stationary bandit problems[END_REF] is O( p T log(T)). Note that this method only requires the storage of the last t rewards of each arm which requires less resources than the UCB1 algorithm used without SW.

MAB with reset

This section introduces an alternative solution to SW-MAB to deal with non-stationary environment, the MAB with reset or reset-MAB. In this solution, a UCB1 algorithm is used as in 2.5.3 but the MAB is periodically reset. The number of plays between resets is denoted by T reset . Every T reset play, the reset-MAB resets itself by forgetting its past decisions and collected rewards and starting over [START_REF] Rahman | A Beam-Switching Scheme for Resilient mm-Wave Communications With Dynamic Link Blockages[END_REF][START_REF] Allesiardo | The non-stationary stochastic multi-armed bandit problem[END_REF], as shown in Figure 2.9. The reset compels the MAB to update the reward of each arm regularly, notably during the initialisation phase at each reset. Ideally, the parameter T reset is set low enough to be inferior to the interval between two breakpoints but high enough to allow sufficient plays and hence enough sampling to learn from the collected rewards. The resolution of the exploration-exploitation dilemma lies here in the tuning of the T reset parameter. The reset-MAB solution is described in Algorithm 7.

Algorithm 7 MAB with reset algorithm

n = 0, x j = 0, X j = 0, n j = 0; j = 1, 2 for t = 1, 2 do Select arm t Receive reward x t n j = 1, n = n + 1 x t = x t end for for t = 3 to T exp do Select arm j = argmax j=1,2 (x j + r 2 ln(n) n j ) Receive reward x j X j = X j + x j n j = n j + 1, n = n + 1 x j = X j n j if mod (n, T reset ) == 0 then n = 0, x j = 0, X j = 0, n j = 0; j = 1, 2 end if end for

Radio Environment Map (REM)

The Radio Environment Map (REM) concept, introduced by [START_REF] Zhao | Overhead analysis for radio environment map enabled cognitive radio networks[END_REF], consists of spatially geo-located measurements that have been interpolated to build the whole map of the measured metric, as described by Figure 2.10. The aim is to be able to predict a certain metric for a given location to support and enable certain optimization algorithms [START_REF] Birkan | Radio environment map as enabler for practical cognitive radio networks[END_REF][START_REF] Zhao | Network support-The radio environment map[END_REF]. After the collection of samples of measurements, the construction of REM involved the interpolation of these measurements to the whole map. In the context of the thesis the metric used is the RSRP. The relation between the location l 0 of a user u and the RSRP received by the users from a BS m is given in decibels:

RSRP (l 0 ) = v(l 0 ) z }| { p m + 10k log 10 (D (l 0 )) + s (l 0 ) + # (l 0 ) , (2.26)
where p m is the transmit power of m, D (l 0 ) is the geometrical distance be- tween m and u located at l 0 , s (l 0 ) captures the shadowing effect and # (l 0 ) models any measurement error by a centered uncorrelated Gaussian process with variance s 2 . The interpolation method used is Kriging [START_REF] Cressie | Fixed rank Kriging for very large spatial data sets[END_REF] with Covariance Tapering, which offer high performance in terms of prediction accuracy [START_REF] Han | Radio environment map construction by kriging algorithm based on mobile crowd sensing[END_REF][START_REF] Furrer | Covariance tapering for interpolation of large spatial datasets[END_REF]. For further details on the construction of REMs using Kriging with CT, the reader can refer to [START_REF] Mahbubul Alam | Performance evaluation of covariance tapering for coverage mapping[END_REF].

Conclusion

This section concludes this chapter that exposes the theoretical and technical models that are used throughout this thesis.This chapter contains the background required to have a good understanding of the rest of the thesis and to make the thesis self-contained.

Chapter 3

SON functionality for the mitigation of interferences in heterogeneous networks

Introduction

5G eMBB use cases require high throughputs and massive connectivity. To enhance the User Throughput and the capacity of the network, 5G resorts to M-MIMO as well as cell densification [START_REF] Romanous | Network densification: Challenges and opportunities in enabling 5G[END_REF][START_REF] Ge | 5G ultra-dense cellular networks[END_REF][START_REF] Peng Tian | Deployment analysis and optimization of Macro-Pico heterogeneous networks in LTE-A system[END_REF]. M-MIMO offers improved SINR thanks to beamforming. Beamforming, as presented in Section 1.1.1 focuses the energy on the user to increase the signal strength. It offers reduced interferences and spatial multiplexing through MU-MIMO.

To enhance further the capacity of a M-MIMO cell, the network can be densified as in 4G. Densification can be achieved through spatial densification [START_REF] Bhushan | Network densification: the dominant theme for wireless evolution into 5G[END_REF], namely the multiplication of cells in an area. The throughput of the users depends on the capacity of the cell which can be increased by adding Small Cells (SCs) in the coverage of a Macro Cell (MC). SCs have a smaller coverage than MCs due to a lower pilot power, around 30 dBm. While SCs offload the traffic of the MC, their deployment also decreases significantly the pathloss to a user, hence increasing the signal received by the UE. Therefore, SC densification improves the throughputs and the capacity of a network.

Inside the coverage of a MC, the location of the SCs are important. If the distance between of MC and SC is too small, the impact of the signal transmitted by the MC is too important and the coverage area of the SC will be highly reduced, hence the SC will have little to no impact on the capacity and the quality of the network. To avoid such issue, SCs' deployment is recommended close to the edge of the MC.

Even if the location of the SCs is optimized to maximize their impact, high interferences from the MC still limit their efficiency when they use the Chapter 3. SON functionality for the mitigation of interferences in heterogeneous networks same frequency band. In 4G, the enhanced Inter-Cell Interference Coordination (e-ICIC) feature was implemented to manage interferences in densified network and its performance was further improved when optimized with SON algorithm [START_REF] Tall | Self organizing strategies for enhanced ICIC (eICIC)[END_REF]. e-ICIC feature is based on the Almost Blank Subframe (ABS) feature introduced by 3GPP [59] and has been the subject of various studies [START_REF] Bartoli | Adaptive muting ratio in enhanced inter-cell interference coordination for LTE-A systems[END_REF][START_REF] Weber | Scheduling strategies for HetNets using eICIC[END_REF][START_REF] Rácz | On the impact of inter-cell interference in LTE[END_REF][START_REF] Tang | Joint resource allocation for eICIC in heterogeneous networks[END_REF][START_REF] Gospel Ruben | Scheduling for interference mitigation using enhanced intercell interference coordination[END_REF][START_REF] Quek | Adaptive interference coordination in multi-cell OFDMA systems[END_REF][START_REF] Dirani | A cooperative Reinforcement Learning approach for Inter-Cell Interference Coordination in OFDMA cellular networks[END_REF]. This feature mutes most part of the MC transmission during a certain portion of the sub-frames, as depicted in Figure 3.1. During the mute of the MC, the SCs are able to serve their users with reduced interference. Alternatively, the feature can be applied in the frequency domain, as shown in Figure 3.2. In this setting, the bandwidth is split between the MC and the SCs. This leads to frequency avoidance and remedies the issue of interferences while reducing momentarily the frequency resources of the cell. Only one cell is chosen for transmission, namely the one offering the smaller pathloss. ZF precoding could also be a solution to the interference problem in heterogeneous cells. This precoding technique, introduced in [START_REF] Spencer | Zeroforcing methods for downlink spatial multiplexing in multiuser MIMO channels[END_REF] takes into account the co-channel interference through an inversion matrix for instance. When the ZF technique is applied to a group of cells (i.e. distributed M-MIMO), it requires intensive processing and strict requirements on latency and backhaul capacity.Reference [START_REF] Hosseini | Massive MIMO and small cells: How to densify heterogeneous networks[END_REF] deals with interferences in heterogeneous network deployment with a non-cooperative solution. This solution aims at transmitting data to users through independent streams which are orthogonal to the subspace of the main interfered users.

In this chapter, we introduce a collaborative scheduling for a M-MIMO cell with SCs located in its coverage zone. This scheduler offers a low complexity coordination scheme. Each SC sends a single number to the MU scheduler of the MC that enables the MC scheduler to optimize the performance of the SCs and MC. It is noted that the virtualization of the 5G RAN and particularly of the MAC eases the exchange of information between the different cells of the heterogeneous cell.

Optimization problem formulation

In this section, we present a 5G heterogeneous cell and express the optimization problem triggered by the interferences of the MC on the SCs.

5G heterogeneous cell

Let us consider a 5G heterogeneous cell with a MC and several SCs, as shown in Figure 3.3. This section models the MC and the SCs present in the heterogeneous cell. We denote by m the MC and by s a SC belonging to the set of SCs S, not to be mistaken with S a signal from an antenna. The GoB of the MC m is denoted by B m

MC model

The MC is a 5G MC with M-MIMO system. GoB is used for initial access and synchronization. Each beam is activated one after the other in a beam sweeping scheme. The UE sends back to the BS the ID of the beam offering the best signal and get attached to it. Data transmission is achieved with TDD through MRT beamforming. MRT focuses the energy of the signal on the user using time reversal concept, as introduced in [START_REF] Titus | Maximum ratio transmission[END_REF].

S b,m u b,m = p b m |(h b u b,m ) H c u b,m )| 2 Â s2S I s u b,m + Â m 0 I m 0 u b,m + s 2 (3.1)
The scheduler of the MC can schedule at most k users simultaneously and at most one user per beam.

SC model

We make the assumption that a SC is located in the coverage of one beam only for the purpose of simplicity, though the adaptation to overlapping SCs is straightforward. The scheduler of a SC s is a SU-scheduler and can schedule at most one user simultaneously.

The signal received by a user u s of a SC s, denoted by S s u s is given in (3.2). g s u s denotes the gain (including pathloss, antenna gain and fast fading) from a SC s to a user u s . p s denotes the pilot power of the SC s. The set of users attached to a small cell s, a macro cell m or a beam b are respectively denoted by U s , U m and U b . The interference on a user u s from a SC s from another SC s and another MC m 0 are denoted by I s 0 u s and I m 0 u s respectively and their expression are given in the next paragraph.

S s u s = p s |g s u s | 2 Â s 0 6 =s I s 0 u s + Â m 0 I m 0 u s + s 2 (3.2)

Interference model

The interference created by a small cell s on the user u is written as:

I s u = p s |g s u | 2 (3.3)
The interference generated by a macro cell m 0 on u reads:

I m 0 u = p m 0 Â u 0 2U m 0 ,u 0 6 =u |(h m 0 u ) H c m 0 u 0 | 2 (3.4)
It should be noted that the interference from a SC on the users of the MC is expected to be significantly lower than the interference from the MC on the users of a SC, as a result of the MC presenting a much higher transmission power than the SC. Therefore, our solution will focus on tackling the issue of the interference from the MC on the SCs solely.

Study of the impact of the scheduling

The scheduling policy of a SC s for t = t M refers to the choice of the user u s that will be served at time t M . The scheduling policy of the MC refers to the choice of a set U k (t M ) of at most k max users to be served at time t M . We denote the set of beam that have to be activated to serve the users of U k by B k . We define by R m,b u (t M ) and R s u (t M ) the instantaneous throughput of the user u at time t M belonging to respectively the MC through the beam b and the SC Chapter 3. SON functionality for the mitigation of interferences in heterogeneous networks s. Denote by R m,b u (t M ) the average throughput received by a user u during the time window [t 0 , t M ]. We assume that R m,b u (t M ) = 0 if the user u is not scheduled at time t M and that R m,b u (t 0 ) = 0 . e is a small positive parameter defining the degree of weighting decrease in the exponential moving average used in the calculation of R m,b u (t M ) as given in (3.5).

R m,b u (t M ) = (1 e)R m,b u (t M 1 ) + eR m,b u (t M ) (3.5)
We define the global utility of the heterogeneous cell, U global as the sum of utilities of the macro-and small cells in its coverage area. This utility function maximizing the mean user throughput in a proportional fair manner is given by (3.6). d is a positive number as small as possible to avoid problematic behavior when the logarithm's argument is null.

U global = Â b2B Â u2U b log(R m,b u (t M ) + d) + Â s2S Â u2U s log(R s u (t M ) + d) (3.6)

Impact of the MC scheduling on the global utility

To study the impact of scheduling policy of the MC at t M+1 on the evolution of the utility U global , let us assume that the scheduling policies for the MC and the SCs have already been chosen and applied during the time window [t 0 , t M ]. Consider the MC scheduling policy at t M+1 , which consists in choosing a set U k of at most k max users. The contribution of a scheduled user u m , d scheduled u m , to U global reads (3.7), according to [START_REF] Combes | Scheduling gain for frequency-selective Rayleighfading channels with application to self-organizing packet scheduling[END_REF].

d scheduled u m = log((1 e)R m u m ,t M + eR m u m ,t M +1 + d) log(R m u m ,t M + d) (3.7)
The approximation of the variation d scheduled u m with a first-order Taylor expansion is given in (3.8)

d scheduled u m = R m u m ,t M+1 e R m u m ,t M R m u m ,t M + d + o(e) (3.8)
If u m is not scheduled at time t M+1 , then the variation d u m is given by (3.9).

d notscheduled u m = log((1 e)R m u m ,t M + d) log(R m u m ,t M + d) (3.9)
The variation d notscheduled u m can also be approximated with a first-order Taylor expansion. This approximation is given in (3.10).

d notscheduled u m = e R m u m ,t M R m u m ,t M + d + o(e) (3.10)
We denote by U m the utility function of the MC m and by U k (t M+1 ) the set of users u m of the MC m scheduled at time t M+1 . The variation of utility U m for the MC m between t M and t M+1 is the sum of the variation of utility for every user attached to the cell, as written in (3.11).

DU m =  u m 2U k (t M+1 ) R m u m ,t M+1 + d  u m 2U m e R m u,t M R m u m ,t M + d + o(e) (3.11)
U m (t M+1 ) is given in (3.12) and is the sum of U m (t M ), depending solely on previous scheduling decisions, and of w m (t M+1 ), which depends on the decision taken for the time slot t M+1 . The scheduling decision for the MC consists in a set of at most k max users. Within a beam, a single user having the best PF metric is scheduled. The PF metric is defined in Section 2.4.1. Therefore, a scheduling decision for MC amounts to picking a set of beams B k max , of at most k max beams, and selecting the users with the highest PF criterion attached to these beams.

U m (t M+1 ) = U m (t M ) + w m (t M+1 ) (3.12)
where

U m (t M ) = Â b2B Â u2U b (log(R b u,t M + d) e R b u,t M R b u,t M + d ) (3.13)
and

w m (t M+1 ) = e  b2B k max u2U b ( R b u,t M+1 R b u,t M + d ) (3.14)

Impact the SC scheduling on the global utility

As for the study of the impact of the MC scheduling on the global utility, we assume that the scheduling policies for the MC and the SCs have already been chosen and applied for the time window [t 0 , t M ]. We now consider scheduling policy for the time t M+1 for the SC for one user at a time. results in the variation in the utility and is written as (3.15). The variation of utility for the BS s d u s following the scheduling of the user u s belonging to s at time t M+1 is denoted by d scheduled u s and is written as (3.15).

d scheduled u s = log((1 e)R s u s ,t M + eR s u s ,t M +1 + d) log(R s u s ,t M + d) (3.15)
Chapter 3. SON functionality for the mitigation of interferences in heterogeneous networks

The approximation of this variation is given in (3.16).

d scheduled u s = R s u s ,t M+1 e R s u s ,t M R s u s ,t M + d + o(e) (3.16)
If a user u s 2 s is not scheduled at time t M+1 , the decrease in the utility function U s due to this decision is denoted by d notscheduled u s and is given in (3.17).

d notscheduled u s = log((1 e)R s u s ,t M + d) log(R s u s ,t M + d) (3.17)
The approximation of this variation with a first-order Taylor expansion is given (3.18).

d notscheduled u s = e R s u s ,t M R s u s ,t M + d + o(e) (3.18)
Finally, the utility function U s of a SC s at time t M+1 of the heterogeneous cell is given in (3.19). This utility function is two-fold. U s (t M ) is the part of the utility function depending solely of the scheduling decisions taken up to time t M , whereas w s (t M+1 ) accounts for the variation of utility triggered by the scheduling policy for t M+1 . U s (t M ) and w s (t M+1 ) are given respectively in (3.20) and (3.21).

U s (t M+1 ) = U s (t M ) + w s (t M+1 ) (3.19)
where

U s (t M ) = Â u2U s (log(R s u,t M + d) e R s u,t M R s u,t M + d ) (3.20)
and

w s (t M+1 ) = e max u2U s ( R s u,t M+1 R s u,t M ) (3.21)

Optimization problem

We consider an heterogeneous cell which consists in a MC, denoted by m, and of a set S m of SCs s.The aim of our work is to maximize the MUT of the heterogeneous cell. Therefore, this translates in the optimization of U global , given in (3.6). The system is also under a beam skipping constraint (See Section 2.4.2). This constraint aims at minimizing inter-beam interferences by forbidding the simultaneous activation of adjacent beams. The optimization of the mean user throughput in the 5G heterogeneous cell can hence be translated as the following optimization problem, given in (3.22).

MU scheduler for the Macro Cell

maximize

B k max 2B U global (t M+1 ) = U m (t M ) + Â s2S m U s (t M ) + w m (t M+1 ) + Â s2S m w s (t M+1 ) subject to 8(b i , b j ) 2 B k max , |i j| 2 (3.22)
The problem (3.22) is a complex combinatorial optimization problem. The solution aims at maximizing the global throughput in a fair manner. A heuristic solution is proposed to allow real-time solution compatible with operational systems

MU scheduler for the Macro Cell

This section introduces a collaborative MU-scheduler for the MC. Regarding the scheduling in the MC, let us remind that at most one user attached to a beam b 2 B m can be scheduled simultaneously. The aim of the collaborative scheduler is to optimize the MUT of the heterogeneous network. This means protecting the SCs while still offering a good quality of service to the users of the MC.

Let us consider a beam b 2 B m that highly interferes the SC s. When a user attached to the beam b is served, there is a high probability that the users of the SC are highly interfered. On the other hand, if the beam b is muted, no user attached to this beam is allowed to be served and the beam utility is null. There is thus the trade-off to be dealt with.

The first step consists in assessing the gain obtained by the scheduling of the user of a beam. Without collaboration, a simple MU-scheduler with a beam skipping constraint, as introduced in Chapter 2, is applied. If a user u b attached to a beam b is scheduled at time t M , it is assumed that this user has the best PF-criterion among the set of users U b , where U b is the set of users u attached to beam b.

u sched b (t M ) = arg max u2U b R b u,t M+1 R b u,t M + d (3.
d b = e max u2U b R b u,t M+1 R b u,t M + d (3.24)
Chapter 3. SON functionality for the mitigation of interferences in heterogeneous networks

The beam interfering the most a SC s is referred to as its interfering beam. In our current design, we consider only one interfering beam per SC. Note that the generalisation to the case of more than on interfering beam is straightforward. We define a beam as activated if one of its user is being served. Each SC sends a loss factor d s to the MC. This loss factor accounts for the loss in performance of the SC due to the scheduling of the beam interfering the SC the most.

To calculate this loss factor d s , the SC requires two expected rates per UE. The first rate is denoted by R s,I u,t M+1 , where I stands for interfered, and is the rate that the UE expects to receive at t M+1 if the interfering beam of the SC is active. The second rate required is the rate expected by the UE at t M+1 if the interfering beam of the SC is not active, this rate is denoted by R s,NI u,t M+1 , where the subscript NI means Not Interfering. To compute these two rates, the user u proceeds as follows: The UE u computes one of the two rates based on actual received signals. Through a simple test like a threshold comparison or a likelihood ratio, it assesses if the received signals equals R s,I u,t M+1 or R s,NI u,t M+1 . The SC stores once the interferences observed by UEs with and without the interfering beams. These values are averaged to obtain a mean value of interference level with and without the interference of the interfering beam. The UE u uses these values to compute the complement rate, for instance R s,I u,t M+1 if the actual signal lead R s,NI u,t M+1 or R s,NI u,t M+1 if the actual signal lead R s,I u,t M+1 . The UE finally sends these two terms to the SC.

When the interfering beam of SC s is activated, the contribution of the SC s to the global utility function U global is negatively impacted. This loss factor d s is expressed in (3.25) and is calculated as the difference between the PF criteria of the highest PF-ranked user with and without the activation of the interfering beam.

d s = emax u2s ( R s,I u,t M+1 R s,NI u,t M+1 R s u,t M ) (3.25)
d s is transmitted to the macro cell scheduler by the different SCs. We define the quantity D b in (3.26) that the BS calculates to assess the contribution of a beam b to the global utility taking into account the corresponding loss factor d s of the SC that b interferes.

D b = d b + d s (3.26)
We denote by B candidates the set of beam that have at least one user attached to them. Once the scheduler of the MC receives the loss factors of the SCs,

Validation of the scheduler through numerical results

47

it computes the gain factor D b of each beam b 2 B candidates . Beam skipping, given in Section 2.4.2, is then applied to the set of candidates beams B candidates . The beam with the highest gain factor D b is selected and added to the set of beams to be scheduled B k max . Then the adjacent beams to the selected one are removed from the set of candidate beams B candidates following the beam skipping constraint. These two steps, selection and removal, are repeated until k max beam have been selected or until the set of candidates beams is empty. Once the set B k max is known, the user selection for the macro cell becomes straightforward. It is recalled that each beam b serves at most one user at a time which is selected using a standard PF criterion applied to the users covered by that beam. Finally, the addition of complexity of the algorithm to the baseline is O(N users ), where N users is the total number of users in the heterogeneous cell. The algorithm for the MU collaborative scheduling of the MC is presented in 8. The SCs' scheduler remains unchanged and is performed independently. It implements a SU-PF scheduling, as presented in Section 2.4.

Algorithm 8 MU Beam Selection

Validation of the scheduler through numerical results

In this section we assess the efficiency of the MU collaborative scheduler through numerical experiments.

Chapter 3. SON functionality for the mitigation of interferences in heterogeneous networks

Deployment scenario

Network description

We consider a heterogeneous network consisting in a MC, and 10 SCs located within the MC's coverage area. The 5G MC belongs to a tri-sectoral network with 18 surrounding LTE sectors placed over a hexagonal grid. The MC is using a GoB for control and synchronization and eigen beamforming with MRT precoding for the transmission of data. The SC relies on an omnidirectional antenna. The location of the SCs in the MC can be seen in Figure 3.4. The SCs are located in this scenario close to the edge of the cell where they allow to achieve the best densification gain, as this is where the users with the lowest SINR are located. The color code in Figure 3.4 is only used for sake of clarity and does not give any indication about the strength of the signal. The characteristic of this network are summarized in Table 3.1. The pathloss model follows the Hata model [START_REF] Gpp | Evolved Universal Terrestrial Radio Access (E-UTRA); Further advancements for E-UTRA physical layer aspects[END_REF](See Table 3.2).

Validation of the scheduler through numerical results

49

Traffic description

We consider elastic traffic with users arriving randomly according to a Poisson process, as introduced in Section 2.2.1. In each small cell s, the number of arrival per second is l s users/s. The number of arrival per second in the MC m is denoted by l m users/s. The users of each cell are distributed uniformly in space. A new user arrives in its cell, downloads a file of a size with a mean equal to s f and leaves the cell. The characteristics of the traffic for this experiment is presented in Table 3.3.

Scenario

We compare two scenarios. In each scenario, the SCs use a SU-PF scheduler. In the baseline scenario, the MC uses a MU-PF scheduler with a beam skipping constraints, whereas in the second scenario the MC uses the MU collaborative scheduler. The aim is to assess the performance of the collaborative scheme.

Chapter 3. SON functionality for the mitigation of interferences in heterogeneous networks

Results

Comparison of MUT

We first compare the MUT in the macro cell and the small cells for a fixed l m and for different l s . Figure 3.5 shows the MUT for the heterogeneous cell for the baseline and the collaborative scheduling. The MUT gain achieved by the collaborative scheduler with respect to the baseline varies from 4 to around 20 percent. We observe that the gain is bigger for higher small cell traffic. For instance, for l s = 3 users/sec, the global MUT varies from 7 Mbps to 7.3 Mbps only whereas for a higher l s , with l s = 4 users/sec, the MUT increases from 2.2 to 2.6 Mbps, providing better gain. One can observe that with higher SCs' load, the interference coordination brings about higher gain which is related to the non-linear relation between the load and the arrival rates. Figure 3.9 presents the Jain's index [START_REF] Rajendra | A quantitative measure of fairness and discrimination[END_REF] for the heterogeneous cell for different small cell users' arrival rates. One can observe that the overall fairness is improved by the collaborative scheduling, with Jain's index gain varying from 5 to 12 percent.

Conclusion and extensions

This section has presented a collaborative scheduling technique to manage 5G heterogeneous network with M-MIMO macro cells and small cells deployment. The solution is based on exchanging a loss factor that each small cell provides to the macro cell, that estimates the performance utility loss due to the macro cell scheduling. This approach enables the MC to achieve good performance while protecting the SCs. While the optimization of MU scheduling is a combinatorial problem, it is shown that the exchange of the loss factor allows to design a very low complexity scheduling algorithm. The MU scheduler presented in this chapter is thus feasible in operational networks.

The collaborative scheduling provides throughput gain that grows with the increase of traffic demand. In the simulations, the MUT gain of up to around 20 percent is obtained for the heterogeneous cell comprising a MC and several SCs while achieving enhanced fairness.

The proposed solution has been developed for TDD system. The solution is simpler for FDD system, such as GoB implementation in the frequency domain, since the same beams are used for the transmission of both control and data signals. Comparing to state of the art CoMP approach, the solution requires smaller amount of information exchange between small-and macro cells. The coordinated scheduling solution can benefit from virtual network architecture, e.g. Mobile Edge Computing (MEC) that simplifies information exchange between the network nodes. The architecture proposed by O-RAN, for instance, can support this solution.

Chapter 4

Interferences management between M-MIMO cells 4.1 Introduction

The energy focusing capability achieved by the M-MIMO technology with beamforming has clear benefits in terms of enhanced capacity and EE [START_REF] Cui | Energy-efficiency of MIMO and cooperative MIMO techniques in sensor networks[END_REF][START_REF] Xu | Energy efficiency optimization for MIMO broadcast channels[END_REF][START_REF] Knr Surya | Energy efficiency in massive MIMO-based 5G networks: Opportunities and challenges[END_REF]. Unfortunately, such gains come with greater interference on the border of the cells [START_REF] Blum | MIMO capacity with interference[END_REF][START_REF] Tse | Fundamentals of wireless communication[END_REF]. Traditional approaches for collaboration between adjacent cells with M-MIMO deployment such as CoMP [START_REF] Irmer | Coordinated multipoint: Concepts, performance, and field trial results[END_REF][START_REF] Woo | Joint network-wide opportunistic scheduling and power control in multi-cell networks[END_REF] require intensive signal processing, exchange of important amount of information and have strict requirements on the backhaul capacity. CoMP [START_REF] Bassoy | Coordinated multi-point clustering schemes: A survey[END_REF] is a standardized feature which mitigates interference between cells. It can be divided into three main categories of coordination schemes:

• Joint Transmission: In this scheme, the CSI, the scheduling information and the data are shared between two or more base stations. The data is transmitted to a given user by several BSs simultaneously. This technique enables to replace the interference by additional useful signal. It offers good performance at the expense of high backhaul bandwidth. The joint transmission can be coherent or incoherent. Coherent joint transmission means that joint precoding is used and the transmission is synchronized between the different transmission points. In noncoherent transmission, each transmission point individually precodes its own transmission.

• Dynamic Point Selection: With this technique, CSI, scheduling information and data are also shared between multiple transmission points but only one transmission point transmits at a time to a given user. For each subframe, the best server is chosen to transmit, depending on the current channel condition.

• Coordinated Scheduling/Beamforming: Here, only one transmission point sends the data to a given user. Hence, the data is not shared among the BSs. However, CSI is shared and scheduling and beamforming information are coordinated to minimize interference.

Chapter 4. Interferences management between M-MIMO cells

CoMP offers good performance but at the expense of extensive processive for precoding and high backhaul load requirements which may impact the latency. CoMP has been introduced in 5G standards and denoted as distributed6MIMO or as Multi-Transmission Reception Points [START_REF] Holma | 5G technology: 3GPP new radio[END_REF]. One centralized unit controls several radio units through low latency connections. CoMP has been a success for UL transmissions in Long Term Evolution (LTE) but lacked UE support in the DL. In that matter, 5G shows better potential to benefit from CoMP technology. MU-MIMO scheduling is a combinatorial problem. Therefore, this optimization problem is considered in the literature as:

• An aggregated utility problem with greedy suboptimal solutions. In this context the scheduler takes a sequence of decisions, each of which is aiming at increasing the global utility. For instance, [START_REF] Dimic | On downlink beamforming with greedy user selection: performance analysis and a simple new algorithm[END_REF] proposes a Greedy User Selection algorithm to maximize the sum-rate. This method demands the calculation of precoders and power allocation at each user selection and shows degraded performance when the CSI knowledge is not perfect.

• A metaheuristic optimization problem. This category relies on stochastic approximation and have a high probability of converging towards the optimal solution. However, there is no theoretical proof of convergence or performance. A set of a feasible solution is constructed iteratively until a stopping criterion is met. The solutions in this category are often inspired by nature, such as particles swarm in [START_REF] Hei | Novel scheduling strategy for downlink multiuser MIMO system: Particle swarm optimization[END_REF], where the behavior of swarms is mimic to manage DL scheduling.

PF in the context of MU-MIMO has been addressed by multiple authors. For example, the authors in [START_REF] Liu | Proportional fair scheduling for multi-cell multi-user MIMO systems[END_REF] develop a PF framework for multi-cell multiuser MIMO scheduling, in conjunction with ZF precoding. They define a PF metric for a set of users that is given as the sum of the PF metrics of the individual users, and argue that it is equal to the rate of change of the global PF utility function (i.e. sum of logarithms of the accumulated throughputs). The selection of set of users to be scheduled is first solved for the single-cell case and then generalized to the multi-cell case. This chapter proposes a MU-collaborative scheduler for M-MIMO to enhance interference management at a beam level. Automatic Neighbor Beam Relation (ANBR) is a feature allowing to automatically establish relations between beams. The exploitation of input from the ANBR enables to further optimize resource management functions with beam level spatial resolution. The ANBR provides the MU-scheduler with binary matrix, with non-zero elements representing interfering beam relations while taking into consideration current traffic conditions. The ANBR matrix can be derived 4.2. Interference problem with M-MIMO cells 57 from the Automatic Neighbor Relation (ANR), a beam management feature that allows to evaluate different types of relations between gNodeBs (gNBs) or beams. We show how the MU-schedulers of neighboring cells can take advantage of the ANBR matrix to minimize collisions between interfering beams. The proposed approach introduces an implicit collaboration between neighboring cells that requires small amount of information exchange and a low complexity.

The chapter is organized as follows. Section 4.2 analyzes the issue of interfering cells. The architecture supporting the solution is described in Section 4.3. Sections 4.4.1 and 4.4.2 introduces respectively the concepts of static and dynamic ANBR while Section 4.5 develops a MU -collaborative scheduling solution relying on the ANBR. Numerical results are presented in Section 4.6 followed by concluding remarks in Section 4.7. We consider a MC m with a M-MIMO antenna, using a GoB type of beamforming for both control and data channels. Hence a signal model that differs from previous chapter. It is noted that the approach can be generalized to other beamforming techniques such as Eigen value beamforming. The M-MIMO antenna is equipped with N = N x ⇥ N z radiating elements, and is serving N m mobiles. Each mobile has a single receiving antenna. Assume that k max is the maximum number of mobiles (the term users can be used interchangeably) that the BS can serve simultaneously and that the BS serves k users at a given time slot with k  k max . Denote by p max the maximum transmission power of the BS. The power p m that the BS m transmits to user u, equals |h m b u (u)| 2 is modeled as the product between the pathloss, the antenna gain G m b u (u) of the beam b u serving user u and measured at the direction of user u, and the fast fading term Z(u). Its expression is given in (4.2), where c and g are constants that depend on the type of environment. The latter is modeled as a realization (per user) of a Nakagami distribution, and can be parameterized for different propagation environments [START_REF] Charash | Reception through Nakagami fading multipath channels with random delays[END_REF].

Interference problem with M-MIMO cells

|h m b u (u)| 2 = c d g u,m G m b u (u)Z(u) (4.2)
The SINR of a user u attached to cell m is written as (4.3), where I m 0 u expresses interference produces by a cell m 0 on the user u and is detailed in the next paragraph.

S m u = p m |h m b u (u)| 2 Â m 0 I m 0 u + s 2 (4.3)
The beams are highly focused with high gain. Therefore, two beams with overlapping coverage area can highly interfere each other. Denote by I b 0 ,m 0 u the interference received by the user u from the beam b 0 of the cell m 0 , by p m the power that the BS m transmits in the direction of u through the beam b 0 . h m 0 b 0 u 0 (u) models the channel condition between the cell m 0 and the user u through the beam b 0 , as detailed in Section 2.3.2. The interference generated by a cell m 0 on u is written as the sum of interferences from its active beams I b 0 u :

I m 0 u = Â b 0 2B m 0 I b 0 u (4.4)
where

I b 0 u = p m 0 Â u 0 2b 0 ,u 0 6 =u |h m 0 b 0 u 0 (u)| 2 (4.5)

Interference problem with M-MIMO cells 59

The probability of users of two distincts beams interfering each others depends on the beams' configuration and on the propagation environment. If the antenna grain diagrams associated to the beams do no not overlap, interbeam interference can still occur due to reflections, however the probability of significant interference is reduced. The location of the users inside the beams is also important. The further away the user u m is from its BS m, the weaker the received signal from its BS m gets and the higher the interference from the other BS m 0 becomes. Therefore, on Figure 4.3, user u 1 receives a strong signal from m through b and a low interference from the beam b 0 , whereas the opposite occurs for users u 2 . Users u 1 and u 1 0 are less likely to be interfered by the other cell as they are located further away from their interfering BS but their gain or loss in case of collaboration is more difficult to predict due to the use of M-MIMO. On one hand, collaborating involves reduces resources but also decreased interference level. In this simplified scenario, a coordination scheme between the two cells could benefit users u 2 and u 0 2 and either upgrade or downgrade the QoS for u 1 and u 2 . This illustration is simplistic for pedagogy purpose but reveals the tradeoff at stake with the coordination of two cells for interference management. Our aim is to maximize the MUT for the two cells we consider. We want to offer the best SINR possible to every user while protecting the users on the edge. We next show the architecture framework of the solution.

Architecture framework

This section introduces the network architecture framework of our solution.

Distributed SON (D-SON)

D-SON is distributed at the edge of the network, for instance in e-NodeB for 4G or edge cloud for 5G. This location offers a more dynamical reaction, closer to real-time than C-SON.

ANBR-based interference management

We present in this section the ANBR solution that is divided into two modular applications: the static ANBR and the dynamic ANBR. 2 ). Therefore, the matrix component corresponding to this couple has for value 1 while the other components take the value 0. The static beam relation matrix is derived from user measurements during a period of time (e.g. days).

Dynamic ANBR

The dynamic ANBR is deployed either as a SON function in the D-SON in 3GPP architecture or as a xApp in the Near-RT-RIC in O-RAN architecture. In both architectures, dynamic ANBR receives from static ANBR the static matrix A. From the entries of the matrix A and the dynamic traffic data, the dynamic ANBR produces a dynamic beam relation matrix Q that indicates to the schedulers of the neighboring cells the selected beams for the coordination. The matrix Q is updated in a time scale of the traffic dynamics, e.g. in the order of a second.

Categorization of users

Coordination should only be applied when the potential overall gain exceeds the possible overall loss. To this end, we divide the users into two categories. Let us consider U s b -the set of users served by a beam b defined by (4.6). In the example given in Figure 4.3, U s b would include the users u 1 and u 2 , and U b 0 the users u 0 1 and u 0 2 .

U s b = {u 2 m|b = arg max b2B m C m,b u } (4.6)
The second category of users consists in the users of the neighboring cell m 0 that are likely to benefit from the beam b being in collaboration, i.e. highly interfered users mainly located on the border of cell m 0 . The indicator A 1 (b, u 0 ), given in decibels in (4.7). The indicator A 1 (b, u 0 ) specifies if a user u 0 has a Signal to Interference Ratio (SIR) calculated with respect to the interfering beam b below a threshold g th and is given in decibels in (4.7).

A 1 (b, u 0 ) = ⇢ 1 if C m 0 u 0 I b u 0 < g th 0 otherwise. (4.7)
The set of users u 0 2 m 0 verifying the condition A 1 (b, u 0 ) is denoted by U i b and its expression is given in (4.8). In the example given by Figure 4.3, the set U i b contains u 0 2 and the set U i b 0 only u 2 .

U i b = {u 0 2 m 0 |A 1 (b, u 0 )} (4.8)
The cardinalities of U s b and U i b are denoted by n s b and n i b and are given by (4.9) and (4.10) respectively. In the configuration presented in

n i b = |U i b | (4.9)
n s b = |U s b | (4.10)

Generation of the dynamic beam relation matrix Q

Once the categorization of the users is determined, the dynamic ANBR is used to construct the dynamic beam relation matrix Q. For each entry 1 of the static beam relation matrix A, the module checks if the collaboration will benefit the majority of users or not. This decision is based on the comparison of the ratio between the number of users that beam b interferes in m 0 and the number of users it serves in m to a threshold h th . If for both beams, the ratio is above a predetermined threshold, then the corresponding entry for the couple in the dynamic matrix Q is 1 and is 0 otherwise. The threshold is determined throuhj a, exhaustive optimization performed once (See Section 4.6.2). This condition is written in (4.11).

A 2 (b) = ⇢ 1 if n i b n s b > h t h 0 otherwise. (4.11)
The matrix Q is thus defined as follows:

Q b,b 0 = A b,b 0 ⇥ 1 {A 2 (b)} ⇥ 1 {A 2 (b 0 )} (4.12)
Chapter 4. Interferences management between M-MIMO cells

The rationale for (4.12) is the following: if Q b,b 0 = 1, the beams b and b 0 will be in collaboration. The benefit of a collaboration of a beam increases with the number of users the beam interferes, while it decreases with the number of users it serves. By imposing both conditions A 2 (b) and A 2 (b 0 ) we ensure that both cells will gain from the coordination and avoid a potential QoS loss in one of the cells (which is unacceptable for a mobile operator) even if the overall gain is positive.

The algorithm for the matrix generation is described in Algorithm 9.

Algorithm 9 Matrix Generation

Input: A Init: Q =[0] of the same dimension as A for all A b,b 0 = 1 do n i b |U i b |, n s b |U s b | n i b 0 |U i b 0 |, n s b 0 |U s b 0 | Q b,b 0 = A b,b 0 ⇥ 1 {A 2 (b)} ⇥ 1 {A 2 (b 0 )} end for
The usage of this matrix by the schedulers for interference management is described in the next section.

ANBR-based collaboration

This section introduces the MU collaborative scheduler that allows the management of interferences between two neighboring M-MIMO cells. Denote the rate of a user u 2 m at time t M+1 if scheduled by R b u,t M+1 , its average rate during a time window [t M T , t M ] by R b u,t M , and by d > 0 a positive number chosen as small as possible. k max denotes the maximum number of users that a BS can serve simultaneously. We denote by U candidates the set of users that can still be scheduled and by U k -the set of users already selected for scheduling. This section presents two types of collaboration: one in the time domain, called time domain ANBR, and the second in the frequency domain, called frequency domain ANBR.

Time domain ANBR

In the time domain, interference coordination is performed by muting interfering beams, one cell at a time. The MU collaborative scheduler works as follows. Consider the scheduling of the cell m (or m 0 ) described in Algorithm 10.

ANBR-based collaboration

Algorithm 10 Time ANBR-assisted MU Scheduler

Input : Q Init: U k = {}, U candidates = {u 2 m} if TTI is even then for all b for which Q b,b 0 = 1 do Remove u 2 b from U candidates end for end if while |U k | < k max or U candidates 6 = ∆ do u select argmax u2U candidates R b u,t M+1 R b u,t M +d U k U k [ u select Define N (u select )
the set of users attached to the beam of u select and to the adjacent beams

U candidates U candidates \N (u select ) end while
At the initialization phase, U candidates contains all the users attached to m (or m 0 ). The scheduler ranks the users of U candidates with respect to a PF criterion, namely

R b u,t M+1 R b u,t M +d
. The non-zero elements of Q indicate which pairs of beams should be consecutively muted. For all Q b,b 0 = 1, the beams b 2 B m are muted at an even TTI, whereas b 0 2 B m 0 -at an odd TTI. The users of the muted beam are removed from U candidates . The scheduler then selects the topranked candidate and removes it from the candidate list. The schedulers also remove from the set of candidates U candidates the users attached to either the same beam or the adjacent beams, to perform beam skipping, as described in Section 2.4.2. We repeat these operations until k max candidates are selected or until the set of candidates U candidates is empty.

Frequency domain ANBR

In the frequency domain, the schedulers receive from the dynamic ANBR function the dynamic beam relation matrix Q as well as the bandwidth attributed to its beam when in collaboration. We assume full frequency reuse in each cell. When coordination is applied, non-overlapping frequency bandwidth is allocated to the interfering beams, as shown in Figure 4.6.The scheduling is then performed following a MU PF scheduler while adapting the bandwidth used for the collaborating beams, as shown in

Input : Q Init: U k = {}, U candidates = {u 2 m} while |U k | < k max or U candidates 6 = ∆ do u select argmax u2U candidates R b u,t M+1 R b u,t M +d U k U k [ u select Define N (u select
) the set of users attached to the beam of u select and to the adjacent beams

U candidates U candidates \N (u select ) if 9b 0 2 B m 0 so that Q(b, b 0 ) = 1 then
Reduce bandwidth of u select end if end while

The next section presents the performance assessment of the ANBR-based interference coordination, introduced in this chapter, via numerical simulation.

Numerical results

Simulation scenario

Deployment scenario

Consider two neighboring M-MIMO cells in a tri-sectorial 5G networks. The network comprises 19 sectors (cells): a central sector and two tiers of 18 neighboring sectors (6 sites located on a hexagonal grid) surrounding it. The central sector and one of its direct neighbors denoted hereafter as cell 1 and cell 2 respectively implement the coordinated self-organizing scheduling (see Figure 4.7). The reference baseline solution does not implement coordination but a MU-PF scheduling with beam skipping, as presented in Section 2.4.2. The simulation parameters are summarized in Table 4.1.

Traffic description

The simulation follows a full buffer scenario where the number and location of the users are constant during the simulation and are presented in Figure 4.7. The distribution of users shows a hotspot at the border of the two cells implementing the MU collaborative scheduling based on ANBR. In Figure 4.7, a red rectangle surrounds the hotspot zone with high traffic located around the cell edge area of the two cells. Each cell has 35 users in the hotspot area and 10 users in the rest of the cell, geographically uniformly distributed in each zone. The full buffer model allos to assess the impact of the coordination approach in a static traffic scenario.

Parameters tuning for dynamic ANBR

An exhaustive search determines the best values for the thresholds h th and g th defined previously in (4.7) and (4.11). This search is achieved by defining a uniform grid of 10 ⇥ 10 points (h th ,g th ), with h th varying from 1/10 to 1 and g th -from 1 to 10. For each point of the grid we compute the MUT gain using the time-ANBR coordination scheme as depicted in Figure 4.8. The gain increases with the decrease in h th while the MUT gain is not sensitive to variations in g th except for small values. Too small value of g th results in too few mobiles that can benefit from the coordination between the two cells. Similarly, a small value of h th allows more beam relations to be included in the matrix Q and more users will participate in the coordinated scheduling. A gain of 105% in MUT with respect to the baseline is achieved on a plateau of 27 points, indicating little sensitivity of the thresholds (see Figure 4.8). The parameters defining conditions A 1 and A 2 in (4.7) and (4.11) respectively are h th = 1/3 and g th = 5dB. It is noted that g th takes into account interference from one beam only (thus explaining its relatively high value). On that note, the users' color code in Figure 4.7 is the following: Red squares correspond to users verifying the condition A 1 and sky blue and yellow squares -to the rest of the users attached to cell 1 and 2 respectively.

Performance analysis

This section analyzes the performance of the collaborative MU scheduling compared to a MU-PF scheduler. Figure 4.11 presents the MUT for the two cells for the baseline, in blue, and the collaborative scheduler based on frequency and time ANBR, in red and yellow respectively. The use of ANBR increases the performance of the system. Although time-ANBR outperforms slightly frequency-ANBR, both ANBR have similar results. One can see that the time domain coordinated solution increases MUT by 78 and 122 percents for cell 1 and cell 2 respectively. With the frequency domain collaboration scheme, the MUT is enhanced from 5.2 to 9, 4 Mbps and from 3.2 to 7.2 Mbps, for cell 1 and 2 respectively. The range of throughputs in the two cells are different due to the difference of location of the users and the distance of the users to their BS. Since most of the users are located near the cell edge (area located in the red rectangle in Figure 4.7), most of the users benefit from the coordinated scheduling. Frequency domain collaborating beams still interfere beams outside the collaborative scheme, accounting for the difference in performance between time domain and frequency domain collaboration. When a beam is in time domain collaboration, the time slots during which the beam is muted are given to a beam which is not in collaboration. Therefore, the shared time resources are re-used by the BS. On the other hand, in frequency domain collaboration, if a user uses less frequency bandwidth, the unused band can't be allocated to another user. However, as frequency domain collaboration does not require synchronicity between BSs, it is easier to implement and still efficient. Frequency-ANBR offers the advantage of not requiring any time synchronization between the two cells and can be easier to implement. It is important to note that in other scenarios with different traffic distributions, similar performance gain are expected mainly for the group of users near the cell edge or for highly interfered users, and not for all, which is the target of the proposed solution. We note as well that users that do not participate in the coordinated scheduling (e.g. non-red users in Figure 4.7) benefit from rate increase, as by muting some of the interfering beams, noncollaborating users experience interference. Moreover, non-collaborating beams have more opportunities to be scheduled when collaborating beams are muted. Figure 4.12 compares the user throughputs with and without collaboration based on frequency domain ANBR for each user of cell 1 located outside the hotspot area. Users with lowest throughputs are closed to the edge of the cell thus benefitting the most from the collaboration. As a result, their throughputs are significantly improved with the collaboration. Users with high throughputs are located closer to the BS and will benefit less from interference reduction, showing reduced gain or small loss if the benefit of the collaboration is not enough to counterbalance the sharing of their frequency band. However the loss is significantly lower than the gain of benefiting users. This concludes the performance analysis that enables to highlight the important gain achieved by the MU collaborative scheduler introduced in this chapter.

Conclusion and extensions

This chapter introduced the ANBR concept and its application to interference management of 5G M-MIMO cells. We have demonstrated how a combination of static and dynamic ANBR can be used to coordinate the MU schedulers of a pair of neighboring cells with M-MIMO deployment. The strong interference at cell edge motivates the coordination approach.

Static ANBR offers a static beam relation matrix that accounts for the probability of interference between any pairs beams of two neighboring relation. A dynamic beam relation matrix is generated from the static ANBR matrix along with the traffic conditions and is used to consecutively mute certain interfering beams in the two cells. The beam relations are updated at the time scale of arrivals and departures of users, namely in the order of a second, thus making this approach much more attractive than traditional techniques such as CoMP which operates at a millisecond time scale.

In the use case analyzed, very significant gains are achieved. Greatest gains are expected for groups of users located at highly interfered zone, typically close to cell edge. The solution is applied to GoB M-MIMO technique, and is applicable to both time domain (TDD) and frequency domain (FDD) technology. It can be generalized also to other beamforming solutions such as eigen-based beamforming.

The solution proposed in this chapter is heuristic-based, having the merit of being very low complexity. A generalization of the solution to a technique based on Machine Learning is presented in Chapter 5.

Chapter 5

Application of MAB to inter-cell interference management

Introduction

Chapter 4 describes a heuristic low cost approach for interference coordination between M-MIMO cells based on the ANBR concept. The significant gain achieved is made possible by exploiting information at beam level resolution. This chapter investigates the use of Multi-Armed-Bandit (MAB) for the interference coordination problem, with the objective of achieving higher gains to those achieved with the heuristic ANBR-based approach.

The MAB introduced in Section 2.5 offers several advantages that motivate its usage in interference mitigation such as a low complexity which eases the integration within operational system. Moreover, MABs converge rapidly and require no training phase. Several MABs can also easily be deployed in parallel, making our solution easily scalable.

MAB shows the particularity of being stateless, as mentioned in Chapter 2. This limits the type of use cases that can be treated. However, interesting use cases have been successfully investigated in the literature. For instance, [START_REF] Daher | Cognitive management of self-organized radio networks based on multi armed bandit[END_REF] presents the integration of MAB for SON management. The adaptation of MAB to machine type communications through sleeping MAB is studied in [START_REF] Ali | Sleeping multi-armed bandits for fast uplink grant allocation in machine type communications[END_REF]. Finally, the reliability of the millimeter wave (mm-wave) is improved through the use of MAB in [START_REF] Rahman | A Beam-Switching Scheme for Resilient mm-Wave Communications With Dynamic Link Blockages[END_REF][START_REF] Aykin | MAMBA: A multi-armed bandit framework for beam tracking in millimeter-wave systems[END_REF][START_REF] Matthew | Multi-armed bandit beam alignment and tracking for mobile millimeter wave communications[END_REF].

In the MAB-based Interference Control (MIC) solution, we employ MABs that decide whether sets of interfering beams from neighboring cells should collaborate. The MABs send the decision to the scheduler of the two cells and receive a reward in exchange, as illustrated by Figure 5.1.

MIC solution

This section first focuses on the MAB algorithm and then addresses its implementation in the MIC framework.

MAB algorithm adaptation for MIC

The dynamicity of the traffic, related to arrival and departure of users or mobility, needs to be taken into account by the MAB algorithm. To this end, the MAB needs to consider input during a limited history. This can be done through two methods, namely the reset MAB or a MAB with sliding windows. For sake of clarity, we consider a MAB dedicated for a given set of collaborating beams B MAB from two neighboring cells m 1 and m 2 . The role of the MAB is to instruct the schedulers of m 1 and m 2 whether or not to coordinate the set of beams B MAB . The choice of rewards for the MAB will determine different implementations of MAB and will be addressed in the sequel.

Reset MAB

The theoretical background of this algorithm is presented in Section 2.5. The reset-MAB takes decision according to UCB algorithm but is periodically reset. The goal is to reset the MAB to constraint it to adapt to frequent changes in the reward distribution. The reset parameter is hereafter denoted by T reset . After T reset decisions, the parameters of the MAB are re-initialized, MAB explores each arm at least once. We consider a MAB with two arms, j = 1, 2, corresponding to the following two decisions: '1' stands for 'coordination'and '2' -for 'no coordination'. Denote by n j the number of time the arm j has been chosen, by T exp the number of iterations of the experiment and by x j and X j respectively the reward and cumulative reward received by the arm j. Algorithm 12 presents the reset-MAB algorithm.

Algorithm 12

Reset MAB Algorithm in MIC context n = 0, x j = 0, X j = 0, n j = 0; j = 1, 2 with 1 for 'coordination', 2 for 'no coordination' for t = 1, 2 do Send t to schedulers of cell m 1 and m 2 Receive reward x t from schedulers of cell m 1 and m 2 n j = 1, n = n + 1 x t = x t end for for t = 3 to T exp do j = argmax j=1,2 (x j + r 2 ln(n) n j ) Send j to the schedulers of cell m 1 and m 2 Receive reward x j from schedulers of cell m 1 and m 2 X j = X j + x j n j = n j + 1, n = n + 1

x j = X j n j if mod (n, T reset ) == 0 then n = 0, x j = 0, X j = 0, n j = 0; j = 1, 2 end if end for

MAB with Sliding-Window (SW)

The MAB with sliding window adapts smoothly to changes in the network as the SW-MAB only considers the W last rewards received by an arm, with W the size of the sliding windows. The Algorithm 13 presents the SW-MAB used for the MIC solution.

Algorithm 13 SW-MAB Algorithm in MIC context x j = 0, X j = 0, n j = 0 with j = 1, 2 with 1 for 'coordination', 2 for 'no coordination' for t = 1, 2 do Send t to schedulers of cell m 1 and m 2 Receive reward x t from schedulers of cell m 1 and m 2 n j = 1, n = n + 1 x t = x t end for for t = 3 to T exp do j = argmax j=1,2 (x j + r 2 ln(n) n j ) Send j to schedulers of cell m 1 and m 2 Receive reward x j from schedulers of cell m 1 and m 2 n j = max(n j + 1, W), n = n + 1

x j = Â n s=n W+1 x j,s n j end for

Implementation of MICs

This section considers deployment type of the MAB, namely centralized vs distributed, and the type of reward used by the MAB.

MAB distribution

The static ANBR, presented in Section 4.4.1, identifies groups of beams from the neighboring cells that can potentially benefit from coordination. In the example shown in Figure 5.2, we can identify 3 sets, namely A, B and C. These sets are neither overlapping nor contiguous. Two MAB distributions are possible:

• Centralized implementation In a centralized implementation, a single MAB will be deployed to supervise the 2 cells and will decide the collaboration of sets A, B and C. The trhee sets are thus considered as a single collaborating set of beams. It decreases the complexity of MIC solution.

• Distributed implementation

In this implementation, a MAB will be implemented per set of beams of the neighboring cells. In the case presented in Figure 5.2, a distinct MAB will consider the beams for each of the sets A, B and C. This implementation offers a finer granularity and better performance, as shown in Section 5.3.

Rewards sent by the MU-schedulers

The MAB sends its decision and the set of involved beams B MAB to the schedulers (See Figure 5.1). The scheduler applies the decision and then has to send back a reward to the MAB, as described in Algorithm 14. The exploitation term of the MAB function, see (2.24), depends on the choice of rewards. The reward should reflect the impact of the coordination or non coordination on the performance of the network. The reward chosen is the MUT averaged over a period of time T reward for the selected set of users, which can consists in either:

• The users attached to the beams belonging to B MAB , namely a local reward.

• All users of the cells involved in the coordination, namely a global reward.

Global reward means that the overall performance are taken into account, namely the aim is not only to improve the performance for the beam involved in the collaboration but for the two cells in their whole. However this leads Chapter 5. Application of MAB to inter-cell interference management to a noisier system and can downgrade the performance. Local reward takes only the local situation and can miss phenomena linked to the collaboration. The performances of both configurations are discussed in Section 5.3.

The period T reward should be long enough to taking into account the consequences of the decision of the MAB in the learning process but short enough to enable the MAB to learn from the reward and take new decisions before the trafic distribution changes. Denote by U k the set of at most k max users scheduled by the MU-scheduler and U candidates = {u 2 m} the set of users candidate for scheduling. We denote by a j the decision of a MAB j, by U b the set of users attached to a beam b. B j,m 1 and B j,m 2 denote the set of beams controlled by the MAB j and belonging respectively to m 1 and m 2 . Algorithm 14 shows how the scheduler of cell m 1 takes into account the coordination decisions of the distributed MABs, namely the incorporation of the muting decisions to the coordinated cells m 1 andm 2 . Note that the cell m 2 applies the same scheduling algorithm but coordination is applied only at odd TTIs. When the scheduler receives the decision a j from a MAB j, it applies the collaboration decision until the next reception of a decision and sets a counter t j . After T reward TTIs, the counter t j reaches 0 and the reward x j for j is computed and is sent to the MIC for the MAB j to process. In this use case, there is an hotspot in all the three couples of interfering and collaborating beams. Hence both centralized and distributed MABs take identical collaboration (muting) decisions. It will be shown that in certain conditions the distributed MAB deployment will outperform the centralized one.

Performance analysis of hotspot deployment scenario

• Local versus global reward

Global reward introduces some noise from the non-collaborating users, but have negligible impact on the MAB decisions. The loads in both cells is shown in Figure 5.7 for the ANBR-based, the MIC and the baseline solutions. The load of a cell is estimated as the proportion Chapter 5. Application of MAB to inter-cell interference management of time during which at least one user is attached to the cell. The load decreases in cell 1 from 0.88 for the baseline to 0.73 for the ANBR solution, and to 0.56 for the MIC-based solution. The lower load is a direct result of the improvement of user throughputs, that allows the user to download files and leave the system faster. Similarly in cell 2, we note that the load is 50% higherfor the baseline solution than that of the MAB implementation. Note that the users' location with respect to the beams and the cell border explain the difference of performance between both cells.

Analysis of the impact of the type of deployment and rewards

The traffic scenario presented in Figure 5.8 enables to observe and analyse the impact of the configuration of the type of reward,i.e. global or local reward, and the type of deployment of the MAB, i.e. distributed or centralized. The traffic distribution presented in Figure 5.8 aims at outlining the differences between the different configurations. There are two hotspots respectively at the edge of beams 9 of cell 1 and 16 of cell 2 and in the beam 14 of cell 1, near the BS. The hotspot areas are framed in red in the figure. As in the previous scenario, colored dot indicates the couple of beams in the collaboration scheme. In the case where collaboration is implied, the beams 9 of cell 1 and 16 of cell 2 and the beams 14 of cell 1 and 11 of cell 2 form distinct collaborating sets. Areas with no traffic have been removed from the figure. Since users of beam 14 experience reduced interference from beam 11, thanks to the absence of traffic at their border, one could expect limited collaboration gain. This scenario examplifies the case where the distributed MAB solution is advantageous with respect to the centralized MAB one.The time window over which the reward is calculated is set to 10 TTIs,the windows size is set to 10 for the SW-MAB and the reset period to 10 for the reset MAB. It is noted that as the traffic distribution remains the same throughout the simulation, the window size and the reset period have low impact on the results. The arrival rate is 6 users per second and the mean file size is 6 Mbits.

Impact of the type of rewards

The comparison between MIC using local and global rewards is considered next. Figure 5.10 presents the MUT for the two different rewards' choice together with the baseline solution. As we can see, the global reward solution outperform slightly the local one for the MUT averaged over both cells for the duration of the experiment. However, the difference of performance is as small as previously mentioned.

Conclusion

This chapter presents an alternative to the solution for the interference management for 5G M-MIMO cells presented in Chapters 4 relying on ML. The use of MAB techniques, and particularly the Reset-MAB and SW-MAB allows to dynamically learn optimal coordination decisions. Typical time for the MAB to take a new decision (i.e. a MAB algorithm step) is of the order of 10 ms. Hence after a few tens of milliseconds the MAB reaches a good decision, which allows it to adapt to traffic variations. We have presented different implementations of the MAB algorithm, namely centralized (with a single MAB) and distributed (with a distinct MAB per set of interfering beams), with local and global rewards. In certain cases, the centralized and distributed solutions provide the same results, namely when the sets of beams to be coordinated are adjacent. In other cases, the candidate beams for coordination may be separated in distinct groups. In such cases, the distributed MAB implementation can outperform the centralized one and is therefore recommended. For a given set of beams, a centralized or several distributed-MABs will regularly opt for the collaboration or the non-collaboration of these beams. Experiences shows in Section 5.3 that distributed-MAB achieved better performances because of finer granularity than centralized-MAB deployment. The comparison between global and local rewards for the MAB show slight performance improvement using the global ones. The indirect coupling between the coordinated and the noncoordinated users via the scheduler can explain such a result. Non-coordinated users will see their performance improved when the coordinated ones achieve higher throughputs and thus leave the network quicker. Finally, by introducing MAB-based learning in the coordination decisions, better decisions are made when comparing to the ANBR-based solution, which is a heuristic solution.

Chapter 6

REM-based interference management 6.1 Introduction

Chapter 4 and 5 introduce a solution for inter-cell interference management for 5G M-MIMO networks. Both ANBR-and MIC-based solutions offer a spatial resolution at a beam level. Exploitation of users' geolocation data provides finer granularity and allows to enhance the performance of the previous solutions. Geolocation data is retrieved through Global Positioning Systems (GPS) or using network-assisted solutions [START_REF] Kanhere | Position location for futuristic cellular communications: 5G and beyond[END_REF][START_REF] Bartoletti | 5G localization and context-awareness[END_REF]. 5G new technology allows a precision up to the meter for outdoor and to the centimeter for indoor. Radio Environment Map (REM) [START_REF] Zhao | Overhead analysis for radio environment map enabled cognitive radio networks[END_REF] is a spatial representation of key metrics constructed from field measurements and spatial interpolation. It allows from the positioning of a UE to retrieve the expected value of the metric at this location. Our aim is to exploit REM to offer an enhanced MU-MIMO coordination solution to mitigate inter-cell interference, as illustrated by Figure 6.1. The MU-collaborative scheduler uses the location of the users in the network to retrieve their RSRP from their serving and interfering beams. This knowledge enables the schedulers to choose a set of users for scheduling that minimizes the impact of interference. U candidates and U 0 candidates are still not empty and the two sets of already selected UEs for scheduling, U k and U 0 k 0 , have not yet reached the maximum size of k max . In the second case, only a UE u 0 attached to m 0 can be selected for scheduling. U 0 candidates is not yet empty and U 0 k 0 has less than k max UEs, while U candidates is empty or U k is of size k max . Here, after updating U 0 k 0 , we only perform the beam skipping technique and the constraint of one UE per beam. The third case is symmetric to the second one with the roles of u 0 and u interchanged.

Numerical results

Simulation scenario

Consider the two neighboring macro cells m and m 0 with M-MIMO deployment. Full buffer traffic model is considered. The traffic is distributed uniformly in two areas within the cells' coverage as shown in Figure 6.2. A hotspot area close to the cell-edge of the two macro cells is enclosed by a green line, with 35 UEs attached to each cell and presented by small circles. Two areas outside the hotspot area and closer to the BSs are enclosed by a black line with 10 UEs each, presented by small triangles. 45 UEs are attached to each BS m and m 0 , colored in blue and black, respectively. The simulation parameters for the network, channel and traffic characteristics are listed in Table 6.1. We assume that the REMs have been provided for each beam.

Performance analysis

The implementation of the coordination algorithm requires to set the threshold c th which is determined numerically using a simple exhaustive search. Figure 6.4 shows the Mean User Throughput (MUT) calculated for different values of c th . A maximum value is obtained for c th = 8 which is selected as the threshold for all the numerical experiments. Next, we investigate how the REM based coordination benefits UEs inside and outside the hotspot area. To this end, we compare in Figure 6.5 the MUT results obtained in each area using REM and the baseline solution. The performance gain inside and outside the hotspot area is of about 190 and 20 percent. Little coordination is expected outside the hotspot area, where interference level is relatively low with respect to that close to the cell edge at the hotspot area. However, the coordinated MU-scheduling mechanism switches off interfering beams at certain TTIs that benefits non-coordinated UEs further away from the cell edge.

Conclusion

This chapter has introduced a low complexity solution for interference coordination between BSs with M-MIMO deployment and GoB implementation. The coordination is performed at each BS by the MU-scheduler that utilizes REMs to derive the coordination decision. The REMs provide the RSRP maps for the GoB of the serving cell and the neighboring interfering cells. The quality of the REMs in terms of resolution and precision is critical for the derivation of correct coordination decisions. To this end, Kriging with the Covariance Tapering spatial interpolation has been used that provides good prediction accuracy and computational complexity. The REM based coordination achieves around 100 percent MUT gain with respect to the baseline solution without coordination. The high resolution information provided by the REMs has shown its usefulness when comparing the solution with the one obtained using beam level resolution, which achieves around 50 percent lower MUT. However, the three solutions presented in this thesis are complementary. The ANBR-based solution offers a lower complexity solution than the two others whereas the REM-based solution takes advantages of the geolocation data. Eventually, MIC-based solution shows better performances and does not require geolocation data.

Lastly, the REM based coordination solution can be adapted to other M-MIMO implementations. If the GoB is used only for the control channels, while data channels rely on eigen based beamforming, the REMs can be constructed using UE measurements reported over SSB beams to which they are attached.

Chapter 7 Conclusion

This chapter summarizes the contributions of the thesis and open the discussion for future work and extensions.

Contributions of the thesis

The first contribution aims at managing the interference in a 5G heterogeneous network. M-MIMO uses beamforming focusing the signal to achieve higher throughput. In a heterogeneous cell, this triggers high interference with the small cells located within the cell edge region. e-ICIC can be implemented as a SON function that provides a solution to interference in heterogeneous cells in 4G. In the context of 5G with MU-MIMO, the extension of e-ICIC is inefficient. A low cost coordination mechanism between the small cells and the interfering macro-cell is proposed based on beam muting. Beam muting means that the beams will not serve their users during a TTI or more. The loss of capacity is reduced thanks to MU-scheduling. The trade-off lies between the gain of not muting a beam and the loss occurring at the small cells due to the scheduling of their interfering beams. Thus the collaborative MU-scheduler can take into account the loss of the small cells when scheduling. This leads to great gains for the whole heterogeneous network and especially on the small cells' side.

We next investigate the management of inter-cell interference in 5G M-MIMO networks through three approaches, namely ANBR-,MIC-and REM-based solutionS. ANBR deals with high interference between neighboring macrocells. This solution consists in two modules. The first ANBR module, namely static ANBR, is located in the management plane and generates the static ANBR matrix using measurements over a long period (i.e. days). The second module, namely the dynamic ANBR, uses actual traffic conditions to select from the static matrix the beams which are likely to benefit from interference coordination. The coordination scheme is updated at each event, i.e. each departure or arrival of a user in the network. The dynamic ANBR identifies and sends the indices of beams to the schedulers for the coordination. The beams in coordination will be muted every two TTIs but the benefits from Chapter 7. Conclusion the interference reduction exceed the impact of muting. The impact of muting beams is lessened by the use of MU. This approach is a heuristic-based solution, in which thresholds are set using off-line exhaustive search. The approach provides significant performance gains, assessed through simulations that show that not only users within collaborating beams but the whole cells are gaining from the application of the ANBR-based coordination.

The ANBR approach takes decisions using information at beam level resolution. The decisions to coordinate are based on conditions using fixed optimized thresholds for all the beams of the GoB. The second solution tries to improve the decisions using Machine Learning, namely through MAB algorithms. The decisions are based on information at beam level, but are optimized online using the MAB, to offer a more dynamic system and lessen the risk of non-adapted parameters. Different MAB algorithms have been tested with different implementations: centralized versus distributed implementations, local versus global reward and SW-versus reset-MAB. For a set of beams from two neighboring cells, the MAB decides whether these beams should collaborate or not and sends its decision to the schedulers of both cell that apply the collaboration when required. In exchange, the schedulers send back a reward that allows the MAB to quickly learn the best decision. The MAB algorithms implemented in our work are the reset-and the SW-MAB that both offers a dynamic adaptation to the change of contexts that occur in the network. MAB algorithms applied to interference management offers significant enhancement as shown in Section 5.3. Distributed solution is recommended while global reward provides slight performance improvement with respect to the local one.

The last approach to tackle the inter-cell interference problem is based on the exploitation of geolocalized data. REM-based collaborative scheduling benefits from geolocation data of UEs along with the information provided by the REMs to optimize the scheduling of neighboring 5G M-MIMO cells. REMs for each beam of the GoBs are utilized. They are estimated from measurements through Kriging technique with covariance tapering and provide the RSRPs of each beam with a granularity of 1x1 m. This finer granularity allows to improve the criterion for collaboration and increases significantly the performances of the network. The coordination is performed at a userlevel rather than at a beam-level, through the comparison of the estimated SIR of users to a threshold, determined by an exhaustive search performed once. The results achieved by the REM based coordination outperforms significantly the ANBR approach, thanks to the finer granularity of information. The REM-based solution shows slightly higher results than the MIC-based approach.

7.2. Future works and perspectives 107

Future works and perspectives

In this thesis we have addressed the problem of interference coordination in the context of M-MIMO deployment. We have proposed different solutions applied to scenarios involving two coordinating cells surrounded by interfering neighboring cells. The problem can be generalized to several cells implementing coordination, and specific adaptation of the algorithms will be needed. One can expect a small number of beams with significant interference to participate in the interference coordination involving more than two cells. Efficient selection method of the interfering beams needs to be developed.

The interference coordination solutions has been applied to GoB type of M-MIMO. A challenging problem is to extend the work to non-GoB technology, namely to eigen beamforming based on channel reciprocity. It is recalled that eigen beamforming is applied to data channels whereas control channels can use GoB. In the REM-based solution for example, one has to figure out how can the REM be used to assess localized interference from beams generated using eigen beamforming transmissions.

In the solutions for interference coordination based on ANBR and on the MIC solutions, the coordination beams are muted sequentially, on even and odd TTIs. The coordinating beams can involve different number of beams from the neighboring cells implementing the interference coordination. An interesting problem that is left for investigation to optimize the muting sequence of beams taking into account the number of beams from each cell in the collaborating set and the corresponding served traffic.

Lastly, the concept of ANBR and Machine Learning can be used to different problems involving M-MIMO. Examples are inter beam intra-and inter cell handover optimization. The objective of such optimization can be related to improving the mobility procedure itself, similarly to mobility robustness optimization (MRO) in 4G networks, or to reduce beam failure by better balancing the load between beams.

3. 1

 1 enhanced Inter-Cell Interference Coordination (e-ICIC) in time domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 e-ICIC in frequency domain . . . . . . . . . . . . . . . . . . . . 3.3 Heterogeneous cell with beamforming . . . . . . . . . . . . . . 3.4 Macro cell and Small Cells (SCs) coverage map . . . . . . . . . 3.5 Mean User Throughput (MUT) in the macro cell and its attached small cells . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 MUT in small cell 5 for l s = 3.4 . . . . . . . . . . . . . . . . . . 3.7 Cell-Edge Throughput (CET) in small cell 5 . . . . . . . . . . . 3.8 CET in Small Cell 5 for l s = 3.4 . . . . . . . . . . . . . . . . . . 3.9 Jain Fairness Index . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Interfering M-MIMO cells . . . . . . . . . . . . . . . . . . . . . 4.2 Two interfering M-MIMO cells . . . . . . . . . . . . . . . . . . 4.3 Users position in overlapping beams . . . . . . . . . . . . . . . 4.4 Implementation of ANBR in 3GPP architecture . . . . . . . . . 4.5 Beam relation matrix illustration . . . . . . . . . . . . . . . . . 4.6 Frequency collaboration . . . . . . . . . . . . . . . . . . . . . . 4.7 Traffic map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8 Exhaustive search for the tuning of the parameters g th and hth xxiv 4.9 Users served and interfered per beam in cell 1 . . . . . . . . . 4.10 Users served and interfered per beam in cell 2 . . . . . . . . . 4.11 MUT comparison . . . . . . . . . . . . . . . . . . . . . . . . . . 4.12 Users outside hotspot zone in cell 1 . . . . . . . . . . . . . . . . 4.13 Users in hotspot zone in cell 2 . . . . . . . . . . . . . . . . . . . 4.14 Users outside hotspot zone in cell 2 . . . . . . . . . . . . . . . . 4.15 Users in hotspot zone in cell 2 . . . . . . . . . . . . . . . . . . . 5.1 MAB interaction with the scheduler . . . . . . . . . . . . . . . 5.2 Centralized and distributed MAB deployment . . . . . . . . . 5.3 Two adjacent cells implementing GoB . . . . . . . . . . . . . . 5.4 Map with hotspot scenario . . . . . . . . . . . . . . . . . . . . . 5.5 Mean user throughput for cell 1 and cell 2 . . . . . . . . . . . 5.6 Mean user throughput for cell 1 and cell 2 . . . . . . . . . . . 5.7 Load for cell 1 and cell 2 . . . . . . . . . . . . . . . . . . . . . . 5.8 Traffic scenario for deployment analysis . . . . . . . . . . . . . 5.9 Mean user throughput comparison between centralized-and distributed-MIC solutions . . . . . . . . . . . . . . . . . . . . . 5.10 Mean user throughput comparison between MIC solutions with local and global rewards . . . . . . . . . . . . . . . . . . . . . . 6.1 REM-based collaborative scheduler . . . . . . . . . . . . . . . . 6.2 Traffic distribution of UEs in cells m and m 0 . . . . . . . . . . . . 6.3 Graphic representation of REM . . . . . . . . . . . . . . . . . . 6.4 MUT as a function of the threshold c th . . . . . . . . . . . . . . 6.5 MUT inside and outside the hotspot areas using REM and the baseline solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 MUT performance for ANBR-, REM-and MIC-based scheduler xxv List of Tables 3.1 Network and Traffic characteristics . . . . . . . . . . . . . . . . 3.2 Channel characteristic . . . . . . . . . . . . . . . . . . . . . . . 3.3 Traffic characteristic . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Network characteristics . . . . . . . . . . . . . . . . . . . . . . 4.2 Channel characteristics . . . . . . . . . . . . . . . . . . . . . . . 4.3 Traffic characteristics . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Network characteristics . . . . . . . . . . . . . . . . . . . . . . 5.2 Channel characteristics . . . . . . . . . . . . . . . . . . . . . . . 5.3 MAB's parameter configurations . . . . . . . . . . . . . . . . . 6.1 Network parameters and characteristics of both channel and traffic. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii Dedicated to Marie-Victoire Charpentier, this thesis won't change the Parisian landscape, but I sure hope it makes you proud.

FIGURE 1 . 1 :

 11 FIGURE 1.1: Services in 5G network

FIGURE 1 . 2 :

 12 FIGURE 1.2: Radio Intelligent Controllers for RAN management

1. 1 .

 1 Context of the work need to switch beams often. Synchronization Signal Blocks (SSBs) are used for cell search. These blocks are broadcasted by the BS and represent 20 resource blocks in frequency and 4 symbols in time. A SS/PBCH block consists in PBCH, a Primary PSS (PSS) and a Secondary SS (SSS), as shown in Figure 1.3.

FIGURE 1 . 3 :

 13 FIGURE 1.3: Structure of a SSB

FIGURE 1 . 4 :

 14 FIGURE 1.4: Synchronization burst

1. 1 . 9 Frame

 19 Context of the work Number (SFN)

FIGURE 1 . 5 :

 15 FIGURE 1.5: OFDMA resource structure

Chapter 1 .

 1 Introduction• Downlink resource allocation type 0:

FIGURE 1 . 6 :

 16 FIGURE 1.6: Bitmap sent by the BS to the UE

FIGURE 2 . 1 :

 21 FIGURE 2.1: Antenna model

FIGURE 2 . 2 :

 22 FIGURE 2.2: Coordinates of a beam direction

FIGURE 2 . 4 :FIGURE 2 . 5 :

 2425 FIGURE 2.4: Gain pattern for 3 adjacent beams of the GoB using linear array

FIGURE

  FIGURE 2.6: MAB interaction with the environnement

FIGURE 2 . 8 :

 28 FIGURE 2.8: Sliding window strategy

FIGURE 2 . 9 :

 29 FIGURE 2.9: MAB with reset

FIGURE 2 .

 2 FIGURE 2.10: REM construction

FIGURE 3 .

 3 FIGURE 3.1: e-ICIC in time domain

FIGURE 3 . 3 :

 33 FIGURE 3.3: Heterogeneous cell with beamforming

23 )

 23 When a beam b is scheduled, it increases the global utility U global defined in (3.6) by d b . We call d b the gain of beam b and define it in (3.24).

  Init: B k max = {}, B candidates = {b 2 B|9u 2 b} for all s 2 S do Compute loss factor d s using (3.25) Send d s to m end for for all b 2 B candidates do Compute gain factor d b using (3.24) if 9s 2 S interfered by a beam b then D b d b + d s else D b d b end if end for while |B k max | < k max and B candidate 6 = ∆ do b select argmax b2B candidates D b B k max B k max [ b select Define N (b select ) the set containing b select and its adjacent beams B candidates B candidates \N (b select ) end while

FIGURE 3 . 4 :

 34 FIGURE 3.4: Macro cell and SCs coverage map

FIGURE 3 . 5 :

 35 FIGURE 3.5: MUT in the macro cell and its attached small cells

FIGURE 3 . 6 : 4 Chapter 3 .

 3643 FIGURE 3.6: MUT in small cell 5 for l s = 3.4

FIGURE 3 . 7 :FIGURE 3 . 8 :

 3738 FIGURE 3.7: CET in small cell 5

FIGURE 3 . 9 : 54 Chapter 3 .

 39543 FIGURE 3.9: Jain Fairness Index

  Let us consider two M-MIMO cells denoted by m and m 0 each having a GoB denoted respectively by B m and B m 0 . The two cells are presented in Figure 4.1. The color code aims at differentiating the beams of the GoBs of the two cells and does not indicate the strength of the signal or any other metric.

FIGURE 4 . 1 :

 41 FIGURE 4.1: Interfering M-MIMO cells

  by C m,b u the useful signal received power of user u from beam b of BS m, by d u,m -the distance between m and u, and by s 2 -the noise power. C m,b u can be written as a function of the channel gain h m b u (u) as follows in (4.1), C m,b u = p m |h m b (u)| 2 . (4.1)

Figure 4 .

 4 2 illustrates this configuration in a simplistic manner.

FIGURE 4 . 2 :

 42 FIGURE 4.2: Two interfering M-MIMO cells

FIGURE 4 . 3 :

 43 FIGURE 4.3: Users position in overlapping beams

Figure 4 .

 4 Figure 4.4 presents an architecture in which our solution can be deployed.In this figure, we can observe the following entities:

FIGURE 4 . 4 :

 44 FIGURE 4.4: Implementation of ANBR in 3GPP architecture

4. 4 . 1

 41 Static ANBR Static ANBR is either deployed as a C-SON function in a 3GPP architecture or as an application called r-App located in the Non-RT-RIC in the O-RAN architecture. Static ANBR provides a static beam matrix relation A m,m 0 for any two cells m and m 0 , with elements A m,m 0 b,b 0 . For simplicity of notations we omit the superscripts m and m 0 from the matrix A in the rest of the chapter. Each element A b,b 0 corresponds to a couple of beam b and b 0 belong respectively to the BSs m and m 0 . A b,b 0 = 1 if the beam b of cell m and beam b 0 of cell m 0 are likely interfere each other, and 0 otherwise.

Figure 4 .

 4 Figure 4.5 illustrates the construction of a static beam relation matrix for the simplified network given in Figure 4.2. In the case of Figure 4.5, each column represents a beam b i of m and each row represents a beam b 0 i of m 0 . Based on the figure of the beam, we observe that the couple of beams with high probability of interference is the couple (b 2 , b 02 ). Therefore, the matrix component corresponding to this couple has for value 1 while the other components take the value 0.

FIGURE 4 . 5 :

 45 FIGURE 4.5: Beam relation matrix illustration

Figure 4 . 3 , n i b and n i b 0 are equal to 1 while n s b and n s b 0

 430 are equal to 2.

Figure 4 . 6 .

 46 Algorithm 11 presents this solution.

FIGURE 4 . 6 :

 46 FIGURE 4.6: Frequency collaboration

FIGURE 4 . 7 :

 47 FIGURE 4.7: Traffic map

FIGURE 4 . 8 :

 48 FIGURE 4.8: Exhaustive search for the tuning of the parameters g th and hth

Figure 4 .

 4 Figure 4.9 depicts the distribution of the served -and interfered users per beam for cell 1, as presented in Section 4.4.2 The number of users attached to-and served by each beam is indicated in blue while the number of users of cell 2 that are being interfered by the beam is shown in red. The ratio of these two values is used in the algorithm of the dynamic ANBR.

FIGURE 4 . 9 :

 49 FIGURE 4.9: Users served and interfered per beam in cell 1

Figure 4 .

 4 Figure 4.10 shows the same categorization for cell 2. One can see that cell 2 has more interfered users which is due to the users' location with respect to the beams and the cell border.

FIGURE 4 . 10 :

 410 FIGURE 4.10: Users served and interfered per beam in cell 2

FIGURE 4 .

 4 FIGURE 4.11: MUT comparison

FIGURE 4 . 12 :

 412 FIGURE 4.12: Users outside hotspot zone in cell 1

Figure 4 .

 4 Figure 4.13 presents the user throughputs for the users of cell 1 located inside the hotspot area. In this location, every user gains from the collaboration. The gains are higher in the hotspot than for users located outside but the throughputs remains lower in this area, as the users are on the edge of the cell and receives lower signals.

FIGURE 4 . 14 :

 414 FIGURE 4.14: Users outside hotspot zone in cell 2

Figure 4 .

 4 Figure 4.15 presents the performance of the users located in the hotspot of cell 2. The results are similar to those of the hotspot users of cell 1.

FIGURE 4 . 15 :

 415 FIGURE 4.15: Users in hotspot zone in cell 2

FIGURE 5 . 1 :

 51 FIGURE 5.1: MAB interaction with the scheduler

FIGURE 5 . 2 :

 52 FIGURE 5.2: Centralized and distributed MAB deployment

Figure 5 .

 5 Figure 5.4 presents the first traffic scenario in which a hotspot is located at the border between the two cells, identified by the red rectangle in Figure 5.4. The arrival rate of users is equal to 5 users per second per cell, with 80% of the traffic located in the hotspot area and the remaining 20 % being uniformly distributed in the cell. Users arrive, download a file with mean size of 6 Mbits and leave the network. The beams are numbered in the figure for sake of clarity.

FIGURE 5 . 4 :

 54 FIGURE 5.4: Map with hotspot scenario

Figure 5 .

 5 Figure 5.6 presents the MUT in both cells m 1 and m 2 for the different interference management solutions. TheANBR solution (in red) achieves better results than the baseline (in blue), with 83% increase in MUT for cell 1 compared to the baseline solution, from around 12 to around 22 Mbps. The MIC solution (in yellow) outperforms both solutions with up to 125% gain with respect to the baseline solution, namely 27 Mbps in cell 1. Cell 2 shows the same tendency. The baseline solution achieves a MUT of 15 Mbps compared to 21 for the ANBR solution and 25 Mbps for the MIC implementation.

FIGURE 5 . 6 :

 56 FIGURE 5.6: Mean user throughput for cell 1 and cell 2

FIGURE 5 . 7 :

 57 FIGURE 5.7: Load for cell 1 and cell 2

FIGURE 5 . 8 :

 58 FIGURE 5.8: Traffic scenario for deployment analysis

Figure 5 .

 5 9 presents the MUT in both cells for the baseline, the centralized-and the distributed-implementations. Both MAB-based solutions outperform the baseline solution. The MUT in cell 1 increases for instance from 51.5 to 55.8 Mbps for the centralized-MAB-based solution and 62.9 Mbps for the distributed-MAB-based solution. The distributed implementation offers a finer granularity and enables to avoid an all-or-nothing coordination scheme. While centralized implementation enables to lower the complexity of the solution, the deployment of a distributed MIC solution ensures better performance.

FIGURE 5 . 9 :

 59 FIGURE 5.9: Mean user throughput comparison between centralized-and distributed-MIC solutions

FIGURE 5 . 10 :

 510 FIGURE 5.10: Mean user throughput comparison between MIC solutions with local and global rewards

FIGURE 6 . 1 :

 61 FIGURE 6.1: REM-based collaborative scheduler

Figure 6

 6 .3 show an example of REMs for two given beams.

FIGURE 6 . 2 :FIGURE 6 . 3 :

 6263 FIGURE 6.2: Traffic distribution of UEs in cells m and m 0 .

FIGURE 6 . 5 :

 65 FIGURE 6.5: MUT inside and outside the hotspot areas using REM and the baseline solutions.

Figure 6 .

 6 Figure 6.6 presents the MUT for both cells with theANBR-the MIC-and the REM-based scheduler. We can see that the MIC solution offers similar performance to the REM-based collaborative solution. The gap between the two performance is accounted for the exploration phase required by the MAB. However the MAB presents the advantage to offer the same order of achievement while not requiring geolocation data nor the availability of REMs.

FIGURE 6 . 6 :

 66 FIGURE 6.6: MUT performance for ANBR-, REM-and MICbased scheduler

  profiter d'une granularité plus fine. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 REM based MU-MIMO coordinated scheduling . . . . . . . . 6.3 Numerical results . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.1 Simulation scenario . . . . . . . . . . . . . . . . . . . . . 6.3.2 Performance analysis . . . . . . . . . . . . . . . . . . . . 6.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Contributions of the thesis . . . . . . . . . . . . . . . . . . . . . 7.2 Future works and perspectives . . . . . . . . . . . . . . . . . . Services in 5G network . . . . . . . . . . . . . . . . . . . . . . . 1.2 Radio Intelligent Controllers for RAN management . . . . . . 1.3 Structure of a Synchronization Signal Block (SSB) . . . . . . . . 1.4 Synchronization burst . . . . . . . . . . . . . . . . . . . . . . . 1.5 OFDMA resource structure . . . . . . . . . . . . . . . . . . . .

	vii xxi xxiii
	5.4 6 REM-based interference management
	List of Figures 6.1 7 Conclusion 1.1
	7.1 Bibliography

1.6 Bitmap sent by the Base Station (BS) to the User Equipment (UE) 2.1 Antenna model . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Coordinates of a beam direction . . . . . . . . . . . . . . . . . . 2.3 Multi-paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Gain pattern for 3 adjacent beams of the GoB using linear array 2.5 Gain pattern for 3 adjacent beams of the GoB using tapered array 2.6 MAB interaction with the environnement . . . . . . . . . . . . 2.7 Explore-then-Exploit . . . . . . . . . . . . . . . . . . . . . . . . 2.8 Sliding window strategy . . . . . . . . . . . . . . . . . . . . . . 2.9 MAB with reset . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10 REM construction . . . . . . . . . . . . . . . . . . . . . . . . . .

  2.6: MAB interaction with the environnement

	Chapter 2. Theoretical and technical background
	Algorithm 3 MAB process
	for t = 1 to T exp do
	Agent selects an arm k t 2 A Environment computes a reward x k t (t)
	Agent observes x k t (t)
	end for

TABLE 3 .

 3

	1: Network and Traffic characteristics
	Network parameters
	Number of macro BSs m	1
	Number of beams b in the GoB	16
	Number of small BSs s	10
	Number of interfering macros 3x6 sectors
	Intersite distance	500 m
	Bandwidth	20 MHz

TABLE 3 .

 3

	2: Channel characteristic
	Channel characteristics
	Thermal noise	-174 dBm/Hz
	Macro Cell Path Loss 128.1 + 37.6log 10 (d) dB
	(d in km)
	Small Cell Path Loss 140.7 + 36.7log 10 (d) dB
	(d in km)

TABLE 3 .

 3

	3: Traffic characteristic
	Traffic characteristics
	Macro users' arrival rate l m	3.5 users/sec/cell
	Small users' arrival rate l s	[3, 3.4, 3.8, 4] users/sec/cell
	Macro/small traffic spatial distribution	Uniform
	Service Type	FTP
	Average file size	5 Mbits

TABLE 4 .

 4 The channel is modelled as presented in Chapter 2 and the parameters are exposed in Table4.2.

	1: Network characteristics
	Network parameters
	Number of macro BSs		2
	Number of interfering macros	6x3 sectors
	Macro-cell layout	Hexagonal trisector
	Number of beams b per macro cell	16
	Bandwidth	20 MHz
	Channel model TABLE 4.2: Channel characteristics
	Channel characteristics
	Thermal noise	-174 dBm/Hz
	Path Loss (d in km)	128.1 + 37.6 log 10 (d) dB
	Nakagami-m shape parameter	2
	Intersite distance	500 m

TABLE 4 . 3

 43

	: Traffic characteristics
	Traffic characteristics
	Number of user in the hotspot zone	35
	of each cell
	Number of user outside the hotspot	10
	zone in each cell
	Traffic distribution in hotspot zone	uniform
	Traffic distribution outside hotspot zone	uniform
	Service Type	Full buffer, data

TABLE 5 .

 5

	2: Channel characteristics
	Channel characteristics
	Thermal noise	-174 dBm/Hz
	Path Loss (d in km)	128.1 + 37.6 log 10 (d) dB
	Nakagami-m shape parameter	2
	Intersite distance	500 m

TABLE 6 .

 6 1: Network parameters and characteristics of both channel and traffic.

	Network parameters
	Number of BSs	2
	Number of beams N B per BS	16
	BS transmit power	46 dBm
	Bandwidth	20 MHz
	Channel characteristics
	Thermal noise per Hertz	174 dBm/Hz
	Path Loss (D in km)	128.1 + 37.6 log 10 (D) dB
	Nakagami-m shape parameter	5
	Shadowing Log-normal	6 dB
	Intersite distance	500 m
	Traffic characteristics
	Number of UEs inside the hotspot zone per cell	35
	Number of UEs outside the hotspot zone per cell	10
	Traffic distribution inside the hotspot zone	Uniform
	Traffic distribution outside the hotspot zone	Uniform
	Service Type	Full buffer, data

  Chapter 6. REM-based interference management FIGURE 6.4: MUT as a function of the threshold c th .

		3.5
		3
		2.5
	MUT (Mbps)	1.5 2
		1
		0.5
		0
		th =1	th =2	th =3	th =4	th =5	th =6	th =7	th =8	th =9	th =10	th =11

The three solutions presented in the thesis are complementary and enable to have an effective solution for various network implementations.

Acknowledgements

I am deeply grateful to Zwi Altman for sharing his knowledge, time and advice with me. His expertise and his humanity both made the journey really enjoyable and are truly inspiring.

I would like to express my gratitude to Eitan Altman, for his supervising and his contribution to my work.

The performance of users located outside the hotspot area in cell 2 is given in Figure 4.14. The figure shows that the results are similar than for the users outside the hotspot area in cell 1, which is expected. Once again, the loss that are observed are smaller than the gain of other user. That fact added to the improvement for every user inside the hotspot area account for the global enhancement of the performance of the two cells.

the set of users attached to the beam of u select and to the adjacent beams U candidates U candidates \N (u select ) end while end for for every MAB j do t j = t j 1 if t j = 0 then Compute reward for j Send reward to the MAB j end if end for

Numerical results

This section assesses the performance of the MIC solution.

Simulation scenario

The simulation scenario is the same as that presented in Chapter 4 except for the traffic distribution, and is briefly summarized for sake of clarity. Let us consider two neighboring M-MIMO cells in a hexagonal 5G networks. The network comprises 19 sectors (cells): a central sector and two tiers of Chapter 5. Application of MAB to inter-cell interference management 18 neighboring sectors (6 sites located on a hexagonal grid) surrounding it. The central sector and one of its direct neighbors denoted hereafter as cell 1 and cell 2 respectively implement the MIC-based collaborative scheduling (see Figure 5.3). Each GoB contains 16 beams, labeled on Figure 5.3 for sake of clarity. The reference baseline solution implements a MU-PF scheduling without coordination and with beam skipping, as presented in Section 2.4.2. The simulation parameters are summarized in Table 5.1. The channel is modeled as presented in Chapter 2 and the parameters are listed in Table 5.2.

Chapter 5. Application of MAB to inter-cell interference management beam couples) solutions are considered. We test the module for every possible configuration of the parameters, given in Table 5.3, and present the results for the distributed-SW-MAB with global reward implementation. The performance of the MIC-based solution is compared with MU-PF scheduling with beam skipping as a baseline and to a ANBR solution implementation, as introduced in Chapter 4. Figure 5.6 presents the results for the distributed SW-MAB with global reward. Every combination of the parameters shown in Table 5.3 leads to comparable outputs, as explained below:

• SW-versus Reset Algorithm Both MAB algorithms converge very rapidly and provide practically the same decisions.

Chapter 6. REM-based interference management

This chapter presents a low complexity solution with a finer granularity than that of beam which relies on the usage of REMs. This approach is presented in Section 6.2 and is assessed through experimentation in Section 6.3.

REM based MU-MIMO coordinated scheduling

The coordinated scheduling aims at avoiding simultaneous scheduling of UEs from neighboring cells that experience strong interference from the serving beams. In order to significantly reduce intra-cell interference, a beam skipping technique is applied as in the other chapters. We recall that one UE per beam can be scheduled at a time. As in the previous chapters, let R b u u,t M+1 and R b u u,t M be the instantaneous rate of UE u 2 m attached to the beam b u when it is scheduled at t M+1 and its average rate calculated over a time window [t M W , t M ], respectively. The average rate of u at t M+1 is given as 3.5 [START_REF] Combes | Scheduling gain for frequency-selective Rayleigh-fading channels with application to selforganizing packet scheduling[END_REF].

The scheduling process is performed iteratively. At each new TTI (i.e. scheduling period), we consider two sets, U and U 0 of all active UEs attached to m and m 0 , respectively. We recall that U k and U 0 k 0 stand for the sets of UEs to be scheduled in the new TTI by m and m 0 , respectively and are first initialized to empty sets. Then, U and U 0 are sorted according to the PF criterion and the top-ranked UEs u and u 0 are given in (6.1) in (6.2).

where x is a small positive number.

The inter-cell inter-beam scheduling constraint is addressed as follows. We assume that the REM of each beam have already been computed and are accessible by the collaborative scheduler. Let us consider the user u and u 0 located respectively in the cell m at l u and in the cell m 0 at l u 0 . The REMs provides the scheduler with the predicted RSRP originating from the beams b u and b 0 u 0 and measured by the UEs u and u 0 . The REM associated with b u provides the predicted wanted signals ĉb u m (l u ) at l u and the predicted in- terfering signal ĉb u m (l u 0 ) at l u 0 . The REM associated with beam b 0 u 0 provides 6.2. REM based MU-MIMO coordinated scheduling 97 the predicted wanted signal ĉb 0 u 0 m 0 (l u 0 ) at l u 0 and the predicted interfering sig- nal ĉb 0 u 0 m 0 (l u ) at l u . We can proceed and estimate the corresponding Signal-to- Interference Ratios (SIRs) of u and u 0 which are given by Ĝu

where Îb u ,u 0 (l u 0 ) is defined as

1, if b u is the biggest interfering beam to u 0 at l u 0 , 0, otherwise. (l u , l u 0 ) falls below c th , coordination is triggered by muting b 0 u 0 in even TTIs, i.e. 2t, t = 0, 1, 2, . . . , and b u during odd TTIs, i.e. 2t + 1. Thus, u and u 0 will be scheduled at 2t and 2t + 1, respectively. Hence, coordination is implemented as an inter-cell inter-beam constraint for the scheduler, allowing to avoid simultaneous scheduling of a pair of UEs (u, u 0 ) with highly interfering serving beams. After adding u (resp. u 0 ) to U k (resp. U 0 k 0 ), we remove all w 0 2 U 0 (resp. w 2 U) for which Ĝmin

Similarly, following the selection of UE u, all UEs from the same beam as well as those from adjacent beams (i.e. beam skipping) are removed from U . The iterative process continues while U k has less than k max UEs and U candidates is not empty or U 0 k 0 has less than k max UEs and U 0 candidates is not empty. The detailed coordinated scheduling solution is presented in Algorithm 15, where three cases are distinguished. The first one considers the case where two UEs, u and u 0 served by m and m 0 , respectively, are selected for scheduling.

< k max and U candidates 6 = ∆) and card U 0 k 0 < k max and U 0 candidates 6 = ∆ then Select u using (6.1) and select u 0 using (6.2) Compute Ĝmin

served by b u and by its adjacent beams from

Équipe d'encadrement

Eitan Altman, Professeur, Avignon Université/Inria Directeur de thèse Zwi Altman, Ingénieur de recherche, Orange Labs Co-directeur de thèse