Maxime, Mélanie, Stéphane, Rémi, Kimi Anastasia Lorin

REMERCIEMENTS

"Peut-on écrire une thèse sans mourir d'ennui ou devenir à moitié fou ? oui, et mieux encore, il faut vivre la thèse comme un défi" 1 , et cette thèse a été un véritable défi de trois ans mais surtout, le travail intellectuel le plus stimulant de ma vie.

Ce travail n'aurait jamais pu exister sans mes encadrants, Alexandre Ferrieux et Isabelle Hamchaoui auprès de qui j'ai énormément appris depuis trois ans et demi que nous travaillons ensemble. Aussi, je ne remercierai jamais assez mon directeur de thèse, le Pr. Xavier Lagrange pour sa confiance, son suivi et ses excellents conseils tout au long de cette thèse. Je souhaiterais également remercier mes rapporteurs de thèse pour leur temps et leurs commentaires pertinents et mon jury de thèse d'avoir accepté de juger mon travail.

Je tiens à remercier tous mes collègues d'Orange (ex-CNET): les équipes ITEQ et AMOX, la bibliothèque technique, la section syndicale, Ziad, Benoît, Olivier pour ces discussions passionnantes que nous avons eues ensemble à l'occasion d'un café, d'un repas.

Également merci à Laurent Thomas et à toute la communauté OAI de faire vivre ce superbe projet qu'est OpenAirInterface.

De manière plus personnelle, merci à l'émission "C'est pas sorcier" pour mon goût pour les Sciences depuis 20 ans. Dany et Raz pour votre contenu qui m'a déterminé à avancer. A tous mes camarades, Jean, Dani, la section du Trégor, Camille M., les fédés des jeunes bretons... la lutte continue. Merci à mes amis qui m'ont soutenus (et supportés) ces

RÉSUMÉ EN FRANÇAIS

Le développement des réseaux cellulaires LTE (4G) a permis au cours de la précédente décennie l'émergence de services demandeurs en débit (ex. la vidéo) pour les utilisateurs mobiles. Cependant, les réseaux LTE présentent des caractéristiques de latence qui ne sont pas compatibles avec un certain nombre de services, comme le jeu vidéo dans le cloud. La latence est le temps nécessaire à un paquet de données pour traverser un segment du réseau. La latence est particulièrement importante dans le réseau d'accès cellulaire (Radio Access Network, RAN) comme le relève la littérature et l'expérience terrain des opérateurs. La latence minimale d'aller-retour (AR) avec la technologie LTE est de l'ordre de 20 ms, beaucoup plus que dans l'accès fixe où il est courant d'avoir des latences inférieures à 10 ms. Surtout, la latence du réseau d'accès en réalité dépasse régulièrement les 50 ms jusqu'à plusieurs centaines lorsque le point d'accès est congestionné. Les services nécessitant de la faible latence sont aussi souvent sensibles à la variation de cette latence dans le temps, ce qui est appelée la gigue. La gigue dans le RAN LTE se révèle aussi très importante, principalement dû aux changements dans le temps des caractéristiques du lien radio, les retransmissions pour pallier les pertes, et à la concurrence d'accès aux ressources radio des différents utilisateurs.

La dernière génération de technologie cellulaire 5G [START_REF] Dahlman | The next generation wireless access technology[END_REF] a parmi ses objectifs la réduction de la latence pour tendre vers la milliseconde dans le RAN. Pour cela, certains mécanismes d'adaptation aux conditions radio et de récupération des pertes ont été améliorés. Surtout, il a été introduit une nouvelle méthode de gestion des flux de bout en bout pour contrôler et garantir des latences faibles, basée sur du découpage du réseau (slicing), de la différentiation des flux et la gestion active de qualité de service (QoS). Ces mécanismes de gestion active de la latence ne sont actuellement pas utilisés pour les flux Internet qui sont généralement transmis dans un mode "Best-Effort" (au mieux). Or, il s'agit du mode de transmission par défaut des flux utilisateur dans le réseau. De ce fait, la latence minimale dans le RAN pour les utilisateurs une fois que la 5G Standalone sera déployé ne devrait être réduite que de l'ordre de 10 ms, tout en ne garantissant pas plus qu'en LTE cette latence minimale au cours du temps. L'objectif de ce travail de thèse est de réduire la latence et la gigue introduite par le v réseau d'accès cellulaire, LTE et 5G pour les transmissions de flux Internet.

La méthode utilisée repose sur l'utilisation d'une plateforme d'expérimentation en laboratoire (voir Chapitre 2) composée de terminaux du commerce et d'une station de base (BS) LTE/5G issue du projet open-source OpenAirInterface 2 [2]. Cette méthode permet de contrôler l'environnement radio (avec un canal radio optimal, sans variations, sans perturbations externes) et les latences du réseau tout en étant représentatif des latences utilisateur avec des terminaux du commerce et une station de base de recherche. L'étude des latences se concentre sur le RAN du côté de l'opérateur du réseau ; le coeur réseau et les terminaux sont considérés comme des boîtes noires. À l'inverse, le code source de la station de base utilisée est ouvert, ce qui permet une analyse poussée des mécanismes des protocoles radio. Les latences sont mesurées par paquet à l'aide de 2 captures réseaux simultanées avec une synchronisation des horloges à l'entrée et à la sortie du RAN (méthode passive de mesure contrairement à une méthode active qui utilise un flux pour mesurer la latence). Cette méthode permet d'obtenir un suivi fin de l'évolution de la latence au cours du temps avec une précision sous la milliseconde, mais cette méthode ne permet pas d'analyser la composition interne de la latence.

La première contribution de cette thèse est la proposition d'une nouvelle méthode de mesure des latences internes au RAN, LatSeq (voir Chapitre 3). Nous définissons la latence interne comme la succession de délais subit par un paquet de données dans le système de la station de base. Nous modélisons la succession des délais comme un graphe dont les arêtes correspondent à des segments du trajet du paquet, les sommets comme un lien entre 2 segments (par exemple entre 2 couches protocolaires). LatSeq est implémenté dans la station de base (BS) sous la forme de sondes dans le code qui observent le passage des paquets entre les différents segments. Ces sondes sont conçues pour avoir le minimum d'impact possible en termes de délais et de coût de calcul sur le système observé. L'ensemble des traces produites par les sondes sont écrites au fur et à mesure du fonctionnement du RAN dans un fichier de traces. Le trajet des paquets au sein de la BS sont alors générés par un algorithme de reconstruction de graphe. Les sondes fonctionnent en temps-réel, mais la reconstruction est un processus plus lourd en calcul qui est fait a posteriori. En sortie, l'outil fournit une représentation graphique des latences (voir Fig. 3.10a) par paquet à un niveau de grain qui peut aller jusqu'à la fonction logicielle avec une précision de l'ordre de la microseconde. Une évaluation a été réalisée pour montrer le faible impact de la méthode sur le fonctionnement de la station de base 2. https://openairinterface.org/ vi (16 ns par mesure) et pour montrer la précision de la mesure de la latence. LatSeq (pour Latency Sequence analysis) a fait l'objet d'une publication en conférence internationale [3], d'une publication du code de l'outil en open-source 3 et d'une contribution au projet OpenAirInterface 4 .

Les principaux résultats d'expérimentations sur la plateforme expérimentale équipée de Latseq montrent que : 1) la voie descendante (du réseau vers les terminaux) est la source la plus importante de délais lorsque la cellule est chargée par plusieurs flux concurrents (phénomène du "bufferbloat") ; 2) le délai subit par un paquet sur la voie remontante (des terminaux vers le réseau) dans la BS est toujours faible, même si les retransmissions, la segmentation et le "in-order delivery" RLC peuvent en générer ; 3) la voie remontante représente la principale source de gigue du RAN et une latence plus importante que celle de la voie descendante quand la cellule est vide.

L'analyse des latences internes montre que le mécanisme d'accès au canal radio de la voie remontante est responsable de la variation importante de la latence d'un paquet à un autre et de l'apparition de rafales de transmission dans le profile de transmission du trafic [4] (voir Fig. 4.10a). En effet, comme pour la voie descendante, l'allocation des ressources radio est réalisée par l'ordonnanceur MAC de la station de base, par contre à la différence de la voie descendante, la connaissance de l'état des buffers de transmission et des besoins en transmission sont répartis entre les terminaux. Pour résoudre ce problème dû à l'architecture centralisée des cellules radio, les normes LTE et 5G définissent : Le "Scheduling Request" (SR) qui permet à un terminal de demande un accès au canal de la voie remontante ; Le "Buffer Status Report" (BSR) qui permet à un terminal de transmettre à la station de base l'état d'occupation des buffers de transmission ; Le "Uplink Grant" (UG) qui est transmis par la station de base aux terminaux via le canal de contrôle (DCI) et qui indique les ressources radio allouées à un terminal ainsi que le codage et la modulation à utiliser. Le processus est le suivant (voir Fig. 4.15): Le terminal a des données à transmettre et demande donc à la BS des ressources radio avec un SR. La station de base prend en compte cette demande en envoyant un UG au terminal lui allouant des ressources radio sur la voie remontante. Puis le terminal transmet des données sur les ressources radio qui lui ont été réservées et y ajoute un BSR pour indiquer la quantité de données encore à transmettre. L'algorithme d'ordonnancement va prendre en compte cet état du buffer pour allouer la quantité de ressources radio nécessaire pour vider le buffer 3. https://github.com/Orange-OpenSource/LatSeq 4. https://gitlab.eurecom.fr/oai/openairinterface5g/-/tree/latseq vii de transmission. C'est le mécanisme de demande explicite d'accès et la connaissance partielle par la station de base de l'état des buffers des terminaux qui génèrent des délais d'accès et des variations dans la latence d'un paquet à l'autre. Il s'agit de cette dernière observation, corrélée avec des observations dans un réseau cellulaire commercial d'Orange (voir paragraphe 4.2.2) et des observations issues de la littérature (voir paragraphe 4.2.6) qui justifient le reste de ce travail de thèse sur le besoin de réduire la gigue et la latence de la voie remontante dans le RAN en travaillant sur le mécanisme d'accès et d'allocation des ressources radio.

La principale contribution de cette thèse [5] est une solution au problème de l'estimation par la station de base (BS) des besoins en transmission des terminaux (voir Chapitre 6) dénommé "Enhanced-BSR". Pour cela, il est proposé d'implémenter un modèle d'estimation au niveau de la BS qui utilise les rapports BSR, l'estimation du débit de transmission du terminal et le taux d'utilisation des blocs de transport (TB) pour fournir une prédiction en continu de l'occupation du buffer de transmission du terminal. Cette information est ensuite utilisée par l'algorithme d'ordonnancement pour répartir les ressources entre les terminaux. Le modèle d'estimation linéaire utilisé s'adapte en continu pour garantir une utilisation des ressources radio allouée à plus de 80%. Le modèle assez simple qui est utilisé permet d'obtenir une bonne estimation des besoins en transmission d'un terminal dont le trafic est régulier avec un débit source qui varie peu à des courtes échelles de temps. Cela correspond typiquement à un transfert TCP, du cloud gaming ou de la vidéo. Les avantages majeurs de cette solution, c'est qu'elle est compatible avec les normes LTE et 5G actuelles, qu'elle ne nécessite pas de modifications de l'algorithme d'ordonnancement ou des terminaux. Une limitation importante est le gaspillage de ressources radio lorsque le modèle surestime les besoins en transmission du terminal.

Une évaluation de la solution montre une nette diminution de la gigue ainsi que de la latence remontante en particulier pour les transferts longs et à débit constant passant d'un délai AR de 52 à 19 ms et d'une gigue de 10 ms au lieu de 47 ms (voir Fig. 6.4). La latence et la gigue sont aussi diminuées pour un trafic non déterministe en rafale, mais occasionne une sous-utilisation des ressources radio allouées. Une conséquence intéressante de la réduction de la gigue et la latence de la voie remontante est à observer sur les trafics TCP descendant (par exemple pour un téléchargement ou le chargement d'une page Web). En effet, TCP est un protocole connecté, par lequel le transfert des données se fait par une voie (par exemple la voie descendante pour un téléchargement), et les données sont acquittées en utilisant la voie inverse (la voie remontante pour un viii téléchargement). L'acquittement est utilisé pour garantir la transmission des données, mais aussi pour contrôler la congestion en adaptant le débit de transmission aux capacités du réseau. Les algorithmes de contrôle de congestion TCP les plus utilisés sur internet estiment la capacité du réseau à partir des pertes et de la latence. Nous avons observé une amélioration du débit réalisé et une meilleure stabilité du débit de transfert lorsque notre solution est implémentée (voir le débit en Fig. 6.7c comparé au débit avant avec la Fig. 6.7a). Cette expérimentation montre que la réduction de la gigue et de la latence remontante n'est pas seulement bénéfique pour les services sensibles à la latence, mais aussi pour l'augmentation des débits et pour une meilleure utilisation des ressources [4] radio dans le RAN par les trafics TCP. Or, la sous-utilisation des ressources radio par TCP dans les réseaux cellulaires est un problème souvent évoqué dans la littérature ( 70% en

Introduction

INTRODUCTION

Motivation

5G is the next generation cellular network technology, released in 2018 in replacement to LTE. According to [7] the number of 5G subscribers should increase to surpass LTE in 2026 (Figure 0.1a). In the same time, customers increasingly favor mobile access for their internet consumption [START_REF] Sandvine | Global Internet Phenomena Report[END_REF]. Meanwhile, traffic intensive services has developed on mobile such as video streaming, gaming and conferencing. In consequence, The amount of traffic carried by mobile networks grows exponentially since 2016 as depicted in Fig 0 .1b. Since the capacity of LTE networks seems saturated, especially in dense zone with high throughputs requirements at peak hours, as a most efficient radio access technology, 5G-New Radio (NR) [START_REF] Dahlman | NR -The New 5G Radio-Access Technology[END_REF] will make it possible to support this increase in communication needs 5 . By 2026, 5G should be used for the transmission of 130 EB per month worldwide in a total of 240 EB for all technologies. This will alleviate the traffic load for LTE networks but at the cost of a new infrastructure deployment. 5G-NR [START_REF] Dahlman | The next generation wireless access technology[END_REF] is designed as the successor of the LTE Radio Access Technology (RAT) [START_REF] Jeffrey | What will 5G be?[END_REF] by the Third Generation Partnership Project (3GPP) consortium. It was designed to be the global standard for the air interface of 5G networks. 5G-NR shares many similarities with LTE, in the manner to perform radio transmissions, however, the radio transmissions techniques have been enhanced to meet requirements defined by International Telecommunication Union (ITU) IMT-2020 for 5G [START_REF] Series | IMT Vision-Framework and overall objectives of the future development of IMT for 2020 and beyond[END_REF]. These requirements are commonly represented in the form of the triangle in Figure 0.1c with 3 types of services targeted [START_REF] Navarro-Ortiz | A survey on 5G usage scenarios and traffic models[END_REF]. enhanced Mobile BroadBand (eMBB) for extreme throughputs towards 10 Gbps, massive Machine Type Communications (mMTC) for dense machine-to-machine communications and Ultra-Reliable Low-Latency Communications (URLLC) for extremely low latency towards 1 ms [START_REF] Jeffrey | What will 5G be?[END_REF]. Nowadays, cellular networks are theoretically able to provides 1 Gbps with 10 ms latency. This is a complex challenge to meet required values knowing that there is a fundamental trade-off between throughput, reliability and latency in radio Figure 0.1 -Emergence of 5G in mobile habits and new services (Source: Ericsson [7]).

transmissions [START_REF] Soret | Fundamental tradeoffs among reliability, latency and throughput in cellular networks[END_REF].

That is particularly true for the latency which has been somewhat neglected in favor of throughput in the past. However, there has been a strong motivation to reduce latency in recent years with the emergence of services that require it. Self-driving automotive, industry automation, tactile internet and cloud gaming are usually given as examples of services which requires low latency and low jitter to be deployed in mobile networks [START_REF] Navarro-Ortiz | A survey on 5G usage scenarios and traffic models[END_REF][START_REF] Americas | New services and applications with 5G uRLLC. Whitepaper. 5G Americas[END_REF]. In fact, low latency is important in machine-to-machine communications timescale more than in machine-to-human interactions timescale.

Standards and recent researches propose a number of Radio Access Network (RAN) techniques to satisfy the needs of low latency in cellular networks [START_REF] Parvez | A survey on low latency towards 5G: RAN, core network and caching solutions[END_REF], including shortened processing and better radio resource management. Also, 5G redefines the end-to-end Quality of Service (QoS) model and network architecture to support specific services mentioned earlier. Besides, QoS architecture which prioritizes the access to radio resources of a user or a flow over another is not fully inline with the Internet philosophy of the "net neutrality". Because the Best Effort (BE) continues to be the main carrying mode of data volume in cellular networks (LTE, 5G), it is particularly interesting to focus on BE-enabled techniques to provide lower latency for the largest number of customers and for the largest number of innovative internet services. work capacity variations and retransmissions that occur at the radio channel, favoring the constitution of a bufferbloat. The bufferbloat also impacts the efficiency of transport protocols, such as Transmission Control Protocol (TCP) and Quic (QUIC) (both represent more than 98% of internet traffic [START_REF] Schumann | Impact of Evolving Protocols and COVID-19 on Internet Traffic Shares[END_REF]) on achieving good usage of network capacity. The efficiency of TCP over cellular networks is often shown to be suboptimal with a radio capacity utilization of 60% [6], with a cost for operators who bought expensive radio spectrum. The question of downlink bufferbloat is well studied with many solutions using cross-layer techniques or active queue management in the RAN. In the same time, the bufferbloat [START_REF] Guo | Understanding On-Device Bufferbloat for Cellular Upload[END_REF] and more generally the latency of the uplink channel was less studied also because the downlink latency had a more important contribution to RAN Round-Time Trip (RTT) and because the majority of traffic was in the downlink. That will be no longer true with more and more latency-sensitive uplink services. 5G takes these evolutions into account with a more efficient uplink channel in terms of throughput (OFDMA, wider bandwidth) and latency (enhanced HARQ, optimized access method). These standardized methods are necessary but not sufficient to tackle uplink latency and bufferbloat for BE traffics.

The objective of this thesis is for reducing latency and jitter of the uplink channel in the 5G and LTE RANs for a majority of mobile customers. The fine-grain characterization of latency in the network is not an obvious problem, we also try to solve this problem in this thesis. These subjects are of great interest for a network operator, so this work is funded by Orange under a CIFRE contract. and analytical expressions how grant-based uplink access is a source of a non-negligible delay and jitter. Contribution 3 : Addressing the uplink latency and jitter problem with Enhanced-BSR, an evolution of buffer estimation algorithm for MAC layer One of the cause of mentioned uplink access latency, lies in a bad knowledge of the mobile terminal needs in transmission by the base station. The proposed solution enhanced-BSR opts for an estimation method of the UE needs in transmission. We have proposed a new buffer status estimation algorithm for the base station's MAC uplink scheduler which utilizes "Scheduling Request" (SR) and "Buffer Status Reports" (BSR) and transport block utilization. The implementation is 3GPP 5G and LTE standards compliant and was made in the OAI base station. We demonstrated with experiments on selected types of traffic (TCP transfer and uplink traffics) that the new estimation could accurately predict UE transmission buffer length. The outcome is a more efficient radio resource allocation by the BS dynamic scheduler for access latency and jitter resulting in an important reduction of the uplink jitter in the presented radio context and mobile traffics.

Dissertation outline and organization

The rest of the thesis is organized in chapters as follows. The links between chapters are illustrated in Figure 0.2. Elements of understanding concerning cellular networks and internet protocols, followed by a review of latency in the access network and their mitigation are given in Chapter 1. The testbed and research methods are detailed in Chapter 2. Chapter 3 presents the LatSeq tool developed for internal latency analysis which completes the testbed. A detailed analysis of latency in the RAN is provided in Chapter 4 with a focus on the uplink channel. Furthermore, Chapter 5 motivates the need to reduce jitter and uplink latency to improve TCP efficiency with ultimately better utilization of radio resources and to reduce end-to-end latency. Our proposed method to reduce uplink latency jitter with a predictive estimation model of the transmission needs, enhanced-BSR, is explained in Chapter 6. Following that, a preliminary work explores perspective opened by enhanced-BSR for the uplink scheduling in Chapter 7. The manuscript ends with a general conclusion and perspectives of this thesis work.

Chapter 1

BACKGROUND

Abstract : This chapter provides the elements of understanding for the rest of the thesis. The first section is an overview of 5G and LTE cellular systems, with a focus on the Radio Access Network (RAN) and the radio resource allocation. The second section presents the internet protocol suits with an introduction to Transport Control Protocol (TCP), congestion control and Quic protocol. Finally, the last section is dedicated to the latencies in the networks and their mitigation techniques. After latency definitions, to issue of bufferfloat is exposed followed by a short review of latency mitigation techniques in 5G.

5G and Cellular Networks

A new generation of Radio Access Technology (RAT) is launched every decade since 1980. 3G in 1990, 3.5G HSDPA in 2002, 4G in 2009. Each generation bringing new capabilities to mobile networks for instance, web-browsing with 3G and video streaming with 4G. In 2010's Long Term Evolution (LTE)-Advanced and Advanced pro was launched with incremental improvements of LTE. The Fifth Generation (5G) RAT is introduced in 2019 to come after LTE technologies. After a long phase of exploration and standardization Third Generation Partnership Project (3GPP) issued Release 15 1 describing New Radio (NR), the new 5G RAT2 [START_REF] Dahlman | NR -The New 5G Radio-Access Technology[END_REF] in accordance with requirements edicted by International Telecommunication Union (ITU). Release 15 describes initial 5G RAT i.e. how Radio Access Network (RAN) works and how to connect it to a mobile core. Since then, Release 16 3 (2020) [START_REF] Parkvall | 5G NR Release 16: Start of the 5G Evolution[END_REF] and Release 17 4 have brought improvements in many areas such as signalling, architecture and services provided.

5G Non-StandAlone mode (NSA) (using an LTE mobile core) is already deployed for commercial uses with more than 600 million subscribers during the first quarter of 2022 and expected to be 1 billion by then end of this year worldwide [7]. Figure 0.1a depicts the expected number of subscribers for the different RAT until 2027 and shows a rapid growth of the 5G usage in the next few years, mainly in replacement of 3G and 2G. It comes with an explosion of usage and data consumption as illustrated in Fig. 0.1b permitted by new 5G capabilities. The popularity of the video streaming everywhere and the development of new data intensive services such as mobile cloud gaming, Virtual Reality (VR) and remote work saturate a number of cells in dense area, reaching capabilities of LTE. 5G must be able to support the need for capacity in 2030 thanks to a redesigned radio layer and mobile network architecture. When theoretically the maximum peak datarate in LTE is 1 Gbps, 5G offers a maximum of 10 Gbps in downlink and uplink. In this task, an increase of spectral efficiency, number of cells and of spectrum bandwidth are needed, implying a number of challenges in radio layer.

A fundamental movement in past years has been the convergence between the various wireless networks and fixed networks. LTE/5G and Wifi uses the same modulation scheme, home fiber and 5G seek to provide 1 Gbps symmetric to customer. In the same idea, it is desired that the latencies of fixed and mobile networks converge. ITU IMT-2020 defines an objective of 1 ms RAN Round-Time Trip (RTT) for 5G (see Fig. 0.1c), when the minimum latency achievable in LTE is about 20 ms. Latency is probably the most difficult challenge to solve in 5G because there are at the same time fundamental limits related to the radio channel [START_REF] Soret | Fundamental tradeoffs among reliability, latency and throughput in cellular networks[END_REF] and questions of architecture of the mobile network.

Latency mitigation techniques for 5G will be discussed in Section 1.3.3. In this section, we first give an insight of current and upcoming cellular network architectures. After that, an overview of cellular Radio Access Network (RAN) protocol stack §1.1.2.1 and physical radio channel will be presented §1.1.2.2. Radio resource allocation schemes will be detailed in §1.1.3 with a special focus on the uplink channel.

Cellular network architecture

The cellular network connects mobile terminals (i.e. User Equipments (UEs)) to the Internet Protocol (IP) network (i.e Packet Data Network (PDN)). Both LTE and 5G cellular networks consists in 2 parts, the RAN (i.e. Evolved Universal Terrestrial Radio Access (E-UTRA) for LTE and Next Generation RAN (NG-RAN) for 5G) and the Mobile Network Core (MNC) (i.e. Evolved Packet Core (EPC) for LTE and 5G Core (5GC) for 5G) [START_REF] Fan | An Overview of Low latency for Wireless Communications: an Evolutionary Perspective[END_REF]-III. An overview of LTE and 5G cellular network architectures are depicted in Figure 1.1. On the left are the mobile terminals and on the right the IP network. UEs are connected to Base Station (BS), called Evolved NodeB (eNB) in LTE and gNodeB (gNB) in 5G through the radio air interface in the RAN. BSs are then connected to MNC through backhaul network (in practice, a high capacity optical fiber of hundred kilometers connecting the base station to the datacenter where the core is located). MNC is the edge interface with PDN.

Radio Access Network

The radio access network consists in cells served by one or more BSs and UEs. Area covered by a cell depends on the UE density, geography of the area, antenna power and frequency used. BS provides radio functionalities (Radio-Frequency (RF), power amplification, beam forming, etc.), baseband processing (data coding, modulation, radio resource mapping, etc.) and radio link layer-2 (Medium Access Control (MAC), Radio Link Control (RLC), Packet Data Convergence Protocol (PDCP), Radio Resource Control (RRC) protocols). BSs are connected together with X2 interface (Xn in 5G) to handle mobility of a user between cells with handover procedure (i.e. performing user context and data migration from one cell to another). BSs cooperate to offer the best throughput and energy efficiency (e.g. balancing connected UEs between BSs to share traffic load). Also, 5G is designed to operate with the LTE network [START_REF]Samsung. 4G-5G Interworking[END_REF]. It is necessary in 5G NSA since the control place is provided by eNB and EPC. In the future, interworking between the 4G and 5G network will improve connectivity while distributing the traffic load.

UEs in the cell are either connected and active or sleeping and inactive. After an initial access procedure, the UE receives a Radio Network Temporary Identifier (RNTI) that is an identifier for this UE in this cell for the duration of the session. RNTI is transported by the MAC layer header and serves to route and decode radio messages. Also, a Data Radio Bearer (DRB) for user traffic and a Signal Radio Bearer (SRB) for the control-plane traffic are instantiated for the UE. Radio bearers consist in practice on virtual circuits identified with a Logical Channel IDentifier (LCID) associated to layer software entities.

More DRBs can be instantiated in addition to the default bearer to set up virtual circuits for specific traffics e.g. voice carrying (see Voice over LTE (VoLTE) bearer in [START_REF] Zhang | LTE optimization engineering handbook[END_REF]-11), latency-sensitive slices [START_REF] Afolabi | Network slicing and softwarization: A survey on principles, enabling technologies, and solutions[END_REF][28]-13.7. A QoS Class Identifier (QCI) (or 5 QoS Identifier (5QI) [START_REF] Gpp | System architecture for the 5G System (5GS)[END_REF]) is associated to the bearer which specifies the level of Quality of Service (QoS) requirements in terms of delay, datarate and priority 5 6 .

LTE Channel Quality Indication (CQI)

Radio Access Network

RAN ensures the connectivity of mobile devices to the mobile network. At each end of the radio channel there are devices and a base station. 3GPP Technical Specifications (TS) 36.300 [START_REF] Gpp | Evolved Universal Terrestrial Radio Access (E-UTRA) and Evolved Universal Terrestrial Radio Access Network (E-UTRAN)[END_REF] and 38.300 [START_REF] Gpp | Overall description[END_REF] define respectively for LTE and 5G radio protocol architecture to permit interoperable communication between UE and BS through the physical radio interface. The protocol stack is discussed in Figure 1.2 and in the following. Thereafter, we make a brief presentation of the physical radio layer ( §1.1.2.2). Figure 1.2 summarizes protocols in use in the RAN. The same stack is implemented in both BS and UE modem. OSI layer 2 includes Packet Data Convergence Protocol (PDCP), Radio Link Control (RLC) and Medium Access Control (MAC) protocols. "Application", "Transport protocol" and "IP" are given for information to represent user plane while RRC [START_REF] Gpp | Evolved Universal Terrestrial Radio Access (E-UTRA); Medium Access Control (MAC) protocol specification[END_REF][START_REF] Gpp | Medium Access Control (MAC) protocol specification[END_REF] represents the radio control plane. User plane and Control plane traffics shares the same protocol entities after the PDCP. However, user and control traffic are isolated in 2 different bearers, respectively in DRB and SRB. Also, as can be seen in the figure, 5G has taken over the protocol architecture of LTE with great similarities between these protocols. Thus, many remarks on the radio layer 2 in LTE remain true in 5G. The 5G Service Data Adaptation Protocol (SDAP) entity is put aside here because it will not be implemented during this thesis. SDAP is related to the new 5G QoS framework with QoS flows and not directly for radio transmission operations.

LTE and 5G protocol stack

Packet Data Convergence Protocol (PDCP) layer [START_REF] Gpp | Evolved Universal Terrestrial Radio Access (E-UTRA); Packet Data Convergence Protocol (PDCP) specification[END_REF][START_REF] Gpp | Packet Data Convergence Protocol (PDCP) specification[END_REF] is responsible for security (ciphering and integrity protection), routing (mapping of radio bearers to RLC entities, routing to cell group in dual connectivity, inter-cell handover handling) and header compression (compression of IP header with Robust Header Compression (ROHC) to reduce overhead). PDCP also implements packet retransmission and in-sequence delivery functionalities to ensure reliability of the radio link in complement to RLC and MAC layers. Latencies associated to PDCP layer are:

-Processing delay for data encryption -In-sequence delivery when enabled could cause Head-of-Line (HoL) blocking -The handover procedure with the transfer of the PDCP entity buffer's contents from one BS to another, with also a risk of a packets discarding.

Radio Link Control (RLC) protocol [START_REF] Gpp | Evolved Universal Terrestrial Radio Access (E-UTRA); Radio Link Control (RLC) protocol specification[END_REF][START_REF] Gpp | Radio Link Control (RLC) protocol specification[END_REF] prepares packets from PDCP entity for the MAC layer (segmentation 7 and sequence numbering). Packets awaiting to be transmitting are stored inside the transmission buffer. RLC entity identified with a LCID belongs to one radio bearer. According to the traffic carried by the logical channel, there are 3 entity modes:

-Transparent Mode (TM) where the RLC entity is totally transparent without segmentation and retransmission. It is used for control-plane broadcast where the information should reach multiple users. -Unacknowledged Mode (UM) supports segmentation but no retransmissions, used when error-free delivery is not required. It is recommended in the literature for VoLTE but not for internet services (despite TCP protocol could handle a low error rate like one at the output of the MAC layer after Hybrid-ARQ (HARQ) process).

and concatenation in LTE

-Acknowledged Mode (AM) is the default mode for the user plane DRB. RLC Automatic Repeat reQuest (ARQ) procedure is implemented for error detection and recovery handling. An ARQ feedback loop for acknowledgment and retransmission buffer are needed to ensure retransmission process, adding latency, control signalling and RAN configurations. It makes AM more robust and reliable, but this induces delay from the control overhead as opposed to cutting losses.

Latency associated to the RLC layer are:

-Buffering latency since the RLC entity stores packets from IP layer for radio layer.

It is very dependent on the buffer size, except that we know that buffers in mobile networks are big (several megabytes), to accommodate variations in capacity and retransmission needs. -Retransmission latency for AM entity and the ARQ feedback delay to detect and recover lower layers losses. For that reason, the UM should be preferred for latencysensitive flows such as video streaming, and voice-over-ip.

Medium Access Control (MAC) [START_REF] Gpp | Evolved Universal Terrestrial Radio Access (E-UTRA); Medium Access Control (MAC) protocol specification[END_REF][START_REF] Gpp | Medium Access Control (MAC) protocol specification[END_REF] entity is common to all bearers in a BS or a UE and multiplexes logical channels. Because of that, BS's MAC entity is responsible for scheduling of users and bearers for both uplink and downlink. UE's MAC entity is in charge of the prioritization of the instantiated bearers. Scheduling and radio resource allocation schemes will be discussed in detail Sec. 1.1.3. MAC encodes Transport Blocks (TBs) to be transmitted through radio physical layer (conversely decode them) by concatenating RLC data segments from multiple logical channels, MAC header and eventually padding to fit with the scheduled Transport Block Size (TBS).

The MAC Hybrid-ARQ (HARQ) process, up to 8 in parallel, handles most of the radio link errors (10% error at the first MAC transmission [START_REF] Soret | Fundamental tradeoffs among reliability, latency and throughput in cellular networks[END_REF]-Table I, less than 1% at the RLC layer and less than 0.1% at the PDCP layer) with recovery and retransmission of TB. HARQ is the combination of an Automatic Repeat reQuest (ARQ) and a Forward Error Correction (FEC). FEC refers to a more general method which tries to detect and correct errors in a transmission. When an unrecoverable error appears, the decoding HARQ entity sends a Not-ACK (NACK) to the transmitting HARQ counterpart for a retransmission. 5G modified HARQ method to be more efficient in error recovery with Incremental Redundancy (IR) technique and more latency efficient with a shortened ACK feedback loop.

Finally, MAC layer manages number of radio-related network functions such as random access control (for initial access) and Link Adaptation (LA) mechanism. The role of the LA is to adapt Adaptive Modulation and Coding (AMC) used in Physical Layer (PHY) using CQI. TBS is also determined by the MAC according to AMC.

Latency related to the MAC layer are:

-Processing delay for multiplex and de-multiplex transport blocks -Scheduling latency depending on the current cell load, position of the user and the scheduling strategy -DL and UL signalling procedures to access radio channel -HARQ feedback loop and retransmission delays of about 8 ms in LTE.

For further information, an in-depth representation of the protocol stack is given in Appendix C.C.4.

Radio physical layer

There are fundamental trade-offs between capacity, coverage, latency, reliability, and spectral efficiency in cellular networks [START_REF] Soret | Fundamental tradeoffs among reliability, latency and throughput in cellular networks[END_REF]. Network capacity is the theoretical maximum amount of data that the link can support [START_REF] Ishac | Defining Network Capacity[END_REF]. The objective of the radio link layer is to achieve the maximum achievable network capacity.

To tend towards the Shannon capacity, many improvements have been introduced with the NR technology [START_REF] Dahlman | The next generation wireless access technology[END_REF] from modulation and coding scheme to Multi-User Multiple-Input Multiple-Output (MIMO) (see II.C of [START_REF] Jeffrey | What will 5G be?[END_REF]-II.C). In reality, main differences between LTE and 5G RATs are related to physical layer to increase capacity and to support lowlatency [START_REF] Fan | An Overview of Low latency for Wireless Communications: an Evolutionary Perspective[END_REF]-Table II. However, 5G like LTE utilizes Orthogonal Frequency-Division Multiple (OFDM) as transmission scheme, quadrature-amplitude for modulation and support Frequency-Division Duplexing (FDD) and Time-Division Duplexing (TDD) duplexing modes.

In Time-Division Duplexing (TDD), uplink and downlink multiplexing is made in time with an alternation of the uplink and downlink transmissions. Frequency-Division Duplexing (FDD) is very common in commercial network, uplink and downlink radio transmissions are multiplexed in frequency with one band for the uplink and one band for the downlink. It is robust and permit a continuous transmission of data in both directions in the same time. Band 7 for example reserves band 2500 -2570 MHz for uplink and 2620 -2690 MHz for downlink.

After that, the radio channel is organized as a time-frequency resource grid composed of Physical Resource Blocks (PRBs) as depicted in Figure 1.3. Resource grid is shared between all users in the cell. The figure depicts a Single-Input Single-Output (SISO) situation, when for a MIMO system, there is one different resource grid for each antenna.

In the time domain, transmissions are organized by frame of 10 ms, subframe of 1 ms and 2 slots per sub-frame. At most one TB of dynamic size is transmitted during a Transmission Time Interval (TTI) of 1 ms through the air interface.

PRBs are divided in frequency-domain into 12 subcarriers and in time into 14 OFDM symbols carrying encoded and modulated data. The PRB is allocated to one user, but the bandwidth composed of several PRBs could be allocated to several users in the same time with the Orthogonal Frequency-Division Multiple Access (OFDMA) model. OFDMA allows flexible and dynamic sharing of the bandwidth between UEs. It is used for the downlink channel in both LTE and 5G and also for the uplink in 5G. The access method for the uplink in LTE is slightly different since it uses the Single-Carrier FDMA (SC-FDMA) and is not represented on the figure .   Data are encoded into a TB with a size determined by the number of PRB and the Modulation and Coding Scheme (MCS). In particular, the MCS is associated to a Quadrature-Amplitude (QAM) scheme [START_REF] Gpp | Evolved Universal Terrestrial Radio Access (E-UTRA); Physical layer procedures[END_REF]-Table - [START_REF] Gpp | User Equipment (UE) radio access capabilities[END_REF]. Some OFDM symbols of a resource block are used for control channels 9 , for instance Primary Synchronization Channel (PSCH), Secondary Synchronization Channel (SSCH), Physical Downlink/Uplink Control Channel (PXCCH) or Physical HARQ Indicator Channel (PHICH). Usually, it is admitted to have 14% of control overhead on radio capacity.

The Physical Downlink Control Channel (PDCCH) carries downlink scheduling assignments, information necessary for the mobile to be able to receive, demodulate and decode downlink user channel data and transport control elements to permit uplink transmissions. The payload carried on the Physical Downlink Control Channel (PDCCH) is the Downlink Control Information (DCI) encoded with the RNTI of the device. The Downlink Control Information (DCI) message is of different format, format 0 carries information related to the uplink scheduling grants and format 1 information related to the downlink assignments.

To learn more about the LTE physical layer, an excellent tutorial is proposed in [START_REF] Zarrinkoub | Understanding LTE with MATLAB: from mathematical modeling to simulation and prototyping[END_REF] which presents physical layer from theory to practice with a MATLAB implementation. In addition, the research work presenting an implementation of an end-to-end 5G NR simulation [START_REF] Patriciello | An E2E simulator for 5G NR networks[END_REF] propose a tutorial on the physical and MAC layer in 5G.

Radio resource allocation

In the previous Sec. 1.1.2.2, we gave an overview of the physical radio layer. Especially, we detailed the radio time-frequency resource grid composed of PRBs shared between UEs. In this section, we discuss the radio resource allocation schemes with a focus on the uplink radio resource grid (Fig. 1.3) at the RAN-level.

Allocation schemes

RAN is essentially a scheduled system where time-frequency radio resources are assigned to mobile devices. Scheduling for the uplink and downlink channels follows roughly same principles except for the fundamental asymmetry that exists between the two. The transmission power and data packets are distributed among UEs when in the downlink power resource and packets are centralized within the BS. Moreover, BS is responsible for the scheduling for the whole cell.

Downlink dynamic scheduling

In the downlink, the default radio resource allocation is made dynamically by the BS MAC scheduler. For each TTI the scheduler shares PRBs between DRBs with Protocol Data Unit (PDU) in the RLC transmission buffer. The quantity of PRBs (i.e. the TBS) allocated to a DRB is decided according to a scheduling strategy, parameters (QoS requirements), and the network status (buffer occupancy, channel quality). The UE is informed of the reception of a transport block in a TTI with parameters to decode it from the DCI in the front of the TTI.

Uplink dynamic scheduling

In the uplink, the default radio resource allocation is also made dynamically by the BS MAC scheduler. But at the difference of the downlink, the user have to request access to the channel with a Scheduling Request (SR) ( §1.1.3.2), that we call explicit scheduling request access. In response the BS MAC scheduler ( §1.1.3.2) grants user with few radio resources with an Uplink Grant or grant of uplink capacity (UG), that is called grant-based access. SR is completed by the Buffer Status Report (BSR) ( §1.1.3.2). The procedure is depicted in Fig. 1.4 in the left part. This grantbased access is primarily used for uplink radio resource allocation because it solves the contention problem with a dynamic allocation that comes with a shared radio resource grid. It is particularly adapted for rapidly changing radio channel capacity and traffic datarate with many served users. We will discuss more about uplink dynamic scheduling in the further section § 1.1.3.2.

Fast Uplink access or pre-scheduling or configured-grant is a grant-based technique where the BS transmits a UG without the prior reception of a SR. The procedure is depicted in Fig. 1.4-"Fast-UL". The BS typically predicts when the UE should need access to the uplink channel to transmit new data. A small share of the bandwidth is commonly allocated regularly (few resource blocks). Thus, a good Signal-to-Noise Ratio (SNR) is necessary to transmit a IP packet in one transmission, limiting the technique to the users with a good radio conditions. Fast Uplink Grant (Fast-UL) is considered as an important low-latency enabler in RAN on the condition of a good estimation of data packet arrival by the BS (e.g. [START_REF] Condoluci | Soft resource reservation for low-delayed teleoperation over mobile networks[END_REF] studied the soft-reservation for latency-sensitive teleoperation in mobile networks). UE could transmit padding when the transmission buffer is empty, but it is not mandating as with Semi-Persistent Scheduling (SPS) (further complexity partly solved by using HARQ NACK and soft-combining).

However, resources are allocated to one user with the risk of a false estimate which would lead to radio resource wastage impacting the rest of the cell performance. The explicit SR method is more conservative than the Fast-UL concerning the radio resource allocation by granting only the resources requested by the UE.

Semi-Persistent Scheduling is a grant-free semi-static allocation technique by which a UE is provided with reserved radio resource during a period of time at a configured periodicity. The periodicity and the reserved resource blocks are network-configured with an RRC message prior to the activation of the SPS. L1/L2 RRC control activate and deactivation the SPS channel. Figures 1.4-"SPS" illustrates a data transmission when SPS is enabled with a fixed periodicity for transmission. TB padding occurs when there are no data left in the transmission buffer. SPS was standardized primarily for Voice over IP [START_REF] Jiang | Principle and performance of semi-persistent scheduling for VoIP in LTE system[END_REF][START_REF] Seo | Performance modeling and stability of semipersistent scheduling with initial random access in LTE[END_REF] but it is also as a solution for latency and jitter sensitive services with a deterministic traffic pattern. Dynamic scheduling is able to take into account radio variation of traffic generation and radio channel capacity while semi-static scheduling needs prior transmission parameters. 3GPP standards describe procedures and mechanisms associated to dynamic scheduling in [31-33, 39, 47-49].

MAC scheduler LTE and 5G standards concerning uplink scheduling look very similar except 5G introduces a new QoS model and more scheduling flexibilies at the physical layer [START_REF] Pedersen | Agile 5G Scheduler for Improved E2E Performance and Flexibility for Different Network Implementations[END_REF][START_REF] Patriciello | An Improved MAC Layer for the 5G NR Ns-3 Module[END_REF]. The 3GPP standard [START_REF] Gpp | Overall description[END_REF]- §10.3 defines the dynamic radio resource allocation by the BS and specifies that UEs always monitors the PDCCHs in order to find possible grants for uplink transmission when its downlink reception is enabled (see Discontinuous Reception (DRX) mechanism).

The section §10.1 of the technical specification [START_REF] Gpp | Overall description[END_REF] describes the basic scheduler operations which are:

-Taking into account the explicit scheduling request, UE buffer status and the QoS requirements -May assign resources taking account the radio conditions at the UE -Assign radio resources in a unit of a slot, with a minimal quantum of radio PRB The operation definition does not standardize how to perform the scheduling according to Scheduling Request (SR)/Buffer Status Report (BSR) or QoS requirements. Scheduling algorithm is a hot topic of research for RAN performances [START_REF] Khuram Ashfaq | Comparative analysis of scheduling algorithms for radio resource allocation in future communication networks[END_REF][START_REF] Li | User Scheduling for Uplink OFDMA Systems by Deep Learning[END_REF][START_REF] Moustafa | A hybrid downlink scheduling approach for multi-traffic classes in LTE wireless systems[END_REF][START_REF] Mahlouji | Analysis of Uplink Scheduling for Haptic Communications[END_REF][START_REF] Francesco | Learning radio resource management in RANs: Framework, opportunities, and challenges[END_REF][START_REF] Satheesh Balakrishnan | A review of MAC scheduling algorithms in LTE system[END_REF][START_REF] Eirini | Uplink resource allocation in SC-FDMA wireless networks: A survey and taxonomy[END_REF][START_REF] Abu-Ali | Uplink scheduling in LTE and LTE-advanced: Tutorial, survey and evaluation framework[END_REF] Especially, 5G scheduler comes with a larger number of options due to physical layer flexibilities [START_REF] Hyoungju | Introduction to ultra reliable and low latency communications in 5G[END_REF] and QoS framework [1]-6.2. The MAC technical specification document [START_REF] Gpp | Medium Access Control (MAC) protocol specification[END_REF]- §5.4 defines the Uplink Shared Channel (UL-SCH) data transfer. In particular, section §5.4.4 defines the procedure for UE to request UL-SCH resources for new transmission with a Scheduling Request (SR).

Scheduler is channel-dependent since 3G where transmissions to a device take place when the radio channel conditions are favorable and exploit channel variations. BS MAC scheduler is also in charge of the Link Adaptation (LA) [START_REF] Francesco | Scheduling and Link Adaptation for Uplink SC-FDMA Systems-A LTE Case Study[END_REF], including Power Control (PC). PC has an important role since it determines the transmission energy that should use UE to transmit with a target Block Error Rate (BLER) of maximum 10% giving a MCS. The transmission power is an important source of energy consumption by mobile [START_REF] Kanti | Power consumption analysis, measurement, management, and issues: A state-of-the-art review of smartphone battery and energy usage[END_REF], and an important performance criteria for mobile users and it is reduced with a too high transmission power while a too low transmission power generates radio link errors and retransmissions.

The outcome of the uplink scheduler is the radio resource assignment indicated by the UG.

Uplink Grant indicates in the downlink DCI time-frequency radio resource allocation to the UEs. The information in the DCI formats 0-0 or 0-1 precises the UE's RNTI, information about resource (frequency-domain allocation, time-domain allocation), about transport block (MCS, redundancy version, HARQ process number), about antenna (antenna ports, MIMO scheme) and about uplink power control. An UG is necessary to transmit data over a PRB and is the final outcome of the scheduling process that occurs every TTI.

Scheduling Request A SR is triggered when a new data arrives in the UE transmission buffer. Thus, the UE will send SR (the number of pending SR is limited by a configuration) until it has new transmission. Pending SR sent by UE are reset when a UG is received. SR periodicity configuration [START_REF] Gpp | Evolved Universal Terrestrial Radio Access (E-UTRA); Physical layer procedures[END_REF] carried by the scheduling request is limited (basically one bit) but could be transmitted regularly, typically, every 10 ms.

Buffer Status Reporting TS 3GPP 38.321 [START_REF] Gpp | Medium Access Control (MAC) protocol specification[END_REF] section §5.4.5 details the procedure to provide the serving base station with information about uplink data volume in the RLC entity, i.e. Buffer Status Report (BSR). The data volume calculation is detailed in the [START_REF] Gpp | Radio Link Control (RLC) protocol specification[END_REF]- §5.5. The BSR MAC CE (in Fig. 1.6a) reports the buffer occupancy associated to a logical channel group (≃DRB) with an index of 6-bits length. While the buffer status reporting is made per DRB group, the UG is given to one UE. This scheme has been discussed at the 3GPP in 2006 [START_REF]Comparison of UL buffer reporting/scheduling schemes in LTE[END_REF][START_REF] Ericsson | On the Granularity of Uplink Scheduling in LTE[END_REF]. Pradap et al. in [START_REF] Kv Pradap | Uplink buffer status reporting for delay constrained flows in 3GPP long term evolution[END_REF] reviews the different uplink buffer status reporting schemes considered by the 3GPP working group.

The reporting scheme is one trade-off. The other one is the BSR index table :   -In the width of the index range. An increase of this range implies an increase of the MAC CE size and then the MAC control overhead. -In the maximum value reported by the range. A too little value limits the maximum uplink throughput reporting and a too large value decrease the accuracy of the reported value. -The distribution function that takes a buffer size in input and returns an index.

This function could be defined to split the range equality, but the standard choose to give importance to lower values. For high index values, the accuracy of reported bytes decreases.

The BSR reported values of buffer occupancy according to the LTE's BSR

Three types of event trigger a BSR:

-When a new UL data is available in logical channel with a higher priority : Regular BSR -To fill padding bits when a UG allocates more bytes than it is necessary to empty the buffer : Padding BSR -At the expiration of a timer, the periodicBSR-Timer : Periodic BSR When transmission is running, the regular BSR should not be triggered in our experiment since we are using only the default non-QoS bearer. However, the transmission of a regular BSR is the result of an internal modem's process that it is neither networkconfigured nor standardized. The events that trigger BSR on our setup are the padding of a TB and the expiration of the periodicBSR-Timer when a TB is generated.

When the reported BSR value in LTE is 63, the base station estimates there are at least Q max BSR = 150 kB in the transmission buffer even if the UE transmission buffer is larger. The upper bound of BSR index Q max BSR table and the BSR periodicity T BSR limit the estimated uplink packet generation datarate of the UE such as:

U max reported = Q max BSR /T BSR (1.1)
The value of periodicity is network-configured and usually set to 16, 32 or 64 ms10 For a BSR reported every 64 ms, it implies D max reported = 18.75 Mbps. In the other side, a reported buffer size of 150 kB will be cleared in 11 or 400 TTIs in LTE (for a 20 MHz bandwidth) according to the MCS. Thus, the periodicity of BSR should be adapted according to the cell capacity and cell bandwidth to efficiently report the UE uplink traffic generation datarate. However, it is not the case in operated networks with fixed default values.

Summary

In summary, The default radio resource allocation in the radio access network is dynamic and the result of the BS MAC scheduler. The UG informs devices for an uplink transmission opportunity. We presented 3 uplink access methods in Fig. 1.4. SPS and Fast-UL efficiently reduce uplink access latency. A latency analysis of each allocation scheme is provided in [START_REF] Mahlouji | Analysis of Uplink Scheduling for Haptic Communications[END_REF] and in [START_REF] Camilo | Ultra-low latency in next generation LTE radio access[END_REF]. However, SPS suffers from a lack of adaptation to condition changes and control overheads and Fast-UL is subject to radio resource wastage when UGs are not adapted to the transmission needs.

The explicit SR method is based on scheduling request and buffer status reporting to adapt UGs to transmission needs. The typical time-sequence radio resource allocation committed in the standard is illustrated in Figure 1.4-"Explicit Scheduling Request". A new data arrives at the UE transmission buffer that triggers a Scheduling Request (SR) with a certain delay. The scheduling process is done dynamically every TTI. The BS in return allocates dynamically some resources to the UE with a grant to which UE responses with a Buffer Status Report (BSR) to report the volume of data awaiting to be transmitted and some user data if it leaves some space in the TB after the control plane elements and signalling. SR and BSR mechanisms main properties are summarized in Table 1.1.

Technical specifications do not explicit the link between the BSR, the SR, QoS and the UG as they do not explain how the BS should take into account the BSR and the SR internally. However, we understand the BSR is used internally by the BS to maintain a buffer status estimation that should be used in scheduling operations.

Internet Protocol Suite

The Internet protocol suite commonly uses HyperText Transfer Protocol (HTTP) [START_REF] Fielding | HTTP Semantics[END_REF] as application layer, Transport Layer Security (TLS) [START_REF] Rescorla | The Transport Layer Security (TLS) Protocol Version 1[END_REF] as security layer, Transmission Control Protocol (TCP) as transport layer and IP [START_REF]Internet Protocol[END_REF] for network layer communications. The delivery of data packets between client and server endpoints is ensured by the transport layer protocol. By 2022, TCP is largely dominant worldwide [START_REF] Schumann | Impact of Evolving Protocols and COVID-19 on Internet Traffic Shares[END_REF] with ≈ 90% of the traffic followed by Quic, ≈ 8% (30% in mobile networks 11 ) and User Datagram Protocol (UDP) (≈ 1.5%). TCP transports the biggest part of the internet HTTP traffics including web browsing, video streaming, communications and document sharing [START_REF] Sandvine | Global Internet Phenomena Report[END_REF]. Quic [START_REF] Iyengar | QUIC: A UDP-Based Multiplexed and Secure Transport[END_REF] in the next decade should replace TCP with HTTP/3 [START_REF] Bishop | HTTP/3. RFC[END_REF] for the most of these usages. UDP is a datagram transport protocol used by other protocols (e.g. Quic) which is ultimately not so used for user traffic except for latency-sensitive real-time applications.

Transmission Control Protocol (TCP)

TCP [START_REF]Transmission Control Protocol[END_REF] is a reliable transport protocol which uses Acknowledgment (ACK) and retransmissions to ensure the reception of all bytes. TCP is connection-oriented which mean that a full-duplex communication between endpoints is established after a handshake procedure (exchange of SYN, SYN-ACK, ACK packets) and closed at the end of the transfer (with FIN and FIN-ACK packets). To keep the communication reliable, TCP implements packet loss detection and recovery mechanisms. TCP also adopts to control the congestion (TCP must use a congestion control algorithm [START_REF] Dr | TCP Congestion Control[END_REF]) of the link with a continuous estimation of path capacity to avoid losses. That is the condition to avoid saturating network and losses due to a buffer overflow.

Congestion control

The Congestion Control Algorithm (CCA) is in charge of the congestion control and is able to do it using the feedback loop providing by the return ACK flow [START_REF] Van Jacobson | Congestion avoidance and control[END_REF]. This feedback loop is illustrated in Figure 1.7. Acknowledgement packets initially commit the correct reception of a packet by the client. Thereafter, the transmission control continues the transfer by sending a new data packet. The volume of data "in-flight" in the network 11. https://www.rcrwireless.com/20220211/opinion/readerforum/heres-why-quic-is-still-worryingmobile-operators-reader-forum In addition to report packet loss in the network, the ACK flow carries implicit hints on the network congestion [START_REF] Custura | Rethinking acks at the transport layer[END_REF]. Implicit hints include the measurement of the RTT, the variation of the RTT over time and the traffic transmission pattern. Congestion Control Algorithm (CCA) is composed of a congestion estimator and a congestion window controller. It must adapt the congestion window to the network conditions throughout the transfer duration. There are several versions of TCP which implement different congestion control algorithms [START_REF] Polese | A Survey on Recent Advances in Transport Layer Protocols[END_REF]-Table II. Among them, CUBIC and Bottleneck Bandwidth and RTT (BBR) are very common on the internet (> 60% [START_REF] Mishra | The Great Internet TCP Congestion Control Census[END_REF]). The Cubic algorithm (default CCA in Linux) estimates the available network path capacity from data packet losses. BBR initially developed by Google, uses a combination of RTT and bottleneck bandwidth estimations to estimate network path capacity. Research and developments around BBR are very important, and some experts announce that in the near future BBR will take over from cubic without replacing it [START_REF] Mishra | Are we heading towards a BBRdominant Internet?[END_REF]. Artificial Intelligence (AI)-based CCA (e.g. [START_REF] Winstein | Tcp ex machina: Computer-generated congestion control[END_REF]) and CCA adapted to mobile networks [START_REF] Lorincz | A Comprehensive Overview of TCP Congestion Control in 5G Networks: Research Challenges and Future Perspectives[END_REF][START_REF] Poorzare | Challenges on the Way of Implementing TCP over 5G Networks[END_REF] are also active research topics.

Quic

Quic [START_REF] Iyengar | QUIC: A UDP-Based Multiplexed and Secure Transport[END_REF] is the next generation of transport protocol for internet services [START_REF] Molavi | Taking a long look at QUIC: an approach for rigorous evaluation of rapidly evolving transport protocols[END_REF][START_REF] Langley | The quic transport protocol: Design and internet-scale deployment[END_REF]. Quic is based on decades of research and experiences of internet transport protocols. It meets the needs for more efficient internet transmissions, enhanced security and user privacy. Especially, the security aspect is put at a high level for Quic with a total packet encryption, including protocol header, that is a major change compared to TCP. It ques-tions the utilization of middle-boxes in the network for traffic protocol optimizations 12 (Performance Enhancing Proxies [START_REF] Liu | On improving TCP performance over mobile data networks[END_REF]). Quic must be able to replace TCP on the internet to join evolution of user needs and latest network innovations. It already accounts for a significant portion of the Internet traffic [START_REF] Sandvine | Global Internet Phenomena Report[END_REF]. Just as TCP, Quic guarantees the reliability of user data transport but with a revisited acknowledgement mechanism [START_REF] Iyengar | QUIC Loss Detection and Congestion Control[END_REF]. A CCA is also used following a similar model to TCP with BBR implemented by default for a number of protocol stacks 13 .

12. TCP Optimization Enhancements for a Better Mobile Experience, F5 (2017) : https://www.f5. com/company/blog/tcp-optimization-enhancements-for-a-better-mobile-experience [START_REF] Soret | Fundamental tradeoffs among reliability, latency and throughput in cellular networks[END_REF]. List of open-source Quic stack implementations : https://github.com/quicwg/base-drafts/ wiki/Implementations

Latency in Networks and Mitigation

In this section, we discuss latency in networks and their mitigation with a special focus on latency in mobile networks. Previous Sections 1.1 and 1.2 respectively introduced mobile networks technologies and Internet protocols. Latency is studied and defined in 3GPP documents for mobile networks and in Request For Comments (RFC) documents from transport area working group of Internet Engineering Task Force (IETF) for the Internet area 14 . The definitions of latency found in standards are presented in subsection 1.3.1. The subsection also discusses the distribution of end-to-end latencies. Afterwards, we present in subsection 1.3.2 the well-known network bufferbloat that is an important latency issue in networks, in particular in mobile networks, and which is an active research topic. In subsection 1.3.3, is presented the techniques of mitigation for latency implemented in 5G and proposed in the literature. Latency 3GPP TS 22.261 [START_REF] Gpp | Service requirements for the 5G system[END_REF] defines end-to-end latency as "the time that takes to transfer a given piece of information from a source to a destination, measured at the communication interface, from the moment it is transmitted by the source to the moment it is successfully received at the destination". It corresponds to the definition given in RFC7679 [START_REF] Almes | A One-Way Delay Metric for IP Performance Metrics (IPPM)[END_REF] of the One-Way delay metric where the given piece of information is an IP packet. In Figure 1.8, end-to-end latency does not differentiate the channel, uplink or downlink. The latency associated to a path direction is called One-Way Delay (OWD) on the figure and can be of uplink for latency from mobile to IP network or downlink for latency from IP network to mobile. 5G RAT Key Performance Indicators (KPIs) are defined in 3GPP TS 38.913 [START_REF] Gpp | G; Study on Scenarios and Requirements for Next Generation Access Technologies[END_REF]-7. Especially it defines the control plane latency, the user plane latency and the latency for infrequent packets.

Definition of latency

The control plane latency is understood as the time to move from idle to active state (ready to transfer data) for a UE (i.e. the time to effectuate Random Access procedure Channel (RACH) and initial access procedure Appendix A.7.6).

The RAN One-Way Delay (OWD) is defined as "the one-way transit time between a packet being available at the IP layer of the UE/RAN edge node and the same packet being available at the RAN edge/UE. The RAN edge node is the node providing the radio access network interface towards the core network". The RAN RTT is the sum of uplink and downlink RAN OWDs. Generally, when a latency for a 5G system is given, it refers to the RAN RTT.

Latency for infrequent small packets is a mix of RAN OWD and control plane latency because it is the time it takes to successfully deliver an application layer packet from the mobile device to the egress point in the RAN when the device starts from its idle state.

The mobile network latency is the latency of the network nodes serving UEs, i.e. base station, backhaul network and mobile network core, excluding the air interface and mobile. In Fig. 1.8, on-BS latency and on-device latency are also shown as latency segments.

The Round-Time Trip (RTT) is the delay for a packet to reach from the source to the destination and a response back to the source. RTT is usually measured with ping [START_REF] Kalidindi | A Round-trip Delay Metric for IPPM[END_REF] and is equal (subject to some uncertainties due to the method of measurement with a program) to the sum of uplink and downlink OWDs. Network RTT is a KPI of the network since it measure the latency experienced by a user while it interacts with a distant machine via the network.

Network segments Each network segment on the path illustrated in Fig. 1.8 comes with different types of delay regarding its property. There are queueing delay due to buffering, propagation delay due to physical limits of the information speed (2.10 5 km/s in a fiber, 3.10 5 km/s in the air), processing delay due to network processing and error recovery delay. For instance, server OS segment consists in on-chip processing delays, latency of RLC layer segment inside BS is dominated by queueing delays and latency of radio interface, bounded by the TTI duration. In total, the mobile network latency is composed of:

T = T RAN + T Backhaul + T Core (1.2)
Where T Backhaul is the latency of the backhaul network (mostly transport latency), T Core , the latency of the core network (associated to routing latency and transport to PDN) and T RAN , the latency introduced by the RAN, and the focus of this thesis.

Traditionally, the effects of quick varying radio channel capacity and transmissions losses in the RAN are handled using multiple packet buffers with different latency characteristics.

-A transmission buffer which stores arriving data packets to be transmitted. It is made large in mobile networks to take advantage of new available radio capacities in a millisecond timescale and conversely to avoid buffer overflow packet flow when the available capacity shrinks. -An handover buffer to proceed to device mobility from one base station to another seamlessly without losses. -An aggregation buffer to collect data from different bearer and generate a TB in each transmission opportunity. -Retransmission buffers related to MAC HARQ and RLC ARQ processes.

-An in-sequence buffer to avoid excessive reordering.

Jitter Until now, we have defined the absolute latency of an individual packet.

The Packet Delay Variation (PDV) is given for a pair of packets within a packet flow between 2 measurement points. In other words, the PDV is the difference between the OWD of a selected pair of packets (RFC 3393 [START_REF] Demichelis | IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)[END_REF]). PDV is also called latency variation or jitter, even if the last designation is viewed as confusing [START_REF] Demichelis | IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)[END_REF]-1.1. ITU-T document Y.1540 16 (2019) §6.2.4 calls it 2-point packet delay variation and gives the preferred method for summarizing the delay variation based on quantile. Quantile-based limits of PDV method is to select upper and lower quantiles of the delay variation distribution and then measure the distance between those quantiles.

In the following, we use this method with the 10 th percentile and the 90 th percentile as illustrated with the Cumulative Distribution Function (CDF) of latency for a given connection in Fig. 1 we indicate minimum latency, which is the minimum OWD observed for a given segment and tail latency, also known as high-percentile latency, which is high latencies that packets experienced infrequently.

Jitter is an important metric for the buffer sizing [START_REF] Showail | Buffer sizing in wireless networks: challenges, solutions, and opportunities[END_REF] and to determine the dynamics of queues withing a network. Each segment of the network generates more or less jitter. For instance, in-sequence delivery of PDCP causes "flushing" effect that generate lot of latency variation while a non-congested processing function generates latency but not latency variation.

Summary Three metrics are used to characterize a network segment, the OWDs, the RTT and the jitter. Users experienced the end-to-end latency, composed of a series of latency with different characteristics introduced by each segment on the network path. This thesis focus on the latency of the RAN.

Bufferbloat

Some network segments latency are dominated by the queueing delay (or buffering delay). Queueing delay appears when packets arrive at a network node at a higher rate than the node can process them and puts them into a buffer. It is a key component of delay in packet switched network. It has been observed by Gettys et al. in [START_REF] Gettys | Bufferbloat: Dark buffers in the internet[END_REF] (2011) that today's computer networks are suffering from unnecessary latency and poor system performances. They define the bufferbloat (i.e. excess queueing latency) phenomenon, which is the existence of excessively large and persistently full buffers inside the network. Bufferbloat is a major concern of the research since then to unleash network performances. Optimal buffer sizes should be equal the Bandwidth-Delay Product (BDP), however, as the delay is hard to estimate, larger buffers have been deployed (the problem of buffer sizing) to raise throughputs and prevent losses. It has favored the appearance of bufferbloat.

It is prevalent on last-mile access network (e.g. LTE, Wifi.) [START_REF] Hoiland-Jorgensen | Bufferbloat and Beyond -Removing Performance Barriers in Real-World Networks[END_REF] since this network segment aggregates multiple flows and have recourse to large buffer because of the channel capacity variation and the needs of error recovery. An important work on bufferbloat in Wifi has been done by Toke Hoeiland-Joergensen in [START_REF] Hoiland-Jorgensen | Bufferbloat and Beyond -Removing Performance Barriers in Real-World Networks[END_REF]. Especially, it indicates bufferbloat is very real and common in radio access network and tends to be significant, most often on the order of several hundreds milliseconds (more than 20000 km equivalent in an optical fiber).

A direct solution to tackle bufferbloat is to control dynamically to maximum queue size instead of letting it fills up. Hoeiland as many research works proposes aqm [START_REF] Kuhn | Characterization Guidelines for Active Queue Management (AQM)[END_REF][START_REF] Bortzmeyer | IETF Recommendations Regarding Active Queue Management[END_REF] solutions [START_REF] Khademi | The new AQM kids on the block: An experimental evaluation of CoDel and PIE[END_REF][START_REF] Adams | Active Queue Management: A Survey[END_REF][START_REF] Høiland-Jørgensen | Piece of cake: a comprehensive queue management solution for home gateways[END_REF][START_REF] Barreiros | QoS-Enabled networks: Tools and foundations[END_REF] where queues are not yet considered as infinite First-In First-Out (FIFO) queues but implement a packet dropping policy (e.g. Adaptive Random Early Detection (ARED), Controlled Delay (CoDel) [START_REF] Høiland-Jørgensen | The Flow Queue CoDel Packet Scheduler and Active Queue Management Algorithm[END_REF], Proportional Integral controller Enhanced (PIE) [START_REF] Pan | Proportional Integral Controller Enhanced (PIE): A Lightweight Control Scheme to Address the Bufferbloat Problem[END_REF]). Bufferbloat is also prevalent in cellular networks [START_REF] Zhang | Transport layer performance in 5G mmWave cellular[END_REF][103]-2 and causes many problems for low latency services but also for Transmission Control Protocol (TCP) performances [103]-3. The bufferbloat is concentrated at the bottleneck, therefore at the level of the Radio Access Network (RAN). In the downlink path of RAN, it is the Radio Link Control (RLC) transmission buffer which stores the packets before they are multiplexed and transmitted over the radio channel.

Figure 1.10 depicts the Round-Time Trip (RTT) over time of a TCP transfer in a commercial mobile network. At the beginning of the transfer, the RTT is close to the end-to-end RT T min (≈ 30 ms). Thereafter, the RTT increases with some peaks due to the packet scheduling in the RAN but never reduce with a steady latency of 1 second, which is considerable. The data volume in-flight in the network reaches a maximum of 1.6 Mbytes (= 1100 Internet Protocol (IP) packets) without packet losses. Thus, we estimate the RLC buffer associated to one Data Radio Bearer (DRB) (i.e. one User Equipment (UE)) is at least 1.6 MB, it is ten time larger than the Bandwidth-Delay Product (BDP) that is equal to 5.375 × 0.03 = 0.161 MB 17 . However, a large buffer is necessary to handle rapid fluctuation of the radio channel capacity (no starving when new network capacities are available), and transmission loss (when the network capacity is degrading).

Bufferbloat tackling is a complex challenge [START_REF] Zhang | The bufferbloat problem over intermittent multi-gbps mmwave links[END_REF] and an important issue for cellular networks performances [6] not only for low latency services but also for transport protocols. [START_REF] Briscoe | Reducing Internet Latency: A Survey of Techniques and Their Merits[END_REF] for mobile network case). P: processing, V: propagation, Q: queueing, S: scheduling, E: link error, I: medium acquisition, H: handover.

Latency Mitigation in 5G

It's time for low latency in cellular networks. Briscoe et al. in [START_REF] Briscoe | Reducing Internet Latency: A Survey of Techniques and Their Merits[END_REF] (2016) investigate techniques for reducing internet latency. In [START_REF] Briscoe | Reducing Internet Latency: A Survey of Techniques and Their Merits[END_REF]-Fig. 2 they propose a classification of source of delay and techniques for reducing latency. They identified 5 types of delays which are: §II) Structural delays, related to network architecture; §III) Interaction between endpoints (transport and session initialization, E2E packet loss and recovery); §IV) Delays along transmission paths which cover propagation delays, link error recovery, queueing delay, medium acquisition... ; §V) Delays related to link capacities such as insufficient capacity or under-utilized capacity; §VI) Intra-end host delays, mostly operating system and network stack delays.

We use their classification to establish Table 1.2 which summarizes 5G mechanisms and related research for latency reduction in mobile networks. Some related studies are given in the last column. We do not present a comprehensive survey of low latency for 5G. To going further, [START_REF] Fan | An Overview of Low latency for Wireless Communications: an Evolutionary Perspective[END_REF] (2021) gives an overview of low latency for wireless communications with an evolutionary perspective from 2G to 5G. [START_REF] Parvez | A survey on low latency towards 5G: RAN, core network and caching solutions[END_REF] (2018) is a comprehensive survey on low latency for 5G with an end-to-end analysis, RAN, core, network and caching. [START_REF] Nasrallah | Ultra-low latency (ULL) networks: The IEEE TSN and IETF DetNet standards and related 5G ULL research[END_REF] describes Time-Sensitive Networks (TSN) and Deterministic Networks (DetNets) standards and related 5G research for 5G ultra-low latency networks.

5G system is designed to go towards low latency transmissions, 10 ms for best effort bearer and 1 ms for time-critical bearer. To ensure that, 5G employs both end-to-end and RAN tools.

End-to-end tools in relation with RAN End-to-end toolbox includes the traffic differentiation, the network softwarization, the content placement and the network architecture and management [170]. The new 5G Quality of Service (QoS) framework introduces the end-to-end QoS flow architecture where packets are carried in virtual circuit associated to a QoS class (e.g. Default best-effort, latency-optimized, service-optimized, etc.) Differentiation of traffics allows the Medium Access Control (MAC) scheduling to prioritize a flow over another and radio resource reservation (see Table 1.2-"Flow and circuit scheduling"). It comes with the network slicing which enables a virtual isolation of end-to-end network on the same physical network infrastructure.

Slicing [START_REF] Afolabi | Network slicing and softwarization: A survey on principles, enabling technologies, and solutions[END_REF] comes from the concepts of software-defined networking (SDN) [START_REF] Astuto | A survey of software-defined networking: Past, present, and future of programmable networks[END_REF] and network function virtualization (NFV). In practice, Slicing is the reservation of a bandwidth part to a slice for a specific service. QoS flows and slicing are useful tools for latency-sensitive latency scheduling.

Cloud-RAN (C-RAN) [START_REF] Wu | Cloud radio access network (C-RAN): a primer[END_REF][START_REF] Salman | Cloud RAN: Basics, advances and challenges[END_REF] allowed by network softwarization redefines the Base Station (BS) network functions placement in a pool to share the same infrastructure (see Table 1.2-"Virtualization"). For instance, handover between base stations is facilitated with a shorter handover latency. Moreover, BSs higher layers and control plane could be placed in the same datacenter facility as the 5GC to reduce control place latency.

There has been a trend in recent years to bring content closer to clients with copy of the content stored in a decentralized way e.g. with Content Delivery Network (CDN) (see Table 1.2-"Content placement"). This network architecture is completely taken into account in the design of the 5G architecture. The content could be now co-located in a data centre with the core and the cloud-RAN. Mobile Edge Computing (MEC) 18 redefines the content and computing location closer to the client, reducing efficiently latency [START_REF] Varghese | Revisiting the Arguments for Edge Computing Research[END_REF]. In this architecture, the packet path is shortened to the RAN and a backhaul network. However, even if this architecture could be employed for all type of traffics without traffic differentiation (they are using IP routing), the deployment cost is very high, limiting its utilization for only few latency-specific services (e.g. cloud gaming, XR [START_REF] Alriksson | XR and 5G: Extended reality at scale with time-critical communication[END_REF]).

RAN tools Radio Access Network (RAN) segment counts for the larger part of latency in the mobile network with a baseline RTT of 20 ms in LTE for a total mobile network latency usually around 30 ms. Third Generation Partnership Project (3GPP) published several documents in 2016 where latency reduction is studied for LTE [START_REF] Gpp | Study on latency reduction techniques for LTE[END_REF]. Some practical recommendations have been given concerning network configurations for low latency [START_REF] Zhang | LTE optimization engineering handbook[END_REF]. These studies inspired the development of New Radio (NR). 5G-NR Radio Access Technology (RAT) is able to tend towards low latency thanks to key features [START_REF] Dahlman | The next generation wireless access technology[END_REF] that distinguish it from LTE. Latency we found in RAN are of different types: 1) Processing (i.e. service time in queueing theory) delay; 2) Propagation delay; 3) Queueing (or buffering) delay; 4) Scheduling delay (i.e. insufficient capacity); 5) Link error recovery delay; 6) Medium acquisition delay; 7) Handover latency. In table 1.2, paragraphs below are put in relation with delays and techniques identified by [START_REF] Briscoe | Reducing Internet Latency: A Survey of Techniques and Their Merits[END_REF] in the column "Paragraphs".

Processing delay Delays associated to data processing functions at every layer of the radio protocol stack (Fig. 1 The data serialization (Table 1.2-"Concatenation and segmentation") has been revisited for Packet Data Convergence Protocol (PDCP) and RLC layers to avoid latency due to segmentation and in-sequence delivery. In-sequence delivery means a fully decoded Protocol Data Unit (PDU) with a sequence number N could not be delivered until the N -1 PDU are not fully decoded and transmitted. Thus, a retransmission at a lower layer of an old PDU might block the transmission of several PDUs. In-sequence has been removed from PDCP layer (since after PDCP, the segmentation is handled by IP layer for user-plane) and revisited for RLC layer. Especially, RLC entities are not allowed anymore to perform concatenation of multiple PDCP Service Data Units (SDUs) into one PDU since the MAC is already in charge of multiplexing multiple RLC PDU. This simplifies the PDU decoding at the counterpart with a low impact on achieved Transport Block (TB) utilization rate and control overhead.

Low-Density Parity-Check Code (LDPC) (user-plane) and polar codes (control-plane) for data encoding replaces the LTE turbo-coders with a gain in terms of spectral and computational efficiency (Table 1.2-"Leveraging interfaces"). LDPC in particular changes the transmission scheme by introducing more flexibilities. As in LTE, in NR as well a huge TB is split into multiple Code-Blocks (CBs), but NR introduce the concept of Code-Block Group (CBG) that are coded and decoded independently without prior buffering. As soon as a CB is decoded, it is transmitted to MAC layer to reassemble the transport block. Since the TB structure has been modified to put headers inside the TB instead of all at the beginning, a block of data could be transmitted as soon as it is decoded when in LTE we had to wait the decoding of the whole TB (which could be huge in 5G) before transmitting all blocks of data. More efficient coder/decoder and early transmission of CBs, the Transmission Time Interval (TTI) could be shortened below 1 ms.

In general, the current hardware performs better than when LTE was launched, which enables a reduction of the processing times defined in the standard. The processing time for data encoding and decoding defined as 3 ms in the two cases in LTE is now more flexible and lowered to one TTI duration in 5G.

Propagation delay

The time of propagation is bounded to the TTI duration, it is the time to fully receive a transport block. The TTI of 1 ms endorses the propagation of the radio wave in the air (300 km per ms), the physical transmission system delay (i.e. antenna) and the decoding time at the Physical Layer (PHY) layer. The adding of multiple numerologies in 5G enables transmission in a fraction of millisecond ( §1.1.2.2, Table 1.2-"Short transmission time"). There are also a propagation delay in the backhaul network (using fiber) to carry data packet to the network core and after that in the Packet Data Network (PDN). A closer content palliates to an important propagation delay in the mobile network.

Queueing delay is the most important source of latency and occurs when end hosts transmit more packets than the RAN can sustain. The bufferbloat presented in §1.3.2 is the most illustrative bad consequence. There are several ways to mitigate the congestionrelated delays in the RAN. The queueing management provides an efficient solution to queuing delays (Table 1.2-"Smaller network buffer","Queuing management"). RLC transmission buffer is usually designed large to handle rapid fluctuation of radio channel with the drawback to generate an important queueing latency when it is full. A research direction is to adapt the RLC buffer size dynamically to the channel conditions. Another promising research direction is the utilization of AQM systems combining queueing management, end hosts and congestion signalling (e.g. Explicit Congestion Notification (ECN) [START_REF] Johansson | Motivation to improved ECN handling in NR[END_REF]) for cellular networks.

The most active project on this subject is Low Latency, Low Loss, Scalable throughput (L4S) applied to cellular networks. L4S [START_REF] Ericsson | Use of L4S in 5GS[END_REF] is a method that provides fast indication of congestion from RAN, scalable end hosts (Congestion Control Algorithm (CCA) which reacts proportionally to congestion hints) and a queueing management based on 2 queues (one for L4S capable flows and one non-L4S capable). L4S comes from the RITE project 19 , a project funded by the European Union for reducing internet latency. L4S is well discussed in the literature and in Internet Engineering Task Force (IETF) working groups and is the subject of studies to be applicable to 5G networks like in Bruhn [START_REF] Brunello | Low Latency Low Loss Scalable Throughput in 5G Networks[END_REF] (2020).

In the same idea, the cooperation between end points and network is a promising direction to mitigate congestion and fully utilized radio channel capacity. For that, adapted congestion control and explicit congestion hints are employed (Table 1.2-"Under-utilized capacity").

Scheduling delay is related to the multiple aspects of cellular networks. Principles behind the scheduling improvements are the reduction of the access latency, the fastest possible adaptation to fluctuations in the radio channel and the wider bandwidth for increased radio channel capacity (Table 1.2-"Packet scheduler","leveraging interfaces").

We presented in §1.1.3.2 the different radio resource allocation schemes with grantfree and grant-based access. The L2 pre-emption reduces drastically the access latency by quickly schedule an urgent latency critical packet with a short TTI that overwrites another ongoing downlink transmission. DL Semi-Persistent Scheduling (SPS) and Fast Uplink Grant (Fast-UL) reduces also latency by cancelling the needs of waiting for a data arrival (DL) or the reception of a Scheduling Request (SR) (UL). Fast-UL is considered as an important factor of access latency reduction in the uplink, but it comes with the problem of the estimation of transmission needs.

Grant-based dynamic scheduler benefits from radio frame structure flexibilities (e.g. OFDM-symbol level scheduling), higher numerologies (e.g. shorter TTI), duplex scheme (e.g. full-duplex transmission scheme) and flexible scheduling timing (e.g. timing between grant and transmission) [START_REF] Pedersen | Agile 5G Scheduler for Improved E2E Performance and Flexibility for Different Network Implementations[END_REF] to reduce access latency for a wide range of users. All of these combined with a more efficient channel sounding and signalling, schedulers will be more agile to adapt quickly to channel variations. Finally, with Advanced Antenna System (AAS), massive Multiple-Input Multiple-Output (MIMO) [START_REF] Jeffrey | What will 5G be?[END_REF]-II.C, beamforming, wider bandwidth, higher Modulation and Coding Scheme (MCS) and higher frequencies, schedulers will be able to leverage interface capacity and mitigate insufficient capacity in all circumstances. Several propositions of scheduler already show how to use efficiently newly available radio capacities, especially for mmWave radio channel [START_REF] Jeffrey | What will 5G be?[END_REF]-II.B.

Link error recovery delay Link error recovery are mostly handled by the Hybrid-ARQ (HARQ) process. 5G uses HARQ retransmission procedure using incremental redundancy to improve successful transmission decoding probability with the soft combination of information from multiple transmission attempts. Also, retransmissions are supported on a finer granularity with CBG, limiting the impact of an error on a part of the transport block at few CBG instead of the entire TB. The HARQ ACK/NACK timing is flexible of 1 to 15 instead of fixed 4 TTI in LTE. The asynchronicity of HARQ is an important difference in error recovery compared to LTE and an important enabler for reducing radio link latency. It was common in LTE to see bursts of data after 8 ms of pause because of an HARQ retransmission. Retransmissions in 5G should be less visible on latency Empirical Cumulative Distribution Function (ECDF). Research is still active on HARQ with multiple proposed method to increases the recoverability of data [START_REF] Duhamel | Recent advances in HARQ communications[END_REF].

The RLC Acknowledged Mode (AM) entity takes a part in the remaining radio link recovery after HARQ process with the RLC Automatic Repeat reQuest (ARQ) process.

In this task, [START_REF] Levasseur | Impact of acknowledgments on application performance in 4G LTE networks[END_REF] shows the impact of acknowledgments on application performance in LTE and propose to adjust RLC timers in §5.3 to limit this latency. [START_REF] Austrheim | Implementing immediate forwarding for 4G in a network simulator[END_REF] discusses the desirability of removing ARQ at the RLC layer for latency-sensitive traffic by using an Unacknowledged Mode (UM) entity instead of AM entity. It could be made only if traffic flows are differentiated between those loss-sensitive and those latency-sensitive.

Initial access delay

The initial access is the procedures by which a UE established the connection with a cell through the Random Access procedure Channel (RACH) procedure and the a Radio Resource Control (RRC) session is established/re-established, ready to transmit and receive data. The latency reduction for the initial access delay is mainly related to the RRC session re-establishment and an optimized state machine transition from inactive to active with the arrival of the intermediate IDLE state.

Handover latency Mobile device often moves from one cell to another when it is moving. Handover has been identified as an important source of latency spikes during the transfer of a base station to the other [129]-6.2. Handover procedure begins with control signalling exchange following the re-routing of the flows from the mobile core and the transfer of the PDCP transmission buffer from the old BS to the new one.

The increased cooperation between BSs and multiple attachments makes the handover procedure smoother for the mobile users, limiting latency spikes. Also, the physical colocation of BS higher-layer network functions in the same pool will decrease the transfer latency of one BS to another. Summary 5G-NR RAT introduces many improvements for reducing latency in mobile networks at RAN and end-to-end levels. Latency reduction in RAN will come from an optimized processing time, a better link error recovery handling, a dynamic and flexible scheduler, and a work to tackle queueing latency. The objectives are both to reduce the baseline RAN RTT and to contain high latencies inherent to cellular radio networks (i.e. bufferbloat). 5G gives tools to operate low latency networks, but, a lot of research work and experiments remain to be done to effectively brings achieve low latency. We give the example of the complex agile dynamic scheduler or the Active Queue Management (AQM).

Objectives

This thesis is written in the field of computer networking. This field can be viewed as being comprised of three different aspects namely mathematics, engineering and science. Mathematical aspect provides theoretical foundations in particular of what is achievable and computable. Computer networking field has always been throughout its history linked to engineering. A huge engineering work has been necessary to build physical part of computer networks in a sense that internet is probably the most important and complex machine build by the humankind. Software engineering is now everywhere in the network developments especially when we consider the tendency of network softwarization. Scientific method is important in computer when the network itself can become the subject of study.

Simulation (e.g. ns-3)

+ -Realism

Emulation (e.g. mahi mahi) In networking, the scientific studies are usually based on one of the three following approaches (see Fig. 2.1): 1) Simulation where the network is entirely simulated with a computer program; 2) Lab experiments where the system under study is placed in a controlled environment with sometime emulation for non-essential parts; 3) Real-world experiments where the system under study is placed in a representative environment. The research presented in this thesis takes a use of three methods [START_REF] Imran | A survey of simulators, emulators and testbeds for wireless sensor networks[END_REF] with always the idea that observations and ideas developed are as close as possible to real world systems.

Lab testbed commercial network experiments

The research process we followed in this work is primarily based on a lab experiment approach, complemented by simulation and commercial network experiments when necessary. The lab testbed we present in the next section is a trade-off between the realism of a commercial network experimental setup that represents customer experience of the networks and a computer simulation where the mobile network is modeled. In fact, too often work based only on simulations are finally never deployed on the internet because idealisations often leading to very different behaviour on networking devices.

The objectives of the lab testbed are multiple:

1. It should exhibit the issues related to latency in radio access networks, completed by the literature and the commercial network trials;

2. It should permit a deep and comprehensive analysis of the latency root causes, especially, it implements probes and capture points.

3. It is used as a support for the evaluation of the proposed solutions.

The testbed reproduces a cellular radio access network (i.e. mobile terminals, base station) completed with a mobile network core and a local server to close the path between the client and the server.

The following Section 2.2 presents the lab testbed setup aspects. Section 2.3 describes the application which generates the network traffic. The last Section 2.4 of the chapter explains the analysis tools and methods for latency analysis in use in the testbed.

Lab testbed with OpenAirInterface

In this section, we describe the main testbed used in this thesis. It makes a link between real-world cellular networks and simulations. In one hand, the proposed experimental setup is a white box for latency analysis in contrast to deployed networks. On the other hand, ideas developed on a real radio interface ensures that they can be implemented in real systems (representativity of experiment in contrast to simulations). For these reasons, an open-source software cellular network is the most suitable solution. We chose to use OpenAirInterface (OAI)[176] [2] as the base of the presented lab testbed.

A quick review of the existing solutions for cellular network research is proposed on the next section followed by a presentation of testbed and finally a brief technical introduction to OAI radio air interface.

Comparison of RAN solutions

Solution LTE 5G Open * Advantages Limitations Amarisoft ✓ ✓ ✗ •Already deployed • Closed sources • Expensive Radisys ✓ ✓ ✗ • Advanced open-RAN • Closed sources • Expensive srsLTE ✓ ✓ ✓ • Stable • Modularity • Small community • Good code quality OpenAirInterface ✓ ∼ ✓ • Active project, • 5G developments • Research oriented, On-going • Large community • Code quality Table 2.1 -Solutions for cellular network research testbed. * Open: Open-Source.
The lab testbed is a research platform designed around a private cellular network. We considered 4 solutions of private cellular network presented in Table 2.1. An evaluation of different solutions, especially of OAI, is provided in a recent work of Vilakazi et al. [START_REF] Vilakazi | Evaluating an Evolving OAI Testbed: Overview of Options, Building tips, and Current Performance[END_REF].

Amarisoft 1 provides LTE BS (BS), 5G BS (gNodeB (gNB)) and UE software which run on off-the-shelve hardware. The callbox is an easy-to-deploy closed linux machine on which runs a gNB and a core network. It is designed for LTE and 5G testing of user equipment with advanced configurations. Amarisoft solution is a very powerful machine for cellular network research.

In the same idea, Radisys 2 offers a complete 3GPP compliant RAN software. Radisys 5G software runs also on generic hardware and claims to have open interoperable interface. srsLTE 3 [178] is one of the most important open software for mobile wireless network. It implements a complete LTE chain in an open-source way 4 . A very good point to srsLTE is the code maturity. However, at January 2020, there were no communication about a 5G release. OpenAirInterface (OAI) 5 6 [2] is an open-source software that proposes a complete implementation of LTE network and currently actively develop 5G Standalone (SA). Developments in the project are for a big part the contribution of the French Eurecom research laboratory, but it is also supported by important companies (e.g. Fujitsu, Qualcomm, Nokia, Meta, Orange), universities and organizations (e.g. bcom, chinese academy of sciences, university of Kent). OpenAirInterface Software Alliance (OSA) which drives the OAI project to make partnership with bigger open network project (e.g. open RAN, magma core).

De Javel et al. have presented in 2021 (after our initial evaluation

) a new opensource 5G RAN, free5GRAN [START_REF] De | Towards a new open-source 5G development framework: an introduction to free5GRAN[END_REF] which put priorities on modularity, and experiment reproduction for research and education purpose.

A common property of the presented solutions is the use of a generic hardware when the software is the major part of the work. Softwarization of networks is a long-term tendency by which network functions are moved from specific hardware and low software engineering to generic hardware and high software engineering. More generally, the layer 2 which ensures physical transmission in the Open Systems Interconnection (OSI) model is a software which runs on a generic computer. The physical layer 1 where bytes are converted into Orthogonal Frequency-Division Multiple (OFDM) symbols is rather implemented in a specific software-defined radio hardware.

The differentiation of the projects could be made on the advancement of the 5G RAT implementation and the open-sourceness of their solution as an enabler for research. The 2. https://www.radisys.com/solutions/openran 3. https://www.srslte.com/ 4. https://github.com/srsran/srsran 5. https://openairinterface.org/ 6. https://gitlab.eurecom.fr/oai/openairinterface5g

usage of an open-source project means that we could manipulate, read and add code to verify hypothesis, that is more interesting for research purposes than closed-source Application Programming Interface (API)-based project. Bringing all these arguments together, OAI corresponds the most to our use-case. In addition, the developer community around the OpenAirInterface project is very active with multiple side-projects, workshop and additional features for OAI. Finally, there is an experience at Orange of OAI as the company is an important member of the OSA and a long-time contributor to the project.

An important limitation of OAI is the late release of 5G in July 2021 and then, the 5G stack code maturity. In the following experiments and implementations, the LTE version will be used instead of 5G for these reasons. That is not a problem as we explained in paragraph 1.1.2.1, LTE and 5G have in common the same Layer-2 protocols and mechanisms. In many cases, a short discussion is sufficient to extrapolate LTE results to 5G. When a 5G feature is necessary, we could emulate it in the LTE BS's stack. We are using the OAI Evolved NodeB (eNB) and Evolved Packet Core (EPC). EPC is consisting on 3 entities, Home Subscriber Server (HSS) for SIM card registry, Mobile Management Entity (MME) for paging and Serving Gateway (SGW) to connect to IP network. We do not pursue a deep understanding of EPC since our subject of study is the RAN mechanisms. In the following of this work, the focus on engineering aspects is put on the OAI base station as the main object of the radio access technology study.

Mobile network core

We set up only one Best Effort (BE) bearer at the UE RRC connection establishment and do not consider any QoS bearers as it is a common configuration in operational networks. The EPC is made as transparent as possible, especially to perform RRC procedures.

Radio Access Network testbed

Base Station hardware

The USRP B210 is the key to bring real-time LTE communication to RAN with a capacity of 61.44 MS/s (mega symbol per second) far more than the capacity of the LTE

User Equipment

We opted for a COTS smartphone as UE instead of OAI's soft UE. The first reason of that choice is the will to focus on the base station in this work. As we already argued, we think that an important research work has to be made at the base station and thus, we consider UEs as black boxes which are 3GPP standard-compliant. Also, a smartphone is a complex system on which different modules interact in such a way that they are not described in any standards. For instance, the LTE modem is of course compliant to 3GPP standard to communicate with LTE network. However, standard does not describe in detail some functions and then, they are implemented differently by founders. Also, modem has complex interactions with an operating system (e.g. Android or iOS) and drivers. The search for realism with the RAN testbed implies to use these commercial black boxes called smartphone instead of a less representative UE i.e. a Linux OS, x86 architecture, OAI driver and USRP modem.

The 2 models of smartphone we are using are:

-LeEco Le 2 7 (2016) smartphone running rooted Lineage 16 OS (derivative from Android 9) and communicating in LTE with Qualcomm snapdragon 652 modem. -Xiaomi Mi 10T 5G 8 (2021) smartphone running rooted Android 10 and communicating in LTE and 5G with Qualcomm 865.

The smartphone embeds all the necessary to generate and receive data and produce logs. Smartphones are controlled with the Android Data Bridge (ADB) interface via USB. We have connected 2 users on the RAN to assess MAC radio resource allocation between 2 users but not more to keep the BS stable.

Air Interface

In this testbed, the Air interface is made ideal (SN R > 12 dB) and does not vary in time. Though such a channel is not present in a real-world RAN, this ensures a controlled air interface with a stable excellent channel quality rather than mobile users with uncontrolled radio interface phenomena. To do that, UEs are put in RF shield to protect them from outdoor interference and avoid multipath.

In this configuration, we cannot assess the ability of the RAN: 1) to recover air interface losses and errors; 2) to handover between cells; 3) to adapt modulation and coding scheme to air interface capacity variations. In the same time, it allows us to assess other aspects of RAN without to include in analysis unclear issues from air interface channel quality variations.

RAN configurations

We use the Band 7 as defined by Evolved Universal Terrestrial Radio Access (E-UTRA) for air interface (see Table 2.2) as the 5G frequencies are not yet stable in OAI. Band 7 uses FDD with 10 MHz bandwidth (=50 PRBs) Single-Input Single-Output (SISO) at 2.68 GHz. Downlink modulation is up to 64-QAM and uplink modulation to 16-QAM. With such configuration, theoretical achievable throughput is about 37.5 Mbps (3GPP [39]- §7.1.7) in the downlink and 17 Mbps in the uplink (3GPP [39]- §8.6.2).

In addition, Appendix B.B.1 shows the RRC configuration sent by the mobile network core to the RAN at the establishment of the default DRB for PDCP, RLC, MAC and PHY layers. One bearer is established per user. At the RLC layer, the Acknowledged Mode (AM) is used, which ensures a better reliability with acknowledgements. UE's MAC retxBSR-Timer is introduced to improve BSR robustness by avoiding deadlock situation. 320 ms after a BSR without grant, UE will send another regular BSR. The Scheduling Request periodicity is set to 10 ms according to the SR config index. Discontinuous Reception (DRX) is not configured in the experiments.

Measurements

A valuable contribution of this testbed architecture design is the capture at multiple points of the packets' timestamp (see Figure 2.4). Network captures are made at the UE, base station machine and server network interface. Network captures observe latency and throughput but also gives a more complete information of how network is transmitting data with a transmission pattern. Such data files are valuable for a deep analysis of latency-related RAN phenomena.

Traffic generation

A number of evaluation and validation scenarios are provided by International Telecommunication Union (ITU) and 3GPP. Scenarios [START_REF] Gpp | G; Study on Scenarios and Requirements for Next Generation Access Technologies[END_REF]-6 describe the environment (e.g. urban or rural), users (e.g. stationary or pedestrian), deployment (e.g. sites density) and traffic patterns. A survey on scenarios adopted is presented in [START_REF] Navarro-Ortiz | A survey on 5G usage scenarios and traffic models[END_REF], especially in section III.B, the 3GPP scenario parameters are described (i.e. what is a rural deployment or what is a FTP transfer). The testbed under this classification is an indoor hotspot-eMBB with 1 or 2 users per transmission without speed mobility. However, we do not use the traffic model proposed by the 3GPP (summarized in [START_REF] Navarro-Ortiz | A survey on 5G usage scenarios and traffic models[END_REF]-Table V). Traffic models are described with a file of size S and a packet transmission distribution D. For instance, a FTP traffic (e.g. an TCP long transfer) is described by a truncated lognormal distribution with parameters σ = 0.35 and µ = 14.45 that is an important simplification of the TCP transfer dynamic (the same arguments for web-browsing and video streaming).

We prefer to utilize linux internet stack to get representative results. Transport protocols and endpoint applications have been chosen to be representative of the common mobile networks usage by the customers. Transport protocol stacks (introduced in §1.2) and the applications are depicted in Figure 2.5. We also use a User Datagram Protocol (UDP) tool for the generation of a determined traffic profile, less for representativeness than to test the behavior of the RAN under certain conditions. This section presents the characteristics of the traffic generator used in the testbed.

Iperf Long TCP and UDP traffics are generated by iperf tool [START_REF] Tirumala | Iperf: The TCP/UDP bandwidth measurement tool[END_REF] installed on the server and on the UE. Iperf has a non-negligible advantage over an HTTP traffic generated for instance by watching a Youtube video that is the fine control on transmission parameters of the experiment and a lean application layer. Watch video on internet or load webpage with chromium is factually more realistic to what experienced real user but with a loss on control on transmission parameters (e.g. datarate) or with undefined application behaviour (e.g. excess of buffering). Iperf is used to evaluate the transmission performances of the RAN for TCP and UDP long flow traffic. Especially, iperf TCP to test mechanisms related to TCP, congestion control and latency handling and TCP adaptation to RAN capacities. TCP protocol stack reference is the one of the Linux kernel 4.15.0 9 . Iperf UDP for traffic pattern generation with different controlled datarate and packet size.

With iperf, we are able to generate an UDP data flow with a determined datarate, packet size and jitter in the packet inter-departure. Uni-directionnal UDP flows are very useful to generate determined traffic profile that should not be representative of a real application behaviour but interesting the network response to the traffic profile i.e. how it will transport it. cURL HTTP requests are performed with cURL. Compared to raw TCP downlink traffic, HTTPS requests have characteristics to be a short flow of few megabytes and for which the time to download is more important than the TCP goodput. lsquic For the OAI testbed, we also set up a Quic client/server as Quic protocol. Pushed by Google, Quic will carry the most of the data in cellular networks in the upcoming years. Many android applications are using natively Quic protocol such as Youtube with Cronet 10 . Cronet is the chromium's networking stack. It implements Google Quic protocol and supports IETF Quic standard. However, in the same idea as Iperf compared to Chromium, we rather prefer to use a toy client/server for protocol testing. The objective is that to minimize undefined interactions with upper layers and focus testing on interaction between IETF Quic protocol and cellular network. We use lsquic 11 [181] toy client and server. We do not see elsewhere the use of lsquic on Android smartphone as client and server. The compilation and installation procedure are detailed in a note of blog 12 . The method proposed is an important contribution of our testbed design since it allows testing how Quic reacts to cellular network conditions. The toy server allows us to generate Quic traffic and HTTP/3 [START_REF] Bishop | HTTP/3. RFC[END_REF] requests. lsquic implements Bottleneck Bandwidth and RTT (BBR) congestion control algorithm with a PTPC (Packet Tolerance PID Controller) method for the Acknowledgment (ACK) generation that is not representative of the majority of the quic stack.

Flent

We complete the UDP and TCP using the Flent [START_REF] Høiland-Jørgensen | Flent: The flexible network tester[END_REF] tool that is today usually used in the bufferbloat and transport protocol community to run tests on real network hardware. It is based on netperf tool (an alternative to iperf developed by HP) and provides a helpful software overlay with scenarios and visualizations to increase reliability and making easier network experiments. At the best of our knowledge, Flent was never used in an RAN testbed. The difficulty was to adapt and compile netperf server for Android and Flent.

Analysis tools and methodologies

In this section is presented the methods used to determine latency and impact of radio segment on traffic flow. It concludes the chapter concerning research method after a complete review of testbeds and traffic generation.

Active RTT measurement

A common method to get end-to-end latency experienced by a user in a network is by measuring simultaneously to a traffic flow, the RTT with an Internet Control Message Protocol (ICMP) ping traffic. A ping request is sent to a destination host and in response the host respond with an echo packet. Sent regularly, the ping samples actively the RTT of the network. ICMP ping does not inform about the composition of the latency by segment, e.g. uplink/downlink or RAN/core. Moreover, ping takes "snapshot" RTT at a specified frequency leading to a sampling issue. It does not inform about traffic transmission pattern.

In parallel of that, IETF standardized with RFC5357 [START_REF] Hedayat | A Two-Way Active Measurement Protocol (TWAMP)[END_REF] a Two-Way Active Measurement Protocol (TWAMP). TWAMP has better accuracy to measure latency, jitter and packet loss than ping, however it reveals to be more complex to be put in place in the mobile environment.

Passive measurements

The technique of passive measurements [START_REF] Morton | Active and Passive Metrics and Methods (with Hybrid Types In-Between)[END_REF] consists of taking network captures at the edge of the RAN. "Passive" because the capture simply observes packets on network without interact with them or with the network. [START_REF] Allman | Comments on bufferbloat[END_REF] employs this technique to analyse the RTT and latency variation of TCP connections. The method illustrated in Figure 2.6 uses 2 measurement points at which timestamped packets are copied in a network capture. The 2 points of measurements should be at the edges of the RAN. Practically, measurement point is a tcpdump capture at the Operating System (OS) cellular network interface. In the UE, the network capture is made at the network interface mounted by Android when the modem is enabled. Then, captures do not include cellular network protocol stack. The other capture is made at the BS's interface connected to the backhaul and the core, captures include all the cellular network stack. Thanks to the 2-point measurements, a packet is observed at the entry and the exit of the RAN. The RTT is computed from one capture by observing the departure of a packet and the response corresponding to it without adding active measurement packet in the network. The uplink delay is computed from 2 captures, as the time for a packet departing from UE to reach the output of the RAN. Conversely, the downlink delay is the time for a packet from the arrival on BS machine to reach UE network interface. Packets are uniquely identified with their IP identifier included in the IP header. The measurement of network performances is then known at a packet-level. Moreover, network captures give a very useful information on the manner data are transmitted through the RAN with the traffic profile plotted in Figure 2.7. The traffic profile is the cumulative volume of the transmitted bytes over time. The traffic profile of a continuous transmission will lookalike a line. Bursty transmissions will have a stepped traffic profile.

Traffic profile includes a lot of information that the simple measurement of the delays does not have. For instance, the increase of the delay could reveal the build up of a temporary queue due to a lack of transmission opportunity, or it could be the sign of a too important transmission datarate compared to network capacities, leading also to the build up of a queue. In the first case the traffic profile shows emphasized steps while in the second case the traffic profile shows a transmission as a line with a slope inferior to the generation datarate. That is significant in the following of this work to quantify the source of latency and jitter among application transmission pattern, network capacities, retransmissions, resource allocation scheme. We think that is an aspect too often put apart in work concerning latency in wireless networks.

Network calculus [START_REF] Van Bemten | Network calculus: A comprehensive guide[END_REF] mathematical method is largely based on the concept of the traffic profile and consider each segment of the network as a node with internal characteristics which modify a traffic profile in input to a traffic profile in output. A convenient representation of traffic pattern at 3 capture points is given in Figure 2.7. The horizontal difference gives for selected bytes the time it spends in the network. The vertical difference gives at the selected time the amount of bytes in the network. In our case, the vertical difference is limited by the buffers' length in the base station and in the UE modem. The maximal slope of the line orange is limited by the network instantaneous throughput (in the downlink or uplink).

However, the task of synchronizing the 3 captures made with 2 different clocks at subms level is challenging. It is challenging especially because of this need to have a clock synchronization below the radio slot level.

Network captures synchronization

The 2 machines on which are realized the captures are quite a bit different. We could not ensure to have a sub-ms synchronization between CPU clocks of UE and BS over a long timescale. The solution of Network Time Protocol (NTP) server is not suitable because it does not warrant such degree of synchronization. More generally, networkbased synchronization does not have a sufficient degree of precision with about 2 -3 ms. The solution found in the cellular networks to synchronize nodes, especially base station is usually to use a GPS signal. But for a sake of simplicity and for the cost, we opt to a solution with the measurement of the timestamp on both UE and BS at the beginning of the experiment and at the end. This approach is based on the assumption of that the clock drift during a seconds-experiment is contained to lower than one millisecond.

As a reminder, the UEs are connected to the BS machine through ADB interface (i.e. USB). We tried a first approach by which in parallel we get the current timestamp of the host machine and the timestamp of the UE using ADB remote shell. After evaluation, we found a standard deviation of 4.5 ms that is too important to have a good synchronization of captures. Root causes are multiple, among ADB server request, the USB interface and android shell and kernel delays, we could not evaluate which one is in cause of this variation of the measurement.

The solution we finally opt for is as follows: 1) Mount UE as a tethering server at which the host connect to; 2) Capture packets on the UE and on host tethering network interfaces; 3) Send 5 pings packet through USB interface that will be captured at the both endpoints; 4) Stop capture and unmount tethering; 5) For all pings in the 2 captures, compute the timestamp difference and uncertainty due to USB interface; 6) Return the mean timestamp difference in second between the 2 endpoints and return the uncertainty.

After evaluation, we found a standard deviation of about 0.35 ms over 5 seconds that is below 1 ms. Before an experiment, the androsync script is run and returns a timestamp offset between the 2 network capture points. We ensure that for a ten seconds transmission, the time drift between the 2 capture points is below the radio subframe. We consider this precision sufficient to pursue radio access network latency studies.

Conclusion

In our study, we are interested in the latency performance of RAN segment in controlled good radio conditions. The testbed presented in section 2.2.3 is designed to focus on radio link layer. We selected the open-source project OpenAirInterface as LTE RAN solution after a short review in Sec. 2.2.1 of the existing RAN solutions. The BS is connected to an OAI EPC for the server and an USRP SDR card for the RF transmission. 2 COTS UEs are connected to the LTE cellular network with a default BE bearer.

Testbed main limitation is related to the air interface model that is too simplified to be realistic. We can add to limitations:

-No handover procedure linked to a mobility event -The lack of Remote Radio Head (RRH) and antenna segment -A long development process to implement a feature by the complexity of the code (1.13M Source Lines of Code (SLOC)). We reviewed in section 2.3 the traffic generation method. A strong focus was put on transport layer protocol, more than on a limited number of targeted service. We justify this choice by the perspective of generalization we would like to give to this work in a sense where we think that a major part of services will be based on UDP/TCP/Quic protocols. To avoid undefined behaviours, implementations and configurations issues from higher level protocol, we place our study at the layer just above IP. The RAN should guarantee the best transmission as possible in terms of throughput, latency and jitter to end-to-end protocol. For that, TCP and Quic servers are simple as possible to avoid hypothetical higher layer issues. To evaluate radio capacity and traffic pattern transmission, we use UDP with a variable datarate and packet size.

Finally, section 2.4 presents the latency calculation method with both active and passive measurement. As we show, two network capture points are sufficient to have an information at the packet-level of the total RAN latency on the condition of having a synchronization of the clocks of the 2 captures.

The next step is to obtain such fine knowledge of the latencies inside the RAN i.e. inside the BS node. The BS is usually a black box which pulls latency metrics for instance per layer or per QoS bearer. In the following chapter we present a novel method to measure at packet-level internal latencies inside the base station.

Chapter 3

LATSEQ

Abstract : This chapter presents a major contribution of this thesis that is a novel measurement tool, called LatSeq, for Radio Access Network (RAN) internal latency analysis. After a review of motivation behind this work, we introduce the notions of internal latency, packet fingerprint and packet journey that are used for the RAN latency analysis. We particularly detail the key design of the tool to achieve low impact on the observed system and the relevant fine-grain latency statistics. The evaluation with a showcasing of the tool demonstrates these properties. Finally, we discuss the importance of the contribution for the open source community.

Motivation

Offering low latency communications is undoubtedly one of the foremost 5G promises. However, coping with latency objectives under few milliseconds creates tough challenges for the air interface design but also for QoS management and resource allocation on the radio segment. In general low latency mechanisms proposed in the literature were only evaluated in simulated environments. Several open-source platforms of end-to-end mobile networks have recently emerged: OAI and srsLTE are oriented toward research purposes. Both open-source and commercial Base Station (BS)s report a bunch of metrics to evaluate the quality of the network and possible issues on Key Performance Indicator (KPI). Usually, latency is given per layer or per QoS bearer from a sampling process. In OAI many timers are set up to measure the contribution of different network functions to the total latency. We quickly understand the limit of this approach to pursue a comprehensive evaluation of the latency in the RAN. We would like in one hand to correlate delays reported by the BS with the latency measured with the 2-point passive captures technique and in the other hand, to get composition of delays for an individual packet or at least to a packet flow.

A comprehensive tool for latency analysis added in software BSs is needed and constitutes an important contribution of this thesis. The tool should be able to measure the delays in each layer entity but also delay values related to the same packet should not be captured independently of each other: what should be built is the sequence of successive latencies across the layers. Making measurements inside a running BS is a challenge in itself as it requires measuring delays of the order of a microsecond for thousands of high throughput connections and ten of diverse users. Logging these large numbers of events necessarily incurs extra delays, while we are precisely seeking a low-impact tool to preserve the BS operation. To meet these double harsh constraints of fine granularity and performance we develop a tool called LatSeq -"Latency Sequence analysis". The principle behind LatSeq could be summarized as follows: During in-running BS, solely log packets' information needed to compute in offline packet's sequence of latencies. From this principle comes the design of the tool presented in section 3.3. Before that in the next section 3.2.1 we define the notions of internal latencies and sequence of latencies extensively used in this work.

Definitions

Internal latency

The end-end-end latency experienced by a packet is divided between heterogeneous segment by which this packet is carried (see Figure 1.8). The three fundamental sources of latency in computer networks is the propagation delay, the buffering and the processing delay. What affects the QoS is the end-to-end latency in light green in Figure 1.8. It aggregates all the types of latency accumulating on the packet's path. But latency characteristic is radically different between an international fiber and a wireless access segment. The question is, in the RAN, what is the type of latencies, their magnitude and their location. With the setup presented in section 2.2, the cellular network latency per packet is known accurately, but it tells nothing on RAN internal latencies in blue. The solution we propose complete the study of end-to-end latency in red in Figure 1.8 with a focus on base station internal latency in green. Defining the latency of a packet handled through the radio access network is in itself difficult. A packet is processed by a collection of layer entities and is prone to segmentation or reassembly, concatenation and retransmission. This induces 4 cases for internal latency as shown in Fig. 3.1. The latency can be considered either for each packet segment or for all segments related to the same packet. Moreover, depending on the point of view (from inside the node, from outside the node, from software, from hardware), delays are not of the same nature and not always measurable. We define the internal latency as the time between the moment when the packet is fully received by the node (here the base station) from the input interface and the moment when all the segments making up the packet leave the software part of this node, to be transmitted and possibly retransmitted. The node is a logical environment with a unified reference clock and delimited by an input and an output physical interface. The definition catches the latency effectively experienced by the packet due to software part of the node. The different cases we encounter in the cellular network node are illustrated in 3.1. For a packet that is buffered and processed by the node as a whole (case (a)), the internal latency represented by blue arrow is the difference of time between the arrival and the departure of the packet. The worst case corresponds to the case (d) when a multiple retransmission occurs, delaying the effective correct transmission of the packet. A lost packet is considered to have an infinite internal latency. This definition of internal latency is particularly suitable for softwarized networks where network functions are implemented as a software that is increasingly the case with the development of cloud RAN and network function virtualization. In the same time, this definition of internal latency does not include latencies due to access or physical layer.

A second challenge is to track a packet or a data segment along its journey inside the node. In the following, we call a packet or a data segment in any layer a data unit. The node consists in a succession of layers and functions that process and buffer packet. Layers entity is a coherent environment that provide a network service. For instance, the MAC layer ensures the (de-)multiplexing of packet for physical layer and uses a buffer to perform retransmissions. The MAC entity is associated to a baseband unit. Each layer manages a set of local identifiers to differentiate entities. Local identifiers are used to route packets through layers. At the bound of two layers, local identifiers link two entities. Inside the entity, a common manner to differentiate packets and track through entities is to assign them a temporary identifier. For example, each UE is identified at the MAC layer by the Radio Network Temporary Identifier (RNTI). Also, several flows can be transmitted to the same UE by use of different DRBs. At the RLC level, each data unit is referenced by a Sequence Number (SN) and multiple logical channels can be multiplexed and identified by a Logical Channel IDentifier (LCID). Then, a given packet with a SN number will be routed through MAC and RLC layer using DRB identifier, RNTI and LCID. Aggregating the identifiers from different layers makes it possible to track a packet along its path. Note that some identifiers can be redundant (e.g. the DRB number and the DRB) but this does not present a problem.

Packet fingerprint

It comes what we define the packet fingerprint at different layers. The packet fingerprint is generated whenever a measurement is made to track a packet segment during its life within the node. It includes the identification of the packet with the mentioned local identifiers but also the position of this packet in the node and a precise timestamp as illustrated in Fig. 3.2a. The syntax of the packet fingerprint is given in Figure 3.2b. The main elements are the following ones:

1. The timestamp with a precision of at least one microsecond, largely below the length of an OFDM symbol of 1/14 ms;

2. The location of the packet on the path indicates the direction, (i.e. downlink or uplink) followed by the network segment observed;

3. The packet fingerprint local identifier part is composed of a set of packet-related or layer-related identifiers with the name of the identifier followed by its value.

The packet fingerprint does not include all the local identifiers accessible at a layer and does not include packet header or packet payload at all. We remark in the case of radio access network that there are also global identifiers that are common for a packet flow. That is the case of the RNTI that is related to a unique user session and accessible from entities' data-structure almost all along the path from high PHY layer to PDCP. It is convenient to add global identifiers for a question of filtering.

In addition to latency measurement, that is the primary objective of this tool, we added another set of values that is related to packet properties not used to identify a packet. In computer networks, it is basically the size of the packets, but also information related to the radio context of the packet, for instance the buffer occupancy at the arrival of the packet in the buffer. The cost for the packet fingerprint captures is limited and provides valuable data for latency analysis.

The real-time capture of fingerprints in-running BS poses a difficult engineering challenge. The main advantage of the proposed approach is to limit the quantity of information that has to be captured of a packet to retrieve packet path and latencies. Contrary to regular passive measurements that capture by copying bytes of entire (or simply the header with optimizations) packet, we copy only values of related packet and entity data structures that are useful to identify a packet at a position. We call the packet's life across layers the packet journey as the list of successive fingerprints for a given packet. Journeys of packet are mixed together in the trace file generated during the execution of the BS. In Figure 3.3, fingerprints represented by rectangles, the packet path is IP-PDCP-RLC-MAC but due to segmentation and concatenation, a given fingerprint can be included in several journeys. Conversely, fingerprints related to different journeys are interleaved in the trace file. As a consequence, journeys are not distinguishable at first glance and should be rebuilt using appropriate processing.

Packet journey

In a way, journeys are an oriented graph. Each fingerprint can be seen as a vertex of a graph. When the destination field of a fingerprint is the same as the source of another one, there is an edge between the two vertices if both fingerprints share the same local identifiers' value. We are thus defining subgraph each one corresponding to a packet journey. These subgraphs are Directed and Acyclic (DAG) [START_REF] Thulasiraman | Graphs: theory and algorithms[END_REF]. Such graphs are wellused to represent a network of processing elements. All fingerprints generated at a given input share the same source field. For example, the fingerprints of all downlink packets arriving at the BS on the wired interface are characterized by src=ip.in. Similarly, when a data unit is transmitted to the radio interface, the destination field of all its possible leaf fingerprints are dest=phy.out. The 4 typical graphs possible in the network node are illustrated in Figure 3.3. When a data unit is dropped (or loss) by a layer, a new journey is generated replacing the previous one as shown in the case d).

From the principles of packet fingerprints and packet journeys, we build the matrix of internal latency (Fig 3 .4). The vertical axis corresponds to the timestamp, and the horizontal axis represents the packet segment path. The fingerprints (blue points) are put on the grid according to timestamp and packet segment path. Coherent blue points related to a packet journey are linked together with the blue line. The total internal latency experienced by a packet (or a packet segment) in the BS corresponds to the difference of time between the incoming and the outgoing edges of the journey graph. Different internal latencies are calculated with the timestamp value assigned to the vertices of the graph. In addition to the latency related to an individual packet, our approach is able to provide latency statistics per packet path and per layer. The third axis is used as a free parameter for filtering. For instance on the figure the third axis is used to split packet journeys per UE. All the information related to the internal latency in the BS is hold in the matrix with each point corresponding to a packet fingerprint. We have implemented our proposed internal latency measurement tool for the OAI RAN project. LatSeq is composed of two functional blocks as shown in Figure 3.5 that corresponds to the 2 challenges we have previously taken up, the real-time measurement and the packet journey rebuilding. The real-time capture of fingerprints is associated to the measurement part. The packet journey rebuilding and the computation of internal latency is associated to the analysis part. Another practical hard work comes when we implement the tool in an existing project, which is the placement of the relevant measurement points. In this section we detail the rationales, the implementation of LatSeq and the integration with OAI eNB code.

Implementation

Mechanism

Real-time measurement of packet fingerprint

The measurement part (Fig. 3.5 (A) and (B)) captures fingerprints on a running BS and records them in a trace file. Thus, we have measurement points on the packet path that necessary introduce a processing overhead to the BS. The design of the measurement part focuses on reducing the processing overhead introduced by the measurement in itself. We reduce the overhead by minimizing the number of tasks to perform on the packet path and by isolating time and IO consuming tasks (i.e. writing to the trace file) from the rest of the BS program. It exists a separation between the measurement and the conversion stage (see Appendix C.C.1 where the LatSeq main thread runs in parallel to the OAI threads where measurements are made). Such design isolates the time and IO-consuming task of fingerprint generation from the rest of the program where measurements are performed. The strategies and mechanisms utilized to achieve low-impact challenge are summarized in the Table 3.1 and developed in the following.

Measurement points

The point of the electron microscope (Fig. 3.5-(A)) is a macro-instruction called LATSEQ_P. It is inserted in the code with one LATSEQ_P corresponding to one measurement point. The format of this macro-instruction is LATSEQ_P("dir src-dest", "data_id1%d...", data_id1, ...). For instance the code in Listing 3.2 associated to the latseq measurement point at the rlc.tx.am-rlc.seg.am segment: #i f d e f LATSEQ LATSEQ_P( "D␣ r l c . tx . am--r l c . s e g . am" , " l e n%d : r n t i%d : drb%d . sdu%d . r s n%d . r s o%d " , sdu->s i z e , e n t i t y ->ue_rnti , e n t i t y ->channel_id , sdu->sdu_num , pdu ->sn , pdu->s o ) ;

#endif Listing 3.2 -measurement point code
At the compilation of the program, the macro-instruction is replaced by its inlined form into the OAI code. The inlined form avoid a costly function call operation. There are as many times the LATSEQ_P code as there are measurement points on the packet path justifying the idea to drastically reduce the extra-delays associated to the measurement process.

The arguments contain all the necessary information for the fingerprint to be formatted in the trace file. Because the macro-instruction handles a variable number of data identifiers, we use the variadic macro mechanism. Variadic functions are functions that can take a variable number of arguments. Variadic function consists of at least one fixed variable and then an ellipsis (i.e. "f(a, ...)") as last parameter. After a quick evaluation of the assembly code generated, the variadic macro-instruction reveals to be the most efficient in terms of instruction number compared to variadic functions. In fact, the variadic function needs a function call instruction (that costs instructions) and a call method to get access to arguments. Interesting properties of variadic functions are the readability of the code and the number of source line of code compared to variadic macro-instructions. For measurement points the performance has been preferred over code length and readability. Practically, when a LATSEQ_P is put with 3 arguments (the segment string, the identifier string and one data identifier), LATSEQ_P("p", "f%d", i1) is converted during the compilation time (and not at the execution as with variadic function) by LATSEQ_P3 that is the macro-instruction for the inlined function log_measure1(p, f, i1).

The log_measure(p,f,...) performs 2 tasks, takes a timestamp and copies arguments value into a data structure. The timestamp presents in the fingerprint is not given in argument but taken at the LATSEQ_P measurement. The CPU timing information in x86 architecture is obtained by using the rdtsc assembly instruction (the cost is one assembly instruction or few CPU cycles) and returns the CPU counter in CPU cycle since the boot (not a time in second). The timestamp counter (TSC) is a hardware counter in x86 CPU, therefore low-overhead and high resolution compared to commonly used gettimeofday() function in C. The rdtsc value in CPU cycle is not converted at the stage but directly used as timestamp for fingerprint. To get timestamp in second with a microsecond precision, the CPU frequency has to be known precisely such as:

t i = t 0 + (rdtsc i -rdtsc 0 )/F (3.1)
where rdtsc 0 is the CPU cycle at the arbitrary start of LatSeq with corresponding UNIX time t 0 and F is the CPU frequency in cycle per second (of order of GHz). The CPU frequency is not a priori known with precision and subject to variation over time e.g. quartz drift or internal power mode management. Also, the flag tsc_constant should be set on multicore CPU to ensure a cycle counter synchronization between cores on which are executing different threads of the program. To keep the precision of rdtsc at the conversion stage to UNIX timestamp, we log regularly (i.e. every second) a synchronization fingerprint with the format: Concerning the arguments, the macro-instruction code puts the strings and arguments in a latseq_element data-structure presented in Figure 3.6. The LatSeq structure contains the timestamp, the string constant (relative to a measurement point) of packet path segment, the string constant for the identifier format (relative to a measurement point) and a fixed-size array of arguments (to avoid time-consuming allocation and de-allocation of dynamic array). The arguments are simply copies in the LatSeq element with a low instruction cost. As soon as the latseq_element is filled, the OAI program is resumed.

r d t s c _ v
In total, the decompilation of the log_measure function with 2 data identifiers gives 37 x86 CPU instructions to perform a fingerprint capture (See Appendix C.C.2). Moreover, the instructions to get the LatSeq element could be optimized at the run time with a pre-fetch. Each data identifier added costs 3 instructions: mov -0x8(%rbp ) , %rax mov -0x24(%rbp ) , %edx mov %edx , 0 x1c(%rax ) Listing 3.4 -Assembly code to copy a data identifier On a GHz CPU, each measurement point costs about twenty nanoseconds. For 10 measurement points, we delay a packet far less than a microsecond for measurements.

Fingerprints collector and conversion

The conversion stage of latseq_element into a character string fingerprint is centralized in a dedicated thread. latseq_element data-structure are stored locally before to be collected and converted by the dedicated thread. As illustrated with the LatSeq datastructure Figure 3.6, the latseq_element is temporarily stored in a lockless ring buffer. The principle is as follows : As the packets are observed, fingerprints are put in the ring buffer at the writer head place. In the same time, the reader converts fingerprints that are behind the writer head. No fingerprints are lost if the reader is faster than the writer in average. The buffer size is here to handle an instantaneous burst of fingerprints production or a pause in the conversion. The buffer is said lockless because the writer and the reader can interact with the buffer without the usage of a mutex. A mutex is usually used to synchronize reading/writing of a variable by different threads. The ring buffer and the two heads, one to read and the other to write avoid the need of a mutex, mutex that could lead to unnecessary lock of the program. This buffer uses static memory that is local to the instrumented thread. It uses the Thread-Local Storage is a local memory to a thread instantiated at the initialization of the thread. In other words, there are as many ring buffers as the number of instrumented threads (not the number of latseq point as instrumented thread could have many latseq points).

Another solution is to write fingerprints in a non-thread local buffer. Concurrent thread access to such a buffer require an inter-thread synchronization. Thus, this would be inefficient.

The data collector is the low-priority LatSeq-specific thread. Assigning low-priority to this thread is conservative regarding CPU utilization and to minimize its impact on the BS. The data collector loop like measurement point, is optimized in terms of CPU instructions to meet high performance / low-impact requirements (See Appendix C.C.3). As illustrated in Figures 3.5 (also in Appendix C.C.1), this thread visits each ring buffer in a round-robin fashion. It gets one latseq element in a ring buffer and formats it to a string (one line) and goes to the next ring buffer. If the visited ring buffer is empty, then the collector goes directly to the next buffer. The data collector is the only thread that empties the ring buffers, and it runs asynchronously. The size of ring buffer is determined according to the thread that generates the most fingerprints. It is known that IO operations costs a lot for a program, in particular writing into a file.

The outcomes of the LatSeq measurement module is a flow of packet fingerprints written either on the standard output or a temporary file in a shared memory for further processing, illustrated with the screen capture in Figure 3.7. For future work we also propose to write the fingerprint flow to a network socket for a distant machine to process real-time the fingerprint flow.

Measurement points in the LTE stack

The implementation of LatSeq is done by adding measurement points LATSEQ_P in the OAI BS source code for the LTE stack. We instrumented the OAI code with 34 carefully chosen measurement points. Indeed, too many measurement points would be superfluous with not significant latency, as too few would make it impossible to rebuild journeys. Measurement points complete the latency measurement testbed presented in Fig. 2.4 with Fig. 3.8.

Packet path in the LTE stack

LatSeq has been implemented in the LTE stack. There are 2 main packet paths, one from PDN to UE (i.e. in the downlink) and one from UE to PDN (i.e. in the uplink). In the downlink the input measurement point is set in the PDCP layer and the output in the PHY layer. In the uplink, the input is set in PHY and the output in the PDCP interface with GPRS Tunneling Protocol (GTP). The others in-stack LatSeq points are represented in the Appendix C.C.4. The number of measurement points depends at least on the number of layers because of local identifiers needed to link the input to the output.

The method we suggest putting useful measurement point for being able to rebuild the packet path is to define one entry function with the incoming data structure (for instance, the received bytes array from the antenna in the uplink) and one output function (for instance, the sendmsg function at the network interface). Following them, find functions that manipulate data structures and associated data identifiers. The necessary measurement points are those that link 2 different data structures containing user data or 2 different functions that manipulate a different set of data identifiers. When the minimal set of measurement points between entry and output is defined, more measurement points could be added to surround a specific network function, we want to know more about its specific latency.

Let's take an example in the uplink: With the following local data identifiers, we could not link directly to the same packet the first fingerprint with the eighth. But we could link the fingerprint 1. and 2. thanks to the common point phy.in.proc of the packet segment and thanks to local identifiers fm (frame number) and subfm (subframe number) which match between the two fingerprints. Sequentially, lcid (LCID), rsn (RLC sequence number), rso (RLC sequence offset), psn (PDCP sequence number) and drb (DRB id) are local identifiers to track packets between segments. Solely the fingerprint 7 is not necessary to link segments (as psn used to link RLC to IP layer through PDCP is already known at the frontier of RLC and PDCP layer with the segment rlc.reas.am-pdcp.rx) but all other points are necessary to track packet. The interest of pdcp.rx-gtp.out measurement point is to take a precise measurement of delay introduced by the pdcp.rx point. Note that at the output segment, we could have just indicated the psn value to link the last segment with the second-tolast segment. However, we recall that latseq is integrated on global package of end-to-end latency measurement where latency is measured with passive network captures. GTP and IP protocols are used to carry packet respectively in mobile core network and in packet data network. They define a tuple of values to identify packets and flows. In particular, the IP protocol defines the IP id (identification) field that is used to distinguish IP packets. The ipid flag is accessible before the GTP encapsulation (conversely after the GTP decapsulation) at the measurement point since it has a fixed position in the IP header with the code: Listing 3.6 -IP ID catching at the GTP layer Afterwards, we correlate the packet IP id of the packet fingerprint with the IP id from network captures, it provides an end-to-end continuity for packet tracking.

We distinguish 3 types of segments in our implementation : Processing-related segments; Buffer-related segments; and Retransmission segments.

It is convenient to differentiate the type of delays. A latency due to a processing function could be reduced by optimizing the function or simplifying its task when a latency of a buffer is due to a service access (e.g. RLC buffer because of MAC scheduling to radio access). For that, we box the buffer with measurement points to isolate latency experienced by packets in buffers.

Contextual measurement points

In an extension of LatSeq we added a new type of measurement points not related to packets but contextual, the type "information" I. Indeed, we found that radio context is really meaningful to analyse latency along with the internal latencies from fingerprints. For instance, the RLC buffer latency is related to the mac scheduling decisions and the retransmission latency is related to channel quality. The fingerprints generated by the implemented I measurement points are listed below: As for packet-related fingerprints, the "I" fingerprints is composed of a timestamp, a location in the stack and data properties. The information fingerprints had revealed very useful to enlighten the latency last segment not covered by LatSeq i.e. the UE air interface modem.

The work to implement measurement points should not be played down. It is a work of an expert of the radio stack that knows packet path, data structures, functions and standards. But the implementation of itself is easy, with only a macro instruction LATSEQ_P with the suitable arguments to put inside the source code.

Internal latency analysis

The output of the LatSeq measurement is the packet fingerprint file, including a flow of fingerprints related to different UEs, layers, packets, etc. The rebuilding phase (Fig. 3.5-(C)) where fingerprints are put together to rebuild packet journey is necessary to compute internal latency. Latency data are then filtered and plotted to provide a comprehensive analysis of the internal latency.

Packet journey rebuilding algorithm

The main task of the analysis module is to rebuild the journeys of the different packets, associating each journey with a graph of causally linked fingerprints. The configuration of the rebuilder defines the input points (e.g. ip.in for the downlink, phy.ant.in for the uplink) and the output points (e.g. phy.out.ant for the downlink and ip.out for the uplink). The rebuilding algorithm tries to find a path between the input and the output points constituted of fingerprints (edge in the graph vocabulary) that match with data identifiers and time causality.

The algorithm is as presented in Algorithm 11 : When a fingerprint that corresponds to an entry segment is found, then a new journey is created (recursive_build). For a first segment as A-B, the next segment should be B-X. The list of fingerprints is unfolded when fingerprint with a segment match with an unfinished journey, the local identifier are compared. The global identifier quickly filter fingerprints that not belong to journey. All the local identifiers known for the journey should match with the local identifiers present in the fingerprints in a logical AND style. Any local identifier mismatch means the current fingerprints does not belong to the journey (match_m). A journey is considered completed as soon as the output point is reached.

1 { 2
" u i d " : " 30 " , 3 " d i r " : 0 , 4 " f i l e " : " /tmp/ main_ocp . 0 1 0 1 2 0 2 2 . l s e q " , 5

" t s _ i n " : 1 6 0 4 0 8 6 5 2 5 . 3 3 2 0 5 3 , 6

" t s _ o u t " : 1 6 0 4 0 8 6 5 2 5 . 3 3 2 7 9 1 , 7

" path " : 0 , 8 " c o m p l e t e d " : t r u e , 9

" p r o p e r t i e s " : { " l e n " : " 62 " } , 10 " g l o b " : { " r n t i " : " 5 3 4 2 1 " } , 11

" s e t _ i d s " : { " u i d " : " 30 " , " t e i d " : " 4 " , " g s n " : " 253 " , " drb " : " 1 " , " psn " : " 88 " , " sdu " : " 88 " , " r s n " : " 72 " , " r s o " : " 0 " , " l c i d " : " 3 " , " r e q f m " : " 686 " , " t x r e q " : " 0 " , " h a r q " : " 3 " , " s f n " : " 1 1 7 0 1 " , " fm " : " 731 " , " subfm " : " 5 " } , 12

" s e t " : [ [ 3 3 , 1 6 0 4 0 8 6 5 2 5 . 3 3 2 0 5 3 , " i p . i n --g t p . i n " ] , [ 3 4 , 1 6 0 4 0 8 6 5 2 5 . 3 3 2 0 5 4 , " g t p . i n --pdcp . i n . g t p " ] , [ The outcome is a line per journey in a JavaScript Object Notation (JSON) format as presented in Listing 3.8 that contains among other things the related local identifiers and timestamp associated to segments. For this downlink journey, the internal latency is 0.738 ms and the 2 segments that contribute the most to the latency is the RLC transmission buffer and the PHY encoding process.

The algorithm 1 complexity is O(n), where n is the number of fingerprints when fingerprints are ordered and all packet journeys belong to the "linear" case. The complexity of the algorithm grows with the number of segmentations and concatenations (Fig. 3.3) that are present in the cellular networks. Indeed, contrary to the linear case where we can definitively complete a journey when the output point is reached, segmentation, concatenation or retransmissions generates recursive journey rebuilding. The number of journeys to consider for a new fingerprint grows exponentially with the size of the fingerprint file. The classification of a fingerprint to a journey is quick at the start of the fingerprint flow but very slow at the end when we have to loop over thousands of journeys to consider for concatenation, segmentation and retransmission. We solve the problem with configurations for the algorithm that reduce the research space. Three configuration values limit the search space in time, respectively, the concatenation (100 ms), the segmentation (100 ms) and the retransmission (32 ms). The values are chosen to limit the error of rebuilding while limiting as low as possible the search space. For a new fingerprint, we update, with the timestamp the set of journeys that are open for a new segmentation, a concatenation or a retransmission. Thus, the number of journeys to look at for a given journey is contained during the processing and does not explode. Note that the algorithm allows to rebuild "on-the-flow" by feeding the rebuilding algorithm with the measurement part output.

Filtering results

The LatSeq measurement part generates fingerprints for all data units forwarded by the BS. Should we want to restrain observation to a subset, for example, to a given family of journeys (e.g. a flow), then a filtering module is required. Implementing this filter on the measurement side is not possible, as the journey to which each fingerprint belongs is calculated later, in the analysis part. The filter module should necessarily be implemented after the rebuilder. With adequate filters in the analysis module, it is possible to investigate the behaviour of a protocol layer, a specific UE or a mechanism (e.g. segmentation). A wide range of filters can be defined. We give a few examples below:

-By selecting journeys with the same RNTI, the behavior of a specific UE can be studied. -By additionally filtering on a local identifier (e.g. an IP packet number), the packet's entire life can be explained. -The internal latency for a given bearer can be measured with a filter on a DRB value. -The behaviour of any layer can also be analyzed with a filter on a src or dest value.

Visualization and statistics generation

We developed a set of scripts to compute statistics and tools for visualizations. In particular, a tool to fill the matrix with internal latencies in Fig. 3.4. The output is a CSV file containing internal latency per layer, per journey, per UE. Standard visualization tools such as gnuplot2 could then be used to plot internal latency for instance in Figures 3.9 An important contribution of our approach is the waterfall plot as a powerful representation (see Figures 3.10). This type of plot enlightens latency causality in a packet journey in a time-sequence plot. A small part of uplink transmissions is isolated showing the packet journey of 2 IP packets but 3 radio transmissions. The first IP packet has been transmitted over 2 radio frame and reassemble at the RLC layer. For the two IP packets, the decoding delay at phy.in.proc segment constitutes a significant part of the total uplink latency. This visualisation is a valuable tool not to study the latency in the all but for a deep understanding of radio transmission phenomena on a small set of selected journeys, for instance, to debug high tail latency.

In this work, we focus on latency but LatSeq opens the door of a wide variety of analyses. For example, thanks to the len field, the instantaneous throughput can be computed, possibly at each layer or for each user. Extra information can be added in the properties field of the fingerprint to perform analyses based on other criteria. Also, in an extension of the tool, we added the "I" fingerprints which gather some information of the radio context. Especially, radio latency is correlated to buffer occupancy, scheduling decision, and retransmissions. We can put side by side the waterfall of the packet transmission with the downlink and uplink information (see Appendix C.C.5).

In the downlink, we plot the RLC transmission buffer occupancy, the reported channel quality, the MCS corresponding to scheduling decision and HARQ retransmissions.

In the uplink, we plot also the RLC reception buffer occupancy, the computed channel quality (Channel Quality Indication (CQI)), the BSR and SR that are related to uplink scheduling decisions and HARQ retransmissions.

These representations are not new and are somewhat equivalent to some debugging tool of Amarisoft but interesting when putting in regard with latency waterfall.

Evaluation

A major point for the relevancy the proposed method of measurement is its low impact on the BS performances even while recording a huge amount of packet fingerprints. The throughput could attain Gbps or more than 100.000 data packets per second in a 5G BS 100.000 packets per second with 10 measurement points on the path means it could be generated 1.000.000 fingerprints per second.

The rebuilding phase is not time-constrainted contrary to the measurement in the BS. We evaluate the delay added by the measurement points on the packet path and the cost on BS performances of the fingerprint generation.

Performance results are summarized in Table 3.2. The fingerprint generation experiment (subsection 3.4.1) evaluates the measurement part (Fig. 3.5-(A)), with the time to generate a fingerprint (in the table with one data identifier i.e. variadic macro-instruction argument). The data collector part (Fig. 3.5-(B)) is evaluated both in terms of memory usage (subsection 3.4.2) and CPU usage (subsection 3.4.3). An analysis of the assembly code generated shows we add 37 machine code instructions per LatSeq measurement points (Appendix C.C.2). On a unit test setup, we measured the execution time of the LatSeq macro on 1 million runs from 1 to 10 arguments (e.g. data identifiers). The execution time according to the number of arguments is plotted in Figure 3.11. The average time goes from 10.98 ns for 1 argument to 18.56 ns for 10. On a CPU with a frequency of 3.0 GHz, 14 ns (5 arguments) is equal to 42 CPU cycles3 that is not so far from the 37 found in Appendix C.C.2. The increase of the execution time is linear to the number of arguments. In fact, the number of assembly instructions associated to the copy of one identifier to ring buffer is fixed, and add a fixed amount of execution time (Listing 3.4). In conclusion, for a given packet measured at 10 different points along the path, measurement points incurs of 200 ns (for 10 data identifiers), 5 times lower than our objective of a microsecond resolution.

Unitary measurement point generation time

Memory usage

Concerning the memory, it is allocated a ring buffer of typically 256 latseq measurement elements per instrumented thread. An element is a data-structure of 180 bytes, thus the ring buffer size in memory is of 46 kB. The number of instrumented threads depends on OAI source code structure, but it is linked to the number of MAC/PHY encoding/decoding process and the number of data radio bearers. OAI BS instantiates 8 threads for MAC/PHY (linked to the number of HARQ processes), and one thread per data radio bearer. The number of ring buffers is then equal to 8 + the number of DRB. By default, one default bearer is instantiated at the connection establishment of a UE. Then the size in memory of LatSeq measurement is directly linked to the number of connected UE. For one UE, the size in memory of LatSeq is then about 450 kB which is a limited impact on memory requirements.

In-running global performance

To evaluate LatSeq contribution to the global performance, we measure its CPU utilization (with perf kernel tool) when fingerprints are generated at full speed (i.e. max disk writing speed of 264 MBps in our setup). The screen capture in Appendix C.C.6 shows a part of 0.22% of LatSeq on the total cpu usage by the BS program. Thus, the collector thread is not CPU-bound (usage < 100%). Moreover, the BS's linux load average (that comprises the CPU and IO operation) is not so impacted by the presence of LatSeq in both baseline and loaded condition, respectively +0.02 (with 0.03 of incertainty) and +0.07 (with 0.075 incertainty). LatSeq has then a visible impact on machine performances but limited and principally due to the data collector's capacity bottleneck that is on the writing-to-file side, not on the fingerprint-formatting one. We can then roughly estimate LatSeq's capacity as the maximum writing speed in ring buffers, that is about the full disk writing speed. With disk writing speed of 264 MBps, for a fingerprint size of 100 bytes, it corresponds to about 2.64 M fingerprints per second.

Over the assumption that each packet forwarded by the BS generates 10 fingerprints, then the maximum throughput we can measure accurately is 64 × 2.64.10 6 /10 = 16.9 MBps (in both side) in the (worst) case of 64-bytes packets. Moreover, as a running BS has no need for disk access, LatSeq does not threaten the system's performance even at full disk throughput. A manner to accelerate LatSeq is to write fingerprint on the high-speed RAM with shared memory. The fingerprints are then processed and filtered (e.g. per UE) from the shared memory by rebuilding and filtering scripts and the results is finally written on the disk. We also propose as further work to write fingerprints on a network interface dedicating to BS monitoring with a server which receives fingerprints.

Related work

The idea of measuring latency inside the RAN partly was inspired by [START_REF] Li | Mobileinsight: Extracting and analyzing cellular network information on smartphones[END_REF] where they measure in-device latency. The measurement method and technical solutions was inspired by several works in [START_REF] Ibm | Logging in multi-threaded applications efficiently with ring buffer[END_REF][190][START_REF] Simpson | [END_REF][START_REF] Afanasyev | Optimize multi-threaded logging using lock-free queue and separate thread[END_REF]. These solutions described mainly high performances logging systems not specific for cellular networks.

A number of tools are provided with the OpenAirInterface project. Two of them are to be cited for the packet measurement and network monitoring, T_tracer 4 and Wireshark captures. T_tracer is a framework to debug and monitor softmodem. With a set of measurement points inside the BS, T_Tracer pushes data to monitoring tools. The monitoring tool called enb5 provided traces of data unit at PHY, MAC, RLC and PDCP layers. We did not use the measurement part of T_tracer because it has been designed for debugging and monitoring and not for the performances.

Another module of T_Tracer is the possibility to put passive capture point for wireshark 6 directly in the LTE stack. The measurement point copies the content of a packet to a local port where wireshark, a graphical network capture software could listen and capture LTE packets. In our setup it would have more passive measurement points but inside the RAN segment that is not possible otherwise. A clear limitation is that to have the same complete view of internal like with LatSeq we should multiply the number of passive network capture points, but these captures have a high cost. When a capture is performed, even if we do not count the measurement point cost in itself, the whole packet is captured with the payload. Even with a smart truncation of the packet to keep only the header associated to the layer, the cost in terms of CPU instruction is always higher than what we propose with LatSeq fingerprints capture. In addition to packet header as local identifier, LatSeq indicates the packet segment of the fingerprints. The rebuilding could be made without any knowledge of the underlying packet path architecture when with network capture, the packet path should be indicating as a configuration. The T tracer solution is to be quoted because it is a powerful tool for BS debugging but does not meet the logging performances required for realistic experimentation not in a debugging mode.

Showcasing

burst of transmission to debug is encircled

In this section, we showcase the capabilities of LatSeq to effectively exhibit internal latencies inside BS in coordination with the passive network capture. We observed in deployed cellular networks an effect of bursty transmissions of packets, particularly marked in the uplink. We believe it is relevant to explore these phenomena on our testbed (presented in Chapter 2) with LatSeq.

On the presented testbed, four passive capture points are put on the packet path as illustrated in Fig. 2.4. A downlink TCP traffic is generated from server to terminal which generates in the uplink a flow of acknowledgments. Each acknowledgment is typically a 100-bytes IP packet. The TCP time-sequence from network captures is plotted in Fig. 3.12.

The data packets and their corresponding acknowledgments are represented with the cross. Three problems had to be highlighted here:

1. The high latency in the uplink compared to the downlink 2. The uplink delay jitter from one packet to another 3. The bursty transmission of packets in the uplink when in the downlink the packet are regularly transmitted

The answer to this question cannot be resolved nor with the network captures only neither with latency statistics from the BS. With LatSeq measurements the situation become clearer compared to the black box represented by the large white empty space on the time-sequence plot.

When we zoom on the waterfall of the transmission burst, we found that all acknowledgment packets are transmitted in the same transport block (in yellow on Fig. 3.13). Then all packet issues to the same uplink grant. The reassembling of the IP packets are made at the RLC layer, confirming that all packets are packed on the UE side in the transport block. The packet journeys do not show HARQ or RLC ARQ retransmissions. We also eliminate the possibility of a RLC in-sequence delivery because there are no concatenation with an earlier transport block.

The radio context confirms that there are no disturbances in the air interface that could incur decoding error and retransmission. However, the explicit scheduling request (SR/BSR) sent by the UE are correlated with the transmission by the BS of an Uplink Grant or grant of uplink capacity (UG). The size of the grants transmitted after a BSR reception allow the transmission of several packets especially since the acknowledgment packets are small (around 100 bytes). In this example, the uplink latency should be correlated with the radio resource allocation with the transmission of an UG.

As partial conclusion, LatSeq demonstrates on this short example how it could help the analysis of latency in the RAN. In addition to radio context information, it is a very powerful tool for debugging but also to get relevant latency statistics per layer.

Contribution to open source community

This work would not have been possible without an open source RAN or an open source OS or even without the open source community around OpenAirInterface. As a consequence, and in the pure philosophy of open source, LatSeq is made available to everyone with a permissive license (BSD 3-clause) for any other project. The implementation is made as independent as possible from the OAI project and the sources and examples are available on Orange open-source github group 7 . We had already some contacts with external people about the tool and how to use it with great interest. The wide range applicability of the solution thanks to its general concept and the different communications made in intern at Orange and in external at conferences should contribute to the distribution of the tool.

In a second time, we pushed to the official OAI repo the LatSeq branch 8 of the implementation of measurement points to be ease the usage of the tool by the OAI community. We started the process after a presentation at the OAI summer workshop that showed great interest from the OAI community, especially for 5G Ultra-Reliable Low-Latency Communications (URLLC) developments. We collaborate with Eurecom to implement LatSeq to the OAI 5G stack 9 since we implement it for LTE only in a first time. The hope to have the tool to survive after this thesis is a main contribution of this work for the Software-Defined RAN (SD-RAN) open source community.

7. https://github.com/Orange-OpenSource/LatSeq 8. https://gitlab.eurecom.fr/oai/openairinterface5g/-/merge_requests/1432/ 9. https://gitlab.eurecom.fr/oai/openairinterface5g/-/tree/latseq_5GSA

Conclusion

In this chapter we present a novel approach to measure latencies inside a software BS. Based on definitions of the packet fingerprint as the measurement input, the packet journey for the path of a packet and the matrix of internal latency we implemented tool called LatSeq.

This tool is composed of a low-impact measurement extension for the OpenAirInterface BS and a module to process and visualize the journeys of packets and various statistics. This paves the way to a very wide range of fine-grained and comprehensive internal latency analyses as illustrated with the matrix of delay (Fig. 3.4). LatSeq provides for an understanding of interactions between the different layers in terms of delays especially with the usage of the waterfall representation of delays (Fig. 3.10a). We showcase this ability on a real-world experiment. This makes LatSeq a valuable help to optimise LTE and 5G radio interfaces.

It should be noted that LatSeq is sufficiently independent of the OAI platform's code and structure to be generalized to a wide variety of open-source software, e.g. srsLTE.

The code of the tool and its implementation on OAI has been released on open-source. This is an important aspect for the dissemination of this work and a contribution to the OAI community.

-

RAN Latency Characterization in the Lab Testbed

It could be found in the literature many work about latency measurements in realworld mobile networks [16-18, 20, 22, 193-196][93]-3.1. Also, the latency is frequently evaluated as a performance indicator in simulation studies when it is not the center of the focus. Our lab testbed allows going deeper in the analysis of per-packet latency on a selected set of internet traffic in a Best-Effort configuration. This work is a prior to the thesis direction followed thereafter. This section is a baseline to understand latency related to OAI RAN segment. Table 4.1 summarizes the experimental trials that was performed. Firstly, an evaluation of the RAN RTT with ICMP pings was performed for a latency baseline. Internet traffic is in majority composed of HyperText Transfer Protocol (HTTP)/TCP/IP flows. Experiments with short and longer HTTP transfer are used to evaluate the transfer pattern and duration of such internet flows. Finally, UDP long transmission and Flent experiments exhibit the well-known bufferbloat issue we encounter in the RAN.

Baseline RAN RTT with Ping

A very popular tool among network testers is the ping command1 This command uses ICMP protocol to send a ping request to a target, the target respond with a ping reply that gives the network RTT. The particularity of ping is that the request is directly handled by the OS kernel, which avoids as much as possible application delays (see Fig. 1.8). The results of this section are used as baseline latency of the RAN (i.e. RTT, downlink/uplink OWD) for this thesis according to the RAN configuration presented in Table 2.2.

Different parameters are affecting packet latency :

-The positioning of the user in the cell modifies the received signal strength. A low or varying signal strength increases the probability of radio loss (with HARQ error recovery latency) and diminishes the MCS and the achievable throughput and thus, increases the packet transmission time. -The content positioning with respect to mobile user affects the packet path length (i.e. propagation delay). -The scheduling strategy may decide to prioritize latency over throughput or prioritize inter-user fairness over latency with a big impact on the resulting latency. -The packet size is also determinant in the latency because a larger packet will take several TTIs to be delivered when a small packets could be successfully delivered with few Physical Resource Blocks (PRBs). -The packet datarate is the source of queueing latency when it is higher than the radio channel capacity. Especially, queueing latency shows a peak when a burst of packets arrives with a high instantaneous datarate.

Step -The Network load causes the reduction of radio resources allocated per user and inter-cell interference this may cause a reduced Signal-to-Noise Ratio (SNR) and hence a lower MCS. Network load impacts the achievable throughput.

Baseline RTT In Figure 4.1 the ECDF of ping RTT for different packet sizes (7.000 points) is plotted for the testbed without any other load. A packet of 64 bytes corresponds to the default ping packet size. Packets are sent with an inter-departure time of 500 ms, resetting the network between each packet sending. The RTT ping median is 30 ms, the average is 32 ms, the variance is 5 ms, the minimum RTT is of 20 ms and 90% of packets experience a RTT of less than 40 ms.

The minimum value of 20 ms can be confirmed by an analysis of the different operations in the downlink and the uplink. LTE user-plane OWDs in the downlink and the uplink are presented in Tables 4.2, where t FA is the time for the frame alignment before transmission, t SR is the time before the transmission of a scheduling request, t HARQ the time to recover from a radio loss with HARQ (e.g. 8 ms in LTE) and P BLER the probability of radio loss. The minimum OWD is 3.5 ms and 10 ms for downlink and uplink, respectively. This gives a minimum RTT of 13.5 ms. The remaining delay is split between 0.5 ms for the backhaul/core/server latency and 6 ms inside the UE modem. HARQ process adds at most 8 × 3 = 24 ms before failing to recover radio loss. P BLER target for Link Adaptation (LA) is usually 10%. Thus, the expected value of OWD due to HARQ is 0.888 ms (<< 24 ms). HARQ introduces a jitter that was well studied in [START_REF] Sahu | End-to-end uplink delay jitter in LTE systems[END_REF]. The remaining loss (< 1% after HARQ process) are recovered by the RLC AM entity at the t-reordering timer expiration.

RTT according to the packet size In [START_REF] Muhammad | Iperf, Ping, and slicing in RAN testing[END_REF], Rekoputra evaluates the latency in an OAI RAN for different configurations. He varied the packet sizes and the ping packet inter-departure in a low-loaded RAN. He founds a high jitter and a difference of delay for different packet sizes. Furthermore, he compared the "baseline" ping with a Cloud-RAN (C-RAN) version of OAI and concluded that C-RAN leads to a better latency performances with lower latency. We do not have the C-RAN version he ran. ECDFs of RAN RTT presented in Figure 4.1 according to the packet size (we vary the packet size in uplink from 64 to 8192 bytes) shows different results concerning the increase of the RTT in respect with the increase of the packet size.

From 64 to 8192 Bytes, minimum, median and 90 th percentile RAN RTT increases of about 5 ms.

For the 99 th percentile packet RTT (higher 10 packets latency per size) the latency is not an increasing function of the size. Like every rare event, these high values are difficult to analyze, but, thanks to network captures and latseq traces, we know that is a bad scheduling in UE kernel or modem, decoding errors or the need of a connection re-establishment (with a random access procedure). Also, for a packet size below the Maximum Transmission Unit (MTU) size of 1.472 bytes2 where the RTT experienced does not seem to be impacted by the size of the packet. For packet sizes over the MTU, the uplink IP packet is segmented. Bigger than 1500-bytes packets are segmented at the UE Network Interface Card (NIC) into multiple IP packets. Packet bigger than 1472 bytes at the mobile networks ingress interface are segmented because of the GTP encapsulation into multiple GTP/IP packets. The maximum uplink Transport Block Size (TBS) for a MCS = 20 is 2384 bytes. Thus, a packet with a size inferior to 2384 bytes should be transmitted in one TB. For bigger packets that could not fit in a TBS, packets are segmented into several SDUs carried by several TBs at the UE RLC layer. Successive transport blocks are transmitted with an interval of 1 ms corresponding to the TTI duration.

We illustrate the operation of segmentation/transmission/reassembling with the example of a 4500-byte IP packet on the uplink. The packet is segmented in 3 MTU size IP packets by the UE NIC and transmitted within at least 2 TB through the air interface.

The packet journey on the uplink inside the base station is plotted in Figure 4.2. We observe that 2 successive radio subframes (i.e. 1 ms interval) carry 2 transport blocks of size 2384 and 2249 bytes. The four parallel lines at the physical layer corresponds to the decoding of codeblock group of the same transport block. The first transport block includes the first IP packet entirely transmitted and forwarded to the PDN and a part of the second one. The third transport block includes the other part of the second packet and the third packet entirely. The second packet is reassembled at the RLC layer with the 2 TBs, that is made visible in Figure 4.2 with the vertical line at the rlc.reas.am segment. The delay introduced by the transmission of the 3 packets over 2 successive subframes is then 1 ms and depends on the achievable transport block size i.e. the achievable uplink throughput. A packet of size 8 kB is in this configuration divided into 6 IP packets, 99 RTT in a loaded cell In the BE paradigm, it is a very common situation where two different applications, with different traffic patterns share the same DRB i.e. the same network pipe. In this situation, the 2 flows are concurrent to access to the same shared resources which are PDCP entity ciphering functions, radio resources, etc. The worst situation for a mouse flow (i.e. low datarate flow) could encounter is the sharing of the same DRB with a long large flow (elephant-type flow). To emulate this situation, we continue to use ICMP ping at 5 packets per second for short time sensitive traffic, and we use shaped UDP traffic with a throughput equal to 85% of the cell capacity in the downlink (30 Mbps). Shaped traffic means that the maximum sending rate is controlled and packet are transmitted with a smoothed inter-departure time without burst. For UDP packet of size 1472 bytes, it means 2.5 packets arrives every subframe of 1 ms. Iperf tool is used to perform this experiment.

The downlink transmission profile shows a constant transmission of packets every radio subframe (i.e. every millisecond), as there are no other user in competition to radio resources access. As we do not reach the radio capacity, we only had 7 packets dropped (over 61665 UDP packets) following 2 HARQ retransmissions failed.

We plot in Figure 4.3 baseline latency and latency when the DRB is loaded. Ping RTT in function of packet size for an empty cell, corresponding to the previous situation with median RTT around 32 ms. When the concurrent flow is present, the median RTT has raised up to 200 ms, that is equivalent to a propagation delay of 40.000 km in an optical fiber.

Another remark when we observe the ECDF, points are less concentrated around the median, but extreme values (max and min) are less spread, 30 ms (for a median at 200 ms) instead of 50 ms (for a median at 32 ms). The bandwidth is regularly given to the UE, the access time in the uplink is reduced. In the same time, the value are spread around the median because the small ping packet are in competition with larger UDP packets in the downlink, the difference of delay jitter introduced by the uplink is intensified by the downlink queue bottleneck.

In summary, this experiment illustrates the consequence on the RAN RTT of a persistent queue in the BS for a BE DRB. The problem could be tackled by avoiding intra-bearer resource sharing. Since by radio frame there are 29.8 packets of long flow and 0.05 ping packet, it is more interesting to organise a round-robin sharing between them with 2 DRBs (with an adapted QoS Class Identifier (QCI) for time sensitive flow). The scheduler multiplexes DRBs, not the UE or the flows. Slicing is a promising solution to differentiate flow types with reserved radio resources (see latency mitigation section 1.3.3).

Connection establishment, RTT and HTTP/S transfer

We evaluate in this section a real-world latency case with the HTTP downlink transmissions.

Short HTTP transfer

The first experiment is a HTTP transfer to get small file of 10 kB. It shows the impact of the attachment delay and baseline RTT on the transfer duration since an important part of the transfer consists in awaiting responses during the connection establishment, RRC connection, TCP handshake and HTTP response (see Fig. D.A.4).

Radio connection establishment

The attachment latency, consisting in RACH procedure, contention resolution and RRC procedure [START_REF] Gpp | Evolved Universal Terrestrial Radio Access (E-UTRA) and Evolved Universal Terrestrial Radio Access Network (E-UTRAN)[END_REF] (See Appendix A.A.1) delays takes at least 50 ms3 and 76 ms in an average. In the 3GPP standard [START_REF] Gpp | G; Study on Scenarios and Requirements for Next Generation Access Technologies[END_REF], control plane latency is defined as "the time to move from a battery efficient state (e.g. IDLE) to the start of continuous data transfer (e.g. ACTIVE)". It is equivalent to 2 full round-trips even with a low loaded mobile network core and an RRC establishment success of 100%, that is not negligible for short transfers in terms of latency and induced energy consumption. Energy cost This experiment shows that very short flows of few kilobytes are impacted by the initial access procedure delay and the RTT due to multiple packet exchanges for connection establishment and closing. While the transmission of the user data lasts for 6 ms, the UE modem should be turned on for 178 ms, with a cost in terms of energy.

Limitations Latencies related to radio link errors are not included in the above experiment. MAC HARQ ( §1.1.2.1) recovers most of the link errors, RLC ARQ recovers remaining MAC errors. 1 HARQ acknowledgment/retransmission cycle takes 8 ms (less in 5G), a significant latency compared to the RAN RTT.

DRX is an important mechanism for energy consumption saving by regularly turning off the radio channel. It follows typically a cycle alternating between sleeping and awaking [START_REF]Samsung. 4G-5G Interworking[END_REF]199]. No DRX is configured in our experiments (see Table 2.2). The impact of DRX sleep cycle on latency could be important [START_REF] Pocovi | On the suitability of LTE air interface for reliable lowlatency applications[END_REF] with long sleep cycle to save battery [START_REF] Zhang | LTE optimization engineering handbook[END_REF].

In addition, the handover in mobility between two base stations is not evaluated despite the handover generates latency and head-of-line blocking as highlighted by [START_REF] Tan | Supporting mobile VR in LTE networks: How close are we?[END_REF] for mobile Virtual Reality (VR) traffic in LTE networks.

Longer HTTP transfer

At 2022, webpages weight is commonly less than 4000 kB (equivalent to a 850 ms transfer at full bandwidth in our lab testbed) and constituted of 65 elements from different server locations 4 . The second experiment is an HTTP transfer using Transport Layer Security (TLS) encryption representing the typical downloading of a web resource of 80 kB length (i.e. 55 full MTU IP packets). The TLS encryption adds 2 RTTs of negotiation before the data transfer begins (see Client-Server hello in Appendix D.A.4). Contrary to the first experiment where all packets are transmitted in one burst, the TCP sender may act with several transmission/ACK rounds. Figure 4.5a represents the traffic profile for this HTTPS transmission. The firsts 10 kB of data are transmitted in one burst after the connection establishment that is achieved at second 0.15. Thereafter, the packets are transmitted as soon as the TCP sender transmit new bytes at the reception of an acknowledgment.

The experienced RTT at the beginning of the connection (during TCP handshake and connection establishment) is contained in a range of 20-35 ms in accordance with the measured baseline RTT in section 4.1.1. In the same time, the TCP sender seems not to be able to send enough packets to fill the bandwidth. In average, the achieved goodput is 2.5 Mbps, only 7% of the available radio capacity. It means that with a full usage of the radio capacity, the HTTP object could be transmitted in 17 ms and not in 270 ms as here.

The TCP time-sequence plotted in Figure 4.5a shows how packets are transmitted and acknowledged. The TCP sender increases its sending rate at the reception of a new ACK but it does not to reach in the time window of the transmission the network capacity, that is the primary objective of a TCP sender at the beginning of a connection. The transmission rate increase is also cut back after second 1.75 because of an increase of latency (Fig. 4.5b after second 0.40) from 20 to 90 ms. The uplink traffic profile in red in Fig. 4.5a explains the increase of the latency by a delayed transmission of ACK in the uplink, confirmed with associated LatSeq traces.

A webpage is typically constituted of dozen of HTTP elements of few kilobytes. The above experiment is repeated for each HTTP element to load a webpage (in parallel and in sequence according to element dependencies). Multiple HTTP transfers use a significant part of the available channel capacity (50% compared to 7% for one HTTP transfer) with an intra-bearer competition between flows with different RTT (depending on server location).

Exhibiting bufferbloat

Bufferbloat is an important issue of latency in cellular networks (see §1.3.2). The bufferbloat occurs where excessive buffer incurs long latency, substantial jitter and suboptimal throughput. This section exhibits the bufferbloat and its impact on throughput and real-time response of applications. In particular, we show that uplink is more subject to bufferbloat than downlink.

RRUL tests

The Realtime Response Under Load (RRUL) tests 5 is a common test in the bufferbloat research community. To perform this test, Flent 6 [START_REF] Høiland-Jørgensen | Flent: The flexible network tester[END_REF] was used. This test puts a pressure on the network with 4 TCP flows in downlink and 4 TCP in uplink LatSeq demonstrates that delays are concentrated on the downlink RLC queue.

Since the UE is consistently scheduled over all the resource blocks in the downlink, the traffic profiles may suggest that the intermittent transmission is due a concurrency of access to resource blocks with other TCP flows. The concurrency between TCP flows in the downlink is visible by two elements:

-A larger queueing delay between the green and the purple line, meaning a larger number of packets in the RLC transmission buffer. -In the downlink interface (purple line), the packets are not yet transmitted regularly every millisecond but transmitted without a clear pattern as the radio resources are shared between flows without differentiation.

ACK are quickly generated after the reception of a data packet and then are buffered in the UE's uplink transmission buffer. The total standing RLC downlink queue is about 400 kB and in average 90 kB for this TCP flow, showing the fair share of the downlink buffer. The Bytes-in-Flight (BIF) shows the convergence of the TCP sending rate towards the fair share use of the link with the other TCP flows. Remember that this situation is very particular since there are no delay between RAN and server, there are no competition with other connected user to schedule, and the network interface capacity does not vary in time (i.e. the MCS remains high and constant). However, this experiment in the downlink shows how the downlink queue builds up with concurrent TCP connections.

TCP uplink flows

In the uplink, data packets are carried by the uplink segment while small acknowledgment packets are traversing downlink segment. The achieved datarate is only 6.45 Mbps (less than 36.7 % of radio resources used). The average throughput is 1.61 Mbps with a fairness index of 0.9858 among TCP flows. About 1 % of bytes are considered as lost by the TCP sending stack. The figure of uplink RTT (Fig. 4.8b) shows that the retransmissions are triggered because of an RTT over to 600 ms. Compared to the downlink, the uplink flows show a much higher RTT of around 450 ms. For both uplink and downlink segments, the delay is more important for TCP uplink flows than for downlink. The uplink segment represents the major part of the RTT, with a contribution of 300 ms visible in Figure 4.8a between the magenta and brown curves. The downlink segment delays acknowledgment packets of 100 ms (between brown and pink). A comprehensive study on-device uplink bufferbloat is provided by [START_REF] Guo | Understanding On-Device Bufferbloat for Cellular Upload[END_REF] in [START_REF] Guo | Understanding On-Device Bufferbloat for Cellular Upload[END_REF].

The uplink and downlink latency different results reveals fundamental asymmetries between radio downlink and uplink side. This test exhibits two of them about the scheduling method:

-The difference of achievable throughput is less important in the uplink than in the downlink for a given bandwidth. Indeed, because of the limited uplink emitting power, and the limited computation capabilities, the modulation coding scheme in uplink is limited to small value. -The difference of radio resource management inherent to the centralized architecture of the cell. The scheduling of uplink transmissions is operated by the BS at the reception of SR and BSR sent by the UE. We observed radio frame where the UE does not request uplink radio resources and then, BS does not grant bytes to it, increasing the bufferbloat effect in the uplink.

Uplink Segment as a Source of Latency and Jitter

This section presents in detail experiments and findings related to the RAN uplinkchannel latency. Especially, we demonstrate the importance of uplink radio access and radio resource allocation scheme on the uplink latency characteristics.

Uplink transfer experiment

Experimental setup

We employ a long UDP flow (10 seconds transmission) at a datarate of 1 Mbps in the uplink. The UE generates traffic and the server receives UDP packets. The datarate is chosen to generate a continuous flow of packets without reaching the bottleneck of radio capacity. Packet length is set to have the same size as a TCP pure acknowledgments packets (= 24 bytes of UDP payload, 80 bytes in total). That is small packets compared to Ethernet MTU size packets used in the downlink for data transfer. Notice that small packets (< 200 bytes) in the uplink are very common among applications for instance TCP downlink acknowledgments, cloud gaming applications, remote control. This traffic generation configuration ensures to generate at least one packet every millisecond, maintaining a non-zero transmission buffer at each TTI. The RTT is measured all along the experiment with an ICMP ping as illustrated in Figure 4.9. important jitter between ping measurements from 20 to 80 ms with a maximum to 110 ms. The significant variation of RTT from one measurement to another looks surprising because the UDP packets are generated at a Constant BitRate (CBR) and thus, the increase of RTT must be constant.

Results

Latency Main results are presented in

Traffic profile

The uplink traffic in Fig. 4.10b (zoom on a time window of 240 ms) exhibits a "stepped" transmission with burst of 3 TTIs length every 64 ms carrying the most part of the traffic volume interspersed with less regular small transmissions carrying one or two packets. The consequence of that is an important variation of the RTT, exacerbated by the transmission in "bursts". We define the burst of transmission as the transmission of at least three data packets over one or more consecutive TTIs. On the traffic profile presented, there are 3 bursts of transmission carrying in average 60 packets over 3 TTIs.

Reported buffer

The reported buffer size by BSR 7 (see Fig. 4.10c for BSR signal) is almost constantly the same with index 42 (= [ 5476 , 6411 ] bytes) at least every 64 ms. The BSR value in addition to report a transmission buffer size of around 6000 bytes also shows the stability in time between buffer filling and buffer emptying. However, the 7. §1.1.3.2 for a BSR signal description added queueing delay measured of ≈ 25 ms is not coherent with the buffer length and the radio capacity because a quick calculation suggests that the queueing delay should be 6000 × 8/17.10 6 = 3 ms.

Conclusion

This experiment arises 3 questions about uplink-channel transmission: 1) An important uplink latency; 2) An important jitter from a packet to another; 3) A transmission per burst rather than a continuous transmission.

The latency ( 1) is the consequence of packets transmission delayed into a burst of transmission. The jitter ( 2) is also due to the transmission in bursts of packets arriving at different moments but also due to small packet transmission between bursts. The bursts of transmission (3) come just after the reception of a BSR with a great correlation in time with a delay shift of 8 ms. It has to be noted that the burst size corresponds to the reported buffer length. The small transmissions do not appear to be correlated to the reception of a BSR and they are weakly correlated to the reception of a SR.

The hypothesis is that the transmission pattern, cause of the observed latency and jitter, is a consequence of the radio resource allocation process. Indeed, there are no scheduling prioritization between UEs, air interface capacity variability or radio link retransmissions that could generate this kind of bursty transmissions. We remade the experiment with a lower constant MCS and we got the same result on the traffic pattern with bursts of transmission that lasts over more TTIs.

Uplink latency and jitter in a commercial RAN

We seek to reproduce the previous experiment in a commercial RAN to validate previous conclusions in an other LTE system.

Commercial Radio Access Network setup

In this setup, the radio access network is considered as an object of study on which we perform experiments under different conditions. Radio access network configurations are the same as those used by other users and terminal is immobile so as not to add uncontrolled parameters due to propagation condition variations. The parameter of these experiments are the type of traffic relaying by the cellular network and the load of the cell. End-to-end latency comprises the RAN latency but also on-terminal latency, core The band 3 is used, corresponding to a frequency of 1800 MHz (FDD, uplink: 1710 -1785 MHz, downlink: 1805 -1880 MHz) associated to a bandwidth of 10 MHz. The antenna uses the MIMO technology 2×2. A quick calculation gives a maximum achievable throughput for the cell of 75 Mbps in the downlink and 25 Mbps in the uplink. Huawei is the vendor for both the base station and the cellular modem receiver. The terminal is in a fixed location and in good radio condition i.e. the SNR is high and constant. Such radio conditions minimize latency due to radio layer retransmissions and concentrate latency study on access and scheduling latency. The cellular network interface is made using a Huawei E5377 mobile hotspot 8 with a theoretical maximum throughput of 150 Mbps. The server uses a gigabyte interface. The operating system of the host machine on which client and server are installed is a Linux kernel 4. Network captures are made on the 2 network interfaces as indicated in Fig. 4.11.

The cell serves regular customers in BE way, with a fair sharing of the radio resources. All along the day and mobility events, the load of the cell is varying according to the customer usage of the network. Fig. 4.12 represents the achieved throughput for the testing terminal all along the day. The achieved throughput is the consequence of the instantaneous cell load and the sharing of the radio resources among users. The highest throughput is achieved during the night and the lowest during the afternoon when the number of connected users is the most numerous and the most active. The plot shows a limit of throughput at 43 Mbps during some measurements at night when the cell is almost empty. We use as hypothesis that 43 Mbps is the maximum achievable throughput in the given setup.

8. https://assistance.orange.fr/equipement/cles-3g-et-dominos/huawei-domino-4g-e5377

For the following observations, we perform 2 UDP uplink transfers at 1 Mbps (< 43 Mbps, the radio channel capacity) at the 2 moments pointed with arrows in Fig. 4.12. The first when the network is empty at night and one at noon when the cell is busy.

Observations on the uplink traffic profile

Empty cell case

The traffic profile at a small timescale window of 70 ms (see Fig. 4.13) exhibits a similar uplink traffic pattern as we found in section 4.2.1: 1) An important uplink latency for some packets; 2) An important jitter from one packet to another, especially between packet transmitted "alone" and those transmitted in a burst; 3) The presence of regular burst of transmission preceded 5 ms earlier by an uplink transmission.

In this experiment, the scheduler has no reasons to delay a grant requested by the device while there are PRBs left in a TTI (generated datarate inferior to the cell capacity) and there are no competition with another device.

The duration between 2 bursts of transmission is 13 ms in 90% of cases. With the same reasoning as with the experiment on the OAI testbed, a BSR transmission should be associated to a burst of transmission. The possible BSR periodicity around 13 ms is 8 or 16 ms [START_REF] Gpp | Radio Link Control (RLC) protocol specification[END_REF]. The time between a TB transmission and a burst is 5 ms, that is the minimum achievable latency in LTE-A for the SR-to-Grant-to-Transmission round.

Loaded cell

In a loaded cell where the UE is in competition with other UEs the traffic profile in Fig. 4.14 shows the same pattern with bursts and small transmissions with some irregularities in the interval between bursts. Also, the pattern between burst is not yet clear. The new variables introduced by the resource sharing aspect are : 1) the delay between the explicit scheduling request (BSR or SR) and the grant; 2) the size of grants. The competition between UEs to access to a given TTI induces a variable delay that is not yet equal to 4 ms but greater. The competition between selected UEs for a given TTI reduces the achievable instantaneous throughput (explaining why some bursts of transmission span over 2 or 3 consecutive TTIs instead of one previously). The increased latency and jitter due to resources sharing is expected.

We show with these experiments that the traffic pattern even in the best conditions is not optimal concerning latency, and jitter, and it depends on specific RAN configurations.

Analysis of the uplink radio resource allocation scheme 4.2.3.1 Static code analysis of the OAI base station stack

This section reviews the implementation of the standards that is made in the OAI RAN project [201]. It completes the pure reading of the 3GPP standards in paragraph 1.1.3.2.

At the reception of a SR, the BS sets for the connected UE the flag ul_SR and puts the UE in an active mode (Listing D.D.2). In response, the uplink dynamic scheduling algorithm that is executed each TTI, will assign by default 3 Resource Blocks (RBs) to the UE as soon as the BS as the estimated buffer B ( Q) is equal to 0. The code associated to these instructions of the scheduler is in appendix, Listing D.D.2. Moreover, the bydefault MCS of 10, which is the highest MCS index to have a transmission in Quadrature Phase Shift Keying (QPSK), gives a TBS of 63 bytes 9 . This value of MCS is modified later if a previous CQI has been calculated for this UE. The BS unsets the SR flag as soon as RB are assigned to a user. When a BSR is received at slot n, the buffer length estimation is simply updated to the value Q bsr + reported according to the BSR index table (Listing D.D.3) such as:

Qn = Q bsr + (4.1)
At the reception of a RLC PDU after TB decoding (Listing D.D.4), the estimated buffer Qn is decreased by the size L of the PDU such as:

Qn = [ Qn-1 -L] + (4.2)
The selection of UE U to be scheduled in OAI depends on SR, BSR and an inactivity period. The code associated to the function UE_is_to_be_scheduled(...) that performs this selection is given in appendix, Listing D.D.5. The conditions to schedule a UE could be summarized as:

-The estimated UE transmission buffer is not empty (related to BSR-based estimation) -A SR has been received, and no grants has been generated yet -The UE is connected and active but no activities have been detected since 2 radio frame (= 19 ms)

After that, the scheduler shares PRBs K between U according to its algorithm and the scheduling strategy §1.1.3.2. At the end, a grant is issued with the number of resource block K i to the UE i, the resource block map, the MCS to use, the transmission power and other control flags (Listing D.7.6).

Illustration with an uplink traffic profile

Figure 4.15 illustrates the consequence of the radio resource allocation process implementation on the uplink traffic profile. At the bottom of the figure, the generation of small packets (less one hundred bytes-length) at the UE side which are accumulating in the transmission buffer. At the middle, the air interface transmission and at the top, the egress interface of the base station with the packets transmitted to the core network. Applying the code analysis above, we get the following transmission events: the reception of a SR 4. The BSR reports the size of the buffer containing packets number 2 to 8. It is indicated with BSR(5 ∼ 6) on the figure where the BSR reports in reality a range of bytes corresponding to 5 or 6 packets in our example.

5. An UG is sent accordingly that incurs the transmission of these packets in a burst of 6 packets 8 ms later.

6. For an unknown reason with certain model of smartphone in same radio context, the UE stops to send SR for a while, such as it has a DRX configured, but we verify that our network core does not configure a DRX. We think about an energy consumption manager implemented in the modem.

7. However, because the UE is RRC connected, after an inactivity_timer period the base station "wake up" the UE with a UG as if the base station received a SR. The UG size should not correspond to the transmission needs since no BSR has reported its size yet.

8. In response, the UE transmits data but neither a periodic BSR because the timer has not expire yet, nor a regular BSR because it is the low priority default logical channel nor a padding BSR because there are data in the buffer to transmit. 9. Later, a periodic BSR is transmitted at the expiration of the periodic BSR timer periodic_BSR-timer. Afterwards, a burst of transmission occurs with packets numbered from 11 to 25. The uplink OWD experienced by the packet 11 is about 55 ms to 10 ms for the packet 25 for the same data burst.

Hence, that is not a mystery to observe a burst of transmission 8 ms after a small UG, which betrays the reception of a BSR and an update of the estimated buffer. In fact, the inter-arrival of the bursts is almost constant at 64 ms, corresponding to the BSR periodic timer configured of 64 ms. At the update of the buffer status, the BS allocates resources in consequence to empty the buffer at the maximum datarate. Given the radio capacity, it takes 3 TTIs to empty the buffer built up during 64 ms. With a 20 MHz bandwidth with the highest MCS it could take only one 1 TTI to empty entirely the buffer. Conversely, with a 1.4 MHz bandwidth (= 6 PRBs) with a middle value MCS, it would take at least 50 TTIs. In last situation, the effect of jitter due to BSR reception should be vanished.

An unexpected consequence of the default allocation following a scheduling request is the uplink delay jitter generated in uplink for small packet flows. Indeed, in the presented mechanism above, a big packet of 1500 should be segmented at the RLC layer level without to be completed with two or three small transmissions but reassembled at the burst reception with the remainder of the packet, reducing the interpacket-jitter but with a cost in latency for this unique packet.

The same analysis of the source code could be made on the srsLTE 10 [202] project and leads to the same conclusions about the radio resource allocation according to the reception of SR and the buffer status estimation from BSR.

Patriciello et al. [START_REF] Patriciello | An E2E simulator for 5G NR networks[END_REF] in their implementation of the 5G-NR RAN precises also: "Note that the first scheduling assignment [following a SR] is blind since the gNB does not know the buffer size at the UE yet. In this regard, since this is implementation-specific, we assume that the first scheduling opportunity consists of the minimum amount of OFDM symbols that permits at least a 4 bytes transmission.". Thus, the same transmission pattern should be observed in their NS-3 5G simulator.

Influence of RAN configurations

In the previous section, we described the mechanism of allocation for the default RAN configuration and a CBR traffic. This section explores the influence of configurations, SR periodicity, BSR triggering and bandwidth allocation on the uplink traffic profile and uplink latency. With the sr-ConfigIndex of 157 instead of 5, a SR could be sent every subframe (i.e. every 1 ms). This configuration avoids the necessity for the UE of waiting the next opportunity, eliminating the SR triggering delay. According to [START_REF] Tan | Supporting mobile VR in LTE networks: How close are we?[END_REF]- §6.1, it is 80% of packets that waits for a SR. First, we cannot see any differences for the transfer phase because, this phase is mostly controlled by the BSR reception. Then, a shorter SR periodicity is not used in this phase to reduce latency. More importantly, in the ping phase. The SR is triggered as soon as a new packet arrival as shown in Figure 4.16c where the inter-SR transmission during the transfer is equal to 1 ms (the mechanisms in the modem to trigger SR are not a priori known, it depends on the founder implementation), generating huge amount of SR. The BS answers to the scheduling request with a UG of fixed size of 3 PRBs. Because the UE will receive the UG through the Physical Downlink Control Channel (PDCCH) only 4 ms later (common scheduling delay in LTE), the UE continues to send SR until the limit of 4 (defined by the dsr-TransMax configuration).

Scheduling Request periodicity

The RAN RTT is clearly reduced by reducing the SR transmission opportunity periodicity from 31 ms to 25 ms (see Fig. 4.16a and more importantly, more stable with a standard deviation of 1 ms contrary to 5 ms with default 10 ms SRs When the RAN is loaded, the RTT is reduced of 5 ms while the effect on jitter is less visible.

The traffic profile in Figure 4.16b shows more "little transmissions" between bursts of transmission. It confirms that without a BSR, the BS reacts in a conservative way and only assigns the amount of PRBs necessary to empty the estimated transmission buffer.

The scheduling round time (SR-to-transmission time) is 5 ms at minimum in LTE-A and commonly 8 ms. This value should tend to the millisecond in 5G with proper radio configurations of "K"s timings. The SR periodicity is typically to 10 or 20 ms according to [START_REF] Park | Exll: an extremely low-latency congestion control for mobile cellular networks[END_REF], [START_REF] Tan | Supporting mobile VR in LTE networks: How close are we?[END_REF]-Table 7, values also used in OAI and srsLTE [START_REF] Gomez | srsRAN source code for BSR management[END_REF][START_REF] Nikaein | OAI source code for MAC layer[END_REF]. But it does not constitute on itself a solution to reduce uplink latency because it does not reduce bursty transmissions while it costs in terms of energy for the UE to transmit.

Buffer Status Reporting periodicity

We set the BSR periodic timer at the DRB establishment to 8 ms instead of 64 ms. Thus, BSR timer is expired at every SR (SR transmission opportunity periodicity to 10 ms). The impact of this configuration is clearly visible on the uplink traffic profile in Figure 4.17a. Every 10 or every 27 ms if there are no SR received, a small UG with a pre-allocated number of PRBs is issued. In response, all TBs includes a periodic BSR MAC Control Element (MAC CE) because the timer has expired. 8 ms later a burst of transmission corresponding to the reported BSR is visible. Thus, the transmission bursts occur more regularly, every 18 or 35 ms reducing the average RTT from 55 to 37 (see Fig. 4.17b) and the jitter from one packet to another from 50 ms to 30 ms in relation with traffic profile.

This experiment shows the significant role of the BSR reporting on uplink dynamic scheduling and then, on the latency characteristic. However, reducing BSR periodicity has a cost in terms of control overhead. The regular BSR is a control element of 1 -4 bytes-length in the MAC TB. A BSR transmitted every 64 ms as it is configured in OAI costs for a pool of 20 connected UEs 10 kbps. With a configuration of 8 ms in a loaded cell with up to 100 devices, the cost of BSR increases to 400 kbps only to reports buffer status. In the same time, it has a cost in terms of processing for both UE and BS for an assumed limited benefit. That is a trade-off between the knowledge accuracy we want at the base station for the scheduling and the costs we want to pay in terms of resource blocks and processing.

Allocate all the bandwidth to UE at an explicit scheduling request

With this third experiment (Figure 4.18), we modified the uplink scheduler to allocate all the bandwidth of a slot to the UE each time it is scheduled. By then, the UE always receives the maximum radio resource capacity on a given slot. That is an unrealistic in practice, but it helps to understand what happens if the scheduler always guest the buffer status without reporting delay, inaccuracy and periodicity. The UDP uplink traffic profile depicted in Fig. 4.18b shows a clear regularity of the transmissions with small burst of Contrary to the previous experiments, we observe many SR triggering, every 10 ms in fact. The cycle of 27 ms due to eNB scheduler is not yet visible. Periodic BSR continues to be transmitted with a value of 0 every 70 ms (> 64). The question that comes is why a SR is triggered at every SR transmission opportunity whereas it was not the case in the baseline experiment. Our hypothesis is that, because BS allocates every time a large band (and not few PRBs), the UE modem never enters a "sleep" mode for battery efficiency, in other words, the high rate uplink keeps the modem on when a SR transmission opportunity occurs. This experiment demonstrates the existence of a "sleep" mode for uplink interface, but we could not pursue the investigation since it is an internal modem mechanisms. We repeated the experiment with 2 different models of smartphones and observed the same phenomena that in the baseline case we have few SR compared to this fullband allocation experiment.

Concerning the latency measured with the ping RTT (in Fig. 4.18a), the latency and the jitter is clearly decreased before and during the uplink traffic generation. The minimum RTT is reduced to 12 ms that is close to the minimum achievable in LTE that is 10.5 ms. The mean RTT is about 21 ms instead of 30 with a jitter of 14 ms without the uplink traffic. More interestingly when the generation of the uplink traffic, the mean RTT drops to 18 ms with a lower jitter of 10 ms that looks counter-intuitive. In fact, the generation of the uplink traffic increases the number of transmission opportunities with a regularly generated SR without filling all the TB. Then the ping RTT benefits from the regularly allocated TTI without being in competition to be carried in the transport block. Conversely, when the ping is the only traffic, it should trigger the SR and wait for the grant since the inter-departure of the ping is far more than 10 ms.

This experiment shows the importance of a short regular scheduling with the appropriate amount of PRBs to tend towards low latency and low jitter. But this amount of PRBs to allocate is not easy to determine under the resource usage constraint and partial knowledge of transmission buffer length.

Summary

In this section we investigated the influence of RAN configurations, i.e. the SR periodicity configuration, the periodic BSR configuration and the scheduler response to a scheduling request, on the uplink traffic profile and the RAN RTT. ECDF of RTTs obtained for the different configurations is presented in Figure 4.19. In solid lines, baseline RTT and in dashed line, when the DRB is loaded with a 1 Mbps uplink UDP traffic.

The best way to reduce latency efficiently, is to assign all the bandwidth (all the PRBs of the next TTI) to the UE each time it requests radio resources (i.e. at the reception of a SR or a non-zero BSR), but this implies a low spectral efficiency, an important transmission and coding energy costs for the UE and an unfair sharing of radio resources between UEs. Without modifying the scheduling algorithm, both reduction of the SR and BSR periodicity reduce the latency compared to the baseline configuration. The reduction of the SR periodicity to 1 ms clearly reduced the uplink latency and more importantly, it reduces the dispersion around the median when the ICMP flow is alone, whereas the reduction of the BSR periodicity to 8 ms reduces the latency but not the dispersion compared to baseline. When the ICMP flow shares the DRB with the UDP flow, the best strategy to reduce latency is to reduce the BSR periodicity (the median of the baseline is 55 ms, 50 with a short SR periodicity and 35 ms with a short BSR periodicity). Reducing the BSR periodicity has a beneficial consequence on the uplink latency jitter by reducing the size of the transmission bursts and the effect of "small transmission packets between big bursts".

In conclusion, to reduce baseline latency, we should select a SR configuration such as it increases its number of transmission opportunity but to reduce jitter and burstiness of a traffic flow, the buffer status reporting periodicity is the best trade-off configuration.

On the buffer knowledge

The allocation of the radio resource grid to devices requires knowledge of their buffer status for an efficient allocation of radio resources according to the needs. The detailed analysis in the preceding paragraphs of the standards ( §1.1.3.2), of the source code ( §4.2.3) completed by experiments in Section 4.2.4 inform us that the base station's knowledge of the UE's buffer status are directly related to the transmission of the BSR. The illustration of that is plotted in Figure 4.20 where the ground-truth RLC transmission buffer occupancy is in blue and the reported value represented with black triangles (data from the UDP uplink experiment in paragraph 4.2). After the reception of each BSR, a corresponding UG causes the emptying the buffer of the corresponded reported value. It lets the buffer not completed emptied as there are a delay between the transmission of a BSR and the corresponding TB transmission. Moreover, the indexing of the reported byte volume induces an inaccuracy of 15% for 6000 bytes in the buffer. This inaccuracy in the reporting is clearly visible on the figure with 4 BSR reporting index 44 after second 0.6 and on BSR reporting index 45 at second 0.9 by a threshold effect, changing the grant size of about 1000 bytes.

The figure is very illustrative of the buffer status sampling problem posed by the periodic buffer status reporting with a long period. The knowledge of the buffer state that the scheduler has is limited. The importance of the information reported by BSR for the scheduler is discussed in [START_REF] Kv Pradap | Uplink buffer status reporting for delay constrained flows in 3GPP long term evolution[END_REF] in particular for delay-sensitive traffics.

In 2009 in his thesis [START_REF] Francesco | Scheduling and Link Adaptation for Uplink SC-FDMA Systems-A LTE Case Study[END_REF], F.D. Calabrese dedicates the section §7.4.2 to the question of the impact of buffer knowledge on the streaming service. He shortly compared the throughput performances according to 2 types of traffic, a bulk CBR traffic with large packet and a bulk CBR traffic with small packet (768 Bytes) for an ideal knowledge of the buffer status and partial knowledge with a buffer status reports (the periodicity is not detailed by the author). He found that the partial knowledge impacts the average cell throughput, especially for the uplink small packet case with a loss of 8% and an increased bandwidth utilization due to padding and delays. In his work, he did not develop the latency and burstiness aspect related to the partial knowledge of the buffer status.

Related observations

Some work already noticed the intrinsic bursty nature of the uplink access [START_REF] Bruhn | Performance and Improvements of TCP CUBIC in Low-Delay Cellular Networks[END_REF][205]- §4.2.5. However, it is often hide and explained as the capacity variability of the air interface, the scheduling and the HARQ retransmissions [START_REF] Sahu | End-to-end uplink delay jitter in LTE systems[END_REF]. We found 3 of them which notes the link between the latency and the jitter with the uplink traffic profile and the dynamic scheduling.

In "End-to-end delay jitter in LTE Uplink" (2020) [START_REF] Sahu | End-to-End Delay Jitter in LTE Uplink: Simple Models, Empirical Validation & Applications[END_REF], Sahu models and addresses the problem of achieving a delay jitter target for uplink RAN. Especially, this work provide a model of uplink scheduling flow in NS3 and with the associated queue length process. He uses an M/G/1 fluid model to estimate the delay jitter. A strong limitation we found, he indicated himself is the manner of BSR is generated. As he wrote "When an IP packet arrives in the PDCP queue, UE [as it does not have an uplink allocation] firstly sends an uplink SR to the eNB so that eNB can give an initial uplink allocation. This generally happens in LTE and not valid for NS3. In NS3, directly BSR is generated which indicates the amount of resource requirement and in the very next subframe, the eNB MAC scheduler gives the allocation of TBS (in bytes) for the corresponding BSR.", but that is not true in both OAI, srsLTE and deployed networks. Thus, it models correctly the dynamic of queue length variation in time according to grant allocation, but it does not model the lack of knowledge on buffer status as we pointed out.

Bruhn, Kuehlewind and Muehleisen in "Performance and Improvements of TCP Cubic in low-delay cellular networks" (2022) [START_REF] Bruhn | Performance and Improvements of TCP CUBIC in Low-Delay Cellular Networks[END_REF] studied the link between delay variations in mobile networks and SR procedure (see II.A and II.C) in an Ericsson RAN testbed similar to the commercial network setup we used in §4.2.2. They depict in Fig. 3.a) the RTT sampling with ACKs over time and show the important variation of delay from one burst of transmission to another. Their analysis of the situation is in agreement with ours, but we think from our experiments that they underestimate the role of the BSR in the jitter and the creation of transmission bursts. [START_REF] Park | Exll: an extremely low-latency congestion control for mobile cellular networks[END_REF] pointed out in section §3.2 the difference between downlink and uplink traffic profile due to the grant-based uplink dynamic scheduling process. Figure 7 in this work illustrates well the difference of scheduling between downlink and uplink because of the SR periodicity. They correctly analysed the importance of the SR periodicity on the RTT variation over time. Especially in their work, they propose in section §4.4 a method to calculate the minimum RTT of the RAN from the uplink traffic observation and an estimation of SR periodicity. With the arguments we develop earlier in this chapter, we think they have missed a part of the analysis on their presented uplink traffic profile by not taking into account the contribution of the BSR to the transmission bursts which we see repeating every 40 ms or so (indicating a 32 ms BSR periodicity configuration in the given example). Ingemar Johansson from Ericsson already exhibited in [START_REF] Johansson | Congestion Control for 4G/5G Networks IETF 96[END_REF] and [START_REF] Johansson | Congestion control for 4G and 5G access[END_REF] (2016) the bursty nature of the uplink radio access network. Especially he pointed out the fundamental difference between uplink and downlink scheduling where packets are transmitted as quickly as resource allocation allows with a "smooth" pattern when the uplink scheduling shows bundled packets that affect RTT. Figure 4.22 extracted from the IETF presentation of Johansson plots the uplink traffic profile at the PDCP layer at the UE (blue) and the base station (green) and at the RLC layer. Moreover, on RLC plot he indicates the SR and the UGs. The uplink traffic profile demonstrates that packets are waiting for an UG in the RLC transmission buffer. However, we regret that the BSRs are not reported on the plot and are not studied as a source of transmission bursts. This work of Johansson is part of larger study on transport protocols and 4G/5G networks [START_REF] Johansson | Congestion control for 4G and 5G access[END_REF]. Thus, it focuses on the consequence of the bursts and RTT jitter more than on the analysis of the root causes.

Graphic removed to respect copyright

Conclusion

In this chapter we investigated latency characteristics and jitter in the LTE RAN with an OpenAirInterface lab testbed.

The first section characterized the RTT of the RAN and the bufferbloat with a focus on the uplink channel. The uplink channel is less studied in the literature in spite of the interesting questions it poses. We dedicated the second section to the study of uplink traffic profile, and latency. We found a link between the access latency, latency variation and the grant-based allocation scheme. In particular this study is based on both controlled-variable experiments, commercial network experiments, analytical study of the code and on analysis of related observations from the literature.

The conclusion is that uplink dynamic allocation parameters made with a concern for a compromise between performance and control/energy costs can lead to important delays and above all significant jitter with bursty transmissions. The question, already summarized by Johansson in [START_REF] Johansson | Congestion control for 4G and 5G access[END_REF] that comes is the impact of the uplink channel traffic profile on the data flow and services.

Chapter 5

WHY TACKLE THE UPLINK-CHANNEL

LATENCY AND JITTER ? TCP inefficiency over cellular networks Legacy congestion control protocols such as TCP and its variants are known to perform inefficiently over cellular networks. In particular, they fail to take advantage of the whole cellular network capacity. This observation is made by a certain number of studies [6,[START_REF] Pötsch | Future Mobile Transport Protocols[END_REF][START_REF] Robert | Behaviour of common TCP variants over LTE[END_REF][START_REF] Srivastava Ashutosh | An experimental evaluation of low latency congestion control for mmwave links[END_REF]. In [START_REF] Liu | On improving TCP performance over mobile data networks[END_REF], Liu et al. found a network capacity usage of 43.7%, 53.9% in [START_REF] Austrheim | Implementing immediate forwarding for 4G in a network simulator[END_REF], 52.6% in [START_REF] Huang | An in-depth study of LTE: effect of network protocol and application behavior on performance[END_REF] and in [6], authors found an LTE network capacity utilization by TCP of 68% (with tuned parameters [START_REF] Briscoe | Reducing Internet Latency: A Survey of Techniques and Their Merits[END_REF]- §V-C.2).a)). This even worse in 5G with an estimated utilization of only 22% of the available capacities [6]. The inefficiency of TCP over cellular networks comes from a highly variable channel capacities over short time scales (path loss variation and obstruction associated to mobility) and unpredictable changes in RAN latency (e.g. handover, network load, HARQ loss) [START_REF] Schulte | Mobile network delay characteristics and interactions with the transport layer[END_REF][START_REF] Zhang | The bufferbloat problem over intermittent multi-gbps mmwave links[END_REF][START_REF] Pieska | TCP performance over 5G mmWave links-Tradeoff between capacity and latency[END_REF]. From that matter of fact, the topic of TCP over mobile network has becoming very active [START_REF] Polese | A Survey on Recent Advances in Transport Layer Protocols[END_REF][START_REF] Lorincz | A Comprehensive Overview of TCP Congestion Control in 5G Networks: Research Challenges and Future Perspectives[END_REF] since the apparition of mobile internet services with 3G and LTE, but the utilization of mobile network specific TCP variant in the network remains limited [START_REF] Mishra | The Great Internet TCP Congestion Control Census[END_REF].

TCP Cubic experiment in testbed TCP Cubic is one of the most widely deployed TCP variants on the today internet [START_REF] Mishra | The Great Internet TCP Congestion Control Census[END_REF]. Figure 5.1 depicts the throughput achieved over time of TCP Cubic on the OAI testbed (see Table 2.2 for network configurations) with a constant network capacity. In this experimentation, 2 radio errors have been recovered by the HARQ process, resulting in an error-free radio link. The average achieved goodput for a 10 seconds transfer is 22 Mbps, 59% of the network capacity. The throughput regularly decreases as the sender congestion control believes to detect an indication of a congested network. TCP CCA uses acknowledgments to detect packet loss in the network and react to congestion [START_REF] David | Relaxing Restrictions on Explicit Congestion Notification (ECN) Experimentation[END_REF] (Fig. 1.7). For a downlink transfer, an ACK flow is carried by the uplink channel.

RTT and transmission pattern

In addition to loss reporting, ACKs are used by the TCP stack to estimate RTT of the network. Therefore, the sampling of the RAN RTT is directly correlated to the frequency of ACK generation. The RTT measured by the TCP sender over time is plotted in Figure 5.3 and shows important variation from one ACK packet to another. RTT varies from 30 ms to 120 ms with a mean at 60 ms and does not reflect the reality of the network status that is under-utilized. In this example, the RTT presents typical cycle between one extremum to another. There are plotted the TCP traffic profile at the entry of the RAN in the downlink as it is transmitted by the TCP sender, the traffic profiles at the UE radio interface and traffic profiles at BS's RLC layer in the uplink and at the output of the RAN. The RTT is measured between green and red curves. In (A), the RTT corresponds to the minimum RAN RTT as the downlink queue is empty and an uplink transmission opportunity is available. In (B), the last data packet of the initial TCP transmission burst is composed of 25% of queuing latency in the downlink and 75% of access latency due to a lack of UG; Burst of transmission in (C) after the reception of a BSR by the BS and packet transmitted alone in (D) even though the UE transmission buffer is not empty have been discussed in Section 4.2. It can be seen from figure 5.4 the under-utilization of the air interface capacity with the "pauses" in the transmission of the purple traffic profile. The purple line slope when the downlink buffer is non-empty corresponds to the peak capacity of the downlink interface, i.e. 37.5 Mbps.

Congestion control

The lack of data in the downlink buffer is due to 2 TCP mechanisms related to the congestion window. The congestion window maintains the number of bytes in the network not yet acknowledged below the limit fixed by the congestion control algorithm. The reception of an ACK frees up space in the window, allowing TCP sender to transmit new bytes. That is why after the reception of a burst of ACK, the sending of a burst of new data in the downlink can be observed. A solution adopted by TCP BBR [START_REF] Cardwell | BBR: Congestion-based congestion control[END_REF] is to pace [START_REF] Polese | A Survey on Recent Advances in Transport Layer Protocols[END_REF] the sending datarate to the estimated network bottleneck capacity. Pacing keeps the downlink buffer just full but not fuller 3 , preserving downlink latency from buffering delays. However, the packet sending rate is still limited by the congestion window.

The congestion window is controlled dynamically by the TCP CCA. The objective of CCA is to maintain and adapt a congestion window such as the sending rate is equal to the network bottleneck capacity (e.g. radio link capacity) so avoid buffer overflow losses. TCP Cubic CCA depends on packet loss (or estimated loss after DUP ACK and Retransmission TimeOut (RTO) expiration) as many other TCP variants (i.e. loss-based) to detect congestion or changes in network conditions. Others variants use RTT rises to detect congestion (i.e. delay-based) and others combine RTT and estimated bandwidth such as BBR (i.e. hybrid). All of these algorithms have in common to be dependent of the smoothed RTT [START_REF] Gurtov | Effect of delays on TCP performance[END_REF] calculated from ACK feedback loop and exposed by the TCP stack. . Areas in red represents moments when sRT T + 4 × RT T V AR > RT T , sign of a possible congestion [START_REF] Van Jacobson | Congestion avoidance and control[END_REF]. According to this calculus, the network is most of the time in a congestion state even though it is in reality a lack of uplink grants to transmit ACKs. It is therefore not surprising to see Cubic struggling to increase congestion window.

TCP BBR and Quic experiments

Under the same configurations, the state-of-theart of CCA, TCP BBR [START_REF] Atxutegi | On the use of TCP BBR in cellular networks[END_REF] outperforms Cubic with 27.5 Mbps and 73% utilization of the network capacity. However, BBR also suffers from the significant variation of RTT as it tries to reach optimal BDP with a varying delay [START_REF] Kumar | TCP BBR for ultra-low latency networking: challenges, analysis, and solutions[END_REF].

The best network capacity utilization is obtained for Quic transport protocol (see an introduction to Quic in §1.2.3 and Quic traffic generation in §2.3) and BBR CCA with 86% (= 32.2 Mbps).

In [START_REF] Molavi | Taking a long look at QUIC: an approach for rigorous evaluation of rapidly evolving transport protocols[END_REF], authors evaluated Quic under different scenarios and also found Quic generally outperforms TCP, but they additionally identified performance issues on cellular networks with a poor radio resource utilization.

ACKs have been redesigned in Quic protocol [START_REF] Iyengar | QUIC Loss Detection and Congestion Control[END_REF] to carry more information [START_REF] Custura | Rethinking acks at the transport layer[END_REF] but less often [START_REF] Iyengar | QUIC Acknowledgement Frequency[END_REF][START_REF] Fairhurst | Current QUIC performance is compromised compared to TCP for asymmetric paths[END_REF]. We got an Ack ratio from 1 : 10 to 1 : 36, depending on loss rate. As for TCP, Quic sending stack estimates from ACK sampling, the minimum RTT, the smoothed RTT and the RTT variation [86]-5. As there are 29 ms between each ACK, the uplink transmission of them is done similarly to the transmission of pings in section 4.1.1 with an uplink latency of 20 to 30 ms, less subject to RTT variation from one ACK to another. The reduction of the ACK frequency comes with a less reactivity to channel degradation, materialized by the increase of downlink packet delay associated to a buffer filling up.

Short TCP transfer and Hystart

In addition to that, most applications are using TCP for short transmissions, such as Web browsing that opens multiple TCP connections to load objects from different servers 7 . At the start of the flow, TCP is continually 7. 72 objects per webpage in average according to live measurements of https://webview.orange.com seeking out network bottleneck capacity by increasing its rate until it detects a congestion, following which TCP enters into congestion avoidance phase. Slow-start phase that characterizes TCP connection beginning has already been identified as a limitation 8 in LTE networks [START_REF] Huang | An in-depth study of LTE: effect of network protocol and application behavior on performance[END_REF][START_REF] Peters | TCP HyStart Performance over a Satellite Network[END_REF][START_REF] Bruhn | Behavior of TCP CUBIC in Low-Latency Mobile Radio Networks[END_REF][START_REF] Koyama | Slow start algorithm for mobile broadband networks including delay unrelated to network congestion[END_REF]. Exiting the start-up phase too quickly [START_REF] Bruhn | Performance and Improvements of TCP CUBIC in Low-Delay Cellular Networks[END_REF] does not allow network capacity to be reached quickly at the start of the connection. Cubic CCA takes several rounds trip to reach the estimated network capacity after an early exit from slowstart. That is illustrated in Figure 5.6 with the ECDF of throughput achieved for TCP connection carrying either 1 MB or 10 MB. The mean achieved throughput of a 10 MB transfer is 22 Mbps while it is only 7 Mbps for a 1 MB file. Similar observations are made in [START_REF] Bruhn | Behavior of TCP CUBIC in Low-Latency Mobile Radio Networks[END_REF][START_REF] Agarwal | Mind the delay: the adverse effects of delay-based TCP on HTTP[END_REF][START_REF] Garcia | Examining TCP short flow performance in cellular networks through active and passive measurements[END_REF][START_REF] Järvinen | Effect of competing TCP traffic on interactive real-time communication[END_REF], impacting HTTP transfer duration. In average, TCP exits slow-start after 175 kB transmitted (i.e. 120 ms), and hits for the first time the network capacity after having transmitted 3.2 MB (i.e. 1.65 sec). The early exit of slow-start is triggered by the increase of latency, as an implicit sign of congestion (example in Fig. 4.5b), the same sign of congestion as in Fig. 5.5. Tackling latency jitter at the beginning of the TCP transmission has multiple positive impacts : It increases radio resource utilization in downlink; It makes the transfer duration shorter for a better user experience and also reduces the time during which user modem has to be turned on.

Summary

The uplink latency and jitter is an issue for TCP transport protocol to efficiently use radio access network capacity (figure 5.7). By delaying ACKs, the uplink channel hides the congestion state of the downlink channel, which fools the transmitting TCP stack and leads to falsely congestion detection. That is particularly visible at the beginning of the connection where TCP seeks to reach network capacity but fails because of an increase of RTT due to uplink allocation scheme. Moreover, the inter-arrival times 8. Especially Hystart algorithm [START_REF] Ha | Hybrid slow start for high-bandwidth and longdistance networks[END_REF] that is by-default used by Linux TCP stack and bursts sizes fluctuate such in it does not follow a simple Poisson process as assumed in [START_REF] Winstein | Stochastic forecasts achieve high throughput and low delay over cellular networks[END_REF] and in several mobile network queueing models. The next generation Quic protocol is also sensible to these variations of RTT to estimate the network congestion status. The issue should be the same and even worse in 5G with many latency challenges associated to it: highly variable radio channel capacity 9 [START_REF] Zhang | Transport layer performance in 5G mmWave cellular[END_REF][START_REF] Srivastava Ashutosh | An experimental evaluation of low latency congestion control for mmwave links[END_REF][START_REF] Welzl | Future Internet Congestion Control: The Diminishing Feedback Problem[END_REF][START_REF] Zaki | Adaptive congestion control for unpredictable cellular networks[END_REF], frequent handover, buffer sizing and blockage effect [START_REF] Poorzare | Challenges on the Way of Implementing TCP over 5G Networks[END_REF][81]-Table 2.

-Considering return path latency in access networks. Flavien Ronteix-Jacquet. NetSatDay meets NoF: "Fast convergence of congestion control", July 2021, Toulouse / Virtual, France. Link to presentation : https: //hal.archives-ouvertes.fr/hal-03327308 Communication 9. Figures of throughput in [START_REF] Zhang | Transport layer performance in 5G mmWave cellular[END_REF] are very illustrative of variability of the 5G mmWave channel In section 4.2 of Chapter 4 we exhibited with an uplink transfer how the current grantbased access method generates latency and bursts. Figure 6.1 illustrates the link between the RTT we observed on a commercial network (see setup in Sec. 4.2.2 and publication [4]) and the traffic profile captured showing transmission "steps". The circled region shows an increase of the latency due to a lack of grants. The corresponding uplink traffic in exhibits a very bursty profile, in line with related observations presented in section 4.2.6. Bursts of transmissions (big step on the traffic profile) occur after a BSR is received. Such delays and delay variation are not justified by a radio capacity bottleneck or by retransmissions due to poor radio environment, but only by allocation based on grants and BSR. BE default radio bearer utilizes the grant-based scheme for uplink radio resource allocation.

In the same time, we motivated the needs to minimize bursty uplink transmission pattern and jitter in Chapter 5 with the example of TCP. TCP congestion control has difficulties to adapt its sending transmission window to the network capacity variability and quickly varying RTT. Thus, the network capacity is not fully utilized (see Fig. 5.1) that is a source of internet latency identified in paragraphs 1.3.3 and 1.3.2. TCP represents the major part of the internet traffic, that is carrying by the default BE bearer in the RAN.

Our approach

Our approach redefines the estimation of UEs buffer status at the BS left unmodified the grant-based access method, the BS's MAC scheduler and the UE unchanged. A continuous estimation of the UE's transmission buffer is performed at the difference of the sampling made with "legacy" BSR mechanism. The model of estimation for each UE uses BSR measurements and the transport block utilization rate and returns the estimation buffer occupancy. We successfully show that a simple linear estimation model reduces effectively uplink latency and jitter for non-QoS flows. More than an algorithm, we propose a new architecture for the uplink scheduler that includes an estimation model.

Related work

The firsts relative work concerning estimation of transmission buffer length is the implementation done in RANs using BSR. We studied it in Section 4.2.3. The conclusion is that estimation is partial and depends on a BSR reception event.

Several other studies have proposed model of estimation for the purpose of radio resource allocation. Applied to SPS or Fast-UL ( §1.1.3.1), the estimation is used to preschedule grants before the UE has to request it with a SR or a BSR. Fast-UL especially is an efficient solution to lowering uplink latency [START_REF] Johansson | Congestion Control for 4G/5G Networks IETF 96[END_REF]. [START_REF] Li | Predictive pre-allocation for low-latency uplink access in industrial wireless networks[END_REF] uses a predictive pre-allocation for low-latency uplink access in the industrial wireless networks. [START_REF] Moberg | Prohibiting unnecessary scheduling requests for uplink grants[END_REF] prohibits unnecessary SR for uplink grants by using the SPS for lower priority data to not delay high priority which uses Sounding Reference Signal (SRS). [START_REF] Nordström | Scheduling of delay-sensitive traffic[END_REF] is a predictive scheduling to solve delay problem through proactively scheduling a mobile station without actual knowledge of the data it wants to transfer. A close work to our proposition is the prediction based on transmission history proposed by [START_REF] Feng | A Predictive Semi-Persistent Scheduling Scheme for Low-Latency Applications in LTE and NR Networks[END_REF] for SPS. They show how the prediction of size new arrived data to adapt semi-persistent scheduling is beneficial for uplink latency. Their approach needs to be implemented in both UE and BS because of SPS configuration and does not take advantages of dynamic scheduler strategy. The prediction method is also different as our approach, based on auto-regressive model using the buffer occupancy time series. The patent [START_REF] Ankel | Uplink prescheduling[END_REF] proposes to predict using the history of packet generation the arrival of new packet to pre-allocate radio resources. These works are designed towards specific services such as industry machine to machine communications. For a majority of them, they are taking place in grant-free allocation schemes with SPS and does not take advantage of BSR reporting, dynamic MAC scheduling and modem optimizations. However, mathematical models for estimation developed in these work have to be taken into account when designing an traffic generation estimation algorithm.

In §4.2.4, we concluded the decrease of BSR transmission periodicity diminishes the uplink latency of long transmissions. Instead of reducing the periodic BSR timer value, other studies propose to better estimate the buffer occupancy reported to the BS, as proposed by [START_REF] Aby | An optimized method of buffer status reporting for uplink data in lte[END_REF], taking into account the so-called buffer reporting delay. In that work, the BSR value is modified in the UE, to integrate the estimated data filled in the buffer during the time for the BSR to become effective. Unfortunately, this attractive approach requires a modification of the UE, which may reveal quite difficult to implement and deploy. In a recent work [START_REF] Zhang | Predicting Buffer Status Report (BSR) for 6G Scheduling using Machine Learning Models[END_REF] (2021) Zhang also tries to predict the BSR to integrate information of data arriving after the transmission of the BSR for reducing uplink scheduling latency. For that, he evaluates several state-of-the-art supervised machine learning algorithm such as random forest, XGBoost and Long Short Term Memory (LSTM). However, this solution does not solve the problem of the continuous estimation of the UE transmission needs. He solely proposes to palliate a limitation of the BSR, which is the delay between its reception and the effective transmission following an UG. [START_REF] Kv Pradap | Uplink buffer status reporting for delay constrained flows in 3GPP long term evolution[END_REF] proposes a "No Delay Information" scheme to limit the feedback buffer status reporting schemes for delay-constrained DRB. They illustrate by their solution that delay information at the base station is important for scheduling real-time flows. [151][243] propose to estimate delay experienced by packets from BSR (not the actual queue length) for delay-aware scheduling purpose. In [START_REF] Bun | Method for link buffer size and queue length estimation for bandwidth-varying mobile data networks[END_REF], the queue length estimation is also used for a better congestion control in bandwidth-varying mobile network. The queue length estimation is also used like us for the uplink dynamic scheduling [START_REF] Rizk | Queue-aware uplink scheduling with stochastic guarantees[END_REF] [START_REF] Chandrasekhar | Timer-based scheme for user equipment queue state estimation[END_REF] but their approach of estimation is different and less general. Guo et al. in [START_REF] Guo | Understanding On-Device Bufferbloat for Cellular Upload[END_REF] also highlights some limitations of BSR in §6.2 and proposes, as a part of a bigger solution, a buffer estimation method in order to reduce on-device delay (i.e. uplink bufferbloat). The authors combine BSR, padding statistics (i.e. transport block utilization) and grants to estimate throughput and finally derives from it an estimation of transmission buffer length. This solution is implemented at UE's modem for active queueing management purpose, so the estimated value cannot be used by BS's scheduler for resources allocation.

Design and Implementation

We propose to include a new module called Enhanced-BSR (eBSR) in the MAC entity of the BS as shown in Fig. 6.2. Its role is to provide an estimation of the UE buffer occupancy that is more frequently updated than the BSR. The estimation is based on uplink signalling i.e. BSR, SR, padding size. The proposed estimation algorithm models the traffic source as being of constant bitrate over a time period.

Estimation algorithm principles

Queuing model

In our proposal each UE i is modeled as a queueing system that is filled by a single traffic source of piecewise constant datarate D and emptied by uplink transmissions L following a UG g. At slot t, the number of bytes Q i t in the buffer of UE i is given by:

Q i t = [Q i t-1 + D i t • T tti -L i t ] + (6.1)
where D i t is the packet generation datarate, T tti is the slot duration (e.g. T tti = 1 ms) and L i t is the number of bytes correctly received by the BS. All the above parameters are known to the UE. The BS knows L i t but neither Q i t-1 nor D i t . The computations below are per UE but for the sake of simplicity, we remove the upper index i in the following.

The objective of the estimation model is to provide the BS with an estimate Qt of Q t . At each time slot and for each UE, the base station computes:

Qt = [ Qt-1 + Dt • T tti -L t ] + (6.2)
The key parameter in (Equation 6.2) is the estimated uplink source rate Dt . To explain how Dt is computed, we introduce d t and g t as the number of bytes transmitted and the number of bytes granted for slot t, respectively. Note that L t ≤ d t ≤ g t . When d t = L t , the transport block is correctly received by the BS (no HARQ retransmission). When d t = g t , all the resource blocks allocated by the BS are used by the UE. Even if the BS does not receive the transport block, it can determine L n and it knows d n and g n . We define the transport block utilization computed at each reception of a transport block as Value of D is updated according to events as illustrated in figure 6.3. At reception of the first SR, variable D is initialized to a default value D0 (e.g. to 128 kbps, which is the LTE air interface capacity for a low channel quality) and then increased or decreased as described below.

ρ n = d n /g n , note that 0 ≤ ρ n ≤ 1.

Estimation update

Enhanced-BSR

Adjust SR

D ̂t +Δd

After that, the update of Q is made continuously at each TTI with the equation 6.2 and D is updated at a reception of a SR, a BSR, of a TB or at the activation of a probing phase. D is increased and decreased by a value ∆d at the triggering of one event (Adjust BSR, Adjust SR, Adjust PRB usage, Probing). The estimation ends when the estimated traffic generation has stopped.

Increase D : A non-zero BSR reception indicates a data accumulation at the UE queue. Q is updated with the reported buffer size Q bsr and D is increased accordingly to account for the extra bytes generated in the last BSR period of duration T bsr according to:

Dn = Dt-1 + ( n m=n-T bsr (g m -L m ) -Q bsr )/T bsr (6.3)
In Equation 6.3 the sum includes the difference between granted and estimated bytes in the last BSR period since the queue build-up could also be the result of the scheduling policy. We also propose a probing mechanism which aims to deliberately increase D for a period of time when no BSR has been received for a period of time.

Probing has been added because network-configured BSR periodicity could be as long as 1 second which incurs a delay to adapt to an increase of D. Probing phase tries to "sense" an increase of D before the reception of a BSR occurs by a voluntary increase of D. If a low ρ is observed, the estimation undo to the previous D.

Decrease D : A novelty of this work is the use of ρ t defined as the proportion of resource used to transmit user data. Its role is to monitor the effective use of the granted radio resource and the relevance of the estimation of UE transmission needs.

An important goal of the estimation algorithm is to avoid radio resource waste caused by an over-estimation of the UE buffer. A too large UG compared to the current buffer size is denoted by a low value of ρ t . Thus, as soon as ρ t indicates wasted radio resources, D is decreased to lower the estimation of UE transmission needs. ρ t < ρ thresh (with ρ thresh = 0.8) triggers a decrease of D such that:

Dn = Dn-1 • ρ n (6.4)
The threshold of ρ thresh is a parameter of the algorithm. We use this value because it is common to set control plane overhead at 14% for goodput calculation [START_REF] Gpp | User Equipment (UE) radio access capabilities[END_REF] and because we observed ρ t > ρ thresh when UG are correctly sized. Also, ρ t is not computed when a HARQ error occurs in slot t as the content of the transport block in terms of user data, control signalling and padding is not known. When the traffic generation has stopped, ρ t is expected to be 0. We use this property to end the estimation when ρ t < ρ end (with ρ end = 0.1) and Dt-1 < Dend (e.g. Dend = 128 kbps).

In practice, an exponential smoothing on D of parameter α = 0.8 is used to converge towards D while quickly reacting to D decrease (An example of D convergence is given in Appendix E.E.2a).

Fallback to "legacy" BSR : We employ a conservative method to preserve radio resource utilization which stops the estimation and fallbacks to "legacy" BSR estimation when the algorithm does not converge i.e. when the achieved ρ < 0.8 for 10 consecutive transmission in a time frame of 200 ms. The time frame is chosen to let multiple BSR transmissions and adaption rounds. With a BSR timer at 64 ms, it means the model could adapt estimated datarate 3 times from BSR. If during this phase of adaption, the model does not converge with a ρ < 0.8, then it means the source datarate is not deterministic enough. That is the case when the traffic generation pattern does not comply with the linear model of the proposed estimation.

Prediction for the uplink dynamic scheduler

The output of the algorithm is the estimation of the UE transmission buffer length Qt . This value is then taken into account by the uplink dynamic scheduler to determine UG size according to its own algorithm as illustrated in Fig. 6.2. The first contribution of our approach is to provide a continuous estimation of the UE buffer length from the Equation 6.2. A second important contribution is the prediction of the UE buffer state at the horizon of d slots under the assumption of a relatively constant datarate of D t ∼ D t+d :

Qt+d = [ Qt + d • T tti • Dt - t+d-1 m=t+1 g m ] + (6.5)
Setting d = 8 for LTE is a wise choice since there are 8 subframes between the scheduling decision and the effective transmission by the UE. Thus, the uplink scheduler will send grants for slot t + 8 not based on current Qt but on expected buffer status Qt+8 , effectively reducing uplink access latency for packets arriving in buffer at slot t + 7.

Evaluation

Lab testbed

Base Station We evaluated the performances of the proposed enhanced-BSR algorithm on the lab testbed presented in section 2.2 with the configurations in Table 2.2. We have implemented our proposition in the OAI LTE BS as a MAC-layer module as presented in Fig. 6.2 (around 1.750 source lines of code). The MAC scheduler is left unmodified and selects UE to schedule as described in the Sec. 4.2.3 when an explicit SR signal is received.

Air interface In the following, the air interface is ideal (SNR> 12 dB) and does not vary in time. This strong hypothesis aims at assessing the intrinsic ability of our algorithm to correctly estimate UE transmission needs. The evaluation in more realistic conditions on an error-prone channel is left for further work, though anecdotal evidence on production networks tends to confirm the existence of this delayed estimation problem in real life.

Traffic considered

We consider 5 types of traffic to evaluate different properties of our solution. Each traffic are generated independently.

ICMP Ping traffic is used to evaluate baseline RAN RTT. One 64 bytes-length ping packet is generated every 50 ms.

An uplink traffic pattern is generated using UDP packets to evaluate the scheduling performance in terms of latency and jitter. The latency is measured for each packet thanks to the simultaneous captures at the UE and the BS side. Two traffic patterns are considered. A 1 Mbps CBR traffic with a deterministic inter-packet delay of 0.8 ms (100 bytes-length packets), thereby the transmission buffer is filled every TTI. A nondeterministic traffic where inter-packet delay is variable and random (delay applied with tc netem tool1 ). The average inter-packet delay is 0.8 ms (100 bytes-length packets) with an average datarate of 1 Mbps. The traffic is then very hard to predict for a linear model if the instantaneous datarate differs too much from the average. TCP BBR 30 MB-length transfer in downlink and uplink to show the impact of the uplink access method on throughput, delay and jitter experienced by users. It should be noted, the downlink TCP transfer generates an uplink flow of ACK with similar characteristics to an almost-CBR traffic when the TCP estimated bandwidth has converged.

Experiment

Results

Results discussed in this section are summarized in Table 6.1. Achieved ρ corresponds to the overall transport block utilization achieved over all uplink transmissions. It is calculated by taking the total volume scheduled g by the MAC scheduler and the total volume received d in the RLC buffer. The control signalling at the MAC layer and Signal Radio Bearer (SRB) are then not included in the value d. The targeted achieved ρ is 80% as justified in 6.2.1.2-"Decrease D".

Regarding the impact of Enhanced-BSR on the MAC layer performance, it is limited by the fact that even if one model is instantiated per connected terminal, updating the model at each TTI corresponds only to a few linear mathematical operations. The main cost comes from the search for the UG corresponding to the reception of a TB, an operation which is already partly performed by the MAC layer. The prediction that is generated at each round of scheduling is also a linear mathematical operation that is quick to perform. Finally, the memory needed for each Enhanced-BSR instance is limited to few variables to maintain. Figure 6.4 depicts the ECDF of a ping in optimal condition (no competition, no retransmissions, no variable channel capacity) for eBSR, BSR with a timer of 64 ms and one of 8 ms. The experiments show a clear improvement in terms of latency for 95% of the packet. The median RTT is reduced from 31 ms to 20 ms with a reduced jitter. Moreover, 10% of the packets experienced a RTT of less than 13.5 ms, the baseline minimum RTT ( §4.1.1).

At the reception of a SR, the model is initialized with an initial estimated datarate. Because of that, the estimated buffer length is not zero and the scheduler will allocate radio resources as if it receives a BSR with the SR. The counterpart of this is a poor estimation of the buffer length, leading to a ρ of only 31.6%. After the reception of a padding TB, the estimation is stopped. There are no obvious solution to set model initial values since it depends on the type of traffic that is not priorly known. The model converges with the firsts value of transport block utilization and BSR.

UDP uplink transfer with eBSR

Constant BitRate traffic That is the best case because this traffic pattern matches the modeling of the source datarate as being of constant bitrate. The difference between the baseline and with our proposition is made clear with the uplink traffic profiles presented in Figure 6.5. At the top (Fig. 6.5a), we found the characteristic traffic pattern studied in Sec. 4.2.1 with bursts of transmission after the reception of a BSR. At the bottom (Fig. 6.5b) the traffic pattern is "stepped" with a pace of 10 ms, corresponding to SR According to the results presented in Table 6.1, the latency is reduced (visible also on the traffic pattern) but more importantly, the jitter (difference between 10th and 90th percentile) is crushed from 50 to 10 ms thanks to the more regular transmission of packets with burst of same size in opposition to the baseline transmission. It complies with latency/jitter-sensitive applications needs with a limited impact of the transport block usage.

Moreover, the convergence of the model is visible on the traffic profile 6.5b where D converges towards D after few scheduling rounds. Before second 7.22, the traffic profile slope is lower than 1 Mbps which means an under-estimation of the model, corrected after the second 7.22 and the reception of a BSR. After that, the estimated datarate D is equal to the actual source datarate D = 1 Mbps with a good transport block usage around 80%, the model objective (see Appendix E.E.2a and E.E.2b respectively).

Bursty traffic That is the worst case for the prediction because of the non-deterministic nature of this traffic in contrast with the deterministic nature of the modeled source datarate. The latency and jitter are not as reduced as for the deterministic traffic with a poor transport block usage (<< 80 %) resulting in a fallback to "legacy" BSR estimation (see paragraph §6.2.1.2-"Fallback to legacy BSR"). The fallback to the legacy is a conservative policy to preserve TB utilization in sacrificing latency jitter.

TCP transfer and eBSR

We motivated in Chapter 5 the need to tackle latency and jitter of the uplink channel by TCP transmissions. Contrary to the use case presented in Chap. 5, we prefer to evaluate TCP performances with BBR CCA instead of Cubic because we think BBR will be adopted widely in the upcoming years. Also, even if BBR is the state-of-the-art of TCP CCA in terms of throughput (see Fig. 5.7), it does not perform optimally over cellular networks [START_REF] Atxutegi | TCP Performance over Current Cellular Access: A Comprehensive Analysis[END_REF], that constitutes an interesting challenge for our proposition and an argument for its adoption in cellular networks.

Uplink transfer

In the uplink, we do not observe a clear improvement either on latency or on throughput. In fact, TCP BBR has a good utilization of the uplink radio capacity of more than 95% of the available capacity. Our proposition has less impact on the uplink traffic pattern than the scheduler in itself when TCP reaches the radio capacity. The internet TCP congestion control machine maintains a number of bytes in the network contrary to the UDP traffic seen before. Thus, even when scheduler over-estimates the UE transmission needs, the TBS is filled with TCP packets present in the transmission buffer. It explains the almost full usage of the TBS measured with the achieved ρ. Also, our solution improves the first megabyte transmit duration on average in 450 ms instead of 570 ms thanks to a delayed exit of the slow-start phase.

Downlink transfer

The TCP downlink transfer experiment in the proposed setup is characterized by the presence of 2 feedback loops of estimation. The one of the server TCP stack and the one of the enhanced-BSR model as illustrated in Figure 6.6. The TCP estimation loop probes the RAN capacity using the ACK flow. Uplink channel latency and jitter disturb the ACK flow impacting the TCP bandwidth estimation. The enhanced-BSR model estimates the datarate of ACK flow using the TB usage and BSRs. When the TCP throughput raises up, the number of ACK increases that is detected by In the downlink, we observe with BBR a minor average throughput improvement from 28 Mbps to 32 Mbps but with a greater stability of the instantaneous throughput as depicted in figures 6.7a and 6.7c. Especially, with enhanced-BSR TCP BBR converges to the estimated network capacity after 3 seconds of transfer with very few deviations from this value contrary to the baseline where the instantaneous throughput is varying a lot during the transfer.

The most noticeable result concerns the RTT and the jitter, with a reduced mean RTT from 55 to 32 ms and a standard deviation of delay from 25 to 3 ms. The RTT is very stable after the BBR has converged at second 3 as plotted in Fig. 6.7d compared to Fig. 6.7b. The RTT reduction is for a part the result of a reduced uplink OWD (uplink ACK flow is similar to a CBR traffic) but for a bigger to the reduction the downlink queueing latency. Indeed, the average DL RLC buffer size was equal to 292.2 kB in the baseline but 186.6 kB (-36%) with eBSR. The uplink jitter reduction has a beneficial on the slow start phase of TCP that was identified as an issue in cellular networks (Chap. 5 with the network capacity hit in 198 ms instead of 259 ms, a difference of 2 RAN RTT). Also, the first megabyte is transmitted quickly in 282 ms instead of 340 ms, a reduction of 17%.

Results on TCP Cubic with hystart should be more important. Preliminary results give a median throughput on the first megabyte at 12 Mbps compared to the 7 Mbps found in Fig. 5.6. It is in accordance with observations concerning impact of slow-start early exit in [START_REF] Bruhn | Performance and Improvements of TCP CUBIC in Low-Delay Cellular Networks[END_REF].

Conclusion and Further studies

We have proposed a 3GPP-compliant modification in the base station's MAC scheduler to estimate UE's uplink transmission buffer status. The proposed algorithm takes into account the standardized SR and BSR signalling, but also the transport block utilization and grants, to perform a continuous estimation of UE's transmission needs.

In a lab testbed using an OpenAirInterface LTE BS we demonstrate the ability of the proposed solution to estimate and predict UE's transmission needs with as a result a reduction of uplink latency. The uplink traffic profile reveals to be less bursty, which benefits TCP downlink transmissions with better throughputs obtained and greater stability in the experienced RTT.

The main limitations reported by the experimental evaluation are:

-The difficulty to handle efficiently bursty non-deterministic traffics; -The convergence time of estimation D towards D impacting performance for short and sporadic traffic; -The dependence on uplink transmissions occurrence to update ρ; -The un-differentiation of dataflows with different packet generation characteristics in the uplink traffic mix.

We see 4 further studies for future validation of the proposition.

A more comprehensive evaluation with a more realistic air interface and user traffic should be conducted to fully validate the estimation algorithm, especially to study interactions with the endpoints and scheduling algorithm when users are in competition for radio resources with varying radio capacities.

A more comprehensive evaluation on a wider range of traffics, for instance with TCP Cubic (that continue to be widely common on the internet), video live-streaming (DASH protocol) and cloud gaming. TCP Cubic and cloud gaming especially should benefit from a more stable uplink latency.

The model when enabled could be the target of malicious users. It raises security considerations at this sensible point of the network. For instance, as the model continuously estimate the UE needs in transmission, a user could generate important traffic that overcome the cell's capacities. Such attack works for a scheduling algorithm that does not guarantee the fairness between users, with the consequence to disturb connections in cell area for the other users.

The TCP downlink transfer experiment revealed in paragraph 6.3.2.3 the interaction of the TCP feedback loop to sense the network capacities with the enhanced-BSR loop to sense the uplink application transmission needs. The increase of the transmission datarate in the downlink should as a result increase the acknowledgement datarate in the uplink that is estimated by the enhanced-BSR estimation model. We think that this aspect should be investigated as well as the contribution to the reduction of the bufferbloat because we observed during experiments a reduction of DL buffering with both BBR and Cubic.

Motivation

In the development of enhanced-BSR in Chapter 6, the uplink dynamic scheduler has not been modified. In this configuration, UG transmissions frequency is directly linked to SR transmission periodicity (see Fig. 6.5b). When we disable explicit SR, the continuous estimation of buffer status leads to a persistent scheduling of the UE every TTI resulting in a dreadful battery consumption.

We propose in this chapter to work on an uplink scheduling algorithm which tries to reconcile: 1) the latency jitter; 2) the radio resource utilization; 3) the energy consumption. Latency, achieved throughput and energy consumption constitute a fundamental trade-off in radio networks [START_REF] Soret | Fundamental tradeoffs among reliability, latency and throughput in cellular networks[END_REF]199,[START_REF] Gueguen | Dynamic tradeoff between energy and throughput in wireless 5G networks[END_REF]. In this work, the usage of enhanced-BSR and Fast-UL makes the uplink scheduling more similar to the downlink scheduling. A particular attention is also put on achieved throughput, experienced latency and inter-UE fairness to comply with BE context. The aim of the BS MAC scheduler is to assign to each user PRBs while respecting a certain number of constraints due to the air interface with an objective of maximizing radio resource usage. Grant-based method is preferred for BE users (without QoS constraints) in place of grant-free access. Grant-based access is characterized by the usage of UG to resolve the contention problem of the shared medium between emitting UEs. UG is the exclusive right given to a scheduled UE i to transmit over a PRB k on a TTI t with a given Modulation and Coding Schemes. Grant-free accesses are not considered in this work since they are not designed for BE traffics. The UG is the outcome of the BS MAC scheduling process as illustrated in Fig. 7.1. The optimization challenge of mapping UEs to RBs comes with a prohibitive complexity with an exhaustive search. Multiple strategies have been proposed in the literature to solve scheduling problem, especially for the downlink side [START_REF] Satheesh Balakrishnan | A review of MAC scheduling algorithms in LTE system[END_REF][START_REF] Capozzi | Downlink Packet Scheduling in LTE Cellular Networks: Key Design Issues and a Survey[END_REF] even if the uplink scheduling is more and more considered with the raise of uplink traffic and services [START_REF] Eirini | Uplink resource allocation in SC-FDMA wireless networks: A survey and taxonomy[END_REF][START_REF] Abu-Ali | Uplink scheduling in LTE and LTE-advanced: Tutorial, survey and evaluation framework[END_REF][START_REF] Al-Khatib | Channel-and bufferaware scheduling and resource allocation algorithm for LTE-A uplink[END_REF]. We distinguish two families of scheduling algorithm [START_REF] Moustafa | A hybrid downlink scheduling approach for multi-traffic classes in LTE wireless systems[END_REF] depicted in Fig. 7.2, those QoS-aware which take into account QCI for QoS requirements and those QoS-unaware mostly in use for BE. QoS-unaware scheduling strategies are of 3 families [START_REF] Khuram Ashfaq | Comparative analysis of scheduling algorithms for radio resource allocation in future communication networks[END_REF]. Round-Robin (RR) rules optimize fair sharing of radio resources between users. Best-CQI / Max-Rate (BCQI) or max-rate optimizes cell aggregated throughput even if it implies to starve some users to serve another. PF algorithm optimizes throughput while maintaining fair share of radio resources between users.

Radio resource allocation

A practical breakdown of the radio resource allocation problem is to consider two subproblems [START_REF] Piro | Two-level downlink scheduling for real-time multimedia services in LTE networks[END_REF]. The first one selects a set of UEs U that could transmit data in the TTI t. The next one performs the mapping between U and the set of RB K of the given TTI t. We call them respectively, Time-Domain (TD) and FD scheduling since TD selects UEs scheduled for a given time slot and FD selects time-frequency resource blocks to UEs scheduled.

Time-domain scheduling

Time-Domain scheduling is usually linked to SR access method (called in the following "explicit-SR") in grant-based access (see Figure 1.4). TD distributes connected UEs between TTIs. Reception of a SR usually causes the scheduling of the UE for the next TTI by the TD just like the reception of a HARQ NACK. The buffer-aware scheduler also takes into account the estimated buffer to select UEs. However, it has to be noted that TD has free rein to select users, even they do not have request access as soon as it is connected and active (not DRX inactive). Fast-UL are typically grants issued by the TD not in response to a SR (see Fig. 1.4). Fast-UL (or pre-scheduling) has been identified as an efficient solution to reduce uplink access latency [START_REF] Johansson | Congestion Control for 4G/5G Networks IETF 96[END_REF]. Figure 7.3 extracted from [START_REF] Zhang | LTE optimization engineering handbook[END_REF] shows the beneficial impact on latency of regular Fast-UL. The smaller is the Fast-UL periodicity, the lower are the latency and the jitter. The results for the pre-scheduling of 1 ms is similar to what we got in short SR experiment in §4.2.4.1.

Frequency-domain scheduling

FD scheduler is always closely linked to Link Adaptation (LA) stage of MAC layer which attributes MCS to an UG as shown in Fig. 7.1. The frequency domain scheduler presented and proposed in the following is channel-aware, i.e. the scheduling decision is also based on capacities achievable by each UE. The literature generally considers the scheduling problem as a problem of maximization of a given metric λ under a set of constraints. QoS-unaware schedulers usually choose a metric λ related to datarate, to select the user i that will get the PRB k in maximizing λ i,k . For instance, the fair-sharing of radio resources between terminals is a common constraint in BE context.

Energy efficiency considerations

The transmission in the uplink is battery consuming for the UE [START_REF] Kanti | Power consumption analysis, measurement, management, and issues: A state-of-the-art review of smartphone battery and energy usage[END_REF]. More than in downlink, uplink transmissions cost in terms of energy for the terminal since it must transmit with a defined power to optimize channel SNR. We use the energy model proposed by Lauridsen et al. in [199]-2. This energy model describes the energy consumption of device's LTE modem. Because we consider only uplink, the power consumption P ([Watt]) is defined by:

P =m idle • P idle + (7.1) m idle • {P con + m T x • m Rx • P Rx+T x + m T x • [P T x + P T xRF (P) + P T xBB (d)]}
where m x corresponds to the share of time spent in the x state; P idle the power associated to idle state when UE is connected but not active; P con the baseline energy consumes when modem is on; P Rx+T x is the power to send or receive a signal; P T x the baseline power for transmission module; P T xRF the energy consumed to transmit with a power P ([dBm]) and P T xBB the energy consumed to transmit at a datarate d (with d t = g t /T tti [Mbps]). Parameters P x used for simulation are extracted from [START_REF] Lauridsen | Empirical LTE smartphone power model with DRX operation for system level simulations[END_REF] and updated for 5G in [199]-6. Transmit power P depends on the channel conditions. P T xRF is constant for P ≲ 0 and increases exponentially for P > 0 dBm. P T xBB is almost constant for lower and higher MCS, then datarate at which we transmit does not have a big impact on the power consumption. See in Appendix A.3 the typical energy consumption of a device over time from power up to data transmission. The major limitation of this energy model is that it does not integrate the RACH procedure and the DRX mode for a TTI-level computation of energy. However, the scheduler does not operate during RACH and DRX sleep. Thus, we are able to estimate energy consumption at each TTI using the Equation 7.1.

Proposed uplink scheduling method using traffic prediction

The proposed algorithm consists mainly to select which users has to be scheduled in a given TTI t. The important idea of this proposition is to use the principle of pre- Sawtooth transmission pattern according to the channel capacity (related to a UE) and cell load (related to the RAN). The adaptation of Transmission Opportunity (TxOp) periodicity is presented in the next section §7.4.1. The radio resource utilization goes through a distribution of terminals over several time slots ( §7.4.2). Inter-UE fairness is ensured by the usage of a PF FD which will be discussed in §7.4.3. The notations are summarized in the table of list of symbols in Appendix.

T cycle Q t T active T cycle (a)

Adaptive periodicity of transmission opportunities

According to [199,[START_REF] Gueguen | Dynamic tradeoff between energy and throughput in wireless 5G networks[END_REF], the uplink transmission energy consumption is reduced by: (L1) Minimizing the number of users transmitting in the same TTI; (L2) For a given mean throughput, it is preferred to transmit on a short period of time but with a high instantaneous throughput; (L3) The energy cost to turn on the modem is fixed. Thus, to minimize energy the number of "turn on" instructions should be reduced. It is preferred to transmit during 3 TTIs in a row instead of one out of two during 6 TTIs.

Starting from the above constraints, we propose to use a "cycle" pattern for transmission as shown in Figure 7.4a. During T i,cycle -T i,act TTIs, no UGs are sent to UE i: its transmission buffer Q i is filling. After that, during T i,act one or multiple UGs are sent in accordance with (L3) to empty the buffer at the highest rate to minimize duty cycle T i,act T i,cycle (L2). Contrary to pre-scheduling where usually T act = 1 and T cycle fixed, we propose to update them at the beginning of every cycle as follows:

T i,act t = ⌈ Qi t+T sched +Ttx /d i t ⌉ (7.2)
That corresponds to the number of TTIs necessary to empty the predicted buffer if all the bandwidth is given to the UE i. Thus, the period of time during which the UE is active depends on buffer length and channel capacity for UE i. Thereafter, the T i,cycle update is given by:

T i,cycle t = (T i,act t • di • T tti ) + ( Di • T tti ) -Qi t Di • T tti (7.3) s.t. T min ≤⌈T i,cycle t ⌉ ≤ T max (7.4) T min =T harq (7.5)
T max =⌈T harq • f ( Ū act )⌉ (7.6) Equation 7.3 means the cycle length should be long enough to empty in T act TTIs the buffer generated during the cycle period T cycle and the remaining buffer from the previous cycle. It depends on estimated uplink datarate, channel capacity, and buffer size. T cycle is lower bounded by (7.5), the feedback loop delay for retransmissions and (7.6) to avoid too big transmission bursts. The choice to bound (7.5) to T harq is justified by the fact that if an HARQ retransmission, the new transmission will coincide with the retransmission. Ū act in (7.6) corresponds to the average number of UEs served by the BS and is linked to the cell load. Cell load ξ cell is typically defined as percentage of used PRBs vs available PRBs. f is a function to be determined, for instance a log-like function depending on the number of actively transmitting UEs in the cell.

A consequence of our proposal is the generation of a "sawtooth" pattern illustrated in Fig. 7.4b. According to cell load (if there is inter-UEs competition for access to radio resources) and channel capacity, the sawtooth pattern changes with the cycle length and the burst size. The best case occurs 1) when channel capacity permits the transmission of all the buffer in one TTI at a maximum datarate and 2) a low cell load with low concurrency with other UEs for access to resources. Conversely, the worst case occurs when there are more UEs in the cell and when the channel capacity for the UE is low.

Distribution of terminals over frames

Following (L1) constraints the algorithm tries to minimize the number of UEs active on a TTI. At the update of the couple (T i,act , T i,cycle ), we also update the beginning slot of the cycle with a shift of ∆t such as:

arg min ∆t { ∆t+t+T i,cycle τ =∆t+t+T i,cycle -T i,act U act τ }, ∆t ∈ [t; t + T i,cycle ] (7.7)
Where U act τ is the number of active users in the TTI τ . In other words, the objective is to find where to shift the beginning of the transmission cycle such that the average number of active users in a TTI is minimized. In addition, distributing active users over radio frame is beneficial to frequency-domain scheduling to achieve fairness between them while decreasing complexity and increasing resource usage.

Link adaptation and fairness with Proportional-Fair

We use PF as frequency-domain scheduling strategy because it has good radio resource usage performances, and it is fair by-design [52]-Table 2. At each scheduling round t, the algorithm selects for each PRBs k in K the UE i which maximizes the metric λ:

max i∈Uact λ i,k (7.8) with λ i,k = d i k (t) Di t (7.9) and Di t = (1 -β) Ri (t -1) + βd i t , β = 0.1 (7.10)
Where d i,k t is the achievable throughput for the UE i over PRBs k and Di t the mean achieved throughput including throughput already scheduled d i t in the current TTIs t. PF is channel-aware at PRB scale but not buffer-aware. Link adaptation and physical constraints are not studied in this work.

A back-pressure is applied by PF on the TD scheduler through the Qi value. In fact, for T i,act t=1 d i t < Di • T i,cycle (with d i (n) scheduled bytes in the TTIs t), it comes a rise of Qi , that will be taken into account to update active and cycle duration. d i t < d i,K t when the bandwidth is shared between multiple TTIs, and it is the result of FD scheduling.

The complexity of proposed scheduling method is linear in the number of users and PRBs. It is similar to PF solutions with a prediction in a short future that comes with extra complexity to plan over multiple TTIs. The computation overhead of (T act , T cycle ) is low and comes from overhead due to SRS-to-CQI and enhanced-BSR model processes.

Preliminary Evaluation

Simulation setup

A strong part of this work focus on the dynamic of radio resource allocation in uplink channel. The OAI-based lab experiment has a central role into understanding and assessment of proposed methods. However, we were confronted with the complexity of development process inside OAI code and instabilities related to it. For that reason, we set up a simulation platform to accelerate solution development on a simplified model of uplink radio allocation. The simulator, written in Python, is simplified to only simulated uplink radio resource allocation part with an emulated variable air interface. More details about the alloc_sim simulator are given in appendix F.7.6. Especially, the cellular network system model used is illustrated in appendix F.F.1 but it uses the design presented in Figure 1.5. The system model focuses on the uplink allocation with 4 elements:

1. The uplink traffic generator 2. The radio resource control plane 3. The uplink radio channel 4. The buffer-aware, channel-aware MAC scheduler Uplink traffic generator The uplink traffic generation is the same as for OAI testbed presented in the research method section 2.3. The simulator manipulates real IP packets with the linux iptable nfqueue1 (see details in Appendix F.7.6).

Radio resource control plane

The control-plane mainly consists in signals used for grant-based radio resource allocation, SR, BSR and UG. The grant-based allocation scheme is described in §1.1.3.2. We use the same implementation as in OAI and srsLTE studied in §4.2.3. We also implement pre-scheduling with fixed periodicity and SPS for comparison purposes. Unless otherwise stated, the configurations used are those presented in Table 2 Radio interface loss is set randomly at the transmission block level. At any error, the block is considered as faulty and induces a retransmission 8 ms later. The generally targeted Block Error Rate (BLER) for the selection of the MCS and the transmission power is 10%, the value we use in the simulations.

MAC scheduler MAC scheduler follows the model presented in §1.1.3.2 and in Fig. 1.5. Each TTI the uplink scheduler calculates the radio resource grid for incoming time slots. It is buffer-aware as it uses the estimated buffer size to generate grants. We implemented the OAI model of estimation ( §4.2.3) and enhanced-BSR. It is also channel-aware as it implements a link adaptation algorithm (linked to the channel capacity from commercial traces). We implemented 3 QoS-unaware algorithm (those described in §7.2.1), PF, RR and BCQI for comparison purposes. However in the next section, we will only consider the PF algorithm because it is the most common strategy to ensure both fairness and throughput performances in the cell. Comparisons with others QoS-unaware scheduling algorithms are let for a further work. In this section we provide a preliminary performance evaluation of the proposed uplink scheduling algorithm. We limit simulation to one cell with users randomly distributed in the cell. The packet arrival process for each user is 50 bytes/ms, corresponding to a deterministic CBR traffic at 400 kbps. BSR periodicity is set to 32 ms, SR periodicity to 10 ms and HARQ feedback loop to 8 ms. We compared our proposed TD scheduling algorithm to current explicit SR-based with the same PF FD scheduling. We vary the number of users from 1 to 25 users and the bandwidth from 1.4 to 20 MHz (6 RBs to 110). Simulation is performed for 5000 TTIs (=5 sec) and repeated 20 times. Figures 7.6 compare cell aggregated throughput Dcell , mean uplink delay T , and radio resource usage ξcell for different simulation parameters. Other overall results are summarized in Table 7.1.

Performance analysis

It is clear that the proposed FD algorithm outperforms the baseline in terms of latency and jitter in all experiments. It also increases aggregated cell throughput for wide bandwidth congested cases thanks to a better radio resource usages. Wide bandwidths are becoming very common with 5G RAN when uplink channel will see its datarate increase in the incoming years. In the same time, energy consumption remains equivalent between the 2 methods by reducing number of Transmission Opportunity (TxOp)s (L2), (L3) and the number of average active users in a TTIs (L1). Our proposition optimizes the trade-off between throughput, latency and energy for PF FD scheduling without modifications of the existing signalling or QoS management. In Figures 7.7 are depicted the uplink OWD evolution over 1.5 sec for 2 UEs for 10 MHz and 1.4 MHz. The evolution of CQI over time, retransmission events and uplink datarate are the same in all the cases. The first thing to observe is the reduction of latency thanks to enhanced-BSR prediction instead of BSR-32 ms estimation. For a wider bandwidth of 10 MHz (Fig. 7.7a and Fig. 7.7b), our algorithm achieves also a lower jitter while minimizing the number transmitting TTIs. Another good characteristic is the increase of experienced latency by UE 0 even if the bandwidth is enough wide to maintain a low latency when the capacity decreases. Such implicit signal of change in the radio environment is useful for applications to adapt datarate over time while with PF it was almost completely hidden. For Fig. 7.7c and Fig. 7.7d the bandwidth is only of 1.4 MHz. When the channel capacity decreases for UE 0 the latency increases quickly to a hundred milliseconds with PF and with our proposition, but the jitter remains contained. After TTI 1300, the UE 0 's channel quality recovers and the latency quickly decrease because the PF algorithm ensures a fair share of the throughput. During the recovery period, PF prioritizes UE 0 over UE 1 , implying a temporary decrease of achieved datarate and increase of latency and jitter for the UE 1 that could be the hint of inter-UE competition at the bottleneck for the endpoint.

Summary

This section presents some preliminary results on a research direction following Enhanced-BSR which is the uplink scheduler. We have proposed a QoS-unaware scheduling method which uses Enhanced-BSR traffic prediction and Fast-UL mechanism. The main idea is to issue UGs with a variable periodicity to minimize latency and jitter while maintaining radio resource usage, fairness and energy consumption. Simulation results show a clear improvement in terms of latency, radio resource usage and throughput compared to PF scheduling without any significant increase of energy consumption. There is still a lot of work to do, but preliminary results are promising for future developments. Further, it is important for a future publication to evaluate the proposed solution with state-of-the-art scheduling algorithms in a NS-3 simulation environment.

CONCLUSION

General conclusion

5G changes the current LTE paradigm to go towards low latency cellular networks. The low latency in cellular networks is often presented as a key enabler for emergent new internet services sensitive to latency such as cloud gaming, augmented reality, tactile internet. 5G-NR standardization integrates a number of radio transmission enhancements in order to reduce latency compared to the last generation of cellular network i.e. LTE. The RAN and end-to-end 5G toolbox, including a new flow management and QoS framework, is utilized and completed by an important research work to achieve the hard challenge of 1 ms latency in cellular networks. The main objective of this thesis was to reduce latency and jitter in the RAN.

In order to achieve this goal, we set up an experimental testbed using an OpenAirInterface base station and commercial devices with the objective to analyze RAN-related latencies. The radio channel conditions have been made perfect and stable in time to limit spurious HARQ retransmissions. The RAN RTT was measured with 2 passive measurement points and 1 active ICMP ping measurement. We have found limitations in measuring latency inside the BS where it is difficult to monitor latency of an individual packet inside the base station.

We then proposed a tool for fine-grain internal latency analysis inside the BS. This tool called LatSeq was designed to have a low impact on performance of the observed system. For that, we adopted a software architecture which ensures that latency-critical measurement part is made as fast as possible while the heavy analysis part is made offline. Data packets are observed during their journey inside the BS with measurement points that generate packet fingerprints. The packet fingerprint includes a microsecond-accurate timestamp, a position of the packet inside the base station and identifiers related to this packet. Position and identifiers are used to rebuild in offline the packet journey in the RAN. Packet journey is plotted in a waterfall format which is an easy and comprehensive tool for latency analysis.

After that, we performed a set of experiments to characterize latency inside the RAN for a non-QoS internet traffic (i.e. Best-Effort). Especially, we illustrated the baseline RAN RTT under different configurations, the impact of RTT on HTTP transfer, and we exhibited the well-known bufferbloat phenomena which is the existence of a persistent queue inside the network bottleneck. The latency due to the bufferbloat represents in general 2 or 3 orders of magnitude that of the RAN. From these first experiments, we found an important latency and jitter caused by the uplink-channel. We have completed these observations with trials in a commercial mobile network and related observations in the literature. We have explained this latency and important jitter under ideal transmission conditions by the grant-based radio resource allocation scheme and the RAN configurations. According to the RAN configuration concerning the scheduling strategy, the SR periodicity and the BSR periodicity, the median RAN RTT obtained goes from 19 ms to 31 ms and the jitter from 3 ms to 13 ms. We have pointed out the importance of the BS knowledge of the UE buffer size for the scheduling performance, latency and jitter.

We motivated the need to tackle uplink latency jitter due to the bad estimation of the transmission buffer length with the example of the TCP performance in mobile networks. Indeed, we showed that TCP throughput suffers from rapid variation of the RTT measured with acknowledgment feedback loop. Especially, it leads to spurious RTO triggering and misestimate of the network congestion status, resulting in a datarate lowered than the available radio channel capacity. The problem still exists for the state-of-the-art of congestion control (i.e. BBR) and for Quic, the next generation of transport protocol.

We have proposed Enhanced-BSR, a new MAC layer module, which redefines the manner the UE buffer size estimation is made by the BS. For that, we have used a model that continuously estimates the buffer size that is then used by the BS uplink dynamic scheduler to allocate radio resources according to a scheduling strategy. The algorithm proposed in the first proposed version models the UE as a queue filled by an estimated CBR data source and emptied with UG. The estimation is corrected at the reception of a BSR and the transport block utilization following an UG. We have demonstrated the ability of the model to efficiently reduces latency and jitter in 3GPP LTE and 5G RAN without having resort to QoS bearer or further RAN configurations. We have shown the benefit of our solution on TCP transmissions with an increased throughput and a controlled end-to-end RTT.

Finally, this thesis includes a number of technical contributions. We have developed and implemented a new measurement tool inside OAI base station which requires an effort of software design to achieve stringent time requirements and a good knowledge of the RAN protocol stack. LatSeq is intended to be integrated into the OAI open-source project. Also, LatSeq has been designed to be implemented in a wider range of project and thus, it has been the subject of a publication of the code in open-source. Enhanced-BSR was also developed and integrated in the OAI eNB. The code composed of 1750 source lines of code implement the linear model proposed. A patch of this code is available on request. Finally, we have developed a toy uplink radio channel to accelerate experimental development of radio resource allocation scheme and uplink dynamic scheduling algorithm. It is less representative than a NS-3 simulation, but the computational cost and the development time are clearly reduced for quick experiments.

Perspectives

Further latency studies We have explained in the Sec. 4.2 a root cause of the uplink channel jitter comes from the partial knowledge of the UE's transmission buffer status that has the base station's scheduler. We began to study the different aspects and consequences of the partial buffer knowledge on the scheduling efficiency §4.2.5 and the transmission pattern §4.2.4. However, this work is still limited and should be completed by a comprehensive mathematical, analytical study, and simulations. Indeed, the majority of work concerning the uplink scheduling utilize as assumption a perfect knowledge of the buffer at a given TTI, like in the downlink. This assumption is false for a long BSR timer for the default BE bearer, thus relation between the scheduling algorithm and the partial buffer knowledge should be investigated.

The chapter 5 motivated the need to tackle uplink-channel jitter with the example of the TCP acknowledgments flow for downlink transmissions. In the same time, the Quic protocol is replacing TCP in number of mobile applications. The question of Quic parameters for mobile networks remains open in the IETF working group discussions 2 3 . Especially, there are the open questions of ACK frequency [START_REF] Iyengar | QUIC Acknowledgement Frequency[END_REF] and return link for Quic in the mobile context. For instance, the study of latency jitter could bring arguments to the ACK frequency determination, with the trade-off between network capacity sensing and burst of transmission generation.

LatSeq We presented LatSeq in Chapter 3, a novel solution for internal latency measurement. LatSeq was implemented in the OAI eNB LTE stack for analyzing internal latency in the RAN ( §3.3.2). We chose the LTE stack because the 5G-NR was not stable at this moment in 2020 and 2021. OAI 5G-NR stack is gaining stability in 2022 and Lat-Seq should get interest to be implemented for the gNB 4 Especially if it could be done in collaboration with the OAI community. Challenges around this implementation into the 5G stack are the same as presented in Sec. 3.3.2, i.e. relevant points for packet journey rebuilding and latency analysis. It is an ongoing work 5 .

Research objectives behind LatSeq could be extended beyond the sole latency analysis on a testbed.

We see a first application to the high tail latency troubleshooting. Fig. 4.1 shows that for an extreme minority of packets, the RAN latency could be very important somewhere one magnitude above the median latency. The knowledge on the individual packet path inside the BS is a helpful tool for RAN troubleshooting. Indeed, it discriminates the source of the latency, BS or UE or backhaul, the protocol, etc. With the development of private 5G networks for enterprise 6 which need robustness and reliability, we hope to demonstrate the utility of LatSeq for network debuggability.

We envision a second application of LatSeq for latency monitoring of end-to-end Slicing with Service-level agreement (SLA). As said Roland Mestric, Nokia's head of product marketing 7 , "The SLAs really matter. These industries [...] run some critical business on [Slicing] services. That's where the slicing comes into play as a way to deliver this quality.", the end-to-end slicing is associated to a QoS contractualized with SLAs. SLAs are based on numerical KPIs for instance maximum latency or limited burst of transmission. In this task, LatSeq with an adapted sampling rate (to limit the number of log data generated) can report latency KPI for packet journeys for a given slice. We have to demonstrate the advantage of LatSeq over BS reported latency values in particular, compared to ground truth and verifiability. Furthermore, old architecture of BS used in 3G and LTE employs a monolithic system where all network functions of the BS are physically located at the bottom of a tower at the center of the cell, tower at the top of which the antennas are installed. A novel architecture initialized with LTE advanced and developed in 5G employs a decentralized, disaggregated architecture. The new gNB is composed of a Radio Unit (RU), a Distributed Unit (DU) and a Centralized Unit (CU). According to the split option in use, several network functions of CU are softwarized and centralized in a datacenter. C-RAN architecture [START_REF] Wu | Cloud radio access network (C-RAN): a primer[END_REF][START_REF] Salman | Cloud RAN: Basics, advances and challenges[END_REF] is very flexible to adapt radio access architecture to the characteristics of the cell (e.g. rural or dense area) and operator's network infrastructure. LatSeq tool should take into account this architecture evolution and break up with its current monolithic architecture. This decentralized architecture raises issues of synchronization and data transfer. The interest of LatSeq for latency monitoring in these complex, decentralized and software-based infrastructures is certain.

In summary, the future of LatSeq has its place within 5G, C-RAN and slicing.

Enhanced-BSR

The initial proposition of Enhanced-BSR in Chapter 6 uses a linear model of estimation for the traffic resource, which means the source is considered as CBR traffic with a uniform distribution of packet arrival. The model proves its efficiency for uplink CBR transfer, paced uplink TCP flow and TCP uplink ACK flows but, it is not well suited for short transmissions and bursty traffic patterns. In addition to that, the energy cost for the UE to be scheduled more regularly should be evaluated. However, according to the model of energy presented in §7. 3 [199], the energy impact for the UE of a more frequent scheduling is compensated by a transmission at a higher datarate when the UE is transmitting (i.e. the UE modem is turn on to transmit over a larger set of PRBs). The next step should explore the other estimation method to be adapted to a wider range of traffic patterns. This model of estimation has to use at least BSR sampling and TB utilization to provide BS MAC scheduler a continuous estimation of the UE transmission buffer length. There are already traffic pattern prediction algorithms proposed in the literature [START_REF] Li | Predictive pre-allocation for low-latency uplink access in industrial wireless networks[END_REF][START_REF] Feng | A Predictive Semi-Persistent Scheduling Scheme for Low-Latency Applications in LTE and NR Networks[END_REF][START_REF] Li | A Learning-Based Pre-Allocation Scheme for Low-Latency Access in Industrial Wireless Networks[END_REF][START_REF] Sun | Latency Performance Analysis of Predictive Resource Allocation[END_REF][START_REF] Ali | Fast uplink grant for machine type communications: Challenges and opportunities[END_REF][START_REF] Afrin | An adaptive buffer based semipersistent scheduling scheme for machine-to-machine communications over LTE[END_REF], [START_REF] Nuwan | Enabling mMTC and URLLC in 5G: Initial Access, Traffic Prediction, and User Availability[END_REF]-4,[129]-7.2.1. for the context of radio resource allocation in the IoT and machine-to-machine communication. Online learning of the traffic pattern in a given cell with a Machine Learning techniques (e.g. [START_REF] Zhang | LNTP: An end-to-end online prediction model for network traffic[END_REF] with Long Short-Term Memory (LSTM)) is an interesting approach for a more general context without knowing a priori the type of traffic to estimate in the cell. Also, in the experiment, the linear model of eBSR was unique for all users in the cell, with the same configurations and for all cells using the OAI eNB program without taking into account cell particularities (e.g. rural or urban context). It is clear that different models should be used in the same BS according to the expected traffic types associated to a bearer (it is hard to set up a specific eBSR model to a unique UE since the RNTI is likely to change in the time). Slicing and QoS identifier associated to it is a good indicator of the expected traffic pattern to adapt eBSR model and configurations in consequence.

So far, we have only talked about buffer size prediction with eBSR. The linear model we proposed also gives the estimated waiting time of the queue such as Ŵq = D × Q. This value could be used by a delay-based scheduler to minimize the latency of the last packet arrived into the buffer that is an efficient scheduling strategy to limit queueing delay (at the expense of aggregated cell throughput and inter-UE throughput fairness).

An important perspective of this research work was developed in the latter chapter, which is the development of a dynamic scheduling algorithm for the uplink channel. The objective of this thesis is to reduce latency in the RAN for BE traffic. The radio resource allocation scheme and strategy have a significant role on the RAN latency. The allocation strategy controlled by the scheduler have to take multiple trade-offs into consideration related to the radio transmissions. One of them is the trade-off between latency, throughput and energy efficiency. The scheduling algorithm we propose to develop preserves throughput and energy efficiency but have to decrease latency and jitter in the uplink. For that, it should benefit from estimations provided by the enhanced-BSR model, that is the key difference with the related work in the literature. Finally, the scheduler will have to ensure a certain throughput fairness between users in the context of BE enhanced Mobile BroadBand (eMBB). Milestones have been laid in the last chapter with encouraging preliminary solutions and results as a research perspective. arrives, the RRC procedure is simplified to a "wake up" RRC message directly to the known Access Management Function (AMF) machine of the network core. The initial access latency is then clearly reduced in 5G, that should benefit for short flow duration and better energy efficiency. The RRC control-plane functionality operates between the RRC located in the BS. RRC is responsible for handling the RAN-related control-plane procedures. The device can be in different states depending on the traffic activity. In LTE, the disconnected and connected states are defined, 5G defines a new state for connected but inactive device. LatSeq traces in Appendix D.1 illustrate fingerprints (highlighted in yellow) associated to HTTP packets 12, 13, 14 in the downlink. The packets of size 1472 bytes at the GTP layer are concatenated into a transport block of size 4584 bytes per TTI. Thus, the data packets are transmitted at the radio capacity of 36.7 Mbps. The downlink latency of the packet 5 is composed of 0.015 ms for the BS OS network stack, 0.55 ms in the RLC transmission buffer and 0.35 ms for the processing, when the downlink latency for the packet 14 is composed of 0.055 ms for the BS OS stack, 3.4 ms in the RLC transmission buffer and 0.2 ms for the PHY processing. Data packets are acknowledged and the connection is closed with packets 15-19. Acknowledgment packets are transmitted in the uplink in 46 ms, far more than the 6 ms for the data packets in the downlink. PDCP ciphering and deciphering, especially, in-sequence delivery could induce important latency and jitter.

RRC and Control plane

B Research Methods

D Uplink segment as a source of latency and jitter

Radio resource allocation schemes

Grant-based access is characterized by the usage of UG to resolve the problem of shared medium between emitting UEs. UE is the exclusive right given to a scheduled UE i to transmit over a RB k on a TTI t with a given Modulation and Coding Schemes. Grants are issued by the Base Station uplink scheduler to indicate radio resources allocated to a user in response to a scheduling request. SR and BSR are the 2 control signaling to request radio resources to MAC scheduler.

The aim of BS MAC scheduler is to assign to each user RBs while respecting a certain number of constraints due to the air interface with an objective of maximizing radio resource usage. We also implemented 2 grant-free methods which are SPS and Fast-UL. Those methods are regularly citing as promising solutions to address 5G latency challenge. As a reminder, the principle of Fast UL is to issue a grant even if the UE does not request it in prevision to a transmission needs. The SPS is the method especially used in Voice over LTE (VoLTE) and deterministic traffics to pre-allocate regularly resources. When SPS is set up, a set of resource block is reserved at a fix period for the UE.

alloc_sim testbed

The simulation setup is based on alloc_sim program (consisting in 2.500 SLOC written in python) which implements a set of UEs connected to a BS through an emulated variable air interface. At every simulation event (a new TTI) the program performs the following procedure: After that, new packet arrivals are added to UE transmission buffers. This amount of data is determined either by the configured traffic pattern as a simulation parameter or by the size of the real linux transmission buffer. The configured traffic pattern takes 3 parameters, the packet size, the inter-packet interval and the time range during which the traffic is generated.

When the radio context is updated with data to transmit and the channel properties, UEs send if necessary according to network configurations, scheduling request to the BS. BS collects these scheduling requests for the scheduling stage. In the same time, transmissions from previous grants are performed. Transmissions consist only on removing bytes from UE transmission buffer to add them into the base station reception buffer. As the time in RAN is discretized by slot, the transmission between UE and BS is considered as instantaneous in a TTI slot. At the reception, there is a probability to have HARQ errors that will trigger a retransmission. Along with user data bytes, it is transmitted in the TB, the BSR MAC CE. The code related to the process of SR and BSR signalling is inspired of what is made in OAI (see section 4.2.3).

Sequentially after the effective data transmission, is run the scheduling process to allocate radio resources in the incoming slots. At the core of the radio resource allocation process is the uplink dynamic scheduler. The real value of this work is to ease implementation of scheduling algorithm with standard interfaces in a friendly language for algorithm development (Python). We implemented 4 common best-effort scheduling algorithms, RR, PF, BCQI and Exponential/PF (EXP/PF). The stage of power control is limited to 2 modes, low power and high power according to CQI threshold. The outcomes of the uplink dynamic scheduler is a radio resource blocks map (rbmap) for the slot 8 ms in the future which indicates for each RB, the UE number allowed to transmit on it and the MCS to use (describing the number of bytes carried by the RB).

All along the simulation, packets are observed at their arrival in the transmission buffer (UE) and at their departure from the reception buffer (BS). The output of that is 2 network captures, one gives the packet timestamp at the entry of the RAN and one the timestamp at the output. 1-point capture is used to determine traffic pattern and instantaneous throughput at traffic and user level. The 2 captures are synchronized in time with the common system slot clock. 2-point captures are used to compute time spent by packets in the RAN, e.g. uplink latency. Thanks to capture, latency is known at both individual packet level and aggregated cell level, also by flow and by UE.

1. Layer 1 is reduced to a resource grid of block which can carries an amount of bytes according to MCS. In reality, the process to convert bytes to transport block to symbols is in-self, a field of study in latency minimization.

2. The air interface is emulated by 2 values, the CQI and the BLER. In reality the air interface is very much more complex than that with propagation, fading and interference phenomena. We choose to do not implement 3GPP models of propagation and fading loss [START_REF] Gpp | Evolved Universal Terrestrial Radio Access (E-UTRA); User Equipment (UE) radio transmission and reception[END_REF]. We prefer to use channel quality "experienced" by users in a real cell.

3. The 3GPP RRC control plane is totally absent of the simulation. UEs does not have to proceed to RACH procedure to connect to the cellular network or operate handover. It is known that such procedures are source of latency and control overhead but as the simulation aims to focus on radio resource allocation, we put apart control plane not directly related to it.

We designed a simulation program to simplify study work of the uplink radio resource allocation. This program makes easy the development of scheduling algorithm, as well as all mechanism, related to radio resource grid sharing. Ones could argue that the simplification is too important to be representative of real system. Firstly, the simulation testbed is a preliminary stage to experiments realized on OAI testbed. Also, it comes with the advantage of focusing on algorithms design avoiding long development time, by then, accelerating research and development. The objective of this thesis is to propose mechanisms for reducing latency in the RAN. In this perspective, we set up a software-defined RAN experimentation platform based on the OpenAirInterface project and designed a tool for fine-grained measurement of RAN internal latencies called LatSeq. We show that the round-time trip latency typically reaches 30 ms in LTE divided between processing, queueing, retransmission and medium acquisition delays. We also measure a significant jitter on the uplink that affects the RAN latency but also the performance of TCP traffics, which are very common on the Internet.

LatSeq highlights the role of the radio resource allocation mechanism in the jitter and latency of the uplink channel. We show how the partial knowledge of the transmission buffer length by the scheduling algorithm from Buffer Status Reports (BSRs) and Scheduling Requests (SRs) induces bursts of transmissions, significant jitter and under-utilization of the radio channel capacity.

We then propose a new mechanism for estimating the terminal transmission buffer length which is compatible with 3GPP LTE and 5G standards. We demonstrate that this method makes it possible to reduce the latency of the uplink channel and even more the jitter experienced by packet flows.

  Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Thesis Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dissertation outline and organization . . . . . . . . . . . . . . . . . . . . . . . . 1 Background 1.1 5G and Cellular Networks . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 0 . 2 -

 02 Figure 0.2 -Thesis chapters outline

Contents 1 . 1 8 1. 1 . 1 9 1. 1 . 2 12 1. 1 . 3

 118119121213 5G and Cellular Networks . . . . . . . . . . . . . . . . . . . . . Cellular network architecture . . . . . . . . . . . . . . . . . . . Radio Access Network . . . . . . . . . . . . . . . . . . . . . . . Radio resource allocation . . . . . . . . . . . . . . . . . . . . . 17

1. 2 27 1. 3 29 1. 3 . 1 32 1. 3 . 3

 227329313233 Internet Protocol Suite . . . . . . . . . . . . . . . . . . . . . . . 26 1.2.1 Transmission Control Protocol (TCP) . . . . . . . . . . . . . . 26 1.2.2 Congestion control . . . . . . . . . . . . . . . . . . . . . . . . . 26 1.2.3 Quic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Latency in Networks and Mitigation . . . . . . . . . . . . . . . Definition of latency . . . . . . . . . . . . . . . . . . . . . . . . 29 1.3.2 Bufferbloat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Latency Mitigation in 5G . . . . . . . . . . . . . . . . . . . . . 36

Figure 1 . 1 -

 11 Figure 1.1 -LTE and 5G user-plane networks architecture with a focus on the user-plane.

Figure 1 . 2 -

 12 Figure 1.2 -LTE and 5G protocol stack and 3GPP Technical Specifications.

Figure 1 . 3 -

 13 Figure 1.3 -The OFDMA uplink radio resource grid shared between 2 UEs (blue and orange). The radio resources allocated to physical control signalling are not represented.

Figure 1 . 4 -

 14 Figure 1.4 -Uplink radio resource allocation schemes. Explicit Scheduling Request is the by-default used allocation scheme.

Figure 1 . 5 -

 15 Figure 1.5 -Uplink grant-based dynamic scheduling

Figure 1 . 7 -

 17 Figure 1.7 -Acknowledgments for TCP congestion control.

Figure 1 . 8 -

 18 Figure 1.8 -End-to-End latency segments.

  .

9 .Figure 1 . 9 -

 919 Figure 1.9 -CDF of latency for a given connection and definitions

Figure 1 . 10 -

 110 Figure 1.10 -Packet delay exhibiting bufferbloat.

Figure 2 . 1 -

 21 Figure 2.1 -Degree of realism of the different research methods.

Figure 2 . 2 -

 22 Figure 2.2 -Mobile network core architecture.

  (a) Testbed elements. (b) With the Radio-Frequency (RF) shield for a radio environment isolation.

Figure 2 . 3 -

 23 Figure 2.3 -Photos of the OAI RAN testbed

Figure 2 . 4 -

 24 Figure 2.4 -The 2 passive capture points on the testbed pointed with red arrows.

52

 52

Figure 2 . 5 -

 25 Figure 2.5 -Network protocols and applications in use on the testbed.

Figure 2 . 6 - 2 -

 262 Figure 2.6 -2-points passive capture latency measurement.

Figure 2 . 7 -

 27 Figure 2.7 -TCP time-sequence plot reading.

Contents 3 . 1 89 3. 7

 31897 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 3.2 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.2.1 Internal latency . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.2.2 Packet fingerprint . . . . . . . . . . . . . . . . . . . . . . . . . 66 3.2.3 Packet journey . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 3.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 3.3.1 Real-time measurement of packet fingerprint . . . . . . . . . . 71 3.3.2 Measurement points in the LTE stack . . . . . . . . . . . . . . 76 3.3.3 Internal latency analysis . . . . . . . . . . . . . . . . . . . . . . 79 3.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 3.4.1 Unitary measurement point generation time . . . . . . . . . . . 85 3.4.2 Memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 3.4.3 In-running global performance . . . . . . . . . . . . . . . . . . 86 3.5 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 3.6 Showcasing LatSeq: TCP ACK packet bursting in uplink . . Contribution to open source community . . . . . . . . . . . . 91 3.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Figure 3 . 1 -

 31 Figure 3.1 -Internal latency cases. a) Linear; b) Segmentation; c) Concatenation; d) Retransmission

  (a) LatSeq fingerprint principle (b) LatSeq fingerprint format

Figure 3 . 2 -

 32 Figure 3.2 -LatSeq fingerprint.

Figure 3 . 3 -

 33 Figure 3.3 -LatSeq journeys cases. a) Linear; b) Segmentation; c) Concatenation; d) Retransmission

Figure 3 . 4 -

 34 Figure 3.4 -Matrix of internal latency. One axis for the time, one for the location and one for a filtering parameter, e.g. UE.

Figure 3 . 5 -

 35 Figure 3.5 -LatSeq architecture.

Figure 3 . 6 -

 36 Figure 3.6 -LatSeq fingerprint data structures.

Figure 3 . 7 -

 37 Figure 3.7 -Screen capture of the LatSeq fingerprints file.

Figure 3 . 8 -

 38 Figure 3.8 -LatSeq points in the RAN testbed.

7 Listing 3 . 5 -

 735 phy . i n . ant--phy . i n . p r o c l e n 1 phy . i n . proc--mac . harq . up l e n 6 6 : : ue0 . c b s e g 0 . fm281 mac . harq . up--mac . demux l e n 3 0 : r n t i 5 8 1 0 6 : ue0 . l c i d 3 mac . demux--r l c . rx . am l e n 3 0 : r n t i 5 8 1 0 6 : drb1 . l c i d 3 . r s n 1 3 8 . r s o 0 . r f i 2 r l c . rx . am--r l c . r e a s . am l e n 3 0 : r n t i 5 8 1 0 6 : drb1 . r s n 1 3 8 . r s o 0 . r f i 2 6 1 6 2 3 7 7 5 1 9 9 . 7 8 4 4 6 9 U r l c . r e a s . am--pdcp . rx l e n 8 6 : r n t i 5 8 1 0 6 : drb1 . r s n 1 3 8 . r s o 0 . psn0 . psn148 7 1 6 2 3 7 7 5 1 9 9 . 7 8 4 4 7 5 U pdcp . rx--gtp . out l e n 8 6 : r n t i 5 8 1 0 6 : drb1 . psn148 8 1 6 2 3 7 7 5 1 9 9 . 7 8 4 5 0 2 U gtp . out--i p . out l e n 8 4 : r n t i 5 8 1 0 6 : drb1 . psn148 . t e i d 1 .gsn149 . i p i d 0 x e 9 e Consecutive LatSeq fingerprint of a packet segment

  uint16_t i p i d = data_req_p->b u f f e r [ data_req_p->o f f s e t + 4 ] << 8 | data_req_p->b u f f e r [ data_req_p->o f f s e t + 5 ] ;

  mac . harq . up nack0 : ue0 : harq1 . fm281 . subfm7 2 mac . i n d b s r 8 . l e n 3 1 : ue0 : l c g i d r l c . r x b u f . am o c c 3 0 : r n t i 5 mac . s c h e d . down mcs28 . t b s 7 . nrb3 : r n t i 5 r l c . t x b u f . am o c c 8 6 : r n t i 5 mac . s c h e d . up mcs10 . t b s 6 3 . nrb3 : ue0 : fm283 . subfm4 10 1 6 2 3 5 1 9 9 . 8 0 4 4 6 5 I mac . i n d phr40 : ue0 : Listing 3.7 -Radio context capture with LatSeq "I" fingerprints

Listing 3 . 8 -

 38 , 1 6 0 4 0 8 6 5 2 5 . 3 3 2 0 5 4 , " pdcp . i p --pdcp . t x " ] , [ 3 6 , 1 6 0 4 0 8 6 5 2 5 . 3 3 2 0 5 5 , " pdcp . tx--r l c . t x . am" ] , [ 4 0 , 1 6 0 4 0 8 6 5 2 5 . 3 3 2 4 8 1 , " r l c . t x . am--r l c . s e g . am" ] , [ 4 1 , 1 6 0 4 0 8 6 5 2 5 . 3 3 2 4 8 2 , " r l c . s e g . am--mac . mux " ] , [ 4 2 , 1 6 0 4 0 8 6 5 2 5 . 3 3 2 4 8 3 , " mac . mux--mac . t x r e q " ] , [ 4 3 , 1 6 0 4 0 8 6 5 2 5 . 3 3 2 4 8 4 , " mac . t x r e q --mac . h a r q " ] , [ 4 4 , 1 6 0 4 0 8 6 5 2 5 . 3 3 2 5 0 9 , " mac . harq--phy . o u t . p r o c " ] , [ 4 5 , 1 6 0 4 0 8 6 5 2 5 . 3 3 2 7 9 1 , " phy . o u t . p r o c --phy . o u t . a n t " ] ] 13 } Rebuilding output of a journey in a JSON format

  . The cumulative volume of internal latency for journey and the contribution of the RLC layer in the downlink and in the uplink.

Figure 3 . 9 -

 39 Figure 3.9 -ECDF of total internal latency and latency in the RLC layer for Downlink and Uplink.

  (a) One packet in the downlink.(b) Two packets in the uplink.

Figure 3 . 10 -

 310 Figure 3.10 -Waterfall representation of packet journeys.

Figure 3 . 11 -

 311 Figure 3.11 -Time to generate a fingerprint according to the number of arguments.

Figure 3 . 12 -

 312 Figure 3.12 -Traffic profile of the LatSeq showcasing experiment. Blue: At the entry of BS in DL; Purple: At the UE; Orange: At the output of the BS in UL. The burst of transmission to debug is encircled

Figure 3 . 13 -

 313 Figure 3.13 -Waterfall of a burst of transmission in the uplink.

Figure 4 . 1 -

 41 Figure 4.1 -Ping RTT for different packet sizes in an empty RAN.

Figure 4 . 2 -

 42 Figure 4.2 -Packet journey on the uplink for a 4.500-bytes IP packet.

Chapter 4 -Figure 4 . 3 -

 443 Figure 4.3 -ICMP ping RTT in a loaded cell at 85 % of network capacity in the downlink compared to baseline (ms).

10 Figure 4 . 4 -

 1044 Figure 4.4 -Time components of the HTTP completion time.

  (a) TCP traffic profile. Green: DL RAN ingress; Purple: UE DL reception; Red: UL RAN egress. (b) Associated RTT.

Figure 4 . 5 -

 45 Figure 4.5 -Webpage HTTP transfer.

5 .Figure 4 . 6 -

 546 Figure 4.6 -Flent Real-time Response Time under Load experiment exhibiting bufferbloat effect.

  .

Figure 4 .

 4 7a shows the traffic profile at different measurement points and Figure4.7b shows the associated RTT. The RTT is in a range of 70 -100 ms. A finer analysis with traffic profiles and

  (a) Associated traffic profiles. Green: DL RAN ingress; Purple: UE DL reception; Red: UL RAN egress. (b) The corresponding RTT and bytes-in-flight. Blue: RTT over time; Red: Bytes-in-flight

Figure 4 . 7 -

 47 Figure 4.7 -Flent RRUL downlink experiment analysis for a selected TCP flow.

  (a) Associated traffic profiles. Magenta: UE UL egress; Brown: UL RAN egress; Ping: UE UL ingress. (b) The corresponding RTT and bytes-in-flight. Blue: RTT over time; Red: Bytes-in-flight

Figure 4 . 8 -

 48 Figure 4.8 -Flent RRUL uplink experiment analysis for a selected TCP flow.

Figure 4 . 9 -

 49 Figure 4.9 -Uplink experiment transfer scenario

Figures 4. 10

 10

  . The variation of RTT before, during, and after the generation of UDP uplink traffic is plotted in Fig.4.10a. Without the UDP traffic, the RTT varies from 20 to 40 ms as we already pointed out in Fig.4.1. When the UDP traffic is generated, the average RTT increases quickly to 50 ms with an

  (a) Ping RTT (ms) measurement over time (s) (b) Associated uplink traffic profile (c) Associated BSR (green) and SR (brown) signalling

Figure 4 . 10 -

 410 Figure 4.10 -Results of baseline uplink transmission at 1 Mbps experiment.

Figure 4 . 11 -

 411 Figure 4.11 -Commercial network experimental setup.

Figure 4 . 12 -

 412 Figure 4.12 -Evolution of the achieved throughput in the cell all along the day.

Figure 4 . 13 -

 413 Figure 4.13 -Uplink traffic profile associated to a transmission at 1 Mbps in empty commercial network.

Figure 4 . 14 -

 414 Figure 4.14 -Uplink traffic profile associated to a transmission at 1 Mbps in loaded commercial network.

  (a) Ping RTT measurement (in ms) (b) Associated uplink traffic profile (c) Inter-SR time (ms)

Figure 4 . 16 -

 416 Figure 4.16 -Experimental results with a Scheduling Request periodicity at 1 ms and an uplink transmission at 1 Mbps.

  (a) Ping RTT measurement (in ms) (b) Associated uplink traffic profile

Figure 4 . 17 -

 417 Figure 4.17 -Experimental results with a Buffer Status Reporting periodicity at 8 ms and an uplink transmission at 1 Mbps.

  (a) Ping RTT measurement (in ms) (b) Associated uplink traffic profile

Figure 4 . 18 -

 418 Figure 4.18 -Experimental results for a full bandwidth allocation to an explicit scheduling request and an uplink transmission at 1 Mbps.

Figure 4 .

 4 Figure 4.19 -ECDF of latency per RAN configuration. Default : SR=10 ms, BSR=64 ms, PF scheduler; Short SR : SR=1 ms; Short BSR : BSR=8 ms.

Figure 4 . 20 -

 420 Figure 4.20 -Buffer reported (black markers) with BSR (in bytes) versus actual RLC transmission buffer occupancy (blue line).

Figure 4 . 21 -

 421 Figure 4.21 -Traffic profile of uplink ACK flow extracted from [203].

Figure 4 . 22 -

 422 Figure 4.22 -Traffic profile of an uplink transmission reported by Johansson at the IETF 96 (2016) [208].

Figure 5 . 1 -

 51 Figure 5.1 -TCP Cubic instantaneous throughput in blue compared to the available LTE radio link capacity in red.

Figure 5 . 2 -

 52 Figure 5.2 -ACK packet generation (orange) and Data packet generation (blue) per 100 ms over time

Figure 5 . 3 -

 53 Figure 5.3 -Measured RTT by the TCP sender from ACKs over time.

50 Figure 5 . 4 -

 5054 Figure 5.4 -Zoom on TCP traffic profile for a typical transmission cycle. Green: At the entry of the BS (Downlink); Blue/Purple: at the UE (Downlink/Uplink); Orange/Red: At the output of the BS (Uplink).

Figure 5 .

 5 Figure 5.4 focuses on the traffic profiles corresponding to a RTT cycle. There are plotted the TCP traffic profile at the entry of the RAN in the downlink as it is transmitted by the TCP sender, the traffic profiles at the UE radio interface and traffic profiles at

Figure 5 . 5 -

 55 Figure 5.5 -Left : Packet RTT. Right : Packet RTT, smoothed RTT and RTT variation by the TCP stack for the typical transmission profile cycle in Fig. 5.4.

Figure 5 . 6 -

 56 Figure 5.6 -Achieved throughput for a TCP Cubic (Hystart enabled) transfer of 1 MB and 10 MB.

Figure 5 . 7 -

 57 Figure 5.7 -RAN capacity utilization.

Figure 6 . 1 -

 61 Figure 6.1 -RTT and uplink traffic profile of a TCP downlink transmission in a commercial network (see §4.2.2 and [4]); Left: RTT variation over time; Right: Uplink traffic profile. Dashed orange line for uplink traffic profile before the air interface; After the air interface at the BS with the solid line. Guessed SRs and BSRs signalling are indicating on the bottom of the plot.

Figure 6 . 2 -

 62 Figure 6.2 -Proposed uplink scheduling with enhanced-BSR algorithm.

Figure 6 . 3 -

 63 Figure 6.3 -Enhanced-BSR estimation model state machine.

Figure 6 . 4 -

 64 Figure 6.4 -ECDF RTT for BSR= 64 ms, BSR=8ms and Enhanced-BSR enabled.

  (a) With BSR= 64 ms (b) With Enhanced-BSR enabled

Figure 6 . 5 -

 65 Figure 6.5 -Enhanced-BSR results on uplink traffic profile for a 1 Mbps CBR transmission.

Figure 6 . 6 -

 66 Figure 6.6 -Double feedback loop with TCP and enhanced-BSR.

  (a) Average throughput with BSR = 64 ms (b) TCP RTT with BSR = 64 ms (c) Average throughput with Enhanced-BSR (d) TCP RTT with Enhanced-BSR enabled

Figure 6 . 7 -

 67 Figure 6.7 -TCP BBR downlink transfer results.

Figure 7 . 1 -

 71 Figure 7.1 -Uplink dynamic scheduling model including enhanced-BSR prediction.

Figure 7 . 2 -

 72 Figure 7.2 -Classification of scheduling strategies.

Figure 7 . 3 -

 73 Figure 7.3 -Empirical distribution function of uplink latency for different values of Fast-UL periodicity. (Extracted from [26])

Figure 7 . 4 -

 74 Figure 7.4 -Adaptive periodicity of transmission opportunities pattern

Figure 7 . 5 -

 75 Figure 7.5 -Example of MCS traces from a commercial network used for the air interface simulation.

Figure 7 . 6 -

 76 Figure 7.6 -Uplink scheduling results for variable number of users and for different bandwidths.

Figure 7 . 7 -

 77 Figure 7.7 -Uplink delay (ms) evolution over time according to scheduling algorithm (Proposition vs PF) and bandwidth.

  (a) without distribution (b) with distribution of UEs over frames

Figure 7 . 8 -

 78 Figure 7.8 -Uplink delay according to UEs distribution strategy.

Figure A. 2 -

 2 Figure A.2 -RRC state machine.

Figure B. 1 -

 1 Figure B.1 -Data Radio Bearer configurations.

Figure C. 2 -

 2 Figure C.2 -Assembly code of a LatSeq measurement point.

Figure C. 3 -

 3 Figure C.3 -Assembly code of a LatSeq data collector.

Figure C. 4 -

 4 Figure C.4 -LatSeq measurement point in the 3GPP stack.

Figure C. 5 -

 5 Figure C.5 -Radio context representation.

Figure C. 6 -

 6 Figure C.6 -CPU usage of LatSeq thread using Linux prof tool.

Figure D. 1 -

 1 Figure D.1 -HTTPS transfer related network captures and LatSeq traces.

  Figure E.2 -Convergence of the model i.e. D → D using the TB utilization ρ.

1. Update air interface capacity 2 .

 2 Generate new packet arrivals 3. Trigger SR 4. Perform TB transmission and apply losses on TB 5. Decode MAC CE BSR 6. Run scheduling algorithm 7. Perform retransmission 8. Generation the RB map and send UG

D

Titre:

  Réduction de la Latence et de la Gigue dans le Réseau d'Accès Radio 5G Mot clés : 5G, Latence, Gigue, Mesure, OpenAirInterface, Ordonnancement remontant Résumé : Les réseaux cellulaires présentent des latences importantes, en particulier au niveau du réseaux d'accès radio (RAN). La nouvelle génération de technologie cellulaire 5G, en plus d'augmenter les débits de transmission, doit offrir de faibles latences en connectivité mobile. L'objectif de cette thèse est de proposer des mécanismes de réduction de la latence dans le RAN. Dans cette perspective, nous avons mis en place une plateforme d'expérimentation software-defined RAN basée sur le projet OpenAirInterface et conçu un outil de mesure fine des latences internes au RAN appelé LatSeq. Nous montrons que la latence aller-retour atteint typiquement 30 ms en LTE répartie entre délais de traitement, de mis en file d'attente, de retransmissions et d'acquisition du medium. Nous avons aussi mesuré une gigue importante sur la voie remontante qui influe sur la latence du RAN mais aussi sur les performances des trafics TCP, très répandus sur Internet. LatSeq met en évidence le rôle du mécanisme d'allocation des ressources radio dans la gigue et la latence de la voie remontante. Nous montrons comment la connaissance imparfaite de la taille du buffer de transmission par l'algorithme d'ordonnancement à partir des rapports Buffer Status Reports (BSR) et des demandes d'accès (SR) induit une transmission par rafale, une gigue importante et une sousutilisation de la capacité du canal radio. Nous proposons alors un nouveau mécanisme d'estimation de la taille du buffer de transmission des terminaux compatible avec les standards 3GPP LTE et 5G, et démontrons que cette méthode permet de réduire la latence de la voie remontante et la gigue subie par les flux de paquets. Title: Reducing Latency and Jitter in the 5G Radio Access Network Keywords: 5G, Latency, Jitter, Measurements, OpenAirInterface, Uplink scheduler Abstract: Cellular networks present high latencies, especially in the Radio Acccess Network (RAN). The new generation of cellular technology 5G, in addition to increasing transmission throughput, must offer low latency in mobile connectivity.

  Mobile Network Core with a focus on user-plane. Radio bearers end in the core network. The architecture of the LTE EPC is depicted at the middle top on Fig.1.1. Control-plane and user-plane traffics are routed by the Serving Gateway (SGW), to the network functions and Packet Gateway (PGW) respectively. User plane traffic is then routed to the PGW that is the IP edge for EPC. PGW is a kind of proxy that allocates IP addresses to users and connect them to the PDN. The architecture of 5GC has been redesigned to be more flexible and more open with a "cloud-native" strategy. The principle is the same, User Plane Function (UPF) plays the role of PGW and ensures to connectivity of user traffic to Internet and Access and Mobility Management Function (AMF) plays the role of SGW in terms of mobility and access management. The new 5G core architecture is designed to manage end-to-end flows in the mobile network in accordance with the new QoS framework.

table : https://www.sharetechnote.com/html/ Handbook_LTE_QCI.html 6. 5G 5QI table : https://www.sharetechnote.com/html/5G/5G_5QI.html

  7.1.7.1-1 and [40]-Table-5.1.3.1-1,2, according to the channel quality. The variation of the modulation order and coding rate gives the spectral efficiency achievable for a symbol in bits per second per Hz, from 0.15 to 5.55 bit/s/Hz in LTE and 0.23 to 7.44 bit/s/Hz in 5G. Consequently, an LTE PRB could carry from 3 bytes 8 to 124 bytes of data. The bandwidth goes from 1.4 MHz to 20 MHz and 100 MHz in 5G with carrier-aggregation and MIMO rank, increasing proportionally the available radio resources and achievable throughput

  8. 15 • 10 3 × 12 × 0.15/1000/8 9. A graphical generation of an LTE radio resource grid in https://www.sqimway.com/lte_ resource_grid.html

Table 1 .

 1 consists of a set of Physical Uplink Control Channel (PUCCH) resources on which SR flag could be transmitted. The quantity of information

		Scheduling Request	Buffer Status Report
	Layer	PHY	MAC
	Channel	PUCCH	Uplink MAC CE
	Size	A part of an OFDM symbol 1 to 4 TB bytes
	Information carried Request access	RLC tx buffer size
	Granularity	1 bit	6 bits (LTE)
	Tx opportunity	every 1-80 subframes	at least every 1-2560 ms
		(typically 10)	(typically 32)

1 -SR and Buffer Status Report (BSR) comparison.

table ,

 , 's extended BSR table and 5G's 5-bits BSR table are plotted in Figure1.6b. The y-axis uses a log scale, showing that between 2 index in higher value, the accuracy of the reported buffer status decreases exponentially. A Maximum Transmission Unit (MTU)length packet is reported with the index 33 in LTE, 26 with the extended table and 17 with the 5-bits 5G's BSR table. 5G standard defines 2 BSR formats, one with an index coded on 5 bits (32 possible values) and one with 8 bits (256 possible values). The 5 bits table encodes buffer occupancy from 0 to 150 kB as in with LTE table with a reduced accuracy on higher values.

		1x10 6
				LTE
			LTE extended table
		5G 5-bits buffer size field
		100000
	Buffer size value (Bytes)	1000 10000					Ethernet MTU size
		100					Minimum IP MTU size
		10
		8	16	24	32	40	48	56
					Index
	(b) Distribution of reported value by BSR per index in LTE
	and 5G.
		Figure 1.6 -3GPP Buffer Status Reporting.

(a) BSR MAC control element format. LCG ID : Logical Channel Group Identifier; Buffer Size : BSR table index

LTE

  .2).

	18. Redhat	documentation	:	https://www.redhat.com/en/topics/edge-computing/
	what-is-multi-access-edge-computing

Table 2 .

 2

	Parameter	Value
	Base Station	OAI eNodeB (lte-softmodem)
	#UE	2
	Antenna format	SISO
	Air Interface quality	SNR > 12 dB
	E-UTRA Band	7
	Frequency	2.56 GHz (UL) / 2.68 GHz (DL)
	Radio frame structure	FDD
	Bandwidth	10 MHz
	#PRB	50
	Max. modulation	16-QAM (UL) / 64-QAM (DL)
	Max. achievable datarate 17 Mbps (UL) / 37.5 Mbps (DL)
	Radio slot duration	0.5 ms
	Radio subframe duration 1 ms
	HARQ max retx	4
	HARQ retx delay	8 ms
	Radio Bearer QoS	Best-Effort
	RLC mode	Acknowledged Mode
	RLC t-Reordering	32 ms
	DRX	Not configured
	Scheduling algorithm	Round-Robin / Proportional-Fair
	SR periodicity	10 ms
	BSR periodicity	64 ms

2 -OAI lab testbed configurations.

2.

2.3.1 Experimental testbed elements

  Alongside BS the RAN is consisting on 2 Commercial Off-The-Shelf (COTS) smartphones and a Software-Defined Radio (SDR) card Universal Software Radio Peripheral (USRP) B210 (Ettus). The photo in Figures 2.3 illustrates those elements. RAN configurations are summarized in Table 2.2.

Algorithm 1

 1 Rebuild packet journey

	Require: I	▷ List of input points
	Require: O	▷ List of output points
	procedure rebuild_journeys(f ingerprints)
	F ← f ingerprints	▷ List of fingerprints ordered by timestamp
	J ← dict()	▷ Dictionary of output journeys
	while F.head is not empty do	▷ For all fingerprints in the file
	while F.head not in I do	▷ Find a new possible journey start
	F.head ← F.head.next
	end while
	f ← F.head	▷ A new input fingerprint
	G ⊂ F	▷ Subset of F where the journey should be completed
	Initialize a new journey empty j
	J ← J ∪ recursive_build(G, f, j)
	F.head ← F.head.next	▷ Proceed to the next fingerprint
	end while
	return J
	end procedure
	procedure recursive_build(G, f, j)
	end for
	return J ′
	end if
	G.head ← G.head.next
	end if
	end while
	return empty	▷ Journey incomplete
	end procedure
	procedure match_m(g.gid, g.lid, f.gid, f.lid)	▷ Classify the fingerprint in the current journey
	if g.gid ∪ f.gid then
	return empty
	end if
	locals ← (g.lid ∪ f.lid)	▷ Set of local identifier in common between f and g
	for all l in locals do
	if g.lid[l] ̸ = f.lid[l] then	▷ On local identifier value is different, not in the journey
	return empty
	end if
	end for
	return locals
	end procedure

while G.head is not empty do if G.head.src = f.dst then ▷ Candidate for next journey m ← match_m(G.head.gid, f.gid, G.head.lid, f.lid) ▷ gid: set of global identifiers, lid: set of local identifiers if m is empty then ▷ data identifiers do not match continue end if S ⊂ G ▷ Subset of G where to find possible segmentation Find possible segmentation f ′ in S j ← j ∪ G.head ▷ Add the fingerprint to the journey ▷ We cannot remove G.head because of concatenation case if G.head in O then J ′ ← j for all h in f ′ do ▷ Rebuild all segmentation journeys J ′ ← J ′ ∪ recursive_build(S, f ′ , j)

Table 3 . 2

 32

	Experiment in low-load conditions	Unit	Baseline µ σ	LatSeq µ σ
	Fingerprint generation time with 1 argument ns	0	10.98 0.2
	Data collector memory usage	kB	0	450
	Linux load average	%	0.79 0.03 0.81 0.03
	Linux load average in loaded condition	%	1.18 0.065 1.25 0.075

-LatSeq impact on BS performance results. µ : mean, σ : standard deviation

Table 4 .

 4 On Radio Access Network Uplink Latency and Jitter : Measurements and

	Publication
	Analysis. Flavien Ronteix-Jacquet. IEEE ITC-33, September 2021, Va-
	lence / Virtual, France. Link to paper : https://hal.archives-ouvertes.
	fr/hal-03420681

1 -Summarized latency experiments.

Table 4 .

 4

		Value	Step	Value
	BS proc.	1 ms	SR align.	t SR (≈ 10 ms)
	TTI align.	t FA (< 1 ms)	SR tx	1 ms
	Data TTI	1 ms	BS sched.	3 ms
	UE proc.	1.5 ms	UG tx	1 ms
	HARQ retx	t HARQ × P BLER	UE proc.	3 ms
	Total downlink 3.5 + t FA + (ms) t HARQ × P BLER	Data TTI BS proc. HARQ retx t HARQ × P BLER 1 ms 1 ms
	(a) In the Downlink	Total uplink 10 + t SR +
			(ms)	t HARQ × P BLER
			(b) In the Uplink

2 -User-plane One-Way Delay (OWD) in LTE RAN.

7.2.1 Scheduling algorithm

	PRB utilization	N UEs	MAC Uplink Scheduler
	UE Scheduling Request (SR)	enhanced-BSR enhanced-BSR Enhanced-BSR	Time Domain (TD)	M UEs	Frequency Domain (FD)	K UEs	UL grants UEs-to-PRBs Allocation
	Buffer status
	reports (BSR)		HARQ manager	CSI manager	CQI	Adaptive Module and Coding (AMC)	Power Control (PC)
	HARQ feedback
						Outer Loop	Link Adaptation
						LA		(LA)
	SRS
	measurements

  Transmission pattern over time with cycle of T cycle length, with transmission ON for T active ms.

		Q
	Channel Capacity	(datarate achievable by UE)	latency + jitter ++ latency -jitter -	t	latency ++ jitter ++ latency + jitter (b)
				Cell load

  .2.

		28
		26
		24
		22
		20
	MCS index	10 12 14 16 18
		8
		6
		4
		2
		0
		0	250	500	750	1000	1250	1500	1750	2000	2250	2500	2750	3000	3250	3500	3750	4000	4250	4500	4750	5000
											Time (ms)

Table 7 .

 7 1 -Summarized uplink scheduling results. baseline PF / proposed for a simulation of 5 seconds.

	Configuration	Cell throughput (kbps)	Unused RBs (%)	Mean #user per TTI	Mean #TxOps per UE	Electric Charge (mAh)	Mean delay (ms)	Mean jitter (ms)
	1.4 MHz / 1 UE	393 / 397	21.1 / 44.0	0.27 / 0.25	545 / 494	1.37 / 1.29	32.0 / 16.1	8.2 / 3.1
	1.4 MHz / 5 UEs 1131 / 1149	0.3 / 0.0	4.22 / 3.69	1691 / 1477	1.13 / 1.14	444 / 424	247 / 240
	5 MHz / 5 UEs	1984 / 2065	32.0 / 25.5	1.0 / 0.7	403 / 296	2.28 / 1.92	29.16 / 10.82 9.03 / 6.77
	5 MHz / 25 UEs 5032 / 5073	0.39 / 0.0	18.87 / 15.86	1509 / 1258	0.96 / 0.96	422 / 492	238 / 275
	20 MHz / 25 UEs 10702 / 13111	33.0 / 14.8	4.58 / 2.88	362 / 232	2.54 / 2.89	27.0 / 16.38	9.30 / 13.17

  i bps Datarate generated at time t by user Di 3GPP . . . . . . . . . . . . . . . . . . . Third Generation Partnership Project 5GC . . . . . . . . . . . . . . . . . . . . 5G Core 5QI . . . . . . . . . . . . . . . . . . . . 5 QoS Identifier AAS . . . . . . . . . . . . . . . . . . . . Advanced Antenna System ACK . . . . . . . . . . . . . . . . . . . Acknowledgment ADB . . . . . . . . . . . . . . . . . . . Android Data Bridge AI . . . . . . . . . . . . . . . . . . . . . Artificial Intelligence AM . . . . . . . . . . . . . . . . . . . . Acknowledged Mode AMC . . . . . . . . . . . . . . . . . . . Adaptive Modulation and Coding AMF . . . . . . . . . . . . . . . . . . . Access and Mobility Management Function API . . . . . . . . . . . . . . . . . . . . Application Programming Interface AQM . . . . . . . . . . . . . . . . . . . Active Queue Management ARED . . . . . . . . . . . . . . . . . . Adaptive Random Early Detection ARQ . . . . . . . . . . . . . . . . . . . Automatic Repeat reQuest BBR . . . . . . . . . . . . . . . . . . . Bottleneck Bandwidth and RTT BCQI . . . . . . . . . . . . . . . . . . . Best-CQI / Max-Rate BDP . . . . . . . . . . . . . . . . . . . Bandwidth-Delay Product BE . . . . . . . . . . . . . . . . . . . . . Best Effort BIF . . . . . . . . . . . . . . . . . . . . Bytes-in-Flight BLER . . . . . . . . . . . . . . . . . . . Block Error Rate BS . . . . . . . . . . . . . . . . . . . . . Base Station BSR . . . . . . . . . . . . . . . . . . . . Buffer Status Report C-RAN . . . . . . . . . . . . . . . . . . Cloud-RAN 235 CB . . . . . . . . . . . . . . . . . . . . Code-Block CBG . . . . . . . . . . . . . . . . . . . Code-Block Group CBR . . . . . . . . . . . . . . . . . . . Constant BitRate CCA . . . . . . . . . . . . . . . . . . . Congestion Control Algorithm CDF . . . . . . . . . . . . . . . . . . . Cumulative Distribution Function CDN . . . . . . . . . . . . . . . . . . . Content Delivery Network CoDel . . . . . . . . . . . . . . . . . . . Controlled Delay COTS . . . . . . . . . . . . . . . . . . . Commercial Off-The-Shelf CP . . . . . . . . . . . . . . . . . . . . . Control-Plane CPU . . . . . . . . . . . . . . . . . . . Central Processing Unit CQI . . . . . . . . . . . . . . . . . . . . Channel Quality Indication CU . . . . . . . . . . . . . . . . . . . . Centralized Unit CWND . . . . . . . . . . . . . . . . . . Congestion Window DCE . . . . . . . . . . . . . . . . . . . Direct Code Execution DCI . . . . . . . . . . . . . . . . . . . . Downlink Control Information DRB . . . . . . . . . . . . . . . . . . . Data Radio Bearer DRX . . . . . . . . . . . . . . . . . . . Discontinuous Reception DU . . . . . . . . . . . . . . . . . . . . Distributed Unit E-UTRA . . . . . . . . . . . . . . . . . Evolved Universal Terrestrial Radio Access ECDF . . . . . . . . . . . . . . . . . . Empirical Cumulative Distribution Function ECN . . . . . . . . . . . . . . . . . . . Explicit Congestion Notification eMBB . . . . . . . . . . . . . . . . . . . enhanced Mobile BroadBand eNB . . . . . . . . . . . . . . . . . . . . Evolved NodeB EPC . . . . . . . . . . . . . . . . . . . . Evolved Packet Core EXP/PF . . . . . . . . . . . . . . . . . Exponential/PF Fast-UL . . . . . . . . . . . . . . . . . Fast Uplink Grant FD . . . . . . . . . . . . . . . . . . . . Frequency-Domain

	t	bps Estimated uplink datarate at time t LIST OF ACCRONYMS
	g	bytes Grant size
	L	bytes Number of bytes transmitted
	B	MHz Bandwidth
	ρ n	%	Transport block utilization
	ξ cell	%	Cell load
	P con	W	Power consumption when the UE is connected and in active state
	P idle	W	Power consumption associated to idle state when UE is connected
			but not active
	P Rx+T x	W	Power to send or receive a signal
	P T x	W	Power for transmission module
	P T xRF	W	Power consumption associated to RF transmission
	P T xBB	W	Power consumption to encode at a bitrate
			234

ShareTechnotes : http://www.sharetechnote.com/html/5G/

TR 21.916 : https://www.3gpp.org/release-16

TR 21.917 : https://www.3gpp.org/release-17

In Huawei, srsLTE, and OpenAirInterface network configurations respectively

In particuler, see IP Performance Measurement group (IPPM) IETF group[START_REF] Parvez | A survey on low latency towards 5G: RAN, core network and caching solutions[END_REF]

https://www.itu.int/rec/T-REC-Y.1540-201912-I/en

RITE project webpage : https://riteproject.eu/dctth/

www.amarisoft.com

6 3 4 . 7 7 0

5 5 U mac . demux--r l c . rx . am l e n 1 0 7 : r n t i 2 2 8 0 5 : drb1 . l c i d

. r s n 3 6 . r s o 0 26 3

. 7 7 0 2

7 U r l c . rx . am--r l c . r e a s . am l e n 1 0 7 : r n t i 2 2 8 0 5 : drb1 . r s n 3

. r s o 0 3 6 3 4 .

7 0 3 6 0 U r l c . r e a s . am--pdcp . rx l e n 1 6 3 : r n t i 2 2

0 5 : drb1 . r s n 3 6 . psn29Listing 3.1 -Consecutive LatSeq fingerprint of a packet segment An individual packet fingerprint in fact cannot be used as is to compute latency, two are necessaries to capture instant latency at a segment. The listing above shows 3 fingerprints from a capture. The fingerprint 1. indicates that the packet with RLC SN 36 has been measured at the bound of RLC and MAC layers at second 634.770255. The destination of the packet at this fingerprint is the rlc.rx.am point. The second fingerprint 2. source's point is rlc.rx.am. Because the DRB identifier and the RLC SN are in common in fingerprints 1. and 2., then they correspond to the same packet but at 2 different layers and at 2 different moments. The latency experienced by this packet at the rlc.rx.am is then of 2 µs. A similar analysis leads to conclude that the third fingerprint belongs to the same packet at the bound of the RLC and the PDCP layer with a delay of 103 µs introduced by the rlc.rx.am segment. Thus, common local identifiers and matching source/destination fields reflect the continuity of the packet's life across layers.

Algorithm python code in Github: https://github.com/Orange-OpenSource/LatSeq/blob/ master/tools/latseq_logs.py

http://www.gnuplot.info

x86 instruction table : https://www.agner.org/optimize/instruction_tables.pdf

https://gitlab.eurecom.fr/oai/openairinterface5g/-/wikis/T

https://gitlab.eurecom.fr/oai/openairinterface5g/-/wikis/T/enb

https://gitlab.eurecom.fr/oai/openairinterface5g/-/wikis/T/wireshark

"How to measure network latency" : https://kadiska.com/measure-network-latency/

There are 28 bytes of header overhead to remove from the MTU because of GTP/IP encapsulation in the RAN

https://www.techplayon.com/lte-control-plane-and-user-latency-calculation-fdd/

Live measurement campaign in Orange France network: https://webview.orange.com (visited in August 2022)

In the srsLTE implementation[START_REF] Gomez | srsRAN source code for BSR management[END_REF], the BS assigns by default 64 bytes instead of PRBs directly at

The BS in response sends a UG that corresponds to a TBS of 3 PRBs, corresponding to 68 bytes ∼ one packet (UG (1)).

3. The UE uses this opportunity to transmit BSR and completes the TB with some user data. The packet 1 is small enough to be transmitted entirely within the TB.The OWD experienced by the packet 1 is then about 13 ms.

Github.com/srsRAN : https://github.com/srsRAN/srsRAN

Fork from Linux

ACK Ratio or AR

Internet congestion control using the power metric: Keep the pipe just full, but no fuller, Leonard Kleinrock[START_REF] Kleinrock | Internet congestion control using the power metric: Keep the pipe just full, but no fuller[END_REF]

Source code of the TCP RTT update function in Linux TCP stack: tcp_ack_update_rtt(...) https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/net/ipv4/tcp_ input.c?h=v4.[START_REF] Parvez | A survey on low latency towards 5G: RAN, core network and caching solutions[END_REF]

https://datatracker.ietf.org/doc/html/rfc6298

http://sgros.blogspot.com/2012/02/calculating-tcp-rto.html

Netem wiki : https://wiki.linuxfoundation.org/networking/netem

https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/

IETF Quic working group on Github : https://github.com/quicwg

IETF Quic working group mailing list archive : https://mailarchive.ietf.org/arch/browse/ quic/

FDD . . . . . . . . . . . . . . . . . . . Frequency-Division Duplexing FEC . . . . . . . . . . . . . . . . . . . . Forward Error Correction FIFO . . . . . . . . . . . . . . . . . . . First-In First-Out FPGA . . . . . . . . . . . . . . . . . . Field Programmable Gate Arrays gNB . . . . . . . . . . . . . . . . . . . . gNodeB GTP . . . . . . . . . . . . . . . . . . . GPRS Tunneling Protocol HARQ . . . . . . . . . . . . . . . . . . Hybrid-ARQ HoL . . . . . . . . . . . . . . . . . . . . Head-of-Line HSS . . . . . . . . . . . . . . . . . . . . Home Subscriber Server HTTP . . . . . . . . . . . . . . . . . . HyperText Transfer Protocol ICMP . . . . . . . . . . . . . . . . . . . Internet Control Message Protocol IETF . . . . . . . . . . . . . . . . . . . Internet Engineering Task Force IP . . . . . . . . . . . . . . . . . . . . . Internet Protocol IPPM . . . . . . . . . . . . . . . . . . . IP Performance Measurement group IR . . . . . . . . . . . . . . . . . . . . . Incremental Redundancy ITU . . . . . . . . . . . . . . . . . . . . International Telecommunication Union JSON . . . . . . . . . . . . . . . . . . . JavaScript Object Notation KPI . . . . . . . . . . . . . . . . . . . . Key Performance Indicator L4S . . . . . . . . . . . . . . . . . . . . Low Latency, Low Loss, Scalable throughput LA . . . . . . . . . . . . . . . . . . . . . Link Adaptation LCID . . . . . . . . . . . . . . . . . . . Logical Channel IDentifier LCP . . . . . . . . . . . . . . . . . . . . Logical Channel prioritization LDPC . . . . . . . . . . . . . . . . . . . Low-Density Parity-Check Code LTE . . . . . . . . . . . . . . . . . . . . Long Term Evolution MAC . . . . . . . . . . . . . . . . . . . Medium Access Control MAC CE . . . . . . . . . . . . . . . . . MAC Control Element 237 MCS . . . . . . . . . . . . . . . . . . . Modulation and Coding Scheme MEC . . . . . . . . . . . . . . . . . . . Mobile Edge Computing MIMO . . . . . . . . . . . . . . . . . . Multiple-Input Multiple-Output MME . . . . . . . . . . . . . . . . . . . Mobile Management Entity mMTC . . . . . . . . . . . . . . . . . . massive Machine Type Communications MNC . . . . . . . . . . . . . . . . . . . Mobile Network Core MTU . . . . . . . . . . . . . . . . . . . Maximum Transmission Unit NG-RAN . . . . . . . . . . . . . . . . . Next Generation RAN NIC . . . . . . . . . . . . . . . . . . . . Network Interface Card NR . . . . . . . . . . . . . . . . . . . . New Radio NSA . . . . . . . . . . . . . . . . . . . . Non-StandAlone mode NTP . . . . . . . . . . . . . . . . . . . Network Time Protocol OAI . . . . . . . . . . . . . . . . . . . . OpenAirInterface OFDM . . . . . . . . . . . . . . . . . . Orthogonal Frequency-Division Multiple OFDMA . . . . . . . . . . . . . . . . . Orthogonal Frequency-Division Multiple Access OS . . . . . . . . . . . . . . . . . . . . . Operating System OSA . . . . . . . . . . . . . . . . . . . . OpenAirInterface Software Alliance OSI . . . . . . . . . . . . . . . . . . . . Open Systems Interconnection OWD . . . . . . . . . . . . . . . . . . . One-Way Delay PC . . . . . . . . . . . . . . . . . . . . . Power Control PDCCH . . . . . . . . . . . . . . . . . Physical Downlink Control Channel PDCP . . . . . . . . . . . . . . . . . . Packet Data Convergence Protocol PDN . . . . . . . . . . . . . . . . . . . Packet Data Network PDU . . . . . . . . . . . . . . . . . . . Protocol Data Unit PDV . . . . . . . . . . . . . . . . . . . Packet Delay Variation PF . . . . . . . . . . . . . . . . . . . . . Proportional-Fair PGW . . . . . . . . . . . . . . . . . . . Packet Gateway 238 PHR . . . . . . . . . . . . . . . . . . . Power Headroom PHY . . . . . . . . . . . . . . . . . . . Physical Layer PIE . . . . . . . . . . . . . . . . . . . . Proportional Integral controller Enhanced PRB . . . . . . . . . . . . . . . . . . . Physical Resource Block PUCCH . . . . . . . . . . . . . . . . . Physical Uplink Control Channel QAM . . . . . . . . . . . . . . . . . . . Quadrature-Amplitude QCI . . . . . . . . . . . . . . . . . . . . QoS Class Identifier QoS . . . . . . . . . . . . . . . . . . . . Quality of Service QPSK . . . . . . . . . . . . . . . . . . . Quadrature Phase Shift Keying QUIC . . . . . . . . . . . . . . . . . . . Quic RACH . . . . . . . . . . . . . . . . . . Random Access procedure Channel RAN . . . . . . . . . . . . . . . . . . . Radio Access Network RAT . . . . . . . . . . . . . . . . . . . Radio Access Technology RB . . . . . . . . . . . . . . . . . . . . Resource Block RF . . . . . . . . . . . . . . . . . . . . . Radio-Frequency RFC . . . . . . . . . . . . . . . . . . . . Request For Comments RLC . . . . . . . . . . . . . . . . . . . . Radio Link Control RNTI . . . . . . . . . . . . . . . . . . . Radio Network Temporary Identifier ROHC . . . . . . . . . . . . . . . . . . Robust Header Compression RR . . . . . . . . . . . . . . . . . . . . Round-Robin RRC . . . . . . . . . . . . . . . . . . . Radio Resource Control RRH . . . . . . . . . . . . . . . . . . . Remote Radio Head RRUL . . . . . . . . . . . . . . . . . . Realtime Response Under Load RTO . . . . . . . . . . . . . . . . . . . Retransmission TimeOut RTT . . . . . . . . . . . . . . . . . . . . Round-Time Trip SA . . . . . . . . . . . . . . . . . . . . . Standalone SC-FDMA . . . . . . . . . . . . . . . . Single-Carrier FDMA 239 SD-RAN . . . . . . . . . . . . . . . . . Software-Defined RAN SDAP . . . . . . . . . . . . . . . . . . . Service Data Adaptation Protocol SDR . . . . . . . . . . . . . . . . . . . . Software-Defined Radio SDU . . . . . . . . . . . . . . . . . . . . Service Data Unit SGW . . . . . . . . . . . . . . . . . . . Serving Gateway SISO . . . . . . . . . . . . . . . . . . . Single-Input Single-Output SLOC . . . . . . . . . . . . . . . . . . . Source Lines of Code SN . . . . . . . . . . . . . . . . . . . . . Sequence Number SNR . . . . . . . . . . . . . . . . . . . . Signal-to-Noise Ratio SPS . . . . . . . . . . . . . . . . . . . . Semi-Persistent Scheduling SR . . . . . . . . . . . . . . . . . . . . . Scheduling Request SRB . . . . . . . . . . . . . . . . . . . . Signal Radio Bearer SRS . . . . . . . . . . . . . . . . . . . . Sounding Reference Signal TB . . . . . . . . . . . . . . . . . . . . Transport Block TBS . . . . . . . . . . . . . . . . . . . . Transport Block Size TCP . . . . . . . . . . . . . . . . . . . Transmission Control Protocol TD . . . . . . . . . . . . . . . . . . . . Time-Domain TDD . . . . . . . . . . . . . . . . . . . Time-Division Duplexing TLS . . . . . . . . . . . . . . . . . . . . Transport Layer Security TM . . . . . . . . . . . . . . . . . . . . Transparent Mode TTI . . . . . . . . . . . . . . . . . . . . Transmission Time Interval TxOp . . . . . . . . . . . . . . . . . . . Transmission Opportunity UDP . . . . . . . . . . . . . . . . . . . User Datagram Protocol UE . . . . . . . . . . . . . . . . . . . . User Equipment UG . . . . . . . . . . . . . . . . . . . . Uplink Grant or grant of uplink capacity UL-SCH . . . . . . . . . . . . . . . . . Uplink Shared Channel UM . . . . . . . . . . . . . . . . . . . . Unacknowledged Mode UPF . . . . . . . . . . . . . . . . . . . . User Plane Function 240 URLLC . . . . . . . . . . . . . . . . . . Ultra-Reliable Low-Latency Communications USRP . . . . . . . . . . . . . . . . . . . Universal Software Radio Peripheral VoLTE . . . . . . . . . . . . . . . . . . Voice over LTE VR . . . . . . . . . . . . . . . . . . . . Virtual Reality

Code to filter UEs to be scheduled hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . dci_format = NFAPI_UL_DCI_FORMAT_0; hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . r n t i = r n t i ; hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . t r a n s m i s s i o n _ p o w e r = 6 0 0 0 ; hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . r e s o u r c e _ b l o c k _ s t a r t = UE_template_ptr->pre_first_nb_rb_ul ; hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . number_of_resource_block = r b _ t a b l e [ rb_table_index ] ; hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . mcs_1 = mcs ; hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . c y c l i c _ s h i f t _ 2 _ f o r _ d r m s = c s h i f t ; hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . f r e q u e n c y _ h o p p i n g _ e n a b l e d _ f l a g = 0 ; hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . new_data_indication_1 = n d i ; hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . t p c = t p c ; hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . c q i _ c s i _ r e q u e s t = c q i _ r e q ; hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . dl_assignment_index = UE_template_ptr->DAI_ul [ sched_subframeP ] ; hi_dci0_pdu->dci_pdu . dci_pdu_rel8 . harq_pid = harq_pid ; hi_dci0_req_body->s f n s f =

THE CAUSES OF LATENCY IN THE RADIO ACCESS NETWORKS

Abstract : Thanks to the experimental platform and comprehensive latency measurement tools that we have put in place, we experimentally evaluate the sources of latency in the Radio Access Network (RAN). We first present a set of experimental evaluations to explore latency in the OpenAirInterface (OAI) lab testbed for an error-free stable radiochannel condition. We found that the uplink channel is responsible for important latency and jitter. We investigate in the last section this issue in depth and demonstrate the role of the uplink radio resource allocation scheme based on SR, BSR and UG. We put this observation on the experimental testbed with a trial in an operational cellular network and literature. The experiment takes place in an Orange's cellular network the configuration in Table 4.3. Packets transfer are made between a client and a server but both are located in the same device (i.e. the UE), which creates a local breakout as shown in Fig. 4.11. Client and server generate UDP and/or TCP traffics according to the type of service for which we would observe experienced latency. The client is connected to the cellular network interface. The server application uses the Ethernet interface. Thus, network captures and logs for client and server shares the same clock with a perfect synchronization, easing the problem of end-to-end latency analysis. Also, we avoid the problem of uncontrolled internet delays by connecting the server at the nearest point of the core network. We were not able to verify the information and puts it as hypothesis, that latency introduced by the core network is negligible and it does not constitute a latency bottleneck. Thus, the end-to-end latency measurement is equivalent to the Radio Access Network (RAN) Round-Time Trip (RTT). 1. When the first packet arrives in the UE transmission buffer, a SR is triggered and sent at the next SR transmission opportunity.

Chapter 6

ENHANCED-BSR

Abstract : In LTE and 5G mobile networks, radio resources are dynamically allocated. In the uplink, the Base Station (BS) needs to know how much data are awaiting in the transmission buffer to be transmitted for each device to properly share the radio resources among them. In this chapter, using a simple scenario and measurements made on an operational network, we show that the BS works on systematically outdated information about the buffer size, which generates delay spikes. Thus, we propose to add a predictive estimation of the buffer size, called enhanced Buffer Status Report (eBSR). It is implemented in OpenAirInterface BS at the Medium Access Control (MAC) layer without changes in the scheduler, and it is compliant with LTE and 5G. We evaluated it on a lab testbed with various traffic profiles. Results show a clear improvement on uplink latency and jitter, together with a limited impact on radio resource usage efficiency. Firstly, the initial access procedure by which a device finds a cell to connect with, receive the necessary system information and request connection through random access is made ultra-lean.

TOWARDS AN UPLINK SCHEDULER COMBINING LATENCY, THROUGHPUT AND ENERGY EFFICIENCY

Abstract In this chapter, we envision a research perspective for Enhanced-BSR with the development of an uplink scheduling algorithm which takes advantage of Enhanced-BSR traffic prediction and Fast-UL to reduce latency, increase radio resource utilization in the cell while containing the energy consumption. We propose a Frequency-Domain (FD) scheduling algorithm which reveals good properties compared to the baseline Proportional-Fair (PF) scheduler in preliminary evaluations performed in a simulator.

APPENDICES A Background Initial Access

Secondly, RRC procedures have been shorten by reducing the number of steps [START_REF]Samsung. 4G-5G Interworking[END_REF]. The most notable evolution of RRC process in 5G is the introduction of a RRC "inactive" state between the Long Term Evolution (LTE)'s RRC "idle" and RRC "connected" states. In this state, no data transfer are possible but the RRC context and the connection to core network are kept established. In consequence of that, when a new data transfer The current consumption of the device is really depends on the connection state and the transmission power.

Energy consumption

Internet Transport Protocols

Client (UE) Server

/ * S e l e c t UE t o be s c h e d u l e d i n t h e n e x t s l o t * / // D e c i d e s i f t h e UE i s t o be s c h e d u l e d i n t h i s s l o t

. * / // r e t u r n a pre-a l l o c a t e d r e s o u r c e g r i d w i t h a number o f pre-a l l o c a t e d RBs

(2) The eBSR dynamic illustrated in Appendix E.1 shows the update of the estimated datarate and buffer size according to triggering events.

-(1) Reception of an SR, initialize the estimation model with a value of estimated datarate -(2) Reception of a transport block with padding, reduction of the estimated datarate (indicated with "DATA (0.5)", 50% of utilization) -( 3) Another reception of a transport block with padding -(4) Probing for extra data when a BSR is not received for a too long time -(5) Reception of a transport block with padding, rollback to the previous estimated datarate - (6) The UG is not sufficient to empty the estimated buffer size - (7) At the reception of a non-zero BSR, the estimated buffer size is updated as well as the estimated datarate - [START_REF] Sandvine | Global Internet Phenomena Report[END_REF] The model detects the end of the transmission after the reception of padding BSR and low TB utilization.

F Alloc_sim

Base Station

Active UEs, Queue length, ...

Control

Uplink Scheduler

Link Adaptation

UE i

Channel model

The uplink interface

In the uplink, BS receives packets and acknowledges them. The BS is also in charge of scheduling and issues grants for UEs. The UE generates traffic to be transmitted and send control signalling to request access. The attachment procedure of the UE is not implemented. The UE becomes connected as they send their first SR. The layer 2 of UE and BS, composed of MAC, RLC and PDCP layer in the 3GPP protocol stack is simplified by solely one transmission and one reception queue to store packets. Queues correspond to RLC queues in the 3GPP model. Packet segmentation is enabled when a TBS is not large enough to transmit entirely a data packet.

A great advantage of this simplification is to put apart undefined interactions between resource allocation and complex control plane of 3GPP model. These interactions are assessed in the more realistic OAI testbed.

The limitation it can be found of this model is to not implement control signalling and control mechanism specific to layers, e.g. in-sequence delivery and ARQ at RLC layer or The energy consumption by UE is calculated at each slot according to the Lauridsen energy model presented in the paragraph 7.3. The simulation program can run on itself for uplink allocation simulation experiments. We also design it to be integrated in a linux environment to interact with real IP traffics as presented in Figure F.2. Such emulation mode is interesting to experiment the behaviour of application traffic flow carried by an uplink RAN. To do that, alloc_sim is paced to write every millisecond on stdin how many bytes are correctly transmitted by the RAN and to read on stdout the number of bytes awaiting to be transmitted at UE. The nfqsched programs make the interface between alloc_sim the linux iptable NFQUEUE. nfqueue 8 for netfilter queue is an iptable target which delegate the decision on packets to a user-space software i.e. nfqsched. With the appropriate linux iptable rules, the packets are stored in a buffer waiting to be forwarded to OUTPUT chain. In user-space, the program that uses the libnetfilter_queue connect to the NFQUEUE and get the messages from kernel and in return issue verdict on the packet. The emulation reproduces the traffic pattern and specific latency of uplink channel on the real IP traffic. This setup looks like the Direct Code Execution (DCE) of ns-3 but in a simpler and lighter way or Mahimahi emulation but with a real uplink system emulated.

Nfqueue

Summary

We are aware that the model proposed for this simulation is a bit far from the reality real cellular network. Especially, 8. https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/

LIST OF SYMBOLS