Lamia Gaied

Marc Lippert

Laurent Keirsbulck

Maxence Bigerelle

Emna Berrich

Fethi Aloui

Berrich Emna

Betouche

Influence of Surface Irregularities on Hydrodynamic Instabilities in Couette-Taylor Flow

Keywords: Taylor-Couette Flow, Surface Roughness, Rectangular groove, Hydrodynamic instabilities

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

ABSTRACT

In this work, we propose an experimental study on the hydrodynamic instabilities due to the roughness of the wall in a Couette-Taylor flow. The study was carried out without an axial flow. In our experiments, the inner cylinder rotates at a certain angular velocity and the outer cylinder is maintained fixed.

The main objective of the study is to highlight the geometric parameters effect on the flow of the flow. We mainly focused on the effect of roughness of the wall on the appearance of instabilities in the studied configuration (delay or advance in its appearance). For this purpose, a qualitative study was performed using Kalliroscope particles to visualize the instabilities. Then, we applied polarography to different types of roughness as a quantitative study. Hence the possibility to characterize the flow regimes of the first instabilities (TVF, WVF, MWVF, and TN).

Our experience has shown that surface irregularities, actually, have an effect on the appearance of the first instabilities depending on the size and nature of the roughness. In fact, the surface roughness not only increases the friction on the wall but it also greatly influences the transport of the momentum and mass in a given flow. The flow, therefore, undergoes more friction when the inner cylinder (in rotation) has a rough surface. This friction slows the velocity of the fluid particles; which delays the onset of instability. The movement of the particles will therefore, be dampened by the irregularities at the level of the rough surface. Moreover, the results also proved that as soon as the Taylor cells are formed, the roughness is no longer a particle motion damper, but on the contrary, it promotes the continuation of the disturbance of the flow. The flow then becomes less slow down in the hollow of the surface irregularities; which leads to less friction. For higher Taylor numbers, the study showed a faster transition from one regime to another. The study of the flow of a fluid around a solid body with a rough or structured surface occupies a considerable interest as it is encountered in the verification of reliability and the performance of rotating machines (turbines or hydrodynamic bearings). The design and modeling of these machines with a high degree of performance is one of the associated applications.

R i

This configuration is analogous to the configuration known as Couette Taylor flow, a flow between two concentric cylinders: the outer cylinder is fixed and the inner cylinder is rotatable. An axial flow is also taken into account so the flow is called Couette Taylor-Poiseuille flow.

Many studies have been done on the thermal part (heat transfer) for configurations where the inner cylinder is smooth. In the case where the cylinders of the system have roughness, some scientists are focused their studies mainly on the heat transfer.

For the specific problem of rotating machines (rotors) with non-smooth surfaces, the study of phenomena in the gap between the two cylinders becomes a major problem to optimize their performance.

A better understanding of the characteristics of the flow field in this confined gap still requires specific deep studies. This is necessary in order to improve their design and develop more efficient models. Since Couette's studies [START_REF] Couette | Etude sur le frottement des liquids[END_REF], Couette-Taylor flow has influenced many scientists, experimentally, numerically and theoretically. Up to now, several experimental, theoretical and recent numerical studies have been dedicated to this problem, and particularly to the Taylor vortices [2][3][4][START_REF] Rehimi | Caractérisation expérimentale des structures tourbillonnaires derrière un cylindre en milieu confiné par PIV et par polarographie[END_REF][START_REF] Abassi | Investigations expérimentales et modélisations numériques par la méthode de Lattice Boltzmann (LBM) pour l'étude des transferts dans les écoulements tourbillonnaires[END_REF].

Regarding the thermal part, most studies reported the case of smooth cylinders. Only a few studies include references to an axially grooved cylinder.

Among the experimental studies, mention is made of Cadot et al. [START_REF] Cadot | Energy injection in closed turbulent flows: Stirring Probe 7 Probe 8 through boundary layers versus inertial stirring[END_REF] and Van den Berg et al. [START_REF] Van Den | Smooth and rough boundaries in turbulent Taylor-Couette flow[END_REF] who are interested in the study of Taylor quilt system having ribs in the form of square or triangular regularly distributed on the surface of cylinders, at large Reynolds numbers (Re i =3×10 5 ).

In these references [START_REF] Cadot | Energy injection in closed turbulent flows: Stirring Probe 7 Probe 8 through boundary layers versus inertial stirring[END_REF][START_REF] Van Den | Smooth and rough boundaries in turbulent Taylor-Couette flow[END_REF], the most interesting conclusion is that in the case where the system consists of two rough cylinders, the drag coefficient is independent of the Reynolds number.

The authors noted that in the case of smooth cylinders, the total dissipation is thus dominated by the contribution of the boundary layers, whereas it is dominated by the contribution of mass transfer for the case of the striated cylinders.

Numerically, Tsukahara et al. [START_REF] Tsukahara | Dns study of the turbulent Taylor-vortex flow on a ribbed inner cylinder[END_REF] studied the influence of ribs on the transport phenomenon and flow characteristics for different Reynolds number up to 3.2 × 103. They concluded that the roughness causes the increase in compressive strength that is attenuated in the presence of a strong adverse pressure gradient behind each rib.

Sodjavi, et al., 2018 illustrated some photographs of the fow at dfferent high Reynolds numbers and surface roughness (smooth cylinder and G12, G6 and G24 grooved cylinders configurations). The flow structures were viusualized using microscopic iriodin particles. They concluded that for fully developed turbulent flows G12 and G24 fows have similar structural behaviour, strictly different from that of G6. They shows that the use of grooved inner cylinder increases the friction coefficient and that the groove contribution rate is a function of λ/k. Within the Reynolds number range of their study (2 104 to 13 104), the friction coefficient ratio between the rough and the smooth cylinders is approximately constant. These studies cited above were carried out with rough structured elements that are arranged to obstruct the average flow. The surface roughness is used either by design requirement or as heat transfer promoter. They are used to be more effective in producing friction resistance than in the case of smooth surfaces.

In order to reduce the frictional drag of the skin, the rough elements aligned with the average flow constitute another category of surface roughness. In this case, the thickness of the boundary layer becomes less than the approximate height of the roughness as well as the resistance to flow becomes dominant. In this context, one can quote the studies of Hall and Joseph [START_REF] Hall | Rotating cylinder drag balance with application to riblets[END_REF], Moradi and Floryan [START_REF] Moradi | Flows in annuli with longitudinal grooves[END_REF], Greidanus et al. [START_REF] Greidanus | Turbulent Taylor-Couette flow over riblets: drag reduction and the effect of bulk fluid rotation[END_REF] and Zhu et al. [START_REF] Zhu | Direct numerical simulation of Taylor-Couette flow with grooved walls: torque scaling and flow structure[END_REF].

Understanding the mechanisms associated with increasing friction drag in a Taylor quilt flow in the case where the cylinder is axially grooved requires a thorough understanding of the Taylor-Couette flow.

In this work, we propose an experimental study on the hydrodynamic instabilities due to the roughness of the wall in a Couette-Taylor flow. The study was carried out with and without axial flow.

The main objective of the study is to highlight the geometric parameters effect on the flow of the flow. We mainly focused on the effect of roughness of the wall on the appearance of instabilities in the studied configuration. For this purpose, a qualitative study was performed using Kalliroscope particles to visualize the instabilities. Then, we applied polarography to different types of roughness as a quantitative study. Hence the possibility to characterize the flow regimes of the first instabilities (TVF, WVF, MWVF, and TN).

II. EXPERIMENTAL SETUP

A Photograph of the experimental apparatus for the study of the Taylor-Couette flow is given in Fig. 1. It consists of two vertical coaxial cylinders made of transparent Plexiglas that create a gap occupied by a fluid. The inner cylinder is driven by a α-step motor with a variable speed. The radii of inner and outer cylinders are ri = 88.6mm and ro =100mm, respectively. Thus, the gap d =R o -R i = 11.5mm and the radius ratio is η = R i /R o =0.886. The length of the fluid between the two cylinders h = 450mm, which gives an axial aspect ratio of Г=H/d=39.47.

The inner-cylinder grooved configuration is presented in Figure 1 As can be seen, the grooved cylinders are equipped with 36 identical axial rectangular cavities which are arranged periodically. They are named "Gn" where n is the number of grooves. Our configuration corresponds to Gn36. The ratio of the groove width w = 12.7 mm to its depth k = 3 mm is equal to w/k = 0.9. The ratio of the groove separation distance λ = 2 Π r i /n = 15.47 to the groove depth k is generally used as the key parameter to distinguish the different wall roughness configurations [START_REF] Motozawa | Experimental investigations on frictional resistance and velocity distribution of rough wall with regularly distributed triangular ribs[END_REF][START_REF] Tsukahara | DNS study of the turbulent Taylor-Vortex on a ribbed inner cylinder[END_REF]. In our configuration, this ratio λ/k correspond approximately to 5.15 for G36.

The relevant dimensionless parameter that characterizes the flow dynamic is Taylor number. It describes the relation between viscous dissipation and centrifugal force. It is defined as:

. .

i i R d d Ta R ν Ω =
The main objective of this study is to evaluate the action of axial grooves on the behavior of Taylor-Couette flow. Thus, the grooved inner cylinder configuration is compared to the case of a Taylor Couette flow with a smooth surface. The outer cylinder has a smooth surface.

II.1. Visualization of the flow by kalliroscope particles

The Kalliroscope allows a qualitative description of the flow. It is a suspension of reflective particles. These particles are anisotropic, of dimensions (30 x 6 x0.07) and density of 1.62 g/cm3. With a high index of refraction (n = 1.85), they provide good visibility of the flow even at a low proportion.

II.2. Polarography technique

Polarography technique is an electrodiffusional method that focuses on the relationships between two conductive systems (electronic: electrodes and ionic: solution) during charge transfer composed of one or more electrons (oxidationreduction reaction). The polarography is based on the plots of Limiting Diffusion current I delivered by the probe while it is polarized by a direct voltage (between 0.5 and 0.7V) according to a given polarogramme, established function of the fluid aqueous solution characterizations and the flow regimes. The electrochemical solution that was used in our study is the ferriferrocyanide of potassium with a concentration of 25 mol/m 3 . The choice of the indifferent inert electrolyte was done to especially limit noised diffusional limiting currents. As inert electrolyte, we used potassium sulfate K 2 SO 4 at a concentration of 200 mol/m 3 .

II.3. Wall shear stress determination

The simplifying assumptions adopted for the resolution of convection-diffusion equation generate that the Lévêque (1928) wall shear stress solution for quasi-steady, can be written as:

3 2 ( ) 0.807 q Lev D Sh t S S l   = =     (1)
where Sh(t) represents the Sherwood number, depending on the limiting current I, the diffusional coefficient of the electrochemical solution D, the Faraday number (F=96485 C/mol), the active potassium ferricyanide ion concentration C 0 , the electrochemical probe width ℓ ( 0.82 d = where d is the active diameter of the probe), A its area and finally n the number of electron exchange during the oxido-redox electrochemical reaction. This Sherwood number can be written as follow:

( ) 2 ( ) ( ) ( ) 3 q Sob q S t S t S t t t θ ∂   = +   ∂   (3)
where:

2 1 2 3 3 3 ( ) 0.486 ( ) q t l D S t θ - - = (4)
For low Péclet numbers, the inverse method is frequently suggested. The details of the method were developed for the first time by Beck et al. (1985), who introduced it to solve the thermal transfer problems, then, by Mao and Hanratty (1990) and Maquinghem (1999) to solve the mass transfer problems.

FIGURE 2: ELECTROCHEMICAL PROBES LOCALIZATION ON THE INNER WALL OF THE OUTER CYLINDER

III RESULTS AND DISCUSSION III.1. Mean flow characteristics

We investigate the effect of grooving on the general behavior of the flow. To characterize flow states in the grooved and smooth cases, we perform scans velocity. The question now is if roughness disturbs the flow and accelerates the transition apparition or the inverse.

To attempt to answer the question of whether the roughness makes the flux more or less stable with respect to the appearance of vortices than in the stable case, we compare the state of the flow with the case where the smooth inner cylinder.

The low dimensions of the Kalliroscope particles allow its transport by the flow. Then, the visualizations allow us to describe the flow structures and to detect the flow instabilities transitions.

III.2. Taylor Vortex Flow (TVF)

In smooth Couette-Taylor System (CTS), the first flow transition i.e. the transition from laminar Couette Flow to Taylor Vortex Flow corresponds to a critical Taylor number of 41 for A radius ratio is η = ri/ro=0.855 and an aspect ratio of Г=31 (Berrich, 2011). The time-evolution of Taylor number for CTS with grooved cylinder is presented on figures 3 and 4. It shows that the flow is steady. It corresponds thus to laminar Couette flow. For this Taylor number, we studied the timeevolution of the mass transfer. The mass transfer is almost constant (figure 5). In fact, no vortices appear for Ta=41. To conclude, for the same Taylor number, in the case where the inner cylinder is smooth vortices appear while when the cylinder is grooved no vortices appear (figure 6). Thus, grooves relay flow transition from laminar Couette Flow to Taylor Vortex Flow. By comparing the results in the cases where the inner cylinder is smooth or with grooves, the first vortex structures begin to develop in the same way from the ends of the latter going along the gap between the two cylinders. For cylinder with grooves, the first instabilities appear at Ta≈52 (figure 7). As anticipated, the grooves produce velocity modulations and the resulting interaction with the flow breaks the primary instability responsible of Couette-Taylor roll formation, so the delay in the appearance of the first instabilities.

The evolution of the parietal velocity gradient, obtained according to the method of Levêque and sobolik (1987) as a function of time, is illustrated by the figure 9. Note that the two signals are completely in phase.

III.3. Wavy Vortex Flow (WVF)

In order to check the dependence of the flow structure on the Roughness, Figure 10, shows the flow characteristics for the second instability.

When the Taylor number increases, a second instability appears at Ta=103. It is characterized by the presence of wavy vortices. The evolution of the Taylor number is given in Figure 11.

Ta=105 Ta=115

FIGURE 10: FLOW VISUALIZATION FOR THE WVF REGIME

Thus, the visualization shows that the rollers take the same periodic wave shape as in the case of smooth cylinder around the cylinder in the azimuthal and axial direction, thus breaking the axisymmetry of the rollers.

Turning to a number of Taylor Ta = 105, we notice the appearance of a new period which is superimposed on that which appeared in the previous regime (Figure 11).

The instantaneous mass transfer rate evolutions for WVF demonstrates that the wavy vortices are periodic in the axial direction with a period λ and in the azimuthal direction with a wave number m and in time with a frequency f 1 .

For a radial ratio η = 0.87, Edwards et al. (1991) [START_REF] Edwards | Onset of wavy vortices in the finite-length Couette-Taylor problem[END_REF] showed that the critical Taylor number corresponding to this second transition, and is equal to Ta c2 = 234 for 3 pairs of vortices and Ta c2 = 52 for 17 pairs of vortices. This may be due to the effect of the Ekman vortices that are constituted at the motionless wall of the air gap [START_REF] Sobolik | Wall Shear Rates in Taylor Vortex Flow[END_REF].

Figure 11 shows that in this case where we are at the beginning of the wavy Taylor vortex regime, the wave number m is equal to 4. From figure 12 it can then be seen that the rapid transition from one flow regime to another results in a rapid intensification of the mass transfer and consequently of the parietal velocity gradient. This intensification is relative to the development of corrugated Taylor vortices, thus characterizing the WVF regime. The increase in the azimuthal wave causes fluctuations in the parietal velocity gradient determined by the method of Sobolik et al. (1987).

III.4. Modulated Wavy Vortex Flow (MWVF)

By further increasing the number of Taylor, the flow goes through a third transition to Ta c3 =142. This transition is triggered when the main azimuthal wave of the WVF regime is then more homogeneous and rolls start to be unstructured (figure 13).

FIGURE 13: EXPERIMENTAL RESULTS FOR MWVF REGIME AT Ta=167

This transition is characterized by the appearance of a low frequency which present the modulation frequency of the rolls (figure 14). This regime appears with a delay compared to the case where the surface of the inner cylinder is smooth. The parietal velocity gradient curves obtained by the two methods of SOBOLIK and LEVEQUE (figure 15) show several oscillations. Fluctuations have become difficult to enumerate because of the superposition of different frequencies.

III.5. Turbulent Taylor Vortex Flow (TTVF)

At the instability threshold characterizing the transition between MWVF and nascent turbulent flow, the vorticity field is qualified by the homogenization of the flow in the presence of turbulent structures. This transition occurred at Ta=583 (figure 16). It is characterized by a continuous spectrum without characteristic frequency (figure 17). Sodjavi, et al., 2018 illustrated that for fully developed turbulent flow (high Reynolds numbers), the ratio of friction coefficient increases while increasing groove numbers (G6, G12, G24). But, they didn't study the effect of groove numbers on flow regimes instabilities. In our case G36 configuration, the turbulent flow appears at Ta = 583 (Re = 209) while it reaches 1210 for a smooth CT inner cylinder (Berrich, 2011).

IV. CONCLUSION

This paper presents an experimental study of the roughness effect on flow behavior in a Couette-Taylor system characterized by a radius ratio η = ri/ro=0.886 and an axial aspect ratio of Г=h/d=39.47. Two experimental techniques were used: visualizations using Kalliroscope particles and the polarography technique i.e. the Electro-Diffusion method. The spatio-temporal evolutions of the flow issued from the visualizations and the wall shear rates evolutions determined from the signal issued from the electro-diffusion probes show that surface roughness can modify the instability character of the flow: delay of the instability apparition.

For these studies, the direction of the main flow is obstructed by the ribs. Therefore, a strong pressure gradient is generated leading to increase transfer of momentum from fluid to the wall, so intense the surface friction. Then, the first instabilities appear with a delay.

  .b.

FIGURE 1 :

 1 FIGURE 1: (a): SCHEMATIC VIEW OF THE CT: 1 Storage tank; 2 Pump; 3 By-pass; 4 Flowrate control valve; 5 Flowmeter; 6 CT feeder tank; 7 Drain valve; 8 Support; 9 Supply holes for the homogeneous filling of the CT gap (8 holes); 10 CT grooved inner cylinder; 11 CT outer cylinder; 12 Flabby; 13 Motor shaft; 14 Electrical motor; 15 Overflow; 16 Ball bearing; 17 Elastic coupling; 18 Protection; 19: Circular probe; (b): PHOTOGRAPHY AND SKETCH WITH DIMENSIONS OF THE EXPERIMENTAL SETUP; (c):.DETAILS OF THE GROOVED INNER CYLINDER WITH DIMENSIONS.

  Top view of the grooved cylinder

  The Leveque solution becomes invalid for disturbed regime with unsteady flow. Thus, a correction of the Lévêque solution (1928) is proposed by Sobolik et al. (1987):

FIGURE 3 :

 3 FIGURE 3: VISUALIZATION OF THE COUETTE FLOW REGIME AT: (a) t = t 0 ; (b) t = t 0 +48

FIGURE 4 :

 4 FIGURE 4: FLOW VISUALIZATION FOR THE COUETTE FLOW REGIME AT Ta=41

FIGURE 5 :

 5 FIGURE 5: EVOLUTION OF MASS-TRANSFER RATE (SHERWOOD NUMBER Sh) FOR Ta=41By increasing the Taylor number, the spatiotemporal evolution of the reflected intensities field becomes non homogeneous along the system.

Figure 6

 6 Figure 6 shows that shear rates are no longer constant because of the transition apparition from Couette laminar regime to Tayor Vortex regime. The average shear rate variations of the wall depend mainly on the axial and radial position at the inner wall of the outer cylinder.

FIGURE 6 :

 6 FIGURE 6: EVOLUTION OF MASS-TRANSFER RATE (Sh) FOR Ta=75

FIGURE 7 :

 7 FIGURE 7: FLOW VISUALIZATION OF THE APPEARANCE OF THE FIRST TAYLOR CELLS AT Ta=75

FIGURE 8 :

 8 FIGURE 8: THE APPEARANCE OF THE FIRST TAYLOR CELLS

FIGURE 9 :

 9 FIGURE 9: EXAMPLE OF TEMPORAL EVOLUTION OF THE PARIETAL VELOCITY GRADIENT DETERMINED FROM THE METHODS OF LEVEQUE AND SOBOLIK (1987) FOR Ta=75 (PROBE 7)

FIGURE 11 :FIGURE 12 :

 1112 FIGURE 11: EVOLUTION OF MASS-TRANSFER RATE FOR Ta=105

FIGURE 14 :

 14 FIGURE 14: EVOLUTION OF MASS-TRANSFER RATE FOR Ta=183

FIGURE 15 :

 15 FIGURE 15: EXAMPLE OF TIME EVOLUTION OF THE WALL VELOCITY GRADIENT DETERMINED USING THE METHODS OF LEVEQUE AND SOBOLIK (1987) FOR Ta=183

FIGURE 16 :ProbeFIGURE 17 :FIGURE 18 :

 161718 FIGURE 16: EXPERIMENTAL RESULTS FOR TTVF REGIME AT Ta=583

Table 1

 1 summarizes the different Taylor transitions corresponding to each flow regime.

Table 1 .

 1 Experimental results for Critical Taylor numbers

	Flow	Experimental Critical	Experimental Critical
	regime	Taylor number value	Taylor number value Ta c
		Ta c smooth surface	Rough surface
		Berrich [ 18]
	TVF	45.45	50
	WVF	60.6	103
	MWVF	606	142
	TTVF	1210	583

Probe 7 Probe 8


Sana Benhamaid
email: sana.benhamaid@hds.utc.fr

Hicham Lakhlef
email: hlakhlef@utc.fr

Abdelmadjid Bouabdallah
email: madjid.bouabdallah@hds.utc.fr

Towards Energy Efficient Mobile Data Collection In Cluster-based IoT Networks

Keywords: Sana Benhamaid, Hicham Lakhlef, Abdelmadjid Bouabdallah Internet of Things, mobile data collection, trajectory planning, deep reinforcement learning

Towards Energy Efficient Mobile Data Collection In

Cluster-based IoT Networks

I. INTRODUCTION

One of the major challenges of IoT networks is the lifetime of the battery limited IoT devices. IoT devices consume large amounts of energy in order to communicate data with their neighbours. These communications will quickly lead to the devices battery depletion. In the literature, various authors studied how to reduce the energy consumption of spatially distributed IoT nodes. Researchers have studied cluster-based energy efficient solutions which consist on regrouping IoT devices into clusters and elect a node depending on its energy capability. This node will serve as a relay node between the IoT devices of the cluster and the gateways or receivers. The relay node will be responsible for sending data and will allow IoT devices to save their energy by only sending their data to the relay node [START_REF] Song | Qosaware energy-efficient cooperative scheme for cluster-based iot systems[END_REF]. In order to enhance the energy efficiency of cluster-based solutions, authors proposed energy efficient routing protocols applied to these contexts. For example, in [START_REF] Toor | Energy aware cluster based multi-hop energy efficient routing protocol using multiple mobile nodes (meacbm) in wireless sensor networks[END_REF], the authors proposed an energy efficient cluster-based routing protocol for wireless sensor networks that uses different parameters in order to balance the load of energy and decrease the energy consumption of the network. However, using a routing protocol to reduce the energy consumption in an IoT network implies that these devices must be connected to each other which is not always realistically possible if the nodes are placed in isolated or harsh conditions. One of the solutions proposed to tackle the limitation of routing-based energy efficient solutions are the mobile sink-based solutions. In this case, a node will be responsible for collecting the data of all IoT devices in a cluster of devices and transmitting it to the mobile node when the mobile node is in its range. Mobile data collection is a very promising energy saving solution for IoT networks. However, the most challenging part in mobile sink-based solutions is to determine and plan the trajectory of the mobile sink in order to gather data from the nodes. Most existing approaches to mobile data collection are static and only find a solution for a scenario with fixed parameters. These solutions do not consider a change of context in the system such as a change in the amount of data generated by the IoT nodes in the clusters or the mobility of the IoT devices where an IoT device can move from a cluster to another.

Recently, researchers have been interested in using artificial intelligence techniques such as machine learning or deep learning, in order to propose intelligent mobile data collection solutions that will adapt the trajectory of the mobile node depending on the activity of the IoT nodes and the environment. In our solution, we will use a machine learning paradigm called reinforcement learning. Reinforcement learning is suitable for complex environment that need adaptability depending on the context. This technique will allow us to have a path planning solution that will adapt the trajectory of the mobile node in order to maximize the amount of collected data. Unlike other existing solutions, our solution will focus on reducing the energy consumption and plan the trajectory of the mobile node depending on the activity of the clusters (e.g. a cluster which has no data to send will be less likely to be visited by the mobile node).

II. BACKGROUND AND BRIEF DESCRIPTION OF OUR

SOLUTION

A. Background on machine learning

In learning based problems where we need our agent or system to learn a behavior and make informed decisions, machine learning solutions have proved to be the most efficient. Machine learning algorithms learn from data or experiences and apply what they have learned to make informed decisions. They usually involve human intervention that gives feedback to the algorithm. Machine learning algorithms are divided into three categories : supervised, unsupervised and reinforcement learning. For our solution, Fig. 1: Description of our system and a possible scenario and trajectory for the mobile node we chose to use reinforcement learning. In this section, we will present the three machine learning paradigms and why reinforcement learning is the most appropriate approach for our problem rather than supervised or unsupervised learning.

Reinforcement learning is a machine learning paradigm that is inspired from human learning. It is based on the concepts of agent, state, action and reward. The objective of the agent is to learn which actions to take in order to maximize the reward given a state of its environment and a set of possible actions.

Reinforcement learning is different from supervised learning which is the most studied machine learning paradigm. Supervised learning is learning from a set of labeled data provided by a knowledgeable external supervisor [START_REF] Sutton | Reinforcement learning: An introduction[END_REF]. The goal of this type of learning is to build a system able to act correctly when a situation is not included in the training data set. Supervised learning alone is, consequently, not suitable for interactive problems since in our solution the mobile node needs to interact with its environment and learn from its own experience. Reinforcement learning is also different from unsupervised learning. Unsupervised learning is a learning paradigm that aims to find correlations in sets of unlabeled data. This type of learning is not based on correct examples. Finding structure in a reinforcement learning agent experience might be interesting. However, in our solution, the most important objective is to maximize the reward (e.g. maximize the collected data and minimize the energy consumption) which is not covered by unsupervised learning. Consequently, reinforcement learning is the most suitable technique to our problem.

B. Brief description of our solution

Our solution consists on planning the trajectory of a mobile node using a deep reinforcement learning algorithm. As shown in Fig. 1, the mobile node will successively visit clusters and collect data from an elected node in the cluster called relay node. The relay node is responsible for collecting the data from the IoT nodes in the cluster and transmitting it to the mobile node. The objective of our solution is to maximize the collected data while minimizing the energy consumption of the mobile node and IoT devices.

Our solution will also take into consideration the change of activity in the IoT network which means that the mobile node will be less likely to visit a relay node with no data to send. As shown in Fig. 1, the mobile node chose to ignore a cluster that do not meet certain conditions. The mobile node has information on the states of all relay nodes (amount of data to collect). The learning algorithm is also computed on the mobile node. Other solutions proposed reinforcement learning solutions to plan the trajectory of mobile nodes in order to either optimize data collection or minimize the energy consumption and need to perform expensive computations in order to adapt to a change in the context. To the best of our knowledge, our solution is the only one solution that uses deep reinforcement learning in a cluster-based IoT network in order to maximize the data collection, reduce energy consumption of the mobile node and takes into consideration the variation in activity of the different clusters.

III. RELATED WORKS

In order to plan the trajectory of mobile nodes in a data collection context, authors proposed non learning based solutions. For example, in [START_REF] Mozaffari | Mobile unmanned aerial vehicles (uavs) for energy-efficient internet of things communications[END_REF], the authors proposed a framework that optimizes the deployment and mobility of multiple UAV in order to collect data in the uplink from ground IoT devices and minimize the energy consumption of the mobile nodes. In [START_REF] Ghorbel | Joint position and travel path optimization for energy efficient wireless data gathering using unmanned aerial vehicles[END_REF], the authors proposed an algorithm that optimizes the UAV stops for data collection from neighboring sensors and the itinerary followed by the UAV in order to ensure efficient collection of all data with minimum energy consumption. However, in IoT networks, devices do not constantly collect and transmit data, their activity depend on the environment around them or the period of time the node is visited in. In the previous solutions a change in the activity of the IoT network is not considered. Consequently, having a mobile sink periodically collecting data, following a static trajectory and visiting devices that have no data to send may cause energy waste and does not constitute an optimal solution to achieve energy efficiency.

In the literature, most existing data collection and trajectory planning solutions use non-learning based approaches. However, in the recent years, researchers have considered learning based solutions and especially reinforcement learning as a very promising trend in this field. In [START_REF] Hsu | Reinforcement learning-based collision avoidance and optimal trajectory planning in uav communication networks[END_REF], the authors proposed a distributed reinforcement learning approach for path planning and collision avoidance of UAVs. The optimal path design is studied for IoT networks with devices having different communication radio. In [START_REF] Zhang | Trajectory planning of uav in wireless powered iot system based on deep reinforcement learning[END_REF], the authors proposed a deep reinforcement learning scheme that plans the trajectory of a mobile node which is used as a data collector and charger in wireless powered IoT networks. The solution minimizes the average data buffer length, minimizes the residual battery level of the system and avoid devices data overflow. In [START_REF] Bayerlein | Multi-uav path planning for wireless data harvesting with deep reinforcement learning[END_REF], the authors proposed a multi-agent reinforcement learning approach that allows the control the trajectory of a team of cooperative UAVs in order to maximize the collected data under flying time and collision constraints. In [START_REF]Uav path planning for wireless data harvesting: A deep reinforcement learning approach[END_REF], the authors proposed a double deep Q network to plan the trajectory of an UAV on an IoT data harvesting mission. The solution proposed aims to maximize the collected data under flying time and navigation constraints and allows the UAV to adapt to variations in the number of IoT devices. On contrary to our solution this solution does not take into consideration the inconsistencies in the IoT devices activity (e.g a device can collect more data on a certain period of the day more than another).

IV. SYSTEM MODEL AND PROBLEM FORMULATION

A. System Model

We consider an IoT network composed of a number of clusters randomly distributed in a square grid world of size M × M ∈ N 2 with a mobile node collecting data through a relay node from a number K of clusters of IoT devices. We suppose that the relay node is already chosen depending on its energy capability. The k-th relay node is permanently located at [x k (n), y k (n)] in the grid world with k ∈ [1, K] and n = 1, 2, ..., N , represents the number of time slots. We consider the velocity v(n) of the mobile node constant and enough for the mobile node to move from one square of the grid to the one next to it. We consider the velocity as constant since we want our energy efficient solution to be adapted to any type of mobile node. At time step n, the mobile node has an energy capacity w(n) and w c (n) the amount of energy needed to visit the next cluster at time step n. We also consider w f (n) the minimum level of energy needed for the mobile node to move to a safe stop spot z with z in [1, Z]. A safe spot is an area in the square grid where the mobile node will go to when its energy level is not sufficient to visit another cluster in order to recharge its energy.

B. Markov Decision Process

In order to find an optimal data collection policy that maximizes the data collection and minimizes the energy consumption of the mobile node. We will formulate our problem as a Markov decision process problem (MDP). In this section, we define the state space, action space and reward function. We solve this MDP problem using reinforcement learning. The MDP is defined by the tuple (S, A, R, P ) with state-space S, action-space A and reward function R. The state in time t, is given by s t = p t , D t , e t , C where:

• p t ∈ R 2
is the mobile node position on the grid; • D t ∈ R 2×K is the collected data and the remaining data for each cluster head; • e t ∈ N is the remaining energy of the mobile node in time t; • C ∈ R 2×K represents the coordinates of the K cluster heads. The mobile node is limited to fly to one of the four adjacent grids from its current grid in each time slot. The action-space is defined as

A = north, east, south, west
The movement of the mobile node from a position p t is expressed as

p t+1 = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ p t + (-X, 0) if a t = west p t + (X, 0) if a t = east p t + (0, X) if a t = north p t + (0, -X) otherwise
where X is length of a square in the grid. The reward function is a function maps state-action pairs to a real-valued reward, i.e. R : S × A → R. It consist of the following:

• r ed is a reward given if the data collected is superior than a given threshold and the energy consumed is less than a given threshold; • r sd is a reward given if the data collected is superior to a given threshold but the energy consumed is superior than a given threshold; • r ee is a reward given if the data collected is less than a given threshold but the energy consumed is less than a given threshold; • r ne is a penalty given if the data collected is less than a given threshold but the energy consumed is more than a given threshold; • r move is a penalty given each time the mobile moves without collecting data; • r f inish represents a penalty given in case the mobile node energy reaches zero without being in a safe spot.

V. METHODOLOGY

A. Q-Learning

Q-learning is a model-free reinforcement learning technique that helps finding an optimal policy and maximize the expected value of the total reward from the current state and all the consecutive states. For our solution, we chose to use Qlearning since we want the mobile node to freely explore the environment which will help it modify its trajectory in case a new cluster of IoT nodes appears rather than using a more conservative algorithm like SARSA. Formally, for an autonomous agent observing a state s of its environment at time step n s n ∈ S where S is the set of states. The agent executes an action a n ∈ A, where A is the set of possible actions and interacts with its environment. This action changes the state of the environment to a new state s n+1 and the agent will receive a reward r n accordingly. The environment is expected to be non-deterministic as taking the same action in the same state on different occasions may result in different states and different rewards. The agent's goal is to find a policy π that maps a state s n to a probability of choosing action a n and can be represent as following

π : S → P (A)
The reward is the sum of discounted future rewards. To calculate the reward we use γ, where γ ∈ [0, 1] is a discount factor that determines the effect of the future rewards to the current one.

R n = N i=n γ i-n r(s i , a i )
The optimal policy π * is defined as

π * = argmaxE[R n |π]
This optimal policy has an optimal state-action value function that satisfies Bellman optimality equation. It is described by the following

Q π * (s, a) = E[R n+1 + γmaxQ * (s n+1 , a n+1 )]

B. Deep Q-learning

Q-learning is a very efficient algorithm when the environment is limited to small state spaces. However, computing and updating table values for each state-action pair is not efficient when it comes to more complex and sophisticated environments with large state and action such as our environment problem. In these cases, it is more interesting to find an approximation of the Q-value rather than directly computing it. In order to do that, we will use Deep Q-Learning which is technique that uses neural networks to approximate the optimal Q-function Q π (s, a). Deep Q-learning often uses experience replay which is a technique that allows us to store the agent's experiences at each time step n in a data set called replay memory. At time step n, the agent's experience is defined as following e n = (s n , a n , r n+1 , s n+1 )

The main reason for using experience replay is to break the correlation of consecutive samples. In our solution we uses two neural networks. The policy network θ which approximates the optimal policy by finding the optimal Q-function. It accepts the current state s n and finds the evaluation of the value Q(s n , a n , θ). We also use a second network called target network θ to improve the stability of learning. The target network weights are frozen with the original policy network and are updated periodically. It accepts the next state s n+1 and outputs the Q-value Q(s n+1 , a n+1 , θ ). These values are optimized to minimize the loss function defined by

L(θ) = E[(T n -Q(s n , a n )) 2 ]
where T n is the target value which is defined as following

T n = r n + γ n-1 maxQ(s n+1 , a n+1 , θ )
where the Q-value for the next state s n+1 is passed to the target neural network θ for more stability in learning. is the exploration rate which represents the probability that our mobile node will explore the environment and expt is the threshold from which our mobile node will stop exploring the environment and will exploit the experience acquired through the policy network.

VI. CONCLUSION

In this paper, we have introduced a deep Q-learning method with experience replay for trajectory planning of a mobile node in a mobile data collection and cluster-based scenario. We are currently working on training a neural network that uses information about the environment to learn and find an energy efficient trajectory for the mobile node and adapt to significant changes in the context scenario (e.g. the amount of collectible data) without the need for expensive retraining or recollection of data.

  Algorithm 1 DQN-based Energy Efficient Mobile Data Collection initialize action-value with random weights θ and θ , replay memory size H, number of episodes F initialize for exploration and expt for episode :1, ..., F do Initialize the environment and receive initial state s 1 ; Set n = 0; while w > w c do if > expt then Choose a n randomly from action space; else Select a n = argmaxQ(s n , a n , θ); end Execute action a n , compute r n and observe the next state s n+1 ; Store experience (s n , a n , r n , s n+1 ) in the replay memory with a random placement policy; Sample a random mini-batch of G experiences (s i , a i , r i , s i+1 ) from replay memory; Calculate the target value; Update the weights θ = θ every U time step;

	Decrease w;
	Set n = n + 1;
	end
	Decrease ;
	end

Authorized licensed use limited to: Universite de Technologie de Compiegne (UTC). Downloaded on January 10,2022 at 14:49:34 UTC from IEEE Xplore. Restrictions apply.


domain: spi phys math info

text:
Valentin Le Mire

Julien Queyrel

Laurent Castanet

Laurent Féral

Xavier Boulanger

simulateur atmosphérique reposant sur un modèle de prévision météorologique à haute résolution

published or not. The documents may come

Modélisation du comportement pluriannuel de l'atténuation due à la pluie en bande Ka en Guyane française à l'aide d'un

Modélisation du comportement pluriannuel de l'atténuation due à la pluie en bande Ka en Guyane française à l'aide d'un simulateur atmosphérique reposant sur un modèle de prévision météorologique à haute résolution

Valentin Le Mire 1 , Julien Queyrel 1 , Laurent Castanet 1 , Laurent Féral 2 , Xavier Boulanger 3

1 ONERA/DEMR, Université de Toulouse, F-31055 Toulouse -France 2 Laboratoire d'Aérologie 3 CNES/DSO/RF/ITP

I. CONTEXTE

Au cours des dernières années, que ce soit dans le secteur civil ou militaire, une certaine évolution des bandes de fréquences utilisées par les satellites de télécommunication a pu être observée. En effet, si les bandes C et Ku étaient grandement utilisées auparavant pour les systèmes de télécommunication, des bandes de fréquences plus élevées sont aujourd'hui exploitées telles que la bande Ka actuellement et les bandes Q/V voire W qui sont envisagées pour le futur proche. Cette montée en fréquence s'explique principalement par un besoin en bande passante grandissant lié à une augmentation significative du nombre d'utilisateurs et d'applications multimédia.

Une deuxième évolution récente dans le domaine des télécommunications par satellite est l'arrivée en masse sur le marché des pays émergents et en voie de développement. Ces pays étant très peuplés, l'utilisation de bandes de fréquences plus élevées telles que la bande Ka permet de réduire la congestion des bandes de fréquences plus conventionnelles.

Cependant, ces deux évolutions s'accompagnent de perturbations de la propagation du signal dans la troposphère. En effet la traversée de la troposphère est source d'atténuations des signaux radio et plus la fréquence du signal augmente plus celles-ci ont un impact important. Aux bandes de fréquences exploitées ici, ce sont les précipitations qui sont le facteur principal d'atténuation du signal. Ceci est d'autant plus problématique dans le contexte défini précédemment que la plupart des pays émergents et en voie de développement actuellement très actifs sur le marché des télécommunications se trouvent en régions tropicales et équatoriales. C'est ici une observation importante car les climats rencontrés dans ces régions sont caractérisés par d'importantes précipitations qui peuvent parfois être bien plus extrêmes que celles rencontrées en milieu tempéré.

La caractérisation du canal de propagation et plus précisément la caractérisation des précipitations dans ce canal est un sujet d'importance majeure dans le domaine de la propagation. Pour ce faire, dans un premier temps il est nécessaire d'obtenir des données lors d'expérimentation de propagation. Dans un second temps, ces données sont utilisées pour tester les modèles de propagation existants ou pour en développer de nouveaux. Il existe aujourd'hui de nombreux modèles de propagation en bande Ka qui ont été principalement développés pour des climats tempérés mais peu ont été testés en régions tropicales ou équatoriales. Le problème de ces régions est qu'il n'existe aujourd'hui que très peu d'expérimentations de propagation en bande Ka sur des liaisons Terre-Espace, particulièrement en regard du nombre d'expérimentations effectuées en régions tempérées.

Depuis plusieurs années, une alternative envisagée pour entre autre, pallier ce manque de données est l'utilisation de modèles météorologiques à haute résolution associés à des modèles de propagation. Dans cette optique l'ONERA développe et optimise un modèle hybride météorologiqueélectromagnétique nommé WRF-EMM [START_REF] García | Use of WRF model to characterize propagation effects in the troposphere[END_REF]. Ce modèle est composé d'un modèle météorologique à haute résolution WRF [START_REF] Skamarock | A description of the Advanced Research WRF model version 4[END_REF] couplé à un module de calcul d'atténuation troposphérique EMM qui permet de générer des statistiques de propagation sur de longues périodes pour tout type de liaisons de radiocommunications opérant à fréquence élevée. L'utilisation de ce modèle pour reproduire des statistiques d'atténuation due à la pluie en Guyane française est ici présentée sur une période de trois années complètes 2017, 2018 et 2020. En effet, une expérimentation de propagation entreprise par l'ONERA et le CNES sur le Centre Spatial Guyanais (CSG) [START_REF] Boulanger | Propagation Experiment at Ka-Band in French Guiana: First Year of Measurements[END_REF] a permis d'obtenir sur cette durée des données expérimentales d'une disponibilité excellente qui permettent de comparer et évaluer les performances du modèle WRF-EMM. Plus précisément, ces performances sont analysées non seulement d'un point de vue annuel mais aussi d'un point de vue saisonnier et mensuel. En effet, cette région équatoriale subit une saisonnalité très marquée avec des différences de précipitations entre les saisons sèches et humides très importantes, il est donc important de connaitre la capacité du modèle WRF-EMM à reproduire ces effets saisonniers pour obtenir des statistiques d'atténuation due à la pluie non seulement annuelles mais aussi saisonnières et mensuelles.

II. LE MODELE WRF-EMM

Le fonctionnement global du modèle WRF EMM utilisé ici est brièvement présenté en Figure 1. On peut y voir que le modèle météo WRF est nourri par la base de données de réanalyse ERA5 pour produire des cartes atmosphériques tridimensionnelles avec des résolutions de l'ordre de 2_x_2_km² et 5 minutes. Ces cartes permettent d'extraire différents paramètres tels que la température, la pression et les différents contenus en eau dans chaque cube atmosphérique. Le module EMM exploite ensuite ces paramètres pour calculer les séries temporelles d'atténuation sur une liaison satellite spécifique. En particulier, l'atténuation due à la pluie est calculée à l'aide de la théorie de Mie pour des gouttes supposées sphériques et d'une distribution de leur taille équivalente à celle utilisée par le modèle WRF. Vu la grande variabilité saisonnière observée dans cette région, cette analyse pluriannuelle a également été entreprise d'un point de vue mensuel pour évaluer la capacité de WRF-EMM à reproduire une granulométrie plus fine. Cette analyse a montré des très bonnes performances de WRF-EMM malgré une certaine variabilité mensuelle, comme illustré en Figure 3.

Figure 1 -

 1 Figure 1 -Fonctionnement global de WRF-EMM

Figure 2 -

 2 Figure 2 -Statistiques annuelles et saisonières d'atténuation due à la pluie à Kourou pour la période 2017, 2018 et 2020.

Figure 3 -

 3 Figure 3 -Exemples de statistiques mensuelles d'atténuation due à la pluie à Kourou pour la période 2017, 2018 et 2020.