Keywords: spacecraft charging, electron emission, electrostatic discharges, testing, simulation

To represent the time spent by space materials on ground before launch, the measurement of secondary electron emission properties is performed after long duration exposure to ambient atmosphere and humidity. The observed change with respect to pristine samples have an impact on the estimation of worst-case surface charging levels in geostationary orbit, especially for a spacecraft in eclipse. It is therefore recommended to adequately outgas the samples with respect to the expected flight conditions and ageing effects.

I. INTRODUCTION

stimating worst-case surface charging remains a challenge because it depends on the complex interaction between the spacecraft and its environment. One of the key physical processes at play is the electron emission of surface materials exposed to ambient electron irradiation. Flight data have shown that hundreds to thousands of volts of negative potentials are observed on spacecraft at geosynchronous orbit (GEO) when the spacecraft is in the midnight sector, especially when it is in Earth eclipse [START_REF] Choi | Analysis of GEO spacecraft anomalies: Space weather relationships[END_REF]- [START_REF] Thomsen | Statistical properties of the surface-charging environment at geosynchronous orbit[END_REF]- [START_REF] Matéo-Vélez | Spacecraft surface charging induced by severe environments at geosynchronous orbit[END_REF]. High differential voltages of surface dielectrics with respect to the structure have also been measured in the past [START_REF] Mizera | Charging Results from the Satellite Surface Potential Monitor[END_REF] but too few spacecraft are equipped with dedicated instrumentation to allow extrapolating to other situations. Ground experiments are performed by the space industry on small sized coupons to test the response of materials sensitive to charging hazards under flight representative conditions. These data can be used as is and/or serve as inputs of larger scale simulations using numerical tools [START_REF] Roussel | SPIS multi time scale and multi physics capabilities: development and application to GEO charging and flashover modelling[END_REF]- [START_REF] Thiebault | SPIS 5.1, an innovative approach for spacecraft plasma modelling[END_REF]- [START_REF] Mandell | Nascap-2k Spacecraft Charging Code Overview[END_REF]- [START_REF] Muranaka | Development of multi-utility spacecraft charging tool (MUSCAT)[END_REF] that combine the three-dimensional geometrical description of a given spacecraft, the physical mechanisms ruling the behavior of its surface materials and, finally, the energy distribution of worst-case environments [START_REF]ECSS-E-ST-10-04C, Space environment[END_REF]- [START_REF]NASA-HDBK-4002A[END_REF].

This paper shows how the ground experimental conditions used to measure the electron emission yield (EEY) under This work was supported by CNES R&D program 2017. J.-C. Matéo-Vélez, M. Belhaj S. Dadouch, P. Sarrailh, S. L. G. Hess are with ONERA / DPHY, Université de Toulouse, F-31055 Toulouse -France (e-mail: mateo@onera.fr).

Denis Payan is with CNES, FR-31401, Toulouse (e-mail: denis.payan@onera.fr) electron irradiation have an impact on the resulting data and subsequently on the extrapolation to space conditions. Due to the very low escape depth of secondary electrons (a few nanometers), EEY is highly sensitive to first mono-layers. Studies of the electron emission behavior of pi-bonded organic structures as well as bare metals show that the probability of electron emission can change dramatically at sub-monolayer levels and that the change tends to flatten out at a few monolayers ( [START_REF] Hong | Coverage-Dependent Modification of the Surface Electronic Structure of an Organic-Semiconductor-Adsorbate Layer[END_REF]- [START_REF] Hilleret | The secondary-electron yield of air-exposed metal surfaces[END_REF]).

We first present the effect of material surface state sensitivity on EEY. In particular, we show the impact of the contamination and/or oxidation layers that form at the surface of technical materials, i.e. the materials used in spacecraft technologies, because of their inevitable exposure to the atmosphere. Finally, we show how these changes result in different results from a worst case surface charging analysis.

Section II presents the experimental apparatus used to measure the EEY in various conditions aiming at representing the main steps from manufacturing to launch to space environment. Section III presents and analyzes the results of the experimental campaign as for the effect of the evolution of the surface composition. Section IV describes the numerical simulation study performed to identify the effect of EEY measurements on the estimation of worst-case surface charging under GEO environments. The numerical results are presented in Section V and discussed in Section VI.

II. EXPERIMENTAL APPARATUS

A. Facility

The surface analysis by means of X-ray Photoelectron Spectroscopy (XPS) as well as EEY measurements were performed in DEESSE facility located at ONERA Toulouse Center. DEESSE, represented in Fig. 1, is entirely dedicated and designed to the study of electron emission and to surface analysis. A dry turbo-molecular pump associated with an oil-free primary pump allows the system being maintained at an Ultra-High-Vacuum (UHV) level. The sample holder allows the variation of the electron incidence angle from 0° (normal incidence angle) to 80°. An ELG electron gun from Kimball Physics Instrument was used. The electron beam was pulsed during EEY measurements to limit the surface conditioning

B. EEY measurement techniques

To measure the EEY parameters, the incident electron current I f and the sample current I S were monitored. I f was measured using a faraday cup. The sample was negatively biased with respect to the grounded vacuum tank during Is measurement in order to avoid the recollection of the tertiary electrons emitted from sample with few eV in energy and backscattered by the tank walls. The negative bias potential of -9 Volts was sufficiently low to neglect the deflection and deceleration of beam electrons with a few tens of eV. The electron beam was pulsed to avoid surface charging of insulating material, whatever their thickness. The duration of each pulse was 10 µs. A contactless Kelvin probe was used to check that no surface charging built up during irradiation. Knowing I f and I S, the EEY can be deduced thanks to the expression Eq.1 given below [16]:

𝐸𝐸𝐸𝐸𝐸𝐸 = 𝐼𝐼 𝑓𝑓 -𝐼𝐼 𝑠𝑠 𝐼𝐼 𝑓𝑓
Eq.1

C. Material Characterization

A DAR400 X-Ray source from Omicron was used for the XPS analysis. It was equipped with Mg Ka anode. The X-ray incidence angle was set to 45° and the electron energy analyzer acceptance angle was normal to the surface. As the resolution depth of the XPS is close to that of the mean secondary electron escape depth (a few nm), this analysis technique is suitable to investigate the effect of the chemical surface composition on the EEY.

D. Experimental Conditions

Two samples have been investigated in this work. The first one, referred as PNC, is a black conductive paint supplied by CNES. The second one was 100 nm thick silica (SiO 2 ) layer deposited on Si substrate [17]. The EEY yield have been measured after each step.

III. EXPERIMENTAL RESULTS

A. Results for SiO 2

Results for SiO 2 are first described here. Fig. 3 shows the XPS spectra measured on SiO 2 surface after the initial argon sputtering (initial step). The surface is composed mainly by Oxygen and Silicon as expected and also some remained carbon contamination. The evolution of the surface composition during the storage conditions is shown in Fig. 4.

Step 1

Sample cleaning at Reception

total of 100 days 10 days at humid air

Step 2

Storage at ambient air

Step 3

Storage at ambient air

Step 4

Storage at humid air EEY in UHV after baking EEY in HV The main noticeable result is the increase in the carbon content against silicon and oxygen (SiO 2 ). That illustrates how the exposition of the cleaned sample surface to the ambient atmosphere during steps 2 and 3 and thereafter to a wet atmosphere during step 4 leads to the deposition of hydrocarbon contamination layer [START_REF] Vaughan | A new formula for secondary emission yield[END_REF]. Fig. 5 shows the evolution of the normalized EEY after each storage step as a function of primary electron energy. Each measurement was performed ten times. The dispersion of the results was lower than 5 % at all energy. One significant effect of sample contamination during storage is the shift of the incident electron energy E max that corresponds to the maximum of EEY. This shift is more pronounced after the first exposure to the ambient atmosphere during step 2. The following exposure to ambient atmosphere during step 3 and to the wet atmosphere during step 4 has only a moderate effect on E max . The thickness of the built-up hydrocarbon layer after step 2 was probably comparable to the mean secondary electron escape depth. This may explain the quasi-saturation tendency of E max observed in Fig. 6. The second cross-over is shifted from around 20 keV at step 1 down to around 10 keV after step 4, which may affect spacecraft charging estimates.

B. Results for PNC

The evolution of the PNC surface composition during the storage conditions is shown in Fig. 7. The carbon content increased by 65% from step 1 to step 4. Fig. 8 shows the evolution of EEY after each storage step as a function of primary electron energy. Only little change is observed between measurements performed at step 1 and step 4, which should not impact spacecraft charging estimates.

C. EEY Fits

Analytical formulas are more suitable for numerical simulations than raw data. Fig. 9 presents two fits of the SiO 2 EEY at step 4 using the Vaughan [START_REF] Vaughan | A new formula for secondary emission yield[END_REF] and Dekker [START_REF] Dekker | Secondary Electron Emission[END_REF] models. Dekker's model better fits the experimental results than Vaughan's.

IV. NUMERICAL SIMULATION ANALYSIS

In this section, we describe the inputs and results of simulations made to estimate worst-case surface charging of a GEO spacecraft. The version 5.2.4 of the SPIS software, freely available from the Spacecraft Plasma Interaction Network in Europe website (www.spis.org), has been used.

A. Environments

Two environments have been modelled. The first environment is the European Cooperation for Space Standardization (ECSS) worst-case for surface charging at GEO [START_REF]ECSS-E-ST-10-04C, Space environment[END_REF], which is a double Maxwellian fit of the event measured on SCATHA on April 24th, 1979 [START_REF] Gussenhoven | Geosynchronous environment for severe spacecraft charging[END_REF]. This environment is referred as ECSS WC. The second environment, referred as 3M SCATHA in this paper, is the triple Maxwellian fit of the same event [START_REF] Matéo-Vélez | Spacecraft surface charging induced by severe environments at geosynchronous orbit[END_REF]. There are important differences between these two Maxwellian fits. The latter is higher than the former at electron energies between 1-10 keV and inversely above 20 keV. In this paper, the EEY of PNC and silica are used as inputs of the EEY model currently included in SPIS, i.e. the Vaughan model also used in NASCAP [START_REF] Mandell | Nascap-2k Spacecraft Charging Code Overview[END_REF].

B. Spacecraft Design

C. ESD risk criteria

The simulations are compared using two indicators of electrostatics discharge risks (ESD). The first indicator is the time necessary to get an inverted potential gradient (IPG) of +300 V between solar cell cover glasses and their underlying ground structure. The second indicator is the time to get the same +300 V IPG level on OSR.

V. WORST-CASE CHARGING PREDICTION

A. WC absolute charging in eclipse 'at equilibrium'

The configuration leading to the worst absolute spacecraft charging is defined as the configuration leading to the most negative frame potential at equilibrium. It is not necessarily the worst conditions in terms of ESD risks. The worst frame potential of -12.3 kV is obtained with the ECSS WC environment for GEO and with the EEY properties of materials measured after step 4 (after contamination by ambient atmosphere and humidity). The time to reach an IPG of +300 V The second worst situation for absolute charging has been obtained with the ECSS WC environment for GEO and with the EEY material properties measured after step 1. The frame potential is reduced to -11.3 kV due to a slight increase in the secondary electron current, consistent with the highest E max obtained at step 1. The time to reach an IPG of +300 V on a solar cell cover glass and an OSR surface is 120 seconds and 320 seconds, respectively. The risk to trigger ESDs on solar cells is twice that of the previous configuration.

B. WC SA charging in eclipse

The configuration leading to the highest risk to trigger ESDs on the solar arrays (SA) in eclipse condition is with 3M SCATHA environment and with EEY properties measured after step 1, i.e. after cleaning. The equilibrium frame potential is about -5.6 kV, which is half the frame potential, obtained in the configuration reported in Section V.A. The time to reach an IPG of +300 V on a solar cell cover glass and an OSR surface is 45 seconds and 150 seconds, respectively. This is a direct effect of the enhanced flux of ambient electrons of energy between 1 and 10 keV; inducing larger secondary electron emission from surfaces with respect to electrons above 20 keV, see figure 13 of [START_REF] Matéo-Vélez | Spacecraft surface charging induced by severe environments at geosynchronous orbit[END_REF]. It results in increasing the difference of potentials due to high EEY of silica in this energy domain. Material properties obtained after step 1 exhibits a larger EEY than other steps. They results in larger ESD risks. Fig. 11 presents the spacecraft surface potential at equilibrium. External solar panels face the worst conditions with higher IPG levels than closer to the hub. The effect of the barrier of potential, which is imposed by the most negative surfaces on the SA surface, is less pronounced on the outer solar panels.

C. WC OSR charging in eclipse

The configuration leading to the highest risk to trigger ESDs on OSR in eclipse conditions is obtained with 3M SCATHA environment and with EEY properties measured after step 4. The equilibrium frame potential is about -6.8 kV. The time to reach an IPG of +300 V on a solar cell cover glass and an OSR surface is 110 seconds and 120 seconds, respectively.

D. WC SA charging in sunlight

The simulation results are very close to each other when the spacecraft is in sunlight, whatever the EEY properties. This is explained by the dominance of photoemission over EEY at 1 AU. The equilibrium frame potential is about -4 kV under 3M SCATHA environment. The time to reach an IPG of +300 V on a solar cell cover glass and an OSR surface is 40 seconds and 200 seconds, respectively.

VI. DISCUSSION

For the spacecraft geometry and materials used in this paper, the inverted gradient potentials reach +300 V faster in sunlight in the 3M SCATHA environment definition. In these calculations, the +300 inverted gradient level is reached faster with the 3M SCATHA environment specification than with the ECSS WC environment. This environment results in both high level of absolute charging combined with increased EEY on solar cell cover glasses. It is worth reminding that this result is applicable to the present spacecraft configuration only. A different spacecraft design or choice of material properties could develop higher inverted gradient potentials in calculations with the ECSS WC environment than with the SCATHA 3M. Simulating an envelope of severe environments is thus recommended to predict the ESD risk with more confidence.

Exiting the Earth shadow needs also a special care because it allies two worst-case situations in series: equilibrium frame potential WC during eclipse followed by WC for solar arrays.

Another highlight of this paper is the importance to use material properties measured in appropriate conditions to get a proper estimation of the ESD risk in flight conditions. This remains challenging since the surface state of materials after a while in orbit is not yet fully known nor understood. Ultra-violet radiation from the sun, redeposited outgassed materials along with temperature variations, damage from energetic and not so energetic charged particles, exposure to electrical propulsion products if any, all modify the surface material. To be applicable to recently launched spacecraft, further tests could be performed to correlate these low coverage electron emission studies with a careful study of the outgassing rate of these materials, with careful attention paid to stimulated desorption processes due such mechanisms.

Depending on the outgassing levels expected during flight, one would use one EEY curve instead of one another. It shows the necessity to define an envelope of EEY curves to be used as parametric inputs to global scale numerical simulations, especially for materials that cover large areas. Therefore, EEY must be measured 1/ after representative sample storage conditions (including air hygrometry); 2/ under UHV; and 3/ after baking to the expected maximal flight temperature. Depending on the mission, however, actions should be taken during EEY measurements to remove contaminants according to expected flight conditions and possibly to account for ageing effects under radiative environment. Experimentally, this can be achieved using ion sputtering. In any case, it is highly recommended to keep track of sample surface state during tests.

Finally, the numerical models of SPIS should include the Dekker model in the future to better account for the actual shape of EEY.

Fig. 1 .

 1 Fig. 1. ONERA/DEESSE chamber apparatus

Fig. 2 .

 2 Fig. 2. Ground material ageing protocolSeveral sample storage and operating conditions have been applied in order to represent ageing under atmospheric conditions and space conditions, see Fig.2. The PNC and the SiO 2 coating have been aged in four steps:• step 1 : removal of contaminants from the surface with a sputtering flux of 500 eV Argon ions perpendicular to the samples, aiming at restoring the original pristine sample. • steps 2 : sample storage under atmospheric laboratory conditions for 40 and 65 days for SiO 2 and PNC, respectively. Temperature was kept at 23±0.5 °C and moisture level at [45-65 %]. • steps 3 : sample storage under atmospheric laboratory conditions for 62 and 41 days for SiO 2 and PNC, respectively (i.e. for a duration of around 100 days in total for both samples). • step 4 : sample storage under high humidity level representative of conditions met before launch at Kourou, French Guyana (10 days). Humidity level of [93%] was introduced at 40 °C.

Fig. 3 .Fig. 4 .

 34 Fig. 3. XPS spectrum measured on the SiO2 after step 1

Fig. 5 .

 5 Fig. 5. Evolution of the EEY of SiO2 as a function of storage conditions.

Fig 6 .

 6 Fig 6. Evolution of Emax on SiO2 as a function of carbon concentration (contamination).

Fig. 7 .

 7 Fig. 7. Evolution of the elementary PNC surface composition (% of silicon, carbon and oxygen) as function of the storage conditions

Fig. 8 .

 8 Fig. 8. Evolution of the EEY of PNC as a function of storage conditions.

Fig. 9 .

 9 Fig. 9. Vaughan's and Dekker's models fits of the SiO2 normalized EEY at step 4.

Fig. 10

 10 presents the geometry used to model a geosynchronous telecom spacecraft. It is composed of a central hub of dimensions 4.5 m × 2.5 m × 2.5 m covered with the conductive black paint (PNC) mentioned above, except the North and South faces covered partly with indium tin oxide (ITO) and partly with optical solar reflectors (OSR). Four antenna reflectors made of conducting material (mimicked by graphite material) are located around the hub. The larger antenna is circular with a diameter of 3.6 m. Six solar panels are deployed. The size of each panel is 5 m × 3.6 m. The front side is made of coverglass and modelled with SiO 2 material used in the experimental campaign described above. The rear side of the solar panels is made of PNC. The total size of the spacecraft is 39 m.

Fig. 10 .

 10 Fig. 10. Telecom spacecraft geometry

  paint on a solar cell cover glass and an OSR surface are 280 seconds and 210 seconds, respectively.

Fig. 11 .

 11 Fig. 11. Surface potential obtained with SCATHA 3M and with material properties measured after step 1.

  Argon gas is injected through a micro leakage valve and atoms are excited by microwaves generated by a microwave magnetron with a 2.45GHz frequency. The Omicron hemispherical electron analyzer is used. It can record spectra from 2 to 2000eV. Prior to measurements, the tank was baked to 180°C for 48 hours.

	effect (electron beam induced surface modification). A Tectra
	ion gun used for surface sputtering has an energy range from Spacecraft worst-case surface charging: On the 50eV to 5000eV.
	importance of measuring the electron emission
	yield under representative environmental
	conditions
	J.-C. Mateo-Velez, M. Belhaj, S. Dadouch, P. Sarrailh, S. L. G. Hess, D. Payan
	E

ACKNOWLEDGMENTS

This work was funded by CNES R&D program 2017. The authors thank M. Villemant from ONERA for designing figure 1.


Sarah Wassermann

Thibaut Cuvelier

Pavol Mulinka

Pedro Casas

ADAM & RAL: Adaptive Memory Learning and Reinforcement Active Learning for Network Monitoring

Keywords: Stream-based Machine Learning, Active Learning, Reinforcement Learning, ADWIN, Network Attacks, MAW-ILab

Network-traffic data commonly arrives in the form of fast data streams; online network-monitoring systems continuously analyze these kinds of streams, sequentially collecting measurements over time. Continuous and dynamic learning is an effective learning strategy when operating in these fast and dynamic environments, where concept drifts constantly occur. In this paper, we propose different approaches for stream-based machine learning, able to analyze network-traffic streams on the fly, using supervised learning techniques. We address two major challenges associated to stream-based machine learning and online network monitoring: (i) how to dynamically learn from and adapt to non-stationary data and patterns changing over time, and (ii) how to deal with the limited availability of ground truth or labeled data to continuously tune a supervised learning model. We introduce ADAM & RAL, two stream-based machine-learning approaches to tackle these challenges. ADAM implements multiple stream-based machine-learning models and relies on an adaptive memory strategy to dynamically adapt the size of the system's learning memory to the most recent data distribution, triggering new learning steps when concept drifts are detected. RAL implements a stream-based active-learning strategy to reduce the amount of labeled data needed for streambased learning, dynamically deciding on the most informative samples to integrate into the continuous learning scheme. Using a reinforcement learning loop, RAL improves prediction performance by additionally learning from the goodness of its previous sample-selection decisions. We focus on a particularly challenging problem in network monitoring: continuously tuning detection models able to recognize network attacks over time. By continuously learning from and detecting concept drifts within real network measurements, we show that ADAM & RAL can continuously achieve high detection accuracy and limit the amount of training data needed to detect attacks over dynamic network data streams.

I. INTRODUCTION

Network-traffic monitoring and analysis is paramount to understand the functioning of complex large-scale networks, especially to get a broader and clearer visibility of unexpected events. One of the major challenges faced by online networkmonitoring applications is the processing and analysis of large amounts of heterogeneous and fast incoming networkmonitoring data. This sort of data usually comes in the form of high-speed streams, which need to be rapidly and continuously processed. In this context, detecting and adapting to strong variations in the underlying statistical properties of the modeled data makes data-stream analysis a very difficult task.

The application of machine-learning models to networksecurity and anomaly-detection problems has largely increased in the last decade; however, the general approach in the literature still considers the analysis as an offline learning problem, where models are trained once and then applied to the incoming measurements. This approach is very restrictive when dealing with highly dynamic environments, where concept drifts -i.e. changes in the underlying properties of the prediction target -occur often and previous knowledge rapidly becomes obsolete. An additional challenge of learning in inthe-wild networking scenarios is the lack or limited availability of ground truth or labeled data for training purposes. Labeling new incoming data is often an expensive and cumbersome process -especially when done manually and in an online fashion -, and not all data samples are equally valuable.

In this paper, we investigate stream-based approaches to machine-learning-based network security, using different algorithms for the analysis of continuously evolving data. Stream machine-learning analysis consists of processing one data instance at a time, inspecting it only once, and as such, using a limited amount of memory; stream approaches work in a limited amount of time, and have the advantage of being able to perform a prediction at any point in time during the stream.

We introduce novel stream-based, continuous learning strategies to deal with the aforementioned challenges; we conceive, describe, and evaluate ADAM & RAL, two streambased machine-learning approaches to deal with (i) concept drifts in the stream of network measurements and (ii) limited availability of labeled, ground-truth measurements. ADAM relies on simple data-distribution change-detection algorithms to dynamically adapt the learning memory of different streambased machine-learning models to the most recent data distribution, triggering new learning steps when concept drifts are detected. RAL consists of a stream-based active-learning strategy to reduce the amount of labeled data needed for learning, dynamically deciding on the most informative measurements to integrate into the continuous learning scheme. Active learning aims at labeling only the most informative samples to reduce the overall training cost. There is an assorted list of data-querying strategies and algorithms to decide which data samples should be labeled [START_REF] Settles | Active Learning Literature Survey[END_REF]; among them, the most popular strategy is based on uncertainty sampling, which uses the model-prediction uncertainty for the corresponding sample to decide whether to query its label or not. The higher the uncertainty of the model for a given sample, the more interesting the label of this sample becomes for adapting the model. RAL improves model training and prediction performance by additionally learning from the goodness of its previous sample-selection decisions, using a reinforcementlearning scheme. We make RAL freely available on GitHub as a Python package 1 .

We evaluate the performance of the proposed approaches on the detection of different types of network attacks and anomalies, using real network measurements collected at the WIDE backbone network, relying on the well-known MAW-ILab dataset for attack labeling [START_REF] Fontugne | MAWILab: Combining Diverse Anomaly Detectors for Automated Anomaly Labeling and Performance Benchmarking[END_REF]. Results not only show that particular stream-based machine-learning models are able to keep up with important concept drifts in the underlying network data streams while keeping high detection accuracy, but also that it is possible to drastically reduce the amount of labeled data with stream-based active-learning approaches by relying on reinforcement-learning principles.

The remainder of this paper is structured as follows. Section II presents an overview on the related work. Sections III and IV introduce the proposed ADAM and RAL approaches. Section V presents evaluation results on the continuous detection of network attacks and anomalies on real network measurements, using both ADAM and RAL with different machine-learning models. Finally, Section VI concludes the paper.

II. STATE OF THE ART

The application of machine learning to networking problems has been largely explored in the literature [START_REF] Boutaba | A Comprehensive Survey on Machine Learning for Networking: Evolution, Applications and Research Opportunities[END_REF]. There are a couple of extensive surveys on any-domain anomaly-detection techniques [START_REF] Chandola | Anomaly Detection: A Survey[END_REF] as well as network-oriented anomaly detection [START_REF] Ahmed | A survey of network anomaly detection techniques[END_REF], [START_REF] Zhang | A Survey of Anomaly Detection Methods in Networks[END_REF], including machine-learning-based approaches. We refer the interested reader to [START_REF] Boutaba | A Comprehensive Survey on Machine Learning for Networking: Evolution, Applications and Research Opportunities[END_REF] for a detailed survey on the different machine-learning techniques commonly applied to network-traffic analysis. There are multiple recent papers on the application of machine-learning models to networksecurity and anomaly-detection problems [START_REF] Casas | Network security and anomaly detection with Big-DAMA, a big data analytics framework[END_REF], [START_REF] Vanerio | Ensemble-learning Approaches for Network Security and Anomaly Detection[END_REF]- [START_REF] Casas | Super learning for anomaly detection in cellular networks[END_REF]. In [START_REF] Casas | Network security and anomaly detection with Big-DAMA, a big data analytics framework[END_REF], we analyze and benchmark big-data-analytics frameworks for large-scale network-traffic monitoring and analysis. In [START_REF] Vanerio | Ensemble-learning Approaches for Network Security and Anomaly Detection[END_REF], we compare the performance of standard, offline machinelearning models for network security in fixed-line networks, further studying more complex and robust models based on ensemble-machine-learning techniques. Wireless-network monitoring using similar techniques is studied in [START_REF] Casas | Super learning for anomaly detection in cellular networks[END_REF]. In [START_REF] Casas | GML learning, a generic machine learning model for network measurements analysis[END_REF], we introduce GML learning, a generic machine-learning model for network-measurements analysis, which achieves high accuracy for many different network-analysis problems. However, all these approaches consider the offline analysis of network measurements, in batch mode.

The specific application of stream-based machine-learning approaches to network security and anomaly detection is by 1 https://github.com/SAWassermann/RAL far more limited; a relevant and representative example linked to current research is presented in [START_REF] Carela-Español | A streaming flow-based technique for traffic classification applied to 12+1 years of Internet traffic[END_REF], where the authors evaluate stream-based traffic-classification approaches based on Hoeffding adaptive trees [START_REF] Domingos | Mining high-speed data streams[END_REF], using MAWILab data and the MOA machine-learning toolkit, as we do in this work.

Naturally, the data-stream machine-learning domain has a long-standing tradition and many interesting references are worth mentioning when considering the application and evaluation of stream-based machine-learning models; these cover general problems related to the learning properties for streambased algorithms [START_REF] Domingos | Catching Up with the Data: Research Issues in Mining Data Streams[END_REF], [START_REF] Stonebraker | The 8 requirements of real-time stream processing[END_REF], the mining and evaluation processes when dealing with massive datasets [START_REF] Hulten | Mining massive data streams[END_REF], the identification of model-evaluation issues [START_REF] Gama | Issues in evaluation of stream learning algorithms[END_REF], as well as propositions of general frameworks for data streaming [START_REF] Gama | On evaluating stream learning algorithms[END_REF]. Of particular relevance for stream-based machine-learning-model evaluation are the problems of class imbalance and concept drift, which are extensively addressed in [START_REF] Hoens | Learning from streaming data with concept drift and imbalance: an overview[END_REF].

When it comes to active learning, there is a vast literature in the field. For example, [START_REF] Žliobaitė | Active Learning with Evolving Streaming Data[END_REF], [START_REF] Žliobaitė | Active Learning with Drifting Streaming Data[END_REF] present three simple approaches for active learning. Their proposed Randomized Variable Uncertainty approach tackles the problem of streambased active learning, using the model's prediction uncertainty to decide whether to query and trying to detect concept drifts by randomizing the certainty threshold used for labeling decisions. [START_REF] Xu | Active learning over evolving data streams using paired ensemble framework[END_REF] develops an active-learning algorithm with two different classifiers: one "reactive" and one "stable". The stable classifier is trained on all available labeled instances, while the reactive one is trained based on a window of recent instances. [START_REF] Ienco | Clustering Based Active Learning for Evolving Data Streams[END_REF] presents an active-learning technique based on clustering and prediction uncertainty. [START_REF] Krawczyk | Active and adaptive ensemble learning for online activity recognition from data streams[END_REF] conceives an approach relying on a modification of the Naïve Bayes classifier to update the different learners through the queried samples. In particular, they use one-versus-one classifiers to tackle multi-class problems and update the weights of the different classifiers by comparing their predictions to the ground truth. Their technique behaves similarly to RAL. However, the major difference is that [START_REF] Krawczyk | Active and adaptive ensemble learning for online activity recognition from data streams[END_REF] uses information about the classifiers' prediction certainty (without considering the corresponding weights) to adapt the minimum threshold for querying the oracle, while we rely on the usefulness of the decisions taken by RAL to tune the system according to the data stream.

Finally, ideas from reinforcement learning have already infused into the active-learning domain, but mostly into poolbased approaches. In [START_REF] Baram | Online Choice of Active Learning Algorithms[END_REF], [START_REF] Hsu | Active Learning by Learning[END_REF], authors rely on the multiarmed bandit paradigm. [START_REF] Hsu | Active Learning by Learning[END_REF] develops ALBL, which uses a modified version of EXP4 [START_REF] Auer | The Nonstochastic Multiarmed Bandit Problem[END_REF], a weight-updating rule, in order to attribute adaptive weights to different learners based on rewards; the learner to use is then determined through these weights and uses its uncertainty measure to select the samples in the pool to hand to the oracle. The approach described in [START_REF] Baram | Online Choice of Active Learning Algorithms[END_REF] is similar to the one in [START_REF] Hsu | Active Learning by Learning[END_REF], except for the rewardcomputation scheme. The algorithm presented in [START_REF] Song | Stream-based Online Active Learning in a Contextual Multi-Armed Bandit Framework[END_REF], [START_REF] Song | A Contextual Bandit Approach for Stream-Based Active Learning[END_REF] relies on the same principles as the approach we are proposing, but tackles a different problem: Song's goal is to introduce an active-learning component into a contextual-bandit problem, while we are aiming at solving an active-learning problem by using contextual bandits.

III. ADAM -STREAM LEARNING WITH ADAPTIVE MEMORY

We start by introducing ADAM, relying on an ADAptive Memory strategy. Given that we are dealing with continuous data analysis, the approach must be able to identify and adjust to the variation of the statistical properties of the analyzed data, detecting sudden statistical changes or concept drifts. To do so, ADAM relies on ADWIN, a dynamically adjusting window-based approach introduced in [START_REF] Bifet | Learning from Time-Changing Data with Adaptive Windowing[END_REF], which maintains a window of variable size containing training samples. The algorithm automatically grows the window when no change is apparent, and shrinks it when the statistical properties of the stream changes. ADWIN automatically adjusts its window size to the optimum balance point between reaction time and small variance; using the ADWIN adaptation strategy, ADAM implements four stream-based machine-learning algorithms popular in the literature, including incremental k-NN, Hoeffding adaptive trees (HAT), adaptive random forests (ARF) [START_REF] Gomes | Adaptive random forests for evolving data stream classification[END_REF], and SVM through stochastic gradient descent (SGD).

A. Adaptation Strategy

One of the key features of a stream-based machine-learning model is that of continuously adapting to the changes of the underlying statistics describing the current data under analysis, by periodically re-training or re-calibrating. To deal with evolving data, one needs to define strategies for: firstly, detecting when changes occur; secondly, deciding which data to use for a subsequent model re-calibration (or, more generally, keeping updated sufficient statistics); and finally, retraining or re-calibrating the model when a significant change has been detected. Most strategies use variations of a simple sliding window approach, where a window containing the most recent measurements is kept and constantly updated, removing older measurements based on some specific criteria.

The ADWIN algorithm keeps a sliding window W with the most recently observed measurements x n . At each time n, instance x n is generated according to an unknown probability distribution D n . Let m be the length of W , μW the (computed) average of the measurements in W . The idea of ADWIN is straightforward: whenever two large enough subwindows of W exhibit distinct enough averages, we conclude that the corresponding expected values are different, and the older portion of the window is dropped. Algorithm 1 briefly describes the algorithm, where μW0 and μW1 are the averages of the instances in W 0 and W 1 , respectively. Note that it is not needed to define W 0 and W 1 sizes, as the values of those are decided by the algorithm itself. ADWIN is therefore a simple statistical test for different distributions in W 0 and W 1 , which checks whether the observed average in both sub-windows differs by more than the threshold . This threshold is defined based on the global error of the statistical test and the lengths m 0 and m 1 of the corresponding sub-windows. for each n > 0 do 4:

W ← W ∪ {x n } add x n to the head of W 5: if μW0 -μ W1 ≥ for some split of W = W 0 •W 1 then 6:
drop instances from the tail of W

B. Concept Drift Detection

Concept drift happens when the statistical properties of the analyzed dataset abruptly shift in time [START_REF] Gama | A survey on concept drift adaptation[END_REF]. Different change-detection algorithms can be applied to identify the times when the probability distribution of a stochastic process or time series changes. In our problem, such a detection has to be performed in an online manner, i.e. without assuming that the statistics of the complete time series are known in advance. The ADWIN algorithm is by design an online change-detection algorithm. In this paper, we consider an additional change-detection algorithm to analyze the considered dataset: the Page-Hinkley test (PHT) [START_REF] Page | Continuous Inspection Schemes[END_REF]. PHT is a standard statistical test for change detection, commonly used in timeseries analysis. In a nutshell, this test is a sequential adaptation of a test for an abrupt change of the average of a Gaussian stochastic process, and it allows efficient detection of changes in the usual behavior of a process.

IV. RAL -STREAM LEARNING WITH ACTIVE REINFORCEMENT

RAL relies on reinforcement-learning principles, using rewards and contextual-bandit algorithms [START_REF] Auer | The Nonstochastic Multiarmed Bandit Problem[END_REF], as well as prediction uncertainty. The overall idea is summarized in Figure 1. The intuition behind the different reward values is that we attribute a high (positive) reward in case the system behaves as expected, and a low (negative) one otherwise, to penalize it. RAL obtains rewards/penalties as soon as it is asking for ground truth. In a nutshell, it earns a positive reward ρ + in case it queries the oracle and would have predicted the wrong label otherwise (i.e. the system made the right decision to ask for the ground truth: the sample is deemed informative) and a penalty ρ -(i.e. a negative reward) when it asks the oracle even though the underlying classification model would have predicted the correct label (i.e. querying was unnecessary). The rationale for using reinforcement learning is that RAL learns not only based on the queried samples themselves, but also from the usefulness of its decisions. The objective function to maximize is the total reward: n i=1 r n , where r n is the nth reward (ρ + or ρ -) obtained by RAL.

The conceived system additionally makes use of the prediction certainty of the underlying classification model(s). The prediction certainty is defined as the highest posterior classification probability among all possible labels for sample x. More formally, the prediction certainty of a model is equal to max ŷ P (ŷ|x), with ŷ being one of all the possible labels for x. The rationale behind this design choice is that the model's Algorithm 2 RAL algorithm.

1: procedure RAL(x, E, α, θ, ε, η) 2:

x: sample to treat α ← updateDecisionPowers(r, E, decisions, com-mitteeDecision, α, η)

18: θ ← min θ 1 + η × 1 -2 r ρ - , 1 19: function UPDATEDECISIONPOWERS(r, E, decisions, committeeDecision , α, η) 20:
for e ∈ E do 21:

if decisions[e] = committeeDecision then return (ρ -if ŷ(x) = y else ρ + ) prediction uncertainty is an appropriate proxy for assessing the usefulness of a data point. Combining the reward mechanism with the model's uncertainty allows us to tune the sampleinformativeness heuristic to better guide the query decisions. Also inspired by the bandit literature [START_REF] Watkins | Learning from Delayed Rewards[END_REF] and to better deal with concept drifts in the data, we implement an ε-greedy policy, which improves the data-space exploration; we sample a uniform-probability distribution, and if this value is below a certain threshold ε, the system queries the oracle, ignoring the decision of RAL's classification models. We refer to this as the ε-scenario. This ensures that we have a good chance of detecting potential concept drifts: without this policy, the system could end up being too confident about its predictions, and thus never ask the oracle again, even though its estimations are wrong.

Next, we present the details of a committee or multiclassifier version of RAL, relying on multiple models. Nevertheless, it is very easy to use RAL with a single machinelearning model, even if we do not use it in the evaluations of this paper. We provide some comments on this by the end of the section.

The algorithm behind RAL is summarized in Algorithm 2.

Our approach is inspired by contextual bandits [START_REF] Auer | The Nonstochastic Multiarmed Bandit Problem[END_REF]. We rely on a set of experts (i.e. different machine-learning models), referred to as a committee. Each expert gives its opinion for the sample to consider: should the system ask the oracle for feedback or is the expert confident enough about its prediction? To assess a model's prediction certainty, we rely on a certainty threshold θ: if the model's certainty is below θ, the expert is too uncertain about the prediction to make and thus it advises that RAL asks the ground truth. The query decision of the committee takes into account the opinions of the experts, but also their decision power: if the weighted majority of the experts votes against querying, RAL will rely on the label prediction provided by the committee, used in the form of a voting classifier. The decision power of each expert gets updated such that the experts which agree with the entire committee are obtaining more power in case that particular decision is rewarding, i.e. informative (otherwise, these experts get penalized). These weights are updated through the EXP4 rule [START_REF] Auer | The Nonstochastic Multiarmed Bandit Problem[END_REF], with a learning rate η. RAL does not update the decision powers of the different learners in the ε-scenario: the committee did not take the querying decision and thus the weights of the models should not be impacted by this querying action.

The computation of the reward is carried out every time the committee decided to query (i.e. not in the ε-scenario). RAL therefore gets rewarded with ρ + > 0 when it queried the oracle and asking was informative (i.e. the voting classifier would have predicted the wrong label). Conversely, RAL is penalized with ρ -< 0 if the system used the oracle because the committee decided to do so, even though the underlying classifier would have predicted the correct class.

As an additional step, to ensure that RAL adapts as best as possible to the data stream, we do not only tune the weights of the committee members based on rewards, but also the uncertainty threshold θ, denoted in the remainder of this section as θ n to stress that it is influenced by the n -1 samples observed so far. Again, as for the decision powers, θ n is not updated in the ε-scenario. The update rule of θ n we implemented for our tool is written as follows:

θ n ← min θ n-1 × 1 + η × 1 -2 rn ρ - , 1
We now detail the reasoning behind the selection of the update policy used by RAL. We are looking for an update rule of the form

θ n ← min {θ n-1 × (1 + f (r n )) , 1}
where f (r n ) = 1 -exp (a × r n ). The threshold should increase slightly when the reward is positive, conversely when the reward is negative. More formally, the update policy should satisfy the following properties: 1 -θ n should decrease fast in case r n is negative, as this indicates the system queries too often, thus is doing poorly. Therefore, θ n should be adapted fast to improve RAL's performance. 2 -θ n should slightly increase when r n is positive, so that the system does not always keep decreasing the threshold and avoids that θ n drops to 0. The model was right to ask for more samples, and thus the threshold should be increased. Nevertheless, as the system is currently doing well, we do not want the threshold to be too reactive to the queries. 3 -f must have two extrema: a minimum at ρ -< 0 and a maximum at ρ + > 0. 4 -θ n represents a probability. θ n = 0 is not acceptable due to the product form of the update policy, thus the values of θ n must be in the interval (0, 1]. 5 -f (r n ) must be in the interval (-1, 1] to ensure that θ n takes values corresponding to a probability. We exclude -1 from the allowed range of values to avoid that θ n drops to 0.

Properties 1 and 2 lead us to choose the family of functions f : x → 1 -exp (a × x) parameterized by a. Property 5 can be translated into an equation to determine this parameter:

lim r→ρ -f (r) = 1 -exp (a × ρ -) = -1.
After solving this equation, we get a = ln 2 ρ -. As f is strictly increasing, and because a is nonpositive, f will have a maximum when r n = ρ + (thus satisfying property 3). Note that, in order to satisfy property 5, ρ + must be chosen such that f (ρ + ) ≤ 1.

As a final step, we introduce an additional hyper parameter to the update rule, namely the learning rate η. This rate aims at smoothing the evolution of the threshold θ n , i.e. avoiding that θ n changes too dramatically with a single query. We thus have the following update rule:

θ n ← min θ n-1 × 1 + η × 1 -2 rn ρ - , 1
We restrict the values of η to the range (0, 1). Indeed, we still must satisfy property 5 (a value of 1 would violate this one) and η = 0 would lead to an nonreactive system, as the threshold would never adapt.

We acknowledge that RAL includes a non-negligible number of hyperparameters which should be well chosen in order to obtain the best results. While we do not have any rule of thumb on how to define exact values, the following guidelines help RAL learn from the streaming data: 1 -The initial value of θ should be set to a high one (i.e. close to 1) when the number of possible labels is low, to avoid that the model is always too certain about its prediction for the encountered samples. 2 -ε should be higher when dealing with more dynamic datasets, to increase the probability of accurately grasping concept drifts; in general, we would advise using values in the range of 1 to 5%.

3 -η should be small to avoid changing the decision powers of the different learners, i.e. α, and θ n too abruptly; we would advise values below 0.1. 4 -There is no specific range of values for ρ ± which works better than others and these values should be picked considering the situation in which RAL is used: if unnecessary queries are a major issue, one should set ρ -such that its absolute value is much higher than the one of ρ + .

To conclude, and as we said before, RAL can also easily be used with a single classifier instead of a committee of learners. Transforming the committee version into a single-classifier one is straightforward. In that case, RAL becomes very lightweight and the only element of the system that allows it to efficiently adapt to and learn from the data stream is the variation of the uncertainty threshold θ, by relying on the rewards.

V. CONTINUOUS DETECTION OF NETWORK ATTACKS

To evaluate the performance of the proposed algorithms and adaptation strategies, we consider the detection of diverse types of network attacks in real network-traffic measurements collected at the WIDE backbone network, using the wellknown MAWILab dataset for attack labeling. MAWILab is a public collection of 15-minute network-traffic traces [START_REF] Fontugne | MAWILab: Combining Diverse Anomaly Detectors for Automated Anomaly Labeling and Performance Benchmarking[END_REF] captured every day on a backbone link between Japan and the US since 2001. Building on this repository, the MAWILab project uses a combination of four traditional anomaly detectors (PCA, KL, Hough, and Gamma) to partially label the collected traffic.

Given the different nature and goals of ADAM and RAL, we rely on two different evaluation strategies, each of them adapted to the specific challenges tackled. Next, we describe the dataset used in the evaluations, as well as the employed evaluation strategies, along with the obtained results.

A. Data Description

The traffic studied in this paper spans two weeks of packet traces collected in late 2015. From the labeled anomalies and attacks, we specifically focus on those which are detected simultaneously by all four MAWILab detectors. We consider in particular two types of attacks: flooding attacks, and distributed network scans. We train different models to detect each of these attack types separately, thus each detection approach consists of two different binary detectors.

To detect these attacks, we consider a slotted, time-based evaluation. To do so, we split the traffic traces in consecutive time slots of five seconds each, and compute a set of features describing the traffic in each of these slots. In addition, each slot i is assigned a label l i , consisting of a binary vector l i ∈ {0, 1} 2 which indicates at each position if anomaly of type j = 1, 2 is present (l i,j = 1) or not (l i,j = 0) in the current time slot. We compute a large number n of features describing a time slot, using traditional packet measurements including traffic throughput, packet sizes, IP addresses and ports, transport protocols, flags, etc. The total set accounts for n = 245 features, which are computed for every time slot i. Besides using traditional features such as min/avg/max values of some of the input measurements, we also consider the empirical distribution of some of them, sampling it at many different percentiles. This provides much richer information, as the complete distribution is taken into account. We also compute the empirical entropy H(•) of these distributions, reflecting the feature dispersion.

B. ADAM Evaluation Strategy and Performance

A commonly used evaluation scheme within the datastream-mining domain is the well-known prequential approach. Each instance is first used to test a model, and then to train or update it. Prequential evaluation can be used to measure the accuracy of a model since the start of the evaluation, by keeping in memory the complete history of instances and evaluating the model on each new instance, but it is generally applied using sliding windows or decaying factors -as we do it in ADAM -, which forgets previously seen instances in the model-update process and focuses on those instances in the current sliding window or learning memory. As opposed to more traditional k-fold cross-validation, which is generally used in the evaluation of offline machine-learning models based on k shuffles of the complete dataset, prequential cross-validation works on a single stream of data using only one model: its assessment of the stream-based model tends to be weaker due to this.

To avoid this weakness, we evaluate ADAM following a new strategy to evaluate stream-based algorithms [START_REF] Bifet | Efficient Online Evaluation of Big Data Stream Classifiers[END_REF], using prequential k-fold cross-validation; this strategy is basically an adaptation of k-fold cross-validation to the streaming setting, and assumes we have k different models derived from the algorithm we want to evaluate, running in parallel. Each time a new sample arrives, it is used for testing one of the k models selected randomly, and is then used for training by all the other models. As evaluation metric, we take the attack-detection accuracy (ACC).

A commonly applied approach to evaluate the performance of stream-based algorithms is to benchmark them against their corresponding offline, batch implementations. Therefore, we use as baseline the results obtained for the corresponding batch-based algorithms, including k-NN, Hoeffding tree (HTbatch), random forest (RF-batch), and SVM. We compute the difference between the batch accuracy and the prequential one to compare the performance of stream algorithms with respect to their batch variants.

We use the MOA machine-learning library [START_REF] Bifet | MOA: Massive Online Analysis[END_REF] to perform the analysis, including both the hyperparameter calibration (using a grid-search procedure) and the model training and evaluation. MOA is specifically designed for stream-based machine-learning approaches.

First of all, we study the variation of the statistical properties of the considered dataset, in particular detecting concept drifts with the Page-Hinkley test. Figure 2 depicts the cumulative number of changes observed in the dataset, as well as the times when those changes are detected. The test detects 14 abrupt changes during the total measurement time span. The frequency of changes significantly increases in the last third of the dataset, with more than 10 changes detected in the last 4 days. Concept drifts occur from modifications of the underlying characteristics of the prediction target.

Concept drifts can be used to explain sudden shifts in the performance of algorithms as depicted in Figure 2. For example, the window size of the algorithm changes when concept drifts are detected, and thus could have an important impact on performance.

We evaluate the performance of the learning algorithms using ADAM in two binary-classification scenarios -one for each attack type, resulting in two sets of results for each tested algorithm. Figure 3 reports the performance results for each attack type, considering detection accuracy as the performance metric, and using the batch-algorithm accuracy as baseline. More precisely, we report the difference between the 10-fold prequential and batch accuracy. The prequential CV-performance evaluation shows that both ARF and SGD models rapidly converge to the batch-based accuracy results, with minimum accuracy variations when concept drifts occur. The SGD model performs slightly better, even outperforming the batch-based performance. On the other hand, both the k-NN and HAT models do not exhibit any convergence and results tend to oscillate around the batch-algorithm baseline. In the case of HAT, we can highlight the correlation between the detected concept drifts and the performance variations of the model. Interestingly, the HAT model is the one achieving the highest accuracy of the four models (up to 30% improvement with respect to the baseline), but cannot maintain such a performance constantly in time.

C. RAL Evaluation Strategy and Performance

To showcase the performance of RAL, we evaluate and compare it to a state-of-the-art algorithm for stream-based active learning, as well as against a very basic random sampling approach (RS). In particular, we compare RAL to the Randomized Variable Uncertainty (RVU) technique proposed in [START_REF] Žliobaitė | Active Learning with Evolving Streaming Data[END_REF], [START_REF] Žliobaitė | Active Learning with Drifting Streaming Data[END_REF], as this approach also heavily relies on the uncertainty of the underlying machine-learning models to take the querying decisions.

For each benchmarked algorithm, we proceed as follows: first, we subdivide the considered datasets into three consecutive, disjoint parts, i.e. the initial training set, the streaming Ping flood data, and the validation set. The validation set consists of the last 30% of the dataset, the initial training set is a variable fraction of the first samples (varying between the first 0.5%, 1%, 2%, 5%, 10%, and 15%), and the streaming part includes all the remaining samples not belonging to the other two subsets. We then train a model on the initial training set and check its accuracy on the validation part -we refer to this as the initial accuracy. Next, we run the specific active-learning algorithm on the streaming part and let it pick the samples it decides to learn from. We retrain the model after each new queried label. Finally, we evaluate the final model -trained on the initial training set plus the selected samples -, again on the validation set, and analyze this prediction accuracyreferred to as the final accuracy.

In the context of this evaluation, we implement for both RAL and RVU the budget mechanism presented in [START_REF] Žliobaitė | Active Learning with Evolving Streaming Data[END_REF], based on the ratio between the number of queries and the total number of samples observed so far; the system is allowed to issue queries to the oracle as long as this ratio is below a certain threshold, i.e. the budget. For RS, we use a budget indicating the exact number of samples to ask feedback for. For each attack type, we set it to the highest average number of queried samples by either RAL or RVU among all the tests with all the considered initial-training-set sizes.

Similarly to the evaluation of ADAM, all tests are repeated 10 times, and we report both average accuracy and standard errors. For RAL, we indicate the average number of queries performed due to the uncertainty of the underlying model, as well as those issued through the ε-greedy mechanism. For comparison purposes, we also report the average number of queries issued by RVU. We set the values of ρ + and ρ -to 1 and -1, respectively. Based on grid search with the training data in the ranges presented in Section IV, we set θ to 0.9, ε to 2.5%, and η to 0.01. In the case of RVU, we set its parameters based on those recommended in [START_REF] Žliobaitė | Active Learning with Drifting Streaming Data[END_REF]. Finally, we set the budget to 0.05 for both approaches. Results obtained for the Ping-flood and the Netscan attacks are presented in Figures 4 and5. The reported all-streaming accuracy refers to the accuracy obtained by the model in case it queries all the samples seen in the stream. In the context of this evaluation, we use the committee version of RAL. More precisely, the committee is a voting classifier composed of a k-NN model with k = 5, a decision tree, and a random forest with 10 trees. We use the same voting model for RVU and RS. Figures 4(a) and 4(b) clearly show that RAL outperforms both RVU and RS on average. A striking example is the result for the Netscan detection, where RAL obtains final accuracies which are up to 10 percentage points higher than the ones of RVU and RS for the two smallest initial-training-set sizes. To our surprise, RVU is often outperformed by RS. Finally, the Ping-flood-detection analysis shows that the three approaches often yield a final accuracy higher than the all-streaming one, underlining that learning from the entire data stream does not necessary output the best possible accuracies. This could be explained by the high number of concept drifts in the data.

Last, when it comes to the number of queried samples, we see that RAL queries on average significantly less often than RVU -between 20% and 25% fewer queries -, and a non-negligible part of these queries are due to the model's uncertainty, suggesting that the samples picked by RAL for its learning purposes are wisely chosen. Also note how useful turns out to be the ε-greedy policy, as the additional exploration capability helps better deal with the concept drifts in the data, contributing to the better results showed in Figure 4.

The initial accuracy is constant for the two different MAW-ILab attack subsets. This is due to the fact that the first 15% of these datasets consist of points with the same label (more precisely, they represent an attack).

One could wonder whether the performance gain by RAL is worth the complexity of the system. Even though the accuracy gain might not be very significant, RAL's querying strategy has additional advantages over the two other techniques. For instance, RS does not take into account the uncertainty of the model nor query usefulness, meaning that there is a risk to miss interesting samples. Indeed, querying the ground truth when the model is uncertain helps discover underexplored regions the model can learn from and RAL is additionally guided by its reward mechanism. In the specific case of the MAWILab dataset, RS would probably miss interesting attack samples, while RAL has a better chance of querying ground truth for these data points and better learn how to detect attacks. Another advantage of RAL over RVU, besides its better results shown above, is that the querying decisions are influenced by the informativeness of all past queries, not only their sheer execution; RVU does not take that information into account at all, and thus there is a risk that RVU queries unnecessary samples too often. This is especially problematic if querying is very expensive or the oracle has only limited availability.

VI. CONCLUDING REMARKS

Dynamic and adaptive-memory-based learning seems to be a promising learning strategy to adapt to very dynamic environments, where concept drifts occur often. This is a common scenario when dealing with online network-trafficmonitoring applications. We have introduced and evaluated ADAM and RAL, two stream-based machine-learning approaches to tackle important challenges when dealing with data streams. We have shown that ADAM permits to track transient changes and concept drifts along time. Indeed, using ADAM, adaptive learning algorithms can continuously achieve high detection accuracy over dynamic network data streams, when dynamically adapting their learning pace and memory to changes in the underlying statistics of the samples. We have confirmed that both adaptive random forests and SVM through stochastic gradient descent are better for the studied problem, especially in terms of robustness to concept drifts and convergence of results. We have also introduced RAL, a novel Reinforced stream-based Active-Learning approach to tackle the challenges of stream-based active learning, i.e. selecting the most valuable sequentially incoming samples to reduce the amount of learning data to label, using reinforcement-learning principles. RAL does not only learn from the data stream, but also from the relevance of its own querying decisions. RAL provides a completely different exploration-exploitation tradeoff than existing algorithms. Evaluations have shown that RAL provides very promising results, outperforming state-of-the-art techniques, providing higher accuracies with less ground truth. As an additional contribution, we make RAL freely available on GitHub. As a next step, we are currently working on the integration of ADAM and RAL approaches, using the ADWIN adaptation and concept-drift-detection strategy to improve the performance of RAL under dynamic scenarios, as well as by using a dynamic memory-length approach.

22 :

 22 α[e] ← α[e] × exp(η × r) EXP4 23: return α/ e∈E α[e] normalize each value of α 24: function GETREWARD(x, y) 25:

Figure 1 :

 1 Figure 1: Overall idea of the system.

Figure 2 :

 2 Figure 2: Page-Hinkley concept-drift detection. Changes are marked with dashed lines.

Figure 3 :

 3 Figure 3: Prequential 10-fold cross-validation accuracy evaluation. Diagrams show prequential 10-fold CV results for each algorithm for each attack type. Concept drifts detected by the Page-Hinkley test are marked with dashed lines.

Figure 4 :

 4 Figure 4: Prediction accuracy for RAL, RVU, and RS. For each of the tested datasets, RAL outperforms both techniques.

  Ping flood (RVU). (d) Netscan (RVU).

Figure 5 :

 5 Figure 5: Number of queries issued by RAL and RVU. RAL asks for much fewer samples, yet can achieve a better accuracy than RVU.


RUN UP OF SURFACE AND INTERNAL WAVES

H. Branger 1 , O. Kimmoun , N. Gavrilov , V. Liapidevskii 3 , E. Pavlova 2 3 4

1 IRPHE, CNRS, Marseille, France ; e-mail : branger@irphe.univ-mrs.fr 2 ECM, Marseille, France ; e-mail : olivier.kimmoun@ec-marseille.fr 3 LIH, SB RAS, Novosibirsk, Russia ; e-mail : liapid@hydro.nsc.ru 4 POI, FEB RAS, Vladivostok, Russia : epavlova@poi.dvo.ru

The evolution of breaking waves propagating towards the shore and more specifically the run-up phase over the swash-zone for surface as well as for internal waves is considered. The study is based on the laboratory run up experiments for surface waves performed in ECM (The Ecole Centrale Marseille), on the laboratory stratified flow experiments performed in LIH (Lavrentyev Institute of Hydrodynamics) and on the field data describing the internal wave run up from the joint LIH -POI (V.I. Il'ichev Pacific Oceanological Institute) expedition in summer 2006. The presentation is focused on the breaking and energy transfer mechanisms common to surface and internal waves in the swash-zone. The mathematical model taking into account turbulent mixing and dispersion effects is discussed.

Run up of large amplitude breaking waves

Experiments have been conducted in the glass-windowed ECM wave-tank (17m x 0.65m x 1.5m) in Marseille. A 1/15 sloping beach is mounted on the bottom. The wave maker is able to generate regular waves, freak waves (by space-time focalization technique), or valuable solitary waves. Wave profiles have been measured by a set of wave gauges, velocity measurements have been performed by Particle Image Velocimetry (PIV) methods. Synchronized measurements are done at different locations and the space/time velocity field are reconstructed over all investigated zone. The extension of the wave run up, and the celerity of the propagating bore on shore are estimated using video techniques with a vertical laser light sheet. The details of the experimental procedure can be found in [START_REF] Kimmoun | Laboratory PIV Measurements of Wave Breaking on Beach[END_REF].

Much attention is given to the correct synchronization between experiments to have a global estimation of the mean and fluctuating parts of the wave induces velocity flow. Different types of waves have been investigated: regular waves, groups of waves, extreme waves and solitary waves. Different stages of run up process from a spilling breaker to the developed turbulent bore are shown in Figure 1.

Figure 1. Run up of surface waves

Large-amplitude internal waves in shelf zone

The topic of internal waves is studied in the context of wave breaking, vortex formation and mixing in coastal waters. Nonlinear internal waves generated by tides, as well as by interaction of flows with topography, play an important role in the energy transfer from the large-scale motion to small-scale mixing. The analogy with the breaking mechanisms of surface waves is very useful to simulate the dissipation processes on shelf. Vertical and horizontal structure of near-shore stratified flows is important since some abnormal wave shapes can occur close to the breaking limits. Scotti and Pineda (2004) have recently observed bore-like structures with trapped cores in the near shore area of Atlantics. The transition from wave-like motion to the separate moving soliton-like structures containing trapped dense core is the common feature of the run up process of internal waves. It can be observed in any shelf zone with high internal wave activity as well as in laboratory experiments. In figure 2 the run up of lenses of cold water due to interaction of the internal tide front with the bottom is shown. The undisturbed depth of the thermocline is 25 meters. The run up phase of internal tide moves cold water to the shore for a long distance. Since the lenses of cold water are separate, the mean velocity of fluid in such cold cores is equal to the phase velocity of waves. The dissipation due to mixing is very effective in this region too. It means that run up of internal waves influences many dynamical processes in coastal waters (sediment transfer on the shelf, vertical mass exchange, etc).

The laboratory experiments on internal wave run up performed in the glass-windowed LIH wave-tank (3m x 0.15m x 0.6m) in Novosibirsk reveal the dependence of mixing efficiency on the thickness of dense layer before the wave front. In Figure 3 the stages of run up process of an internal wave is shown. One can see the sudden transition from a laminar flow to the high turbulent flow in the core of wave, when the depth of the dense layer before the wave vanishes. Figure 3. Run up of an internal wave (LIH experiment): the soliton-like wave (upper), the bolus formation at the boundary of the swash-zone (middle), the turbulent bore on the "dry bottom".

The mathematical model, which is an extension of the Green-Naghdi model and includes the turbulent layer, has been developed to find the criterion of wave breaking in the steady-state flow over a topography [START_REF] Liapidevskii | Breaking of waves of limiting amplitude over an obstacle[END_REF]. This model and its hyperbolic approximation is adopted to unsteady run up problems for surface and internal waves.

Figure 2 .

 2 Figure 2. Run up of internal waves in the shelf. The total depth is 16.5 m, the temperature difference between the cold core and surrounding fluid is about 10 degrees.

  The work was supported by INTAS under grant 06-9236, by RFBR under grant 05-05-64460, by RAS under grant 4.13.1 and by SB RAS under grant 06-113.