Mariam Kiran
email: mkiran@lbl.gov

Bashir Mohammed
email: bmohammed@lbl.gov

Nandini Krishnaswamy
email: nkrishnaswamy@lbl.gov

DeepRoute: Herding Elephant and Mice Flows with Reinforcement Learning

Keywords: Reinforcement learning, route optimization, path computation ?

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The rise of data hungry services such as mobile, video streaming and Cloud/Internet applications are bringing unprecedented demands to underlying network backbones [START_REF]Cisco: Trending analysis[END_REF]. Wide area networks (WANs) are investigating intelligent and e cient network management techniques to do load balancing, improve used bandwidth and overall optimize network performance. Tra c congestion can directly cause performance deterioration, such as when links are oversubscribed causing bottlenecks [START_REF] Clark | Measurement and Analysis of Internet Interconnection and Congestion[END_REF]. Many services rely on having high-throughput transfers and need high capacity links such as 100s Gbps. However, even for the busiest link, the current average utilization is only between 40-60%, to account for unanticipated peaks [START_REF]Achieving high utilization with software-driven wan[END_REF]. Tra c engineering and path computation techniques such as MPLS-TE (Multiprotocol Label Switching Tra c Engineering) [START_REF] Awduche | Requirements for tra c engineering over mpls[END_REF], Google's B4 [START_REF] Jain | B4: Experience with a globally-deployed software defined wan[END_REF] and Microsoft's SWAN (Software Driven WAN) [START_REF]Achieving high utilization with software-driven wan[END_REF] have proposed manners in which routers can greedily select routing patterns for arriving flows, both locally and globally, to increase path utilization. However, these techniques require meticulously designed heuristics to calculate optimal routes and also do not distinguish between arriving flow characteristics.

Path computation has a number of real-world networking implications. Examples such as load balancing, minimizing congestion and utilizing maximumbandwidth as some cases that can be explored as a reward. WAN networks al-low a number of pathways to exist between pairs of end-hosts.

Internet and WAN tra c usually contains a mixture of flow characteristics, such as long and short flows, which if on the same path, can have detrimental e↵ects on each other. Known as bulk long-living file transfers (elephant flows) are bandwidth-sensitive and short transfers (mice flows) are latency-sensitive, significantly impacting user experience and require innovative ways to manage them [START_REF] Wang | Adaptive path isolation for elephant and mice flows by exploiting path diversity in datacenters[END_REF]. WAN tra c usually contains a 80%:20% flow distribution (mice to elephant ratio) and if mixed on same paths, can cause queuing delays impacted by high latency and low throughput [START_REF] Akella | An empirical evaluation of wide-area internet bottlenecks[END_REF]. Isolating flows to dedicated routes [START_REF] Yan | A network management system for handling scientific data flows[END_REF] is challenging in real-time [START_REF] Wang | Adaptive path isolation for elephant and mice flows by exploiting path diversity in datacenters[END_REF] [START_REF] Trestian | Micetrap: Scalable tra c engineering of datacenter mice flows using openflow[END_REF] and has led to under-utilized paths. Researchers have attempted to recognize arriving flows to e ciently manage them [START_REF] Kiran | Understanding flows in high-speed scientific networks: A netflow data study[END_REF]. In data center networks, these are often prioritized, such as mice latency [START_REF] Carpio | Di↵flow: Di↵erentiating short and long flows for load balancing in data center networks[END_REF] [START_REF] Benson | Microte: Fine grained tra c engineering for data centers[END_REF] or elephant throughput [START_REF] Alizadeh | Data center tcp (dctcp)[END_REF] to improve data center network performance. In WAN, path computation uses optimization to calculate paths taken between end-hosts.

Leveraging research in software defined networking (SDN) and reinforcement learning, we explore this problem of path computation and finding optimal routes in general, as a path resource allocation problem. We use machine learning algorithms to learn and provide viable solutions for dynamic flow management for both elephant and mice flows.

Network routing and machine learning is not new. Broadly speaking, there are two main approaches used here [START_REF] Valadarsky | Learning to route[END_REF]:

1. optimize routing configurations by predicting future tra c conditions depending on past tra c patterns or 2. optimize routing configurations based on number of feasible tra c scenarios with aim to improve performance parameters.

While SDN's centralized control o↵ers great promise, these calculations cause overhead for high-performance networks and need global management to work, which is di cult in a large network [START_REF] Curtis | Devoflow: Scaling flow management for high-performance networks[END_REF]. Recent success of machine learning in complex decision making problems such as Alpha-Go [START_REF] Silver | Mastering the game of go with deep neural networks and tree search[END_REF], cooling datacenters and self-driving cars [START_REF] Xiao | Learning a↵ordance for autonomous driving[END_REF] suggest feasible applications to our problem. Particularly reinforcement learning (RL), actively being applied in robotics [START_REF] Tomar | Mamic: Macro and micro curriculum for robotic reinforcement learning[END_REF], allows agents to learn how to make better decisions by interacting directly with the environment. Using concepts of rewards and penalties it can learn through experience to optimize its objective function.

Revisiting the path computation challenge, in this paper we use RL to find optimal paths (or routes) as a resource management problem. First, the system learns optimal paths by repeatedly selecting available paths between source and destination, given the arriving flow distribution. RL directly interacts with the network environment and learns best paths to minimize the flow completion time given the current network conditions. Our RL approach (DeepRoute) is developed using Q-learning as a value-based gradient reinforcement learning [START_REF] Zhu | A q-values sharing framework for multiple independent q-learners[END_REF].

Our experiments are focused on WAN scenarios, with two implementations, 1. We simulate an environment with synthetic data set of 100 flows, with 80:20 (elephant:mice) distributions, being allocated on 4 possible paths. We compare DeepRoute to either randomly selecting paths or prioritizing one flow type over the other. Here we use the analogy of 100 flows waiting to be allocated on paths and aim to quickly empty the wait queue. 2. We translate the experiment on Mininet network emulator environment. Doing so, we remove some assumptions we drew in the simulation and also understand how DeepRoute will work in a real network environment.

In both cases, we train DeepRoute via experience generating abundance of flow interaction data with the network. Evaluation of the agent's performance is done on test data set, which is previously unseen flow data, to see how well DeepRoute fares against other techniques.

Background

Path Computation and Flow Management in WAN

We explain why path computation is a challenging problem:

-Network tra c demands are continuously changing and are often di cult to predict. Routing tables are continuously changing with new devices joining/leaving the network. However, paths between two endpoints, usually follows one optimized route. If there are too many flows, this leads to potential congestion on the path [START_REF] Domzal | New congestion control mechanisms for flow-aware networks[END_REF].

-Underlying network systems are complex and distributed, with multiple links between source-destination pairs. Understanding link properties such as bandwidth or latency is di cult to model accurately [START_REF] Floyd | Link-sharing and resource management models for packet networks[END_REF].

-Most flow management techniques are developed for data center networks [START_REF] Al-Fares | Hedera: Dynamic flow scheduling for data center networks[END_REF]. WANs, on the other hand, prioritize performance metrics such as minimizing packet loss and bandwidth utilization for tra c diversion [START_REF] Winstein | Tcp ex machina: Computer-generated congestion control[END_REF]. Google's B4 and Microsoft's SWAN, both, make decisions on application characteristics and heuristics.

Compared to approaches designed by B4 and SWAN [START_REF] Jain | B4: Experience with a globally-deployed software defined wan[END_REF][1], we leverage reinforcement learning to provide alternative to heuristic-based path computation problem. We aim to allow networks to learn best routes such to minimize flow completion times of both elephant and mice flows.

Path computation has a number of real-world networking implications. Examples such as load balancing, minimizing congestion and utilizing maximum bandwidth as some cases that can be explored as a reward. WAN networks allow a number of pathways to exist between pairs of end-hosts. These paths can have equal or di↵erent cost distributions such as settings for bandwidth, latency, throughput and more. These settings can determine how quickly the arriving flow will reach its destination, and can be allocated using di↵erent egress ports to choose path to take. ECMP routing is an example of this where it uniformly picks an egress port to reduce congestion on one path, however has seen to su↵er from hash collisions [START_REF] Iselt | Resilient routing using mpls and ecmp[END_REF] and unbalance on di↵erent cost distribution paths. Another problem is the changing tra c conditions in the network. Figure 1 shows a new flow F2 being allocated to one of the paths. However, there might be other previous flows running F0 and F1, already allocated on part of the paths. This means that while the links costs can be set in advance, the available bandwidth on the links is continuously changing and di cult to anticipate when selecting paths.

Tra c engineering such as MPLS-TE, B4 and SWAN use heuristics to design optimal allocations for paths using local and global optimization functions. For DeepRoute, we focus on WAN network path computation, particularly where we want flow-based routing decisions to improve flow completions times for mice and elephants that can allow all available paths to be chosen and utilized.

Reinforcement Learning

A reinforcement learning problem is formulated with an agent, situated in a partially observable environment, learning from past interaction data to make current decisions. The agent receives data in form of environment snapshots, processed in some manner, with specific relevant features. After receiving information and computing the value for future actions given the current state, an agent then acts to change its environment, subsequently receiving feedback on its action in form of rewards, until terminal state is reached. The objective is to maximize the cumulative reward over all actions in the time agent is active. Reinforcement learning research has investigated multiple techniques such as in multi-armed bandit problems, resource allocation or finding routes through a maze [START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF]. Deep reinforcement learning builds upon classical models, replacing the learning with a neural network to approximate policy and value functions. Here, the function approximates the environment state space with actions and rewards. Particularly when the state space is too large to store, this approach has proved feasible in learning approximate conditions. In future, we plan to expand DeepRoute to adapt from Q-learning to neural network learning (deep Q-networks), but is currently out of scope of the work presented here.

Reinforcement learning can be expressed as a Markov Decision Process (MDP) involving sequential decisions. This is given as a tuple (S, A, R, P ), where s 2 S set of states, a 2 A set of actions, R(s, a, s 0 ) represents reward given for executing action a at s and moving to new state s 0 . There is probability P for executing action in state s.

In our problem, the network is modeled as a MDP. We model 4 paths and their current allocations as a state. The agent selects a path and receives a reward after a flow has completed. In the simulation model, this is delayed reward as flows take longer to finish. In the mininet model, we get the reward at every iteration. The reward act as signals to adjust the forwarding-link priorities to enhance or diminish the probability a specific next-hop is selected for the flow. Our agent learns to adjust path selection policies based on experience through continuous modification and rewards.

Q-learning Formulation

A Q-value represents state-action combinations. Better Q-values shows better chances of getting higher rewards which are earned at the end of a complete episode. The Q-value is calculated using a Q-function. It approximates the Qvalue using prior Q-values, a short-term and a discounted future reward. This way the find optimal control policies across all environment states. Q-learning is an o↵-policy reinforcement learning algorithm that uses a table to store all Q-values with possible states and action pairs. This table is updated using the Bellman equation, allowing the action to be chosen using a greedy policy, given as with is discounting factor.

Q(s, a) = R(s, a) + max a 0 Q(s 0 , a 0 )) (1)
Temporal Di↵erence (TD) Learning. In our model, we enable the agent to learn in every action taken, despite it being end of the episode or not. We define an episode to end after 100 flows have been allocated. The TD learning factor updates current Q-value where ↵ is the learning rate,

Q t (s, a) = Q t 1 (s, a) + ↵T D t (a, s) (2)
Therefore, our final equation becomes,

Q t (s, a) = Q t 1 (s, a) + ↵(R(s, a) + max Q(s 0 , a 0 ) Q t 1 (s, a)) (3)
Where 2 [0, 1] represents discounting factor that scales importance of the immediate reward obtained for the action and rewards R obtainable for actions at the new state s 0 . The learning rate ↵ 2 [0, 1] models the rate at which of Q-values are updated.

Related Work

E cient selection of paths for short and long flows has also shown to reduce congestion [START_REF] Munir | Minimizing flow completion times in data centers[END_REF]. Additionally, adaptive tra c management using congestion-based routing has proven to improve overall network utilization [START_REF] Chakravorty | Performance optimizations for wireless wide-area networks: Comparative study and experimental evaluation[END_REF] [START_REF] Jain | B4: Experience with a globally-deployed software defined wan[END_REF]. However, adaptive algorithms in heavy tra c load, could cause oscillatory behavior and cause performance degradation [START_REF] Wang | Analysis of shortest-path routing algorithms in a dynamic network environment[END_REF]. OWAN was developed to optimize bulk transfers on WAN by re-configuring the optical layer showing a 4-times faster flow completion rate [START_REF] Jin | Optimizing bulk transfers with software-defined optical wan[END_REF].

Flow completion time has been used as a vital metric to improve network congestion [START_REF] Noormohammadpour | On minimizing the completion times of long flows over inter-datacenter wan[END_REF]. Additionally, separating elephant and mice flows can have a direct impact on network quality [START_REF] Habibi Gharakheili | Enabling fast and slow lanes for content providers using software defined networking[END_REF]. Other approaches have used calendaring to improve bandwidth utilization and reduce congestion [START_REF] Kandula | Calendaring for wide area networks[END_REF].

In comparison to above approaches, machine learning has been used for network routing. The authors [START_REF] Valadarsky | Learning to route[END_REF] show how learning can help improve the average congestion rate through softmin learning.

Reinforcement learning , in particular, has given interesting results in compute resource allocations such as CAPES [START_REF] Li | Capes: Unsupervised storage performance tuning using neural network-based deep reinforcement learning[END_REF] learning to optimize the Luster file system and DeepRM [START_REF] Mao | Resource management with deep reinforcement learning[END_REF] learning to allocate jobs on processors. Within networks, implementations of reinforcement learning in internet congestion control [START_REF] Jay | A deep reinforcement learning perspective on internet congestion control[END_REF] and developing intelligent TCP congestion algorithms [START_REF] Winstein | Tcp ex machina: Computer-generated congestion control[END_REF] have shown game-changing results in managing bandwidth and bottleneck across individual links. But nearly all of these demonstrations have only been shown on small networks and simulation only. There is also little evidence on how these can be expanded to real WAN environments.

In this paper, we prove the usefulness of reinforcement learning for path computation, but this is presented as early work on how Q-learning approaches can provide benefit and be translated in WAN environments, which we will expand in future work.

Design of DeepRoute

We define a general network topology with unidirectional links denoted as paths p 2 P with varying bandwidth capacity and latency. Flows arrive at time step t = 1 and are assigned a fixed path, for example either 1 ! 2 ! 4 or 1 ! 3 ! 4 (in Figure 1). The arriving flows are generated with a specified size and duration allowing the flow to exist on the path for a time period (longer for elephant flows). Given every flow f i from source s i to destination d i , is defined with flow size v i and duration r i . The flow is assigned to a path p i which would have latency lat pi and available capacity ac pi . The latency influences the actual flow completion time c i = r i + lat pi . At a given time, flows are allocated based on available bandwidth ac pi , which changes depending on tra c patterns.

Similar to SDN [START_REF]Achieving high utilization with software-driven wan[END_REF], we assume a centrally existing routing control. The controller maintains information on current allocations across paths and makes routing decisions based on completion times of previous flows.

DeepRoute in Simulation Model

Figure 2 shows the network topology used in the simulation. The goal is to move all arriving flows from source to destination as quickly as possible. There are 4 possible paths with di↵erent bandwidth and latency settings. In the model, we assume 100 flows are arriving together in one timestep, and the controller allocates each flow to the next available path. The 100 flows contain a distribution of 80% mice flows and 20% elephant flows (Table : 1). Once allocated, the time step progresses to t+1, where the controller tries to allocate the remaining flows.

Objective. When a flow duration r i (in time units) finishes, it computes its completion time c i by adding its duration with path latency. We then inverse this, to give the flow's slowness rate by l i = c i /r i . Similar to [START_REF] Mao | Resource management with deep reinforcement learning[END_REF], we normalize this, to prevent skewing results for longer flows. The objective of DeepRoute is to get as many flows completed soon as possible.

Paths as Resources. We assume 100 flows arrive at Node-1 going to Node-5.

There are 4 possible paths 1 ! 2 ! 5, 1 ! 5, 1 ! 3 ! 5 or 1 ! 4 ! 5, with each path having di↵erent bandwidths and latency attached. bu means total bandwidth units available to be allocated. This changes as flows are allocated on them consuming part of the bu equal to flow size. The allocations last for the flow's duration, and the completion time is computed summing path latency given in tu (time units). We define this as current available bandwidth across all paths, size of the flow being allocated and current allocations on the paths. For example, after an allocation on path0 (1 ! 2 ! 5), the bandwidth availability is now ac p0 = 10 v 1 and this part of state becomes (ac p0 , ac p1 , ac p2 , ac p3 ).

Action Space. There are 4 paths so 4 possible actions. We assume one action is taken per one flow, within one time step. If there are no available paths, the controller skips a time step with no allocations. Once all 100 flows are allocated, it finishes a complete episode. The simulation is run for number of iterations containing many episodes. The total reward is calculated per episode when all 100 flows are allocated.

Reward Calculation. At the end of each episode, the RL agent calculates if any flows have finished and total completion time is recorded.

DeepRoute in Mininet

Figure 3 shows mininet topology forming multiple paths to chose from one endhost to the other. Here, we configure three metrics for the links -capacity, packet loss and latency. Mininet allows us to add delays on links. The capacities (bandwidth) are setup as 2 x 10 Gbps links for path0 and path1, and 2 x 8 Gbps for path2 and path3.

Sending flows. We ping from one end-host to the other and record the time in which the ping reaches the other host. The controller learns which egress port to use, deciding which path to take. This time is recorded as the reward against the path taken. SDN Control. We emulate an SDN network using an Ryu OpenFlow switch. To have a global view, we configure a network controller, Openflow switches, linux hosts and network links. The hosts run a standard Linux kernel and network stack to emulate running real network applications. Traceroute measurements are recorded on the route taken and the specific gateway at each hop. We calculate the total time across each hop as a reward. In this case, every episode is represented by one flow (one ping).

State and Action Space. For dynamic multipath selection, we send a packet from source to destination via one of the four network paths, dynamically allocating flows on links. The flows go through paths: 1 ! 2 ! 5, 1 ! 5, 1 ! 3 ! 5 and 1 ! 4 ! 5.

For initial learning, we use Link Layer Discovery Protocol (LLDP) [START_REF] Congdon | Link layer discovery protocol and mib[END_REF] to obtain link and switch states in the topology, it uses to advertise device identity and abilities, and other devices connected within. LLDP helps maintain a global view of network topology and also retains a multipath environment.

Training DeepRoute

The DeepRoute agent runs in an episodic fashion (with 100 flows in simulation and 1 flow in mininet). The episode terminates when all flows have been allocated.

A Q-value is added with each state and action taken and saved into a Q-table. We design our algorithm based on [START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF]. As the reinforcement learning algorithm uses Bellman's equation, there is a possibility of overfitting to ideal conditions. To prevent this, during the testing phase, we use ✏ for allowing DeepRoute to select random action rather than Q-table values. Q-table is recorded by monitoring Wireshark logs across all interfaces on the controller.

Initially, the OpenFlow switch performs active and passive measurements across all egress links, building the Q-table. The utilization data across the interfaces is collected. Here, we also have a local Q-table at each network node, as well as a global aggregation table managed by the network controller, shown in Figure 5. The wireshark logs allow to collect data on latency, throughput and packet loss. Rewards or the completion time is added afterwards to populate the Q-values in the table constructed. Table 2 shows an example of the Q-table constructed in Mininet. Showing only 4 entries, 1 in each path. We transfer a mixture of flow distributions (8 and 16 GBytes). The arrival time (or completion time) is considered as the reward, added to the Q-table for that state and action.

Evaluation

We evaluate DeepRoute for the following objectives:

-With a distribution of network path parameters, how does DeepRoute compare with other allocation techniques in simulation model?

-How can DeepRoute be translated into real network environments as in a mininet model?

-How much of utilized capacity improvements are observed for the overall network?

Simulation Model Results

Testing Data. We generate a new set of 100 flows of 80:20 (mice:elephant) distribution. This data is unseen by the DeepRoute during training phase. The test data is consistent for all other comparison schemes to validate the results. Comparing with other techniques. We compare DeepRoute with other algorithms -Random, to randomly assign flows on available paths, Prioritize-Mice, to allocate mice flows before allocating elephant flows, and Prioritize-Elephant, allocate elephant flows before mice flows. These have been published in path computation problems [3][32].

Average Path Utilization. Figure 6 shows the path capacity used during the testing phase. Here we see that while Path0 is the most used by the elephantand mice-first techniques, DeepRoute is able to show a more spread of using other paths e ciently. It learns to use Path 1 and Path 2, with lower latency more e ciently. This is also shown in Figure 7, where paths are used more than in other techniques. The random technique just spread use across all paths. Figure 8 shows that DeepRoute is able to completely utilize the network at stable 30% as compared to the other techniques as number of flows increase. Mice and Elephant allocations. Figure 7 shows that DeepRoute is able to spread elephant and mice flows more uniformly across all the paths. We also see that by spreading the load, there is less congestion on one path, which was the case with other techniques.

Comparing with shortest path route. Based on the path configurations, the network would use Path1 the most due to higher bandwidth and lower latency. However, in the simulation DeepRoute learns to allocate on alternate paths, based on the current allocations already active on the paths.

Mininet Model Results

After training, we configure the Ryu SDN controller to select egress based on Q-table. This adjusts forwarding rules dynamically as new flows arrive.

Testing data. We generate 100 pings from one end-host to the other and record the utilization across all the paths. Throughputs recorded. Figure 9 shows the throughput measurements along the 4 paths. Here, because the mininet model is extremely simple, the states learned are the available paths, flow size sent and the time recorded. This allows the controller to learn the shortest, less cost path, Path 1, as the most optimal path to use. As a result, all future tra c goes through this path and less evenly distributed at other paths. This result shows, if the controller was able to learn more features of the current state, we could enhance the controller decisions. This will form basis for future investigations of implementing DeepRoute in real networks.

Discussion

Modelling realistic WAN flows

In this section we explained how we model realistic WAN flows, using di↵erent size parameters in mininet simulation. We use NETEM [START_REF] Hemminger | Network emulation with netem[END_REF] which provides network emulation functionality for testing protocols by emulating properties of wide area networks. To emulate the real-world network scenario, with control on parameters that a↵ect network performance, we change the path delay on all four paths to (300ms, 100ms, 100ms, 300ms). In running our emulation, we consider four key metrics -link capacity (bandwidth), latency, transfer size and flow arrival time (presented in Table 2). To calculate latency, link delays were assigned paths.

For link capacities (bandwidth), the DeepRoute topology has a total link capacity of 36 Gbps linking the path 0 to path 3. This includes (2 X 10 Gbps) on path 0 and 1 as well as (2 X 8 Gbps) on path 2 and 3 respectively. Iperf3 [START_REF] Mortimer | iperf3 documentation[END_REF] is used for measuring performance characteristics, specifically, the TCP version of Iperf3. Each TCP Iperf throughput test is initiated for 60 seconds per path. All end hosts loop through this process for at least 100 flow rounds, thereby measuring throughput on all the network paths (path 0-3) with di↵erent flow distributions. We use wireshark [START_REF] Orebaugh | Wireshark & Ethereal network protocol analyzer toolkit[END_REF], a free and open-source packet analyzer to capture packets on all interfaces.

Performance measurement with load and delay

As flow are sent, the network controller starts the learning process by performing active and passive measurements between all switches (S1-S5). The active measurements are used to measure latency between the switches, while passive measurements are used to obtain load and residual capacity in each link.

Having successfully deployed emulated WAN using NETEM and Mininet, we conducted some performance measurement. With a set of flows, we specify a normal distribution for delay, but since delays are not always uniform, we specify a Pareto distribution (non-uniform delay distribution). As a result, all packets leaving source to destination via path 1 and 2 will experience a delay time of 100ms, while those leaving via path 0 and 3 will experience 300ms. The final results show minimum, average, maximum and standard deviation of the Round-Trip-Time (RTT) and if packet loss is recorded.

Comparing to optimization approaches

The current implementation of Q-table shows that similar results could have been achieved by optimization techniques. However, this implementation is limited by modelling the system as a simple MDP approach. In a real network setting, conditions are much more dynamic with unseen environment conditions. This means the Q-table approach will have to updated the build approximate value functions, such as deep Q-networks, to make decisions in unseen conditions.

Conclusions and Future Work

Recent breakthroughs in deep learning research, made possible by accelerated hardware and big data, in many fields. However, there is still a lack of understanding on how this can be used in network routing research.

By utilizing Q-learning we allow the controller to learn from the environment about the paths and best hops between source and destination. With network environments being very dynamic, with possible packet loss and tra c congestion across some of the best paths, we explore how a DeepRoute controller can learn best possible combinations depending on the tra c arriving and the current network conditions to optimally utilize the network.

Commercial systems that promise improved network performance tend to focus on average typical flows and through exploring edges (when compared to average) press exploration of new tra c engineering models. Our work highlights the need to change approaches to path computation and flow management for new applications like hybrid cloud computing and other use cases cited. While networks are challenged to strike the balance between capacity, throughput, latency and cost, AI applications can have an impact on future deployments. Our results show promise on how DeepRoute can allow e cient use of path capacity and the mininet implementation shows how it can be adapted in a real network environment.

Fig. 1 .

 1 Fig. 1. Example of routing a new flow F1 from 1!4. There are two possible paths to take through Node-3 or Node-2.

Fig. 2 .

 2 Fig. 2. Topology used in the Simulation Model (bu=bandwidth units occupied and tu=time units occupied).

Fig. 3 .

 3 Fig. 3. Mininet Topology.

Fig. 4 .

 4 Fig. 4. Changing number of iterations during training gives di↵erent rewards.

Fig. 5 .

 5 Fig. 5. Q-values for Local and Global levels.

  Capacity (Gbps) Latency (ms) Path Transfer (GB) Arrival Time (ms)

Fig. 6 .

 6 Fig. 6. Occupied bandwidth across the paths with arriving flows. Occupied bandwidth refers to the percentage bandwidth occupied on the path.

Fig. 7 .

 7 Fig. 7. Average flows allocated to all paths.

Fig. 8 .

 8 Fig. 8. Average network utilization.

Fig. 9 .

 9 Fig. 9. Throughput measurement from Source-Destination.

Table 1 .

 1 Elephant and mice flow distributions used in simulation case.

	Flow type Size	of	Flow	Duration of Flow	Distribution %
		(bandwidth units	(time units occu-
		occupied)		pied)
	Elephant Range(3,5)		Range(5,8)	20%
	Mice	Range(1,2)		Range(1,3)	80%

Table 2 .

 2 Sample Q-table showing capacity, latency and flow arrival time.

This work is funded under DOE ASCR Early Career Grant Deep Learning: FP00006145.

Training in Simulation

DeepRoute is trained for multiple iterations as shown in Algorithm 1. Each iteration generates new 100 flows and DeepRoute learns by allocating these flows on the paths. We record the network state (available bandwidth on all 4 paths, current allocations, size of the flow to allocate), action taken (which path is chosen) and reward collected in the episode.

Training iterations.

Training for more iterations, allows the size of the Q-table to grow (from 3532 for 50 iterations, 6238 for 100 and 21640 for 500 iterations). However, Figure 4 shows that the maximum score is achieved by 400 iterations. Therefore this is chosen as the ideal training iterations.

Training in Mininet

The implementation in Mininet removes some assumptions drawn in the simulation model. Here, along with using an OpenFlow switch, we use packet loss and latency, to calculate the reward for every ping received at other host. The


Onyekachi Nwamuo
email: onyekachien@uvic.ca

Paulo Magella De Faria Quinan

Issa Traore
email: itraore@ece.uvic.ca

Isaac Woungang
email: iwoungan@ryerson.ca

Abdulaziz Aldribi

Arguments Against using the 1998 DARPA Dataset for Cloud IDS Design and Evaluation and Some Alternative

Keywords: cloud IDS, cloud security, machine learning, IDS evaluation, Hypervisor-based IDS

Due to the lack of adequate public datasets, the proponents of many existing cloud intrusion detection systems (IDS) have relied on the DARPA dataset to design and evaluate their models. In the current paper, we show empirically that the DARPA dataset by failing to meet important statistical characteristics of real world cloud traffic data center is inadequate for evaluating cloud IDS. We present, as alternative, a new public dataset collected through a cooperation between our lab and a non-profit cloud service provider, which contains benign data and a wide variety of attack data. We present a new hypervisor-based cloud IDS using instanceoriented feature model and supervised machine learning techniques. We investigate 3 different classifiers: Logistic Regression (LR), Random Forest (RF), and Support Vector Machine (SVM) algorithms. Experimental evaluation on a diversified dataset yields a detection rate of 92.08% and a false positive rate of 1.49% for random forest, the best performing of the three classifiers.

Introduction

In today's IT and business world, there has been a significant increase in the public adoption of cloud computing for the production systems and services support, and there seems to be no end in sight [33]. However, the growth in the public adoption of the cloud paradigm has increased organizations exposure to a wide variety of cyber attacks and vulnerabilities. Intrusion detection system (IDS) is one of the key tools being used or explored in combatting cloud attacks.

Until now, the availability of a cloud dataset has been one of the major challenges hampering the progress of the research on cloud IDS. The majority of the works done so far on cloud IDS was done using conventional datasets like the DARPA 1998 or the KDD'99 datasets [START_REF] Ahmad | CloudIDS: Cloud Intrusion Detection Model Inspired by Dendritic Cell Mechanism[END_REF][START_REF] Hussain Bhat | Machine Learning Approach for Intrusion Detection on Cloud Virtua machines[END_REF][START_REF] Chou | An adaptive network intrusion detection approach for the cloud environment[END_REF][START_REF] Hyukmin Kwon | Self-similarity based lightweight intrusion detection method for cloud computingLecture Notes in Artificial Intelligence 6592 Edited by Subseries of Lecture Notes in Computer Science[END_REF]. More so, the datasets used in the works done on a cloud environment are not made available for public use, in some cases for privacy concerns. These factors have denied the cloud researchers of an all-encompassing real-world cloud intrusion dataset to carry out their work on. As shown in previous studies [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF][START_REF] Kandula | The Nature of Datacenter Traffic: Measurements and Analysis[END_REF], there are strong differences between cloud network data and conventional network data in terms of their characteristics such as flow inter-arrival time, packet-level communication, load ratios of the internal / external traffic flow, and so forth. On the other hand, the design of anomaly detection models involves constructing normal activity baselines from previously collected sample activity data. Hence, constructing cloud anomaly detection models using conventional network data would fail to capture adequately cloud network behavior considering the aforementioned differences between cloud network and conventional network data.

The objective of the current work is to provide an empirical justification for the need for a dataset collected specifically in a real cloud environment compared with using a conventional network dataset in developing cloud IDS. Furthermore, we explore the design of cloud anomaly detection using supervised machine learning techniques. Specifically, three machine learning algorithms are studied: logistic regression (LR), random forest (RF) and support vector machine (SVM).

The rest of the paper is structured as follows. Section 2 discusses related work. Section 3 highlights informally the deficiencies of the DARPA dataset and introduces as alternative a real Cloud IDS dataset. Section 4 provides empirical evidence supporting the claim that the DARPA dataset does not meet key characteristics of cloud data. Section 5 presents a new hypervisor-based cloud IDS model using supervised machine learning. Finally, Section 6 makes concluding remarks.

Related Works

To address the lack of public cloud-specific datasets, some researchers have focused on generating new datasets, such as [START_REF] Kholidy | Cidd: A cloud intrusion detection dataset for cloud computing and masquerade attacks[END_REF][START_REF] Moorthy | Virtual host based intrusion detection system for cloud[END_REF][START_REF] Mukkavilli | Generation of labelled datasets to quantify the impact of security threats to cloud data centers[END_REF]; however, to the best of our knowledge none of these datasets are openly available. As a result, many existing cloud IDS proposals have relied on conventional IDS datasets for development and evaluation, using primarily the DARPA IDS dataset or the KDD CUP dataset. We discuss some of these proposals in the following.

Bhat et al. [START_REF] Hussain Bhat | Machine Learning Approach for Intrusion Detection on Cloud Virtua machines[END_REF] proposed an approach for detecting intrusions in virtual machine environment on cloud using traditional and multiclass (hybrid) machine learning algorithms. The following machine learning algorithms were considered: Naïve Bayes Tree (NB Tree) classifier, hybrid of NB Tree and Random Forest. The NSL-KDD'99 dataset was used for evaluation and it was observed that hybrid machine learning models perform better than the traditional or individual algorithms. In using single classifiers, their evaluation generated accuracies of 95%, 91% and 98% for each of Random Forest, K-NN and SVM, respectively, while the combination of NB Tree and hybrid of NB Tree and Random Forest resulted in a high accuracy of 99% and low false positive rate of 2%. More so, the hybrid of Random Forest and weighted K-Means amounted to 94.7% accuracy and 12% false positive rates.

Modi and Patel [START_REF] Modi | A novel hybrid-network intrusion detection system (H-NIDS) in cloud computing[END_REF] presented an approach that integrates hybrid Network Intrusion Detection Systems to cloud computing environment. The experimental set up involves using the Eucalyptus infrastructure for the simulation of a cloud computing environment, while the KDD IDS dataset was used for the evaluation of their work. Their research framework involved the integration of signature-based detection and anomaly detection. They utilized Snort, for the signature-based intrusion detection and three machine learning classifiers viz the Bayesian, Associative (a machine learning model using association rule) and Decision Tree classifiers singularly and collectively for the network anomaly detection. The experimental result of their proposal for the three classifiers and their collective ability yielded a true positive rate (TPR) of 97.14%, and a false positive rate (FPR) of 1.17%.

Muthurajkumar et al. [START_REF] Muthurajkumar | A Rough Set based Feature Selection Algorithm for Effective Intrusion Detection in Cloud Model[END_REF], used the combination of fuzzy SVM and random feature selection algorithms (RSFSA) to propose a cloud intrusion detection model. In their experiment, they built two sets of intrusion detection model, one with the whole data features and the other after introducing feature selection. A dataset consisting of 10% of KDDCUP was used for the experiment and analysis of their approach. The average detection rate from their experimental results before and after applying the RSFSA to the Fuzzy SVM classifier are 86.88% and 94.15%, respectively. Their work confirmed that feature selection plays an important role in the classifiers' detection accuracy. It would have ben interesting to evaluate this proposal using a real cloud intrusion detection dataset.

Chou et al. [START_REF] Chou | An adaptive network intrusion detection approach for the cloud environment[END_REF] proposed an adaptive network-based intrusion detection system for the cloud environment using the DARPA 2000 and the KDD Cup 1999 datasets. Their approach used spectral clustering, an unsupervised learning algorithm to build a decision tree-based detection model for detecting an anomaly in an unlabeled network connection data. They used Bro-IDS to generate connections records from the raw packets. Their experimental result on the DARPA dataset yielded a detection rate of 95% and a false positive rate of 4.5% while the KDD Cup 1999 dataset yielded a detection rate of 90% and a false positive rate of 5%. Their approach is not enough robust as it could not detect DOS and some probing attacks which create a great amount of connections.

Ahmad et al. [START_REF] Ahmad | CloudIDS: Cloud Intrusion Detection Model Inspired by Dendritic Cell Mechanism[END_REF] presented an intrusion detection model that uses Dendritic Cell Algorithm for detecting intrusions in cloud computing environment. The experimental evaluation was conducted using the DARPA 1999 dataset. The network-based attributes were used as signals in their experiments. They carried out their experiment on a total of 187 threat events of Week 4 and Week 5 of the DARPA 1999 dataset and the algorithm achieved a detection rate of 79.43% and a false positive rate of 13.43%. In their work, they demonstrated that using Dendritic Cell algorithm could provide a solution in detecting attacks in the cloud environment. Kannan et al. [START_REF] Kannan | Genetic algorithm based feature selection algorithm for effective intrusion detection in cloud networks[END_REF] proposed a host-based cloud intrusion detection system which uses a genetic algorithm based feature selection and a Fuzzy SVM based classifier for deciding if an event is intrusion or not. The cloud environment was simulated with Proxmox VE 1.8 which is an open source virtualization environment while the evaluation was done using the KDD'99 cup dataset. In the experimental results, a detection rate of 98.51% and a false positive rate of 3.13% were obtained.

Zhao et al. [START_REF] Zhao | An anomaly intrusion detection method based on improved K-means of cloud computing[END_REF] put forward an anomaly detection system based on an unsupervised learning algorithm, namely the K-means clustering algorithm. The dataset chosen for their experiment was the KDD Cup 99 dataset. For the comparative analysis of the performance of their proposed approach, they used the Particle Swarm Optimization (PSO) and Backpropagation (BP) Neural Network algorithms to test the performance of their proposed algorithm. The K-means algorithm performed better than the other two algorithms, yielding a false positive rate (FPR) of 3.56% as against FPR of 6.78% and 5.75% for PSO and BP neural network algorithms, respectively. And in terms of false negative rate (FNR), 7.65% was achieved in contrast to 10.46% and 13.75% obtained using PSO and BP neural network algorithm, respectively. Their study was not carried out with a real cloud IDS dataset but it however worth its salt as it highlighted the possibility of predicting several types of attacks in the cloud.

Xiong et al. [START_REF] Xiong | Anomaly secure detection methods by analyzing dynamic characteristics of the network traffic in cloud communications[END_REF] proposed an anomaly detection method for cloud computing systems based on two approaches, viz the Synergetic Neural Network (SNN) algorithm and the Catastrophe theory (CT) algorithm. They used the DARPA dataset for their experiment and focused their work on the network traffic information. Their experiment yielded an overall average detection rate of 83% on the SNN algorithm and 86.62% overall average detection rate on the CT algorithm. The experiment also yielded an overall average of 8.3% false positive rate on the SNN algorithm and an overall false positive rate of 9.06% on the CT algorithm.

Li et al. [START_REF] Li | A neural network based distributed intrusion detection system on cloud platform[END_REF] proposed an artificial neural network (ANN) based cloud IDS. The experimental part of their proposal involved simulating a cloud environment using Ubuntu Enterprise Cloud (UEC), a Eucalyptus-powered cloud platform and evaluating the result on 10% of the KDD'99 dataset. The experiment yielded an average detection rate of 99% and an average detection time of 37.1 seconds. One of the drawbacks here is that the ANN takes huge training time for large databases, therefore, the anomaly detection algorithm may incur an increased cost if retraining is required due to change in traffic behaviour as in the case of the cloud computing environment. More so, the simulated dataset can not stand in as a real cloud dataset.

Datasets for Cloud IDS Evaluation

In this section, we compare the characteristics of conventional network data with cloud network data, and give an overview of the ISOT cloud IDS evaluation dataset.

Cloud Network vs. Conventional Network Data Characteristics

Great disparity exists when considering the proximity of both the cloud and conventional network data centers. While the cloud data centers are distributed globally, the non-cloud data centers are always situated in a close proximity to their users or on the premises of the serving organizations. The global placement of the cloud datacenters satisfies the requirements for geo-diversity, geo-redundancy and regulatory constraints [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF]. Studies [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF][START_REF] Kandula | The Nature of Datacenter Traffic: Measurements and Analysis[END_REF] have shown that the characteristics of cloud network traffic are different from the conventional network traffic in so many ways as explained in the following.

Figure 1: Network Flow in ISOT-CID

Network Flow: Empirical studies in [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF], [START_REF] Kandula | The Nature of Datacenter Traffic: Measurements and Analysis[END_REF] have shown that the inter-arrival time for 80% of the traffics in the cloud network is usually under 1ms while with the conventional network, this can be between 4ms and 40ms as the traffic does not change that quickly. In their work, it was also noted that the number of active flows for any given second at a switch is at most 10,000 flows and that new flows can also be highly instantaneous in arrival. These studies also went on to explain how the flow inter-arrival time affects the kind of processing that can be done for each new flow and the usefulness of logically centralized controllers for the flow placement. The cloud network traffic is usually bursty in nature with the ON/OFF intervals being characterized by heavy-tailed distributions. Their analysis also shows that in a cloud computing environment, the load ratios of the internal/external traffic flow between the instance to instance or instance and other sources are usually high. In [START_REF] Kandula | The Nature of Datacenter Traffic: Measurements and Analysis[END_REF], it was also discovered that in a conventional network data, 80% of flows are usually smaller than 10Kb in size as compared with a cloud network data. On the one hand, the flow of communication patterns in a cloud network is usually high due to the numerous applications being hosted and high link utilization across the cloud's multiple layers. On the other hand, with the traditional network, the communication flow pattern and the link utilization are usually small in size. Figures 1 and2 show the network flows for a typical cloud environment based on the ISOT-CID and the DARPA 1999 IDS evaluation dataset which was collected by simulating a conventional network environment [START_REF] Aldribi | Hypervisor-Based Cloud Intrusion Detection through Online Multivariate Statistical Change Tracking[END_REF]. In the ISOT-CID environment shown in figure 1, we can see significant variability in the network flows including the hypervisor to hypervisor network flows, the in/out traffic flows from VM to external source not within the cloud environment, the VM to VM network flows, and traffic flow between tenants VMs [START_REF] Aldribi | Hypervisor-Based Cloud Intrusion Detection through Online Multivariate Statistical Change Tracking[END_REF]. The conventional network comprises of limited network flows as can be seen in figure 1.4 which has only two network flows viz external and internal traffic flows [START_REF] Aldribi | Hypervisor-Based Cloud Intrusion Detection through Online Multivariate Statistical Change Tracking[END_REF].

Topology:

The physical topology of a cloud data center follows a canonical 3-Tier architecture which consists of the core layer or the uppermost layer, aggregation layer or the middle layer and the edge layer or the lower link layer. In contrast, the traditional data centers follow a 2-Tiered topology in which the core layer and the aggregation layers are collapsed to form one layer [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF]. In a typical cloud network, data is either centralized or outsourced and provided to the users on-demand irrespective of their geographic location. This relieves the data owner of the full control of their data as the cloud service providers now manage and maintain the data. The cloud data also has the flexibility of being scaled up or down by automated means. Some giant cloud service providers such as Amazon, Google and Microsoft do have cloud data centres dispersed geographically for the provision of universal data access to the various users [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF].

Overview of the ISOT Cloud IDS Dataset

The ISOT-CID is a publicly available dataset that was collected in a real world environment using the infrastructure of Compute Canada, a nonprofit cloud service provider that extends its services in the areas of providing the computational needs of researchers [START_REF] Aldribi | Hypervisor-Based Cloud Intrusion Detection through Online Multivariate Statistical Change Tracking[END_REF]. There were two phases involved in the ISOT-CID data collection procedure, namely Phase 1 in 2016 and Phase 2 in 2018. The data in the two phases were collected on the same production environment based on OpenStack from various cloud layers such as hypervisors, guest hosts layers and the network layer. The dataset size is more than 8 terabytes and it contains data of different formats such as the memory dumps, CPU and disk utilizations, system call traces, system logs and network traffic [START_REF] Equinix | To Maximize the Cloud, Focus on the Network[END_REF]. Another advantage of the ISOT-CID is that it is labelled and includes both normal and attack activities. The current work is based on the network traffic attributes of the ISOT-CID.

The ISOT-CID collection environment contains three hypervisor nodes viz, node A, node B, and node C. The collecting environment is also composed of 10 virtual machines or instances (VM1 to VM10) launched in three different cloud zones A, B, and C [START_REF] Aldribi | Hypervisor-Based Cloud Intrusion Detection through Online Multivariate Statistical Change Tracking[END_REF].

The benign data in the ISOT-CID came from web applications/traffic and administrative activities [START_REF] Equinix | To Maximize the Cloud, Focus on the Network[END_REF]. Some of the web application activities include account registration, blog activities, and web browsing. The web traffic statistics revealed that more than 160 legitimate users were involved in the generation of the normal data which comprises of 60 human users and 100 robots [START_REF] Aldribi | Hypervisor-Based Cloud Intrusion Detection through Online Multivariate Statistical Change Tracking[END_REF]. While the administrative activities cut across instance routine maintenance, system rebooting, application updates, file creation, machine access via SSH and remote server access.

Table 1 presents all the attacks covered in ISOT-CID, such as probing, DoS, information disclosure, R2L, input validation, backdoors and authentication breach, etc. These attacks were grouped into insider or outsider attacks depending on its source [START_REF] Aldribi | Hypervisor-Based Cloud Intrusion Detection through Online Multivariate Statistical Change Tracking[END_REF]. On the one hand, the inside malicious activities were perpetrated by either an insider within the cloud environment who had a root access on the hypervisor nodes or by a compromised VM within the cloud environment used as a stepping stone. Some of the inside attacks were backdoor and Trojan horse, network scanning, password cracking, DoS attacks, and so forth. On the other hand, the outside malicious activities emanated from outside the cloud environment with the ISOT-cloud environment being the primary target. Some of the outside attacks are made up of the application layer (layer 7) and network layer (layer 3) DoS attacks, input validation attacks, SQL injection, path/directory traversal and cryptojacking (unauthorized crypto mining). For instance, Figure 3 shows a timeline for the attack scenario in Phase 1 Day 2 (2016-12-15). The ISOT-CID is composed of three levels of network communications namely: external, internal and local traffic [START_REF] Equinix | To Maximize the Cloud, Focus on the Network[END_REF]. In the ISOT-CID context, the external traffic is the traffic between the instance and an outside machine. The internal traffic or the hypervisor traffic is between the hypervisor nodes. And finally, the local traffic is the traffic between two VMs on the same hypervisor node. The ISOT-CID network data also comes in kinds, one being without payload on both hypervisors and VMs and the other involving the full network traffic only on the hypervisors [START_REF] Aldribi | Hypervisor-Based Cloud Intrusion Detection through Online Multivariate Statistical Change Tracking[END_REF]. The ISOT-CID network traffic/packet statistics are shown in Table 2.

Traffic Characterization

We analyzed the network traffic data of the ISOT-CID and the DARPA 1998 datasets by looking at the network communication patterns at the flow-level. The idea is to show how similar or different they are in terms of data transmission behaviour at the flow level. This is also in line with the work done in [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF] where the authors used traffic engineering techniques to distinguish between cloud data center networks and conventional or traditional networks. We considered three metrics in our empirical data analysis, viz Number of Flows, Flow Inter-arrival Times, and Flow of Traffic characteristics of the two datasets. Our findings based on the distribution reveals that, the number of active flows for the ISOT-CID is between 2,000 to 6000 about 90% of the time. In the case of the DARPA 1998 dataset, the number of active flows is between 20 and 1000 in 90% of the time interval. This empirical observation supports the results of a prior work on data center traffic [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF]. It is also considerable to note that the latency assigned by a controller to a new flow is determinant on the lengths of the flows [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF].

Number of Active Flows Characteristics

Flow Inter-arrival Time Characteristics

Additionally, we examined the empirical CDF of the flow inter-arrival times under a 120 seconds time window on ISOT-CID Phase 2 Day 1 (2018-02-16) dataset and These results suggest that DARPA is characterized by a smaller number of flows than the ISOT_CID dataset. This empirical observation also supports the results of a prior work [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF].

The flow inter-arrival times affects the scalability of the controller because a significant number of new flows arrive at a given switch within an interval of few microseconds [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF]. Therefore, it is recommended to use multiple CPU's per controller and multiple controllers to compute the routes in order to scale the throughput of a centralized control framework.

Flow-Level Communication Characteristics

The aggregate network transmission behaviour of the two datasets were examined using a day's traffic from each respectively. We based the analysis on two network flow metrics, viz the extra-rack traffic and the intra-rack traffic. The extra-rack traffic signifies the traffic leaving the switch rack or internal hosts for other internal hosts or external destinations, this is easily measured while the intra-rack traffic represents the amount of traffic that stays within the rack or node [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF]. On the one hand, the intra-rack traffic for ISOT-CID, was computed by taking the difference between the volume of traffic generated by the instances attached to the hypervisor nodes and the traffic exiti ng the nodes. On the other hand, the intra-rack traffic for DARPA 1998 dataset was computed by taking the difference between the volume of traffic generated by the servers or host attached to the switches and the traffic exiting the switches. Table 3 shows the extrarack and intra-rack traffic compositions, while Table 4 provides the percentage representation of these two metrics. Figure 6 depicts a bar graph showing the rati o of extra-rack to intra-rack traffic in the selected day traffic of both the ISOT-CID and the DARPA 1998 datasets.

Figure 6: Comparison of the ratio of extra-rack to intra-rack traffic for ISOT-CID and DARPA 1998 datasets

The result of the analysis shows that for the ISOT-CID dataset, 75.84% of the traffic is confined to within the hypervisor node in which it was generated while 24.16% of the traffic leaves the nodes. This result is in contrast with the DARPA 1998 dataset in which only 48.11% of the traffic stays within the communication nodes and 51.89% of the traffic leaves the nodes. The result of this network traffic analysis supports the observations made in prior studies [START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF][START_REF] Kandula | The Nature of Datacenter Traffic: Measurements and Analysis[END_REF] of network traffic characterization of data centers.

Hypervisor-based Cloud IDS using Supervised Machine Learning

In this section, we explore the effectiveness of hypervisor-based cloud anomaly intrusion detection using supervised machine learning. Specifically, three machine learning algorithms are studied: logistic regression (LR), random forest (RF) and support vector machine (SVM).

Feature Model

Because the VM instances in the cloud environment share the same hypervisor, to improve the cloud computing intrusion detection process, the feature extraction should be such that it takes into account the correlated behavior of the instances. A network flow on the other hand can be seen as a bidirectional packet streams between two hosts or the movement of network traffic across different network points, usually from a source to a destination and vice versa. We first grouped the captured hypervisor packets in the pcap file formats into a stream of packet flows based on a time window 𝛿𝑡 using a flow based forensic and network troubleshooting traffic analyzing tool called Tranalyzer. Eighty raw features were extracted from the packet headers and some of the raw features are represented in table 5. We used a three-dimensional features space in this thesis work namely frequency-based features, entropy-based features and load-based features based on the work of Aldribi et al. [START_REF] Aldribi | Hypervisor-Based Cloud Intrusion Detection through Online Multivariate Statistical Change Tracking[END_REF].

Two categories of frequency-based features were adopted for each VM instance, namely, the 'in-frequency' and the 'out-frequency' features. The in-frequency represents the frequency of the packets incoming to a specific instance from any source or endpoint while the out-frequency represents the outgoing packets from a specific instance back to the respective sources.

The load-based features were extracted by taking the ratio of the matching in an out frequency features as proposed in [START_REF] Aldribi | Hypervisor-Based Cloud Intrusion Detection through Online Multivariate Statistical Change Tracking[END_REF].

The entropy associated with the probability distribution of network traffic occurrences at ingress and egress points during the observation time window was computed for the specific instances. Given a distribution of probabilities P = {𝑝 1 , 𝑝 2 , . . . . . , 𝑝 𝑁 } having N variables, entropy is defined as

Hs = -∑ 𝑝 𝑖 𝑁 𝑖=1 𝑙𝑜𝑔 2 𝑝 𝑖 (1)
Where 0 ≤ 𝑝 𝑖 ≤ 1 and ∑ 𝑝 𝑖 𝑁 𝑖=1

= 1. And in our case, 𝑝 𝑖 represents the probability of the distinct frequency features of the traffic during the observation time window. For instance, the entropy of the source IP is calculated by first computing the appearing probability associated with the source IP which is gotten by taking the ratio of the number of packets with the specified source IP address and the total number of packets observed in the flow, after that the entropy equation of ( 1) is adopted to get the value. The entropy is minimum (Hs = 0) at maximum flow concentration or when the features exhibit a deterministic behaviour. On the other hand, the entropy Hs is maximum (Hs = log2N) at maximum flow dispersion or when the feature is fully at random.

Data Preparation

The data preparation steps undertaken in this research work in order to get the best features in the dataset are shown in fig 7.

Figure 7: Data Preparation Subsystem

We leveraged the computational power of pandas, an open source software library for Python programming language to developed a python script to extract the aforementioned three feature categories of our feature model. The extracted features were transformed to the same scale between 0 and 1 using the min-max normalization approach. To reduce the dimensionality and complexity of the feature space, we used the CARET R-library, a tree-based feature selection technique. The CARET R-package gives a percentage score to all the features with the noisy features having a percentage score of zero (0) in accordance to their statistical significance, that is information gain.

The most important features are then used to train the machine learning model. Table 6 show the features and their overall significance to the model prediction as indicated by the CARET R-package. divided by 𝛿𝑡.

0.00

Model Evaluation using ISOT-CID

In this research work, the performance of the three machine learning algorithms was measured using the detection rate (DR) (also known as true positive rate (TPR)) and the false positive rate (FPR) which are two metrics commonly used for IDS performance computation.

The observation time window 𝛿𝑡 was set at 120 seconds for the network flow aggregation and feature extraction. The data was processed following the steps described in figure 7. The DR and the FPR obtained from the three machine learning algorithms for each VM instances over different attack days and their respective overall results were computed. The overall performance of the machine learning classification algorithms is summarised in table 7 for ISOT-CID (covering both phases 1 and 2). The random forest algorithm was the best of the three-machine learning algorithms in terms of performance with a detection rate of 92.08% and a false positive rate of 1.49%. Table 8 shows the confusion matrix for the random forest classifier. The matrix shows the number of flows being classified; the cells contain the numbers of true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN). We examined also the effect of varying the size of the observation time window 𝛿𝑡 on the DR and FPR of the three machine learning models. Day 5 of phase 2 hypervisor B data was used for this analysis. Figures 8 and9 show the effect on the detection rate and the false positive rate, respectively. On the one hand, as the observatio n time window increases, the amount of data available for decision increase thereby presenting the model with a more balanced dataset which will in turn aid in better decision making.

On the other hand, as the time window increases, the decision time is delayed which represents an increased window of vulnerability.

Conclusion

Security and privacy remain one of the main issues faced by cloud co mputing adopters and consequently, there is an urgent need for the IT professionals and subject matter experts to come up with a system that can both detect and protect the cloud infrastructure from malicious activities. In this paper, we carried out an empirical analysis on the DARPA intrusion evaluation dataset and showed its deficiencies when compared to the ISOT-CID which is a real cloud computing dataset. The results support previous work done on network traffic characterization of data centres. It is the claim of this work that due the deficiencies, the DARPA dataset should not be used as a genuine dataset in the design and evaluation of cloud IDS.

Also, we investigated cloud intrusion detection using different supervised machine learning models. The performance results obtained using the machine learning algorithms are encouraging meaning that if more effort and study is channeled into it, academia and researchers can come up with a better way to protect the cloud computing environment against intrusions.

In this paper, the empirical study for the characterization of network traffic to substantiate the difference between a cloud dataset and a conventional dataset was only limited to three flow-level metrics viz, the number of active flows, flow inter-arrival time, and flow -level communication patterns. Our future work will consist of extending the presented work by exploring other empirical means to further understand the nature of network traffic of the cloud and conventional dataset/datacenters like in the areas of packet-level communication, link utilizations and many more.

Figure 2 :

 2 Figure 2: 1999 DARPA IDS Evaluation Dataset Network Flow

Figure 3 :

 3 Figure 3: Timeline for Phase 1 Day 2 (2016-12-15) Inside and Outside Attacks

Figure 4

 4 Figure 4 represents the empirical cumulative distribution function (CDF) of the number of active flows at different switches within 120 seconds time window for both ISOT-CID Phase 2 Day 1 (2018-02-16) dataset and DARPA 1998 Tuesday week 4 Training dataset.Our findings based on the distribution reveals that, the number of active flows for the ISOT-CID is between 2,000 to 6000 about 90% of the time. In the case of the DARPA 1998 dataset, the number of active flows is between 20 and 1000 in 90% of the time interval. This empirical observation supports the results of a prior work on data center traffic[START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF]. It is also considerable to note that the latency assigned by a controller to a new flow is determinant on the lengths of the flows[START_REF] Benson | Network traffic characteristics of data centers in the wild[END_REF].

Figure 4 :Figure 5 :

 45 Figure 4: The CDF of the distribution of the number of flows at the edge switch in ISOT and DARPA

Figure 8 :Figure 9 :

 89 Figure 8: Effect of observation time window on the detection rate

Table 1 : Attacks covered in the ISOT-CID dataset [2]

 1

	Attack Target Layer	Insider Attack Types	Outsider Attack Types
	Application Layer		 SQL Injection
			 Web	Vulnerabilities
			Scanning
			 Cross-site	Scripting
			(XSS)
			 Dictionary/Brute Force
			login attack
			 Fuzzers
			 HTTP Flood DOS
			 Directory/Path
			Traversal
	Network Layer	 Trojan Horse	 Synflood Dos
		 Backdoor (reverse shell)	 Unclassified
			(unsolicited traffic)

Table 2 : ISOT-CID Network Traffic Distribution [2]

 2

	Phase	Total Normal Traffic	Total Malicious Traffic	Total Packets
	1	22,356,769	15,649	22,372,418
	2	9502872	2,006,382	11,509,254

Table 3 : Extra-Rack and Intra-Rack traffic composition for ISOT-CID and DARPA 1998, showing the number of packets.

 3

	Flow Metric	ISOT-CID phase 2 Day 1	DARPA Tuesday week 4
		(2018-02-16) dataset	Training dataset
	Extra-Rack Traffic	693426	932843
	Intra-Rack Traffic	2177148	864810
	Total Traffic	2870574	1797653

Table 4 : Percentage composition of Extra-Rack and Intra-Rack traffic for ISOT- CID and DARPA 1998 Flow Metric (%) ISOT-CID phase 2 day 16 dataset DARPA Tuesday week 4 Training dataset

 4

	Extra-Rack Traffic (%)	24.16	51.89
	Intra-Rack Traffic (%)	75.84	48.11
	Total Traffic (%)	100	100

Table 5 : Some of the raw features extracted from the hypervisor network traffic using the traffic analyzer tool (Tranalyzer)

 5

	Feature	Description
	flowInd	The flow index
	timeFirst	Date/time of first packet
	timeLast	Date/time of last packet
	duration	Flow duration
	srcIP	Source IP
	srcPort	Source port
	dstIP	Destination IP
	dstPort	Destination port
	numPksSnt	Number of transmitted packets
	numPksRcvd	Number of received packets
	numBytesSnt	Number of transmitted bytes
	numBytesRcvd	Number of received bytes

Table 6 : Features and their overall importance

 6 Load feature matching the ratio of the total number of packets flowing to and from endpoint 𝔢 𝑖 during [t , t + 𝛿𝑡]. The number of packets flowing from the endpoint 𝔢 𝑖 to 𝔢 𝑑 during [t , t + 𝛿𝑡] divided by 𝛿𝑡. The number of packets flowing from the endpoint 𝔢 𝑠 to 𝔢 𝑖 during [t , t + 𝛿𝑡] divided by 𝛿𝑡. The maximum number of packets over ip flowing out of 𝔢 𝑖 during [t , t + 𝛿𝑡] divided by 𝛿𝑡. The number of packets flowing from specific ip in 𝔢 𝑖 to all dp in all endpoints 𝔢 𝑑 during [t , t + 𝛿𝑡] divided by 𝛿𝑡. The number of packets flowing from all sp in all endpoints 𝔢 𝑠 to specific ip in 𝔢 𝑖 during [t , t + 𝛿𝑡] divided by 𝛿𝑡. Load feature matching the ratio of the number of packets flowing from all source ports in all endpoints 𝔢 𝑠 to and from specific instance port (𝑖𝑝) in 𝔢 𝑖 during [t , t + 𝛿𝑡]. The number of packets flowing from specific sp in all endpoints 𝔢 𝑠 to all ip 𝔢 𝑖 during [t , t + 𝛿𝑡] divided by 𝛿𝑡. Load feature matching the ratio of numbers of packets flowing to and from the endpoint 𝔢 𝑠 to 𝔢 𝑖 during [t , t + 𝛿𝑡]. The total number of packets flowing from 𝔢 𝑖 during [t , t + 𝛿𝑡]

	Feature			Definition	Overall
				Significance
				(%)
	𝒇 𝒊 𝒊𝒏 (𝒕)	The total number of packets flowing to 𝔢 𝑖 during [t , t + 𝛿𝑡] divided	100
				by 𝛿𝑡.
	𝑳 𝒊 (t)			97.382
	𝒇 𝒊,𝒊𝑷,𝒅,𝒅𝒑 𝒐𝒖𝒕	(𝒕)	87.251
	𝒇 𝒊 𝒐𝒖𝒕 (𝒕)	The total number of packets flowing from 𝔢 𝑖 during [t , t + 𝛿𝑡]	86.431
				divided by 𝛿𝑡.
	𝒇 𝒔,𝒔𝑷,𝒊,𝒊𝑷 𝒊𝒏	(𝒕)	85.590
	𝒎𝒂𝒙 𝒊𝒑 {𝒇 𝒊,𝒊𝒑 𝒐𝒖𝒕 (𝒕)}	8.298
	entropy_dp	Entropy of the destination port	8.023
	𝒇 𝒊,𝒊𝒑 𝒐𝒖𝒕 (𝒕)	7.877
	numPksRcvd	Number of received packets	7.877
	numPktsSnt	Number of transmitted packets	6.992
	𝑳 𝒎𝒂𝒙 (𝒊𝒑) (t)	Load feature matching the ratio of the maximum number of	3.098
				packets over the instant port (𝑖𝑝) flowing to and from endpoint 𝔢 𝑖
				during [t , t + 𝛿𝑡].
	𝒇 𝒊,𝒊𝒑 𝒊𝒏 (𝒕)	2.915
	𝑳 𝒊,𝒊𝒑 (t)	2.750
	𝒇 𝒔𝑷,𝒊 𝒊𝒏 (𝒕)	0.00
	𝑳 𝒔,𝒔𝒑,𝒊,𝒊𝒑 (t)	0.00
	𝑳 𝒔𝒑,𝒊 (t)	Load feature matching the numbers of packets flowing from	0.00
				specific source port in all endpoints 𝔢 𝑠 to and from all instance
				ports in endpoint 𝔢 𝑖 during [t , t + 𝛿𝑡].

Table 7 : Comparison of overall performance for ISOT-CID

 7

	Algorithm	Overall
		FPR (%)	DR (%)
	Logistic Regression	2.61	90.52
	Random Forest	1.49	92.08
	SVM	1.84	92.06

Table 8 : Confusion matrix for random classifier based on ISOT-CID

 8

			Prediction
			Attack	Normal
	Reference	Attack	TP=114291	FN=9831
		Normal FP=4428	TN=292775