Akram Hakiri
email: hakiri@laas.fr

Aniruddha Gokhale
email: a.gokhale@vanderbilt.edu

Data-Centric Publish/Subscribe Routing Middleware for Realizing Proactive Overlay Software-Defined Networking

Keywords: CCS Concepts, Networks → Routing protocols, Overlay and other logical network structures, •Software and its engineering → Message oriented middleware, Publish/Subscribe, Overlay Network, Software-defined Networking, Network Virtualization

Software Defined Networking (SDN) has emerged as an attractive solution to allow cloud-to-cloud interconnection and federation. SDN technologies, such as OpenFlow, use both reactive hop-by-hop and proactive approaches to program the switches. The reactive strategy incurs substantial scalability problems for large networks due to the hop-by-hop behavior while the proactive approach is hard to implement in practice due to the need to forecast all possible forwarding rules ahead-of-time. An attractive and more realistic alternative is the proactive overlay SDN approach, however, many challenges must first be overcome to realize it. Existing techniques to program the switches use low-level programming abstractions, which are error-prone and cannot scale. Middleware-based solutions, e.g., using XMPP, are stateful and hence also incur substantial scalability issues. Although content-based publish/subscribe (pub/sub) solutions have been used in the past for SDN, they rely on brokers, which is problematic and incurs unnecessary additional infrastructure elements that pollute the SDN architecture. To address these issues, this paper demonstrates how the strengths of the data-centric, broker-less pub/sub paradigm can be exploited to realize proactive overlay SDN for inter cloud domain federation. To that end, we first present the design rationale and architecture of our solution called PO-SEIDON (Proactive brOkerless SubscribEr Interest-Defined Overlay Networking). Second, we present the messaging protocol between the controller and switches. Finally, we present results of evaluating POSEIDON and illustrate how it improves data delivery and provides high performance at the network-level in proactive overlay SDN.

INTRODUCTION

Service providers are increasingly using data centers for offering services such as video steaming. This approach has been validated by the success of services, such as YouTube and Netflix. This trend is further growing with the advent of Internet of Things (IoT), where the applications are making increasing demands on computation, storage, and network resources, in turn forcing service providers to consolidate groups of data centers in the form of cloud domains. The key requirement is that resources -in our case the network -should be made available any where and any time. Unfortunately, the current network architecture is not suited to support the elastic and on-demand networking resource demands stemming from the federation of cloud domains.

Software Defined Networking (SDN) [START_REF] Nunes | A survey of software-defined networking: Past, present, and future of programmable networks[END_REF] has emerged as a promising technology for network management, e.g., to provide new capabilities for managing individual flows, automate resource provisioning, and provide more agility in deploying, configuring and instantiating cloud networking. The separation of the packet forwarding logic from the data plane, i.e. from the switches, and handing it to an external controller, which embeds all the network intelligence, makes it easier to manage and control the switches and networks since any change must now be made only in the controller.

OpenFlow [START_REF] Mckeown | Openflow: enabling innovation in campus networks[END_REF] is a dominant SDN technology that allows the controller to install flow rules in the underlying switches it controls to enable packet forwarding. OpenFlow provides both reactive and proactive approaches to setup flow rules in the switches. In the reactive flow setup, for every new flow that arrives at a switch, that switch should send a packetin event to the controller so that the latter can send the switch all the flow rules that it should apply to any incoming packets belonging to that flow. This behavior is illustrated by every switch that lies in the flow path. The controller contacts all the switches between the source and destination of a flow in a point-to-point manner to install the desired forwarding rules.

The reactive approach incurs several performance and scalability limitations when the number of switches in a flow increases, which is often the case when connecting distributed cloud domains. The problem is further exacerbated for finegrained flow matching in latency-sensitive, long-lasting short flows because of an explosion in the number of forwarding states [START_REF] Shirali-Shahreza | Rewiflow: Restricted wildcard openflow rules[END_REF]. To address the overhead stemming from this hop-by-hop behavior of the reactive approach, the proactive flow setup approach was conceived. In this approach, the controller populates all possible flow rules into the switch ahead-of-time for all possible traffic that could arrive at that switch. This behavior is akin to typical routing tables in traditional networks.

Although the proactive flow setup approach tries to provide better performance by avoiding the extra-latency of the first packet matching [START_REF] Fernandez | Comparing openflow controller paradigms scalability: Reactive and proactive[END_REF], it is hard to realize in practice be-cause of the need to forecast all possible flow entries that can match against future arriving packets. Moreover, irrespective of the approach used, the OpenFlow switches must buffer a number of short flows in the flow table. However, typical buffer sizes do not exceed a few hundreds of packets thereby causing memory overflows. Subsequently, an extra latency may be incurred in forwarding packets across large networks as the number of new flows to be programmed increases and the SDN controller has to setup all the properties for those flows onto the SDN switches it controls.

With the advent of Internet of Things and cloud federations, it is becoming increasingly important to be able to interconnect multiple cloud domains. Although the proactive SDN approach is promising, as discussed above, the concept is mostly of theoretical value and hard to implement in practice. To meet these urgent needs, we surmise that the easy-to-implement functionality of the reactive approach can be blended into the proactive approach, however, without incurring the scalability issues of the reactive approach by enabling the proactive SDN at an overlay layer. The overlay property stems from the ability to use multiple tunnels to create network slices whose endpoints terminate in a variety of entities, such as virtual switches or physical edge routers. In such an overlay approach, only the endpoint switches of the tunnels must be programmed by the controller instead of every switch in the flow path.

A number of challenges manifest in this proposed approach. The first challenge stems from having to decide which technique to use in the SDN controllers and switches to program the endpoints of the tunnels that reside in the forwarding plane. One approach is to leverage existing Open-Flow APIs. However, these APIs are too low-level and lack intuitive abstractions that can support compositional semantics [START_REF] Han | Blueswitch: enabling provably consistent configuration of network switches[END_REF] to combine a sequence of rules and process them in parallel. Middleware solutions have tried to address these issues. For instance, some approaches have adopted the Extensible Messaging and Presence Protocol (XMPP) [START_REF] Marques | Bgp-signaled end-system ip/vpns[END_REF] to distribute the control plane and management plane information to end servers. XMPP, however, is a stateful protocol that requires a server to retain history information status carried over from the previous session processing. Subsequently, XMPP creates significant overhead of data when multiple users are interconnected and thereby suffers from scalability issues.

The authors in [START_REF] Koldehofe | The power of software-defined networking: Line-rate content-based routing using openflow[END_REF] introduced line-rate, content-based middleware and matching semantics to disseminate routing information to SDN-enabled switches. Similarly, authors in [START_REF] Bhowmik | Distributed control plane for software-defined networks: A case study using event-based middleware[END_REF] and [START_REF] Tariq | Pleroma: A sdn-based high performance publish/subscribe middleware[END_REF] introduced content-based middleware to enable group communication as well as control and management tasks among distributed SDN controllers, respectively. However, a significant limitation of content-based routing systems is the lack of decoupling between publishers and subscribers for distributed SDN. In this approach, both participants remain tightly coupled to a broker for message delivery. Moreover, the broker cannot guarantee message delivery to applications, which may require additional enforcement outside of the existing middleware solutions.

To address these concerns, we propose Proactive brOkerless SubscribEr Interest-Defined Overlay Networking (PO-SEIDON), which is a broker-less, extensible architecture for proactive overlay SDNs that can be realized within Open-Flow. POSEIDON is a publish/subscribe (pub/sub) middleware that exploits the expressiveness power of the data-centric information model to allow the controller to configure and manage the data plane, and to describe the states of the network elements, e.g. creating route instances, exchanging routes and VPN membership information, and managing link status for connection and disconnection. POSEI-DON requires proactively programming only the endpoints of the tunnel at boot-time or at policy creation-time. The data plane can thereafter populate packets over these overlay tunnels. The result is that all packets are forwarded at line-rate, which eliminates any unnecessary latency induced by consulting the controller for every flow thereby improving scalability. To overcome the low-level programming issues, POSEIDON uses the Pyretic [START_REF] Reich | Modular sdn programming with pyretic[END_REF] description language to offer a higher-level of abstraction that is both intuitive and hides the complexity of the underlying network.

The following benefits accrue from our work:

• POSEIDON has full configuration and control of how it processes and forwards packets through the SDN data plane.

• When upgrading the OpenFlow-enabled switches, PO-SEIDON can continue to perform the forwarding operation independently from the underlying firmware without deteriorating network performance.

• The use of a higher-level of abstraction enables POSEI-DON to provide a combination of fine-grained flows at the virtual edge switches and coarse-grained flows in the physical underlay core network devices.

• The use of pub/sub middleware provides higher levels of structure and semantics to build common and computation-independent services.

The remainder of this paper is organized as a follows: Section 2 uses two concrete real-world case studies to elicit the solution requirements for POSEIDON; Section 3 describes the design of POSEIDON and its messaging protocol illustrating how they meet the solution requirements; Section 4 shows how the POSEIDON architecture can address the needs of our motivational case studies; Section 5 presents results of experiments evaluating various properties of PO-SEIDON; Comparison with related work is described in Section 6; and finally, conclusions drawn from this work and lessons learned are presented in Section 7.

MOTIVATIONAL CASE STUDIES

This section describes motivating use cases, which are used to elicit key requirements that must be met by PO-SEIDON.

Use Case 1: Data Center Networking and Virtualization

High bandwidth multimedia applications like YouTube and Netflix are typically hosted over distributed data centers connected over different ISPs which may use data replication on different sites to ensure the availability and reliability of their services. The traffic generated at an ISP's edge network may be aggregated over virtual tunnels with a certain bandwidth and forwarded to end users for performance and efficiency sake. Accordingly, applications should be able to discover and register remote endpoints (e.g., BGP routers, virtual machines, etc), and negotiate network resources to provision and virtualize multiple virtual links to construct an overlay network for groups of users. The provisioning service should allow creating, modifying, and updating network tunnels between the discovered endpoint devices.

A key requirement is that these applications need a mediator layer/service to coordinate the communication between applications in virtualized servers or tenants and their underlying network to provision their QoS requirements. Subsequently, these requirements should be communicated to a control service that takes the configuration from the mediator service to provision and virtualize the underlying network resources. Additionally, ISPs need to constantly monitor the SLA (Service Level Agreement) conditions with remote applications and adjust the network resources if necessary. Therefore, a monitoring service is required to gather network topology information at a higher level of abstraction, and detect and report network failures.

Use Case 2: Internet Exchange Point

A Software-defined Internet Exchange (SDX) [START_REF] Gupta | Sdx: A software defined internet exchange[END_REF] is a large layer 2 domain that interconnects participant router peers of multiple ISPs together using the BGP routing protocol. It offers capabilities to address inter-domain routing issues by creating virtualized connections between routing endpoints. In the traditional approach, each BGP router is required to "learn" by itself the presence of other routers, select the best path for each prefix and re-advertise the new selected path. Additionally, a given router needs to control multiple switches at once and advertise its data simultaneously to multiple ISPs networks. These data need to be queued and made available for joining participants. For example, a router maybe be interested in a subset of traffic that conveys web HTTP traffic to its clients, or a subscribing router may be interested in video streaming traffic for its end users.

SDX may benefit from using the more expressive, proactive overlay SDN policies than conventional hop-by-hop destination based forwarding. SDX may also benefit from the data-centric, pub/sub for disseminating data only based on subscribers' interests.

Solution Requirements

Based on the real-world scenarios above, we derive the following requirements that must be met by POSEIDON:

1. Since the underlying network needs to be dynamically reconfigured, there is a need for the control plane to be notified whenever there is a change in the underlying physical network. This motivates the need for a publish/subscribe-based solution.

2. It is also important to provide the users with a generic policy framework that hides the heterogeneity in the underlying network devices. Thus, it is desirable to have a common interface and messaging capability between the control plane and forwarding plane where new devices can be added seamlessly.

3. It is important to provide the user with a flexible and higher-level of abstraction to program the system that can handle QoS issues and policies.

4. It is important for each instance of the virtualized overlay network instance to run in isolation from other networks, and to allow dynamically scaling up and down the number of applications/users in the virtualized data center. This motivates the creation of tunnels at the endpoint so that users/applications can be placed or migrated seamlessly without impacting the network performance.

POSEIDON ARCHITECTURE

This section details the POSEIDON architecture explaining how it meets the solution requirements of Section 2.3. The fundamental contribution of POSEIDON lies in how it realizes the proactive overlay SDN idea using data-centric pub/sub middleware. The use of data-centric, topic-based pub/sub is critical to our design because information can be represented as topics that can be easily queried using a variety of filter expressions, and can be disseminated to only interested entities thereby improving scalability.

To be compliant with the SDN architecture of separating the control plane from the data plane, and to not pollute it with extra infrastructure elements such as brokers, PO-SEIDON supports only two key elements as illustrated in Figure 1: a POSEIDON agent that resides in the control plane, and a routing agent that resides in the forwarding plane. The former relies on a policy management service in the SDN controller which supports intuitive and expressive interfaces at a higher-level of abstraction for applications to use. The latter receives these higher-level policy descriptions and translates them to low-level configuration commands in the switch thereby completely relieving the applications and developers from low-level programming. POSEIDON provides a southbound interface based on the data-centric, pub/sub paradigm for communication between the two types of agents with a well-defined messaging protocol and message types. POSEIDON ensures no disruption to existing OpenFlow-based SDN deployments since it encapsulates the original OpenFlow messages within its messages that are exchanged between the POSEIDON agent and routing agents. POSEIDON also allows new network devices to be introduced into the system without any disruption to the currently operational system. This feature is derived from the pub/sub system's discovery service, which promotes interoperability between endpoints so that remote participants can (i) dynamically learn about each other by sending participant declaration messages; and (ii) exchange information (e.g., QoS, data types, etc.) to match each other by sending pub/sub declarations. Consequently, POSEIDON provides more flexibility and programmability to users.

Intuitively, the POSEIDON middleware can manage one or more interconnected network slices, where each network slice is composed of a set of configurable SDN switches. Two or more neighboring network slices are interconnected inside a virtual global data space (GDS). Multiple GDSs are then connected to each other through edge SDN switches. PO-SEIDON exploits the expressiveness of data provided by the topics. Each topic is associated with the publication of data between each module. The content of each topic includes internal fields of our pub/sub routing services.

The rest of this section delves into the details of POSEI-DON explaining the rationale for the design decisions.

POSEIDON Agent: The Control Plane

Figure 2 shows the internals of the POSEIDON agent, which comprises four types of nodes: monitor node, policy node, configuration node and control node. The monitor node collects information related to the health of the system including faults, errors, etc. The policy node includes a policy decision point and policy enforcement point both connected to a policy data store. The configuration node communicates with the orchestration layer via REST APIs to the applications. It also communicates with other configuration nodes through the distributed synchronization services provided by the built-in discovery mechanisms of the pub/sub system and with the policy node through a requestresponse model. The control node is used to define the rules for the proactive overlay. We now describe each of these node types in detail providing a rationale for their need in our design.

Configuration Node

The configuration node serves as the link to the cloud orchestration and automation systems using RESTful interfaces, which offer loosely coupled communication and high scalability. RESTful interfaces are used by cloud operators/applications to install configuration states to perform traffic engineering, e.g., load balancing. The configuration node can communicate with both the policy node and control node. For instance, the configuration node can query the policy node for the network virtualization policy used in the orchestration system. Similarly, it can direct the control node to install policy decisions in the network device. The configuration node uses topic-based filters provided by the pub/sub system for these communications. Topic-based filters are important because queries should return only pertinent information thereby reducing control-level traffic.

Policy control

Recall that network switches must be programmed to forward incoming traffic based on some rules specified by the controller. Our goal is to reduce the frequent and potentially non-scalable communication between the SDN controller and switches. Such forwarding rules and network management functions can be driven by different policies.

Figure 3 depicts the policy management service inside the policy node. This service provides policy control functions to the configuration node for ensuring best resource orchestration. To that end, the policy node infers policy decisions from application requirements specified through the configuration node and communicates them to the control node for enforcement. The policy module specifies how packet forwarding/updating abstractions should be realized. The core of the abstract set of policies we use is defined in [START_REF] Foster | Frenetic: A network programming language[END_REF] and [START_REF] Monsanto | Composing software-defined networks[END_REF]. The policies include a set of primitive actions, predicates, query policies, policy combinators and compositions. The primitives can be mixed to create complex policy functions to handle packets. The policy control functions ensure that the POSEIDON agent can continue to honor guaranteed service delivery to end-users. The policies also determine how network resources are allocated as well as how individual subscribers can take into account network flow control and application-oriented flow.

Policy

These policy abstractions are defined using the Pyretic description language [START_REF] Foster | Languages for software-defined networks[END_REF], which enables users to express conflict resolution by composing different policies into a single set of matching rules in the network devices. Hence, rather than supporting many policies, the abstraction policy layer combines policies from multiple participants to generate a set of rules that can be used for route advertisements without flooding the flow tables.

Pyretic provides a higher-level of abstraction to the operators, and automates the mapping of these higher-level descriptions into topic filter expressions used for querying.

The policy database shown in Figure 3 is a repository of multiple abstract policies that can be composed using multiple parallel and sequential composition operators [START_REF] Reich | Modular sdn programming with pyretic[END_REF]. For instance, the sequential composition ( ) redirects the output of one policy to the input of another forming a pipeline of sequential policies. Likewise, the parallel composition operator (+) applies multiple policy functions to the same packet and combines the results.

Control Node

The control node is responsible for installing the rules in the switches. It receives configuration states from the configuration node using content-based filtered topics, which contain the forwarding rules to be sent to the routing agent for installing new rules in the forwarding plane (i.e., data plane). Additionally, it exchanges network route information with other distributed control nodes and routing agents using the POSEIDON protocol. It also sends forwarding policies into the policy database for enabling intelligent decision making without having to contact the controller every time a new flow arrives. The control node holds a global snapshot of the network and the link states which helps it in performing the best route selection. Hence, the controller can configure virtual overlay tunnels based on the high-level information provided by the configuration node and the policy management service inside the policy node of the POSEIDON agent. To that end, the control node uses three types of topic-based messages for communication with the switches. The messaging protocol is described in Section 3.3.

Monitoring, Analysis and Troubleshooting

Monitoring the health of the system is critical. Figure 4 shows the internal structure of the monitor node. A monitor node supervises the network resources and communicates with applications using northbound REST interfaces. These interfaces are used to query statistics and analysis, and retrieve network operational states using the POSEI-DON agent. Thus, the POSEIDON agent performs flexible and robust monitoring built atop the pub/sub model. Furthermore, the monitor node contacts the control node using the distributed management module to collect data from the network. The monitor node uses a data collector to communicate with the policy management module through the content-filter interface. That is, the monitoring service uses filter expressions to select data samples of interest by using SQL-like expression to store the information in the policy database.

Routing Agent: The Data Plane

As depicted in Figure 5, the routing agent is a user space process available on each instance of an OpenFlow-capable switch and acts as a local translator at the forwarding plane. It is responsible for exchanging control states between the POSEIDON Agent and the network devices using the PO-SEIDON protocol.

POSEIDON Agent

OpenFlow Capable switch

Routing Agent

POSEIDON Protocol The routing agent receives high-level configuration information from the POSEIDON agents and translates them to low-level details (i.e., QoS, VLAN tags, etc.), which is understood by the switches in the overlay. To support a uniform abstraction that can cover a range of these low-level possibilities, POSEIDON supports the notion of a built-in data isolation called Partition, which is a scoping mechanism to organize traffic by creating non-overlapping logical virtual data spaces for dynamic association of participants. Partitioning the network into several network slices can increase the scalability of the system by allowing multiple advertisements and subscriptions.

POSEIDON Agent

For virtual LANs (VLANs), Partitions can be seen as VLAN identifiers (i.e., VLAN tags) so that routing agents belonging to the same physical infrastructure can easily be isolated into different VLANs. These partitions are mapped into specific VLAN ports for grouping the sent data into specific receivers. Consequently, the routing agent translates these partitions into VLAN tags and installs the forwarding states within the OpenFlow entries of the SDN switches.

Likewise, a partition can be translated to an IP multicast address to perform multicast VLAN registration to efficiently distribute data across switches thereby reducing the bandwidth consumed by the multicast traffic. In such a case, POSEIDON can check whether routing agents belonging to a specific partition are still alive. The pub/sub discovery process reports whether new virtual switches have joined or left the network. Besides, the routing agent carries and reports analytics, health status, errors, and statistics from the forwarding plane to the distributed POSEIDON agents.

POSEIDON Messaging Protocol

In this section we describe the three types of messages currently supported by the POSEIDON messaging protocol: peer discovery, network configuration, and monitoring messages, which are exchanged between the control node of the POSEIDON agent and the routing agents.

Endpoints Discovery Message

POSEIDON relies on the underlying pub/sub system's built-in discovery protocol to identify the tunnel endpoints matching POSEIDON agents and routing agents. Discovery messages are sent periodically (i.e., regulated by a heartbeat) to check the liveliness of different POSEIDON peers. Listing 1 shows the peer discovery message exchanged between POSEIDON agents and routing agents. The "sender" field describes the IP address of the publishing endpoint. It can also contain the MAC address in the context of a layer 2 participant. The "sender ID" describes the "VLAN ID" of a participant or any other membership information that can uniquely identify a participant within its group. The format is extensible and customizable to the underlying pub/sub system.

Listing 1: Built-in participant Discovery <?xml version="1.0"?> <--Discovery topic --> <struct name="discovery"> <member name="sender" type="string" key="true"/> <member name="sender_ID" type="string" /> <member name="receiver" type="string"/> <!--QoS policies ... --> <member name="LivelinessQoS" type="string"/> <member name="DurabilityQoS" type="string"/> <member name="TopicDataQoS" type="string"/> <member name="GroupDataQoS" type="string"/> <!--more QoS policies ... --> <!--Session Management --> <member name="scope_announce" type="sequence"> <member name="register_context" type="sequence"> <member name="session" type="sequence" key="true"/> </struct> For example, depending on the underlying pub/sub system's features, the discovery message supports reconfigurable QoS policies to control many reliability aspects, such as liveliness. For instance, when using the OMG Data Distribution Service (DDS) pub/sub technology, in Listing 1, "Liveli-nessQoS " is used to ensure automatic discovery and liveness check of endpoints. The endpoint discovery process uses inbound session management to ensure that communication will be established end-to-end. The field "scope announce" in Listing 1 describes the announcement of a session from a participant to all others. The "scope announce" is a sequence of fields that includes the forwarding path, the session heartbeat, and discovery data.

Network Configuration Operation

Once the endpoints are discovered, it is important for the POSEIDON agent to configure all the identified routing agents with flow rules. Listing 2 illustrates a configuration message used by the POSEIDON agent to populate the Routing Information Base (RIB) of the routing agent.

Listing 2: Configure BGP Path <?xml version="1.0"?> <struct name="bgp_advertise"> <member name="sender" type="string" key="true" /> <member name="receiver" type="string"/> <member name="sender_id" type="long"/> <member name= "node" type="string" /> <member name="instance-id" type="long"> <--! BGP Network Layer Reachability Information (NLRI) --> <member name="nlri" type ="string"> <!--Label for label switch routers (LSRs) --> <member name="label" type="long"> </struct>

Monitoring messages

Currently, POSEIDON supports network management and monitoring capabilities through the monitoring message. Listing 3 shows the topic data used by POSEIDON to query the state of a SDN switch. This topic message contains information about the fault, statistics, errors, and health of the network device. It also includes information about the sender of that request (i.e., IP address, VLAN tag), the entity which sent the report and other information including statistics, faults, and errors, etc.

Listing 3: Query Network Statistics <?xml version="1.0"?> <struct name="report_sate"> <member name="sender" type="string" key="true" /> <member name="sender_id" type="long"/> <member name="receiver" type="string"/> <member name= "node" type="string" /> <member name="statistics" type="struct" > <member name="faults" type="struct"> <member name="health" type="struct"> </struct>

VALIDATING THE POSEIDON APPROACH FOR REAL-WORLD USE CASES

The aim of this section is to validate the POSEIDON approach for the same use cases we used to motivate the work.

POSEIDON for Interconnecting Distributed Cloud Networks

Figure 6 illustrates how POSEIDON can be used to scalably interconnect distributed cloud networks. In contrast to the traditional approach that uses physical links (shown as P LI , i = 1..5 in Figure 6) to exchange routing information, POSEIDON provides a topology abstraction to simplify interconnection by virtue of creating a big virtual switch "v sw". This is feasible because the abstracted physical topology can be ported to a collection of distributed routers due primarily to the policy repository (Section 3.1.2). Recall that this repository holds a topology abstraction library that determines whether to aggregate multiple underlying switches into a single overlay or splitting a physical node into several virtual ones.

Listing 4 using the expressive description in Pyretic shows the topology transformation of the underlying network where the physical switches "SW1, SW2, and SW3 " are aggregated to create the virtual switch "v sw" in the overlay network. The next step consists of creating the transformation policy which allows virtualizing the underlying network. To generate the topology abstraction policy that abstracts the underlying switches into the virtual switches, the virtualize() function in Listing 5 takes SW1, SW2, and SW3 as input and renders "v sw" as a topology transformation. The merge abstracted policy takes a list of the underlying physical switches and renders a single overlay virtual switch "v sw". It selects the shortest path to forward packets from one edge router to another. Thereafter, the routing agent at the customer edge router implements translation of the virtualize policy into a single new policy for the underlying router. This abstracted policy creates a virtual tunnel in the overlay network to allow distributing the route's information among autonomous systems and enable a high level of aggregation between customer interfaces and provider's edge devices. Each customer device implements a POSEIDON Agent to translate the configuration sent by the control plane through the POSEIDON protocol.

POSEIDON for Internet Exchange Point

Figure 7 depicts how POSEIDON uses the BGP protocol for simplifying the exchange of network reachability information using a POSEIDON agent in the SDX controller. Unlike traditional approaches where each BGP router should "learn" by itself the other routers, our approach acts as a route reflector that learns BGP route advertisements, selects the best path for each prefix, and re-advertises the new selected path on the appropriate session using the scope announce field in the endpoint discovery message described in Listing 1.

The POSEIDON agent runs an SDN application to drop, modify, update, and forward traffic among different Autonomous Systems (ASs). In particular, it combines BGP The policies include conflict resolution rules to program the virtualized switch and avoid the policies' interference conflicts between edge routers. Additionally, the POSEI-DON agent implements a router reflector to replace the traditional BGP route server [START_REF] Jasinska | Internet exchange bgp route server[END_REF]. These routing decisions are based on per-endpoint policy control and the best-path calculation for each set of paths available between the distributed routes.

The route reflector is an SDN application at the control plane that checks the advertised routes from each AS and performs path filtering at that exchange endpoint. It also allows enforcing the traffic ratio among different routing endpoints. POSEIDON splits the virtual tunnel of AS C into two small virtual tunnels, each conveying the filtered traffic for each AS separately as shown in Figure 8. Consequently, each network provider can apply a customized route selection process to select one or more best routes to each Internet destination.

In traditional approaches, each BGP speaker should be configured separately to provide point-to-point communication with all the other speakers. In contrast, in our approach the big virtual switch in Figure 8 hides the complexity of configuring each BGP speaker separately. The virtualization policy supported by our POSEIDON agent creates virtual tunnels between edge routers. It creates the overlay big switch by virtualizing the underlying network, i.e. the physical BGP edge routers. The resulting virtual overlay switch maps the ingress physical ports into virtual ports to create the illusion that each router has its own switch interface.

Furthermore, the control plane translates the local RIB into the corresponding prefix in each nearest BGP speaker to forward multiple paths for the same speaker on a single BGP session. Path adding/removing is negotiated bidirectionally between the POSEIDON Agents and the service provider routers. The POSEIDON agent ensures that there is no inactive, invalid or even suboptimal path to its peering routers.

PERFORMANCE EVALUATION

This section describes results of experimental evaluation of POSEIDON. Specifically, we describe the results for different network metrics such as bandwidth, latency, network overhead, and scalability.

Experimental Setup

POSEIDON has been evaluated for the two use cases described in Section 4. The experiments were conducted on a SDN testbed running the Mininet SDN tool [START_REF] Lantz | A Network in a Laptop: Rapid Prototyping for Software-defined Networks[END_REF]. Mininet is a Linux-based lightweight SDN virtualization tool for running large number of OpenFlow network equipment, and allows experimenting with different types of network topologies. We also integrated the different modules comprising our solution into the NOX [START_REF] Gude | Nox: Towards an operating system for networks[END_REF] SDN controller, which is written in C++ and Python programming languages. We implemented the reactive OpenFlow in the SDN controller so that each time a new packet arrives at the edge router, it sends a request using PACKET IN message to the controller. The controller examines the packet header of the incoming packets and checks whether a new entry should be created and new actions should be applied to those packets. Then, the controller sends PACKET OUT message to all the underlying routers from the source to the destination. Additionally, to implement the proactive approach, we used the OMG Data Distribution Service (DDS) as the underlying topic-based, data-centric pub/sub middleware. Thus, POSEIDON can create virtual tunnels between edge routers. To install new rules for establishing the tunnel, the controller sends OpenFlow messages to the tunnel endpoints, i.e. the edge routers of that tunnel.

Evaluating the End-to-end Latency

Rationale. End-to-end latency is critical to many SDNenabled applications since it allows routing traffic in predictable time scales. To evaluate the timing performance of our solution, we consider the end-to-end delay as the time duration to send a packet from source to destination. The measurement of the one way delay is not straightforward because packets experience different network delays including processing delay, queuing delay, transmission and propagation delays. Thus, we have considered the two-way latency, i.e., the Round Trip Time (RTT). We then consider one-way delay as half of RTT. Furthermore, to evaluate the effectiveness of our approach, we compared the latency results with those of the traditional, reactive hop-by-hop OpenFlow data dissemination. 1Analysis. Figure 9 depicts the end-to-end latency for both the reactive OpenFlow and our proactive overlay SDN approach. A close inspection of Figure 9 shows that the latency experienced by our proactive overlay solution is close to 20 milliseconds on average (reaching 35 msec in worst case), which is better than the reactive approach that showed worst-case latency up to 75 milliseconds. Indeed, the reactive OpenFlow is constantly changing in reaction to the current network conditions. The volatile reactivity to the changing events introduces high rate of changes because reactive OpenFlow must continually update all the switches along routing paths. Moreover, the OpenFlow agents in every switch should lookup in their flow tables to check whether a matching rule/action for a given packet flow can be found. If no matching is found in the flow entries, the OpenFlow agents should send requests, i.e. PACKET IN messages, to the controller to create new flow entries for that given flow. Accordingly, reactive OpenFlow experiences larger latencies and substantial jitter than our proactive solution, which attempts to address the latency concerns before it becomes a problem in the network. As seen, our overlay routing paths do not introduce extra processing latency in the overlay network and demon-strate less jitter. Rather, our approach creates a virtual tunnel between two end switches along the routing path so that packet processing is enabled only at the beginning and the end of the tunnel. Our solution improves the network latency by enabling proactive overlay SDN anonymity to all applications running on top of overlay routing protocols. Our approach preserves the higher performance characteristics required over the overlay routing path while hiding the complexity of the underlying network topology.

Evaluating Bandwidth usage

Rationale. Since SDN networks must share the same physical infrastructure with other existing applications, the bandwidth usage is a concern. Thus, data dissemination in POSEIDON middleware should be protected against any network fluctuation such as congestion. To evaluate the usage of the shared links, we consider each application generating best effort traffic close to 10 MB/sec. Our aim is to evaluate whether our approach shows better results to protect the flow in the face of bandwidth fluctuation. We also compared our approach with the reactive OpenFlow solution.

Analysis. Figure 10 illustrates the bandwidth usage for both the POSEIDON approach and the reactive OpenFlow. Both approaches allow sending data at 10 MB/s. However, reactive OpenFlow experiences several bandwidth limitations and the traffic becomes irregular. The bandwidth decreases drastically at different time intervals. The packet flow experiences several packet losses. In contrast, in our proactive solution, the throughput decreased only at the initialization phase comprising the creation of the tunnel, i.e., establishing the connection between two end switches across the overlay routing path. Thereafter, the bandwidth becomes regular and protected against the fluctuation during the rest of the experiment. Accordingly, the results show that our solution is successfully able to overcome the limitations of the reactive OpenFlow approach by providing better usage of network bandwidth. This is because the POSEI-DON agent at the controller receives fewer request messages from the data plane since the virtual path is already setup inside the virtual tunnel.

Evaluating POSEIDON Scalability

Rationale. The design choice of our proactive SDN solution is similar to distributed systems as it was designed with scalability concerns in mind to allow its deployment in largescale SDN scenarios. Therefore, to evaluate the scalability of our proposed solution, we consider the average response time in processing the network requests. Specifically, we in-creased the number of the data plane nodes and we evaluate the time latency for every additional introduced node.

Analysis. Figure 11 depicts the latency of the proposed proactive SDN approach as a function of the number of data plane equipment across the path. A close inspection of the figure reveals that the average latency of our proposed solution is close to 30 ms, while the maximum value of this delay remains close 50 ms. Since our proactive approach creates a virtual tunnel between end points, increasing the number of core routers across the data path will not necessarily linearly impact the performance of the network. There are a couple of reasons for this: first, core routers are usually high performance network equipment that forward incoming packets without any additional processing; second, the core routers remain transparent to the network as the virtual tunnel will cross them seamlessly.

Evaluating Network Overhead

Rationale. As POSEIDON is used for interconnecting data centers, there are several critical metrics that should be considered in evaluating the overhead introduced by PO-SEIDON. First, the CPU load is usually impacted by the control modules such as the configuration module to perform connection setup and tear down. Moreover, the forwarding operations performed by the POSEIDON routing agent can come at a cost of increased memory utilization in the overlay switches, which is a critical issue in data center networks. In particular, memory and queuing buffers hold packets and connection state information of the traffic transiting across the switch. Therefore, there is a need to maintain the memory utilization as low as possible to avoid buffer overflow.

Analysis. Figure 12 illustrates the control plane overhead for both the reactive OpenFlow approach and our proactive solution. The figure shows that the control traffic of the former varies between 5 Kbits/sec and 15 Kbits/sec, while the network overhead of the proposed proactive approach is less than 5 Kbits/sec most of the time and increases a little up to 5 Kbits/sec at regular time intervals. Indeed, the reactive OpenFlow requires processing several network requests for network state update, topology discovery, failure recovery, etc. Unlike the reactive approach, our proactive overlay solution sends Endpoints Discovery Message that acts like heartbeat messages at heartbeat periods to discover whether new nodes have joined or left the network. As discussed in Section 3.3, the discovery messages are sent periodically to check the liveliness of different peers. In particular, the LivelinessQoS and the scope announce parameters ensure Figure 12: Control Plane Overhead: Reactive OpenFlow vs. Proactive POSEIDON automatic discovery of end points as well as heartbeat periodic data discovery. Thus, the evaluation of the network overhead confirms our claims on providing better network performance compared to the traditional reactive approach.

RELATED WORK

In this section we survey relevant research efforts and compare them to POSEIDON. Our SDN-based literature survey indicates that there does not exist a clearly identified role for middleware platforms (e.g., whether it should be used in the northbound or southbound or east-west interfaces) with the aim of interconnecting applications to SDN network and providing the freedom to the developers to define the way they would use it in SDN. There have been some efforts, however, to introduce middleware in SDN.

The authors in [START_REF] Phemius | Disco: Distributed multi-domain sdn controllers[END_REF] introduce a broker-based framework called DISCO as an east-west interface to coordinate federated SDN controllers using RabbitMQ [START_REF] Ionescu | The analysis of the performance of rabbitmq and activemq[END_REF] middleware over the Advanced Message Queuing Protocol (AMQP) [START_REF] Vinoski | Advanced message queuing protocol[END_REF]. DISCO allows the brokers to make routing decisions rather than letting the SDN network application perform this task. As a broker-based solution, DISCO routes the control traffic between distributed controllers, however, it does not perform any routing decisions at the SDN forwarding plane, which is left to control plane applications. In contrast, POSEIDON supports self-formed federation that can perform data dissemination/advertisement over the global data space of topics. As such, POSEIDON operates as a repository federation in which individual repositories can participate in a global federation in a fully distributed manner.

Authors in [START_REF] Hakiri | Publish/subscribe-enabled software defined networking for efficient and scalable iot communications[END_REF] proposed a data-centric IoT architecture in which a pub/sub middleware is integrated into SDN as a northbound interface for enabling agile and flexible network orchestration. Similarly, authors in [START_REF] King | The middleware assurance substrate: Enabling strong real-time guarantees in open systems with openflow[END_REF] describe a pub/sub middleware called MIDAS that leverages OpenFlow to control the resource management strategy and low-level switching configurations. Unlike these approaches that consider the middleware layer as a northbound interface, POSEIDON can be considered as a southbound interface to connect distributed overlay SDN data planes.

Likewise, authors in [START_REF] Boughzala | Openflow supporting inter-domain virtual machine migration[END_REF] introduce an OpenFlow-enabled middleware for virtual machine migration in data centers at the SDN control plane. On the other hand, POSEI-DON defines both the control plane functionality as well the data plane forwarding routines for supporting proactive SDN model. It also describes the messaging protocol to connect them together.

Authors in [START_REF] Koldehofe | The power of software-defined networking: Line-rate content-based routing using openflow[END_REF] propose a pub/sub architecture for SDN where the controller establishes line-rate content-based matching semantics to disseminate routing information to SDNenabled switches. The common shortcoming of contentbased systems is their dependency on the application layer to perform pub/sub operations. They also do not map directly to multicast communication unless a routing engine is added for group communication, which may result in higher bandwidth consumption and higher latency [START_REF] Tariq | Efficient content-based routing with network topology inference[END_REF]. POSEIDON natively supports many-to-many communication pattern and can provide reliable group communication for scalable oneto-many and many-to-many data distribution.

Recent efforts have already showed how pub/sub and SDN can be combined to perform content filtering at the level of SDN controllers. For example, LIPSIN [START_REF] Jokela | Lipsin: Line speed publish/subscribe inter-networking[END_REF] uses Bloom filters in data packets to infer the underlay topology from the overlay network, and enable efficient multicast communication. Despite this, LIPSIN requires substantial architectural changes for moving end-point oriented systems to their architecture. Similarly, PLEROMA [START_REF] Tariq | Pleroma: A sdn-based high performance publish/subscribe middleware[END_REF] uses content-based filters in SDN to enable group communication as well as allows running multiple virtual overlay domains. Likewise, authors in [START_REF] Bhowmik | Distributed control plane for software-defined networks: A case study using event-based middleware[END_REF] introduced an event-based middleware to offer distributed SDN control plane management and configuration. Unlike these works, our contribution provides a distributed pub/sub service in the control plane along with a routing agent for enabling proactive overlay data plane communication.

Recently some efforts have explored the use of Extensible Messaging and Presence Protocol (XMPP) service as an alternative to OpenFlow in hybrid SDN networks [START_REF] Marques | [END_REF]. The XMPP middleware is used to distribute control plane and management plane information to end servers that serve to enhance the communication between data centers in overlay networks and physical devices in the underlying network. The disadvantages of XMPP and its related technologies (i.e., roster, presence and routing functions) exist in different contexts. In particular, XMPP is considered to be a standard only for the wire protocol, i.e., XMPP is agnostic in relation to the data being transferred. XMPP is also stateful, which makes it more difficult to scale because each server needs to know the entire state in order to serve a request. Typically, the XMPP clients and servers utilize the domain name system (DNS) to resolve a server's do-main name into an address they can connect to. XMPP also requires centralized services to exchange messages between server-to-server federation, which makes it inefficient in duplicating messages when distributing them to multiple destinations. This is where utilizing POSEIDON for message distribution is more beneficial than XMPP. POSEI-DON uses the built-in pub/sub discovery service to allow publishers and subscribers to dynamically and continuously discover each other without the need to contact any name servers.

The OpFlex [START_REF] Smith | Opflex control protocol[END_REF] control protocol is introduced to configure and monitor all connected devices. The OpFlex protocol is founded on the concepts of declarative policy-driven system to control and program a large set of physical and virtual network devices. OpFlex is a request-response protocol based on JSON-RPC [START_REF] Morley | JSON-RPC 2.0 Specification[END_REF], where each component sends a request to query the information from its peer element. However, there are several disadvantages of RPC with respect to message passing since it may incur a severe degradation in performance due to marshaling/unmarshaling of messages (i.e., context switching increases scheduling costs) and may have to deal with added complexity in configuration for simple scenarios.

Unlike these approaches, POSEIDON is fully distributed so there is no single point of failure in the network during the communication. It also supports reconfigurable resource management policies for efficient use of the bandwidth, network and memory resources. Additionally, it supports the proactive overlay SDN, which makes it a suitable technology to ensure scalability, reliability, and flexibility.

CONCLUSION

Proactive overlay software defined networking (SDN) aims at overcoming the scalability limitations of reactive hop-byhop approaches for large-scale networks and the practical limitations of the pure proactive approach. Although both approaches are supported by the dominant SDN technology OpenFlow, its APIs are too low-level and lacks intuitive abstractions to build reusable SDN applications. To overcome these limitations, in this paper we described the POSEI-DON middleware for realizing proactive overlay SDN. PO-SEIDON proposes an efficient matching of advertisement and subscription using the data-centric publish/subscribe (pub/sub) paradigm to support flexible and programmable SDN networks. The paper demonstrates how POSEIDON can be used in two real-world use cases. The experimental evaluation of POSEIDON validates our claims about (1) the ease of configuring the overlay SDN forwarding tables of the data plane directly through the middleware using a software control layer inside the SDN controller, and (2) its scalability properties.

The following lessons were learned conducting this research, which illustrates the limitations of the work while also highlighting opportunities for further research in this area:

• Applying POSEIDON solution for IoT systems.

The modular architecture of POSEIDON makes it suitable for large number of wired networks such as cloud environment, Cloudlets, and multi-tenant networks. However, there are many reasons to extend POSEI-DON to support Internet of Things (IoT) systems.

To enable POSEIDON support for IoT, it should be extended to integrate new protocols such as 6Low-PAN [START_REF] Kushalnagar | IPv6 over Low-Power Wireless Personal Area Networks (6LoWPANs): Overview, Assumptions, Problem Statement, and Goals[END_REF] and ROLL [START_REF] Winter | RPL: IPv6 Routing Protocol for Low-Power and Lossy Networks[END_REF]. Meanwhile its footprint, stemming from its reliance on a pub/sub middleware, should be reduced to suit the embedded, resourceconstrained systems. We also believe that such an approach will draw attention of the research community to the need to bring agility and programmability of SDN for scalable and high performance IoT networks.

• Securing Flow Forwarding in POSEIDON. PO-SEIDON has the ability to interconnect hundreds of SDN controllers and devices at different scales. Currently, POSEIDON supports built-in security policies to deliver safety required by the control plane. At the data plane, since the flow table entries are written as pieces of software modules, their content may be altered due to the increasing number of DDoS and malware attacks, and phishing activities. Therefore, we believe that sophisticated encryption and authentication mechanisms to overcome hackers, and recovery of packets from failure are needed. For example, role-based and policy-based access control [START_REF] Scott-Hayward | A survey of security in software defined networks[END_REF] could be added to SDN to support data integrity, pedigree, confidentiality, and non-repudiation.

Figure 1 :

 1 Figure 1: POSEIDON Architecture

Figure 2 :

 2 Figure 2: Internal components of the POSEIDON Agent

Figure 3 :

 3 Figure 3: Policy Service within POSEIDON Policy Node

Figure 4 :

 4 Figure 4: Monitoring node

Figure 5 :

 5 Figure 5: Internal view of the routing agent inside OpenFlow capable switch

Listing 4 :

 4 Mapping the Physical devices to Virtual ones def virtual_switch(topo): self.v_sw = 1 self.vport = 1 for (sw, port) in topo.egress_location: vmap[(v_sw, vport)] = (sw, port) vport +=1

Figure 6 :

 6 Figure 6: POSEIDON for Interconnecting Multiple Data Centers over the Internet

Listing 5 :

 5 Virtualize topology abstraction function virtualize(v_sw, merge(name=3, from_switches=[sw1, sw2, sw3]))

Figure 7 : 8 .

 78 Figure 7: POSEIDON for Software Defined Internet Exchange

Figure 8 :

 8 Figure 8: Transforming the underlying topology into single virtual switch ISP C has virtual interfaces to connect to virtual interfaces of ISP A and ISP D through the switch. Listing 6 the parallel composition of the abstracted policy that allows ISP C to filter HTTP traffic towards ISP A and forwards Netflix video streaming on UDP port 443 towards ISP D. Listing 6: Forwarding Policy in ISP C match(dstport = 80) >> fwd(A)) + match(dstport = 443 >> fwd(D))

Figure 9 :

 9 Figure 9: Latency: Proactive POSEIDON vs. Reactive OpenFlow

Figure 10 :

 10 Figure 10: Efficient Bandwidth usage for POSEIDON

Figure 11 :

 11 Figure 11: Evaluating the Scalability

Recall that the pure proactive approach is impractical and mostly of theoretical value.

Acknowledgments

This work was funded by the Fullbright Visiting Scholars Program, NSF CNS US Ignite 1531079 and by AFOSR DDDAS FA9550-13-1-0227. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of Fulbright, NSF and AFOSR.


Prithviraj Patil
email: prithviraj.p.patil@vanderbilt.edu

Akram Hakiri
email: akram.hakiri@laas.fr

Aniruddha Gokhale
email: a.gokhale@vanderbilt.edu

Cyber Foraging and Offloading Framework for Internet of Things

Keywords: Mobile cloud computing, IoT, software engineering, modular and configurable framework

published or not. The documents may come   L'archive ouverte pluridisciplinaire

Cyber Foraging and Offloading Framework for Internet of Things Prithviraj Patil * , Akram Hakiri † , and Aniruddha Gokhale *

I. INTRODUCTION:

Computation mobility or cyber foraging [START_REF] Sharifi | A survey and taxonomy of cyber foraging of mobile devices[END_REF] is a capability that is critical to the success of mobile cloud computing (MCC). Computation mobility refers to the ability of the infrastructure to seamlessly migrate a computation from one node to another node. The major difference between computation mobility and traditional client-server distributed computing model using remote procedure call (RPC) is that in RPC no computation is being executed at the client while in computation mobility, the computation is performed mostly at the client but may get migrated to the server machine for a variety of reasons. Computation mobility is desired in cases where computing devices are available in every size and shape, and where sharing of computing resources can be beneficial.

In the context of MCC, smartphones and tablets can take advantage of computation mobility. By offloading or moving a computation, MCC attempts to alleviate restrictions on smartphone devices (e.g., their computation power, battery life, memory capacity, etc.) by offloading all or some part of its computation to either a neighboring smartphone device or to the remote cloud. A number of computation mobility solutions have been proposed for MCC like MAUI [START_REF] Cuervo | Maui: making smartphones last longer with code offload[END_REF], CloneCloud [START_REF] Byung-Gon Chun | Clonecloud: elastic execution between mobile device and cloud[END_REF], ThinkAir [START_REF] Kosta | Thinkair: Dynamic resource allocation and parallel execution in the cloud for mobile code offloading[END_REF], Scavenger [START_REF] Darø | Scavenger: Transparent development of efficient cyber foraging applications[END_REF] and MobiCloud [START_REF] Huang | Mobicloud: building secure cloud framework for mobile computing and communica-tion[END_REF]. All these offloading solutions rely on partitioning the application running on the smart device (phone or tablet) at different granularities, such as a method call or a process, which is called as a computation unit. Depending on the offloading policy programmed in these frameworks, these computation units will be offloaded to the other devices such that optimum usage of device resources is achieved.

Although these computation offloading solutions have shown significant promise in traditional mobile cloud computing 1 , they incur limitations in the context of Internet of Things (IoT). IoT aims to achieve total connectedness in the environment we live in [START_REF] Gubbi | Internet of things (iot): A vision, architectural elements, and future directions[END_REF]. Computation as we know today is constrained to only personal/server computers or mobile phones/tablets. In the realm of IoT, every person is expected to encounter thousands of new entities that are going to be smarter, powered with computational ability, and which will be able to sense/process/store/send/receive data in one form or another [START_REF] Atzori | The internet of things: A survey[END_REF].

To list a few instance of such smart devices, people are expected to interact in a variety of ways with TVs, watches, air conditioners, clothes, car keys, refrigerators, house doors, house floor/ceiling, air conditioners, vehicles, roads, kiosks, vending machines, bank ATMs, postal boxes, medical devices and many others. This non-exhaustive list of smart objects or "things" makes it immediately apparent that these objects will come in a variety of shapes and sizes. The computing power on these objects will range from few MHz to GHz. Similarly, these objects will illustrate substantial variability in their storage capacity, memory, power, network bandwidth etc.

IoT is thus positioned to be a far larger superset of mobile entities than just phones and tablets. Consequently mobile cloud computing will continue to play an important role for IoT, in turn implying that computational mobility will be a required capability -in fact a much more critical capabilityfor IoT to effectively utilize the resources. However, contemporary computation mobility solutions based on offloading [START_REF] Byung-Gon Chun | Clonecloud: elastic execution between mobile device and cloud[END_REF], [START_REF] Cuervo | Maui: making smartphones last longer with code offload[END_REF], [START_REF] Huang | Mobicloud: building secure cloud framework for mobile computing and communica-tion[END_REF]- [START_REF] Darø | Scavenger: Transparent development of efficient cyber foraging applications[END_REF] have been studied in the context of mobile cloud computing involving just smartphones (and tablets) but not the broad range of objects expounded by IoT. These contemporary solutions have been developed with very specific assumptions about the role, power, mobility, trust etc. of the offloading client and offloading server which is very dynamic in the IoT.

To address these challenges, this paper makes the following contributions:

• We present a novel modular and highly configurable framework for providing seamless computational mobility in the IoT realm. • We describe key capabilities of this framework such as modularity and dynamic configurability for dynamic IoT scenarios in terms of node mobility, trust, cost model etc. • We validate our solution by showing how it can be applied to multitude of IoT scenarios. The rest of the paper is organized as follows: Section II-A describes the similarities and differences in using computation mobility in traditional mobile cloud computing and IoT, and outlines the key challenges that call for a modular and highly configurable framework for computational mobility in IoT; Section II-B describes the components of a computation mobility framework for the IoT scenario; Section III describes the architecture, configuration and implementation of mobility framework; Section IV compares our solution to related efforts in the area of computation offloading in traditional MCC and IoT; and finally Section V provides concluding remarks describing potential future directions and open research problems in this realm.

II. DESIGN CONSIDERATIONS FOR A IOT COMPUTATION MOBILITY FRAMEWORK

We now elicit the design considerations for realizing a computation mobility framework for IoT. First, we understand the problem space by illustrating the different dimensions of complexity that arise in the IoT realm. We then present key desired features of such a framework.

A. Computation Mobility Challenges for IoT

Section I mentioned several computation offloading solutions [START_REF] Byung-Gon Chun | Clonecloud: elastic execution between mobile device and cloud[END_REF], [START_REF] Cuervo | Maui: making smartphones last longer with code offload[END_REF], [START_REF] Huang | Mobicloud: building secure cloud framework for mobile computing and communica-tion[END_REF]- [START_REF] Darø | Scavenger: Transparent development of efficient cyber foraging applications[END_REF] that have been developed for the traditional MCC use cases. In these scenarios, multiple mobile smartphones (or tablets) are distributed across geographical locations and connected to remote cloud machines via cellular or WiFI network. The computation on a smartphone is offloaded either to another smartphone or to the cloud server machine. All these solutions make specific assumptions about the characteristics of smartphones and cloud capabilities as follows:

1) Smartphone is a "weaker" device.

2) Smartphone is a "mobile" device.

3) Remote resources in cloud machine require a "stronger" device. 4) Remote resources may be held in a "non-mobile" device. 5) Remote resources should be "trusted" by smartphones. 6) Cloud machine is may be a "remote" device in the context of a smartphone device. 7) Communication between smartphone and cloud may obey "client-server" protocol where smartphone is always a client. 8) A smartphone communicates with the cloud machine over WiFi or cellular networks.

In the above assumptions, the notion of stronger and weaker can be construed as relative quantitative measures of the computation power, storage power, memory, battery power or any other measurable device resource. A weaker device in the context of traditional MCC is basically a client while stronger device is the server.

Although these assumptions are valid for the traditional MCC use cases, they apply in only a small subset of the much larger IoT realm. IoT consists of thousands of devices capable of sensing, computing, storing, sending and/or receiving data with different capacity. These devices and the services offered within the realm of IoT illustrate several additional characteristics that must be taken into consideration for computation mobility solutions, such as mobile/non-mobile, trusted/un-trusted, weaker/stronger, client/server, WiFi/cellular, free/for profit and many others. In such a complex scenario, different devices talk to each other at different times in different ways which gives rise to a very broad range of complex interactions. The increased complexity limits the applicability of contemporary computational mobility solutions in the IoT context.

In Table I we have attempted to capture many such interactions in IoT. For example in scenario 5 in the Table I, we encounter heterogeneity of communication protocols used by different devices. For example, the smartCars communicate with each other using gossip protocol(at every traffic signal e.g). The smartPhones communicate with each other using peer to peer protocol. A smartPhone and cloud talk to each other using client-server protocols. A smartphone and smart-Car communicate with each other using publish/subscribe protocol.

The list of interactions in Table I is by no means exhaustive; instead we should expect many more such complex interactions among devices as more and newer devices get attached to the IoT. Consequently, in such a varied and dynamic IoT device interactions scenario, computation offloading solutions developed for the traditional MCC are not readily applicable to all IoT interactions. Hence, we need highly configurable, modular and dynamic computation mobility solutions, which can be tailored for and used in each and every use case of IoT.

B. Key Desired Capabilities of IoT Computational Mobility Framework

Based on the different dimensions of complexity alluded to in Table I, we describe the key desired capabilities that can be realized as components or modules of the computation mobility framework for IoT.

Figure 1 shows the various capabilities of the framework at an abstract level. Table II The smartCars communicate with each other using gossip protocol, e.g. at every traffic signal. The smartPhones communicate with each other using peer to peer protocol. A smartPhone and cloud talk to each other using client-server protocols. A smartphone and smartCar communicate with each other using publish/subscribe protocol. 5

Communication channel used smartPhone, smartTV, smartCar and smart Gas-stations

The smartTV and smartPhone communicate over WiFI, smartPhone and smartCar communicate by direct physical connection, smartCar and smart Gas-station communicates over cellular. 6

Trust and security smartPhone, smartTV, smartCar and smart Gas-stations

The smartTV and smartPhone always trust each other but smartCar and smart Gas-station may not trust each other all the time. 7

Stronger in one resource but weaker in another

The smartCar and RSU (Road Side Unit) used as a static relay point

The smartCar is a stronger computational device but has less network bandwidth. The RSU on the other hand is almost a dumb device but has maximum network bandwidth and stable connectivity to the wired network. 8

Application partitioning

The smartPhone and Cloud Some applications running on the smartPhone are partitioned (e.g. using annotations) by developers while others are not. 9

Business or cost smartPhone, smartTV, smartCar and smart Gas-stations A smartCar and smartPhone offload to each other with no money involved; while a smartCar and smartGasStation may offload to each other based on some pricing model. In this subsection, we describe these modules in more detail. We also show the possible variations of each one of those capabilities required to cater to different configurations in the IoT scenario. We will use these component abstractions to design a concrete modular and configurable framework described in Section III. Each capability and its responsibility is described below.

Sensing for Discovery and Data Collection: Node sensing is an important part of the computation mobility framework for IoT. Each node dynamically needs to [START_REF] Abolfazli | Hmcc: A hybrid mobile cloud computing framework exploiting heterogeneous resources[END_REF] he lacks, e.g. node A wants to trade in its storage space against computational power. Payment Methods: For those nodes who want to do business using the computation framework, a payment option must be available. For cost models like those mentioned in bullet 1 and 2 of the above list, any eFinancial service can be used. However, for cost models like those in bullet 3 and 4, we need a secure way agreed by both parties involved. We may need to implement secure software service on the basis of bit-coin for such cost models.

Fault Tolerance Strategies: Different nodes may have different fault management strategies in the offloading framework. For instance, a node N1 may want to offload the same computation C1 to two nodes N2 and N3 so as to tolerate one fault. Node N2 in turn may also want to offload C1 (which was offloaded to it by N1) to N4 (in addition to computing it by itself) to tolerate a fault and retain its credibility in the system. This is desirable especially if nodes are getting charged/paid for offloading computation. Nodes may want to protect their computations against following faults.

1: FT against unreliable network 2: FT against random node mobility 3: FT against dynamic load 4: FT against malicious nodes 5: FT against network partitioning Application Interface: Applications running on the device may need to communicate with the framework in a standard way for one or more of the following purposes:

1: report computations which are computeintensive/missing deadlines. 2: report the required granularity/partitioning of an application, computation priority and other configuration information 3: receive information about offloaded computation like expected time for results 4: receive results back from the framework of the offloaded computation Security/privacy/trust of the Computation: Nodes use different protocols/strategies for various trust management tasks like 1: establish a trust with other device 2: communicate with a trusted device 3: communicate with an untrusted device Framework should be flexible enough to accommodate these variation among nodes.

Communication Protocols & Channels: Different devices employ different strategies for sending and receiving data to other devices. Framework should be flexible enough to take these into consideration. The variation here also affects the way nodes discover each other (described under the first bullet point)

1: publish-subscribe based data dissemination 2: client-server based data dissemination 3: peer-to-peer based data dissemination 4: multi-path data dissemination 5: network channels(cellular, WiFi, radio, wired) Resource Monitoring: Framework also need to make sure it has latest resource usage information of the device. Hardware resources that need to be monitored could be following 1: network b/w 2: processor usage 3: power usage 4: memory usage 5: disk usage 6: sensors Resource provisioning: Nodes also need to make sure they have enough resources before they accept the computation from another node. This is relevant especially to those nodes which scale up and down based on the load like cloud server, e.g. smart Gas-station is serving no smartCars at time t0 and hence running on low power mode or possibly on hibernate. But at time t1, n number of smartCars arrive at gas station, discover a smart Gas-station, sense an opportunity for faster computation and offload some of their heavy computations to it. The gas station needs to make sure it is able to scale up its computing resources to serve those cars and financially benefit from it [START_REF] Mahadev Satyanarayanan | The Case for VM-Based Cloudlets in Mobile Computing[END_REF].

III. IOT COMPUTATION MOBILITY FRAMEWORK: ARCHITECTURE, CONFIGURATION AND IMPLEMENTATION

In this section we concretely describe the internals of the framework by focusing on its software engineering, i.e., class structure of its various components.

A. Architecture

We describe top-level or primary classes (entities) that exist in the software architecture of the IoT computational mobility framework that capture the IoT complexities. The need for these classes/entities in our software framework is motivated by the key capabilities described in Section II-B. These classes/entities components encode one or more of these capabilities. We show the mappings between abstract modules we envisioned in the previous section mapped to concrete class structue of the framework in the Table III.

1. Node Repository: Figure 2 shows the class structure of the node repository. The node repository object is used to store the information about the nodes that device can connect to. The important information of a node required for computation mobility including node mobility, resources, cost, trust etc. is stored in here. It will have functions to set/get information about node and to add or remove nodes.

We store mobility information of a node in the mobility_info object which has parameters like current location, current speed, frequency for location update and history location data. It also has parameter called mobility function. It is used to predict the mobility of node if required by other components e.g. cost function may need to determine network connectivity during given time period in the near future to determine the realistic cost of offloading to a node. This function can be used in two ways. For devices with specialized mobility patterns like a moving car, train, airplane or phones, this function can be set/adapted according to the route set by the user of the moving device. For other devices where mobility is random (like phone with a walking person), this function is derived from the recent history location data.

The Node repository stores resource information about nodes into the resource_info object. Resource information can be configured with various standard and non-standard resources.

2. Resource Info: Figure 3 shows the class structure of the resource_info. It has functions to add resource, remove resource, get/update/predict resource value, add a resource monitor, set frequency for resource monitoring. This class can be used to collect information about various general resource types like network, disk, memory, power, computing power, sensors etc. It can also be used to define special-purpose resources by overriding functions in the base resource class. For example, in the use case 5 from Table I, smartCar may want to offload its computation to other smartCar based on the temperature/fuel efficient of the other smartCar. This object has parameters like measurement_unit, max value, min value, history values, monitor and monitor frequency.

3. Application Repository: Figure 4 shows the class structure of the application repository. Every node in IoT that runs its application(s) would like to use the offloading framework. So the framework stores information about such applications. It provides an API interface to the running applications to talk to the framework. This class of framework needs to be linked to the application running on the device which wants to use the computation mobility framework.

An application can register itself with the framework, set computational granularity for offloading, set priority, set a partitioner strategy or define a specialized partitioner, and provide partitioning points. This class object has parameters like application_id, computation_info, and partitioning_info. It can then register or remove computations with the framework for offloading which is the object of the class computation_info.

The computation_info stores all the information related to a computation such that it can be offloaded, e.g. code_size, code, time_limits, resource_requirements, history_run_times, computation_id and application_id.

4. Communication Repository: Figure 5 shows the class structure of the communication repository. This captures all the variations in the node to node communication. It has functions to discover nodes, update nodes, get node information (mobility information, trust information, resources information etc). It also can offload a computation to a node and receive results or receive a computation from a node and send the results back. It can be configured with various network communication protocols (publish-subscribe, peer-to-

B. Configuration:

Now that we have designed all the required components for the IoT computational mobility framework, we describe how to configure these components to satisfy requirements of individual node or application. Note that there could be different/better ways to configure the framework. The primary purpose of describing it is not to show how to configure framework but to show how easily the framework can be configured.

Algorithm 1 describes one possible approach to configure the communication component. At the start, Algorithm 1 sets the communication channel and communication protocol. Values for these two parameters are provided by the configuration object. The framework reads the configuration values from the .xml file and then populates the configuration object to be used by other components in the framework.

At line 3, it sets a trigger to notify that discovery is required. Framework can use different types of user-defined triggers like 1) timeout: e.g. after every 10 seconds. trusted list otherwise in the non-trusted list. It also gathers more information about nodes like mobility, resource, cost etc.

Algorithm 2 describes a way to configure the framework to implement optimizer solver. The optimizer solver described in the 2 is a general purpose solver. It takes optimizer function from the configuration object as input. It then tries to find the best mappings of offload-able computations to the computation nodes so that specified optimization function is optimized. Of course, we can implement optimization in various other ways however main point here is to show the flexibility of framework to realize such fine-grained configurations and optimizers.

C. Proof-of-Concept Implementation & Validation:

To validate the feasibility of our framework, we have implemented some components of the framework for providing a proof of concept. We have also validated it in the smaller simulated IoT environment. This validation shows that our framework is highly configurable and can be applied to various scenarios in IoT seamlessly.

For validation, we have configured our framework with "process" as a computation granularity. We used existing process level migration solutions for java applications. It can then be used to migrate a process running on one JVM capturing all its context, stack, register, heap values and migrate it to another remote JVM. We ran compute-intensive application (for calculating highest prime number) on all nodes. This application has 110 lines of code and would take 300 seconds on the normal desktop machine.

We validated this framework using the (mini) IoT scenario with multiple computational nodes with varying properties like mobile/static, trusted/un-trusted, closer/remote etc. We simulated computational nodes using the docker containers running in virtual machines and communicating with each other with virtual network interface. We simulated network for these nodes in the Mininet simulator. For mobility, we used random way-point model for mobile nodes. We did not use dynamic trust model but statically defined some nodes as trusted while other as un-trusted. We also did not use dynamic cost model but statically defined cost model for every node before simulation. We used different simulated communication channels (WiFi, 802.11a/b/g, wired etc) and communication protocols(client-server, p2p) for communication among the simulated nodes. This experiment showed that framework is flexibile enough to adapt to various scenarios.

However, we are yet to fully implement various components of the framework including trust management, fault tolerance, payment etc. We plan to achieve full implementation of the framework and exhaustive validation in the future.

IV. RELATED WORK

This section compares our work with prior work on computation offloading frameworks and efforts that provide these to IoT.

A. Computation Offloading Frameworks

There are several survey papers available [START_REF] Hoang T Dinh | A survey of mobile cloud computing: architecture, applications, and approaches[END_REF], [START_REF] Fernando | Mobile cloud computing: A survey[END_REF], [START_REF] Shiraz | A review on distributed application processing frameworks in smart mobile devices for mobile cloud computing[END_REF] that discuss different offloading frameworks in the traditional MCC realm including MAUI [START_REF] Cuervo | Maui: making smartphones last longer with code offload[END_REF], CloneCloud [START_REF] Byung-Gon Chun | Clonecloud: elastic execution between mobile device and cloud[END_REF], ThinkAir [START_REF] Kosta | Thinkair: Dynamic resource allocation and parallel execution in the cloud for mobile code offloading[END_REF], Scavenger [START_REF] Darø | Scavenger: Transparent development of efficient cyber foraging applications[END_REF], MobiCloud [START_REF] Huang | Mobicloud: building secure cloud framework for mobile computing and communica-tion[END_REF]. These prior efforts both similarities and differences in their offloading frameworks.

MAUI [START_REF] Cuervo | Maui: making smartphones last longer with code offload[END_REF] uses a combination of VM migration and code partitioning to conserve the power of mobile devices. It is implemented in the .NET environment and uses a run-time, dynamic partitioning algorithm. The partitioning strategy is intrusive and application developers need to annotate the code at method level to let the framework know about offloadable methods. MAUI however does not address the scaling of execution in cloud.

CloneCloud [START_REF] Byung-Gon Chun | Clonecloud: elastic execution between mobile device and cloud[END_REF] unlike MAUI [START_REF] Cuervo | Maui: making smartphones last longer with code offload[END_REF] can be used with unmodified applications but it requires the clone of the entire device on the cloud machine. It uses both static analyzer and dynamic profiler to partition an application so that it migrates and executes in the cloud, and resumes on the device. It uses a simple cost model based on the execution time. Similarly, RMCC [START_REF] Abolfazli | Rmcc: Restful mobile cloud computing framework for exploiting adjacent servicebased mobile cloudlets[END_REF] is lightweight resource-oriented framework for OS-independent mobile Cloudlets based on Restful web interfaces. RMCC allows realizing computation on the mobile devices to enable green MCC. Likewise, HMCC [START_REF] Abolfazli | Hmcc: A hybrid mobile cloud computing framework exploiting heterogeneous resources[END_REF] is a mobile computation outsourcing model that exploits hybrid cloudbased resources to provide fine-grained management resources to wireless mobile devices.

ThinkAir [START_REF] Kosta | Thinkair: Dynamic resource allocation and parallel execution in the cloud for mobile code offloading[END_REF] tries to combine approaches used by MAUI [START_REF] Cuervo | Maui: making smartphones last longer with code offload[END_REF] and CloneCloud [START_REF] Byung-Gon Chun | Clonecloud: elastic execution between mobile device and cloud[END_REF] and improves their individual shortcomings. It improves MAUI's lack of scalability by creating virtual machines (VMs) of a complete smartphone device on the cloud machine. It also removes the restrictions on applications that CloneCloud induces by adopting an online method level offloading. It supports on-demand resource allocation in the mobile cloud based on workloads and deadlines MobiCloud [START_REF] Huang | Mobicloud: building secure cloud framework for mobile computing and communica-tion[END_REF], though not strictly an offloading framework, is a framework for secure, trustworthy mobile cloud in an unsecured network (e.g. MANET). The MobiCloud framework can be used to create a secure communication in the mobile cloud network with trust management, secure routing, and risk management taken care by the framework.

Scavenger [START_REF] Darø | Scavenger: Transparent development of efficient cyber foraging applications[END_REF] is another framework that employs cyberforaging using WiFi for connectivity, partitions the application and distributes jobs. However, it strictly uses a client-server architecture and hence offloading to surrogates is possible but not vice-versa.

All of the above frameworks provide computation mobility in a very specific way by considering only a subset of variants that we discussed in Section II and hence it becomes almost impossible to deploy these solutions in a scenario where offloading requirements vary based on mobile node, application, security, computational granularity etc.

Also, as we discussed in Section II-A, all the above frameworks work in a scenario where multiple weak mobile smartphones work as client device and are connected to a remote powerful cloud machine which work as a server machine. However, IoT brings a whole new set of different scenarios (refer to Section II-A) where the same device may act as a weak or a powerful, a remote or a non-remote, a mobile or a non-mobile device. In such complex and dynamic scenarios, solutions mentioned above would not work.

B. IoT and computation mobility

These papers [START_REF] Atzori | The internet of things: A survey[END_REF], [START_REF] Gubbi | Internet of things (iot): A vision, architectural elements, and future directions[END_REF] discuss the various aspects of the IoT including big vision, architecture, components and future challenges.

In our literature survey, we could not find any major works which would extend MCC computational mobility like features to the space of IoT. However, we find the concepts described in [START_REF] Bonomi | Fog computing and its role in the internet of things[END_REF], [START_REF] Hong | Mobile fog: a programming model for large-scale applications on the internet of things[END_REF], [START_REF] Satoh | A framework for data processing at the edges of networks[END_REF] (e.g. fog computing, data processing at the edge of network, etc.) very relevant to the framework we have proposed in this paper. The authors in [START_REF] Bonomi | Fog computing and its role in the internet of things[END_REF] propose a fog computing concept which is a highly virtualized platform that provides compute, storage, and networking services between end devices located at the edge of the network and traditional cloud computing data centers. Fog computing is proposed as a compliment to the traditional cloud computing especially in the IoT scheme. Though the paper discusses the need for interoperatibility and mobility support in the fog computing network, it however does not discuss computational mobility in particular.

V. CONCLUSION & FUTURE WORK:

In this paper we discussed the importance of code mobility in mobile cloud computing (MCC), and specifically in the Internet of Things (IoT) environment. We discussed the major differences in the traditional MCC and IoT scenarios arising because of the high degree of hetergeneity in the IoT compared to traditional MCC. We motivated the need to take a fresh perspective at designing a modular and highly configurable computation mobility or offloading framework for IoT since the solutions developed for traditional MCC environment are not applicable in IoT due to tight coupling and lack of configurability. We described the various capabilities of such a framework and show a way to design it by providing a concrete architecture through class diagrams, configuration and proofof-concept implementation.

In the future, we plan to implement the full implementation of our framework as a reference point with every sub-component implemented so as to cover every scenario or device interaction in the IoT. We also plan to deploy the framework in the real-world or simulated IoT scenario and configure it dynamically and test the performance of computation mobility on the resource usage

Fig. 1 .

 1 Fig. 1. Components of a Computation Mobility Framework

Fig. 2 .

 2 Fig. 2. Node Repository Class Diagram

Fig. 3 .

 3 Fig. 3. Resource Info Class Diagram

Fig. 5 .

 5 Fig. 4. Application Repository Class Diagram

TABLE I INTERACTION

 I SCENARIOS AND DIMENSIONS OF COMPLEXITY IN IOT FOR COMPUTATIONAL MOBILITY ) smartTV is a weaker device while smartPhone is a stronger device; (2) smartphone is a weaker device while smartCar is a stronger device; (3) smartCar is a weaker device while smart Gas-station is a stronger device 2Relative mobility of devices smartTV, smartPhones and smartCars(1) smartPhone is a mobile device for smartTV; (2) smartPhone is non-mobile device for the smartCar (as long as the vehicle is on the road) 3Dynamically changing relative strength and weakness of devices multiple smartPhones One smartPhone is stronger than the another smartPhone at time t. But at time t + 1, the situation reverses possibly due to the dynamic load on both the phones or remaining battery power.

	No.	Complexity Dimension	Participants	Scenario
	1 (14 Relative strength and weakness of devices smartTV, smartPhone, smartCar and smartGas-station Communication paradigm used Multiple smartCars, smartPhones
			inside each smartCar and cloud

  Cost or Business Model-based Trade-offs: Some nodes in the IoT may want to offer their resources for profit Different nodes may require different cost models. 1: A node may want to trade in dollars for their time 2: A node may want trade in dollars for their resources 3: A node may want to trade symbiotically, e.g. node A may want to use equal amount of resources in future that it is letting other node use at present 4: A node may want to trade what it has more with what

				TABLE II
		CAPTURING & MODULARIZING HETEROGENEITY IN IOT
	Individual interaction in IoT.	Sensing	Trust	Business	Payment	Mobility	Comm	Resource	Resource	Fault	App	Opt
			Mgmt	Model	Payment	Mgmt	Mgmt	Monitor	Provision	Mgmt	i/f	Solver
	smartCar & smartPhone in the same smartCar
	smartCar & smartPhone in the different smartCar
	two smartCars both at rest
	two smartCars both on the road
	two smartphones at home
	a smartPhone & smartTV (physically connected)
	a smartPhone & cloud
	a smartPhone & smart Gas-station
	a smartCar & smart RSU (Road-Side-Unit)
	4: computational granularity
	5: fault tolerance policy (see below)

TABLE III MODULE

 III TO CLASS STRUCTURE MAPPING

	Class	Sensing	Trust	Business	Payment Mobility	Comm	Resource	Resource	Fault	App.	Optimi-
			Mgmt	Model	Mgmt	Mgmt	Moni-	Provi-	Mgmt	Inter-	zation
							tor	sion		face	Solver
										(pari-
										tioning
										&
										granu-
										larity)
	node_repository :
	mobility_info
	trust_model
	resource_info
	cost_model
	application_info
	communication_info

Although mobile cloud computing is also a relatively new field, we use the term "traditional" to refer to use cases that involve only smartphones and tablets.

ACKNOWLEDGMENTS

This work is supported in part by NSF US Ignite CNS 1531079 and AFOSR DDDAS FA9550-13-1-0227. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NSF and AFOSR.