Marcos Dias De Assuncao

Radu Carpa

Laurent Lefèvre

Olivier Glück

Piotr Borylo

Artur Lason

Andrzej Szymanski

Michal L Rzepka

Marcos Dias De Assunção

Piotr Bory Lo

Artur Lasoń

Andrzej Szymański

Designing and Building SDN Testbeds for Energy-Aware Traffic Engineering Services

Keywords: energy-awareness, segment routing, anycast routing, testbeds

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Advances in network and computing technologies have enabled a multitude of services -e.g. those used for big-data analysis, stream processing, video streaming, and Internet of Things (IoT) [START_REF] Atzori | The internet of things: A survey[END_REF] -that are hosted at one or multiple data centres often interconnected with high-speed optical networks. Many of these services follow business models such as cloud computing [START_REF] Armbrust | Above the clouds: A Berkeley view of Cloud computing[END_REF], which allow a customer to rent resources from a cloud and pay only for what is really consumed. Although these models are flexible and benefit from economies of scale, the increasing amount of data transferred over the network requires continuous expansion of installed capacity in order to handle peak demands. Existing work argues, however, that the amount of electricity consumed by network infrastructure may become a bottleneck and further limit the Internet growth [START_REF] Kilper | Energy challenges in access and aggregation networks[END_REF].

Given that high performance wired networks are seldom fully utilised, many organisations attempt to curb their energy consumption by reducing the amount of resources that are active during off-peak periods. Several technologies have been employed for this purpose, e.g. putting resources into low power consumption modes [START_REF] Gupta | Greening of the internet[END_REF], adapting links' data transmission rates [START_REF] Gunaratne | Reducing the energy consumption of ethernet with adap-tive link rate (ALR)[END_REF][START_REF] Miyazaki | High speed 100GE adaptive link rate switching for energy consumption reduction[END_REF], and grouping and transferring packets in bursts [START_REF] Nedevschi | Reducing network energy consumption via sleeping and rate-adaptation[END_REF]. Their utilisation, in general, results in lower overall energy use. On the other hand, traffic engineering [START_REF] Awduche | Overview and principles of internet traffic engineering[END_REF], initially created to reduce network bottlenecks by shifting traffic to underutilised links, has been also investigated as a network-wide approach to improve energy efficiency by, for instance, consolidating traffic on a limited number of links, and thus enabling the remaining links to enter the low power consumption modes [START_REF] Vasić | Energy-aware traffic engineering[END_REF][START_REF] Carpa | Segment routing based traffic engineering for energy efficient backbone networks[END_REF]. As a result, the already difficult problem of optimising the use of network resources became even more challenging.

To simplify configuration and management operations, traffic-engineering schemes are increasingly relying on SDN as it separates control and data planes and provides a centralised view of (i) the network topology, (ii) running applications and, (iii) traffic demands; which are important requirements to program a network and change its virtual topology according to traffic conditions. In our previous work [START_REF] Carpa | Segment routing based traffic engineering for energy efficient backbone networks[END_REF][START_REF] Carpa | Responsive Algorithms for Handling Load Surges and Switching Links On in Green Networks[END_REF][START_REF] Assuncao | On designing sdn services for energy-aware traffic engineering[END_REF], we investigated SDN enabled traffic engineering to redirect data flows and reduce energy consumption. We also considered SDN based energy-aware anycast routing [START_REF] Borylo | Anycast Routing for Carbon Footprint Reduction in WDM Hybrid Power Networks with Data Centers[END_REF] extended by introducing cloud services differentiation [START_REF] Borylo | Fitting Green Anycast Strategies to Cloud Services in WDM Hybrid Power Networks[END_REF], models of cooperation between SDN controller and cloud orchestration software [START_REF] Borylo | Green Cloud Provisioning Throughout Cooperation of a WDM Wide Area Network and a Hybrid Power IT Infrastructure[END_REF] and the concept of interplay between fog and cloud infrastructures supported by wide area software defined networking [START_REF] Borylo | Energy-Aware Fog and Cloud Interplay Supported by Wide Area Software Defined Networking[END_REF]. As experimenting with production networks is rarely possible, the proposed techniques have been evaluated using a discrete-event simulation tool (OM-Net++ [START_REF]OMNeT++ Discrete Event Simulator[END_REF]), which provided very promising and valuable results. However, to fully investigate the subject and assess whether simplifications made during simulations have not led to biased results, it is necessary to design proof of concept implementations by using testbeds.

This work describes challenges and requirements towards building platforms for evaluating energy-aware traffic engineering applications and porting simulations to such testbeds as SDN services. We discuss the design and implementation of two approaches: an SDN application that uses segment-routing to redirect flows in backbone networks in order to free certain links [START_REF] Carpa | Segment routing based traffic engineering for energy efficient backbone networks[END_REF] and energy-aware anycast strategies considering the type of energy used to power target data centres. We describe how custom platforms, called The GrEen Traffic engineering testBed -Segment Routing (GETB-SR) and The GrEen Traffic engineering testBed -Anycast Routing (GETB-AR), are used for evaluating the proposed applications. We compare results gathered in the testbeds with simulation-based results and present challenges faced while designing energy-aware traffic engineering mechanisms as SDN services in testbed environments.

The rest of this paper is organised as follows. Section 2 discusses energy-aware traffic engineering requirements for platforms used for evaluation and SDN concepts. The testbeds used for building proofs of concept are presented in Section 3. The SDN applications developed for validating and evaluating the performance of the traffic-engineering strategies, their life cycles, results and issues regarding deployment in the testbed are described in Section 4. Section 5 discusses related work and Section 6 concludes the paper.

Energy-Aware Traffic Engineering and SDNs

Internet traffic engineering deals with issues of performance evaluation, optimisation, and deployment of technology for measuring, characterising, modelling and controlling network traffic. One of its goals is to control and optimise the routing function to steer traffic through the network in an effective way [START_REF] Awduche | Overview and principles of internet traffic engineering[END_REF], providing appropriate Quality of Service (QoS) and efficient use of network resources. Over the years, interest has grown on applying traffic engineering as a network-wide technique to improve the energy efficiency of network resources [START_REF] Vasić | Energy-aware traffic engineering[END_REF][START_REF] Zhang | GreenTE: Poweraware traffic engineering[END_REF][START_REF] Borylo | Anycast Routing for Carbon Footprint Reduction in WDM Hybrid Power Networks with Data Centers[END_REF]; such efforts are hereafter termed simply as Green Traffic Engineering (GreenTE). Although obtained results are promising, much of the work remains based on numerical analyses and simulation. In an attempt to validate our findings using a real testbed, we identified certain GreenTE requirements that experimental platforms should provide, some of which are summarised in Table 2.

The requirements are grouped in hardware resources, information about traffic, energy-optimisation mechanisms, protocols for enabling traffic engineering, management and control, and measurement of power consumption and performance evaluation. Ideally, modelling and simulation should reflect the behaviour of a real system, but Table 2 provides some assumptions and simplifications found in literature. Whilst some elements may look obvious, many of them may affect the overall assessment of the analysed solution. Testbeds and actual measurements of performance and energyconsumption may eliminate most of them and may reveal side-effects not captured during simulations.

An important requirement of traffic-engineering comprises the ability to gather information about the state of the network, the needs of applications, source of electricity used to power network elements (i.e. renewable, non-renewable) and to configure the behaviour of network elements to steer traffic flows accordingly. Such functions, embedded into data and control planes, were traditionally performed in a decentralised manner, but more recently many traffic-engineering schemes have considered the centralisation of control functions enabled by technologies such as SDNs. SDN separates control and data planes, which in practical terms means that network devices perform tasks that ensure data forwarding (i.e. the data plane) whereas management activities (i.e. the control plane) are factored out and placed at a central entity termed as the SDN controller. SDN has evolved from several technologies, such as Open-Flow, which aim to provide a remote controller with the power to modify the behaviour of network devices via well-defined forwarding instructions. Effort has been SDNs simplify many of the traffic-engineering requirements on gathering traffic information, perform-ing management and control. As described in the next section, in the GETB-SR we use ONOS, an initiative to build an SDN controller that relies on open-source software components, provides northbound abstractions, and has southbound interfaces to handle OpenFlow capable and legacy devices [START_REF]Introducing ONOS: A SDN network operating system for service providers[END_REF]. In addition to a distributed core that enables control functions to be executed by a cluster of servers, ONOS provides two interesting northbound abstractions, namely the Intent Framework and the Global Network View. The intent framework, depicted in Figure 1, allows an application to request a network service without knowledge of how the service is performed. An intent manifested by an application is converted into a series of rules and actions that are applied to network devices. An example of intent is setting up an optical path between switches A and B with amount C of bandwidth. The global network view, as the name implies, provides an application with a view of the network and APIs to program it. The application may treat the view as a graph and perform several tasks that are crucial to traffic engineering, such as finding shortest paths. ONOS provides an application that partially implements SPRING, a framework to enable segment routing currently being standardised by IETF 1 . SPRING provides features for traffic engi-neering as it enables an application to specify paths for data flows while avoiding certain network links.

Simultaneously, in the GETB-AR we use Ryu, a component-based software defined networking framework aimed at creating new network management and control applications [START_REF]Ryu sdn framework[END_REF]. It provides southbound API to control network equipment using various protocols (OpenFlow versions 1.0, 1.2 1.3, 1.4 and 1.5, Netconf or OF-config). Ryu also defines northbound API for deploying SDN applications such as energy-aware traffic engineering mechanisms. Applications in Ryu are software entities running in individual threads and sending asynchronous events to one another [START_REF]Ryu project team: RYU SDN Framework -English Edition. Ryu SDN Framework Community[END_REF]. The Ryu controller was chosen for the GETB-AR testbed as an alternative solution to the ONOS framework used for GETB-SR and to acquire wide experience on configuration and implementation of various SDN controllers. Each application in Ryu has a dedicated FIFO queue to hold incoming events while appropriate event handlers are called for various event types. The application programming model is depicted in Figure 2. The potential of the Ryu framework is proved by commercial deployments in NTT proprietary cloud data centres. Fig. 2 Application architecture within Ryu framework [START_REF]Ryu project team: RYU SDN Framework -English Edition. Ryu SDN Framework Community[END_REF].

Proof-of-Concept Platforms

We built two proof-of-concept testbeds: GrEen Traffic engineering testBed -Segment Routing (GETB-SR) and The GrEen Traffic engineering testBed -Anycast Routing (GETB-AR). The aim of both testbeds is to thoroughly investigate energy-aware traffic engineering approaches proposed in our previous work but evaluated only using discrete-event simulation. Additionally, based on the conducted research, we present challenges faced while implementing energy-aware traffic engineering mechanisms as SDN services in testbed environment.

GrEen Traffic engineering testBed -Segment Routing (GETB-SR)

Figure 3 illustrates the GETB-SR platform and its main components, depicting the deployment of switches, an SDN controller and applications. At smaller scale, the platform comprises components that are common to other infrastructures set up for networking research [START_REF] Kim | Proceedings of the asia-pacific advanced network. OF@TEIN: An OpenFlow-enabled SDN Testbed over[END_REF][START_REF] Melazzi | An openflow-based testbed for information centric networking[END_REF][START_REF] Sallent | FIBRE project: Brazil and Europe unite forces and testbeds for the internet of the future[END_REF]. Moreover, we attempt to employ software used at the Grid5000 testbed [START_REF] Bolze | Grid'5000: a large scale and highly reconfigurable experimental Grid testbed[END_REF] 2 to which we intend to integrate the platform.

To use the platform, a user requests: a slice or a set of cluster nodes to be used by an application as virtual switches or serving as traffic sources and sinks, an OS image to be deployed and a network topology to be used (step 1). We crafted several OS images so that nodes can be configured as SDN controllers and OpenFlow software switches, as discussed later. A baremetal deployment system is used to copy the OS images to the respective nodes and configure them accordingly [START_REF] Jeanvoine | Kadeploy3: Efficient and Scalable Operating System Provisioning[END_REF], whereas a Python application configures VLANs and interfaces of the virtual switches emulating optical switches interconnecting the nodes in order to create the user-specified network topology.

Once the nodes and the network topology are configured, the user deploys his or her application (step 2 in Figure 3). All cluster nodes are connected to enclosure Power Distribution Units (ePDUs)3 that monitor the power consumption of individual sockets [START_REF] Rossigneux | A generic and extensible framework for monitoring energy consumption of OpenStack clouds[END_REF]. This information on power consumption may be used to evaluate the efficiency of an SDN technique (step 3). The data plane comprises two types of OpenFlow switches, namely software-based and hardware-assisted. The former consists of a vanilla OVS [START_REF] Pfaff | The design and implementation of open vSwitch[END_REF], whereas the latter OVS offloads certain OpenFlow functionalities to NetFPGA cards [3]4 . We use a custom OpenFlow implementation for NetFPGAs, initially provided by the Universität Paderborn (UPB) [START_REF]The netfpga-10g upb openflow switch[END_REF], that performs certain OpenFlow functions in the card, e.g. flow tables, packet matching against tables, and forwarding.

Although the NetFPGA cards are by default programmed as custom OpenFlow switches, a user can reprogram them for different purposes by copying a bitstream file to their flash memories and rebooting the system. The current testbed comprises eight serversfive Dell R720 servers equipped with a 10Gbps Ethernet card with 2 SPF+ ports each and three HP Z800 servers with NetFPGA cards with 4 SPF+ ports each. All servers also have multiple 1Gbps Ethernet ports. The infrastructure and the use of ONOS satisfy some requirements of energy-aware traffic engineering, namely providing actual hardware, allowing for traffic information to be gathered, using actual network protocols, enabling the overhead of control and management to be measured, and monitoring the power consumption of equipment. Some energy-optimisation mechanisms, however, are still emulated, such as switching off/on individual switch ports. Although the IP cores of the Ethernet hardware used in the NetFPGA cards enable changing the state of certain components, such as switching off transceivers, that would require a complete redesign of the employed OpenFlow implementation. It has therefore been left for future work.

GrEen Traffic engineering testBed -Anycast Routing (GETB-AR)

The GETB-AR testbed was placed on an 8-core 2.83GHz server running VMware ESXi 5.5, with maximum shared allocation of 20GB RAM for all machines. Virtual disks of the testbed machines were placed on a local RAID 10 array of 4x15k RPM disks. All machines run with 512MB vRAM, except for the VM hosting the SDN controller, which has 2048MB vRAM. VM disks are thinprovisioned 16GB linked clones. There are 14 machines running OVS switches (to support NSFNet topology and validate results presented in previous work [START_REF] Borylo | Anycast Routing for Carbon Footprint Reduction in WDM Hybrid Power Networks with Data Centers[END_REF]). All machines are connected with one vNIC to a management network allowing for remote access and 7 NICs for testbed connectivity. A set of 21 vSwitches was created for connectivity among OVS machines. An additional set of 14 vSwitches and a set of 14 client machines were created to provide user traffic. A separate set of 14 vSwitches and a set of 14 machines were created to provide simulated Data Centre (DC) facilities. Thus, all the testbed traffic is being exchanged within the single physical server.

A Ryu SDN controller application was deployed to handle traffic in accordance with the energy-aware anycast strategies introduced in previous work [START_REF] Borylo | Anycast Routing for Carbon Footprint Reduction in WDM Hybrid Power Networks with Data Centers[END_REF]. Accurate routing of both anycast and unicast traffic relies on up-to-date network data including topology, available resources and other control information gathered from network nodes, and user input via a REST API interface. The acquired data is used to create a network graph continuously updated by the controller's topology discovery module. When a new connection is initialised, its first packets are forwarded to the controller and thoroughly examined to determine unique flow identifiers based on L3/L4 headers. The best path is calculated using one of the available algorithms taking into account available destination nodes, resource constraints and active routing policy. Flow entries are then inserted into each intermediate node's flow table, causing all subsequent packets that belong to the same flow to be forwarded automatically to an appropriate destination, without further notice of the network controller. The southbound interface uses the OpenFlow protocol version 1.3, enabling cooperation with compliant hardware switches. Figure 5 illustrates the GETB-AR platform. To model optical network properties we used MPLS tunnels that reserve assumed amount of resources on the links along the whole path. The aforementioned mechanisms regard to the management and control as well as traffic information GreenTE requirements listed in Table 2. Future extensions. In the current GETB-AR testbed configuration, information about the type of power used at each DC is provided manually through the SDN controller northbound API extended in our testbed. Further development is envisioned to acquire information about the percentage of green energy used in all the nodes automatically. This task is similar to the discovery of network topology where Link Layer Discovery Protocol is utilised along with an appropriate SDN controller extension. We consider two approaches to implement this function.

The first one, the Power Grid Orchestration (PGO), assumes that all nodes have an interface to the power grid network, which allows to estimate the share of renewable energy in the energy supplied to the node. This additional information has to be forwarded to the SDN controller with the use of an appropriate Open Flow protocol extension. The SDN controller is supposed to update its database and modify the behaviour of the traffic engineering applications. The main advantage of PGO is the automatic acquisition of information on the power status of all network nodes and possible use of this knowledge for traffic engineering purposes.

The second concept, called the Open Stack Orchestration (OSO), involves SDN integration in the area of green networking with OpenStack software. In this case, the information about the power status of all DC nodes will come from the OpenStack module responsible for efficient utilisation of the controlled resources. This is possible due to the use of energy-aware interface between SDN and OpenStack. The OpenStack software may estimate and distribute a synthetic index based on many factors, including the share of green energy consumed in the node, current price of energy, utilisation of computing resources, the number and complexity of computing tasks scheduled in a given time scale and many others. The main disadvantage of this solution is that distribution of a single index value for each DC node might be insufficient for other traffic engineering applications. Both approaches to gather node power conditions are of course not exclusive and may be used together.

Another possible extension of GETB-AR testbed is to implement sample network services. We work towards implementing security mechanisms in a cloud computing infrastructure composed of public and private resources. Our research in the area of secure hybrid cloud infrastructure [START_REF] Kurek | Taking back control of privacy: a novel framework for preserving cloud-based firewall policy confidentiality[END_REF] is coordinated with the GETB-AR design, implementation and configuration. Public cloud infrastructure is composed of all DC nodes established in the GETB-AR testbed, private cloud computing resources are represented by all virtual machines attached to network nodes. Some experiments and evaluation of an ongoing work within the secure green networking testbed are planned for near future.

Green Traffic Engineering Use Cases

In this section, we discuss two energy-aware trafficengineering approaches. The first uses segment routing to reroute traffic in order to free links that henceforth become candidates to be switched off, whereas the second establishes paths for anycast requests considering the type of energy used to power target data centres. A crucial contribution of this section is the comparison of results obtained in the testbeds with those using discrete-event simulations.

Segment-Routing Service

Our strategies for routing data flows so that underutilised links may be freed and powered off [START_REF] Carpa | Segment routing based traffic engineering for energy efficient backbone networks[END_REF] from the observation that networks are seldom highly utilised, and that most traffic often follows diurnal and weekly patterns. The SPRING framework is used because unlike in MultiProtocol Label Switching (MPLS)-TE, link and switch IDs called Segment Identifiers (SIDs), are global within an autonomous domain, hence allowing for source-routing. In this way, a flow may be classified at an ingress router and steered through a given path. This section describes the service life cycle and discusses issues that the testbed enables us to identify and investigate. Fig. 6 Start phase of the segment-routing application.

Service Life Cycle

The service, which is a custom version of the ONOS segment-routing application, uses a series of ONOS com-ponents, including its topology information, flow-rule services, and traffic flow objectives. As shown in Figure 6, a service Manager triggers the creation of remaining components when it is launched. The energyaware module, which comprises the proposed trafficengineering algorithms, registers a flow-rule listener to measure flow traffic and link utilisation. The configuration component loads a file that specifies how switches are connected to local networks; information which is then augmented by a topology discovery process. Once the topology is updated, default shortest-path rules are created to guarantee that hosts from a network connected to a switch may reach hosts linked to another switch. A rule consists of a forwarding objective comprising a traffic selector and a treatment. Selectors and treatments result in sets of OpenFlow instructions that are passed to the switches. MPLS push/pop forwarding objectives are created for switches that do not have ports in the source and destination segments -i.e. are neither ingress nor egress switches -and normal IP forwarding objectives are built otherwise. While the service is running, the energy-aware module is notified about changes in topology as well as link utilisation, and periodically evaluates whether there are links to switch off/on. If changes in the link availability are required, the energy-aware module requests a flow-rule update to the Flow-Rule Population module.

GreenTE Issues

Although switching off underused links may be effective from energy efficiency perspective, sudden bursts in traffic may lead to congestion, hence requiring links to be made available. Our previous work [START_REF] Carpa | Responsive Algorithms for Handling Load Surges and Switching Links On in Green Networks[END_REF] proposed algorithms that may react rapidly to traffic bursts by switching links back on when traffic increases. Performance evaluation using discrete-event simulation and UDP-like traffic has shown that the approach successfully reacts to traffic bursts without incurring considerable packet loss. It is assumed, however, that the SDN controller gathers the information about link utilisation from switches every second and that a decision to power a given link back on may be taken and enforced quickly. We performed a simple test and measured the time needed for a controller to decide whether a link should be switched on. A small network topology was considered, as depicted in Figure 7. The Figure also shows the ONOS graphical interface and a data flow (green lines). The network starts with a minimal number of links turned on, forming a spanning tree, and with a TCP flow that nearly exceeds the utilisation threshold, above which the controller decides to turn on more links to handle congestion. A second flow is then injected, thus exceeding the threshold and forcing the controller to switch links on; we measure the time from flow injection to a switch-on decision. In the simulation, the decision takes on average 1.075 seconds, with most of the time spent gathering information on link utilisation. In the testbed, the time is on average 20% higher than in simulation.

We notice that the difference in results between simulation and real testbeds are generally due to simulations assuming zero delay at multiple parts of the processing pipeline and the manner network events are handled. While a single delay simplification would have marginal impact on the results, multiple delays along the packet processing pipeline can account for up to 30% difference in the time to react to changes. Examples of delay simplifications during discrete-event simulations include: instantaneous insertion of forwarding rules into the data path, immediate update of routing tables, fast propagation of flow counters from the simulated hardware ports to the software of the SDN switch, and instantaneous processing of IP UDP/TCP packets. Generally, the only delay properly handled by a simulator is packet queueing time.

Moreover, existing work has already shown that updating the data path forwarding rules is slow in current commercial SDN switches [START_REF] He | Measuring control plane latency in sdn-enabled switches[END_REF]. Google employees report 5 that their SDN-based WAN had an outage due to this issue on propagation of forwarding rules. Improvements can be made in the simulation software to account for some delay, and in the hardware design itself to reduce the time to propagate rules.

Other issues that we investigated concern the stability of the algorithms and the impact of traffic re-routing on TCP flows. Unlike traditional networks where changes in link availability are sporadic, under GreenTE frequent changes may be the rule. Re-routing TCP flows, however, may lead to serious performance degradation due to segments arriving out of order, which in turn result in multiple duplicate ACKs and hence triggering the TCP congestion algorithms at source. Even though the algorithms in the simulator mimic the behaviour of their corresponding theoretical models, they differ from the actual network software implementations provided by certain operating systems. The Linux kernel, for instance, includes several non-standard optimisations [START_REF] Sarolahti | Congestion control in linux tcp[END_REF]. While simulations highlighted that rerouting TCP flows severely impacts the throughput of the transported TCP flows, empirical evaluation on the testbed demonstrated almost no impact under the same conditions. We believe that existing work that wraps real network software stack into simulators6 may help minimise this issue.

Energy-Aware Anycast Routing

One of the approaches to green networking is based on using energy produced from renewable energy sources as this results in carbon footprint reduction. This approach is especially important in the context of networks connecting Data Centres (DCs). Providing numerous cloud computing services requires huge computing power, which in turn, results in enormous power requirements by DCs. Therefore, powering DCs with renewable energy is expected to reduce CO 2 emission significantly. In hybrid power networks, selected DCs are powered using energy from renewable sources while the rest uses energy obtained from conventional sources. As one of the paradigms of cloud computing is to offer the same services in different DCs at the same time, it is possible to choose one of many possible locations to provide the requested service. A corresponding routing scheme is called anycast [START_REF] Develder | Optical Networks for Grid and Cloud Computing Applications[END_REF]. Another important feature of cloud computing is to offer services in an ondemand fashion. That is why incoming requests are unpredictable and the underlying network must be investigated under a dynamic traffic scenario [START_REF] Develder | Optical Networks for Grid and Cloud Computing Applications[END_REF]. Therefore, features like global view of the current network topology and allocated paths as well as possibility to dynamically handle incoming requests are essential. Both features are ensured by the SDN idea with a centralised controller. The GETB-AR testbed is used to evaluate anycast strategies that aim to reduce greenhouse gases emission by processing resource-intensive requests in data centres powered from renewable energy sources.

The anycast routing problem consists of a graph G(V, E) representing the physical network, where V is the set of nodes and E is the set of network links. A subset of V DC ⊂ V comprises data centres that serve user requests and V C ⊂ V denotes source switches/routers to which users are connected. V gDC comprises green data centres that are powered by renewable energy sources, whereas V bDC consists of brown data centres that use traditional energy sources. The following relations are met:

V gDC V bDC = ∅, V gDC V bDC = V DC , V DC V C = ∅ and V DC V C = V .
The strategies aim to prioritise the use of V gDC while respecting several connectivity and availability constraints. Figure 8 illustrates the virtual infrastructure and topology developed on the GETB-AR platform.

Anycast strategies

In previous work [START_REF] Borylo | Anycast Routing for Carbon Footprint Reduction in WDM Hybrid Power Networks with Data Centers[END_REF], we proposed three anycast routing strategies (randomGreen, closestGreen and closes-tGreenWithPenalty) focused on reducing the carbon footprint. All the strategies require very limited additional control information and hence may be easily implemented using a centralised network controller consistent with the idea of SDN. The proposed strategies are compared to the single anycast strategy, which is the base reference strategy, well known and widely described in literature. However, the general idea behind the proposed heuristics is to provide some kind of green DC preference while choosing a destination for an anycast request and, at the same time, consider multiple DCs as targets for the request to improve network performance. Thus, to ensure a comprehensive assessment of the three proposed strategies, we also provided two additional reference strategies, random and closest, which consider multiple DCs as targets for an anycast request but do not prefer green DCs over brown ones. In this paper we provide only short descriptions of those strategies, but detailed information along with pseudocode can be found in previous work [START_REF] Borylo | Anycast Routing for Carbon Footprint Reduction in WDM Hybrid Power Networks with Data Centers[END_REF].

In the single strategy an anycast request is served by first randomly choosing a single destination d within V DC . Then it checks whether the wavelength continuity constraint can be met by any of the three alternate optical paths between the source and randomly selected destination node. If a lightpath is available between the source s ∈ V C and the destination d, then the request is accepted; otherwise it is rejected. In the random strategy the network controller iteratively tries to establish the lighpath between s ∈ V C and a random d ∈ V DC . The strategy ends when the first available d is found or none of the possible destinations is available after examining all possible destinations. In case of the closest strategy, the closest (in the hop manner) destination d is chosen and the lightpath is established between s and d. If none of d ∈ D is available then the request is rejected. In the randomGreen the SDN controller iteratively tries to establish the lighpath between s and a random d ∈ V gDC . If all d ∈ V gDC are tried without success then the same scheme is performed for d ∈ V bDC . In the closestGreen strategy the network controller performs operations analogous to the closest strategy for all d ∈ V gDC . If none of d ∈ V gDC is available then the SDN controller repeats the same operation for all d ∈ V bDC . If none of d ∈ D is available then the request is rejected. The closestGreenWithPenalty strategy works analogously to the closest strategy but with one significant difference. The distance from s to d ∈ V bDC is multiplied by the penalty factor, where the penalty is an input parameter of the strategy. The closestGreenWithPenalty strategy with penalty = 1.0 is equivalent to the closest strategy.

To sum up, the three proposed strategies (random-Green, closestGreen and closestGreenWithPenalty) create an opportunity to reduce carbon footprint by favouring green DCs over brown ones. However, as a side effect, the average lightpath length and resource utilisation may significantly increase in case of randomGreen and closestGreen strategies, where green DCs are firmly prioritised. Thus, the closestGreenWithPenalty strategy was proposed to flexibly balance the trade-off between carbon footprint reduction and the average lightpath length. The higher value is assigned to the penalty parameter the more green DCs are preferred over brown ones and, at the same time, the higher is the average lightpath length. Each of the three proposed strategies is expected to reduce the blocking probability of anycast requests in comparison to the single strategy. Thus, for a comprehensive assessment, the impact of the proposed strategies on network performance will be additionally investigated in comparison to the auxiliary reference strategies, random and closest. The closest strategy is especially significant in the context of network performance as it is expected to ensure the shortest average lightpath length and thus the lowest network resource occupancy. We use a simple shortest path (least hop) routing algorithm and firstfit wavelength assignment with k-alternate paths [START_REF] Szymanski | Performance evaluation of the grade-of-service-based routing strategies for optical networks[END_REF].

The implementation of anycast strategies is an essential part of the controller's application and relies on its knowledge of network state and available resources. The network graph created by the topology discovery module comprises vertices and edges representing network nodes and network links. Neighbour adjacency is discovered by exchange of LLDP packets containing unique identifiers. Each vertex holds node-specific data including its type (brown DC, green DC, client) and interface configuration. Information about available and reserved wavelengths is held by graph edges. This information, along with penalty values and flow-specific parameters is utilised in the process of path computation. Once a packet in message is received by the controller, its contents are extracted to determine ingress node and flow parameters, namely IP addresses and TCP/UDP port numbers. After an optimal path, with regard to active anycast strategy, is chosen by the routing algorithm, OpenFlow instructions are passed to each switch along the desired flow route. Each intermediate node is supplied with two flow entries specifying appropriate output ports to enable bidirectional transmission. Once the path is set up, it may be removed anytime, allowing for flexible network reconfiguration.

Results

Two metrics were used to asses the anycast strategies. The first one estimates carbon footprint and is the average ratio of the power consumed from nonrenewable sources to the amount of DC traffic switched in all DCs (brownKiloW atts/(Gb/s)). Thanks to this normalisation we obtain comparable results under different traffic loads. The second metric is the blocking probability, calculated as the ratio of rejected requests to all requests. An external traffic generator module was implemented to evaluate performance of anycast strategies in the testbed environment. Each of the 14 hosts runs a client application able to generate multiple TCP/UDP streams using the iperf tool. Both anycast and unicast connections are set up on demand while compliance with traffic parameters described in previous work [START_REF] Borylo | Anycast Routing for Carbon Footprint Reduction in WDM Hybrid Power Networks with Data Centers[END_REF] is enforced by a central supervisor generating relevant connection requests. The SDN controller collects statistics of each flow, e.g. transmission time, end nodes and assigned resources. Data regarding the complete routing and network state is available to the controller while end of transmission timestamps are obtained directly from end nodes. A dedicated controller module was implemented to gather and store historical data about flows. As a result of detailed flow database analysis, both indicators, e.g. brownKiloW atts/(Gb/s) and blocking probability were calculated.

Figures 9 and 10 present carbon footprint of the analysed anycast strategies with different data centres indicated as green. In each figure, the left-hand graph presents results obtained in the discrete-event simulator and the right-hand one shows results obtained in the GETB-AR testbed. Figures 11 and12 show the corresponding total blocking probabilities.

The main differences between results obtained in discrete-event simulations and GETB-AR testbed are related to the absolute values. Additionally, differences between particular strategies are less pronounced in the testbed comparing to the simulations. The reason behind such observations is a difference in modelling of network resources in both environments. Finally, results obtained in GETB-AR testbed oscillate between different network loads while this effect cannot be observed for simulation-based results. It is a result of more realistic approach to generation of user requests in virtual machines instead of simulating such requests as programming objects in simulation environment. All of the issues indicate that any simulation-based conclusions should always be drawn with proper caution. However, those minor disparities do not affect a general conclusion that preliminary assessment of anycast strategies performed in discrete-event simulations is proved to be true with results obtained in the GETB-AR testbed. Simulation results show that the three proposed strategies randomGreen, closestGreen and closestGreenWith-Penalty may significantly reduce CO 2 emission in comparison to all reference strategies, and decrease blocking probability in comparison to the single strategy. Furthermore, in each simulation scenario the closestGreen-WithPenalty strategy retains network performance at a level comparable with the closest strategy, which is expected to provide the lowest network utilisation. Additionally, the penalty parameter allows the closestGreen-WithPenalty strategy to elastically balance the tradeoff between carbon footprint reduction and network performance.

Several issues not observed in discrete-event simulations needed to be addressed during development and deployment of the testbed. These were mainly related to detection and assessment of network events and limited computation resources available to the controller and switch software causing packet processing delays. While a connection setup is relatively easily detected by the controller when packets with an unknown flow identifier are received, it is difficult to tell whether the connection is still active at a particular moment. Considering a variety of services being present on the network, there are no common measures such as timeouts indicating expired connections. Although TCP flows may be monitored for presence of FIN and RST flags, such method is not suitable for UDP streams. Still, instant deallocation of unused resources and network state database update is essential for energy-aware routing algorithms to work efficiently. One of the proposed solutions involved applying idle timeouts to handle expired flows. However, such approach could cause several connections to be disrupted prematurely or allocated resources to be freed belatedly. To alleviate the issue, additional controller REST API calls were implemented for client application to notify controller of connection teardown immediately after the end of transmission. This, however, requires client applications to be aware of flow setup mechanism and additional control modules to be implemented by their developers.

Another issue faced in the testbed environment was caused by limited performance of the SDN controller. Although satisfactory switching rates may be achieved by Open vSwitch software deployed on VMWare virtual machines, bursty traffic may drain the controller's resources and make it a network bottleneck. Considering the time required to parse incoming packet in Open-Flow message, calculate efficient path and push flow information to all intermediate flow tables, additional precautions should be taken to prevent inappropriate handling of network streams. This includes additional mechanisms ensuring all flows to be detected only on reception of their initial packet and packet out messages being sent in the correct order. Again, the issue is consistent with GreenTE requirement on management and control presented in Table 2.

Related Work

Several solutions have been proposed to make networks more energy efficient, comprising improvements in used materials, encoding and decoding techniques, power efficient transceivers and other improvements in network equipment. Whilst our algorithms may benefit from improvements in hardware and transmission, we focus on techniques that operate at the routing level. In this area, solutions range from putting network interfaces into sleep mode [START_REF] Gupta | Greening of the internet[END_REF] to increasing idle periods of certain links by changing flow paths [START_REF] Vasić | Energy-aware traffic engineering[END_REF]. A detailed review of the state of the art on this topic is presented in previous work [START_REF] Carpa | Segment routing based traffic engineering for energy efficient backbone networks[END_REF].

In the present work, we focused on describing the importance of a platform to evaluate energy-aware trafficengineering algorithms. Infrastructure for research and development of distributed systems have been established over the years [START_REF] Bolze | Grid'5000: a large scale and highly reconfigurable experimental Grid testbed[END_REF][START_REF] Peterson | Plan-etLab architecture: An overview[END_REF]2], including platforms for SDN solutions [START_REF] Banikazemi | Meridian: an SDN platform for cloud network services[END_REF] and SDN testbeds [START_REF] Kim | Proceedings of the asia-pacific advanced network. OF@TEIN: An OpenFlow-enabled SDN Testbed over[END_REF][START_REF] Melazzi | An openflow-based testbed for information centric networking[END_REF][START_REF] Sallent | FIBRE project: Brazil and Europe unite forces and testbeds for the internet of the future[END_REF][START_REF] Ooteghem | Sustaining a federation of future internet experimental facilities[END_REF]. Our approach and previously described platforms have many similarities, but we focus on providing an infrastructure that may be used for both evaluating SDNbased solutions and assessing their energy efficiency. Most of research in the area of testbed-based evaluation of energy-aware mechanisms is focused on data centres. For example, a green DC testbed has been proposed in previous work [START_REF] Goiri | Designing and managing data centers powered by renewable energy[END_REF]. It adjusts the workload handling process to the availability of solar and wind energy and optimises air conditioning with regard to the outside air temperature. A platform for energy-aware analysis of an internal DC network was also proposed [START_REF] Thanh | Enabling experiments for energy-efficient data center networks on OpenFlow-based platform[END_REF], where the authors combined hardware with emulation techniques and proposed an extension to the Open-Flow protocol in order to measure energy consumption of infrastructure components as well as link occupancy. Another extension to the OpenFlow protocol was also evaluated in a testbed environment [START_REF] Vu | Power aware OpenFlow switch extension for energy saving in data centers[END_REF], where the aim was to reduce energy consumption by turning off switches and their ports in an internal DC network along with changing clock frequency in the equipment.

Conclusions

This paper discussed challenges towards building testbeds and requirements imposed on those testbeds and SDN platforms for validating and evaluating energy-aware traffic-engineering algorithms. We presented two SDN applications: the first one uses segment routing to reroute traffic in order to free certain network links which can then be switched off, whereas the second establishes paths for anycast requests considering the type of energy used to power target data centres. We also illustrated the use of the testbeds. Specifically, in the GETB-SR testbed we discussed challenges on improving the stability of routing algorithms and TCP flows in networks employing GreenTE mechanisms, and by utilising the GETB-AR testbed we compared effectiveness of energy-aware anycast strategies evaluated in the testbed and contrasted it with results obtained using discrete-event simulation. We also presented issues specific of testbed environment along with challenges toward deployment of green anycast strategies in the GETB-AR.

Fig. 1

 1 Fig. 1 ONOS Intent Framework.

Fig. 3

 3 Fig.3Overview of the GETB-SR platform.

Fig. 4

 4 Fig.4The GETB-SR platform.

Fig. 5

 5 Fig.5Overview of the GETB-AR platform.

Fig. 7

 7 Fig. 7 ONOS GUI showing a data flow avoiding the shortest path.

Fig. 8

 8 Fig. 8 Overview of the evaluated scenario.

Fig. 10

 10 Fig.10Brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with V gDC ∈ {9, 11}.

Table 1

 1 GreenTE requirements and commonly adopted approaches.

		GreenTE	How Requirements are Tackled by Solutions
		Requirements	Simulation	Testbeds
				Simplified and approximate
	Hardware resources	software abstractions of hardware, energy consumption, access time	Often real equipments running in a controlled environment
				to resources
	Traffic information		Commonly assumed that information about flows can be gathered without perturbing the network; centrally available	Monitoring protocols coexist with other network functions, excessive monitoring can impact normal traffic when sharing network resources
	Energy-optimisation mechanisms (e.g. Link/port switch on/off, Adaptive Power Idle (LPI)) Link Rate (ALR), Low	Simplified models, assumptions made when implementing support not always available on simulators, parameter details	Actual ALR and LPI, simulated or actual link/port switch off/on
				Partial implementation of
	Network protocols (e.g.	evaluated schemes, often relying	Normally complete protocol stack,
	MPLS-TE, RSVP, SPRING,	on lower-level protocols that	presence of side-effects that may
	OpenFlow)		present already approximate	be neglected by simulation tools
				behaviour
	Management and control	Commonly assumed that the overhead of configuration and control is negligible	Either dedicated infrastructure allocated to management or it shares resources used by normal traffic; overhead can be measured
					Use of managed PDUs,
	Monitoring of power	Monitoring is performed by	wattmeters for measuring the
	consumption and		gathering stats derived from	consumption of power lines,
	performance evaluation	consumption models	infrastructure for gathering
					energy consumption stats
	made towards standardising the interface between con-
	troller and the data plane, generally termed as south-
	bound API, and the manner the controller exposes net-
	work programmability features to applications, com-
	monly called northbound API. An example of such an
	application is cloud infrastructure utilising a network
	to provision cloud services for customers. In this case
	the northbound interface may carry information about
	availability of computing resources or renewable energy
	in multiple data centres.
	SDN Applications	App 1	App 2	App 3
		Application Intent Framework
	Intents are	Distributed Core
	translated into
	instructions for	Southbound
	network devices
		Openflow	NetConf	Southbound Interface

  stem

	Anycast Strategies Configuration		Traffic Status
	User				Notifications
			UI	REST API
	VMware ESXi	VM Deployment	Energy-Aware Anycast Routing	Network State DB	Traffic Supervisor
			Ryu SDN Controller
			OpenFlow
	VM Deployment	Open vSwitch	Open vSwitch	Open vSwitch
		Traffic	Traffic		Traffic
	VM	Generator	Generator	Generator
	Deployment
		Linux Host	Linux Host	Linux Host
		Dataplane
				Connection
				Requests

  Fig.9Brown kilowatts needed to handle 1 Gb/s of DC requests in the NSF network with V gDC ∈ {4, 11}.

				Obtained using Simulations						Obtained in GETB-AR
	Brown kW per 1 Gb/s of DC requests	0 10 20	1	1.2	1.4	1.6	1.8	2	2.2	2.4	Brown kW per 1 Gb/s of DC requests	0 10 20	1	1.2	1.4	1.6	1.8	2	2.2	2.4
						Offered DC load									Offered DC load
						single	random			randomGreen
						closest	closestGreen		closestGreenWithPenalty (penalty=1.9)
				Obtained using Simulations						Obtained in GETB-AR
	Brown kW per 1 Gb/s of DC requests	0 10 20 30	1	1.2	1.4	1.6	1.8	2	2.2	2.4	Brown kW per 1 Gb/s of DC requests	10 20 30	1	1.2	1.4	1.6	1.8	2	2.2	2.4
						Offered DC load									Offered DC load
						single	random			randomGreen
						closest	closestGreen		closestGreenWithPenalty (penalty=2.9)

  Fig. 11 Total blocking probability of both traffic types in the NSF network with V gDC ∈ {4, 11}.Fig. 12 Total blocking probability of both traffic types in the NSF network with V gDC ∈ {9, 11}.

			Obtained using Simulations				Obtained in GETB-AR
	Total blocking probability	0.1 0.2							Total blocking probability	0.1 0.2
		0
		1	1.2	1.4	1.6	1.8	2	2.2	2.4	1	1.2	1.4	1.6	1.8	2	2.2	2.4
					Offered DC load							Offered DC load
					single	random		randomGreen
					closest	closestGreen	closestGreenWithPenalty (penalty=1.9)
			Obtained using Simulations				Obtained in GETB-AR
		0.3								0.3
	Total blocking probability	0.1 0.2							Total blocking probability	0.1 0.2
		0
		1	1.2	1.4	1.6	1.8	2	2.2	2.4	1	1.2	1.4	1.6	1.8	2	2.2	2.4
					Offered DC load							Offered DC load
					single	random		randomGreen
					closest	closestGreen	closestGreenWithPenalty (penalty=2.9)

https://tools.ietf.org/wg/spring/

https://www.grid5000.fr

http://www.eaton.com/Eaton/index.htm

http://netfpga.org/site/#/systems/3netfpga-10g/details/

https://atscaleconference.com/videos/lessonslearned-from-b4-googles-sdn-wan/

[START_REF]Ryu sdn framework[END_REF] http://www.wand.net.nz/ ~stj2/nsc

Acknowledgments

This work was financially supported by the CHIST-ERA ERA-Net SwiTching And tRansmission (STAR) project [1].


PhD Serena Ivaldi
email: serena.ivaldi@inria.fr

Lars Fritzsche

Jan Babič

Dr. rer Freek Nat Stulp

PhD Michael Damsgaard

Dr. Bernhard Graimann

PhD Giovanni Bellusci

PhD Francesco Nori

PhD Jan Babic
email: jan.babic@ijs.si

Anticipatory models of human movements and dynamics: the roadmap of the AnDy project

Keywords: robotics, wearable sensors, ergonomics, machine learning, prediction, anticipation, MSD, human movement, cobots, exoskeletons, humanoids

come

Introduction

In modern societies, the demand for physical assistance to humans is increasing. In factories, for example, production workers execute repetitive tasks in non-ergonomic postures that, on the long run, often cause musculo-skeletal diseases. The increasing age of the workers poses additional risks for muscolo-skeletal accidents in physically demanding tasks. Many workstations are now equipped with co-bots sharing the workspace with workers. Despite the fact that the low-level control of these machines allows the humans to physically interact with them in a safe way, most workstations actually implement the concept of "human-robot co-existence", that is the robot and the human share the space, work side by side, but have very few limited interactions. Particularly, there is little "collaboration", that literally means "working together". If robots want to go beyond the mere "act" in a shared space, they must not only be proficient in physical interaction, but also in physical collaboration. Not all (physical) interactions necessarily are collaborations. Collaboration is a higher and more cognitive form of interaction, where the two partners mutually understand each other, mutually predict and control the exchanged forces, posture and movements while being able to anticipate those of the other partner.

To fully describe the whole-body dynamics of humans collaborating with other humans, and humans collaborating with robots, we need new technologies that allow us to not only estimate the motion, but also to predict its outcome, so that the robots can properly react to the human actions and provide efficient collaboration. These hardware and software technologies are the goals of the H2020 European project AnDy (www.andy-project.eu).

The aim of this paper is to present the roadmap of the AnDy project, presenting the key methodologies and technologies that we will develop for tackling our ambitious objectives.

In AnDy, our requirement is to develop safe dependable robotic systems that are able to collaborate with people by predicting and anticipating their movements. Having such skills requires understanding the biomechanics of collaboration, to have models that we can use to integrate cognition in the robot. To develop these models, we need to track and model the human whole-body dynamics motions. However, current robotic technologies are not yet able to provide these measures in real-time: therefore robots have a blind spot. To predict the future forces and postures of the human, we need first a sensing device that can measure the human whole-body dynamics, which makes the robot aware of the entire human body posture. This is very different from the current cases where the robot only controls the physical interaction with the human, and is thus aware of the human only at the contact level. In other terms, the current limitations of robots in observing human dynamics makes them blind to the human dynamics, and "unaware" of the human effort, which leads to inefficient and non-ergonomic collaboration.

In AnDy, our first milestone is to resolve this blind spot, by developing a novel sensor suit, the AnDySuit, capable of measuring human whole body dynamics in real-time. The big data sets collected with the AnDySuit will be used to generate ergonomic and anticipatory models. Those models will be used online by the robots to adapt their on-line control and make collaboration more efficient and ergonomic. It will combine wearable sensing technologies based on inertial sensors, EMG, tactile and force/torque sensors.

To evaluate the AnDySuit and its application for ergonomic human-robot collaboration, we devised three validation scenarios covering a wide spectrum of technical challenges in a crescendo of complexity. In the first validation scenario, the robot is an industrial cobot, which tailors its controllers to individual workers to improve ergonomy. In the second, the robot is an assistive exoskeleton that optimizes human comfort by reducing physical stress. In the third, the robot is a humanoid, which offers assistance to a human while maintaining the balance of both.In all three cases the robot leverages the measurements of the AnDySuit and the models to predict the intention, action and efforts of the human. This knowledge is the basis to develop novel controllers for efficient human-robot collaboration.

Further, the three scenarios with the three different robots (cobot, exoskeleton and humanoid) allow us to investigate the collaborative action and strategies of the robot in different contexts of manufacturing and industry, which increases the impact of our research.

Methods

The ability to collaborate with humans requires that the robots have the ability to provide physical assistance to people that needs help in performing a certain task. The fundamental assumption is that physical collaboration requires the robot to understand what the partner is doing and predict what the partner is going to do. This requires a model of the partner, and since we are not only interested in the motion but also in the effort, we need a comprehensive model that contains kinematics and dynamics information of the human.

Developing models requires observations, and to make appropriate observations to study a phenomenon often we need to create new tools. This paradigm is recurrent in our history since centuries: for example, the Newtonian telescope was used to observe celestial orbits, which led to the universal gravitation law. In a similar way, to develop anticipatory controllers that enable robots to collaborate with human partners (AnDyControl) we need predictive models of the human motion and dynamics (AnDyModel); to create these models, we need to collect observations of collaborating partners (AnDyDataset), with a novel tool (AnDySuit) that allows us to fully measure the kinematics and dynamics of the humans.

Key technologies

Methodologically, the concept above translates into four sequentially enabling objectives, which define our roadmap:

⎯ The AnDySuit: this is going to be a breakthrough technology that allows us to fill the blind spot of measuring human-robot physical interaction. The AnDySuit is a wearable device that monitors in real-time the kinematics (position, velocity, acceleration at the joints/limbs) and dynamics of the human (contact forces, joint torques and muscle activations): that is, it provides the whole-body posture and whole-body dynamics. By processing the kinematics and dynamics measures, it allows retrieving important variables for collaboration, such as the contact stiffness, the gaze direction (here approximated by head direction), and for ergonomics, such as the body posture.

⎯ The AnDyDataset: the AnDySuit will enable us to record the motion and dynamics of a human performing different tasks, of two humans collaborating, and of a human collaborating with the robot. The kind of recordings we will do are unique in their kind, as we will mix wearable inertial measures, force/torque measures, EMG, motion capture, etc. This will allow us to record kinematics descriptors of the actions (e.g., marker placement, estimated joint angles) as well as dynamics descriptors (e.g., contact force at the hands and feet, arm stiffness). Data collection will take place both in research labs and in real manufacturing scenarios. Experimental protocols with ethics committee approval will be obtained by the partners leading the data collection.

⎯ The AnDyModel: the AnDyDataset will enable the development of models of human motion and dynamics and human collaboration. Accurate dynamics models will rely on the musculo-skeletal models of AnyBody, but model reduction and strong optimization will be obtain real-time signals. We aim at three different types of models: ergonomics models, that are used to retrieve relevant metrics for assessing the ergonomics impact of the motions, based on the metrics of the standard evaluation tools (e.g., EAWS, OCRA) (GLAESER et al., 2014); classification models, that are used to recognize the current ac-tivity of the human; and predictive models, that are used to predict the future evolution of the dynamics and motion of the human (e.g., the goal position of a reaching action, as well as the joint torques that are necessary to accomplish the movement). For the latter, we will explore different machine learning tools, ranging from probabilistic movement primitives to deep neural networks [START_REF] Droniou | Deep unsupervised network for multimodal perception, representation and classification[END_REF].

⎯ The AnDyControl: the final objective is to develop online control strategies for physical human-robot collaboration. Thanks to the AnDySuit, we can measure the human motion and its dynamics in real-time. These observations can be used to make predictions using the prior information provided by the AnDyModel, for example predicting the goal of a human movement, which forces or trajectories to expect. The AnDyControl here has the goal to adapt the robot control strategy in real-time, taking into account such predictions, with the purpose to optimize the collaboration, particularly the ergonomics criteria of the human movement.

The four objectives are summarized in Figure 1, which also represents the logical and temporal structure of the project and the interconnection between the objectives. The development of the AnDySuit is the first goal of the project, as it defines the main dependency for the dataset collection. Since it is mostly based on wearable sensing technologies that are already available on the market, it is reasonable to expect a high TRL (technology readiness level) for this device.

All the other objectives have a lower expected TRL, with the most ambitious predictive models and controllers for collaboration being those with the lowest.

The development of models and controllers will basically proceed in parallel, in a crescendo of complexity. In order to reduce the risks of a purely sequential methodology, we defined a number of sub-objectives at incremental levels of difficulty with respect to two main metrics that are particularly relevant for collaborative tasks: mutualcare and anticipation.

Mutual-care is related to the amount of care for the partner, i.e., the concern that the actor has for the partner (e.g., partner's equilibrium, partner's posture, partner's effort) in the collaborative task execution. In a situation of minimal care, the robot does not particularly care about the forces and postures that the human assumes during the collaboration. If the robot starts "caring" about the human, it means it takes into account human-related variables in the design of its control action. For example, it evaluates the ergonomics postures of the human during the collaborative task, to adapt its action to maximize the comfort of the human. To show better "care", the robot must then monitor the posture and forces in real-time, all along the collaboration, to react and optimize the collaborative action.

Anticipation is related to the amount of synchronization needed to successfully complete the collaborative task, and it is strongly related to the prediction capabilities of the robot. In a situation of minimal anticipation, the robot does not need to anticipate the human partner's positions or forces to perform the collaborative task. If the robot is capable of classifying and recognizing the current action performed by the human, it can choose appropriate control settings to adapt to each action/task. If it can recognize the goal of a reaching movement, or the intent of motion, it can proactively provide a support control action. If it is able to predict the positions and forces that the human partner will do, it can use this information to choose its future control actions and postures as well.

A sketch of the collaborative objectives in terms of the increasing levels of anticipation and mutual-care is shown in Figure 2. Interestingly, the complexity of models and controllers is reflected also in the type of robots used for demonstrating the collaborative tasks.

We stress on the fact that the collaborative scenarios will use the AnDySuit, which provides the online measurements of the human motion and forces, the AnDyModels to extract ergonomics metrics, predict the intentions and motions, and the AnDyController to provide suitable human-careful controllers.

The key difference with respect to previous projects is to provide technologies for human-aware and human-careful physical collaboration. Human-awareness refers to the fact that robot should be able to monitor human motions and forces. Human-care refers to the robot ability to provide a collaborative interaction that takes into account the human needs (e.g. ergonomic posture, minimal fatigue, equilibrium maintenance). In the literature, most prior works dealing with collaboration provide prediction (i.e. human-awareness) but often with limited human monitoring. Very often human-robot interaction is modeled as an exchange of forces, without explicitly modeling neither the human posture, nor the human joint torques. For example, in the manufacturing domain often only the contact forces or the forces at the end-effector are considered. In the healthcare domain, exoskeletons typically access the human posture and exchanged forces (eventually projected into joint-torques) but this information is always local (i.e. limited to body parts where the exoskeleton is attached); the remaining body postures and forces are often neglected. Few projects have been focused on the importance of monitoring human body posture, typically measured with depth sensors (e.g. kinect) or wearable devices (e.g. the Xsens MVN). In this sense, in AnDy we put forward the concept of "human-careful", as opposed to simple "human-aware", meaning that the robot takes into account the whole body measures from the human in the synthesis of its collaborative strategies.

In this sense, the AnDySuit is going to be a breakthrough technology that allows us to fill the blind spot of measuring human-robot physical interaction. This wearable device that monitors in real-time the kinematics (position, velocity, acceleration at the joints/limbs) and dynamics of the human (contact forces, joint torques and muscle activations): that is, it provides the whole-body posture and whole-body dynamics. By processing the kinematics and dynamics measures, it allows retrieving important variables for collaboration, such as the contact stiffness, the gaze direction (here approximated by head direction), and for ergonomics, such as the body posture. These quantities will be used by the robots to optimize their control strategies, with the aim to provide ergonomically safe actions for the human partners. By optimizing humanrelated metrics, we will realize a human-careful control.

Results

The achievements of the project will be experimentally evaluated on different robotic platforms, addressing different collaborative scenarios. Three types of robots are addressed in AnDy: collaborative robotic manipulators, or "cobots", exoskeletons and humanoid robots.

Cobots are already diffused in manufacturing, and so far have been mostly used in co-existance scenario where human operators share the same workspace of the robot, working side-by-side or performing tasks in coordination. For these robots, performing more collaborative tasks is an innovation requiring the online ergonomics estimation of the human partner. The main cobots we will use in AnDy are the KUKA iiwa and LWR. For exoskeletons, the main problem we will address is how to adapt control settings to the action of the human operator in a smooth and automatic way. An interesting research is the quantification of the effects of the action of the exoskeleton on the human body, which requires the measure of kinematics and dynamics of the human thanks to the AnDySuit. The main platform we will use in AnDy is an exoskeleton for overhead work by Ottobock. Humanoids robots, especially legged robots, are still research prototypes. However, in the future it is reasonable to expect that humanoids will find their use in manufacturing, in applications where cobots or even manipulators mounted on a mobile base would not be able to perform the desired tasks. The three collaborative scenarios with the three types of robots are represented in Figure 3. Some European projects are currently focused on producing new humanoids for aircraft manufacturing (H2020 COMANOID -http://comanoid.cnrs.fr) and for technical support (H2020 SECOND HANDS -http://secondhands.eu). In our project, we will use the humanoid robot iCub, which is equipped with force and tactile sensing and allows us to tackle the scientific challenges of collaborative human-humanoid interaction.

Scenario 1 (cobots)

A key feature of collaborative robots / cobots in assembly is that they share a workspace with human workers. Important distinctions between different types of cobots are whether they avoid or exploit contact with humans, and whether they are purely reactive or also predictive [START_REF] Haddadin | Collision detection and reaction: A contribution to safe physical human-robot interaction[END_REF][START_REF] Haddadin | On making robots understand safety: Embedding injury knowledge into control[END_REF]. In contrast, SISBOT et al. ( 2010) generate motion plans that take human social rules into account. [START_REF] Dragan | Generating Legible Motion[END_REF] plan motions such that they become easier for humans to interpret. Kinesthetic teaching exploits human-robot physical contact by enabling humans to demonstrate motions to robots by physically moving the (passive) robot [START_REF] Stulp | Learning Compact Parameterized Skills with a Single Regression[END_REF]. Virtual guides also enable human-robot contact, by constraining the motion to certain pre-defined trajectories or regions [START_REF] Raiola | Co-manipulation with Multiple Probabilistic Virtual Guides[END_REF]. Dynamic, active physical interaction between a human and a robot is particularly challenging. AnDy enables this by 1) giving direct, on-line measurements of the human posture and forces exerted by the human through AnDySuit; 2) by not only reacting to, but also predicting human motion with AnDyModel by analyzing previous executions of tasks stored in AnDyDataset; 3) by taking the ergonomy of the human into account whilst generating motions. Monitoring humans with the AnDySuit will also facilitate the task of programming force-controlled robots for complex assembly tasks. At present the cost of cobots and associated programming is not cost-effective for small production lots (e.g. SMEs) and to flexible and customized productions. ANDY will enable force-controlled assemblies through collaborative human-cobot assemblies that will be more flexible, easier to customize and in the end more cost effective and suitable for SMEs.

Scenario 2 (exoskeletons)

Robotic assistive devices (i.e. exoskeletons) are being developed to either augment the abilities of able-bodied humans or to improve the condition of the people with impaired physical abilities [START_REF] Rahman | Passive exoskeletons for assisting limb movement[END_REF]. Such devices assist movement abilities of different body parts (reviewed by [START_REF] Dollar | Lower extremity exoskeletons and active orthoses: Challenges and state-of-the-art[END_REF]. The purpose of such devices is to provide useful mechanical power by operating synchronously with the person that is wearing it [START_REF] Gams | Effects of robotic knee exoskeleton on human energy expenditure[END_REF]. The human robot interface is crucial for a safe and comfortable interaction with an exoskeleton device [START_REF] De Rossi | Sensing pressure distribution on a lower-limb exoskeleton physical human-machine interface[END_REF].

From the technical perspective there are two branches of exoskeleton development: passive power assist devices and active power assist devices. In general, active power assist devic-es are more powerful, but require actuators and power source, which make them quite heavy; they can be found mostly in rehabilitation, orthopedics, assistive devices [START_REF] Yamamoto | Stand alone wearable power assisting suit -sensing and control systems[END_REF][START_REF] Kawamoto | Power assist method based on Phase Sequence and muscle force condition for HAL[END_REF]. On the other hand, passive power assist devices are mainly used to reduce the burden of the user rather than to amplify their forces [START_REF] Imamura | Motion-Based-Design of Elastic Material for Passive Assistive Device Using Musculoskeletal Model[END_REF][START_REF] Walsh | A quasi-passive leg exoskeleton for load-carrying augmentation[END_REF]. One of the crucial elements of active assistive robots and exoskeletons are the actuators. A promising alternative to conventional stiff actuators is the use of elastic elements to create an intrinsic compliant behavior. This allows a safer human-robot interaction, shock absorbance, and leads to a greater energy efficiency than stiff actuators (VANDERBORGHT et al., 2012). An expansion of compliant actuators is variable stiffness actuators (VSAs) that allow changing the stiffness of the driven joint. These actuators fit the requirements of certain bioinspired robots, since humans use the ability of changing joint stiffness to change, for example, the walking speed [START_REF] Kim | Leg stiffness increases with speed to modulate gait frequency and propulsion energy[END_REF], and to accommodate changes in surface stiffness during walking [START_REF] Ferris | Running in the real world: adjusting leg stiffness for different surfaces[END_REF]. VSAs can, in principle, be used to extend the versatility of spinal support systems. Today, especially for the assembly of heavy or bulky parts, weight compensators/balancers are used. Since these systems do not compensate for inertial forces, even small mistakes lead to work-related injuries (lower back pain, spine injuries) [START_REF] Kroeger | Cooperation of human and machines in assembly lines[END_REF]. In this context, AnDy will establish a novel paradigm for ergonomic interventions through a combination of the four interrelated technologies (AnDySuit, AnDyDataset, AnDymodel and AnDyControl). With AnDy the workers working in human-unfriendly environments will get a valuable assistant that will make their job safe from the possible injuries. The solutions in AnDy also have high potential to become an indispensable diagnostic tool for specialists working in physical medicine and rehabilitation, occupational medicine, and others. A driving force behind the efforts in this scenario is the fact that musculoskeletal problems account for the largest part of work incapacities and that there is a clear demand for solutions that would mitigate these problems. Our project will utilize such sensory solutions (i) to develop models for studying loading of the human joints and then (ii) to leverage exoskeleton control principles and create solutions that will allow a natural merge of exoskeleton technology with the vast spectrum of available sensory solutions. Moreover, the control strategy developed in AnDy for the exoskeletal system will smooth the breakthrough of human-exoskeletal interaction in the field of wearable/assistive robots that support workers in a variety of tasks. Our goal is to improve working ergonomics with prevention of muscle skeletal disease, which will have a direct influence on improving the quality of manufacturing and related processes with special attention given to the ageing workforce. To make sure that the innovation potential of this scenario is as high as possible, useracceptance and cost-effectiveness will be also addressed.

Scenario 3 (humanoid robots)

In the human-humanoid interaction experiments with the iCub performing collaborative assembly with 56 adults, IVALDI et al. ( 2017) gained a lot of knowledge about the dynamics of social signals and physical signals (e.g., contact forces). However, in these experiments switching control strategies to adapt the compliance were controlled by the human and there was no prediction of human intent. On the other hand, in the CoDyCo project there have been numerous advancements in the whole-body interaction of the iCub with its environment [START_REF] Nori | iCub Whole-body Control through Force Regulation on Rigid Noncoplanar Contacts[END_REF], also with humans perturbing the robot equilibrium. There is a need to make the two research streams converge for the case of collaboration. This convergence should happen at several levels: first at the control level, as the robot should be able to cope with the contact forces; second, at the prediction level, as the robot should extract information from the haptic exchange. The first problem is about how to switch between controllers where contact forces must be rejected or followed, which are about functional roles. Traditionally, many works adopted a conservative approach of assigning the robot a passive follower role, e.g. implemented with a low-level impedance control loop in pHRI. Based on fixed roles (master/slave, leader/follower), the robot can easily estimate the human intention and improve the collaboration (GROTEN et al., 2010). Extracting information from haptic signals has been addressed in several papers using local force/torque sensing (BUSSY et al., 2012;BERGER et al., 2013); however, DUMORA et al. ( 2012) noted that wrench measurements provide incomplete information to detect the operator's intent of motion, which was confirmed by REED (2012). Haptic feedback can temporarily be sufficient to achieve subtasks, if contextual information about the task phase is provided. However, more non-verbal cues, e.g., gaze, may be necessary to recognize the partner intent during collaborative tasks and synchronize the dyadic activity [START_REF] Ivaldi | Robot initiative in a team learning task increases the rhythm of interaction but not the perceived engagement[END_REF]. Humanoid robots such as the iCub are well poised to investigate these issues.

In AnDy, we want to exploit the humanoid to push further the complexity of the collaboration. First, the humanoid will be balancing on its feet, therefore it will have to carefully control the exchanged forces and anticipate the human forces to prevent falling; there will be a bilateral control action in place, and the robot will need to balance the control strategy between leader and follower, predicting the human action. Finally, the human and the humanoid will have to realize a complex assembly action, for example a collaborative carrying or a snap-fit. As in the other scenarios, the AnD-ySuit provides the human posture and dynamics, AnDyModel provides models of human motion: the AnDyControl here is pushed to a bigger level of complexity. From the robot control point of view, this task explores the trade-off between physical interaction and postural tasks when the robot balance is at stake. Control strategies mixing soft and hard priorities will be considered and combined with suitable learning strategies to cope with uncertainties, following previous results [START_REF] Nava | Momentum Control of Humanoid Robots with Series Elastic Actuators[END_REF][START_REF] Modugno | Learning soft task priorities for safe control of humanoid robots with constrained stochastic optimization[END_REF]. Furthermore, this task allows studying whole-body control strategies for humanoid robots with several tasks and several constraints (some from the human, some from the humanoid, some originated by the closed-loop chain formed during their collaboration) in presence of partial and noisy observations of the state.

Conclusion

The H2020 European Project AnDy started in January 2017 and lasts 4 years. The goal of AnDy is to endow robots with the ability to control physical collaboration through intentional interaction. To achieve this goal, AnDy relies on three technological pillars: a wearable sensing device, called AnDySuit, which tracks the human mo-tions and forces; AnDyModel, that is ergonomic and human movement models that also provide prediction of movement in collaborative tasks; AnDyControl, which combines ergonomic models with cognitive predictive models of human dynamic behavior in collaborative tasks, learned from data acquired with the AnDySuit.

In this paper, we described the roadmap of the project, its motivation and its objectives. As described in the previous section, the research outlined in the roadmap builds on prior results from the different partners, such as the learning and control frameworks developed in the last years in the framework of the European Projects CoDyCo (https://www.codyco.eu/) and Spexor (http://www.spexor.eu/), for humanoid robots, cobots and exoskeletons.

.

Figure 1 :

 1 Figure 1: [Notice: this figure may be difficult to interpret if not printed/visualized in colors] A sketch of the roadmap of the AnDy project and its objectives.

Figure 2 :

 2 Figure 2: [Notice: this figure may be difficult to interpret if not printed/visualized in colors]. A sketch of the collaborative control objectives, with respect to the different levels of mutual-care and anticipation, used as metrics for collaboration.

Figure 3 :

 3 Figure 3: the three collaboration scenarios with the three different robot types: cobot or robotic manipulator, exoskeleton and humanoid robot.

This work is supported by the European Commission through the project H2020 AnDy (GA n.731540). More info at: http://andy-project.eu/