K R Jayaram
email: jayaramkr@hp.com

Elastic Remote Methods

Keywords: programmable elasticity, scalability, distributed objects

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Elasticity, the key driver of cloud computing, is the ability of a distributed application to dynamically increase or decrease its use of computing resources, to preserve its performance in response to varying workloads. Elasticity can either be explicit or implicit. Implicit vs. Explicit Elasticity. Implicit elasticity is typically associated with a specific programming framework or a Platform-as-a-Service (PaaS) cloud. Examples of frameworks providing implicit elasticity in the domain of "big data analytics" are map-reduce (Hadoop [13] and its PaaS counterpart Amazon Elastic Map Reduce [START_REF]Elastic Map Reduce[END_REF]), Apache Pig [START_REF]Apache Pig[END_REF], Giraph [START_REF] Giraph | [END_REF], etc. Implicit elasticity is handled by the PaaS implementation and is not the responsibility of the programmer. Despite being unable to support a wide variety of applications and computations, each of these systems simplifies application development and deployment, and employs distributed algorithms for elastic scaling that are optimized for its programming framework and application domain.

Explicit Elasticity, on the other hand, is typically associated with Infrastructureas-a-Service (IaaS) clouds and/or private data centers, which typically provide Thanks to Patrick Eugster and Hans Boehm for helpful feedback.

elasticity at the granularity of virtualized compute nodes (e.g., Amazon EC2) or virtualized storage (e.g., Amazon Elastic Block Store (EBS)) in a way that is agnostic of the application using these resources. It is the application developer's or the system administrator's responsibility to implement robust mechanisms to monitor the application's performance at runtime, request the addition or removal of resources, and perform load-balancing, i.e., redistribute the application's workload among the new set of resources. Programmable Elasticity. To optimize the performance of new or existing distributed applications while deploying or moving them to the cloud, engineering robust elasticity management components is essential. This is especially vital for applications that do not fit the programming model of (implicit) elastic frameworks like Hadoop, Pig, etc., but require high performance (high throughput and low latency), scalability and elasticity -the best example is the class of datacenter infrastructure applications like key-value stores (e.g., memcached [START_REF]Memcached[END_REF], Hyperdex [START_REF] Escriva | HyperDex: a Distributed, Searchable Key-Value Store[END_REF]), consensus protocols (e.g., Paxos [START_REF] Kirsch | Paxos for Systems Builders[END_REF]), distributed lock managers (e.g., Chubby [START_REF] Burrows | The Chubby Lock Service for Loosely-coupled Distributed Systems[END_REF]) and message queues. Elasticity frameworks which rely on externally observable resource utilization metrics (CPU, RAM, etc.) are insufficient for such applications (as we demonstrate empirically in Section 5). A distributed key value store, for example, may have high CPU utilization when there is high contention to update a certain set of "hot keys". Relying on CPU utilization to simply add additional compute nodes will only degrade its performance further. Hence, there is an emerging need for a elasticity framework that bridges the gap between implicit and explicit elasticity, allowing the use fine grained application specific metrics (e.g., size of a queue/heap, number of aborted transactions or average number of attempts to acquire certain locks) to build an elasticity management component right into the application without compromising (1) security, by revealing application-level information to the cloud service provider, and (2) portability across different cloud vendors. Why RMI? Despite being criticized for introducing direct dependencies across nodes through remote object references, Remote Method Invocation (RMI) and Remote Procedure Call (RPC) remain a popular paradigm [START_REF]Apache Thrift[END_REF] for distributed programming, because of their simplicity. Their popularity has led to the development of Apache Thrift [START_REF]Apache Thrift[END_REF] with support for RPC across different languages, and the design of cloud computing paradigms like RAMCloud [START_REF] Ousterhout | The Case for RAMCloud[END_REF] with support for low-latency RPC. However, high-level support for elasticity is limited in existing RPC-like frameworks; and such support is vital to engineering efficient distributed applications and migrating existing applications to the cloud. Contributions. This paper makes the following technical contributions:

1. ElasticRMI -A framework for elastic distributed objects in Java:

(a) with the same simplicity and ease of use of the Java RMI, handling elasticity at the level of classes and objects, while supporting implicit and explicit elasticity. (Section 2) (b) that enables application developers to dynamically change the number of (server) objects available to handle remote method invocations with tion of resource utilization metrics and fine-grained application-specific information to drive scaling decisions. (Section 3) 2. A runtime system that handles all the low-level mechanics of instantiating elastic objects, monitoring their workload, adding/removing additional objects as necessary and load balancing among them. (Section 4) 3. Performance evaluation of ElasticRMI using elasticity metrics defined by SPEC [START_REF]Report on Cloud Computing to the OSG Steering Committee[END_REF] and using four existing real world applications -Marketcetera financial order processing, Paxos consenus protocol, Hedwig publish/subscribe system and a distributed coordination service. (Section 5)

ElasticRMI -Overview

Figure 1 illustrates the architecture of ElasticRMI. In designing ElasticRMI, our goals are (1) to retain the simple programming model of Java RMI, and mask the low level details of implementing elasticity like workload monitoring, load balancing, and adding/removing objects from the application developer. (2) require minimal changes, if any, to the clients of an object (3) make ElasticRMI applications as portable as possible across different IaaS cloud implementations.

Elastic Classes and Object Pools

An ElasticRMI application consists of multiple components (implemented as classes) interacting with each other. The basic elasticity abstraction in Elas-ticRMI is an elastic class. An elastic class is also a remote class, and (some) of its methods may be invoked remotely from another JVM. The key difference between an elastic and a regular remote class is that an elastic class is instantiated into a pool of objects (referred to as elastic object pool), with each object executing on a separate JVM. But, the presence of multiple objects in an elastic object pool is transparent to its clients, because the pool behaves as a single remote object. Clients can only interact with the entire object pool, by invoking its remote methods. The interaction between a client and an elastic object pool is unicast interaction, similar to Java RMI. The processing of the method invocation, i.e., the method execution happens at a single object in the elastic object pool chosen by the ElasticRMI runtime, and not the client. ElasticRMIs runtime can redirect incoming method invocations to one of the objects in the pool, depending on various factors and performance metrics (Section 4 describes load balancing in detail). The runtime automatically changes the size of the pool depending on the pool's "workload". ElasticRMI is different from certain frameworks where the same method invocation is multicast and consequently executes on multiple (replicated) objects for fault-tolerance.

Shared State and Consistency

In Java RMI, the state of a remote (server) object is a simple concept, because it resides on a single JVM and there is exactly one copy of all its fields. Clients of the remote object can set the value of instance fields by calling a remote method, and access the values in subsequent method calls. In ElasticRMI, on the other hand, the entire object pool should appear to the client as a single remote object -thereby necessitating coordination between the objects in the pool to consistently update the values of instance fields. For consistency, we employ an external in-memory key-value store (HyperDex [START_REF] Escriva | HyperDex: a Distributed, Searchable Key-Value Store[END_REF] in our implementation) to store the state (i.e., public, private, protected and static fields) of the elastic remote object pool. The key-value store is not used to store local variables in methods/blocks of code and parameters in method declarations. Local variables are instantiated on the JVM in which the object resides. The key-value store is shared between the objects in the pool, and executes on separate JVMs.

Stubs and Skeletons

ElasticRMI modifies the standard mechanisms used to implement RPCs and Java RMI, as illustrated by Figure 1. The ElasticRMI pre-processor analyzes elastic classes to generate stubs and skeletons for client-server communication. As in Java RMI, a stub for a remote object acts as a client's local representative or proxy for the remote object. The caller invokes a method on ElasticRMI's local stub which (1) initiates a connection with the remote JVM, serializes and marshals parameters, waits for the result of the method invocation and unmarshals the return value/exception before returning it to the sender, and (2) performs load balancing among the objects in the elastic pool as necessary. The stub is generated by the ElasticRMI preprocessor and is different from the client application, to which the entire object pool appears as a single object, i.e., the existence of a pool of objects is known to the stub but not to the client application. In the remote JVM, each object in the pool has a corresponding skeleton, which in addition to the duties performed in regular Java RMI, can also perform dynamic load balancing based on the CPU utilization of the object and redirect all further method invocations to other objects in the pool after ElasticRMI decides to shut it down in response to decreasing workload.

Instantiation of Object Pools in a Cluster

An elastic class can only be instantiated by providing a minimum and maximum number of objects that constitute its elastic object pool. Obviously, instantiating all objects in the pool on separate JVMs on the same physical machine may degrade performance. Hence, ElasticRMI attempts to instantiate each object in a virtual node in a compute cluster. Virtual nodes can be obtained either (1) from IaaS clouds by provisioning and instantiating virtual machines, or (2) from a cluster management/resource sharing system like Apache Mesos [START_REF] Hindman | Mesos: A Platform for Fine-grained Resource Sharing in the Data Center[END_REF]. Our implementation of ElasticRMI uses Apache Mesos [START_REF] Hindman | Mesos: A Platform for Fine-grained Resource Sharing in the Data Center[END_REF] because it supports both clusters of physical nodes (in private data centers) or virtual nodes (from IaaS clouds). Mesos can also be viewed as a thin resource-sharing layer that manages a cluster of physical nodes/virtual machines. It divides these nodes into "slices" (called resource offers or slave nodes [START_REF] Hindman | Mesos: A Platform for Fine-grained Resource Sharing in the Data Center[END_REF]), with each resource offer containing a configurable reservation of CPU power (e.g., 2 CPUs at 2GHz), memory (e.g., 2GB RAM), etc. on one of the nodes being managed. Mesos implements the "slice" abstraction by using Linux Containers (http://lxc.sourceforge.net) to implement lightweight virtualization, process isolation and resource guarantees (e.g., 2CPUs at 2GHz) [START_REF] Hindman | Mesos: A Platform for Fine-grained Resource Sharing in the Data Center[END_REF]. While instantiating an elastic class, the ElasticRMI runtime requests Mesos for a specified number of slave nodes, instantiating an object on each slave node.

Mesos aids in portability of ElasticRMI applications, just like the JVM aids the portability of Java applications. Mesos can be installed on private data centers and many public cloud offerings (like Google Compute Engine, Amazon EC2, etc.). As long as Mesos is available, ElasticRMI applications can be executed, making them portable across different cloud vendors.

Automatic Elastic Scaling

The key objective of ElasticRMI is to change the number of objects in the elastic pool based on its workload. The "workload" of an elastic object can have several application-specific definitions, and consequently, ElasticRMI allows programmers to define the workload of an elastic class and specify the conditions under which objects should be added or removed from the elastic pool. This can be done by overriding select methods in the ElasticRMI framework, as discussed in detail in (the next) Section 3.1. During the lifetime of the elastic object, the ElasticRMI runtime monitors a elastic object pool's workload and decides whether to change its size either based on default heuristics or by applying the programmer's logic by invoking the overridden methods discussed above.

If a decision has been made to increase the size of the pool, the Elasti-cRMI runtime interacts with the Mesos master node to request additional compute resources. If the request is granted, ElasticRMI runtime instantiates the additional object, and adds it to the pool (See Figure 2 for an illustration). If the decision is to remove an object, ElasticRMI communicates with its skeleton to redirect subsequent remote method calls to other objects in the pool. Once redirection starts, ElasticRMI sends a shutdown message to the object. The object acknowledges the message, and waits for all pending remote method invocations to finish execution or throw exceptions indicating abnormal termination. Then the object notifies the ElasticRMI runtime that it is ready to be shutdown. ElasticRMI terminates the object and relinquishes its slice to Mesos. This slice is then available to other elastic objects in the cluster, or for subsequent use by the same elastic object if a decision is made in the future to increase the size of its pool.

Programming With ElasticRMI

This section illustrates the use of ElasticRMI for both implicit and explicit elasticity through examples, along with an overview of how to make such decisions with a global view of the entire application. Our implementation also includes a preprocessor similar to rmic which in addition to generating stubs and skeletons, converts ElasticRMI programs into plain Java programs that can be compiled with the javac compiler.

ElasticRMI Class Hierarchy

A distributed application built using ElasticRMI consists of interfaces declaring methods and classes implementing them. The key features of ElasticRMI API (Figure 3) are:

java.elasticrmi is the top-level package for programming ElasticRMI server classes.

Fig. 4: Example of two distributed cache classes implemented in ElasticRMI

-An elastic interface is one that declares the set of methods that can be invoked from a remote JVM (client). All elastic interfaces must extend Elasti-cRMI's marker interfacejava.elasticrmi.Elastic, either directly or indirectly. -The ElasticObject class implements all the basic functionalities of Elasti-cRMI. An application-defined class becomes elastic by implementing one or more elastic interfaces and by extending ElasticObject.

Programming With Implicit Elasticity

ElasticRMI supports implicit elastic scaling, using average CPU utilization across the objects in an elastic pool as the default coarse-grained metric. A time interval, referred to as the burst interval (default 60s) is used to decide whether to change the size of elastic object pool. ElasticRMI measures the average CPU utilization of each object in the elastic pool every 60s -objects are added (in increments of 1 object) when the average utilization exceeds 90% and removed when average utilization falls below 60%. Figure 4a shows an example of a distributed cache class (e.g., a web cache, content/object cache) that relies on ElasticRMI's implicit elasticity mechanisms. The programmer simply implements a cache store based on some well-known algorithm and specifying the minimum and maximum size of the pool, without worrying about adapting to new resources and load balancing.

Programming With Explicit Elasticity

ElasticRMI also allows programmers to explicitly define the workload of an elastic class and specify the conditions under which objects should be added or removed from the elastic pool. Workload definitions can either be coarse-grained or fine-grained.

Coarse-grained Metrics. A programmer can override the default burst interval, and the average CPU utilization thresholds used for changing the number of objects in the pool by calling the appropriate methods (setBurstInterval(...), setCPUIncrThreshold(...) and setCPUDecrThreshold(...)) in java.elasticrmi.ElasticObject which is available in all elastic classes since they extend ElasticObject (see Figure 3). Figure 4b shows an example of a cache class that changes the CPU and memory (RAM) utilization thresholds that trigger elastic scaling. The core Elasti-cRMI API includes specific methods to set CPU and memory thresholds because they are commonly used for elastic scaling -if both CPU and RAM thresholds are set, the runtime interprets them using a logical OR, i.e., in the example shown in Figure 4b, the ElasticRMI runtime increases the size of the pool by in increments of 1 object every five minutes, either if average CPU utilization exceeds 85% or if average memory utilization exceeds 70% across the JVMs in the elastic object pool.

Fine-grained Metrics. ElasticRMI provides additional support, through the changePoolSize method (see Figure 3) which can be overridden by any elastic class. The runtime periodically (every "burst interval") invokes changePoolSize to poll each object in the elastic object pool, about desired changes to the size of the pool. The method returns an integer -positive or negative corresponding to increasing or decreasing the pool's size. The values returned by the various objects in the pool are averaged to determine the number of objects that have to be added/removed. The logic used to decide on elastic scaling is left to the developer, and it may be based on (1) parameters of the JVM on which each object resides, (2) properties of shared instance fields of the elastic object, or of data structures used by the object, e.g., number of pending client operations stored in a queue, and (3) metrics computed by the object like average response time, throughput, etc. ElasticRMI allows classes to use only a single decision mechanism for elastic scaling, i.e., if changePoolSize is overridden, then scaling based on CPU/Memory utilization is disabled. float putLatency = sMap . get ( " put " ) . getLatency () ; float getLatency = sMap . get ( " get " ) . getLatency () ; if ( putLatency > 100 || putLatency > 3* getLatency ) { if ( a v g L o c k A c q F a i l u r e > 50) return 0; if ( a v g L o c k A c q L a t e n c y >= 0.8* putLatency ) return 0; else return 2; } } } Fig. 5: A distributed cache class which relies on ElasticRMI's explicit elasticity support using fine-grained application-specific metrics.

Figure 5 illustrates the use of changePoolSize to make scaling decisions. The CacheExplicit2 class is implemented to use metrics specific to distributed object caches, e.g., avgLockAcqFailure (which measures the failure rate of acquiring write locks to ensure consistency during a put operation on the cache) and avgLockAcqLatency (which measures the average latency to acquire write locks) to make decisions about changing the size of the elastic object pool. In Figure 5, the CacheExplicit2 class does not add new objects to the pool when there is a lot of contention. When the failure rate for acquiring write locks (avgLockAcqFailure is greater than 50%) or when the predominant component of putLatency is avgLockAcqLatency, no additional objects are added to the pool because there is already high contention among objects serving client requests to acquire write locks. When these conditions are false, the size of the pool is increased by two -controlling the number of objects added is another feature of changePoolSize.

Making Application-Level Scaling Decisions. The mechanisms described above involve makes scaling decisions local to an elastic class, and may not be optimal for applications using multiple elastic classes (where the application contains tiers of elastic pools). ElasticRMI also supports decision making at the level of the application using the Decider class. It is the developer's responsibility to ensure that elastic objects being monitored communicate with the monitoring components, either by using remote method invocations or through message passing. The ElasticRMI runtime assumes responsibility for calling changePoolSize method of the monitoring class to get the desired size of each elastic object pool, and determines whether objects have to be added or removed. Due to space limitations, we refer the reader to our tech report for additional details [START_REF] Jayaram | Elastic Remote Methods[END_REF].

The ElasticRMI Runtime

The runtime (1) handles shared state among the objects in an elastic object pool, (2) instantiates each object of a pool, (3) performs load balancing, and ( 4) is responsible for fault-tolerance.

Shared State and Consistency

The objects in the elastic object pool coordinate to update the state of its instance fields (public, private, protected) and static fields. For consistency, we use HyperDex [START_REF] Escriva | HyperDex: a Distributed, Searchable Key-Value Store[END_REF], a distributed in-memory key-value store (with strong consistency). Using an in-memory store provides the same data durability guarantees as Java RMI which stores the state of instance fields in RAM in a single Java virtual machine heap. HyperDex is different from the distributed cache in the examples of Section 3, which is used for illustration purposes only. The Elasti-cRMI preprocessor translates reads and writes of instance and static fields into get(...) and put(...) method calls of HyperDex. Figure 6 shows a simple elastic class C1 and how it is transformed by the Elas-ticRMI preprocessor to insert calls to Hy-perDex (abstracted by Store) and ElasticRMI's runtime (abstracted by ERMI). For variable x, C1$x is used as the key in Store. For synchronized methods, Elas-ticRMI uses a lock per class named after the class -the example in Figure 6 uses a lock called "C1".

If an elastic class has an instance or static field f , a method call on f is handled as follows:

-If f is a remote or elastic object, the method invocation is simply serialized and dispatched to the remote object or pool as the case may be. -Else, if m is synchronized. the method call is handled as in Figure 6, but the runtime acquires a lock on f through HyperDex. In this case, ElasticRMI guarantees mutual exclusion for the execution of f.m(...) with respect to other methods of f . -Else, (i.e., m is neither remote, elastic nor synchronized), f.m(...) involves retrieving f from HyperDex and executing m(...) locally on an object in the elastic object pool, and storing f back into HyperDex after m(...) has completed executing.

ElasticRMI aims to increase parallelism and hence the number of remote method executions per second when there is limited or no shared state. Increasing shared state increases latency due to the network delays involved in accessing HyperDex. Having shared state and mutual exclusion through locks or synchronized methods further decreases parallelism. However, we note that this is not a consequence of ElasticRMI, but rather dependent on the needs of the distributed application. If the developer manually implements all aspects of elasticity by using plan Java RMI and an existing tool like Amazon Cloud-Watch+Autoscaling, he still has to use something like a key-value store to handle shared state. ElasticRMI cannot and does not attempt to eliminate this problem -it is up to the programmer to reduce shared state.

Note that ElasticRMI does not guarantee a transactional (ACID) execution of m(...) with respect to other objects in the pool, and using synchronized does not provide ACID guarantees either in RMI or in ElasticRMI.

Instantiation of Elastic Objects

An elastic class can only be instantiated by providing a minimum (≥ 2) and maximum number of objects that constitute its elastic object pool (see Figure 3). During instantiation, if the minimum number of objects is k, ElasticRMI's runtime creates k objects on k new JVM instances on k virtual nodes (Mesos slices), if k virtual nodes are available from Mesos [START_REF] Hindman | Mesos: A Platform for Fine-grained Resource Sharing in the Data Center[END_REF]. If only l < k are available, then only l objects are created. Under no circumstance does ElasticRMI create two or more JVM instances on the same slice obtained from Mesos. Then, Elasti-cRMI instantiates the HyperDex on one additional Mesos slice, and continues to monitor the performance of the HyperDex over the lifetime of the elastic object. ElasticRMI may add additional nodes to HyperDex as necessary. Elasti-cRMI also enables administrators to be notified if the utilization of the Mesos cluster exceeds or falls below (configurable) thresholds, enabling the proactive addition of computing resources before the cluster runs out of slices.

Load Balancing

Unlike websites or web services, where load balancing has to be performed on the server-side, ElasticRMI has the advantage that both client and server programs are pre-processed to generate stubs and skeletons respectively. Hence, we employ a hybrid load balancing model involving both stubs and skeletons -note that all load balancing code is generated by the pre-processor, and the programmer does not have to handle any aspect of it explicitly. Please also note that this section describes the simple load balancing techniques used in ElasticRMI, but we do not claim to have invented a new load balancing algorithm.

On the server side, the runtime, while instantiating skeletons in an elastic object pool, assigns monotonically increasing unique identifiers (uid) to each skeleton, and stores this information in HyperDex. The skeleton with the lowest uid is chosen by the runtime to be the leader of the elastic object pool, called the sentinel. This is similar to leader election algorithms that use a so-called "royal hierarchy" among processes in a distributed system. The sentinel, in addition to performing all the regular functions (forwarding remote method invocations) to its object in the pool, also helps in load balancing. The client stub created by the ElasticRMI preprocessor (see Section 2.4) has the ability to communicate with the sentinel to invoke remote methods. While contacting the sentinel for the first time, the stub on the client JVM requests the identities (IP address and port number) of the other skeletons in the pool from the sentinel.

For load-balancing on the client-side, the stub then re-directs subsequent method invocations to other objects in the object pool either randomly or in a round-robin fashion. If an object has been removed from the pool after its identity is sent to a stub, i.e., if the sending itself fails, the remote method invocation throws an exception which is intercepted by the client stub. The stub then retries the invocation on other objects including the sentinel. If all attempts to communicate with the elastic object pool fail, the exception is propagated to the client application.

For load-balancing on the server side, the sentinel is also responsible for collecting and periodically broadcasting the state of the pool -number of objects, their identities and the number of pending invocations -to the skeletons of all its members. We use the JGroups group communication system for broadcasts. If the sentinel notices that any skeleton is overloaded with respect to others, it instructs the skeleton to redirect a portion of invocations to a set of other skeletons. To decide the number of invocations that have to be redirected from each overloaded skeleton, our implementation of the sentinel uses the first-fit greedy bin-packing approximation algorithm (See http://en.wikipedia.org/wiki/ Bin packing problem). As mentioned in the previous paragraph, client-side load balancing occurs at the stub while server-side load balancing involves skeletons and the sentinel which monitor the state of the JVM and that of the elastic object pool to redirect incoming method invocations.

Fault Tolerance

Existing RMI applications implement fault tolerance protocols on top of Java RMI's fault-and fault tolerance model, where objects typically reside in main memory, and can crash in the middle of a remote method invocation. !e want to preserve it to make adoption of ElasticRMI easier. In short, ElasticRMI does not hide/attempt to recover from failures of client objects, key-value store (HyperDex) or the server-side runtime processes and propagates corresponding exceptions to the application. However, ElasticRMI attempts to recover from failures of the sentinel and from Mesos-related failures. Sentinel failure triggers the leader election algorithm described in 4.3 to elect a new sentinel, and mesosrelated failures affect the addition/removal of new objects until Mesos recovers.

Evaluation

In this section, we evaluate the performance of ElasticRMI, using metrics relevant to elasticity. Due to space limitations, we refer the reader to our tech report for additional details [START_REF] Jayaram | Elastic Remote Methods[END_REF]

Elasticity Metrics

Measuring elasticity is different from measuring scalability. (Recall that) Scalability is the ability of a distributed application to increase its "performance" proportionally (ideally linearly) with respect to the number of available resources, while elasticity is the ability of the application to adapt to increasing or decreasing workload; adding or removing resources to maintain a specific level of "performance" or "quality of service (QoS)" [START_REF]Report on Cloud Computing to the OSG Steering Committee[END_REF]. Performance/QoS is specific to the application -typically a combination of throughput and latency. A highly elastic system can scale to include newer compute nodes, as well as quickly provision those nodes. There are no standard benchmarks for elasticity, but the Standard Performance Evaluation Corporation (SPEC) has recommended elasticity metrics for IaaS and PaaS clouds [START_REF]Report on Cloud Computing to the OSG Steering Committee[END_REF].

Agility. This metric characterizes the ability of a system provisioned to be as close to the needs of the workload as possible [START_REF]Report on Cloud Computing to the OSG Steering Committee[END_REF]. Assuming a time interval [t, t ], which is divided into N sub-intervals, Agility maintained over [t, t ] can be defined as:

1

N ( N i=0
Excess(i)

+ N i=0 Shortage(i))
where (1)Excess(i) is the excess capacity for interval i as determined by Cap prov(i)-Req min(i), when Cap prov(i) > Req min(i) and zero otherwise. ( 2) Shortage(i) is the shortage capacity for interval i as determined by Req min(i)-Cap prov(i), when Cap prov(i) < Req min(i) and zero otherwise. (3) Req min(i) is the minimum capacity needed to meet an application's quality of service (QoS) at a given workload level for an interval i. (4) Cap prov(i) is the recorded capacity provisioned for interval i, and ( 5) N is the total number of data samples collected over a measurement period [t, t ], i.e., one sample of both Excess(i) and

Shortage(i) is collected per sub-interval of [t, t ].
Elasticity measures the shortage and excess of computing resources over a time period. For example, a value of elasticity of 2 over [t, t ] when there is no excess means that there is a mean shortage of 2 "compute nodes" over [t, t ]. For an ideal system, agility should be as close to zero as possible -meaning that there is neither a shortage nor excess. Agility is a measurement of the ability to scale up and down while maintaining a specified QoS. The above definition of agility will not be valid in a context where the QoS is not met. It should be noted that there is ongoing debate over whether Shortage and Excess should be given equal weightage [START_REF]Report on Cloud Computing to the OSG Steering Committee[END_REF] in the Agility metric, but there are disagreements over what the weights should be otherwise.

Provisioning Interval. Provisioning Interval is defined as the time needed to bring up or drop a resource. This is the time between initiating the request to bring up a new resource, and when the resource serves the first request.

ElasticRMI Applications for Evaluation and Workloads

We have re-implemented four existing applications using ElasticRMI to add elasticity management components to them. This does not involve altering the percentage of shared state or the frequency of accesses (reads or writes) to said state. Marketcetera [START_REF] Miller | Marketcetera Automated Trading Platform[END_REF] Order Routing. Marketcetera is an NYSE-recommended algorithmic trading platform. The order routing system is the component that accepts orders from traders/automated strategy engines and routes them to various markets (stock/commodity), brokers and other financial intermediaries. For fault-tolerance, the order is persisted (stored) on two nodes. The workload for this system is a set of trading orders generated by the simulator included in the community edition of Marketcetera [START_REF] Miller | Marketcetera Automated Trading Platform[END_REF].

Apache Hedwig [START_REF][END_REF]. Hedwig is a topic-based publish-subscribe system designed for reliable and guaranteed at-most once delivery of messages from publishers to subscribers. Clients are associated with (publish to and subscribe from) a Hedwig instance (also referred to as a region), which consists of a number of servers called hubs. The hubs partition the topic ownership among themselves, and all publishes and subscribes to a topic must be done to its owning hub. The workload for this system is a set of messages generated by the default Hedwig benchmark included in the implementation.

Paxos [START_REF] Kirsch | Paxos for Systems Builders[END_REF]. Paxos is a family of protocols for solving consensus in a distributed system of unreliable processes. Consensus protocols are the basis for the state machine approach to distributed computing, and for our experiments we implement Paxos using a widely-used specification by Kirsch and Amir [START_REF] Kirsch | Paxos for Systems Builders[END_REF]. The workload for this system is the default benchmark included in libPaxos [START_REF] Libpaxos | [END_REF].

DCS. DCS is a distributed co-ordination service for datacenter applications, similar to Chubby [START_REF] Burrows | The Chubby Lock Service for Loosely-coupled Distributed Systems[END_REF] and Apache Zookeeper [START_REF] Zookeeper | [END_REF]. DCS has a hierarchical name space which can be used for distributed configuration and synchronization. Updates are totally ordered. The workload for this system is the default benchmark included in Apache Zookeeper [START_REF] Zookeeper | [END_REF].

Workload Pattern

To measure how well the system adapts to the changing workload, we use two patterns shown in Figures 7a and7b. These two patterns capture all common scenarios in elastic scaling which we have observed by analyzing real world applications. The abrupt pattern shown in Figure 7a has all possible scenarios regarding abrupt changes in workload -gradual non-cyclic increase, gradual decrease, rapid increases and rapid decrease in workload. A cyclic change in workload is shown by the second pattern in Figure 7b. So, together the patterns in Figures 7a and7b exhaustively cover all elastic scaling scenarios we observed. Note however, that although the pattern remains the same for varying the workload while evaluating all the four systems, the magnitude differs depending on the benchmark used, i.e., the values of points A and B in Figures 7a and7b are different for the four systems depending on the benchmark. Point A, for example, is 50,000 orders/s for Marketcetera, 75,000 updates/s for DCS, 24,000 consensus rounds/s for Paxos and 30,000 messages/s for Hedwig. We set Point B at 20% above Point A -note that the specific values of Points A and B are immaterial because we are only measuring adaptability and not peak performance.

Overprovisioning and CloudWatch

We compare the ElasticRMI implementation of the applications in Section 5.2 with the existing implementations of the same applications in two deployment scenarios -(1) Overprovisioning and (2) Amazon AutoScaling + CloudWatch [START_REF]Amazon Auto Scaling[END_REF][1].

The overprovisioning deployment scenario is similar to an "oracle" -the peak workload arrival rate i.e., point A for the abruptly changing workload and point B for the cyclic workload are known a priori to the oracle; and the number of nodes required to meet a desired QoS (throughput, latency) at A and B respectively is determined by the oracle through experimental evaluation. The oracle then provisions the application on a fixed set of nodes -the size of which is enough to maintain the desired QoS even at the peak workload arrival rate (A and B respectively). In a nutshell, the over provisioning scenario can be described as "knowing future workload patterns and provisioning enough resources to meet its demands". Overprovisioning is the alternative to elastic scaling -there are going to be excess provisioned resources when the workload is below the peak (A and B), but provisioning latency is zero because all necessary resources are always provisioned. In the CloudWatch scenario, we use a monitoring service -Amazon CloudWatch to collect utilization metrics (CPU/Memory) from the nodes in the cluster and use conditions on these metrics to decide whether to increase or decrease the number of nodes. The ElasticRMI implementation of the above applications, however, uses a combination of resource utilization and application-level properties specific to Marketcetera, DCS, Paxos and Hedwig respectively to decide on elastic scaling. Since ElasticRMI and CloudWatch are two different systems, we also compare the ElasticRMI implementation of the four applications with another version, referred to as ElasticRMI-CPUMem in Figure 7, where no application-level properties are used but only the the conditions based on CPU/Memory utilization in CloudWatch are used. Fig. 8: Provisioning latency in seconds for ElasticRMI and Overprovisioning (which is always 0). Provisioning latency for Amazon CloudWatch is not plotted because it is in minutes and hence well above that of both ElasticRMI and Overprovisioning. You can see repeating patterns corresponding to the cyclic workload.

Agility Results

In this section, we compare the Agility of the ElasticRMI implementation of all four systems against Overprovisioning, CloudWatch and ElasticRMI-CPUMem.

Marketcetera Order Routing. The relevant QoS metrics for the order routing subsystem are order routing throughput, which is the number of orders routed from traders to brokers/exchanges per second and order propagation latency, which is the time taken for an order to propagate from the sender to the receiver. We compare the elasticity of the three deployments of the order routing system described in Section 5.4. The results are illustrated in Figure 7. Figures 7c and7d plot the agility over the same time period as in Figures 7a and7b for all the four deployments. From Figures 7c and7d, we observe that the agility of ElasticRMI is better than CloudWatch, ElasticRMI-CPUMem and overprovisioning. Ideally, agility must be zero, because agility is essentially a combination of resource wastage or resource under-provisioning. We observe that for abruptly changing workloads, agility of ElasticRMI is close to 1 most of the time, and increases to 5 during abrupt changes in workload. We also observe that the the agility of ElasticRMI oscillates between 0 and a positive value frequently. This proves that the elastic scaling mechanisms of ElasticRMI perform well in trying to achieve optimal resource utilization, i.e., react aggressively by trying to push agility to zero. In summary, the average agility of ElasticRMI for abruptly changing workload is 1.37. As expected, the agility of overprovisioning is the worst, up to 24× that of ElasticRMI. This is not surprising, because its agility does reach zero at peak workload, when the agility of ElasticRMI is 5, thus illustrating that overprovisioning optimizes for peak workloads. The average agility of overprovisioning is 17.2 for the cyclical workload and 24.1 for abruptly changing workload. CloudWatch performs much better than overprovisioning, but it is less agile than ElasticRMI. Its agility is approximately 3.4× that of ElasticRMI on average for abruptly changing workloads, and it does not oscillate to zero frequently like ElasticRMI. The agility of ElasticRMI-CPUMem is approximately equal to CloudWatch and is 3.02× that of ElasticRMI on average -this is in spite of ElasticRMI-CPUMem and CloudWatch being different systems. having different provisioning latencies. This is because the same conditions are used to decide on elastic scaling and because the provisioning latency of CloudWatch is well within the sampling interval of 10 minutes used in Figure 7.

Figure 7d shows that the agility of ElasticRMI is again better than that of CloudWatch and overprovisioning for cyclic workloads. We also observe that as in the case of abrupt workloads, the agility of ElasticRMI tends to decrease to zero more frequently than the other two deployments. Figure 7d also demonstrates the oscillating pattern in the agility of overprovisioning -the initial agility is high (and comes from Excess), and as the workload increases, Excess decreases, thereby decreasing Agility and bringing it to zero corresponding to Point B. Then Excess increases again as the workload decreases thereby increas-ing Agility. This repeats three times. As expected, the agility of ElasticRMI-CPUMem is similar to that of CloudWatch.

Hedwig. The relevant QoS metrics for Hedwig are also throughput and latency -the number of messages published per second and time taken for the message to propagate from the publisher to the subscriber. Figures 7e and7f illustrate the agility corresponding to our experiments with Hedwig. From Figures 7e and7f, we observe similar trends as in the case of Marketcetera order processing. ElasticRMI has lower agility values than the other two deployments, and the agility of ElasticRMI tends to oscillate between zero and a positive value. The agility values of CloudWatch are more than 4.5× that of ElasticRMI, on average for abrupt workloads and 3× that of ElasticRMI for cyclic workloads. As expected, the agility of over provisioning is the highest, and is worse than the values observed for Marketcetera in the case of cyclic workloads. We also observe a similar oscillating trend in the agility values of the overprovisioning deployment as in Marketcetera, but the agility values oscillate more frequently because Req min(i) -the minimum capacity needed to maintain QoS under a certain workload changes more erratically than Marketcetera due to the replication and at-most once guarantees provided by Hedwig for delivered messages.

Paxos The relevant QoS metrics for Paxos are the number of consensus rounds executed successfully per second, and the time taken to execute a consensus round. Figures 7g and7h illustrate the agility corresponding to our experiments with Paxos. From Figures 7g and7h, we observe similar trends to Hedwig and Marketcetera. The agility of CloudWatch in this case is 6.6× than of Elas-ticRMI, on average for abrupt workloads and 2.2× that of ElasticRMI for cyclic workloads. We also observe that the agility of ElasticRMI returns to zero (the ideal agility) most frequently among the three deployments.

DCS The relevant QoS metrics for DCS are the number of updates to the hierarchical name-space per second and the end-to-end latency to perform an update as measured from the client. Figures 7i and 7j illustrate the agility corresponding to our experiments with DCS. From Figures 7i and7j, we observe that the agility of CloudWatch in this case is 7.2× than of ElasticRMI, on average for abrupt workloads and 3.2× that of ElasticRMI for cyclic workloads.

Provisioning Latency

Figures 8a and 8b plot the provisioning latency of ElasticRMIfor both abrupt and cyclic workloads. We observe that the provisioning latency of ElasticRMI is less than 30 seconds in all cases, which compares very favorably to the time needed to provision new VM instances in Amazon CloudWatch (which is in the order of several minutes, and hence omitted from Figure 8). Provisioning latency is zero for the overprovisioning scenario, and that is the main purpose of overprovisioning -to have resources always ready and available. Also, we observe that as the workload increases, provisioning interval also increases, due to the overhead in determining the remote method calls that have to be redirected and also due to increasing demands on the resources of the sentinel object in ElasticRMI's object pools.

Related Work

J-Orchestra [START_REF] Tilevich | J-Orchestra: Automatic Java Application Partitioning[END_REF][START_REF] Tilevich | Portable and Efcient Distributed Threads for Java[END_REF] automatically partitions Java applications and makes them into distributed applications running on distinct JVMs, by using byte code transformations to change local method calls into distributed method calls as necessary. The key distinctions between J-Orchestra and ElasticRMI are that (1) J-Orchestra tackles the complex problem of automatic distribution of Java programs while ElasticRMI aims to add elasticity to already distributed programs and (2) ElasticRMI partitions different invocations of a single remote method.

Self-Replicating Objects (SROs) [START_REF] Ostrowski | Self-replicating Objects for Multicore Platforms[END_REF] is a new elastic concurrent programming abstraction. An SRO is similar to an ordinary .NET object exposing an arbitrary API but exploits multicore CPUs by automatically partitioning its state into a set of replicas that can handle method calls (local and remote) in parallel, and merging replicas before processing calls that cannot execute in replicated state. SRO also does not require developers to explicitly protect access to shared data; the runtime makes all the decisions on synchronization, scheduling and splitting/merging state. Live Distributed Objects (LDO) [START_REF] Ostrowski | Programming with Live Distributed Objects[END_REF] is a new programming paradigm and a platform, in which instances of distributed protocols are modeled as live distributed objects. Live objects can be used to represent distributed multi-party protocols and application components. Shared-state and synchronization between the objects is maintained using Quicksilver [START_REF] Ostrowski | Quicksilver Scalable Multicast (QSM)[END_REF], a group communication system. Automatic scaling is not supported and must be explicitly implemented by the programmer using the abstractions provided by LDO.

Quality Objects (QuO) [START_REF]Quality Objects (QuO)[END_REF] is a seminal framework for providing quality of service (QoS) in network-centric distributed applications. When the requirements are not being met, QuO provides the ability to adapt at many levels in the system, including the middleware layer responsible for message transmission. In contrast to QuO, ElasticRMI attempts to increase quality of service by changing the size of the remote object pool, and does not change the protocols used to transmit remote method invocations.

Conclusions

We have described the design and implementation of ElasticRMI and have demonstrated through empirical evaluation using real-world applications that it is effective in engineering elastic distributed applications. Our empirical evaluation also demonstrates that relying solely on externally observable metrics like CPU/RAM/network utilization decreases elasticity, as demonstrated by the high agility values of CloudWatch. We have shown that our implementation of Elas-ticRMI reduces resource wastage, and is sufficiently agile to meet the demands of applications with dynamically varying workloads. Through an implementation using Apache Mesos, we ensure portability of ElasticRMI applications across Mesos installations, whether it is a private datacenter or a public cloud or a hybrid deployment between private data centers and public clouds. We have demonstrated that ElasticRMI applications can use fine-grained application specific metrics without revealing those metrics to the cloud infrastructure provider, unlike CloudWatch.

Fig. 1 :

 1 Fig. 1: ElasticRMI -Overview

Fig. 2 :

 2 Fig. 2: ElasticRMI -Server Side

Fig. 3 :

 3 Fig. 3: A snapshot of the ElasticRMI server-side API.

  public class CacheExp licit2 extends ElasticObject { float avgLockAcqFailure , a v g L o c k A c q L a t e n c y public int chan gePoolS ize () { HashMap < String , float > sMap ; sMap = g e t M e t h o d C a l l S t a t s () ;

Fig. 6 :

 6 Fig. 6: Handling shared state through an in-memory store (HyperDex [23] here).

  Pattern for abruptly changing workload for all four systems. The pattern remains the same, but the meaning and magnitude vary for the four systems. Cyclical workload example for all four systems. As in Figure7a, the pattern remains the same for all four systems but the meaning and magnitude are different.

Fig. 7 :

 7 Fig. 7: Elasticity Benchmarking of Marketcetera order routing, Hedwig, Paxos and DCS. We compare the ElasticRMI implementation of these applications with two other systems described in Section 5.4.

  Provisioning latency -Cyclic Workload


Franc ¸ois Pacull Maxime Louvel

Maria Isabel Vergara-Gallego

Coordination Scheme Editor for Building Management Systems

This paper presents our work on the Coordination Scheme Editor (CSE). This editor provides a web interface to design building automation rules. Rules are designed with a drag and drop programming. Then rules are validated, generated and activated in the buildings.

CSE allows to define template rules. The rules are then instantiated in building rooms with the available sensors and actuators. CSE also offers a monitoring interface to know which scenario is impacted by one or several faulty sensors/actuators.

The CSE relies on the LINC middleware to provide coordination across several building automation systems and sensors/actuators technologies.

I. INTRODUCTION

Building management systems (BMS) have traditionally been proprietary and closed systems. A BMS consists of hardware and software components; in general, software components are proprietary, based either on closed solutions or open industrial standards. Typical examples of standard solutions for BMS are LONworks, KNX, BacNet, and Modbus [START_REF] Kastner | Communication systems for building automation and control[END_REF]. When a building is built, the hardware components are installed and a software commissioning is done. Once again this operation is done using proprietary software and, in most of the cases, it requires the intervention of a skilled technician.

Buildings are usually built to last 20 years or more, and, during their lifetime, they have to evolve. Indeed, buildings are usually retrofitted every two or three years to add new equipment, new facilities (e.g. walls, building extension or new material), or simply to reconfigure a poorly functioning building. Another type of evolution is reconfiguration for users comfort or usage. For instance, a wall can be added/removed to create/remove rooms. After such modifications, the BMS must be reconfigured to adapt the building behaviour. To do so, a skilled technician has to come in the building and perform the reconfiguration of the BMS using a proprietary software.

The complexity increases when considering multiple technologies in one building. Today there is a trend to integrate open industry standards meaning that a unique building may integrate several technologies, or several BMS, that work independently and that have to be configured independently. Another trend is to install new devices, such as sensors, that are not originally provided by the BMS. For instance, indoor air quality sensors exist but they are not connected to existing BMS. Typically, when these sensors are installed data are collected by a dedicated system, an off-line analysis is done and guidelines are provided to the facility manager.

The facility manager then calls a technician to update the BMS accordingly. Another example is the emerging market of the Energy Sub-Contracting Companies (ESCOs) [START_REF] Stuart | The us esco industry: recent trends, current size and remaining market potential[END_REF]. ESCOs want to manage the building more efficiently to reduce the energy bill and take some money in the loop. To make their job more efficient they will want to install specific sensors that should interact with the BMS.

Several solutions have been proposed to connect multiple BMS and technologies through a software layer [START_REF] Bruckner | Latest trends in integrating building automation and smart grids[END_REF], [START_REF] Veichtlbauer | Generic middleware for userfriendly control systems in home and building automation[END_REF]. The work proposed in this paper goes further, by providing a graphical user interface that permits a user (e.g. the facility manager or the building occupants) to monitor the BMS and define the interactions between the connected devices. The use of such interface does not require any special skill. The proposed approach is called Coordination Scheme Editor (CSE). The CSE is a software component based on the LINC [START_REF] Louvel | Linc: A compact yet powerful coordination environment[END_REF] coordination middleware. LINC has already been used in building automation applications integrating heterogeneous systems [START_REF] Pacull | Self-organisation for building automation systems: Middleware linc as an integration tool[END_REF], [START_REF] Ducreux | Resource-based middleware in the context of heterogeneous building automation systems[END_REF]. The CSE provides a web interface to design, monitor, and update BMS rules. This interface is based on the latest web standards (HTML5, SVG and javascript) and blockly [START_REF]Blockly: A visual programming editor[END_REF] to offer an easy drag and drop programming. Once the rules are defined by the user, LINC verifies, compiles, and executes the rules using the real devices. In addition, when a sensor or an actuator becomes faulty, alerts are raised in the CSE to show all the rules impacted by the error.

This paper is structured as follows. First, Section II gives an overview of the middleware LINC; then, Section III describes the CSE and how it can be used to generate, verify, and execute new rules in the building. Section IV explains how the CSE has been used to generate rules in different contexts. Finally, Section V summarizes some existing related approaches and Section VI concludes the paper and presents future directions.

II. OVERVIEW OF LINC

To make this paper self-contained, this section presents an overview of the middleware LINC. More details on LINC can be found in [START_REF] Louvel | Linc: A compact yet powerful coordination environment[END_REF]. LINC provides a uniform abstraction layer to encapsulate software and hardware components. This abstraction layer relies on the associative memory paradigm implemented as a distributed set of bags containing resources (tuples). Inspired by Linda [START_REF] Carriero | Linda in context[END_REF], bags are accessed through three operations:

• rd(): takes a partially instantiated tuple as input parameter and returns from the bag a stream of fully instantiated tuples matching the given input pattern; • put(): takes a fully instantiated tuple as input parameter and inserts it in the bag; • get(): takes a fully instantiated tuple as input parameter, verifies if a matching resource exists in the bag and consumes it in an atomic way. Bags are grouped within objects according to application logic. For instance, all the bags used to manage a set of devices that communicate using the same communication technology can be grouped in the same LINC object. LINC objects are able to execute rules that manipulate resources in its own bags or in the bags of other objects.

The rest of this section describes the properties of LINC and the existing frameworks that facilitate the development of building automation applications and user interfaces. Finally, it describes how rules can be injected into the system, and activated/deactivated at run-time.

A. Coordination rules

The three operations rd(), get() and put() are used within production rules [START_REF] Cooper | Rule-based Programming with OPS5[END_REF]. A production rule is composed of a precondition phase and a performance phase.

Precondition phase: The precondition phase is a sequence of rd() operations which detect or wait for the presence of resources in several given bags. The resources are, for instance, values from sensors, external events, or results of service calls. In the precondition phase:

• the output fields of a rd() operation can be used to define input fields of subsequent rd() operations; • a rd() is blocked until at least one resource corresponding to the input pattern is available. Performance phase: The performance phase of a LINC rule combines the three rd(), get() and put() operations to respectively verify that some resources (e.g. the one(s) found in the precondition phase) are present, consume some resources, and insert new resources.

In this phase, the operations are embedded in one or multiple distributed transactions [START_REF] Bernstein | Concurrency control and recovery in database systems[END_REF], executed in sequence. Each transaction contains a set of operations that are performed in an atomic manner. Hence, LINC guarantees that actions belonging to the same transaction, are either all executed or none. This ensures properties such as:

• Some conditions responsible for firing the rule (precondition) are still valid at the time of the performance phase completion (e.g. the presence sensor still detects someone); • All the involved bags are effectively accessible (e.g. bags encapsulating sensors or an actuators).

B. LINC for buildings management systems

In the context of building automation, LINC provides the PUTUTU framework [START_REF] Pacull | Self-organisation for building automation systems: Middleware linc as an integration tool[END_REF], [START_REF] Ducreux | Resource-based middleware in the context of heterogeneous building automation systems[END_REF], which integrates more than 20 wireless and wired technologies commonly used in BMS and WSAN (Wireless Sensor/Actuator Networks). PUTUTU is formed of a set of LINC objects that encapsulate different communication technologies. These objects inherit from three basic objects which contain special bags to facilitate the manipulation of information and the development of BMS rules. The three basic objects are:

• WSAN Sensor Object: This object is used to encapsulate sensors. Among others, it provides three bags: Sensors to store sensor values in the form: (id, value), Type to associate the identifier of a sensor with the type of data it provides (i.e. (id, type)), and Location to manage the spatial location of the sensor (i.e. (id, location)). The first two bags are usually filled by the driver that encapsulates the technology and the last bag is filled by the application when the binding is done. • WSAN Actuator Object: This object encapsulates actuators. As the WSAN_sensor object it provides a bag Type to keep information about the device type and a bag Location to keep information about the location of the device. It provides also the bag Command which is used to send a command to the actuator. Then, to control an actuator, a resource is put in the bag Command. The resource contains the id of the actuator, the command to apply, and possible parameters. When the resource is added, the command is sent to the actuator through the driver encapsulating the protocol. • WSAN Sensor Actuator object: This object inherits from the last two objects and it is used to encapsulate technologies that provide sensors and actuators devices. Other components, such as BMS software tools and databases, can be mapped to obtain information regarding the building, to add information regarding new discovered devices, and to reconfigure the devices. For instance, in [START_REF] Ducreux | Dynamic reconfiguration of building automation systems with linc[END_REF] the software tools of a LONWorks [START_REF] Ryssel | IEHandbook LONWorks[END_REF] BMS have been encapsulated in LINC in order to automatically reconfigure the BMS.

C. User interfaces

LINC provides graphical user interfaces based on the Model-View-Controller(MVC) [START_REF] Krasner | A description of the model-viewcontroller user interface paradigm in the smalltalk-80 system[END_REF] approach. The MVC mechanism has been implemented above the bag paradigm. This mechanism permits to create interfaces that represent the real system, the user is able to interact with the real system, and the interface is updated when there is an event from the system. A framework that provides user interfaces has been developed [START_REF] Ducreux | Rapid prototyping of complete systems, the case study of a smart parking[END_REF]. The framework permits creating web interfaces based on the latest standards (HTML5, javascript and CSS). A LINC user interface is provided by a LINC object and can be accessed through any web browser. The framework provides three main types of objects which are based on the MVC mechanism and that are used to create user interfaces:

• Object SVGInterface: This object is able to render Scalable Vector Graphic (SVG) files. The SVG file is previously designed and javascript code is attached to support user interaction (e.g. click on predefined graphical elements). Graphical elements in the SVG are modified dynamically (e.g. change color, shape, text, hide, show, and so on), accroding to events from the system, thanks to the MVC mechanism. • Object Plot: This object generates different types of plots (as SVGs) and pies to visualize data. • Object Layout: This object provides the layout of the main interface. The interface contains several LINC interfaces, that can be provided by different LINC objects (SVGInterface, plot, layout, or other) and may also include external web interfaces. The layout is defined using an HTML web-page or a SVG file previously designed.

D. Rule generation and execution

In LINC, rules can be added in a running system. The rule is simply added as a resource in the form of (r_id, context, rule). When the resource is added, the rule is verified and compiled. If there is no error, the rule is executed by a LINC object. The context is an information meaningful for the application. It can be used to activate or deactivate a group of rules. For instance, a building may have different rules for different day periods, or it could have some rules for normal functioning and others for emergency situations.

Additionally, the execution of a rule is also controlled by resources in the RulesId bag of coordinator objects. This bag contains tuples shaped as "(rule id, context, status)", the status can be either ENABLED or DISABLED. When the rule is compiled, a rd to this bag is added at the beginning of the precondition and the performance of the rule, to check if the rule is ENABLED. Adding these rd operations permit stopping the execution of a rule by simply replacing the resource ("rule_id", "context", "ENABLED") to ("rule_id", "context", "DISABLED"). Indeed, the rd operation of the rule status causes the transaction to fail. In this way, a rule can enabled/disabled by adding a resource on a bag. Combining this with the context information, it is possible to enable/disable all the rules of a given context.

III. COORDINATION SCHEME EDITOR

This section describes how the CSE works. First it describes its architecture. Then, it details how new rules can be defined and verified using the graphical interface of the CSE. Finally, a brief overview of the building monitoring interface provided by the CSE is presented.

A. CSE Overview and Architecture

The architecture of the CSE is illustrated in Figure 1. Existing communication technologies for building automation are integrated with LINC using the PUTUTU framework [START_REF] Pacull | Self-organisation for building automation systems: Middleware linc as an integration tool[END_REF]. In this way, heterogeneous sensors and actuators can be accessed and manipulated by the application. Besides, the system integrates a database that keeps information regarding the devices deployed in the building(s). This information permits the application to associate a device identifier with the type of information it can provide, the actions it can perform, its location, and so on.

The user interface has been developed using the MVC GUI framework provided by LINC. As seen in Figure 2,

B. Rule design

Rules can be defined by the user, through the creation of Templates. Defined Templates are then activated in the desired room(s). The blockly interface provides blocks that can be selected by the user to create or update the template.

The user can also create a dedicated rule (without a template) by directly choosing the devices available on a specific room of the building. The same interface is used for dedicated rules or templates.

1) Templates definition: A template permits the creation of generic rules, using a set of devices, that can be activated later on a given room or set of rooms. The template can correspond to a scenario (i.e. a set events generating actions for actuators), a soft-sensor (i.e. combination of real sensors or external values), or a soft-actuator (i.e. to act on several actuators). Template Scenario: A template scenario defines a rule that combines several sensors and actuators. A scenario is formed of a set of conditions, such as values of sensors, to trigger actions to a set of actuators. Figure 3 illustrates the definition of a template scenario that detects the presence of someone in the room, using a presence sensor; then, it determines if the room temperature is higher than "26 degrees Celsius", using a temperature sensor. If the two previous conditions are true, the ventilation of the room is turned on.

Template Soft-Sensor: The value of a soft-sensor is a function that combines the values of a set of real sensors. Figure 4 shows an example of definition of a soft-sensor. In this case, the value of the soft-sensor will be "true" if the presence sensor value is "true" and the humidity is higher than or equal to "50%".

Fig. 4: Soft Sensor Template Creation Example

Template Soft-Actuator: Acting on a soft-actuator means acting on several actuators. For instance, all the lights in a room can be combined on a single actuator, so that turning on this actuator will turn on all the lights in the room.

C. Template activation

Once a template is defined, the user can activate it on a room. For that the user can choose the desired building and floor. Then, the list of defined templates and possible rooms will be displayed on the interface, and the user can bind a template to a room.

Once the user submits the activation, LINC will check if the room has all the devices needed by the template. If this is the case, LINC will generate the appropriate rule, using the available devices, and add the rule the system to execute it. Listing 1 shows the rule generated for an instance of the template scenario shown in Figure 3. Line 1 and 7 are used to control the rule execution. Line 2 corresponds to the first condition of the template (presence == "T rue") and matches resource ("pr_23", "True"). Line 3 and 4 correspond to the second condition of the template (temperature > 26).

Listing 1: Rule generated from Template

A rule instantiated from a template can be deactivated by removing the corresponding resource in the bag RulesId of the coordinator object (i.e. the CSE object). It is also possible to disable all the instantiated rules of template thanks to their context information (e.g. "temp_sc").

D. Dedicated rules

Using the devices available in the building, it is also possible to write rules. In this case no template is defined, the devices are chosen from the list of available devices. The Figure 5 shows how a dedicated rule is created, available devices per room can be chosen from the list in the right side of the interface, then they can be used as a blockly block, in the blockly workspace, to create the rule.

Dedicated rules are also verified before their execution, and they can be activated/deactivated when desired.

E. Rule verification

When a template is instantiated or a rule is designed, several verifications can be done. If an error is detected, it is shown on the interface and the rule is not activated. Examples of errors that can be detected are:

• values tested for the sensors are out of the sensor range;

• invalid command for an actuator;

• invalid type in free entry text (e.g. a string when expected an integer). The CSE also provides an interface to monitor the rules deployed in the buildings. The floor plan of a room is displayed along with sensor values and actuator staus.

In addition, the monitoring interface informs when a new device, that has not been bound, is discovered and allows the user to bind the new device to a given location. After the binding is done, the new devices can be used in rules. Figure 6 displays the monitoring interface where the building plan is visualized on the right, and new discovered devices appear on the left part of the window.

Finally, CSE offers a monitoring interface of all the rules designed with CSE. It is possible to filter the rules based on building, room, template generation/instantiation date, and the status of the rule. The status of a rule can be activated, stopped, deleted or error. Error status means that one or more devices by the rule became faulty. Thanks to the template instantiation, the facility manager can then easily link the faulty device to a template scenario. For instance, if the presence sensor "pr_23" becomes faulty, used in Listing 1, the rule "r0033" is impacted. This rule was generated by instaniating the template 3 on a given room. Hence the facility manager will know that the temperature of the room is not properly controlled anymore.

IV. CASE STUDIES

A. Lighting system control

This use case has been implemented in the FP7 SCUBA European project [START_REF]SCUBA: Self-Organazing, Co-operative, and robUst Building Automation[END_REF]. The demonstrator integrates the LON-Works [START_REF] Ryssel | IEHandbook LONWorks[END_REF] Building Automation System (BAS) with external information. LONWorks is a technology largely used to control devices in large buildings. However, the BAS uses only information coming from within the building. When the building is built (or retrofitted), a LONWorks technician comes and manually configures the connections between the sensors (e.g. movement or buttons) and the actuators (e.g. light, shutter or Heating Ventilation and Air Conditioning (HVAC)).

The goal of this scenario is to control the lighting system of the building, using external information regarding the power consumption of the building and the current energy price. The idea is to decrease the light intensity to remove/decrease picks of power consumption or to save money when the energy price First, the LONWorks system has been integrated in LINC [START_REF] Ducreux | Dynamic reconfiguration of building automation systems with linc[END_REF] as well as the ligthing algorithm, running in the cloud. Then, different devices of the building were mapped in the CSE according to their position in the building. A soft sensor and a soft actuator have been defined. The soft sensor takes inputs from the lighting algorithm. The soft actuator is the LONWorks device controlling the lights in the room. Finally, three template scenarios were defined to adapt the ligth intensity. The first one switches off the light when the room is empty. The second one sets the light to 500 lux when the room is occupied and no constraints are imposed on energy consumption. The last one sets the light to 150 lux if the external algorithm detects that energy price is high or the demand is high in the building. With the CSE the facility manager can create his own rules according to his objectives.

B. Building emulation

The second use case is a portable demonstrator, that emulates a buidling containing several sensors and actuators technologies. Figure 7 is a picture of the demonstrator. The demonstrator integrates a lighting control device (Plugwise plug that can be on or off), a shutter (graphical interface on a small screen), and emulates a presence and luminosity sensor (EnOcean switch with four buttons). The building control and the CSE run in a Raspberry PI platform.

With CSE we designed several scenarios to adapt the light and the shutter according to the information coming from the EnOcean button. Figure 8, shows a rule example, where the shutters are controlled according to sensor values (EnOcean switch). This emulation shows that we can easily implement rules with CSE integrating several heterogenous technologies.

V. RELATED WORK With the increase of embedded communicating devices in applications, such as building automation and smart homes, visual programming frameworks are an interesting approach to provide the user with the tools necessary to control their own system. Such framework must deal with several challenges. First they have to support the integration of heterogeneous devices and software/hardware components. Then they should allow the generation, verification, and execution of distributed rules. And finally, they should provide an intuitive visual programming language. Based on LINC, the CSE answers all these challenges and provides a complete solution.

Dealing with heterogeneity in these applications has been studied in the literature [START_REF] Veichtlbauer | Generic middleware for userfriendly control systems in home and building automation[END_REF], [START_REF] Fernbach | Interoperability at the management level of building automation systems: A case study for bacnet and opc ua[END_REF], however, no solution integrates a friendly user interface with a mechanism to coordinate distributed devices while hiding heterogeneity. Blockly has already been used to program wireless sensor network applications in environments [START_REF] Serna | A visual programming framework for wireless sensor networks in smart home applications[END_REF]. Similar to the CSE, the authors provide a web interface based on blockly permitting the definition of rules. Then, the proper code is generated. Nevertheless, this approach does not support heterogeneous devices (i.e. only CoAP capable devices are encapsulated). Other visual programming approaches such as ClickScript [START_REF] Mainetti | A novel architecture enabling the visual implementation of web of things applications[END_REF] and Labview [START_REF] Johnson | LabVIEW Graphical Programming: Practical Applications in Instrumentation and Control[END_REF] require some programming skills and are not suited for non-experienced users.

VI. CONCLUSION

This paper presented the Coordination Scheme Editor (CSE). CSE is a web-based tool used to design, monitor and control building automation rules. CSE relies on the LINC middleware and rules can thus be designed with heterogeneous hardware and software technologies.

CSE offers a visual programming language based on the latest web standard (HTML5, SVG and javacript) and Blockly. This provides an easy to use HMI than can be used by facility manager, building owners or event building occupants. CSE interface allows to design template rules that can be instantiated across buildings from a single web interface. At design and instantiation time, rules are verified to check for instance that only valid commands are sent to actuators. CSE also provides a monitoring interface that can display the status of all the rules. The monitoring interface also shows all the rules and templates impacted by a faulty sensor.

Future work will focus on providing more advanced verification with CSE. Indeed currently CSE only verifies that the generated rules are correct. However there is no verification on the potential conflicts between rules. A typical conflict is when two rules can be activated at the same time (i.e. same triggering condition) and they have different effect in the system (e.g. start vs. stop the ventilation).

Fig. 2 :

 2 Fig. 1: CSE architectures

Fig. 3 :

 3 Fig. 3: Scenario Template Creation Example

Fig. 5 : 5 : : { 7 [ 9 [

 5579 Fig. 5: Dedicated Scenario Example

Fig. 6 :

 6 Fig. 6: CSE monitoring interface

Fig. 8 :

 8 Fig. 8: Building Emulation Scenario

ACKNOWLEDGMENT

This work has funded by the Artemis ARROWHEAD (grant 332987) and the H2020 TOPAs project (grant 676760).