Mickaël Riviere
email: mickael.riviere@esiroi.re

José D Padrón

Carlos T Calafate
email: calafate@disca.upv.es

Juan-Carlos Cano
email: jucano@disca.upv.es

Tahiry Razafindralambo
email: tahiry.razafindralambo@univ-reunion.fr

Improving Emergency Vehicles Flow in Urban Environments Through SDN-based V2X Communications

Keywords: Software-Defined Networks, SUMO, V2X, emergency vehicles, DENM messages

Vehicular networks have emerged in the past years as one of the most promising technologies in the context of Intelligent Transportation Systems. Several applications have recently been envisaged to improve the lives of citizens in different ways, particularly in the field of safety. In this work we focus on a different direction by leveraging on vehicular communication technologies and software-defined networking (SDN) to achieve a solution able to improve the travel time of emergency vehicles (EVs) when traveling in urban environments. In particular, we devise a solution where the route to be followed by EVs is shared with an SDN-enabled infrastructure, being then smartly rebroadcasted using those roadside units (RSUs) that fall within the region the EV will pass by in a near future. This way, standard vehicles that are ahead of the EV are notified about its arrival, and clear the required lane to facilitate a faster and obstacle free path for the EV's journey. Simulation experiments based on the SUMO traffic simulator for the city of Valencia, Spain, show that, if properly tuned, the proposed solution is able to reduce the travel time of EVs by nearly 15%, while maintaining the travel time of other vehicles mostly unaltered.

I. INTRODUCTION

The field of Intelligent Transportation Systems (ITS) is gaining momentum due to its potential at improving the lives of citizens from many perspectives, including timeliness, efficiency, fuel economy, environmental friendliness, etc. [START_REF] Garg | A systematic review on intelligent transport systems[END_REF]. In this context, vehicular ad hoc networks (VANETs) [START_REF] Vahdat-Nejad | A survey on context-aware vehicular network applications[END_REF] have emerged as a means for vehicles to directly communicate with each other, or with the aid of infrastructure elements known as RoadSide Units (RSUs) [START_REF] Sou | Enhancing vanet connectivity through roadside units on highways[END_REF]. Such networks pave the way for a plethora of services such as safety, real-time traffic monitoring, congestion detection, information and entertainment, etc. [START_REF] Shrestha | Challenges of future vanet and cloud-based approaches[END_REF].

Recently, the adoption of Software-Defined Networking (SDN) in VANET environments has emerged as a solution to provide improved flexibility and programmability to these networks [START_REF] Al-Heety | A comprehensive survey: Benefits, services, recent works, challenges, security, and use cases for sdn-vanet[END_REF], [START_REF] Ku | Towards software-defined vanet: Architecture and services[END_REF]. In this context, advanced services can be envisioned that involve a high level of network dynamism. Hence, in this paper, we focus on a novel application that combines the strengths of SDN and VANETs to reduce the travel time of emergency vehicles (EVs) in urban environments. In particular, we focus on situations where the traffic density in a city is moderate or high, which often causes an emergency vehicle to travel slower than expected as other vehicles often do not know exactly which action to take when they detect its presence, and such detection often occurs only when the EV is already quite near. The proposed solution addresses this problem by partially sharing the EV's established path with other vehicles via Decentralized Environmental Notification Messages (DENM) [START_REF] Festag | Cooperative intelligent transport systems standards in europe[END_REF]. Such solution is possible by combining VANET communications with SDN, whereby messages are sent from the EV to the nearest RSU, and then from another RSU to the relevant vehicles along the path (V2I communications paradigm). In particular, the use of SDN technology will enable a smart and dynamic redirection of DENM messages to warn road users ahead that an EV is approaching, and that a specific lane must be cleared to facilitate its trip. An optimized redirection of DENM messages promotes the reduction of network traffic by broadcasting only the DENM messages at relevant points, and reaching only those receivers that should actually take an action on the messages, instead of merely targeting all vehicles in circulation indiscriminately. This helps at drastically reducing the radio resources' consumption, being a significant contribution of the current work compared to prior approaches.

Our solution has been implemented and validated by combining the Ryu SDN controller [START_REF] Pemberton | Ryu openflow controller[END_REF] with the SUMO traffic simulator [START_REF] Lopez | Microscopic traffic simulation using sumo[END_REF]. Based on experiments in the city of Valencia, Spain, we are able to show that the proposed solution is quite effective, allowing to reduce the travel time of EVs by nearly 15%. In addition, we find that our solution does not introduce a meaningful penalty for other vehicles circulating along the EV's route, being that their travel times remains similar.

The remainder of the paper is organized as follows: in the next section we provide an overview of some related works. Then, in section III, we provide an overview of our proposal, including also technical details regarding its implementation. Section IV details the simulation framework we have set forth in order to undertake the desired study. Experimental results are then presented and discussed in section V. Finally, conclusions and future research directions are provided in Section VI.

II. RELATED WORK

The improvement of the emergency services is an issue that has been addressed by several research projects. In particular, we can find in the literature different works where VANET technologies are leveraged to reduce the travel times of EVs. Back in 2009, Buchenscheit et al. [START_REF] Buchenscheit | A vanet-based emergency vehicle warning system[END_REF] introduced the concept of active warnings for EVs to notify other vehicles about their needs. Years later, Jayaraj and Hemanth showed the potential of VANETs for emergency vehicle signaling [START_REF] Jayaraj | Emergency vehicle signalling using vanets[END_REF]; this solution uses DSRC communication to warn nearby vehicles, and facilitate the passage of the EV. Such approach was extended more recently [START_REF] Tanjima | Provide emergency ambulance service in road accident using vanet[END_REF] by shifting the focus to RSUs.

The focus on traffic lights is another approach that can be beneficial to bring down the travel time of EVs [START_REF] Noori | Modeling the impact of vanet-enabled traffic lights control on the response time of emergency vehicles in realistic large-scale urban area[END_REF]. In this direction, Sumi and Ranga [START_REF] Sumi | An iot-vanet-based traffic management system for emergency vehicles in a smart city[END_REF] proposed an IoT-VANET-based traffic management system that will ease traffic movements for EVs. Similarly Costa et al. [START_REF] Da Costa | Poster: Safesmart -a vanet system for efficient communication for emergency vehicles[END_REF] proposed a system that uses the current infrastructure of the transceivers in traffic lights and the features of DSRC/ITS-G5 to provide communication between the EVs and the transceivers in traffic lights.

RF signals can also be used to segregate a certain part of a road, hence reducing the traffic on the EV's route [START_REF] Nagaraja | Thefted vehicle identification system and smart ambulance system in vanets[END_REF]. Other solutions exist to secure the messages that are exchanged using various encryption methods and security algorithms; such methods enable traffic discovery so that messages exchanged between the EVs and the vehicles cannot be diverted, and may influence the actual traffic [START_REF] Chen | A secure ambulance communication protocol for vanet[END_REF]. However, these different solutions have little or no effect on traffic to facilitate the movement of emergency vehicles. Indeed, they simply collect information and adapt the EV's route accordingly.

In this paper, our approach follows the same research line as that in [START_REF] Sumi | An iot-vanet-based traffic management system for emergency vehicles in a smart city[END_REF], [START_REF] Da Costa | Poster: Safesmart -a vanet system for efficient communication for emergency vehicles[END_REF], emphasizing on V2I communications to improve EV travel times. In particular, the proposed solution will use an SDN network to dynamically handle both short and long range messages, avoiding message broadcasting in areas where the notification has little or no relevance. Hence, our solution will warn vehicles on the same stretch of road to move away, but will also warn vehicles ahead along the EV's route that an incoming EV is expected; this allows freeing up lanes so that the entire EV route can benefit from reduced travel times, while minimizing the usage of radio resources.

III. PROPOSED SOLUTION

VANET technologies are capable of offering support for advanced services, which may, for instance, help to improve traffic efficiency. To this end, the use of Cooperative Awareness Messages (CAM) and DENM messages [START_REF] Festag | Cooperative intelligent transport systems standards in europe[END_REF] can be used to gather periodic status data from nearby vehicles, or to provide safety services such as cooperative collision avoidance. DENM messages are often generated by a Central ITS service manager following a request from one of the services that requires the notification of an asynchronous event. In a decentralised situation, specific details of the event informed in the message can warn us about the level of danger using a set of four possibilities: informative, obstacle, danger, the highest danger.

This paper attempts to provide a solution to improve the travel time of EVs by combining the Vehicle to Infrastructure (V2I) and Infrastructure to Vehicle (I2V) paradigms, along with SDN technology, to achieve as optimized communications solution. In particular, SDN will support the exchange of DENM messages between the EV and other road users through the vehicular infrastructure; such messages will notify vehicles about the imminent lane occupancy of the EV in their vicinity, allowing them enough time to free that lane before the EV's arrival.

To understand how this goal is achieved, Figure 1 shows an overview of the proposed network architecture. The network infrastructure includes an edge domain where SDN-enabled Roadside Units (RSUs) are located at the lower infrastructure layer, and whose connectivity is made possible by the higher infrastructure layers, all of them managed by the SDN controller. Whenever the EV detects a new RSU, it will contact it to announce its future route. Such information is sent upwards to the SDN controller, which exchanges this information with a specific ITS application developed for our solution. Such application includes geographic data about each of the RSUs deployed. Based on such data, it decides which RSU (or RSUs) is/are the best candidate(s) to disseminate the received message. Such selection is detailed in Algorithm 1. Its operation is quite simple; first, it must retrieve several data about the EV, such as its position, the lane ID, or the last RSU visited. The algorithm will then use this information to determine the next RSU along the path. If the EV starts its journey and does not know any RSU, then the one with the highest signal strength is chosen. Once the next RSU(s) is chosen, a message is sent with the necessary information to warn the other vehicles.

Afterward, such information is provided to the SDN controller, which reconfigures the flow tables of switches and RSUs, thereby defining the new path for this flow. From this point on, the DENM message that triggered this reconfiguration, as well as all other DENM messages that follow (when arriving from the same RSU), will be delivered to the target RSU(s), which will broadcast them. Since the goal is to empty the lane that the EV is using, all vehicles traveling in that same lane will be requested, upon message reception, to change their lane, so as to free the EV's lane for a period τ . This is the case of vehicle C in the figure .  As final remarks regarding this solution, it is worth noting that it is applicable to large cities, where major streets and avenues have more than one lane for a certain travel direction. Also, for safety and abuse prevention purposes, only an EV can send messages requesting the release of a lane. Standard vehicles can only receive messages and act upon them. In addition, when receiving a lane change message, this can result in a mere warning to the driver for standard vehicles, or a mandatory instruction in the case of autonomous vehicles. Regarding the period of time τ during which the lane is

IV. SIMULATION FRAMEWORK

In this section we will describe the simulation framework used for our study, which involves three different tools: i) Mininet version 2.3.0 [START_REF] Team | Mininet releases[END_REF], in order to emulate the network. ii) Ryu version 4.34 [START_REF] Pemberton | Ryu openflow controller[END_REF], which serves as an external controller to the network. iii) SUMO version 1.14.1 [START_REF] Lopez | Microscopic traffic simulation using sumo[END_REF], which allows us to simulate the traffic within a city. Firstly, Mininet allows us to design and emulate the proposed network. To this end, we devise a tree topology consisting of 4 switches (openVSwitch) and 8 access points.

Notice that Mininet is capable of working with the OpenFlow technology, which is a basic requirement for our SDN-based solution.

Secondly, the RYU controller allows the dynamic creation of the flow table entries, and the redirection of the various network messages. This allows us to dynamically redirect messages towards the next RSU along the EV's route by having awareness of the current EV position, the geographic location of all RSUs, and the EV route itself, determining the most appropriate RSU at any time.

Thirdly, SUMO is used for the simulation of the road, specifically to recreate a part of the city of Valencia. It is worth mentioning that the vehicles simulated on SUMO are also connected to Mininet in order to identify them as mobile network hosts. A total of 200 cars are present in the simulation, and they constantly move along a randomly-defined path along the streets of the city.

The behavior of the simulation is the following. A car will have to make a journey from a point A to a point B. Each time it detects a new RSU, it will send a message (DENM) to the controller. Following an analysis by the ITS application, this message will be then redirected to the next RSU placed strategically along the EVs path, being received by all the vehicles within its radio range. Such vehicles should react to the message by switching to another lane whenever they can interfere with the EV.

To have a complete view of the proposed simulation framework, Figure 3 shows a diagram of our network and the different tools involved. The main goal is that the EV sends its DENM to the SDN controller through the RSUs deployed throughout the city, and the different intermediate equipment of the SDN network. The message will then be redirected to the next access point and retransmitted to the other vehicles.

Using the devised simulation framework, we are able to study the impact of varying different parameters on the EV travel time performance, and we can also measure network metrics like network (re)configuration time, message redirection delay, and number of packets involved.

V. SIMULATION RESULTS

In this section we present our simulation results based on the simulation framework described in the previous section. Our goal is to determine the degree of effectiveness of our SDN-based solution, where algorithm 1 is used to optimize traffic, compared to the default situation, where no mechanism is available to favor the EV along its path. To this end we perform different experiments under similar conditions, and where we vary the period τ , which allows us to regulate for how long does the EV request a specific lane to be left empty, a period of time that should be respected by other vehicles. In particular, we test with τ values of 20, 30, 40, 50 and 60 seconds (one for each experiment).

Regarding performance metrics, a comparison between the EV and regular vehicles in terms of travel time and average speed will be carried out. This is in order to see if an improvement is visible, and what is the impact of our approach on the global traffic. In particular, our end goal is to reduce the EV's travel time as much as possible while attempting to maintain the impact on the other vehicles to a minimum. Having said this, we start by first analyzing the travel time and average speed of the EV to determine the actual improvements achieved when adopting different values for the τ parameter.

Table I shows the different values achieved in terms of average speed and travel time for each value of τ , being the default situation (no SDN network) available as reference. In terms of average speed, we can observe a tendency (from 20 to 50s); in particular, when τ is increased, the average speed also increases. In fact, the optimal results are achieved for τ = 50, which results in an average speed of 9.73 m/s; this is 16.11% faster than the default situation. Likewise, the lowest value obtained (τ = 20) still allows obtaining a better performance than the default situation, with an average speed that is 9.72 % higher.

In terms of travel time, we can see the same trend than for the average speed, only in the opposite direction. Again the best value is also achieved for τ = 50. For this time period, the travel time decreases by 25 seconds, a 13.88% less than the default situation.

In order to contrast the results, figure 4 shows the variation of the speed of the EV throughout time. In particular, it shows the difference between the default situation (figure 4a), and the best-case situation, achieved for τ = 50 (figure 4b). In the latter experiment we can see that, from the beginning of the EV's movement, the average speed experienced is higher. In addition, there are also fewer times when the EV has to slow down or stop completely.

It is expected that improving the EV travel time has an impact on the overall traffic, has vehicles have to move aside in order to clear a specific lane; this is prone to cause more temporary congestion to regular vehicles. So, an ideal solution should reduce the EV's journey time while minimizing the impact on the global traffic. If we focus on the results available in table II, we can observe the behavior of the overall traffic regarding travel time. As we can observe, the impact is minimal. In fact, the average travel time when compared to the default situation increases, at most, by 1.08 % (for τ = 40); for the optimum value found above (τ = 50), we see that such increase is limited to 0.59 %, which can be considered nearly negligible. To gain further insight onto the differences between the default situation and the optimal one, figure 5 shows the distribution of the travel time for vehicles in both cases. Notice that, for the default situation, the minimum values are lower, and there are slight differences around 160s and 195s, meaning that travel times tend to concentrate around these values as a result of having fewer available lanes, which means that more queuing is prone to occur to leave room for the EV, as expected.

Overall we find that, in order to achieve a correct balance between the EV trip optimization and the impact on the rest of the traffic, a 50s lane reservation period seems to be a good compromise. In this way, we will obtain a travel time improvement of 25s for the EV in a 3-minute route, being that other vehicles only experience an increase of about 1 second (on average). Finally, the question arises as to the network's ability to manage all its connections from a global system perspective. As it can be seen in table III, acceptable values are achieved, although these could be improved with better/dedicated hardware. Notice that we are in a situation where a single controller manages 12 network devices that simultaneously receive information from 200 vehicles. Although only one EV is triggering network updates, this is enough to create a slight bottleneck effect that affects network performance. In fact, although the network creation time seems quite short compared to other values, it still takes more than 1s. We also ended up with a message redirection time of about 6s to contact about 70 vehicles. This is primarily due to the sum of several factors: the time spent to handle the first message, the overhead involved in the delivery to the controller, the processing time, and the update time of the network configuration. These different values could be improved by using other controllers to alleviate the network load, and by optimizing the application-layer software developed.

VI. CONCLUSIONS AND FUTURE WORK

As ITS services and underlying technologies continue to expand, novel solutions can be envisioned that go well beyond current state of the art, achieving improved performance in unforeseen ways. In this paper we have presented such a solution by leveraging on the potential of VANET and SDN technologies. In particular, we have devised a strategy that allows emergency vehicles to improve their travel times in congested urban environments by requesting those vehicles occupying the lane they intend to use in a near future to free that lane. This is made possible by a combination of elements including: (i) VANET-enabled vehicles and EVs, (ii) an SDN-based infrastructure that includes RSUs along the city, and (iii) a sophisticated application that is aware of RSU deployment, and that can interpret EV messages, so as to select the adequate geographic locations for message dissemination; such approach allows reaching all required vehicles along the EV's path (for its near future locations), while minimizing the amount of radio resources consumed.

Our proposed solution as been formally presented in this paper, and evaluated through simulation by combining both Mininet and SUMO tools. Experimental results for a realistic setting in the city of Valencia, Spain, have shown that we are able to improve the travel time of EVs by nearly 15% when requesting vehicles occupying the lane used by the EV to move aside for a specified period τ ; experimentally we have shown that a period of 50s achieves the best results in terms of reducing the EV travel time, while maintaining the the time penalty experienced by other drivers at negligible levels.

As future work we plan to extend our solution, which currently relies on message exchange via RSUs only, to also support direct communication between the EV and other vehicles so as to free the lane reservation made as soon as the EV passes by. Such approach is particularly applicable to autonomous vehicles, as regular drivers are well aware that the EV has passed. We also plan to extend our study to other city profiles, characterized by a different number of lanes and different traffic loads, to assess the gains achieved by our solution under different settings.

Fig. 1 :

 1 Fig. 1: Proposed network architecture.

Fig. 2 :

 2 Fig. 2: Scenario used for testing. Blue circles indicate the location of the RSUs, and the EV's route is shown in red.

Fig. 3 :

 3 Fig. 3: Simulation framework.

Fig. 4 :

 4 Fig. 4: EV speed vs. time.

Fig. 5 :

 5 Fig. 5: Distribution of travel time for regular vehicles (Default vs τ = 50).

TABLE I :

 I Average speed and travel time for the EV when varying the τ parameter.

		Default	τ = 20	τ = 30	τ = 40	τ = 50	τ = 60
	Avg. speed	8.38	9.17	9.25	9.64	9.73	9.54
	(m/s)
	∆ (m/s)	-	+0.79	+0.87	+1.26	+1.35	+1.16
	∆(%)	-	+9.42	+10.38	+15.03	+16.11	+13.84
	Travel time	180	164	166	159	155	161
	(s)
	∆ (s)	-	-16	-14	-21	-25	-19
	∆(%)	-	-8.88	-7.77	-11.66	-13.88	-10.55

TABLE II :

 II Average travel time for regular vehicles when varying the τ parameter.

		Default τ = 20	τ = 30	τ = 40	τ = 50	τ = 60
	Travel time	185.6	186.1	187.4	187.6	186.7	187.4
	(s)
	∆ (s)	-	+0.5	+1.8	+2.0	+1.1	+1.8
	∆(%)	-	+0.27	+0.97	+1.08	+0.59	+0.97

TABLE III :

 III Network performance stats.

Message redirection time 6005.4 ms ± 0.34% Network configuration time 1260 ms ± 8.63% Average number of vehicles reached 69 Number of packets exchanged 234 ± 16.46%

ACKNOWLEDGMENT This work is derived from R&D project PID2021-122580NB-I00, funded by MCIN/AEI/10.13039/501100011033 and "ERDF A way of making Europe".


Abdelkader Lahmadi
email: abdelkader.lahmadi@loria.fr

Laurent Andrey
email: laurent.andrey@loria.fr

Olivier Festor
email: olivier.festor@loria.fr

Performance of Network and Service Monitoring Frameworks

Keywords: performance evaluation, measurement metrics, measurement methodology, management efficiency, management scalability, management delays, management impact

The efficiency and the performance of management systems is becoming a hot research topic within the networks and services management community. This concern is due to the new challenges of large scale managed systems, where the management plane is integrated within the functional plane and where management activities have to carry accurate and up-todate information.

We defined a set of primary and secondary metrics to measure the performance of a management approach. Secondary metrics are derived from the primary ones and quantifies mainly the efficiency, the scalability and the impact of management activities. To validate our proposals, we have designed and developed a benchmarking platform dedicated to the measurement of the performance of a JMX manager-agent based management system.

The second part of our work deals with the collection of measurement data sets from our JMX benchmarking platform. We mainly studied the effect of both load and the number of agents on the scalability, the impact of management activities on the user perceived performance of a managed server and the delays of JMX operations when carrying variables values. Our findings show that most of these delays follow a Weibull statistical distribution. We used this statistical model to study the behavior of a monitoring algorithm proposed in the literature, under heavy tail delays distribution. In this case, the view of the managed system on the manager side becomes noisy and out of date.

I. INTRODUCTION

The rapid growth of the Internet over the last 20 years has been startling. However, efforts to manage its services and their underlying networks have often fallen afoul due to poor performance of the management systems in place. The problem is not that management systems and protocols do not exist, but rather that the lack of performance models, tools and benchmarking platforms to assess their cost and well understand their needs on resource consumption are not well studied. Furthermore, studying the performance of the value-added functional plane without taking in consideration the cost of management activities and its impact would lead to inaccurate estimation of the quality of service and might impact the business benefit. Consequently, questions arise like: what is the cost of a management system, its impact on a managed system ? how does a management system scale with the growth of a managed system ?

The same problem has arisen in other computer science disciplines (databases, distributed systems, IP networks,etc). An extensive literature exists, and many standards have emerged in these disciplines to assess the performance of the proposed systems and architectures.

In the network and services management community however, it was surprising that no agreement on conventions for evaluating the performance of management systems was established so far. Existing performance metrics like response times, throughput, cost, quality and scalability in the network and service management literature are inconsistent and confusing. As a result, no common foundation has been established to evaluate the performance of management systems so far. Such studies are burdened with the lack of common metrics and usually their results are not comparable, their experiments are not reproducible and they are not representative.

One approach to solving the above lacks is to develop benchmarking platforms and collect measurement data sets to identify and well define the most relevant performance metrics and their measurement methodology.

The aim of our work was to provide common performance metrics to evaluate the performance and the cost of management frameworks using both measurement and analytical techniques for common unrealistic and realistic management scenarios. We have focused our performance evaluation studies on JMX [START_REF] Sun | Java T M management extensions, instrumentation and agent specification, v1.2[END_REF], the de facto standard to manage Java based applications.

The remainder of this paper is organised as follows. In section II, we describe related works devoted to the performance evaluation in general and those dedicated to the management plane in particular. In section III, we identify the major problems that the performance evaluation of management frameworks studies did encounter. We detail our approach and our proposed set of metrics to evaluate the performance of a management framework in section IV. In section V, we present the main results that we have obtained. We conclude and address future work in section VI.

II. RELATED WORKS

Many studies have addressed the performance of management applications. We classify these studies according the research goal behind them. Mainly, we have identified three goals: comparing management approaches performance, studying the performance of a new feature within an existing management approach, purely studing the performance of a new management approach. In [START_REF] Pavlou | On Management Technologies and the Potential of Web Services[END_REF], [START_REF] Pras | Comparing the Performance of SNMP and Web Services-Based Management[END_REF], researchers compare SNMP protocol usage based on polling to web service based management. In an effort to investigate the security overhead in network management, [START_REF] Corrente | Security Performance Analysis of SNMPv3 with Respect to SNMPv2c[END_REF], [START_REF] Du | Implementation and performances analysis of snmp on a tls/tcp/ base[END_REF] assess the performance of SNMP with security support. One of the rarely attempts to make a dedicated measurement on a management protocol was undertaken by Pattinson [START_REF] Pattinson | A Study of the Behaviour of the Simple Network Management Protocol[END_REF] for the SNMP protocol.

Performance evaluation is well studied in other disciplines for other purposes. In [START_REF] Jain | The art of Computer Systems Performance Analysis[END_REF], the author gives a cohesive collection on performance evaluation that include -measurement techniques and tools, experimental design and analysis, simulation, queueing models and a section on probability theory and statistics. Our work is conducted in accordance to guidelines provided in this textbook. The IPPM working group within the IETF has published many RFCs about performance metrics and their measurement methodology dedicated to IP networks [START_REF] Paxson | Framework for IP Performance Metrics[END_REF]. Their work inspired us to define and develop a set of metrics dedicated to monitoring applications. The work of Woodside et al [START_REF] Jogalekar | Evaluating the scalability of distributed systems[END_REF] has provided a metric to assess the scalability of a distributed system. We adopted this metric and extended it to assess the scalability and the impact of a monitoring application.

III. PROBLEM STATEMENT

Our analysis of existing studies on the performance evaluation of management frameworks, allows us to identify two major problems within these studies. The first problem is that existing metrics are not adequate. Major metrics used by performance analysis studies have a confusing semantics. For example, monitoring delay is called response time, execution time, round trip delay, delivery delay. Their metrics support only one activity and can not express others. For example, a request-response delay metric cannot express notification delays. These metrics, rather than giving a clear quantification of the performance, leave a misunderstood burden on the performance of a monitoring application. The quantification of the scalability is a good example. The second major problem in the current performance analysis is that existing methodologies are not adequate. This problem engenderd the following flaws in existing performance studies:

• Results of different performance studies may not be comparable. For example, one study may have measured the delay of retrieving a certain MIB table using GetNext PDUs, whereas the other study uses GetBulk PDUs. • Experiments may not be reproducible, which means that other researchers trying to perform the same experiment may not be able to achieve similar results. This problem may be caused by the fact that the experiments have not been described with sufficient details, leaving the researcher trying to reproduce the experiments with too many options. This problem may also be caused, however, by some events interfering with the experiment, which basically means that the results are not correct. • Performance studies may not be representative, which means that monitoring usage in real networks may be largely different from the one assumed in a study. Therefore, we find that these problems originate from the lack of common measurement scenarios and metrics to evaluate a monitoring approach.

IV. OUR APPROACH

We have developed an approach [START_REF] Lahmadi | Performances des fonctions et architectures de supervision de rseaux et de services[END_REF] for the performance evaluation of a monitoring application, that relies on three goals. These goals are the following:

• A well defined set of of performance metrics with a clear terminology. They should cover management challenges and they should be independent from the underlying technology; • A well defined methodology for their measurement and to guarantee reproducibility. • Design a set of experimental procedures that cover monitoring challenges. The scalability and the development of benchmarking platforms we here targetted. To achieve these goals, we developed a set of contributions that we detail in the following sections. We have defined tow categories of metrics to evaluate the performance of a monitoring application.

A. A Unified Performance Metrics

In the first category, we put a set of metrics that we qualify as primary metrics. These primary metrics are the direct output of a measurement process applied to a monitoring scenario under test. As depicted in Figure 1, this category includes three types of metrics which are: speed, cost and quality. The speed metrics deal with management data access either locally or remotely. These metrics capture the quantity and related timing of management data under some management activity. The cost metrics deal with the overhead of management activities and the resources that they consume. The storage cost refers to the amount of management data stored at a management entity. The communication cost refers to the total amount of information that needs to be transmitted through the network between two or many management entities. The computational cost reflects the processing activity at a management entity. The quality metrics deal with the spatial and temporal errors of management data. By spatial error, we mean that we obtain an error metric per management attribute that summarizes its deviation from the real value of the manageable attribute. The temporal error acts as an error metric for each time slot (a monitoring round, for example) summarizing the time deviation from that instant while acting over a management attribute (collect, notify, update). We note, that a fundamental trade-off exists between the three categories of metrics. These primary metrics need to be extended when they are measured in a particular monitoring scenario involving one to one management entities or many management entities. Therefore, we define two types of qualifiers to be used of these metrics according to the involved monitoring scenario. These qualifiers are the following:

• One-to-one: collects the measured metric vector in a manager to an agent scenario; • One-to-many: collects the set of singleton measurement between a manager and many agents; When the monitoring scenario involves a manager, an intermediate and an agent, the measured metrics needs to be decomposed to quantify the per-entity contribution to the endto-end performance.

The second category of metrics are qualified as derived metrics. These derived metrics are computed from the measured primary metrics. The motivation behind the definition of these metrics is to quantify the efficiency of a monitoring approach within a single quantity in order to be able to compare them. The efficiency metric of a monitoring application puts in relation the three types of primary metrics that we have defined above. Thus, we obtain the following formulae:

G(k) = R(k) C(k) xQ(k) (1)
Where G(k) is the efficiency quantity computed under a performance factor that takes a value denoted by k. R(k) denotes a speed metric quantity, C(k) denotes a cost metric quantity and Q(k) denotes a quality metric quantity. For example, to compute this efforts we can use the throughput of a monitoring system in terms of the number of collected attributes per second as a speed metric, the cost of management activities in terms of resource consumption (CPU, memory and network bandwidth) as a cost metric and the quality of monitoring operations in terms of their respect of a tolerable delay.

The definition of the efficiency derived metric will allow us to define more metrics that cover monitoring challenges which are the scalability and the impact of a monitoring application. The scalability metric will be detailed in section V-B and the impact of a monitoring activity on the performance of the managed system will be detailed in section V-A.

B. A Benchmarking platform

To better understand the performance of management frameworks, we have developed a benchmarking platform [START_REF] Andrey | A jmx benchmark[END_REF] dedicated to JMX management paradigms. We selected this framework because it provides inherent manageability to Java technology enabled applications and services, therefore, the impact of management activities is more visible. The JMX framework has not a predefined management information model, instead it offers several types of managed objects (MBeans) to instrument resources and supports many protocols (TCP/RMI, HTTP, SOAP) for the communication between a manager and an agent. Nevertheless, our platform is flexible and modular enough to be extended to other management frameworks like SNMP. We have focused on the manager-agent model with synthetic tests where we have varied the number of managed objects (MBeans), their types, the monitoring rates and the number of agents. Despite the high saleability of benchmarking results [START_REF] Jain | The art of Computer Systems Performance Analysis[END_REF] which is its key justification, our experience on the benchmarking of the JMX framework, shows us how much this technique is time consuming and error-prone. It also reveals its limited coverage of the space of performance factors values. For example, in [START_REF] Lahmadi | On the impact of management instrumentation models on web server performance: a jmx case study[END_REF], a complete coverage of all our measurement series to identify the impact of the three integration models of an agent within a managed systems (as depicted in Figure 3) needed 3 months of measurement, or 1 month at best if we parallelize measurements. Thus, we believe that analytical and simulation techniques are more suitable to investigate deeper the performance of management frameworks. However, before doing simulation we need to model the behaviour of management activities.

V. RESULTS

In this section, we present the main results that we have obtained by using our benchmarking platform and the set of performance metrics that we have defined.

A. Monitoring Impact

We have built a derived metric to compute the impact of a monitoring activity on the performance of the managed system. This metric has implications on network management models like the SNMP, as well as on services and applications management frameworks.

To develop this metric, we used the efficiency metric described by equation 1. We denote by E(k) the productivity of a managed system. The productivity is defined as:

E(k) = F(k) F(k) + G(k) (2)
Where F(k) is the managed system efficiency defined in the same manner as the monitoring efficiency G(k) and computed by equation 1. This productivity captures the ratio between the functional efficiency of a managed system defined by the real work that offers to the users and the real efficiency that it needs to accomplish this work. Therefore, we define the Management Impact Metric as follows.

MIM(k 0 , k) = 1 - E(k) E(k 0 ∈ [0, 1] (3)
Where k denotes an impact factor that can be the number of monitoring requests. In [START_REF] Lahmadi | On the impact of management on the performance of a managed system: a jmx-based management case study[END_REF], we illustrated this metric on the monitoring of a managed J2EE server1 JBoss using the JMX framework. In this case, we varied the monitoring rate as an impact factor. In a second stage, we have extended this initial work to study the impact of instrumentation models as described in [START_REF] Kreger | Java and JMX: Building Manageable Systems[END_REF] on both the performance of the management and an instrumented web server [START_REF] Lahmadi | On the impact of management instrumentation models on web server performance: a jmx case study[END_REF]. We did show that the users perceived performance in terms of the number of HTTP transactions/s and their respective delays are highly affected by the management activity in the boot driver and component models while a daemon integration model limits the management activities impact on the functional plane. However, we showed that under low monitoring rates in the order of 200 requests/second, the three integration models have a small impact on the web server performance. Figure 3 depicts the throughput and the delays of the web server, monitored by a driver agent model. In this experiment, we varied the operational load in terms of number of web clients and the monitoring load in terms of getAttributes per second.

B. Monitoring Scalability Assessment

The development of a monitoring efficiency metric that we described in section IV-A allowed us to the define a unique metric to assess the scalability of a monitoring application. This metric relies on the one proposed in [START_REF] Jogalekar | Evaluating the scalability of distributed systems[END_REF] to quantify the scalability of distributed systems. Therefore, we define the scalability degree Ψ(k 1 , k 2 ) of a monitoring as follows.

Ψ(k 1 , k 2 ) = G(k 2 ) G k 1 ( 4
)
where k 1 and k 2 are two different values of a scale factor. Generally, k 1 is fixed to an appropriate value to compute a baseline efficiency of a monitoring application. The value of k 2 will vary and the scalability degree is computed to capture the degradation or enhancement of the monitoring application scalability. A monitoring application scales well when

Ψ(k 1 , k 2 ) ≥ 1 or Ψ(k 1 , k 2 ) ≈ 1.
In [START_REF] Festor | évaluation du passage l'échelle des systèmes de gestion : métriques et modèles[END_REF], we have illustrated this metric on the scalability assessment of a centralized monitoring application involving one manager and many agents. We studied its scalability degree analytically by using the performance model developed in [START_REF] Chen | A model and evaluation of distributed network managemetn approaches[END_REF]. We found analytically that the scalability limit is mainly influenced by the monitoring delays. Thus, if the delays remain constant, the scalability degree stays close to 1. We also found that network transit delays when they become higher, contribute to enhance the scalability of such an approach. When the number of agents becomes high, the network will delay the responses from the agents and decrease the load on manager. However, this network transit delay has to be bound to keep an acceptable monitoring quality, mainly the timeliness. Figure 4 depicts the scalability degree of a JMX-based monitoring application while we varied the number of agents between 70 and 700 agents by a step of 70. We observe that the scalability limit of the monitoring application is close to 350 agents that corresponds to a scale factor of 5.

C. Monitoring Delays Characterisation

When assessing the impact and the scalability of a monitoring application, the primary key metric is the monitoring delays. We define the monitoring delay as the delay that experiences a monitoring attribute within an activity, to retrieve/alter its value. This delay is measured using the one-toone, one-to-many metrics defined in section IV-A. Based on the developed benchmarking platform, we have elaborated in [START_REF] Lahmadi | On delays in management frameworks: Metrics, models and analysis[END_REF], a delay model for the manager-agent paradigm. We have collected measurements within two main scenarios. The first scenario use a single manager/single agent setup, where we used the one-to-one delay metric. The second scenario uses a single manager/multiple agents setup, where we used the one-to-many metric. For both scenarios, we applied statistical analysis on the collected data sets to identify their underlying statistical distributions. In the first scenario, we varied the monitoring rate between 1 and 1000 polled attributes per second. We find that the one-to-one delay closely follows a LogNormal distribution. In the second scenario, we varied the number of agents between 70 and 700 agents. We found that the one-to-many delays closely follow a normal distribution with a small number of agents, and becomes more heavy tailed and approximates a Weibull distribution with a considerable number of agents.

The delays scaling behaviour is interesting because we can quantify the temporal accuracy of management data collected by a manager to a somewhat delay tolerance. This parameter is considered as an upper bound on the delay. Any collected attribute from an agent experiences delays, that hold with high probability and is determined empirically based on the managed environment. Figure 5 depicts the measured and predicted timeliness of a polling activity involving one manager and 420 agents. The timeliness is defined as the fraction of agents that respond with delays lower than the monitoring interval which is equal to 1 second in this case. We observe that the measured and predicted timeliness are close to 0.5 which means that half of the agents respond with delays greater than 1 second. The methodology uses to model management delays can be applied to other management related performance metrics such as the number of polled/notified attributes. After the identification of the statistical distribution of monitoring delays within JMX-based applications, we used the empirically obtained models to simulate the effect of these delays on the behavior of the monitoring algorithm running on the manager node. We developed Matlab scripts to simulate a centralized monitoring algorithm behavior by generating synthetic delays from a Weibull distribution. The parameters of the distribution are obtained from our measurements as described above. We used a monitoring algorithm that computes an aggregation function using single values from many agents. This algorithm is similar to the simple-rate algorithm described in [START_REF] Dilman | Efficient reactive monitoring[END_REF]. For each simulation, we fix the number of agents, we generate the synthetic delays and we compute the real aggregation function from the values remaining on the agents and the observed one computed on the manager. We find that the discrepancy between the observed aggregation function computed on the manager and the real one computed without monitoring delays is very important when the number of agents is close to 700. Thus, the monitoring view on the manager experiences a temporal distortion when the number of agents increases. This is due to the monitoring delays. Figure 6 depicts the result of the simulation of the monitoring algorithm with a number of agents equal to 700 and the synthetic delays generated from a Weibull distribution.

VI. CONCLUSION AND FUTURE WORKS

The aim of our work is to provide a common evaluation methodology and performance assessment metrics of a management framework. For this purpose, we defined a set of primary metrics to capture the speed, the cost and the quality of management operations. We designed and developed a benchmarking platform to measure these metrics within realistic and unrealistic scenarios. We illustrated our work with the manager-agent JMX-based management framework, widely used to manage J2EE applications on which several Internet services rely (retailers, banks, government institutes,etc). Our findings show that the management activities have a deep impact on the performance of a managed service if they are optimized. This impact depends on the monitoring rate and the integration model of the agent within the managed system. We also show that high monitoring rates degrade both the performance of management and managed systems, especially delays that become more random and their underlying statistical distribution more heavy-tailed. Thus, we believe that optimizing management activities by minimizing their cost and rates while maximizing their coverage and business benefit needs to be fitted and well defined within solid optimization frameworks.

Figure 1 .

 1 Figure 1. Definition space of primary performance metrics dedicated to monitoring frameworks evaluation.

Figure 2 .

 2 Figure 2. Architecture of the JMX-based instance of our benchmarking platform.

Figure 3 .

 3 Figure 3. The impact of a driver model of a monitoring agent on a web server performance in terms of (a) throughput and (b) HTTP transactions delays.

Figure 4 .

 4 Figure 4. Scalability assessment of the manager-many agents model, where we varied the number of agents as a scale factor between 70 to 700.

Figure 5 .

 5 Figure 5. Predicted and measured timeliness of a polling activity with one manager and 420 agents.

Figure 6 .

 6 Figure 6. Effect of delays on the behavior of a monitoring algorithm with 700 agents attached to the manager in a polling mode interaction.

We used JBoss as managed server: http://www.jboss.org


Abdelkader Lahmadi
email: abdelkader.lahmadi@loria.fr

Laurent Andrey

Olivier Festor

Design and Validation of an Analytical Model to Evaluate Monitoring Frameworks Limits

Keywords: Monitoring, Scalability, Modeling techniques

It is essential that a monitoring system is being designed with performance and scalability in mind. But due to the diversity and complexity of both the monitoring and the monitored systems, it is currently difficult to reason on both performance and scalability using ad hoc techniques. Thus, both simulation is required and analytical models based on well established techniques such as queueing theory have to be developed. In this paper we provide an analytical modeling of the behaviour of the commonly used manager-agent monitoring frameworks within two scenarios: single manager-single agent and single manager-multiple agents. The two designed models enable the automation of the estimation of the scalability limit of the two types of management monitoring schemes regarding a performance metric like the monitoring delay. We validate our developed models through simulation based on parameters values obtained from the performance measurement of a life running JMX-based monitoring applications with the two scenarios.

I. INTRODUCTION

Despite the increasing use of on site measurement techniques, analytical models still play an important and complementary role in the performance evaluation of communication networks and protocols, as they are often more rigorously defined and provide an elegant method to obtain performance evaluation and results. The increase in both size and requirements on management systems which have to deal with more managed systems and management data while at the same time being able to maintain an accurate and near real time vision of the managed systems puts high pressure on the monitoring part and algorithms of the management system. In this context, modeling of monitoring frameworks needs more attention and the development of queueing based model becomes crucial to both assess their performance and verify results obtained from measurement. Such models are also very important to address the following problems:

(i) managed system capacity provisioning, which enables a managed system to determine how much capacity to allocate to monitoring activities in order to still be managed; (ii) management cost identification, which enables the resource consumption to be determined for a given management configuration; (iii) monitoring system tuning, which enables management system bottlenecks to be identified for the purpose of tuning.

However, modeling management frameworks is not trivial for the following reasons. First, various management frameworks have vastly different organisational, communication and information models, underlying protocols and technologies with different performance characteristics. Furthermore, in a management system : (i) there may be resources constraints at the agent side, usually integrated in different ways on the managed system; (ii) management data caching, filtering and aggregation [1] may be performed directly at by agent which complicates the performance modeling; (iii) real management workload has different time scales, where some requests need small time scales, and others are less frequent with larger time scales.

Finally, the management community has a lack of input on workloads that management systems experience in real life. Management of networks and services is usually achieved through the use of a management framework consisting of a management algorithm running on different entities and using an underlying management protocol. Each entity provides a certain management facility to its preceding entity and uses the facility provided by its successor to carry out its part of the overall management task. For instance, a typical management framework, consists of three entities-a manager that is responsible for triggering management requests initiated by human beings (administrator) operations or a hard and software modules running a management algorithm; an agent that implements managed objects with their derived attributes, operations and notifications. The agent is also responsible to fetch management data from the managed system, respond to the manager requests and issues notifications when a threshold is crossed. The third entity is the real managed system that provides the primary functions to overcome service's users requirements [START_REF] Pras | Network management architecture[END_REF]. In such a management framework, incoming management operations trigger requests, processed by the agent, and fetch management data from the managed system. In this paper, we developed models based on closed network of queues for a manager-agent pattern, where each queue represents one monitoring entity. Although, management frameworks may be different, they are constructed using this pattern as a subsystem. Thus, a key contribution of this work is that a complex model of a management framework is reduced to model management requests handling at the individual entities and across them. We validate our developed mathematical formula using queueing simulation where parameters values are obtained from performance measurement of a Java Management eXtension (JMX) [START_REF] Sun | Java T M management extensions, instrumentation and agent specification[END_REF] based management platform to monitor a Java based web server.

The remainder of this paper is structured as follows. Section II provides an overview of related works, mainly those where some analytical models are proposed to assess the performance of monitoring systems. Section III depicts some properties of monitoring frameworks that impact their scalability. In section IV, we develop our analytical models for the main scenarios : single manager-single agent and single manager-multiple agents. We also provide an analysis of these models. In section V, we provide simulation results that verify the developed mathematical formula. Section VII presents our conclusion.

II. RELATED WORK

The main distinction of our case study is the performance and scalability evaluation of monitoring systems. The techniques used in this study have benefited from the literature on systems performance evaluation (e.g., Jain [START_REF] Jain | The art of Computer Systems Performance Analysis[END_REF] and Gunter et al. [START_REF] Bolch | Queueing networks and Markov chains: modeling and performance evaluation with computer science applications[END_REF]).

In the network and services management community rigorous analytical models are rare. To our knowledge, only the following works have provided queueing models dedicated to management frameworks so far. In [START_REF] Stamatelopoulos | Performance and efficiency in distributed enterprise management[END_REF], the authors propose a queueing model of a manager within an Enterprise management system. In [START_REF] Pattinson | A study of the behaviour of the simple network management protocol[END_REF], the authors use some queueing modeling features to simulate the behaviour of an SNMPv1 agent. The thesis of Subramanyan [START_REF] Subramanyan | Scalable snmp-based monitoring systems for network computing[END_REF] contains a model of the behaviour of a hierarchical SNMP based management platform using queueing networks. In [START_REF] Chen | A model and evaluation of distributed network managemetn approaches[END_REF], authors provide some formulas to assess the performance of different management approaches (centralized, distributed and mobile agents). These mathematical results are empirical rather than developed from well defined modeling approaches.

Performance modeling of Internet services and applications is well studied in other disciplines for other puroposes [START_REF] Urgaonkar | An analytical model for multi-tier internet services and its applications[END_REF], [START_REF] Diao | Stochastic modeling of lotus notes with a queueing model[END_REF] but these models usually omit the monitoring part from their models despite the fact that the quality of a service derived from performance quantities is driven by the monitoring part and its impact is not negligible. Thus, coupling services and monitoring performance models is a point of interest to provide service quality requirements. The very recent work of Beitgang [START_REF] Breitgand | Using the right amount of monitoring in adaptive load sharing[END_REF] does such a coupling. There, authors assess the impact of monitoring with a load sharing strategy on servers ressources. Our work follows a similar path to provide well defined monitoring performance models that will be usefull to reliably optimize monitoring systems to fit their monitored environments mainly large scales ones.

III. MONITORING FRAMEWORKS MODELS

In our work, we focus only on the manager-agent pattern since it is the most used and any traditional monitoring framework can be seen as a composition of this pattern. Recent management frameworks such as autonomic management still relies on this pattern where an autonomic entity is a local loop of a manager and an agent. The question that arises when using a monitoring framework with this pattern is related to it's scalability limit while increasing a scale factor. We want to be able to predict this limit with regard to monitoring delays. The factors that affect this delay are mainly the number of agents, managed devices, monitoring variables and their monitoring rates. This limit defines the range of values of a scale factor that guarantees a suitable delay with respect to a tolerance value. We study limit prediction models for two monitoring scenarios which are the single manager-single agent and single manager-multiple agents. Figure 1 depicts their monitoring models, where management variables are collected by a builtin algorithm on the manager that connects to a single or to many agents to retrieve variables values. We consider a periodic polling workload issued by an implicit monitoring algorithm [START_REF] Pras | Network management architecture[END_REF] on the manager. This type of workload is interactive since each polled variable issues a monitoring operation when its monitoring interval arises.

The performance of monitoring has to be characterised by some suitable metrics. As stated in [START_REF] Lahmadi | On delays in management frameworks: Metrics, models and analysis[END_REF], the delay that monitoring calls experience is a good metric for a monitoring algorithm performance. The attribute delay is defined as the time experienced by a monitoring attribute to retrieve its value from an agent to a manager.

A. Monitoring workload model

Monitoring traffic generated either with polling or notification activity can be modeled as an ON/OFF process where a monitoring variable alternates between ON (busy) and OFF (idle) states. This is depicted in Figure 2.

A monitoring variable enters the ON state, when it attempts to retrieve its value by calling a monitoring operation. This operation needs one or more monitoring protocol messages interlevead by the ITM (Inter-Message) times. Each message may carries one or more values of a monitoring variable. Therefore, the duration of the ON period depends on the variable granularity (simple or tabular), the network conditions and monitoring protocol operations (logical or single). After all values of the monitoring variable are fetched, the variable goes into the OFF state. The duration of ON state and OFF state correspond to monitoring delay and monitoring interval, respectively. OFF time denotes any time that the variable is inactive. ON time is the time taken to fetch all values of a monitoring variable in a monitoring operation. The ON state can be split into successive monitoring messages used to deliver variable's values.

IV. ANALYTICAL MODELS

We have developed a simple queueing network model that applies to a manager agent pattern shared by an interactive set of monitoring variables with the ON/OFF activity model described above. We consider, in this work, closed queueing networks where the number of monitoring variables in the system is always constant. Each queue within the model represents a monitoring entity, i.e. the monitoring algorithm, the manager and the agent. Here, we assume an implicit monitoring algorithm running on the manager using polling operations. Each monitoring variable incurs a single monitoring protocol message to carry its value from the agent.

Here, we consider a finite population of M interactive monitoring variables with ON/OFF traffic, which are sharing the manager queue to request their values from one or many agents. We assume that the M variables OFF times distribution fits an exponential distribution whose mean is ∆. Herein, we assume a naive monitoring algorithm where monitoring calls of different variables occurs continuously and independently at a constant rate. We model the monitoring algorithm, that fetches variables values, as an infinite server (IS). The respective service times of a monitoring message (which is a component of the ON times) have a general distribution whose mean is S. Figure 3(a) depicts the single manager-single agent scenario modeled with a closed queueing network. This monitoring pattern fits perfectly to the well etablished machine repair model [START_REF] Bolch | Queueing networks and Markov chains: modeling and performance evaluation with computer science applications[END_REF]. The figure 3(b) depicts the single manager-multiple agents scenario modeled with a closed queueing network model. The latter maps to to the well known central server model [START_REF] Buzen | Queueing network models of multiprogramming[END_REF]. We assume processorsharing queues, and our results hold for any general service time distribution as long as the monitoring intervals (think times) have an exponential distribution. We have a closed-form solution for the two models and we can describe the delays with simple parameters. In this work, our model analysis is based on the use of Little's Law [START_REF] Bolch | Queueing networks and Markov chains: modeling and performance evaluation with computer science applications[END_REF]. This law states that the response time of a queue is equal to the mean number of jobs within the queue to its throughput. If one denotes by K the queue length, γ the throughput and T is the response time, then Little's law is given by :

T = K γ (1)
The performance model parameters are computed using the Mean Value Analysis (MVA) technique [START_REF] Bolch | Queueing networks and Markov chains: modeling and performance evaluation with computer science applications[END_REF]. This method is based on two fundamental equations and it allows us to compute the mean values of measures of interest such as mean waiting times, throughput, and the mean number of jobs at each node. It is based on Little's law and the theorem of the distribution at the arrival time (in short, arrival theorem) for closed product-form networks. This technique allows us to identify scalability limits for each scenario with respect to a scale factor variation. It provides mean values of metrics rather than a full distribution.

Below is a list of parameters used for expected response times using the selected queueing models:

• M : the number of monitoring attributes polled by an algorithm residing on the manager.

• N : the number of agents attached to the single manager.

• ∆ : the mean monitoring interval of monitoring attributes.

• R : the response time from a queueing theory view that represents monitoring delays.

• γ : the throughput in terms of completed jobs. In our context it represents the number of polled attributes.

• µ i : service rate at node i (manager or agent) • S i : service time at node i.

• e i : the visit ratio of a node i.

• D i : service demand D i = e i × S i at node i. We use closed queueing networks with an interactive traffic (ON/OFF model). The response time T in Little's law, expressed by equation 1, represents the total cycle time. We thus obtain: T = R + ∆. The mean number K of customers within the closed queueing network is constant, hence we obtain K = M . Little's law can thus be transformed as follows:

R = M γ -∆ (2)
We use the above formula to estimate the monitoring delay R with the two monitoring scenarios. Following, we denote by the symbol SS the single manager-single agent model and by the symbol SM the single manager-multiple agents model.

A. Single manager-single agent model analysis

We begin by considering the monitoring delay of a single agent as depicted in Figure 3(a). The monitoring delay R SS computed from 2 is given by:

R SS (M ) = M γ SS (M ) -∆ SS (3)
We observe that the scalability limit regarding delays is the number of monitoring attributes M and their monitoring times ∆. Thus when M increases or ∆ decreases, the response times increase. A monitoring system needs to provide a suitable delay to guarantee a good timeliness of polled attributes. Consequently, we need to bound monitoring delays to some threshold value. In this case, we take the monitoring interval ∆ as this threshold.

We will now assess the bound on the number M of variable that provides bounded delays to the monitoring interval ∆. Let respectively D max , D sum and D avg be the maximum, respectively the sum and the average of service demand by queue of all non IS-node. We have :

D max = max(D i ), D sum = D i ( 4
)
where i ∈ M anager, agent and D avg = D sum

M

From the balanced job bound analysis [START_REF] Bolch | Queueing networks and Markov chains: modeling and performance evaluation with computer science applications[END_REF] we have the upper bound of the throughput denoted by

γ max = min( 1 D max , M ∆ + R min ) (5)
where

R min = max(M × D max -∆, D sum + ((M -1) × D avg × D sum D sum +∆ )) (6)
is the lower bound of response time. The highest possible overall throughout is restricted by each node in the network especially by the bottleneck nodes. Thus, we get γ SS (M ) ≤ γ max . From the model in Figure 3(a), the visit ratios are given by : e A = 1 and e M = 2. At first we determine that :

γ max = 1 D max , where D max = max{e M × S M , e A × S A } = 2 × S M .
herein, we observe that the bottleneck node is the manager. Using the upper bound ∆ on the response time R SS (M ), we get as a bound on the number M of monitoring attributes :

M * ≤ ∆ S M (7)

B. Single manager-many agents model analysis

We use the same methodology as above to determine the monitor in delays (i.e. those for which values where received in time) within the model depicted in Figure 3(b). We assume that each Monitoring attribute generates N jobs to retrieve its value from the N agents. We assume also that jobs are asynchronous and do not account for any synchronisation between them. Thus, the monitoring delay R SM computed from 2 is given by :

R SM (M, N ) = M × N γ SM (M, N ) -∆ SM (8)
We observe that the scalability limits are the number of monitoring attributes, the number of agents and the monitoring times ∆ SM . As the first scenario, we would like to predict the number of agents that provide a monitoring delay less or equal to the monitoring time. We determine that the throughput γ SM ≤ 1 (N +1)×S M since the visit ratio of the Manager is equal to N +1. Given equation 8, the bound on the number of agents N * is given by :

N * ≤ 2 × ∆ × γ max M (9)
V. SIMULATION

We have simulated the queueing models that we propose in section IV for the polling based manager-agent model with the single manager-single agent scenario and the single manager-multiple agents one. We developed a Matlab script that implements the Mean Value Analysis (MVA) algorithm to compute our performance metrics mainly the monitoring delays while varying a scalability factor.

The simulation input parameters, mainly the service rates for the manager and agents are obtained from the measurement of a running JMX based manager-agent pattern application monitoring a web server. Table V summarizes the input parameters for the queueing model that we consider.

We note that all measurements of service times are obtained from low monitoring rates (1 request/second) to avoid measuring the queueing latency in addition to the service times. This method has been successfully employed in similar problems [START_REF] Urgaonkar | An analytical model for multi-tier internet services and its applications[END_REF].

A. Results

First we provide results of the single manager-single agent scenario where we have varied the number of monitoring variables M from 1 to 1000 with a mean monitoring interval ∆ = 1 second. Using the parameters in table V and the equation 7, we obtain as theoretical bound of attributes M * close to 877. In this case, delays are less or equal to 1 second (monitoring interval). As depicted in Figure 4, we find that simulation results give the same value as the mathematical formula 7. Secondly, we want to predict the maximal number of agents N * within the single manager-multiple agents scenario and compare it with simulation results. Using values of table V, we observe that γ max = 1 (N +1)×S M . We transform the equation 9 and we obtain:

N * ≤ 1 2 8 × ∆ M × S M + 1 -1 (10)
In the simulation of a single manager-multiple agents scenario, we fixed M = 1 and ∆ SM = 1 and we varied the number of agents N. Using equation 10 and values from the table V, we obtain the optimal number of agents that guarantee a monitoring delay less or equal to 1 second close to N * = 41 agents. As depicted in Figure 5(a), we observe that the theoretical result corresponds to the value obtained using simulation.

VI. DISCUSSION

Our analysis confirms an overall intiuition that the single manager-single agent has a better scalability limit regarding the number of attributes than the single manager-multiple agents regarding an increase in the number of agents. According to our mathematical formulation, designing a monitoring system with the single manager-single agent pattern as a management entity like an autonomic entity is more efficient than traditional central manager with many agents. Despite that this result seems to be trivial from an empirical manner, in this work we validate it in a more rigorous analytical manner.

Our models are simple and usefull. Their underlying assumptions appear however to be heavy and don't reflect all real world deployed monitoring applications. First, we consider in this work a concurrency level of the monitoring algorithm equal to 1, however main monitoring algorithms have a concurrency level greater than one. Usually they rely on a multithreading approach to fetch variables values. Second, we considered in the modeling phase that monitoring events follow a Poisson process with an exponential distributed monitoring intervals. This assumption needs to be validated from real work monitoring traffic traces. Recent work in [START_REF] Schönwälder | SNMP traffic analysis: Approaches, tools, and first results[END_REF] has started traffic analysis of real world monitoring applications. Their data sets will be very useful to identify statistical assumptions of queueing models of monitoring applications.

VII. CONCLUSION

In this paper, we respond to the following question: "how to predict the scalability of a monitoring application following the manager-agent pattern?" By using simple analytical models based on the well developped queueing theory. To this end we have developped two analytical models for two major scenarios: the single manager-single agent and the single manager-multiple agents. The performance metric of interest is the delay that experiences a manager to retrieve the values of monitoring variables from a set of agents. This delay is important to collect accurate data that reflects a real time view of the monitored system or when monitored attributes are short live like SIP 1 transactions for example. Our aim is to predict the scalability limits of each scenarios with regard to delays when the number of agents and attributes increases. We have shown that our analytical models are in line with simulations. We also assessed that the single manager-single agent scales better than the second with respect to their scale factors.

While the model is performant, it needs to be enhanced to support other monitoring features among which the concurrency level is the most important one, especially in the single manager-multiple agents scenario where attributes are in some case retrieved in parallel from agents. This will be the main objective of our future work in this modeling activity. A second activity will be to apply the model to SNMP-based systems. This mainly consists in measuring the parameters needed to feed the model from real world systems. This latter activity is pursued in the context of the FP6-IST EMANICS Network of Excellence.

Fig. 1 .

 1 Fig. 1. Fundamental topologies of traditional monitoring frameworks. (a) single manager-single agent.(b) single manager-many agents

Fig. 2 .

 2 Fig. 2. ON/OFF model for a monitoring variable workload within (a) polling activity and (b) notification activity.

Fig. 3 .

 3 Fig. 3. queueing models for two monitoring scenarios: (a) single manager-single agent, (b) single manager-many agents.

Fig. 4 .

 4 Fig. 4. Monitoring delays of the single manager-single agent scenario as a function of the number of polled attributes

Fig. 5 .

 5 Fig. 5. Performance of the single manager-multiple agents scenario as a function of the number of agents. (a) monitoring delays. (b) total throughput in terms of attributes/s.

TABLE I MODEL

 I INPUT PARAMETERS FOR SIMULATION RESULTS.