Sapan Bhatia
email: sbhatia@cc.gatech.edu

Charles Consel
email: charles.consel@inria.fr

Julia Lawall

Minimizing Cache Misses in an Event-driven Network Server: A Case Study of TUX

We analyze the performance of CPU-bound network servers and demonstrate experimentally that the degradation in the performance of these servers under highconcurrency workloads is largely due to inefficient use of the hardware caches. We then describe an approach to speeding up event-driven network servers by optimizing their use of the L2 CPU cache in the context of the TUX web server, known for its robustness to heavy load. Our approach is based on a novel cache-aware memory allocator and a specific scheduling strategy that together ensure that the total working data set of the server stays in the L2 cache. Experiments show that under high concurrency, our optimizations improve the throughput of TUX by up to 40% and the number of requests serviced at the time of failure by 21%.

Introduction

To make information services readily available at the scale at which they are sought on the Internet, network servers need to be scalable. Two main factors contribute to this requirement. First, server loads are known to be highly variable, and can increase manifold with a sudden increase in the popularity of the served resources, stressing servers unexpectedly. The Slashdot effect [START_REF] Wald | The 1999 Southern California Seismic Network Bulletin[END_REF] on web servers is a well-known example of this phenomenon. Second, recent advances in signal transmission media have achieved high data-transmission rates with low error rates, shifting the performance bottleneck from the physical media to the network server itself. This makes it essential for servers to be optimized for performance, and programmed so as to leverage the increasing computational bandwidth of the underlying hardware to achieve scalability.

When the amount of data manipulated by the server is large, its throughput is limited by I/O activity such as disk reads. Even though such servers often perform aggressive caching, the amount of data requested concurrently is so large and varied that some amount of disk activity is inevitable. Much research has been done to improve the scal-ability of such servers. Event-driven programming with some degree of explicit support to make for scalable I/O operations [START_REF] Pai | Flash: An Efficient and Portable Web Server[END_REF][START_REF] Elmeleegy | Lazy asynchronous I/O for eventdriven servers[END_REF] has emerged as the paradigm believed to be the most robust to heavy loads because of its low OS overhead and offering of flexible scheduling.

With the increase in the size of the main memory available on server systems, the contribution of I/O operations to the request processing overhead of servers has diminished considerably. This phenomenon has led to the emergence of a second category of servers with distinct performance characteristics. Specifically, workloads on such servers involve resources that fit completely in the main memory of the server. These servers are referred to as CPU-bound network servers.

In this paper, we consider the performance of CPUbound network servers, in which the contribution of I/O operations to the average processing time of requests can be neglected. In practice, many common configurations of web, telephony and other servers fall into this category. A common feature of such servers is fact that the request traffic they receive is non-uniform, with marked load spikes triggered by specific real-world events. 1 Heavy workloads on such servers refer to data that not only remain in core, but is often very small in size, consisting of a small set of highly solicited files or records. Also, some categories of servers such as telephony servers [START_REF]The Internet Engineering Task Force, 1895 Preston White Drive, Suite 100[END_REF] inherently access relatively small amounts of data to perform their function.

To be scalable and efficiently handle heavy loads, CPUbound network servers must effectively exploit data locality. This is becoming especially important with the increasing disparity between processor speeds and memory access times. Although this importance has been emphasized in the past [START_REF] Ruan | Evaluating the impact of simultaneous multithreading on network servers using real hardware[END_REF], little is done in present-day network servers to expressly avoid memory accesses. Set-associative caches and compiler-enabled cache optimizations [START_REF] Larus | Using cohort scheduling to enhance server performance[END_REF] reduce cache collisions in contemporaneously accessed data to a certain extent. However, under heavy loads, these optimizations are often rendered ineffectual. Increasing the number of concurrent requests also increases the amount of data treated by the server at a time, generating a high rate of data-cache misses.

We perform a workload-oriented optimization of CPUbound network servers, favoring in-core workloads. Using a cache-aware memory allocator and a specific scheduling strategy, we try to ensure that the current working set of the server remains in the L2 cache. If it overflows, then we ensure that it does so gracefully. Our approach does not impact the behavior of the server with respect to I/Obound workloads. Furthermore, since increasing the size of the served resources moves the bottleneck of operation towards the underlying transport layer and the network, the throughput benefits of our approach are best observed for workloads requiring access to small amounts of data.

The cache optimizations described in this paper are optimistic, and are guided by code annotations provided by the server programmer. The annotations guide a kit of program analysis and transformation tools that we have developed. These tools parse and analyze event-driven servers written in the C language, and modify their memory management behavior to use our optimization strategy. They also generate program-specific library functions that can be used to modify the scheduling behavior of the application. We have used these tools to optimize the TUX web server. 2 TUX is widely regarded to be a high-performing Web server [START_REF]The SPECWeb99 Benchmark[END_REF], and thus, speedups in TUX are a strong indication of the effectiveness of our approach.

The design and implementation of the TUX web server is used to describe caching problems in servers as well as to illustrate our approach in detail in the rest of the paper. The remaining sections of the paper are organized as follows: Section 2 explains slow-downs in servers caused due to cache inefficiencies. Section 3 gives an overview of our optimization approach. Section 4 presents a case-study in which we apply our approach to TUX. Section 5 evaluates our approach experimentally. Section 6 presents related work, and finally, Section 7 concludes.

Cache-Related Slowdowns

Cache usage is said to be poor when cached items are evicted regularly prior to being used, resulting in frequent memory accesses. Poor cache usage is observed in two main situations that result in data and instruction cache misses respectively. These situations are discussed below.

Data-cache misses due to an explosion in total perrequest data. When the amount of per-request data exceeds a certain threshold, the total live data set at certain stages can no longer be accommodated in the hardware caches, causing the eviction of live cache items. Such situations are most commonly observed in high-concurrency workloads, which increase the volume of per-request data by virtue of an increased number of concurrent requests.

Figure 1 illustrates this behavior for an unmodified TUX web server running on Linux 2.6.7. 3 We find that through-2 TUX is commercially distributed as the Redhat content accelerator 3 Our experiments with other servers (e.g. thttpd, boa, mathopd and put degrades and L2 data cache misses increase steadily as concurrency increases, up to about 4000 concurrent connections. From here on, performance continues to degrade even though L2 cache misses decrease sharply. We attribute the former effect to an explosion in the amount of per-request data and the latter to the fact that the server is close to overload. When the server is overloaded, it does not have sufficient resources to accept new incoming connections, causing incoming connections to either remain incomplete or be rejected [START_REF] Banga | Better operating system features for faster network servers[END_REF]. This reduces the number of concurrent requests and hence the amount of per-request data as well.

Instruction-cache misses due to request scattering. Servers inherently benefit from the instruction cache by virtue of executing the same instructions repetitively in different request contexts. When a batch of instructions is applied to consecutive incoming requests, the second and consecutive applications can be expected to yield instruction cache hits. Very often, though, batches are received at intervals of time that are shorter than the time of treatment of a single request but longer than that of individual stages. This causes requests to be scattered in the processing graph of the server, resulting in competition amongst the stages involved for the instruction cache.

Such an increase in the number of i-cache misses being used can be observed in the leftmost part of Figure 1(b). As concurrency decreases below 40, we find that throughput drops (and response time increases), as low overall concurrency implies low stage concurrency. One prominent indi-Apache) have yielded results that are comparable to those in TUX. cation of this effect is the decreasing number of bus transactions required for instruction fetches (i.e., L2 instruction cache misses), as concurrency increases. Thus, although the total L2 cache misses increase, the decrease in L2 i-cache misses for concurrency less than 40 compensates for them, and there is an improvement in throughput. Exceptionally, for concurrency less than 10, we observe low performance in spite of few cache misses. The apparently low performance during this phase results from the fact that the CPU and caches are under-utilized, and the load is far too low to exercise the full computational bandwidth of the server.

We conclude from these two contradicting considerations, that the strategy used to manage requests in a server must make a tradeoff between these two effects. In particular, throttling the number of requests treated at various stages to avoid an explosion in per-request data must be balanced with accumulating requests to favor the instruction cache. Since the degradation in performance due to the former effect is the dominant one, our strategy revolves around trying to eliminate data-cache misses, while at the same time reconciling the instruction-cache criterion.

Cache Optimizing CPU-Bound Servers

In this section, we present our approach to use a cacheaware memory allocator, Stingy, in coordination with a specific scheduling strategy to optimize an event-driven network server. We will describe its application to TUX in much greater detail and with concrete examples in the following section. A presentation of this methodology from the programming-language point of view, focusing on the program analyses involved can be found elsewhere [START_REF] Bhatia | Memorymanager/scheduler co-design: Optimizing eventdriven servers to improve cache behavior[END_REF].

In a nutshell

Although servers manipulate very specific data, typical global optimizations [START_REF] Bonwick | The slab allocator: An object-caching kernel memory allocator[END_REF][START_REF] Calder | Cache-conscious data data placement[END_REF] performed to organize them in a cache-friendly way are ad-hoc and do not necessarily lead to efficiency. Our approach is to deploy a memory allocator that allocates memory objects from a region that is laid out with a knowledge of the underlying cache. This layout ensures that every location in the region maps into a unique region in the L2 cache. The result is that the objects allocated are also guaranteed not to conflict with one another. This strategy is much more aggressive than the current approaches for cache-friendly allocation in OSes such as the Slab allocator [START_REF] Bonwick | The slab allocator: An object-caching kernel memory allocator[END_REF], and exploits the flexibility of scheduling of the application and the knowledge of its structure, as they permit one to predict the lifetime properties of the various data manipulated.

Currently, only the data manipulated by the server are managed by the allocator. Library functionalities on which the application depends use their default allocation strategy. The same applies to dynamic services. We set aside these extensions for future work. For dynamic services, we envisage specific interfaces to the server that enable them to manipulate memory objects in a cache-aware way. In this work, however, we provide the basis for our optimization methodology in the context of static workloads.

Our optimizations are applied to an event-driven server through automated tools. These tools include an analysis stage and a transformation stage. The analysis determines the amount of stack, global and per-request data used by the server. The transformations then generate a customized memory allocator that maintains the memory objects discovered by the analysis in a cache-aligned memory region, and modify the server to request these objects from the allocator.

Specifically, for each object allocated as part of the perrequest data, the allocator maintains a pool of pre-allocated objects that are initialized and handed out when the server requests them. Since it is inefficient to have the allocator make the decision as to whether a free object is available for every allocation separately, we propagate part of this effort into the scheduler, which applies it to batches of requests. The scheduling strategy also includes a criterion to try to exploit instruction cache locality by accumulating requests. These aspects are described in more detail in the following sections.

From the implementer's point of view

The approach to optimizing an existing server consists of four steps, each performed with the help of a specific tool. The steps to be performed are: (i) Analyzing memory usage, (ii) Generating a customized memory allocator, (iii) Modifying the server to use the customized allocator (iv) Modifying the server's scheduler. The first three steps are automated, subject to a preparatory step to bring the server into a format that can be analyzed by the tools. The fourth step must be carried out manually, but can use certain library functions generated by the first three.

Case Study: TUX

In this section, we describe the effort of applying our approach by explaining the steps listed in the previous section in the context of TUX. However, first, we give an overview of how to prepare the server for our tools.

Preparatory step

The tools that are used to apply our approach operate on event-driven programs whose scheduling and memory management activities can be summarized using the constructs specified in Figure 2. Figure 2 also gives the names of the concrete functions in TUX implementing these constructs. If certain statements bypass these constructs by accessing low-level data structures directly, then wrapper functions must be introduced. This may happen, for instance, if a stage is queued in the context of a request by directly manipulating the data structure defining the request context instead of doing so by invoking Schedule Stage. In TUX, we did not require any such rewriting. The above interface is described through source-code annotations such as the ones illustrated in Figure 3. The add tux atom function is identified as the interface construct QueueStage. Its first argument is labeled with "T", indicating that it represents v o i d a d d t u x a t o m ( t u x r e q t * r e q , a t o m f u n c t * atom ) a t t r i b u t e ( ( Q u e u e S t a g e ( " T " , " S " ) ) ) ; v o i d * t u x m a l l o c ( i n t s i z e ) a t t r i b u t e ( ( M a l l o c ( " i n t " ) ) ) ; v o i d k f r e e ( v o i d * mem) a t t r i b u t e ( ( F r e e ( "O " ) ) ; s t a t i c i n t e v e n t l o o p ( t h r e a d i n f o t * t i ) a t t r i b u t e ( ( S c h e d u l e r ) ) ; t u x r e q t * t u x m a l l o c r e q ( ) ; t u x r e q t * t u x m a l l o c r e q w r a p ( i n t s i z e ) { r e t u r n ( t u x m a l l o c r e q ( ) ) ; } Fig. 3. Example annotations and wrappers for TUX.

the task context, and the second with "S", indicating that it represents the stage to be queued. Similarly, the functions tux malloc and tux free are identified as the Malloc and Free constructs respectively. The tux malloc req function, used to allocate a request data structure, cannot be labeled directly as it does not accept any argument corresponding to the size of the allocated data. This function must hence be wrapped in a new function that accepts as argument the object size.

Memory analysis with memwalk

The tool memwalk provides conservative approximations of three quantities: (i) The amount of stack used by the program (ii) The amount of per-task state allocated and deallocated, categorized for the various objects. that may not be live at the same time can share such a subregion. The schema of this memory map is shown in Figure 5. The memory map consists of three parts: (i) the cache slab, which overlaps with the L2 cache, (ii) the low slab, which lies below the cache slab (iii) the high slab, which lies above the cache slab. The cache slab contains all the data of the server under normal circumstances, and is the part in which the stack, global data and per-request data are arranged. It is laid out in such a way that distinct locations in it map into distinct locations in the L2 cache. On the x86 architecture, this amounts to using a range of physically contiguous memory. The role of the low and high slabs is described later in this section.

The remainder of the section provides details on the arrangement of the specific regions:

The stack is maintained at the lowest addresses of the cache slab, with the low slab to back it up. Since the stack grows downwards, a stack overflow causes data on the stack to spill into the upper part of the low slab. This situation can result if the estimation of stack utilization is too optimistic, underestimating the amount of stack memory required. Although this spill of data into the low slab may cause cache misses, this arrangement ensures that such a misestimation does not overwrite other program data or cause a memory access violation. Global data are maintained in the region just above the stack area. This choice is motivated by the fact that the size of globals is known before-hand and fixed, and so we are assured that they will not need to be spilled into the regions above or below.

Per-request data are maintained in the upper (dominant) region of the cache slab. Each object is allocated a portion of this region, with the size as calculated in the analyses described in the previous section. The reason we choose the uppermost region of the cache is to be able to spill data into the high slab in case it is infeasible to store it all in the cache slab. Such situations arise particularly when the size of an object allocated is workload-specific, in which case, a conservatively approximated amount of space can be reserved in the cache slab, and the worst-case amount in the high slab.

Modifying the server to use the customized allocator

The tool stingify replaces all the old allocations and deallocations of per-request data with invocations to the Stingy allocator. The size and other parameters are replaced with the identifier of the per-request object retrieved by the tool memwalk. The per-request data in TUX is aggregated into a relatively small number of structures, limiting the benefits of a tool for this activity. However, when the number of objects is large, the utility of stingify is greater, as manual replacements require effort and are prone to errors. Furthermore, it can be used to perform quick replacements when experimenting with different configurations of the server.

Modifying the scheduler

The last step in enabling the Stingy allocator is to modify the scheduler to support it. Before we describe the concrete modifications to make and propose strategies to best go about the process, we will describe the usual implementation of schedulers in event-driven programs. A high-level view of the scheduler of TUX is shown in Figure 6(b). This scheduler is similar, but not identical. The key difference between the scheduler of TUX and the one shown in Figure 6(a) is that the former considers the requests with I/O completed as a part of the total set of active requests to be treated. A stage may terminate at an arbitrary point, and the request made eligible to be scheduled in the next stage. All requests waiting to be processed are thus considered by the scheduler. Requests that have just completed an I/O action are added to the set of active requests asynchronously by the helper processes. The scheduler orders requests on the basis of their priority. Accepting new requests is given the highest priority by treating all incoming requests before considering requests at other stages.

Schedulers in event-driven programs

Our scheduling strategy

Our scheduling strategy requires the inclusion of two criteria in the scheduler. The first is support for the Stingy allocator. The scheduler must check if enough per-request memory is available for a request before it is elected. This is done by invoking the query function generated by the tool stingygen. The second criterion is to favor the instruction cache by bringing requests in early stages of processing up to the mark with requests in advanced stages. These criteria can be handled by defining additional request priorities.

We first change the scheduler of TUX to iterate through stages instead of individual requests, considering the entire lot of requests active at a particular stage. Once we have done so, we sort requests on the basis of three new priorities: (i) Requests that have attained maximal flow at a particular stage by using up all the per-request memory allocated for them at that stage are given the highest priority, as no more requests can be accumulated with them. (ii) Requests for which the amount of per-request memory available is insufficient are given the lowest priority, as they will likely cause cache misses. (iii) The remaining requests are given a medium priority that is lower than the priority of the first class of requests, as it is possible that requests in early stages of processing may eventually come to the level of these requests, increasing the size of the batch. This priority is weighted, with more favorable weights given to requests that are earlier in the course of treatment, as compared to those that are advanced. This final scheduler is illustrated at the bottom of Figure 6.

Benchmarks

We now describe an evaluation of our optimizations by comparing the performance of unmodified and optimized versions of TUX. We first discuss our benchmarking methodology, focusing on the characteristics of the server we wish to measure, the tools used to measure them and finally describe our results.

w h i l e ( 1 ) { / / U p d a t e s y n c h r o n o u s t i m e r s , s t a t i s t i c s e t c . DoUpdates ( ) ; / / H a n d l e n o t i f i c a t i o n s r e c e i v e d t h r o u g h s i g n a l s H a n d l e S i g n a l s ( ) ;

/ / E x t r a c t s e t o f r e q u e s t s w i t h I / O c o m p l e t e d r e q u e s t s w a i t i n g = P o l l I O ( c u r r e n t r e q u e s t s ) ; / / T r e a t w a i t i n g r e q u e s t s f o r e a c h r e q i n r e q u e s t s w a i t i n g { / / Look up c u r r e n t s t a g e o f t h e r e q u e s t c u r s t a g e f n = G e t C u r r e n t S t a g e ( r e q ) ; S c h e d u l e S t a g e ( r e q , c u r s t a g e f n ) ; } } (a) Standard w h i l e ( 1 ) { / / A c c e p t any new i n c o m i n g r e q u e s t s i f ( N e w R e q u e s t s W a i t i n g ( ) )

A c c e p t N e w R e q u e s t s ( ) ; i f ( ! Empty ( a c t i v e r e q u e s t s ) ) { f o r e a c h r e q i n a c t i v e r e q u e s t s { c u r s t a g e f n = G e t C u r r e n t S t a g e ( r e q ) ; S c h e d u l e S t a g e ( r e q , c u r s t a g e f n ) ;

} } i f ( n o t h i n g t o d o )
S l e e p ( ) ; } (b) TUX w h i l e ( 1 ) { / / O( 1 ) e l e c t i o n o f t h e h i g h e s t p r i o r i t y s t a g e / / Get t h e h i g h e s t p r i o r i t y ( eg . 5 ) and u s e i t t o / / g e t t h e c u r r e n t s t a g e q u e u e . c u r p r i o r i t y = G e t C u r r e n t H i g h e s t P r i o r i t y ( ) ; c u r s t a g e = G e t N e x t S t a g e ( c u r p r i o r i t y ) ; c u r b a t c h = G e t A c t i v e R e q u e s t s ( s t a g e ) ; f o r e a c h r e q i n c u r b a t c h { c u r s t a g e f n = G e t C u r r e n t S t a g e ( r e q ) ; S c h e d u l e S t a g e ( r e q , c u r r e n t s t a g e f n ) ;

} i f ( n o t h i n g t o d o )
S l e e p ( ) ; } (c) TUX Modified Fig. 6. Scheduler of a typical event-driven server limited by I/O

Tools

We considered a variety of server benchmarking tools to use in our experiments. We needed a tool that was well identified and also captured the key property we were interested in, namely, the performance of a server under workloads of specific concurrencies.

There are three main regions in a typical server's performance regime with respect to increasing concurrency. The first, as we mentioned in Section 2, is the phase in which the load is well below exercising the full computational bandwidth of the server. In this phase (the elastic zone 4 ), the throughput tends to increase linearly with additional load. When the computational bandwidth of the server is neared, i.e., for in-core workloads, when CPU utilization nears 100%, the server enters it's plastic zone. In this stage, performance starts to degrade due to inefficiencies in caching. Finally, when the size of the incoming request stream increases beyond a final threshold, it goes into its failure zone. Then, connections begin to get dropped due to queue overflows, requests get detained for long periods of time due to lack of CPU allocation, and the server starts to become unproductive.

One popular index of measurement is the uniform load, in terms of the number of requests per second that a server can handle before it enters the failure zone, and becomes saturated. httperf [START_REF] Mosberger | httperf -a tool for measuring web server performance[END_REF] is a tool that is known for being able to sustain server overload by avoiding client-side bottlenecks, like the number of available file descriptors, the size of socket buffers etc. We use this tool in our work to compute the maximum number of requests the server can handle before failure.

Although httperf is suitable for measuring this value of maximum simultaneous connections, it is not optimal for a controlled application of high-concurrency workloads. This is because httperf (and like benchmarks) simply generate requests uniformly at regular intervals of m/rate, where m is the number of requests in a burst. Concurrency in this case can only be escalated when the server is close to overload, since otherwise the server responds to request bursts in periods that are too small to allow concurrency to build up. This escalation in concurrency close to overload is a result of the detention of requests over long periods of time in the failure zone of the server.

For this reason, we decided to use Apachebench [START_REF]Apache HTTP server project[END_REF], which serves this second purpose. Apachebench takes the desired concurrency, c, of requests as a command line parameter, and keeps the total number of parallel requests in the server close to c, measuring total throughput for the benchmarked period. With Apachebench, we measure performance in the server's plastic zone. Apachebench has been used to evaluate servers under high request concurrency before [START_REF] Von Behren | Capriccio: scalable threads for internet services[END_REF], and is used commonly in industry.

Environment

We ran the load generators on a system with two Xeon processors running at 3GHz each, with 1MB of cache and with an Intel e1000 Gigabit Ethernet card. The server ran on an Intel Pentium IIIM running at 1.4GHz, with 1MB of L2 cache. Running the Netperf [START_REF]company Information Networks Division, Netperf: A network performance benchmark[END_REF] benchmark for both client/server pairs quickly showed that even for raw data transfers using the protocol stack, the bottleneck of data transfer was on the server side. The measurements provided in this paper were obtained with Linux kernel 2.6.7. The experiments conducted consisted of repeatedly requesting a set of small files. quests per second generated for it by httperf. Note that this load is generated uniformly over the period of benchmarking. The maximum number of concurrent connections over a benchmarked period is also displayed at points at regular intervals in these graphs. We observe that the peak performance of server, i.e., the load handled just before entering its failure zone increases by about 21%. Figure 7(b) shows the variation of requests serviced per second with increasing concurrency in the two servers. Figure 7(c) shows the number corresponding variation in L2 cache misses. We note that requests serviced increase by up to 40% for a concurrency of about 2500 and L2 cache misses decrease by up to 75%.

Results

Analysis

Apachebench As mentioned earlier, we use apachebench to analyze performance in the plastic zone of the servers, and httperf to analyze their failure zones. We observe that in Figure 7 over the plastic zone, the number of L2 cache misses decreases drastically in the modified versions of the servers. As a result of this decrease, performance now stays relatively consistent over the entire zone. Early on, when concurrency is in the neighborhood of 40, the increase in performance can also be expected to be due to a reduction in i-cache misses.

httperf To understand performance improvements close to the failure zone, we must keep in mind that there is an escalation in concurrency as a server approaches overload. This trend can be observed in Figure 7(a). Since our modifications make the servers more robust to high concurrencies, the modified servers can handle this load close to overload better than the unmodified ones. The result is that the point at which the server fails is delayed, and the server scales to a higher peak performance.

Related Work

The scalability of servers has been an intensively researched topic in the systems community. Much of this research has been done in the context of servers with a considerable amount of I/O activity. In this section, we will focus on the works that are most pertinent in the context of CPU-bound servers.

Chandra and Mosberger introduced multi-accept servers [START_REF] Chandra | Scalability of Linux event-dispatch mechanisms[END_REF] that were shown to bring about significant gains in performance as compared to traditional servers. Brecht et al. [START_REF] Brecht | Accept()able strategies for improving server performance[END_REF] have shown that performance could be enhanced with small modifications in the above strategy. The results of both these works concur with the observations presented in ours, in that, (i) The former advocates that servers aggressively accept requests and treat requests in as large batches as possible, improving locality with respect to instructions and static data.(ii) The latter advocates that these batches be limited in size, to prevent the total working data set of the server from exploding. Our approach strikes a balance between these two policies in an adaptive way and derives itself from the characteristics of the underlying cache.

Larus and Parkes have presented another cache-aware scheduling strategy called Cohort scheduling [START_REF] Larus | Using cohort scheduling to enhance server performance[END_REF]. Our scheduling strategy includes a policy that effectively implements a variant of Cohort scheduling, favoring the batching of requests as long as it can be done without causing datacache misses. Larus and Parkes applied Cohort scheduling to a threaded server, by changing the implementation of the underlying threading library. In this work, we have applied what amounts to cohort scheduling to an event-driven server using automated tools. Better instruction-cache locality was also the goal of Blackwell [START_REF] Blackwell | Speeding up protocols for small messages[END_REF], in his work on optimizing TCP/IP stacks. He showed that by processing several packets in a loop at every layer, one could induce better reuse of the corresponding instructions.

Cache-conscious data placement has been used to optimize the caching behavior of generic programs [START_REF] Calder | Cache-conscious data data placement[END_REF][START_REF] Chilimbi | Cacheconscious structure layout[END_REF][START_REF] Chilimbi | Cacheconscious structure definition[END_REF]. These works use program analysis and profiling information to efficiently arrange objects in memory, and fields within objects. While the goal of these efforts is to reduce the number of cache misses in generic programs, our work focuses on the specific problem of reducing data cache misses in event-driven servers, since they have a well defined structure and concurrency behavior.

Recent work has advocated policies for resource aware scheduling. The Capriccio threading library [START_REF] Von Behren | Capriccio: scalable threads for internet services[END_REF] is one example, in which scheduling aims to balance the utilization of various resources at blocking points in the server application. These blocking points are points at which potentially blocking system calls are invoked, and are extracted auto-matically. The resources tracked by Capriccio were memory, CPU and file descriptors. Blocking points in a program can be seen as end points of implicit stages in a server. In relation to Capriccio, our work could be seen as a special kind of resource-aware scheduling which aims to constrain cache usage. Similar to Capriccio, the SEDA architecture [START_REF] Welsh | SEDA: An architecture for well-conditioned, scalable internet services[END_REF] had dynamic resource controllers, which dynamically adapted resource usage at various stages based on observed performance. SEDA also did not specifically explore caching inefficiencies in CPU-bound servers.

Conclusion & Future Work

In this paper, we analyzed the performance of eventdriven servers that are CPU-bound and demonstrated that caching inefficiencies can lead to poor performance. We presented an approach consisting of a cache-aware memory allocator and a specific scheduling strategy that together ensure that L2-cache-miss rate in the application is minimized. Our approach is applied with the help of a set of tools that operate on event-driven programs written in the C language. We described a case study in the context of a high-performing real-world server, TUX. Our optimizations improves the throughput under high concurrency by up to 40%, reduces L2 data-cache misses by about 75% and improves the maximum load handled before failure by 21%.

As future work, we are looking to apply our approach to other CPU-bound servers, such as telephony servers that serve the SIP protocol. We are also considering additional OS-support for the Stingy allocator and specific interfaces that allow our approach to be extended to dynamic services.

Fig. 1 .

 1 Fig. 1. (a) Throughput degradation in the TUX web server with increasing concurrency (b) Corresponding increase in L2 cache misses (d-cache on the left and i-cache on the right)

Fig. 2 .

 2 Fig. 2. Set of abstractions supplied as input to the analysis tools.

Fig. 4 .

 4 Fig. 4. Memory usage analysis of TUX.

Fig. 5 .

 5 Fig. 5. Layout of the Stingy allocator's memory pool.

Figure 6 (

 6 Figure 6(a) illustrates a typical scheduler in a server limited by I/O. The scheduler consists of a loop that starts by handling global activities like updating stats, updating timers and checking for time-outs, handling signals etc. Next, it polls for requests that have just completed a read or write to or from an I/O device, and are waiting to be serviced. It

Figure 7 (

 7 Figure 7(a) illustrate a plot between the number of requests serviced per second by TUX, and the number of re-

Fig. 7 .

 7 Fig. 7. (a) Throughput of TUX with increasing concurrency. (b) Corresponding increase in L2 cache misses (c) Peak performance of TUX for uniform load.

  Queue Stage : S × T → void A function that queues a request to be executed at a particular stage. add tux atom Schedule Stage : S × T → void A function to execute a stage in the context of a request. tux schedule atom Scheduler : void → void Entry point into the implementation of the scheduler. event loop M alloc : int → O A function to allocate a block of memory for object O. kmalloc,kmalloc req,get abuf, sock alloc,kmem cache alloc F ree : O → void A function that frees the memory allocated for object O. kfree, kfree req,sock release, kmem cache free,free abuf

	Construct	Description	Implementation in TUX
	Where,
	S ⊂ [0, ∞)	is the set of stages.
	T ⊂ [0, ∞)	is the set of request.
	O	is the set of objects used by various stages in the course of processing requests.

As an example, we found out that the information site of the Sydney Olympic Games, which was served entirely out of RAM[START_REF] Ramaswamy | [END_REF]

The terms elastic zone, plastic zone and failure zone are borrowed from material sciences terminology.

, which contains the stages of an event-driven server at its nodes and events at the edges.

The SCG extracted for TUX is shown in Figure 4 (edge labels omitted).

Generating a customized allocator using stingygen

The tool stingygen accepts the output of the tool memwalk and generates a memory map that contains an area dedicated to each region of memory identified by memwalk. The region corresponding to per-request data contains a sub-region for each object type. Two objects http_post_header [nf=1,sc=20] http_send_body [nf=7,sc=72] http_process_message [nf=45,sc=234] http_dirlist_head [nf=6,sc=72] list_directory [nf=2,sc=642] do_dir_line [nf=3,sc=1140] http_dirlist_tail [nf=9,sc=80] do_send_abuf [nf=71,sc=56] flush_request [nf=1,sc=0] http_lookup_vhost [nf=6,sc=413] parse_request [nf=47,sc=846]


Frédéric Jouault

Jean Bézivin

Charles Consel

Ivan Kurtev
email: ivan.kurtev@univ-nantes.frcharles.consel

Fabien Latry
email: fabien.latry@labri.fr

Building DSLs with AMMA/ATL a Case Study on SPL and CPL Telephony Languages

Domain-Specific Languages (DSLs) enable more concise and readable specifications than General Purpose Languages (GPLs). They are for this reason increasingly used. This DSL approach presents, however, many challenges. One of them is the prototyping and implementation of the numerous DSLs that are required to replace a single GPL. This work presents a case study of implementing two telephony languages: SPL and CPL. It shows how a DSL building framework like AMMA can be used to this purpose.

Introduction

Domain-Specific Languages (DSLs) are increasingly used. They directly represent domain concepts at the syntactical level. This enables concise specifications that may even be understood or specified by non-programmer domain experts.

They are, however, many issues that DSL designers encounter. For instance, deriving a DSL from domain knowledge requires more than just placing a domain expert and a language engineer in the same room. Methodologies need to be developed. Some solutions to this first problem may be found in [START_REF] Thibault | Domain-Specific Languages: From Design to Implementation Application to Video Device Drivers Generation[END_REF].

Another problem is the implementation of these DSLs. Firstly, although a single General Purpose Language (GPL) is often enough to build complex systems, many DSLs are often required for the same task. Each DSL can indeed only capture a limited aspect of a system. Consequently, the development of a DSL cannot be as expensive as for a GPL. Secondly, it may help the DSL designers in their work to implement proof-of-concept prototypes at different stages. We therefore consider that inexpensive development and rapid prototyping are essential for the success of the DSL approach. We tackle here this problem of implementing DSLs.

Our proposal is to use DSL building frameworks [START_REF] Bézivin | Model-based DSL Frameworks[END_REF]. Examples of such frameworks are GME [START_REF] Gme | The Generic Modeling Environment, Reference site[END_REF][START_REF] Karsai | Component Generation Technology for Semantic Tool Integration[END_REF] (Generic Modeling Environment), Microsoft DSL Tools [START_REF] Greenfield | Software Factories: Assembling Applications with Patterns, Models, Frameworks, and Tools[END_REF], and the one we developed: AMMA [START_REF] Bézivin | Model-based DSL Frameworks[END_REF][START_REF] Bézivin | Modeling in the Large and Modeling in the Small[END_REF] (ATLAS Model Management Architecture). In this work, we report on an experiment consisting of the implementation of two languages specific to the domain of internet telephony. The first one is SPL [START_REF] Burgy | Language Technology for Internet-Telephony Service Creation[END_REF] (Session Processing Language), and the second one is CPL [START_REF] Lennox | Call Processing Language (CPL): A Language for User Control of Internet Telephony Services[END_REF] (Call Processing Language).

The outcome of this experiment provides an interesting example of DSL building. Three aspects of each DSL are taken into account: abstract syntax, concrete syntax, and dynamic semantics. Moreover, our case study allows for different approaches to be illustrated. SPL has a textual concrete syntax whereas CPL is XMLbased. Additionally, both languages being in the same domain, one can be defined using the other.

The paper is organized as follows. Section 2 presents AMMA. Sections 3 and 4 respectively describe how SPL and CPL are built. Section 5 concludes.

ATLAS Model Management Architecture

This section briefly presents AMMA and three of its core DSLs: KM3 (Kernel MetaMetaModel), ATL (ATLAS Transformation Language), and TCS (Textual Concrete Syntax). A more complete description can be found in [START_REF] Bézivin | Model-based DSL Frameworks[END_REF].

Overview

AMMA is built on a model-based vision of DSLs, which is presented in [START_REF] Bézivin | Model-based DSL Frameworks[END_REF]. A DSL is considered as a set of coordinated models. Each of these models represents one facet of the language. For instance:

• Abstract Syntax. Domain concepts and their relations are captured in a metamodel called a Domain Definition MetaModel (DDMM).

• Concrete Syntax. Concrete syntaxes of DSLs can be represented as models. One possibility is to specify a transformation from concrete to abstract syntax. Section 4.3 gives an example of this applied to CPL. Another possibility is to represent a concrete syntax as a model conforming to an EBNF metamodel (i.e. a grammar) or to the TCS metamodel (see below).

• Semantics. DSLs have several kinds of semantics that may be captured by models. For instance, dynamic semantics can be captured as an Abstract State Machine [START_REF] Börger | High Level System Design and Analysis using Abstract State Machines[END_REF] model. Alternatively, the semantics of DSLA may be implemented in terms of the semantics of DSLB by writing a transformation from DSLA to DSLB.

AMMA provides a set of core DSLs that are used to specify each model of a DSL. Figure 1 shows four of these core DSLs: KM3, ATL, TCS, and ASM (Abstract State Machines). Three other DSLs: DSLx, CPL, and SPL are also shown. Each of these DSLs is represented by a set of models defined using AMMA core DSLs. The first three core DSLs are described in the next section. As for the last one (i.e. ASM), please refer to [START_REF] Di Ruscio | Extending AMMA for Supporting Dynamic Semantics Specifications of DSLs[END_REF], which describes how we extended AMMA with it.

KM3

The KM3 language is intended to be a lightweight textual metamodel definition language. It enables easy creation and modification of metamodels. The metamodels expressed in KM3 have good readability properties. These metamodels may be easily converted to/from other notations like Emfactic or XMI. KM3 has a clear semantics, partially presented in [START_REF] Jouault | KM3: a DSL for Metamodel Specification[END_REF].

Figure 1 shows that the DDMM of KM3 is expressed in KM3 (box DDMM : KM3). Its concrete syntax is defined in TCS (box CS : TCS). Its semantics is implemented by a transformation to Ecore [START_REF] Budinsky | Eclipse Modeling Framework[END_REF] written in ATL (box KM32Ecore : ATL). Other models are not shown here. For instance, a mapping to MOF 1.4 also exists, and formal semantics of KM3 is also expressed in Prolog. A library Figure 1. AMMA core DSLs of KM3 metamodels and their translations to various formats like Ecore, MOF 1.4, and pictures is available on [START_REF]ATLAS MegaModel Management (AM3) Home page[END_REF].

ATL

ATL is a hybrid model transformation DSL. Its declarative part enables simple specification of many problems, while its imperative part helps in coping with problems with higher complexity. Informal semantics of ATL is presented in [START_REF] Jouault | Transforming Models with ATL[END_REF] along with a non-trivial case study. More than forty different scenarios accounting for more than a hundred individual transformations are available on [START_REF]ATLAS MegaModel Management (AM3) Home page[END_REF].

Figure 1 shows that the DDMM of ATL is expressed in KM3 (box DDMM : KM3). Its concrete syntax is defined in TCS (box CS : TCS). Its semantics is implemented by a transformation to ATL Virtual Machine (see [START_REF] Jouault | On the Architectural Alignment of ATL and QVT[END_REF]) written in ATL (box ATL2VM : ATL). Partial formal semantics of ATL is also expressed in ASM [START_REF] Di Ruscio | Extending AMMA for Supporting Dynamic Semantics Specifications of DSLs[END_REF] (box DS : ASM).

TCS

TCS is a DSL aimed at specifying context-free textual concrete syntaxes of DSLs. From such specifications, models can be serialized into their textual equivalent and text can be parsed into models. In other words, a TCS specification defines a bidirectional translation between a textual representation of a model and its internal representation. The choice of context-free languages was mainly motivated by the observation that programming languages use them extensively. TCS models provide a way to attach syntactic elements, such as keywords and symbols, to elements of the DDMM of a DSL. A detailed description of TCS is out of the scope of this paper.

Figure 1 shows that the DDMM of TCS is expressed in KM3 (box DDMM : KM3). Its concrete syntax is defined in TCS (box CS : TCS). Its semantics is implemented by a transformation to EBNF written in ATL (box TCS2EBNF : ATL).

Session Processing Language

Overview

SPL programs are used to control telephony agents (e.g. clients, proxies) implementing the SIP (Session Initiation Protocol) [START_REF] Rosenberg | SIP: Session Initiation Protocol[END_REF] protocol. SIP concepts are directly available in the language. Consequently, SPL programs are able to concisely and simply express any telephony service. Additionally, SPL is capable of guaranteeing critical properties that could not be verified with a GPL. SPL programs run on a Service Logic Execution Environment for SIP.

Listing 1 gives a simple example of an SPL service. Every incoming call is redirected to SIP address sip:phoenix@barbade. enseirb.fr. The target address is declared on line 3. Lines 6-8 correspond to the definition of the action to perform on incoming calls. The return statement at line 7 forwards the call.

Abstract and Concrete Syntaxes

Abstract syntax of SPL is specified in KM3. This corresponds to the box DDMM : KM3 of the SPL DSL in Figure 1. Listing 2 gives an excerpt of the SPL metamodel. Lines 1-5 define Service, which has a name, and can contain declarations and sessions. A Declaration (lines 7-9) has a name and may be a VariableDeclaration (lines 11-14), which has a type and an optional initialization expression (initExp, line 13). A Session (line 16) may be a Registration (lines 18-20) containing other sessions or a Method (lines 22-27). A Method has a return type, a direction (see the Direction enumeration at lines 35-39), a name, and statements. Expression, TypeExpression, and Statement (lines 29-33) are abstract classes, which are extended to specify the full SPL language (not given here). The full version of this metamodel can be found in AM3 [START_REF]ATLAS MegaModel Management (AM3) Home page[END_REF] metamodel library. l i t e r a l in ; l i t e r a l out ; 39 } Concrete syntax of SPL has been implemented in TCS according to the syntax specified in [START_REF]SPL: The Session Processing Language, Reference site[END_REF]. This corresponds to the box CS : TCS of the SPL DSL in Figure 1. A grammar is automatically derived from both the KM3 metamodel and the TCS model to parse SPL programs into SPL models. SPL models can also be serialized to programs using a TCS interpreter written in Java. We only describe SPL syntax informally here by referring to Listing 1. The Service SimpleForward spans over lines 1-11. Line 3 contains a VariableDeclaration of type uri initialized with the address to forward to. Lines 5-10 correspond to a Registration, which contains a single Method (lines [START_REF] Bézivin | Modeling in the Large and Modeling in the Small[END_REF][START_REF] Burgy | Language Technology for Internet-Telephony Service Creation[END_REF][START_REF] Lennox | Call Processing Language (CPL): A Language for User Control of Internet Telephony Services[END_REF]. This Method returns a response and is called on incoming (i.e. Direction::in) invites.

Dynamic Semantics

Several definitions of SPL semantics can be given. For instance, static and dynamic (by means of transition rule) semantics for SPL is specified in [START_REF]SPL: The Session Processing Language, Reference site[END_REF]. We have also developed an executable specification of SPL dynamic semantics using (ASM). This corresponds to the box DS : ASM of the SPL DSL in Figure 1. Part of this ASM model has been automatically derived from the SPL KM3 metamodel. This corresponds to the definition of SPL data model. Other parts like environments and state machines are not present in the metamodel. They had to be manually translated from the specification.

Call Processing Language 4.1 Overview

CPL is a standard [START_REF] Lennox | Call Processing Language (CPL): A Language for User Control of Internet Telephony Services[END_REF] scripting language for the SIP protocol. It offers a limited set of language constructs. CPL is supposed to be simple enough so that it is safe to execute untrusted scripts on public servers. Listing 3 gives a CPL example, which is equivalent to the SPL example given in Listing 1. The incoming element at lines 5-9 declares the actions that have to be performed on incoming calls. The location element adds the target address to the current environment. The proxy element forwards the call to the adress found in the current environment.

Abstract Syntax

Abstract syntax of CPL is specified in KM3. This corresponds to the box DDMM : KM3 of the CPL DSL in Figure 1. Listing 4 gives an excerpt of this metamodel. The complete version is available in the AM3 [START_REF]ATLAS MegaModel Management (AM3) Home page[END_REF] CPL concrete syntax is XML-based. TCS is therefore not really useful here. The solution we implemented is the following. We use a generic XML parser to go from the XML document to an XML model conforming to an XML metamodel. This has an extremely low cost since these XML parser and metamodel are provided as part of AMMA. In a second step, we transform our XML model into a CPL model using ATL. This corresponds to the box XML2CPL : ATL of the CPL DSL in Figure 1. Listing 5 gives an excerpt of this XML2CPL transformation (line 1). It transforms an XML model into a CPL model (line 2) by using a library of XML helpers (line 4) providing the getElemsByNames operation on XML elements. A single rule is shown: rule CPL (lines 6-17), which transforms the root of the XML document into a CPL element. Nested incoming element is attached to this root (lines [START_REF]ATLAS MegaModel Management (AM3) Home page[END_REF][START_REF] Jouault | Transforming Models with ATL[END_REF][START_REF] Jouault | On the Architectural Alignment of ATL and QVT[END_REF]. Figure 2 shows the full transformation scenario. The CPL script Sample.cpl conforming to the CPL schema is first translated into an XML model conforming to an XML metamodel. Then it is transformed into a CPL model by XML2CPL.atl. The core transformation CPL2SPL.atl is then applied to generate an SPL model. The latter is then serialized into an SPL program using the TCS interpreter on the TCS syntax definition of SPL. This full transformation scenario (called CPL2SPL) is available in the AM3 [START_REF]ATLAS MegaModel Management (AM3) Home page[END_REF] transformation library.

Conclusion

This paper has briefly presented our vision of DSLs as sets of models and its concretization: the AMMA DSL building framework. Details on the implementation with AMMA of two languages (SPL and CPL) specific to the domain of internet telephony have been given. This case study illustrates how AMMA core DSLs can be used to capture different facets of a DSL. KM3 is used to express Domain Definition MetaModels (e.g. of KM3, ATL, TCS, ASM, CPL, and SPL). Concrete syntaxes are defined in TCS, for instance for: KM3, ATL, TCS, ASM, and SPL. Dynamic semantics can be formally defined in ASM, which we have done for ATL and SPL.

Moreover, transformations from any DSLA to any DSLB can be implemented in ATL. This can, for instance, be used to implement the semantics of DSLA in terms of the semantics of DSLB (e.g. from CPL to SPL, see section 4.3). Such a transformation may then be used to translate programs expressed in DSLA into programs expressed in DSLB. Another use of ATL is to implement concrete syntaxes of DSLs (e.g. CPL using XML2CPL, see section 4.3).

Listing 1 . 5 r

 15 Simple SPL program: forwarding a call 1 s e r v i c e SimpleForward { 2 p r o c e s s i n g { 3 u r i us = 'sip:phoenix@barbade.enseirb.fr' ; 4 e g i s t r a t i o n { 6 r e s p o n s e incoming INVITE ( ) { 7 r e t u r n forward us ;

Listing 2 . 4 r 6 7

 246 SPL metamodel excerpt 1 c l a s s Service { 2 a t t r i b u t e name : S t r i n g ; 3 r e f e r e n c e declarations [ * ] o r d e r e d c o n t a i n e r : ֒→Declaration ; e f e r e n c e sessions [ * ] o r d e r e d c o n t a i n e r : Session ; 5 } a b s t r a c t c l a s s Declaration { 8 a t t r i b u t e name : S t r i n g ; 9 } 10 11 c l a s s VariableDeclaration e x t e n d s Declaration { 12 r e f e r e n c e type c o n t a i n e r : TypeExpression ; 13 r e f e r e n c e initExp [0 -1] c o n t a i n e r : Expression ; 14 } 15 16 a b s t r a c t c l a s s Session {} 17 18 c l a s s Registration e x t e n d s Session { 19 r e f e r e n c e sessions [ * ] o r d e r e d c o n t a i n e r : Session ; 20 } 21 22 c l a s s Method e x t e n d s Session { 23 r e f e r e n c e type c o n t a i n e r : TypeExpression ; 24 a t t r i b u t e direction : Direction ; 25 a t t r i b u t e name : S t r i n g ; 26 r e f e r e n c e statements [1 - * ] o r d e r e d c o n t a i n e r : Statement ֒→ ; 27 } 28 29 a b s t r a c t c l a s s Expression {} 30 31 a b s t r a c t c l a s s TypeExpression {} 32 33 a b s t r a c t c l a s s Statement {} 34 35 enumeration Direction { 36 l i t e r a l inout ;

Listing 3 . 5 <i n c o m i n g> 6 <l 7 <p r o x y /> 8 </ l o c a t i o n> 9 <

 356789 Simple CPL script: forwarding a call 1 <? xml v e r s i o n ="1.0" e n c o d i n g ="UTF -8" ?> 2 <c p l xmlns =" urn:ietf:params:xml:ns:cpl" 3 x m l n s : x s i ="http: //www.w3.org /2001/ XMLSchema -instance" 4 x s i : s c h e m a L o c a t i o n =" urn:ietf:params:xml:ns:cpl cpl.xsd ֒→"> o c a t i o n u r l ="sip:phoenix@barbade.enseirb.fr"> / i n c o m i n g> 10 </ c p l> Lines 2-4 declare the default XML namespace and its location.

4 5 8 9

 48 library. A CPL script is rooted by a CPL element (lines 1-3), which contains an Incoming (line 9) element. NodeContainers (lines 5-7) like Incoming and Location (lines 13-15) may contain a Node (line 11) like Action (line 17). Proxy (line 21) is a special kind of SignallingAction (line 19) itself a special kind of Action. Listing 4. CPL metamodel excerpt 1 c l a s s CPL { 2 r e f e r e n c e incoming [0 -1] c o n t a i n e r : Incoming ; 3 } a b s t r a c t c l a s s NodeContainer { 6 r e f e r e n c e contents [0 -1] c o n t a i n e r : Node ; 7 } c l a s s Incoming e x t e n d s NodeContainer {}

10 11 a

 11 b s t r a c t c l a s s Node {} 12 13 c l a s s Location e x t e n d s Node , NodeContainer { 14 a t t r i b u t e url : S t r i n g ; 15 } 16 17 a b s t r a c t c l a s s Action e x t e n d s Node {} 18 19 a b s t r a c t c l a s s SignallingAction e x t e n d s Action {} 20 21 c l a s s Proxy e x t e n d s SignallingAction {} 4.3 Concrete Syntax and Dynamic Semantics Both CPL concrete syntax and semantics are handled by model transformations in ATL.

Listing 5 . 3 4

 53 XML to CPL transformation excerpt, written in ATL 1 module XML2CPL ; 2 c r e a t e OUT : CPL from IN : XML ; u s e s XMLHelpers ;

12 t 15 )

 1215 : CPL ! CPL ( 13 incoming <-s . getElemsByNames ( 14 S e q u e n c e {'incoming '} ->first ( ) A second transformation (CPL2SPL) provides an implementation of CPL semantics by translating CPL concepts into their SPL equivalent concepts. This corresponds to the box CPL2SPL : ATL of the CPL DSL in Figure 1. Listing 6 provides an excerpt of this transformation. Line 2 declares source and target models respectively conforming to CPL and SPL. Rule CPL2Program (lines 4-19) transforms the root CPL element (lines 5-6) into an SPL program (lines 8-10), an unnamed service (lines 11-15) and a dialog (lines 16-18).

Figure 2 .

 2 Figure 2. Full CPL to SPL transformation scenario

  SPL ! Dialog ( 17 methods <-S e q u e n c e {s . incoming , s . outgoing} 18 ) 19 }

Acknowledgments

This work has been partially supported by ModelWare, IST European project 511731.