Henri Casanova

Anshul Gupta

Frédéric Suter

Toward More Scalable Off-Line Simulations of MPI Applications

The off-line (or post-mortem) analysis of execution event traces is a popular approach to understand the performance of HPC applications that use the message passing paradigm. Combining this analysis with simulation makes it possible to "replay" the application execution to explore "what if?" scenarios, e.g., assessing application performance in a range of (hypothetical) execution environments. However, such off-line analysis faces scalability issues for acquiring, storing, or replaying large event traces.

We first present two previously proposed and complementary frameworks for off-line replaying of MPI application event traces, each with its own objectives and limitations. We then describe how these frameworks can be combined so as to capitalize on their respective strengths while alleviating several of their limitations. We claim that the combined framework affords levels of scalability that are beyond that achievable by either one of the two individual frameworks. We evaluate this framework to illustrate the benefits of the proposed combination for a more scalable off-line analysis of MPI applications.

Introduction

Analyzing and understanding the performance behavior of parallel applications implemented with the Message Passing Interface (MPI) on various compute platforms is a long-standing concern in the High Performance Computing (HPC) community. A popular approach is to conduct post-mortem analysis, i.e., based on event traces recorded during an instrumented execution of the application. As the scale of computing platform increases (into the exascale era), it is crucial that post-mortem analysis be scalable. We identify below three main scalability challenges faced by state-of-the-art approaches.

Scalability challenge #1 -The scale of the performance analysis is limited to the largest, dedicated, homogeneous platform available. This is because most tools for the performance analysis of message passing applications require traces with precise and uniform measured elapsed times for all events (computation, communication) included in the trace.

Scalability challenge #2 -The sheer number of recorded events leads to traces so large that transferring, storing, and processing these traces are themselves analysis bottleneck. One possible solution to this challenge is to forego recording full traces but instead to record only a subset of the events and to record only aggregated statistical information [START_REF] Shende | The Tau Parallel Performance System[END_REF][START_REF] Geimer | A Scalable Tool Architecture for Diagnosing Wait States in Massively Parallel Applications[END_REF][START_REF] Knüpfer | Score-P: A Joint Performance Measurement Run-Time Infrastructure for Periscope, Scalasca, TAU, and Vampir[END_REF]. Using such lossy information has proved sufficient for the automated or semi-automated detection of performance bottlenecks, load imbalance, or undesired behaviors such as late senders/receivers or wait states. However, they prevent more in-depth analysis such as Ganttchart visualization, or even debugging. Those analysis tools [START_REF] Knüpfer | The Vampir Performance Analysis Tool-Set[END_REF] that focus on providing such advanced capabilities must thus trade off scalability for a higher trace resolution.

Scalability challenge #3 -One advanced message passing application performance analysis approach is off-line simulation. While understanding the performance behavior of an MPI application on an existing parallel platform is useful, in many cases one wishes to consider other platforms than the one on which the event trace has been obtained, perhaps hypothetical platforms that are not even available. Off-line simulation consists in simulating the application execution by "replaying" event traces. A common use case is when a platform is yet to be specified and purchased. In this case, simulation can be used to determine a cost-effective hardware configuration appropriate for the expected application workload. More generally, simulation can be used to understand the performance behavior of an application by varying the hardware and software characteristics of hypothetical platforms, e.g., compute and network speeds, network topologies, collective communication algorithms. Finally, simulating application execution may be useful even when the target platform is available. For instance, in simulation it is possible to bypass actual computations performed by the application and only simulate the corresponding delays of these computations. In this case the simulated replay produces erroneous application results, but the application's performance behavior may be preserved. It is then possible to conduct development activities for performance tuning in a way that saves time and resources when compared to real executions. This is important due to the costs associated to using largescale platforms (monetary charges, electrical power consumption). While off-line simulation for replaying event traces is a powerful and attractive proposition, it faces its own scalability challenges due to the high computational complexity of discrete-event simulation. For instance, many off-line simulators require the simulation to be executed using an amount of compute resources commensurate to that in the platform to be simulated.

In summary, the three scalability challenges are: (i) trace acquisition is limited to the largest dedicated, homogeneous platform available; (ii) trace size may be so large that transferring, storing, and processing traces are analysis bottleneck; and (iii) off-line simulation of a large-scale application execution for advanced performance analysis may itself require the use of a large-scale platform. Two recent and complementary works, ScalaTrace [START_REF] Noeth | ScalaTrace: Scalable Compression and Replay of Communication Traces for High-Performance Computing[END_REF] and Time-Independent Trace Replay [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF], have each partially addressed these scalability challenges.

The main objective of ScalaTrace [START_REF] Ratn | Preserving Time in Largescale Communication Traces[END_REF] is to produce compact event traces of near-constant size while preserving structural information and temporal event order. This is achieved via a number of techniques to identify and encode patterns in event sequences so as to avoid recording all individual events, and alleviates the scalability challenge #2. ScalaTrace comes with a tool, ScalaReplay [START_REF] Wu | Elastic and Scalable Tracing and Accurate Replay of Non-Deterministic Events[END_REF], to replay application execution directly from the compact traces. Traces generated by ScalaTrace contain time-related information. As a result, traces must be acquired on a dedicated, homogeneous cluster (scalability challenge #1). Furthermore, trace replay with ScalaReplay must be executed at that same scale (scalability challenge #3). Note that ScalaReplay does not allow for the straightforward exploration of "what if?" scenarios because it performs a "live replay" of the application. In fact, in [START_REF] Wu | ScalaExtrap: Trace-Based Communication Extrapolation for SPMD Programs[END_REF] the authors mention that it would be useful to use ScalaTrace traces with an off-line simulation tool.

Time-Independent Trace Replay project [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF] focuses on scalable trace acquisition and scalable trace replay. These objectives are achieved by eliminating all time-related information from the traces. As a result, trace acquisition can be performed on a heterogeneous, non-dedicated platform (e.g., a set of diverse clusters over a wide-area network), with multiple application processes possibly collocated on each compute node, thereby alleviating scalability challenge #1. The use of time-independent traces complexifies trace replay since a simple scaling of event durations is no longer possible. Instead, one must perform discrete-event simulation based on computation and communication volumes. To this end, Time-Independent Trace Replay leverages the state-of-the-art simulation abstractions and models in SimGrid [START_REF] Casanova | Versatile, scalable, and accurate simulation of distributed applications and platforms[END_REF], which make it possible to run the simulation quickly and on a single computer (e.g., a laptop), thus alleviating the scalability challenge #3. The main scalability drawback of this approach is the trace size (scalability challenge #2).

Given the above considerations, in this work we propose to combine the approaches in ScalaTrace and Time-Independent Trace Replay, namely, to use ScalaTrace's compact traces but in a time-independent manner. The resulting combined approach should allow for the off-line simulation of MPI applications with scalable trace acquisition, scalable trace sizes, and scalable trace replay. To the best of our knowledge, no previously proposed framework has achieved these three objectives simultaneously. More specifically, this work makes the following contributions:

• We extend ScalaTrace's tracing mechanism to record volumes of instructions and produce compact time-independent traces;

• We enable application simulation based on ScalaTrace traces with the SimGrid simulation framework;

• We validate our combined approach through a series of experiments with representative benchmarks.

This paper is organized as follows. In Section 2 we provide necessary background information on Sca-laTrace and Time-Independent Trace Replay. In Section 3, resp. Section 4, we describe and evaluate the combination of the approaches in these two projects in terms of trace collection, resp. off-line simulation. In Section 5 we discuss related work. We conclude in Section 6 with a brief summary of our results and with a discussion of future directions.

Background

ScalaTrace

ScalaTrace [START_REF] Noeth | ScalaTrace: Scalable Compression and Replay of Communication Traces for High-Performance Computing[END_REF] is a tracing tool initially designed to characterize the communication behavior of large-scale applications. Its salient capability is that it produces and processes compact traces of near-constant size, while preserving structural information and temporal event order. This is achieved by taking full advantage of the regularity of the traced applications and implementing several techniques such as calling sequence identification, recursion-folding signatures, location-independent encoding of ranks, event aggregation, or causal cross-node reordering. The building blocks of a ScalaTrace trace are Regular Section Descriptors (RSDs) that capture MPI events in a single loop and Power-RSDs (PRSDs) that specify recursive RSDs nested in multiple loops. Time-related information is also encoded in a scalable way by ScalaTrace [START_REF] Ratn | Preserving Time in Largescale Communication Traces[END_REF]. Instead of logging absolute timestamps, as done in other tracing tools, ScalaTrace keeps track of relative delta times, i.e., the time spent either between two consecutive MPI calls (compute delta) or within an MPI call (communicate delta). In order to preserve meaningful time information while keeping traces compact, ScalaTrace stores statistical information (i.e., average, minimum, and maximum delta times) for each event. This aggregate information is completed with histograms of delta times that better capture outliers and distribution properties. Finally, ScalaTrace distinguishes CPU bursts, and their corresponding histograms, not only by call stack but also by path sequence. Hence, the entry path to and exit path from a loop can be distinguished from the iteration path within the loop.

ScalaTrace comes with a tool, ScalaReplay, that replays traces without having to decompress them. ScalaReplay is itself an MPI program that must be executed with as many processes as that used to obtain the trace. It traverses the trace and triggers all the MPI calls with their original payload sizes. For instance, a RSD such as RSD1: <100, MPI_Send1, MPI_Recv1> leads to issuing a hundred pairs of MPI_Send and MPI_Recv calls and the emulation of the interleaving CPU bursts upon replay. This emulation is either done using sleeps or busy waits of appropriate delays that are stored in the trace. These delays are selected to reflect the distribution across the bins of the corresponding histograms. Significant improvements in ScalaReplay II [START_REF] Wu | Elastic and Scalable Tracing and Accurate Replay of Non-Deterministic Events[END_REF] include an elastic data element representation that allows ScalaTrace to produce compact traces for applications that show inconsistent behavior across time steps.

Time-Independent Trace Replay

Unlike ScalaTrace, the Time-Independent Trace Replay project does not attempt to reduce trace sizes. One of its objective is to improve trace acquisition scalability. This is achieved by eliminating all time-related information from the collected trace, i.e., by collecting only computation volumes (numbers of instructions) and communication volumes (numbers of bytes). This simple idea makes it possible to remove several constraints regarding the nature and scale of the trace acquisition platform. First, it is no longer necessary to acquire a trace on a homogeneous platform. The compute nodes may have different compute speeds, although processors must belong to the same Instruction Set Architecture family (so that computation volumes are comparable). The network topology can be arbitrary as well since network delays are not recorded. Consequently, the trace acquisition platform can be composite, e.g., a set of different clusters interconnected via a wide-area network. Furthermore, since no timing information is recorded, it is possible to "fold" the application execution by running multiple MPI processes on the same compute node provided the RAM capacity of the node is not exceeded. Results in [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF] show how using folding on a composite platform it is possible to acquire a trace for a 16,384-process execution of the LU NAS Parallel Benchmark (NPB) [START_REF] Bailey | The nas parallel benchmarks -summary and preliminary results[END_REF] using either 778 compute nodes aggregated from 18 geographically distant clusters or a single cluster with 20 nodes with each 24 cores and 48 GiB of RAM.

From the perspective of off-line simulation the use of time-independent traces is both a challenge and a scalability opportunity. The challenge stems from the fact that without timing information the off-line simulation cannot be based on simple scaling of communication and computation delays, but must rely on precise discrete-event simulation techniques that use accurate simulation models. The scalability opportunity is that, using state-of-the-art simulation techniques [START_REF] Casanova | Versatile, scalable, and accurate simulation of distributed applications and platforms[END_REF] and in particular techniques designed specifically for simulating MPI applications [START_REF] Bédaride | Toward Better Simulation of MPI Applications on Ethernet/TCP Networks[END_REF], it is possible to conduct such simulations on a small-scale platform, e.g., a laptop computer, rather than using a full-scale platform for "live replay" of the traces. Experimental results in [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF] show that it is possible to simulate application executions with reasonable accuracy (simulated execution times within a few % of actual execution times). The scalability of the simulation is not as convincing. While the simulation techniques are shown to scale in the context of on-line simulation [START_REF] Casanova | Versatile, scalable, and accurate simulation of distributed applications and platforms[END_REF][START_REF] Bédaride | Toward Better Simulation of MPI Applications on Ethernet/TCP Networks[END_REF], the results in [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF] show that the size of the traces, i.e., the large number of individual events encoded in each trace, is a limiting factor. As a result, the simulation time is shown to grow roughly linearly with scale of the simulation.

3 Making ScalaTrace Time-Independent

Motivations

ScalaTrace uses a compact trace format but traces contain time-related information, which limits the scalability of the trace acquisition process. By contrast, Time-Independent Trace Replay removes several of the scalability constraints for trace acquisition thanks to the use of composite platforms and/or "folding". However, its scalability is limited due to the size of the traces, which are not stored compactly. Our first objective in this work is to combine both approaches so as to obtain a solution with scalable trace acquisition and trace size.

In this section we provide motivation for our work by demonstrating that the "folding" technique in Time-Independent Trace Replay, which is key to trace acquisition scalability, cannot be leveraged with timedependent traces (such as ScalaTrace traces). Let us first focus on the recorded compute delays. We execute the LU NPB (Class C with 64 processes) with various folding factors (i.e., number of MPI processes per compute node) on the graphene cluster in the Grid'5000 testbed [START_REF] Bolze | Grid'5000: A Large Scale And Highly Reconfigurable Experimental Grid Testbed[END_REF], which comprises 144 2.53GHz Quad-Core Intel Xeon x3440 nodes. For several values of the folding factor (2, 4, 8, 16, 32), we compare the distribution of the measured delays normalized to those obtained in the non-folded mode (i.e., one MPI process per compute node). Since the processes have to compete for CPU resources in folded executions, a natural, perhaps naive, expectation is that the measured compute delays increase roughly linearly with the folding factor.

Figure 1 shows the distribution of the ratios between the measured delays in a folded execution and those in a non-folded execution, versus the folding factor. We find that, contrary to the naive expectation, folding the execution does not have the same impact on all compute events. If we consider all events (leftmost set of boxes), we see that the Inter-Quartile Range (IQR) is large, meaning that folding impact different measured delays differently. The figure also shows data restricted to the most time-consuming events, keeping the top 50%, 25%, 10%, and 5% events in terms of measured delays. These events, and in particular the top 5% events, have delays roughly multiplied by the folding factor up to a folding factor of 16. For a folding factor of 32, the IQR is very large. We conclude that scaling down measured delays to factor out the impact of Folding Factor (Folded/Non-Folded) Time Ratio

Events

All Top 50% Top 25% Top 10% Top 5%

Figure 1: Distribution of the ratio between the measured delta times of a folded execution and those of a regular execution of the LU benchmark (Class C with 64 processes) across events for different folding factors.

execution folding is non-trivial, hence likely precluding the use of execution folding to increase the scalability of the trace acquisition process. ScalaTrace records the time spent in each MPI call as communication times. These are used by ScalaReplay [START_REF] Ratn | Preserving Time in Largescale Communication Traces[END_REF] to replay communication delays and to determine when certain asynchronous calls have to be emitted. Accurate recording of communication delays is in itself more challenging than that of computation delays. Large platforms are typically shared by multiple users who are granted access to compute nodes via a resource management system (e.g., a batch scheduler). So while compute nodes can be dedicated to a user, network links are typically shared and communication delays can be impacted by so-called nearby jobs [START_REF] Bhatele | There Goes the Neighborhood: Performance Degradation due to Nearby Jobs[END_REF]. Furthermore, communication delays can be impacted by operating system noise and running daemons. In the end, even on a homogeneous platform at scale, communication times can be difficult to measure consistently. Still using the same LU benchmark, but using various instances (B-8, B-16, C-8, C-16), we analyze the variation of the delays recorded by ScalaTrace. For each trace we compute the sum of the products of the number of occurrences of an event and of the recorded average delay for that event, in microseconds. In other words, we compute the cumulative computation time and communication time for each trace. The number of runs is between 5 and 7, depending on the instance.

Figure 2 shows the variability of the cumulative computation and communication times across different runs of the same application, launched using the OAR batch scheduler on Grid'5000. We made sure to request entire compute nodes from the batch scheduler and then used various number of cores within each node as described hereafter. In this way, our results are not impacted by sharing a compute node with other jobs. As expected, the computation times are stable across runs. However, there are some outliers due to executions with different numbers of cores per compute node. For B-8, B-16, and C-8, the first run was executed on 4 nodes using only one core per node, while the other runs used only one node (using all 4 cores). For C-16, the first six runs use one core per node, and the last one uses 4 cores on one node. When all the cores of a node are used, the application suffers from memory bus contention and leads to larger execution times. The number of cores used per node also impacts recorded communication delays, but this impact is counterintuitive: using all 4 cores of a node increases the amount of intra-node communications (i.e., through shared memory), which should be faster than inter-node communications (i.e., over the network). We observe the opposite. This is because the cumulative communication times include time spent in the MPI_Wait function. Slowed down computation, due to memory bus contention, in fact improves process synchronization and reduces these wait times. Overall, these results highlights a drawback of time-related information in traces. Even though the execution environment is homogeneous in terms of compute resources, the process mapping impacts the measured times. It implies that a timed trace must be accompanied with a precise description q q q q q q q q q q q q q q q q q q q q q q q q q q Computation Communication 5.0e+08 Run Number Cumulated Delta Times (in microseconds) of its acquisition platform, including the mapping of processes to cores, to make consistent replay possible.

Instance q q B-16 B-8 C-16 C-8
Regardless of the impact of the mapping of MPI processes to cores and compute nodes, the main observation from Figure 2 is that significant variations in cumulative communication time can be observed across runs. The B-16 and C-8 instances show wide variations (e.g., +76.11% for B-16, -18.24% for C-8). These executions were performed back-to-back with the same compute nodes and for the same mapping of processes to cores. For B-16, a slightly larger cumulative computation time, which leads to smaller wait times, also contributes to the communication time variations. But this is not the case for the third run of the C-8 instance. For this run, the cause of the variation is external (we suspect network contention caused by nearby jobs).

Although time-dependent traces are used in most previously proposed tools, including ScalaTrace, they have several drawbacks. In this section we have provided quantitative evidence that, indeed, a key scalability enhancing technique proposed in [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF], execution folding, is not applicable to time-dependent traces. Furthermore, our results show that obtaining consistent and realistic time measurements when tracing events is not straightforward. The mapping of processes to cores of compute nodes has to be taken into account. More importantly, communication delays can be heavily impacted by multiple factors so that two traces recorded back-to-back on the same set of compute nodes can have widely different recorded communication times. For these reasons, and as in [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF] we completely forego the use of time-dependent traces in this work.

Implementation

The standard approach for instrumenting MPI applications, used by both ScalaTrace and Time-Independent Trace Replay, is to use the MPI profiling layer (PMPI). The MPI standard exposes two interfaces for each MPI function, one prefixed with MPI and the other prefixed with PMPI, the former calling the latter directly. This provides developers with the opportunity to insert their own instrumentation code in the implementation of all MPI_ functions. ScalaTrace inserts pre and post stubs in these functions. In pre stubs the recording of the metric of interest, i.e., time, is stopped for the computing phase that precedes the MPI call and a new record is started for the impending communication phase. Conversely, post stubs stop the recording of the metric for the finished communication phase and start the recording for the impending computation phase. These stop and start actions are implemented by the RecordStat() and ResetStat() functions of the Stat class, which in turn call the end() and start() functions of the child class StatTime. Both these functions call gettimeofday() and the end() functions computes the delta elapsed time since the last call to start(). Finally, events, i.e., trace entries, are associated with two instances of a StatTime object, one for computation phases, and another for communication phases.

Given the above clean design, modifying ScalaTrace so that it records the number of instructions executed in each computation phase instead of time is straightforward. We first added a new class, StatInst, which inherits from the Stat class and overrides the start() and end() functions. Instead of calling gettimeofday, these functions call the PAPI_accum_counters() function that accesses the current value of a specific hardware counter, as provided by the portable PAPI interface [START_REF] Browne | A Portable Programming Interface for Performance Evaluation on Modern Processors[END_REF]. We count instructions using the PAPI_TOT_INS counter as in [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF]. The end() function computes the delta in number of instructions since the last call to start(). Counter initialization and destruction are performed in the MPI_Init and MPI_Finalize functions. The number of bytes transferred in each communication phase is simply computed based on the parameters passed to MPI communication functions. Switching from time-dependent to time-independent tracing with our modified implementation of ScalaTrace is done through a single compilation flag.

Results

Impact on Trace Size

As mentioned in Section 1, the main scalability limit for Time-Independent Trace Replay is the verbosity of its trace format. Without any compression, the trace size directly depends on the number of recorded events. With the instrumentation method and corresponding trace format proposed in [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF], called MinI therein, trace size grows roughly linearly with the number of processes. Reducing trace size was the principal motivation for considering ScalaTrace, whose main strength lies in the compactness of the traces it produces. This compactness preserves the complete sequence of events executed by the processes, which can be retrieved by unrolling a ScalaTrace trace. q q q q q q q q q q q q q q q q 1e+02 1e+03 1e+04 Figure 3 plots the number of events stored in a trace vs. the number of processes for the LU NPB Class B and C. Compact refers to the ScalaTrace format, while unrolled corresponds to the total number of events executed by the processes computed once the trace is unrolled (which is the same number of events in the trace when stored in the verbose Time-Independent Trace format). The total number of events executed grows linearly with the number of processes. This is because in this benchmark each process performs roughly the same amount of computation regardless of the number of processes (i.e., the problem size is scaled with the number of processes). This number also grows by a constant factor of 1.6 when going from class B to class C. For these instances, which perform millions of individual events, the number of compact events stored in ScalaTrace traces remains low (less than one hundred) and almost constant. q q q q q q q q q q q q q q q q 1e+02 2e+02 1e+03 The results in Figure 3 demonstrate the capacity of ScalaTrace to take a full advantage of the regularity of an application execution, in this case the LU NPB. This drastic reduction in the number of stored events directly impacts the trace size. Figure 4 plots trace size vs. number of processes the LU NPB Class B and C, for both the MinI instrumentation method in Time-Independent Trace Replay (data from Figure 3, page 10 in [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF]) and ScalaTrace. With 15 ASCII characters per event on average and millions of events stored, MinI traces can be up to 8.8 GiB for class C with 1,024 processes, with trace size increasing roughly linearly with the number of processes. By contrast, ScalaTrace traces show only moderate size increase when the number of processes increases, remaining below 150 KiB. For a given problem size, we note some slight variations in the trace size that are related to the communication pattern of the application, e.g., an instance executed with only four processes has fewer events than instances with 8 processes. Importantly, increasing the problem size leads to only a slight trace size increase. This is because ScalaTrace stores large volumes (of instructions or bytes) in event histograms to ensure trace compactness.

Figures 3 and4 combined show that replacing the verbose Time-Independent traces used in [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF] by the compact traces produced by ScalaTrace solves the major scalability issue of trace size without losing the capacity to replay the exact sequence of events that composes an MPI application.

Impact of Folded Acquisition

The main advantage of time-independent traces that store volumes of executed instructions instead of delta times to represent the CPU bursts between MPI calls is that such information is oblivious to the acquisition conditions. Indeed, even though several processes share a single CPU or are scattered across heterogeneous compute nodes, the measured numbers of instructions should not vary. This is confirmed by Figure 5 that shows the distribution of the ratio between the measured number of instructions of a folded execution and those of a non-folded execution across events in the trace for different folding factors (for the LU NPB, Class C with 64 processes). The number of measured instructions does not vary significantly when the folding factor increases, except for a few outliers. We attribute these outliers to limited hardware counter resolution and to operating system noise. Overall observed variability is less than 5%. This is by contrast to the results in Figure 1, thus demonstrating that storing time-independent information enables the use of folding for increasing the scalability of the trace acquisition process. While similar good results are reported in [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF] for unrolled (time-independent) traces, the results in Figure˜5 are for compact ScalaTrace (time-independent) traces. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 0.950 0.975 1.000 Folding factor (Folded/Regular) Instruction Ratio Figure 5: Ratios between the measured number of instructions for events in a folded execution and that in of a regular execution of the LU benchmark (Class C with 64 processes) vs. the folding factor.

Replaying ScalaTrace Traces in Simulation 4.1 Motivations

The "live trace replay" implemented by ScalaReplay must be executed on a platform at scale. This limits both the scalability of the analyses and the range of "what if?" scenarios that can be explored. Arguably, ScalaReplay is merely a means for its developers to assess whether the compactness of the traces does not lead to problematic information loss. From this standpoint, requiring a homogeneous platform at scale for trace replay is not necessarily a major drawback. ScalaReplay can actually be used to explore "what if?" scenarios by modifying the content of the traces. For instance, one could scale the delta times in the traces to emulate different compute or network delays such as those expected on a hypothetical platforms. However, such scaling is limited and cannot be used to explore all scenarios, such as those with transient and/or local performance degradations, different mappings of processes to compute nodes, different network topologies, or different collective communication implementations. By contrast, Time-Independent Trace Replay leverages SimGrid to enable the exploration of these scenarios, without any tempering of the traces, using off-line simulation. SimGrid leads to accurate simulation for the communication part of the application thanks to the thoroughly validated network model proposed in [START_REF] Bédaride | Toward Better Simulation of MPI Applications on Ethernet/TCP Networks[END_REF].

Implementation

The latest release of ScalaTrace includes a redesign of both the trace format and replay engine to cope with applications with inconsistent behavior across time steps. ScalaReplay can be used in two different modes: normal or histo. In the normal mode, each process simply traverses the trace and replays events while respecting the structure and temporal ordering of the original application. MPI calls are emitted with their original parameters and mock payloads of the right size. CPU bursts are replaced by sleeps or busy waits whose durations are extracted from the trace. The histo mode enables probabilistic trace replay, meaning that senders randomly select receiver ranks from histograms. Receive operations are not posted upon parsing the trace, but instead postponed according to communication delays stored in the trace. A complex mechanism is implemented to handle pending asynchronous receive calls and to ensure that every send is matched.

In this work, we focus on applications such as the NPB, whose behavior is consistent across time steps. Moreover, the implementations of these applications rely on the MPI parameter MPI_ANY_SOURCE, a wildcard for the sender's rank when doing a receive operation. According to the ScalaTrace development team, this parameter causes problematic behavior when using the histo mode. Consequently, we use the normal mode in all our experiments.

To enable the simulated replay of ScalaTrace traces, we leverage the fact that ScalaReplay is an actual MPI application. As such, it can be simulated seamlessly by SMPI [START_REF] Clauss | Single Node On-Line Simulation of MPI Applications with SMPI[END_REF][START_REF] Bédaride | Toward Better Simulation of MPI Applications on Ethernet/TCP Networks[END_REF]. The SMPI framework comprises: (i) an API that implements all the communication operations of the MPI-2 standard (and a subset of the MPI-3 standard) and all the collective communication algorithms of MPICH2 and OpenMPI with their selection logic; and (ii) an emulated MPI runtime. The API is implemented as one of SimGrid's user APIs, while the runtime is built on top of SimGrid's simulation kernel. SMPI supports MPI applications written in C, C++, or Fortran, which must be linked to the SMPI library. In a simulated execution with SMPI all MPI calls are intercepted, and the simulated duration of each call is determined by underlying simulation models. The compute bursts between the MPI calls are emulated, i.e., code for a CPU burst is actually executed on the simulation host, possibly scaled by a user-specified factor. Note that MPI allows the simulation of unmodified MPI applications (provided that they are compiled with smpicc, provided in the SimGrid distribution, instead of mpicc).

To simulate ScalaReplay with SMPI we had to address a couple of issues. First, simulating ScalaReplay as is with SMPI would be a simulation of the replay engine itself, not of the traced applications. This is because in ScalaReplay the replay logic (i.e., trace traversal, event extraction, delta times computations, . . . ) is interleaved with the MPI calls emitted to replay the traced application. To resolve this issue we have added a command line option to SMPI so that it does not perform the simulation of the CPU bursts, in our case ScalaReplay's replay logic. Second, we have modified ScalaReplay to replace the existing busy waits that emulates the delta times between MPI calls by the appropriate SMPI function, smpi_execute(duration), where duration is the delta time extracted by ScalaTrace. When time-independent traces are replayed, we rely on the smpi_execute_flops(numinst) SMPID function, where numinst is the number of instructions extracted from the trace for the current burst. We have added compilation flags so that ScalaReplay allows simulated replay with time-dependent or time-independent traces.

Results

The main objectives in this work is to extend the capabilities of ScalaTrace by enabling a simulated, as opposed to live, replay of the compact traces it produces. More precisely, thanks to the modifications of the ScalaReplay replay engine described in the previous section and the capabilities offered by SMPI, we can simulate application execution using a single host but based on a ScalaTrace trace that has been obtained on a large-scale platform. This makes it possible to explore "what if?" scenarios, without having to modify the input trace, and in particular match the hardware characteristics of an existing platform.

Figure 6 shows execution time vs. number of processes for various instances of the LU NPB (class B and C). Three kinds of results for shown: (i) Simulated execution times based on unrolled time-independent traces (MinI); (ii) Simulated execution times based on compact time-independent traces (ScalaTrace); and (iii) Real execution times on the graphene cluster described in Section 3. The simulations are instantiated so as to match the characteristics of the real-world cluster. With Time-Independent Trace Replay (i.e., with SimGrid), this simply entails creating an XML file that describes the performance rates of the compute nodes, the latencies and bandwidths of the network links, and the physical network topology. We show results for the real-world cluster only up to 128 processes since this particular cluster has only 144 compute nodes. However, we are able to run simulations of platforms 8 times larger, i.e., of an hypothetical graphene cluster that would have around 1,024 compute nodes. We are able to do this because we can obtain valid time-independent traces on composite, non-homogeneous platforms on which multiple processes can run on q q q q q q q q q q 5 10 20 a single compute node (using folding). Being able to simulate large homogeneous platforms for which we would not have been able to obtain a time-dependent trace is an essential motivation for this work. This experiment is at a relatively modest scale (since many homogeneous clusters with more than 1,024 exist), but regardless of the maximum size of the homogeneous physical cluster at hand, or approach can be used to simulate a larger cluster scalably thanks to its use of time-independent compact event traces.

Overall, results in Figure 6 show that all three curves follow the same trends. There is little difference between the two simulated execution times, showing that simulation accuracy is not significantly diminished by using compact ScalaTrace traces, even though some information is lost in these traces compared to MinI traces. We see that simulation is not very accurate for small-scale executions (i.e., with 8 or 16 processes), with an underestimation of the execution by up to 25.6%). Accuracy improves as the number of processes increases (e.g., relative error with respect to the real-world execution is always below 11.5% for 64 processors or more). Since the main objective is to simulate the execution of applications with thousands of processes, this trend is encouraging. A possible source of simulation inaccuracy is given in [START_REF] Wu | Elastic and Scalable Tracing and Accurate Replay of Non-Deterministic Events[END_REF] and is related to trace parsing in the normal mode. The emission of some MPI calls might be delayed, which increases execution time. This issue is solved by the histo replay mode, but as explained earlier this mode cannot be used for our applications. Further investigation, likely in collaboration with the ScalaTrace team, is necessary to improve the accuracy of the simulated replay even at small scales. Overall, these results show that our approach benefits from and combines the strength of ScalaTrace and Time-Independent Trace Replay so as to push the accuracy and scalability of off-line simulations of MPI applications further.

Related Work

Many simulators have been proposed that, like the simulator described in this work, follow the off-line simulation approach. For instance, since 2009 five off-line simulators of MPI applications have been described in the literature [START_REF] Hoefler | LogGOPSim -Simulating Large-Scale Applications in the LogGOPS Model[END_REF][START_REF] Tikir | PSINS: An Open Source Event Tracer and Execution Simulator for MPI Applications[END_REF][START_REF] Zhai | PHANTOM: Predicting Performance of Parallel Applications on Large-Scale Parallel Machines Using a Single Node[END_REF][START_REF] Hermanns | Verifying Causality between Distant Performance Phenomena in Large-Scale MPI Applications[END_REF]. These simulators use different simulation models to compute simulated durations of CPU bursts and communication operations. Computation delays are typically computed by scaling the durations of the CPU bursts in the trace [START_REF] Tikir | PSINS: An Open Source Event Tracer and Execution Simulator for MPI Applications[END_REF][START_REF] Hermanns | Verifying Causality between Distant Performance Phenomena in Large-Scale MPI Applications[END_REF]. The alternate approach is on-line simulation, by which the actual application code is executed on a host platform that attempts to mimic the behavior of a target platform with different hardware characteristics. Part of the instruction stream is intercepted and passed to a simulator [START_REF] Riesen | A Hybrid MPI Simulator[END_REF][START_REF] Zheng | BigSim: A Parallel Simulator for Performance Prediction of Extremely Large Parallel Machines[END_REF]. We refer the reader to [START_REF] Casanova | Simulation of MPI Applications with Time-Independent Traces[END_REF] for a detailed discussion of state-of-the-art off-line and on-line simulators.

Many tools have be proposed for the post-mortem performance analysis of parallel applications, typically relying to aggregated statistical information [START_REF] Shende | The Tau Parallel Performance System[END_REF][START_REF] Geimer | A Scalable Tool Architecture for Diagnosing Wait States in Massively Parallel Applications[END_REF][START_REF] Knüpfer | The Vampir Performance Analysis Tool-Set[END_REF][START_REF] Knüpfer | Score-P: A Joint Performance Measurement Run-Time Infrastructure for Periscope, Scalasca, TAU, and Vampir[END_REF]. Performance profiles generated by these tools can be used to detect performance bottlenecks, load imbalance, or undesired behaviors such as late senders/receivers or wait states. The common approach to perform this kind of analysis is to rely on pattern detection and visualization rather than on the replay of the traces. Most of these tools can also produce detailed traces in the OTF2 format [START_REF] Eschweiler | Open Trace Format 2: The Next Generation of Scalable Trace Formats and Support Libraries[END_REF] which is much more verbose than that of ScalaTrace.

Recent work from the ScalaTrace team is closely related to this work. ScalaJack [START_REF] Srinath | ScalaJack: Customized Scalable Tracing with In-situ Data Analysis[END_REF] is a redesign of ScalaTrace that uses aspect-oriented programming to allow for user-customizable instrumentation. As such, it provides means to trace numbers of instructions for CPU bursts as explained in Section 3. A combination of ScalaBenchGen, a ScalaTrace tool to automatically benchmarks representative of event traces, and an off-line simulator called xSim has been proposed in [START_REF] Lagadapati | Tools for Simulation and Benchmark Generation at Exascale[END_REF]. xSim relies on parallel discrete event simulation. Is is highly scalable provided it is executed on a large number of resources, whereas our proposed approach executes a SimGrid simulation on a single machine. Also, unlike this work it cannot process traces directly, but instead simulates synthetic benchmark generated from the traces. Note that software for these two projects are not publicly available at the time of writing.

Conclusion and Future Work

In this paper we have proposed an approach for scalable off-line simulations of MPI applications. We have combined two existing and complementary tools, ScalaTrace and Time-Independent Trace Replay, so as to capitalize on their respective strengths while alleviating several of their limitations. We have explained our motivations, described a usable implementation and presented results that illustrate the feasibility and the benefits of the combination. This work was done in the spirit of Open Science and reproducible research. Both ScalaTrace and SimGrid are free software available online. Code, traces, and experiments are publicly available on http://github.com/frs69wq/ScalaTrace-TI. This work has highlighted a number of open issues, which we plan to address in future work in collaboration with the ScalaTrace development team.

Figure 2 :

 2 Figure 2: Variability of cumulated delta times across several traces of different instances of the LU benchmark.

Figure 3 :

 3 Figure 3: Evolution of the number of events stored in traces in the compact format of ScalaTrace and once unfolded for different instances of the LU benchmark.

Figure 4 :

 4 Figure 4: Evolution of the trace size in the MinI and SclaTrace formats for different instances of the LU benchmark.

Figure 6 :

 6 Figure 6: Simulated and actual execution times for class B and C LU benchmark vs. number of processes.

Acknowledgments

This work is partially supported by the SONGS ANR project . Experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER, and several Universities as well as other organizations (see https: //www.grid5000.fr).


Christian Licht
email: clicht@univ-montp2.fr

Somsak Orankitjaroen
email: somsak.ora@mahidol.ac.th

Patcharakorn Viriyasrisuwattana
email: patcharakorn.vi@ssru.ac.th

Thibaut Weller
email: thibaut.weller@univ-montp2.fr

Thin linearly piezoelectric junctions

published or not. The documents may come

Introduction

We present various asymptotic models, indexed by p = (p 1 , p 2 ) ∈ {1, 2, 3, 4} 2 , for a thin piezoelectric junction between two linearly piezoelectric (p 2 = 1) or elastic (p 2 > 1) bodies. Index p 1 is relative to the magnitude of the piezoelectric coefficients of the adhesive, characterized by a single parameter μ, with respect to that of the constant thickness 2ε of a layer containing the adhesive. More precisely, we assume that h := (ε, μ) takes values in a countable set with a sole cluster point h ∈ {0} × [0, +∞] so that: ⎧ ⎪ ⎨ ⎪ ⎩ p 1 = 1 : μ1 := lim h→ h(εμ) ∈ (0, +∞) p 1 = 2 : μ1 := lim h→ h(εμ) = 0, μ2 := lim h→ h(μ/2ε) = +∞ p 1 = 3 : μ2 := lim h→ h(μ/2ε) ∈ (0, +∞) p 1 = 4 : μ2 := lim h→ h(μ/2ε) = 0.

(1)

As previously said, index p 2 characterizes the status of the adherents but also that of the interfaces between adherents and adhesive: ⎧ ⎪ ⎨ ⎪ ⎩ p 2 = 1 : the two interfaces are electromechanically perfectly permeable, p 2 = 2 : the two interfaces are electrically impermeable, p 2 = 3 : one interface is electrically impermeable while the other is electroded, p 2 = 4 : the two interfaces are electroded.

(2)

The space R 3 is assimilated with the physical Euclidean space with basis {e 1 , e 2 , e 3 }. Let be a domain, with Lipschitz- continuous boundary, whose intersection S with {x 3 = 0} is a domain of R 2 of positive two-dimensional Hausdorff measure H 2 (S). Let ± := ∩ {±x 3 > 0} and ε be a small positive number, then adhesive and adherents occupy B ε := S × (-ε, ε), ε ± := ± ± εe 3 , respectively; let ε = ε

+ ∪ ε -, S ε ± := S ± εe 3 , O ε := ε ∪ B ε ∪ ± S ε
± . Let ( mD , mN ), ( eD , eN ) be two partitions of ∂ with H 2 ( mD ), H 2 ( eD ) > 0 and 0 < δ := dist( eD , S). For all in { mD , mN , eD , eN }, ± , ε

± , ε de- notes ∩ {±x 3 > 0}, ± ± εe 3 , ∪ ± ε ± , respectively; if (γ D , γ N ) is a partition of γ := ∂ S, we denote γ D , γ N , γ × (-ε, ε) by ε DI , ε NI , ε lat .
⎧ ⎪ ⎨ ⎪ ⎩ (σ h p , D h p ) = μM I e(u h p ), ∇ϕ h p in B ε ∀p 2 ∈ {1, 2, 3, 4} , (σ h p , D h p ) = M ε E e(u h p ), ∇ϕ h p in ε if p 2 = 1, σ h p = a ε E e(u h p ) in ε if p 2 > 1 (3)
where

(M ε E , a ε E )(x) = (M E , a E )(x ∓ εe 3 ) ∀x ∈ ε ± (4) ⎧ ⎨ ⎩ (M I , M E ) ∈ L ∞ S × ; Lin(K) such that M P = a P -b P b T P c P ; ∃κ > 0, κ|k| 2 ≤ M P (x)k • k, ∀k ∈ K := S × R 3 , a.e. x ∈ , ∀P ∈ {I, E} (5)
and Lin(S 3 ) is the space of linear operators on the space S N of N × N symmetric matrices whose inner product and norm are noted • and | • | as in R 3 (the same notations for the norm and inner product stand also for K).

Lastly we have to add the following conditions on S ε ± :

⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ p 2 = 2 D h p • e 3 = 0 on S ε ± , p 2 = 3 D h p • e 3 = 0 on S ε + , ϕ h p = ϕ h p 0 on S ε -, p 2 = 4 ϕ h p = ϕ h p 0 on S ε ± , (6)
the electric potential ϕ h p 0 being given on S ε + or S ε ± . It will be convenient to use the following notations:

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ k := (ê, ĝ) ê := e αβ , 1 ≤ α, β ≤ 2, ĝ := (g 1 , g 2 ), ∀ k = (e, g) ∈ K ẽ ∈ S 3 ; ẽαβ = e αβ , 1 ≤ α, β ≤ 2, e i3 = 0, 1 ≤ i ≤ 3, ∀ e ∈ S 3 k(r) = k(v, ψ) := e(v), ∇ψ ∀r ∈ H 1 (O; R 3 × R) e(v) ∈ D (S; S 2 ); e(v) αβ = 1 2 (∂ α v β + ∂ β v α ), 1 ≤ α, β ≤ 2, ∀ v ∈ D (S; R 3 ) (7)
and the same symbol e(•) shall also stand for the symmetrized gradient in the sense of distributions of D (O; R 3 ), O ∈ O ε , , \ S, B ε , ε or D (S; R 2 ). An electromechanical state with vanishing electric potential on ε DI and on ε eD when

p 2 = 1 will belong to V ε p := H 1 ε mD (O ε ; R 3 ) × ε p 2 , with ⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩ ε 1 = H 1 ε DI ∪ ε eD (O ε ) ε 2 = H 1 ε DI (B ε ) if H 2 ( ε DI ) > 0, H 1 m (B ε ) if H 2 ( ε DI ) = 0 ε 3 = H 1 ε DI ∪S ε - (B ε ) ε 4 = H 1 ε DI ∪ ± S ε ± (B ε ) (8)
where, for any domain

O of R N , N = 2, 3, H 1 (O; R M ) denotes the subspace of H 1 (O; R M ), M = 1
⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ Given ( f , F , d E , d I ) in L 2 ( ; R 3 ) × L 2 ( mN ; R 3 ) × L 2 ( eN ) × L 2 γ N × (-1, 1) with lat d I = 0 when p 2 = 2 and H 2 γ D × (-1, 1) = 0, ϕ oI in H 3/2 (R) vanishing in {|x 3 | > 1 + δ/2} , and ϕ oE in H 1 ( ) vanishing on S, then: f ε (x) = f (x ∓ εe 3 ) a.e. x ∈ ε ± , f ε (x) = 0 a.e. x ∈ B ε , F ε (x) = F (x ∓ εe 3 ) a.e. x ∈ ε mN± , d ε (x) = (μ/ε) 1/2 d I (x, x 3 /ε) a.e. x ∈ B ε , d ε (x) = d E (x ∓ εe 3 ) a.e. x ∈ ε eN± if p 2 = 1, ϕ h p o (x) = ϕ oE (x ∓ εe 3 ) + ε p DI ϕ oI (x ± (1 -ε)e 3 ) a.e. x ∈ ε ± ε p DI ϕ oI (x, x 3 /ε) a.e. x ∈ B ε (9) where p DI is such that p DI = 0 if ∂ 3 ϕ oI = 0 in S × (-1, 1), p DI = 1 if ∂ 3 ϕ oI = 0 in S × (-1, 1). We also introduce the element ϕ o of H 1,1 ( , S) := {ψ ∈ H 1 ( ) whose trace γ o (ψ) on S belongs to H 1 (S)} defined by ϕ o (x) = ϕ oE (x) + (1 -p D I )ϕ oI (x ± e 3 ) a.e. x ∈ ± . We note ϕ o the trace on γ D of ϕ o and set ϕ oI = 1 2 ϕ oI (•, 1) -ϕ oI (•, -1) .
Then, if M p and L p are defined by:

⎧ ⎪ ⎨ ⎪ ⎩ M p (s, r) := ε M ε E k(s) • k(r) dx + μ B ε M I k(s) • k(r) dx, if p 2 = 1 ε a ε E e(u) • e(v) dx + μ B ε M I k(s) • k(r) dx, if p 2 > 1 L p (r) := f ε • v dx + ε mN F ε • v dH 2 + ε DI ∪ ε d ε ψ dH 2 ε = ε eN if p 2 = 1, ε = ∅ if p 2 > 1. ( 10
)
Seeking an equilibrium state leads to the problem

(P h p ) : Find s h p in (0, ϕ h p 0 ) + V ε p such that M p (s h p , r) = L p (r), ∀r ∈ V ε p which, by
Stampacchia theorem, has a unique solution.

The asymptotic models

In the following C , C will denote various constants independent of h that may vary from line to line. It will be convenient in the cases p 2 > 1 to use the same symbol s h p for (u h p , φh p ) where φh p denotes the extension into ε of ϕ h p by 0.

Without loss of generality, we suppose H 2 ( mD+ ) > 0; moreover, when p 1 = 4, we assume H 2 ( mD± ) > 0, H 2 ( eD± ) > 0.

We recall that if the same symbol u h p denotes a continuous extension from H 1 ( ε

+ ; R 3 ) into H 1 ({x 3 > ε} ; R 3 ) and η is a C ∞ o (R) cut-off function such that η = 1 on [- δ 3 , δ 3 ], 0 ≤ η ≤ 1 and 0 < dη dx 3 ≤ 4 δ on δ 3 ≤ |x 3 | ≤ 2δ 3 , η = 0 on |x 3 | ≥ 2δ 3 (11)
then Korn inequality imply

B ε |∇u h p | 2 dx ≤ S×(-ε,δ-ε) |∇ηu h p | 2 dx ≤ C S×(-ε,δ-ε) |e(ηu h p )| 2 dx ≤ C |e(u h p )| 2 L 2 (B ε ;S 3 ) + |e(u h p )| L 2 ( ε + ;S 3 ) ( 12
)
We will propose our models by studying the asymptotic behavior of s h p when h goes to h in three steps. As announced in [START_REF] Licht | Bonding a linearly piezoelectric patch on a linearly elastic body[END_REF], we may proceed, when p 1 = 1, similarly but by taking due account of the realistic nonvanishing electric loading on ε lat easily handled though the standard inequalities:

ε NI ψ 2 dH 2 ≤ C B ε |∇ψ| 2 dx ∀ψ ∈ H 1 ε NI (B ε ) if H 2 ( ε DI ) > 0 ε lat ψ 2 dH 2 ≤ C B ε |∇ψ| 2 dx ∀ψ ∈ H 1 m (B ε ) if H 2 ( ε DI ) > 0 (13)
When p 1 > 1, we proceed in the spirit of [START_REF] Licht | Dynamics of elastic bodies connected by a thin soft inelastic layer[END_REF][START_REF] Licht | Dynamics of elastic bodies connected by a thin soft viscoelastic layer[END_REF][START_REF] Aitmoussa | Modélisation et études des singularités de contraintes d'un joint collé très mince[END_REF].

Step 1 (a priori estimates): By taking r = s h p -(0, ϕ h p 0 ) in the variational formulation of (P ε p ), (12), (13) and

S |s h p (•, ε) -s h p (•, -ε)| 2 dx ≤ 2ε B ε |∇s h p | 2 dx (14) imply: μ|k(s h p )| 2 L 2 (B ε ;K) + |k(s h p )| 2 L 2 ( ε ;K) ≤ C . ( 15
)
Step 2 (convergence of (s h p )): The following two tools are well suitable to describe the asymptotic behavior of the electromechanical state in the adherents and adhesive, respectively. First, let T ε be the mapping from

H 1 ( ε ; R 3 × R) into H 1 ( \ S; R 3 × R) defined by: (T ε r)(x) = T ε (v, ψ) (x) = (T ε 1 v, T ε 2 ψ)(x) := (v, ψ)(x ± εe 3 ) ∀x ∈ ± . (16) Note that T ε s h p = (T 1 u h p , 0) if p 2 > 1! For any w in H 1 ( \ S; R N ), N ∈ {1, 3}, if γ ± o (w ± ) denotes the trace on S of its restriction w ± to ± , [ [w] ] stands for γ + o (w + ) -γ - o (w -). Next, as for all r of H 1 (B ε ; R 3 × R), one has B ε r(x) - 1 2ε ε -ε r(x, x 3 ) dx 3 2 dx ≤ C ε 2 B ε |∇r| 2 dx (17)
we introduce the following element of L 2 (S; K):

k p (ε, r) := 1 (2ε) q ε -ε k(r)(•, x 3 ) dx 3 , q = max(2 -p 1 , 0) (18)
and, obviously, there holds

k p (ε, r) = (e( U p ), ∇ p ), S h p := (U h p , h p ) := 1 (2ε) q ε -ε s h p (•, x 3 ) dx 3 . ( 19
)
So ( 12), ( 14) and (15) imply:

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ |k(T ε s h p )| L 2 ( \S;K) ≤ C , |[[T ε s h p ]]| 2 L 2 (S;R 3 ×R) ≤ C ε + ε μ |k p (ε, s h p )| 2 L 2 (S;K) ≤ C ε -2q • ε μ |U h p | 2 L 2 (S;R 3 ) ≤ C ε 2(1-q) 1 + ε μ ε 2(q-1) | h p | 2 L 2 (S) ≤ Cc * (h) c * (h) = 1 + ε μ if p 2 = 1, 1 εμ if p 2 = 2, ε 2 + ε μ if p 2 > 2 |e( U h p ), ∇ h p | 2 L 2 (S;S 2 ×R 2 ) ≤ C 1 ε 2q • ε μ |S h p -γ ± o ((T ε s h p ) ± )| 2 L 2 (S;R 3 ×R) ≤ C ε + ε μ if p 1 = p 2 = 1 |U h p -γ ± o ((T ε 1 u h p ) ± )| 2 L 2 (S;R 3 ) ≤ C ε + ε μ if p 1 = 1, p 2 > 1. (20)
Thus, if a ⊗ S b denotes the symmetrized tensor product of a and b in R 3 , we deduce:

Proposition 2.1. There exists sp = ( ūp , φp ) in H 1 mD ( \ S; R 3 ) × H 1 eD ( \ S) such that T ε s h p weakly converges in H 1 ( \ S; R 3 × R) toward some sp ; φp = 0 when p 2 > 1 and sp belongs to H 1 ( ; R 3 × R) when p 1 ≤ 2.
When p 1 = 4, k p (ε, s h p ) weakly converges in L 2 (S; K) toward some kp = (ē p , ḡp ), and there exists

(U p , p ) in H 1 (S; R 3 × R) such that ( U h p , h p ) converges weakly in H 1 (S; R 2 × R) toward ( U p , p ), (U h p ) 3 converges strongly in L 2 (S) toward (U p ) 3 . Moreover i) when p 1 = 1, U p = γ 0 ( ūp
)
for all p 2 , while ¯ p is equal to γ 0 ( φp ) when p 2 = 1 or to γ 0 (ϕ 0 ) when p 2 ≥ 3. Furthermore, the trace on γ D of ¯ p is equal to φo while kp = e( ūp ), ∇ ¯ p , ( ḡp ) 3 = ϕ oI when p 2 = 4;

ii) when p 1 = 2, U p = 0, ¯ p = 0 and kp = 0; iii) when p 1 = 3, U p = 0 and ēp = [ [ ūp ] ] ⊗ S e 3 for all p 2 , while ¯ p and ḡp vanish only when p 2 = 2, ( ḡ(3,1)

) 3 = [ [ φp ] ], ( ḡ(3,4) ) 3 = 0.
Actually, in the next step, we will show that ( ūp , φp ) is necessarily the unique solution to a variational problem so that the whole sequences converge. To identify kp , when p 1 > 1, it suffices to go to the limit in the identity

B ε k(s h p )(x) • τ i (x) + s h p (x) • div τ i (x) dx = S [[T ε 1 u h p ]] i θ 1 + [[T ε 2 ϕ h p ]] θ 2 dx (21)
with τ i = (θ 1 e 3 ⊗ S e i , θ 2 e 3 ), θ j ∈ C ∞ o (S), j = 1, 2, i = 1, 2, 3 and to use the convergence to 0 in the sense of distributions of k p (ε, s h p ) by due account of (20)!

Step 3 (identification of (s p , kp )):

When p 1 > 1, we simply go to the limit in the variational formulation of (P h p ) by using suitable test-functions r ε p . For all w 1 in H 1 mD ( ; R 3 ) and all ζ 1 in H 1 eD ( ) vanishing in a neighborhood of γ D , let (w 1,ε , ζ 1,ε ) be defined by

w 1,ε , ζ 1,ε (x) = (w 1 , ζ 1 )(x ∓ εe 3 ) a.e. x ∈ ε ± (w 1 , ζ 1 )(x, 0) a.e. x ∈ B ε For all w 2 in H 1 mD ( \ S; R 3 ) and all ζ 2 in H 1 eD ( \ S) vanishing in a neighborhood of γ D , let (w 2,ε , ζ 2,ε ) be defined by (w 2,ε , ζ 2,ε )(x) = (w 2 , ζ 2 )(x ∓ εe 3 ) a.e. x ∈ ε ± (w a , ζ a )(x, x 3 /ε) + |x 3 | ε (w s , ζ s )(x, x 3 /ε) a.e. x ∈ B ε with (w a , ζ a )(x) = 1 2 [(w 2 , ζ 2 )(x, x 3 ) -(w 2 , ζ 2 )(x, -x 3 )] (w s , ζ s )(x) = 1 2 [(w 2 , ζ 2 )(x, x 3 ) + (w 2 , ζ 2 )(x, -x 3 )] Note (see the proof of Lemma 4.1 of [3]) that e(w 2,ε ) - [[w 2 ]] ⊗ S e 3 2ε , ∇ζ 2,ε - [[ζ 2 ]]e 3 2ε L 2 (B ε ;K) ≤ C |(w 2 , ζ 2 )| H 1 (B ε \S;R 3 ×R) . So r ε p = (v ε p , ψ ε p ) reads as: ⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩ v ε p = w min (p 1 -1,2),ε , 1 ≤ p 2 ≤ 4 ψ ε p = ⎧ ⎪ ⎨ ⎪ ⎩ ζ min (p 1 -1,2),ε p 2 = 1 (θ 1 + x 3 θ 2 )/ε, θ 1 , θ 2 ∈ C ∞ 0 (S) p 1 = 3, 4, p 2 = 2 (1 + x 3 /ε)θ, θ ∈ C ∞ 0 (S) p 1 = 3, 4, p 2 = 3 0 i f (2 ≤ p 1 ≤ 4, p 2 = 4) or (p 1 = 2, p 2 = 2, 3) (22)
When p 1 = 1, one proceeds in two steps. First we prove

(M kp ) 2 p = 0 ( 23
)
where k i p denotes the projection on K i p of any element k of K with:

⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ K = K 1 p ⊕ K 2 p ⊕ K 3 p , p 2 ≤ 2 : K 1 p := {(e, g) ∈ K; e i3 = 0, g 3 = 0} , K 2 p := (e, g) ∈ K; ê = 0, ĝ = 0 , K 3 p := {0} , p 2 > 2 : K 1 p = (e, g) ∈ K; e i3 = 0, ĝ = 0 , K 2 p = (e, g) ∈ K; ê = 0, g = 0 , K 3 p = {(e, g) ∈ K; e = 0, g 3 = 0} ( 24
)
For that, we simply use test functions ρ ε p built as follows: given

(w, ψ) in C ∞ o (S; R 3 × R), ρ ε p (x) = (x 3 + ε)(w, I p 2 ψ)(x) a.e. x ∈ B ε 2ε(w + , I p 2 ψ + )(x -εe 3 ) in ε + , 0 in ε - (25)
where (w

+ , ψ + ) is an extension into H 1 mD+ ( + ; R 3 ) × H 1 eD+ ( + ) and I p 2 = 1 if p 2 ≤ 2, I p 2 = 0 if p 2 > 2.
Hence, as Proposition 2.1 yields ( kp ) 3 p = 0, we deduce

(M I kp ) 1 = M I p ( kp ) 1 ; M I p := M 11 Ip -M 12 Ip (M 22 Ip ) -1 M 21 Ip (26) with M ij Ip , 1 ≤ i, j ≤ 3 being the decomposition of M I in linear operators mapping K i p into K j p . Next, given (v, ψ) in H 1 mD ( ; R 3 ) × H 1 eD ( ) ∩ H 2 ( ; R 3 × R) ψ vanishing in a neighborhood of γ D , we define r ε p = (v ε p , ψ ε p ) by: ⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩ v ε p (x) = v(x, 0) -x 3 ∇ v 3 (x, 0), v ε 3 (x) = v 3 (x, 0) a.e. x ∈ B ε v ε p (x) = v(x ∓ εe 3 ) ∓ εR ± ∇ v 3 (•, 0), 0 (x ∓ εe 3 ) a.e. x ∈ ε ± ψ ε p (x) = ψ(x ∓ εe 3 ) in ε ± , ψ(x, 0) in B ε if p 2 = 1 ψ ε p (x) = ψ(x, 0) in B ε if p 2 = 2 ψ ε p (x) = 0 in B ε if p 2 ≥ 3 (27)
where R ± is a continuous lifting operator from H 1/2 (S; R 3 ) into H 1 mD± ( ± ; R 3 ). As ( e( v), ∇ψ) belongs to K 1 p almost everywhere in S, (26) yields lim

h→ h B ε μM I k(s h p )k(r ε p ) dx = S M I kp • ( e( v), ∇ψ) dx = S M Ip ( k1 p ) • ( e( v), ∇ψ) dx (28)
while, obviously, we have:

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ lim h→ h ε M ε E k(s h p ) • k(r ε p ) dx = M E k(s p ) • k(v, ψ) dx if p 2 = 1 lim h→ h ε a ε E e(u h p ) • e(v ε p ) dx = a E e( ūp ) • e(v) dx if p 2 ≥ 2 lim h→ h L (1,p 2 ) (r ε (1,p 2 ) ) := ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ f • v dx + mN F • v dH 2 + eN d E ψ dH 2 + ( μ1 ) 1/2 γ N 1 -1 d I (•, x 3 ) dx 3 ψ dH 2 p 2 = 1 f • v dx + mN F • v dH 2 + ( μ1 ) 1/2 γ N 1 -1 d I (•, x 3 ) dx 3 ψ dH 2 p 2 ≥ 2 (29)
Lastly, Jensen inequality and the previously established weak convergences achieve the proof of the following convergence result which supports our asymptotic models in the form of variational problems P p where the convention ∞ × 0 = 0 is understood.

Theorem 2.1. If p 2 = 1, when h goes to h, T ε s h p converges strongly in H 1 ( \ S; R 3 × R) toward sp the unique solution to P (p 1 ,1) :

Find (u, ϕ) in (0, ϕ o ) + V p 1 × p 1 such that M (p 1 ,1) (u, ϕ), (v, ψ) = L (p 1 ,1) (v, ψ) ∀(v, ψ) ∈ V p 1 × p 1
where

M (p 1 ,1) (u, ϕ), (v, ψ) := ⎧ ⎪ ⎨ ⎪ ⎩ M E k(u, ϕ) • k(v, ψ) dx + μ1 S M Ip e( û), ∇ϕ • e( û), ∇ϕ dx p 1 = 1 M E k(u, ϕ) • k(v, ψ) dx + μ2 S M I [[u]] ⊗ S e 3 , [[ϕ]]e 3 • [[v]] ⊗ S e 3 , [[ψ]]e 3 dx p 1 ≥ 2 L (p 1 ,1) (v, ψ) := ⎧ ⎨ ⎩ f • v dx + mN F • v dH 2 + eN d E ψ dH 2 + ( μ1 ) 1/2 γ N 1 -1 d I (•, x 3 ) dx 3 ψ dH 2 p 1 = 1 f • v dx + mN F • v dH 2 + eN d E ψ dH 2 p 1 ≥ 2 V 1 := v ∈ H 1 mD ( ; R 3 ); v ∈ H 1 (S; R 2 ) , V 2 := H 1 mD ( ; R 3 ), V 3 = V 4 := H 1 mD ( \ S; R 3 ) 1 := ψ ∈ H 1 eD ( ); ψ ∈ H 1 γ D (S) , 2 := H 1 eD ( ), 3 = 4 := H 1 eD ( \ S) If p 2 > 1, when h goes to h, (T ε 1 u h p , h p ) converges strongly in H 1 ( \ S; R 3 ) × H 1 (S) toward ( ūp , p ) the unique solution to P p : Find (u, φ) in 0, q γ o (ϕ o ) + V p 1 × p such that M p (u, φ), (v, ψ) = L p (v, ψ) ∀(v, ψ) ∈ V p 1 × p where M (1,p 2 ) (u, φ), (v, ψ) := ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ a E e(u) • e(v) dx + μ1 S M Ip e( û), ∇φ • e( v), ∇ψ dx p 2 = 2 a E e(u) • e(v) dx + μ1 S M Ip e( û), ( ḡp ) 3 ) • e( v), 0 dx p 2 ≥ 3 with M Ip e( û), ( ḡp ) 3 ) • 0, e 3 ) = 0 if p 2 = 3 ( ḡp ) 3 = ϕ oI if p 2 = 4 a E e(u) • e(v) dx + μ1 S M I,p (e( û), ϕ oI ) • (e( v), 0) dx p 2 = 4 L (1,p 2 ) (v, ψ) := f • v dx + mN F • v dH 2 + ( μ1 ) 1/2 γ N 1 -1 d I (•, x 3 ) dx 3 ψ dx ∀p 2 ≥ 2 and for 2 ≤ p 1 , p 2 ≤ 4 M (p 1 ,p 2 ) (u, φ), (v, ψ) := ⎧ ⎨ ⎩ a E e(u) • e(v) dx + μ2 S M I ([[u]] ⊗ S e 3 , (∇φ, ( ḡp ) 3 )) • ([[v]] ⊗ S e 3 , (∇ψ, 0)) dx with M I ([[u]] ⊗ S e 3 , (∇φ, ( ḡp ) 3 )) • (0, e 3 ) = 0 if p 2 = 2, 3 ( ḡp ) 3 = 0 if p 2 = 4 or p 1 = 2 L (p 1 ,p 2 ) (v, ψ) = f • v dx + mN F • v dH 2 (p 1 ,2) := H 1 γ D (S) or H 1 m (S) according to the positivity of the length of γ D , (p 1 ,3) = (p 1 ,4) := {0} , p 1 = 2 (2,p 2 ) := {0} , 2 ≤ p 2 ≤ 4

Concluding remarks

In the case of piezoelectric adhesive and adherents (p 2 = 1), our results extend those obtained in elasticity (see [START_REF] Aitmoussa | Modélisation et études des singularités de contraintes d'un joint collé très mince[END_REF][START_REF] Acerbi | Thin inclusions in linear elasticity: a variational approach[END_REF][START_REF] Licht | A modelling of elastic adhesive bonded joints[END_REF][START_REF] Licht | Some new mathematical modelings of junctions[END_REF]).

The asymptotic behavior of the adhesive strongly depends on the magnitude of the stiffness compared to that of the thickness. When the magnitude of the stiffness is of the order of the inverse of the thickness, the adhesive is replaced by a material piezoelectric surface perfectly bonded to the adherents. When it is lesser, the adhesive is replaced by an electromechanical constraint between the two adherents which can be perfect adhesion, electromechanical pull-back or free separation, according to the order of magnitude of the stiffness which is, respectively, larger, equal or lower than that of the thickness.

There is a large discrepancy between our results and that of [START_REF] Serpilli | An asymptotic model of a multimaterial with a thin piezoeletric interphase[END_REF][START_REF] Serpilli | Asymptotic analysis of a multimaterial with a thin piezoelectric interphase[END_REF] obtained by formal or questionable arguments. Similarly, in the case of a thin piezoelectric layer embedded between two elastic adherents, depending on the magnitude of the stiffness, the adhesive is replaced by a material elastic surface perfectly bonded to the adherents or by a mechanical constraint between the adherents. In the case of electrically impermeable interfaces, the material surface has a non-local elastic behavior (since the additional state variable of electric nature φ can be eliminated!), the constitutive equations being derived from the asymptotic behavior of a thin piezoelectric plate acting as a sensor (case p = 1 in [START_REF] Weller | Asymptotic modeling of thin piezoelectric plates[END_REF]). When one interface is electrically impermeable while the other is electroded, the material surface is an elastic membrane. When the two interfaces are electroded, the material surface is an elastic membrane with residual stress. In these last two cases, the constitutive equations are derived from the asymptotic behavior of a thin piezoelectric plate acting as an actuator (case p = 2 in [START_REF] Weller | Asymptotic modeling of thin piezoelectric plates[END_REF]). The mechanical constraint is perfect adhesion, elastic pull-back or free separation according to the order of magnitude of the stiffness. In the case of electrically impermeable interfaces, the elastic pull-back is of non-local nature (since the state variable of electric nature φ, additional to the relative displacement, can be eliminated). In the two other cases, the elastic pull-back is local. When the two interfaces are electroded, it is similar to the purely elastic case, while, if only one interface is electroded, piezoelectric and dielectric coefficients enter the constitutive equations.

The realistic dual situation [START_REF] Geis | Asymptotic models for piezoelectric stack actuators with thin metal inclusions[END_REF] in which an elastic layer is embedded between two piezoelectric bodies can be treated within the same framework.

  The structure made of the adhesive and the two adherents, perfectly stuck together along S ε ± , is clamped on ε mD , subjected to body forces of density f ε and to surface forces of density F ε on ε

	a given electric potential ϕ h p 0 is applied on ε	mN and vanishing on ε lat . Moreover,
	If σ h p , u h p , e(u h p ), D h p , ϕ h

DI and, when p 2 = 1, on ε eD , while electric charges of density d ε appear on ε NI and, when p 2 = 1, on ε eN .

p stand for the fields of stress, displacement, strain, electric displacement and electric potential, respectively, the constitutive equations of the structure, for all p 1 in {1, 2, 3, 4}, read as: