Dorian Leroy
email: dorian.leroy@inria.fr

Benoît Lelandais
email: benoit.lelandais@cea.fr

Marie-Pierre Oudot
email: marie-pierre.oudot@cea.fr

Monilogging for Executable Domain-Specific Languages

Keywords: executable DSLs, runtime monitoring, logging Binary.eval() VarRef.eval() FunctionCall.eval()

Runtime monitoring and logging are fundamental techniques for analyzing and supervising the behavior of computer programs. However, supporting these techniques for a given language induces significant development costs that can hold language engineers back from providing adequate logging and monitoring tooling for new domain-specific modeling languages. Moreover, runtime monitoring and logging are generally considered as two different techniques: they are thus implemented separately which makes users prone to overlooking their potentially beneficial mutual interactions. We propose a language-agnostic, unifying framework for runtime monitoring and logging and demonstrate how it can be used to define loggers, runtime monitors and combinations of the two, aka. moniloggers. We provide an implementation of the framework that can be used with Java-based executable languages, and evaluate it on 2 implementations of the NabLab interpreter, leveraging in turn the instrumentation facilities offered by Truffle, and those offered by AspectJ.

CCS Concepts: • Software and its engineering → Domain specific languages; Software testing and debugging.

Introduction

Complex and data-intensive behaviors such as those observed in scientific computing require insightful data to analyze, whether for verification, validation, or maintenance purposes. Logging and runtime monitoring are two fundamental techniques used -mostly by software developers -to gather such data about the health and behavior of a running system [START_REF] Brat | Experimental evaluation of verification and validation tools on martian rover software[END_REF][START_REF] Pecchia | Industry practices and event logging: Assessment of a critical software development process[END_REF][START_REF] Schiavio | Nodemop: Runtime verification for node. js applications[END_REF][START_REF] Yuan | Characterizing logging practices in open-source software[END_REF]. However, both techniques are usually employed independently and in an ad hoc way, requiring the use of dedicated language constructs (e.g., printf), or of language-specific libraries written in the same language as the program being instrumented (e.g., calls to the Logger class in log4j) [START_REF] Chen | A Survey of Software Log Instrumentation[END_REF].

While this is acceptable for developments involving a single general purpose language (GPL), it becomes problematic when dealing with multiple languages, in particular with executable domain-specific languages (DSLs), for two main reasons. First, due to the restricted expressiveness of an executable DSL, equipping it with logging and runtime monitoring facilities requires to introduce the corresponding concerns into the language, even though such concerns might not be part of its domain. Second, the fact that logging and runtime monitoring libraries are dedicated to specific languages (e.g., Apache log4x for C++, PHP, Java and .NET) means that to reuse them in an opportunistic way for an executable DSL, logging support must be incorporated directly into the execution semantics of the DSL. This goes against the principle of separation of concerns, and induces significant language development costs. There is thus a need for language-agnostic runtime monitoring and logging to provide such capabilities for users of any modeling language and reducing development costs of individual DSLs.

Moreover, runtime monitoring and logging often depend on one another. While both techniques can be used on their own, there are numerous examples of them being used in a complementary manner. For example, runtime monitoring frameworks can operate on data collected from the system through logging, or, conversely, logging can be triggered or altered when specific behaviors are detected by runtime monitors [START_REF] Adolf | Stream runtime monitoring on UAS[END_REF]. Yet, a unified framework fitting both the runtime monitoring and the logging needs of modelers -and especially of domain experts with no technical background in computer science -still has to be proposed.

To tackle this challenge, we propose MoniLog, a reusable domain-specific language dedicated to the definition of moniloggers, which combine runtime monitoring and logging in a polyglot manner. MoniLog users define moniloggers in external specification files, using domain concepts to indicate how their model should be instrumented. A standard library provides basic logging functionalities to users of the language, who can then extend it with libraries tailored to their specific needs. Moniloggers can be triggered upon occurrences of their associated event. Whether a monilogger is actually triggered or not can be further conditioned by properties on the execution state of both the running model and the MoniLog framework. These properties can be expressed with the expression language embedded in MoniLog, or using languages available on the execution platform (e.g., Python or JavaScript on GraalVM [START_REF] Würthinger | One VM to rule them all[END_REF]). The actions triggered by moniloggers consist in logging messages or emitting events (both being able to carry data extracted from the execution state), evaluating expressions, and starting or stopping moniloggers.

The approach is implemented as a language with an Xtext concrete syntax, and with an operational semantics leveraging two dynamic instrumentation mechanisms: Truffle [START_REF] Van De | Fast, flexible, polyglot instrumentation support for debuggers and other tools[END_REF] and AspectJ [START_REF] Kiczales | An overview of AspectJ[END_REF]. We demonstrate the expressiveness of the approach and the interactions of logging and runtime monitoring through a demonstration case on the NabLab language [START_REF] Lelandais | Fostering metamodels and grammars within a dedicated environment for HPC: the NabLab environment (tool demo)[END_REF], which is dedicated to numerical simulations. We evaluate the overhead induced by the approach over three representative monilogging scenarios for NabLab, and show that this overhead ranges from 14% on executions shorter than 30s, to 46% on executions shorter than 10s (overhead included), making it suitable for debugging activities.

The remainder of this paper is organized as follows. Section 2 presents the background and the motivation behind this work. Section 3 provides an overview of the contribution. Section 4 introduces the MoniLog language. Section 5 defines the semantics of MoniLog. Section 6 discusses the implementation details of MoniLog. Section 7 presents our evaluation of the proposed approach. Section 8 discusses related work. Finally, a conclusion and future research directions are presented in Section 9.

Background & Motivation

In this section, we first introduce the techniques known as runtime monitoring and logging, and precisely scope the executable DSLs we consider in the approach. We then motivate our approach using an illustrative example.

Background

Runtime Monitoring. Runtime monitoring consists in observing the internal state of a system during its execution to check whether the system satisfies or violates, on this particular execution, a correctness specification usually provided as a temporal property [START_REF] Francalanza | A foundation for runtime monitoring[END_REF][START_REF] Leucker | A brief account of runtime verification[END_REF]. In this paper, we focus on a refined category of online monitoring called synchronous monitoring [START_REF] Cassar | A survey of runtime monitoring instrumentation techniques[END_REF]. With synchronous monitoring, the runtime monitor is tightly coupled with the system. More precisely, the execution of the system is suspended whenever the monitor checks, according to past events, whether a received event violates the property or not. Compared to asynchronous monitoring, this allows timely (as opposed to late) detection of property violation or satisfaction, at the cost of performance decrease.

Logging. Logging is a technique that consists in collecting data on the behavior of a system. It is widely used to perform activities such as failure characterization, debugging, security analysis, or performance modeling [START_REF] Pecchia | Industry practices and event logging: Assessment of a critical software development process[END_REF]. While several well-established logging frameworks such as IBM's Common Event Infrastructure or Apache log4x are dedicated to the collection of event logs, the insertion of logging points in the system is left to the developer. Thus, logging points are introduced statically in the resulting system, for example by using Aspect-Oriented Programming or by inserting direct calls to the chosen logging API(s) [START_REF] Chen | A Survey of Software Log Instrumentation[END_REF].

Executable DSLs. An executable DSL is composed of an abstract syntax defining the concepts of the considered domain and an execution semantics defining the meaning of these concepts 1 . In this paper, we focus on DSLs where (𝑖) the abstract syntax is provided as a metamodel defined using a metamodeling language (e.g., MOF [START_REF]Meta Object Facility (MOF) Core Specification[END_REF] or Ecore [START_REF] Steinberg | EMF: Eclipse Modeling Framework, 2nd Edition[END_REF]), and (𝑖𝑖) the execution semantics is provided as an operational semantics (i.e., an interpreter).

The considered operational semantics are those composed of a data structure representing the model state and a set of execution rules altering this model state. The model state is defined in an execution metamodel extending the abstract syntax metamodel using a non-intrusive extension mecha- nism, such as package merge [START_REF]OMG Unified Modeling Language (OMG UML)[END_REF] or the open class mechanism [START_REF] Clifton | MultiJava: Modular open classes and symmetric multiple dispatch for Java[END_REF]. The execution rules then perform an in-place endogenous transformation on this model state. This model transformation effectively results in the execution of the model, and can be implemented by various means (e.g., programming languages, model transformation languages).

Figure 1 shows an excerpt of the definition of NabLab, an open-source executable DSL dedicated to numerical simulation 2 , which will be used as a running example throughout this paper. The abstract syntax of NabLab is defined as a metamodel (a. in Figure 1), and its operational semantics is composed of a metamodel representing the model state, and a set of execution rules operating on this model state (respectively b. and c. in Figure 1). A Module contains Variable, Iterator, Job, and Function elements. This is exemplified on Figure 2, which shows an excerpt of a NabLab module solving the heat equation. This module declares five variables (lines 3-4), one iterator (line 6), and a job (line 8) operating on variables defined by the module and the iterators.

Job elements constitute the top-level callable elements, and are called implicitly by their containing Module. The ordering of those calls is delegated to the execution semantics defined in the Module.execute rule, and relies on the 2 https://github.com/cea-hpc/NabLab data flow encompassing all jobs in the module. Job elements contain a sequence of Instruction elements to be executed when the job is called. For the sake of brevity, we only cover here the three main types of instruction of the language: (𝑖) VarDec instructions declare a new Variable, (𝑖𝑖) VarAssign instructions assign the result of an Expression to a Variable, and (𝑖𝑖𝑖) Loop instructions iterate through their index over an array of values resulting from the evaluation of an Expression. The values manipulated by NabLab programs result from the evaluation of Expression elements, which can be UnaryExpressions (e.g., negation, sign inversion), BinaryExpressions (e.g., arithmetic operations over scalars, vectors and matrices), FunctionCalls with a set of args, or VarRefs.

In Figure 2, the ComputeUn job iterates over the array of all cells in the discretized spatial domain (i.e., the mesh), which is obtained though a call to the cells() function. For each such cell j, the loop assigns a value to u for the current n iteration (denoted by u^{n+1}{j}). The assigned value depends on (𝑖) the value of f for j, as denoted by f{j}, (𝑖𝑖) the value of u for j on the previous iteration, as denoted by u^{n}{j}, (𝑖𝑖𝑖) the value of outgoingFlux for j, as denoted by outgoingFlux{j}.

Motivation

Runtime monitoring and logging frameworks are languagespecific, being targeted at GPLs. This means that, for frameworks relying on instrumentation directed by external specifications, the developer has to identify -generally through some form of query language (e.g., AspectJ pointcuts) -the locations to be instrumented to send events to runtime monitors [START_REF] Cassar | A survey of runtime monitoring instrumentation techniques[END_REF] or to act as logging points [START_REF] Amir-Mohammadian | Correct audit logging: Theory and practice[END_REF][START_REF] Shea | Lis is more: Improved diagnostic logging in sensor networks with log instrumentation specifications[END_REF]. For frameworks that provide an API to call directly, this is more intrusive as developers have to introduce these calls in their code. In both cases, the mechanisms offered by the framework are dedicated to a specific language. This remains reasonable in the context of GPLs, as (𝑖) the costs induced by developing and maintaining a framework relying on instrumentation are offset by the number of potential users, and (𝑖𝑖) GPLs offer enough expressiveness to provide a framework relying on a set of APIs to be directly called by developers. However, in the context of DSLs, several obstacles arise that hamper the provision of such frameworks.

Indeed, runtime monitoring and logging must be performed at the domain level to be usable by modelers with no technical background in computer science. For frameworks relying on instrumentation, this means adapting them to the syntactic constructs of each new DSL to be supported, which is tedious and error-prone. Frameworks relying on direct calls to their APIs cannot be used with DSLs without ignoring a few principles of MDE for two main reasons. First, enabling such external API calls may require to introduce syntactic constructs that are not part of the domain of the DSL. Second, this hampers proper separation of concerns, as runtime monitoring and logging concerns become intertwined with the domain concerns addressed by the model.

For example, in the case of NabLab, output operations are designed for production use, not debugging. Consequently, these operations allow to output the content of specific variables (e.g., u in Figure 2), at a specific frequency, both specified in the configuration file of a NabLab application. Thus, to implement conditional logging of a variable, it would be possible to modify the model to compute an artificial variable and output its content under certain circumstances, but this would require to alter the original model and application configuration. Moreover, this workaround (𝑖) does not allow to output intelligible logging messages that help understanding the underlying defects in the model, and (𝑖𝑖) might not be feasible altogether, if the expressivity of the language does not allow to derive the data required for debugging.

In addition, even when frameworks are available for a given GPL, runtime monitoring and logging are often used separately, each technique instrumenting a system independently. Yet, both techniques can benefit from being integrated together. For example, in NabLab, runtime monitoring can be used to monitor the trend of specific (possibly derived) variables, and produce logs displaying the values of those variables whenever they fail to follow the expected trend. However, while the techniques are mutually beneficial, existing frameworks for runtime monitoring and for logging do not interact with one another out of the box. Realizing the integration of such frameworks in the context of GPLs mostly means forgoing some of the benefits of runtime monitoring and logging frameworks. In the context of executable DSLs, it also means working at the implementation level of the DSL, when the language permits it (e.g., black-box language interpreters typically prevent this).

In summary, existing frameworks for runtime monitoring or logging fulfill their purpose because they are dedicated to GPLs, which have the required expressivity to both define and use them. In the case of DSLs, where this expressivity cannot be assumed, there is a need for language-agnostic approaches enabling both logging and runtime monitoring at the domain level. Furthermore, as runtime monitoring and logging often interact with each other, such an approach needs to unify both techniques for modelers to benefit from these interactions while keeping the domain abstractions brought by DSLs. In the next section, we provide an overview of our proposed approach to tackle these challenges.

Overview

Figure 3 provides an overview of the proposed approach for monilogging, and supported by our Monilog language and its associated framework. On the left part, the figure shows in grey the hypotheses of the proposed approach. We consider executable DSLs that are composed of an abstract syntax allowing to define models, and an operational semantics. More precisely, we restrict the scope of the proposed approach to languages whose operational semantics is defined according to the visitor or the interpreter design patterns. From such a definition of an executable DSL, an interpreter is generated to run on top of one or several specific execution platforms (e.g., GraalVM or the HotSpot JVM), and support the execution of the conforming models. We also assume that the execution platform supports dynamic pointcut mechanisms (e.g., Truffle for GraalVM, AspectJ for any JVM), which is depicted on the middle lower part of Figure 3 by the depends on relation between the dynamic pointcut implementation and the execution platform.

On the middle upper part, Figure 3 shows in orange an instrumentation interface defined by language engineers for a specific executable DSL. This instrumentation interface bridges the gap between the DSL and the MoniLog language, both at design time and at runtime. At design time, it allows MoniLog users to define moniloggers for models conforming to the DSL that the interface complements. At runtime, this interface is used to instrument the interpreter executing those models with the defined moniloggers, through the generation of a dynamic pointcut implementation. On the right part, Figure 3 shows in blue the MoniLog language and associated framework constituting the contribution of this paper. The MoniLog language allows to define moniloggers, i.e., runtime monitors, loggers and combinations of both. Moniloggers are mapped to ECA (Event-Condition-Action) rules, which have been used to model reactive behavior in various domains [START_REF] Emanuel Almeida | Event-condition-action systems for reconfigurable logic control[END_REF][START_REF] Bailey | An event-condition-action language for XML[END_REF][START_REF] Perumal | ECA-based interoperability framework for intelligent building[END_REF][START_REF] Zoumboulakis | Active rules for sensor databases[END_REF].

Intuitively, this mapping is as follows. Events can either be (𝑖) execution events exposed by the instrumentation interface, i.e., starting or returning calls to rules of the execution semantics on specific model elements, or (𝑖𝑖) user-defined events emitted by other moniloggers. Conditions are boolean expressions referring to the execution state of either the running model (through the instrumentation interface) or the instrumentation context, and are written in the expression language embedded with MoniLog, or in any language supported by the platform on which the model is being executed. Actions mostly consist of calls to standard or user-defined appenders sending messages to their configured destination according to a given layout, or emissions of user-defined events that can carry data derived from the execution state and can trigger additional moniloggers. Similarly to conditions, actions can contain expressions referring to the execution state -through the instrumentation interface -and to the instrumentation context.

At runtime, the MoniLog framework then allows to use moniloggers with models conforming to any DSL satisfying the prerequisites of the approach. Moniloggers are managed by the MoniLog interpreter, which runs on the same execution platform as the DSL interpreter. As moniloggers follow the ECA paradigm, the MoniLog interpreter is a kind of ECA engine that triggers its ECA rules (i.e., its registered moniloggers) in reaction to events, and that is able to query the execution state of the running model to resolve the condition and action parts of the triggered moniloggers.

The events received by the MoniLog interpreter originate from two sources: the dynamic pointcut implementation (shown in orange on the middle lower part of Figure 3), and actions of triggered moniloggers. The dynamic pointcut implementation is generated from the instrumentation interface of the DSL and the specifications of the moniloggers to be applied to the execution, as shown on Figure 3. This allows to insert calls to the MoniLog interpreter before and after the execution by the DSL interpreter of the rules exposed as execution events by the DSL instrumentation interface. In practice, the actual instrumentation framework for which dynamic instrumentation specifications are generated depends on the execution platform. For example, in the case of GraalVM as an execution platform, the Truffle instrumentation framework can be used for any language defined using the Truffle language implementation framework, while in the case of non-Truffle-based languages (or other JVMs), Aspect-Oriented Programming (AOP) can be used to perform the instrumentation, e.g., using AspectJ.

MoniLog Language

In this section, we present the MoniLog language in more detail. In particular, we detail the metamodel constituting the abstract syntax of MoniLog, and exemplify it on an example MoniLog specification.

Figure 4 provides this aforementioned example, while Figure 5 highlights different parts of the metamodel of the MoniLog language. Figure 5a shows an overview of the language. At the root of conforming models is a MoniLogSpecification element, which contains Monilogger, Event, Layout, and Appender elements, which can all be reused across moniloggers and MoniLog specification files. All of these are ParameterizedElements: they can declare a list of parameters for which a value must be supplied when they are called, and to which expressions may refer. A Monilogger is split in three parts: the events, the conditions, and the actions parts. Figure 5b, 5c, and 5d cover these parts in more details.

The event part of a Monilogger points to an Event contained in a MoniLogSpecification. Event elements can all specify a set of parameters, as denoted by their extending the ParameterizedElement class. As shown in Figure 5b, Event elements can either be ExecutionEvent or UserEvent elements. An ExecutionEvent refers to an ExposedExecutionEvent defined in the instrumentation interface of the DSL to which the model to instrument conforms. An ExposedExecutionEvent specifies an ExecutionRule from the operational semantics of the DSL that, when called with a (possibly empty) set of specific model elements (denoted as Objects conforming to Concepts of the DSL), generates two ExecutionEvents: one before and one after the call to the execution rule. For example, the ComputeUnReturned event defined in Figure 4 refers to an instance of ExposedExecutionEvent named Com-puteUn, which itself refers to the ComputeUn Job shown in Figure 2, and points to the Job.execute rule defined in the execution semantics of NabLab shown in Figure 1. Finally, an UserEvent is an event whose occurrences are directly emitted by a monilogger, as part of an EmitEvent action (see below). UserEvents are typically used when multiple ExecutionEvents can trigger a given monilogger.

The condition part of a Monilogger allows users to specify under which conditions a monilogger should perform its associated Actions when triggered by a given Event occurrence. The condition part consists in an Expression whose result is interpreted as a boolean. As shown on Figure 5c, an Expression can be a BinaryExpression, a UnaryExpression, an AtomicExpression, or a VariableAssignment. The first two allow to define arithmetic and boolean expressions primarily based on AtomicExpressions, while VariableAssigments set the target Variable to the result of the value Expression.

Aside from usual elements such as primitive values (not detailed here for the sake of brevity), AtomicExpression elements can be VariableReferences, ContextReferences, Lan-guageCalls, or LayoutCalls. VariableReference elements point to a declared Variable, which can be a global variable, a parameter of the current monilogger, a parameter of the triggering event of the monilogger, or one of its local variables. Con-textReference elements point to ExposedObject elements provided by the instrumentation interface of the DSL. Ex-posedObjects in turn point to an element of the instrumented model (an Object) conforming to a Concept of the DSL). Thus, where VariableReferences allow to refer to variables specific to the state of the MoniLog interpreter, ContextReferences allow to refer to the state of the running model. In Figure 4, the condition part of the LogTemperature monilogger (line 17) contains two VariableReferences (lastTimeLogged and outputInterval, which are global variables declared at line 9), and a ContextReference (context(t_n)), which refers to the t_n variable (i.e., the simulation time at the previous iteration over n) in the state of the instrumented model. This condition is used to log values at a period differing from that of the simulation.

LanguageCall elements perform a call to a callable element of another model from a potentially different language, with a set of VariableValue elements provided by the args reference to supply values for the parameters of callable element. In Figure 4, such a call to another language is performed on line 23. This call is performed on the stdev function defined in the JavaScript file imported under the utils alias in the specification, by providing the current value of the u_n variable in the model state (i.e., the array of temperature values over the simulated domain at the current iteration of the Iterator over n). Finally, we describe Layout-Call elements below, as they are more relevantly used in the action part of a monilogger than in its condition part.

The action part of a Monilogger allows to specify the actions to be undertaken by the monilogger upon occurrences of its observed event, if its associated conditions is validated. As shown on Figure 5d, the MoniLog language allows to (𝑖) evaluate an Expression through an EvaluateExpression element, (𝑖𝑖) deliver data at a given destination through Ap-penderCall elements, (𝑖𝑖𝑖) emit occurrences of a UserEvent, which may trigger other moniloggers, or (𝑖𝑣) start or stop Moniloggers. In particular, the last three may require to supply arguments under the form of VariableValues.

MoniLog comes with a standard library of Appenders (console and file appenders), but can be extended to provide additional ones suited to the needs of modelers, such as appending data to a database. AppenderCall elements expect a message to append, but can also take additional parameters such as a file path in the case of the file appender. The specific message to append is typically provided through a LayoutCall pointing to a Layout element to obtain a message formatted in a specific way (e.g., according to specific XML or JSON schemes) from a set of VariableValue elements. Similarly to Appenders, the MoniLog language comes with a standard library of Layout (the string formatting layout), but the language can be extended to provide additional Layout types, better suited to the domain of a DSL (e.g., formatting data for inspection in visualization software such as ParaView [START_REF] Ahrens | Paraview: An end-user tool for large data visualization[END_REF]). In Figure 4, the LogTemperature monilogger uses the FileAppender and StringLayout to perform its main action. This action consists in computing the standard deviation of the u_n variable from the model state, and outputting a line containing the current simulation time and the computed standard deviation to a file at a path provided by the filePath variable. The second action performed by the monilogger is to increment the lastTimeLogged variable through an EvaluateExpression.

The last type of action provided by MoniLog is starting and stopping moniloggers, available as StartMonilogger and StopMonilogger elements. Both of these kinds of action point to a particular Monilogger on which to perform the action. StartMonilogger actions can also provide a set of Parameter-Value elements to pass arguments to the monilogger, which will then be available for reference in the expressions it con-tains. In Figure 4, the CoarsenInterval monilogger, which is in charge of increasing the interval of simulation time between each log once the simulation time has reached 0.01, uses the StopMonilogger action on itself to stop monitoring the t_n variable after it has been triggered once.

Formal Semantics

In this section we introduce the execution semantics for MoniLog, formalizing the possible composition of loggers and runtime monitors, as well as the coordinated execution of several moniloggers, in concert with the running model.

General Definitions

In the remainder of this section, we abstract the running model as a labeled transition system (LTS) [START_REF] Arnold | Finite transition systems: semantics of communicating systems[END_REF]. We denote the silent event with 𝜏, and any ExecutionEvent with 𝜆, which we refer to as execution events in the remainder of this Section. We also denote Λ the set of all execution events. We denote with 𝑠 𝜆 -→ 𝑠 ′ the fact that there is an execution rule in the operational semantics of the DSL allowing to transition from state 𝑠 to state 𝑠 ′ by producing an execution event 𝜆, i.e., there is a transition labelled 𝜆 between 𝑠 and 𝑠 ′ in the LTS of the running model. Similarly from 𝑠 to 𝑠 ′ , resulting from calls to execution rules that are not exposed by the instrumentation interface, and thus do not produce an executable event.

𝑚 1 ◁ 𝑠 𝜏 - → 𝑚 1 ◁ 𝑠 ′ 𝑚 2 ◁ 𝑠 𝜏 - → 𝑚 2 ◁ 𝑠 ′ SilSC 𝑚 1 • 𝑚 2 ◁ 𝑠 𝜏 - → 𝑚 1 • 𝑚 2 ◁ 𝑠 ′ 𝑚 2 = ⟨𝑒, as, 𝜎⟩ 𝑒 ≠ exec 𝑚 1 ◁ 𝑠 𝜏 - → 𝑚 ′ 1 ◁ 𝑠 SilMC 𝑚 1 • 𝑚 2 ◁ 𝑠 𝜏 - → 𝑚 ′ 1 • 𝑚 2 ◁ 𝑠 𝑚 1 ◁ 𝑠 𝜆 - → 𝑚 ′ 1 ◁ 𝑠 ′ 𝑚 2 ◁ 𝑠 𝜆 - → 𝑚 ′ 2 ◁ 𝑠 ′ MonC 𝑚 1 • 𝑚 2 ◁ 𝑠 𝜆 - → 𝑚 ′ 1 • 𝑚 ′ 2 ◁ 𝑠 ′ 𝑚 1 ◁ 𝑠 𝜋 -→ 𝑚 ′ 1 ◁ 𝑠 LogC 𝑚 1 • 𝑚 2 ◁ 𝑠 𝜋 -→ 𝑚 ′ 1 • 𝑚 2 ◁ 𝑠 ⟨𝑎, 𝜎 1 , 𝑠⟩ ⇓ {𝜎 ′ 1 , 𝜇} event(𝑚 2 ) = 𝜇 EmitC ⟨exec, emit(𝑎) :: as, 𝜎 1 ⟩ 𝑚1 • ⟨mon, 𝜀, 𝜎 2 ⟩ 𝑚2 ◁ 𝑠 𝜏 - → ⟨exec, as, 𝜎 ′ 1 ⟩ 𝑚1 • ⟨trig, 𝜀, 𝜎 2 ⟩ 𝑚2 ◁ 𝑠 ⟨𝑎,
Concerning moniloggers, we denote any log event resulting from an AppenderCall with 𝜋, and any event emitted by the EmitEvent action with 𝜇. The status of a monilogger can be mon, i.e., the monilogger is waiting for a triggering event, trig, i.e., the monilogger has been triggered and will evaluate its condition, or exec, i.e., the monilogger is currently evaluating its condition or executing its actions. Given a monilogger 𝑚, event(𝑚) returns the event monitored by 𝑚, condition(𝑚) returns the condition of 𝑚, and actions(𝑚) returns the ::-constructed list of actions of 𝑚. We denote the condition to be evaluated with 𝑐. We denote the action to be executedi.e., the head of the list -with 𝑎, and the actions that remain to be executed with as. 𝜀 denotes the empty list. We denote the context of a monilogger with 𝜎. Finally, we denote the state of a monilogger as a tuple 𝑚 = ⟨𝑒, 𝑎𝑠, 𝜎⟩ (also denoted as ⟨𝑒, 𝑎𝑠, 𝜎⟩ 𝑚 ), with 𝑒 ∈ {mon, trig, exec}.

The evaluation of expressions and calls to external models constituting conditions and actions of moniloggers are abstracted as follows. For a given executable model element elt, monilogger context 𝜎, and execution state 𝑠, we denote ⟨elt, 𝜎, 𝑠⟩ ⇓ {𝜎 ′ , res} the fact that elt evaluates to res with resulting monilogger context 𝜎 ′ . In the case of conditions, res ∈ {tt, ff}, denoting a satisfied or unsatisfied condition. Concerning actions, start, stop, and emit denote Start-Monilogger, StopMonilogger, and EmitEvent elements, while 𝑎 denotes AppenderCall or EvaluateExpression elements.

Finally, we introduce the ◁ instrumentation relation between a system and a monilogger. This relation is based on the instrumentation relation described by Francalanza et al.

in [START_REF] Francalanza | A foundation for runtime monitoring[END_REF]. We denote with 𝑚 ◁ 𝑠 the fact that a model in current state 𝑠 is instrumented with a monilogger in current state 𝑚. Figure 6 and Figure 7 present in details the structural operational semantics of the instrumentation of a running model with one (for Figure 6) or several moniloggers (for Figure 7). We present both case in the remainder of this section, starting with instrumentation with a single monilogger.

Single Monilogger Composition

According to Sil, the running model can only silently transition to a new state if the monilogger is in the mon state. The Mon rule states that the running model can only transition non-silently alone to a new state if the label of the transition does not match the triggering event of the monilogger. Conversely, the Trig rule states that when the label of the transition does match the triggering event of the monilogger, the running model and the monilogger transition together to a new state, where the status of the monilogger becomes trig. These three rules enforce the synchronous nature of the instrumentation: execution of the running model is suspended while the monilogger reacts to a triggering event.

Once in a trig state, a monilogger can evaluate its condition. If the condition evaluates to ff (CondF), the monilogger silently transitions to a new state where it resumes monitoring events. Not that any side effect resulting from the evaluation of the condition is preserved. If the condition evaluates to tt (CondT), the monilogger silently transitions to an exec state containing the list of actions to execute as.

Once in an exec state, the Log rules covers the execution of actions other than stop, start, and emit. This rule consists in evaluating the action, which can yield a log event 𝜋 (for calls to appenders), and an updated instrumentation context 𝜎 ′ (for expressions to evaluate). The Restart rule allows a monilogger to return to a mon state when it has no action left to execute. The Stop rule covers the stop action, and simply terminates the instrumentation of the running model, letting the execution of the running model run to completion. As the start and emit actions have no use in the context of a single monilogger, they have no associated rule in Figure 6.

Multiple Monilogger Composition

For the sake of brevity, we only show the instrumentation of a system with two moniloggers, but it can be extended to 3 or more moniloggers. We hereby introduce the • symmetrical and associative relation between monilogger states, which is defined through the rules in Figure 7.

The first rule shown in Figure 7, SilC, states that the The SilMC rule allows a monilogger to silently transition to a new state if no other monilogger is in an exec state. This means that once a monilogger is in an exec state, it is the only one on which rules can be applied. In other terms, once the condition of a monilogger has been validated, its actions must be completely executed before another potentially triggered monilogger starts executing. The running model can only resume executing once all moniloggers have either transitioned back to a mon state, or have been stopped.

𝑚 1 • 𝑚 2 ◁ 𝑠
The remaining rules handle each kind of action in the context of multiple moniloggers. The LogC rule allows a monilogger to execute their AppenderCall or EvaluateExpression actions. The EmitC rule defines the semantics of the EmitEvent action, which emits a user-defined event 𝜇, thereby allowing other moniloggers to transition from mon states to trig states. The StartC rule states that the Start-Monilogger action initializes a new monilogger with context 𝜎 2 resulting from the evaluation of arguments 𝑎 provided to the action. This new monilogger is composed with the currently executing one. Finally, according to the Stop rule, when a StopMonilogger action targets another monilogger, it is simply removed from system.

Implementation

The abstract syntax of MoniLog is implemented as an Ecore metamodel, and its concrete syntax is provided as an Xtext grammar. The MoniLog interpreter is provided under two implementations: a Truffle-based implementation leveraging the Truffle instrumentation framework [START_REF] Van De | Fast, flexible, polyglot instrumentation support for debuggers and other tools[END_REF], and a pure Java implementation relying on AspectJ [START_REF] Kiczales | An overview of AspectJ[END_REF]. The former can run on any JVM, while the latter runs on GraalVM. Both implementations are available on GitHub 3 .

To be used with models conforming to a given DSL, both implementations rely on an instrumentation interface provided by language engineers. The instrumentation interface of an executable DSL consists of (𝑖) a model transformation creating the design-time, model-specific instrumentation interface which exposes model elements that can be read or instrumented under the form of ExposedObjects, and to which users can refer when defining moniloggers, (𝑖𝑖) a context wrapper used by the MoniLog interpreter and providing the value of each model element exposed as an ExposedObject to evaluate ContextReferences, and (𝑖𝑖𝑖) a pointcut for execution rules exposed as ExposedExecutionEvents, to enable the emission of ExecutionEvents at runtime.

The model transformation providing the model-specific instrumentation interface depends on the formalism used to define the abstract syntax of the DSL. In the case of NabLab, which uses Ecore, the model transformation was defined in Xtend, and consists in populating the model-specific instrumentation interface with ExposedObjects mapped to every Job, Function, and Variable elements found in the model.

In the case of Truffle, the context wrapper must follow the Truffle interoperability protocol, allowing any Truffle-based tool and language implementation to read from and write to the execution context of the language. This mostly means that the different types manipulated by the DSL should implement the Truffle interoperability protocol, which is also a prerequisite to benefit from other Truffle-based tooling and from interoperability between Truffle-based languages. In the case of AspectJ, the context wrapper must be able to retrieve the primitive value of an ExposedObject, if it represents such a value (e.g., a real matrix in NabLab is converted to a 2-dimensional double array). If a type cannot be mapped to primitive values (i.e., if it is an object value), the context wrapper must instead provide a map providing the value of each property that can be read on the object value.

Concerning the execution rule pointcuts, when using Truffle, these must be provided as part of the definition of the language. This is because the Truffle Language Implementation Framework offers constructs for defining a kind of instrumentation interface addressing runtime instrumentation concerns, i.e., identifying AST nodes to instrument. Thus, the AST nodes that the language engineer wishes to make instrumentable must be defined as such, which boils down to implementing a specific (Java) interface for them. While this is intrusive with regard to the operational semantics of the DSL, this allows to identify which AST nodes to instrument among nodes corresponding to the same execution rule. For example, in NabLab, this allows to instrument only the AST nodes corresponding to a given job (e.g., ComputeUn) among all AST nodes corresponding to the Job.execute rule.

In AspectJ, the runtime elements of the instrumentation interface can be provided a-posteriori, in a non-intrusive way with regard to the operational semantics, through a collection of aspects instrumenting the Java code of the language interpreter. In the case of AspectJ, pointcuts target the Java methods implementing the execution rules to instrument. As such, AspectJ pointcuts are called every time an execution rule is called, even if the model element on which it is called should not be instrumented. For example, in NabLab, the pointcut associated to assignments is called on each call to the VarAssign.execute rule, even if the actual Variable being assigned a value is not the target of any instrumentation.

Evaluation

In this section, we leverage the previously described implementations to answer the following research questions:

RQ#1 How does the proposed approach allow to combine runtime monitoring and logging to extract relevant data from running models? RQ#2 How is the overhead induced by the approach affected by different scenarios?

Leveraging Complementarities

To evaluate whether MoniLog allows to leverage the complementarities between runtime monitoring and logging, we present a demonstration case on a NabLab model solving the heat equation. In this demonstration case, we show how to use MoniLog to 𝑎) compute the standard deviation of the temperature over the simulated domain, 𝑏) monitor that it decreases at each simulation step, and 𝑐) start a logger appending temperature values to a file in case the standard deviation fails to decrease on a simulation step. The combination of 𝑎) and 𝑏) highlights how logging can be leveraged by runtime monitoring, and the combination of 𝑏) and 𝑐) highlights the reverse. Figure 8 shows the corresponding MoniLog specification. We cover its key points below.

As the expressivity of MoniLog does not allow to easily compute the standard deviation of an array, the specification first imports a JavaScript file named Utils.js (line 6), which contains an stdev function, as well as a date function. Next, a setup section (lines 8-11) declares variables that can be used in the instrumentation context: previousStdev holds the value of the previously computed standard deviation and is initialized with an upper bound value of 1.0, currentStdev holds the current standard deviation, and filePath is used to indicate in which file should be logged, using the aforementioned date function from the imported JavaScript file. Then, the ComputeUnReturned event is declared (line 13): it is emitted after each call to the ComputeUn callable element (a Job in this case). The specification then contains two moniloggers: one computing and monitoring the standard deviation of the u_n array (i.e., the temperature computed on the current iteration), and another which starts as inactive and logs the content of u_n to a file once it is activated.

The first monilogger, MonitorStdev, is triggered by occurrences of the ComputeUnReturned event (line 15). When triggered, it computes the standard deviation of the u_n array using the stdev functions from the imported JavaScript file, assigns it to the currentStdev variable, and compares it to the previousStdev variable (lines [START_REF] Leucker | A brief account of runtime verification[END_REF][START_REF] Li | A runtime monitoring and validation framework for web service interactions[END_REF]. Then, if the comparison violates the expected behavior of the model, a message is printed to the console using calls to StringLayout and ConsoleAppender (lines [START_REF] Morin | Model-based, platformindependent logging for heterogeneous targets[END_REF][START_REF]OMG Unified Modeling Language (OMG UML)[END_REF][START_REF]Meta Object Facility (MOF) Core Specification[END_REF], and a line is appended to the output CSV file using calls to CSVLayout and FileAppender (line [START_REF] Pecchia | Industry practices and event logging: Assessment of a critical software development process[END_REF][START_REF] Perumal | ECA-based interoperability framework for intelligent building[END_REF]. Finally, the MonitorStdev monilogger is stopped, and the LogU monilogger is activated (lines [START_REF] Rabiser | Developing and evolving a DSL-based approach for runtime monitoring of systems of systems[END_REF][START_REF] Schiavio | Nodemop: Runtime verification for node. js applications[END_REF]. If however the model behaves as expected, previousStdev is set to currentStdev instead (line 27). The second monilogger, LogU, is initially inactive, as indicated by the @Inactive annotation (line 30). Once started, it is triggered by occurrences of ComputeUnReturned, and appends the content of u_n to the output CSV file (lines 33-34). MoniLog allow to combine freely logging and runtime monitoring concepts, thereby allowing to leverage the complementarities between the two. This specification also shows how language calls allow to provide the required expressivity for some computations. The MoniLog specification shown in Figure 4 also leverage these complementarities, as it alters its logging behavior (i.e., the frequency of log events) according to the state of a property defined over the execution state of the running model.

Induced Execution Overhead

To answer RQ#2, we consider 3 MoniLog specifications, running on both implementations (Truffle and AspectJ) and instrumenting the same NabLab model, each designed to handle a different use-case, as detailed thereafter.

■ monitor-residual: Verifying a temporal property on a variable. This specification uses MoniLog variables to evaluate properties over several execution states. ■ visualize-stdev: Feeding data derived from the execution state to an external visualization. This specification derives data through language calls to JavaScript, which offers the required expressivity, and sends this data to an asynchronous visualizer. ■ dump-variable: Writing data of interest (i.e., the content of the u_n variable) to a file for offline analysis. This specification outputs large amounts of data (around 25 MB for every 0.2 of simulation time).

Experimental Setup and Protocol. Measurements are performed on an Intel® Core™ i7-9850H CPU @ 2.60GHz × 12, running on Ubuntu 20.04.2, and using GraalVM 21.1.0 as the execution platform. The execution overhead of each specification is measured on the Truffle and AspectJ implementations as follows. First, a warm-up phase consisting of 5 executions of the instrumented model takes place. We then collect the execution time of 50 additional executions, and compute the average execution time using geometric mean. We do the same for the average execution time of the model without instrumentation, which we use as a baseline to compute the absolute and relative overhead of each specification. These measurements are done for 5 different simulation time values, ranging from 0.2 to 1.0, allowing to evaluate the induced overhead on a range of execution times. Summaries of the measurements are shown in Figure 9.

Answering RQ#2. As shown on Figure 9a, for the Truffle implementation of the NabLab interpreter, the baseline execution time of the model ranges from 7.49s to 24.90s, for simulation times ranging from 0.2 to 1.0. The execution time of the instrumented model respectively ranges from 9.52-10.91s to 28.37-31.16s, for an absolute overhead ranging from 2.02-3.42s to 3.47-6.26s, and for a relative overhead ranging from ×1.27-1.46 to ×1.14-1.25. Concerning the AspectJ implementation, Figure 9b shows that the baseline execution time ranges from 33.84s to 116.62s, while the execution time of the instrumented model ranges from 41.03-44.18s to 132.66-137.12s. The absolute overhead thus ranges from 7.19-10.34s to 16.05-20.51s, and the relative overhead ranges from ×1.21-1.31 to ×1.14-1.18.

Figures 9c and9d show for each scenario the overhead per call for the Truffle implementation (resp. the AspectJ implementation), which we define as the absolute overhead divided by the number of times a monilogger is triggered. For the dump-variable and visualize-stdev scenarios, overhead per call is significantly higher than for the monitorresidual scenario, which stays relatively stable at less than 0.5ms (resp. around 2.5ms). This is because both scenarios derive data from the u_n array, which induces significant additional overhead with regard to monitor-residual. The fact that monitor-residual is within the same order of magnitude than the other scenarios (as shown on Figure 9a) stems from the fact that moniloggers are triggered on average 3.34 times more in this scenario. In addition, a sharp decrease in overhead per call can also be observed as simulation time increases for the dump-variable and visualize-stdev scenarios, as it starts above 4ms (resp. above 12ms) and ends around 1-1.5ms (resp. above 4ms). We explain this by the fact that, while all scenarios are executed on GraalVM, those two scenarios make use of language calls to JavaScript, which are only optimized if the execution lasts long enough.

Overall, we consider the overhead induced by the approach to be suitable to debugging activities as the absolute overhead stays reasonably low on shorter execution times, and the relative overhead is reduced by around 50% as the execution time increases.

Threats to Validity

An external threat to validity is that the evaluation relies on a single DSL, NabLab. However, this DSL is (𝑖) representative of the kind of languages for which interactive debugging does not work well, (𝑖𝑖) computationally intensive, and therefore a good candidate to evaluate the induced overhead of the approach, (𝑖𝑖𝑖) representative of the languages that benefit from being supplemented with MoniLog, as it does not provide the expressivity required for logging.

Another external threat to validity is that the evaluation only deals with simple temporal properties. However, while MoniLog does not directly support temporal properties requiring the use of automata (e.g., complex temporal properties written in linear temporal logic), their handling can be delegated to dedicated libraries through language calls.

Related Work

To our knowledge, no current approach provides integrated runtime monitoring and logging facilities, even less so for executable DSLs. We thus hereby review the closest works to ours focusing on either runtime monitoring, or logging in the context of executable DSLs.

On the runtime monitoring front, a number of works provide runtime monitoring for specific DSLs or domains (e.g., BPMN [START_REF] Barnawi | BP-MaaS: A Runtime Compliance-Monitoring System for Business Processes[END_REF], web services [START_REF] Li | A runtime monitoring and validation framework for web service interactions[END_REF], however we limit here our analysis to language-and domain-agnostic approaches. In our previous work [START_REF] Leroy | Runtime Monitoring for Executable DSLs[END_REF], we provide a property language for runtime monitoring allowing domain experts to express temporal properties from domain concepts and intuitive temporal patterns. Conversely, in the present work, we aim to (𝑖) complement the expressivity of existing DSLs with that of other languages in the context of runtime monitoring and logging, and (𝑖𝑖) show that logging and runtime monitoring benefit from each other on the other hand.

In [START_REF] Ancona | RML: Theory and practice of a domain specific language for runtime verification[END_REF], Ancona et al. detail the Runtime Monitoring Language (RML), a DSL dedicated to the definition of runtime monitors in a system-and language-agnostic way. Both RML and MoniLog rely on a kind of instrumentation interface. However, the one used by RML fully relies on the data transmitted through event occurrences. Thus, to allow users to refer to variables of the system, either the complete execution state must be included in each event occurrence, or the instrumentation of the system must be tailored to each runtime monitor, incorporating only the required variables in event occurrences. In addition, RML relies on its own expressivity to manipulate data received from events, preventing data processing with dedicated languages or libraries.

In [START_REF] Mertz | Tigris: A DSL and framework for monitoring software systems at runtime[END_REF], Mertz et al. introduce TigrisDSL, a language for specifying relevance criteria for execution events to be monitored. TigrisDSL is mostly complementary with MoniLog, as it provides another way to selected the subset of monitored events. In [START_REF] Rabiser | Developing and evolving a DSL-based approach for runtime monitoring of systems of systems[END_REF], Rabiser et al. present their development process of a DSL for the runtime monitoring of systems of systems. Similarly to RML, this DSL relies on its own expressivity for constraint evaluation, whereas MoniLog can delegate this to languages that are well-suited to the task and with which the user is proficient.

On the logging front, Morin and Ferry propose in [START_REF] Morin | Model-based, platformindependent logging for heterogeneous targets[END_REF] a platform-independent logging framework for ThingML, a DSL for modeling the behavior of distributed systems allowing to generate code for multiple languages such as Java, JavaScript, and Go. This framework provides a kind of instrumentation interface for ThingML, exposing ThingML functions, properties, and events. Compared to the logging aspect of our proposed approach, this framework is designed for systematic logging instrumentation, and thus does not enable conditional logging, nor does it allow users to tailor logging messages, or to log data derived from the execution state (e.g., average, standard deviation).

Conclusion

We presented MoniLog, a new language to define runtime monitors, loggers, and combinations of the two to instrument models comforming to executable DSLs with an operational semantics. The language provides monitoring and logging capabilities to executable DSLs lacking them due to their restricted expressivity. In addition, as it allows to freely combine logging and monitoring constructs, MoniLog allows domain experts to leverage their complementarities.

Concerning future work, we plan to explore the use of MoniLog in the context of polyglot applications, to enable the definition of moniloggers across models conforming to different languages. Adapting MoniLog to executable DSLs with a translational semantics is also part of our lines for future work, in particular in the context of the C++ code generated from NabLab models.

Figure 1 .

 1 Figure 1. Excerpt of the definition of the NabLab DSL.

Figure 2 .

 2 Figure 2. Excerpt of a NabLab Module solving the heat equation.

Figure 3 .

 3 Figure 3. Overview of the Monilog language and associated framework.

1 package heatequation 2 3 5 6Figure 4 .

 1254 Figure 4. Example MoniLog specification file.

Figure 5 .

 5 Figure 5. Metamodel of the MoniLog language.

Figure 7 .

 7 Figure 7. Inference rules of the structural operational semantics of MoniLog (composed moniloggers)

Figure 8 .

 8 Figure 8. Monilogging temperature dispersion.

Answering RQ# 1 .

 1 Overall, the specification shown in Figure 8 demonstrates how the language constructs provided by Total execution time (Truffle).

  Total execution time (AspectJ).

  Overhead Per Call (ms) (c) Execution overhead per call (Truffle).

  Call (ms) (d) Execution overhead per call (AspectJ).

Figure 9 .

 9 Figure 9. Quantitative evaluation results.

  system resulting from the composition can only silently transition to a new model state if each individual monilogger-model system can also silently transition to that same new model state, i.e., Sil can be individually applied to each. The second rule, MonC, states that 𝑚 1 • 𝑚 2 ◁ 𝑠 can only transition to a new model state 𝑠 ′ with observable event 𝜆 if both 𝑚 1 and 𝑚 2 can also transition to a new state with 𝜆, either through the Mon or through the Trig rule.

Quantitative Evaluation Results