Frédéric Loiret

Aleš Plšek

Philippe Merle

Lionel Seinturier

Michal Malohlava

Constructing Domain-Specific Component Frameworks through Architecture Refinement

Recently, a plethora of domain-specific component frameworks (DSCF) emerges. Although the current trend emphasizes generative programming methods as cornerstones of software development, they are commonly applied in a costly, ad-hoc fashion. However, we believe that DSCFs share the same subset of concepts and patterns. In this paper we propose two contributions to DSCF development. First, we propose DomainComponents -a high-level abstraction to capture semantics of domain concepts provided by containers, and we identify patterns facilitating their implementation. Second, we develop a generic framework that automatically generates implementation of DomainComponents semantics, thus addressing domain-specific services with one unified approach. To evaluate benefits of our approach we have conducted several case studies that span different domain-specific challenges.

Introduction

Component-based Software Engineering (CBSE) [START_REF] Szyperski | Component Software: Beyond Object-Oriented Programming[END_REF] has emerged as a technology for the rapid assembly of flexible software systems, where the main benefits are reuse and separation of concerns. The success of this technology has been proved by variety of its applications, from the general component frameworks [START_REF] Bruneton | The Fractal Component Model and its Support in Java[END_REF], [START_REF] Bures | SOFA 2.0: Balancing Advanced Features in a Hierarchical Component Model[END_REF], [START_REF] Coulson | A Generic Component Model for Building Systems Software[END_REF] to domain specific component frameworks (DSCF) addressing a wide scale of challenges -embedded [START_REF] Van Ommering | The Koala Component Model for Consumer Electronics Software[END_REF] or real-time constraints [START_REF] Plšek | A Component Framework for Java-based Real-time Embedded Systems[END_REF], [START_REF] Hansson | SaveCCM -A Component Model for Safety-Critical Real-Time Systems[END_REF], dynamic adaptability [START_REF] Gjørven | Experiences from Developing a Component Technology Agnostic Adaptation Framework[END_REF], distribution support [START_REF] Sentilles | A Component Model for Control-Intensive Distributed Embedded Systems[END_REF], and many others.

DSCF offers a domain-specific component model and a tool-support that allow developers to address domainspecific challenges by using appropriate abstractions available already at the component-model level. To achieve separation of concerns, domain-specific services (such as dedicated memory ares, tasks parameters, security, ...), in the literature [START_REF] Duclos | Describing and using non functional aspects in component based applications[END_REF] referred to also as non-functional requirements/aspects/properties/concerns, are usually deployed in the runtime platform -composed of a set of custom made containers [START_REF] Moreno | Creating custom containers with generative techniques[END_REF].

Today, a plethora of DSCFs emerges. However, based on our experience, from a concise and specifically designed component model [START_REF] Plšek | A Real-Time Java Component Model[END_REF] to a full fledged component framework [START_REF] Plšek | A Component Framework for Java-based Real-time Embedded Systems[END_REF] is a long road. Although the current trend emphasizes generative programming methods [START_REF] Czarnecki | Generative Programming: Methods, Tools, and Applications[END_REF] as cornerstone of software development, generative methods are usually tailored to specific domains and applied in a costly, ad-hoc fashion which prevents from any reuse or amelioration of solutions to a framework construction. We however believe that DSCFs share the same concepts and patterns to their construction and application.

Therefore, this paper brings two key contributions. First, we propose Domain Components -a high-level abstraction of domain-specific services provided by the container. Second, we develop a generic framework employing techniques of generative programming [START_REF] Czarnecki | Generative Programming: Methods, Tools, and Applications[END_REF] to create custom and component-based runtime platforms leveraging development of Domain Components. Moreover, we introduce architecture optimizations independent from the target domains, which contributes to better performance of resulting applications. Finally, to evaluate our approach, we have conducted several case studies addressing domain-specific challenges and we report on the benefits acquired.

To reflect the goals, the paper is structured as follows. Section 2 stipulates the context of this work and clarifies more precisely our goals. In Sect. 3 we introduce the basic concept that we employ -Domain Components and the Generic Framework. In Sect. [START_REF] Bruneton | The Fractal Component Model and its Support in Java[END_REF] we describe HULOTTEour prototype implementation. Section 5 presents the case studies that we have conducted to evaluate the proposed approach. We discuss related work in Sect. 6. Section 7 concludes and draws future directions of our research.

Context and Goals

Domain-Specific Component Frameworks and their Application. Typically, DSCF is composed of a component model and the tool support which permit assembling, deploying and executing demanded applications [START_REF] Bures | Using a Product Line for Creating Component Systems[END_REF]. Moreover, such component model defines the relevant architectural concepts, called domain-specific concepts, according to the A recognized methodology of developing DSCF [START_REF] Crnkovic | Building Reliable Componentbased Systems[END_REF] is composed of several steps as it is illustrated on Fig. 1a. In this case, each component model (step 1) is used to develop functional concerns of the applicationfunctional components. Typically, functional components encapsulate a business logic of an application.

Afterwards, the framework tool-support is employed to create a runtime platform, in Fig. 1a step 2. The runtime platform is composed of a set of containers [START_REF] Moreno | Creating custom containers with generative techniques[END_REF] that encapsulate functional components, and its goal is to relieve the developer from dealing with domain-specific requirements and to implement the execution support. Current trend in developing the runtime platform emphasizes a generative programming approach. While this task can be seen only as an engineering challenge, the runtime platform plays a crucial role in deciding whether the component model itself will be successful in real-life applications, since its implementation has a direct impact on the performance of a given application. Here, different optimizations should be employed to mitigate notoriously known problem of CBSE system -performance overhead (caused e.g. by intercomponent communication). Finally, functional components and the runtime platform are assembled together to form the resulting application, Fig. 1a step 3.

We distinguish two types of development roles involved in this processapplication developer and framework developer. Application developer is responsible for development of functional components and specification of domain-specific requirements -in Fig. 1a step 1. The role of the framework developer is to design and implement the runtime platform generation process, and the domain-specific requirements defined by the application developer -in Fig. 1a step 2 and 3.

Our Proposal. Considering the presented process, we can notice that for each domain, a different process is used. However, the steps 2 and 3 share many similar concepts across different application domains ( from code generation tasks, application instating and deployment, to tool-chain development etc.). Moreover, they are usually implemented in an ad-hoc manner without any reuse. We therefore propose a new development process presented in Fig. 1b. As the cornerstone we use a generic component model that is easily extendable towards different application domains, in Fig. 1b step 1. Consequently, since all domain specific models share the same concept, an unified approach to runtime platform generation can be employed in steps 2 and 3. Therefore, we can summarize the key contributions of our paper:

• A Generic Component Model and Domain Components. We propose Domain Components -a unified approach to specification of domain-specific requirements presented in custom containers. This allows the application developer to easily manipulate domain specific requirements since they are represented as first-class entities and are separated from the functional concerns. Furthermore, we identify common patterns that are used by framework developers to implement semantics of Domain Components.

• A Framework To Build Component Frameworks. We develop a framework, in the literature also refereed as meta-framework [START_REF] Bures | Using a Product Line for Creating Component Systems[END_REF], composed of high-level tools, methods, and patterns allowing framework developers to generate runtime platforms in a generic way according to concerns captured by Domain Components. Within our approach, the platform is built using component assemblies and is based on our generic component model. Moreover, since we are able to reason about the whole system (functional and platform concerns) using common concepts (components, assemblies), various architecture optimizations independent from the target domains can be introduced, which contributes to better performance of resulting applications.

Constructing Domain-Specific Component Frameworks

In this section, we introduce the basic concepts of our generic framework presented in Fig. 1 b). As the cornerstone of our proposal we define a generic component model, depicted in Fig. 2. The model is divided into core-and platform-level. First, in Sect. 3.1, we present the core-level concepts and introduce Domain Components -special components for expression of domain-specific concerns in the application. Furthermore, responsibilities of application and framework developers are exactly defined. Second, in Sect. 3.2 we introduce the runtime platform construction process -the architecture refinement. In this process, framework developer refines the application architecture through the architectural patterns that we define in the platform-level of the model.

A Generic Component Model

The model is based on the popular CBSE principles [START_REF] Bruneton | The Fractal Component Model and its Support in Java[END_REF], containing the basic entities Component, Interface, Binding, Primitive and Composite component. Moreover, the component model adopts the sharing paradigm [START_REF] Bruneton | The Fractal Component Model and its Support in Java[END_REF] -one specific component can be a subcomponent of more than one composite component.

Figure 2. Component Metamodel and Domain Component

A brand new entity that we introduce is Domain Component, inspired by [START_REF] Plšek | A Component Framework for Java-based Real-time Embedded Systems[END_REF]. The main purpose of domain components is to model domain-specific requirements in a unified way. Within our model, domain components are reified as composite components. The sharing paradigm allows developers to fully exploit this concept. By deploying subcomponents into a certain domain component, the developer specifies that these subcomponents or the bindings between them support the domain-specific property represented by the domain component. Moreover, a domain component contains a set of attributes parameterizing its semantics.

The approach of modeling domain-specific aspects as components brings advantages commonly known in the component-based engineering world such as reusability, traceability of selected decisions or documentability of solution itself. Also, by preserving a notion of a component, it is possible to reuse already invented methods (e.g. model verification) and tools (e.g. graphical modeling tools) which were originaly focused on functional components. If we go further and retain domain components at runtime then it is possible to reconfigure non-functional properties represented by domain components on-the-fly.

We illustrate the DomainComponent concept in Fig. 3a. Components Writer, Readers, MailBox, Library and their bindings represent a business logic of the application. The domain component DC1 encapsulates MailBox and Library, thus defining a domain-specific service (e.g. logging of every interface method invocation) provided by these two components. At the same time, component DC2 represents a different service (e.g. runtime reconfiguration) and defines that this service will be supported by components Writer and Readers. Therefore, the domainspecific concerns are now represented as first-class entities and can be manipulated at all stages of component-software development lifecycle.

The role of the application developer is therefore to create and implement functional components and to specify domain-specific requirements by deploying functional components into domain components. While the application developer is aware of the semantics behind domain components, he does not provide their implementation and therefore can fully focus on functional concerns of the application. To give an example, a domain specific component ThreadArea can specify execution context (an executing thread and its properties) of an active functional component, however, the application developer does not need to know how these properties are enforced at runtime. For more examples see Sect. 5.

The role of the framework developer is to define and implement semantics of domain components. First, his responsibility is to define domain components according to the needs of application developers and to define the rules constraining application of domain components at the functional level. Afterwards, the framework developer designs and implements semantics of domain components using the platform-level concepts -see Fig. 2, and the architectural patterns that we introduce in Sect. 3.2.

Architecture Refinement of Domain Components

The key role of the framework developer is therefore to implement semantics of domain-specific components. When considering domain components and the functionality they express, they impact two core architectural concepts: Functional Component and Binding. We further refer to this phenomenon as architectural refinement of core-level concepts through the platform-level concepts.

A functional component typically implements the business part -a code provided by the application developer, and requires the platform part that implements the domain-specific services -the container. By the Functional Component Refinement we mean that the set of domain specific services is determined by the domain components, consequently the container architecture of a functional component is refined with according platform concepts. A domain-specific service can also pose special requirements on the intercomponent communication (e.g. logging, broadcast communication management), in these cases we speak about Binding Refinement. From our current experiment in using our approach, we claim that these two refinement points (components and bindings) allow to specialize the corelevel concepts according to arbitrary domain-specific requirements. We therefore define two architectural patterns,

Architectural Patterns.

The key purpose of architectural patterns is to allow framework developers to define semantics of domain components and thus to refine the application architecture in a systematic and programmatic way. The patterns are designed to implement any type of a domain-specific service that can potentially be reflected by a container, they therefore define architecture invariants, design and composition rules for the platform-level. In Fig. 2, the platform-level presents two architectural patterns: ChainComposite and ContainerComposite, and we clarify them in the remainder of this section. ChainComposite Pattern is defined as a composite component, the subcomponents of such a composite are special componentsinterceptors. Within the ChainComposite pattern, the interceptor components are bounded via their incoming and outgoing interfaces in an acyclic list, as depicted in Fig. 3a. Here, the IN and OUT interface signatures of the interceptors are not necessarily identical, this allows developers to identify interceptors as adaptors of the intercepted execution flow. The interceptor itself could be a composite component allowing framework developer to implement complex intercepting mechanisms. The ChainComposite component at the platform level refines a binding specified at the functional level, thus the pattern is similar to the concept of the connector [START_REF] Medvidovic | The Role of Middleware in Architecture-Based Software Development[END_REF].

ContainerComposite Pattern, initially introduced in [START_REF] Pessemier | A Component-based and Aspect-Oriented Model for Software Evolution[END_REF], is also specified as a composite component and reifies a container of a functional component. As defined in Fig. 2, it is composed of ChainComposite components and Controller components. The ContainerComposite pattern is applied on a primitive (see example in Fig. 3b) or composite functional component as follows:

• A set of Controller components implementing various domain-specific services and meta-data influencing the whole component (e.g. lifecycle management, recon-figuration management) is composed in the container. Moreover, a special control interfaces are provided to allow an access to these services from outside of the component.

• For each interface of the functional component a ChainComposite pattern is used. ChainComposite components can be interconnected by TRAP interfaces with the controllers, thus allowing centralized management of strategies for interception mechanisms.

Architectural Refinement Process.

Once we specify the functional architecture containing domain components and also architectural patterns for these domain components we employ the architecture refinement process -a process where the core-level architecture specified by the application developer is refined into an architecture where both functional architecture and runtime platform architecture are designed using the platform-level concepts. As a result of this process we obtain a runtime platform architecture where both functional and domain-specific concerns are represented. The crucial point of the architecture refinement process is therefore the propagation of domain-specific concerns into the architecture. The important feature of the architecture refinement process is its variability and extensibility to allow employing different refinement strategies as well as support for new domain-specific components, validation and optimizations. All properties stated above are reflected in the implementation of the architecture refinement process called HULOTTE, described in Sect. 4.

HULOTTE Framework

In this section we describe HULOTTE framework -an extensible tool-set that we have developed to implement the architecture refinement process. However, rather than to implement the whole process in a single transformation step that can be error-prone and hard to extend, we employ a step-wise refinement process [START_REF] Batory | Scaling Stepwise Refinement[END_REF] in order to refine the high-level concepts in our architecture gradually in several stages. This technology allows framework developers to easily modify and extend this process with new domaincomponent definitions and semantics. Consequently, we employ methods of generative programming to compose functional code implemented by the application developer with the runtime platform implementation.

To develop the framework, we have applied the technology for development of extensible tool-sets introduced in [START_REF] Leclercq | Supporting Heterogeneous Architecture Descriptions in an Extensible Toolset[END_REF]. HULOTTE is thus developed purely using CBSE paradigm allowing framework developers seamless extensions towards different refinement strategies. The HULOTTE framework, depicted in Fig. 4, consists of three main units front-end processing a description of a functional architecture stored in ADL, middle-end responsible for a stepby-step architecture refinement, and backend which serves as a target domain specific implementation generator. The motivation for decomposition of the process into three independent units is to separate responsibilities and concerns between the transformation steps. The front-end allows us to process architectures represented by different notations (e.g. Fractal-ADL, UML, ACME) and to transform them into an independent internal representation. Consequently, the middle-end, executing the architecture refinement process, is independent from the architecture description format. Finally, the back-end permits generation of different types of target implementations according to deployment requirements (e.g. C for embedded devices, Java for enterprise applications). In the remainder of this section we highlight interesting issues of each part of the HULOTTE framework.

Front-end implements the translation layer that proceeds an architecture description -in our case given in an extended Fractal-ADL (see [START_REF] Plšek | A Component Framework for Java-based Real-time Embedded Systems[END_REF] for an example), and transforms it into an internal EMF-model based representation.

The translation process gradually proceeds ADL artifacts (component, interface, domain component, binding) and for each applies a dedicated translation component responsible for extracting the information and building an appropriate representation in the internal model. The translation process can be extended by appending a new translator component. The new translator typically reflects a domain-specific extension of ADL (e.g. DistributedNode, ThreadArea presented in Sect. 5).

Middle-end is the central part of the HULOTTE framework and implements the refinement process. Its task is to process the architecture description in the form of the EMF model produced by the front-end, apply defined architecture refinements -creating, connecting, or merging model elements according to employed transformations. Internally, the middle-end is composed of three processing units -PlatformBuilder, Validator, and Optimizer.

PlatformBuilder is responsible for the model refinement and consists of a chain of component builders (for implementations of interceptors, controllers, and components) where each chain participates in the refinement process. From the builders the runtime platform components are instantiated either by loading definitions from an off-theshelf component library or programmatically, via the highlevel API provided by the framework. The selection and execution order of chains is controlled by MainBuilder Dispatcher that recursively explores the platform architecture and applies appropriate builder chains. Moreover, refining the internal structure as a chain of ComponentBuilders encourages extensibility of the whole process, since a new domain-specific builder can be easily introduced.

Validator verifies that resulting platform architectures are in conformance to the architectural constraints and invariants of domain components. The task is not only to verify whether the architectural patterns were applied correctly but also to assert that domain components were specified with respect to their constraints (e.g. to arbitrarily apply two different domain components over the same functional component is sometimes not meaningful, see the Limitations of the Approach in Sect. 5.3).

Optimizer introduces optimization heuristics in order to mitigate the common overhead of component-based applications. The heuristics focus on reducing interceptions in inter-component communication which usually causes performance overhead, and on merging architecture elements in order to decrease memory footprint. A detailed description of the heuristics provided by our framework is out of the scope of this paper, we refer the interested reader to [START_REF] Plšek | A Component Framework for Java-based Real-time Embedded Systems[END_REF]. Moreover, since a complete architecture of the system is available at this stage, additional architecture optimizations can be introduced while still being independent from the target domain.

Back-end part of the framework is also highly configurable in order to reflect current target domain and chosen implementation language. In the case of our implementation of HULOTTE, the back-end is a collection of Java code generators generating Java classes from particular model elements.

Evaluation

Case Study: A Framework for Real-time Java based Systems

The initial case study introduces a component-based framework for RTSJ-based real-time and embedded systems [START_REF] Plšek | A Component Framework for Java-based Real-time Embedded Systems[END_REF]. As the cornerstone of the framework we have defined a domain-specific component model [START_REF] Plšek | A Real-Time Java Component Model[END_REF] which fully reflects the specifics of Real-time Specification for Java (RTSJ) [START_REF] Bollela | The Real-Time Specification for Java[END_REF]. The key motivation for this case study is to employ the domain component concept and the HULOTTE framework in order to achieve a better separation of concerns of RTSJ systems and to mitigate complexities of the RTSJbased development process. Therefore, the domain component concept is used to represent RTSJ concerns. We define MemoryArea domain component to express different allocation areas of RTSJ systemsheap, scoped memory, and immortal memory. Furthermore, ThreadDomain component is defined to represent various execution concepts enforced by RTSJnon-realtime, real-time and non-heap real-time, and to distinguish between active and passive functional components. Consequently, the HULOTTE patterns were used to implement defined domain components. The ChainComposite pattern is employed to implement MemoryArea components by providing correct switching between allocation contexts and supporting cross-area communication. Similarly, the ContainerComposite patterns implements containers of components deployed in the ThreadDomain component.

Real-time Collision Detector. To apply our domain specific framework, we have implemented a large case study -Real-time Collision Detector (RCD) introduced in [START_REF] Andreae | Scoped Types and Aspects for Real-time Java Memory Management[END_REF]. The RCD algorithm is about 25K Loc and its task is to proceed a periodic stream of aircraft positions and determine if any of these aircrafts are on a collision course.

Figure 5a shows a snippet of the RCD architecture designed in our approach. The Runner component represents the starting point of the application, by deploying it in the ThreadDomain:NHRT component we precisely define its execution context. Furthermore, ScopedMemory2 encapsulates functional components responsible for computations performed in every iteration of the algorithm and thus implements deallocation of temporal data between every iteration, results of these computations are stored in a StateTable component defined as a persistent by the ScopedMemory1 domain component. Finally, the patterns introduced in Sect. 3.2 were employed to implement domain components. In Fig. 5b we demonstrate application of the ChainComposite pattern that implements cross-scope communication between MotionCreator and StateTable component; and application of the ContainerComposite pattern that was used to implement the ThreadDomain component for the Runner component.

Evaluation. When developing the RCD example we can witness several benefits of our approach. The domain specific component model [START_REF] Plšek | A Real-Time Java Component Model[END_REF], designed through the domain component concept, allowed us to construct a specific framework [START_REF] Plšek | A Component Framework for Java-based Real-time Embedded Systems[END_REF] addressing fully the challenges of RTSJ-based software development. The domain components simplified expression of RTSJ specific properties, since these properties are present in the architecture as first-class entities. A full separation of functional and real-time concerns is achieved, therefore, the functional code is more readable -reflecting the functional needs of the application without any constrains imposed by the real-time properties. As the second benefit of our approach we consider application of the HULOTTE tool-chain for automatic generation of the runtime platform implementing RTSJ-related code, which is highly error-prone when implementing by hand. Moreover, performed benchmarks published in [START_REF] Plšek | A Component Framework for Java-based Real-time Embedded Systems[END_REF] showed that our approach does not introduce any overhead comparing to purely object-oriented methods.

Other Applications and Lessons Learned

Apart from the presented case study, we have validated our approach in studies spanning various domains.

In [START_REF] Malohlava | Introducing Distribution into a RTSJ-based Component Framework[END_REF] we have introduced a new domain component -DistributionNode (DN), to address challenges of distribute applications. A functional component in DN will be thus deployed on the corresponding node together with its runtime support extended towards the specifics of distributed communication. The role of the framework developer is therefore to apply the ChainComposite pattern on each distributed binding, consequently corresponding stubs and skeletons will be refined as a subcomponents of the ChainComposite and automatically generated by the HULOTTE framework. Moreover, the framework generates each DN component as a self-standing application allowing deployment of the components into the corresponding nodes. The evaluation showed that the approach brings significant ease into a distributed system development.

In [START_REF] Plšek | Ambient-Oriented Programming in Fractal[END_REF] we have addressed the challenges of ambient and ubiquitous computing by designing dedicated domain components. The HULOTTE framework allowed us to implement specifics of ambient communications using the architectural patterns.

Furthermore, we have conducted a study on reflective and reconfigurable services, a challenging topic addressed by many component frameworks [START_REF] Bruneton | The Fractal Component Model and its Support in Java[END_REF], [START_REF] Bures | SOFA 2.0: Balancing Advanced Features in a Hierarchical Component Model[END_REF], [START_REF] Coulson | A Generic Component Model for Building Systems Software[END_REF], [START_REF] Gjørven | Experiences from Developing a Component Technology Agnostic Adaptation Framework[END_REF]. In our approach, such requirements can be specified by dedicated domain component -ReconfigurationDomain.

The achieved result corresponds to the Fractal component model [START_REF] Bruneton | The Fractal Component Model and its Support in Java[END_REF]. The non-functional properties are specified by domain components and architectural patterns are used during their development, see Fig. 6, thus reducing the complexity for application developers. Moreover, our approach outperforms Fractal by using the Domain Component concept, since we are able to exactly specify where the introspection and reconfiguration services will be supported, which allows us to pay for the runtime flexibility only where needed. When conducting these studies, we learned that the approach scales very well for various domain-specific properties orthogonal to business logic, while reducing complexity of developed applications. Moreover, the framework structure allowed us to easily extend the set of supported domains.

Limitations of the Approach

In this paper we focus on definition of domain components and their integration in the HULOTTE framework.

However, an open research issue still remains specification of policies and constraints that regulate application of domain components at the functional level. Since some domain-specific services are non-orthogonal -competing or dependent on each other, their application must be exactly delimited in a form of policies that will manage non-trivial combinations of domain-specific services. This is however out of the scope of the paper, we plan to pursue this topic in our future work.

Related Work

Applying generative methods [START_REF] Czarnecki | Generative Programming: Methods, Tools, and Applications[END_REF] to propose a general approach to component framework development is not a novel idea. Bures et. al. [START_REF] Bures | Using a Product Line for Creating Component Systems[END_REF] summarize properties and requirements of current component-based frameworks and proposes a generative method for generating runtime platforms and support tools (e.g. deployment tool, editors, monitoring tools) according to specified features reflecting demands of a target platform and a selected component model. Comparing to our approach, the authors provide the similar idea of generation runtime platform, however they merely focus on runtime environment and related tools and neglect a definition of component model requirements by claiming that the proposed approach is generative enough to be tailored to reflect properties of contemporary component models.

Similarly, Coulson et. al. [START_REF] Coulson | A Generic Component Model for Building Systems Software[END_REF] argue for the benefits and feasibility of a generic yet tailorable approach to componentbased systems-building that offers a uniform programming model that is applicable in a wide range of systems-oriented target domains and deployment environments.

Furthermore, many state-of-the-art domain-specific component frameworks propose a concept of containers with controllers refined as components, e.g. DiSCo framework [START_REF] Prochazka | DisCo Space-Oriented Middleware: Architecture of a Distributed Runtime Environment for Complex Spacecraft On-Board Applications[END_REF] addressing future space missions where key challenges are hard real-time, embedded constraints, different levels of application criticalities, and distributed computing. Cechticky et al. [START_REF] Cechticky | A Generative Approach to Framework Instantiation[END_REF] presented the generative approach to automating the instantiation process of a component-based framework for such on-board systems.

On the other hand, aspect-oriented programming (AOP) is a popular choice to non-functional services implementation. Moreno [START_REF] Moreno | Creating custom containers with generative techniques[END_REF] argued that non-functional services should be placed in the container and showed how generative programming technique, using AOP, can be used to generate custom containers by composing differen non-functional features. This corresponds with our approach, however, as we have shown in [START_REF] Pessemier | A Component-based and Aspect-Oriented Model for Software Evolution[END_REF], aspects can also be represented as domain components -AspectDomain component, thus allowing developers to leverage the aspect-techniques to the application design layer, and to represent them as components.

Conclusion and Future Work

The recent boom of domain-specific component frameworks (DSCF) brings a challenge of constructing them effectively by enforcing reuse, since ad-hoc approach to their implementation still dominates.

This paper brings the following contributions. First, we have proposed Domain Components -a unified approach that clarifies specification and manipulation of domainspecific requirements presented in custom containers. Moreover, we have identified common patterns that facilitate implementation of Domain Component semantics. Second, we have developed a generic framework that uses generative programming methods to instantiate domain-specific applications together with their runtime platform. Finally, the whole approach is highly transparent since it is based on a component model with a configurable tool-set, which allow developers to easily extend it towards various domains.

To evaluate benefits of our approach, we have conducted various case studies that span different domain challenges. The results showed that our approach supports clear separation of functional and non-functional concerns of applications. Furthermore, proposed architectural patterns together with employed generative programming methods mitigate complexities of implementation of domain-specific concerns. Additionally, as we have shown in [START_REF] Plšek | A Component Framework for Java-based Real-time Embedded Systems[END_REF], our approach introduce various optimizations that reduce the usual overhead of component-based applications.

As for our future work, an open research issue still remains consistent and symmetric approach to construction of containers needs to be specified in a form of policies that will manage non-trivial combinations of domain-specific services.

Figure 1 .

 1 Figure 1. Development Methodologies of Domainspecific Component Application

Figure 3 .

 3 Figure 3. Domain Components and Architectural Patterns

Figure 4 .

 4 Figure 4. Overview of the Internal HULOTTE Implementation Structure

Figure 5 .

 5 Figure 5. Real-Time Collision Detector

Figure 6 .

 6 Figure 6. Applying the Architectural Patterns to Achieve Reconfigurable Components


Sung-Ho Lee
email: sung-ho.lee@cea.fr

Valerio Olevano

Benoit Sklénard
email: benoit.sklenard@cea.fr

A generalizable, uncertainty-aware neural network potential for GeSbTe with Monte Carlo dropout

A Bayesian neural network potential (NNP) achieved with the Monte Carlo dropout approximation method is developed for GeSbTe alloys. The Bayesian NNP is shown to be more generalizable than its classical counterpart, yielding reasonable predictions on structures that are not directly in the training configurations, and is able to output uncertainty estimates for the predictions. Its application to a molecular dynamics (MD) simulation is also presented, and the validity of the obtained trajectory is evaluated by comparing it to Density Functional Theory (DFT).

Introduction

GeSbTe (GST) alloys are interesting materials for a variety of technological applications like phase change memories or photonic devices [START_REF] Elliott | Chalcogenide Phase-Change Materials: Past and Future[END_REF]. Molecular dynamics (MD) simulations are often used to survey their properties and to optimise their composition, but studying finite temperature properties such as phase transition or thermal conductivity often require long simulations of extended systems that are too expensive for ab initio methods. Neural network potentials (NNP), trained on Density Functional Theory (DFT) references, can be used to drive an MD to overcome this limitation, which were recently shown to enable simulations at scales that were previously impossible at a near-DFT level accuracy [START_REF] Behler | Four Generations of High-Dimensional Neural Network Potentials[END_REF].

NNPs, however, can silently fail on structures that lie outside the learned configuration space, which poses some issues when relying upon them for atomistic simulations. One solution is to apply the Bayesian paradigm to the neural network to capture the model's uncertainty along with its predictions. This has previously been demonstrated for the different phases of carbon [START_REF] Wen | Uncertainty quantification in molecular simulations with dropout neural network potentials[END_REF], but never to more complex materials containing more than one chemical species.

In this work, a Bayesian NNP using the Monte Carlo (MC) dropout technique will be used on GST materials to evaluate its ability to estimate the predictive uncertainty as well as its superior generalizability compared to classical NNPs.

Methodology

Classical and Bayesian HDNNP

A feedforward neural network (NN) is the most common form of a neural network. It is organised as a set of layers with each layer, in turn, containing a set of nodes. Each j th node of an l th layer y [l] j in a feedforward NN is connected to all the nodes of the previous layer via the following relationship

y [l] j = σ [l]        i w [l] i j • y [l-1] i + b [l] j        (1)
where σ is known as an activation function that introduces nonlinearity to the network, and w and b are weight and bias parameters that are optimised during training. The index i runs over all nodes of the l -1 th layer. This is also represented schematically in Fig. 1.

The High Dimensional NNP (HDNNP) [START_REF] Behler | Four Generations of High-Dimensional Neural Network Potentials[END_REF] is a type of NNP that is composed of a set of Atomic Neural Networks (ANN) that each takes one atom of a given system as the input and outputs a so-called "atomic energy", whose sum provides the total energy of the system. The ANNs are simple feedforward NNs that share their weights and biases amongst the same chemical elements.

Classically, feedforward neural networks contain point weights and output point estimates. In contrast, Bayesian neural networks place a probability prior over the weights and the estimates are obtained as a distribution by sampling from the posterior [START_REF] Jospin | Hands-On Bayesian Neural Networks-A Tutorial for Deep Learning Users[END_REF].

Formally, this can be expressed as follows. Starting with the Bayes' theorem

P(H|D) = P(D|H)P(H) P(D) = P(D, H) H P(D, H ′ )dH ′ (2)
substituting θ, representing the weights and biases of the neural network, for the hypothesis (H), and the input, target pair of the training dataset (D x , D y ) for the data (D) and using the relationship P(A|B)P(B) = P(A, B) the following equation is obtained Lastly, with this Bayesian posterior of the parameters that take into account the training dataset, the predictive posterior can be used for Bayesian inference.

P(θ|D) = P(D y |D x , θ)P(θ) θ P(D y |D x , θ ′ )P(θ ′ )dθ ′ (3)
P(y|x, D) = θ P(y|x, θ ′ )P(θ ′ |D)dθ ′ (4)
In practice, especially for complex functions like neural networks, computing and sampling from the posterior is intractable, mainly due to the difficulty associated with computing the evidence θ P(D y |D x , θ ′ )P(θ ′ )dθ ′ [START_REF] Graves | Practical Variational Inference for Neural Networks[END_REF]. This is therefore performed indirectly by sampling θ i ∼ P(θ|D) for i = 0 to N [START_REF] Jospin | Hands-On Bayesian Neural Networks-A Tutorial for Deep Learning Users[END_REF]. Monte Carlo dropout is one such approach to approximately sample from the posterior by using dropout at inference time [START_REF] Gal | Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning[END_REF].

Dropout was originally developed as a regularisation technique in which a set fraction of nodes of a neural network are randomly "dropped" (ie. set to zero) during training to prevent overfitting [START_REF] Srivastava | Dropout: A Simple Way to Prevent Neural Networks from Overfitting[END_REF]. Similar to Eq. 1, dropout is expressed mathematically as follows

r [l] i ∼ Bernoulli(p) y i [l-1] = r [l] i • y [l-1] i y [l] j = σ        i w [l] i j • y i [l-1] + b [l] j        (5)
In standard dropout, the mask r is disabled at inference time and the values are rescaled according to the probability p to account for the increased number of connections. Therefore, for a given input, the output of the network is always constant. In contrast, the masking is maintained in Monte Carlo dropout and multiple forward passes through the network are performed for each input. The mask is drawn randomly from the Bernoulli distribution at each iteration which results in a distribution of outputs. The mean and standard deviation extracted from this distribution is then taken as the prediction and the corresponding uncertainty.

Computational Details

To generate the datasets, ab initio molecular dynamics (AIMD) calculations based on Density Functional Theory (DFT) were carried out using the VASP code [START_REF] Kresse | Efficient iterative schemes for ab initio totalenergy calculations using a plane-wave basis set[END_REF][START_REF] Kresse | From ultrasoft pseudopotentials to the projector augmented-wave method[END_REF] at temperatures from 300 K to 1500 K at 300 K intervals for 20 ps for hexagonal GeTe, Sb 2 Te 3 and Ge 2 Sb 2 Te 5 (containing 192, 240 and 216 atoms, respectively). 200 snapshots were then selected for each trajectory to compute accurate energies and forces. All DFT calculations were performed using the Perdew-Burke-Ernzerhof (PBE) [START_REF] Perdew | Generalized gradient approximation made simple[END_REF] exchange-correlation functional and included the D3 dispersion correction to describe Van der Waals (VdW) interactions [START_REF] Grimme | A consistent and accurate ab initio parametrization of density functional dispersion correction (DFT-D) for the 94 elements H-Pu[END_REF].

Our in-house library FFLearn was used to generate the NNPs, employing the symmetry functions proposed by Behler and Parrinello [START_REF] Behler | Generalized Neural-Network Representation of High-Dimensional Potential-Energy Surfaces[END_REF][START_REF] Behler | Atom-centered symmetry functions for constructing highdimensional neural network potentials[END_REF] to encode the local environments around each atom for neural network input. After testing different configurations, all NNPs were defined to have 2 hidden layers with 15 nodes each. For the Bayesian NNPs, nodes were dropped with a probability of 10% save for the output layer that was kept at 0% to utilise all the available information, and sampled 100 times during inference.

Results and Discussion

For a neural network potential to be practicable for atomistic simulations, it must have the ability to generalize as much as possible to previously unseen configurations, and, perhaps more importantly, to recognise when it cannot.

Since a neural network potential can be thought of as a complex function modelling the potential energy surface (PES), regions on the PES that are outside the learned domain can lead to erroneous predictions. Being able to recognise when this occurs would make the NNP more reliable.

Comparison of Generalizability

In order to assess the generalizability, two NNPs (one Bayesian and one classical) were trained on the hexagonal GeTe 300 K, 600 K, and 900 K datasets (the training set) and made to predict the energies of the 1200 K and 1500 K snapshots (the test set). Root mean squared error (RMSE) between the predictions and the reference DFT energies was used as the accuracy metric, and the results are summarised in Table 1.

The higher temperature of the test sets was used to emulate structures drawn from outside the configuration space immediately spanned by the training set. The Bayesian NNP shows some promising results for the 1200 K dataset, with an RMSE of 53.1 meV/atom. The error becomes much greater for the 1500 K dataset which is expected to contain structures that are more amorphous-like, thus containing too many unknown configurations. Meanwhile, the classical NNP is unable to produce any valid results for either datasets.

Uncertainty estimation

The 1500 K example in the previous section highlights the importance of being able to estimate the predictive uncertainty. Fig. 2 shows that this is achievable with the Bayesian NNP, exhibiting a clear correlation between the uncertainty estimates and the prediction errors (computed as the absolute difference between the DFT reference energy and the NNP prediction). It is important to note, however, that this uncertainty is an estimate rather than a true reflection of the error, and care must be taken when interpreting it. Namely, they do not necessarily have a strictly linear relationship.

Making use of the uncertainty estimates typically involves setting a threshold, above which the structure is considered unknown to the model. Following this, an MD powered by an NNP (hereafter NNP-MD) can, for example, raise a warning or be stopped. The identified structure can also be used to improve the potential by means of active learning [START_REF] Krogh | Neural Network Ensembles, Cross Validation, and Active Learning[END_REF] or on-thefly learning [START_REF] Jinnouchi | On-the-Fly Active Learning of Interatomic Potentials for Large-Scale Atomistic Simulations[END_REF].

Predictions on Ge 2 Sb 2 Te 5

We now focus on Ge 2 Sb 2 Te 5 (GST225), a material that lies on the GeTe-Sb 2 Te 3 tie-line on the ternary diagram. To compare the NNPs' generalizability to ternary alloys, a classical and a Bayesian NNP were each trained on the two extrema (GeTe and Sb 2 Te 3 ) and tested on Ge 2 Sb 2 Te 5 at different temperatures. The results are summarised in Table 2, and the lowest error case (1200 K) is represented graphically in Fig. 3. Similar to the results in Sec. 3.1, the classical HDNNP fails to give meaningful predictions.

On the other hand, the relative success of the Bayesian NNP is surprising, given that it has not been supplied with any direct knowledge of the Ge -Sb or the Ge -Sb -Te relationships, or any permutations thereof. Nonetheless, these results hint at the possibility of modelling other GST compositions along the GeTe-Sb 2 Te 3 tie-line with only a minimal dataset on the GST itself. As the dataset generation is the most expensive step to training NNPs, this could prove to be a very powerful development. However, more work is required to confirm these findings on different stoichiometries of GST, and to understand the mechanism behind this extrapolation capability before physically relevant properties can be calculated.

It should be noted that the particularly low RMSE for the 1200K dataset for the Bayesian NNP has no physical basis. Neural network training is a stochastic process that involves a few different random sampling operations. For example, the initial weights must be drawn from a probability distribution and the training dataset is randomly shuffled during training. In addition, as neural networks are very high dimensional, there exist many local minima. As such, different initial conditions practically always converge to different minima, which leads to slightly different results being produced for each new trained potential. Consequently, the lowest RMSE may lie at another temperature if the experiment was repeated, but the fluctuations in the values will remain minor. Though this effect is amplified here as GST225 lies far outside the trained configuration space (due to the lack of the Ge -Sb and the Ge -Sb -Te interactions in the training data, as mentioned above), it is still clear that the Bayesian NNP can give much more meaningful predictions than the classical.

Radial Distribution Function

To validate the Bayesian NNP on its ability to predict kinetic properties of periodic systems, an MD simulation was performed in the canonical (NVT) ensemble with the Nosé-Hoover thermostat at 1200 K for 20 ps followed by a simulation in the microcanonical (NVE) ensemble for 10 ps for the hexagonal GeTe using the LAMMPS [START_REF] Thompson | LAMMPS -a flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales[END_REF] package. The potential was trained on all GeTe datasets mentioned in Sec. 2.2. The size of the simulation cell was intentionally kept relatively small (192 atoms; same as the training set structures) in order to compare against DFT. From the NVE trajectory, a time-averaged total radial distribution function (RDF; g(r)) was extracted, taking into account all interatomic distances (Ge -Ge, Ge -Te and Te -Te). As shown in Fig. 4, the RDF shows a good agreement between the NNP and DFT. In addition, the NNP-MD was completed ∼35 times faster than the AIMD, running at a speed of 231.98 timesteps/s, compared to 6.65 timesteps/s for the DFT. Reducing the number of sampling steps for the Bayesian NNP will further increase the performance, and it is likely that even sampling only 10 times will result in only a minimal difference in the accuracy of the final prediction [START_REF] Gal | Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning[END_REF]. Moreover, as HDNNPs scale quasi-linearly with the number of atoms [START_REF] Behler | Four Generations of High-Dimensional Neural Network Potentials[END_REF], while DFT follows cubic scaling with the number of electrons, this performance gain is only expected to increase as the system size grows.

In contrast, a classical NNP trained under the same conditions was unable to run an MD due to unphysically high forces being predicted, causing the simulation to crash. As a result, no RDF for the classical NNP could be computed for comparison.

Conclusion

In this work, a Bayesian neural network potential for GeSbTe was developed using Monte Carlo dropout [START_REF] Gal | Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning[END_REF]. It was shown to have an improved generalizability compared to its non-Bayesian counterpart on hexagonal GeTe, with insightful estimates on the uncertainties of the predictions. Molecular dynamics simulations performed with a Bayesian NNP exhibited good agreement with the ab initio MD, shown in the form of the radial distribution function. In addition, a significant improvement of the computational cost was also observed. These indicate that neural network potentials can reliably be used for studies of physical properties requiring atomistic simulations at large scales, such as the thermal conductivity. Furthermore, it surprisingly was able to reasonably calculate the energies of hexagonal Ge 2 Sb 2 Te 5 , even in the absence of information on the two-body Ge -Sb and the three-body Ge -Sb -Te relationships in the training dataset. Future work will involve investigating this in more detail to better understand its underlying mechanism to try to take advantage of this for a cost-efficient training of a general GST potential.

Figure 1 :

 1 Figure 1: Schematic representations of a standard feedforward neural network (left) and a neural network with dropout, with the dropped nodes crossed out (right).

Figure 2 :

 2 Figure 2: Uncertainty estimates for the hexagonal GeTe validation and test sets against the prediction errors given by the Bayesian NNP.

Figure 3 :

 3 Figure 3: Energy predictions on Ge 2 Sb 2 Te 5 at 1200 K given by the classical and Bayesian NNPs trained on GeTe and Sb 2 Te 3 , compared to the DFT reference.

Figure 4 :

 4 Figure 4: A comparison of the total radial distribution function for GeTe at 1200 K between the AIMD trajectory and the NNP-MD trajectory.

Table 1 :

 1 RMSE (meV/atom) of hexagonal GeTe between the DFT energy and the NNP predictions.

		Validation 1200 K 1500 K
	Bayesian	3.18	53.1	1339.5
	Classical	1.49	561.5	9185.0

Table 2 :

 2 RMSE (meV/atom) of Ge 2 Sb 2 Te 5 between the DFT energies and the NNP predictions at all temperatures.

	Temperature (K)	300	600	900	1200 1500
	Bayesian	70.4	50.0	32.0	14.6	25.6
	Classical	992.7 964.2 933.6 902.2 455.5

Acknowledgment

This work was performed using HPC resources from GENCI-IDRIS (Grant 2021-A0110911995) and was partially funded by European commission through ECSEL-IA 101007321 project StorAIge and the French IPCEI program.


A crisis of the reproducibility of studies reported in leading science journals emerged a few years ago [START_REF] Peng | The reproducibility crisis in science: A statistical counterattack[END_REF][START_REF] Barba | The hard road to reproducibility[END_REF][START_REF] Munafo | A manifesto for reproducible science[END_REF][START_REF] Baker | 1,500 scientists lift the lid on reproducibility[END_REF][START_REF] Collins | Policy: NIH plans to enhance reproducibility[END_REF][START_REF] Begley | Six red flags for suspect work[END_REF] . One of the multiple origins of this crisis is that many statistically significant results obtained in some studies were not replicated or supported by other ones [START_REF] Perrin | Preclinical research: Make mouse studies work[END_REF][START_REF] Begley | Drug development: Raise standards for preclinical cancer research[END_REF][START_REF] Begley | Reproducibility in science: improving the standard for basic and preclinical research[END_REF] . One of the reasons given is that many published results are false positives: authors wrongly generalize (i.e., infer) their results to the target population after obtaining a statistically significant result in their study sample [START_REF] Ioannidis | Why most published research findings are false[END_REF][START_REF] Nuzzo | Scientific method: statistical errors[END_REF] . P-hacking and HARKing are ones of the inappropriate methods of analyzing and interpreting study results, leading to false positive results. P-hacking (or "p-fishing") occurs when researchers collect data without a predetermined sample size, select data without a priori identification of inclusion and exclusion criteria, or select statistical analyses until nonsignificant results become significant [START_REF] Greenland | Multiple comparisons and association selection in general epidemiology[END_REF][START_REF] Guller | Interpreting statistics in medical literature: a vade mecum for surgeons[END_REF][START_REF] Head | The extent and consequences of p-hacking in science[END_REF][START_REF] Bender | Adjusting for multiple testing--when and how?[END_REF] . HARKing occurs when researchers present a post hoc hypothesis based on their results as if it were an a priori hypothesis [START_REF] Kerr | HARKing: hypothesizing after the results are known. Personality and social psychology review : an official[END_REF] . However, even in the absence of p-hacking, HARKing, association biases [START_REF] Delgado-Rodriguez | Bias[END_REF] , or any other errors in scientific reporting, the probability to wrongly infer statistically significant results to the target population is very high is some situations. The purpose of the paper is to help researchers in veterinary science who do clinical research as well as clinicians interpreting those research findings appreciate the degree of (un)certainty when generalizing the study results to the target population of studied animals, according to the characteristics of the clinical study. This appreciation is also necessary to practice evidence-based veterinary medicine [START_REF] Lanyon | Evidence-based veterinary medicine: a clear and present challenge[END_REF] , especially when critically appraising evidence within the more general framework of the clinical decisionmaking process [START_REF] Vandeweerd | Is evidence-based medicine so evident in veterinary research and practice? History, obstacles and perspectives[END_REF][START_REF] White | Systematic evaluation of scientific research for clinical relevance and control of bias to improve clinical decision making[END_REF] .

I. GENERAL CONCEPTS

Throughout this paper, the context will always be the following: researchers seek to provide evidence that there is a true association between one exposure (e.g., neutering status, being treated with a treatment versus placebo, surgical versus medical intervention) and one outcome (e.g., disease occurrence, tumor remission, all-cause death). The statistical tests mentioned in this paper will therefore test such associations. The conclusions drawn from other statistical tests (e.g., those testing the normality of one distribution) will not be addressed. Furthermore, to facilitate the reading of the paper, the expression "significant association" will be used for "association classified as being statistically significant based on the p-value (p ≤ α)"; such expression does therefore not refer to the clinical significance of the association [START_REF] Kelsey | A contrary view on statistical significance[END_REF][START_REF] West | 5 ways statistics can fool you-Tips for practicing clinicians[END_REF] .

II. HYPOTHETICAL STUDIES

Two hypothetical studies will illustrate the concepts presented in the paper. For simplification purpose, these studies will be considered as feasible and ethical.

The context of the first study (study #1) is the following: Mullin et al. conducted a nonrandomized study to assess the association between doxorubicin chemotherapy use (versus no therapy use) and death occurrence in dogs with presumptive cardiac hemangiosarcoma [START_REF] Mullin | Doxorubicin chemotherapy for presumptive cardiac hemangiosarcoma in dogs[END_REF] . The statistically significant difference in death occurrence between the two groups suggested a potential effect of doxorubicin chemotherapy on time to death. In this context, the investigators of study #1 designed a randomized clinical trial to confirm the beneficial effect of doxorubicin chemotherapy within the first four months of use. To do so, they use the figures provided by the Kaplan-Meier curves in the paper of Mullin et al. [START_REF] Mullin | Doxorubicin chemotherapy for presumptive cardiac hemangiosarcoma in dogs[END_REF] (45% and 5% of alive dogs at four months, respectively) in order to calculate the sample size with 80% of statistical power (n=79 dogs in each group). Then they followed the dogs during four months and compared the two groups of dogs on death occurrence by using the Kaplan-Meier method and the log-rank test.

Study #1 can be considered as "confirmatory" since it is conducted to confirm the result of the previous study of Mullin et al.

The context of the second study (study #2) is the following: it has been suggested that masitinib monotherapy use has promising potential in treating canine epitheliotropic T-cell lymphoma [START_REF] Holtermann | Masitinib monotherapy in canine epitheliotropic lymphoma[END_REF] .

In this context and based on these premises, the investigators of study #2 conducted a randomized clinical trial to assess the effect of masitinib in dogs diagnosed with multicentric lymphoma on (partial or complete) remission of their lymphoma. To do so, they randomly allocate 80 dogs with multicentric lymphoma in one out of two groups: one group receiving masitinib plus prednisone (n=40), the other one receiving prednisone only (n=40). This number of 80 dogs was not based on an a priori sample size calculation, but based on the available time to recruit dogs within in a predefined period. Then they followed the dogs during three months and compared the two groups on the presence of partial/complete lymphoma remission at three months. Study #2 can be considered as "exploratory" since it is the first one studying such association between masitinib plus prednisone use (versus prednisone use only) and lymphoma remission in such population of dogs with multicentric lymphoma.

III. REVIEW OF STATISTICAL CONCEPTS

Some statistical reminders about the null hypothesis, type-I and type-II errors, and statistical power are briefly provided below. These points are covered in more detail elsewhere [START_REF] Shott | Detecting statistical errors in veterinary research[END_REF] .

A. The null-hypothesis and its acceptance or rejection A statistical test testing the association between one exposure and one outcome is based on a "null-hypothesis" which is the absence of such association in a predefined (target) population [START_REF] Lehmann | The Fisher, Neyman-Pearson Theories of Testing Hypotheses: One Theory or Two?[END_REF] .

For instance, the null-hypothesis of the log-rank test performed in study #1 is: "there is no association between doxorubicin chemotherapy use (versus no therapy use) and time to death in dogs with presumptive cardiac hemangiosarcoma.". If the null-hypothesis is rejected, one concludes that the study provides evidence supporting that there is a true association in the population between the exposure and the outcome. Most of the time, the rejection or acceptance of the null-hypothesis is based on the p-value provided by the statistical test: if the p-value is less than or equal to a threshold value (α), the association is classified as "significant" in the study sample, and it is concluded that the null-hypothesis is false (rejection of the nullhypothesis). (Such use of the p-value and the "significant" / "non significant" approach has however contributed to the reproducibility crisis and is questioned by many scientists [START_REF] Sterne | Sifting the evidence-what's wrong with significance tests?[END_REF][START_REF] Jeffery | Liberating the (data) population from subjugation to the 5% (P-value)[END_REF][START_REF] Mcshane | Abandon Statistical Significance[END_REF][START_REF] Amrhein | Scientists rise up against statistical significance[END_REF] .)

B. Type-I and type-II errors and statistical power When there is no true association between the exposure and the outcome in the population (or "when the null-hypothesis is true"), the probability of obtaining a significant association (p ≤ α) in the study sample is α (also known as the probability of type-I error).

When there is a true association between the exposure and the outcome in the population (or "when the null-hypothesis is false"), the probability of not obtaining a significant association (p > α) in the study sample is equal to β (also known as the probability of type-II error).

Therefore, is such situation, the probability of obtaining a significant association is (1-β), which is the value of the statistical power of the study.

IV. REVIEW OF DIAGNOSTIC TEST CONCEPTS

The sensitivity (Se) of a diagnostic test is the probability of a positive test result when the disease (or any health-related condition) is present, and the specificity (Sp) is the probability of a negative test result when the disease is not present. The positive predictive value (PPV) of a diagnostic test is the probability that an animal with a positive test result would have the disease.

In a sample of N animals, Se, Sp, and PPV can be estimated by calculating the proportion of true-positive animals among diseased animals (Se), the proportion of true-negative animals among disease-free animals (Sp), and the proportion of true positive animals among testpositive animals (PPV).

From the frequencies of Table 1, Se, Sp and PPV can be expressed as:

𝑆𝑒 = 𝑇𝑃 𝑇𝑃 + 𝐹𝑁 𝑆𝑝 = 𝑇𝑁 𝐹𝑃 + 𝑇𝑁 𝑃𝑃𝑉 = 𝑇𝑃 𝐹𝑃 + 𝑇𝑃 (1)
Table 1 can be re-expressed by using proportions instead of frequencies. To do so, let  be the prevalence rate of the disease in the population; it is therefore the probability of having the disease for an animal randomly drawn from this population before the result of the diagnostic test was known. If one randomly draws a sample of N animals from the population in which the prevalence rate is , there will be N. diseased animals and N.(1-) disease-free animals.

Among the N. diseased animals, there will be N..Se true-positive animals; among the N.(1-) disease-free animals, there will be N.(1-).Sp true-negative animals. Table 2 presents the frequencies of Table 1 by using Se, Sp, and  in a sample of N animals.

Using Formula (1) of the PPV by replacing TP with N..Se and by replacing FP with N.(1-).(1-Sp), one obtains:

𝑃𝑃𝑉 = 𝜋. 𝑆𝑒 (1 -𝜋). (1 -𝑆𝑝) + 𝜋. 𝑆𝑒 (2)
The Appendix illustrates the equivalence between Formula (1) and Formula (2) by using data from a hypothetical study. Therefore, in a situation where the prevalence rate of the disease () is low (for instance, 1%), a diagnostic test which has a very good Se (for instance, 80%) and an excellent Sp (for instance, 95%) will have a poor PPV (e.g., 14% provided by Formula (2) by using the previous values of , Se, and Sp; see Appendix for illustrative data). A PPV of 14% would be interpreted as: only 14% of animals with a positive test for the studied disease actually have the disease, while 86% of animals with a positive test are actually disease-free.

V. APPLICATION OF DIAGNOSTIC TEST CONCEPTS TO STATISTICAL TESTS

Although the interpretation of diagnostic tests and the interpretation of statistical tests have some important divergent concepts, some aspects of diagnostic test interpretation can serve as a reasonable metaphor for interpreting statistical tests [START_REF] White | Interpreting statistics from published research to answer clinical and management questions[END_REF] . To apply the concept of sensitivity, specificity, and positive predictive value of a diagnostic test to a statistical test testing the association between one exposure and one outcome, one must define what are the analogous terms for having the disease, for being disease-free, and for having positive and negative test results. With a statistical test, researchers seek for evidence that there is a true association in the population (i.e., evidence supporting that the null-hypothesis is false). A significant association in the study sample is in favor of the evidence they are seeking. In the situation of diagnostic tests, clinicians seek for evidence that the animal has the studied disease. A positive test for this disease is in favor of the evidence they are seeking. Therefore, the analogies described in Table 3 can be made [START_REF] Browner | Are all significant P values created equal? The analogy between diagnostic tests and clinical research[END_REF] .

The sensitivity of a diagnostic test is the probability of obtaining a positive test result when the disease is present. By analogy (see Table 3), the sensitivity of a statistical test is therefore the probability of obtaining a significant association when the null-hypothesis is false. This is analogous to the statistical power of a study (1-β). The specificity of a diagnostic test is the probability of obtaining a negative test result when the disease is absent. By analogy (see Table 3), the specificity of a statistical test is the probability of obtaining a non-significant association when the null-hypothesis is true. Since α is the probability of obtaining a significant association when the null-hypothesis is true, by analogy, the specificity of a statistical test is therefore (1α).

VI. RELEVANCE OF CALCULATING THE PPV OF A STATISTICAL TEST

In practice, a clinician is much more interested in the PPV of diagnostic tests than the sensitivity and specificity of such tests. When a positive test result is obtained for an animal, one would like to know the probability that the animal has the studied disease (i.e., the PPV for having the disease). By analogy with statistical tests (see Table 3), when a significant association is obtained, one would like to know the probability that there is a true association in the population (i.e., the PPV for the null-hypothesis being false). For instance, the PPV of the log-rank statistical test performed in study #1 is the probability that there is a true association between doxorubicin chemotherapy use (versus no therapy use) and time to death in dogs with presumptive cardiac hemangiosarcoma, if the investigators obtained a significant association in their study sample.

The analogy for a low PPV of a statistical test indicates that the probability that there is a true association in the population is low even when the association is significant. In such situations of low PPV, it would not be surprising that a significant result in one study is difficult to replicate as being significant in another study of the same association targeting the same population [START_REF] Ioannidis | Why most published research findings are false[END_REF] .

To calculate the PPV of a statistical test, Formula (2) will be used by replacing the sensitivity, specificity, and  for diagnostic tests by their analogous values for statistical tests. For a statistical test, we previously determined that Se=(1-β) and Sp=(1-α). We must now interpret the value of  for statistical tests.

When researchers plan to design a study to test an association between one exposure and one outcome, they have some level of uncertainty that this association actually exists in the population. They must have such level of uncertainty because, if they knew with 100% certainty that the null-hypothesis is false, a study would not be necessary. Before conducting a study, researchers therefore have in mind an a priori probability that the null-hypothesis is false in the studied population, which lies between 0 excluded and 1 excluded. For instance, because the association between doxorubicin chemotherapy use (versus no therapy use) and time to death in dogs with presumptive cardiac hemangiosarcoma has been previously suggested, the investigators of study #1 should have a higher level of certainty that this association truly exists, compared to the level of certainty of the investigators of study #2, where the association tested in study #2 has never been studied before.

In the context of diagnostic tests,  was the probability of having the disease for an animal randomly drawn from a population before the result of the test was known. By using the analogies presented in Table 3, for a statistical test,  would be the a priori probability that the null-hypothesis is false (i.e., the a priori probability that the association truly exists in the population). The "a priori" expression means "before the result of the statistical test is obtained from the study". (This expression refers to the notion of "prior information" in Bayesian statistics [START_REF] Greenland | Bayesian perspectives for epidemiological research: I. Foundations and basic methods[END_REF] , in the context of the PPV of statistical tests [START_REF] Browner | Are all significant P values created equal? The analogy between diagnostic tests and clinical research[END_REF][START_REF] Lash | The Harm Done to Reproducibility by the Culture of Null Hypothesis Significance Testing[END_REF] .) As Browner and Newman wrote, the value of  for statistical tests is based on "biologic plausibility, previous experience with similar hypotheses, and knowledge of alternative scientific explanations" [START_REF] Browner | Are all significant P values created equal? The analogy between diagnostic tests and clinical research[END_REF] . Therefore, in an exploratory study where researchers are the first ones to assess an association between one exposure and one outcome in a specific target population, it must be admitted that the a priori probability that such association truly exists () is low, despite the potential strong pathophysiological basis for this exploratory study [START_REF] Ioannidis | Why most published research findings are false[END_REF] .

We can rewrite Formula (2) of the PPV of a diagnostic test by replacing Se and Sp by their analogous value for a statistical test ((1-α) and (1-β), respectively):

𝑃𝑃𝑉 = 𝜋. (1 -𝛽) (1 -𝜋). 𝛼 + 𝜋. (1 -𝛽) (3) 10/22
With  being the a priori probability that the null-hypothesis is false (i.e., the probability that the association truly exists), α the type-I error, β the type-II error, and (1-β) the statistical power.

The interpretation of Formula ( 3) is as follows. Suppose that the statistical power of study #2 is 80% when testing the association between masitinib plus prednisone use (versus prednisone use only) and lymphoma remission in dogs with multicentric lymphoma, and α is set at 5% (α=0.05). Suppose that the probability that this association truly exists is (i.e., =0.01); this low value of  can be explained by the fact that study #2 is exploratory and therefore involves much a priori uncertainty about the existence of such association. Based on the characteristics of study #2 (a statistical power of 80% and its exploratory status with =0.01), the PPV calculated by using Formula (3) is 0.14 (14%). This value of 14% means that if the investigators of study #2 conduct this study and obtain a significant association, the probability that this association truly exists in the population of dogs with multicentric lymphoma is (only) 14%.

VII. FALSE POSITIVE REPORT PROBABILITY OF A STATISTICAL TEST

In the context where researchers would like to estimate the probability of wrongly concluding that there is a true association in the population after obtaining a significant one in the study sample, the complement of the PPV (1-PPV) is the most relevant indicator. This complement of PPV is called "false positive report probability" (FPRP) [START_REF] Wacholder | Assessing the probability that a positive report is false: an approach for molecular epidemiology studies[END_REF][START_REF] Held | Reverse-Bayes analysis of two common misinterpretations of significance tests[END_REF] . The FPRP is therefore the probability that there is no true association in the population after obtaining a significant association in the study sample. In other words, the FPRP quantifies the probability of wrongly concluding that there is a true association in the population after obtaining a significant one in the study sample. For instance, the FPRP of the log-rank statistical test used in study #1 is the probability of wrongly concluding that there is an association between doxorubicin chemotherapy use (versus no therapy use) and time to death in dogs with presumptive cardiac hemangiosarcoma after obtaining a significant association in the study sample.

11/22

We obtain the expression of the FPRP after calculation from Formula (3):

𝐹𝑃𝑅𝑃 = 1 -𝑃𝑃𝑉 = (1 -𝜋) × 𝛼 (1 -𝜋) × 𝛼 + 𝜋 × (1 -𝛽) (4)
With  being the a priori probability that the null-hypothesis is false (i.e., the probability that the association truly exists), α the type-I error, β the type-II error, and (1-β) the statistical power.

Figure 1 provides numerical examples of FPRP values according to selected values of  and (1β).

VIII. MISINTERPRETATION OF THE TYPE-I ERROR AND P-VALUE

In the vast majority of cases, the Type-I error α is set at 5% (α=0.05), which is considered as a low value. Many researchers wrongly think that since α is low, the conclusion following a significant association is accompanied by a (same) low probability of error [START_REF] Goodman | p values, hypothesis tests, and likelihood: implications for epidemiology of a neglected historical debate[END_REF][START_REF] Gliner | Problems With Null Hypothesis Significance Testing (NHST): What Do the Textbooks Say?[END_REF] . Similarly, pvalues are commonly misinterpreted as the observed probability to wrongly reject the nullhypothesis, which means that researchers commonly and mistakenly interpret the p-value as if it were the FPRP [START_REF] Greenland | Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations[END_REF] . For instance, if the p-value obtained in study #2 is 0.03, the investigators of study #2 would probably conclude with a mistaken belief that there is strong evidence for a true association between doxorubicin chemotherapy use (versus no therapy use) and time to death in dogs with presumptive cardiac hemangiosarcoma, with a 3% risk of error [START_REF] Jeffery | Liberating the (data) population from subjugation to the 5% (P-value)[END_REF][START_REF] Goodman | A dirty dozen: twelve p-value misconceptions[END_REF][START_REF] Wasserstein | The ASA's Statement on p-Values: Context, Process, and Purpose[END_REF] . The p-value is actually the probability of the observed or more extreme results, if the null-hypothesis were true and if there were no bias when estimating the association. (The p-value has therefore no meaningful interpretation per se since its value is conditional on an hypothesis that nobody would know with 100% certainty whether it is true or false [START_REF] Wagenmakers | A practical solution to the pervasive problems of p values[END_REF] .)

a priori probability that the null-hypothesis is false is compatible with the current state of knowledge in the field [START_REF] Held | Reverse-Bayes analysis of two common misinterpretations of significance tests[END_REF][START_REF] Colquhoun | The reproducibility of research and the misinterpretation of p-values[END_REF] .

X. CLINICAL SUMMARY

The probability of wrongly concluding that there is a true association in the population between one exposure and one outcome after obtaining a significant association in the study sample is neither equal to α nor to the p-value. Such probability (the "false positive report probability", or FPRP) depends on the characteristics of the study, namely its statistical power and its "confirmatory" versus "exploratory" status based on previous results in the same field.

In the case of an exploratory study (i.e., no previous studies have studied the same association in the same population yet), which is not uncommon in veterinary clinical research, researchers cannot be convinced that there is a true association in the population after obtaining a significant one in their study sample. However, in a study designed to confirm a result that other highquality studies have previously obtained, researchers can start to be confident when they conclude that there is a true association after obtaining a significant one in their study sample.

Furthermore, and importantly, unless the true association is strong, a small sample size (which is also not uncommon in veterinary clinical research) prevents one from being confident when concluding, even after obtaining a significant association in the study sample.

Researchers can start to be confident when they conclude that there is a true association in the population between one exposure and one outcome after obtaining a significant one in the study sample if (1) the statistical power of the study is high (at least 80%), and (2) the amount of knowledge on the subject of the study allows to estimate the a priori probability that this association truly exists of at least 20%. In such a situation, when the association is significant, the probability of wrongly concluding that this association truly exists is 20% (FPRP=20%).

One may think that such probability of 20% is too elevated, compared to the 5% that most researchers have in mind when they conclude after obtaining a significant association.

However, to reach an FPRP of 5%, the study must have the following characteristics: having a statistical power of 80% and being confirmatory with an a priori probability that there is a true association as high as 54% (=0.54). Unfortunately, such confirmatory studies would not likely be designed because researchers would be concerned that most funding sources and journals would prioritize projects as being more innovative [START_REF] Munafo | A manifesto for reproducible science[END_REF][START_REF] Hagen | Novel or reproducible: That is the question[END_REF] . This is the reason why confirmatory studies should be much more encouraged than they actually are [START_REF] Mogil | No publication without confirmation[END_REF] .

The readers must keep in mind that the interpretation of the probability of wrongly concluding that there is a true association in the population after obtaining a significant one in the study sample (i.e., the FPRP) assumes the absence of p-hacking, HARKing, association biases, or any other errors in scientific reporting. Even in this ideal world, the calculation of the FPRP, as it has been presented in this paper (and in other ones [START_REF] Ioannidis | Why most published research findings are false[END_REF][START_REF] Wacholder | Assessing the probability that a positive report is false: an approach for molecular epidemiology studies[END_REF] ), is still likely to be too optimistic [START_REF] Colquhoun | The reproducibility of research and the misinterpretation of p-values[END_REF] (i.e., the FPRP is likely to be even higher). Researchers as well as clinicians must nonetheless be aware that a significant association often provide a weak evidence that this association truly exists in the population. Such awareness is a necessary step to communicate more cautiously when writing the clinical relevance of the results of one study, potentially leading to clinical decisions thereafter. More generally, it is a necessary step for better veterinary research and (self-)evaluation of scientific research when practicing evidence-based veterinary medicine. "There is a true association in the population" ("the nullhypothesis is false") "To be disease-free" "There is no true association in the population" ("the null-hypothesis is true")

"To obtain a positive result" "To obtain a significant association" ("p ≤ α")

"To obtain a negative result" "To obtain a non-significant association" ("p > α")

Figure 1 .

 1 Figure 1. Values of the false positive report probability (FPRP) according to the statistical

Table 3 .

 3 Analogies between diagnostic and statistical tests.Situation of diagnostic testsCorresponding situation for statistical tests "To have the disease"

Table 1 .

 1 Distribution of frequencies within a sample size of N animals according to the resultsof a diagnostic test and the absence/presence of the disease.

	Result of a diagnostic test	Disease present	Disease absent	Total
	Positive	True Positive (TP) False Positive (FP)	FP+TP
	Negative	False Negative	True Negative
				TN+FN
		(FN)	(TN)
	Total	TP+FN	FP+TN	N

Table 2 .

 2 Distribution of frequencies within a sample size of N animals according to sensitivity

(Se) and specificity (Sp) of a diagnostic test, and according to the proportion  of diseased animals in the sample.

Table 4 .

 4 Illustrative example of distribution of frequencies within a sample size of 1493 animals.

	Result of a diagnostic test	Disease present	Disease absent	Total
	Positive	12	74	86
	Negative	3	1404	1407
	Total	15	1478	1493

Acknowledgments

I wish to thank Prof. Fanny Storck and Dr. Elodie Darnis for their very helpful comments they provided, and Dr. Jeremy Beguin for helping me in finding illustrative examples in the field of oncology. I am finally grateful for the very helpful comments and suggestions by the editor and two referees on the originally submitted version.

IX. FACTORS CONTRIBUTING TO A HIGH FALSE POSITIVE REPORT PROBABILITY

The Formula (4) indicates that the FPRP of a statistical test performed in a study whose objective is to provide evidence that there is a true association between one exposure and one outcome depends on the probability of type-I error (α), on the statistical power of the study (1β), and on the a priori probability that the null-hypothesis is false ().

A. Impact of the value of the type-I error (α) Suppose that, in study #1 designed with 80% of statistical power, the a priori probability that the null-hypothesis is false is 20% (i.e., =0.20). If α is set to 1% (α=0.01), the FPRP calculated from Formula (4) is 5%; if α is set to 5% (α=0.05), the FPRP increases to 20%. Therefore, and more generally, the higher the type-I error (α), the higher the FPRP. This point is one of the origins of a scientific movement that questions the type-I error threshold of 5% (α=0.05), and proposes to lower it to 0.5% (α=0.005) [START_REF] Benjamin | Redefine statistical significance[END_REF] . This movement is however not shared by all scientists [START_REF] Trafimow | Manipulating the Alpha Level Cannot Cure Significance Testing[END_REF] , and the convention for the type-I error (α) threshold set at 5% is likely to persist for years. From now on throughout the paper, the value of α will be set at 5% (α=0.05).

B. Impact of the value of the statistical power

Suppose again that study #1 is designed with an a priori probability that the null-hypothesis is false of 20% (=0.20). With a statistical power of 80% (by recruiting 79 dogs per group), the calculated FPRP is 20%. Suppose now that the investigators were finally able to recruit 39 dogs only per group instead of 79, the statistical power decreases to 50%. In this new situation, the calculated FPRP increases to 29% (see Figure 1). More generally, the lower the statistical power, the higher the FPRP. Since the statistical power of a study is directly related to its sample size, the FPRP increases with decreased sample size. This point indicates that a low statistical power (or small sample size) not only decreases the chances of obtaining a significant result when there is a true association, but it also makes any obtained significant result more likely to be false positive.

C. Impact of the a priori probability that the null-hypothesis is false Suppose again that study #1 is designed with a statistical power of 80% (with 79 dogs per group) and that the a priori probability that the null-hypothesis is false is 20% (=0.20). The calculated FPRP is 20%. Suppose that study #2 is designed with 80% of statistical power as well (with 40 dogs per group), but with the a priori probability that the null-hypothesis is false of 1% (=0.01), a low value due to its exploratory status. With such characteristics, the calculated FPRP of study #2 is 86% (see Figure 1). More generally, the lower the a priori probability that the null-hypothesis is false, the higher the FPRP. This point indicates that an exploratory study obtaining a significant association in the study sample is more likely to wrongly conclude that the association truly exists compared to a confirmatory study. This reasoning above is well known by clinicians using diagnostic tests [START_REF] Altman | Diagnostic tests 2: Predictive values[END_REF][START_REF] Grimes | Uses and abuses of screening tests[END_REF] : a diagnostic test can have excellent sensitivity and specificity but have a very low PPV (and therefore a high FPRP) if the disease prevalence rate is very low. With statistical tests, a similar phenomenon occurs: a statistical test can be very sensitive (excellent statistical power) and very specific (low threshold of type-I error α), however a significant association in the study sample can very poorly predict the existence of a true association in the population if the a priori probability that this true association exists is very low.

The most difficult task to appreciate the probability of wrongly concluding that the association truly exists in the population (i.e., the value of the FPRP) is to appreciate the a priori probability that the null-hypothesis is false [START_REF] Browner | Are all significant P values created equal? The analogy between diagnostic tests and clinical research[END_REF] . Such appreciation is beyond the scope of the paper. Briefly, some authors suggested a "reverse-Bayes" reasoning [START_REF] Matthews | Why should clinicians care about Bayesian methods?[END_REF] , which consists in setting the statistical power of the planned study and the desired FPRP value, then in seeing whether the value of the Appendix To illustrate Formula (2), suppose the data shown in Table 4 from a hypothetical study of 1493 animals, including 20 diseased animals and 66 animals with a positive test result for a diagnostic test.

By using the classical formulas of Se, Sp, and PPV (see formulas under Table 1), we obtain: