Gustavo Sousa

Walter Rudametkin

Laurence Duchien

Automated Setup of Multi-Cloud Environments for Microservices Applications

Keywords: multi-cloud, microservices, cloud management, variability management, software product lines

Multi-cloud computing has been proposed as a way to reduce vendor dependence, comply with location regulations, and optimize reliability, performance and costs. Meanwhile, microservice architectures are becoming increasingly popular in cloud computing as they promote decomposing applications into small services that can be independently deployed and scaled, thus optimizing resources usage.

However, setting up a multi-cloud environment to deploy a microservices-based application is still a very complex and time consuming task. Each microservice may require different functionality (e.g. software platforms, databases, monitoring and scalability tools) and have different location and redundancy requirements. Selection of cloud providers should take into account the individual requirements of each service, as well as the global requirements of reliability and scalability. Moreover, cloud providers can be very heterogeneous and offer disparate functionality, thus hindering comparison.

In this paper we propose an automated approach for the selection and configuration of cloud providers for multi-cloud microservices-based applications. Our approach uses a domain specific language to describe the application's multi-cloud requirements and we provide a systematic method for obtaining proper configurations that comply with the application's requirements and the cloud providers' constraints.

I. INTRODUCTION

Multi-cloud computing is the use of resources and services from multiple cloud providers where there is no agreement between providers to offer an integrated view of the system [START_REF] Grozev | Inter-cloud architectures and application brokering: taxonomy and survey[END_REF]. It is up to the customer or a third-party to integrate services from different providers. Multi-cloud computing is a way to avoid vendor dependence and to better exploit offerings in the cloud market by employing a combination of public and private cloud resources. Customers can build configurations that better fit their needs while reducing their dependence on any given provider.

The microservices architectural style [START_REF] Lewis | Mars) Microservices: a definition of this new architectural term[END_REF] provides an approach for building scalable applications. In a microservices architecture, applications are composed of small services that run in separate processes and can be deployed to different environments. Besides the benefits from separating concerns and of modularization, this approach scales services independently, allowing better use of resources.

We argue that these two approaches have strong synergism. On the one hand, building a multi-cloud application in-volves deploying to different clouds. Having independent wellisolated services facilitates this. On the other hand running microservices across private and public clouds from different providers allows for improved scalability and reliability.

However, building a multi-cloud solution is complex. Customers must consider factors such as functional requirements, configuration options, pricing policy, data center location, availability levels, etc. Cloud providers have different management interfaces and use different concepts for their offerings, making them difficult to compare. In addition, each of the application's services may have different requirements. They may be written in different programming languages, using different application frameworks or databases, and may have different scalability and availability requirements. Thus, a cloud provider that supports one of the application's services might not support another. Also, local regulations may require data to be stored within a given region, imposing further constraints.

We propose an automated approach for the selection and configuration of cloud providers for multi-cloud microservices-based applications. Our approach relies on ontology reasoning [START_REF] Sirin | Pellet: A practical OWL-DL reasoner[END_REF] and software product line techniques [START_REF] Pohl | Software Product Line Engineering: Foundations, Principles and Techniques[END_REF] to get from a high-level description of multi-cloud requirements to a configuration for a multi-cloud environment. While some approaches deal with aspects of multi-cloud management [START_REF] Ardagna | MODAClouds: A modeldriven approach for the design and execution of applications on multiple clouds[END_REF], [START_REF] Wright | A constraints-based resource discovery model for multi-provider cloud environments[END_REF], [START_REF] Petcu | Experiences in building a mOSAIC of clouds[END_REF], [START_REF] Ferry | Cloud MF: Applying mde to tame the complexity of managing multi-cloud applications[END_REF], [START_REF] Dastjerdi | CloudPick: a framework for QoS-aware and ontology-based service deployment across clouds[END_REF], they do not take into account the rich variability that exists in cloud providers' configuration options, losing valuable insight. Work done to manage variability in the cloud [START_REF] Quinton | SALOON: a platform for selecting and configuring cloud environments[END_REF] does not consider multi-cloud requirements. Our approach goes further than existing work by dealing with multi-cloud requirements for microservicesbased applications, while considering provider configuration variability.

Our key contributions are:

• An approach to build multi-cloud environments that handles the intrinsic variability in existing providers. • A method for translating high-level multi-cloud requirements to a set of provider specific feature model configurations. • Domain specific languages for specifying an application's multi-cloud requirements. • An experimental assessment of the approach using an example application and four popular cloud providers.

In Section II we discuss the motivation for multi-cloud computing and issues identified when setting up multi-cloud environments. In Section III we present our approach, and in Section IV we discuss its implementation. Finally, we discuss related work in Section V and conclusions in Section VI.

II. MOTIVATION

Motivations for using multiple clouds have been discussed in recent research [START_REF] Ferry | Towards model-driven provisioning, deployment, monitoring, and adaptation of multi-cloud systems[END_REF], [START_REF] Petcu | Towards multi-clouds engineering[END_REF]. A list of 10 reasons were identified in a survey conducted by Petcu [START_REF] Petcu | Multi-cloud: Expectations and current approaches[END_REF]. We classify these motivations into three broad categories:

• Reducing vendor dependence. This includes protecting applications from cloud outages or failures, but also avoiding vendor lock-in. In case no cloud satisfies all of the applications requirements, combining clouds becomes mandatory. Also, regulations may require sensitive data to be stored in a particular location, such as the user's country.

Multi-cloud applications exist for different reasons and approaches to manage them should take this into account.

To illustrate the complexity faced when building a multicloud system, we introduce an example scenario of an ecommerce application that requires the use of multiple clouds. This application is composed of the following services:

• a product catalog that keeps a record of product information and inventory; • a recommendation service that records purchases and recommends new products for customers; • a user management service that manages a user database and a credit card database; • an order management service that tracks orders and their statuses; • a payment service responsible for interacting with banking and credit card partners; • as well as a web frontend and a mobile API gateway to provide access to end users' devices.

Each of these services may be developed independently, using different technologies. Therefore, each service has different functional requirements. For instance, while the web frontend is a web application written in Java, the recommendation service is a standalone Python application with an embedded database. Moreover, according to its development process, a service may also require extra tools for continuous integration, issue tracking, staging resources, etc.

In addition to functional requirements, this example scenario should also meet the following:

• instances of web frontend and product catalog should be deployed to at least two different providers to ensure essential services are always available, even in the case of a provider failure. • data concerning European customers should be stored in a country that is a member of the European Union.

• the credit card database should be kept in a private cloud controlled by the company.

Apart from these requirements, there are no further restrictions on application services. Thus, when setting up a multicloud environment to deploy this application, we will be looking to optimize costs or improve the quality of service while complying with application requirements and constraints. Even from this simple example we can see that taking all relevant factors into account for setting up a multi-cloud environment can be a very complex task, which calls for automation and supporting tools.

In this paper, we present an automated approach for the selection of cloud providers and generation of proper configurations that considers the following concerns:

Cloud providers heterogeneity: Cloud providers usually offer functionality as services at different levels of abstraction such as infrastructure (IaaS), platform (PaaS) and software (SaaS). Even at the same abstraction level, providers employ varying terminology and features which usually do not map directly those of competing providers. These differences complicate further the comparison between providers.

Cloud providers variability: Cloud providers may have complex rules concerning their configuration. According to the provider, some functionality may only be available in a given region or for a given user plan. Some features may have conflicts while other may have dependencies. These as well as other complex constraints can be found in cloud providers and should all be considered to obtain a proper configuration.

Multi-cloud requirements for microservices-based applications: Microservices can be developed by different teams, relying on different technologies and methodologies and may therefore require functionalities at different levels of abstraction. In a multi-cloud environment, microservices can be deployed across private and public clouds from different providers to implement scalability and redundancy mechanisms or to comply with location constraints.

III. APPROACH

Our approach to deal with the concerns identified in Section II relies on (i) a high-level domain-specific language for describing multi-cloud requirements; (ii) feature models to manage the variability in cloud providers; and (iii) ontologies for dealing with the heterogeneity across cloud providers.

Our approach, depicted in Fig. 1, makes a distinction between the roles of cloud provider experts and developers. A cloud provider expert is responsible for providing formal descriptions of cloud providers' offerings through feature models and ontology mappings. On the other hand, developers interact with the approach by providing requirements of a multicloud environment. These requirements are matched against providers' descriptions to generate a multi-cloud configuration,

Cloud Provider Ontology

Cloud Provider Mappings

Cloud Provider Variability Model (Feature Model)

Cloud Experts

Application Ontology

Developer

Multi-Cloud Application Requirements

Global Cloud Ontology

Multi-Cloud Configuration

IT operations

Fig. 1. Overview of the actors and artifacts involved in the proposed approach which includes a selection of clouds providers and information on how to configure them for deploying the application.

A. A DSL for multi-cloud application requirements

We propose a language for defining multi-cloud requirements based on three main constructs: service requirements, cloud variables and instance groups. The service requirements are intended to describe the functionality required by a service to run. These can be virtual machines, runtime environments, databases, software management tools, or any other feature that is offered by a cloud provider. Fig. 2 depicts part of the requirements of the example given in Section II. For instance, the web frontend service requires a WebContainer with at least 2 GB of RAM that has a JavaWebContainer that implements version 2.5 of the Servlet API.

The concepts mentioned in service requirements are part of a global cloud ontology. If a concept is not in the ontology, the developer can use the application specific ontology to include new concepts. For example, one could add a new ontology class called HighPerformanceWebContainer and express that it is equivalent to a WebContainer that has more than 4 virtual CPUs. Once a new concept is added it can be used as part of a service requirements description.

The cloud variables describe conditions on the clouds where the services are to be deployed. This allows specifying requirements, such as, instances of a service should be placed in clouds maintained by different providers, or, that a given service should be deployed to a cloud located in a specific region. In Fig. 2, we have a condition that cloud B should be located in WesternEurope and that cloud A should be a different provider than B.

Finally, instance groups describe where to deploy services and how they will be instantiated. This construct lets developers specify constraints on the number of instances at the cloud or global levels. In our example, the web frontend service will In essence, the proposed language enables developers to define requirements for each service individually, relying on concepts from the cloud ontology. The ontologies are extensible and allow defining complex requirements involving logical operators and composition rules. Finally, cloud variables and instance groups can be used to define scalability and redundancy rules across providers or regions, as well as location and colocation rules for services.

B. Feature models for managing cloud variability

Feature models are widely used to model commonalities and variability across a software product line or system family [START_REF] Chen | Variability management in software product lines: A systematic review[END_REF]. A feature model is usually depicted as a tree diagram whose nodes represent features that can be selected to build a software product. While the tree hierarchy describes a composition relationship between features, additional constraints allow for defining extra relationships between features.

In our approach, feature models are used to model the variability in the configurations of a given cloud provider. This means that a feature model describes what are the services and resources offered by a given provider, how they are related and what are the constraints between them.

PHP

Postgre SQL The feature models employed in our approach, as the one shown, are extended with cardinalities and additional constraints. Therefore, we rely on previous developments done on cardinality-based feature models [START_REF] Czarnecki | Formalizing cardinalitybased feature models and their specialization[END_REF] and on modeling cloud providers with feature models [START_REF] Quinton | Cardinality-based feature models with constraints: A pragmatic approach[END_REF]. For expressing relative cardinalities and the complex constraints found in cloud providers, we also rely on the results of our previous work on extending feature models with relative cardinalities [START_REF] Sousa | Extending feature models with relative cardinalities[END_REF].

A relative cardinality is the number of instances of given feature in relation to one of its ancestors in the feature model tree hierarchy. For example, in constraint C 2 of Fig. 3, [0..3] (Gear, Application) is the cardinality of feature Gear in relation to Application in the case of feature Free being selected. This means that the maximum number of Gears in an Application is three when the Free plan is chosen.

Feature models have already been used in [START_REF] Quinton | Automated selection and configuration of cloud environments using software product lines principles[END_REF], [START_REF] Quinton | SALOON: a platform for selecting and configuring cloud environments[END_REF] to manage the variability in a cloud provider's configuration. However, in their approach the feature models employed consider only configurations for single service applications. For instance, in the case of the OpenShift PaaS, it allows for defining a configuration for either a Java or a Python application, but it does not allow defining a configuration for an application composed of two services, written in different languages or deployed to different regions.

This limitation exists because their implementation of feature models does not consider that individual instances of a feature can have a different decomposition of its subfeatures. But it is also due to the limitation of existing feature modeling constructs, which could not describe the complex constraints involving relative cardinalities that surface once we consider configuring multiple services through feature models.

In a summary, by modeling cloud provider variability we guarantee that the configurations generated by our approach are valid according to providers' constraints. In addition, the use of feature models extended with relative cardinalities allows for modeling the complex constraints identified when service-based applications are considered.

C. Ontologies for managing cloud heterogeneity

Feature models can capture the variability in a cloud provider but they cannot describe the semantics of these features. In a feature model we can specify that choosing a Free plan implies using only small size Gears, but we can not describe what is a small size Gear. For describing semantics of features we rely on the use of ontologies and mappings from features to ontology concepts. An ontology is a formal definition of concepts in a given domain and how they are interrelated. It is usually defined by a set of classes and objects that are related through properties.

In our approach, mapping a feature model to an ontology is done through a domain-specific language. Fig. 4 shows part of the mapping from the OpenShift PaaS feature model to the cloud ontology. In this example, we can see a mapping from feature Tomcat to ontology class JBossEWS. This mapping states that an instance of feature Tomcat represents an object of the ontology class JBossEWS. The concepts referenced in the mapping are also part of a global cloud ontology that includes the main concepts of the cloud computing domain.

However, while in some cases a feature may provide an instance of a cloud concept by itself, in other cases a set of features may be needed. For example, the Cartridge feature in the OpenShift PaaS (Fig. 3) represents an executable container that can be used to run a web application server or a database. In the cloud ontology, executable containers are represented by the class AppContainer, which has a set of properties such as CPU power, available RAM, disk size, the kind of software it can run, in which cloud it is deployed, etc. The properties of a given instance of feature Cartridge depend on how its subfeatures are selected. Therefore, an instance of feature Cartridge will have 1GB of RAM if it is of medium size, it will provide a JBoss EWS 2.0 as a servlet container if Tomcat feature is selected, and it will be deployed to Amazon US-East-1 cloud if the US feature is selected.

To deal with this scenario the mapping language allows describing properties that depend on the selection of features. The mapping for feature Cartridge in Fig. 4 illustrates how this can be done. In this case, the mappings work as a template for an ontology object, whose property values depend on the selection of features. Using ontologies has the advantage of allowing reasoning over concepts to find equivalence and specialization relationships between providers offerings and service requirements. For example, by reasoning upon ontologies we can discover that the JBoss EWS 2.0 provided by OpenShift feature Tomcat can be matched to the requirements of web frontend service (see Fig. 2), which requires a Java Servlet Container implementing version 2.5 of the Servlet API.

To enable further extensibility of our approach, application specific and provider specific ontologies can be optionally defined by developers and cloud experts respectively. Developers can define application specific concepts, as well as their relationship to global concepts, which can then be used directly in the description of the application's requirements. Similarly, provider specific concepts can be used to simplify the description of feature model mappings.

In a summary, ontologies are used to bridge the gap between service requirements and cloud providers. By using ontologies we can describe service requirements that are as complex as an ontology class description and reason upon them to find cloud providers that may offer instances of these classes.

IV. IMPLEMENTATION

To evaluate the feasibility of our approach, we developed a set of modeling and processing tools for modeling to find a multi-cloud configuration that suits the application's requirements. The following subsections provide more information about the implementation of these tools and their evaluation.

A. Modeling

To model the artifacts required by our approach, we designed three domain-specific languages for describing (i) multi-cloud requirements; (ii) cloud provider variability; and (iii) cloud provider mappings. We then used the xText [START_REF] Bettini | Implementing Domain-Specific Languages with Xtext and Xtend[END_REF] framework together with Eclipse Modeling Framework [START_REF] Steinberg | EMF: Eclipse Modeling Framework[END_REF] to provide an integrated editor for these languages. For describing the ontologies we use the standard OWL 2 Web Ontology Language [START_REF] Bechhofer | Encyclopedia of Database Systems[END_REF], for which many modeling and reasoning tools are available [START_REF] Sirin | Pellet: A practical OWL-DL reasoner[END_REF], [START_REF] Motik | Hypertableau Reasoning for Description Logics[END_REF]. The developed modeling tools are available at the accompanying site 1 .

B. Reasoning

The problem of finding a multi-cloud configuration that suits a set of application requirements can be seen as an assignment problem subject to constraints. As illustrated in Fig. 5, an instance of this problem is described by the requirements of an application and a set of available cloud providers. From application requirements we have a set of services, which are assigned by the developer to a cloud variable through a set of instance groups (see Section III-A). On the other side, we have a set of cloud providers that offer varying functionality. A solution to an instance of this problem is an assignment from cloud variables to cloud providers, which is depicted as a set of dashed lines.

Due to the number of providers, the search space for this problem can be very large, making it difficult to find a solution in a reasonable time. In addition, verifying the suitability of a solution is a complex and time consuming task as it requires reasoning on ontologies to convert application requirements to provider specific constructs and evaluating variability in feature models. These factors, when combined, may make this problem intractable. To deal with these difficulties our search algorithm employs backtracking with constraint propagation and constraint learning [START_REF] Dechter | Constraint Processing[END_REF]. We start with a set of constraints obtained from cloud conditions and increase our constraint base as verification of assignments fail. Doing so, we avoid performing expensive validations beforehand and gradually reduce the search space as new constraints are learned.

Verification is done in three steps: 1) validating that assigned providers comply with cloud conditions; 2) checking that service requirements can be mapped to a set of features in the assigned provider; and 3) verifying that there is a valid configuration where all services assigned to a provider can be deployed. Partial results from each phase are kept in a cache to avoid repeated calculations for similar assignments.

1) Validating cloud conditions: Cloud conditions are those described in application requirements as part of the cloud variables description (see Section III-A). In Fig. 2, for example, cloud variable B has as a condition that it should be assigned to a cloud located in WesternEurope. Evaluating these conditions is relatively simple and consists in checking if each assigned cloud variable complies with its constraint. However, this may still require some reasoning over ontologies to identify, for example, that a cloud located in France meets the criteria of being located in WesternEurope. To do so, we translate a cloud condition into an ontology class, which represents the class of clouds that complies with the condition, and then verify if the assigned provider cloud is an instance of this class. Fig. 6a shows an example of cloud condition for cloud variable B (see Fig. 2) and the corresponding ontology class described in the OWL 2 Manchester Syntax.

2) Mapping service requirements to provider features: After verifying that the assigned providers comply with cloud conditions we still need to check if each selected provider supports all functionality required by the services assigned to it. To do so, we use the cloud provider mappings to check, for each assigned service, if its required functionality can be mapped to a set of features in the provider's feature model. First each service is converted into an ontology class, which defines the class of objects that provides service requirements. Fig. 6b shows the requirements for web frontend service (see Fig. 2) and its corresponding ontology class. Then, we use ontology reasoning together with cloud provider mappings to find a selection of features that provides an instance of this required class. Fig. 6c shows the result from mapping web frontend service to the OpenShift PaaS.

In this example, the web frontend service requires a WebContainer, but feature Cartridge in OpenShift provides just an AppContainer (see Fig. 4). However, Open-Shift's specific ontology describes that a WebContainer is the same of an AppContainer with the data property isWeb set to true. OpenShift's mapping for feature Cartridge defines that the property isWeb is set to true if and only if the feature Web is also selected. Thus, an instance of feature Cartridge provides a WebContainer if its subfeature Web is selected.

The end result of this process is a feature selection defining a set of features that provide the requirements of a service. For the web frontend example, the expression obtained defines that an instance of feature Cartridge can provide service requirements if features medium, Tomcat, EU , Scalable and 5 instances of feature Gear are also selected. If a mapping from the required ontology class to a feature selection can not be found, then the assigned provider does not support the service requirements.

3) Verifying feature constraints and generating a complete configuration: After mapping service requirements to a selection of features, we know that the assigned providers support the services assigned to them. However, as there may be limits in the resources offered by a provider, conflicting features, or other constraints in cloud offerings, we still need to check if all services assigned to a provider can be deployed together and to generate a configuration for each provider. This process is done by evaluating the cloud providers' feature models against the feature selection expressions obtained in the previous step. This is a complex and time consuming task that involves translating the feature model to a constraint satisfaction problem and solving it to find a valid configuration that matches the requirements. This translation is based on existing work [START_REF] Mazo | Transforming attribute and cloneenabled feature models into constraint programs over finite domains[END_REF] and our previous developments [START_REF] Sousa | Extending feature models with relative cardinalities[END_REF] in cardinality-based feature models. If a solution to the corresponding constraint problem is found it means that the assignment is valid. Besides this, the output solution also defines for each selected provider a complete configuration of the corresponding feature model, detailing a hierarchy of feature instances that should be selected.

Our strategy for processing application requirements enables us to get from a set of requirements for microservices-based multi-cloud applications to a concrete set of selected features in each cloud provider. By decomposing this process into three steps we avoid doing more expensive calculations when not needed and we reuse intermediate results.

C. Experimentation

To validate the effectiveness of our approach and evaluate the performance of the developed tools we elaborated an example application based on the requirements described in Section II and measured the time consumed at each stage of processing, as well as the total time to get from requirements to a concrete multi-cloud configuration.

The application requirements were described using the domain-specific language for multi-cloud requirements and include 13 services, with 5 cloud variables and 15 instance groups. In addition, we elaborated feature models and ontology mappings for four popular cloud providers: Open-Shift2 , Heroku3 , Google4 and Jelastic 5 . Information from cloud providers was obtained from their documentation and through the use of their configuration tools.

We performed 20 executions of the developed tool to convert application requirements to a multi-cloud configuration. All experiments were run on a MacBook Pro Computer with a 2 GHz Intel Core i7 processor and 8GB of memory. From the obtained results, shown in Table I, we can see that translating from high-level application requirements to a concrete configuration took on average 6.62s. For this set of four providers 32 valid solutions, in an universe of 1024 possible assignments, were found in an average time of 24s. To evaluate the performance in the presence of more providers we randomly generated extra 46 providers in a way that no generated provider would completely support the features required by the example application. With this new set of 50 providers, 52s were need to find the same 32 solutions. This indicates, that despite the time needed to process and generate a valid solution, invalid assignments can be discarded more quickly and reduce the search space.

D. Limitations

Our approach focus on the selection and configuration of cloud providers for a multi-cloud scenario and does not deal directly with the deployment of applications. Generation of deployment scripts from feature models has been discussed in the literature [START_REF] Quinton | Automated selection and configuration of cloud environments using software product lines principles[END_REF], and similar techniques could be integrated in our approach. We also still do not consider costs and quality of service for optimizing the selection as these depend heavily on application usage. Finally, for the conducted experiments, cloud provider feature models and ontology mappings were manually defined relying providers' documentation, and are thus susceptible to errors.

V. RELATED WORK

Several recent works were proposed to deal with aspects of the management of multi-cloud systems. CloudMF [START_REF] Ferry | Cloud MF: Applying mde to tame the complexity of managing multi-cloud applications[END_REF] relies on a domain-specific language and models@runtime to deal with heterogeneity across providers and support provisioning and deployment of multi-cloud systems. We also employ domain-specific languages for modeling requirements and provider descriptions, but in addition we also support the selection of cloud providers and variability management. The mOSAIC [START_REF] Petcu | Experiences in building a mOSAIC of clouds[END_REF] and soCloud [START_REF] Paraiso | soCloud: a service-oriented component-based paas for managing portability, provisioning, elasticity, and high availability across multiple clouds[END_REF] projects propose a multi-cloud PaaS for interoperability across multiple clouds. Instead of proposing developers a new platform service, our approach aims at supporting them to choose and combine existing platform, infrastructure and software services. Wright et al. [START_REF] Wright | A constraints-based resource discovery model for multi-provider cloud environments[END_REF] propose a constraints-based method for composing multiprovider environments. Like in our work, their goal is to find an assignment of services to cloud providers according to some constraints. However, requirements are defined as low-level constraints and services can use only infrastructure resources. CloudPick [START_REF] Dastjerdi | CloudPick: a framework for QoS-aware and ontology-based service deployment across clouds[END_REF] is a framework that supports the selection of IaaS providers and the deployment of a network of virtual appliances (preconfigured virtual machine images). Cloud4SOA [START_REF] Kamateri | Cloud4SOA: A semanticinteroperability paas solution for multi-cloud platform management and portability[END_REF] also proposes the use of ontologies to provide interoperability across PaaS providers. Like in our approach, ontologies are used for semantically mapping requirements to providers' offerings. However, both approaches do not manage variability and constraints of cloud providers configurations.

Managing variability and heterogeneity in the cloud has also been identified as an important issue that is subject of substantial work. In [START_REF] Ferreira Leite | Automating resource selection and configuration in inter-clouds through a software product line method[END_REF] and [START_REF] García-Galán | Automated configuration support for infrastructure migration to the cloud[END_REF], feature models are used to model variability in infrastructure resources (e.g. virtual machine, storage, network, etc) and support provider selection. Feature models are also used in a similar way in [START_REF] Dougherty | Model-driven auto-scaling of green cloud computing infrastructure[END_REF], but to optimize energy consumption. In our work, we consider not only variability in a resource or service configuration but across all the functionality offered by a provider.

Ontologies are also proposed as a way to deal with heterogeneity and the semantic gap between requirements and cloud providers offers [START_REF] Petcu | Experiences in building a mOSAIC of clouds[END_REF], [START_REF] Amato | A framework for semantic interoperability over the cloud[END_REF], [START_REF] Cretella | A semantic engine for porting applications to the cloud and among clouds[END_REF]. However these approaches do not consider the variability in providers' offerings and apply reasoning upon predefined instances.

The SALOON [START_REF] Quinton | SALOON: a platform for selecting and configuring cloud environments[END_REF] approach also relies on feature models to capture cloud provider configuration options and ontologies to map from requirements to cloud features. However, it does not consider applications that can be composed of multiple services and do not support multi-cloud requirements.

Our approach shares with existing work the use of ontologies to deal with heterogeneity across cloud providers and of software product line techniques for managing variability in cloud provider configurations. However, our work takes into account variability at the provider level, not only for a given resource type (virtual machine, storage, etc) and not just for one application service.

In addition, we also employ domain-specific languages and model driven-engineering techniques to achieve separation of concerns between different user roles. We support the description of requirements for microservices-based applications and mechanisms to achieve multi-cloud requirements of location, scalability and redundancy. Overall, our approach differentiates from others by taking into account variability in cloud configuration options, multi-cloud requirements and service composition to build a suitable environment. VI. CONCLUSION Multi-cloud computing has the potential of reducing vendor dependence, increasing application reliability and optimizing resource usage. However, the wide number of available cloud providers, their high heterogeneity and their intricate configuration options, make it very complex to exploit these benefits. In this paper, we describe an approach to deal with this complexity, by supporting the selection and configuration of multicloud environments for microservices-based applications.

Our approach relies on a domain-specific modeling language for defining multi-cloud requirements and a combination of reasoning tools to obtain: (i) a valid assignment of application services to cloud providers and (ii) proper configurations for these providers. We implemented this approach as a set of tools for modeling the required artifacts and reasoning upon them. We conducted some experiments to show that valid configurations can be obtained in reasonable time.

For future work, we intend to include pricing and quality of service information as part of our analysis and investigate search algorithms and heuristics to deal with this as an optimization problem. This goes into the direction of our longterm goal, which is towards self-adaptive multi-cloud environments capable of identifying optimization opportunities as application requirements and cloud market evolve.

Fig. 2 .

 2 Fig. 2. Sample multi-cloud requirements

Fig. 3

 3 depicts part of a feature model elaborated to describe the variability in the OpenShift PaaS provider. OpenShift

Fig. 5 .

 5 Fig. 5. Multi-cloud configuration as an assignment problem

numInstances 1 .. 5

 15 cloud B { self.location = WesternEurope } Cloud that dataCenterLocatedIn some (partOfRegion value WesternEurope) (a) Cloud variable definition and corresponding ontology class options AutoScale } } WebContainer that hasMemory some (DataAmount that hasUnit value MB and hasValue some integer [>= 2048]) and hasSoftware some (JavaWebContainer that providesAPIs some (Servlet that hasVersion value 2.5)) and deployedInCloud value AmazonEUWest1 and options value AutoScale and maxNumInstances some integer[>= 5] (b) Service requirements and corresponding ontology class Cartridge { medium, Tomcat, EU, Scalable, 5 Gear } (c) Feature selection for web frontend service in OpenShift PaaS.

Fig. 6 .

 6 Fig.6. Sample translations from requirements, to ontology classes, to features.

•

  Optimizing performance and costs. Different cloud providers offer specific advantages. Combining them can improve QoS or costs. Multi-clouds can also be used to offload processing to other clouds when dealing with peaks, or to deploy services closer to the end user.

• Complying with requirements, constraints or regulations.

http://researchers.lille.inria.fr/ ∼ sousa/mmcloud/

http://www.openshift.com/

http://www.heroku.com/

http://cloud.google.com/

http://jelastic.com/


Nacim Betrouni
email: nacim.betrouni@inserm.fr

Camille Munck

Wael Bensoltana

Grégory Baert

Anne- Sophie Dewalle-Vignion

Arnaud Scherpereel

Serge Mordon

Real-time light dosimetry for intra-cavity photodynamic therapy: Application for pleural mesothelioma treatment

Keywords: photodynamic therapy, intra-cavity application, light distribution modeling, dosimetry, real time tracking, imaging

Complete and homogeneous illumination of the target is necessary for the success of a photodynamic therapy (PDT) procedure. In most applications, light dosimetry is done using detectors placed at strategic locations of the target. In this study we propose a novel approach based on the combination of light distribution modeling with spatial localization of the light applicator for real time estimation and display of the applied dose on medical images. The feasibility approach is demonstrated for intrapleural PDT of malignant pleural mesothelioma.

1-Introduction

Photodynamic therapy (PDT) is an emerging technic in oncology. Its mechanism of action is based on the interaction of a photosensitizer (PS) drug, a light source for the PS activation and oxygen in tissues. The combination of these three elements induces chemical reactions leading to the creation of reactive oxygen species (ROS) and tumor cell death.

The achievement of the desired therapeutic effect of a PDT procedure relies on the accurate PS concentration distribution and on the continuous and uniform light delivery to the tissues. This last issue is particularly critical due to the variability in target geometry: solid, hollow organs and cavities,… For cavities, various types of light applicators were investigated as bare-fibers, having a directed radiation pattern and being applicable for small lesions, cylindrical diffusers [START_REF] Beyer | Systems for light application and dosimetry in photodynamic therapy[END_REF] with scattering dome applicators which has a homogeneously distributed radiation pattern with various active lengths and lastly, flat or flexible 2D applicators as textile [START_REF] Mordon | Light emitting fabric technologies for photodynamic therapy[END_REF] or blanket [START_REF] Lianga | Maximizing fluence rate and field uniformity of light blanket for intraoperative PDT[END_REF] allowing obtaining planar illumination.

In addition to these applicators, sensors are used to monitor light application by the continuous measurement of the light fluence rate in different locations in the cavity.

Most often, these sensors consist of isotropic probes collecting light from every direction [START_REF] Zhu | Real-time treatment feedback guidance of pleural PDT[END_REF]. This dosimetry method allows only obtaining punctual measurements of light distribution whereas continuous spatial feedback is required to estimate received dose at each target point. This is particularly the case for Malignant Pleural Mesothelioma (MPM) treatment.

Treating MPM remains a challenge with two main alternatives: palliative chemotherapy for inoperable patients, and multimodal treatment including surgery, combined with chemotherapy and radiotherapy for the others.

Surgery offers the best chance of survival for this still incurable disease, however after the most complete tumor resection, microscopic tumor cells persist and surgery should be associated with an adjuvant local treatment. PDT appeared as a potential option for an effective intra-operative complement to surgery. After extensive preclinical studies, first clinical results were reported by the University of Bern group [START_REF] Ris | Photodynamic therapy with chlorins for diffuse malignant mesothelioma: initial clinical results[END_REF]. Currently, the most complete studies, with a major impact on survival and minimum toxicity, are certainly those led by the Pennsylvania team [START_REF] Friedberg | A phase I study of Foscan-mediated photodynamic therapy and surgery in patients with mesothelioma[END_REF][START_REF] Friedberg | Photodynamic Therapy and the Evolution of a Lung-Sparing Surgical Treatment for Mesothelioma[END_REF][START_REF] Friedberg | Radical pleurectomy and intraoperative photodynamic therapy for malignant pleural mesothelioma[END_REF]. The photosensitizer (Porfimer sodium (Photofrin)) is administered to the patients 48 hours before a maximal surgical tumor resection. Then, a light applicator connected to a laser source (wavelength 635 nm) is moved inside the pleural cavity, to illuminate the cavity walls (the target), filled with dilute intra-lipid solution, until a fluence of 20 J.cm - 2 is obtained. The light delivery monitoring is achieved by 7 isotropic probes placed at strategic locations in the thorax and connected to a dosimetry system ( [START_REF] Dimofte | In vivo light dosimetry for pleural PDT[END_REF], [4;10]).

However, this method does not provide information about the light delivered between these 7 locations. In this study, we describe a light applicator and introduce a new method for light dosimetry during PDT procedures of intra-cavities with complete and continuous feedback of the applied light dose. We present the feasibility of this approach on a phantom mimicking an intraoperative thorax cavity.

2-Materiel and Methods

The global approach proposed herein is based on three ideas. First, we present an unique action model that characterizes the light distribution around the applicator.

The second idea is about using a spatial locator for real time tracking of the applicator movements inside the cavity in order to cumulate spatially and temporally the light dose applied to the target. The last idea requires the use of imaging to define the target and to be used as a spatial reference map for the real time display of the applied light dose.

2-1 Light applicator construction and setting

The proposed light applicator is an adaptation of the model used in the previous trials [START_REF] Friedberg | Phase II trial of pleural photodynamic therapy and surgery for patients with non-small-cell lung cancer with pleural spread[END_REF]. It consists of a cylindrical diffuser held in an endo-tracheal tube in which a carbon tube was inserted, to harden the tube and to correct its curvature. The cuff of the endo-tracheal tube is filled with intra-lipid solution at the concentration of 0.01% to act as a scattering agent.

For the real time localization of the applicator, an electromagnetic 3D tracking device is used to track the applicator movements. This tracking system (TrackStar, Ascension Technology Corporation, Burlington, VT, USA) is composed of a control unit with plugs to connect a transmitter and up to 4 six degrees-of-freedom sensors.

One sensor (2 mm diameter at the distal end and 1.2 mm diameter for the cable) is fixed just above the cylindrical diffuser (Figure 1.a). It allows the localization of the sensor, and therefore the light wand, giving a 3D position and orientations regarding to the reference defined by the transmitter with a precision of 0.5 mm. Figure 1.b represents the light wand assembled.

2-2 Characterization of a light distribution model

Light distribution around the diffusing tip of the applicator was characterized by direct measurements of the light fluence rate (irradiance (W.cm -2 )) using sensors completed with a digital photography to define fluence rate isosurfaces.

Measurements were done in a plastic tank of black color, limiting light reflection, filled with 0.01% dilute intralipid. The tip of the applicator was fixed horizontally in the tank through a cable gland and the optical fiber was connected to a medical diode laser 635 (Ceralas®, Biolitec).. For the fluence rate measurements, an isotropic probe (Model IP 85, Medlight®S.A., Switzerland), collecting light in a large solid angle, was connected to an optical power meter (Model 841-PE,Newport® Corporation, Irvine, CA, USA). The isotropic probe was fixed vertically in a plastic tube and moved above the diffusing tip of the light wand, using a millimetric precision benchmark with vertical and horizontal freedom degrees.

A calibration factor was defined to convert the punctual power measurements (W) into fluence rate values (W.cm -2 ). It was estimated by correlating pairwise power measures obtained by the isotropic probe and fluence measures obtained by a 1cm² surface detector (Model PD300, OPHIR, Israel).

The previous method allowed only obtaining measurements for discrete positions on the light applicator. This dosimetry method with the powermeter only gave punctual measurements of the light distribution. In order to obtain a more complete and continuous light distribution characterization, a digital photography was taken above the tank. After spatial and chromatic calibrations of the picture, the measurements done previously and their respective positions were matched with the image to define iso-surfaces as represented in figure 2.

Complete description of the method is provided in Munck et al. [START_REF] Munck | Illumination profile characterization of a light device for the dosimetry of intra-pleural photodynamic therapy for mesothelioma[END_REF].

2-3 Tridimensional Imaging

Radiation therapies (radiotherapy and brachytherapy) are always image-based techniques. Imaging allows to define the targets, to estimate their volumes and in some cases, to define organs at risk to be spared. It is also used to optimize the ballistics and as a reference map for the dose display.

Regarding PDT, imaging is being increasingly used especially for interstitial applications as in prostate [START_REF] Betrouni | An image guided treatment platform for prostate cancer photodynamic therapy[END_REF], Glioblastoma [START_REF] Stepp | ALA and malignant glioma: fluorescence-guided resection and photodynamic treatment[END_REF] and head and neck [15;16] diseases. As for radiotherapy, the goal is mainly to define the targets to treat.

For MPM management, 3D computed tomography (CT) imaging is the reference for pleural evaluation, fortumor staging and follow-up. We propose to use these images in the dosimetry process of the PDT procedure. First, these images are used to estimate volume and surface of the pleural cavity. Thereby, the total required fluence and light application duration can be estimated before the treatment.

Another benefit to CT scan images is to use them as a reference space for the real time light dose display during the treatment. However, as these images are acquired before the procedure, a spatial registration is required to compute the transformation between the pre-operative space and the per-operative space. Fiducial markers consisting of 5 mm diameter capsule filled with paraffin oil can be attached to the patient around the region to be imaged. These markers are multimodality imaging compatible and able to create clear patterns on the images.

Spatial mapping of light dose

In order to compute a 3D light dose (D), the action model established for the applicator is combined with the positions returned by the spatial locator. The dose is matched with the images space using the following transformation:

𝑇 "##$%&'()* %+',-. = 𝑇 1*'2.+%((-* %+',-. ° 𝑇 "##$%&'()* 1*'2.+%((-* (1)
𝑇 "##$%&'()* 1*'2.+%((-* is formed using the positions and the orientations given by the tracking device and continuously updated according to the tracking frequency rate, while

𝑇 1*'2.+%((-* %+',-.
is the registration transformation that locates the patient regarding to the localization system. It is computed by mapping the coordinates of the fiducial markers on the images and their coordinates in the tracking device space (transmitter space). These last coordinates are obtained using a second sensor that points them successively.

Once the 3D spatial mapping done, for each voxel point p(x, y, z), the cumulated dose is estimated as:

𝐷 #(6,8,9) = 𝐼𝑟𝑟𝑎𝑑𝑖𝑎𝑛𝑐𝑒𝑀𝑜𝑑𝑒𝑙 𝑥, 𝑦, 𝑧 . 𝑑𝑡 1*-'(+-2( (%+- (2)
Where IrradianceModel is the above defined unique action model that characterizes the light applicator.

3-Validation of the approach 3-1 Protocol

Experiments are done on an anatomically realistic phantom consisting of a humansized hemithorax, made of plaster, and 3D printed into a right-side intra-operative and post tumor resection pleural cavity (Figure 3).

Four fiducial markers were attached to the phantom and CT images were acquired using standard protocol used for MPM exam (Figure 4).

A PDT procedure is simulated by moving the light applicator inside the cavity. The evaluation protocol consists in comparing the light dose measured by our proposed method and by the method of reference wi the 7 isotropic probes. First, the 3D positions of the probes are obtained using a spatial locator for each position; the model estimated fluence is extracted from the matrix D (equation 2). These values are referred as Model_Measurements. The values collected by the probes are referred as Ref_Measurements.

Six independent experiments (Exp1 to Exp6) were carried out by varying the light applicator movements (speed, trajectory,..) inside the cavity.

3-2 Results

Figure 5 shows the accumulated 3D light distribution (matrix D) combined with the CT images to locate the treated and untreated regions. A special color look up table (LUT) gives relative display regarding a reference dose value. Each color represents 10% of this dose.

Figures 6 depicts the temporal evolution of the cumulated fluence (J.cm -2 ) measured by the two methods in one of the 7 intrathoracic locations (probes) respectively for (Exp1, Exp2, Exp5), and (Exp3, Exp4, Exp6). From this figure, it appears that the Ref_Measurements have a linear evolution. This result can be explained by the fact that the probes were continuously collecting light through the intralipid liquid even when the applicator was far from their positions, while the Model_Measurements do not have this ability because the model has a limited volume as defined in section 2.2.

For the 7 probes, the mean error between the reference measurements and the estimated measurements was about 10% with under-estimation for the proposed method. Since this under-estimation was constant for the six experiments, a correction factor of 1.1 was established and a new experiment was carried out after correcting the Model_Measurements with this value. Figure 7 presents the temporal evolution of the mean light dose calculated in one position for all experiments after correction.

The difference between the two methods measurements decreased below 1%.

4-Discussion

The success of a PDT procedure depends on the optimal dosimetric configurations of the photosensitizer and the light. The first issue relies on the determination of the PS distribution to the target tissues, which remains particularly difficult to monitor in clinical conditions. In most applications, based on dose escalation trials, a defined concentration is considered as optimal with sufficient and homogeneous distribution in tissues, and limited toxicity. For Photofrin, used in PDT for MPM, a dose of 2mg/ Kg is commonly administered to the patient.

The second issue, light delivery, is to apply a sufficient light dose to each target point.

The approach proposed herein constitutes a new trend in dosimetry for intrapleural PDT. Indeed, instead of the continuous measurement of the light delivered, using dedicated materiel (probes, spectrometer, power-meter,…), our dosimetry method relies on a unique action model that characterizes the light propagation from the light applicator. The model was established by defining the light distribution profile in the intra-lipid solution. It takes into account the absorption and the diffusion around the applicator but not the multiple scattering that occurs within the intralipid solution. This can explain the systematic under-estimation of the model.

Anyway, in this kind of treatment, the procedure is based on a continuous movement of the light source inside the cavity, which means that the time spent at a specific spatial position is short leading to a non significant impact of the light scattering.

The second originality of our dosimetry method resides in the real time tracking of light applicator's movements inside the cavity, using a spatial locator. Thus, by combining the model and the positions, a complete 3D light dose is estimated.

Spatial locators were already used in laser applications to track the light applicator movements ( [START_REF] Mordon | Endovenous laser treatment of the great saphenous vein: Measurement of the pullback speed of the fiber by magnetic tracking[END_REF], [START_REF] Sadick | Advantages of real-time magnetic tracking of the cannula for controlled laser assisted lipolysis[END_REF]) and were proved to be efficient tools to monitor light application, in addition to being safe and clinically suitable.

Actually, from a clinical point of view, the real time update and display of the applied light dose brings the physician three main benefits. The first is the continuous feedback about the light delivered to the treated area. It improves the homogenization of the light dose by adapting the light delivery to untreated or not completely treated regions. The second benefit comes from the fact that visual assessment of the dosimetry is more natural than the punctual evaluation of discrete measurements. Moreover, the use of a gradient dose display allows a quick appreciation of the dose being delivered. Ultimately, one can obtain images as those used in radiotherapy treatment planning (Figure 8).

The last benefit regards the procedure time. The proposed approach reduces the intervention time by avoiding the setting of the measurements probes inside the thorax which take about 15 minutes in te case of intrapleural PDT for MPM.

Figures captions

Figure 1 :

 1 Figure 1: The light applicator. (a) The position of the 3D sensor according to the cylindrical diffuser. (b) Assembly of the applicator.

Figure 2 :

 2 Figure 2:. Illumination profile of the applicator when connected to laser emitting 3 W.

Figure 3 :

 3 Figure 3: (Left) Thoracic phantom with surgical opening and isotropic probes placement in the pleural wall. (Right) Photodynamic therapy procedure simulation.

Figure 4 :

 4 Figure 4: 3D Computerized Tomography of the thoracic phantom with patterns created by the fiducial markers. Images are displayed using three orientations: transversal, sagittal and coronal.

Figure 5 :

 5 Figure 5: 3D light dose display combined with CT images of the thorax cavity phantom. Crossing lines (top left viewer (sagittal), top right viewer (coronal) and center right viewer (axial)) indicate the light applicator position inside the cavity.

Figure 6 :

 6 Figure 6: Cumulated fluence measured in one unique position over time by the two methods: our proposed model based method (Model_Measurments) and the reference method based on a probe and a powermeter (Ref_Measurments). Six independent experiments were done. (a) Measurements: Exp 1, Exp 2 and Exp 5. (b) Measurements: Exp 4, Exp 3 and Exp 6.

Figure 7 :

 7 Figure 7: Temporal evolution of the light dose (fluence J.cm -2 ) measured in one position by the two methods: Reference method based on a dosimeter (power-meter)and Model method assuming a unique profile emission from the light applicator after correction.

Figure 8 :

 8 Figure 8: Example of a treatment planning display for a lung radiotherapy showing the distribution of radiation doses delivered to the tumor. The prescription dose is 50 gray.

Conflict of interest

All the authors have no conflict of interest to declare.