Trung-Dung Le

Verena Kantere
email: vkantere@uottawa.ca

Laurent D'orazio

Dynamic estimation and Grid partitioning approach for Multi-Objective Optimization Problems in medical cloud federations

Keywords: Cloud computing, Multiple Linear Regression, Cloud federations, Genetic Algorithm, Non-dominated Sorting Genetic Algorithm

Data sharing is important in the medical domain. Sharing data allows large-scale analysis with many data sources to provide more accurate results. Cloud federations can leverage sharing medical data stored in different cloud platforms, such as Amazon, Microsoft, etc. The pay-as-you-go model in cloud federations raises important issues of Multi-Objective Optimization Problems (MOOP) related to users' preferences, such as response time, money, etc. However, optimizing a query in a cloud federation is complex with increasing the variety, especially due to a wide range of communications and pricing models. The variety of virtual machines configuration also leverages the high complexity in generating the space of candidate solutions. Indeed, in such a context, it is difficult to provide accurate estimations and optimal solutions to make relevant decisions. The first challenge is how to estimate accurate parameter values for MOOPs in a cloud federation consisting of different sites. To address the accurate estimation of parameter values problem, we present the Dynamic Regression Algorithm (DREAM). DREAM focuses on reducing the size of historical data while maintaining the estimation accuracy. The second challenge is how to find an approximate optimal solution in MOOPs using an efficient Multi-Objective Optimization algorithm. To address the problem of finding an approximate optimal solution, we present Non-dominated Sorting Genetic Algorithms based on Grid partitioning (NSGA-G) for MOOPs. The proposed algorithm is integrated into the Intelligent Resource Scheduler, a solution for heterogeneous databases, to solve MOOP in cloud federations. We validate our algorithms with experiments on a decision support benchmark.

Introduction

Cloud federation is a paradigm of interconnecting the cloud environments of more than one service providers for multiple purposes of commercial, service quality, and user's requirements [START_REF] Kurze | Cloud federation[END_REF]. Besides of vendor lock-in and provider integration, a cloud federation has several types of heterogeneity and variability in the cloud environment, such as wide-range communications and pricing models.

Cloud federations can be seen as a major progress in cloud computing, in particular for the medical domain. Indeed, sharing medical data would improve healthcare. Federating resources makes it possible to access any information even on distributed hospital data on several sites. Besides, it enables to access larger volumes of data on more patients and thus provide finer statistics.

For example, patient A has just come back from a tropical country B. He has a rare disease from there. The hospital cannot recognize his disease. The clinic in country B records some cases like his. However, the two databases of the hospital and the clinic are not in the same database engine, or cloud provider. If a cloud federation exists to interconnect the two clouds, his disease could be recognized and he can have a treatment soon.

In cloud federations, pay-as-you-go models and elasticity thus raise an important issue in terms of Multi-Objective Optimization Problems (MOOPs) according to users preferences, such as time, money, quality, etc. However, MOOPs in a cloud federation are hard to solve due to issues of heterogeneity, and variability of the cloud environment, and high complexity in generating the space of candidate solutions.

Let's consider a query Q in a example below.

Example 1. A query Q in the medical domain, based on TPC-H3 query 3 and 4:

SELECT p . UID , p . PatientID , s . PatientName , p . PatientBrithDate , p . PatientSex , p . EthnicGroup , p . SmokingStatus , s . PatientAge , s . PatientWeight , s . PatientSize , i . GeneralName , i . GeneralValues , q . UID , q . SequenceTags , q . SequenceVRs , q . SequenceNames , q . SequenceValues FROM Patient p , GeneralInfoTable i , Study s , S equ en ce At tr ib ut es q WHERE p . UID = s . UID AND p . UID = i . UID AND p . UID = q . UID AND p . PatientSex = 'M ' AND p . SmokingStatus = ' NO ' AND s . PatientAge >= x AND q . SequenceNames LIKE '%X -ray % ' This query is transformed into a logical query plan using logical operation, such as select, project, join, etc. Depending on the physical operators, a query optimizer generates a query execution plan to execute a logical plan. Actually, various Query Execution Plans (QEPs) are generated with respect to the number of nodes, their capacity in terms of CPU, memory and disk and the pricing model. Table 1 presents an example of possible QEPs for Q. Choosing an execution plan is a trade-off between objectives such as the response time or the monetary cost, and depends on users' preferences: a user A may prefer minimizing his budget (QEP1); a user B may want the lowest response time (QEP2); a user C may look for a trade-off between time and money (QEP3).

Assuming that the query is processed on Amazon EC2. The master consists of a m2.4xlarge instance (8 virtual cores with 68.4 GB of RAM). Workers consist of m3.2xlarge instances (8 virtual cores and with 30 GB of RAM). If the pool of resources is 70 VCPU with 260GB of memory, the number of QEPs is thus 70 x 260 = 18,200. The problem is then how to search and optimize such a query in a real environment, when the pool of resources is more variable, with respect to multiple dimensions (response time, monetary cost, etc.). Since generating QEPs maybe infeasible due to high complexity, we aim to find an approximate optimal solution.

In this paper, we address several challenges for the development of medical data management in cloud federations. The first challenge is how to estimate accurate parameter values for MOOPs without precise knowledge of the execution environment in a cloud federation consisting of different sites. The execution environment may consist of various hardware and systems. In addition, it also depends on the variety of physical machines, load evolution and wide-range communications. As a consequence, the estimation process is complex. The second challenge is how to find an approximate optimal solution in MOOPs using an efficient Multi-Objective Optimization algorithm. Indeed, MOOPs could be solved by Multi-Objective Optimization algorithms or the Weighted Sum Model (WSM) [START_REF] Helff | Weighted Sum Model for Multi-Objective Query Optimization for Mobile-Cloud Database Environments[END_REF] or be converted to a Single-Objective Optimization Problem (SOOP). However, SOOPs cannot adequately represent MOOPs [START_REF] Glaßer | Approximability and Hardness in Multi-objective Optimization[END_REF]. Also, MOOPs leads to find solutions by Pareto dominance techniques. Since generating a Pareto-optimal front is often infeasible due to high complexity [START_REF] Zitzler | Performance assessment of multiobjective optimizers: an analysis and review[END_REF], MOOPs need an approximate optimal solution calculated by Pareto dominance techniques.

The estimation process can be classified into two classes: without [START_REF] Nykiel | MRShare: Sharing Across Multiple Queries in MapReduce[END_REF][START_REF] Sidhanta | OptEx: A Deadline-Aware Cost Optimization Model for Spark[END_REF][START_REF] Wu | Predicting query execution time: Are optimizer cost models really unusable?[END_REF]] and with machine learning algorithms [START_REF] Doka | IReS: Intelligent, Multi-Engine Resource Scheduler for Big Data Analytics Workflows[END_REF]. In a cloud federation with variability and different systems, cost functions may be quite complex. In the first class, cost models introduced to build an optimal group of queries [START_REF] Nykiel | MRShare: Sharing Across Multiple Queries in MapReduce[END_REF] are limited to MapReduce [START_REF] Dean | MapReduce: simplified data processing on large clusters[END_REF]. Besides, a PostgreSQL cost model [START_REF] Wu | Predicting query execution time: Are optimizer cost models really unusable?[END_REF] aims to predict query execution time for this specific relational DBMS. Moreover, OptEx [START_REF] Sidhanta | OptEx: A Deadline-Aware Cost Optimization Model for Spark[END_REF] provides estimated job completion times for Spark 4 with respect to the size of the input dataset, the number of iterations, the number of nodes composing the underlying cloud. These works mention the estimation of only execution time for a job, and not for other metrics, such as monetary cost. Meanwhile, various machine learning techniques are applied to estimate execution time in recent research [START_REF] Akdere | Learning-based Query Performance Modeling and Prediction[END_REF][START_REF] Ganapathi | Predicting Multiple Metrics for Queries: Better Decisions Enabled by Machine Learning[END_REF][START_REF] Tozer | Q-Cop: Avoiding bad query mixes to minimize client timeouts under heavy loads[END_REF][START_REF] Xiong | ActiveSLA: A profit-oriented admission control framework for Database-as-a-Service providers[END_REF]. They predict the execution time by many machine learning algorithms. They treat the database system as a black box and try to learn a query running time prediction model using the total information for training and testing in the model building process. It may lead to the use of expired information. In addition, most of these solutions solve the optimization problem with a scalar cost value and do not consider multi-objective problems.

A well known Pareto dominance technique to solve the high complexity of MOOP is Evolutionary Multiobjective Optimization (EMO). Among EMO approaches, Non-dominated Sorting Genetic Algorithms (NSGAs) [START_REF] Deb | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints[END_REF][START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF] have lower computational complexity than other EMO approaches [START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF]. However, these algorithms still have high computational complexity. We presented Nondominated Sorting Genetic Algorithm based on Grid partitioning (NSGA-G) [START_REF] Le | An efficient multi-objective genetic algorithm for cloud computing: NSGA-G[END_REF] to improve both computation time and qualities of NSGAs. It has more advantages than other NSGAs. Two versions of NSGA-G will be shown to compromise computation and quality.

In this paper, we introduce a medical system on a cloud federation called Medical Data Management System (MIDAS). It is based on the Intelligent Resource Scheduler (IReS) [START_REF] Doka | IReS: Intelligent, Multi-Engine Resource Scheduler for Big Data Analytics Workflows[END_REF], an open source platform for complex analytics workflows executed over multi-engine environments. In particular, we focus on:

(1) a dynamic estimation and (2) a Non-dominated Sorting Genetic Algorithm for Multi-Objective Optimization Problems. The first contribution is Dynamic linear REgression AlgorithM (DREAM) to provide accurate estimation with low computational cost. DREAM is then implemented and validated with experiments on a decision support benchmark (TPC-H benchmark). The second contribution is Non-dominated Sorting Genetic Algorithm based on Grid partitioning (NSGA-G) to improve both quality and computational efficiency of NSGAs, and also provides an alternative for Pareto-optimal of MOOPs. NSGA-Gs are validated through experiments on DTLZ problems [START_REF] Deb | Scalable Test Problems for Evolutionary Multiobjective Optimization, Evolutionary Multiobjective Optimization[END_REF] and compared with NSGA-II [START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF], NSGA-III [START_REF] Deb | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints[END_REF], and the others in Generational Distance [START_REF] Veldhuizen | Evolutionary Computation and Convergence to a Pareto Front[END_REF], Inverted Generational Distance [START_REF] Coello | Solving Multiobjective Optimization Problems Using an Artificial Immune System[END_REF], and Maximum Pareto Front Error [START_REF] Veldhuizen | Multiobjective evolutionary algorithms: classifications, analyses, and new innovations[END_REF] statistic.

This paper is an extended version of [START_REF] Le | An efficient multi-objective genetic algorithm for cloud computing: NSGA-G[END_REF][START_REF] Le | Dynamic Estimation for Medical Data Management in a Cloud Federation[END_REF]. In particular, they are grouped together to become a system, MIDAS. Besides, the theory of NSGA-G in [START_REF] Le | An efficient multi-objective genetic algorithm for cloud computing: NSGA-G[END_REF] is expanded to two versions: NSGA-G using Min point and using Random metric. The remaining of this paper is organized as follows. Section 2 presents the research background. DREAM is presented in Section 3. Section 4 shows NSGA-G. Section 5 presents experiments to validate DREAM and NSGA-Gs. Finally, Section 6 concludes this paper and lists some perspectives.

Background

In this section, we introduce an architecture of the system, concepts and techniques, allowing us to implement the proposed medical data management on a cloud federation. First of all, an overview of the Medical Data Management System (MIDAS) and the benefits of cloud federation where our system is built on are introduced. After that, an open source platform, which helps our system managing and executing workflows over multi-engine environments is described. The concept of Pareto plan set related to Multi-Objective Optimization Problem (MOOP) in MIDAS is then defined. In addition, Multiple Linear Regression and Non-dominated Sorting Genetic Algorithm are also introduced as the basic foundation of our proposed algorithms for MOOP.

Cloud federation

This section shows the definition and the example related to a cloud federation. A cloud federation enables to interconnect different cloud computing environments. Cloud computing [START_REF] Armbrust | A view of cloud computing[END_REF] allows to access on demand and configurable resources, which can be quickly made available with minimal maintenance. According to the pay-as-you-go pricing model, customers only pay for resources (storage and computing) that they use. Cloud Service Providers (CSP) supply a pool of resources, development platforms and services. There are many CSPs on the market, such as Amazon, Google and Microsoft, etc., with different services and pricing models. For example, Table 2 shows the pricing of instances in two cloud providers in 2019. The price of Amazon instances are lower than the price of Microsoft instances, but the price of Amazon is without storage. Hence, depending on the demand of a query, the monetary cost is low or high at a specific provider.

In the medical domain, cloud federation may lead to query data across different clouds. A demand query running in that cloud federation could be concerned about the price of time and money of the execution query. It is a Multi-Objective Optimization Problem (MOOP). For example, federating resources makes it possible to access any information on a person with distributed hospital data on various sites. Various big data management system could be used to manage the medical data, which has the 3Vs characteristics of Big Data [START_REF] Abadi | The Beckman report on database research[END_REF]: high volume, high variety, and high velocity. The data also stores that belong in different clouds are shown in Fig. 1. This example shows that the data can be stored in three different clouds, such as Amazon Web Services, Microsoft Azure, Google Cloud Platform. Pay-as-you-go models in clouds lead to solving Multi-Objective Optimization Problem to find a Query Execution Plan (QEP) according to users preferences, such as time, money, quality, etc. MOOPs often use Pareto dominance techniques in finding an optimal solution. In particular, in a query processing problem, let a query q be an information request from databases, presented by a set of tables. A Query Execution Plan (QEP), denoted by p, is the evaluation of a query that can be passed to the executor. The set of QEPs is denoted by symbol P. The set of operators is denoted by O. A QEP, p, can be divided into two sub-plans p 1 and p 2 if p is the result of function Combine(p 1 , p 2 , o), where o ∈ O.

The execution cost of a QEP depends on parameters, which values are not known at the optimization time. A vector x denotes parameters value and the parameter space X is the set of all possible parameter vectors x. N is denoted as the set of n cost metrics. We can compare QEPs according to n cost metrics which are processed with respect to the parameter vector x and cost functions c n (p, x). Let denote C as the set of cost function c.

Let p 1 , p 2 ∈ P, p 1 dominates p 2 if the cost values according to each cost metric of plan p 1 is less than or equal to the corresponding values of plan p 2 in all the space of parameter X . That is to say:

C(p 1 , X ) C(p 2 , X ) | ∀n ∈ N, ∀x ∈ X : c n (p 1 , x) ≤ c n (p 2 , x). (1)
The function Dom(p 1 , p 2 ) ⊆ X yields the parameter space region where p 1 dominates p 2 [START_REF] Trummer | Multi-objective parametric query optimization[END_REF]:

Dom(p 1 , p 2 ) = {x ∈ X | ∀n ∈ N : c n (p 1 , x) ≤ c n (p 2 , x)}. (2)
Assume that in the area x ∈ A, A ⊆ X , p 1 dominates p 2 , C(p 1 , A) C(p 2 , A), Dom(p 1 , p 2 ) = A ⊆ X . p 1 strictly dominates p 2 if all values for the cost functions of p 1 are less than the corresponding values for p 2 [START_REF] Trummer | Multi-objective parametric query optimization[END_REF], i.e.

StriDom(p 1 , p 2 ) = {x ∈ X | ∀n ∈ N : c n (p 1 , x) < c n (p 2 , x)}. (3)
A Pareto region of a plan is a space of parameters where there is no alternative plan has lower cost than it [START_REF] Trummer | Multi-objective parametric query optimization[END_REF]:

P aReg(p) = X \ ( p * ∈P StriDom(p * , p)). (4)

IReS

Cloud federation model needs to integrate cloud services from multiple cloud providers. It raises an important issue in terms of heterogeneous database engines in various clouds. Among various heterogeneous database system described in Table 3, IReS platform considers both heterogeneous systems and MOOP in clouds.

Intelligent Multi-Engine Resource Scheduler (IReS) [START_REF] Doka | IReS: Intelligent, Multi-Engine Resource Scheduler for Big Data Analytics Workflows[END_REF] is an open source platform for managing, executing and monitoring complex analytics workflows. IReS provides a method of optimizing cost-based workflows and customizable resource management of diverse execution and various storage engines. Especially, IReS platform helps us to organize data in the multiple clouds as a cloud federation. Interface is the first module which is designed to receive information on data and operators, as shown in Fig. 4. The second module is Modelling, as shown in Fig. 4, is used to predict the execution time by a model chosen by comparing machine learning algorithms. For example, Least squared regression [START_REF] Rousseeuw | Robust Regression and Outlier Detection[END_REF], Bagging predictors [START_REF] Breiman | Bagging predictors[END_REF], Multilayer Perceptron in WEKA framework 5 are used to build the cost model in Modelling module. The module

Research

Heterogeneous MOOP Proteus [START_REF] Karpathiotakis | Fast queries over heterogeneous data through engine customization[END_REF] × Polystore Query rewriting [START_REF] Papakonstantinou | Polystore Query Rewriting: The Challenges of Variety[END_REF] × BigDAWG Polystore System [START_REF] Elmore | A demonstration of the BigDAWG polystore system[END_REF] × ClooudMdsQL [START_REF] Kolev | CloudMdsQL: querying heterogeneous cloud data stores with a common language[END_REF][START_REF] Kolev | The CloudMdsQL Multistore System[END_REF] × MuSQLE [START_REF] Giannakouris | MuSQLE: Distributed SQL query execution over multiple engine environments[END_REF] × MISO [START_REF] Lefevre | MISO: souping up big data query processing with a multistore system[END_REF] × Polybase [START_REF] Dewitt | Split query processing in polybase[END_REF] × Estoscada [START_REF] Bugiotti | Invisible Glue: Scalable Self-Tuning Multi-Stores[END_REF] × IReS tests many algorithms and the best model with the smallest error is selected. It guarantees the predicted values as the best one for estimating process. Next module, Multi-Objective Optimizer, optimizes Multi-Objective Query Processing (MOQP) and generates a Pareto QEP set. In Multi-Objective problem, the objectives are the cost functions user concerned, such as the execution time, monetary, intermediate data, etc. Multi-Objective Optimization algorithms can be applied to the Multi-Objective Optimizer. For instance, the algorithms based on Pareto dominance techniques [START_REF] Coello | Evolutionary algorithms for solving multi-objective problems, Second Edition[END_REF][START_REF] Deb | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints[END_REF][START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF][START_REF] Jain | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point Based Nondominated Sorting Approach, Part II: Handling Constraints and Extending to an Adaptive Approach[END_REF][START_REF] Knowles | The Pareto archived evolution strategy: a new baseline algorithm for Pareto multiobjective optimisation[END_REF][START_REF] Le | An efficient multi-objective genetic algorithm for cloud computing: NSGA-G[END_REF][START_REF] Srinivas | Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms[END_REF][START_REF] Zhang | MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition[END_REF][START_REF] Zitzler | SPEA2: Improving the Strength Pareto Evolutionary Algorithm[END_REF] are solutions for Multi-objective Optimization problems. Finally, the system selects the best QEP based on user query policy and Pareto set. The final QEP is run on multiple engines, as shown in Fig. 4.

Multiple Linear Regression

In many database management systems, predicting cost values is useful in optimization process [START_REF] Wu | Predicting query execution time: Are optimizer cost models really unusable?[END_REF]. Recent researches have been exploring the statistical machine learning approaches to build predictive models for this task. They often use historical data to train and test the cost model as a Single-Objective Problem (SOP). Besides, Linear Regression is an useful class of models in science and engineering [START_REF] Soong | Fundamentals of probability and statistics for engineers[END_REF]. In this section, we describe the background of this model.

This model is used in the situation in which a cost value, c, is a function of one or more independent variables x 1 , x 2 , ..., and x L . For example, execution time c is a function of data size x 1 of first element in join operator and data size x 2 of second element in that join operator.

Given a sample of c values with their associated values of x i , i = 1, 2, ..., L. We focus in the estimation the relationship between c and the independent variables x 1 , x 2 , ..., and x L based on this sample. Cost function c of Multiple Linear Regression (MLR) model [START_REF] Soong | Fundamentals of probability and statistics for engineers[END_REF] is defined as follows:

c = β 0 + β 1 x 1 + ... + β L x L + , (5)
where β l , l = 0, ..., L, are unknown coefficients, x l , l = 1, ..., L, are the independent variables, e.g., size of data, computer configuration, etc., c is cost function values and is random error following normal distribution N (0, σ 2 ) with zero mean and variance σ 2 . The fitted equation is defined by: where ĉti , ĉmo are execution time and monetary cost function; x P a , x Ge are the size of Patient and GeneralInfo tables, respectively, and x nodeA , x nodeB are the number of virtual machines created to run query Q.

ĉ = β0 + β1 x 1 + ... + βL x L . (6
There are M historical data, each of them associates with a response c m , which can be predicted by a fitted value ĉm calculated from corresponding x lm as follows: ĉm = β0 + β1 x 1m + ... + βL x Lm ; m = 1, ..., M.

Let denote

A =       1 x 11 x 21 ... x L1 1 x 12 x 22 ... x L2 . . . . . . . . . . 1 x 1M x 2M ... x LM       , (8)
C =       c 1 c 2 . . c M       , (9)
B =       β0 β1 . . βL       . ( 10
)
Non-dominated Sorting Filter Front

P t Q t F 1 F 2 F 3 F 4 F 3 P t+1 R t
Fig. 2: NSGA-II and NSGA-III procedure [START_REF] Deb | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints[END_REF][START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF].

To minimize the Sum Square Error (SSE), defined by:

SSE = M m=1 (c m -ĉm ) 2 , (11)
the solution for B is retrieved by:

B = (A T A) -1 A T C. (12)

NSGA

After having the prediction cost values of MOOPs, we need to use Multi-Objective Optimization algorithms to find an optimal solution. Among Multi-objective Optimization algorithm classes, Evolutionary Multiobjective Optimization (EMO) shows their advantages in searching and optimizing for the MOOPs [START_REF] Coello | Evolutionary algorithms for solving multi-objective problems, Second Edition[END_REF]. Among EMO approaches, Non-dominated Sorting Genetic Algorithms provide low computational complexity of non-dominated sorting, O(M N 2 ) of NSGAs [START_REF] Deb | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints[END_REF][START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF] comparing to O(M N 3 ) of other Evolutionary Multi-Objective Optimization (EMO), where M is the number of objectives and N is the population size.

NSGA process Initially, NSGAs start with a population P 0 consisting of N solutions. In MOOPs, a population represents a set of candidate solutions. The size of P 0 is smaller than the number of all candidate solutions. Each solution is on a specific rank or non-domination level (any solution in level 1 is not dominated, any solution in level 2 is dominated by one or more solutions in level 1 and so on). At first, the offspring population Q 0 containing N solutions, is created by the binary tournament selection and mutation operators [START_REF] Deb | Simulated Binary Crossover for Continuous Search Space[END_REF]. Where the binary tournament selection is a method of selecting an individual from a population of individuals in a genetic algorithm, and the mutation operation is a method to choose a neighboring individual in the locality of the current individual. Secondly, a population R 0 = P 0 ∪ Q 0 with the size of 2N will be divided into subpopulations based on the order of Pareto dominance. The appropriate N members from R 0 will be chosen for the next generation. The non-dominated sorting based on usual domination principle [START_REF] Chankong | Multiobjective decision making: theory and methodology[END_REF] is first used, which classifies R 0 into different non-domination levels (F 1 , F 2 and so on). After that, a parent population of next-generation P 1 is selected in R 0 from level 1 to level k so that the size of P 1 = N and so on.

The difference among NSGA-II, NSGA-III and other NSGAs is the way to select members in the last level F l . To keep the diversity, NSGA-II [START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF] and SPEA-II [START_REF] Zitzler | SPEA2: Improving the Strength Pareto Evolutionary Algorithm[END_REF] use crowding distance among solutions in their selection. NSGA-II procedure is not suitable for MOO problems and the crowding distance operator needs to be replaced for better performance [START_REF] Ishibuchi | Sensitivity of performance evaluation results by inverted generational distance to reference points[END_REF][START_REF] Köppen | Substitute Distance Assignments in NSGA-II for Handling Many-objective Optimization Problems[END_REF]. Hence, when the population has a high-density area, higher than others, NSGA-II prefers the solution which is located in a less crowded region.

On the other hand, MOEA/D [START_REF] Zhang | MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition[END_REF] decomposes a multiple objectives problem into various scalar optimization subproblems. The diversity of solutions depends on the scalar objectives. However, the number of neighborhoods needs to be declared before running the algorithm. In addition, the estimation of good neighborhood is not mentioned. The diversity is considered as the selected solution associated with these different sub-problems. Experimental results in [START_REF] Deb | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints[END_REF] show various versions of MOEA/D approaches which fail to maintain a good distribution of points.

An Evolutionary Many-Objective Optimization Algorithm Using Referencepoint Based Non-Dominated Sorting Approach [START_REF] Deb | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints[END_REF] (NSGA-III) uses different directions to maintain the diversity of solutions. NSGA-III replaces the crowding distance operator by comparing solutions. Each solution is associated to a reference point [START_REF] Deb | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints[END_REF], which impacts the execution time to built the reference points in each generation. The diversity of NSGA-III is better than the others, but the execution time is very high. For instance, with two objectives and two divisions, three reference points will be created, (0.0,1.0), (1.0,0.0) and (0.5,0.5), as shown in Fig. 3. After selection process, the diversity of population is better than NSGA-II with solutions close to three reference points. However, comparing all solutions to each reference point makes the computation time of NSGA-III very high. In addition, NSGAs often compare all solutions to choose good solutions in F l . Therefore, when the number of solutions or objectives is significant, the time for calculating and comparing is considerable.

Application In some cases, some objectives are homogeneous. In the reason of the homogeneity between the multi-objectives functions, removing an objective do not affect to the final results of MOO problem. In other cases, the objectives may be contradictory. For example, the monetary is proportional to the execution time in the same virtual machine configuration in a cloud. However, cloud providers usually leases computing resources that are typically charged based on a per time quantum pricing scheme [START_REF] Kllapi | Schedule optimization for data processing flows on the cloud[END_REF]. The solutions represent the tradeoffs between time and money. Hence, the execution time and the monetary cost cannot be homogeneous.

As a consequence, the multi-objective problem cannot be reduced to a monoobjective problem. Moreover, if we want to reduce the MOO to a mono-objective optimization, we should have a policy to group all objectives by the Weighted Sum Model (WSM) [START_REF] Helff | Weighted Sum Model for Multi-Objective Query Optimization for Mobile-Cloud Database Environments[END_REF]. However, estimating the weights corresponding to different objectives in this model is also a multi-objective problem.

In addition, MOO problems could be solved by MOO algorithms or WSM [START_REF] Helff | Weighted Sum Model for Multi-Objective Query Optimization for Mobile-Cloud Database Environments[END_REF]. However, MOO algorithms are selected thanks to their advantages when comparing with WSM. The optimal solution of WSM could be unacceptable, because of an inappropriate setting of the coefficients [START_REF] Fonseca | An Overview of Evolutionary Algorithms in Multiobjective Optimization[END_REF]. Furthermore, the research in [START_REF] Khan | Iterative non-deterministic algorithms in on-shore wind farm design: A brief survey[END_REF] proves that a small change in weights may result in significant changes in the objective vectors and significantly different weights may produce nearly similar objective vectors. Moreover, if WSM changes, a new optimization process will be required.

In conclusion, MOOP approaches leads to using Pareto dominance techniques. A pareto-optimal front is often infeasible [START_REF] Zitzler | Performance assessment of multiobjective optimizers: an analysis and review[END_REF]. NSGAs show the advantage in searching a Pareto solution for MOOP in less computational complexity than other EMO [START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF]. However, they should be improved the quality to solve MOOP when the number of objectives is significant.

Motivation

In the context of medical data management, the background of concepts and techniques related to cloud federations, we introduce a medical system on a cloud federation called Medical Data Management System (MIDAS). It is based on the Intelligent Resource Scheduler (IReS) [START_REF] Doka | IReS: Intelligent, Multi-Engine Resource Scheduler for Big Data Analytics Workflows[END_REF], an open source platform for complex analytics work-flows executed over multi-engine environments.

MIDAS It is a medical data management system for cloud federation. The proposal aims to provide query processing strategies to integrate existing information systems (with their associated cloud provider and data management system) for clinics and hospitals. Fig. 4 presents an overview of the system. Integrating the system within a cloud federation allows to choose the best strategy for MOQP. MIDAS can be developed based on the platform which can execute over multi-engine environments on clouds. Fig. 4 also shows an example of MIDAS, where three database engines are installed and run in three clouds of different providers.

We choose IReS platform to consider the advantage as shown in Table 3. IReS platform is installed in every machine in MIDAS. The different cloud resource pools allow the system to run in the most appropriate infrastructure environments. The system can optimize workflows between different data sources on different clouds, such as Amazon Web Services8 , Microsoft Azure9 and Google Cloud Platform10 . The proposed system is developed based on the Intelligent Resource Scheduler (IReS) for complex analytics workflows executed over multiengine environments on a cloud federation.

Machine learning algorithm

The machine learning algorithms in IReS need entire training datasets to estimate the running costs, which are calculated by determining the cost of processing a job. It may lead to use expired information. Hence, the proposal algorithm aims to improve the accuracy of estimated values with low computational cost. Our proposed method is integrated into IReS to predict the cost values with low computational cost in MOQP of a cloud environment.

Multi-Objective Optimization In addition, MOQP could be solved by Multi-Objective Optimization algorithms or the Weighted Sum Model (WSM) [START_REF] Helff | Weighted Sum Model for Multi-Objective Query Optimization for Mobile-Cloud Database Environments[END_REF]. However, Multi-Objective Optimization algorithms may be selected thanks to their advantages when comparing with WSM. The optimal solution of WSM could be not acceptable, because of an inappropriate setting of the coefficients [START_REF] Fonseca | An Overview of Evolutionary Algorithms in Multiobjective Optimization[END_REF]. Furthermore, the research in [START_REF] Khan | Iterative non-deterministic algorithms in on-shore wind farm design: A brief survey[END_REF] proves that a small change in weights may result in significant changes in the objective vectors and significantly different weights may produce nearly similar objective vectors. Moreover, if WSM changes, a new optimization process will be required. Hence, our system applies a Multi-Objective Optimization algorithm to the Multi-Objective Optimizer to find a Pareto-optimal solution. When the WSM changes, the final result just is determined by using the Pareto-optimal set at the final step. Furthermore, generating a Pareto-optimal front is often infeasible due to high complexity [START_REF] Zitzler | Performance assessment of multiobjective optimizers: an analysis and review[END_REF]. MOOPs leads to finding an approximate optimal solution by Pareto dominance techniques. A well known approach to solve the high complexity of MOOP is Evolutionary Multiobjective Optimization (EMO). Among EMO approaches, Non-dominated Sorting Genetic Algorithms (NSGAs) [START_REF] Deb | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints[END_REF][START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF] have lower computational complexity than other EMO approaches [START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF]. However, this algorithm still has high computational complexity. We need to find an approach to improve the computational complexity and quality of NSGAs.

In conclusion, our solutions aim to improve the accuracy of cost value prediction with low computational cost and to solve MOQP by Multi-Objective Optimization algorithm in a cloud federation environment. Besides, we also find a method to search and optimize MOOPs by finding an approximate optimal solution in the high complexity of generating a Pareto-optimal front.

Dynamic Regression Algorithm

The first technique in MIDAS relates to the estimation of accurate cost values in the variable environment of a cloud federation. Most of cost models [START_REF] Fard | A Multi-objective Approach for Workflow Scheduling in Heterogeneous Environments[END_REF][START_REF] Nykiel | MRShare: Sharing Across Multiple Queries in MapReduce[END_REF][START_REF] Wu | Predicting query execution time: Are optimizer cost models really unusable?[END_REF] depend on the size of data. Hence, our cost functions are functions of the size of data. In particular, cost function and fitted value of Multiple Linear Regression model are previously defined in Section 2.4. The bigger M for sets {c m , x lm } is, the more accurate MLR model usually is. However, the computer is slowing down when M is too big.

Furthermore, the target of Multi-Objective Query Processing is the Multi-Objective Optimization Problem [START_REF] Zhang | MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition[END_REF], which is defined by:

minimize(F (x) = (f 1 (x), f 2 (x)..., f K (x)) T ), (13)
where x = (x 1 , ..., x L ) T ∈ Ω ⊆ R L is an L-dimensional vector of decision variables, Ω is the decision (variable) space and F is the objective vector function, which contains K real value functions.

In general, it is hard to find a point in Ω that minimizes all the objectives together. Pareto optimality is defined by trade-offs among the objectives. If there is no point x ∈ Ω such that F (x) dominates F (x * ), x * ∈ Ω, x * is called Pareto optimal and F (x * ) is called a Pareto optimal vector. Set of all Pareto optimal points is the Pareto set. A Pareto front is a set of all Pareto optimal objective vectors. Generating the Pareto-optimal front can be computationally expensive [START_REF] Zitzler | Performance assessment of multiobjective optimizers: an analysis and review[END_REF]. In cloud environment, the number of equivalent query execution plans is multiplied.

Example 3. Assuming that a query is processed on Amazon EC2. If the pool of resources includes 70 vCPUs and 260GB of memory, we assume that a configuration to execute a query plan is created by the number of vCPUs and the size of memory which is the multiple of 1GB. In particular, a configuration can be 01 vCPU and 260GB of memory and the others is 70 vCPUs and 01GB of memory. Hence, the combination of different configurations to execute this query would be 70*260=18,200.

Example 3 shows that a query plan can generate multiple equivalent QEPs in cloud environment. The smaller M for sets {c m , x lm } is, the faster speed for the estimation cost process of Multi-Objective Query Processing for a QEP is. In the system of computationally expensiveness in cloud environment as in Example 3, a small reduction of computation for an equivalent QEP estimation will become significant for a large number of equivalent QEPs estimation.

The most important idea is to estimate MLR quality by using the coefficient of determination. The coefficient of determination [START_REF] Soong | Fundamentals of probability and statistics for engineers[END_REF] is defined by:

R 2 = 1 -SSE/SST, ( 14
)
where SSE is the sum of squared errors and SST represents the amount of total variation corresponding to the predictor variable X. Hence, R 2 shows the proportion of variation in cost given by the Multiple Linear Regression model of variable X. For example, the model gives R 2 = 0.75 of time response cost, it can be concluded that 3/4 of the variation in time response values can be explained by the linear relationship between the input variables and time response cost. Table 4 presents an example of MLR with different number of measures. The smallest dataset is M = L + 2 = 4 [START_REF] Soong | Fundamentals of probability and statistics for engineers[END_REF], where M is the size of previous data and L is the number of variables in Equation [START_REF] Breiman | Bagging predictors[END_REF]. In general, R 2 increases in parallel with M . In particular, R 2 should be greater than 0.8 to provide a sufficient quality of service level. As a consequence, M should be greater than 5 to provide enough accuracy. Hence, when the system requires the minimum values of R 2 is equal to 0.8, M > 6 is not recommended. In general, R 2 still rises up when M goes up. Therefore, we need to determine the model which is sufficient suitable by the coefficient of determination.

Training set DREAM coefficient of determination

New training set Modelling Fig. 5: DREAM module [START_REF] Le | Dynamic Estimation for Medical Data Management in a Cloud Federation[END_REF].

Our motivation is to provide accurate estimation while reducing the number of previous measures based on R 2 . We thus propose DREAM as a solution for cloud federation and their inherent variance, as shown in Fig. 5. DREAM uses the training set to test the size of new training dataset. It depends on the predefined coefficient of determination. The new training set is generated in order to have the updated value and avoid using the expired information. With the new training set, Modelling uses fewer data in the building model process than the original approach does.

Cost modeling without machine learning [START_REF] Nykiel | MRShare: Sharing Across Multiple Queries in MapReduce[END_REF][START_REF] Sidhanta | OptEx: A Deadline-Aware Cost Optimization Model for Spark[END_REF][START_REF] Wu | Predicting query execution time: Are optimizer cost models really unusable?[END_REF] often uses the size of data to estimate the execution time for the specific system. Besides, the machine learning approach [START_REF] Doka | IReS: Intelligent, Multi-Engine Resource Scheduler for Big Data Analytics Workflows[END_REF] can use any information to estimate the cost value. Hence, our algorithm uses the size of data as variables of DREAM. In [START_REF] Bugiotti | Invisible Glue: Scalable Self-Tuning Multi-Stores[END_REF], ĉ is the cost value, which needs to be estimated in MOQP, and x 1 , x 2 , . . . are the information of system, such as size of input data, the number of nodes, the type of virtual machines. If R 2 ≥ R 2 require , where R 2 requires is predefined by users, the Algorithm 1 Calculate the predict value of multi-cost function [START_REF] Le | Dynamic Estimation for Medical Data Management in a Cloud Federation[END_REF] 1: function EstimateCostValue(R 2 require , X, Mmax) 2:

for n = 1 to N do 3:

R return ĉN (p) 14: end function model is reliable. In contrast, it is necessary to increase the number of set value. Algorithm 1 shows a scheme as an example of increasing value set: m = m + 1.

In this paper, we focus on the accuracy of execution time estimation with the low computational cost in MOQP. The original optimization approach in IReS uses Weighted Sum Model (WSM) [START_REF] Helff | Weighted Sum Model for Multi-Objective Query Optimization for Mobile-Cloud Database Environments[END_REF] with user policy to find the best candidate solution. However, the optimal solution of WSM could be not acceptable, because of an inappropriate setting of the coefficients [START_REF] Fonseca | An Overview of Evolutionary Algorithms in Multiobjective Optimization[END_REF]. Besides, Multi-Objective Optimization algorithms have more advantages than WSM [START_REF] Fonseca | An Overview of Evolutionary Algorithms in Multiobjective Optimization[END_REF][START_REF] Khan | Iterative non-deterministic algorithms in on-shore wind farm design: A brief survey[END_REF]. They lead to find solutions by Pareto dominance techniques. However, generating a Pareto-optimal front is often infeasible due to high complexity [START_REF] Zitzler | Performance assessment of multiobjective optimizers: an analysis and review[END_REF]. One of well known Multi-Objective Optimization algorithm class is Non-dominated Sorting Algorithms (NSGAs). Hence, after having a set of predicted cost function values for each query plan, a Multi-Objective Optimization algorithm, such as NSGA-G [START_REF] Le | An efficient multi-objective genetic algorithm for cloud computing: NSGA-G[END_REF] is applied to determine a Pareto query execution plan set. At the final step, the weight sum model S and the constraint B associated with the user policy are used to return the best QEP for the given query [START_REF] Helff | Weighted Sum Model for Multi-Objective Query Optimization for Mobile-Cloud Database Environments[END_REF]. In particular, the most meaningful plan will be selected by comparing function values with weight parameters between ĉn [START_REF] Helff | Weighted Sum Model for Multi-Objective Query Optimization for Mobile-Cloud Database Environments[END_REF] at the final step, as shown in Algorithm 2. Fig. 6 shows the different between two MOQP approaches. Our algorithms are developed based on the MLR described above using x i for size of data and c i for the metric cost, such as the execution time, energy consumption, etc.

Non-dominated Sorting Genetic Algorithm based on Grid partitioning

After having the prediction cost values of MOOPs by DREAM, we need to use Multi-Objective Optimization algorithms to find an optimal solution. Hence, the second technique relates to looking for an efficient approach for searching and optimizing in MIDAS is introduced in this section. NSGAs [START_REF] Deb | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints[END_REF][START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF] end if 8: end function known approaches to optimize MOOPs. Our previous work [START_REF] Le | An efficient multi-objective genetic algorithm for cloud computing: NSGA-G[END_REF] proposed Nondominated Sorting Genetic Algorithm based on Grid partitioning (NSGA-G) to improve both diversity and convergence of NSGAs while having an efficient computation time by reducing the space of selected good solutions in the truncating process. NSGA-G is an algorithm based on genetic algorithms (GAs). The convergence of GAs is discussed in [START_REF] Cerf | Asymptotic convergence of genetic algorithms[END_REF]. The difference between many GAs is the qualities of diversity and convergence. We will describe the strategy to improve the qualities of NSGAs while having an efficient computation time as below.

At the t th generation of Non-dominated Sorting Genetic Algorithms, P t represents the parent population with N size and Q t is offspring population with N members created by P t . R t = P t ∪ Q t is a group in which N members will be selected for P t+1 .

Main process

This section describes more details about the main process of NSGAs. Algorithm 3 shows the steps of the processing. First, the Offspring is initialized in Algorithm 3 Main process [START_REF] Deb | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints[END_REF][START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF].

1: function Iterate(P opulation) 2:
Of f springs ← ∅ 3:

while Of f springs.size < populationSize do 4:

P arent = Selection(P opulation) 5:

Of f springs = Of f springs ∪ Evolve(P arent) 6:

end while 7:

P opulation = P opulation ∪ Of f springs 8:

P opulation = T runcate(P opulation) 9:

return P opulation 10: end function Algorithm 4 Non-dominated Sorting [START_REF] Deb | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints[END_REF].

Require: R 1: function Sorting(R) 2: RinRank ← ∅ 3: rank = 1 4: remaining ← R 5:
while RisN otEmpty do 6:

F ront ← non -dominatedP opulation(remaining, rank) 7:

remaining = remaining \ F ront 8:

RinRank = RinRank ∪ F ront 9:

rank + + 10:

end while 11:

return RinRank 12: end function Line 2. The size of Offspring equals to the size of Population, i.e., N . Hence, a parent is selected from the population and evolved to become a new offspring. A new population with the size of 2N is created from Offspring and the old population. After that, the function Truncate will cut off the new population to reduce the members to the size of N , as shown in Line 8.

Non-Dominated Sorting

Before the truncating process, the solutions in the population with a size of 2N should be sorted in multiple fronts with their ranking, as shown in Algorithm 4. First, the Non-dominated sorting operator generates the first Pareto set in a population of 2N solutions. Its rank is 1. After that, the process is repeated until the remain population is empty. Finally, 2N solutions are divided into various fronts with their ranks.

Filter front process

NSGA-G using Min point NSGA-G finds the nearest smaller and bigger grid point for each solution. For example, Fig. 7 shows an example of a two-objective The first strategy avoids computing multiple objective cost values of all solutions in the population, the space is divided into multiple small groups by Grid Min Point and Grid Max Point, as shown in Fig. 7. Each group has one Grid Min Point, the nearest smaller point and one Grid Max Point, the nearest bigger point. Only solutions in a group are calculated and compared. The solution has the smallest distance to the nearest smaller point in a group will be added to P t+1 . In this way, in any loop, we do not need to calculate the crowding-distance values or estimate the smallest distance from solutions to the reference points among all members in the last front, as shown in Fig. 7. In any loop, it is not necessary to compare solutions among all members in F l , as F 3 in Fig. 2. The second strategy chooses randomly a group. The characteristic of diversity is maintained by this strategy. Both strategies are proposed to improve the qualities of our algorithm. Algorithm 5 shows the strategy to select N -

l-1 j=1 F j members in F l .
Algorithm 6 Filter front in NSGA-G using Random metric. The two lines 2 and 3 in Algorithm 5 determine the new origin coordinates and the maximum objective values of all solutions, respectively. After that, they will be normalized in a range of [0, 1]. All solutions will be in different groups, depending on the coefficient of the grid. The most important characteristic of this algorithm is randomly selecting the group like NSGA-III to keep the diversity characteristic and remove the solution among members of that group. This selection helps to avoid comparing and calculating the maximum objectives in all solutions.

1: function Filter(F l , M = N -l-
To estimate the quality of the proposed algorithm, three qualities, Generational Distance [START_REF] Veldhuizen | Evolutionary Computation and Convergence to a Pareto Front[END_REF], Inverted Generational Distance [START_REF] Coello | Solving Multiobjective Optimization Problems Using an Artificial Immune System[END_REF] and the Maximum Pareto Front Error [START_REF] Veldhuizen | Multiobjective evolutionary algorithms: classifications, analyses, and new innovations[END_REF], are used.

NSGA-G using Random metric

In MOOP, when the number of objectives is significant, any function used to compare solutions leads to high computation. NSGA-G using Min point uses Grid partition to reduce the number in groups, but it still needs a function to group all objectives value to a scalar value. In order to decrease the execution time, this section proposes a random method to compare solutions among a group. This approach does not generate any reference point or an intermediate function to estimate the value of solutions. The natural metric values are chosen randomly to remove the worst solution in the different groups.

All the steps in this algorithm are similar to NSGA-G using Min point, as shown from Line 2 to Line 6. Loop While has one more step of choosing metric randomly. Function selectRandomMetric is used to select a natural metric among the objectives in MOOP. The important characteristics of this algorithm are randomly selecting the group like NSGA-G and using natural metric among various objectives. It aims to keep the diversity characteristic, and reduce the comparing time. This selection helps to avoid using an intermediate function in comparing and calculating the values of solutions.

Selecting the size of grid

The proposed approach uses Grid partitioning to guarantee that the solutions are distributed in all the solution space. Assuming that there is a problem with N objectives. The last front should remove k solutions. By normalizing the space of solution in the range of [0, 1] and dividing that range to n segments, a solution belongs to one of n N groups in that space. In terms of Non-dominated principle, a group including a solution in that space have many other groups which contain Non-dominated solutions. These groups are called Non-dominated groups. All the groups in this situation make a set groups, called front group.

The proposed idea is to keep the diversity characteristic of the genetic algorithm by generating k groups and removing k solutions. Hence, the ideal front group is designed so that it has k groups.

Simple front group From a group in the normalizing space in range of [0, 1], a simple plane covers it and includes Non-dominated groups. In the space of N axes, the number of groups is n N . Hence, the simple front group is the simple plane. The number of groups in that front group is n N -1 . Therefore, if the last front needs to remove k solutions, the number of grid n is determined as follows

n = k 1 N -1 . (15)
For example, Fig. 8 shows a problem with 3 objectives. In each axis coordinate, the size of grid is 4, and the maximum number of groups in all space of N axis coordinates is 4 3 . A simple front group includes 4 3-1 = 16 groups. If the last front needs to remove 15 solutions, the number of grid when we choose simple front group is n = k Max front group From a group in the normalizing space in range of [0, 1], a simple plane covers it and includes Non-dominated groups. In the space of N axis coordinates, the number of groups is n N . Max front group has the largest number of groups includes N planes. Hence, the number of groups in Max front group is n N -(n -1) N . Therefore, if the last front needs to remove k solution, the number of grid n is determined as follows

n N -(n -1) N = k. (16)
For instance, Fig. 9 shows a problems with 3 objectives. In each axis coordinate, the size of grid is 4, the maximum number of groups in all space of N axis coordinates is 4 3 . Max front group includes 4 3 -3 3 = 64 -27 = 37 groups.

Validation

DREAM

The previous section introduces two algorithms for the Multi-Objective Optimization Problem in MIDAS. DREAM and NSGA-G have been implemented on top of IReS platform. They have been validated with experiments.

Implementation Our experiments are executed on Galactica private cloud11 with a cluster of three machines. Each node has four 2.

Results

To estimate the quality of price models which are estimated by DREAM in comparison with other algorithms, Mean Relative Error (MRE), a metric used in [START_REF] Akdere | Learning-based Query Performance Modeling and Prediction[END_REF] is used and described as below:

1 M i=1 M |ĉ i -c i | c i , (17)
where M is the number of testing queries, ĉi and c i are the predict and actual execution time of testing queries, respectively. IReS platform uses multiple machine learning algorithms in their model, such as Least squared regression, Bagging predictors, Multilayer Perceptron. In IReS model building process, IReS tests many algorithms and the best model with the smallest error is selected. It guarantees the predicted values as the best one for estimating process. DREAM is compared to the Best Machine Learning model (BML) in IReS platform with many observation window (N , 2N , 3N and no limit of history data). The smallest size of a window, N = L + 2 [START_REF] Soong | Fundamentals of probability and statistics for engineers[END_REF], where L is the number of variables, is the minimum data set DREAM requires.

As shown in Table 5 and 6, MRE of DREAM are the smallest values between various observation windows. In our experiments, the size of historical data, which DREAM uses, are small, around N .

NSGA-G

Various earlier studies on Multiple Objective Evolutionary Algorithms (MOEAs) introduce test problems which are either simple or not scalable. DTLZ test problems [START_REF] Deb | Scalable Test Problems for Evolutionary Multiobjective Optimization, Evolutionary Multiobjective Optimization[END_REF] are useful in various research activities on MOEAs, e.g., testing the performance of a new MOEA, comparing different MOEAs and a better understanding of MOEAs. The proposed algorithm is experimented on DTLZ test problems with other famous NSGAs to show advantages in convergence, diversity and execution time.

Implementation Our experiments use Multiobjective Evolutionary Algorithms (MOEA) 13 framework in Open JDK Java 1.8. All experiments are run on a machine with following parameters: Intel(R) core(TM) i7-6600U CPU @ 2.60GHz × 4, 16GB RAM.

Experiments For fair comparison and evaluation, the same parameters are used, such as Simulated binary crossover [START_REF] Deb | Simulated Binary Crossover for Continuous Search Space[END_REF] (30), Polynomial mutation [START_REF] Deb | Simulated Binary Crossover for Continuous Search Space[END_REF] (20), max evaluations (10000) and populations (100) for eMOEA [START_REF] Coello | Evolutionary algorithms for solving multi-objective problems, Second Edition[END_REF], NSGA-II, MOEA/D [START_REF] Zhang | MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition[END_REF], NSGA-III and NSGA-G14 , during 50 independent running to solve two types of problems in DTLZ test problems [START_REF] Deb | Scalable Test Problems for Evolutionary Multiobjective Optimization, Evolutionary Multiobjective Optimization[END_REF] with m objectives, m ∈ [START_REF] Breiman | Bagging predictors[END_REF][START_REF] Coello | Evolutionary algorithms for solving multi-objective problems, Second Edition[END_REF]. These algorithms use the same population size N = 100 and the maximum evaluation M = 10000. We apply Simple front group approach, equation 15, to determine the grid in both of NSGA-Gs with Min point and Random metric experiments. We use the Generational Distance (GD) [START_REF] Veldhuizen | Evolutionary Computation and Convergence to a Pareto Front[END_REF], Inverted Generational Distance (IGD) [START_REF] Coello | Solving Multiobjective Optimization Problems Using an Artificial Immune System[END_REF] and the Maximum Pareto Front Error (MPFE) [START_REF] Veldhuizen | Multiobjective evolutionary algorithms: classifications, analyses, and new innovations[END_REF] to compare the quality of NSGA-Gs to other NSGAs.

GD measures how far the evolved solution set is from the true Pareto front [START_REF] Yen | Performance Metrics Ensemble for Multiobjective Evolutionary Algorithms[END_REF], as shown in following:

GD = n i=1 d 2 i n , (18)
where d j = min j ||f (x i )-P F true (x j )|| shows the distance objective space between solution x i and the nearest member in the true Pareto front (P F true ), and n is the number of solutions in the approximation front. Lower value of GD represents a better quality of an algorithm. IGD is a metric to estimate the approximation quality of the Pareto front obtained by MOO algorithms [START_REF] Bezerra | An Empirical Assessment of the Properties of Inverted Generational Distance on Multi-and Many-Objective Optimization, Evolutionary Multi-Criterion Optimization[END_REF], which can measure both convergence and diversity in a sense. IGD is shown in the following equation [START_REF] Yen | Performance Metrics Ensemble for Multiobjective Evolutionary Algorithms[END_REF]:

IGD = v∈P Ftrue d(v, X) |P F true | , ( 19
)
where X is the set of non-dominated solutions in the approximation front, d(v, X) presents the minimum Euclidean distance between a point v in P F true and the points in X. Lower value of IDG represents the approximate front getting close to P F true , and not missing any part of the whole P F true . MPFE shows the most significant distance between the individuals in Pareto front and the solutions in the approximation front [START_REF] Yen | Performance Metrics Ensemble for Multiobjective Evolutionary Algorithms[END_REF]. This metric is shown in the following equation:

M P F E = max i d i . (20)
In all tables show the experiments, the darkest mark value show the least value in various algorithm experiments, and the brighter mark value is the second least value among them.

Study on test problems In this section, we use DTLZs, and WFG [START_REF] Huband | A review of multiobjective test problems and a scalable test problem toolkit[END_REF] test problem to experiment NSGA-Gs. Advantages of two versions of NSGA-G are present in 748e-04 6.923e-03 6.908e-03 3.218e-03 7.617e-04 WFG3 5 4.027e-04 0.000e+00 2.549e-03 1.941e-03 2.011e-03 1.061e-05 DTLZ1 6 2.903e+00 2.137e+00 9.131e+01 1.820e+00 6.839e+00 4.907e-01 DTLZ3 6 2.226e+01 1.332e+01 1.252e+02 1.760e+01 2.389e+01 5.457e+00 WFG1 6 1.207e-03 8.842e-04 8.000e-03 6.753e-03 3.559e-03 7.417e-04 WFG3 6 4.104e-04 0.000e+00 2.523e-03 1.639e-03 1.800e-03 5.384e-05 DTLZ1 7 7.790e-01 8.949e-01 2.228e+01 2.601e-01 1.407e+00 8.201e-02 DTLZ3 7 1.719e-01 4.449e-02 1.309e+00 3.610e-02 1.619e-01 5.628e-03 WFG1 7 1.048e-03 8.219e-04 6.825e-03 5.613e-03 3.891e-03 6.405e-04 WFG3 7 4.011e-04 3.055e-06 2.390e-03 1.871e-03 1.665e-03 5.926e-05 DTLZ1 8 5.823e+00 5.851e+00 1.130e+02 1.276e+00 9.933e+00 4.660e-01 DTLZ3 8 2.071e+01 1.941e+01 1.604e+02 1.355e+01 3.001e+01 4.757e+00 WFG1 8 1.377e-03 9.406e-04 9.023e-03 7.659e-03 4.454e-03 6.469e-04 WFG3 8 3.655e-04 2.689e-05 1.692e-03 1.301e-03 9.662e-04 6.578e-05 DTLZ1 9 8.374e-01 3.626e+00 3.074e+01 3.544e-01 2.772e+00 1.003e-01 DTLZ3 9 4.673e-02 7.112e-02 6.293e-01 8.922e-03 1.052e-01 2.843e-03 WFG1 9 1.309e-03 8.924e-04 8.882e-03 7.551e-03 4.020e-03 6.816e-04 WFG3 9 3.597e-04 2.576e-05 1.298e-03 1.208e-03 7.634e-04 5.365e-05 DTLZ1 10 7.375e-01 1.519e+00 2.091e+01 2.705e-01 2.207e+00 3.021e-02 DTLZ3 10 4.785e-02 1.116e-01 6.793e-01 7.345e-03 1.118e-01 2.939e-03 WFG1 10 1.369e-03 1.385e-03 8.551e-03 6.364e-03 3.648e-03 6.692e-04 WFG3 10 3.259e-04 0.000e+00 1.196e-03 1.265e-03 6.945e-04 4.352e-05

First, two versions of NSGA-G often show that they are faster than the other algorithms in all experiments of average computation time, Table 8, 10, and 12.

Second, NSGA-Gs are also better than other NSGAs in terms of quality in GD and MPFE experiments, as shown in Table 7, and 11. Except for the IDG experiment, as shown in Table 9 the quality of NSGA-G with Random metric is not as good as other ones. However, the fastest algorithm among NSGAs is NSGA-R NSGA-II MOEA/D NSGA-III NSGA-G DTLZ1 5 3.437e-01 1.027e+00 3.741e+01 6.226e-01 3.465e+00 4.637e-01 DTLZ3 5 5.568e-01 4.794e-01 3.576e+00 3.969e-01 1.098e+00 1.589e-01 WFG1 5 1.298e-01 2.924e-01 1.234e-01 7.202e-02 1.365e-01 2.906e-01 WFG3 5 4.167e-02 3.850e-01 1.272e-01 1.417e-01 7.899e-02 3.987e-01 DTLZ1 6 4.975e+00 6.617e+00 2.469e+02 2.903e+00 9.524e+00 2.688e+00 DTLZ3 6 1.131e+02 4.698e+01 5.199e+02 4.207e+01 8.253e+01 2.761e+01 WFG1 6 1.722e-01 3.705e-01 1.531e-01 7.460e-02 1.596e-01 3.341e-01 WFG3 6 5.367e-02 5.424e-01 1.488e-01 1.630e-01 1.065e-01 5.146e-01 DTLZ1 7 7.034e-01 4.042e+00 1.938e+01 4.718e-01 7.695e-01 8.458e-01 DTLZ3 7 7.320e-01 4.310e-01 4.852e+00 2.878e-01 3.826e-01 2.524e-01 WFG1 7 1.437e-01 3.547e-01 1.371e-01 7.114e-02 1.403e-01 3.199e-01 WFG3 7 6.134e-02 6.325e-01 1.573e-01 1.705e-01 1.169e-01 6.122e-01 DTLZ1 8 1.234e+01 1.212e+01 4.166e+02 3.101e+00 1.073e+01 2.849e+00 DTLZ3 8 1.501e+02 6.557e+01 7.623e+02 3.720e+01 1.011e+02 2.665e+01 WFG1 8 1.284e-01 3.186e-01 1.251e-01 6.956e-02 1.238e-01 2.692e-01 WFG3 8 6.487e-02 6.477e-01 1.593e-01 1.704e-01 1.115e-01 6.094e-01 DTLZ1 9 4.009e-01 3.676e+00 5.490e+00 3.932e-01 6.185e-01 5.747e-01 DTLZ3 9 3.029e-01 4.578e-01 1.713e+00 2.398e-01 2.584e-01 2.401e-01 WFG1 9 1.167e-01 2.921e-01 1.193e-01 6.477e-02 1.131e-01 2.561e-01 WFG3 9 6.758e-02 6.897e-01 1.621e-01 1.675e-01 1.078e-01 6.237e-01 DTLZ1 10 9.350e-01 9.074e+00 1.357e+01 6.061e-01 1.499e+00 1.028e+00 DTLZ3 10 4.368e-01 5.440e-01 2.368e+00 2.000e-01 3.965e-01 1.912e-01 WFG1 10 1.147e-01 3.043e-01 1.167e-01 6.273e-02 1.102e-01 2.671e-01 WFG3 10 6.759e-02 6.676e-01 1.670e-01 1.696e-01 1.043e-01 6.102e-01 NSGA-R NSGA-II MOEA/D NSGA-III NSGA-G DTLZ1 5 2.008e+01 1.195e+01 8.083e+02 1.765e+01 3.548e+02 4.912e+00 DTLZ3 5 1.079e+01 1.564e+00 2.545e+01 1.604e+00 1.546e+01 5.798e-01 WFG1 5 1.332e-01 1.763e-02 2.620e-01 2.042e-01 1.709e-01 1.588e-02 WFG3 5 1.583e-01 0.000e+00 9.601e-02 6.763e-02 1.139e-01 0.000e+00 DTLZ1 6 2.937e+02 5.789e+01 1.583e+03 5.168e+01 3.920e+02 7.665e+00 DTLZ3 6 1.045e+03 2.861e+02 1.825e+03 1.913e+02 7.048e+02 7.409e+01 WFG1 6 2.288e-01 1.619e-02 3.790e-01 3.086e-01 2.649e-01 1.372e-02 WFG3 6 1.690e-01 0.000e+00 1.090e-01 7.179e-02 9.973e-02 2.058e-03 DTLZ1 7 1.193e+02 4.205e+01 8.990e+02 9.095e+00 1.081e+02 2.998e+00 DTLZ3 7 1.138e+01 2.539e+00 1.768e+01 3.267e-01 4.447e+00 1.286e-01 WFG1 7 2.461e-01 1.443e-02 3.670e-01 2.775e-01 2.428e-01 1.545e-02 WFG3 7 1.630e-01 6.556e-04 1.017e-01 6.336e-02 7.499e-02 2.411e-03 DTLZ1 8 4.798e+02 2.375e+02 1.982e+03 4.991e+01 5.619e+02 8.178e+00 DTLZ3 8 1.458e+03 3.881e+02 2.152e+03 1.856e+02 9.085e+02 6.259e+01 WFG1 8 2.722e-01 1.486e-02 4.113e-01 3.155e-01 3.020e-01 1.039e-02 WFG3 8 1.499e-01 0.000e+00 9.124e-02 5.919e-02 6.697e-02 2.380e-03 DTLZ1 9 1.732e+02 1.234e+02 9.926e+02 1.264e+01 3.271e+02 1.976e+00 DTLZ3 9 7.820e+00 3.242e+00 1.899e+01 2.121e-01 6.489e+00 9.978e-02 WFG1 9 2.388e-01 1.108e-02 3.644e-01 2.316e-01 2.435e-01 7.929e-03 WFG3 9 1.516e-01 4.995e-04 8.803e-02 5.787e-02 8.046e-02 1.736e-03 DTLZ1 10 1.097e+02 1.138e+02 9.838e+02 8.148e+00 3.040e+02 2.231e+00 DTLZ3 10 6.727e+00 2.405e+00 1.556e+01 1.584e-01 5.933e+00 7.632e-02 WFG1 10 3.030e-01 1.372e-02 4.268e-01 2.544e-01 3.118e-01 9.250e-03 WFG3 10 1.468e-01 3.964e-04 7.328e-02 5.557e-02 6.889e-02 2.378e-03 often NSGA-G with random metric. It can be accepted for the trade-off between quality and computation time.

Study on the evaluation In the previous experiments, we survey algorithms with various problems and the constant number of max evaluation. This section selects a specific problem and shows the observation of algorithms while the process is running. In particular, we choose DTLZ3 problem with eight objectives, called DTLZ3-8. Besides, we focus on reducing the execution time of NSGAs algorithm. Hence, this section compares two versions of NSGA-G algorithms to others in NSGA class, such as NSGA-II and NSGA-III. Two versions of NSGA-G with Min point and Random metric are called NSGA-G and NSGA-R, respectively. The results in Fig. 10 and 11 show that two versions of NSGA-G are faster than others. Both their convergence and diversity are better than NSGA-II and NSGA-III.

In conclusion, NSGA-Gs often show better quality and faster execution time in most cases, such as DTLZs, WFGs. One main conclusion of these experiments is that NSGA-G with a Random metric is often the least expensive in terms of computation.

Conclusion

This paper is about medical data management in cloud federation. multi-engine environments. DREAM aims to address variance in a cloud federation and to provide accurate estimation for MOQP. Experiment results with DREAM and TPC-H benchmark are quite promising with respect to existing solutions. Further more, we introduce Non-dominated Sorting Algorithms based on Grid partitioning (NSGA-G) in searching and optimization MOOP. We validated NSGA-Gs with DTLZ, WFG test problems, and MOEA framework. The experiments show that NSGA-Gs often show better quality and faster execution time than other NSGAs in most cases, such as DTLZs, WFGs. One main conclusion of these experiments is that NSGA-G with a Random metric is often the least expensive in terms of computation.

In the future, we plan to validate our proposal with more cloud providers (and their associated pricing model and services) and data management systems. We will also define new strategies to choose QEPs in a Pareto Set. Further more, the size of population in each generation iterate is constant in many NSGAs. The suitable value of population size is still a question of NSGAs. Future works include a deeper study on the impact of the size of the population.

Fig. 1 :

 1 Fig. 1: Motivating Example on using cloud federation.

Fig. 3 :

 3 Fig.3: An example of using the crowing distance in NSGA-II.

Fig. 4 :

 4 Fig.4: Architecture of MIDAS[START_REF] Le | Dynamic Estimation for Medical Data Management in a Cloud Federation[END_REF].

Fig. 6 :Algorithm 2 : else 6 :

 626 Fig.6: Comparing two MOQP approaches[START_REF] Le | Dynamic Estimation for Medical Data Management in a Cloud Federation[END_REF]

Fig. 7 :Algorithm 5

 75 Fig.7: An example of using Grid points.

Fig. 8 :

 8 Fig. 8: A simple front group.

Fig. 9 :

 9 Fig.9: A max front group.

  NSGA-II MOEA/D NSGA-III NSGA-G DTLZ1 5 2.595e-01 4.418e-01 2.251e+01 4.264e-01 3.090e+00 1.977e-01 DTLZ3 5 1.861e-01 5.528e-02 1.130e+00 8.650e-02 3.079e-01 1.678e-02 WFG1 5 1.133e-03 9.

Fig. 10 :Fig. 11 :

 1011 Fig. 10: Inverted Generational Distance of 4 algorithms with DTLZ3-8.

Table 1 :

 1 Multiple Objectives for Query Execution Plans

	QEP VMs Price ($/60min) Time (min) Monetary ($)
	QEP1 20	0.02	60	0.4
	QEP2 80	0.02	22	0.59
	QEP3 50	0.02	26	0.43

Table 2 :

 2 Example of instances pricing in 2019.

	Provider Machine vCPU Memory (GiB) Storage (GB) Price ($/hour)
	Amazon a1.medium 1	2	EBS-Only	0.0255
		a1.large	2	4	EBS-Only	0.0510
		a1.xlarge	4	8	EBS-Only	0.1020
		a1.2xlarge	8	16	EBS-Only	0.2040
		a1.4xlarge 16	32	EBS-Only	0.4080
	Microsoft	B1S	1	1	2	0.0104
		B1MS	1	2	4	0.0208
		B2S	2	4	8	0.0416
		B2MS	2	8	16	0.0832
		B4MS	4	16	32	0.1660
		B8MS	8	32	64	0.3330
	2.2 Pareto plan set
	Pareto dominance techniques are often used in Multi-Objective Optimization
	Problem (MOOP), such as Evolutionary Multiobjective Optimization (EMO) [14,
	15, 27, 31, 44, 53, 54]. In the vast space of candidate solutions of Multi-Objective
	Optimization Problem (MOOP), a candidate solution may be not better than
	another one because of trade-off between various objective values. Pareto sets
	are used in this situation to optimize a MOOP.

Table 3 :

 3 Recent heterogeneous database system researches.

)

  Example 2. A query Q[START_REF] Le | Dynamic Estimation for Medical Data Management in a Cloud Federation[END_REF] could be expressed as follows:SELECT p . PatientSex , i . GeneralNames FROM Patient p , GeneralInfo i WHERE p . UID = i . UIDwhere Patient table is stored in cloud A and uses Hive[START_REF] Thusoo | Hive -a petabyte scale data warehouse using Hadoop[END_REF] database engine6 , while GeneralInfo table is in cloud B with PostgreSQL database engine7 . This scenario leads to concern two metrics of monetary cost and execution time cost. We can use the cost functions which depend on the size of tables of Patient and GeneralInfo. Besides, the configuration and pricing of virtual machines cloud A and B are different. Hence, the cost functions depend on the size of tables and the number of virtual machines in cloud A and B.

ĉti = βt0 + βt1 x P a + βt2 x Ge + βt3 x nodeA + βt4 x nodeB ĉmo = βm0 + βm1 x P a + βm2 x Ge + βm3 x nodeA + βm4 x nodeB

Table 4 :

 4 Using MLR in different size of dataset[START_REF] Le | Dynamic Estimation for Medical Data Management in a Cloud Federation[END_REF].

	Cost	x1	x2 M R 2
	20.640 0.4916 0.2977
	15.557 0.6313 0.0482
	20.971 0.9481 0.8232
	24.878 0.4855 2.7056 4 0.7571
	23.274 0.0125 2.7268 5 0.7705
	30.216 0.9029 2.6456 6 0.8371
	29.978 0.7233 3.0640 7 0.8788
	31.702 0.8749 4.2847 8 0.8876
	20.860 0.3354 2.1082 9 0.8751
	32.836 0.8521 4.8217 10 0.8945

  are well

	Initial Population		All Candidates
	Objective		Weighted Sum
	values		Model Values
	Satisfied Termination Criteria?	Termination Population	Comparing Scalar Values
	Fitness Distribution	Weighted Sum Model Values	The best QEP
	Genetic	Comparing
	Operation	Scalar Value
	Insert Parent	The best QEP
	Multi-Objecitve Optimization based	Multi-Objecitve Optimization
	on Genetic Algorithm	based on Weighted Sum Model

  4 GHz CPU, 80 GiB Disk, 8 GiB memory and runs 64-bit platform Linux Ubuntu 16.04.2 LTS. The system uses Hadoop 2.7.3 12 , Hive 2.1.1, PostgreSQL 9.5.14, Spark 2.2.0 and Java OpenJDK Runtime Environment 1.8.0. IReS platform is used to manage data in multiple database engine and deploy the algorithms.

Table 5 :

 5 Comparison of mean relative error with 100MiB TPC-H dataset[START_REF] Le | Dynamic Estimation for Medical Data Management in a Cloud Federation[END_REF].

	Query BMLN BML2N BML3N BML DREAM
	12 0.265 0.459 0.220 0.485 0.146
	13 0.434 0.517 0.381 0.358 0.258
	14 0.373 0.340 0.335 0.358 0.319
	17 0.404 0.396 0.267 0.965 0.119

Table 6 :

 6 Comparison of mean relative error with 1GiB TPC-H dataset[START_REF] Le | Dynamic Estimation for Medical Data Management in a Cloud Federation[END_REF].Experiments TPC-H benchmark with two datasets of 100MB and 1GB is used to have experiments with DREAM. Experiments with TPC-H benchmark are executed in a multi-engine environment consisting of Hive and PostgreSQL deployed on Galactica private cloud. In TPC-H benchmark, the queries related to two tables are 12, 13, 14 and 17. These queries with two tables in two different databases, such as Hive and PostgreSQL, are studied.

	Query BMLN BML2N BML3N BML DREAM
	12 0.349 0.854 0.341 0.480 0.335
	13 0.396 0.843 0.457 0.487 0.349
	14 0.468 0.664 0.539 0.790 0.318
	17 0.620 0.611 0.681 0.970 0.536

Table 7 ,

 7 8, 9, 10, 11, and 12. Metrics, such as GD, IDG, MPFE, are used to estimate the qualities of the different algorithms. These experiments compare both of NSGA-Gs with Min point and Random metric to other algorithms.

Table 7 :

 7 Generational Distance

Table 8 :

 8 Average compute time in Generational Distance experiment

	m eMOEA	NSGA-R NSGA-II MOEA/D NSGA-III NSGA-G
	DTLZ1 5 3.604e+01 6.642e+01 5.508e+01 2.000e+02 2.241e+02 6.366e+01
	DTLZ3 5 5.398e+01 6.440e+01 7.074e+01 1.870e+02 2.714e+02 6.212e+01
	WFG1 5 1.379e+02 6.658e+01 6.636e+01 1.899e+02 2.594e+02 6.720e+01
	WFG3 5 8.562e+02 8.162e+01 6.074e+01 1.864e+02 3.077e+02 8.370e+01
	DTLZ1 6 4.552e+01 5.582e+01 5.632e+01 1.918e+02 1.662e+02 5.672e+01
	DTLZ3 6 9.340e+01 6.572e+01 6.362e+01 1.971e+02 1.783e+02 6.638e+01
	WFG1 6 1.961e+02 9.826e+01 7.392e+01 2.049e+02 2.157e+02 7.286e+01
	WFG3 6 1.083e+03 7.580e+01 6.642e+01 1.967e+02 2.384e+02 7.782e+01
	DTLZ1 7 6.206e+01 5.834e+01 6.208e+01 2.290e+02 1.621e+02 5.964e+01
	DTLZ3 7 1.568e+02 6.992e+01 7.024e+01 2.405e+02 1.817e+02 7.022e+01
	WFG1 7 2.585e+02 7.806e+01 8.042e+01 2.473e+02 2.085e+02 7.810e+01
	WFG3 7 1.469e+03 8.030e+01 9.184e+01 2.896e+02 2.821e+02 9.950e+01
	DTLZ1 8 8.762e+01 5.998e+01 6.640e+01 2.450e+02 2.327e+02 6.244e+01
	DTLZ3 8 2.235e+02 7.618e+01 7.652e+01 2.536e+02 2.535e+02 7.424e+01
	WFG1 8 3.100e+02 8.034e+01 8.710e+01 2.625e+02 2.924e+02 8.206e+01
	WFG3 8 1.464e+03 7.912e+01 7.772e+01 2.542e+02 3.268e+02 8.346e+01
	DTLZ1 9 1.157e+02 6.264e+01 7.034e+01 2.524e+02 3.095e+02 6.590e+01
	DTLZ3 9 2.978e+02 7.694e+01 8.422e+01 2.678e+02 3.422e+02 7.828e+01
	WFG1 9 3.846e+02 8.442e+01 9.426e+01 2.731e+02 3.844e+02 8.668e+01
	WFG3 9 1.677e+03 8.954e+01 8.166e+01 2.595e+02 4.373e+02 8.642e+01
	DTLZ1 10 1.527e+02 6.510e+01 7.584e+01 2.740e+02 4.204e+02 6.874e+01
	DTLZ3 10 3.860e+02 8.132e+01 8.916e+01 2.883e+02 4.641e+02 8.370e+01
	WFG1 10 4.747e+02 8.996e+01 1.005e+02 2.941e+02 5.175e+02 9.272e+01
	WFG3 10 1.881e+03 8.576e+01 8.640e+01 2.802e+02 6.035e+02 9.128e+01

Table 9 :

 9 Inverted Generational Distance

	m eMOEA

Table 10 :

 10 Average compute time in Inverted Generational Distance experiment

	m eMOEA	NSGA-R NSGA-II MOEA/D NSGA-III NSGA-G
	DTLZ1 5 3.384e+01 5.500e+01 5.176e+01 1.840e+02 2.139e+02 5.276e+01
	DTLZ3 5 9.490e+01 8.072e+01 5.954e+01 2.942e+02 2.803e+02 6.146e+01
	WFG1 5 1.453e+02 7.752e+01 8.710e+01 1.957e+02 2.988e+02 8.220e+01
	WFG3 5 9.067e+02 8.638e+01 5.950e+01 2.087e+02 3.137e+02 8.416e+01
	DTLZ1 6 4.982e+01 6.264e+01 5.860e+01 2.209e+02 1.894e+02 6.534e+01
	DTLZ3 6 9.604e+01 6.984e+01 6.554e+01 2.182e+02 1.958e+02 7.078e+01
	WFG1 6 2.188e+02 8.088e+01 7.810e+01 2.452e+02 2.282e+02 8.362e+01
	WFG3 6 2.601e+03 9.036e+01 6.638e+01 3.200e+02 3.094e+02 1.215e+02
	DTLZ1 7 6.754e+01 5.880e+01 6.122e+01 2.517e+02 1.620e+02 6.066e+01
	DTLZ3 7 1.587e+02 7.172e+01 6.986e+01 2.525e+02 1.798e+02 7.168e+01
	WFG1 7 2.579e+02 7.696e+01 8.294e+01 2.587e+02 2.185e+02 7.768e+01
	WFG3 7 1.272e+03 7.836e+01 7.194e+01 2.487e+02 2.284e+02 8.888e+01
	DTLZ1 8 8.430e+01 5.996e+01 6.610e+01 2.537e+02 2.322e+02 6.328e+01
	DTLZ3 8 2.358e+02 7.418e+01 7.808e+01 2.608e+02 2.535e+02 7.446e+01
	WFG1 8 3.158e+02 7.960e+01 8.682e+01 2.704e+02 2.903e+02 8.344e+01
	WFG3 8 1.432e+03 8.044e+01 7.712e+01 2.513e+02 3.242e+02 8.364e+01
	DTLZ1 9 1.237e+02 6.278e+01 6.978e+01 2.563e+02 3.120e+02 6.646e+01
	DTLZ3 9 3.174e+02 7.882e+01 8.330e+01 2.721e+02 3.418e+02 7.838e+01
	WFG1 9 3.827e+02 8.586e+01 9.338e+01 2.718e+02 3.837e+02 8.594e+01
	WFG3 9 1.696e+03 8.290e+01 8.142e+01 2.607e+02 4.369e+02 8.654e+01
	DTLZ1 10 1.436e+02 6.472e+01 7.536e+01 2.753e+02 4.187e+02 6.876e+01
	DTLZ3 10 4.003e+02 8.566e+01 8.872e+01 2.897e+02 4.572e+02 8.270e+01
	WFG1 10 4.635e+02 8.924e+01 1.008e+02 2.915e+02 5.137e+02 9.116e+01
	WFG3 10 1.902e+03 8.662e+01 8.612e+01 2.802e+02 6.022e+02 9.028e+01

Table 11 :

 11 Maximum Pareto Front Error

	m eMOEA

Table 12 :

 12 Average compute time in Maximum Pareto Front Error experiment .476e+02 7.960e+01 8.678e+01 2.612e+02 2.932e+02 8.164e+01 WFG3 8 1.482e+03 8.250e+01 7.690e+01 2.497e+02 3.244e+02 8.380e+01 DTLZ1 9 1.031e+02 6.208e+01 6.984e+01 2.514e+02 3.068e+02 6.554e+01 DTLZ3 9 3.043e+02 7.924e+01 8.222e+01 2.634e+02 3.368e+02 7.806e+01 WFG1 9 3.935e+02 8.856e+01 9.290e+01 2.700e+02 3.807e+02 8.676e+01 WFG3 9 1.660e+03 9.016e+01 8.028e+01 2.594e+02 4.347e+02 8.622e+01 DTLZ1 10 1.507e+02 6.436e+01 7.442e+01 2.728e+02 4.151e+02 6.830e+01 DTLZ3 10 3.933e+02 8.494e+01 8.852e+01 2.865e+02 4.593e+02 8.244e+01 WFG1 10 4.769e+02 9.296e+01 9.974e+01 2.904e+02 5.110e+02 9.182e+01 WFG3 10 1.875e+03 8.474e+01 8.594e+01 2.784e+02 6.013e+02 9.096e+01

	m eMOEA	NSGA-R NSGA-II MOEA/D NSGA-III NSGA-G
	DTLZ1 5 4.128e+01 5.408e+01 5.408e+01 2.401e+02 2.308e+02 5.522e+01
	DTLZ3 5 5.676e+01 6.470e+01 5.944e+01 2.074e+02 2.982e+02 6.294e+01
	WFG1 5 1.623e+02 8.048e+01 7.232e+01 2.239e+02 2.815e+02 7.082e+01
	WFG3 5 9.397e+02 7.952e+01 6.154e+01 2.043e+02 3.174e+02 1.023e+02
	DTLZ1 6 4.550e+01 5.556e+01 5.634e+01 1.924e+02 1.662e+02 5.686e+01
	DTLZ3 6 9.168e+01 6.554e+01 6.418e+01 1.985e+02 1.787e+02 6.656e+01
	WFG1 6 1.958e+02 7.512e+01 7.434e+01 2.072e+02 2.170e+02 7.650e+01
	WFG3 6 1.136e+03 7.774e+01 6.724e+01 1.967e+02 2.406e+02 7.916e+01
	DTLZ1 7 8.734e+01 6.188e+01 6.164e+01 2.453e+02 2.058e+02 6.204e+01
	DTLZ3 7 1.622e+02 7.046e+01 7.170e+01 2.699e+02 1.812e+02 8.968e+01
	WFG1 7 2.674e+02 8.156e+01 8.470e+01 2.574e+02 2.154e+02 8.036e+01
	WFG3 7 1.461e+03 8.358e+01 7.426e+01 2.546e+02 2.334e+02 8.180e+01
	DTLZ1 8 8.920e+01 6.054e+01 6.592e+01 2.443e+02 2.349e+02 6.252e+01
	DTLZ3 8 2.360e+02 7.318e+01 7.644e+01 2.536e+02 2.555e+02 7.426e+01
	WFG1 8 4

http://www.tpc.org/tpch/

https://spark.apache.org/

https://www.cs.waikato.ac.nz/ml/weka/

http://hive.apache.org/

https://www.postgresql.org/

https://aws.amazon.com/

https://azure.microsoft.com/

https://cloud.google.com/

https://horizon.isima.fr/

http://hadoop.apache.org/

http://moeaframework.org/

https://gitlab.inria.fr/trle/moea