Sonia Ikken
email: sonia.ikken@telecom-sudparis.eu

Eric Renault
email: eric.renault@telecom-sudparis.eu

Amine Barkat
email: amine.barkat@polimi.it

Tahar Abdelkamel Tari

Éric Renault

Tahar Kechadi
email: tahar.kechadi@ucd.ie

Cost

Keywords: Collaborative Cloud Storage Environment, Big data Scheduling, Storage Cost Saving, Intermediate data Dependency

come

I. INTRODUCTION

Big data workflows have become an important paradigm since the introduction of scientific workflow and the need to formalize complex data-intensive scientific processes. One common characteristic of big data workflow applications is the existence of intermediate data during the execution of workflow instances. This leads to the generation of a massive amount of intermediate results as data dependencies need to be hosted and managed over a cloud infrastructure. Handling large intermediate data dependencies in cloud infrastructure is important for such operations and need a long time for execution since those dependencies need to process intermediate results from different storage locations. As some intermediate data are very large to be relocated efficiently, this operation must take into account the dependencies between intermediate data in selecting their locality. Furthermore, scientific users share important intermediate data dependencies for cooperation and reproduction of new intermediate results. This has enabled researchers to collaboratively work with other professionals or scientific users around the world and to handle and share intermediate data workflow enormously larger in size than before. By offering storage services in several geographically distributed datacenters, cloud infrastructure enables big data workflow applications to offer low latency access to scientific user data. However, the ever increasing volumes of scientific intermediate data address the need to interpret, move and store them more efficiently to the most appropriate datacenter. One fundamental issue in dealing with such scales of scientific user intermediate data results for a workflow application is how to efficiently place them in a distributed cloud datacenter while ensuring the dependency and scalability of the placed data such that the total storage cost of the cloud provider is minimized. On another note, cloud storage providers offer geographically distributed datacenters providing several storage classes with different prices. They can collaborate by sharing their respective resources and dynamically adjust their hosting capacities in response to their data applications. An important problem faced by cloud users is how to exploit these storage classes to serve an application with data requirements at minimum cost. A federation of existing cloud storage services supports the scientific users with a unified and combined view of storage and data services across several providers and applications. Recently, several studies take advantage of pricing plan variety of different resources in a cloud storage federation, where the cost can be optimized by trading through negotiation a storage versus compute and network resources as well as cost optimization of data distribution across cloud providers [START_REF] Rebai | Improving profit through cloud federation[END_REF], [START_REF] Moreno-Vozmediano | Iaas cloud architecture: From virtualized datacenters to federated cloud infrastructures[END_REF], [START_REF] Vernik | Data on-boarding in federated storage clouds[END_REF], [START_REF] Bermbach | Cloud federation: Effects of federated compute resources on quality of service and cost[END_REF] (here, we are disregarding in profit improvements). None of these studies investigated the trade off between network and storage cost to optimize cost of data workflow placement across a federated cloud storage provider. Our study is motivated by these pioneer issues as none of them can simultaneously answer the aforementioned questions (i.e., placement and cost saving of data workflow in cloud storage federation).

Hence, addressing these questions makes the following main contributions: by exploiting cloud storage federation characteristics and data workflow requirements, we formalize and model the input and output parameters of the system cost model. We then propose a cost optimization problem in which the optimal cost of transferring, storing and requesting intermediate data is calculated. The exact intermediate data file dependency is assumed to be known a priori in order to focus on the data workflow placement problem itself. In this approach, we proposed an exact algorithm based on the integer linear programming model that takes into account the dependency requirements (valuable and unnecessary correlation) of these intermediate data for making decisions, and reallocates intermediate data requests with dependencies in a single datacenter to reduce the total storage cost. This paper is organized as follows: Sec. II describes the system model based on the cloud storage federation scenario

II. SYSTEM MODEL

A. Scenario Assumptions

The scenario introduced in Fig. 1 The set of considered datacenters can be modeled as a matrix noted DChome d which combines a home datacenter in order to emphasize the origin of the generated data. For the sake of convenience, we here only consider storage and data transfer resources of D as computing resource provisioning is similar to that of the storage resource. Hence, to underscore the limited amount of available storage space on each federated datacenter k, each member in the federation knows the storage resource quotas that is capped at SCF k offered by the other federation members as well as their internal capacity that is capped at SCL k . Denoted BCF k , the data bandwidth shared between the home datacenter and the destination datacenters to transfer one unit of intermediate data file. Let DBmax k be the maximum data bandwidth quota (in GB per month) provided among federated datacenters.

Each provider that hosts federated datacenters cooperatively proposes a storage cost noted OSC k , and induces a local storage cost noted LSC d for the internally hosted intermediate data. A storage federation price varies according to busy storage or data bandwidth resources formulated by S busy . Each federated datacenter provides a maximum cloud storage quota noted QCmax k to every users. Then, the busy quota is reduced from the maximum capacity of storage and bandwidth resources. However, cloud storage providers must rise to the variation of the price in real-time regarding to various factors such as data access demands, cloud storage market rates, and datacenter localization. To unlock these constraints, a pricing strategy [START_REF] Toosi | Resource provisioning policies to increase iaas provider's profit in a federated cloud environment[END_REF] is adapted to determine the storage federation cost of the providers that varies according to their busy quotas in a way that when the busy quota is high the monetary cost goes up and when it is low the the monetary cost goes down. This pricing mechanism allows them to dynamically set their insourcing/outsourcing storage or transfer prices by establishing the monetary cost in exchange of offering data storage space or data bandwidth or selling storage resources. Below, the mathematical formula determines the insourcing / outsourcing prices for the data placement demands:

S = SCmax k -S dis SCmax k * (S price -ME price ) + ME price (1)
Basically, equation (1) considers the minimum effective price (ME price ) that is a reference monetary cost of the amount of data storage or data transfer that providers do not fall below in order to address economic issues. The affected cost (S price ) for the end-users is fixed and varies according to standard on-demand cloud storage pricing plan based on reservation contract or prepaid scheme [START_REF] Mazrekaj | Pricing Schemes in Cloud Computing: An Overview[END_REF]. Moreover, the maximum capacity of the storage federation QCmax k is given by the totality of the storage or transfer quota offered by each datacenter. A very important point to consider during the intermediate data placement on the storage federation is a data transfer in or from other federation members. It should be mentioned that in most cloud storage services, the monetary cost of data transfers is more expensive than the data storage itself, so this is an important cost factor that must be considered in our placement problem. Therefore, the transfer cost of data insourcing and data outsourcing are noted IT C k and OT C k respectively. The outsourcing and insourcing storage and transfer costs (OSC k , LSC d , IT C k and OT C k ) are updated using equation ( 1), and depend on their available outsourcing versus local capacity and the minimum effective storage price of each generated intermediate data file.

B. Matrix Model for Intermediate data dependency

As stated earlier, each federated datacenter k receives intermediate data placement requests from multiple home datacenters. When new intermediate data files are generated by an instance workflow, correlations between each file pair are revealed. These correlations correspond to the inter-file dependencies generated from several workflow instances.

Fig. 2 illustrates an example of intermediate data dependency and the corresponding matrix model. Each intermediate data file i has a dependency with itself and with some other file j, so if a set of intermediate data files ID i that has a correlation with file j, this one is also correlated with the same set ID i (symmetry). Therefore, input parameters of the data placement model are dependency values Dep i,j that are collected in matrix DEP .

However, various skewed of dependencies can be specified between the intermediate data during a freak execution of a workflow instance. Scientist users may encounter these errors 1during the execution that causes unnecessary dependencies that need to be adjusted or re-run. Hence, some intermediate data dependencies are not valuable. In order to consider this situation, a float parameter λ j i is defined in the data placement model that denotes scientific users's tolerance of dependency files i and j:

λ j i =
0 no tolerance to process independently i and j 1 otherwise.

(

) 3
The generated intermediate data dependencies with λ j i = 1, operate over a set of I/O requests between each other in the selected datacenter federation. Some operations can be involved in remote access requests: data input adjustment, reprocessing or data re-utilization. For each I/O access, there is a cost noted IOP C i,j . The values of the dependency matrix DEP is dynamically maintained for each set of generated files.

III. EXACT ALGORITHM

In the following we resort to an exact approach called Exact Federation Big Data Workflow Placement algorithm (ExactFed BDWP) which leads to a mathematical programming approach based on an Integer Linear Programming (ILP) model. The ExactFed BDWP algorithm is solved using a branch-and-bound method describing a set of valid inequalities of the big data workflow placement problem cited above and the objective function to optimize under linear constraints. Some of these constraints are obtained according to a practical system in the cloud data placement considering service scenarios and storage capacities. As shown in Fig. 3, the storage selection decision has to lead to the minimum data storage cost in compliance with the intermediate data dependency constraints (one datacenter hosting several intermediate data dependency files) and a maximum storage utilization for the federated datacenters. Hence, we discuss the cost model and the objective function that should be minimized considering the objective and the following procedures for the ExactFed BDWP algorithm execution:

• From a set of input parameters Dep i,j , each federated datacenter exposes its insourcing/outsourcing storage cost based on the Equation (1). • The problem is solved on each workflow instance when a new intermediate data file is generated from a home datacenter. Such cost is assumed to change according to the federation features.

In the following we introduce the use of the integer bivalent variables 0-1 x k id that tells which datacenter d hosts intermediate data file i to be placed in federated datacenter k. A glossary of all the used notations and their descriptions in the proposed exact model is shown in Table 1. Using these notations, the global objective function is given by equation ( 4):

MinCost = d =k idk x k id • size i • (OSC k + IT C k + OT C d ) + d=k idk x k id • size i • LSC k + d =k ijdd k y kk ijdd • Dep ij • λ j i • IOP C i,j (4)
The objective function for the optimal scheduling and placement of intermediate data files can be expressed as the minimization of the cost of transferring, storing data workflow in the federated datacenters, where x k id = 1 is used to indicate that intermediate data file i is transferred and placed in federated datacenter k and x k id = 0 otherwise. Ideally, the ExactFed BDWP algorithm should minimize the I/O transaction cost also when intermediate data dependencies are scheduled separately to different datacenter, i.e. y kk ijdd = 1 and y kk ijdd = 0 otherwise. Objective function ( 4) is subject to several linear and integrity constraints expressed respectively by equations ( 5) to (16):

ik x k id = IDN d ∀d ∈ D; d = k (5) dk x k id = 1 ∀i ∈ ID i ( 6
)
x k id + x k jd = 2 ∀i, j ∈ ID i ; ∀k, d, d ∈ D ( 7
)
x k id + x k jd ≤ 1 ∀i, j ∈ ID i ; ∀k, d, d ∈ D ( 8
)
x k id + x k jd -y kk ijdd ≤ 1 ∀i, j ∈ ID i ; i = j; ∀k, k , d, d ∈ D ( 9
)
kk dd

y kk ijdd ≤ k x k id ∀d ∈ D; ∀i, j ∈ ID i ( 10
)
k =d id x k id • size i ≤ SCF k ∀k ∈ D (11) k=d id x k id • size i ≤ SCL d ∀k ∈ D ( 12
)
k =d id x k id • size i • BCF k ≤ DBmax k ∀k ∈ D (13) id x k id • size i ≤ SCmax k ∀k ∈ D ( 14
) d IDN d = 1 ∀i ∈ ID i ( 15
)
Dep ij = Dep ji ∀i, j ∈ ID i ; ∀Dep ij ∈ DEP (16)
Equation ( 5) expresses the placement constraint in federated datacenters which store the set of ID i files. Equation [START_REF] Yuan | A data placement strategy in scientific cloud workflows[END_REF] indicates that there is only one datacenter hosting file i. Equations ( 7) and ( 8) express dependency constraints that are derived from the values of dependency tolerance λ j i which indicates the splitting (λ j i = 1) or the merging (λ j i = 0) of the intermediate data files during their placement in the federation. Equations ( 9) and [START_REF] Negru | Cost optimization for data storage in public clouds: a user perspective[END_REF] define the relations between bivalent variables x k id , x k jd and y kk ijdd . Equations ( 11) and ( 12) define the storage capacity constraints in each home and federated datacenter whith a limited quota. The data bandwidth quota offered to transfer intermediate data files from or out of the federated datacenter is expressed by equation ( 13) and the aggregation of these quotas cannot exceed the provided maximum bandwidth (DBmax k ). The storage resources limitation of the federation is expressed by equation ( 14). Equation (15) indicates the uniqueness constraint of each generated file i from a single workflow instance. The relation of dependency between files i and j is symmetric and is expressed by equation (16).

IV. PERFORMANCE EVALUATION

A. Setting Parameters

In order to evaluate the proposed model and to show the influence of using federated cloud storage characteristics, a set of simulations have been performed with different input parameters. The evaluation model is performed under AMPL tools and the CPLEX solver2 as an ILP optimization program to solve objective function [START_REF] Bermbach | Cloud federation: Effects of federated compute resources on quality of service and cost[END_REF]. The assessment concerns the cost optimization of intermediate data dependency placement. To create a dynamic environment and unpredictable situations, between 3 and 18 geographically distributed datacenters have been randomly selected from three cloud providers. Among these datacenters, 8 are owned by Amazon S3 (AM) 3 , 4 by Google Cloud Storage (GO) 4 and 6 by Microsoft Azure (MZ) 5 . Each datacenter is restricted by storage and bandwidth capacities ranging in [10GB, 1000GB] and in [1GB,10GB] respectively. Each outsourcing and insourcing demand is composed of a random inter-file dependency ranging in [50*50,1000*1000] organized in a matrix with 1 to 2 GB size per file. Intermediate data files are affected to their home datacenter in a binary matrix (DChome d ). The insourcing/outsourcing storage and transfer monetary costs are given by equation [START_REF] Rebai | Improving profit through cloud federation[END_REF]. The I/O request cost and the affected monetary cost (S price ) for the end-users including OSC k , IT C k and OT C k are set according to the pricing plan for each provider. Table 2 summarizes these different price ranges. For an economic market purpose, the minimum effective price (ME price ) is set randomly and is higher or close to the affected cost for the scientific community (ME price =S price *0,45). The binary value of λ j i is set randomly for each new generated intermediate data file.

B. Compared scenarios & performance metric descriptions

Since previous studies on data placement and cost saving of intermediate data dependencies in cloud storage federation differ and are not sufficiently close to the placement problem (see Sec. V) that our approach deals with (the involvement 5), ( 7), ( 8), ( 11), ( 12) and ( 14). The intermediate data placement is scheduled entirely in each home datacenter thanks to the unlimited storage capacity (no loss). The outsourcing/ insourcing storage costs are obviously not integrated to solve the non-federation scenario considering just local dependencies. In a capacitybased scenario, the federated datacenters (nodes in the cluster Hadoop) randomly select the outsourcing storage to schedule the intermediate data files to the federation members only when their own resources are not available (nodes capabilities). Here, the selection is done arbitrarily to outsource intermediate data files without considering the dependencies (constraints 7 and 8).

We applied the following metrics to analyze the performance of the ExactFed BDWP algorithm with the compared scenarios: (i) Total storage cost: this metric is defined by the objective function computed by equation 4 that measures the cost of transferring, storing and requesting intermediate data files to fulfill the evolving big data workflow requirements. This corresponds to the sum of all defined costs. (ii) Federation utilization: this metric shows the fairness of the intermediate data distribution on a selected datacenter in the federation. It is defined as the ratio between the amount of storage space used by intermediate data placement (both local and federation members) and the maximum amount of storage space for all intermediate data files placement. (iii) Convergence time: this metric measures the execution time of the ExactFed BDWP algorithm in order to assess how fast the algorithm finds a solution to fulfill the intermediate data dependency placement.

C. Simulation results

The total storage cost evaluation of all algorithms through simulations is presented in Fig. 4 and5. The results that show the fair use of the federation are summarized in Fig. 6 capacity-based scenarios while the number of datacenter is set to 9. The simulation results (100$, 137$ and 150$ respectively for the extreme case) correspond to the aggregation of those obtained for each federated datacenter that participate and receive outsourcing and insourcing storage demands of intermediate data placement with varying amounts of files (DEP matrix size variation). The general observation is that ExactFed BDWP algorithm testifies significant cost savings as compared to the benchmark scenarios. As expected, results show that the ExactFed BDWP algorithm outperforms the compared strategies with 27.00 % in average total storage cost saving as regarding to the non-federation scenario and 33.33 % as compared to the capacity-based scenario while matrix size reached 1000*1000 which corresponds to 1000 files of 1 GB. Fig. 5 extends the cost saving evaluation for the ExactFed BDWP algorithm by reporting performance as a function of dependency file pairs size (Dep i,j ) while the number of datacenters is set to 9. With the increase of the size of the matrix and dependency files, the amount of stored intermediate data obviously increases in ExactFed BDWP, non-federation and capacitybased algorithms (100$, 137$ and 150$ respectively), and this influences much more the two scenario comparisons. Admittedly, the cost of insourcing/outsourcing file transfers are not included in the non-federation scenario, whereas the margin between its prices and those offered dynamically in the federation approach impacts on the total cost since the affected monetary costs are not negotiated and consider only local dependencies and take a fixed cost of home datacenter regardless to the price ranges. Similarly, for the capacity-based scenario that exhibits the highest average cost in the both Fig. 4 and5, their costs corresponding to insourcing/outsourcing transfer, storage and I/O cost substantially contrast with the total cost saving. It does not optimize the movement of intermediate data since it places them randomly to the different datacenters until the capacity is full without taking its dependencies in to consideration. Although the number of dependencies increases, the insourcing/outsourcing cost of intermediate data dependencies is minimized in the federation and this influences the total storage cost saving. In addition, the I/O request cost is minimized when there is a dependency tolerance. The very important point in the federation is the placement balancing and fairness between federation members D for the intermediate data distribution. To achieve this, the amount of dependency files is set to 2000 GB (1000*1000 matrix size of 2GB per file) and are placed in federated datacenters and by both comparison scenarios by randomly setting the home datacenters from the providers P . The results are summarized in Fig. 6, 7 and 8 for a number of datacenters set to 6, 10 and 18 respectively those being selected for file placement decisions.

Figures show that the intermediate data placement by the federated datacenters are the most balanced overall from heterogeneous capacities and prices (with the exception of the unlimited capacities for no-federation scenario). The Ex-actFed BDWP algorithm involves 6/6, 9/10 and 15/18 of the selected federated datacenters to schedule dependency files as compared to 4/6, 5/10 and 5/18 for the capacity-based scenario and 3/6, 3/10, 5/18 only for the scenario without federation in Fig. 6, 7 and 8 respectively. In fact, the storage federation contributes greatly and maintains the data distribution balancing among the members whatever the cloud storage provider participating (AM, GO and MZ). Moreover, the negotiated attractive prices influence on the placement decision as each provider tries to offer dynamic pricing that balances the placement decision based on their capacity (equation ( 1)), thus maintaining a collaboration among federated datacenters by fairly placing the intermediate data dependency distribution in the cloud federation, thus reducing the charge for the scientific community. By contrast, as long as the no-federation scenario participates individually, the placement and storage cost is not optimal due to the unlimited capacities (the entire amount of data stored is linear with expected cost) and fixed prices, and consequently the amount of intermediate data dependencies are distributed only in the home datacenters. Fig. 9 and 10 pursue the analysis for the ExactFed BDWP algorithm time execution by reporting performance respectively as a function of the matrix size (1000 GB of intermediate data files) vs. the federation size that ranges in 3 to 18 datacenters, and dependency sizes (obtained from the aggregation of dependency file pairs Dep i,j ) with values 2000, 20000 and 200000 vs. the dependency values (the number of datacenters is fixed to 9 for the results in Fig. 10). As the problem is NP-hard (limitations of the branch-and-bound method), the execution time of the ExactFed BDWP algorithm grows like the matrix size, especially when the number of datacenters is beyond 9 as reported in Fig. 9 (from 2 seconds to 4 minutes for all simulated instances). Fig. 10 illustrates the evaluation results of the influence of dependency constraints (expressed in equation [START_REF] Zhao | A data placement strategy for data-intensive scientific workflows in cloud[END_REF] and equation ( 8)) on the performance of the ExactFed BDWP algorithm. This corresponds to the cases: random default constraints (one could not force different the dependency constraints), splitting constraint (dependency files must be scheduled to different destination datacenters), and strong constraints (dependency files must to be scheduled to the same destination datacenters) while the number of datacenters in the federation is set to 9.

For small number of dependencies (2000) the Ex-actFed BDWP algorithm exhibits close performance irrespective of the constraints. However, for the larger number of dependencies (20000 and 200000), more important differences appear with splitting, default and strong dependency constraints standing out as an intermediate data dependency placement problem whose resolutions are one minute to fifteen minutes. With the splitting constraint, the dependency file placement problem is solved faster as the space of feasible solutions is small (no inter-file dependencies to be considered in the resolution space). By contrast, in the case of strong dependency constraint the placement problem becomes harder to solve as there is a huge inter-file dependency to be considered in the resolution space while satisfying all storage requirements.

V. RELATED WORK

Data storage optimization has received a lot of attention in data workflow systems, and some previous works used the cost model to provide certain features from cloud storage scenario. Works presented in [START_REF] Yuan | A data dependency based strategy for intermediate data storage in scientific cloud workflow systems[END_REF], [START_REF] Yuan | A data placement strategy in scientific cloud workflows[END_REF], [START_REF] Zhao | A data placement strategy for data-intensive scientific workflows in cloud[END_REF] are closer to our focusing problem. Authors in [START_REF] Yuan | A data dependency based strategy for intermediate data storage in scientific cloud workflow systems[END_REF] elaborated a cost-effective strategy for storing intermediate data workflow in a single cloud storage provider using the Amazon-based fixed pricing. The proposed model focuses on running a scientific workflow system in a cloud and automatically deciding whether intermediate dataset should be stored or deleted in the cloud storage provider considering user's tolerance of computation delays. Since our allocation strategy does not remove intermediate data sets before the end of the workflow system processing, as shown in the introduction, all intermediate data sets are stored in multi-datacenters involving a cloud storage federation with a cost storage optimization. This differs considerably from [START_REF] Yuan | A data dependency based strategy for intermediate data storage in scientific cloud workflow systems[END_REF] which cannot be compared to our model. Authors in [START_REF] Yuan | A data placement strategy in scientific cloud workflows[END_REF] present a matrix-based k-means clustering strategy for data placement in scientific cloud workflow. The authors stress the movement of large volumes of data that can automatically be allocated among datacenters based on the data dependencies. The optimization is done only at the data movement level and the authors did not defined a storage cost optimization during the intermediate data placement which differs from our approach. In addition, our approach takes into account the type of dependency in order to further optimize the data movement and storage cost. Authors in [START_REF] Zhao | A data placement strategy for data-intensive scientific workflows in cloud[END_REF] are closer to the work in [START_REF] Yuan | A data placement strategy in scientific cloud workflows[END_REF]. The authors present a data placement strategy based on data dependency clustering for scientific workflow in a heterogeneous cloud. The storage cost in this work is not considered and authors focus on the cost of the data redistribution only.

A cloud storage cost-based selection decision is presented in [START_REF] Ruiz-Alvarez | A model and decision procedure for data storage in cloud computing[END_REF], [START_REF] Agarwala | iCostale: adaptive cost optimization for storage clouds[END_REF], [START_REF] Negru | Cost optimization for data storage in public clouds: a user perspective[END_REF]. Authors in [START_REF] Ruiz-Alvarez | A model and decision procedure for data storage in cloud computing[END_REF] propose an optimization problem for selecting the best storage services and they take into consideration application requirements and user priorities. A total storage cost is solved and workload requirements are not addressed in this work. In [START_REF] Agarwala | iCostale: adaptive cost optimization for storage clouds[END_REF], the authors identify an adaptivecost optimization system for multi-cloud storage to decrease the storage cost, a compression and placement algorithm are used to reduce data cost in cloud storage. However, the data correlation is not considered on the optimization cost model. Moreover, our approach highlights the trade-off between cost and data dependencies that is not the goal in [START_REF] Agarwala | iCostale: adaptive cost optimization for storage clouds[END_REF]. The authors in [START_REF] Agarwala | iCostale: adaptive cost optimization for storage clouds[END_REF] reduce the storage cost by compressing data and choose the lowest provider price. Intuitively, their cost model is a little more economical compared to our (by reducing the data size), but the fact of compressing data does not allow to deal with the dependencies in each storage providers which is a disadvantage for our optimization case. But the use of the federation storage price can balance considerably the costs of our optimization model in our favor in some input parameters. Authors in [START_REF] Negru | Cost optimization for data storage in public clouds: a user perspective[END_REF] propose a storage-cost optimization based on linear programming model using multiple public cloud storage providers. This work differs from our model since the type of data and collaboration-aspect-based cloud providers are not envisaged. Moreover, objectives of their optimization model are not exactly similar to ours, one of which is latency and the exclusion of communication cost between providers.

Since our optimization goal differs considerably from that of [START_REF] Yuan | A data dependency based strategy for intermediate data storage in scientific cloud workflow systems[END_REF], [START_REF] Yuan | A data placement strategy in scientific cloud workflows[END_REF], [START_REF] Zhao | A data placement strategy for data-intensive scientific workflows in cloud[END_REF], [START_REF] Ruiz-Alvarez | A model and decision procedure for data storage in cloud computing[END_REF], [START_REF] Agarwala | iCostale: adaptive cost optimization for storage clouds[END_REF], [START_REF] Negru | Cost optimization for data storage in public clouds: a user perspective[END_REF], [START_REF] Negru | Cost optimization for data storage in public clouds: a user perspective[END_REF] it is not advisable to compare performance to these approaches.

VI. CONCLUSION

The present work introduced intermediate data placement cost saving solution through a collaborative cloud storage environment. An exact federation algorithm (ExactFed BDWP) based on an integer linear programming (ILP) model and the branch-and-bound method has been proposed to solve the problem of the inter-file placement (symmetric dependencies) that takes into account the storage federation characteristics. The ExactFed BDWP schedules and places fairly intermediate data files taking into account their dependency requirements, size and cost saving over distributed datacenter. A binary symmetric matrix is defined to represent the dependency for each pair of generated file in the same matrix, and home datacenter hosting matrix are used to outsource intermediate data storage to the federation. The ExactFed BDWP algorithm was tested and evaluated by the way of simulations on a set of data files generated randomly with two different scenarios. An effective and optimal solution in terms of total storage cost saving was shown by the ExactFed BDW algorithm against the other scenarios. The execution time of the ExactFed BDWP algorithm increases with the size of the dependency matrix, and the number of datacenters involved in the federation. However, for a realistic numbers of federated datacenters, the ExactFed BDWP algorithm remains fast and achieves optimal scheduling and placement of intermediate data dependency.

The convergence time of ExactFed BDWP algorithm also matters in terms of swift response to additional data placement since some workflow applications require the placement of large data sets corresponding to a very complex dependency between a set of data files (asymmetric). This can thus put very stringent requirements on extended data placement, thus we plan in future work to consider a dependency types between a set of intermediate data.

  illustrates a cooperation of cloud storage providers noted by P which involves federated datacenters D that are geographically distributed providing online mass storage to the scientific community collaborations to schedule a set of intermediate data noted by ID i . i being a single file with its respective size noted size i , d and d' indicate home datacenters where intermediate data are generated by a workflow instance and are temporarily stored.

Fig. 1 .

 1 Fig. 1. Federated cloud datacenters scenario. The set of federated datacenters D are aggregated and interconnected in the form of an inter-cloud. They use native peer-to-peer communication link to shift intermediate results from a busy disk entity to those with an available capacity, and efficiently use storage resources of the datacenters and balance intermediate data placement requirements. The bandwidth resource of each link is used to transfer the intermediate data files between federated datacenter noted k. The intermediate data placement decision on one or multiple locations of the cloud storage federation involves the use of datacenters converged in local and storage federation for data re-utilization internally or by the other collaborative users. After the workflow instances have generated intermediate data locally noted IDN d on each datacenter, a memory temporary stores these intermediate data results before their scheduling and placement decision.The set of considered datacenters can be modeled as a matrix noted DChome d which combines a home datacenter in order to emphasize the origin of the generated data. For the sake of convenience, we here only consider storage and data transfer resources of D as computing resource provisioning is similar to that of the storage resource. Hence, to underscore the limited amount of available storage space on each federated datacenter k, each member in the federation knows the storage resource quotas that is capped at SCF k offered by the other

Fig. 2 .

 2 Fig. 2. Intermediate data dependency matrix

  Finding the optimal placement requires the computation of storage cost for each possible instance solution from each intermediate data placement request (input parameters) on a federated and local datacenters. The federated datacenters use a cost model and the ExactFed BDWP algorithm according to the data dependency to schedule their intermediate data placement requests to the storage federation.

Fig. 3 .

 3 Fig. 3. Overview of the data storage cost federation approach: input/output parameters.

  , 7 and 8 Finally, Fig. 9 and 10 depict the execution time of the ExactFed BDWP algorithm.

Fig. 4

 4 Fig. 4 depicts the results of minimizing the total storage cost of the ExactFed BDWP algorithm with no-federation and

Fig. 4 .

 4 Fig. 4. The results of the optimal total storage cost regarding to dependency matrix size variation (DEP ).

Fig. 5 .

 5 Fig. 5. Results of the optimal total storage cost regarding to the number of dependency file pairs (Dep i,j ).

Fig. 6 .

 6 Fig. 6. Intermediate data distribution results for 6 federated datacenters.

Fig. 7 .

 7 Fig. 7. Intermediate data distribution results for 10 federated datacenters.

Fig. 8 .

 8 Fig. 8. Intermediate data distribution results for 18 federated datacenters.

Fig. 9 .

 9 Fig. 9. Execution time of the ExactFed BDWP algorithm by varying the number of federated datacenters.

Fig. 10 .

 10 Fig. 10. Execution time of the ExactFed BDWP algorithm with the different dependency parameters values λ j i while the number of datacenters is fixed to 9.

  Sec. III derives an exact optimization approach for allocating intermediate data on federated Cloud storage. Sec. IV shows and discusses the simulation results obtained with compared scenarios. Related work is reviewed in Sec. V. Finally, Sec. VI concludes the paper and presents some future work.

2017 IEEE 19th International Conference on High Performance Computing and Communications; IEEE 15th International Conference on Smart City; IEEE 3rd International Conference on Data Science and Systems 978-1-5386-2588-0/17 $31.00 © 2017 IEEE DOI 10.1109/HPCC-SmartCity-DSS.2017.67

with target assumptions.

TABLE I .

 I STORAGE COST FEDERATION MODEL: PARAMETERS &DECISION VARIABLES.

	Parameters	Descriptions
	P	Set of cloud storage providers
	D	Set of federated datacenters
	d, d , k, k	Designate home datacenters d and d' and federated datacenters k
		and k'
	Depi,j	Dependency coefficient value of intermediate data dependency as
		input to the model to designate affinity between files i and j
	DEP	Dependency matrix including all dependency values of Depi,j
	DChomed A home datacenter matrix which brings home datacenter d to
		generate intermediate data files
	LSCd	Local storage cost (dollar per GB) of cloud storage provider hosting
		datacenter d
	OSCk	Outsourcing storage cost (dollar per GB) of cloud storage provider
		hosting datacenter k
	sizei	Size of intermediate data file i
	λ j i	Scientific users's tolerance of intermediate data dependencies of i and j which is a binary value, λ j i = 0 indicates scientific users have no tolerance to process independently i, j, and 1 otherwise.
	i	Single intermediate data file
	IDi	Set of intermediate data files
	IDNd	Number of intermediate data generated and stored temporary in
		datacenter d
	IT Ck	Insourcing transfer cost (dollar per GB) proposed by provider to
		transfer data from home datacenter k
	OT Ck	Outsourcing transfer cost (dollar per GB) proposed by a provider
		to transfer data to another datacenter k'
	IOP Ci,j	Cost of I/O requests (dollar per operation) of intermediate data files
		i and j on federated datacenters
	SCFk	Storage space quotas offered and shared (GB per month) by each
		datacenter k in the storage federation
	SCmaxk	Maximum storage space quotas (GB per month) offered and shared
		by all the federated datacenters
	SCLd	Storage space quotas (GB per month) available by each home
		datacenter d
	BCFk	Data bandwidth (per data unit) quota offered and shared between
		home datacenter and destination datacenter to insource and out-
		source intermediate data storage
	DBmaxk	Maximum data bandwidth (GB) quotas provided among federated
		datacenters
	Variables	Definitions
	x k id	A binary variable, x k id = 1 if intermediate data file i is sched-uled from datacenter storage d to outsourced datacenter k, and 0
		otherwise.
	y kk ijdd	A binary variable, y kk ijdd = x k id * x k jd
	• The ExactFed BDWP algorithm keeps the interme-
	diate data dependencies in a single datacenter while
	saving their storage, transfer and transaction costs. The
	intermediate data with dependency tolerance should
	be optimized according to the cost of I/O-demands
	requested among federated datacenters.

TABLE II

 II

		.	STORAGE PRICES OF THE THREE CLOUD STORAGE
				PROVIDERS.
	Prices	I/O	cost	Storage	Data	Data transfer
	--	($/10000	($/GB)	transfer	OUT ($/GB)
	Cloud	operations)		IN
	storage				($/GB)
	providers
	Amazon	[0.005-0.01]	[0.004-0.04]	[0]	[0-0.25]
	S3
	Google	[0-0.005]	[0.007-0.023]	[0]	[0.08-0.23]
	Cloud
	Storage
	Microsoft	[0.015-0.0345]	[0.08-0.125]	[0]	[0-0.181]
	Azure
	of intermediate data dependencies at the lowest cost in the
	federated placement), we resort to a comparison with two
	following strategies: no-federation aspect on the one hand and
	a capacity-based placement strategy used in default Hadoop
	implementation on the other hand. Datacenters in the non-
	federation scenario turn in an autonomous way and depend on
	their own storage space resources to place intermediate data
	files. To elaborate this scenario, a relaxation of the ILP was
	built and consists in eliminating constraints (

It can be an information about a replacement task, integrating a new input data that improves the reliability of the workflow instance execution.Accordingly, between each pair of files (i, j), value λ j i parameter is defined. The amount and the size of intermediate data dependencies feeds the expected storage cost when scheduling intermediate data IDi on the federated locations. These later collaborate by sharing their respective storage resources and dynamically adjust their hosting capacities according to their intermediate data placement requirements.

http://ampl.com/products/solvers/solvers-we-sell/cplex/

https://aws.amazon.com/fr/s3/pricing/

https://cloud.google.com/storage/pricing

https://azure.microsoft.com/fr-fr/pricing/details/storage/


domain: shs info

text:
Marc M Anderson

Karën Fort

Evaluating the Acceptability of Ethical Recommendations in Industry 4.0: An Ethics by Design Approach

Keywords: artificial intelligence, ethics, heavy industry, Industry 4.0, operationalizing

de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

AI Ethics in general suffers from a lack of operationalization. [START_REF] Morley | Ethics as a service: a pragmatic operationalisation of AI ethics[END_REF] have highlighted a number of reasons which might lie behind what they call "a significant gap … between theory and practice within the AI ethics field." Meanwhile, various approaches have been taken to address this gap, for example those summarized by [START_REF] Prem | From ethical AI frameworks to tools: a review of approaches[END_REF], but nearly all remain either at a very general level, or if they engage an operational level they remain very much within the range of technical adjustments to the algorithm or data, i.e. they disconnect an engagement of ethical concerns from the humans involved, for example the users or the software engineers working for the technology companies. 3There are three motivations behind this research. First, operationalizing AI ethics depends upon finding methods to implement ethics at a ground level, but a large part of implementation involves the willingness of software engineers to actually implement the methods. There are few studies which explore, as we do here, the question of what software engineers, developing AI systems in technology companies, are ready to put in place with regard to AI ethics. [START_REF] Widder | Dislocated Accountabilities in the AI Supply Chain: Modularity and Developers' Notions of Responsibility[END_REF] provide strong general insights into this question, but do not go as far as a quantifiable level for operationalization. Secondly, the issue of the potential of implementing AI ethics at operational levels as it overlaps with general ethical implementation concerning the humans and human contexts affected by the AI, has not been explored, either in terms of categorization or quantifiability, a deficiency which • 1 marc.anderson@inria.fr; dr.marcanderson@gmail.com LORIA, UMR 7503, Université de Lorraine, Inria and CNRS, Campus Scientifique, 615 Rue du Jardin-Botanique, 54506 Vandoeuvre-lès-Nancy, France

• 2 karen.fort@loria.fr Sorbonne Université/LORIA, UMR 7503, Université de Lorraine, Inria and CNRS, Campus Scientifique, 615 Rue du Jardin-Botanique, 54506 Vandoeuvre-lès-Nancy, France we try to redress. Thirdly, and closely related, the question of whether there is an actionable typology of AI Ethics categories for operational level implementation which is more fine-grained and includes the overlap described above, deserves study.

Beginning from an approach of giving operational level ethical recommendations, our interest has been focused on knowing not merely how many ethical recommendations have been achieved but what types of recommendations are more or less likely to be implemented. Thus, a categorization of recommendations was required, but a bottom up categorization, i.e. one which grew out of the specific issues we encounteredin their human contextand which could accommodate those issues, rather than one which discarded issues which didn't fit with a pre-developed categorization. This would help understand how the industrial partners and tech company partners see the ethical angle, help make suggestions or advance potential future lines of research to improve implementation of recommendations in poorly implemented categories, and apply insights from some categories toward better implementation of recommendations in others.

In this article we thus present a quantitative review of the general and categorized results of direct ethical recommendations given to partners of an EU Horizons 2020 project. In Part 2, we begin with an overview of project context and timeline, the methodology adopted, and our approach for assessing the implementation result status of recommendations. In Part 3, we describe our category definitions along with examples, and our categorization process for recommendations. Part 4 presents, in various figures, implementation results relative to recommendations, in terms of overall results for the project as a whole, then by category, and finally broken down both by category and implementing partner, followed up, in each case, by observations and discussion.

Both the ethics team assessment and the project partners assessment are given and discussed. Part 5 discusses limitations with regard to industrial technical contexts, while Part 6 offers some suggestions for future research regarding less implemented recommendation categories. Finally, we conclude with some reflections on interesting aspects of our findings.

To the best of our knowledge our approach to quantifying the implementation results of specific ethical recommendations is entirely new. In other words, there is no 'state of the art' with regard to the approach we are describing. Even on the subject of embedded and practical approaches to applying ethics of AI in industrial settings the existing research literature is sparse verging on non-existent. Much of the literature, such as [START_REF] Tahaei | Toward Human-Centered Responsible Artificial Intelligence: A Review of CHI Research and Industry Toolkits[END_REF] consists of extremely broad overviews, which assume a generic notion of industry practically unsuited to our aims, and which engage or summarize ethics under the usual few abstract headings such as explainability and fairness. Research which does address heavy industry e.g. [START_REF] Singh | Identifying issues in adoption of AI practices in construction supply chains: towards managing sustainability[END_REF]) also tends to remain at an abstract level, or as with [START_REF] Ciobanu | AI Ethics of Industry 5.0 -From Principles to Practice[END_REF] it remains at both an abstract and a technical level.

The publication of (Anderson and Fort, 2022), who develop a bottom up approach for ethics of AI in industry and that of [START_REF] Berrah | Industrial performance: an evolution incorporating ethics in the context of industry 4.0[END_REF]) who develop a system for integrating ethics as a dimension of performance evaluation, while admitting that quantifiable performance indicators for ethical integration are difficult to attain, are the nearest to our approach.

Project Overview & Methodology

The project began in 2020, with the aim of researching various aspects of AI integration in heavy industry settings in three manufacturing sites located in France, Belgium, and Germany.

Ethics by design had to be scaled up gradually since at the beginning there was only the initial project proposal to go by. The first opportunity for consideration of the project's ethical issues was thus offered in the preliminary stage, approximately a three-month period, in which the contexts of the possible Use Cases (hereafter UC) were presented by the manufacturing partners, question and answer sessions were held, and certain UCs were selected to go forward in the project.

At this stage of the project the aim was to develop a baseline of the current situation of the humans involved in each UCprimarily the shop floor level operators or maintenance technicianswithin their work environment, in relation to the other workers, machines, and automated systems already in place, tasks being done, time to carry out those tasks, and physical spaces.

A set of baseline questions were asked to flesh out these contexts. Then an initial summary of ethical interest in each UC was made, in keeping with a call to the other partners to summarize their interest in each UC in terms of what problems they had interest in and competence in engaging. On its part, the ethics team summarized potential ethical difficulties in the UCs and rated each of them in terms of these difficulties. The first ethics team recommendationsor pre-recommendationswere thus appraisals of each UC at three levels of difficulty: high, medium, or low in terms of ethical issues, with an outline of why each was rated as it was. The rating was arrived at by an appraisal of what was proposed for the AI, in comparison with the fleshed out initial context of the UC.

The ethics team did not specifically recommend not choosing certain UCs, but made clear which were the most problematic in terms of ethical issues. Subsequently, on that basis, the partnersthe ethics team was not part of the decision committeedecided to eliminate the ethically most problematic UC from the project. The ethics team's reasons for rating this UC as high in terms of ethical issues, was, among others: the operator had very little time to act or react so that AI suggestions might add stress to an already small time frame, and the human agency aspect was ambiguous. The UC was abandoned even though its solution, which had been envisioned as a way to increase efficiency and eliminate downtime in the initial stage of the production process in question, was the stated top priority of one of the industrial partners. Other UCs were also eliminated from consideration for other reasons, e.g. the tech company partners did not find the problems within them to fit with their technical competencies or interests.

The ethics team considers the elimination of the ethically problematic UC, after discussion, to be a major success, and a full implementation, but could not include it in the quantified results because it was different in kind and unique in terms of quantification. But it is important to mention it here because it belongs to that category of actions centered around the often-forgotten or dismissed question, highlighted by [START_REF] Hagendorff | Blind spots in AI ethics[END_REF], of whether in certain contexts AI systems should be used at all. This question is rarely discussed in AI ethics, but here an ethical appraisal which resulted in not using the technology was followed by the partners in a significant way.

Recommendations

After the initial UC selection, the embedded ethicist participated in the ongoing design and technical meetings of the project, which is estimated at about 50 to 60 meetings per year for each of the first two years. At the same time, the ethics team participated in the development of the work package tasks of the project as well as in reviewing and contributing to the related deliverables. Thus, the initial sets of recommendations were specified according to a particular UCapproximately the first 7 months of the projectand were concerned with the initial solution plans for each UC relative to its work and worker context. Later recommendations tended to be specified according to a particular UC context within a task. Since the latter were dealing with written deliverables as well as design plans discussed in meetings, the recommendations regularly concerned the planned solution described in the deliverable and sometimes they also concerned the wording or content of the deliverable itself. The recommendations thus tended to become more precise as the AI service solution became more developed. In this way recommendations evolved, were integrated with one another, and gave way to new recommendations as solutions advanced and changed. This approach is only possible if there is really an ethics by design in practice, rather than merely in theory.

A total of 130 recommendations have been issued to date. The recommendations were given formally, i.e. by means of a written document which described the UC context, reviewed the ethical issues in question relative to the context, and gave precise numbered recommendations to address the ethical issues. The following are some anonymized examples of actual recommendations, with other examples provided below in Table 2.1 according to category.

• x) Recommend that you formally clarify to the operator if the operator's role changes with regard to checking tread alignment, e.g. no longer has to check or checks less frequently. If the operators will no longer check, clarify this to the TBM operators.

• x.x-x) (Partner X) Regarding labeling of good and bad images for alarm: Recommend that you clarify who will label the images and estimate how many images need to be labelled and how long it will take.

• x.x-x) (All) Recommend that for Task X.X Use Cases where explicit and implicit feedback will be combined (xxxx_2; xxxx_5), you develop a best practice of tagging the data resulting from that feedback to indicate that active operator choices (explicit feedback) make up part of it.

The ethical recommendations documents were dated and disseminated to the designated ethical contacts for each project partner as well as other involved partner members, and they were deposited on the project Microsoft Teams ethics channel files, in conformity with the practice adopted for other work within the project. The recommendations were also discussed in technical meetings, both as the ethics team became aware of them and as the technical solutions progressed. Sometimes a partner asked for special meetings to discuss potential ethical issues. Additional recommendations or additional aspect of recommendations might then be added.

Assessment

The Ethics team kept track of the state of implementation of each recommendation as tasks, deliverables, and UCs progressed. They also outlined what a full implementation would consist of. Based on this ongoing assessment, and in the interest of keeping the complexity of the approach within reason, it was decided to put the results of implementation under three categories. Fully implemented indicates that a recommendation was fully implemented according to what we expected, or has been committed to with evidence that it will be by project end. Partially implemented indicates that some effort was made toward implementation, or in the case of recommendations with several aspects, one or more but not all aspects were implemented. Not implemented indicates that no evident or proven effort was made for that recommendation.

The default assumption is thus negative tending. A recommendation is considered as not implemented unless at least some evident effort was or is being made toward implementation, and is considered partially implemented unless all aspects are evidently implemented or in process of implementation.

In addition, 9 recommendations were classed as NA (not applicable), by the ethics team, because changes in a proposed solutionoften independent of ethical recommendationsrendered them irrelevant.4 These 9 have not been included in the total, thus reducing it to 121.

Clearly the approach is imperfect in terms of being exact, because unless the ethics team waits until the project is complete there is still some small chance that all recommendations could be implemented. On the other hand, in most cases the time for implementation is practically passed in the project. Thus, a certain pragmatic imperfection has to be accepted. But this imperfection aligns with the project ethics team's embedded ethics by design approach, whose aimsin contrast with an all or none approachare that 1) better some ethics, as much as we can get, be actually implemented than none at all, and 2) a demonstration of ethics by design starting from a ground up approach and relying on very specific recommendations to change the design path of a technology development project be shown to produce some results which can be improved in later efforts, and 3) insights from the approach be gathered and suggestions be made toward bettering future efforts in ground up applied ethics of AI and technology.

3 Categorization of Ethical Recommendations5

Definitions of Categories

Recommendation Category Definition Protocol

Adopt a specific set of instructions regarding errors, new tasks, etc.

Example: ETH X UC X ID X -Recommend that you develop a protocol to address the allowed for 5 to 10% of cases when the AI misrecognizes text, i.e. state the steps the operator will take in case of AI error.

Human centering

Tailor aspects of development to individual users and develop services collaboratively with users (e.g. work with the operator to design something)

Example: ETH ID X.

Methodology

Manual annotation, in this case categorization, is not about measuring a physical reality (such as the height of Mont Blanc), but about quantifying a phenomenon [START_REF] Desrosières | Pour une sociologie historique de la quantification : L'Argument statistique[END_REF], which implies agreeing beforehand on conventions of equivalence: for example, in order to count unemployed people, there first has to be agreement on what defines unemployment.

These conventions should be then documented, for example in annotation guidelines, and the consensus should be measured, using inter-annotator agreement metrics [START_REF] Artstein | Inter-coder agreement for computational linguistics[END_REF]. This methodology was used to categorize the ethical recommendations of the project.

Process and Results

One of the authors categorized the recommendations already made at that point in the project -120 total -into 15 categories (annotation #1). They wrote annotation guidelines, defining precisely each category. The second author then read the guidelines and, without consulting the first round of annotations, did their own (annotation #2). They disagreed on the categories for 60 recommendations (50% of the cases), fully agreed in 38 cases (32%) and hesitated between the annotation #1 and another category in 22 cases (18%). The strict observed agreement is therefore 31.66%. If we consider the ambiguous cases as part of the agreement, we reach 50%.

These results show that the categories needed to be reviewed. First some definitions in the guidelines such as the one for 'Timeliness', were improved (the emphasis on order of implementation was removed and the category was defined fully in terms of getting another recommendation implemented 'earlier'), then some categories were merged (e.g. 'Human centering' with 'Feedback,' and 'Training' with 'Adaptation') 6 and new ones added, such as 'GDPR.' Finally, all the recommendations were reviewed one last time, with each agreement case discussed and a consensus decision made on them.

Four categories were assigned to at least 15 recommendations: Human centering, Design, Responsibility and Workload. On the other end of the spectrum, four categories were used less than five times (four times for each): Evaluation, Training, GDPR and Verify effects. These results are satisfying, as there is no prevalence of one category in particular and no useless categories either.

It should be noted that new recommendations were added in the meantime and that 130 recommendations have now been made in total at this point of the project, with 9 of these being re-categorized as NA, as explained above.

Implementation Results of Recommendations

Comments on Methodology

The ethics team assessment of the results was an ongoing process, in which partner moves toward implementation were recorded as they arose, from about the sixth month of the project onwards. A final review of the status of all recommendations was then carried out prior to writing. We then presented all partners with an online worksheet of all recommendations and our assessment of their status, with instructions to give their own assessment of the implementation status of recommendations. If they agreed with the ethics team assessment they need make no changes. If they disagreed with our assessment, they could upgrade or downgrade, and give reasons for the change, as well as reasons why the recommendation was not implemented. After a five week limit, the data was gathered from the partner assessment for comparison with our own assessment. That data is presented in the Figures displaying partner assessment result. It can be considered as the independent partner assessment of the implementation status of recommendations, although in many respects it agrees with the ethics team assessment.

Note also that the ethics team did not directly push for implementation of the recommendations. In other words, the ethics team did not engage in 'applied moralizing'. The team's role was viewed as that of giving the recommendations and the reasoning behind them, discussing them in meetings, being available for any partners who needed ethical advice, and occasionally inquiring whether particular recommendations had been addressed for purposes of keeping track of results.

Overall results

In Fig. 1 and Fig. 2 below, the overall results as a proportion of the recommendations which were kept, are indicated, for each of the three outcomes: fully implemented, partially implemented, not implemented. 7

7 Note that the overall results as assessed by the Project Partners are based upon 108 recommendations kept and 22 rendered NA (as opposed to 121 kept and 9 NA for the ethics team). This difference is due to project partners deciding to formally abandon one UC fairly late in the project, thus rendering all of its recommendations NA for the project partner assessment. The ethics team, however, decided to retain their assessment of the 13 recommendations of the UC, since that assessment had been completed before the formal abandonment occurred.

Observations Regarding Overall Results

As observed in Fig. 1, overall, 36% of the recommendations were implemented fully according to the ethics team assessment, with a further 33% partially implemented. 31% of the recommendations remained unimplemented.

As the project is not completely finished as of writing, there is a marginal possibility that some of the implementation results may change. This possibility is taken into account by the project partners in particular, as can be seen in the somewhat more optimistic overall assessment of implementation results presented in Fig 2 .  For the partners, 52% of the recommendations were fully implemented, 25% were partially implemented, and 23% remained un-implemented.

Discussion of Overall Results

The overall results show that ethics can be operationalized in the industrial AI context at ground level through the approach of embedding the ethicist in the technological development process. The difference between the rate of success as assessed by the ethics team and that assessed by the partners is also quite consistent. We did not for example, find the partners simply re-assessing the results so as to upgrade everything to fully implemented, which they were free to do. In fact, some partners even downgraded some results from full to partial, or from partial to not implemented. For example, deliverable commitments in one UC to a training and observation phase for process engineers with regard to digital twins, which the ethics team assessed as partial fulfillment, were downgraded to no by the respective industrial partner, and clarifications regarding AI model training input by industrial partners which the ethics team had accepted as full implementation, were downgraded to partial by one of the tech company partners.

Moreover, the upgrade toward fully implemented in the partner assessment of Overall Results from 36% to 52% drew equally from the partially and not implemented portions. In many cases, as indicated by the partners to us, the upgrade of certain recommendations to a full or partial implementation result is based upon the partner(s) insistence that full implementation will be and can only be achieved at the final deployment phase of AI services. The ethics team is confident that the results presented in our own assessment give an accurate picture of what has been achieved, but by including the partner assessment, there is room for arguing that the more objective figure lies somewhere between the two assessments. One could still question whether this rate of success for ethics is practical however. There are at least two responses to that question. First, that in a purely ethical sense practicality is arguably complex and unique in comparison with other domains, and second, that if being practical is to be measured in the same sense as quantitative evaluations of technical success, then the expectations for improvement should remain comparable to those for technical success.

So, in the first place, regardless of numbers, the approach demonstrates a practicality appropriate to ethics as historically understood, in terms of success in getting the partners of the project to consider ethics at a specific level of generalityhere the shop floor and tech design leveland in provoking ethical responses to the recommendations. Tech company partners and industrial partners are prompted and habituated to some degree, to think ethically, i.e. to expand their thinking beyond the narrowness of the purely technological solution, an outlook which one hopes will remain with them beyond the project. One can also begin to get some idea of what kinds of categories of recommendations are welcomed (e.g. Human Centering), or avoided (e.g. Evaluation), and thus of where to direct future reflection and then develop a practical approach to address those which are avoided.

In the second place, if the evaluation of technical achievements and the initial goals of the project are taken as a guide, the project aims for efficiency improvements under various technical categories which range between 1-9%. Thus, if the approach is to be measured by similar standards, it is fair to say that the total outcomes of 36% fully implemented and 33% partially implemented is at least as good as the project technical outcomes. Industrial process and manufacturing automation typically envision modest gains of 10-25% as practical, e.g. in efficiency (ZVEI, 2012). They do not aim at wholesale transformations of manufacturing processes. It would not be fair to ask more of a practical ethical approach insofar as it is held to the same quantitative standards and applied in similar contexts.

From a pragmatic ethical viewpoint, this suggests that the above two paradigms need not be exclusive. Ethics can and should have its quantitative tending aspect coexisting usefully alongside its more ideal and qualitative aspect. The former gradually builds into the latter, and looks to the latter as a simple map to keep oriented.

Results by Category

In Fig. 3 and Fig 4 . below, the results according to category are given, for each of the three outcomes, as a proportion of the total recommendations under that category.

Fig. 3 Results of Ethical Recommendations by Category as Assessed by Ethics Team

In Fig. 5 and Fig 6 . below, the same information is given in chart form, with the categories most fully implemented as a percentage of the total recommendations in the category in descending order from left to right, with higher total recommendations for a category deciding ties in percentage, e.g. in Fig. 5, Responsibility has 21 recommendations compared to only 3 for Valorize Experience, even though both are at 33%.

Observations Regarding Results by Category

Although somewhat offset by the fact that the number of recommendations for the categories in question are relatively small, the results by category, Fig. 3, contain two clear extremes. All recommendations for Ethical rewording and for De-anthropomorphization of AI descriptions within deliverables were carried out. No recommendations under the category of Evaluation were carried out. This did not change in the partner assessment, as seen in Fig. 4.

In the ethics team assessment, in each of the three categories with the largest total number of recommendations, well over half the recommendations in each category were either fully or partially implemented. In descending order of number of recommendations, these were Human Centering, followed by Responsibility, and then Design. The partner assessment did not differ substantially, Fig. 4, except for the category of Human Centering, where many implementations judged to be partial by the ethics team, were upgraded to fully implemented.

Discussion of Results by Category

In the results separated by category, what seems clear is that the recommendations which are most difficult to get implemented are those which aim to set user focused quantitative benchmarks for error and reliability -Evaluationand recommendations to quantitatively estimate various aspects of new tasks to be given to the worker -Workload.

The reasons behind the poor implementation of Evaluation type recommendations may be that setting a benchmark can be taken as implying the possibility that the system is not, and may never be, reliable enough to use ultimately, and so we should not use it. Such an implication tends against the notion that tech can and should always find a solution, a notion deeply ingrained in software engineers and one which works against ethical engagement, as [START_REF] Avnoon | Contextualizing the ethics of algorithms: A socioprofessional approach[END_REF] and [START_REF] Clark | In)accessibility and the technocratic library: Addressing institutional failures in library adoption of emerging technologies[END_REF] argue.

If estimating quantitative figures in advance seems to be difficult and avoided, nonetheless it is clearly not impossible, since the project partners did it for recommendations under the best implemented larger categories of Human Centering, Design, and Responsibility. That quantitative evaluation is avoided is also counter-intuitive given the already noted software engineer urge to find quantitative and technical solutions. It seems then, that the root of the problem for recommendations of the Evaluation type does indeed lie in the ingrained notion that the technology can always succeed eventually, and conversely that it cannot fail or be inappropriate to a context. The relative failure of the Workload recommendations in the project context seems more related to work and its historical and institutional frameworks as such, than in the attitudes of technology companies, even though it has a definite crossover with the above issue of publicizing problems with technology in quantitative terms.

Asking for workload estimates tends, in itself, to create more work to some degree, and it may not be easy to carry out such quantitative estimates. Both of these are issues that appear to be soluble however.

More problematic is that workload recommendations bring attention to the fact that developing and integrating a technology is often just adding more work somewhere else, [START_REF] Crawford | The atlas of AI: Power, politics, and the planetary costs of artificial intelligence[END_REF] and sometimes work that is never factored in, even though the overall stated goal is generally to adopt the technology to lessen the workload. If it turns out that in some cases there is merely a shifting of workload to others areas where it is not accounted for, then of course this practically contradicts the stated aim.

It does not actually contradict it however, unless formally acknowledgedan acknowledgement that many of the ethics team's Workload recommendations attempted to getso that it is known in a manner such that the shop floor worker also becomes aware of it. Then it also tends toward requiring some justification and legal adjustment in relation to work contracts.

In support of this explanation, in the several instances in the project where the Workload recommendations were followed there was a tendency for the project partners to take on the extra workload themselves, or shift it from shop floor level to management. This is a relatively good result in view of operationalizing ethics, in the sense that bringing the extra workload out in the open with recommendations toward quantitative estimations, is something applied ethics can help to tackle, and it is at least a neutral result in terms of the worker. But it is not a satisfying result. A satisfying result would require a public and honest rethink on the reasons we are deploying technologies, and one which proceeds from the given context.

It also illustrates a significant problem faced in operationalizing ethics in industry and beyond: that law and ethics have a difficult relationship and what is legal may not be ethical. To impose upon a worker in a contract, may not be ethical in terms of the contradictions involved with regard to the stated aims for technology adoption.

It may be perfectly legal however, or legal because unchallenged. In that case, the legal aspect will tend to win locally, unless and until the ethical aspect gains a wide enough hearing to begin to question and then change the law.

Results by Category and Partner

In Fig. 7 and Fig. 8 below, are given, in heat map format, the results according to category, for each of the three outcomes but now sorted additionally by responsible partner or partners (anonymized). The colour intensity of each square indicates the combined proportions of fully, partially, or not implemented, relative to the total recommendations under that categorynumber given in the squareassigned to that partner. Note that here the total number of recommendations for a category does not correspond exactly to those given in the table and figures above because responsibility for some recommendations was formally assigned to more than one partner, and sometimes the same recommendation was carried out (or partially) by one responsible partner but not by the other(s).

Observation Regarding Results by Category and Partner

In the heat map of results by category, Fig. 7, as assessed by the ethics team, further broken down according to partner responsible for a recommendationsometimes multiple partners -, we observe that the conjunction of some partners with some categories produced a high degree of partial or full implementation relative to the total recommendations given for that category. Thus, to give as an example the specific results which arrive at the colour intensity indicated: tech company Partner 3 (P3) fully or partially implemented 16 of 16 of the Human Centering recommendations for which they were responsible and Industrial Partner 1 (IP1) implemented 8 of 8 fully or partially in the same category, while tech company Partner 2 (P2) partially or fully implemented 6 of 9 Responsibility recommendations and 5 of 5 Design recommendations. On the other hand, Industrial Partner 2 (IP2) only implemented 3 of 8 Responsibility recommendations.

In the heat map of implementation results by category and partner as assessed by the project partners themselves, Fig. 8, some movement toward the blue or fully implemented colour intensity can be observed both for some categories and some partners. But also, some category rows and some partner columns remain unchanged, e.g. those of the Evaluation category and the results of tech company Partner 8 (P8). Moreover, the same general trends in the project partner assessed heatmap can be seen, e.g. strong implementation in the Human Centering and Design rows, and relatively strong implementations overall by Partner 2 (P2) and Partner 3 (P3).

Discussion of Results by Category and Partner

In the results broken down according to partner in the heatmaps, it is clear that, allowing for the categories just discussed, some partners are much more willing to implement recommendations. This held between industrial partners viewed separately as well, since one of these partners had a proportionally higher rate of full or partial implementation than the other. It should be noted also that one tech company partner more or less ignored ethical recommendations altogether, suggesting that they 'didn't really understand' the ethical aspect. Other partners were mixed in their engagement depending on the issue at stake and their interest in particular UCs. For these partners their responses to ethical recommendations were sometimes ambiguous, e.g. 'if it is efficient, we will do x', which can indicate that efficiency takes precedence over ethical concerns, but also that there is a 'way out' if implementation proves bothersome.

The Human Centering category had the best results in the heatmaps both as assessed by the ethics team and by the partners themselves, after adjustment for the number of recommendations. Recommendations to consider the workers (users) and collaborate with them in developing the services seem to be readily taken up most of the time. This may be helped by the factas noted anecdotally by the industrial partnersthat the workers will tend not to use a new service if it doesn't work properly, so it is important to get the service right.

Primarily, however, it is also encouraged by the fact that Human Centering recommendations tend to be positive, in the sense that they require additional work, but they do not obstruct lines of technical development pursued by the partner. They can be operationalized by giving the workers options in the service, by considering how to build on technology already used by the workers, by clarifying who will do what, by discussion with process engineers and heads of work teams, and by giving short surveys which uncover the worker's background knowledge and expectations and adjusting accordingly. In other words, though the ethics team tried to make all the recommendations positive rather than prohibitory, Human Centering lends itself particularly well to this approach.

As noted earlier, tech company Partner 3 (P3) had the best record of full or partial implementation, proportional to the number of recommendations they were responsible for, closely followed by Partner 2 (P2). Multiple individuals within P3 engaged with us in implementing ethical recommendations over various deliverables and issues. Also, interestingly, and perhaps because of the ethical engagement in question, as can be seen in Fig. 7 and Fig 8 ., a good portion of the upgrade from partially to fully implemented in the partner assessment, was made by P3. In other words, the partner which was most interested in ethical engagement, was also the partner most inclined to view their engagement as more successful.

The higher proportions of successful implementations for partners who were responsible for the largest numbers of recommendations seems to indicate a strong commitment in those partners to attempting to engage with the ethical aspect of the project. This impression was backed up by the ethics team's subjective assessments of different partner reactions to recommendations generally: the partners who had the highest proportions of full or partial implementation were also those who had a tendency to contact the ethics team regularly either for clarifications or to askwithout promptingto set up meetings to discuss potential ethical issues in UCs which they led. These partners did not mind engaging the ethical aspect and even seem to have welcomed the engagement.

Limitations of our Study

Allowance should be made for the partners not having experienced a hands on and ground up approach. The ethics team considers that approach to be a relatively new approach to applied ethics of AI in industry. The partners did not know what to expect when the project began, and the ethics team developed the approach as the project went along. Accordingly, it may be that some of the earliest recommendationssome were made in the first four monthsremained un-implemented wholly or in part because the partners were still unsure of the ethics approach. It was observed that some of the more willing partners became more comfortable with the general approach as the project progressed.

The main limitation, in parallel with the above observation, is that, across categories, implementation of recommendations depends very much on the attitude which the partner, as an organization, takes toward ethical concerns. This makes it difficult to quantify ethical results such that they could be taken to be completely objective and unbiased. But it also points out several facts which promise a way forward for ethics at the organizational level: an organization can have an ethical attitude, and if it can, then that ethical attitude can be cultivated.

It also leads logically to a related limitation: the roles of particular individuals in the ethical implementation results was very obvious, but the specific influences of those roles are missing from the results. A range of interest was observed among individual participants in each partner organization, with some individuals showing sustained interest in ethical operationalization. In terms of the written deliverables for which an ethical issues section containing recommendations was explicitly included, the task leader in charge of the deliverable often pushed to have the section completed as part of the deliverable completion. This was not just a matter of mere routine however. Some deliverable ethical issues sections were left relatively undone, while in other instances the task leader suggested to include an ethical issues section in a deliverable for which none had been planned.

The bias which limits the objectivity of the study in quantifying ethics also shows that an approach of operationalizing ethics at ground level is the right way to go precisely because it is developing the applied ethics techniques to deal with such biases and locating where those techniques need to be applied. In other words, to know that some organizations do not have a culture conducive to applying even direct ethical recommendations, leads obviously to asking why, and then to asking how such a culture can be instilled in the organization, and finally to asking how certain individuals working in the organization could serve as entry points for instilling an ethical culture.

Future Research

In a Deweyan friendly ethics the relation between the individual and society is the main focus of a practical application [START_REF] Dewey | [END_REF]. In this project that relation is recreated at the level of the work organization. The 'individual' is the industrial worker or the individual software engineer and the 'society' is the group of industrial and tech company project partners. This is very fertile ground upon which to operationalize ethics. It can also frame the questions to be asked going forward.

Consequently, research needs to be done on how the internal culture of the partner organization (industrial or tech company) and the relation between the organization and the individual working within it, influences their response to a ground up ethical approach such as this one. Further research should also be centered around the paradigm of relative and incremental ethical results in industrial and other contexts, i.e. progress should be made on an ethical approach which aims at bettering a context or proposed solution, rather than an either-or approach simply transposed by fiatusually to no effectfrom high level norms.

For AI ethics in the heavy industry context and work context more generally, there is also an opportunity to study how an applied ethics approach such as the one presented here can locate and bring out into the open instances of the uneasy contradiction mentioned earlier, between additional work, caused by the adoption of AI and related technologies, which is legally allowed but unethical, as opposed to extra work which is legally allowed but also ethically consistent. Studies which could bring this contradiction out in the open of a public discussion seem to be especially called for.

Further, the fact that the partner assessment of results, arguably did not differ by a huge margin from the ethics team assessmentparticularly since many upgrades were in a few categories and by a specific partneralso presents an opportunity. A good line of further studies here would be how much parallel ethical assessments and the possibility of peer review of ethical assessments at operational level, influence how tech company partners or industrial partners rate their own performance. AI Ethics in the industrial context could serve practically to integrate such studies with psychological and other related research in order to uncover positive best practices to be applied.

A final question is whether a ground level ethics operationalizing approach works as well in a more limited time frame, e.g. less than a year, or, conversely, does it get better results over a period longer than three years?

Conclusions

In this paper the results of a bottom up approach to AI ethics as applied in an EU Horizons 2020 project, which is now in its third year, have been presented. These results show how many of the specific ethical recommendations, given to project partners to date, were implemented, either fully, partially, or remain unimplemented. The results are described first as a percentage of total recommendations made, then as sorted under fifteen categories, and finally as sorted by both category and implementing partner. The process of categorizing recommendations is also described.

The ground up approach in question has been based upon carefully considering context, making recommendations, and monitoring and evaluating results. This paper is thus the more quantitative result of a larger attempt to operationalize AI ethics in Industry 4.0. From this more quantitative result, it was argued that the approach works and that it discloses two difficulties in operationalizing ethics in this context: the tendency of tech companies to reject the possibility of not using the technology in most cases and the tendency of new technological integrations to add new work covertly which offsets and contradicts the stated aims behind the new technology additions.

This paper also discloses several interesting and less obvious results which deserve further study, namely that both the individual cultures of organizations like those participating in the project and particular individuals within such organizations, have a major influence on ethical operationalization. With these leads the authors are confident that future research can work to better understand how ground level operationalization of AI ethics in industrial settings can be advanced.

Fig. 2

 2 Fig. 2 Overall Results of Ethical Recommendations as Assessed by Project Partners

Fig. 4 Fig. 5

 45 Fig.4 Results of Ethical Recommendations by Category as Assessed by Project Partners

Fig. 6

 6 Fig. 6 Results of Ethical Recommendations by Category as Assessed by Project Partners -Percentage of Total

Fig 8 .

 8 Fig 8. Results of Ethical Recommendations by Category and Project Partner(s) as Assessed by Project Partners

  Design Make changes or additions to technical or procedural elements of the solutionExample: ETH ID X.X-X -Recommend that you separate the x prediction from the x prediction service if possible, so that the operator can devote clear attention to x prediction faults only.Insufficient SpecsClarify aspects of the production or development process (e.g. in what format is operator feedback gathered, how many suggestions will AI give operator, what XAI methods will be used, etc.)Example: ETH X UC XX ID X.X-Xa -"Required interaction with the operator -What do we expect from him?": Recommend that you estimate how many potential suggestions as soon as possible.GDPRCheck whether a solution follows the spirit of GDPR regulations

		Example: ETH ID X.X-X -Recommend that you use alternatives to
		'exploit/exploiting' human activity and knowledge, i.e. alternatives
		which emphasize human participation.
	Workload	Estimate how much, how long, how many, of some new task to be
		done
		Example: ETH X UC XX ID X.X-X -"Required datasets for
		solution development (Quality of Service -Feedback System):"
		Recommend that you estimate in advance how much time this will
		add to operator's or quality manager's workload, either as a whole,
		or per 'meaningful units of product characteristics.'
	Evaluation	Assess whether some aspect of the workplace context is taken into
		account in the proposed quantitative outcome of the solution, e.g.
		acceptable error rate, or set a range for quantitative assessment of
		service, e.g. reliability
		Example: ETH X UC X ID X.X-X -Recommend that you clarify
		whether the blurry image situation for OCR recognition is also
		included in the X% error KPI.
	Training	Recommendations to provide specific training or implement services
		by stages
	Simplification	Try simpler techniques first
		Example: ETH X UC X ID X -Recommend that you consider using
		non-AI statistical process control method if feasible, to avoid greater
		complexity and error.
	Verify effects	Verify whether a proposed implementation would have some human
		effect
		Example: ETH X UC X ID XX -Recommend that you monitor
		cohesion of operator team regularly watching for changes in social
		interaction of the team members; ask the operator team members.
	Timeliness	Implement certain other recommendations in a timely manner
		Example: ETH X UC X ID X (update) -Recommend that you begin
		developing protocol at Deliverable X.X stage.
	Valorize experience	Make better use of human abilities/experience
		Example: ETH X UC X ID X.X-X -"AI training task:" Recommend
		that you dedicate an experienced operator who has a demonstrated
		interest in this to do it and reduce that operator's work
		proportionally.
	Ethical rewording	Reword a text or redraw a diagram to better include the human
		contribution

X-X -(Partner X; Partner Y) Recommend that you carry out a preliminary short survey, e.g. 10 questions, of user background knowledge (process engineers and operators) regarding AI, to be used in adjusting for potential user assumptions during XAI development.

Example: ETH ID X.X-XX -Recommend that you ask operator consent for natural voice feedback related to Task X.X (and tasks of WPX generally as relevant) accompanied by clear information as regards the purpose and storage time of the voice data.

Responsibility

Confirm or change who is responsible for tasks in some part of the process or what their new tasks will be Example: ETH X UC X ID X.X-X -Figure

6

: Recommend that you clarify whether the operator will be expected to review the root cause output.

De-anthropomorphization

Change anthropomorphic wording or thinking about AI Example: ETH ID X.X-X -Recommend that if X.X-X not implemented then have several sessions with process engineers and operators, to present clear mechanistic explanations of AI processes as un-intelligent tools.

Example: ETH X UC X ID Xc -Recommend that you institute a trial period where AI suggestions are first cleared by process engineer, shifting responsibility to the latter. Table 1 Definitions of Ethical Recommendation Categories with Examples 3.2 Process of Categorization and Results

Hereafter tech companies, or tech company partners in case we refer specifically to the technology company partners collaborating within our project. Software engineers, in contrast, refers to the individuals working within these tech companies.

NA recommendations have not been included in the calculation of percentages for implementation.

Note that recommendation categories were developed after the specific recommendations had been given and were not disseminated to the project partners.

The categories that were merged are not presented in the table above.

Acknowledgements

This research was funded by AI-PROFICIENT which has received funding from the European Union's Horizon 2020 research and innovation program under Grant Agreement No. 957391. The authors would like to thank Bertrand Remy at LORIA for generating the heat maps of Figs. 7 and8.

Data Availability Statement

All data generated or analysed during this study are included in this published article [and its supplementary information files].

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.