Rui Peng

Yan-Fu Li
email: yanfu.li@ecp.fr

Jun-Guang Zhang

Xiang Li

A risk-reduction approach for optimal software release time determination with the delay incurred cost

Keywords: software reliability, software release time, parameter uncertainty, multiattribute utility theory (MAUT)

Most existing research on software release time determination assumes that parameters of the software reliability model (SRM) are deterministic and the reliability estimate is accurate. In practice, however, there exists a risk that the reliability requirement cannot be guaranteed due to the parameter uncertainties in the SRM, and such risk can be as high as 50% when the mean value is used. It is necessary for the software project managers to reduce the risk to a lower level by delaying the software release, which inevitably increases the software testing costs. In order to incorporate the managers' preferences over these two factors, a decision model based on multi-attribute utility theory (MAUT) is developed for the determination of optimal risk-reduction release time.

1

Introduction

Software plays key roles in many industrial systems (e.g. power grid, telecommunication network, internet, etc) of modern society. Software reliability is of great importance for the stable operation of such systems. To ensure the reliability, software needs to be systematically tested prior to its release to the market. During the testing phase, the latent software faults are identified, isolated and removed. As a result, software reliability is improved. Based on the time-to-failure data obtained from the testing phase, software reliability can be measured and predicted using appropriate software reliability models (SRMs) [START_REF] Musa | Software Reliability: Measurement, Prediction, Application[END_REF][START_REF] Xie | Software reliability modelling[END_REF][START_REF] Lyu | Handbook of Software Reliability Engineering[END_REF].

Besides measuring and predicting software reliability, the SRMs are often used to support the software project managers making important decisions. A typical application is to advice the managers when to release the software. Consequently, the optimal release time determination during the software testing phase has become an extensively researched topic [START_REF] Okumoto | Optimum release time for software systems, based on reliability and cost criteria[END_REF][START_REF] Yamada | Optimum release policies for a software system with a scheduled software delivery time[END_REF][START_REF] Yamada | Cost-reliability optimal release policies for software systems[END_REF][START_REF] Pham | A software cost model with imperfect debugging, random life cycle and penalty cost[END_REF][START_REF] Pham | A software cost model with warranty and risk costs[END_REF][START_REF] Xie | A study of the effect of imperfect debugging on software development cost[END_REF][START_REF] Huang | Optimal release time for software systems considering cost, testing-effort, and test efficiency[END_REF][START_REF] Boland | Optimal times for software release when repair is imperfect[END_REF][START_REF] Liu | A reliability-constrained software release policy using a non-Gaussian Kalman filter model[END_REF][START_REF] Ho | The determination of optimal software release times at different confidence levels with consideration of learning effects[END_REF][START_REF] Yang | A study of uncertainty in software cost and its impact on optimal software release time[END_REF][START_REF] Lai | A Study of When to Release a Software Product from the Perspective of Software Reliability Models[END_REF][START_REF] Li | Reliability analysis and optimal version-updating for open source software[END_REF]. [START_REF] Okumoto | Optimum release time for software systems, based on reliability and cost criteria[END_REF] originally formulated and studied the optimal software release time determination problem. [START_REF] Yamada | Optimum release policies for a software system with a scheduled software delivery time[END_REF] studied the optimum release policies minimizing the total expected software cost with a scheduled software delivery time. [START_REF] Yamada | Cost-reliability optimal release policies for software systems[END_REF] developed a decision-making model, where both reliability and cost are considered. [START_REF] Pham | A software cost model with warranty and risk costs[END_REF] developed a software reliability-cost model to determine the optimal release policies that maximize the expected net gain in reliability. [START_REF] Pham | A software cost model with imperfect debugging, random life cycle and penalty cost[END_REF], [START_REF] Xie | A study of the effect of imperfect debugging on software development cost[END_REF], and [START_REF] Boland | Optimal times for software release when repair is imperfect[END_REF] investigated the effect of imperfect debugging on release time determination. [START_REF] Huang | Optimal release time for software systems considering cost, testing-effort, and test efficiency[END_REF] highlighted the importance of testing effort and testing efficiency in this decision problem. [START_REF] Ho | The determination of optimal software release times at different confidence levels with consideration of learning effects[END_REF] emphasized the learning effects for release time determination. [START_REF] Liu | A reliability-constrained software release policy using a non-Gaussian Kalman filter model[END_REF] proposed a non-Gaussian Kalman filter model for a reliability-constrained software release policy. More recently, [START_REF] Yang | A study of uncertainty in software cost and its impact on optimal software release time[END_REF] developed a new optimal software release time model, which can control the risk of the project being over-budgeting. [START_REF] Li | Reliability analysis and optimal version-updating for open source software[END_REF] proposed a model for the optimal version-updating time determination of open source software using multiattribute utility theory (MAUT) to combine two decision attributes: development time and reliability. Similar as [START_REF] Yamada | Cost-reliability optimal release policies for software systems[END_REF], [START_REF] Lai | A Study of When to Release a Software Product from the Perspective of Software Reliability Models[END_REF] studied the software release time policy considering both reliability and software cost.

In the release time determination problem, meeting the reliability requirement is of great importance. This is because the customers generally have a minimum reliability requirement, which can be specified in the contract. In order to check whether the reliability requirement is satisfied, SRM is often adopted to predict the reliability of software after its release. Most existing research on release time determination assumes that the parameters of the software reliability model are deterministic and the reliability estimate is accurate [START_REF] Yamada | Cost-reliability optimal release policies for software systems[END_REF][START_REF] Xie | A study of the effect of imperfect debugging on software development cost[END_REF][START_REF] Huang | Optimal release time for software systems considering cost, testing-effort, and test efficiency[END_REF][START_REF] Boland | Optimal times software release when repair is imperfect[END_REF][START_REF] Liu | A reliability-constrained software release policy using a non-Gaussian Kalman filter model[END_REF][START_REF] Ho | The determination of optimal software release times at different confidence levels with consideration of learning effects[END_REF][START_REF] Yang | A study of uncertainty in software cost and its impact on optimal software release time[END_REF][START_REF] Li | Reliability analysis and optimal version-updating for open source software[END_REF]. In practice, however, there exists a risk that the reliability requirement cannot be guaranteed due to the uncertainties in the software testing process which are reflected in the parameters of SRMs, and such risk can be as high as 50% if the software is released at the moment the estimated reliability equals to the reliability requirement (shown in Section 2). It is necessary for managers to reduce the risk to a lower level, and thus the testing process is expected to be longer, which inevitably increases the costs of testing. In order to balance between reducing the risk of unfulfilling the reliability requirement and controlling the cost incurred by release delay, this paper develops a new decision model for software release time determination, using MAUT [START_REF] Fishburn | Utility Theory for Decision Making[END_REF] to optimize the two conflicting objectives simultaneously.

The rest of the paper is organized as follows. In Section 2, the limitations of the existing research on software release time determination are further discussed, which motivate us to develop a new decision model. In addition, the attributes including risk and delay incurred cost are formulated. In Section 3, the decision model based on MAUT is presented, and the procedures of constructing it are described. In Section 4, the proposed decision model is illustrated by a case study. In Section 5, threats to validity are discussed.

Finally, concluding remarks are presented in Section 6.

Model Formulation

Considering the minimum reliability requirement level R 0 , the decision problem is typically formulated as

0 ) | ( R t x R  , (1) where
) | ( t x R
is the conditional software reliability, defined as the probability that the software will operate without failure within time interval ] , (

x t t 
given that it is released at t. The optimal release time T is then the minimum testing time required so that the software reliability reaches the level R 0 . In most software reliability models, there are a set of parameters

  m    ,..., , 2 1  θ
(where m is the number of parameters) used to represent the optimal release time T by a function

  0 2 1 , ,..., , R f T m     . To solve T,
most existing research works assume that these model parameters are known without uncertainty, and

) | ( t x R
can model the actual software reliability exactly [START_REF] Yamada | Cost-reliability optimal release policies for software systems[END_REF][START_REF] Xie | A study of the effect of imperfect debugging on software development cost[END_REF][START_REF] Huang | Optimal release time for software systems considering cost, testing-effort, and test efficiency[END_REF][START_REF] Boland | Optimal times software release when repair is imperfect[END_REF][START_REF] Liu | A reliability-constrained software release policy using a non-Gaussian Kalman filter model[END_REF][START_REF] Ho | The determination of optimal software release times at different confidence levels with consideration of learning effects[END_REF][START_REF] Yang | A study of uncertainty in software cost and its impact on optimal software release time[END_REF][START_REF] Li | Reliability analysis and optimal version-updating for open source software[END_REF].

Risk considerations

In reality, the exact values for these model parameters are often unknown. Instead, they are estimated from the collected time-to-failure data. Parameter uncertainty arises since the estimated parameters are subject to the random variations (or noises) in the data [START_REF] Dai | Uncertainty analysis in software reliability modeling by Bayesian approach with maximum-entropy principle[END_REF]. Due to the uncertainty of parameters, the software reliability computed from SRM is no longer deterministic. Consequently, the optimal release time T given a reliability target is also a random variable.

When the SRM parameters are estimated by the maximum likelihood estimation (MLE) method [START_REF] Nelson | Applied Life Data Analysis[END_REF], it is shown that the optimal release time T given R 0 is asymptotically normally distributed with mean T ˆ and variance   T Var ˆ. Here, T ˆ is obtained from solving (1) with the estimated parameters, and

  T Var ˆ is the variance of T ˆ.
Details about these quantities are presented in the Appendix 1.

Considering the uncertainty in T, the risk that software cannot meet the reliability requirement when it is released at time t can be quantified as

      ) ) (( 1 ) ( | 0 0 T Var T t T t P R t x R P t r         , (2)
where  

x  is the cumulative probability function (CDF) of standard normal distribution.

It is seen that when the mean value of release time, T ˆ is used, there is

) 0 ( 1   =50%
chance that the reliability requirement cannot be guaranteed. Such risk is too high to be acceptable. As a result, reducing the risk to a lower level to improve the confidence on the software reliability becomes an important issue. To account for this, the risk-reduction

release time T R is introduced as,   T Var z T T r R 0   , (3)
where r 0 denotes the acceptable risk level of managers, and 0 r z is the (1-r 0 ) quantile of the standard normal distribution. As seen from ( 3), the release time based on risk reduction requires a delay of

 

T Var z r 0 , which often results in the increase of the testing costs. This is a useful approach if the managers are certain about the risk level required and are committed to achieve it at all costs. On the other hand, it is also easy to elicit a maximum tolerable risk value given the project budget [START_REF] Nan | Impact of budget and schedule pressure on software development cycle time and effort[END_REF].

Cost Considerations

From the managers' perspective, it is also important to control the cost incurred by release delay [START_REF] Pham | A software cost model with warranty and risk costs[END_REF]. Let the testing cost function be denoted by C(t),

the delay incurred cost at time t ( T t  ) is obtained as,   ) ( ) ( T C t C t C p   (4)
The discussions above indicate that reducing the risk and controlling the delay incurred cost are two important but conflicting criteria that should be considered simultaneously when determining the software release time. Therefore, it is natural to incorporate the managers' preference into the decision process to make a compromise between these two criteria. In Section 3, the MAUT is adopted, and a decision model is developed for the determination of optimal release time.

The decision model based on MAUT

The application of MAUT is based on a one-dimensional multi-attribute utility function, which is the measure of the attractiveness of the conjoint outcome of the different attributes. The additive form of the multi-attribute utility function is given by

       n i i i n d u w d d d U 1 2 1 ,... , , (5)
where each attribute is denoted by d i , i=1,2,…n, the attractiveness of each attribute is represented by the single utility function u(d i ) and w i is the scaling constant which represents the importance weight for the utility u(d i ). The sum of the weights is equal to 1 (von [START_REF] Winterfeldt | Decision Analysis and Behavioral Research[END_REF]. By maximizing the multi-attribute utility function, the best alternative (i.e. the best set of values of the decision variables) is obtained, which gives the maximum attractiveness of the conjoint outcome of the attributes.

The main reason for using MAUT in our approach is that the typical management scenarios can be appropriately represented within its structure. In the decision problem formulated, there are two competing objectives to be balanced: minimizing the risk and minimizing the delay incurred cost. Given that the risk reduction and the cost control are both subjective, the single utility function is used to reveal managers' preference towards each attribute. By allocating different values of importance weights to the utilities of the attributes, managers can use the multi-attribute utility function to measure the total attractiveness of the conjoint outcome of the risk and the delay incurred cost given a specified release time.

Another reason for the selection of MAUT is that it has strong theoretical foundations due to the use of the expected utility theory. The utility theory takes managers' risk attitude into account, e.g. risk neutrality, risk aversion and risk proneness [START_REF] Fishburn | Utility Theory for Decision Making[END_REF]. Furthermore, MAUT provides a feasible approach for considering the continuous scale of the alternatives. Specifically, in our problem, the release time as the alternative should be considered in a continuous scale. Last but not least, when managers have other requirements, i.e., the minimization of the total cost in the software development cycle [START_REF] Sgarbossa | A Cost Analysis of Systems Subject to Random Field Environments and Reliability[END_REF], the control of the uncertainty in the total cost function [START_REF] Yang | A study of uncertainty in software cost and its impact on optimal software release time[END_REF], the optimized resource allocation (Ngo-The and Ruhe, 2009), our decision model can be extended by introducing more attributes in the framework of MAUT. The proposed MAUT procedure for our decision problem is discussed in detail below.

Elicitation of single utility function for each attribute

After the quantification of each attribute by ( 2) and ( 4), managers' preference towards the performance of each attribute should be assessed. To represent this, the single utility function is used. Suppose that the highest and lowest expected risks are first selected as 0 0 r and 1 0 r , respectively. In real applications, they provide the lowest and highest satisfactions to the managers, respectively. Cukic et al. [START_REF] Cukic | The theory of software reliability corroboration[END_REF] suggests that the reliability and the confidence of the reliability are usually application specific and predefined. For example, suppose that managers can only accept a risk level below 5%. The superscript of i r 0 ,

  1 , 0  i
is used to represent the corresponding utility value, which is determined so that the management is indifferent between the following two alternatives: 1) getting risk i r 0 with certainty; 2) getting risk 0 0 r with probability (1-i) and 1 0 r with probability i (Keeney and Raiffa, 1976, von Winterfeldt and Edwards, 1986). The single utility function is generally described by the linear or exponential function shown as follows [START_REF] Keeney | Decisions with Multiple Objectives: Preferences and Value Tradeoffs[END_REF]:

  0 0 r r u      or     0 0 exp r r u       , ( 6
)
where  ,  and  are constants which ensure    

1 , 0 0  r u .
To determine which form in (6) should be used, we can compare the certainty equivalent u(50%) and the expected value of the 50-50 lottery (u(0%)+u(100%))/2.

Specifically, if they are equal to each other, the managers are risk neutral and the linear form should be used. In this case, the utility function can be written as

  1 0 0 0 1 0 0 0 1 r r r r r u    
Otherwise, the managers are not risk neutral and the exponential form should be adopted.

In this case, the utility function can be written as

          1 0 0 0 0 0 0 0 exp exp exp exp r r r r r u           
where  is the non-zero solution to

      0 exp 2 exp exp 5 . 0 0 0 0 1 0       r r r    .

Estimation of scaling constants

The following step is the estimation of the scaling constants w 1 and w 2 =1-w 1 , which correspond to the important weights of u(r 0 ) and u(C p ). There are two common methods to assess the scaling constants: certainty scaling and probabilistic scaling (von [START_REF] Winterfeldt | Decision Analysis and Behavioral Research[END_REF]. Given that only two attributes are considered in our problem, the probabilistic scaling technique is used.

In the probabilistic scaling approach, the managers are asked to compare their preference over the two choices: 1) a deterministic joint outcome   with probability 1-p. The managers are first asked to compare the deterministic outcome with the lottery having a 50-50 chance of occurrence. If the managers prefer the certain outcome, the probability p is gradually increased until they are indifferent with these two choices. On the other hand, if the managers prefer the lottery, we decrease p. When the indifference is achieved, p is equal to the scaling constant w 1 for the risk attribute (von [START_REF] Winterfeldt | Decision Analysis and Behavioral Research[END_REF].

Maximization of multi-attribute utility function

By maximizing the multi-attribute utility function

     , ) ( ) ( ) ( ), ( 2 0 1 0 t C u w t r u w t C t r U p p   (7)
the optimal risk-reduction release time is obtained as

    ) ( ), ( max arg 0 * t C t r U T p t R 
. It is worth noting that ( 7) is based on certain independence assumptions. Interested readers can refer to [START_REF] Keeney | Decisions with Multiple Objectives: Preferences and Value Tradeoffs[END_REF] for detailed theoretical discussions. In real-world applications, these assumptions are commonly accepted [START_REF] Brito | Multi-attribute risk assessment for risk ranking of natural gas pipelines[END_REF]de Almeida, 2008, Ferreira et al., 2009). Moreover, it has been shown that even when these assumptions are violated, the additive multi-attribute utility function can provide fairly good approximations [START_REF] Edwards | Use of multiattribute utility measurement for social decision making[END_REF][START_REF] Farmer | Testing the robustness of multiattribute utility theory in an applied setting[END_REF].

Summary of the procedure

The first step of the implementation of the decision model is to quantify the attributes, i.e., the risk and the delay incurred cost. For the risk attribute, based on the standard statistical results, risk can be quantified by (2). For the cost attribute, the generalized cost model is used and it is quantified by (4). The following step is the elicitation of single utility functions for both attributes. After this, the scaling constants for each attribute are determined following procedures in Section 3.2. Finally, based on the single utility functions and the scaling constants, the multi-attribute utility function is obtained as shown in (7). The optimal risk-reduction release time is determined by maximizing it.

Case Study

In this section, the proposed optimal release time determination approach is applied onto the case study used in [START_REF] Pham | A software cost model with warranty and risk costs[END_REF]. By considering the risk and the delay incurred cost simultaneously, the optimal release time is determined by incorporating the managers' preference into the decision process. In addition, sensitivity analysis is introduced to assist checking the robustness of the final decision.

The determination of optimal risk-reduction release time

Step 1: quantification of attributes

The Goel-Okumoto (GO) model [START_REF] Goel | Time-dependent error-detection rate model for software reliability and other performance measures[END_REF]) was adopted in [START_REF] Pham | A software cost model with warranty and risk costs[END_REF] where a denotes the total number of expected faults in the software and b represents the fault detection rate. Furthermore, the reliability of the software system during its operational phase is obtained as

  x t t x R ) ( exp ) | (    (9) and ) | ( t x R
represents the conditional software reliability, which is defined as the probability that the software will not fail given a specified time interval ] , ( x t t  in the operational phase [START_REF] Yang | A study of operational and testing reliability in software reliability analysis[END_REF]. Since x is usually set to 1 without loss of generality, the release time given the reliability target R 0 is . Moreover, from the standard statistical analysis [START_REF] Nelson | Applied Life Data Analysis[END_REF], as shown in the appendix, the variance of the release time is

       ) / 1 ln( ln 1 0 R ab b T ( 10
obtained 758 . 9 ) (  T Var
. Accordingly, the attribute risk can be quantified by substituting these estimated values into (2).

The cost model proposed by [START_REF] Pham | A software cost model with warranty and risk costs[END_REF] consists of two parts, i.e., the expected general testing cost   t C 1 and the expected cost of removing errors during testing phase

  t C 2 as    t c t C 1 1  ,     y t m c t C  2 2  , ( 11
)
where c 1 is the software test cost per unit time,  is the discount rate of the testing cost due to the learning effect, c 2 is the cost of removing an error per unit time during the testing phase and y  is the expected time of removing an error during this period.

According to [START_REF] Pham | A software cost model with warranty and risk costs[END_REF], the coefficients in the cost model can be determined by empirical data and previous experiences of the staff members. We set the parameters as:

c 1 =700, 95 . 0   , c 2 =60 and 1 . 0  y 
, same to the assignments in [START_REF] Pham | A software cost model with warranty and risk costs[END_REF]. It is worth noting that risk is expected to be less than 50% from managers' point of view. Therefore the software should be released after 479 . 41 ˆ T in order to reduce the risk, that is, we just need to consider ) , [   T t

. Accordingly, the delay incurred cost at the time t is obtained as

            y p T m t m c T t c t C    2 1           . ( 12
)
Step 2: Elicitation of single utility functions The following step is to assess managers' preference towards the performance of the risk and the delay incurred cost. Interviews with the managers are needed to elicit reasonable single utility functions. Suppose that management scenarios are as follows:

(1) Managers are risk neutral towards both attributes.

(2) Managers indicate that they can only accept up to a risk level of 5%, and the smaller the risk the better, until the risk can be eliminated.

(3) Managers have an incurred cost budget of $15000 and they are completely unsatisfied when all the money is spent; their satisfaction increases when the expense decreases, and the highest satisfaction level is achieved when no money is spent.

According to the management scenarios above, corresponding explanations on the determination of single utility functions are shown as follows:

(1) Since managers are risk neutral towards both attributes, the linear form of the single utility function is used.

(2) The lowest risk requirement is % 5 0 0  r and the highest risk expectation is 0

1 0  r .
The single utility function for risk is obtained as  

0 0 20 1 r r u   .
(3) The maximum cost budget is 15000

0  p C
and the highest satisfactory cost expectation is 0

1  p C
. The single utility function for delay incurred cost is determined as

  15000 1 p p C C u   .
Step 3: Estimation of scaling constants At this stage, the scaling constant w 1 is estimated first by comparing the deterministic

joint outcome   0 1 0 , p C r
with the lottery consists of  

1 1 0 , p C r
with probability p and

  0 0 0 , p C r
with probability 1-p. Suppose managers claim that they are indifferent between these two choices when p is equal to 0.5, then w 1 =0.5. Since the sum of scaling constants is equal to one, w 2 is equal to 0.5 as well.

Step 4: Maximization of multi-attribute utility function Based on the estimated single utility functions and scaling constants, the multi-attribute utility function can be obtained by ( 7). . However, at this release time, the 50% risk is too high to be acceptable by the managers because the lowest risk requirement % 5 0 0  r is not satisfied.

At this point, the manager has to make a compromise between reducing the risk and controlling the delay incurred cost.

With this consideration, the software testing time is expected to increase. We denote Finally, it should be noted that during the time periods

    0 0 , ˆr T T and       , 0 p C T
, the multi-attribute utility function is dominated by only one of the attributes. More specifically, for the first period, since the lowest risk requirement % 5 0 0  r has not been satisfied, the delay incurred cost is the only attribute contributing to the multi-attribute utility function. Given that the delay incurred cost is increasing over time and managers' satisfaction level is decreasing with it, the multi-attribute utility function is decreasing during this time period. While for the second time period, the multi-attribute utility function is dominated by the risk attribute and it equals to 0.5u(r 0 ). Figure 1 shows that the multi-attribute utility function remains at 0.5 level when release time is greater than

  0 p C T . It implies that the available cost budget 15000 0  p C
is sufficient for the managers to reduce the risk to the best level 0 1 0  r .

Sensitivity analysis

As shown in the Sections above, the optimal risk-reduction release time can be determined by maximizing the multi-attribute utility function. However, since most parameters in the MAUT are obtained based on the subjective assessments of the managers, the optimal risk-reduction release time obtained may not be accurate. In practice, the managers have to know how robust the optimal decision is, and thus sensitivity analysis is needed. More specifically, sensitivity analysis can help to investigate the relative variation of the optimal solution when a specific parameter changes, i.e., the change of cost parameters, scaling constants, etc. The results from sensitivity analysis reveal the stability of the optimal solution.

Sensitivity analysis is generally done by changing one parameter and setting the other parameters at the fixed levels [START_REF] Xie | Software reliability modelling[END_REF]Hong, 1998, Li et al., 2010). The sensitivity of the optimal decision to one parameter x can be quantified by x q S , , defined as the relative change of the optimal risk-reduction release time when x is changed by 100q% [START_REF] Xie | Software reliability modelling[END_REF]Hong, 1998, Li et al., 2010).

) ( ) ( ) ( * * * , x T x T qx x T S R R R x q    (13)
A large value of x q S , indicates that parameter x has significant impact onto the determination of * R T , and * R T is regarded as sensitive to the change of x. Normally, managers should pay special attention to the important parameters as the optimal decision * R T is heavily dependent on the accurate estimates of them (Xie and Hong, 1998, Li et al., 2010).

From a practical point of view, it may not be necessary to conduct sensitivity analysis for all the parameters in this optimal release time problem. For instance, parameters c 2 and y  are expected to be insignificant, because the expected cost to remove errors from time t to T ˆ is negligible. More specifically, given a high reliability requirement such as R 0 =0.95, there will be few faults detected from T ˆ to t. Additionally, as c 1 =700, c 2 =60 and 1 . 0  y  ; compared with the estimated value of c 1 , the product of c 2 and y  is too small to have any impact on the delay incurred cost function in (12). Another example is the determination of 1 0 r and 1 p C , which represents the highest risk reduction expectation and highest cost control expectation, respectively. Since managers always prefer less risk and less cost, setting them to zero can properly describe the best cases for risk reduction and cost control respectively.

In contrast, parameters c 1 and  are much more important since they dominate the change of the delay incurred cost over time. Similarly, 0 0 r and 0 p C are of importance as shown in Figure 1, where   Since the sum of these two weights is equal to one, investigating one factor is sufficient. Results of sensitivity analysis with regard to these parameters are summarized in Table 1. Specially, since parameter  represents the learning effect of the testing team which is not greater than 1, the value of

S

-0.77% -0.48% -0.22% 0.20% 0.38% 0.55% 0 0

, r q S 0.74% 0.47% 0.22% -0.20% -0.39% -0.57% 1 ,c q S 0.74% 0.47% 0.22% -0.20% -0.39% -0.57%

 , q S 2.91% 1.96% 0.99% -0.53%
It can be seen that these parameters do not significantly influence the final solution on * R T since all the absolute values of x q S , are below 3%. In other words, the optimal riskreduction release time obtained is robust to the changes in the parameters. Moreover, results in Table 1 indicate that * R T is positively correlated with w 1 and 0 p C , and negatively correlated with 0 0 r , c 1 and  . Physical meanings of these parameters can actually explain these results. For instance, when w 1 increases, it means that more importance is allocated for the control of risk. As a result, * R T increases as well.

Threats to validity

Based on the standard statistical analysis [START_REF] Nelson | Applied Life Data Analysis[END_REF], there is 50% chance that the software will not meet its reliability requirement when the mean value of the release time, T ˆ is used. However, it should be noted that the standard statistical analysis is an approximation. It is still an open question whether the risk is really as high as 50%. To investigate this problem, an empirical case study is conducted by the Monte Carlo simulation using MATLAB tool.

In particular, the GO model is adopted, where the preset parameters are given by 100  a and 1 . 0  b

. Suppose that the reliability requirement is 95 . 0 0  R , then the real value of optimal release time can be obtained as 73 . 52  real T

. According to the general procedures discussed in [START_REF] Lyu | Handbook of Software Reliability Engineering[END_REF], 10000 failure data sets are generated, and each failure data set is composed of ninety time-to-failure data points. Since each failure data set can produce an estimate of the optimal release time denoted by T ˆ, risk that software cannot meet the reliability requirement can be easily estimated by comparing these T ˆ values with real T , and such risk is estimated as % 21 . 60 ˆ0  r

. Although this result is different from the estimated risk based on the standard statistical analysis, it is another piece of evidence that the risk due to parameter uncertainty cannot be neglected, because the risk can be even higher than 50%.

In addition, the normal distribution is used to quantify the uncertainty of optimal release time. Although this type of approximation technique is widely applied to reliability engineering, it may not be accurate. In this case, incorporating experts' opinion and past experience could be a choice. For example, experts could probably know the distributions of some model parameters based on their past experience on similar software projects.

Based on this type of information, parameter uncertainty can be effectively quantified by combining the Maximum-Entropy Principle (MEP) into the Bayesian approach [START_REF] Dai | Uncertainty analysis in software reliability modeling by Bayesian approach with maximum-entropy principle[END_REF]. By incorporating this quantification of parameter uncertainty into the simulation of optimal release time, the uncertainty of optimal release time can be modeled more sufficiently.

Besides the consideration of risk, the delay incurred cost is incorporated into our decision problem. This is because the risk cannot be overlooked due to the limit cost budget of the project [START_REF] Nan | Impact of budget and schedule pressure on software development cycle time and effort[END_REF]. Management needs to strike a balance between reducing the risk and controlling the delay incurred cost. In other words, given a reliability requirement, we introduce two new important dimensions for the determination of optimal release time: the risk that software cannot meet the reliability requirement due to parameter uncertainty, and the delay incurred cost associated with such risk. However, it should be noted that the formulation here may not be sufficient for release time determination. In reality, managers can also have other requirements, which may include the minimization of the total cost in the software development cycle [START_REF] Sgarbossa | A Cost Analysis of Systems Subject to Random Field Environments and Reliability[END_REF], the control of the uncertainty in the total cost function [START_REF] Yang | A study of uncertainty in software cost and its impact on optimal software release time[END_REF], and the optimized resource allocation (Ngo-The and Ruhe, 2009), etc. When these requirements are considered, our decision model can be extended by introducing more attributes into the framework of MAUT.

Conclusions

The software release problem during the testing phase is of great importance in the software development cycle. This paper discusses in detail when to release software given a reliability constraint. In particular, we highlight the risk in the reliability estimate due to the parameter uncertainty in the SRM. However, reducing such risk inevitably increases the testing costs. Thus, from the management's point of view, a compromise should be made between reducing the risk and controlling the delay incurred cost associated with it. To account this issue, a decision model based on MAUT is developed for the determination of optimal risk-reduction release time. The proposed model provides project managers with a boarder view of the release time determination problem.

It not only allows managers to optimize two criteria simultaneously, but also incorporates managers' preference to the decision process. In this paper, the risk of not fulfilling the software reliability requirement is studied from the aspect of parameter uncertainties in the SRM. Future work can be conducted to analyze the effect of choosing different SRMs on the estimated optimal release time and the risk of not fulfilling the reliability requirement.



              k i i i i n i i n t m t m t m t m L i 1 1 1 ! ) ( ) ( exp ) ( ) ( (14)
By maximizing the likelihood function above, point estimates of model parameters can be determined. The variances of these estimators can be calculated following the asymptotic theory for maximum likelihood estimation [START_REF] Nelson | Applied Life Data Analysis[END_REF]. In particular, the Fisher information matrix is obtained as

                                                                                                        2 2 2 2 1 2 2 2 2 2 2 2 1 2 1 2 1 2 2 2 1 2 1 ln ln ln ln ln ln ln ln ln ) , , ( m m m m m m L E L E L E L E L E L E L E L E L E I                                                  (15)
According to the standard theory of MLE, when the data size is large,  

                                            ( 17
)
where

i T    / is evaluated at   m    , , ˆ2 1 
. Based on the standard statistical analysis [START_REF] Nelson | Applied Life Data Analysis[END_REF], T is asymptotically normally distributed with mean T ˆ and variance   T Var ˆ.

Appendix 2

Table 2 shows the software testing data used in [START_REF] Pham | A software cost model with warranty and risk costs[END_REF], which are summarized as the number of failures per one-hour interval of execution time.

                               

  lowest risk and the highest delay incurred cost; 2) the lottery consists of both attributes at their best levels   probability p and both attributes at their worst levels  

  Figure1shows this multi-attribute utility function as a function of the release time. This multi-attribute utility function is maximized risk and controlling the delay incurred cost.

Figure

  Figure 1 Multi-attribute utility function given different release times

  t increases between these two time points, the single utility function associated with risk increases and the single utility function associated with cost decreases. As the weighted sum of the two single utility functions, the multi-attribute utility function increases first and then decreases. The optimal risk-reduction release time which maximizes the multi-attribute utility function is 586

  points of multi-attribute utility function. Furthermore, scaling constants w 1 and w 2 are also important since they represent the different importance weights allocated to both attributes, which directly affect the final solution of * R T .

  the positive change of  .

  on the covariance matrix, the uncertainty of other quantities, which are functions of parameters  

  to analyze the failure data for reliability assessment. In this work, we use this model as well. It is noted that the procedures are similar if other software reliability models are adopted. Moreover, future research can be carried out to analyze the impact of parameter uncertainties when different models are used for software release time determination. The mean value function and the failure intensity function of the GO

	model are given by
	( t m	)		1 ( a		e	bt 	)	and		t (	)		bt  abe	(8)

Table 1

 1 Sensitivity analysis results given different parameters

		q		-30%	-20%	-10%	10%	20%	30%
	S	q	1 ,w	-1.36% -0.88% -0.43% 0.42%	0.84%	1.27%
	q	,	C	0
						p

Table 2

 2 Failure in 1 hour intervals and cumulative failures

	Hour (i)	Number of failures (n i )	Cumulative failures
	1	27	27
	2	16	43
	3	11	54
	4	10	64
	5	11	75
	6	7	82
	7	2	84
	8	5	89
	9	3	92
	10	1	93
	11	4	97
	12	7	104
	13	2	106
	14	5	111
	15	5	116
	16	6	122
	17	0	122
	18	5	127
	19	1	128
	20	1	129
	21	2	131
	22	1	132
	23	2	134
	24	1	135
	25	1	136
	According to (14), the likelihood function can be written as

Acknowledgment

The research is partially supported by the China NSFC under grant number 71231001, China Postdoctoral Science Foundation funded project under grant number 2013M530531, and the Fundamental Research Funds for the Central Universities under Grant FRF-MP-13-009A and FRF-TP-13-026A.

Appendix 1

Parameters in the software reliability model are estimated on the basis of the recorded time-to-failure data. Maximum likelihood estimation (MLE) technique is generally adopted for such estimation. Based on the standard statistical analysis [START_REF] Nelson | Applied Life Data Analysis[END_REF], the optimal release time T given a reliability target is asymptotically normally distributed with mean T ˆ and variance

 

T Var

Suppose that there are totally m model parameters to be estimated,


Yan-Fu Li

Rui Peng

Service Reliability Modeling of Distributed Computing Systems with Virus Epidemics

Keywords: service reliability, distributed computing system, virus epidemics, continuousstate model, differential equations, universal generating function

Distributed computing (DC) system is widely implemented due to its low setup cost and high computational capability. However, it might be vulnerable to malicious attacks like computer virus due to its network structure. The service reliability, defined as the probability of fulfilling a task before a specified time, is an important metric of the quality of DC system. This paper attempts to model and compute the service reliability for the DC system under virus epidemics.

Firstly, the DC system architecture is modeled by an undirected graph whose nodes (i.e. computers) have a continuous-state model representing its computational capability. Then a set of epidemic differential equations are formulated and solved to obtain the state dynamics of each node under the virus epidemics. A universal generating function (UGF) based approach is proposed to calculate the service reliability of DC system. Numerical results show the effectiveness of the proposed method. The sensitivity analysis on the model parameters, the comparison with centralized computing system and the optimization of defense level parameter are also conducted.

I. Introduction

Distributed computing (DC) system [START_REF] Andrews | Foundations of Multithreaded, Parallel, and Distributed Programming[END_REF] is a collection of multiple autonomous computers that can communicate through a computer network to solve a large computational task. The purpose of the DC system is to coordinate the use of shared resources and provide communication services to the users [START_REF] Ghosh | Distributed Systems-An Algorithmic Approach: Chapman & Hall[END_REF]. Comparing to the centralized computing system, the DC system possesses many advantages, such as high performance, low setup cost, and potential for enhanced reliability [START_REF] Kounev | Performance modeling and evaluation of distributed component-based systems using Queueing Petri Nets[END_REF][START_REF] Lai | A model for availability analysis of distributed software/hardware systems[END_REF][START_REF] Stankovic | A distributed parallel programming framework[END_REF]. Therefore, DC system has gained increasing popularity in many application fields such as distributed software/hardware system [START_REF] Lai | A model for availability analysis of distributed software/hardware systems[END_REF][START_REF] Rutherford | Evaluating test suites and adequacy criteria using simulation-based models of distributed systems[END_REF], distributed power generation [START_REF] Wang | Reliability-constrained optimum placement of reclosers and distributed generators in distribution networks using an ant colony system algorithm[END_REF], distributed sensor system [START_REF] Kerrouche | Strain Measurement on a Rail Bridge Loaded to Failure Using a Fiber Bragg Grating-Based Distributed Sensor System[END_REF], and etc. Like many other computing systems, the service quality of DC system is of high concern to the majority of practitioners. Service reliability, which measures the capability of a system to accomplish its tasks on time, is a very important metric of DC system service quality [START_REF] Dai | A study of service reliability and availability for distributed systems[END_REF]. Many research works have been devoted to modeling and analysis of the reliability (including service reliability) of DC systems [START_REF] Dai | A study of service reliability and availability for distributed systems[END_REF][START_REF] Fahmy | Reliability evaluation in distributed computing environments using the AHP[END_REF][START_REF] Lin | Efficient algorithms for reliability analysis of distributed computing systems[END_REF][START_REF] Lin | Optimizing availability and QoS of heterogeneous distributed system based on residual lifetime in uncertain environment[END_REF][START_REF] Xie | Optimal number of hosts in a distributed system based on cost criteria[END_REF]. However, most of the previous research works have focused on the failures caused by the 'unintentional' defects embedded in the DC hardware infrastructure and the installed software. In practice, external factors such as infective computer virus become widely spread in the current computer networks [START_REF] Shih | Classification methods in the detection of new malicious emails[END_REF]. In this paper, we focus on the type of virus which can reproduce themselves and infect other computers in the network. If the virus successfully parasitizes one computer, it will rapidly copy itself, consume the computing resources (e.g. CPU and memory) of the host, and attempt to infect other healthy computers via network connections (e.g. email, FTP transfer, message exchange, etc). This process will repeat on other infected computers and may eventually lead to a great loss of computational capability of the whole DC system if the situation is not attended to. Protecting the DC system against the virus attacks becomes an increasingly important issue [START_REF] Rutherford | Evaluating test suites and adequacy criteria using simulation-based models of distributed systems[END_REF] and this type of protection is clearly different from the protection against 'acts of nature' or 'accidents' [START_REF] Bier | Protection of simple series and parallel systems with components of different values[END_REF]. For example, the CPU breakdown in a computer usually will not affect the operations of other computers connected in the same network. In the literature of reliability research, many studies have been devoted to intentional attack protections by designing protection strategies for different systems (e.g. power substations, defense systems, etc) [START_REF] Levitin | Optimal defense strategy against intentional attacks[END_REF][START_REF] Hausken | Strategic defense and attack for series and parallel reliability systems[END_REF][START_REF] Levitin | Importance of protections against intentional attacks[END_REF][START_REF] Levitin | Protection vs. redundancy in homogeneous parallel systems[END_REF][START_REF] Shah | Two methods for privacy preserving data mining with malicious participants[END_REF], but few have investigated the attacks with epidemic characteristics, such as computer virus [START_REF] Wierman | Modeling computer virus prevalence with a susceptible-infected-susceptible model with reintroduction[END_REF].

In the field of epidemiology modeling, some research works have addressed the virus spreading issue in computer networks, but the emphasis is on the speed and range of the spreading [START_REF] Wierman | Modeling computer virus prevalence with a susceptible-infected-susceptible model with reintroduction[END_REF][START_REF] Buzna | Modelling the dynamics of disaster spreading in networks[END_REF][START_REF] Murray | The application of epidemiology to computer viruses[END_REF].

3

To bridge the gap, in this work the service reliability of the DC system is modeled and computed under the virus epidemics with the consideration of possible system noises.

The rest of this paper is organized as follows. In Section II, the general model of virus epidemics is proposed: the continuous-state model is used to describe the computational capability of each node in the DC system and the epidemic differential equations are set and solved to obtain the time-dependent state index. In Section III, service reliability is defined based on the virus epidemic model and the universal generating function (UGF) technique is adopted for computing service reliability. Section IV illustrates the proposed model on a numerical example with 1) the sensitivity analysis on the defense level parameter and the processing speed coefficient, 2) the comparison with centralized computing system and 3) the optimization of defense level parameter. Section V concludes this study with some possible future research directions. In this work, the DC system is modeled as an undirected graph

II. Modelling of Virus

) , ( L V G  , where } 1 | { N i v V i   
is the set of computers (nodes), and

} , 1 | { N j i N i l L ij     
is the set of communication channels (links) connecting the nodes. It is noted that many authors have assumed homogeneous elements (no difference between nodes and links) in DC system [START_REF] Dai | A study of service reliability and availability for distributed systems[END_REF][START_REF] Fahmy | Reliability evaluation in distributed computing environments using the AHP[END_REF].

However, for the virus epidemic modeling, nodes are usually treated as infectious components while the links are treated as the noninfectious channels for virus spreading [START_REF] Lin | Optimizing availability and QoS of heterogeneous distributed system based on residual lifetime in uncertain environment[END_REF][START_REF] Murray | The application of epidemiology to computer viruses[END_REF].

A continuous-state reliability model of individual nodes

Markov chain model is one of the conventional approaches for modeling DC system reliability with a number of discrete states [START_REF] Xie | Optimal number of hosts in a distributed system based on cost criteria[END_REF], where each node has two states: online (functioning state)

or offline (failure state) and the transition diagram is established to model the system state changes. However, the size of Markov state space grows exponentially with the increase of the number of nodes and the degradation states [START_REF] Pham | Software reliability and cost models: Perspectives, comparison, and practice[END_REF]. Moreover, when a node is under virus infection, it will not completely lose its computational capability in short time. Once a virus successfully resides onto a node, it attaches itself to some executable files. Its code will be executed when one user attempts to launch an infected program. After the execution of its code, the virus may replicate itself into other programs. Progressively, more and more programs will be infected and the running virus codes will consume greater amount of computing resources and slower down the entire computer. Therefore, a two-state model is not sufficient to describe the degradation phenomena of individual nodes.

A feasible alternative is the continuous-state model, which has been considered by many studies on computer virus epidemiology [START_REF] Wierman | Modeling computer virus prevalence with a susceptible-infected-susceptible model with reintroduction[END_REF][START_REF] Buzna | Modelling the dynamics of disaster spreading in networks[END_REF][START_REF] Murray | The application of epidemiology to computer viruses[END_REF] where the state of each node takes real values to represent the degradation condition of the computational capability. In addition, continuous state models have already been considered by a number of reliability researchers, since many realworld systems and components exhibit the continuous type of degradation [START_REF] Baxter | Continuum structures I[END_REF][START_REF] Brunelle | Review and classification of reliability measures for multistate and continuum models[END_REF][START_REF] Liu | Using neural network function approximation for optimal design of continuous-state parallel-series systems[END_REF][START_REF] Long | Reliability analysis and optimization of weighted voting systems with continuous states input[END_REF].

The continuous-state model defines a state space

] 1 , 0 [  
representing all possible intermediate real-valued levels between the two extremes: '0' is perfect functioning state and '1' is the complete failure state. In addition, ) (t i  denotes the state index of node i at time t. In the next section, we will apply the epidemic functions to solve ) (t i  .

Virus Epidemic Model

In this section, we describe the epidemic differential equations for the virus spreading in our model. From the assumptions made in section 2.1, if a node v i is at healthy state, then 0 ) ( 

t i  .
The deviation from the normal state represents the level of damage to the node. The strength of interactions between node v i and its neighbor v j is defined by the weight w ij attributed to the edge e ij . In this study, the weight is set to be proportional to the speed of the communication channel between two nodes. For node v i , its state is dependent on the cumulative impacts from all its

neighbors i j v  
each proportional to the connection strength w ij with some time delay t ij , and its own ability to defend virus infection. Taking all the above factors into consideration, the status change of node v i can be represented using the following epidemic differential equation:

i i j ij j ij i t t t w t dt d i     ) ( ) ( ) (       (1)
where i  is the set of neighboring nodes that have direct connections to node i and i  is the time independent parameter that represents the ability of node v i to defend virus infection.

i  is an important parameter with physical meanings. For example, the anti-virus software such as

McAfee usually offers different levels of protections ranging from basic to total protections. The more protections the anti-virus offers the higher price it has. Based on the single epidemic equation in (1), we establish the following equation system to model the combined effects of virus spreading and anti-virus mechanism in the entire DC system.

                               N N j Nj N Nj N i i j ij j ij i j j j t t t w t dt d t t t w t dt d t t t w t dt d N i             ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( 1 1 1 1 1 1 1   (2)
There are many methods to solve such differential equation systems like [START_REF] Ghosh | Distributed Systems-An Algorithmic Approach: Chapman & Hall[END_REF]. The Laplace transform

      0 ) ( ) ( dt e t t L st  
is an effective approach with the property that

    ) 0 ( ) ( ) ( '      t sL t L
. After taking the Laplace transform on the both sides of eq. ( 2), the transformed equation system can be solved via linear algebra given the initial conditions

) , ,..., 1 ( 0 ) 0 ( , 1 ) 0 ( i j N j j i      
. Then the inverse Laplace transform is applied on the obtained solutions to finally derive the solutions to (2). In Section 2.3, an illustrative example is presented to obtain the node state index

An Illustrative Example

Fig 1 shows a 5-node DC system with the node 1 infected by a certain type of virus. The edge weight and node virus defending parameters are assigned with some preset values, as shown in Table 1. The edge weight is determined by the speed of data transmission, e.g. megabyte per second (mbps), between two nodes, whereas the virus defense parameter can be determined by the protection level of the anti-virus software installed in each node.

) 0 ( , 0 ) 0 ( , 0 ) 0 ( , 0 ) 0 ( , 1 ) 0 ( 5 4 3 2 1          
. Since the size of virus (about few Kbits) is often very small comparing to the bandwidth of the communication channel, the time delay caused by network traffic is assumed negligible. Based on the conditions mentioned above, the virus epidemics equations are written as follows.

                                  ) ( 1 . 0 ) ( 05 . 0 ) ( 02 . 0 ) ( 03 . 0 ) ( ) ( 125 . 0 ) ( 05 . 0 ) ( 06 . 0 ) ( ) ( 2 . 0 ) ( 02 . 0 ) ( 06 . 0 ) ( 01 . 0 ) ( 09 . 0 ) ( ) ( 05 . 0 ) ( 01 . 0 ) ( 04 . 0 ) ( ) ( 2 . 0 ) (
                        (3)
The eigenvalue method is used to solve (3). The solutions are as follows: As can be seen from Fig. 2, the state index of node 1 first drops steeply from 1 with time and then it begins to reduce more smoothly. This phenomenon is due to the virus defending mechanism.

                                                              t t
The curves of node 2, 3, 4 and 5 are similar. They all start from 0 and gradually increase. The node which has larger weighted edge connecting with node 1 is expected to be infected faster than the others at the beginning. This situation is validated in the form of a steeper increasing curve of ) ( 2 t  . It is also worth noting that the state index of each node becomes relatively stable as time increases.

In the next Section, the service reliability is modeled and computed.

III. Service Reliability Modeling of Distributed System Under Virus Epidemics

Based on the epidemic model of virus spreading presented in Section II, this section derives the service reliability of the DC system. Suppose that the entire task is divided into K sub-tasks which are distributed to the N nodes for processing. One node can process multiple subtasks and a subtask can be distributed onto multiple nodes for processing. The service reliability is usually considered as the probability of successfully completing a target task within a predefined period of time [START_REF] Dai | A study of service reliability and availability for distributed systems[END_REF][START_REF] Dai | Availability modeling and cost optimization for the grid resource management system[END_REF]. In this work, we adopt this definition into the context of virus epidemic and regard the service as successful if it is finished within time T. Prior to the reliability model, the assumptions for the task processing and transmission are presented as follows:

0 1 2 3
1. Each node can execute multiple subtasks simultaneously. A node starts to execute an assigned subtask immediately after it gets all necessary inputs, and the data processing speed depends on its state and the type of subtask.

2. Each link has a data transmission speed (bandwidth), of which a stable portion is occupied by one subtask throughout the entire task transmission and processing period.

This assumption is in line with the multiplexing technique for computer communications.

3. The data transmission time within each node is negligible.

4. For each sub-task at one node, the data is transmitted through a same set of links before and after the execution.

5. The hardware failures due to natural causes are not considered because these failure probabilities are usually very low and the focus of our study is on virus spreading. is the amount of raw data to be processed on certain node and

Service reliability of one subtask distributed to one node

k k d   ) 1 ( 
is the amount of result data to be returned to the initiating node. The data is transmitted between node v i and the node that initiates the subtask, through a same set of links ki L before and after the execution of the sub-task at one node. Thus the data transmission speed for sub-task k transmitted to resource v i is

) ( min j L l ki l s ki j   (6)
where j l is the transmission speed (or bandwidth) of the jth link in ki L occupied by the sub-task k executed by node i. Hence the transmission time of the sub-task k is obtained as follows:

ki k ki s d T  (7)
Let E i denote the set of subtasks distributed to node i. It can be obtained from [START_REF] Wang | Reliability-constrained optimum placement of reclosers and distributed generators in distribution networks using an ant colony system algorithm[END_REF]

) ( )) ( ( )] ( )) ( ( [ ) (                       (9)
The probability that node i can finish processing subtask k can be calculated as

) Pr( ) ( k k ki ki d Y T R    (10) Denote dt t Z ki ki T t ki ki ) (     
. According to the property of Gaussian process, Z ki itself is a normal random variable with mean 0. The variance of Z ki and thus of ki Y can be obtained as follows:

2 1 2 1 2 )) ( ) ( ( ) ( ) ( ) ( dt dt t t E Z E Z V Y V ik ki ki ki T t T t ki ki ki ki ki             (11)
It can be seen that the variance of Y ki is determined by the covariance function of the Gaussian process, which describes the covariance of noises at any pair of time points. Different covariance functions need to be used depending on the mechanism of the noises associated with the subtask processing speed. Specifically, if the covariance of the noises at different time points is 0 and the noise variance at any time t is a constant 2 2 ) (

ki ki t    , we have ) ( )) ( ) ( ( )) ( ) ( ( ) ( 2 1 1 1 2 1 2 1 ki ik ki T t ki ki T t T t ki ki ki t T dt t t E dt dt t t E Y V ik ki ik ki ki ki                       (12)
Furthermore, we have

) ) ( )) ( ( ( 1 ) Pr( ) ( ki T t i ki k k k k ki ki Y V dt t f d d Y T R ki ki             (13)
where ) (  represents the cumulative probability function of standard normal distribution.

Total service reliability

In this section, we utilize the UGF approach to derive the total service reliability based on the reliability index obtained in the section above for individual subtask on one single node. UGF was first introduced by Ushakov in 1986 [START_REF] Ushakov | Universal generating function[END_REF] and it proves to be very effective in evaluating reliability of complex multi-state systems. Much research has been done on incorporating UGF into reliability analysis of various k-out-of-n systems, series-parallel systems, weighted voting systems, acyclic information networks, and manufacturing systems [START_REF] Ding | A framework for reliability approximation of multi-state weighted k-out-of-n systems[END_REF][START_REF] Li | Heterogeneous redundancy optimization for multi-state series-parallel systems subject to common cause failures[END_REF][START_REF] Peng | Element maintenance and allocation for linear consecutively connected systems[END_REF][START_REF] Li | Random fuzzy extension of the universal generating function approach for the reliability assessment of multi-state systems under aleatory and epistemic uncertainties[END_REF]. The UGF of a discrete random value X is defined as a polynomial, , ) (

1 0     W w x w w z p z u ( 14
)
where the variable X has W possible values and p w is the probability that X takes the value x w .

In our case, the UGF of each subtask k distributed to node i is defined as [START_REF] Bier | Protection of simple series and parallel systems with components of different values[END_REF] where φ denotes the empty set. The UGF representing two different subtasks distributed to node i can be obtained as

 z R z R z u ki k ki ki      ) 1 ( ) ( } {
 z R R z R R z R R z R R z q p z q z p z u z u mi ki m mi ki k mi ki m k mi ki h B S l l h B l l S h h mi union ki l h l h ) 1 )( 1 ( ) 1 ( ) 1 ( ) ( ) ( } { } { } , { 1 0 ) , ( U 1 0 1 0 1 0                       (16)
where S k and B l represent the sets of subtasks, and union  is the proposed union composition operator. By iteratively combining the UGF representing all the subtasks distributed on node i, the UGF of the node i can be obtained as

h Q H h h i z P z U    0 ) ( ( 17
)
The UGF of the DC system can be obtained by combining the UGF of all the nodes together as

j Q J j j N union union z P z u z u z u z U       0 2 1 ) ( )... ( ) ( ) ( (18)
The service reliability can be obtained as

}) ,..., 1 { ( 1 ) ( )) ( ( 0 0 K Q P z P z U j J j j Q J j j j            (19)
where ) (  is the redistributive operator designed to obtain the system service reliability.

IV. A Numerical Example

To illustrate the proposed method for DC service reliability computation, the example in Section 2.3 is extended. Suppose that a task needs to be processed within T=5.5 seconds. The resource management server (RSM) has divided the task into three subtasks and dispatched them as E 1 ={1,2}, E 2 ={1}, E 3 ={2}, E 4 ={2,3}, E 5 ={3}, as shown in Fig. 3. In real industrial practice, the algorithm for determining task dispatching is very complicated and interested readers are encouraged to refer to [START_REF] Andrews | Foundations of Multithreaded, Parallel, and Distributed Programming[END_REF]. The size and the percentage of raw data for each subtask are given as

mbits d mbits d mbits d 20 , 30 , 20 3 2 1 
  and 5 . 0 3 2 1      
. For simplicity, let the relationship of processing speed and the node state have the following form:

) ( )) ( 1 ( ) ( t t t ki i ki ki       
where ki  is the processing speed coefficient that links the processing speed to the node state. It

is assumed that 4  ki  and ) ( 1 )) ( ) ( ( 2 1 2 1 t t t t E ki ki     
for k=1,2,3 and i=1,…,5, where 1(TRUE)=1 and 1(FALSE)=0. Subtasks 1 and 2 need to get their inputs from and send back the results to node 1. Subtask 3 needs to get inputs from and send back results to node 5. Fig. 3 The 5-node DC system with distributed subtasks The bandwidths of the edges are provided in the Table 4. It is assumed that L 12 ={e 12 }, L 23 ={e 13 }, L 24 = {e 13 , e 34 }, L 34 = {e 45 }. Note that e 13 is in both L 23 and L 24. In this study, we assume that 50% bandwidth of e 13 is occupied by the transmission of sub-task 2 executed by node 3, and the other 50% bandwidth of e 13 is occupied by the transmission of sub-task 2 executed by node 4.

Table.4 Bandwidths of the edges

A. Reliability calculation

According to [START_REF] Rutherford | Evaluating test suites and adequacy criteria using simulation-based models of distributed systems[END_REF]

      T R T R T R T R
The probability of finishing subtask 3 is 1 Fig. 5 shows the curves of service reliability as a function of processing speed coefficient for different levels of defense. It is assumed that the processing speed coefficient ki  is the same for every k and i. It can be seen that the service reliability has a S-shaped increase when the processing speed coefficient increases and the higher service reliability is associated with the higher defense level. The S-shaped curve is attributed to the reason that the pdf of a normal distribution is higher nearer the center so that the reliability of a subtask distributed to a node increases faster when the absolute difference between The state equation of node 1 can be expressed as

) ( ) ( 1 1 1 t t dt d      It is easy to know that . ) ( 1 1 t e t    
For a fair comparison, it is still assumed that

) ( )) ( 1 ( 4 ) ( t t t ki i ki       and ) ( 1 )) ( ) ( ( 2 1 2 1 1 1 t t t t E k k     for k=1,2,3.
Similarly, we can obtain

          T e Y T R
Thus the reliability of the centralized system is better if

9923 . 0 ) ( ) ( ) ( ) ( 31 21 11 1   T R T R T R T R .
When T=5.5, the reliability of the centralized system is as shown in Fig. 6. The system reliability is greater than 0.9923 when the defense parameter is greater than 3.05.

Fig. 6 Reliability of centralized system as a function of the defense parameter Note that our comparison between centralized computing system and distributed computing system is based on the assumption that the infected node is curable regardless of the severity of the infection. In some other cases, the defense may become invalid when the infected node is overly damaged, i.e., its state index is above some threshold. In such cases, it is more reasonable to use distributed system structure. If some nodes in a distributed computing system have already become incurable, the defense mechanism can still prevent other nodes from infection and they may be able to complete the system mission. However, for the centralized computing system, it fails as long as the single node has become incurable.

D. Optimal allocation of defense level

In case the defense level of one individual node can be controlled by the amount of defense resource allocated to the node, the optimal allocation of defense level can be studied. For a DC system with N nodes, the optimal defense level allocation problem which maximizes the system reliability subject to a given cost can be formulated as C is the maximum allowable cost. For illustration, consider the optimal allocation of defense level for the system presented in part A of this section. Without loss of generality, it is assumed that Note that our model is readily adapted to more general case where the defense parameters take stepwise values instead of constant ones throughout the system mission time. To fulfil this, we need to divide the system mission time into certain periods such that the defense parameters are constant within each but can change in different periods. In this case, we need to first solve equation ( 2) to obtain the node state index in the first period given the initial condition at mission time 0. Then, we can solve (2) for the node state index in the second period given the node state index value at the end of the first period as the initial condition. Iteratively, the node state index throughout the whole system mission can be solved as a piecewise function of time. With the state indices of all the nodes, the system reliability can be calculated in similar procedures. The optimal defense resource allocation will be formulated as

Maximize

)) ,..., ( ),..., ,..., ((

Fig. 1 A 5 -

 15 Fig.1 A 5-node DC system with the node 1 infected

Fig. 2

 2 Fig.2 Plot of ) (t i  over time

  Let k d denote the amount of data related to sub-task k. It is noted that k d contains two different sets of data: 1) the raw data forwarded to the processing node for processing; 2) the output data of the processing node which has to be sent back to the initiating node. Let k  denote the percentage of the raw data in sub-task k, then k k d  

From

  Fig.4a Plot of ) (t i  over time with the defense parameters being 4 times larger than those in

Fig. 5

 5 Fig.5 Service reliability versus processing speed coefficient for different levels of defense

  system reliability and the total cost when the defense levels of the nodes are

  The optimal defense levels of the nodes are found to be )

Spreading in Distributed Computing Systems

	(t	)
			Defense parameter at node v i
			Percentage of the raw data in sub-task k
	k
	d k		The amount of data related to subtask k
	(t	)
	Notations
			Range of continuous state			[	] 1 , 0	, where 0 indicates the perfect
			functioning state and 1 indicates the complete failure state.
	T		System time
	N		Total number of nodes in the computer network
	G		The undirected graph representing the computer network
	V		The set of nodes in the computer network
	v i		Node i in the computer network
	L		The set of communication channels in the computer network
	l ij		The communication channel that links node v i and v j

i  The state index of node i at time t i  The neighborhood set of node v i E i The set of subtasks distributed to node i i  ki  Data processing speed of subtask k by node v i at time t ki  Processing speed coefficient which links the processing speed to the node state ) (T R ki Probability that all the transmission and processing operations of subtask k assigned to node i can be finished by time T K Total number of subtasks

Table . 1

 . Weight of edges and virus defense parameter

	Edge	12 e	13 e	15 e	23 e	34 e	35 e	45 e
	Weight	0.04	0.09	0.03	0.01	0.06	0.02	0.05
	Node	1	2	3	4	5
		0.2	0.05	0.2	0.125	0.1
	The initial conditions imply that					0

Table 1

 1 over time with the defense parameters being twice larger than those in Table

		1
												u1(t)
		0.9										u2(t)
												u3(t)
		0.8										u4(t)
												u5(t)
		0.7
		0.6
	state index	0.5
		0.4
		0.3
		0.2
		0.1
		0	0	1	2	3	4	5	6	7	8	9	10
								second

i 

Acknowledgement:

The research report here is partially supported by the NSFC under grant number 71231001, 71301009 and 71420107023, the Fundamental Research Funds for the Central Universities of China, FRF-TP-14-051A2, and by the MOE PhD supervisor fund, 20120006110025.

From ( 16), we have

Furthermore, we have

According to [START_REF] Levitin | Protection vs. redundancy in homogeneous parallel systems[END_REF], the service reliability is 0.9923.

B. Sensitivity Analysis

In this Section, the sensitivity analysis is performed on two model parameters, namely the defense level and the processing speed coefficient, that are influential to the computation of DC system service reliability.

In order to investigate the effects of defense level on the node states, Fig 4 (a) and 4 (b) shows the expected node states when the defense parameter of each node is as four times/twice bigger as the defense parameter shown in Table 1. With comparison to the curves in Figure 2

V. Conclusions and Future Works

DC system is popular in industry because of its low setup and maintenance cost as well as high computational capability. However, due to the network nature of DC system, it might be vulnerable to virus attacks. This paper focuses on the computation of the service reliability of DC system under virus epidemics. The computational capability of individual node is modeled by a continuous-state model. The network topology of DC system is explicitly modeled as an undirected graph. A set of differential equations are formulated to describe the node state dynamics due to virus spreading. A universal generating function based approach is proposed to calculate the service reliability of the DC system. A numerical example is presented for illustration. The sensitivity analysis on the model parameters, the comparison with centralized computing system and the optimization of defense level parameter are also conducted.

The results show that enhancing the virus defense of each node i is an effective way of recovering the system from virus attacks but it may be costly if every node has to be attended to.

In future research, we will attempt to implement the developed methods onto larger scale DC systems. In addition, it is an essential issue to evaluate the risk in different fields [START_REF] Wu | Business intelligence in risk management: Some recent progresses[END_REF][START_REF] Wu | Computational simulation and risk analysis: An introduction of state of the art research[END_REF][START_REF] Wu | Introduction to the special issue on optimizing risk management in services[END_REF]. Thus another direction is to conduct risk analysis on our continuous state epidemic model.


Syed Islam
email: syed.islam@uel.ac.uk

Adel Noureddine
email: a.noureddine@uel.ac.uk

Rabih Bashroush
email: r.bashroush@qub.ac.uk

Measuring Energy Footprint of Software Features

With the proliferation of Software systems and the rise of paradigms such the Internet of Things, Cyber-Physical Systems and Smart Cities to name a few, the energy consumed by software applications is emerging as a major concern. Hence, it has become vital that software engineers have a better understanding of the energy consumed by the code they write. At software level, work so far has focused on measuring the energy consumption at function and application level. In this paper, we propose a novel approach to measure energy consumption at a feature level, cross-cutting multiple functions, classes and systems. We argue the importance of such measurement and the new insight it provides to non-traditional stakeholders such as service providers. We then demonstrate, using an experiment, how the measurement can be done with a combination of tools, namely our program slicing tool (PORBS) and energy measurement tool (Jolinar).

I. INTRODUCTION

With the proliferation of Software systems and the rise of paradigms such the Internet of Things, Cyber-Physical Systems and Smart Cities to name a few, the rate of energy consumption of Information and Communication Technologies (ICT) is rising at an alarming rate. It is estimated that ICT consumption will rise from 168 to 433 Gigawatts (7% to 14.5%) by 2020 [START_REF] Vereecken | Overall ICT footprint and green communication technologies[END_REF]. Greenhouse Gas emissions (GH) from ICT are also expected to double to 1430 MtCO2e within the same period [START_REF] Webb | SMART 2020: enabling the low carbon economy in the information age, a report by The Climate Group on behalf of the Global eSustainability Initiative (GeSI)[END_REF], highlighting the need for more energy efficient hardware as well as software.

In addition to traditional software properties often cited to aid in program comprehension [START_REF] Storey | Theories, methods and tools in program comprehension: past, present and future[END_REF], we argue that in the near future, it will become vital for software engineers to better understand the energy consumed by their applications. While software energy consumption is already a major software engineering concern in some application areas such as mobile computing (given the impact on battery life, and ultimately the product viability), monitoring and optimising the energy consumption of software systems is gaining considerable traction else where. A recent survey showed that software architects envisage energy becoming a major architectural concern in the next five years [START_REF] Bashroush | Data center energy demand: What got us here won't get us there[END_REF].

Current research focuses on measuring the energy consumption of applications at two main granularities. The first considers the energy consumption of a particular function or a set of functions, while the second level of granularity looks at the energy consumed by the entire software stack or process. Work in this area entails creating accurate models to estimate the energy consumption of software, which sometimes includes the use of complex formulas or additional hardware. Such measurements are vertical in nature and oriented towards measuring the consumption of software constructs. Although such approaches can aid software engineers identify bottlenecks, hotspots or energy smells in the code, they do not lend themselves to better understand consumption at feature-level, which is ultimately what is being consumed by end-users.

In this paper, we propose a novel approach for measuring energy consumption of end-user features, taking a more horizontal cross-cutting view. Our approach will enable newer levels of programme comprehension and understanding. On the one hand, it will empower software engineers analyse the consumption of a given feature, which can cross-cut multiple classes, or even programming languages. On the other hand, it will help cloud service providers isolate and better understand energy consumption of their system features, and where applicable, introduce better charging models for endusers based on real energy consumption.

The rest of the paper is structured as follows. The next section discusses related literature and motivation. Section 3 presents our feature oriented approach. Validation and results are then analysed in Section 4. Finally, Section 5 rounds off the paper with a summary and future work.

II. MOTIVATION & RELATED WORK

Current approaches for monitoring energy range from hardware devices to power models and tools [START_REF] Noureddine | A review of energy measurement approaches[END_REF]. Hardwarebased solutions, such as power meter devices and dedicated integrated circuits or sensors, demand additional investment, complex installations, and treat the entire system as a black box. Other approaches overcome such large granularity limitations by estimating software energy using power models or a calibration process. Tools, such as pTop, PowerSpy, PowerAPI, JouleMeter or Energy Checker provide power and energy insights into software energy footprint but at the cost of limiting usability. For instance, some require a power meter in order to calibrate their models, or for runtime energy monitoring (PowerAPI, PowerSpy, JouleMeter), others require modifications to applications' source code or patching parts of the operating system to effectively monitor software energy consumption (pTop, Energy Checker). Source-based approaches provide energy readings for methods and functions [START_REF] Noureddine | Monitoring energy hotspots in software[END_REF] or for individual lines of code [START_REF] Li | Calculating source line level energy information for Android applications[END_REF]. These approaches measure the energy consumption of software artefacts (e.g., methods, classes, lines of code), mostly ignoring their functionality or the relationships between them.

Modern software systems are typically composed of more than one programming language and often comprise dozens of features that cross-cut hundreds of components. Thus, the comprehension of such information would allow developers to focus their optimisation efforts on entire features consumed by end-users, rather than only software constructs. This, for example, could help cloud providers model their billing structure around the energy consumption of particular system features.

III. APPROACH

Our approach to measuring software energy consumption at the feature-level has two steps. The first step involves the identification and extraction of an executable subset of the original program that implements the feature of interest (feature slicing). The second involves the measurement of the energy consumed by the feature slice (energy measurement). The two steps are discussed below.

A. Feature slicing

The first step in our approach is to identify what parts of the system constitutes a feature of the software. Unlike formal functional components of a software system, what constitutes a feature is often down to the granularity at which the software system is modelled. At higher architectural levels, a feature constitutes work done by several modules to achieve some purpose; However, even a small utility function within that same system can be regarded as a feature when taking a more fine-grained view.

Program Slicing is a technique for identifying the influence or dependencies for a specific point of interest within the code, referred to as the slicing criteria. Program Slicing is well researched and has many applications [START_REF] De | Program slicing: Methods and applications[END_REF], including comprehension, testing, debugging, maintenance, re-engineering, re-use, and refactoring. Although useful for debugging, the original notion of program slicing now known as static slicing is not suitable for feature extraction. The static form is a conservative approach where any possible influences on the slice criterion for all possible inputs are identified. This conservatism over approximates and includes much more dependencies than what a typical execution of a feature would require. For example, if a static slice was taken with respect to variable n on line 38 of program p (Figure 1), the static slice would include the entire program. Several extensions to the original notion of slicing have been proposed over the years that attempt to tackle feature extraction [START_REF] De | Understanding function behaviours through program slicing[END_REF], [START_REF] Wong | Locating program features using execution slices[END_REF], [START_REF] Eisenbarth | Locating features in source code[END_REF]. However, most of these techniques do not yield a slice that is an executable program in its own right. Furthermore, none of these techniques can handle multi-language systems.

Recently proposed Observation-based Slicing [START_REF] Binkley | ORBS: Language-independent program slicing[END_REF] is a technique that finds its origin in Weiser's original motivation for slicing [START_REF] Weiser | Program slicing[END_REF]: uninteresting statements can be deleted (sliced out) of a program to help focus attention on relevant statements (i.e., the slice). Operationally, PORBS 1 (our observation-based slicer) achieves this by tentatively deleting one or more statements and then observing the behaviour of the resulting program. It attempts to find a subset of the program (slice) that preserves the behaviour of the original program with respect to a certain set of inputs and the slice criterion.

Figure 1 shows a program P , which is a utility program with two features, sort and unique. The first, reads a file and outputs contents of the file in sorted order, while the second, reads a file and outputs non-repeating contents. The features are triggered by providing the name of the input file and options -s (sort) or -u (uniq), respectively. We present this simplistic version of the utility tool to explain our approach ignoring details such as error checking. The code has been structured in such a way that the main, readFile and writeList methods will be required for both utilities. Each specific feature is then provided by one of the two methods, namely sort by sort method and uniq by unique method.

Running PORBS slicer on the code with an appropriate input file and a feature options causes PORBS to produce an executable slice implementing the certain feature. For example, Column 2 of Figure 1 marks lines included in the slice for the feature option -s and Column 3 marks the lines included in the slice for feature option -u. Both slices are executable programs on their own right and will produce the same output as the original program (P ) for respective features. PORBS is therefore able to extract executable slices that contains all the code pertaining to the implementation of a particular feature within a program. We are therefore using PORBS in our approach to extract a minimal set of code that implements a certain feature.

B. Energy measurement

We estimate the energy consumption of features using Jolinar2 , an accurate, lightweight and easy-to-use tool for energy measurements. Jolinar is based on our energy models introduced and validated in [START_REF] Noureddine | A preliminary study of the impact of software engineering on greenit[END_REF], [START_REF] Noureddine | Towards a Better Understanding of the Energy Consumption of Software Systems[END_REF]. These models use publicly available hardware OEM specifications to calculate resource utilisation for monitored applications, then estimate their energy consumption based on energy formulas. First, we measure resource utilisation by the hardware (so far, CPU, disk and memory) and apply energy models to estimate how much the hardware modules are currently consuming. Next, we capture the resources used by the monitored application (for example, number of CPU cycles, or number of bytes written to the disk the application is using), and apply a second set of energy models to estimate software energy consumption.

The models in Formulas 1 and 2 takes into consideration modern processors' characteristics, such as DVFS (Dynamic Voltage and Frequency Scaling), CPU's TDP (Thermal Design Power) and real-time processor frequency and voltage changes. The power consumption is calculated every 500 milliseconds, following power variation whenever frequency or voltage changes.

P f CP U = 0.7 × T DP f T DP × V 2 T DP × f × V 2 × t P ID CP U t CP U (d) (1)
P CP U = f ∈f requencies P f CP U × t f CP U f ∈f requencies t f CP U (2)
Disk and memory energy models use a similar approach of capturing resources and estimating energy consumption. Disk energy model uses the number of bytes read and written by the application (see Formula 3), while the memory mode uses the percentage of RAM memory occupied by the application. These models were validated with an error margin of around 3% on average [START_REF] Noureddine | A preliminary study of the impact of software engineering on greenit[END_REF], [START_REF] Noureddine | Towards a Better Understanding of the Energy Consumption of Software Systems[END_REF].

P disk =
IV. VALIDATION

We validate our approach using the use case shown in Figure 1 where we identify two executable program slices, the first one is for the sort feature and the second one for the uniq feature. Following the extraction of these slices Fig. 2. Jolinar's output for uniq slice with 10'000'000 inputs using PORBS, we apply our energy model using Jolinar to estimate the amount of energy consumed by each of the slices when processing a file with 100,000, 1,000,000 and 10,000,000 integer values using a machine running an Intel Pentium T3400 processor running at 2.16 GHz and a Seagate Momentus 5400.5 SATA hard disk.

Figure 2 shows the energy output running for uniq slice with 10,000,000 inputs as provided by Jolinar. Table I outlines the energy results for the different inputs, detailing energy consumed by the CPU, disk and memory, for both the sort and uniq slices. The results show an exponential growth in the energy consumption for the sort feature, going from 10 joules to more than 1415 joules when integer inputs are multiplied by 100. The majority of this energy is due to sorting calculations (therefore the CPU), with an increase of memory energy for larger data sets. Nearly a third of the energy (31.2%) consumed by the 10 million integer sort feature is due to memory access (read/write) for sorting. In particular, Collections.sort(), which is used in the sort feature (see line 26 in Figure 1), requires a temporary storage of up to n/2 object references [START_REF]Java class collections[END_REF]. On the other hand, the uniq feature exhibits a much lower memory energy consumption as the unique method (see line 29, Figure 1) only adds input to a hashset and a list.

Figure 3 and4 show the power consumption of the sort and uniq features with 10,000,000 inputs, respectively. Initially, there is an increase in disk power consumption due to disk activities reading the input file (e.g., method readFile(String s) in lines 14-23 in Figure 1). Memory consumption for sort is constant across the experiment as Java allocates a large memory buffer for sorting and did not require additional memory space. More interestingly, the graph outlines spikes in the CPU power consumption which are explained by the sorting cycles in the algorithm and the copying of data in-between. With a shorter experiment duration, uniq seeks processing power for copying and clearing Java collections objects (a hashset and a list) as seen in Figure 4.

Comprehension of energy consumption by feature can help developers identify the features that are consuming more energy and use additional information (such as importance and likely frequency of use) to decide on where to focus optimisation efforts (e.g. optimise code or dedicate energy efficient resources to frequently used features).

V. SUMMARY AND FUTURE WORK

In this paper, we present a novel approach for better understanding the amount of energy consumed by software at a feature level as compared to traditional methods which analyse energy consumption based on software artefacts (e.g. function or process). We argue the value of such analysis and validate our approach by using PORBS to extract executable slices from an example program for each specific feature. We then use Jolinar to measure the amount of energy consumed by these features. Our validation shows that we are able to identify and measure the energy consumption of features (slices), which could help software engineers conduct a new level of analysis and optimisation (e.g. focus on what matters most).

Future work includes extending our empirical validation using large real-world cloud-based systems where such information is likely to have a huge impact. We also plan to automate the energy monitoring at feature-level, and expand our energy models to additional hardware and platforms in the cloud. Finally, our future work will also consider calculating in real-time the GHG emissions of features based on energy consumption and power grid mix (e.g. % renewables) data in countries where such information is readily available.

Fig. 1 .

 1 Fig. 1. Observation-based Slice for Line 38 with option -s and -u.

Fig. 3 .Fig. 4 .

 34 Fig. 3. Power consumption of the sort slice with 10,000,000 inputs

  Bytes read × (

	DiskP ower read DiskRate read	)
	+ Bytes write × (	DiskP ower write DiskRate write	)

TABLE I ENERGY

 I CONSUMPTION BY THE FEATURES SORT AND UNIQ FOR VARIOUS INPUT SIZES

					sort				uniq
	Input File Size Input Size	CPU	Disk Memory	Total	CPU	Disk Memory	Total
	556K	100000	9.51	0.01	0.23	9.75	5.49	0.01	0.07	5.57
	5.4M	1000000	55.6	0.12	7.46	63.18	8.68	0.12	0.22	9.02
	54M	10000000	972.3	1.18	441.44	1414.92	54.34	1.18	6.53	62.05

http://www.noureddine.org/research/jolinar