Warren Connell
email: wconnel2@gmu.edu

Massimiliano Albanese
email: malbanes@gmu.edu

Sridhar Venkatesan

A Framework for Moving Target Defense Quantification

Keywords: Moving Target Defense, MTD Quantification, Framework

Moving Target Defense (MTD) has emerged as a game changer in the security landscape, as it can create asymmetric uncertainty favoring the defender. Despite the significant work done in this area and the many different techniques that have been proposed, MTD has not yet gained widespread adoption due to several limitations. Specifically, interactions between multiple techniques have not been studied yet and a unified framework for quantifying and comparing very diverse techniques is still lacking. To overcome these limitations, we propose a framework to model how different MTD techniques can affect the information an attacker needs to exploit a system's vulnerabilities, so as to introduce uncertainty and reduce the likelihood of successful attacks. We illustrate how this framework can be used to compare two sets of MTDs, and to select an optimal set of MTDs that maximize security within a given budget. Experimental results show that our approach is effective.

Introduction

Moving target defense offers a great potential in turning the typical asymmetry of the cyber security landscape in favor of the defender [START_REF] Jajodia | Moving Target Defense: Creating Asymmetric Uncertainty for Cyber Threats[END_REF], and many techniques have been developed since the term first surfaced in the literature. However, each technique only addresses a narrow subset of potential attack vectors and different techniques tend to measure their effectiveness in different and incompatible ways. Additionally, in order to provide a comprehensive security solution, multiple techniques should be used jointly, but this requires the selection of an optimal subset of available techniques. Although several surveys note where certain MTDs might not work well together [START_REF] Okhravi | Survey of cyber moving targets[END_REF], or give a qualitative estimate of their effectiveness and cost [START_REF] Farris | Quantification of moving target cyber defenses[END_REF], a quantitative framework that can accommodate any existing or future MTDs is still needed for this area of research to progress past specialized, isolated solutions. To address this pressing need, we present a novel framework that captures the relationships between available MTDs and the information such MTDs may affect through probabilistic measures. It also captures the relationships between services, their weaknesses, and the knowledge required to exploit such weaknesses to probabilistically determine the effectiveness of any given technique or set of techniques, regardless of how they operate. Our framework presents the following desirable attributes: (i) generality -the relationship between MTDs and the knowledge they protect defines an interface that enables to plug any MTD into the framework; (ii) extensibility -the model can be extended to accommodate future MTDs by introducing new elements, such as additional knowledge blocks or classes of weaknesses; (iii) resilienceas the framework addresses generic classes of weaknesses rather than specific vulnerabilities, the model can address both known and unknown (zero-day) attacks; (iv) usability -the framework is simple and intuitive, can be used to compute utility estimates at different levels of granularity, and can incorporate cost in the estimation of utility.

The remainder of the paper is organized as follows. Section 2 discusses related work, whereas Section 3 covers our threat model and underlying assumptions. The framework itself is presented in detail in Section 4 with a simple running example, while a more complex case study is discussed in Section 5. Then, Section 6 shows two applications of the proposed model. Finally, Section 7 discusses potential future work and gives some concluding remarks.

Related Work

Many different metrics are used in the literature to measure the effectiveness of MTDs, such as attacker's success rate [START_REF] Carroll | Analysis of network address shuffling as a moving target defense[END_REF], or metrics for deception, deterrence, and detectability [START_REF] Jafarian | Spatio-temporal address mutation for proactive cyber agility against sophisticated attackers[END_REF]. Still others utilize multiple metrics (productivity, success, confidentiality, and integrity) for both the attacker and the defender [START_REF] Zaffarano | A quantitative framework for moving target defense effectiveness evaluation[END_REF], leading to confusion over the multiple dimensions. However, all these metrics only evaluate a few select MTDs. One expert survey provides a thorough assessment of the effectiveness and cost of many techniques across the spectrum of existing MTDs [START_REF] Farris | Quantification of moving target cyber defenses[END_REF], but the survey is qualitative in nature and potentially subject to reviewer's bias. Our work leverages existing work on attack graphs [START_REF] Jajodia | Managing Cyber Threats: Issues, Approaches, and Challenges, Massive Computing[END_REF], particularly those approaches that evaluate security by looking at how the probability of a successful attack propagates over an attack graph [START_REF] Wang | An attack graph-based probabilistic security metric[END_REF]. The TREsPASS project 1 provides a holistic view of an organization's information security risk. It provides a visualization framework that combines the impact of vulnerability exploitation, physical security breach and social engineering on the target organization. This framework can be used to analyze several properties of multi-step attacks such as the required effort or time, and likelihood of success. However, attack graphs cannot be readily used with every MTD, as they are often tied to specific vulnerabilities. In fact, several MTDs can drastically alter a system's attack surface, requiring to generate an entirely new attack graph every time the MTD changes the system's configuration, which is not feasible in practice.

Our work is also inspired by research on autonomous systems, particularly selfprotecting systems [START_REF] Alomari | An autonomic framework for integrating security and quality of service support in databases[END_REF], which autonomously change their settings to adapt to their environment, implicitly creating a moving target. In order to do so efficiently, they must quantify the effectiveness and cost of all possible changes.

Threat Model and Assumptions

The general nature of our model lets us make very broad, worst-case assumptions about the cyber threats we are trying to protect against. In particular, we assume that attackers can exploit any possible attack vector. Most techniques described in the literature only protect against a narrow subset of possible attacks and no single MTD can protect against all possible attack vectors. This is handled by our model by providing the ability to combine multiple MTDs in a defense-indepth approach. We also make the worst-case assumption that no static defense can prevent an attack, as the attacker has virtually unlimited time to plan and execute an attack and zero-day exploits can evade static defenses. Only MTDs are considered to have an effect on the attacker's success rate, and even then, an MTD may not be perfect. We assume that attackers can be stopped or at least delayed by preventing them from acquiring accurate knowledge about the target system. Our primary focus here is on the reconnaissance phase, when that knowledge is gathered prior to planning and executing attacks. Our goal can be achieved by either preventing attackers from accessing that knowledge or delaying them until that knowledge is no longer useful.

Finally, we make several additional simplifying assumptions throughout the paper that we summarize here. Future work will allow us to revise many of our assumptions in order to further generalize our approach. We assume that services and weaknesses are time-invariant. We also assume that services and knowledge blocks are independent, but multiple services with dependencies could be modeled. We currently assume that each MTD has a predefined optimal configuration of its parameters, and that, if multiple MTDs affect a knowledge block, they do not interact and only the most effective one is considered.

Quantification Framework

In this section, we present the proposed quantification framework, which, as shown for the motivating example of Fig. 1, consists of four layers: (i) a timeinvariant service layer representing the set S of services to be protected; (ii) a weakness layer representing the set W of general classes of weaknesses that may be exploited; (iii) a knowledge layer representing the set K of all possible knowledge blocks required to exploit those weaknesses; and (iv) an MTD layer representing the set M of available MTD techniques.

Mathematical Model

The proposed MTD quantification framework can be formally defined as a 7tuple (S, R SW , W, R W K , K, R KM , M), where: (i) S, W, K, M are the sets (ii) R SW ⊆ S × W represents relationships between services and the common weaknesses they are vulnerable to; (iii) R W K ⊆ W × K represents relationships between weaknesses and the knowledge blocks required for an attacker to exploit them; and (iv) R KM ⊆ K × M represents relationships between knowledge blocks and the MTD techniques that affect them. The proposed model induces a k-partite graph (with

k = 4) G = (S ∪ W ∪ K ∪ M, R SW ∪ R W K ∪ R KM ).
Layer 1: Service Layer The first layer represents the set S of services we wish to protect against attacks. We assume that the services are time-invariant, i.e., the functionality of the services does not change over time, and services cannot be taken down to prevent attacks, as this action would result in a denial-ofservice. We only consider one service in the case studies presented in this paper, but the model could be extended to consider multiple interdependent services, similarly to how an exploit chain might occur within attack graphs [START_REF] Jajodia | Managing Cyber Threats: Issues, Approaches, and Challenges, Massive Computing[END_REF][START_REF] Wang | An attack graph-based probabilistic security metric[END_REF].

Layer 2: Weakness Layer The second layer represents the set of weaknesses W that services are vulnerable to. We choose general classes of weaknesses rather than specific vulnerabilities because there are too many vulnerabilities to enumerate, some vulnerabilities are unknown, and, depending on the MTD used (e.g., OS rotation), specific vulnerabilities may change over time. Using general classes of weaknesses when building the model makes them time-invariant.

The examples used in this paper draw these weaknesses primarily from MITRE's Common Weakness Enumeration (CWE) project [START_REF] Christey | CWE/SANS top 25 most dangerous software errors[END_REF], particularly from those known as the "Top 25 Most Dangerous Software Errors." Although many of the top software errors are primarily the result of bad coding practices and better solved at development time, the top software errors enabling exploits such as SQL Injection, OS Injection, and Classic Buffer Overflow can be addressed at runtime by MTDs (e.g., SQLrand) and make for good general categories of weaknesses. The Microsoft STRIDE Threat Model [START_REF] Howard | Writing Secure Code[END_REF] has also been used as a source of general threats in MTD research [START_REF] Soule | Quantifying minimizing attack surfaces containing moving target defenses[END_REF] and can fill in areas where CWE may be lacking. For example, Information Disclosure (eavesdropping) and Denial of Service are not specifically addressed by CWE. Our example shows two weaknesses, SQL Injection and Buffer Overflow. More weaknesses, such as OS Injection, might be included in a more complex example, while other weaknesses, such as Cross-Site Scripting, would not be applicable to this service.

Layer 3: Knowledge Layer The third layer represents the knowledge blocks K required to effectively exploit weaknesses. This knowledge is required to plan an attack even when no MTD is deployed (such as a victim's IP address) or it may be an additional piece of information required due to the use of an MTD. For example, SQLrand [START_REF] Boyd | SQLrand: Preventing SQL injection attacks[END_REF] adds a keyword to SQL commands, which must be known for a malicious user to perform SQL injection. We assume that knowledge blocks are independent and must be acquired using different methods. For example, IP address and port number should not modeled as separate knowledge blocks because a method to determine one would also reveal the other.

The relationship between the knowledge and weakness layers is many-tomany. A weakness may require several pieces of knowledge to be exploited, and a knowledge block may be key to exploiting several weaknesses. This layer may also be extended as new MTDs -disrupting new and different aspects of an attacker's knowledge -are developed.

In our example, we assume that, in order to execute a SQL Injection attack, the attacker must gather information about the service (e.g., name and version of the specific DBMS) and the network configuration (e.g., IP address). In order to execute a Buffer Overflow attack, an attacker must know the IP address and some information about the vulnerable memory locations. A higher-fidelity version of this model may take a knowledge block and break it down into smaller, more specific items that are specifically targeted by available MTDs.

Layer 4: MTD Layer The fourth layer of the model represents the set M of available MTDs. As MTD techniques provide probabilistic security, we model the impact of an MTD M i on the attacker's effort to acquire knowledge K j by associating a probability P i,j -representing the attacker's success rate -with the relation (K j , M i ). As mentioned in the Section 3, when only static defenses (i.e., no MTD) are deployed, an attacker will acquire the necessary knowledge without significant effort, which we model by associating a probability of 1.

For example, if technique M 1 in Fig. 1 (Service Rotation) reduces an attacker's likelihood of acquiring knowledge block K 1 (i.e., correct version of the service) by 60%, we would label that edge with P 1,1 = 0.4. If an MTD delays an attacker by some factor, we can also express that as a probability that the attacker will not gather the correct information in a timely manner. For example, an MTD that expands addressable memory by a factor of 10 might reduce the attacker's probability of success to 0.1, so P i,j = 0.1. The exact methodology for determining the value of P i,j may vary from MTD to MTD, and we are investigating this problem as a separate line of research that goes beyond the scope of this paper. Specifically, we are developing a general approach to model the tradeoff between cost and effectiveness of MTD techniques, as we vary the values of a technique's tunable parameters and other aspects of the attacker/defender interaction. Ultimately, this approach will enable us to identify the optimal configuration for each technique. Therefore, in this paper, we assume that such optimal configuration has already been identified for each available MTD technique, along with the corresponding value of P i,j and the corresponding cost.

Expressing MTD effectiveness in terms of the probability an attacker will succeed in acquiring required knowledge enables us to analyze multiple techniques using a uniform approach that yields values in the [0,1] range, with a theoretically perfect MTD yielding P i,j = 0, and a completely ineffective MTD yielding P i,j = 1. In our example, we use service rotation to disrupt knowledge about the version of the service, and naïvely assume that rotating between 4 services reduces the attacker's probability of gathering the correct information to P 1,1 = 0.25. We apply an IP address rotation scheme to mask the victim's IP address. It has been shown that perfect shuffling reduces the attacker's likelihood of guessing the correct IP address by 37% [START_REF] Carroll | Analysis of network address shuffling as a moving target defense[END_REF]. Using a conservative estimate, we assume P 2,2 = 0.75. Finally, to protect knowledge of the memory layout, we use a dynamic ASLR scheme. Although dynamic ASLR only adds a single bit of entropy compared to typical ASLR [START_REF] Shacham | On the effectiveness of address-space randomization[END_REF], this further delays the attacker, resulting in a probability P 3,3 = 0.5 of gathering the correct information.

Computing MTD effectiveness

We compute an MTD's effectiveness starting from layer 4 of the model and working our way down to find the overall probability of attacker's success. First, we define P (K j ) as the probability that the attacker has the correct information about knowledge block K j , and compute P (K j ) for each K j in layer 3, based on the active MTDs affecting it. If there is no active MTD, we assume that the attacker is guaranteed to obtain that information, i.e., P (K j ) = 1.

In our example, each knowledge block is affected by only one MTD. When multiple MTDs affect the same knowledge block, we make the simplifying assumption that the resulting effect is driven by the best-performing MTD. Thus:

P (K j ) = 1, if M i ∈ M s.t. (K j , M i ) ∈ R KM ∧ active(M i ) min Mi∈M s.t. (Kj ,Mi)∈R KM P i,j ∧ active(M i ), otherwise (1)
A possible improvement to the model would be to capture the effect of multiple MTDs acting on the same knowledge block by using a function modeling either diminishing returns or some other interaction between multiple MTDs.

Next, we determine the probability P (W k ) that an attacker has gained all the knowledge required to exploit a given weakness W k . Since each knowledge block is independent, this is simply the product of the probabilities associated with all knowledge blocks leading to it, as shown by Equation 2.

P (W k ) = Kj ∈K s.t. (W k ,Kj )∈R W K P (K j ) (2)
In our example, when calculating P (W 1 ) and P (W 2 ) for SQL Injection and Buffer Overflow, respectively, we obtain P (W 1 ) = 0.25 • 0.75 = 0.1875 and P (W 2 ) = 0.75 • 0.50 = 0.375.

Finally, we determine the defender's utility U gained by deploying MTD techniques based on the reduced probability of exploit for each class of weaknesses. In this work, the utility is defined as a function of the probability P (S l ) that an attacker can compromise a service S l by exploiting any of the weaknesses leading to it. P (S l ) can be computed as the probability of the union of non-mutually exclusive events, using the Inclusion-Exclusion Principle [START_REF] Chen | Reduced recursive inclusion-exclusion principle for the probability of union events[END_REF]. With respect to our running example, P (S 1 ) can be computed as follows:

P (S 1 ) = P (W 1 ∪ W 2 ) = P (W 1 ) + P (W 2 ) -P (W 1 ∩ W 2 ) (3)
As W 1 and W 2 are not necessarily independent (as shown in this example), we cannot assume P (W 1 ∩ W 2 ) = P (W 1 ) • P (W 2 ). Instead, we must express each P (W ) in terms of its corresponding independent knowledge blocks K j , that is P (W 1 ) = P (K 1 ) • P (K 2 ), P (W 2 ) = P (K 2 ) • P (K 3 ), and P (W 1 ∩ W 2 ) = P (K 1 ) • P (K 2 ) • P (K 3 ), and then express P (S 1 ) as a function of probabilities P (K j ):

P (S 1 ) = P (K 1 ) • P (K 2 ) + P (K 2 ) • P (K 3 ) -P (K 1 ) • P (K 2 ) • P (K 3 )
which results in P (S 1 ) = 0.25 • 0.75 + 0.75 • 0.5 -0.25 • 0.75 • 0.5 = 0.469 For graphs with 3 or more weaknesses W * ⊆ W, we can expand Equation 3to the generalized form of the Inclusion-Exclusion Principle [START_REF] Chen | Reduced recursive inclusion-exclusion principle for the probability of union events[END_REF]:

P W k ∈W * W k = |W * | i=1   (-1) i-1 • W ∈2 W s.t. |W |=i P   Wj ∈W W j    
Computing the probability of the union of multiple events is an NP-hard problem that cannot be solved in better than O(2 n ) time [START_REF] Chen | Reduced recursive inclusion-exclusion principle for the probability of union events[END_REF]. However, the general nature of the weaknesses in layer 2 of the model limits their numberas opposed to vulnerabilities which may number in the thousands -keeping the computing time manageable.

After computing P (S l ), we can easily compute the defender's utility as U = 1 -P (S l ). Besides this simple approach, the utility could be a sigmoid function of P (S l ) with an inflection point centered around a desired effectiveness. Such functions are commonly used in autonomic computing [START_REF] Alomari | An autonomic framework for integrating security and quality of service support in databases[END_REF]. The complete computation for each of the values in our example is shown in Fig. 2. Note that this choice of utility function relies upon the expectation that at least some measure of protection will be guaranteed for at least one knowledge block for each weakness, otherwise the attacker will be guaranteed to exploit that weakness and reduce the utility to 0. To handle this issue, utility can be defined as a function of the probabilities to exploit each weakness.

Experimental Evaluation

We now present a more complex example to demonstrate the capabilities of our model. As seen in Fig. 3, we consider the same basic service but protect against two additional classes of weaknesses, OS Injection [START_REF] Christey | CWE/SANS top 25 most dangerous software errors[END_REF] and Eavesdropping (related to Information Disclosure from the STRIDE model [START_REF] Howard | Writing Secure Code[END_REF]).

In this case study, more fine-grained knowledge blocks have been considered in order to provide more detail or to fit the specific MTDs selected for the case study. For example, knowledge block Knows(memory) has been broken down into separate blocks related to system call mapping, memory address, and stack direction. Similarly, SQL Injection now explicitly requires knowledge of keywords appended to SQL commands and some knowledge of the database schema, both of which are disrupted by SQLRand. Most importantly, we can now observe the many-to-many relationships between weaknesses, knowledge blocks, and MTDs, and conclude that finding the optimal solution is no longer trivial. However, using approximate yet reasonable values of P i,j for each MTD and cost constraints, we can determine the final utility as a function of selected MTDs using the steps previously shown and find an optimal solution using a problem solving method, such as stochastic hill climbing or evolutionary methods.

As a proof of concept, we can take the model in Fig. 3 and perform all the necessary computations programatically. As mentioned earlier, we are studying the relationship between cost and effectiveness of MTD techniques as part of P(S 1 ): Chance attacker reaches goal U = Utility factor: chance of 0 exploits occurring P i,j = probability of attacker having knowledge, based on disruption from 1 MTD P(K j ) = probability of attacker having knowledge, based on all MTDs P(W k ) = probability of attacker having all required knowledge (exploit occurs) Fig. 3 another line of research. For the purpose of this paper and the evaluation we are presenting, we obtained qualitative values of P i,j and cost from an expert survey [START_REF] Farris | Quantification of moving target cyber defenses[END_REF] which estimates the relative effectiveness and cost of several MTD techniques by grouping them into coarse-grained categories of Low, Medium, or High. Whether or not an MTD is active can be treated as a Boolean variable, with inactive MTDs implying an attacker's probability of success of 1 and a cost of 0. The values from a sample MTD setup are shown in Table 1. The interim calculations for the probabilities of each knowledge block being acquired and each weakness being able to be exploited are also shown.

Applications

In these section, we discuss two different applications of our framework.

Comparing MTDs

Given a set M of MTD techniques, we want to identify the one that provides the highest overall utility. With respect to the example of Fig. 3, we start from the baseline deployment, shown earlier in Table 1, including M 6 (Mutable Networks) and M 8 (ASLR) to ensure we have a utility value to compare with. We then measure the updated utility value after individually adding each of the other MTDs to our baseline deployment. From the results reported in Table 2, we find that M 3 (SQLRand) offers the greatest increase in utility, with M 1 , M 2 , and M 3 being the only ones offering any increase at all. To explain these results, we observe that there is a lower bound on P (S 1 ) that translates into an upper bound on U , defined by max(P (W 1 ), P (W 2 ), P (W 3 (W 4 )).

In other words, the overall defense can only be as strong as the protection against exploitation of its most vulnerable weakness, which in turn benefits from the deployment of multiple MTDs. Therefore, given the baseline conditions, only an MTD that affects the most vulnerable weakness will yield any improvement in our utility value. This procedure could be used iteratively in an attempt to find an optimal solution in a greedy manner, but there would have to be some way to handle cases where no MTD adds any utility (such as random selection).

Selecting Optimal Defenses

Given a set M of MTDs and a budget, we would like to select the optimal set of MTDs that yield the highest utility with a total cost within the budget. As we now have a tool to evaluate the utility of any MTD deployment, we can also solve for the optimal selection of MTDs, given the constraints that the deployment of each MTD is a Boolean variable (either active or not) and that the sum of the costs of selected MTDs be under our budget. For the purpose of evaluating our framework and making the problem interesting, we selected a value of the budget (120) halfway between 0 and the total cost of deploying all available MTDs (i.e., 210). This choice ensured that a solution with utility greater than 0 would be found and that approximately half the MTDs would be chosen as part of the optimal solution. We solved using the generalized reduced gradient non-linear algorithm [START_REF] Lasdon | Nonlinear optimization using the generalized reduced gradient method[END_REF] with random restarts to eliminate finding local maxima. After solving, we obtain an optimal solution with the selected MTD highlighted with a thicker red outline in Fig. 4 and detailed results, including margins of error for our estimates of effectiveness, shown in Table 3.

We can observe that our choice of a utility function forces the selection of a variety of MTDs such that each weakness has at least one MTD affecting one of its knowledge blocks and that protection is evenly distributed over the 4 weaknesses. Visually, we can also observe that an MTD with the ability to affect multiple knowledge blocks is inherently more powerful than one that only affects one. However, if their cost is too high or effectiveness too low, it will still not be chosen as part of an optimal solution. Similarly, an MTD that only affects one knowledge block may be chosen if it is effective, low-cost, or affects a knowledge block that still receives relatively weak protection from other MTDs.

Extending the Framework

Our framework can accommodate any existing MTD as long as we can identify the knowledge blocks it affects, the extent to which it disrupts that knowledge, and how it relates to the weaknesses we plan to protect against. Another important feature of our framework is the ability to be extended to accommodate any future MTD that may be developed. A new MTD that affects existing knowledge blocks may be simply added to the MTD layer of the model, while an MTD that works in ways we have not yet considered might also require the addition of new knowledge blocks. Even a new class of weaknesses could be added to the model if the situation warrants it, making our model "future-proof" against new developments in cyber threats.

Conclusions and Future Work

In this paper, we have introduced a framework for quantifying moving target defenses. Our approach to quantifying the benefits of MTDs yields a single, probability-based utility measure that can accommodate any existing or future MTD, regardless of their nature. Our multi-layered approach captures the relationship between MTDs and the knowledge blocks they are designed to protect and the relationship between knowledge blocks and generic classes of weaknesses that can be exploited using that knowledge. We have shown through case studies that we can compute the joint effectiveness of multiple MTDs as a function of their individual effectiveness and, by doing so, we can make informed deci-sions about which MTD or set of MTDs provide better protection based on the security requirements or cost constraints.

Although the work presented in this paper represents a significant step towards effective MTD quantification, several limitations still exist and will be addressed as part of our planned future work. Specifically, limitations exist in the following areas: (i) probability computation -our methods for computing the probability P i,j of knowledge disruption provide rough estimates, so a procedure needs to be developed to accurately assess the effectiveness of any MTD;

(ii) cost modeling -currently, we adopt a very simple notion of cost, and use cost just as an additional constraint, whereas a more sophisticated notion of cost could be introduced and taken into account in the computation of utility values; and (iii) choice of utility function -the proposed utility function is based on the assumption that all weaknesses need to be at least partially protected by MTDs to prevent the utility from dropping to 0, therefore, if the risk of leaving a specific weakness unprotected can be accepted, other classes of utility functions could be explored. To address these limitations and further refine our model, we plan to work on several aspects of the framework, as briefly described below.

Implementation and validation. To validate the model, we plan to deploy multiple MTDs on our computing infrastructure and then examine their effectiveness both in isolation -in order to determine the value of P i,j for each MTD -and jointly -in order to accurately study the combined cost and performance.

Application to multiple attack phases. Our model aims at disrupting an attacker's knowledge in the reconnaissance phase of the cyber kill chain. While this may be the most cost-effective way to approach cyber security, no defense is perfect, and we need to ensure multiple layers of defense. Some MTDs can disrupt an attacker's ability to maintain a foothold in the system, so we plan to extend our framework to model this additional class of MTDs.

Application to dependent services. Our framework currently models only independent services. Similar to attack graphs, an attacker may need to execute a sequence of exploits to reach a specific goal. Thus, we plan to extend our framework by introducing a meta-model that captures the relationships between services and the MTDs that can protect them from multi-step attacks.

Heuristics. Because of the O(2 n ) runtime to evaluate utility with the current model, it may be necessary to develop heuristics to speed up the evaluation in the case that the number of weaknesses grows to the point where using the model becomes infeasible.

Confidence intervals. Because of the level of uncertainty of our probabilistic values, we may not have a completely accurate utility value. With enough experimental samples, we could introduce confidence intervals into our assertion that a certain MTD or set of MTDs has a higher utility.

Fig. 1 .

 1 Fig. 1. Quantification Framework Layers

5 Fig. 2 .

 52 Fig. 2. Computing MTD Effectiveness

Fig. 4 .

 4 Fig. 4. Case Study Optimal Configuration

Table 1 .

 1 . Case Study Quantification Framework Sample Case Study Evaluation

	MTD	P i,j	Cost Active? P i,j (effective) Cost (effective)
	M 1 (Service Rotation)	P 1,1 0.500 15	No	1.000	0
	M 2 (Intrusion Tolerant Systems) P 2,1 0.900 25	No	1.000	0
		P 2,4 0.900			1.000
		P 2,5 0.900			1.000
	M 3 (SQLRand)	P 3,2 0.300 20	No	1.000	0
		P 3,3 0.300			1.000
	M 4 (IP Rotation/MOTAG)	P 4,4 0.900 25	No	1.000	0
	M 5 (OS Rotation)	P 5,5 0.700 15	No	1.000	0
	M 6 (Mutable Networks)	P 6,4 0.500 20	Yes	0.500	20
		P 6,10 0.500			0.500
	M 7 (Multivariant Systems)	P 7,6 0.500 20	No	1.000	0
		P 7,8 0.500			1.000
	M 8 (ASLR)	P 8,7 0.500 10	Yes	0.500	10
	M 9 (TALENT)	P 9,5 0.500 20	No	1.000	0
		P 9,9 0.500			1.000
	M 10 (Reverse Stack Execution)	P 10,8 0.500 20	No	1.000	0
	M 11 (Distraction Cluster)	P 11,10 0.500 20	No	1.000	0
	Knowledge:				Total Cost	30
	Knows(application)	1.000			Total Budget	120
	Knows(keyword)	1.000
	Knows(DBschema)	1.000			Cost:
	Knows(IP)	0.500			High	25
	Knows(OS)	1.000			Medium	15
	Knows(syscall mapping)	1.000			Low	5
	Knows(mem address)	0.500
	Knows(stack dir)	1.000			Effectiveness:
	Knows(instr set)	1.000			High	0.3
	Knows(path)	0.500			Medium	0.5
					Low	0.9
	Chance of attack success:
	SQL Injection	0.500
	OS Injection	0.250
	Buffer Overflow	0.250
	Easvesdropping	0.250
	Chance of attacker success:	0.500
	Utility	0.500

Table 2 .

 2 Improvement from adding MTDs

	MTD M1 (Service Rotation) M2 (Intrusion Tolerant Systems) M3 (SQLRand) All others
	Utility	0.5625	0.513	0.614	0.5
	Delta	0.0625	0.013	0.114	0.0

Table 3 .

 3 Case Study Optimal Configuration

	MTD	P i,j	C Active? Pi,j (effective) C (effective)
	M 1 (Service Rotation)	P 1,1	0.500 ± 0.05 15	Yes	0.500 ± 0.05	15
	M 2 (Intrusion Tolerant Systems)	P 2,1	0.900 ± 0.05 25	No	1.000	0
		P 2,4	0.900 ± 0.05			1.000
		P 2,5	0.900 ± 0.05			1.000
	M 3 (SQLRand)	P 3,2	0.300 ± 0.05 20	Yes	0.300 ± 0.05	20
		P 3,3	0.300 ± 0.05			0.300 ± 0.05
	M 4 (IP Rotation/MOTAG)	P 4,4	0.900 ± 0.05 25	No	1.000	0
	M 5 (OS Rotation)	P 5,5	0.700 ± 0.05 15	No	1.000	0
	M 6 (Mutable Networks)	P 6,4	0.500 ± 0.05 20	Yes	0.500 ± 0.05	20
		P 6,10	0.500 ± 0.05			0.500 ± 0.05
	M 7 (Multivariant Systems)	P 7,6	0.500 ± 0.05 20	Yes	0.500 ± 0.05	20
		P 7,8	0.500 ± 0.05			0.500 ± 0.05
	M 8 (ASLR)	P 8,7	0.500 ± 0.05 10	Yes	0.500 ± 0.05	10
	M 9 (TALENT)	P 9,5	0.500 ± 0.05 20	Yes	0.500 ± 0.05	20
		P 9,9	0.500 ± 0.05			0.500 ± 0.05
	M 10 (Reverse Stack Execution)	P 10,8	0.500 ± 0.05 20	No	1.000	0
	M 11 (Distraction Cluster)	P 11,9	0.500 ± 0.05 20	No	1.000	0
	Knowledge:					Total Cost	105
	Knows (1,application)	0.500 ± 0.05				Total Budget	120
	Knows (1,keyword)	0.300 ± 0.05
	Knows (1,DBschema)	0.300 ± 0.05				Cost:
	Knows (1,IP)	0.500 ± 0.05				High	25
	Knows (1,OS)	0.500 ± 0.05				Medium	15
	Knows (1, syscall mapping)	0.500 ± 0.05				Low	5
	Knows (1, Mem Address)	0.500 ± 0.05
	Knows (1,stack dir)	0.500 ± 0.05				Effectiveness:
	Knows (1,instr set)	0.500 ± 0.05				High	0.3 ± 0.05
	Knows (1,path)	0.500 ± 0.05				Medium	0.5 ± 0.05
						Low	0.9 ± 0.05
	Chance of attack success:
	SQL Injection	0.023 ± 0.006
	OS Injection	0.063 ± 0.013
	Buffer Overflow	0.063 ± 0.013
	Easvesdropping	0.250 ± 0.035
	Chance of attacker success:	0.313 ± 0.043
	Utility	0.687 ± 0.043

This work was partially supported by the Army Research Office under grants W911NF-13-1-0421 and W911NF-13-1-0317, and by the Office of Naval Research under grant N00014-13-1-0703. We acknowledge Dr. George Cybenko and Dr. Sushil Jajodia for the invaluable inputs they provided in the early stages of this work.

1 http://www.trespass-project.eu


Lin Yuan
email: lin.yuan@epfl.ch

Joël Theytaz
email: joel.theytaz@epfl.ch

Touradj Ebrahimi
email: touradj.ebrahimi@epfl.ch

Context-Dependent Privacy-Aware Photo Sharing based on Machine Learning

Keywords: Privacy protection, online social network, photo sharing, access control, decision making, context, machine learning

Photo privacy has raised a growing concern with the advancements of image analytics, face recognition, and deep learning techniques widely applied on social media. If properly deployed, these powerful techniques can in turn assist people in enhancing their online privacy. One possible approach is to build a strong, automatic and dynamic access control mechanism based on analyzing the image content and learning users sharing behavior. This paper presents a model for contextdependent and privacy-aware photo sharing based on machine learning. The proposed model utilizes image semantics and requester contextual information to decide whether or not to share a particular picture with a specific requester at certain context, and if yes, at which granularity.

To evaluate the proposed model, we conducted a user study on 23 subjects and collected a dataset containing 1'018 manually annotated images with 12'216 personalized contextual sharing decisions. Evaluation experiments were performed and the results show a promising performance of the proposed model for photo sharing decision making. Furthermore, the influences of different types of features on decision making have been investigated, the results of which validate the usefulness of pre-defined features and imply a significant variance between users sharing behaviors and privacy attitudes.

Introduction

Wide spread of smart mobile devices and online social networks (OSNs) make photo sharing an easy and popular activity. However, it has also raised concerns on privacy since the shared content reveals substantial sensitive information about people. Most social networking or photo sharing services provide access control for users to manage their privacy. However, users need to manually set their sharing policies in only a static manner, without the possibility to share their photos to different groups of people dependent to contexts, e.g. the location, time or even nearby people of potential viewer. Most access control mechanisms enforce only binary sharing options, namely "Yes" or "No", which may not provide the best experience when a user just wants to disable partial information in photo sharing. With the latest progress in image analytics, pattern recognition, and deep learning techniques, large scale information is mined from the shared multimedia content. Although seemingly compromising privacy, those techniques can in turn be used to enhance privacy, in such a way of helping people estimate the privacy value of their content or control the access of their content automatically and dynamically.

In this paper, we present a machine learning based model that can accurately predict users photo sharing decisions based on their past decisions. To make photo sharing decisions, the proposed model takes into account not only the content of an image, but also the context information about the image capture and potential requester. To validate the proposed model, we conducted a user study on 23 subjects and three sets of evaluation experiments.

The rest of the paper is structured as follows. Section 2 introduces related works. Section 3 describes in detail the proposed model. Then Section 4 and Section 5 present the user study and performance evaluation. Finally, Section 6 outlines some discussions and Section 7 summaries the paper.

Related Work

A number of studies have been focused on understanding users privacy concern on photo sharing, as well as the potential privacy implications via both subjective [START_REF] Ahern | Over-exposed?: Privacy patterns and considerations in online and mobile photo sharing[END_REF][START_REF] Besmer | Moving beyond untagging: Photo privacy in a tagged world[END_REF] and objective [START_REF] Friedland | Cybercasing the joint: On the privacy implications of geo-tagging[END_REF][START_REF] Pesce | Privacy attacks in social media using photo tagging networks: A case study with facebook[END_REF] studies. A number of approaches to privacy protection in photo sharing have been proposed, including usage control scheme in distributed OSNs [START_REF] Cutillo | Privacy preserving picture sharing: Enforcing usage control in distributed on-line social networks[END_REF], Secure JPEG scrambling image visual information [START_REF] Yuan | Privacy-preserving photo sharing based on a secure JPEG[END_REF][START_REF] Yuan | Secure jpeg scrambling enabling privacy in photo sharing[END_REF][START_REF] Yuan | Privacy-preserving photo sharing based on a public key infrastructure[END_REF], separate coding and sharing of JPEG image by P3 [START_REF] Ra | P3: Toward privacy-preserving photo sharing[END_REF] and tag-based access control [START_REF] Klemperer | Tag, you can see it!: Using tags for access control in photo sharing[END_REF]. In addition, a substantial research effort has been made on estimating the privacy value or detecting privacy-sensitive objects in images. These works include private/public image classifications and privacy-sensitive visual information detection, based on not only learning low-level image features (color, edge, faces and SIFT) [START_REF] Zerr | Privacy-aware image classification and search[END_REF], but also deep learning approaches such as Convolutional Neural Network (CNN) [START_REF] Tran | Privacy-cnh: A framework to detect photo privacy with convolutional neural network using hierarchical features[END_REF][START_REF] Tonge | Image privacy prediction using deep features[END_REF].

Another branch of research has been focused on context-aware information sharing in the scenario of social networks or cloud services. Smith et al. [START_REF] Smith | Social Disclosure of Place: From Location Technology to Communication Practices[END_REF] provided an early investigation on solutions to enable people to share contextual information in mobile social networks. Wiese et al. [START_REF] Wiese | Are you close with me? are you nearby?: investigating social groups, closeness, and willingness to share[END_REF] investigated the impact of various factors on people's willingness to share information. Harkous et al. [START_REF] Harkous | C3P: Context-Aware Crowdsourced Cloud Privacy[END_REF] present a conceptual framework named C3P for automatic estimation of privacy risk of data based on the sharing context. Bilogrevic et al. [START_REF] Bilogrevic | A machine-learning based approach to privacy-aware information-sharing in mobile social networks[END_REF] present SPISM, an information-sharing system that predicts (semi-)automatically sharing decision, based on personal and contextual features. Despite the substantial works on contextual information sharing, very few have considered context information for privacy protection in online photo sharing. To the best of our knowledge, this paper is the first attempt to investigate the feasibility of deploying both content-related and contextual features of images, to automatically make or "recommend" photo sharing decisions. Requester

Request Context • Identity • Location • Neighbors • Time, etc.

Training Decision

Access

Pictures on database Fig. 1. Framework of a photo sharing system based on the proposed model.

3 A Model for Context-/Privacy-Aware Photo Sharing

Security Assumption and Operating Principle

First of all, we assume the photo sharing service providers are trustworthy. Users allow the service to conduct necessary analysis on their photos, and the system is granted the right to enforce access control of users photos. Fig. 1 illustrates a photo sharing architecture of the proposed model. The operating procedures of the model can be described by the following story: Alice (the sender, who wants to upload and share photos with online friends) uploads a set of pictures on the photo sharing service, and the service system analyzes each picture and extracts a set content and contextual features about those pictures. Meanwhile, the system asks Alice a set of questions on her willingness to share each picture to specified individuals in various scenarios. These individuals can be selected from those who visited Alice's profile recently or frequently. Each scenario describes a certain context of a possible requester, who attempts to visualize a picture shared by the sender. The context includes the identity (either real name or social group), location, nearby people and the time when the requester tries to visualize the image. The system then trains a classifier based on Alice's answers for different photos in different scenarios. On the other side, Bob (the requester) visits the profile page of Alice. With the help of the classifier, the system analyzes Bob's context and Alice's photo information, to decide whether or not to show certain photos to Bob, and if yes, at which granularity.

Feature Definition

To train such a classifier, we considered two groups of features: Image Semantic Features (I) and Requester Contextual Features (R). Instead of using low-level image features such as color, texture, composition and SIFT, we believe higher-level semantic features have more immediate relations with privacy.

Image: Category

Major category of the picture, selected from the eight categories identified in Instagram pictures [START_REF] Hu | What we Instagram: A first analysis of instagram photo content and user types[END_REF]: Friends, Activity, Selfie, Food, Pets, Gadget, Fashion and Captioned photo.

I A

Image: Activities Activities involved in the picture, selected from 26 keywords partially defined by [START_REF] Castro | Predicting daily activities from egocentric images using deep learning[END_REF]: working, meeting, reading, presentation, resting, chatting, socializing, family, friends, vacation, TV, cooking, eating, drinking, cleaning, shopping, exercising, traveling, walking, landscape, city, concert, sporting, gaming, gadget and pets.

Who I P

Image: # of People

The number of people in the picture.

Image: Identities

The existence of different identities in the picture. Eight

Image: Location

The semantic location where the image was captured, selected from 12 major location categories adopted from Foursquare Location Categories. Image: Loc. Coordinates Latitude and longitude of the image capture location.

Image: Loc. Frequency

The frequency of the sender being present in such place, selected from Rarely, Sometimes, Often and Almost everyday.

R L Requester: Location

Semantic location of the requester, categorized in Unknown, Friends home, His/her own home, Work place and Public place.

When I T

Image: Time

The time of photo capture in a float value, e.g. 14.5 denotes 2:30 PM.

Image: Day

The day (in a week) of photo capture, selected from Monday to Sunday.

These features include the image category, number/identities of people in image, activities or objects in image and the location and time of image capture. The contextual features of the requester include the requester's identity, location, nearby people and time.

A detailed description of all the features used in our experiments, grouped in different aspects of context, is shown in Table 1. Note that the time of requester is not used in the current experiment because it would be too cumbersome for subjects to read and analyze the complete information containing all contexts.

Photo Sharing Decisions

We defined three photo sharing decisions, corresponding to different levels of photo information disclosure. The three decisions and corresponding descriptions presented in the user study are listed in the following: Decision 1 -Do NOT Share: No, I don't want to share the picture. Decision 2 -Partially Share: Yes, but with some image region protected or/and metadata (GPS, time, etc.) removed. Decision 3 -Entirely Share: Yes, I want to share the picture completely. The reasons of using the specific three sharing decisions instead of conventional binary decisions ("Yes" or "No") are twofold: First, in many scenarios of online photo sharing, people may want to simply remove partial privacy-sensitive visual information in an image, such as ID card, license plate or their children faces. Second, most images shared from smart mobile devices contain metadata such as geotags, camera model and time, which could also compromise privacy. Therefore, an option should be provided for users to partially protect and share their image content.

User Study and Data Collection

We conducted a study that put participants in personalized photo sharing scenarios, and collected an image dataset containing manual-annotated image semantic features and contextual sharing decisions.

To conduct the user study, we developed an Android app1 , named ProShare S. The application allows a user to create an account, take pictures, conduct a set of surveys for each, protect privacy-sensitive image regions, and finally upload them to a dedicated server. The workflow of a user study using ProShare S is illustrated in Fig. 2. Particularly, the survey part is structured in two sets of questionnaires:

Q1 -Image Semantic Information The first questionnaire (Q1) requires the user to add necessary image semantic tags. This questionnaire appears once a picture has been taken from either gallery or camera. The questions in Q1 cover all the semantic features defined in Section 3.2. A build-in face detector offered by Android API is applied to count the number of people in image, which can be manually modified if not correct. Location coordinates and capture time are automatically extracted from image metadata.

Q2 -Contextual Photo Sharing Decisions

Once Q1 is finished, the user is directed to the second questionnaire (Q2), where he/she is presented with 12 sharing contexts/questions. For each context, the user needs to decide how he/she would like to share the picture with the specific requester, by selecting one of the three decisions defined in Section 3.3. An example context is "Would you share this picture with a close friend, when he is at a public place with other people?" The 12 contexts/questions are selected in a special way such that each of the six requester identities appears twice in a random order, with the other contextual features (gender, location, nearby people) sampled at random. In the study, basic user profile is also collected through the App. We therefore present the sharing contexts adaptively based on user's profile. For instance, for a female user we present the requester as "your boyfriend" instead of "girl or boyfriend". We recruited 23 volunteers to participate in our user study, and assigned each of them a task of uploading at least 50 daily pictures of their own and completing corresponding surveys using ProShare S. Each subject was required to complete the task within a week and to try to cover a wide range of image content2 . Finally, 20 out of the 23 subjects successfully finished the required task. We therefore kept only the data of the 20 effective subjects for the later evaluation. A total of 1'018 images including 12216 sharing decisions were contributed by the 20 subjects, each providing 50.9 images on average. Fig. 3 shows the histogram of images in each category and the contextual sharing decisions made on all the images.

Evaluation and Analysis

Methodology

To evaluate the performance of the proposed model for decision making, we conducted three sets of experiments based on the data collected from our user study. We take the working hypothesis that users photo sharing behaviors and privacy attitudes are highly subjective and the difference in users behaviors may cause the proposed model to perform differently between subjects.

The first experiment focused on the performance of the proposed model with respect to each user, namely, within-subject analysis. In the second experiment, we explored a universal one-size-fits-all classifier trained on all users data for predicting a new user's decisions. In the third experiment, we investigated the influences of different image and requester features on the decision making performance of the proposed model.

The WEKA machine learning library [START_REF] Hall | The WEKA data mining software: An update[END_REF] was used in experiments and three representative classification methods were considered: logistic regression, support vector machine (SVM) and random forest. We started with a preliminary test by running a 10-fold cross validation on each user's data using the three methods and random forest always outperformed the other two. We therefore kept using random forest for the rest of the experiment.

To evaluate the decision making performance, the following metrics are used:

-Correct Decision rate: The proportion of correctly predicted decisions.

-Over-Sharing rate: The proportion of cases where image information is shared more than what user expect to share, which compromises privacy. -Under-Sharing rate: The proportion of cases where image information is shared less than what user expects to share, which may compromise usability. -Kappa statistic: Cohen's kappa score [START_REF] Viera | Understanding interobserver agreement: the kappa statistic[END_REF] that measures the chance-corrected agreement between predicted and ground truth decisions.

Within-Subject Analysis

In the first experiment, we used different proportions (from 10% to 90%) of each subject's data to train a classifier, and evaluated the classifier on the rest of the data (evaluation set). This is to examine trade-off between user-burden and prediction accuracy of the proposed model. The evaluation results measured by different metrics across the 20 subjects are shown as box plots in Fig. 4.

In this figure, one observes that the median correct decision rate has already reached 0.75 at a training set of only 10%, which corresponds to only 5 images in average. This means we could already build an acceptable model for half of the users using a very small number of images and their decisions. Above the training set of 50%, most users obtained the correct decision rate higher than 0.8. The median Kappa score at the training set of 10% is below 0.5 and rapidly reaches 0.6 at the training set of 20%. Above the training size of 60%, an almost perfect prediction is observed for half of the users with a median Kappa statistic greater than 0.8. On the other hand, both the over-sharing and under-sharing rates of most users are very low, even at the training set of 10%. However, we observe the over-sharing rate is always higher than the under-sharing rate. A possible explanation is that most users tend to share images and the numbers of different decisions in the dataset are imbalanced. From the results, one also observes a significant variance between users. At the training size of 10%, the maximum difference in correct decision rate between users is up to 0.44. At the training size of 80%, where the optimal performance is obtained for most of the users, such difference still remains around 0.2. Such results agree with our hypothesis made in the beginning of this section that users subjective behaviors may influence the performance of the proposed model.

Cost-Sensitive Decision Making

To address the issue of over-sharing, we introduced the cost-sensitive learning [START_REF] Elkan | The foundations of cost-sensitive learning[END_REF] in our decision making core. The aim is to evaluate the extent to which incorrect decisions can be biased towards the undersharing cases instead of over-sharing, when users concern their privacy more than usability. We specified different error-penalties C i→j (> 1) for over-sharing cases and the penalty of 1 for under-sharing. Therefore, the training process tries to minimize the following cost:

1 i<j 3 (C i→j × N i→j + 1 × N j→i )
, where N i→j denotes the number of cases where Decision i is misclassified classified as Decision j. Specially, we assigned a double error-penalty 2c for the over-sharing cases C 1→3 compared to the other two over-sharing cases. This is because a mistake by classifying "Do NOT Share" to "Entirely Share" may severely compromise privacy. The cost matrix for the cost-sensitive learning is shown in Table 2.

We experimented with a set of values for c (from 1.5 to 5), on each user's data using the same random forest classification. The results at c = 2 and c = 4 are shown in Fig. 5. With an error-penalty c = 2, the over-sharing rate is greatly reduced to a level lower than the under-sharing rate. When increasing c to 4, the over-sharing rate is further reduced, in sacrifice of a significant increase on the under-sharing rate. This indicates a significant trade-off between the capability of privacy protection and system usability. In any cases of cost-sensitive learning, the overall correct decision rate and Kappa statistic do not change much, as the introduced error-penalty mainly acts as a parameter to tune the weights of different incorrect decisions. Fig. 6. Performance of a One-Size-Fits-All classifier.

One-Size-Fits-All Model

In the second experiment, we evaluated a one-size-fits-all model, to examine the potential of building a global classifier trained on the data of all users, to make or "recommendation" decisions for new users. To be fair, for each subject i, we trained a classifier with random forest on the data of the remaining subjects, which was then evaluated on the data of subject i. Cost-sensitive learning was also included in this experiment for comparison. The results over all the 20 subjects are shown in Fig. 6. The median correct decision, over-/under-sharing rates and the Kappa statistic without cost-sensitive learning are 0.636, 0.218, 0.155 and 0.348 respectively. With cost-sensitive learning, the over-sharing rates are reduced below under-sharing, without greatly degrading the correct decisions and Kappa score. The overall performance of such a one-size-fits-all model is not as good as the personalized classifier built on each user's own data. This again implies that users may have very different behaviors and privacy attitudes towards photo sharing. However, such a classifier could already provide an acceptable performance better than a random guess. This experiment provides the insight of building a one-size-fits-all classifier to predict or "recommend" photo sharing decisions for a new user, until the user has enough data to build a personalized classifier.

Influences of Features on Decision Making

At the end, we investigated the influences of different types of features on users photo sharing decisions and on the performance of our prediction model. First, the histograms of three sharing decisions distinguished by different types of features are shown in Fig. 7. The variation in distributions over different feature values indicates the degree of influence of a particular type of features. One observes a significant difference in decision histograms across different requester identities, which implies that the requester identity influences users decision making the most. On the other hand, although the decision histograms do not change much between other contextual features of the requester, there is still a small decrease of the "Entirely Share" decisions at the cases where the requester is at an "Unknown" place or with "Other people" nearby. Image semantic features also influence users decision making significantly. For instance, users prefer sharing photos without people or with a lot of people (≥ 6), to sharing photos with 1 ∼ 5 people. Also, users favor sharing those pictures containing strangers or celebrities, over personal photos with intimate connections like family and close friends.

We then evaluated the performance of decision making on different combinations of image and requester features, by conducting a 10-fold cross validation on each user's data. The correct decision rates of cross validation of all the 20 subjects are shown in Fig. 8. Please refer to the feature notations in Table 1. We gradually remove certain features, and the leftmost and rightmost box plots in Fig. 8 show two extreme cases where all the features (I All + R All ) or only the requester features (R All ) were used. As is shown, when reducing features, the correct decision rate of the majority of subjects decreases, which implies that all those features in general have a positive impact on decision making for most users. When reducing image-related features, a significant variance across different subjects is observed, which indicates that those image features are important for modeling many users sharing decisions. However, for two of those subjects, the prediction model still performs well (correct decision rate higher than 0.9) even using only the requester features (R All ). A possible reason is that the two users made their sharing decisions mostly dependent on the context of requesters, regardless of the image content. One also finds that by removing certain requester contextual features, like requester gender (R G ), location (R L ), or nearby people (R N ), the overall accuracy does not significantly change. With merely the requester identity (R I ) + all image features (I All ), the overall decision making accuracy still remains high. This implies that the requester contextual information than the requester identity has very week or even negative influence on decision making. However, this is not always the case for every subject. Fig. 9 illustrates the results of five example subjects obtained on different combinations of requester contextual features (along with all image features I All ). Here, one observes that the inclusion of requester contextual features other than the requester identity influences decision making quite differently between users. For instance, the correct decision rate of User C obtained on all requester features (∼ 0.8) is much higher than that on only requester identity R I . For User A or D, combining different requester features (R I,L,N or R I,N respectively) generates better accuracy than just using requester identity R I . However, for User B and E, using only the requester identity R I provides the best performance, in which case the other contextual features of requester are considered as noise in machine learning. Such a variance between users again proved our hypothesis that users have different personalized behaviors in photo sharing.

I A ll + R A ll I A ll + R I ,

Discussions

Image Visual Information and Metadata Protection Prior to this study, we have proposed and researched on different approaches [START_REF] Yuan | Secure jpeg scrambling enabling privacy in photo sharing[END_REF][START_REF] Yuan | Image transmorphing with JPEG[END_REF][START_REF] Yuan | Privacy-preserving photo sharing based on a secure JPEG[END_REF][START_REF] Yuan | Privacy-preserving photo sharing based on a public key infrastructure[END_REF] to protect image privacy (visual information and metadata) such that the protected photos can be publicly shown to any party (service provider and individuals) while original photos being secretly shared to authorized individuals. The principal idea of these approaches is to utilize JPEG application markers to secretly preserve partial image information or metadata, which not only enables the reversibility of the obfuscated image but also minimizes the storage burden. Such approaches are collectively named Secure JPEG. In the proposed photo sharing model, Secure JPEG can be used to create a secure version of a photo. Depending on the predicted decision, the system can release the corresponding version (protected or recovered original form) of the image to a requester.

Security Discussion As mentioned in Section 3.1, we assume the service provider in proposed model is trusted. The reasons are twofold: First, it is still not possible to perform certain pattern recognition tasks on client devices efficiently, e.g. image semantic recognition; Second, the system makes sharing decisions in a dynamic way by analyzing both image content and requester context, which means the decision making core must lie on the service provider. However, as the development of pattern recognition on mobile devices, the security requirement of the proposed model can be relaxed. In another specific case of the proposed model, where only requester's identity is taken into account (no other context), the security assumption can be discarded. In this case, the photo sharing decisions are made in a static way equivalent to using an access policy. According to a privacy-preserving photo sharing architecture proposed in our previous work [START_REF] Yuan | Privacy-preserving photo sharing based on a public key infrastructure[END_REF], the access policy can be integrated in a Ciphertext-Policy Attribute-Based Encryption (CP-ABE) [START_REF] Bethencourt | Ciphertext-policy attribute-based encryption[END_REF] and secure photo sharing can be achieved through an "Honest but Curious" untrusted server.

Feature Extraction In this study, most image semantic features were manually annotated. This is because we lack the access and control to a popular social network, and that the automatic tools for extraction of some semantic features (e.g. activities in image [START_REF] Castro | Predicting daily activities from egocentric images using deep learning[END_REF]) are not mature enough. In practice, with the advances in deep learning, content understanding and ubiquitous sensors, automatic extraction of different semantic or contextual features is becoming more accurate and fine-grained.

Conclusion

This paper presents a conceptual model for context-dependent and privacy-aware photo sharing based on machine learning. The proposed model utilizes the images semantic and requesters contextual information to predict photo sharing decisions for users, based on their previous shared photos and past decisions.

To evaluate the proposed model, we first conducted a user study on 23 subjects and collected a dataset containing 1'018 manually annotated images with 12'216 personalized sharing decisions in different contexts. Evaluation experiments have been performed and show a promising performance of the proposed method. Furthermore, the influence of different content-and context-related features on decision making has been investigated, the results of which validated the importance of pre-defined features and implied a significant variance between users sharing behaviors and privacy attitudes.

As our future work, we intend to conduct larger-scale user study based on realistic social networking environment. This will further help us understand users photo sharing behaviors. In addition, we will investigate more sophisticated machine learning or even deep learning approaches to build more accurate and secure photo sharing systems. We believe machines will become intelligent enough to understand people's privacy concerns towards their photo content and this is how we define "privacy-aware".

Fig. 2 .

 2 Fig. 2. Workflow of user study using ProShare S.

FFig. 3 .

 3 Fig. 3. Distribution of (a) images in each category and (b) subjects sharing decisions.

Fig. 4 .

 4 Fig. 4. Performance of sharing decision prediction at different sizes of training sets.

4 Fig. 5 .

 45 Fig. 5. Performance of cost-sensitive decision making with two different values of c.

Fig. 7 .

 7 Fig. 7. Distribution of photo sharing decisions distinguished by different features.

Fig. 8 .

 8 Fig. 8. Correct decision rates of all users obtained on different combinations of features.

Fig. 9 .

 9 Fig. 9. Performance of five example users obtained on combination of all Image Semantic Features (I All ) and different Requester Contextual Features (R).

Table 1 .

 1 Feature notations and definitions.

	ID	Feature	Description
	I C
	What

  types of identities were defined: Sender him/herself, Family, Close friend, Schoolmate or Colleague, Girl or Boyfriend, Acquaintance, Celebrity and Stranger.

		R I	Requester: Identity	The relationship between the requester and the sender, categorized in six types: Family, Close friend, Schoolmate or Colleague, Girl or Boyfriend, Acquaintance and Stranger.
		R G	Requester: Gender	Gender of the requester: Female or Male.
		R N	Requester: Nearby	Whether or not the requester has other people nearby at requesting time.
	Where	I L

The application is publicly available at http://grebvm2.epfl.ch/proshare-s/ proShare-rd2.1.apk.

The instruction and agreement sheet for the user study including several screenshots of the ProShare S App is available at http://grebvm2.epfl.ch/proshare-s/ instruction_sheet_rd2.1.pdf

Acknowledgement

This research was possible thanks to the Swiss National Science Foundation funded project LEADME (200020-149259). We also acknowledge Thomas Mizraji and Vincent Debieux for contributions in the development of ProShare S app. A special thank goes to all the patient subjects who participated in the user study.