Axel Bjørn

Gran

Anne Egeli
email: anne.egeli@safetec.no

Alexander Bjerke Safetec
email: alexander.bjerke@safetec.no

Addressing security in safety projectsexperiences from the industry

Keywords: safety, security, industry

Security and cybersecurity has become familiar to everyone. Nevertheless, security is still some-thing that often denoted as not applicable when companies are developing or operating programmable electronic safety systems. In this fast abstract we try to point of some of the reasons to why it is like that, and discuss the problems by both believing there are no reasonable foreseeable security threats, and by believing that every thinkable attack can hit you. We also describe how we as consultancy try to address security in safety projects, and how guidance address the challenge. Finally, we point to the role of system owners that manage to set up clear expectations to include and integrate both safety and security

I. INTRODUCTION

Over the two last decades, security has become a phrase common for everyone work-ing with programmable electronic critical systems. Going back to year 2000, the EU-funded CORAS project "A Platform for Risk Analysis of Security Critical system" [START_REF] Coras | A platform for risk analysis of security critical systems[END_REF] was one of the first large projects addressing security in critical applications, driven by seven field trials within telemedicine and e-commerce [START_REF] Fredriksen | Experiences from application of mod-el-based risk assessment[END_REF]. Today, the number of security related projects are large. There is also a so wide range of relevant standards so that it becomes wrong to mention one and leaving others out. Although security has become familiar to everyone, security is still something that often is denoted as "NA" (not applicable) when companies are developing or operating programmable electronic safety systems. One aim of our research is to point of some of the reasons to why it is like that. The other aim is to propose some ways to address security in safety projects in an appropriate way based upon experiences from consultancy work in different Norwegian industries.

II. BACKGROUND

Protection from terrorism or other intentional crimes is denoted as security, while safety implies protection from unintentional acts. The difference lies in whether the incident is inflicted intentionally or not; safety risk is characterized by being accidental and security is characterized by being intentional or deliberate. This implies that, in the case of security, an aggressor is present who is influenced by the physical environment and by personal factors [START_REF] Reniers | Continuously and simultaneously optimizing an or-ganization's safety and security culture and climate: the Improvement Diamond for Excel-lence Achievement and Leadership in Safety & Security (IDEAL S&S) model[END_REF]. At the same time that security has become familiar, there have in the academia been a discussion if it is meaningful to distinguish between security and safety as to separate fields of handling risks and crisis [START_REF] Reniers | Continuously and simultaneously optimizing an or-ganization's safety and security culture and climate: the Improvement Diamond for Excel-lence Achievement and Leadership in Safety & Security (IDEAL S&S) model[END_REF][START_REF] Piètre-Cambacédès | The SEMA referential framework: Avoiding ambiguities in the terms "security" and "safety[END_REF][START_REF] Jore | Risk management methodology for protecting against malicious actsare probabilities adequate means for describing terrorism and other security risks? Safety and Reliability of Complex Engineering Systems[END_REF].

III. 3 WHY IS THEN SO HARD TO INCLUDE SECURITY IN SAFETY PROJECTS?

There is a number of good hints and advices on the need of including security also in safety projects provided in a number of international generic and branch specific standards. On the other hand, the experience from working in different client projects within the process industry and transport over the last years tells that it is not that easy. Often security aspects are not addressed at all, or they are addressed rather late in the design and development process. An important question is then why it is so hard to include security in safety projects.

A. Specialists in the security field believe that safety and security are two different phenomenon

Jore and Egeli demonstrated some answers to this question in a study [START_REF] Jore | Risk management methodology for protecting against malicious actsare probabilities adequate means for describing terrorism and other security risks? Safety and Reliability of Complex Engineering Systems[END_REF]. During the spring 2014 they interviewed 15 informants representing 10 different institutions within the oil and gas sector. Five informants worked exclusively with security, while the rest worked with both areas of safety and security. The aim of the study was to generate new in-depth knowledge on the topic of security risk management that can contribute to the development of better security risk analysis methodology. Two findings from the study were:

• 9 out of 15 informants are in favor of specific risk management methodology for the area of security. All informants working exclusively with security supported this argument.

• 8 out of 15 informants claim that probabilities cannot be used in security risk analysis. All informants working exclusively with security supported this argument.

One observation from their study was that specialists in the security field believe that safety and security are two different phenomenon, and they should be approached in different ways. This statement is supported by the lack of historical data for security risks and in particular for security risks such as terrorism. For other security risks, this is not the case. A security crime that most petroleum companies and other businesses presently have to be prepared for is cybercrime. For this type of threat, an enormous number of attacks occur every single day, so there are numerous data available for risk analysts. This would also be the case for other types of security risks, such as burglaries or other types of more "ordinary" crimes. For those security risks where a large dataset exists, these should be included in the risk analysis, although it would be necessary to consider the relevance of these data.

In practical risk management, a company has to address both the area of safety and the area of security. Petersen [START_REF] Petersen | The corporate security professional: A hybrid agent between corporate and national security[END_REF] studied the role of security professionals in cooperate businesses and concluded that security professionals in an organization often are not fully incorporated into the main tasks of the organization. This can have as consequence that they also use risk management methodologies that are not consistent with the rest of the organization. Research has also shown that companies' anti-terrorism policies are often not based on a normative treatment of risk that incorporates likelihoods of attacks. Policy makers' anti-terror decisions may be influenced by the blame they expect from failing to prevent attacks [START_REF] Mcgraw | A policy maker's dilemma: Preventing terrorism or preventing blame[END_REF].

B. Specialists in the safety field believe that there are no security threats Requirement 7.4.2.3 in IEC 61508 [START_REF]Functional safety of electri-cal/electronic/programmable electronic safety-related systems[END_REF] says: "If the hazard analysis identifies that malevolent or unauthorized action, constituting a security threat, as being reasonably foreseeable, then a security threats analysis should be carried out". There is however a problem. The ones that are responsible for the functional safety management are the specialists in the safety field. Firstly, they lack historical data for security risks and in particular for security risks such as terrorism. Thereby they conclude that it is not reasonably foreseeable. Secondly, they may lack the needed guidance, as security is not the main topic of the safety standards, and the number of applicable security standards are large. Finally, they may lack the knowledge about the threats. Actors conducting illegal operation in cyberspace vary from government intelligence and security services, traditional military adversaries, global commercial companies, terrorist-and extremist groups, to organized hacker groups. Motives can include idealism, criminal activity, terrorism, economical gains or geopolitical considerations. To understand how and why an actor can be a security threat, it requires the safety specialist to have insight in the motivation, the mission, the mindset and the methods of the attacker. Hommedal states that every security professional should have a strong understanding of these four M's [START_REF] Hommedal | See it coming: The Four M's of Digital Espionage[END_REF]. However, to be applied in the safety projects, this understanding is also needed within the safety professionals.

C. Some Common Mistake Specialists in the safety field believe that it is just a question about the vulnerabilities

The requirement 7.4.2.3 can also be answered with that everything is reasonably foreseeable. In that case, the next step is to undertake a vulnerability analysis. In many projects, this means to address a set of security standards. This is observed done in two ways: (i) use the requirements in the standards as guidance, or (ii) use the requirements as compliance requirements.

The first strategy works fine since it may imply to first address the threats, then establish some reasonable scenarios and finally ask what kind of means and barriers that should be included. The problems comes when the requirements in the security standards are included as list of requirements that should be complied to. Firstly, this may lead to having security requirements that are in conflict with the safety requirements. This means that this assessment should have taken place early in the project in order to solve the conflict. Addressing it later may mean that it is impossible, or to costly, with the results that the security requirement is dropped. Secondly, this strategy do not address the threats. This means that there may be a larger number of security requirements and barriers that are included, while other security needs are overseen.

IV. CONCLUSION AND FURTHER WORK

Security and cybersecurity has become familiar to everyone, but security is still something that often is denoted as "NA" (not applicable) when companies are developing or operating programmable electronic safety systems. In our research, we try to point of some of the reasons to why it is like that. We discuss both the problems by believing there are no reasonable foreseeable security threats, and by believing that every thinkable attack can hit you. We believe that one of the main challenges for the industry is to balance safety and security risk management, and find ways to integrate security risk in the overall risk management.


Thor Myklebust
email: thor.myklebust@sintef.no

Tor Stålhane
email: stalhane@idi.ntnu.no

Safety Stories -A New Concept in Agile Development

Keywords: user stories, safety stories, IEC 61508, communication, assessor I

Safety stories is a new practice, developed to ensure that the agile requirements management process encompass the important safety requirements together with required measurements and techniques. The safety story card will create a common problem understanding between the software developers and the safety stakeholders. Safety stories are user stories that include one or more safety requirements. If you do not get the safety right it does not matter how well you execute the rest of the project.

INTRODUCTION

The term "user stories" were introduced by K. Beck in the XP-1999 conference [START_REF] Beck | Extreme programming explained: embrace change Page 190[END_REF]. A user story is a short, simple description of a feature, told from the problem owner's perspective. The essence of usercentric requirements elicitation is to focus on what the user wants or needs and not on what the system shall do. User stories are intended to shift the focus from writing about features to discussing them. The discussions are often more important than the written text. It is common practice to also document the acceptance criteria together with each user story.

A typical user story follows the following pattern:

As a <type of user> I want <to perform some tasks> so that I <can achieve some goal, benefit or value> While the user stories usually are simple, the acceptance criteria are often more complex. Unfortunately, there is no agreed-upon common format for the acceptance criteria. Consider the following user story and acceptance criteria example:

As a <customer> I want <to withdraw cash from an ATM> so that I <don't have to wait in line in the bank> Acceptance criteria: Given that the account is creditworthy and that the card is valid and the dispenser contains cash: When the customer requests the cash, then ensure that the account is debited, and ensure that cash is dispensed and ensure that the card is returned.

II. THE SAFETY EPIC

An epic is a high-level user story. It must be refined and split into several user stories before it can be realized. The epics are important for two reasonsthey define the high-level requirements of the system to be developed and it identifies the system's environment. A typical safety epic could be

To satisfy <the overall safety needs> the system must <always be able to reach a safe state>

To keep <the operators> safe, the system must <stop the robot when someone enters the tool cell> Through having safety epics and discussing them right from the start we make sure that safety is on the agenda throughout the project and prevent it from being an add-on considered too late.

III. THE SAFETY STORY

We would like to keep the patterns of the safety stories close to the user story pattern. It is, however, important to bear in mind that safety requirementsand thus, safety stories -have two sources (1) standard requirements and (2) system hazard analyses. A final challenge when it comes to safety stories is that they often will be linked to a functional user story, either directly or indirectly.

The safety requirements introduced by the applicable standard can be formulated as follows:

To satisfy <a safety standard requirement> the system must <achieve or avoid something> It is also possible to describe process-related requirements in this way, but the value of this is doubtful since a user story is supposed to be taken from the backlog and then be finished. A process requirement, on the other, hand, should be on the team's agenda throughout the project. The safety requirements that are not related to the standard should be related to either a user story or to the results of a safety analysis -e.g. a HazId or an FMEA (Failure Mode and Effect Analysis). The FMEA may also include an evaluation of the diagnosis. It is then named FMEDA (Failure Mode and Effect Diagnostic Analysis). The result of a safety analysis is usually either a requirement change, an architectural change or the introduction of a barrier -e.g. a watchdog. The safety story should refer to the analysis that introduced it. This is achieved by linking the safety story to the system, subsystem or function FMEA or FMEDAe.g.:

To keep <function> safe, the system must <achieve or avoid something> Note that linking safety needs to software design is already required by the IEC 61508:2010, part 3.

IV. WHY SAFETY STORIES AND SAFETY EPICS

User stories are written throughout an agile project.

A user-story workshop is often arranged at the start of the project. Everyone in the team should participate and the goal is to produce a product backlog that fully describes the current understanding of the functionality to be delivered. As a lot of things in agile development, creating user stories is also part of achieving efficient communication in the project. Thus, who writes the user story is far less important than who is involved in the discussion.

Even though the product backlog is in some senses the equivalent of a requirements document in a nonagile project, it is important to bear in mind that a user story or safety story is not finished before it is discussed both within the team and with the customer representatives.

Writing safety stories will have several benefits. The most important ones are:

• It will create discussions about how to realize the safety stories, both when writing them, during the sprint backlog refinement discussions and when they are to be implemented. Discussions lead to communication and a common metal model of how to realize the story which will improve system quality. • The safety stories and the improved communication will help to create a safety culture in the team. • We will discuss how to realize the safety story when it is taken out of the backlog. Thus, our decision will be based on more information that if we had decided how to realize it at the start of the project. • Since the safety stories are about "what" and not about "how", they are a good starting point for discussions with the assessor or the safety responsible person. E.g., will the assessor accept that we solve this safety story by implanting this barrier?

V. CONCLUSION

The introduction of safety stories into agile development will enable us to involve developers, assessors and customers into realizing the system's safety requirements, help to build a safety culture and base our safety-related decisions on all currently available information.

ACKNOWLEDGMENT

This work was partially funded by the Norwegian Research Council under grant #228431 (the SUSS project) and SINTEF SEP project Safe Software.


Abdulqawi Saif

Lucas Nussbaum

Performance Evaluation of NFS over a Wide-Area Network

Keywords: performance evaluation, design of experiments, network latency, NFS Compas'2016 : Parallélisme/ Architecture / Système Lorient, France, du

The Cloud and Big Data movements triggered a move to more centralized, remote storage resources. However, the use of such remote resources raises a number of challenges at the protocol level. In this paper, we first evaluate the influence of several factors, including network latency, on the performance of the NFS protocol. Then, we explore how statistical methods such as a fractional factorial design of experiments could have helped to drastically reduce the number of required experiments while still providing a similar amount of information about the impact of the factors.

Introduction

The Big Data movement has put a lot of focus on the importance of data, and on the importance of proper data management approaches. It encompasses changes in the way we store data, by moving from traditional expensive storage arrays to more cost-effective, scalable and flexible Software-Defined Storage solutions such as Ceph [START_REF] Weil | Ceph: A scalable, high-performance distributed file system[END_REF], GlusterFS [1] or iRODS [START_REF] Rajasekar | iRODS primer: integrated rule-oriented data system[END_REF]. At the same time, there is a push towards moving our computing and storage facilities from small on-premises datacenters to larger datacenters or Cloud offerings in order to reduce costs and increase reliability and elasticity. This move towards remote and centralized storage resources raises a number of challenges when aiming at maintaining high-performance access. The protocols that were designed to access storage resources over a local network are not necessarily able to perform efficiently when network latency increases due to the need to transfer data over hundreds or thousands of kilometers. Modern storage systems often provide an object-based interface, where storage objects are read or written using GET/PUT methods that are able to perform well over highlatency network. However, most applications still have not been ported to such interfaces, and still rely on traditional POSIX-like directories and files access. Several designs are possible to provide that interface, such as relying on a kernel driver on the client side (which can be a limitation when supporting a wide range of systems), or using a FUSE filesystem (which has performance impacts). As a result, most modern Software-Defined Storage solutions still provide a traditional NFS or CIFS interface (e.g. GlusterFS [1]), that provides an unintrusive way to access storage from clients. In this paper, we illustrate the challenges raised by the use of a traditional NFS interface in a high-latency WAN setup. After providing some context and related work in Section 2, we present a set of experiments that demonstrate the influence of several factors, including network latency, on NFS experiments in Section 3. Finally, in Section 4, we explore how statistical methods could be used to reduce the number of experiments in such many-factor evaluations. We conclude the paper and highlight some opportunities for future work in Section 5.

Context and Related Work

The Network File System (NFS) was introduced in 1984 to allow sharing resources in a network for heterogeneous client machines [START_REF] Sandberg | Design and implementation of the sun network filesystem[END_REF]. Its first public version (NFSv2) had several limitations (no asynchronous write operations). NFSv3 (1988) overcomes this and added support for the TCP protocol. NFSv4 (2000) came with additional improvements such as compound RPCs to divide any operation into smaller ones, additional security features, client delegations to authenticate the clients to do some operations without contacting the server [START_REF] Charbon | NFSv4 test project[END_REF][START_REF] Shepler | Network file system (NFS) version 4 minor version 1 protocol[END_REF], altogether with other features in order to improve the overall performance. Many studies have focused on the evaluation of the performance of NFS. Chen et al. [START_REF] Chen | Newer is sometimes better: An evaluation of NFSv4.1[END_REF] made an intensive comparison of NFSv3 and NFSv4.1. They also tested the newly implemented features of NFSv4.1 as well as tuning NFSv3 & NFSv4.1 to increase the throughput performance. They show that the performance of NFSv3 and NFSv4.1 depends on the network latency, i.e., NFSv4.1 was faster on a high latency network and slower than NFSv3 on a network with a lower latency. The same authors additionally focused on the performance of NFSv4.1 [START_REF] Chen | Linux NFSv4.1 performance under a microscope[END_REF] by studying its behavior against several factors in Linux as the kernel version and the memory management. Radkov et al. [START_REF] Radkov | A performance comparison of NFS and iSCSI for IP-networked storage[END_REF] compared several versions of NFS (v2, v3 and v4) with iSCSI as representatives of file and block storage systems, and concluded that NFS is comparable with iSCSI in terms of performance. Furthermore, Martin et al. [START_REF] Martin | NFS sensitivity to high performance networks[END_REF] tested NFS sensitivity against latency. They showed that NFSv3 can tolerate high latency more than NFSv2. Others have focused on performance evaluation of client-side tuning. Ou et al. [START_REF] Ou | Is cloud storage ready? a comprehensive study of IP-based storage systems[END_REF] tested NFSv4 besides other storage file systems such as iSCSI and Swift in the context of cloud storage systems. Their important finding, which is related to our context, shows that the client-side configuration plays a significant role over the performance of IP-based storage systems like NFS. Lever et al. [START_REF] Lever | Linux NFS client write performance[END_REF] uncovered several performance and scalability issues in the Linux NFS client that affected write latency and throughput. The main difference between the experiments presented in this paper and the ones presented in previous works is the focus on a high latency environment, using a realistic distributed testbed instead of using an emulated network. Also, we use statistical methods (fractional factorial and full factorial designs). Those are rarely used in Computer Science, but we could find a few examples: Videau et al. [START_REF] Videau | Toward an experiment engine for lightweight grids[END_REF] used a fractional factorial model to design their multi-parameter experiment in order to determine the impact of each parameter on the results. Khoshgoftaar and Rebours [START_REF] Khoshgoftaar | Improving software quality prediction by noise filtering techniques[END_REF] used a full factorial model to evaluate several learning algorithms.

Experimental Evaluation of NFS

In this section, we describe a series of experiments performed on the Grid'5000 testbed [START_REF] Balouek | Adding virtualization capabilities to the Grid'5000 testbed[END_REF]. We used three sites of the testbed, as shown in Figure 1. Thanks to Grid'5000 capabilities, we deployed our own software environment (Debian jessie-x64-nfs image, Linux 3.16.0) and set To allow using NFSv4 beside NFSv3, we configured NFS on the clients by enabling the name mapping daemon IDMAPD which is required by NFSv4. In addition, we make sure that the server accepts 1M as I/O size value be verifying it on nfs_xdr header file. All nodes (server and clients) are connected over a 10 Gbps network (both the Ethernet network on each site and the Grid'5000 intersite network). This setup is representative of one where an NFS client accesses a server over the Internet, with a high-bandwidth link, and varying network latency (local or remote NFS server).

Experimental setup and factors

NFS performance can be affected by several factors (parameters) and configurations. In this paper, we focus on six factors that can have an impact on NFS performance. Those factors are manipulated on NFS clients. Three factors are NFS tuning parameters (changeable through the mount command). All factors are listed below, starting by the tuning ones :

NFS Version: [NFSv3, NFSv4 ].
NFSv3 is the stable and widely used version [START_REF] Tarasov | Virtual machine workloads: The case for new NAS benchmarks[END_REF] while NFSv4 is the latest version released with promising specifications [START_REF] Shepler | Network file system (NFS) version 4 protocol[END_REF].

Synchronicity of I/O operations:

[sync, async]. They are used on the client side (see nfs( 5)) to control the behavior of the client when issuing writes. In asynchronous mode, the client may delay sending write requests to the server, and return from the write request immediately. In synchronous mode, the client will wait until the server replies to return from the write request. This option should not be confused with the server-side sync/async option (see exports( 5)) which controls whether the server must wait until data is written to stable storage (e.g. HDD) before issuing a reply to the client. We use normal Grid'5000 compute nodes with a single hard disk drive, and not nodes which would provide good I/O performance by spreading data over several disks, as such nodes are not available on all needed sites. Thus, the local hard disk drives of our nodes are likely to be a major performance bottleneck. In order to focus on the protocol aspects of performance rather than just on the performance of local storage devices, we do not limit ourselves to experiments using data stored on ext4 filesystems backed by hard disk drives, but we also perform experiments using in-memory data, by setting up a tmpfs filesystem on nodes.

NFS

File size: [100 MB, 5 GB ]. They are used to represent the small and big files.

Latencies: [0.027 ms, 6.87 ms, 13.9 ms]. They represent the measured one-way latencies between the server and Luxembourg, Lyon and Rennes clients, respectively.

Compas'2016 : Parallélisme/ Architecture / Système Lorient, France, du 5 au 8 juillet 2016

We have a total of 96 possible combinations of factors (2 5 x 3 1 -five two-level factors & one three-level factor), such as [tmpfs, NFSv3, sync, 64 KB, 5 GB, Lyon client]. Furthermore, each combination will be tested with sequential reading and writing access patterns. We use the dd command with /dev/zero as source and 1MB as a block size to perform the writing operations. Similarly, the cp command is used to perform the reading operations.

The experiments are executed sequentially by changing the access pattern of each experiment, i.e., every experiment performs a write operation before reading the written file again, then the next experiment is similarly executed and so on. Moreover, to determine the exact read/write time of each experiment, all of them are executed with (1) preventing the cache effect on the server as well as on the clients, and (2) re-mounting the corresponding NFS directory after performing the pair (read and write) operations of each experiment. The statistical validity of the results shown next is increased by replicating each experiment five times to get, in total: 960 executions (96 experiments x 2 access patterns x 5 replications).

Results

This section shows the obtained results. They are presented by separating reading and writing results as shown below. Figure 2 shows that using tmpfs provides much better results than ext4 in all cases, due to the lower speed of HDD. Considering only tmpfs, one can see that the best performance occurred when the files are read from the local client. That can enable us to confirm that the high latencies environment (represented by Lyon and Rennes) strongly affect NFS reading performance. For instance, when a client is changed from Luxembourg to Lyon, the throughput is reduced on all NFS versions by at least 8.6x, 7x on figures 2-(A) and 2-(B), respectively. Changing NFS versions has only a limited impact on the obtained results. Thus, we can assert that those versions have approximately the same level of sensitivity against the applied latency. However, NFSv4 causes a slight decrease in performance when compared with NFSv3. We also found that the performance is slightly improved on tmpfs by increasing I/O size because larger I/O requests reduce the number of I/O requests, and thus the penalty induced by network latency.

Reading results

Writing results

The writing results are presented in figures 3 and 4. As with reads, the sync writing results also exhibit reduced performance when using an HDD, compared to when using tmpfs. The HDD results are nevertheless affected by the different latency like the results of tmpfs which are strongly influenced. The tmpfs results show that the cost of changing the client from the local site to another one with 6.8 ms as latency can decrease the throughput by 17.5x. We also found that NFSv3 shows a better performance over NFSv4 when the local client writes 100 MB file on a server memory. We hypothesize that this is an inconvenient of NFSv4's compound RPCs when performing write operation on a low latency connection and a high-speed storage material. In contrast, NFSv4 shows a better performance when writing on HDD. On the other hand, the asynchronous writing results show an increased performance over the synchronous results on all experiments, but its most significant impact is shown on the remote clients. For instance, the highest throughput of Lyon client is raised to 18.9x using the async option on tmpfs and a 5 GB file. By having write requests return immediately, this mode is able to hide the latency between the server and the client. Of course, this could have an impact on data reliability.

Statistical Analysis of Results

In most Computer Science papers, experimental results are analyzed in a rather raw way, with only a limited use of statistics (computing averages, medians, etc.). However, statistics can be a useful tool, to help validate results by excluding the effects of chance or other unknown factors, and ensuring that all results are generally coherent. In experiments involving many factors such as those described in this paper, statistics can also contribute to a deeper understanding of the results, by providing a way to identify the role of each factor and their interactions. In this section, we show how statistical methods [START_REF] Jain | The art of computer systems performance analysis[END_REF][START_REF] Hinkelmann | Design and analysis of experiments[END_REF][START_REF] Antony | Design of experiments for engineers and scientists[END_REF] (factorial designs) could be used to plan experiments, and identify the relative importance of factors, and their interactions. First, in section 4.1, we use a full factorial design (evaluating all combinations of factors). This design is applied over two-level factors in order to measure the impact of each factor when its value changes from a low level to a high level, providing a way to make a first pass over all factors and decide which ones to exclude from further experiments (because their impact would be limited). For this reason, the factors from section 3.1 are represented as: (1) A : NFS version with (-1) NFSv3 , (+1) NFSv4, (2) B : Storage type with : (-1) tmpfs , (+1) ext4, (3) C : Sync option with : (-1) sync , (+1) async, (4) D : I/O Size with : (-1) 64 KB , (+1) 1 MB, (5) E : File size with : (-1) 100 MB , (+1) 5 GB, and (6) F : Client Latency with : (-1) Luxembourg , (+1) Rennes (only Luxembourg and rennes are used as this method is limited to two-level factors). However, a full factorial design requires a very large number of experiments (320 experiments in our case, as will be explained later). So, in section 4.2, we use a fractional factorial design to analyze the same factors using a reduced number of experiments (45 in our case), at the expense of not obtaining information about interactions between all factors. Ultimately, both models determine if the effects are significant using Student's t-distribution to calculate a 95% confidence interval of each effect.

Full factorial design

In this section, the full factorial design is used over: 2 k * r experiments where k is the number of factors (k = 6) and r is the number of replications (r = 5), that is, 320 experiments. The key idea of 2 k * r design is to isolate the errors through the different executions. This leads to calculating the estimated behavior of the response variable y, which is the estimated time to read/write a file in a given experiment, using this model: y = q 0 + q i x i + q j x j + q k x k + q ij x i x j + q ik x i x k + q jk x j x k + . . . + q ijk x i x j x k + . . . + e where the response variable y is the estimated time to read/write a file in a given experiment, q 0 is the mean response of the model, x i represents the correspondent level of factor i (-1 or +1), q i with i ∈ {A, B, . . . , F} is the impact of the factor i when changing its value from -1 to +1, q ij with i, j ∈ {A, B, . . . , F} ∧ i = j represents the impact of interaction between the factors i and j, and e represents the model errors.

-

Factors and Interactions

Effects

Figure Effects of factors and interactions

The result of applying this model on writing access pattern is shown in Figure 5. It shows the effects of all factors under study, and of their interactions. The x-axis of this figure can be read as described in this example: A, AB, CDE represent those three cases : Factor A (NFS versions), interaction between factors A and B (NFS versions, storage type), and interactions between factors A,B and C (NFS versions, storage type and Sync option), respectively. Moreover, those effects can be read as follows : the larger the absolute value, the bigger the influence on the results.

As a result, we found statistically that the four main factors that make the most variation on the results are: (1) the file size, (2) the storage systems, (3) the sync options, and (4) the client latency. For example, the first case can be interpreted as : when the transferred file is changed from -1 (100 MB) to +1 (5 GB), it makes the most variation on the result (transfer time) and so on. We also found that, there are some significant interactions such as BF, CF, and BCF, and some factors that don't have much impact on the results, such as A (NFS version).

Fractional factorial design

In many cases, full factorial designs generate too many experiments to be usable in practice.

One solution to reduce the number of experiments without reducing the number of factors is to use a fractional factorial design. The general idea is to create a set of experiments that is reduced compared to the one obtained with a full factorial design, but that still, by construction, enables the study of the effect of each factor, even if information about the interactions between some factors is lost. A fractional factorial design (described in more detail in [START_REF] Jain | The art of computer systems performance analysis[END_REF] those columns. We then put the rest of factors D,E,F over an equal number of interactions as shown in table 1 (for example, we chose to use the AB column for the factor D). Note that as a result, the effects of factors D,E,F will be confounded with the interactions AB, AC, BC, which means that their respective effects cannot be separated.

A B C D AB E AC F BC ABC -1 -1 -1 1 1 1 -1 -1 -1 1 1 -1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 1 -1 -1 1 -1 1 -1 -1 -1 -1 1 1 1 -1 1 -1 1 -1 -1 1 1 -1 1 -1 -1 -1 1 1 1 1 1 1 1
In the resulting matrix, each row represents an experiment. For instance, the first row deals with the result of the experiment that has the factors A,B,C at NFSv3, tmpfs, sync levels and the factors D,E,F at 1MB, 5 GB, Rennes levels, respectively.

Fractional factorial design benefits from some matrix properties to do the calculation. It is shown in table 1 that (1) the sum of every column equals to 0, (2) the sum of product of any two columns also equals to 0, and (3) the sum of absolute values of any column equals to 2 k-p . Furthermore, the allocation of factors on interactions is not unique. For example, instead of having D=AB, E=AC, and F=BC, we can use D=AC, F = BC, and E = BC and so on. It is possible to compare two different allocations by calculating its resolution (see [START_REF] Jain | The art of computer systems performance analysis[END_REF], section 19.4).

The result of applying this model on writing experiments is shown in Figure 6. This result shows that the first four important factors that have a significant effect on the writing time are the same factors that were obtained using the full factorial model in the previous section. Thus, this model gives the same conclusion but with reducing the number of experiments from 320 to 45.

Conclusion & Future Work

In this paper, we performed a series of experiments on NFS over Grid'5000, with a wide range of parameters before applying the full and fractional factorial models over those experiments.

The main originality of our work is firstly making a real and complex case study on how we can use the statistical analysis such as the fractional factorial design to reduce the number of experiments. We show that, instead of doing a full factorial experiment, we could have done a fractional factorial design which would have required only 12.5% of full factorial experiments to obtain the same conclusions as shown earlier. Secondly, we show that the network latency has a significant impact on the performance of NFS, but that some tuning allow us to mitigate its effects: with default parameters, performance on a remote client with 6.8 ms as latency is 11.6%, 5.7% of the performance of a local client (client located in the same data center as the server) on reading and writing operations, respectively. But with tuning, the performance can increase up to 20%, 83% of the performance of a local client, which makes the use of NFS over high-latency networks an acceptable option. We aim at extending that work in the future by building an enlarged benchmark to try tuning

Figure 1 :

 1 Figure 1: Experiment topology over Grid'5000 testbed

  I/O size: [64 KB, 1 MB ]. Those values are used because 64KB represents the default value of I/O size of several NFS implementations while 1 MB also represents the maximum I/O value that is used by most NFS servers. Storage type: [Hard disk -ext4, In-memory -tmpfs].

Figure 2 :

 2 Figure 2: Reading results

Figure 4 :

 4 Figure 3: Synchronous write results

Table 1 :

 1 Fractional design with k = 6 & p = 3

		0.5
	Effects	0.0
		-0.5
		A . n f s _ v e r s io n B . S t o r a g e T y p e C . s y n c _ O p t io n D . n f s _ io	E . f il e _ s iz e F . la t e n c y	A B C
		Factors
	Figure 6: Fractional factorial results

Compas'2016 : Parallélisme/ Architecture / Système Lorient, France, du 5 au 8 juillet 2016


Andrew Fielder
email: andrew.fielder@imperial.ac.uk

Emmanouil Panaousis
email: e.panaousis@qmul.ac.uk

Pasquale Malacaria
email: p.malacaria@qmul.ac.uk

Chris Hankin
email: c.hankin@imperial.ac.uk

Fabrizio Smeraldi
email: f.smeraldi@qmul.ac.uk

Game Theory Meets Information Security Management

Keywords: Information security management, game theory, cyber security

This work addresses the challenge "how do we make better security decisions?" and it develops techniques to support human decision making and algorithms which enable well-founded cyber security decisions to be made. In this paper we propose a game theoretic model which optimally allocates cyber security resources such as administrators' time across different tasks. We first model the interactions between an omnipresent attacker and a team of system administrators seen as the defender, and we have derived the mixed Nash Equilibria (NE) in such games. We have formulated general-sum games that represent our cyber security environment, and we have proven that the defender's Nash strategy is also minimax. This result guarantees that independently from the attacker's strategy the defender's solution is optimal. We also propose Singular Value Decomposition (SVD) as an efficient technique to compute approximate equilibria in our games. By implementing and evaluating a minimax solver with SVD, we have thoroughly investigated the improvement that Nash defense introduces compared to other strategies chosen by common sense decision algorithms. Our key finding is that a particular NE, which we call weighted NE, provides the most effective defense strategy. In order to validate this model we have used real-life statistics from Hackmageddon, the Verizon 2013 Data Breach Investigation report, and the Ponemon report of 2011. We finally compare the game theoretic defense method with a method which implements a stochastic optimization algorithm.

Introduction

Due to the growth of cyber attacks against government agencies and companies, there is a need for more investment to protect networks and platforms which allow the exchange and the storage of important information. Large organizations are more likely to employ a security manager whose main task is to lead Information Security Management to mitigate cyber security risks. However many companies do not have dedicated cyber security personnel, but the critical tasks required to mitigate these risks must still be performed. In the latter case these tasks need to be carried out by the system administrators who must allocate part of their time for this. This is determined by other crucial non-security tasks. Since time is a scarce resource for system administrators, the available time to perform cyber security tasks should be ideally optimized. Hence there is a need for a method that gets as an input a time allowance and identifies how much time should be spent on tasks related to the defense of different parts of the infrastructure. The importance of this challenge has also been highlighted by Alpcan and Başar [START_REF] Alpcan | Network security: A decision and game-theoretic approach[END_REF] (p. 9). This is a critical challenge given the fact that an attacker usually aims at exploiting one vulnerability of the system whilst the defending party must protect as many cyber security targets as possible. System administrators seek to maximize the mitigation of the greatest number of attacks against the various assets while an attacker aims to maximize the probability of penetrating the different "security layers" and complete their attack against a preferable target. This problem falls under the entire class of problems ISO 27001 examines with regards to balancing expenditure to the information security risk.

In this paper, we consider the global asset as the collection of all the data assets of an organization. We additionally consider a target to be a piece of hardware that has access to a subset of this asset. In our model the process of allocating resources is to improve the performance of the defense of a system. Our model defines the defense in terms of all the actions that can be performed to improve the defense of a given target. We assume that each target can be attacked by the exploitation of a vulnerability. The adversary follows a unique path to penetrate into the organization's system and compromise a target. Examples of targets, vulnerabilities, as described by our model, and threats against these targets, can be found in 2013 Data Breach Investigation report [1] published by Verizon. By modeling a cyber security scenario in the form of targets that attacks can be performed against and the set of all actions that can protect that target, we are able to create a more compact and computationally feasible model removing the interdependencies between actions. According to our model, the attacker probabilistically chooses a target and plays with probability 1 all the actions required to attack this target. Their decision is based on the different profits they will obtain when they attack this target successfully and the baseline defense applied to each of the attack paths towards the targets. However, we assume that the attacker is not aware of the resource allocation strategy chosen by the defender nor conducts surveillance to identify such strategy.

The research question that this paper attempts to address is "given that different targets have different weights how do we optimally allocate cyber security resources to defend such targets?". The defense of a target is indicated by two security levels, baseline and best practice. The baseline shows the probability of an attack against a target being mitigated, to guarantee that the basic security functionality of the systems is maintained, when no additional administrators time has been allocated. The best practice defense denotes the percentage of attacks that are mitigated assuming that the system is currently running the most up-to-date security software as well as having had all patches applied and any potentially exploitable data removed. These tasks could be carried out by allocating additional system administrators' time. An example of baseline defense and resource allocation towards the provision of extra security is the following. We can assume that an unmonitored spam filter stops a certain percentage of junk e-mail, for example 60% which corresponds to the baseline defense. By having an administrator monitoring and upgrading the spam filters, more attacks can be prevented, and for instance 90% of the spam is mitigated. Thus under any resource allocation strategy, in terms of man hours, where an administrator is assigned to perform a spam filter based action, we have an improvement of 50% in stopping junk emails over the baseline defense.

In this paper we use game theory to model the cyber security environment and challenge of resource allocation as described above. Any resource allocation problem must consider the strategies available to the adversary and the cyber security team of the organization thus making game theory an appropriate tool to model such an environment and provide an effective set of solutions. According to [START_REF] Alpcan | Network security: A decision and game-theoretic approach[END_REF] (p. 40), "A game-theoretic framework for defensive decision-making has a distinct advantage over optimization as it explicitly captures the effects of the attacker behavior in the model, in addition to those of the defensive actions. Plain optimization formulations, on the other hand, focus only on the optimization of defensive resources without taking attackers into account."

The remainder of this paper is organized as follows. Section 2 discusses related work from the state-of-the-art on the intersection of game theory and security. In Section 3, we propose our game theoretic model while in Section 4 we undertake comparisons against alternative methods. Finally, Section 5 concludes this paper by summarizing its main contributions and highlighting future work to be undertaken to further improve the performance and the usability of our model.

Related work

A significant amount of work on the intersection of economics and information security has been published by Grossklags. The authors in [START_REF] Acquisti | Losses, gains, and hyperbolic discounting: An experimental approach to information security attitudes and behavior[END_REF] discuss the factors that influence the decision process of individuals regarding their information security concerns. They contribute to the formulation of information policies and technologies regarding personal information security and privacy. The work published in [START_REF] Johnson | Nash equilibria for weakest target security games with heterogeneous agents[END_REF] examines the weakest target game which refers to the case where an attacker is always able to compromise the system target with the lowest level of defense and not to cause any damage to the rest of the targets. The game theoretic analysis the authors have undertaken shows that the game leads to a conflict between pure economic interests and common social norms. While the former are concerned with the minimization of cost for security investments, the latter imply that higher security levels are preferable. A very thorough work has been published in [START_REF] Grossklags | Secure or insure?: a game-theoretic analysis of information security games[END_REF] where the authors model security interactions by choosing different games such as weakest-link or best-shot to represent practical security scenarios. In these games, decision parameters are a protection level determining the security level chosen for the player's resources and a self-insurance level which mitigates damages when a successful attack occurs.

Security problems have been increasingly studied by using Stackelberg games to model the strategic interaction between a defender and an attacker. Some physical security games such as [START_REF] Kiekintveld | Computing optimal randomized resource allocations for massive security games[END_REF] assume the existence of targets that might be covered or uncovered during an attack. In [START_REF] Kiekintveld | Computing optimal randomized resource allocations for massive security games[END_REF], Kiekintveld et al. make the assumption that it is always preferable for the defender to defend as many targets as possible to obfuscate vulnerabilities over a period of time. The resulting players' payoffs depend only on the attacked target and whether this is covered by the defender or not. An important contribution is the work conducted by Korzhyk et al. in [START_REF] Korzhyk | Stackelberg vs. nash in security games: An extended investigation of interchangeability, equivalence, and uniqueness[END_REF], showing how a leader (defender) should derive their strategy given that the security game could be either a Nash or a Stackelberg game. They also examine the case of a follower (attacker) who can attack more than one target. The authors show that Nash and Stackelberg strategies are the same in the majority of cases only when the follower attacks just one target.

Another recent contribution within the field of physical security has been published in [START_REF] Kiekintveld | Security games with interval uncertainty[END_REF], where the authors address the problem of finding an optimal defensive coverage. Because of the uncertainty about the attacker's payoffs they define such coverage as the one maximizing the worst-case payoff over the targets in the potential attack set. In a cyber security game we can consider that a defender may not know the payoffs for an attacker and they may be able to infer some bounds for those payoffs. Work published by Lye and Wing [START_REF] Lye | Game strategies in network security[END_REF] uses a nonzero sum stochastic game model where an adversary is attacking an enterprise network and a network security administrator is defending the different network assets (e.g. web server, file server, workstation). A thorough survey of research contributions within the broad field of security and privacy in computer networks modeled by game-theoretic approaches has been carried out in [START_REF] Manshaei | Game theory meets network security and privacy[END_REF].

Game Theoretic Formulation

We define a cyber security game as a game-theoretic model that captures essential characteristics of resource allocation decision making (i.e. system administrators' time) to prevent data loss and defend system and network assets of an organization. This is a two-player, non-cooperative static game between a system security team, defender D, which defends an organization's information assets and data against external or insider adversaries who are modeled as an omnipresent attacker A. Our model follows the definitions of the work done by Korzhyk et al. in [START_REF] Korzhyk | Stackelberg vs. nash in security games: An extended investigation of interchangeability, equivalence, and uniqueness[END_REF]. The attacker A might choose to attack any target from the available set of targets T = t 1 , t 2 , . . . , t n whilst D aims to mitigate attacks by defending targets using resources. In our game the targets are data assets accessible by specific hardware components and system administrators are the limited resources. We consider a schedule as a unique allocation of system administrators to perform tasks that will update the targets defense from baseline to best practice. We define the set of available defense schedules as S. The utility of D when t i is attacked and has no resources assigned to it equals the baseline defense utility U bl D (t i ). On the other hand, if D has assigned some resource to defend t i then their best practice defense utility equals U bp D (t i ). Finally, the utility in case where no attack has taken place equals zero.

Likewise for A, we define their utility values as U bl

A (t i ) and U bp A (t i ), respectively. Moreover, we denote the difference between best practice and baseline utilities of a target t i for both players as ∆U D (t i ) and ∆U A (t i ). We observe that by applying a given resource to a target, the utility of D for this target increases and the utility of A decreases; namely ∆U D (t i ) > 0 and ∆U A (t i ) < 0. Similarly to the aforementioned example of a spam filter, let's assume that the baseline defense probability of a spam filter is 60% while the best practice defense probability equals 90%. If the damage caused to the organization due to a successful attack against this spam filter is given by l = -100 then the baseline utility value equals U bl = -40 and the best practice equals U bp = -10.

The normal form of this game is described as follows. A's pure strategy space is the set of targets, while their mixed strategy is denoted by A = a i , where a i represents the probability of attacking a target t i . A schedule (or pure strategy) of D is a feasible unique assignment of resources to cover (i.e. provide best practice defense) the different targets. Assuming that one resource is adequate to provide best practice defense for a target, a pure strategy is represented by a tuple s = s i ∈ {0, 1} n , where s i equals 1 when t i is defended by best practices; or 0 when only baseline defense is in place. Due to limited resources we define the feasible schedules by S ⊆ {0, 1} n . The number of schedules available to m administrators in the case of n targets equals n! (n-m)!m! . For example for a scenario with 4 targets, when the number of system administrators equals two then the available schedules are:

S = {< 1, 1, 0, 0 >, < 1, 0, 1, 0 >, < 1, 0, 0, 1 >, < 0, 1, 1, 0 >, < 0, 1, 0, 1 >, < 0, 0, 1, 1 >}
In this paper we assume homogeneous resources, namely each resource can apply best practice defense equally for each of the targets, allowing all the possible resource allocation schedules to be played. The mixed strategy D = d s of the defender is a probability distribution over the different schedules, where d s is the probability of playing a schedule s ∈ S. We define the coverage induced by the strategy D to be the vector C = c i , where the probability c i of applying the best practice defense for a target i is given by c i = s∈S s i d s . Going back to our previous example with 4 targets and 2 system administrators, assuming a mixed strategy D = 0.3, 0.2, 0.15, 0.15, 0.1, 0.1 for the defender then the coverage vector equals C = 0.65, 0.55, 0.45, 0.35 . Following [START_REF] Korzhyk | Stackelberg vs. nash in security games: An extended investigation of interchangeability, equivalence, and uniqueness[END_REF], we define the utility functions of both players as follows. Definition 1 (utilities of the cyber security game). When a strategy profile D, A is played, the utility values of both players are given by the following:

U D (D, A) = n i=1 E D,C (t i ) = n i=1 a i (c i U bp D (t i ) + (1 -c i )U bl D (t i )) (1) U A (D, A) = n i=1 E A,C (t i ) = n i=1 a i (c i U bp A (t i ) + (1 -c i )U bl A (t i )) (2)
If both players act rationally the game theoretic solution, upon their simultaneous moves is the Nash Equilibrium (NE).

Definition 2 (NE of the cyber security game). A pair of mixed strategies D, A forms an NE if the following are satisfied:

I. D plays a best-response that is U D (D, A) ≥ U D (D , A), ∀ D II. A plays a best-response that is U A (D, A) ≥ U A (D, A ), ∀ A
In a game it is possible that there are many NE. However in presence of unknown attack distributions not all Nash defenses perform equally. Therefore we are interested in the particular NE that most favors the targets with the highest defender's utility. We define the NE's weight as:

1≤i≤n (c i U bp D (t i ) + (1 -c i )U bl D (t i )).
Definition 3. The weighted NE has the maximal weight among all NE.

In the case that there exists more than one NE with maximal weight any of these equilibria can be chosen as the weighted NE. Notice the NE's weight differs from the defender's utility in that the attacker's strategy is ignored. In the remaining of this paper, unless otherwise stated, by NE we refer to the weighted NE.

Characterization of Nash defense

In a real scenario attackers and defenders often have different preferences and criteria for evaluating the financial impact of a successful attack, as such it is unrealistic to expect that cyber security games are necessarily zero-sum. Thus, in our cyber security game, players typically weigh the same outcomes in different ways. In our model we have made the assumption that both players assess the different targets in a proportionally equivalent manner.

In the following we have proven that the Nash defense is a minimax strategy for the defender which implies that it minimaximizes the attacker's utility in the cyber security game. This follows the work reported in [START_REF] Korzhyk | Stackelberg vs. nash in security games: An extended investigation of interchangeability, equivalence, and uniqueness[END_REF]. We represent the set of all defender's Nash strategies of a non-zero sum cyber security game G as S G N E and the set of all defender's minimax strategies as S G M M . We define a zero-sum cyber security game G in which the baseline and best practice utilities of the attacker for each target are equivalent to their utilities in G, therefore:

U bp A (t i ) = U bp A (t i ), ∀t i ∈ T, and U bl A (t i ) = U bl A (t i ), ∀t i ∈ T (3)
Since G is a zero-sum game, we know that

U bp D (t i ) = -U bp A (t i ) and U bl D (t i ) = -U bl A (t i ).
We define a function µ which maps the attacking probabilities of each target in G to the attacking probabilities in G as defined in [START_REF] Korzhyk | Stackelberg vs. nash in security games: An extended investigation of interchangeability, equivalence, and uniqueness[END_REF]:

µ(a i ) = a i = λa i ∆U D (t i ) ∆U A (t i ) , ∀t i ∈ T (4)
where µ(a i ) is the probability of a target t i to be attacked in G, therefore µ(A) = A. Respectively the set of defender's Nash strategies in G is represented by

S G N E
and the set of defender's minimax strategies by

S G M M . Proposition 1. D , A is NE profile in G iff D , µ(A ) is NE profile in G.
Proof. To prove proposition 1, we have combined Lemmas 1 and 2 presented in appendix. We have that A and µ(A ) are the attacker's best responses, in G and G respectively, to the same defender's best response D . Therefore the tuple of strategies D , A is NE profile in G iff D , µ(A ) is NE profile in G. This implies that D is a Nash defense in G iff it is also a Nash defense in G.

Theorem 1. A defender's Nash strategy in the non-zero sum cyber security game is also a defender's minimax strategy in the same game.

Proof. The minimax theorem states that for zero sum games NE and minimax solution coincide. Thus D is a Nash defense in G iff D is a minimax defender's strategy in G namely D ∈ S G M M . This means D minimaximizes the utility of the attacker in the zero-sum game where the defender's strategy is as in G:

D = argmin D max A U A (D, A).
We also have that U A (D, A) equals:

n i=1 a i (c i U bp A (t i ) + (1 -c i )U bl A (t i )) = n i=1 λa i ∆U D (t i ) ∆U A (t i ) (c i U bp A (t i ) + (1 -c i )U bl A (t i ))
From the above

U A (D, A) ≤ U A (D, A ) ⇐⇒ U A (D, A) ≤ U A (D, A ) because
the left hand side of the ⇐⇒ is the right hand side multiplied by the constant(s) λ ∆U D (ti) ∆U A (ti) . Hence minimum and maximum of the ordering are preserved, therefore a minimax in G is also a minimax in G. Using proposition 1 we conclude that a defense strategy D is NE equilibrium in G and G therefore it is a minimax in G and, by the above argument, it is also a minimax in G.

This result guarantees that independently from the attacker's strategy the defender's solution is optimal.

Minimax Solver with Singular Value Decomposition

Given the high number of possible targets within an organization the game formulation is subject to the "state explosion problem" [START_REF] Hankin | Payoffs, intensionality and abstraction in games[END_REF]. Therefore there is a need for a method which will compute Nash defenses for large number of targets. This must be computationally fast and provide reasonable precision in the calculation of the equilibrium. A possible approach is to look for an abstraction of the payoff matrices where only the most relevant features of such matrices are kept and minor features are discarded. We achieve this by using Singular Value Decomposition (SVD) to reduce the rank of the individual payoff matrices. To the best of our knowledge, there are no general results about game solutions for rank reduced payoff matrices. The closest work, undertaken by Kannan and Theobald [START_REF] Kannan | Games of fixed rank: A hierarchy of bimatrix games[END_REF], performs rank reduction of the matrix given by the sum of the payoff matrices of the two players.

Based on the Ponemon report 2013 [START_REF]State of web application security[END_REF], we have illustrated the payoff matrices in Table 1. We notice that these matrices have a particular underlying structure which is mostly captured in the very first singular values. In other words, there are few dominant singular values, usually two or three. By ignoring the singular values after the second or third largest, we obtain reasonably close solutions to the original game solutions and a dramatic speed up of the computation. As an example for a game with 8 targets, 1 security administrator, and payoff matrices illustrated in Table 3, the singular values are 792, 108, 91, 42, 22, 12, 10, 3 . This means the first component already provides a reasonable approximation to the original matrix and the components after the third component can be treated as "noise". The speed-up in performance is mainly caused by the fact that the rank reduction, in our games, results in a large number of strategies becoming dominated. For example in a game with 120 strategies and rank 10, around 3/4 become dominated when the rank is reduced to 6. By comparing the equilibria found in a 10 targets, 2 system administrators game and its SVD rank 2 abstraction (Table 2), we found that there is aperformance improvement of more than 1000 times while the approximate solution only slightly deviates from the precise solution.

Model Comparison

To evaluate our game theoretic model we want to compare the performance of the Nash resource allocation method (Minimax solver with Singular Value Decomposition) which is given by the NE of the cyber security game, against alternative methods. The methods we have selected for comparison are based on approximations to two common sense approaches that one might consider, called Uniform and Weighted as well as a stochastic optimization algorithm called Acceptable Coverage.

The Uniform defense distribution gives no preference to any target that a defender wants to defend meaning that all targets are given equal probability to be defended. This method is a naive approach that assumes no knowledge on the part of the defender to decide how much each target is valued or how likely a target is to be attacked.

The Weighted defense distribution creates a probability distribution based on the value of each target that is being defended. According to this approach, the time a defender allocates to protect a target is proportional to its importance. A distribution of time across schedules is calculated such that each target is covered and the sum of the probability for playing each of the particular sets of defenses equals the intended coverage for each target. For example if a target t 1 should be defended with probability 0.2, then all schedules that include a defense of t 1 should sum to 0.2. This results in more time being scheduled to perform tasks which improve the defense of the targets that are either more vulnerable or more valuable to the organization. While this does not necessarily represent the best possible cyber security decision, this method identifies at least an average decision maker, that provides a reasonable defense by which to measure the improvement that the Nash defense introduces. Acceptable Coverage. We compare the minimax solution to a stochastic optimization algorithm with uncertainty in the attacker's payoffs. This approach is similar to the one published in [START_REF] Kiekintveld | Security games with interval uncertainty[END_REF] where the authors define a defensive coverage as the one maximizing the worst-case payoff over the targets in the potential attack set3 . For a given target we have two intervals from which the attacker gets their utility based on the applied defense; baseline or best practice. These are represented as [U min,bl A , U max,bl A ] and [U min,bp A , U max,bp A ], respectively. When uncertainty about the attacker's payoff is introduced, the concept of attack set gets more involved because we do not know what the maximum expected payoff for the attacker is. One is then led to define a potential attack set for a coverage as the set of all targets that could give the attacker the maximum expected value, for any attacker's payoffs that can be extracted from the intervals of the payoffs. Given a coverage C the attacker can be guaranteed a payoff of at least the maximum of the minimum values over all targets;

R(C) = max ti E min A,C (t i ) where E min A,C (t i ) = a i (c i U min,bp A (t i ) + (1 -c i )U min,bl A (t i ))
. R(C) defines the potential attack set associated to the coverage C as

A(C) = {t i |E max A,C (t i ) ≥ R(C)}.
Following these ideas we look for a defender's strategy giving the best utilities guaranteed. As such the defender will choose a strategy providing best coverage, where a coverage is better than another coverage if the former guarantees both a higher minimum expected utility for the defender and a lower maximum expected utility for the attacker compared to the latter. More formally we define the coverage ordering as follows:

C ≥ C ⇐⇒ min i (E D,C (t i )) ≥ min i (E D,C (t i )) ∧ max j (E min A,C (t j )) ≤ max j (E min A,C (t j ))
Given a set of coverages maximal with respect to the above ordering a coverage is acceptable if it is the maximal element, with respect to the defender's utility, of this set. If a coverage is acceptable then we have considered all attacker's payoffs that can be extracted from the payoffs intervals otherwise there would be a coverage above it, contradicting maximality.

Comparisons. We have undertaken simulations to identify how effective a Nash defense performs against distributions created by the Uniform, Weighted and Acceptable Coverage methods. For the purposes of testing we consider an organization like an online store whose assets include users payment details, website configurations and data related to operational procedures. The most valuable target would be a database server with the most of clients' details. This could be compromised by an SQL injection attack. The second most valuable target would be the store's website that an attacker could deface by modifying data of the web server the website is located at. The rest of the targets could consist of the mail server and different workstations. The device of each user, seen as a target, has a value related to the privileges that a user obtains. To undertake our comparisons, the values of utilities for each target have been taken from a report on data breaches [START_REF]State of web application security[END_REF], where the utility per target is given by mid-value for each of the ranges of damage reported, and are presented in Table 1. We consider two different sizes of online stores in terms of reputation, number of clients and amount of data held. The larger store has 10 targets while the smaller one has 8. We assume that the organization is limited to the number of system administrators that they have available to them and cannot cover all the targets at target ID t1 t2 t3 t4 t5 cost ($) 10k 30k 50k 75k 150k target ID t6 t7 t8 t9 t10 cost ($) 250k 350k 450k 750k 3m one time. We test the defense of a system with up to 3 administrators. These choices seem reasonable for such organizations in our proof-of-concept comparisons. The utilities for the attacker in this scenario are set at 20% of the value that the defender has for each given target. This is to represent that an attacker will still receive a large payoff for successfully attacking a target. However while the defender may have long term damage, the attacker's profit will generally be given by the immediate impact that their actions have. For the comparisons we have used 2 specific attack distributions given by the Hackmageddon [2] and Verizon data sets [1], where any of the attack distributions, shown in Tables 3 and4, represents an unknown attacker that is attempting to breach a target.

While we have identified the range of damages that can be expected from a successful attack, the values themselves do not necessarily reflect a single attack, so we consider varying the utilities both players perceive for each target. By varying the utility of each target, we are capable of identifying if the Nash method performs better than either of the other methods independently of the cost of each target. We vary the utilities as presented in Table 1 by adjusting the values perceived by the defender, where the experimental bounds allow for a deviation of up to 10% from the stated utilities for each target. The baseline defense has been fixed at a 50% breach rate for each target while the best practice defense reduces that rate to 20%. These numbers have been decided artificially for the model validation.

Performance evaluation. We define the performance of a solution as the average amount of damage expected from any single attack, where the expected damage is calculated as the result of an attack on a given target. In each single attack one or more targets are defended following a schedule determined by the specified defender's strategy. If a target that is being attacked is not defended, then the damage equals the baseline damage. If the target is defended, the loss equals the damage with the best practice defense. To measure the performance of a solution we have created a simulated environment, in Python, used to perform attack sampling. For all comparisons performed, a sample size of 100,000 attacks was used. Such a sample is referred to as a run in the results. In the following we present the results, where 25 runs have been performed in each case and the average of the percentage improvement and the standard deviation seen across the runs have been plotted. We have set the rank values for SVD to 5 in both 8 and 10 target scenarios. These values allow us to minimize the runtime for each of the comparisons while maximizing the accuracy of the results.

The percentage improvement seen in comparison to the uniform defense shows a minimum improvement of approximately 15%. Comparably, the smallest average improvement for Nash defense over the Weighted is around 7%. In addition to this, the maximum improvement seen in the Nash solution over the Uniform is approximately 50%, where the maximum improvement over a Weighted defense does not exceed 25%. The improvements between different numbers of administrators for defending a single system identify the impact that the addition of an administrator has in improving the defense. We see that with 8 targets for both attack distributions the addition of more administrators increases the improvement seen in the defense of the system by more, over the common sense approaches. A large difference in improvement indicates that adding an additional administrator will have a greater impact on the defense of the system than in the case of a lower level of improvement. This growth however saturates, as when the number of administrators tends towards the number of targets the improvement seen tends to zero. This happens because all targets can be covered thus the expected damage is minimized across the whole system for all defender strategies.

When comparing the Nash and Acceptable Coverage (AC), we see that unlike the comparisons to the common sense approaches, the Nash defense does not always perform better. In contrast to the approximate performance as measured against the common sense approaches, the average improvement does not exceed 10% for either method. With the 8 target scenario, the results show that the AC performs better for 1 administrator regardless of the data set used; specifically the AC method appears to be approximately 2% to 3% better. However, when there are 2 system administrators, the Nash defense performs between 2% and 6% better depending on the data set. With 3 system administrators the results show that on average, for a Hackmageddon attacker, the AC method performs better, but the Nash defense is preferable with a Verizon attacker. It should be considered that the deviation of these results shows that in some circumstances the improvement of one method over the other can be less than 1%. For 10 targets and 1 administrator we see that with both datasets there is no improvement in the average performance of the Nash solution over AC, with a very small standard deviation. This indicates that the allocation of system administrators' time to targets is similar for both Nash and AC, where the randomized sampler gives the deviation in the results. For more than 1 administrator it appears to show a relatively large positive improvement for the Nash defense over the AC, where the range improvement is approximately 2% and 5% for 2 and 3 administrators respectively using a Hackmageddon attacker. For a Verizon attacker the improvements are close to 6% and 8% for 2 and 3 administrators. As has been noted above, there is a small deviation in the approximate scheduler used for the AC method, which may account for some of the improvement seen by the Nash defense and the reason for the large deviations in some of the results.

Conclusion

In this paper, we have introduced a cyber security model along with game theoretic tools to prove that common sense techniques are not as efficient at providing effective defense schedules as the ones computed by game theory. Our future work includes interviews with system administrators to define levels of values for the different model components. We are also interested in validating the effectiveness of the model when considering an organization where the number of targets available to be attacked will range between 25 and 100. The efficiency of performance for SVD is important given that there is a trade-off in time and accuracy, where we have seen that even for small games it is computationally inefficient to derive the equilibria. As such an interesting extension to this work would be to measure the performance of SVD in terms of efficiency when large games are played. We have described an environment where the defense of a system is pro-active, but we don't consider the scheduling of time in a reactive manner in order to recover systems after a successful attack. In order to recover a system, the time of available system administrators needs to be assigned to this task, which limits the number of system administrators available to maintain the defense of the system. Therefore a further extension of the game considers the concept of optimally dividing the available resources between the recovery of the system and the maintenance of pro-active security. One limitation of the current model, that future work will address, is that in many cases the methods taken by an attacker to break into a system may have steps that are relevant to more than one target. In this case there are specific actions that can be performed by a system administrator that effectively cover multiple targets.

Lemma 1. An attacker's strategy A is best response to a defender's strategy D in G iff A = µ(A ) is the attacker's best response to D in G.

Proof. To prove this lemma we must prove that

U G A (D, A ) -U G A (D, A ) ≥ 0 ⇔ U G A (D, A ) -U G A (D, A ) ≥ 0, ∀A
Solving (4) for a i we have that a i = 1 λ a i ∆U A (ti)

∆U D (ti) and as we know that ∆U A (t i ), ∆U D (t i ), λ ≥ 0 the following holds:

U G A (D, A ) -U G A (D, A ) ≥ 0 ⇔ n i=1 a i (c i U bp A (t i ) + (1 -c i )U bl A (t i )) - n i=1 a i (c i U bp A (t i ) + (1 -c i )U bl A (t i )) ≥ 0 ⇔ n i=1 (a i -a i )(c i U bp A (t i ) + (1 -c i )U bl A (t i )) ≥ 0 ⇔ n i=1 1 λ ∆U A (t i ) ∆U D (t i ) ( a i -a i )(c i U bp A (t i ) + (1 -c i )U bl A (t i )) ≥ 0 ⇔ n i=1 ( a i -a i )(c i U bp A (t i ) + (1 -c i )U bl A (t i )) ≥ 0 ⇔ U G A (D, A ) -U G A (D, A ) ≥ 0
U G D (D , A) -U G D (D , A) ≥ 0 ⇔ n i=1 a i (c i U bp D (t i ) + (1 -c i )U bl D (t i )) - n i=1 a i (c i U bp D (t i ) + (1 -c i )U bl D (t i )) ≥ 0 ⇔ n i=1 a i (c i ∆U D (t i ) + U bl D (t i )) - n i=1 a i (c i ∆U D (t i ) + U bl D (t i )) ≥ 0 ⇔ n i=1 a i (c i -c i )∆U D (t i ) ≥ 0 ⇔ n i=1 1 λ ∆U A (t i ) ∆U D (t i ) a i (c i -c i )∆U D (t i ) ≥ 0 ⇔ n i=1 a i (c i -c i )∆U A (t i ) ≥ 0 ⇔ n i=1 a i (c i -c i ) ≥ 0 ⇔ ∆ U D (ti)≥0 n i=1 a i (c i -c i )∆ U D (t i ) ≥ 0 ⇔ U G D (D , A) -U G D (D , A) ≥ 0

Fig. 1 .

 1 Fig. 1. Improvement of Nash defense over the different methods for both Hackmageddon (H) and Verizon (V) attackers.

Lemma 2 .

 2 A defender's strategy D is best response to an attacker's strategy A in G iff D is also best response to the attacker's strategy A = µ(A) in G.Proof. To prove this lemma we must prove thatU G D (D , A) -U G D (D , A) ≥ 0 ⇔ U G D (D , A) -U G D (D , A) ≥ 0, ∀D .We prove this inequality as follows

Table 1 .

 1 Financial cost of an organization based on the findings of the Ponemon 2011 report[START_REF]State of web application security[END_REF].

	Rank	Time (s) Value
	10 (no SVD) 204 -615.744
	8	60	-614.854
	6	1.8	-617.732
	4	0.4	-616.124
	2	0.1	-616.651
	Table 2. Times and defender utilities
	for a 10 targets 2 system administrators
	games using SVD.

Table 3 .

 3 Probability of Attack per Target based on Statistics from Hackmageddon [2].

	ID	t1	t2	t3	t4	t5
	8 t. 0.060 0.060 0.060 0.080 0.133
	10 t. 0.021 0.021 0.021 0.021 0.063
	ID	t6	t7	t8	t9	t10
	8 t. 0.222 0.222 0.244 -	-
	10 t. 0.008 0.125 0.208 0.208 0.229

Table 4 .

 4 Probability of Attack per Target based on Statistics from Verizon [1].

In security games without uncertainty, the attack set is defined as the set of all targets that give the attacker the maximum expected payoff, given some coverage.