Maxime Compastié
email: compastiem@yahoo.fr

Rémi Badonnel

Olivier Festor

Ruan He

From Virtualization Security Issues to Cloud Protection Opportunities: An In-Depth Analysis of System Virtualization Models

Keywords: Security Management, System Virtualization, OS-level Virtualization, Cloud Infrastructures, Unikernel

Virtualization methods and techniques play an important role in the development of cloud infrastructures and their services. They enable the decoupling of virtualized resources from the underlying hardware, and facilitate their sharing amongst multiple users. They contribute to the building of elaborated cloud services that are based on the instantiation and composition of these resources. Different models may support such a virtualization, including virtualization based on type-I and type-II hypervisors, OS-level virtualization, and unikernel virtualization. These virtualization models pose a large variety of security issues, but also offer new opportunities for the protection of cloud services. In this article, we describe and compare these virtualization models, in order to establish a reference architecture of cloud infrastructure.

We then analyze the security issues related to these models from the reference architecture, by considering related vulnerabilities and attacks. Finally, we point out different recommendations with respect to the exploitation of these models for supporting cloud protection.

Introduction

System virtualization is an essential key to the development of cloud infrastructures and their services. In these environments, cloud service providers (CSP) expose resources to consumers, while these ones exploit these resources to run services or to compose new elaborated ones that can be offered to other consumers [START_REF] Mell | The NIST Definition of Cloud Computing[END_REF]. The multiplicity of stakeholders questions the security at several levels and, consequently, questions the security of the underlying system virtualization: (i) the cloud service level agreement (SLA) specifies the availability of virtualized resources, (ii) the broad network access to cloud resources and the potential multi-tenancy requires the isolation of virtualized resources, (iii) the growing regulation with respect to data protection puts additional constraints on the CSPs to guarantee the confidentiality and integrity of resources [START_REF][END_REF]. Recently, important efforts have been focused on performance improvement, leading to the emergence of new virtualization technologies capable of reducing virtualized environment footprints. In particular, containerization methods move the virtualization layer from the OS-hardware border to the OS-application one, in order to share the OS kernel. Also, unikernel-based virtualization permits to decrease the complexity of virtual environ-ments, by elaborating minimal operating systems specifically built for dedicated applications.

The development and deployment of these virtualization models are still in the early stage, while the technologies and related products are not fully mature. It therefore remains unclear what kind of specific security issues it introduces. We envision that the attack surface of virtualization could be impacted by the hypervisor isolation, cross-layer invocation and containerization. These factors make virtualization extremely difficult to establish in-depth defense security, requiring the critical security issues to be addressed in a holistic way. In particular, from a vertical perspective, applications inside a virtual machine (VM) are diversely incorporated with several layers and/or components, ranging from hypervisor to guest OS. Thus, any misconfiguration of a VM instance or hypervisor could eventually allow attackers to penetrate into the applications. From an horizontal perspective, as each layer is composed of heterogeneous components from different providers, the trust relationship between these components, either hardware or software, is hard to be established for the assessment of security properties. It is therefore challenging to securely and seamlessly incorporate these components.

Our contribution addresses the security of system virtualization models used for cloud infrastructures. We provide an in-depth description of the virtualization models to conduct a security analysis. Contrary to [START_REF] Sahoo | Virtualization: A Survey on Concepts, Taxonomy and Associated Security Issues[END_REF], we do not focus on the security question related to the operation of cloud environments. The analyzed virtualization models are not limited to full-fledged VM system virtualization, as considered in [START_REF] Pearce | Virtualization: Issues, Security Threats, and Solutions[END_REF], but we also include OS-level virtualization and unikernel VM system virtualization. Our approach is close to [START_REF] Pattaranantakul | NFV Security Survey: From Use Case Driven Threat Analysis to State-of-the-Art Countermeasures[END_REF], but our analysis is not bound to any cloud operation use-case. It rather permits to infer recommendations for our software-defined security strategy for distributed clouds, as elaborated in [START_REF] Compastié | Software-defined Security for Distributed Clouds[END_REF]. To evaluate the benefits and limits of virtualization models for cloud security, this article is organized as follows. First, we describe in Section 2 different virtualization models, including virtualization based on type-I and type-II hypervisors, OS-level virtualization and unikernel virtualization. The purpose is to give a basic understanding of their design principles and relationships, and to establish a reference architecture that synthesizes these models and serves a support for the security analysis. Second, we analyze in Section 3 the security of the different virtualization models, based on this reference architecture. In that context, we identify and classify the vulnerabilities that may affect the components of this architecture. We then quantify their probability of occurrences, and detail the related security attacks, in view of existing security threats. Third, we infer in Section 4 different counter-measures and recommendations with respect to the exploitation of these models for cloud security. This includes the integration of security mechanisms during the design of resources, the minimization of the attack surface based on the generation of highly specialized resources, and the adaptation of cloud protection based on security programmability.

System Virtualization Models

First of all, we will describe the considered system virtualization models, in order to establish a reference architecture of cloud infrastructure, that will serve as a basis for the security analysis. In that context, we will also remind some important principles related to system virtualization.

Context

According to [START_REF] Popek | Formal Requirements for Virtualizable Third Generation Architectures[END_REF], system virtualization can be defined as the use of an encapsulating software layer surrounding an operating system, which conforms to the behavior expected from a physical hardware. It is first envisioned as an enabler for multi-tasking: it has enabled hardware resource multiplexing, and thus, the concurrent execution of several OSHowever, more efficient approaches dealing with this issue have outrun the interest for system virtualization for decades. In 1990s, the emergence of personal computing and the limited compatibility of software appliances to one or a few number of operating systems has renewed the interest for system virtualization. The desktop virtualization enables the usage of an appliance developed on one operating system with another one. The large-scale deployment of cloud computing during 2010s has induced a new application field. As the cloud paradigm requires the decoupling of software resources from hardware resources, system virtualization has become one of the enablers of this paradigm.

We usually define a regular system architecture according to a two-state mode execution environment (non privileged mode and privileged mode), as illustrated in Fig. 1. Programs running in the non privileged mode rely on a subset of non-critical instructions sets (which will not impact the state of the machine, defined later), while programs in the privileged mode are allowed to use the whole instruction set. When a program running in the non privileged mode wants to invoke a privileged instruction (instruction only allowed to run in the privileged mode), it sends an interruption, called trap. Such a trap will trigger the execution of a predefined routine in the privileged mode. All user applications which are in the non privileged mode are loaded to the memory address space as user space. The programs in the privileged mode are loaded to the memory address space as kernel space. One program can only run in one specific mode. When a user application wants to use a privileged instruction, it is trapped through a specific interruption. The trapping mechanism stops the execution of the current application, stores the execution context, loads the corresponding routine, and executes the routine in the privileged mode. When the routine execution terminates, the execution context will be switched back to the user application which is in the non privileged mode. The set of system calls is defined as a subset of routines which can be invoked by applications in the non privileged mode.

System Virtualization

The system virtualization architecture derives from the regular system architecture. It enables concurrent execution of multiple OS. Within this architecture, two categories of instructions exist: sensitive and non-sensitive. Non-sensitive instructions can be executed directly on the hardware. However, sensitive need to be controlled since they modify the state of a machine which may impact a concurrent execution. The software module necessary to implement this virtualization architecture is called Virtual Machine Monitor (VMM), whose properties are defined in [START_REF] Popek | Formal Requirements for Virtualizable Third Generation Architectures[END_REF]. A VMM provisions an abstracted and isolated environment for each OS, which is called Virtual Machine (VM). It is a program interfacing between programs in each VM and hardware resources.

Fully and Partially Virtualizable Architectures

A fully virtualizable architecture is defined such that all its sensitive instructions are controlled. The VMM uses the trapping mechanism to realize this control. Hence, an architecture is considered as fully virtualizable, if and only if all its sensitive instructions are part of the privileged instructions.Thus, intercepting the instructions from unprivileged mode permits to handle the execution of sensitive instructions, while the privileged ones can be controlled using traps. The existence of sensitive but unprivileged instructions prevents the VMM from using trapping. Such instructions do not trap and thus do not trigger VMM instruction simulations, enabling the VM to access the hardware resources directly. For instance, the x86 architecture does not comply with the fully virtualizable architecture [START_REF] Robin | Analysis of the Intel Pentium's Ability to Support a Secure Virtual Machine Monitor[END_REF] since some instructions accessing sensitive registers are invokable from non-privileged mode. Such an architecture is called partially virtualizable architecture. It is then required to treat these non-virtualizable instructions with other mechanisms than instruction trapping. Three methods are usually considered to address this issue.

The binary translation method relies on a live interpretation of all sensitive instructions by VMM routines. Once a VM program has its code loaded into memory and its execution started, the VMM scrutinizes ongoing instructions to dynamically substitute sensitive ones by VMM routine calls before their usage. The para-virtualization approach consists in the modification of the program source code employing sensitive instructions before their instantiations, substituting them by equivalent VMM routine calls (equivalence property) which are referred to hypercalls. Considering the case of an OS in a virtual machine, the para-virtualization related code modifications are only located in the kernel of the VM, to make it cope with the VMM through hypercalls.The hardware-assisted virtualization consists in a solution based on CPU capabilities. More precisely, when hardware-assisted virtualization is available, the CPU proposes a root operation mode intended to host the VMM, while the non-root operation mode supports the VM code execution. These modes are not related to the privileged and non-privileged ones: the root mode enables programs to define which instructions, executed in the non-root mode, will trap back.

VMM Implementation Types

As a common usage, VMMs (also called hypervisors) are implemented with extra-features enabling VM management (management console) and virtual hardware provisioning to VMs. In this article, we refer to an hypervisor as a VMM embedded with a management console, a memory manager, an input/output subsystem and a networking subsystem. We define a host system as the only set of OS and applications to have access to hardware resources not mitigated by the hypervisor. A guest VM stands for a VM with a restricted access to hardware resources. We distinguish two types of VMMs. The type-I hypervisor corresponds to a VMM directly operating the hardware resources. The host system runs alongside guest VMs, and both of them have to cope with the hypervisor. Their main difference at runtime is that the host system is allowed by the hypervisor to access the hardware without restriction and has administrative privileges over the hypervisor, while the guest VMs are handled by the virtual hardware environment. The Xen hypervisor [START_REF]The Xen Project, the Powerful Open Source Industry Standard for Virtualization[END_REF][START_REF] Barham | Xen and the Art of Virtualization[END_REF] and the KVM hypervisor [START_REF] Kivity | KVM: the Linux Virtual Machine Monitor[END_REF] are two examples of type-I hypervisor implementations. The type-II hypervisor runs as an application of the host system. It cannot directly access the hardware, but relies on the host OS routines to access hardware resources. Oracle Virtualbox [12] and QEMU [START_REF] Qemu | Available from[END_REF] are two well-known examples of type-II hypervisors.

OS-Level Virtualization

Hypervisors and VMs contribute to a proven virtualization architecture, currently being used in numerous use cases among which cloud computing, software testbed environments and software analysis framework. However, from an application isolation perspective, embedding OS kernel and virtual hardware environments with applications in a VM may be challenged by two optimizing issues. First, each VM embeds an OS kernel instance, meaning a resource cost not related to VM application exploitation. Second, the trapping mechanism induced by the VMM is a CPU-cycle greedy mechanism.

OS-level virtualization2 attempts to increase efficiency by eliminating OS kernel from the isolation scope. It keeps only the set of applications and dependent libraries within a container. They run in parallel with other programs of the host system and both of them access the OS kernel and hardware resources, through the common OS kernel interfaces exposed to programs residing in the user space (system calls) [START_REF] Kolyshkin | Virtualization in Linux[END_REF]. The host OS kernel is in charge of required virtual resources (i.e. filesystem access, networking) and container isolation enforcement. The container execution environment benefits from kernel features and its configuration is performed by the container engine. This extra layer runs at the host system application level. By these means, OS-level virtualization enables almostbaremetal [START_REF] Felter | An Updated Performance Comparison of Virtual Machines and Linux Containers[END_REF][START_REF] Soltesz | Container-based Operating System Virtualization: A Scalable, High-performance Alternative to Hypervisors[END_REF] performances, outrunning the regular system virtualization ones, at the cost of coupling containers with a specific operating system. LXC [START_REF]Linux Containers -LXC -Introduction[END_REF], Docker [START_REF] Docker -Build | Ship, and Run Any App[END_REF] and gVisor [START_REF]gVisor: Container Runtime Sandbox[END_REF] are three thriving examples of container engines supporting the Linux OS.

Unikernel Virtualization

Containerization has brought an application-centric perspective in the virtualization debate, as the applications and their dependencies are the only embedded system parts in the portable environment. OS-level virtualization induces the dependency of in-container applications upon the host system OS kernel. It incidentally restricts the set of applications eligible to virtualization to the ones supporting this OS kernel. Additionally, in the performance area, containerized applications have to cope with host system OS kernel routines. Since containers cannot embed routines running in privileged mode, they cannot implement optimized privileged routines for dedicated purposes. Unikernel virtualization tackles these two issues, by transposing Library OS concept to the system virtualization era. Through the bypassing of legacy support constraints, it enables the refining of the system layer, reducing footprints of virtualized applications.

Library OS

A library OS exports the hardware resource management outside the OS kernel. In practice, this management relies on a set of libraries, implementable by any application whose access to the related resources is required. Each application is able to implement the most adapted resource managers to its mission, while the dispatching of resource management amongst applications relieves the need of intermediate layers when accessing the hardware. In return, this architecture comes at the cost of a restricted portability and compatibility toward managed resources. The managers are expected to be resource-specific to contribute to their efficiency. However, the applications are not likely to implement all of them to support all different types of hardware resources. The Drawbridge [START_REF] Porter | Rethinking the Library OS from the Top Down[END_REF], the V++ cache kernel [START_REF] Cheriton | A Caching Model of Operating System Kernel Functionality[END_REF] and the Exokernel [START_REF] Engler | Exokernel: An Operating System Architecture for Application-level Resource Management[END_REF] projects are three architectures for library OS implementation exporting resource management in user space. The first converts Microsoft Windows 7 into a library-OS compliant architecture at the top of a VMM for process isolation purpose (pico-process) at the trade-off of keeping hardware management in the kernel space of the host OS to ensure compatibility. The second one only exploits memory address space segregation in order to keep critical OS kernel features in the kernel space such as memory management, interprocess communications and thread management. The last one supports application multi-tasking and a secured hardware resource management, which is endorsed by libraries and is secured by a dedicated layer. Examples of Exokernel implementations include Aegis [START_REF] Engler | Exokernel: An Operating System Architecture for Application-level Resource Management[END_REF], ExOS [START_REF] Engler | Exokernel: An Operating System Architecture for Application-level Resource Management[END_REF] and XOmB [START_REF] Larkby-Lahet | Xomb: an Exokernel for Modern 64-bit, Multicore Hardware[END_REF].

Unikernels

A unikernel corresponds to an OS architecture featuring a specialized, sealed, single purpose library OS. They can be run in virtual machines on the top of a hypervisor [START_REF] Madhavapeddy | Unikernels: Library Operating Systems for the Cloud[END_REF]. Each unikernel system addresses an application, its configuration, and the minimal dependencies required to run, and is configured and packed into an image, before their instantiation. Running inside a VM relieves these library OSes from supporting a vast amount of hardware configurations, and focus only on the virtual hardware environment exposed by an hypervisor. Additionally, the unikernel architecture does not provide backward compatibility. This approach contributes to lightweight unikernel VMs compared to regular VMs. MirageOS [START_REF] Madhavapeddy | Unikernels: Library Operating Systems for the Cloud[END_REF] and In-cludeOS [START_REF] Bratterud | IncludeOS: A Minimal, Resource Efficient Unikernel for Cloud Services[END_REF] are two examples of common unikernel solutions.

A single unikernel image addresses the integration of one application, with its minimal set of dependencies. These dependencies address the application runtime, and libraries for both the software and the hardware resource management, in line with the library OS concept. Software components addressed by unikernel images may be tightly related to a development platform or may remain independent. The first option requires the usage of a dedicated tooling and programming language to design a unikernel appliance. All the software components undergo the same processing (i.e. static analysis, code optimization, code compilation, linking and packing), contributing to the performance and lightweight properties of the resulting image. From the system architecture perspective, this option permits to include more system layer components in the assessment scope of programming language properties. The second option can be assimilated to the first one if a specific runtime environment is packed in the image. The in-situ software management capability is not provided in the unikernel image. Instead, it is performed externally, directly on the image before its instantiation, by the building of a toolstack, relying on a package manager, or be only uprooted on code inclusion mechanisms. This ex-situ approach contributes to lighten the unikernel footprint when instantiated.

An instantiated unikernel uses a single address space for its application and runtime, including the hardware management. This permits to cope with the overheads of context switches issued by process scheduling and system calls. Unikernel instances do not implement processes, but still rely on multi-threading to support running parallel tasks. Built unikernel images can be conceived to be instantiable by a hypervisor, and cope with the related virtual hardware environment. This capability enablement relies on the inclusion of hypervisor-specific hardware resource management routines into unikernel images, in order to address a specific virtual hardware environment. In the case of the exploitation of unikernel VMs, type-I hypervisors are usually privileged to minimize the layers mitigating the unikernel VM access to hardware resources. The routines of the unikernel VMs are executed in the privileged mode to avoid the trapping overhead, when accessing the virtual hardware environment.

Synthesis

We have presented different virtualization models, including virtualization based on hypervisors of type-I, virtualization based on hypervisors of type-II, OS-level virtualization and unikernel-based virtualization. We infer a virtualization reference architecture, depicted on Fig. 2 that synthesizes these virtualization models. This architecture will serve as a basis to our security analysis and is composed of four levels:

• The hardware level, represented at the bottom row of the figure, encompasses physical resources composing the host machine, such as the CPU, the volatile memory (RAM), the I/O interfaces (for persistent storage and extension cards) and the network adapter for networking purposes.

• The host OS level, detailed at the homonym row in the figure, is exploited only for type-II hypervisor and containers engines, and accounts for host OS kernel, base utilities and core libraries used by these VM softwares. Type-I hypervisor and unikernel-related hypervisor may completely manage themselves hardware resources, or delegate this management to privileged VM (e.g. storage or network device handling).

• The hypervisor level, mentioned in the figure in the next row, refers to hypervisors and container engine softwares. We assume it handles VMM capabilities (VM instruction trapping and sensitive instructions handling), VM memory management, VM networking (virtual network adapter and inter-VM networks) and VM I/O interfaces. The hypervisor is configured through a management console, enabling a system administrator or a service to manage the hypervisor and VM configurations.

• The virtual machine level refers to VMs, containers and unikernels, and corresponds to the top row of the figure. This level includes applications, their configurations, their runtime environment and the libraries they depend on. Utilities are not systematically embedded in unikernels, since their features are embedded as application libraries, when they are required.

In the same manner, OS kernel is neither integrated in containers (containers rely on host OS kernels) nor in unikernels (the OS kernel is reshaped in a set of libraries, provided with the application).

Virtualization provides important properties with respect to security [START_REF] Pearce | Virtualization: Issues, Security Threats, and Solutions[END_REF], in particular:

• Isolation: VM access to physical resources is regulated by the hypervisor. This control affects inter-VM access as well, and confers resource isolation capabilities to virtualization. Moreover, by allocating quotas to the physical resources to VMs, virtualization also promotes resource consumption isolation.

• Oversight/introspection: the hypervisor is able to inspect a VM resource usage and thus observe its internal state, leading to the oversight capability. As the hypervisor is also in charge of VM resource allocation, internal state modifications may be performed, defining the introspection capability.

• Snapshotting: the hypervisor enforces access control amongst a VM and physical resources. This positioning enables the hypervisor to control the VM execution, by interrupting and resuming it. The content of allocated resources can be saved and reallocated as well, paving the way for VM internal state exportation and restoration. From a security perspective, this feature permits reversing back a VM from a given insecured state to a previous secure one.

In the meantime, virtualization makes the system architecture more complex, by introducing new components (e.g. hypervisor) and redefining interactions amongst system architectural components (e.g. privileged instruction trapping). This may also introduce new security issues that are analyzed in the following section.

Security Analysis based on the Reference Architecture

We conduct a security analysis based on the reference architecture. First, we investigate the vulnerabilities of this architecture, draw a corresponding taxonomy, and evaluate qualitatively the criticality of vulnerabilities (in terms of occurrences) with respect to virtualization models. We then determine the threats affecting the architecture, and analyze related attacks in view of identified vulnerabilities.

Identification of Vulnerabilities

We first identify and classify vulnerabilities related to the reference architecture, as depicted on Fig. 3. The criticality of these vulnerabilities with respect to virtualization models is synthetically exposed in Table 1, and is justified with the presentation of vulnerabilities. We investigate vulnerabilities carried by the VM as a part of a top-down study based on the reference architecture. We assume a VM hosting an application, its library dependencies and an OS kernel. We then consider the vulnerabilities affecting the hypervisor (or the container engine) running on the host machine. Nested virtualization is seen as a particular case of VM applications. In that context, we detail vulnerabilities related to the following four main categories: the VM application, the VM guest OS, the hypervisor, and the execution environment of the hypervisor.

VM Applications

The vulnerabilities of VM applications are typically related to memory management (with respect to runtime variable type checking, memory deallocation, kernel inference in user space, and developpement software flaws), and to software interfaces (with respect to access control, possible code injection, and concurrency).

Memory Management. An application instantiated in a VM relies on memory management routines provided by language interpreters, OS standard libraries and kernel internal routines. The lack of type checking on variables at runtime introduces trivial memory management based exploitation such as buffer overrun [START_REF] Hastings | Purify: Fast Detection of Memory Leaks and Access Errors[END_REF][START_REF] Qin | SafeMem: Exploiting ECC-memory for Detecting Memory Leaks and Memory Corruption During Production Runs[END_REF] or integer overflows. This vulnerability is emphasized by the size of the code base of the applications deployed on virtual environments. Consequently, type-I and type-II hypervisors are the most subjected to this vulnerability, while OS-level virtualization benefits from OS kernel removal. As unikernels have their code base highly constrained, their virtualization model is the least affected. Memory deallocation also induces security issues. Except for interpretor-based execution environments equipped with a memory garbage collector, no memory space is ensured to be automatically deallocated when stopped being used. This may lead to process memory leaks [START_REF] Hastings | Purify: Fast Detection of Memory Leaks and Access Errors[END_REF][START_REF] Qin | SafeMem: Exploiting ECC-memory for Detecting Memory Leaks and Memory Corruption During Production Runs[END_REF][START_REF] Bovet | Understanding the Linux Kernel: from I/O ports to Process Management[END_REF], and data persistence issues [START_REF] Solomon | User Data Persistence in Physical Memory[END_REF]. This vulnerability is related to traditional system architecture with multiple processes, and impacts type-I and type-II hypervisors as well as OS-based virtualization. The single application support of unikernels leaves physical memory cleaning to the hypervisor at the application termination. Kernel inferences in userspace may also be possible. The OS kernel is in charge of system management, by providing routines that are used by application processes. Nevertheless, the kernel is also able to manage this address space for various purposes, such as process swapping, free space reclamation [START_REF] Kook | Optimization of out of Memory Killer for Embedded Linux Environments[END_REF], and exception handling. All virtualization models are exposed to this vulnerability. While it is obvious in a regular system architecture, unikernels rely on a built-in runtime for the whole support of an application. Finally, we should remind that any program comes with software flaws and bugs issued from their development cycles, inducing new exploitable vulnerabilities [START_REF]CWE/SANS Top 25 Most Dangerous Software Errors[END_REF]. This echoes to the unikernel pleading to limited code base, while OS-level containers benefit from not embedding an OS kernel.

Software Interfaces. In VMs, the software programs rely on interfaces to communicate with the users and other programs. These ones are able to send and to receive data through several channels including memory buffers, I/O interfaces and network sockets. In the area of access control, the improper implementation and configuration of authorization and authentication mechanisms can generate several vulnerabilities such as bad privilege assignment (configuration) or weak authentication [33,[START_REF] Jaeger | Analyzing Integrity Protection in the SELinux Example Policy[END_REF]. While virtualization models exploiting regular system architectures have to carry multiple applications and base utilities with their own interfaces, unikernel models only expose the interfaces of the application it supports the execution of. Consequently, the access control vulnerability may be considered as less critical than in other virtualization models. Software interfaces may be used to corrupt the execution of a program, when it does not proceed to the necessary checks on the input data. This can lead to the exploitation of inconsistencies in data structures [35,36], non-controlled string format exploitation [START_REF] Scut | Exploiting Format String Vulnerabilities[END_REF] and, more directly, code injection [38,[START_REF] Martin | Code-injection Vulnerabilities in Web Applications -Exemplified at Cross-site Scripting[END_REF]. When an interface can be accessed by several programs at a time, synchronization mechanisms are mandatory. If not, this interface may be concerned by concurrency vulnerabilities [38,[START_REF] Netzer | What Are Race Conditions?: Some Issues and Formalizations[END_REF]. As all virtualization models support multi-threading in their virtualized environment, they are all equally affected by these vulnerabilities.

VM Guest OS

The vulnerabilities of the VM guest OS concern the software management (including dependency solving, service degradation and configuration issues) and the OS kernel oversight (including kernel criticality, specific security mechanisms, access to user space, and hardware exposure).

Software Management. When provisioning a virtual machine to prepare a service, the application is installed over its execution environment, and may be upgraded to a new version to apply security fixes, and eventually benefits from newly available features. In practice, such software management is performed by a package manager (e.g. APT [START_REF]Apt -Debian Wiki[END_REF]) or by a dedicated installer (E.g. 0install [42]) especially shaped for dedicated appliances. Nevertheless, solving dependency may generate vulnerabilities [START_REF] Cosmo | Package Upgrades in FOSS Distributions: Details and Challenges[END_REF]. This particularly affects virtualization models supporting legacy system architectures. Software management can be performed at several points (e.g. base utilities, runtime, application itself) and imply multiple providers, increasing the vulnerable surface. On the opposite side, the software management in unikernels uses a pool of software components provided by the unikernel itself, and is performed only at build time [START_REF] Madhavapeddy | Unikernels: Library Operating Systems for the Cloud[END_REF]. Besides, software management processes may lead to the degradation of services, as the upgrade usually requires to restart and reload configurations [START_REF] Crameri | Staged Deployment in Mirage, an Integrated Software Upgrade Testing and Distribution System[END_REF]. This can be exploited to make the service unavailable during a certain time period. In all the considered virtualization models, this vulnerability can be mitigated by performing upgrades on virtual environments that are not in production. This mitigation is usual for unikernels, as software management is performed ex-situ. Finally, software management may also induce configuration vulnerabilities. This may be due to packages whose initial configuration is too permissive, as shown by [START_REF] Heyens | MongoDB Databases at Risk[END_REF]. It may also relate to the update of configuration files during upgrades, with inconsistencies related to the new version of an application. Type-I and type-II virtualization models are the most affected due to the complexity of the architecture, while OS-level virtualization benefits from the absence of an OS kernel in containers, and unikernels are protected by their inherent constrained nature.

OS Kernel Oversight. The OS kernel is a software running in privileged mode in charge of basic system resource management. The criticality of the kernel for a system contributes to vulnerabilities with respect to tasks that it supports. These vulnerabilities are emphasized on monolithic kernel architectures (such as Linux), while they are further limited in micro-kernel architectures [START_REF] Liedtke | On Micro-kernel Construction[END_REF]. These vulnerabilities are bounded to the resilience of OS kernels and their isolation with the applications. OS-level containers are out of the scope, as they do not embed any OS kernels. Type-I and type-II virtualization models are the least affected, as the OS kernel exploits the system architecture features (i.e. privilege levels) to isolate itself from applications. On the opposite, unikernels do not propose strong barriers to isolate hardware management routines from the application ones. The OS kernel carries its own design principles, which may lead to the inapplicability of security mechanisms from the OS kernel to protect applications. For instance in Linux, process address space isolation is a non-sense in kernel space, as the notion of process is not defined there [START_REF] Bovet | Understanding the Linux Kernel: from I/O ports to Process Management[END_REF]. Unikernel is one known exception, as its design is based on a common framework for both the development of hardware resource management and applications [START_REF] Madhavapeddy | Unikernels: Rise of the Virtual Library Operating System[END_REF]. Therefore, this vulnerability affects most significantly the type-I and type-II hypervisor virtualization models, while neither the OS-level nor the unikernel virtualization models are concerned. The access to user space is another source of vulnerabilities.

For instance, the Linux kernel contains routines to read or write memory allocated to applications without restriction, (copy_from_user() and copy_to_user()), enabling it to intervene with no control [START_REF] Bovet | Understanding the Linux Kernel: from I/O ports to Process Management[END_REF]. The kernel has thus access to the application internal structures supporting both the application data and code. Finally, the OS kernel has an unrestricted access to hardware resources, since it runs in privileged mode. These resources can be either physical or virtual. The hardware exposure constitutes another vulnerability. All the virtualization models (except the OS-level one) are concerned. Unikernels are packed only with the hardware management routines that are necessary to the proper application operation, reducing potential risks.

Hypervisor

The hypervisor is subject to several vulnerabilities related to inter-VM crosstalks (such as co-residence, common networking infrastructure, and other resource sharing issues), VM-hypervisor crosstalks (such as resource sharing with the host, implementation of virtualization, and hypervisor oversight issues) and the management console (such as hypervisor oversight and non-linear VM execution issues).

Inter-VM Crosstalks. By controlling VM access to physical hardware, the hypervisor enforces inter-VM isolation.

Each VM is thus not able to access the resource, or communicate with another one, except if it is tolerated by the hypervisor. This isolation is especially challenged, when several VMs are sharing the same hypervisor on the same hardware. This situation is referred as VM coresidence [START_REF] Ristenpart | Get off of My Cloud: Exploring Information Leakage in Thirdparty Compute Clouds[END_REF]. Indeed, VMs rely on the same resources (CPU, memory, I/O, networking interfaces), and the same hypervisor. Compromising one of these shared resources paves the way for creating a hidden channel mitigating the hypervisor isolation, as stated by [START_REF] Bazm | Side-channels Beyond the Cloud Edge: New Isolation Threats and Solutions[END_REF]. This vulnerability affects the virtualization models in different manner. VMs with type-I hypervisor, type-II hypervisor and unikernels rely on a virtual hardware environment to interact with each other. These interfaces are specifically provisioned to each VM to let the hypervisor enforce the isolation. Unikernels restrict the interfaces, by supporting the least necessary virtual hardware resources for the unikernel execution. On the contrary, co-located containers in OS-level virtualization are part of a common host OS, that provides a wider pool of shared resources. This leaves them more vulnerable to these isolation issues. In the area of networking infrastructures, the hypervisor may provide a networking feature to support communications amongst hosted virtual environments and external resources. It can either rely on a virtualized network sustained only by the hypervisor or by an external program (e.g OpenVSwitch [50]). The networking configuration has to enforce the isolation amongst resources to only authorized communications. A misconfiguration leads to potential vulnerabilities enabling to bypass the isolation. Hypervisors may also feature other resource sharing, whose misconfiguration enables communications amongst VMs. The most obvious one is persistent storage sharing amongst several VMs (volume). Several VMs may be an instance of a common image issued from a registry [START_REF] Tak | Understanding Security Implications of Using Containers in the Cloud[END_REF]. The tampering of this image can compromise the related allocated VMs. This vulnerability affects equally all the considered virtualization models.

VM-Hypervisor Crosstalks. The hypervisor controls the execution of VMs, while interfacing between them and the host OS. It enforces the isolation between the host system supporting the hypervisor and the VMs. Hypervisors and VMs communicate through interfaces. The virtual hardware environment is composed of virtualized hardware resources exposed to the VM and serving as a communication medium with the hypervisor. These resources may be subject to vulnerabilities [52,[START_REF] Matousek | Don't Get Bitten[END_REF][START_REF] Kortchinsky | Black Hat USA[END_REF]. Hypervisors may also provide private communication channels (with their VMs), that may not enforce proper security checks [55]. Vulnerabilities may also be due to the implementation of virtualization methods. From a software engineering perspective, hypervisor routines can be flawed, and carry software vulnerabilities [56]. It may also affect their implementations [START_REF] Quist | Detecting the Presence of Virtual Machines Using the Local Data Table[END_REF]. System virtualization may exploit hardware mechanisms that can mitigate the effects of software flaws. This contributes to make them a bit more secured.

Finally, the hypervisor controls the execution of VMs, and has a holistic view over the usage of virtualized resources. The oversight of hypervisors can be seen as a potential vulnerability, as they can access and modify the exposed virtual hardware resources [START_REF] Nance | Virtual Machine Introspection: Observation or Interference[END_REF]. Models related to system virtualization are less prone to these vulnerabilities. OSlevel virtualization exposes the whole system call interface of the host OS kernel to containers, jeopardizing its isolation.

Management Console. For configuration and monitoring purposes, each hypervisor proposes an interface for handling it and the hosted VMs. The management console can be used by administrators or other entities, such as cloud orchestrators.The first vulnerability relates to the oversight based on the management console. The users of the management console have a complete control over VM life-cycle, and can modify the configuration of the virtual hardware environment. This control can be performed independently from what the software in VMs can support, affecting its execution. Users are also able to build new and unsecured communication channels [START_REF] Li | Secure Virtual Machine Execution under an Untrusted Management OS[END_REF]. All virtualization models are equally affected by this vulnerability, as each one proposes a management console. This control also permits to affect the monotonicity of VM executions, as shown in [START_REF] Pearce | Virtualization: Issues, Security Threats, and Solutions[END_REF][START_REF] Garfinkel | A Virtual Machine Introspection Based Architecture for Intrusion Detection[END_REF]. For instance, performing a suspend-and-resume operation against a VM enables potential time and replay attacks. All virtualization models are equally affected by this vulnerability. These features are quite common in hypervisors, but may also be performed over container-based solutions.

Execution Environment of the Hypervisor

The execution environment (host OS or hardware) of hypervisors is also concerned by different vulnerabilities related to the host OS and the hardware.

Host OS. Hypervisors supporting the execution of VMs have to rely on an OS to access physical hardware resources. This OS can be an internal part of the hypervisor (e.g VMware ESXI), be a VM itself controlled by the hypervisor (e.g. Dom0 in Xen), or be out of the control of the hypervisor (e.g. Oracle Virtualbox). The hypervisor is dependent of the behavior of this OS. The OS-level virtualization is also affected by these vulnerabilities, as it is an application supported by a host OS. We consider that host OS vulnerabilities are the same than the ones detailed in Section 3.1. These vulnerabilities are predominant in type-II hypervisor and OS-level virtualization models, as host OSes are necessarily present. Type-I hypervisor and unikernel virtualization models are less affected, as the OSes can be managed and combined in the hypervisor.

Hardware. As nested virtualization is considered as outof-scope of this analysis, we consider cases where the hypervisor (and eventually the host OS) are running on a bare-metal hardware. The hardware layer is affected by vulnerabilities, independently from the considered virtualization model. The hypervisor is a software component requiring a hardware infrastructure to operate. This hardware has an oversight above the running hypervisor, the running VMs and the current communications of the virtual interfaces exploiting the physical one. The persistent storage of the hardware infrastructures also affects the stopped VMs, and their own storage. This makes hardware an attractive target to compromise VMs. The physical properties of hardware can be exploited as side channel sources [START_REF] Kocher | Differential Power Analysis[END_REF]. Hardware manufacturers embed firmwares with their devices, such as CPU microcodes and extension card firmwares. Some of them do not accept firmware upgrades, because of the design constraints or by the lack of support from manufacturers, leaving security flaws unpatched. Other ones support it, but may require the system to run in a limited mode [START_REF] Wojtczuk | Attacking Intel Trusted Execution Technology[END_REF]. Finally, the physical access to hardware resources constitutes an important vulnerability, as components may be added or modified in the hardware layer, altering the behavior of underlying hypervisors and VMs.

Considered Threats and Attacks

We exploit the STRIDE threat assessment methodology [START_REF] Torr | Demystifying the Threat Modeling Process[END_REF] to analyze the different threats and attacks related to our reference architecture. In accordance with it, we consider a set of six main threats, namely spoofing, tampering, repudiation, information disclosure, denial of service and elevation of privileges. In that context, we assume several hypotheses regarding this architecture and the attacker:

• The instantiated virtual machine applications expose interfaces that are accessible remotely.

• The attacker is located remotely, and can use the legitimately exposed interfaces.

• The hardware layer is not accessible to the attacker. We consider physical intrusion attacks out of the scope of system virtualization.

• The assets targeted by the attacker are applications and data located in virtual machines.

• The objective of the attacker is to negatively impact the availability (denial of service), the integrity (alteration) or the confidentiality (retrieving) of an asset.

• A component is considered secured if not compromised, but which may contain inherent software flaws known by the attacker. The attacker can exploit them to compromise a component, and gain influence or control over it.

We classify attacks with regard to threats, according to the notion of compromises: we consider a component as compromised, when the attacker is capable of executing an arbitrary code. We introduce a three-level classification based on the extension of [START_REF] Sahoo | Virtualization: A Survey on Concepts, Taxonomy and Associated Security Issues[END_REF]: compromise-free attacks (that are not related to compromises), compromising attacks (that enable compromises), and compromise-based attacks (that are based on compromises). Fig. 4 provides an overview of this classification, while Table 2 details the relationships with the threats with regard to the reference architecture.

Compromise-Free Attacks

We first detail compromise-free attacks, attacks that do not aim at or do not require the compromise of a component of the architecture. As targets of these attacks, we mainly focus on virtual machines and hypervisors. However, the execution environment of hypervisors might also be considered to some extent, as an objective for an attacker.

Virtual Machine As a Target

We consider in this category VM denial of service attacks, which consist in blocking or disrupting the operation of a virtual machine. These attacks do not aim at gaining control over the resource nor exfiltrating data, but only affect its availability. Such an attack can be performed from the hypervisor (exploiting the management console oversight vulnerability), by using the management console (or a service controlling it) to shutdown the targeted virtual machine, as detailed in [START_REF] Li | Secure Virtual Machine Execution under an Untrusted Management OS[END_REF]. A more discrete manner to proceed such a denial of service is to reconfigure the virtual hardware environment to cause the VM dysfunctioning (e.g. reducing allocated RAM) to reduce the footprint of the attack. The network may also be used to perform such attacks. Based on the hardware exposure vulnerability of guest and host OS kernels, a massive workload from the network may make the VM applications and the VM OS kernels unavailable. This can typically consist in a distributed denial of service attack. Software flaws are also sources of software failures (e.g. [64] related to a given appliance) and can be exploited to perform denial of service attacks. VM appliances may be attacked using their intrinsic flaws, or the ones of their dependencies. These attacks exploit mostly the memory management and concurrency vulnerabilities, but might also rely on software interface vulnerabilities. The OS kernel of the VM may also be used for that respect (kernel criticality vulnerabilities). According to [START_REF] Gu | Characterization of Linux Kernel Behavior under Errors[END_REF], linux kernel faults are mainly due to unsecure software development.

We then focus on attacks exploiting software flawed design. Software applications may carry flaws in their design, but not specifically related to their code. These attacks can be performed without requiring the execution of arbitrary code, and thus, without requiring a compromise. They relates to the tampering threat of in-VM applications, their runtime, the base utilities and the in-VM OS kernel. For instance, misconfigurations are typically exploited to perform such attacks, as detailed in [START_REF] Lin | A Study and Implementation of Vulnerability Assessment and Misconfiguration Detection[END_REF][START_REF] Barrère | A SAT-based Autonomous Strategy for Security Vulnerability Management[END_REF]. They may be the consequence of initial inadequate settings, or changes due to software upgrades or manual administration. The impact of such attacks can be high, when they relate to authorization and authentication rules (access control vulnerabilities). Software management may also be exploited for that purpose. Outdated dependencies, required by some VM appliances, can also introduce vulnerabilities. The upgrade process can also lead to man-in-the-middle (MITM) attacks during the collection of packages, as pointed out in [START_REF] Luettmann | Man-in-the-middle attacks on auto-updating software[END_REF][START_REF] Cappos | A Look in the Mirror: Attacks on Package Managers[END_REF]. In particular, it is possible to prevent the execution of software updates, including security patches. These attacks relate to the dependency solving error vulnerabilities and the memory management vulnerabilities. Finally, attacks may also be based on the non-linear/non-monotonic execution vulnerabilities. As exposed in [START_REF] Pearce | Virtualization: Issues, Security Threats, and Solutions[END_REF], an attack might restore a VM to a vulnerable state, by using the hypervisor facilities.

Hypervisor as a Target

The compromise-free attacks also concern denials of service related to hypervisors, similar to the previous ones:

• The management console represents an attack vector, since it can shut down the hypervisor (management console oversight vulnerability).

• Network protocols can be used to flood the hypervisor and/or its execution environment based on DDoS attacks.

• Hypervisors are also composed of software components, having their own vulnerabilities that are exploitable by attackers.

We can also notice that VMs can indirectly contribute to hypervisor denials of service. For instance, authors of [START_REF] Pearce | Virtualization: Issues, Security Threats, and Solutions[END_REF] pointed out VM sprawling flaws, enabling the rapid spread of misbehaving VMs. This leads to the exhaustion of the hypervisor resources.

Compromising Attacks

The second main category of our classification is compromising attacks. This compromise often serves as a basis for more elaborated attacks. We analyze here attacks leading to the compromise of components of the reference virtualization architecture. We distinguish attacks affecting the virtual machines, the hypervisor and its execution environment.

Virtual Machines

The attacks compromising virtual machines first include software exploitation attacks, that consist in forcing an application in a VM to execute an arbitrary code. As the application runs in an unprivileged mode, the injected payload cannot rely on privileged instruction sets. This affects in-VM applications, their runtime and base utilities. Considered actions include the tampering of resources, the privilege elevation of an attacker, and the disclosure of information. This attack may exploit several vulnerabilities. In addition to memory management vulnerabilities [START_REF] Pincus | Beyond Stack Smashing: Recent Advances in Exploiting Buffer Overruns[END_REF], the code injection in software interfaces may also be used for the insertion of payloads. The dependency solving error [START_REF] Abate | A Modular Package Manager Architecture[END_REF] in software management may also prevent software upgrades, in order to maintain security flaws. Compromised OS kernels can also contribute to such attacks (access to userspace vulnerabilities). Authors of [START_REF] Checkoway | Iago Attacks: Why the System Call API is a Bad Untrusted RPC Interface[END_REF] show that applications can be subverted by an OS kernel based on forged returns from system calls. Other flaws, such as misconfigurations and additive dependencies, may be used from unprotected interfaces. The works in [START_REF] Luettmann | Man-in-the-middle attacks on auto-updating software[END_REF][START_REF] Cappos | A Look in the Mirror: Attacks on Package Managers[END_REF] also investigate a MITM attack against the software management system, with the objective of inserting or maintain-ing unprotected interfaces on applications, accessible by the attacker.

We can also consider OS kernel exploitation attacks aiming at the execution of an arbitrary code by the kernel. These attacks benefit from the privileged execution mode, which extends the attack surface. A concrete example of such an attack is the use of rootkits [START_REF] Op | [END_REF], enabling an attacker to gain the whole control over a system. The related threats include tampering, privilege elevation, and disclosure of information. A compromised application in the user space may be sufficient, but not necessary for performing these attacks. Moreover, modular OS kernels increase the risks, as the process loading modules constitutes an additional payload vector.

Finally, we can point out hyperjacking / hypervisor injection attacks. A typically example is the VMbased rootkit (VMBR) attack, which permits to integrate a malicious hypervisor at the bottom side of the operating system [START_REF] Carbone | Taming Virtualization[END_REF]. Its introspection capability enables it to overpass any security mechanisms [START_REF] Pearce | Virtualization: Issues, Security Threats, and Solutions[END_REF]. King et al. propose in [START_REF] King | SubVirt: implementing malware with virtual machines[END_REF] a persistent VMBRs targeting both Windows OS and Linux, and hosting malicious services undetectable from these OSes. The blue pill attack, described in [START_REF] Carbone | Taming Virtualization[END_REF], is a VMBR attack capable of infecting a host without any reboot. The VMBR attack takes advantage from the hypervisor oversight vulnerability to get introspection capabilities, and to be undetectable by OS-level detection mechanisms.

Hypervisor

The compromising attacks may also target the hypervisor of the virtualization architecture. Amongst these attacks, we can highlight hypervisor direct exploitation attacks [START_REF] Horn | Exploiting the Xen hypervisor[END_REF]. Even if they are not expected to be exposed as much as their VM appliances, hypervisors remain sensitive to these direct attacks. The threats related to these attacks affect the hypervisor itself, by tampering it, repudiating the traceability of its behavior, disclosing information of its configuration and the VM configuration, and elevating the privilege of a VM user to those of the hypervisor administrator. For instance, the work in [START_REF] Wojtczuk | Subverting the Xen hypervisor[END_REF] illustrates the Xen hypervisor (and the privileged domain) compromise by a rootkit using a weakness in the DMA implementation and the Xen debug register. [START_REF] Ormandy | An Empirical Study into the Security Exposure to Hosts of Hostile Virtualized Environments[END_REF] also identifies several common attacks against hypervisors, by considering the case of local attackers. Hypervisor extension mechanisms (extension packs, module framework) are also critical components for the hypervisor, as they are capable of loading code, such as a malicious payload, in its instance. Such attack is described in [START_REF] Ormandy | An Empirical Study into the Security Exposure to Hosts of Hostile Virtualized Environments[END_REF], where the Xen loadable module framework permits to load arbitrary code in the Xen address space. These attacks take advantage of the resource sharing with host and virtualization implementation vulnerabilities.

Another important compromising attacks correspond to command/control channel exploitation attacks. The command/control channel is a privileged communication medium between the hypervisor and a VM, as depicted in [START_REF] Waldspurger | Memory Resource Management in VMware ESX Server[END_REF]. This attacks targets the hypervisor by using this channel as a medium to compromise it. The work in [START_REF] Carpenter | Hiding Virtualization from Attackers and Malware[END_REF] investigates how to use the command channel testing feature, in order to recognize a virtual machine environment.These attacks may typically be related to the resource sharing with host vulnerability.

Execution Environment of the Hypervisor

Compromising attacks may also include firmware exploitation attacks against the execution environment of the hypervisor. Hypervisors are running at the top of a hardware layer, except in the case of nested virtualization. They are therefore constrained by the hardware layer, which is composed of devices with their own firmwares. These firmwares may carry their own flaws, providing the necessary material to an attacker to compromise them. The corresponding threats are the hardware tampering, the information disclosure based on side channels, and the elevation of privileges to get control on the software components running over the hardware. For instance, such an attack is showcased in [START_REF] Wojtczuk | Following the White Rabbit: Software attacks against Intel VT-d technology[END_REF], in order to compromise the firmware of network interface cards. These attacks are emphasized by the hard constraints regarding the upgrade procedure of these firmwares. Such upgrades are typically limited by the degradation of services during the upgrade process, or by the lack of support from manufacturers. These attacks are mainly based on the hardware oversight and hardware upgradability vulnerabilities.

Compromise-Based Attacks

The last category of attacks are compromise-based attacks. The two first categories (compromise-free and compromising attacks) are often the first steps towards establishing a more sophisticated attacks targeting a virtualization architecture. Consequently, we consider that compromise-based attacks require the prior compromise of a component to be performed. These attacks typically include the case of malicious virtual machines attacking the remainder of the virtualization architecture, and the case of a malicious hypervisor trying to snoop on the virtual machines it is in charge of.

Malicious Virtual Machines

The first category of attacks relies on malicious virtual machines. An hypervisor hosts several VMs, which share the same physical resources. They can be competing to access these resources. The hypervisor resource monopolization attacks consist in a malicious VM taking control on an hypervisor to gain an exclusive access to these resources. This leads to the violation of the hypervisor resource isolation. These attacks on physical CPU are described in [START_REF] Zhou | Scheduler Vulnerabilities and Coordinated Attacks in Cloud Computing[END_REF]. Xen scheduler vulnerability is used by a malicious virtual machine to steal compute cycles to other co-located machines. They typically exploit the co-residence and the virtualization method implementation vulnerabilities.

The VM hopping [START_REF] Jasti | Security in multitenancy cloud[END_REF] attacks consist in a malicious VM that directly targets another VM from the virtualization environment. The related threat concerns the VM applications, their runtime and utilities, and their OS kernel. They permit the tampering and the elevation of privilege in the virtualization architecture. These attacks can typically use common networking infrastructure and other resource sharing vulnerabilities to access the targeted VM. They can exploit software interfaces, configuration and hardware exposure vulnerabilities to perform a compromise.

The attacks regarding the VM monitoring from a VM consist in collecting information about a VM, without compromising it. They can typically rely on a passive monitoring of another virtual machine. The related threats are about VM applications, their runtime and their OS kernel, through the disclosure of information and the breaching of the non-repudiation principle. These attacks are mainly conditioned by the exploitation of side channels, and the co-residence of VMs. The work in [START_REF] Barresi | CAIN: Silently Breaking ASLR in the Cloud[END_REF] illustrates the by-passing of an OS protection based on side channels. The work in [START_REF] Xiao | One Bit Flips, One Cloud Flops: Cross-vm Row Hammer Attacks and Privilege Escalation[END_REF] makes use of co-residence and physical properties to affect memory pages owned by other VMs, to proceed to in-memory information leakage or even to build a hidden channel between both VMs. The co-residence issue is especially challenging in a public cloud infrastructure. The work in [START_REF] Ristenpart | Get off of My Cloud: Exploring Information Leakage in Thirdparty Compute Clouds[END_REF] details the steps to reach a VM co-residence with the targeted VM. Finally, authors of [START_REF] Zhang | Cross-VM Side Channels and Their Use to Extract Private Keys[END_REF] explore how co-residence can contribute to cryptography key leaks based on CPU L1-cache. These attacks are based on the hardware physical property, the virtualization method implementation and the inter-VM crosstalks vulnerabilities.

The VM escape to a VM attack [START_REF] Pearce | Virtualization: Issues, Security Threats, and Solutions[END_REF] aims at compromising the hypervisor, in order to access another VM. It is very similar to a VM hopping, and corresponds to the same threats, but relies on the hypervisor compromise to break the isolation. VM-hypervisor crosstalks vulnerabilities are typically involved in these attacks. The VM escape to an host relies on the hypervisor compromise from the malicious VM, in order to gain further control. In most of the cases, these attacks target legitimate interfaces between the hypervisor and the malicious VM to compromise the hypervisor, such as the VENOM [START_REF] Geffner | VENOM Vulnerability[END_REF] and the Cloudburst attack [START_REF] Kortchinsky | Black Hat USA[END_REF]. These attacks take advantage of the VM-hypervisor crosstalks vulnerabilities.

Malicious Hypervisor

The second category of attacks relies on a malicious hypervisor. The VM introspection / monitoring attack from the host exploits a malicious hypervisor to analyse the behavior of a VM and to infer its internal state. Different introspection techniques are presented in [START_REF] Nance | Virtual Machine Introspection: Observation or Interference[END_REF] to track the activities of a virtual machine. For instance, [START_REF] Li | Secure Virtual Machine Execution under an Untrusted Management OS[END_REF] demonstrates an approach for extracting a secret key from a memory dump. Authors of [START_REF] Morbitzer | SEVered: Subverting AMD's Virtual Machine Encryption[END_REF] propose a strategy to extract secrets from a AMD SEV-protected VM: a mali-cious hypervisor manipulating the mapping between guest physical memory and the host physical memory enables a malicious distant client to obtain secret data. A shown in [START_REF] Li | Secure Virtual Machine Execution under an Untrusted Management OS[END_REF], the management console can also serve as a support to perform such a malicious monitoring. These attacks are typically based on resource sharing with the host and hypervisor oversight vulnerabilities.

The inter-VM communication introspection attacks may also provide interesting informations regading VM communications with other hosted VMs or hosts, through I/O or networking subsystems that are handled by the hypervisor. This raises privacy issues related to communication interception and introspection. The threat affects the OS kernel, the runtime environment and the applications in VMs, causing potential disclosure of informations. The hypervisor oversight and resource sharing with the host vulnerabilities contribute to these attacks. The VM mobility attacks consists in an attacker using the export feature of the hypervisor to obtain the VM storage device, the virtual hardware environment configuration, and the current state of the memory and vCPU related to a VM. They can typically be based on the management console vulnerabilities.

Table 3 describes the relationships among the vulnerabilities of the reference architecture and the attacks that they leverage.

Counter-Measures and Recommendations

After having classified security attacks, we propose different counter-measures and recommendations with regard to the reference architecture. Therefor, we consider two major requirements for counter-measures. To limit their cost on resource addressing protection, these ones should have a minimal impact on the operability of protected resources. Moreover, to ensure that counter-measures contributes to the protection effort, they should not impact on the security benefits brought by the virtualization itself.We provide a threefold classification for these countermeasures. Fig. 5 describes this classification, while Table 4 exposes the benefits and limits of identified countermeasures in terms of coverage, requirements and costs. Table 5 specifies the relationships among threats and countermeasures/recommendations. This classification includes counter-measures consisting in integrating security mechanisms at the design of resources, counter-measures aiming at reducing the attack surface of resources, and countermeasures enabling a higher adaptation through security programmability.

Integration of security mechanisms at design time

A first category of counter-measures consists of addressing the protection of resources (or components) at design time through the integration of security mechanisms. This may concern both the virtual machines and the hypervisor.

Protection of virtual machines

The protection of virtual machine resources can be performed at various levels. Counter-measures can be considered at the OS kernel. For instance, the address space layout randomization (ASLR) method permits to prevent the exploitation of memory management vulnerabilities [START_REF] Shacham | On the Effectiveness of Address-space Randomization[END_REF]. From an architectural viewpoint, monolithic OS kernels may facilitate security attacks due to the lack of isolation amongst kernel subsystems. PerspicuOS [START_REF] Dautenhahn | Nested Kernel: An Operating System Architecture for Intra-Kernel Privilege Separation[END_REF] addresses this issue, by fragmenting the OS kernel code with an isolation of privileges. The nested kernel is the OS kernel subpart whose memory access is privileged, while the outer kernel corresponds to subsystems relying on inner kernel API for memory access. These APIs virtualize the MMU (Memory Management Unit) to enforce protection on the outer kernel memory access. However, the ability to enforce kernel-based counter-measure is subjected either to the existence of relevant implementations in the OS kernel, or the ability of the latter to be extended. This statement jeopardize the security of operating system restricting the access to their programming interfaces. The isolation of the OS kernel components argues in favor of unikernels, since the minimalism of their architecture leads to the dispatching of hardware management routines accross several unikernel VMs.

The counter-measures also concern the applications of virtual machines, as they represent a major entry point. The variety of applications and runtime environments to be considered in that context overpasses the scope of our analysis. Without constrains on the runtime environment, this often supposes specialized security mechanisms adapted to each applications. This comes with the burden to secure a more important code base. In the meantime, the protection of applications from untrusted execution environments can be addressed generically. OS-based solutions such as the XOMOS OS [START_REF] Lie | Implementing an Untrusted Operating System on Trusted Hardware[END_REF] are part of the solution to this issue. Shielding the application execution is also an interesting approach to protect applications from a malicious OS. For instance, Haven [START_REF] Baumann | Shielding Applications from an Untrusted Cloud with Haven[END_REF] exploits the Intel SGX technology to provide a protective layer against OS-based and physical attacks. However, both of them comes with heavy requirements on the the hardware operating the application. Therefore, constraining the application runtime is a reasonable trade off to enable a extensive protection on application with a limited requirements on the platform. The usage of unikernels constitutes a solution to have a minimal code base and to reduce the attack surface. In addition, as the runtime environments are constrained by the unikernel framework, this restricts the complexity of their support from a security perspective.

From a software management viewpoint, package managers may be secured and may contribute to prevent attacks on virtual machines. Approaches such as [START_REF] Cappos | A Look in the Mirror: Attacks on Package Managers[END_REF] introduce package signature mechanisms to deal with manin-the-middle attacks, as well as package alterations. Complementarily, [START_REF] Abate | A Modular Package Manager Architecture[END_REF] exploits solving techniques for the satisfiability problem (SAT) to cope with dependency solving issues. In the case of unikernels, the software management is performed outside the virtual machines, reducing the consequences of its flaws to the supported appliance. But, these counter-measures are firmly bound to the usage of a package manager implementing these countermeasures.Any software management operation made outside this framework put at stake the security of the system.

Protection of the hypervisor

The hypervisor provides an execution environment to the VMs, enabling them to run in accordance with the VMM properties [START_REF] Popek | Formal Requirements for Virtualizable Third Generation Architectures[END_REF]. In that context, [START_REF] Szefer | A Case for Hardware Protection of Guest VMs from Compromised Hypervisors in Cloud Computing[END_REF] proposes the use of hardware to protect VMs from the host, by enforcing isolation through hardware resource access management and privacy based on a trusted platform module (TPM). However, this counter-measure supposes important prerequisites on the hardware architecture, which may not be considered in practice. Additionally, [START_REF] Li | Secure Virtual Machine Execution under an Untrusted Management OS[END_REF] introduces a method to protect the hypervisor from host VMs by filtering management hypercalls and checking their integrity at the invocation. Virtual CPU context integrity checks as well as virtual memory encryption are enforced during the execution of the hypervisor. The overhead affecting the management operation can be more remarkable on VMs with a short lifespan. In the area of OS-level virtualization, the Intel safeguard extension is used by SCONE [START_REF] Arnautov | SCONE: Secure Linux Containers with Intel SGX[END_REF] to protect containers against untrusted execution environments. This framework provides the necessary building blocks to design enclaved linux containers, that are re- silient against tampering attacks from the OS, with a limited trusted computing base (TCB). The system calls are performed asynchronously with the assistance of a module outside the enclave, and therefor requires a modification of the host OS. Complementary, the exploitation of command channels can be avoided by obfuscation methods, as detailed in [START_REF] Carpenter | Hiding Virtualization from Attackers and Malware[END_REF], but this mitigation degrades the functionality of the hypervisor. [START_REF] Zhou | Scheduler Vulnerabilities and Coordinated Attacks in Cloud Computing[END_REF] also describes a solution to CPU monopolization attacks, by switching virtual CPU scheduling to alternative methods (e.g. exact scheduler, Bernouilli scheduler and uniform scheduler) to limit flaw exploitations. This counter-measure can heavily impact VM usage, and the scheduling method should be assessed to meet operationnal constrains (e.g. service level agreement).

The granularity of the hypervisor architecture also impacts on security. In order to avoid a monolithic architecture, [START_REF] Colp | Breaking Up is Hard to Do: Security and Functionality in a Commodity Hypervisor[END_REF] uses hypervisor virtualization features to decompose management OS capabilities across dedicated service VMs. These VMs are framed by coercitive security constraints, such as hypercall restrictions, security audits and frequent reboots. However, this approach only focuses on the management console segmentation, and does not address the internal subsystems of the hypervisor. The NOVA [START_REF] Steinberg | NOVA: A Microhypervisor-based Secure Virtualization Architecture[END_REF] hypervisor goes further, by proposing a microhypervisor architecture. A minimum trusted computing base (TCB) resides on the host kernel (micro-hypervisor ), while each virtual machine dedicated VMMs are executed in the user space, meaning host device drivers and remaining services. The lack of maturity of this approach introduces limitations regarding guest OS compatibities.

It is also important to cope with networking and storage security. In that context, the enforcement of access control security policies on VM resources is explored by the sHype hypervisor [START_REF] Sailer | Building a MAC-based security architecture for the Xen open-source hypervisor[END_REF]. This solution, based on Xen, enables the enforcement of policies related to the mandatory access control (MAC) on VM resource access, including network communications, standard I/O communications and shared memory. More precisely, the reference monitor enforces the policy on event channel operations, shared memory requesting and domain management operations. The presented solution is however limited to access control enforcement with the MAC model on a single hypervisor instance. TVDSEC [START_REF] Tupakula | TVDSEC: Trusted Virtual Domain Security[END_REF] addresses the access control policy enforcement over several hypervisor instances. In return, only the enforcement over control flows is considered. Networking may also serve for performing resource quarantines. For instance, the NICE framework [START_REF] Chung | NICE: Network Intrusion Detection and Countermeasure Selection in Virtual Network Systems[END_REF] exploits a network controller to put in a quarantine state VMs that are potentially compromised. This framework is however only compatible with Openflow-configured networks.

The integration of security mechanisms at design time is an important topic that induces several research challenges in terms of heterogeneity and distribution. The identified counter-measures clearly highlighted this heterogeneity of resources requiring security mechanisms and the interfaces to be used for their configuration. While the light-weight architecture of unikernels may facilitate the integration of security mechanisms, we believe that other virtualization models could benefits of on-going research efforts in the system management area. In particular, the research efforts currently done in building and developing Infrastructure-as-Code solutions [START_REF] Artac | DevOps: Introducing Infrastructure-as-Code[END_REF] could contribute to the design of security-constrained applications with complex runtimes in other virtualization models. In addition, we think that orchestration languages, such as the Topology and Orchestration Specification for Cloud Applications (TOSCA) or the OpenStack Heat Orchestration Template (HOT), constitute interesting supports to be extended in order to facilitate this integration at design time. They permit to specify and orchestrate cloud services whose resources may be deployed over several infrastructures, considering potential inter-tenant and intercloud collaborations. The services are described in the form of topologies with a set of resources and their relationships. It is possible to specify orchestration processes related to a service, these ones impacting on the state of resources relationships. Research efforts should be considered to enable the specification of security requirements with different security levels that could be associated to orchestration and instanciation processes, and the extension of these languages for defining and automating the integration of security mechanisms, when elaborating the services.

Minimization of the attack surface

A second category of counter-measures concern the minimization of the attack surface, by verifying the properties and restricting the capabilities of resources. In particular we focus on verification and capability dropping techniques.

Formal verification can be applied to both the OS kernel and the applications of virtual machines. It permits to assess the security properties of the components of the architecture, but also to verify the design of security mechanisms (e.g. cryptographic libraries). For instance, Klein et al. [START_REF] Klein | seL4: Formal Verification of an OS Kernel[END_REF] introduce the design and the implementation of a micro-kernel, SEL4, featuring formal specifications. It relies on the Haskell purely functional programming language. The source code can be automatically translated into a formal specification that is then checked. The Hyperkernel [START_REF] Nelson | Hyperkernel: Push-Button Verification of an OS Kernel[END_REF] project proposes an OS kernel that can be assessed using the Z3 satisfiability modulo theories (SMT) prover. This OS makes use of the hardware-based virtualization feature to enforce an isolation between the kernel residing in root-mode and the process running in non-root mode. These approaches may imply an important cost for modifying/extending the code framed for formal specifications, and often suppose to define a modular architecture to prove the properties of components independently. Alternative approaches have also emerged to protect applications and guarantee security properties, without implying formal checking. Typically, the virtual ghost framework [START_REF] Criswell | Virtual Ghost: Protecting Applications from Hostile Operating Systems[END_REF] permits to protect user applications from an untrusted operating system. It enables the applications to control their own operating system-proof sandbox, and a layer interleaving the OS and the hardware is responsible for enforcing the sandbox isolation. The protection against control flow injections is established at the application compilation time by LLVM intermediate code inspections. Formal verification methods are also applied to check the hypervisors. For instance, work in [START_REF] Freitas | Formal methods for security in the Xenon hypervisor[END_REF] has formally specified parts of an hypervisor based on Xen. But the authors acknowledge that this approach is too expensive to be applied to an entire product.

It is also possible to apply capability dropping techniques. Hardening methods are part of them and permit to reduce the attack surface of a system, by imposing a system to restrict parameter values and use specific (security) software components. Most applications come with their very specific recommendations for hardening. This requires a specific knowledge of the applications to be protected while making configuration with multiple applications and runtimes complex to be extensively handled. The images of resources can also be hardened in such a way that their applications are dedicated to a specific purpose, while considering their lifetime is restricted in accordance. A directory of unikernels is proposed in [START_REF] Madhavapeddy | Jitsu: Just-In-Time Summoning of Unikernels[END_REF], facilitating the usage of constrained and ephemeral virtualized resources. This approach is suited only for virtualized resources with featuring a very short boot time, excluding other virtualized resources than unikernel. In that context, we can also restrict the virtual hardware environment. Embedding unused hardware increases the exposure to attacks. An example of such protection is given in [START_REF] Williams | Unikernel Monitors: Extending Minimalism Outside of the Box[END_REF], where the authors export the virtual hardware environment stack to the VM itself. The solution is based on a modified version of the KVM hypervisor delegating its monitor to the VMs. It increases the isolation between the hypervisor and the VM, and permits a finely tailored virtual hardware environment for VMs. However, this implies important modifications in the code base of the hypervisor, requiring a per-hypervisor engineering, which has not been performed on other hypervisors than KVM.

The outsourcing of in-VM software management contributes also to the code base limitation efforts in virtualized environments. Most operating system images are provided with their software management tools. Therefore, the presence of these tools in the VM images can be questioned, as most of the software management operations are performed before the operation of applications. Delegating the software management to the image manufacturing process contributes to protect VM applications against related attacks and compromises. An example of such outsourcing is given in [START_REF] Cappos | A Look in the Mirror: Attacks on Package Managers[END_REF], in accordance with security criteria. However, this approach supposes that (i) the manufacturing chain of VM image is trusted and (ii) the VM instances cannot receive security updates, leaving long-leaving VM instances vulnerable to new attacks, and arguing in favor of short-living VM instances.

The minimization of the attack surface also poses important research challenges in terms of adaptation to new hardware facilities and in terms of automation for resource (re-)generations. These efforts are of course constrained by the hosting environment providing the applications and by the services to be provided. These constraints define a baseline that is specific to both the infrastructures and the applications to be operated. For instance, the new opportunities offered by hardware acceleration, in particular the exploitation of Graphical Processing Units (GPU) for virtualization, is challenging this minimization. The application has to handle the specificities of GPU processing, leading to more heavy virtual machines. Moreover, new solutions have to be investigated to make sure the hypervisor properly control the access to the GPU, considering the multiple competing and vendors-specific solutions [START_REF] Hong | GPU Virtualization and Scheduling Methods: A Comprehensive Survey[END_REF] to implement GPU virtualization. Another major issue is to automate the generation of virtualized resources with a minimal attack surface. Unikernels constitute a thriving system architectural model that provides the necessary material for a comprehensive design of virtual machines. The limited tooling they carry at runtime imposes to properly build them, before their allocation. New management frameworks and algorithms are required to generate these virtualized resources, including unikernels but not limited to them, in an on-the-fly manner. This automation will provide a better and dynamic adaptation to contextual changes, and may also contribute to moving target defense strategies.

Adaptation based on security programmability

Counter-measures are efficient at reducing the attack surface, but they have to constantly be adapted over time to cope with new threats and attacks. The programmability of resources and their security mechanisms contribute to this required adaptation. It can be driven by an orchestration activity relying on monitoring results.

The monitoring of resources can be performed based on introspection techniques.

For instance, VMwatcher [START_REF] Jiang | Stealthy Malware Detection Through Vmm-based "Out-of-the-box" Semantic View Reconstruction[END_REF] permits to determine the internal state (memory and filesystem) of virtual machines. This enables to outsource some security mechanisms such as in-VM malware detection, but require minor extension of the hypervisor to work. Authors of [START_REF] Christodorescu | Cloud Security Is Not (Just) Virtualization Security: a Short Paper[END_REF] propose similar techniques, but applied from the guest OS during the execution phase. The proposed solution relies on a whitelist to identify authorized applications, making it complex to set up in virtual environments operating multiple applications. Active monitoring approaches, such as the LARES architecture [START_REF] Payne | Lares: An Architecture for Secure Active Monitoring Using Virtualization[END_REF], are also applied to observe virtual machines based on dedicated agents or probes. This approach requires to modify accordingly the monitored resources. The monitoring from hypervisor also enables the detection of compromised OS-kernel. Approaches such as the NICKLE framework [START_REF] Riley | Guest-Transparent Prevention of Kernel Rootkits with VMM-Based Memory Shadowing[END_REF], permit to detect rootkits in the OS kernel by operating a dedicated hypervisor. Finally, the hypervisor introspection is also applicable to VM virtual hardware environments, as developed by Slick [START_REF] Bacs | Slick: An Intrusion Detection System for Virtualized Storage Devices[END_REF] and TVDSEC [START_REF] Tupakula | TVDSEC: Trusted Virtual Domain Security[END_REF]. The first one relies on a modification of the hypervisor (experimented in KVM) to track the activity of VMs on virtual storage, while the second one focuses on tracking VM networking flows. This approach permits to detect compromised VMs only when malicious interactions with external resources are initiated. The monitoring may also consist in the identification of vulnerable configurations related to components of the virtualization architecture. For instance, the approach in [START_REF] Barrère | A SAT-based Autonomous Strategy for Security Vulnerability Management[END_REF] proposes a solving method exploiting the satisfiability problem (SAT) for supporting resource management. It permits to detect the presence of configuration vulnerabilities in a preventive manner, considering the maintenance operations that are available. The database of the the vulnerabilities has to be provided and be adapted to the configurations of inspected VMs. The analysis of monitoring results can then drive the orchestration of security mechanisms in virtualization and cloud environments. An efficient and consistent orchestration is an important aspect to deploy and coordinate security mechanisms. Some efforts, such as [START_REF] Pattaranantakul | A First Step Towards Security Extension for NFV Orchestrator[END_REF], exploit orchestration languages to support access control policies, specifically in the area of virtualized network functions (VNF). Policy mining methods may also contribute to assess the consistency of orchestration activities, by extracting high-level policies from entities enforcing lowlevel ones. Work in [START_REF] Hachana | Mining a high level access control policy in a network with multiple firewalls[END_REF] applies this approach to extract network-wide access control policies from the configuration of multiple distributed firewalls.

Orchestration activities depend on the programmability of resources and of their security mechanisms. To enforce security decisions taken by an orchestrator, the resources of the virtualization architecture can be dynamically reconfigured through programmability facilities. These reconfigurations enable to reduce the attack surface (e.g. setting up authentication) or to mitigate the attacks (e.g. isolating a compromised host through firewalling). Considered resources include the different components of the virtualization architecture, but also security mechanisms. We can distinguish different cases: [START_REF] Mell | The NIST Definition of Cloud Computing[END_REF] components supporting reconfigurations at runtime can be managed through their configuration interfaces, (2) statically-configured components that are restartable, can be modified by changing the static configuration alteration and restarting the instance, (3) components that are non restartable (nor reconfigurable at runtime) can be addressed through dedicated methodologies, such as dynamic software updates developed in [START_REF] Walla | Live Updating in Unikernels[END_REF] using IncludeOS unikernels. Therefore, the orchestration has to acknowledge the resource of the resources to be reconfigured to know which reconfiguration strategy is to be employed. We can also notice some recovery techniques, such as the TASER intrusion recovery system [START_REF] Goel | The Taser Intrusion Recovery System[END_REF], capable of tracking the activity of a virtual machine, and recovering its configuration to a clean state after an attack. Nonetheless, this strategy requires to adapt the virtual machines by inserting a surveillance agent.

Security programmability offers new opportunities to protect resources, but also introduces new research challenges, in particular in the context of the permanentlygrowing Internet-of-Things and the development of services at the edge. In that context, research efforts are required to elaborate new strategies and algorithms taking into account the complementary of endogeneous and exogeneous security mechanisms. Endogeneous mechanisms consist in reducing the exposure to attacks by modifying internally the resources. They consist in hardening the configuration of virtualized resources at different levels (as previously presented), and may be driven by multiple security information data sources, such CVE3 and OVAL4 definitions, in order to prevent the exploitation of vulnerabilities. These mechanisms may not be fully implementable in the case of scarce-resource devices, such as the ones of the Internet-of-Things. Exogeneous mechanisms consist in complementing the resources based on external security functions. The programmability of network resources brings flexibility to the deployment and adaptation of such mechanisms. In particular, it enables to dynamically building and orchestrating chains that are composed of different security functions. These security functions may typically include firewalls, intrusion detection systems, and data leakage prevention mechanisms. They may be provided as middleboxes potentially deployed at the edge, or be directly implemented on the softwaredefined networking layer. Important efforts are required to support an adequate usage of both endogeneous and exogeneous security mechanisms in the case of Internetof-Things devices, this including off-loading management methods as well as verification techniques for checking the consistency of these mechanisms.

Conclusions

System virtualization is both a source of vulnerabilities and an enabler for supporting the protection of cloud infrastructures and services. In this article, we have described different virtualization models, namely virtualization based on type-I hypervisors, virtualization based on type-II hypervisors, OS-level virtualization and unikernel virtualization. We have detailed their design principles as well as their relationships in order to infer a reference architecture, that serves as a support to our security analysis. We have then analyzed the different vulnerabilities that may affect the components of this architecture, and identified related attacks, in view of existing security threats. We have finally highlighted several counter-measures and recommendations with respect to their exploitation for cloud protection. Those counter-measures have been selected with the treats impacting virtualization architect in a cloud infrastructure. These recommendations are synthesized in Fig. 6, in line with different phases related to the life-cycle of cloud resources. Our work has highlighted the insufficiency of architectural counter-measures in hypervisor and virtual machines design. It pleads for a more integrative virtualized resource preparation before their allocation taking into account security requirements. We also argue in favor of the simplification of the system architecture to dwarf their attack surface and alleviate their management burden. Finally, deployed security mechanisms should be actively managed according to the security context of the protected resources and its evolution all along the exploitation. Those recommendations apply to the protection of the multiple assets involved in operation of a cloud service, and can comply with very specific security requirements.

Amongst the analyzed virtualization models, unikernels offer interesting opportunities to reduce the attack surface based on their simplified architecture, and to integrate security mechanisms at an early stage during the building of unikernel images. In addition, the programmability of resources, in phase with the security orchestration, can drive the generation of such unikernel resources in the context of cloud environments. The implementation of this orchestration depends also on the proper specification of security requirements regarding the security features and rules to be considered. Research efforts in the area of policy-based management should take benefits from the orchestration languages and exploit them as a support to integrate security mechanisms at design time. The maturity of technologies and tools related to these virtualization models constitute an important lever to the instantiation of these recommendations. The current efforts on Infrastructureas-Code solutions may facilitate the efficient protection of resources more complex than unikernels. It is important to investigate new strategies for automating the generation of these resources with a proper control of the attack surface, and contributing to a better adaptation to contextual changes. Finally, the development of services at the edge and the integration of Internet of Things (IoT) resources to the Cloud introduces new important challenges in terms of heterogeneity and distribution. In that context, new management and programmability strategies should take into account the complementarity of endogeneous and exogeneous security mechanims to cope with scarce resources.

Figure 1 :

 1 Figure 1: Overview of a regular system architecture.

Figure 2 :

 2 Figure 2: Virtualization reference architecture.

Figure 3 :

 3 Figure 3: Classification of considered vulnerabilities.

Figure 4 :

 4 Figure 4: Classification of attacks.

Figure 5 :

 5 Figure 5: Classification of Countermeasures (or Recommendations).

  ,R,I,D,E S,T,R,I,D,E S,T,R,I,D,E S,T,R,I,D,E S,T,R,I,D,E Orchestration of Security Mechanisms S,T,R,I,D,E S,T,R,I,D,E S,T,R,I,D,E S,T,R,I,D,E S,T,R,I,D,E Security Programmability S,T,R,I,D,E S,T,R,I,D,E S,T,R,I,D,E S,T,R,I,D,E S,T,R,I,D,E Notation: Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service and Elevation of Privilege.

Figure 6 :

 6 Figure 6: Illustrated Synthesis of Recommendations with Respect to Cloud Protection.

  ,

		Type-I Hypervisor	Type-II Hypervisor	OS-level Virtualization	Unikernel Virtualization
	Virtual machine	App Runtime App Base Utilities	App Runtime App Base Utilities	App Runtime	App	Application Hardware Runtime
		OS Kernel	OS Kernel		Base Utilities	Management
	Hypervisor	Exec. Handler	Memory Management	Networking	I/O	Mgmt Console
	OS	Privileged VM /		Base Utilities		Privileged VM /
	Host	In-Hypervisor OS		OS Kernel		In-Hypervisor OS
	Hardware	CPU	RAM		I/O Interfaces	Network Adapter

Table 1 :

 1 Exposure to considered vulnerabilities with respect to virtualization models

			Type-I Hypervisor	Type-II Hypervisor	OS-level Virtualization	Unikernel Virtualization
	Runtime variable type checking
	Memory deallocation
	Kernel inference in userspace
	Development software flaw
	Access control
	Code injection
	Concurrency
	Dependency solving error
	Service degr. during mgmt
	Configuration issue
	Kernel criticality
	Inapplicable sec. mech.
	Access to userspace
	Hardware exposure
	Co-residence
	Shared networking
	Other resource sharing
	Resource sharing with the host
	Implem. of virtualization method
	Hypervisor oversight
	Management console oversight
	Non-linear/non-monotonic VM exec.
	H-OS -Dependency solving error
	H-OS -Service degr. during mgmt
	H-OS -Configuration issue
	H-OS kernel -Kernel criticality
	H-OS kernel -Inapplicable sec. mech.
	H-OS kernel -Access to userspace
	H-OS kernel -Hardware exposition
	Hardware oversight
	Hardware physical property
	Hardware upgradibility
	Hardware physical access
	(Notation:	, ,	stand respectively for low, medium
			and high.)

Table 2 :

 2 Relationships among attacks, leveraged threats and affected components

		Spoofing	Tampering	Repudiation	Info. disclosure Denial of ser-	Privilege eleva-
						vice		tion
					• Application
					Software
			• Software		Exploitation
			Flawed Design		• VM Monitor-ing from VM
	Application	• Hyperjacking • VM Mobility	• Application Software Exploitation • VM Hopping • VM Escape to VM	• VM Monitor-ing from VM • VM Moni-toring from Host	• VM toring from Moni-Host • Inter-VM Com. Intro-spection	• VM Denial of Service	• Application Software Exploitation • VM Hopping • VM Escape to VM
					• Application
					Software
			• Software		Exploitation
			Flawed Design		• VM Monitor-ing from VM
			• Application Software Exploitation	• VM Monitor-ing from VM	• VM toring from moni-Host			• Application Software Exploitation
	Runtime	• Hyperjacking • VM Mobility	• VM Hopping • VM Escape to VM	• VM toring from Moni-Host	• Inter-VM Com. Intro-spection	• VM Denial of Service	• VM Hopping • VM Escape to VM
			• Software Flawed Design		• OS Exploitation Kernel
	OS Kernel	• Hyperjacking • VM Mobility	• OS Exploitation Kernel • VM Hopping • VM Escape to VM	• VM Monitor-ing from VM • VM Moni-toring from Host	• VM toring from Moni-Host • Inter-VM Com. Intro-spection	• VM Denial of Service	• OS Exploitation Kernel • VM Hopping • VM Escape to VM
			• Control Ch.
			Exploitation		• Hypervisor
	Hypervisor		• Hypervisor Exploitation • VM Escape to Host	• Control Ch. Exploitation • Hypervisor Exploitation	Exploitation • Computation Res. Monop-olisation	• Hypervisor Denial Service	of	• VM Escape to Host
	Host OS Kernel		• VM Escape to Host			• Hypervisor Denial Service	of	• VM Escape to Host
	Hardware		• Firmware Exploitation		• Firmware Exploitation	• Hypervisor Denial Service	of	• Firmware Exploitation

Table 4 :

 4 Pros and Cons Comparison Table with Respect to Counter-MeasuresThe cost category evaluates qualitatively the counter-measure configuration process: once during virtualization environment set-up (low), once per VM image building (medium), or at each software update (high).

	Counter-measures	Coverage	Requirements	Cost
	Kernel-based Counter-	OS-Kernel,	Existence of mechanism implementations	Medium
	measure	Host-OS Kernel	for the operated kernels
	Application-based	Application,	Existence of mechanism implementations for	Medium
	Counter-measure	Runtime	all the operated applications and their runtime
	Secure Software	OS-Kernel,	Software management only made in secure	Medium
	Management	Application, Runtime	package managers
	Execution Environment	Hypervisor,	Existence of mechanism implementations in	Low
	Counter-measure	Host-OS Kernel	the operated hypervisors
	Granularity of the	Hypervisor	Modular architecture of the hypervisor	Low
	Hypervisor Arch.
	Networking and Storage	Hypervisor	Exposition of storage and networking	Low
	Protection		interfaces by the hypervisor
	Formal Verification of	OS Kernel, Hypervisor,	Limited code base, design and maintainability	High
	Code	Host-OS Kernel	of the formal proof
	Unecessary Capability	All Software Components Modular design and/or support for disabling	Medium
	Dropping		features at runtime
	Outsourcing of In-VM	Application, Runtime,	Short-living VM instance, secured image	Low
	Software Mgmt	OS Kernel	build environment
	Monitoring of Built	Application, Runtime,	Existence of interfaces to monitor and	Medium
	Resources	OS Kernel	semantic knowledge of monitored data
	Orchestration of Security	Application, Runtime,	Specification of security requirements and	Low
	Mechanisms	OS Kernel	mechanism features
	Security Programmability Application, Runtime,	Exposure of reconfiguration interfaces, or	Medium
		OS Kernel	possibility to restart components

Table 5 :

 5 Relationships among Threats and Countermeasures (or Recommendations)

OS-level virtualization is also referred as containerization. However, some hypervisor-based solutions such as[START_REF]Kata Containers -The Speed of Containers, the Security of VMs[END_REF] have started as well to claim this terminology. For the sake of clarity, we dismiss its usage in this article.

Common Vulnerability Enumeration

Open Vulnerability and Assessment Language