Leonardo Goratti
email: leonardo.goratti@create-net.org

Cristina E Costa

Jordi Perez-Romano

Oriol Sallent

Cristina Ruiz

August Betzler

Sayyad Pouria

Seiamak Khodashenas

Karim M Vahid

Babangida Nasr

Abubakar

P S Khodashenas

S Vahid

K M Nasr

B Abubakar

A Whitehead

M Belesioti

I Chochliouros

Network Architecture and Essential Features for 5G: The SESAME Project Approach

Keywords: 5G, scaas, mobile edge computing, self-organising networks, small cell virtualisation

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

In recent years the way mobile users access and consume contents has dramatically changed due to remarkable progresses of miniaturised electronics and the proliferation of portable and user friendly devices including smartphones and tablets [START_REF] Kim | A Study of Mobile User Engagement (MoEN): Engagement Motivations, Perceived Value, Satisfaction, and Continued Engagement Intention[END_REF]. As shown in [START_REF]On the Pulse of Networked Society[END_REF], during the third quarter of 2015, almost 3.4 billion subscriptions were registered worldwide. At the same time new social services and applications have become so popular that mobile connectivity is a preferred way for users. It is evident that with the current pace of growth even LTE-Advanced (LTE-A) will fail to support such high traffic.

Recently the 5G Infrastructure Public Private Partnership (5G-PPP) [3] has started its activities to create the next generation of mobile network. The SESAME project [4] is one amid the nineteen projects which were recently funded. As mentioned above, the emergence of innovative services and increased network availability motivate and drive further mobile user's engagement, creating a loop of increasing expectations and demand [START_REF] Tojib | The Facilitating Role of Smart-phones in Increasing Use of Value-Added Mobile Services[END_REF]. It is worth registering that mobile broadband penetration has indeed risen to 85.5% in the OECD area, meaning more than four wireless subscriptions for every five inhabitants, while the penetration rate in June 2014 was just 76% [START_REF]OECD Broadband Statistics Update[END_REF]. To cope with this large growth, METIS project [START_REF]Requirements and KPIs for 5G Mobile and Wireless System with Recommendations for Future Investigations[END_REF] has defined a large number of use cases that 5G technology shall serve with superior performance over previous generations of cellular technology.

To meet growing users' demand the solution is offered by denser networks in which small form factor small cell (SC) devices can be deployed in large amounts and operate over licensed spectrum. This trend is also regarded to as network densification. Several approaches and technologies are currently converging in the new 5G mobile network as generally discussed in [START_REF] Andrews | What Will 5G Be[END_REF]. In particular, besides new physical layer solutions, Software-Defined Networking (SDN) [START_REF] Kreutz | Software-Defined Networking: A Comprehensive Survey[END_REF], and Network Functions Virtualisation (NFV) [START_REF] Chiosi | Network Functions Virtualisation: An Introduction, Benefits, Enablers, Challenges & Call for Action[END_REF][START_REF]Network Functions Virtualization (NFV); Use Cases[END_REF] are making their way toward a programmable network solution. Borrowing from cloud computing concepts such as Infrastructureas-a-Service (IaaS) and more recently XaaS (anything-as-a-Service) have been adopted also in the mobile network domain. These approaches facilitate the decoupling between service provider, infrastructure provider and network provider. As a consequence, besides traditional Mobile Network Operators (MNO), new types of Virtual Network Operators (MVNO) and Over-The-Top (OTT) service providers can find unprecedented opportunities in 5G.

The SESAME project develops the concept of Small Cell-as-a-Service (SCaaS), which leverages on the separation between traditional market roles, with the aim to make resources available through network virtualisation. In this context, SESAME will maximise the opportunities offered by opening small cells to multi-tenancy. This latter concept, in opposition to typical mobile operators which deploy their own network infrastructure in competition with others, encourages both traditional and new market entrants to share the infrastructure. In this case operators can differentiate based on their service offers rather than on network connectivity. To manage the dense SC network, including mitigating interference and assign resources dynamically, the ETSI NFV Industry Specification Group (ISG) has developed the Management and Orchestration (MANO) framework that constitutes a solution for managing virtualised small cells [START_REF]Network Functions Virtualisation (NFV); Management and Orchestration[END_REF]. This flexible and dynamic system allows operators to reduce CAPEX and OPEX as required in the next generation of mobile networks.

SESAME Enabling Technologies

Small Cells. Small Cells Small cells have become pivotal in today's 4G access. Small cells installation is an effective way to achieve greater performance and capacity to both indoor and outdoor places: they provide improved cellular coverage, higher capacity and applications for homes and enterprises, as well as in dense metropolitan and rural areas [START_REF] Andrews | Seven Ways that HetNets are a Cellular Paradigm Shift[END_REF]. Their role is crucial for providing services in specific high traffic places such as office areas, dense urban areas, stadiums, shopping malls, concert venues, and generally, places with (tactic or sporadic) high end-users density [START_REF] Osseiran | Scenarios for 5G Mobile and Wireless Communications: The Vision of the METIS Project[END_REF].

Mobile-Edge Computing. ETSI Mobile-Edge Computing (MEC) offers computing capabilities at the network edge and bring different services near to the mobile subscribers. Providing edge cloud capabilities allows to enable accelerated services, content and application thanks to increased network responsiveness. The approach proposed is to deploy a MEC server between the mobile core and the Radio Access Network (RAN). Typical services which can benefit from mobile-edge computing include Internet-of-Things, augmented reality and data caching.

Virtualisation. Virtualisation of the communication infrastructure, such as core/edge network elements and access points/macrocells, has been extensively studied by several industry and research initiatives up to now. Recently, its applicability to the small cell infrastructure has now started to receive increasing attention. The remainder of this paper is organised as follows. In Section 2 we describe the general principles behind the SESAME project. In Section 3 we provide detailed description of the SESAME system and conclusions are drawn in Section 5.

SESAME: Small Cell Coordination for Multi-Tenancy and Edge Services

The SESAME project targets innovations around the placement of network intelligence and services in the network edge through NFV and cloud computing. Through the evolution of the SC concept, already mainstream in 4G, SESAME expects to exploit its full potential in challenging highly dense 5G scenarios. SESAME targets providing SCaaS and to consolidate multi-tenancy in communication infrastructures, allowing several operators/service providers to engage in new sharing models, obtaining higher capacity on the access side and exploiting edge computing capabilities.

The key innovations proposed by SESAME focus on the novel concepts of virtualising SC networks by substantially evolving the SC concept under the paradigms of a multi-operator (i.e. multi-tenancy) enabling framework and an edgebased, virtualised execution environment. SESAME leverages on the capability to deliver intelligence directly to the network's edge, in the form of virtual network appliances. The provisioning of multi-operator SC networks is optimized for the most promising scenarios and use cases.

SESAME develops and will demonstrate an innovative architecture, capable of providing SC networks to multiple operators. SESAME fosters the concept of logical partitioning of the SC network in multiple isolated slices, virtualising and partitioning small cells capacity to multiple tenants. SESAME supports enhanced multi-tenant edge cloud services combining SCs with micro-server facilities. The unique characteristics of the SESAME approach allows new SC operators (real estate companies, municipalities, etc) to enter the value chain deploying access infrastructure in specific high traffic demanding areas, and acting as neutral host providers, offering to existing mobile operators on-demand access to network resources.

SESAME Design Principles

The SESAME architecture was driven by design principles extracted from the identification and analysis of high-impact use cases and stakeholders' requirements. Each stakeholder involved in the process behaves according to different objectives. MNOs can achieve lower total cost of ownership combining own resources with outsourced access capacity and computing; VMNOs benefit of wireless services without owing the physical infrastructure; Venue Owners as new market entrants can benefit from becoming local network operators; End Users can enjoy personalised services with superior quality of experience.

In order to maximise user experience, the architecture should take into account various user requirements such as per-user data rate and latency, robustness and resiliency, mobility, seamless user experience and context-awareness. On the other hand, network requirements must be met in order to allow efficient operation and management. The latter include scalability, network capacity, automated system management and configuration, advanced Self-Organizing Network (SON) features, network flexibility, improved coverage, security and flexible spectrum management.

The set of design principles that drives the SESAME architecture is described below. These are derived from both the analysis of use cases and underlying requirements, as well as considering the state-of-the-art in the technological fields relevant to SESAME (e.g. SCs, NFV, cloud computing), and the most prominent standardisation trends. Principle 1: The SESAME system is sustainable and reconfigurable; Principle 2: SESAME offers an infrastructure shared between operators, transparent and neutral; Principle 3: SESAME accelerates the creation of innovative services with superior quality of experience through mobile edge computing; Principle 4: SESAME develops a system which is capable of optimising the usage of radio, storage and computing resources.

The SESAME System

SESAME proposes a novel 5G platform based on small cells, featuring multi-tenancy and edge cloud capabilities, offered to both network operators and mobile users. In the SESAME system shown in Fig. 1, one key design principle is to support the innovative concept of Virtual Small Cell Network Operators (VSCNO). In particular, VSCNOs use the infrastructure deployed by a SCNO. The SESAME approach enhances the SC with micro-servers, which together form the Light DC execution environment. Micro-servers in the Light DC are connected whereby a dedicated internal network.

High-Level Architecture

The SESAME system in Fig. 1 bases its architecture on the concept of the Cloud-Enabled Small Cell (CESC), a new multi-operator enabled SC that integrates a virtualised execution platform (i.e. the Light DC) for deploying VNFs, supporting automated network management and executing novel applications and services inside the access network infrastructure. The Light DC features low-power processors and hardware accelerators for time critical operations and provides a highly manageable clustered edge computing infrastructure. SESAMEs Light DC is based on low-power 64-bit processors supported by hardware accelerators (e.g. GPU, DSPs and FPGAs). In addition to the internal network connecting the micro-servers, a suitable backhaul connection provides connectivity for the SESAME system to external packet data networks.

The CESC Manager (CESCM) provides optimized management of the CESC deployment and is a key element of the SESAME architecture. It implements orchestration, NFV management, virtualisation of management views per tenant, Self-x features and radio access management techniques. The Portal is used by externals to request resources or apply (re)configuration of parameters. The NorthBound Interface (NBI) is the connecting point between orchestration, Service Level Agreements (SLAs) and VSCNOs. The SLA Monitoring module provides inputs to both VSCNOs and infrastructure providers about correct execution of the environment, and it enables the orchestration subsystem to react accordingly to possible changes in the network.

Small cells connect to the operator's domain and in particular to the Evolved Packet Core (EPC) and the Network Management System (NMS). The NMS is responsible for communicating with the typical Element Management System (EMS) in the small cells domain, in charge of Fault, Configuration, Accounting, Performance and Security (FCAPS) for physical and virtual functions, as will be clarified shortly. As shown in Fig. 1, SDN is used to configure the forwarding behaviour of traffic inside the Light DC and the chain of VNFs.

Essential Features

The essential innovations brought by the SESAME architecture (Fig. 1) delve into virtualising small cells, as well as leveraging on the paradigm of multi-tenancy (i.e. multi-operator) coupled with a virtualised computation environment at the mobile network edge.Multiple network operators will be able to use the SESAME system, each one through its own slice of network resources. The idea is to deploy small cells in which functions are virtualised, and each small cell is also connected to a microserver through appropriate fronthaul technology. A micro-server is based on a non-x86 architecture using 64-bit ARMv8 technology. As anticipated, micro-servers together with the SCs form the CESC, and a number of CESCs form the CESC cluster, which can be shared by multiple operators within a geographical area. SESAME will develop important advances also in network management developing a novel edge-computing architecture and deploying Self-x procedures directly to the network edge. To that end, SESAME will support multi-tenancy through Multi-Operator Core Network (MOCN).

Functional Split in Small Cells

SESAME fully embraces network virtualisation, which allows small cell functions to be decoupled from the physical hardware. Indeed, SC functions can be split between Physical Network Function (SC PNF) and virtual (SC VNF), with one SC VNF which can be connected to multiple SC PNF through the fronthaul. Several studies are currently carried forward by the Small Cell Forum to evaluate the implications of different functional splits between physical and virtual [START_REF]Small Cell Virtualization: Functional Splits and Use Cases[END_REF]. The fact that one SC VNF can connect to several SC PNF offers several advantages: 1) improved coordination of the radio functions (coordinated scheduling, inter-cell interference coordination, etc), 2) enhanced scalability of small cell deployments with simplified management, 3) accelerated life-cycle upgrade enabling new features and 4) flexibility to optimally make the workload placement.

Cloud-Enabled SESAME Environment

SESAME relies on SDN and NFV to use general-purpose computing and storage hardware at the mobile network edge. Along this direction a micro-scale virtualised execution environment proposed by SESAME consists in the Light DC, which offers the cloud execution environment of SESAME. The Light DC is designed to build a clustered infrastructure with high manageability and optimised to reduce power consumption, cabling, space and costs. This execution platform is used to support the required VNFs that implement the different SC features and the cognitive Self-x management operations. In addition, SESAME will develop several service VNFs, which include virtual video transcoding, virtual caching and virtual load balancer, just to name a few. Both SC and service VNFs shall be described by appropriate descriptor files using either TOSCA, JSON or YAML files.

As shown in Fig. 1, the CESCM includes the SESAME MANO amongst other components. The MANO include the NFV Orchestrator (NFVO), the VNF Manager (VNFM) and the Virtualised Infrastructure Manager (VIM) [START_REF]Network Functions Virtualisation (NFV); Management and Orchestration[END_REF]. The SESAME NFVO provides the automated execution environment for VNFs and Network Services (NSs). The NFVO can deploy a new VNF or NS upon receiving the request through the portal or in reaction to the information supplied by SLA Monitoring module through the NBI. The NFVO shall look at the local catalog, solve the VNF placement problem and interact with the VIM to make an effective deployment of Virtual Machines (VMs) over the hardware substrate. Further the VIM is connected to and SDN controller (e.g. OpenDaylight) to make the chain of VNFs. The VNFM is hence in control of the service life-cycle (migration, rescaling, termination etc).

Network Slicing Through Virtualisation

A fundamental feature of SESAME will be the virtualisation of small cells and their utilisation and partitioning into logically isolated slices, offered to multiple operators/tenants. A hypervisor software is used to create the Network Functions Virtualization Infrastructure (NFVI) on top of the bare hardware where VMs are executed under the control of the SESAME MANO. The main aspect of this innovation will be the capability to accommodate multiple operators within the same infrastructure, satisfying the SLA and requirements of each operator separately. This significantly reduces the costs of the deployed infrastructure (cost of ownership, maintenance, etc), since hosted SCs can be treated as an operating resource instead of a capital expenditure. Under this perspective, the creation of neutral host solutions comes to address also the economic viability of investments done by telecommunications operators.

SLA

The SLA Monitoring module is responsible for gathering information on the use, performance and delivery of network services. It monitors the performance of a tenant network, as well as of the whole SESAME infrastructure. Accordingly, the SLA negotiation (encompassing billing issues, accounting and so forth), which lead to liaisons between virtual and physical operators, must be part of an interactive process with the existing support system of the telecommunications operator through appropriate open software. Furthermore, SLA Monitoring will encompass monitoring and analytics as fundamental tools for efficient virtualized network management.

Self-Organising Network Features

In a multi-tenant scenario like the one considered in SESAME, it should be distinguished between Self-x functions that are tenant-specific (i.e. the configuration of parameters can differ from tenant to tenant) and those that are common to all the tenants. In the following, selected Self-x functions are discussed to illustrate the perspective of tenants and the relation to the SESAME architecture.

Self-Planning Functions

Self-planning is the automation of the decision process to roll out new network nodes in specific areas, identifying adequate configurations and settings of radio parameters, as well as proposing capacity extension for those already deployed (e.g. increasing bandwidth and/or adding new carriers). In SESAME planning of a new cell shall consist of the automatic decision that a new SC has to be deployed in a certain geographical position and the RF planning of such new cell (i.e. transmit power and antenna parameters). The decision is one inherent function to the SCNO, which is responsible to manage the infrastructure to satisfy the capacity demand of different VSCNOs. On the other hand, the spectrum planning function specifies the amount of bandwidth required by a SC (either a new cell or one already deployed), the type of spectrum (e.g. licensed/unlicensed, etc) and carries out the automatic assignment of the spectrum. In SESAME this function can be tenant-specific if slicing of the resources is implemented assigning separate carriers to each tenant.

Self-Optimisation Functions

Once the network is in operational state, self-optimization includes the set of functions to improve or maintain the network performance in terms of coverage, capacity and service quality by tuning the different network settings. In SESAME the Coverage and Capacity Optimization (CCO) function is used to adjust RF parameters based on coverage and capacity targets. This task is accomplished by the SCNO and it cannot be left to a single tenant since these parameters affect all the tenants sharing the same physical SC.

Automatic Neighbor Relations (ANR) is responsible for automatically building the Neighbor Relation Table (NRT) of each small cell. This is fundamental for mobility purposes because handovers can only be executed between neighbor cells. In SESAME, to manage mobility in a multi-tenant scenario, the NRT of each SC has to include the relation between the SCs of the SCNO and the cells that belong to a specific tenant.

Mobility Load Balancing (MLB) addresses the problem of uneven traffic distribution in mobile networks. The main target of MLB and traffic steering algorithms is to enable overloaded cells to re-direct part of their traffic to neighbouring less loaded cells, hence alleviating congestion problems. This suits particularly well the deployment proposed by SESAME within the CESC cluster, in which the central decision of the CESCM can provide optimisation of the cluster as a whole. The resulting increased network efficiency using MLB postpones the deployment of additional network capacity, in turn reducing costs. This is usually done through range-expansion [START_REF] Ruiz-Avils | Analysis of Limitations of Mobility Load Balancing in a Live LTE System[END_REF], achieved by either cell coverage parameter adjustments or mobility parameter adjustments. However, in 4G LTE networks, MLB is known to lead to network performance degradation due to the frequency reuse-1 in this technology [17]. SESAME will develop solutions to remedy this inefficiency.

Admission and congestion control are integral parts of any Quality of Service (QoS) mechanism for networks that support different types of traffic. This is the case in SESAME where a variety of different applications are delivered to the tenants of a CESC. SESAME will utilise admission and congestion control algorithms to deliver in effective manner the SLA agreed between VSCNOs and the SCNO. In traditional networks where the infrastructure is owned by a single tenant, admission and congestion control mechanisms utilise resources (e.g. multi-service packet traffic) that this tenant provides to the end-users. SESAME constitutes a more challenging environment since admission and congestion control mechanisms have to take into account the particular features of the tenants to ensure that services which have specific QoS characteristics can be delivered.

Self-Healing Functions

This is the automation of the processes related to fault management and fault correction, usually associated to hardware and/or software problems, in order to keep the network operational while awaiting a more permanent solution to fix it and/or prevent disruptive problems from arising. In SESAME the concept of cell outage can be extended to both the CESC and CESC cluster. Specifically, CESC outage occurs in case of failure of the small cells, the micro-server or VNFs. Cell outage detection is done first collecting information such as alarms, alters, error messages and key performance indicators at both CESC and CESC cluster levels. Afterward, cell outage detection can be done applying different methods, including data mining (e.g. for large small cells deployment) in order to identify possible misbehaviours [START_REF] Wang | Cooperative Cell Outage Detection in Self-Organising Femtocell Networks[END_REF].

Conclusion

In this paper we have illustrated the system architecture developed by the 5G-PPP SESAME project, as well features and peculiarities of this system, in the scope of the future 5G mobile network. We have discussed the essential features of the different modules present in the system and we have clearly highlighted the suitability of SESAME to leverage an accrued virtualised network environment. Moreover, we have showed that SESAME incorporates guidelines from standard bodies such as the ETSI NFV ISG. SESAME shall leverage on a cloud-enabled small cell environment at the mobile network edge materialised by the Light DC. Network virtualisation shall allow SESAME to decouple network services from the bare hardware in unprecedented manner, and Self-x features render the system a suitable host for multitenant operators.

Fig. 1 .

 1 Fig. 1. SESAME System Architecture.

Acknowledgments

The research leading to these results has been supported by the EU funded H2020 5G-PPP project SESAME under the grant agreement no 671596.


Maximos Kaliakatsos-Papakostas

Dimos Makris

Asterios Zacharakis

Costas Tsougras

Emilios Cambouropoulos
email: emilios@mus.auth.grc12makr@ionio.gr

Learning and blending harmonies in the context of a melodic harmonisation assistant

How can harmony in diverse idioms be represented in a machine learning system and how can learned harmonic descriptions of two musical idioms be blended to create new ones? This paper presents a creative melodic harmonisation assistant that employs statistical learning to learn harmonies from human annotated data in practically any style, blends the harmonies of two user-selected idioms and harmonises userinput melodies. To this end, the category theory algorithmic framework for conceptual blending is utilised for blending chord transition of the input idioms, to generate an extended harmonic idiom that incorporates a creative combination of the two input ones with additional harmonic material. The results indicate that by learning from the annotated data, the presented harmoniser is able to express the harmonic character of diverse idioms in a creative manner, while the blended harmonies extrapolate the two input idioms, creating novel harmonic concepts.

Introduction

Machine learning allows a machine to acquire knowledge from data forming concrete conceptual spaces, while conceptual blending [START_REF] Fauconnier | The Way We Think: Conceptual Blending And The Mind's Hidden Complexities[END_REF] between two input spaces allows new spaces to be constructed expressed as new structural relations or even new elements, creating new and potentially unforeseen output [START_REF] Wiggins | Computational modeling of music cognition and musical creativity[END_REF]. In music, harmony is an characteristic and well-circumscribed element of an idiom that can be learned from human annotated musical data using techniques such as Hidden Markov Models, N-grams, probabilistic grammars, inductive logic programming (see [START_REF] Steedman | The blues and the abstract truth: Music and mental models[END_REF][START_REF] Rohrmeier | Towards a generative syntax of tonal harmony[END_REF][START_REF] Conklin | Representation and discovery of vertical patterns in music[END_REF][START_REF] Scholz | Robust modeling of musical chord sequences using probabilistic n-grams[END_REF][START_REF] Pérez-Sancho | Genre classification using chords and stochastic language models[END_REF][START_REF] Raphael | Functional harmonic analysis using probabilistic models[END_REF][START_REF] Whorley | Multiple viewpoint systems: Time complexity and the construction of domains for complex musical viewpoints in the harmonisation problem[END_REF][START_REF] Dixon | Probabilistic and logic-based modelling of harmony[END_REF][START_REF] Granroth-Wilding | A robust parser-interpreter for jazz chord sequences[END_REF][START_REF] Kaliakatsos-Papakostas | Probabilistic harmonisation with fixed intermediate chord constraints[END_REF][START_REF] Kaliakatsos-Papakostas | Learning and creating novel harmonies in diverse musical idioms: An adaptive modular melodic harmonisation system[END_REF] among others). In the context of computational creativity in music, a challenging task tackled in the Concept Invention Theory (COINVENT) [START_REF] Schorlemmer | Coinvent: Towards a computational concept invention theory[END_REF][START_REF] Kaliakatsos-Papakostas | Concept Invention and Music: Creating Novel Harmonies via Conceptual Blending[END_REF][START_REF] Cambouropoulos | Structural blending of harmonic spaces: a computational approach[END_REF] project is to blend different/diverse input harmonic idioms learned from data to create new idioms that are creative supersets of the input ones.

The paper at hand briefly presents the extension of a melodic harmonisation assistant (introduced in [START_REF] Kaliakatsos-Papakostas | Learning and creating novel harmonies in diverse musical idioms: An adaptive modular melodic harmonisation system[END_REF]) that learns harmonic idioms by statistical learning on human data, for inventing new harmonic spaces by blending transitions between chords. The blended transitions are created by combining the features characterising pairs of transitions belonging to two idioms (expressed as sets of potentially learned transitions) according to an amalgam-based algorithm [START_REF] Eppe | Asp, amalgamation, and the conceptual blending workflow[END_REF][START_REF] Confalonier | Upward Refinement for Conceptual Blending in Description Logic -An ASP-based Approach and Case Study in EL ++[END_REF] that implements the theory presented in [START_REF] Fauconnier | The Way We Think: Conceptual Blending And The Mind's Hidden Complexities[END_REF] for conceptual blending, through the categorical-based methodology presented in [START_REF] Goguen | Mathematical Models of Cognitive Space and Time[END_REF]. The transitions are then used in an extended harmonic space that accommodates the two initial harmonic spaces, linked with the new blended transitions.

Statistical learning of harmonies from human annotated datasets

Before blending harmonies, the system learns different aspects of harmony through annotated training data, while it produces new harmonisations according to guidelines provided in a melody input file given by the user. The system learns the available chord types within diverse dataset (according to their root notes) based on the General Chord Type (GCT) [START_REF] Cambouropoulos | An idiomindependent representation of chords for computational music analysis and generation[END_REF] representation, which can be used not only to represent but also to describe meaningful relations between harmonic labels [START_REF] Kaliakatsos-Papakostas | Evaluating the General Chord Type representation in tonal music and organising GCT chord labels in functional chord categories[END_REF] -even in non-tonal music idioms [START_REF] Kaliakatsos-Papakostas | Harmony in the polyphonic songs of epirus: Representation, statistical analysis and generation[END_REF][START_REF] Cambouropoulos | The Harmonic Musical Surface and Two Novel Chord Representation Schemes[END_REF]. The training data include musical scores from many idioms (from Modal harmonisations in the Middle Ages to harmonisations of popular music and jazz in the 20th century), with expert annotations. Specifically, the notes of harmonic manually annotated reductions are regarded for the harmonic learning process, where only the most important harmonic notes are included; additional annotated layers of information are given regarding the tonality and the metric structure of each piece. Accordingly, the format of the user melody input file includes indications of several desired attributes that the resulting harmonisation should have.

After the system is trained, it is able to harmonise a user-given melody that, in this stage, includes manual annotations about harmonic rhythm, harmonically important notes, key and phrase structure. Additionally, the user has the freedom to choose specific chords at desired locations (constraint chords), forcing the system creatively to produce chord sequences that comply with the user-provided constraints, therefore allowing the user to 'manually' increase the interestingness of the produced output.

The cHMM [START_REF] Kaliakatsos-Papakostas | Probabilistic harmonisation with fixed intermediate chord constraints[END_REF] algorithm is used for modelling/learning chord progression probabilities for a given idiom. Then statistical information from the user-defined melody is combined with the chord progression model to generate chord progressions that best represent the idiom. Additionally the algorithm offers the possibility for prior determination of intermediate 'checkpoint' chords [START_REF] Chuan | A hybrid system for automatic generation of style-specific accompaniment[END_REF]). The fixed intermediate chords on the one hand help towards preserving some essence of higher level harmonic structure through the imposition of intermediate and final cadences, while on the other hand allow interactivity by enabling the user to place desired chord at any position. Statistics for cadences are learned during the training process, where expert annotated files including annotations for phrase endings are given as training material to the system. After collecting the statistics about cadences from all idioms, the system, before employing the cHMM algorithm, assigns cadences as fixed chords to the locations indicated by user input. The cadence to be imported is selected based on three criteria: (a) whether it is a final or an intermediate cadence; (b) the cadence likelihood (how often it occurs in the training pieces); and (c) how well it fits with the melody notes that are harmonised by the cadence's chords. Direct human intervention allows the user of the system to specify a harmonic 'spinal chord' of anchor chords that are afterwards connected by chord sequences that give stylistic reference to a learned idiom.

Regarding voice leading, experimental evaluation of methodologies that utilise statistical machine learning techniques demonstrated that an efficient way to harmonise a melody is to add the bass line first [START_REF] Whorley | Multiple viewpoint systems: Time complexity and the construction of domains for complex musical viewpoints in the harmonisation problem[END_REF]. The presented harmoniser, having defined the optimal sequence of GCT chords, uses a modular methodology for determining the bass voice leading presented in [START_REF] Makris | Probabilistic modular bass voice leading in melodic harmonisation[END_REF], which utilises independently trained modules that include (a) a hidden Markov model (HMM) deciding for the bass contour (hidden states), given the melody contour (observations); (b) distributions on the distance between the bass and the melody voice; and (c) statistics regarding the inversions of the chords in the given chord sequence.

The bass voice motion provides abstract information about the motion of the bass, however, assigning actual pitches for a given set of GCT chords requires additional information: inversions and melody-to-bass distance distributions are also learned from data. The inversions of a chord play an important role in determining how eligible is each chord's pitch class to be a bass note, while the melody-to-bass distance captures statistics about the pitch height region that the bass voice is allowed to move according to the melody. After obtaining the exact bass pitch, the exact voicing layout, i.e. exact pitches for all chord notes, for each GCT chord is defined. To this end, a simple statistical model is utilised that finds the best combination of the intermediate voices for every chord according to some simple criteria. These criteria include proximity to a pitch-attractor, evenness of neighbouring notes distances and inner voice movement distances between chords. These criteria form an aggregate wighted sum that defines the optimal setting for all the intermediate notes (between the bass and the melody) in every GCT chord. 3 Blending learned harmonies

In the presented system, the harmony of an idiom is represented by first order Markov matrices, which include one respective row and column for each chord in the examined idiom. The probability value in the i-th row and the j-th column exhibits the probability of the i-th chord going to the j-th -the probabilities of each row sum to one. Figure 2 (a) illustrates a grayscale interpretation of the transition in a set of major-mode Bach chorales. An important question is: Given two input idioms as chord transition matrices, how would a blended idiom be expressed in terms of a transition matrix? The idea examined in the present system is to create an extended transition matrix that includes new transitions that allow moving across chords of the initial idioms by potentially using new chords. The examined methodology uses transition blending to create new transitions that incorporate blended characteristics for creating a smooth 'morphing' harmonic effect when moving from chords of one space to chords of the other. An abstract illustration of an extended matrix is given in Figure 2 (b).

In an extended matrix (Figure 2), when using transitions in I i only chords of the i-th idiom are used, while (blended) transitions in A i-j create direct jumps from chords of the i-th to chords of the j-th idiom. Transitions in B i-X constitute harmonic motions from a chord of idiom i to a newly created chord by blending. For moving from idiom i to idiom j using one external chord c x that was produced by blending, a chain of two consecutive transitions is needed (B i-X-j ): c i → c x followed by a transition c x → c j , where c i in idiom i and c j in idiom j respectively. Transitions in C are disregarded since they incorporate pairs of chords that exist outside the i-th and j-th idioms.

Based on this analysis of the extended matrix, a methodology is proposed for using blends between transitions in I 1 and I 2 . Thereby, transitions in I 1 are blended with ones in I 2 and a number of the best blends is stored for further investigation, creating a pool of best blends. A greater number of blends in the pool of best blends introduces a larger number of possible commuting paths in A i-j or in B i-X-j . Transition blending is performed through amalgam-based conceptual blending that has already been applied to invent chord cadences [START_REF] Eppe | Computational invention of cadences and chord progressions by conceptual chord-blending[END_REF][START_REF] Zacharakis | Conceptual blending in music cadences: A formal model and subjective evaluation[END_REF]; in this setting, cadences are considered as special cases of chord transitions-pairs of chords, where the first chord is followed by the second one-that are described by means of features such as the roots or types of the involved chords, or intervals between voice motions, among others. For more information on transition blending the reader is referred to [START_REF] Kaliakatsos-Papakostas | An argument-based creative assistant for harmonic blending[END_REF].

Harmonisation examples

To briefly demonstrate the effect that transition blending has on forming the extended matrix that combines two initial idioms, harmonisations of the first part of 'Ode to Joy' by Beethoven are illustrated in Figure 3. Initially, one can observe that the learned harmonic features from the Bach chorales and the jazz sets (Figure 3 (a) and (b) respectively) are reflected in the harmonisations that the system produces. In the case of the Bach chorales, the most convenient (yet not so musically impressive) sequence of chords is generated, where the V-I pattern is repeated. The jazz harmonisation includes modifications of the usual ii-V-I pattern. By blending the transitions of the two initial idioms, the produced harmonisation (illustrated in Figure 3 (c)) features new structural relations, incorporating chords and chord sequences that are not usual in any of the initial idioms. However, even though the chords and chord transitions per se are unusual, their encompassed features reflect musical attributes of the initial idioms.

Conclusions

This paper describes a melodic harmonisation system which receives as inputs a melody file and two harmonic idioms and produces a melodic harmonisation with the blended harmony of two input idioms. To this end, diverse harmonic datasets were compiled and annotated by experts, while the harmonic description of each idiom was based on chords extracted with the General Chord Type (GCT) algorithm, statistical learning of chord progressions, cadences and chord voicing over these data. Blending of harmonies is performed through blending chord transitions (one chord leading to another) from the input idioms using an algorithmic realisation of conceptual blending based on category theory. Chord transition blending combines features between pairs of transitions from the two input harmonic idioms, producing new transitions that potentially include new chords for both idioms and incorporate blended features. These new blended transitions act as connection points between the two input harmonic idioms, generating the extended idiom that is a blended harmonic superset of the two input ones. A thorough experimental process that evaluates the usefulness of the produced harmonisations in real-world applications (e.g. when the system is used as an assistant for composers) is underway. Initial experimental results indicate that the blended melodic harmonisations are more interesting than the ones produced by using each input idiom separately. Additional experimental processes are expected to provide insights into whether the blended harmonic spaces are perceived as alterations of one of the input spaces (one-sided blends), balanced blends or radically new harmonic idioms, as well as into the role of the user-defined melody in the process of using blended or non-blended harmonising idioms.

Fig. 1 .

 1 Fig. 1. System overview.

Fig. 2 .

 2 Fig. 2. Graphical description of (a) the transition matrix in a set of major-mode Bach chorales and (b) an extended matrix that includes transition probabilities of two initial idioms -like the ones depicted in (a) -and of several new transitions generated through transition blending.

  Ode to joy harmonised in the style of Bach chorales. Ode to joy harmonised in the style of jazz.

  Ode to joy harmonised in the the blended style of Bach chorales and jazz.

Fig. 3 .

 3 Fig. 3. Beethoven's Ode to joy theme harmonised by the system with learned idioms and their blend.

Acknowledgments

This work is supported by the COINVENT project (FET-Open grant number: 611553).


S V Georgakopoulos

K Kottari

K Delibasis

V P Plagianakos

I Maglogiannis

Convolutional Neural Networks for Pose Recognition in Binary Omni-Directional Images

Keywords: Computer Vision, Convolutional Neural Networks (CNNs), omnidirectional image, fish-eye camera calibration, pose classification, synthetic silhouette 1

In this work, we present a methodology for pose classification of silhouettes using convolutional neural networks. The training set consists exclusively from the synthetic images that are generated from three-dimensional (3D) human models, using the calibration of an omni-directional camera (fisheye). Thus, we are able to generate a large volume of training set that is usually required for Convolutional Neural Networks (CNNs). Testing is performed using synthetically generated silhouettes, as well as real silhouettes. This work is in the same realm with previous work utilizing Zernike image descriptors designed specifically for a calibrated fish-eye camera. Results show that the proposed method improves pose classification accuracy for synthetic images, but it is outperformed by our previously proposed Zernike descriptors in real silhouettes. The computational complexity of the proposed methodology is also examined and the corresponding results are provided.

Introduction

Several computer vision and Artificial Intelligence applications require classification of segmented objects in digital images and videos. The use of object descriptors is a conventional approach for object recognition though a variety of classifiers. Recently, many reports have been published supporting the ability of automatic feature extraction by Convolutional Neural Networks (CNNs) that achieve high classification accuracy in many generic object recognition tasks, without the need of user-defined features. This approach is often referred to as deep learning. More specifically, CNNs are state of the art classification methods in several problems of computer vision. They have been suggested for pattern recognition [START_REF] Jia | Caffe: Convolutional architecture for fast feature embedding[END_REF], object localization [START_REF] Oquab | Is object localization for free?weakly-supervised learning with convolutional neural networks[END_REF], object classification in large-scale database of real world images [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF], and malignancy detection on medical images [START_REF] Cheng | Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans[END_REF], [START_REF] Li | Medical image classification with convolutional neural network[END_REF], [START_REF] Suk | Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis[END_REF]. Several reports exist in literature for the problem of human pose estimation and can be categorized in two approaches. The first approach relies on leveraging images local descriptors (HoG [START_REF] Junior | Trainable classifier-fusion schemes: an application to pedestrian detection[END_REF], SHIFT [START_REF] Tamimi | Localization of mobile robots with omnidirectional vision using particle filter and iterative sift[END_REF], Zernike [START_REF] Delibasis | Geodesically-corrected Zernike descriptors for pose recognition in omnidirectional images[END_REF], [START_REF] Hwang | Local descriptor by zernike moments for real-time keypoint matching[END_REF], [START_REF] Zhu | Translation and scale invariants of Tchebichef moments[END_REF], [START_REF] Zhu | Translation and scale invariants of Tchebichef moments[END_REF]) to extract features and subsequently constructing a model for classification. The second approach is based on model fitting processes [START_REF] Yang | Articulated pose estimation with flexible mixtures-of-parts[END_REF][START_REF] Kottari | Fish-eye camera video processing and trajectory estimation using 3d human models[END_REF]. CNNs are trainable multistage architectures that belong to the first approach of classification methods [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF]. Basically each stage comprises of three types of layers, the convolution, the pooling and the classic neural network layer, which is commonly refereed as fully-connected layer. Each stage consists of one of the previous layer type or an arbitrary combination of them. The trainable components of a convolutional layer are mapped as a batch of kernels and perform the convolution operator on the previous layer's output. The pooling layer performs a subsampling to its input with most commonly used pooling function to be the max-pooling, taking the maximum value of the local neighborhoods. Finally, the fully-connected layer can be treated as a special case of kernel with size 1 × 1. To train this network, the Stochastic Gradient Descent is usually utilized with the usage of mini-batches [START_REF] Bottou | Stochastic gradient descent tricks[END_REF]. However, a drawback of the CNNs is the extensive training time required because of the amount of trainable parameters. Due to the inherent parallelism in their structure, the usage of the graphics processing units (GPUs) has been established to perform the training phase [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]. To achieve high quality results, the CNNs require training dataset of large size. Very recently, methods that use deep neural networks in order to tackle the problem of human pose estimation have started to appear in the literature. In [START_REF] Toshev | Deeppose: Human pose estimation via deep neural networks[END_REF] the use the CNN as a regressor for rough joint locator hand-annotations of the Frame Labeled In Cinema (FLIC) dataset is reported. While the results are very promising, the existence of hand-annotations on training set is needed. In this work, we test the suitability of a well-established CNN methodology for pose recognition in synthetic binary silhouette images. To the best of our knowledge CNNs have not been utilized to deal with binary images problems. These types of problems are distinct, because information is limited by the lack of RGB data. Furthermore, the silhouettes of this work are rendered through an omni-directional camera -fisheye. Fisheye cameras are dioptric omni-directional cameras, increasingly used in computer vision applications [START_REF] Kemmotsu | Recognizing human behaviors with vision sensors in a Network Robot System[END_REF], [START_REF] Zhou | Activity Analysis, Summarization and Visualization for Indoor Human Activity Monitoring[END_REF], due to their 180 degree field of view (FoV). In [START_REF] Mei | Single View Point Omnidirectional Camera Calibration from Planar Grids[END_REF], [START_REF] Li | Plane-Based Calibration and Auto-calibration of a Fish-Eye Camera[END_REF], [START_REF] Shah | Intrinsic parameter calibration procedure for a high distortion fisheye lens camera with distortion model and accuracy estimation[END_REF] the calibration of fish-eye camera is reported to emulate the strong deformation introduced by the fish-eye lens. In [START_REF] Wei | Fisheye video correction[END_REF] a methodology for correcting the distortions induced by the fish-eye lens is presented. The high volume of artificial silhouettes required for training the CNN, is produced by using 3D human models rendered by a calibrated fish-eye camera. Comparisons are provided for synthetic and real data with our recent method proposed in [START_REF] Delibasis | Geodesically-corrected Zernike descriptors for pose recognition in omnidirectional images[END_REF].

Methodology

Overview of the method

The main goal of this work is the assessment of the popular CNN technique ability to recognize different poses of binary human silhouettes from indoor images acquired by a roof-based omnidirectional camera. An extensive dataset of binary silhouettes is created using 3D models (

Calibration of the fish-eye camera

The fish-eye camera is calibrated using a set of manually provided points, as described in detail in [START_REF] Delibasis | Refinement of human silhouette segmentation in omni-directional indoor videos[END_REF]. The achieved calibration compares favourably to other state of the art methodologies [START_REF] Rufli | Automatic Detection of Checkerboards on Blurred and Distorted Images[END_REF] in terms of accuracy. In abstract level, the calibration process defines a function F that maps a real world point

j i F , = real x (1)
The resulting calibration is visualized in Fig. 2 for a grid of points virtually placed on the floor and on the walls of the room.

Let B be the binary frame. If we apply the above equation for each real x point of a 3D model and set ( )

1 , = j i B
, then B will contain the silhouette as imaged by the fisheye camera.

Synthetically generated silhouettes

In this work, a number of 3D models (Fig. 3), obtained from [START_REF] Hasler | Multilinear pose and body shape estimation of dressed subjects from image sets[END_REF], [25], are utilized. These models have known real-world coordinates stored in the form of triangulated surfaces. However, only the coordinates of the vertices are used for rendering the binary silhouette frames. Rendering of the models and generation of a silhouette in a binary frame B was achieved by using the parameterized camera calibration as following:

( ) ( ) , 1 k B F = real x (2)
where { }

N k k , , 2 , 1 ,

Convolutional Neural Networks

The CNN was implemented with four convolutional layers; each one of the first three is followed by a max-pooling layer, while the fourth convolutional layer is followed by a fully-connected feed-forward Neural Network with two hidden layers (see Fig. 4). There exist four convolutional layers, each one consists of a n × n, filter, for n=5, 4, 3 and 2 respectively. Pooling filters of a 2 × 2 dimension exists between each two successive convolutional layers. The convolutional layers consist of 16, 16, 32, and 32 filters, respectively, while the max-pooling layers utilize 16, 16, and 32 filters. Finally, the Feed Forward Neural Network consists of one layer with 64 neurons, followed by 20 hidden neurons and one output layer of the generalization of logistic regression function, the softmax functions, where maps the CNN's input into the class probabilities and is commonly used in CNNs. In order to exploit the power of the CNNs that relies on the depth of their layers and at the same time considering the limitations of the GPU memory, we feed the network with binary images sized to 113 × 113 pixels that contain the synthetic silhouettes. The structure of the CNN used in this work is constructed as following: The inputs of the network are binary image silhouettes. The network consists of four convolutional layers, two fully connected layers and the output layer, which are succeeded by a max-pooling layer. The aforementioned construction is illustrated in Fig. 4.

Results

The generation of an extensive dataset is very important for the successful application of CNNs. We used the dataset that we generated for our previous work involving evaluation of geodetically corrected Zernike moments [START_REF] Delibasis | Geodesically-corrected Zernike descriptors for pose recognition in omnidirectional images[END_REF]. Thus, for each one of the 5 poses, the 3D model of each available subject was placed at 13x8 different positions defined on a grid with constant step of 0.5 meters. For each position the model is rotated round the Z-axis every π/5 radians. Positions with distance less than 1 meter from the trail of the camera were excluded. Silhouettes were rendered using the camera calibration (Subsection 2.2).The total number of data for all 5 poses equals to 32142, as described in Table 1. The 50% of the silhouettes was used as training test and the rest as test set. The CNN learning algorithm was implemented using the Stochastic Gradient Descent (SGD) with learning rate 0.01 and momentum 0.9 with 30000 iterations on the 50% of the whole dataset and a mini-batch of 50 images. The training of the CNN was performed using the GPU NVIDIA GeForce GTX 970 with 4 GB GPU-RAM and the Convolutional Architecture for Fast Feature Embedding (CAFFE) library [START_REF] Jia | Caffe: Convolutional architecture for fast feature embedding[END_REF]. The confusion matrix for the 5 synthetic datasets is shown in Table 2. The achieved accuracy was 98.08 %, which is marginally better than the accuracy obtained with the geodetically corrected Zernike moments (95.31 % as reported in [START_REF] Delibasis | Geodesically-corrected Zernike descriptors for pose recognition in omnidirectional images[END_REF]). In order to further test the proposed CNN-based methodology, 4 short video sequences were acquired using the indoor, roof-based fish-eye camera, during which the subjects assumed two generic poses: "standing" and "fallen". Each frame of the real video was manually labelled. The training set that was used for training the CNN and the other classifiers in the case of ZMI [START_REF] Delibasis | Geodesically-corrected Zernike descriptors for pose recognition in omnidirectional images[END_REF] for the two generic poses (standing and fallen) was obtained as following: The dataset for the generic "standing" pose consists of the union of the 5 standing poses (of Fig. 3). The dataset for the generic "fallen" pose consists of prone/back and side falling, generated from the generic standing pose by interchanging Z-axis with Y-axis coordinate and Z-axis with X-axis, respectively. This dataset (which does not contain any real silhouettes) was used in order to train the CNN and the other classifiers to evaluate the 2 generic poses and recognise them in the real silhouettes. Comparative results with our previous approach (geodetically corrected Zernike moments [START_REF] Delibasis | Geodesically-corrected Zernike descriptors for pose recognition in omnidirectional images[END_REF]) are shown in Table 3. It can be seen that in real data, our previously proposed geodesically-corrected ZMI (GZMI) have better discriminating power compared to the CNN. Based on our results so far, it appears that the GZMI are more immune to imperfect segmentation than the CNN. Possible explanations on this observation are suggested in the next section. The achieved accuracy of the CNN compares favorably with the accuracy achieved using the recently proposed GZMI [START_REF] Delibasis | Geodesically-corrected Zernike descriptors for pose recognition in omnidirectional images[END_REF] for different sizes of the training subset. Fig. 5 shows, comparatively, the accuracies achieved by the two methods for training set equal to 10% up to 50% of the total dataset of the synthetically generated poses.

Conclusions and further work

The purpose of this work was to assess the ability of convolutional neural networks -CNNs -to correctly classify synthetically generated, as well as real silhouettes, using an extensive synthetic training set. The training set consists exclusively from the synthetic images that are generated from three-dimensional (3D) human models, using a calibrated omni-directional camera (fish-eye). Our results show that the proposed CNN approach is marginally better than the geodesic Zernike moment invariants (GZMI) proposed in our recent work [START_REF] Delibasis | Geodesically-corrected Zernike descriptors for pose recognition in omnidirectional images[END_REF], but appears to be outperformed in the problem of real silhouette classification. The GZMI features were adapted from their classic definition, using geodesic distances and angles defined by the camera calibration. On the other hand, the CNN generates features that minimize classification error during the training phase, but do not correspond directly to physical aspects of omni-directional image formation. The results that are reported in this work indicate that the proposed GZMI features appear to be more robust to noise induced by imperfect segmentation, than the features generated by CNN. However, more experimentation is required to draw more definite conclusions. Thus, it is in our future steps to further investigate the structure and the parameters of the utilized CNN in order to improve the performance.

  [START_REF] Hasler | Multilinear pose and body shape estimation of dressed subjects from image sets[END_REF] and[25]) of a number of subjects in 5 different standing positions. The 3D models are placed in different positions and at different rotations round the Z-axis in the real world room. Then they are rendered through the calibration of the fish-eye camera, generating binary silhouettes. The dataset of binary silhouettes is separated into training and testing subsets. Classification results are calculated using the testing subset, as well as real segmented silhouettes of approximately the same poses. The aforementioned steps of the proposed methodology are shown in the block diagram of Fig.1.

Fig. 1 .

 1 Fig. 1. The steps of the proposed methodology

  silhouette Generation of training set at different locations (orientation changes are not shown)

Fig. 2 .

 2 Fig. 2. Visualization of the resulting fish-eye model calibration, on the FoV of the indoor environment in which experiments have taken place. The landmark points defined by the user are shown as circles and their rendered position on the frame marked by stars.

Fig. 3 .

 3 Fig. 3. The 3D human model poses used in this work.

Fig. 4 .

 4 Fig. 4. An illustration of the employed CNN architecture.

Fig. 5 .

 5 Fig. 5. The accuracy of the test set as a function of the size of the training set comparatively for the two methods. The sizes of the indicated training sets are equal to 10% up to 50% (with a step of 10%) of the total dataset. It can be observed that the accuracy of the proposed CNN (upper blue line) compares favorably to the accuracy of the GZMI (lower magenta line) of [1].

Table 1 .

 1 Dataset for the classification of 5 standing poses.

	Pose (Class)	Data
	1	7446
	2	5096
	3	7056
	4	6272
	5	6272
	Sum	32142

Table 2 .

 2 The confusion matrix achieved for the synthetic data.

	Pose 1 3663	12	17	21	10
	Pose 2	17	2490	13	11	17
	Pose 3	13	12	3473	17	13
	Pose 4	18	17	16	3074	11
	Pose 5	13	11	16	10	3086

Pose 1 Pose 2 Pose 3 Pose 4 Pose 5

Table 3 .

 3 The classification results for the standing/fallen generic classes.

		Accuracy	Sensitivity	Specificity
	real video, CNN	0.6715	0.5281	0.6948
	real video, GZMI	0.8173	0.8854	0.7864


Cansu Gokhan
email: cansugokhann@gmail.com

Ziya Karakaya
email: ziya.karakaya@atilim.edu.tr

Ali Yazici
email: ali.yazici@atilim.edu.tr

Systematic Mapping Study on Performance Scalability in Big Data on Cloud Using VM and Container

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Big data applications continue to receive an ever-increasing amount of attention, thus they become a dominant class of applications deployed over virtualized environments [START_REF] Mytilinis | I/O Performance Modeling for Big Data Applications over Cloud Infrastructures[END_REF]. On the other hand, the resource utilization feature of cloud computing is mostly based on virtualization techniques, which is the common way to run different services on the cloud [START_REF] Morabito | Hypervisors vs. Lightweight Virtualization: a Performance Comparison[END_REF]. By combining these two, most of the big data on cloud environments are using hypervisor to provision the virtual machines. In this technique, the VMs have their own operating systems which run on the virtual hardware resources provided by hypervisor [START_REF] Felter | An updated performance comparison of virtual machines and linux containers, Performance Analysis of Systems and Software (ISPASS)[END_REF]. Although it is proven to be a very useful technique in resource utilization, still there is an inherent overhead because of the hypervisor [START_REF] Mytilinis | I/O Performance Modeling for Big Data Applications over Cloud Infrastructures[END_REF].

In recent years, containers, which are also called "lightweight virtualization", are gaining popularity due to their ability to offer superior performance because they do not have their own operating systems [START_REF] Morabito | Hypervisors vs. Lightweight Virtualization: a Performance Comparison[END_REF]. Instead, they use the OS kernel underlined with the host machine and they work similar to a regular application and are completely isolated from each other as well as from the underlying system. This technique receives its popularity mostly in Linux OS virtualization, since it uses the features provided by Linux OS kernel itself, such as "cgroup", "namespace", etc., in order to completely isolate each container from the rest.

In this study, along with the other research questions, the main purpose of investigation was to identify if there is a gap in the literature and to what extend those techniques are being studied by using experimental approach.

There are three different databases used in this study to search for relevant papers. The authors found 308 papers that appeared to be relevant. After applying the inclusion and exclusion criteria, there were only 62 papers containing significant information either directly or indirectly related with the research questions.

The remainder of this paper is structured as follows : the related works on performance and comparison of VM vs. containers are discussed in Section 2. Section 3 presents our methodology and research questions investigated. In Section 4, the results are summarized and the findings together with discussion is given. Finally, the limitations of this study is appear in Section 5.

Related Work

Readers requiring in-depth information about the technologies and techniques used in virtual machines and containers together with their relationships to hypervisor and underlying operating systems are offered to read the white paper published by Intel [START_REF]Intel White Paper: Linux Containers Streamline Virtualization and Complement Hypervisor-Based Virtual Machines[END_REF].

There are many studies in the literature about big data on cloud environment. One of the Systematic Mapping (SM) studies conducted on this subject is the work of Ibrahim and his colleagues [START_REF] Abaker | The rise of 'big data' on cloud computing: Review and open research issues[END_REF]. They have analysed the scalability issues of storage, but not the scalability issues of performance within the cloud. They have proposed a classification for big data, a conceptual view of big data, and a cloud services model. This model was compared with several representative big data cloud platforms. They have discussed the background of Hadoop technology and its core component, namely MapReduce, and Hadoop Distributed File System (HDFS).

Yanzhang focuses on the scalability performance issues of Hadoop Virtual Cluster with cost consideration [START_REF] He | Scalability Analysis and Improvement of Hadoop Virtual Cluster with Cost Consideration[END_REF]. They compared the scalability performance with respect to scale-up and scale-out methods under different workloads. Hadoop benchmarks and real parallel machine learning algorithms were used to evaluate the scalability performance. Their experimental results showed that the scale-up method outperformed the scale-out method for CPU-bound applications, and the opposite for I/O-bound applications. They also noted that disk and network I/O are the main bottlenecks of cloud platform due to shared resource contention and interference.

The most comprehensive work on performance comparison of virtual machines and Linux containers has been done by Felter and colleagues [START_REF] Felter | An updated performance comparison of virtual machines and linux containers, Performance Analysis of Systems and Software (ISPASS)[END_REF]. Their goal was to isolate and understand the overhead introduced by virtual machines (specifically KVM) and containers (specifically Docker) relative to nonvirtualized Linux on Cloud. They have concluded that both VM and containers are mature technologies, and that both have negligible performance overheads with respect to CPU and Memory performance. Nevertheless, they warned about the use of these technologies in case of I/O intensive works, which is the case in Big Data Application on Cloud.

Yang et al. have discussed the impact of virtual machine on Hadoop [START_REF] Yang | Impacts of Virtualization Technologies on Hadoop[END_REF]. They describe the effect of different virtualization technologies such as KVM, Xen and OpenVZ on MapReduce environment. Also, they evaluate performance and stability of HDFS (Hadoop Distributed File System) on KVM, Xen and OpenVZ. Besides this, Pedro et al. have presented the performance of KVM and OpenVZ using micro benchmarks for disk and CPU [START_REF] Vasconcelos | Performance Analysis of Hadoop MapReduce on an OpenNebula Cloud with KVM and OpenVZ Virtualizations[END_REF].

There are many other papers in the literature that have studied the performance scalability of Big Data Applications. However, only a few of them have focused on performance scalability comparison of Container vs. VM technologies.

Research Methodology

A systematic map study was performed to obtain the current research map on the performance scalability issues in big data on cloud. The guidelines proposed by Peterson and colleagues [START_REF] Kai | Guidelines for conducting systematic mapping studies in software engineering: An update[END_REF] is followed in this study. The mapping study was conducted in three main stages, namely planning, execution and result.

Systematic Mapping Plan

In the planning stage, we defined research questions, search strategy, screening of papers for inclusion and exclusion, classification of papers and data extraction.

Research Questions

The following research questions were identified as relevant to purpose:

1.
To what extend are the published papers on the performance scalability issues in big data on cloud are based on experimental study? 2. What is the percentage of the mostly studied technologies in big data performance scalability issues on cloud environment? 3. Which is the most investigated hypervisor in big data performance scalability issues? 4. What types of containers are being studied in big data on cloud? 5. Which components of the resources are mostly investigated for performance impact on big data analysis? 6. How frequent is the dominating technology being studied in the last five years as a tool in big data on cloud?

Search Strategy

The selected databases for the study are shown in Table 1 in order to identify potentially relevant conference articles and journal publications.

The following keywords were used in order to perform the search for the study: Big data, Cloud Computing, Performance, Scalability, Container, Virtual Machine, Comparision of VM vs. Containers. Search strings were applied to check keyword, title, and abstract fields in order to perform the automatic search in the selected digital libraries.

These strings are given as follows: [ ("Big Data") AND ("Cloud Computing") AND (performance OR Scalability) AND (Container OR VM OR "Virtual Machine")]

Inclusion and Exclusion Criteria The aim of this process is to identify the most relevant studies for the mapping study. According to the research questions, the inclusion and exclusion criteria given in Table 2 were applied to the selected papers.

Classification of Papers

The present work classified the papers according to properties and categories listed in Table 3.

Data Extraction

In order to extract data from the selected studies, we designed a data extraction Excel table. Each selected paper appears as a record Then, the data that is specifically related to research questions were extracted from each study.

Execution of Systematic Mapping

At the execution stage, we conducted a systematic mapping study according to the plan stated in the previous section. The search string was modified for the different syntax as of databases according to the search criteria, and we have found 308 papers as candidate studies from all the selected sources. The title, abstract, and keywords were analysed, and then, some of the articles were eliminated by applying the exclusion criteria. In case of uncertainties as to inclusion of some papers, the introduction and conclusion sections of these articles were also taken into consideration. As a result of eliminating unrelated articles, 62 relevant research papers were selected.

Conclusion

In this paper, we used systematic mapping technique, to obtain a current research map on performance scalability issues in big data on cloud. The guidelines proposed by Peterson and colleagues [START_REF] Kai | Guidelines for conducting systematic mapping studies in software engineering: An update[END_REF] is followed. Of the 308 papers found in three popular databases according to our search string, 62 papers were found to be related either directly or indirectly with the research subject and questions. The papers are then analysed with respect to the research questions.

The following conclusions are deduced from the analysis of those related papers; (i) considering the performed study, 61 of 62 papers were based on experimental studies achieving a majority of the papers; (ii) the mostly studied technologies are Hadoop and MapReduce; (iii) KVM and Xen are the dominant hypervisor technology used in these studies; (iv) Docker is the mostly studied container type, and LXC and OpenVZ are the technologies that are used equally; and (v) CPU and Disk I/O are the two issues that are mostly handled when comparing these technologies. It is better to state that KVM and Xen are opensource, and Spark is relatively newer than Hadoop.

The most important finding of this research is that there are only a handful academic papers which compare the performance scalability of hypervisor-based virtual machines vs. containers for the big data applications on cloud. On the other hand, although there are many researches about the comparison of either VM vs. "bare metal" (physical) on cloud, only three of them compares the performance in Big Data on Cloud. This was the second gap found during this study.

Another most important implication of this study shows that there is a lack of empirical study conducted on the other popular Big Data analysis frameworks, such as Spark, Storm, FLink, etc.

Limitations

In this study, only three databases are searched for relevant papers. These technologies have gained popularity over the past few years, and journal/conference papers may contain more than those of found in these databases. We believe that other databases should also be considered in such analysis, and we plan to further analyse those papers.

3

 3

3. 3

 3 Results of MappingRQ1 Objective: The main objective of answering this question is to identify the proportion of experimental researches already done when compared to others. RQ1 Results : Considering the performed study, 60 of 62 papers were based on experimental studies making them a majority. Fig.1(a)shows the number of experimental and non-experimental studies with respect to publication years. RQ2 Objective: The main objective of answering this question is to identify to what extend the mostly studied technology is dominating the research area. RQ2 Results: The papers were categorized as follows: Hadoop, MapReduce and Spark. If a paper's study could not be defined with a specific technology, it is shown as the category called "Other/Generic".

Fig. 1 .

 1 Fig. 1. Distribution of Technology and VM's

Fig. 2 .

 2 Fig. 2. Components impacting performance and frequency of studies on Hadoop

Fig. 2 (

 2 Fig. 2(a), indicates that 56% of the total studies are focused on CPU, 32% on Memory (RAM), 44% on Network and 69% are focused on Disk I/O components. Also, this figure depicts the number of studies attributing the performance impact to number of VMs and that of containers is 7 (11%). RQ6 Objective: The aim of this question is to determine the frequency of research on the subject within the last five years. RQ6 Results: The result shows that 50 papers out of 62 are about the MapReduce based tool. According to this Fig. 2(b), it is easily seen that the number of papers which used MapReduce as a tool increased suddenly in 2013, and the frequency of papers has decreases after 2014.

Table 1 .

 1 Selected Databases

	Database	Location
	IEEE Explore	http://ieeexplore.ieee.org/
	Science Direct	http://www.sciencedirect.com/
	ACM Digital Library	http://dl.acm.org/

Table 2 .

 2 Inclusion and Exclusion Criteria Inclusion Criteria 1 Studies addressing performance scalability issues in big data on cloud. 2 Journal and/or conference papers. 3 Studies that describe virtual machine and container types in the big data on cloud. 4 Primary or secondary studies. Studies not accessible in full text. 2 Studies that do not address the performance scalability issues in big data on cloud. 3 Studies not presented in English. 4 Prefaces, slides, panels, editorials or tutorials. 5 Studies that do not answer the research questions.

	Exclusion Criteria
	1

Table 3 .

 3 Classification Scheme

	Properties	Categories
	Research Approach	Theory, survey, review, experimental
	Year	Years between 2009 and 2016
	Article title	Name of the article
	Container Type	Docker, OpenVZ, LXC, Linux-VServer
	VM Type	Xen, KVM, Vmware (ESX,ESXi), Others
	Technology	MapReduce, Hadoop, Spark, Storm, FLink
	Component of Hardware Resource CPU, Disk I/O, Network speed, Memory (RAM),
		# of VM/Container

item in this file. The data extraction table consist of article name, year of publication, technology, component of hardware resource, VM types and container types.

List of articles : http://bit.ly/1Ux6H5M