Ronan-Alexandre Cherrueau

Adrien Lebre

Dimitri Pertin

Anthony Simonet

Matthieu Simonin

ENOS: a Holistic Framework for Conducting Scientific Evaluations of OpenStack

Keywords: Performance, Reproducibility, OpenStack, Cloud, Visualization, Control-plane, Data-plane Performance, Reproductibilité, OpenStack, Coud, Visualisation, Control-plane, Data-plane WAN Nova Neutron Nova compute Neutron agents KeyStone Nova compute Neutron cations WAN WAN Nova Neutron Nova compute Neutron agents KeyStone Nova compute Neutron agents

By massively adopting OpenStack for operating small to large private and public clouds, the industry has made it one of the largest running software project. Driven by an incredibly vibrant community, OpenStack has now overgrown the Linux kernel. However, with success comes an increased complexity; facing technical and scientific challenges, developers are in great difficulty when testing the impact of individual changes on the performance of such a large codebase, which will likely slow down the evolution of OpenStack. In the light of the difficulties the OpenStack community is facing, we claim that it is time for our scientific community to join the effort and get involved in the development and the evolution of OpenStack, as it has been once done for Linux. However, diving into complex software such as OpenStack is tedious: reliable tools are necessary to ease the efforts of our community and make science as collaborative as possible. In this spirit, we developed ENOS, an integrated framework that relies on container technologies for deploying and evaluating OpenStack on any testbed. ENOS allows researchers to easily express different configurations, enabling fine-grained investigations of OpenStack services. ENOS collects performance metrics at runtime and stores them for post-mortem analysis and sharing. The relevance of ENOS approach to reproducible research is illustrated by evaluating different OpenStack scenarios on the Grid'5000 testbed.

ENOS: a Holistic Framework for Conducting Scientific Evaluations of OpenStack

Ronan-Alexandre Cherrueau, Dimitri Pertin, Anthony Simonet, Adrien Lebre Inria, Mines Nantes, LINA Nantes, France Email: firstname.lastname@inria.fr , Matthieu Simonin Inria, IRISA Rennes, France Email: firstname.lastname@inria.fr By massively adopting OpenStack for operating small to large private and public clouds, the industry has made it one of the largest running software project. Driven by an incredibly vibrant community, OpenStack has now overgrown the Linux kernel. However, with success comes an increased complexity; facing technical and scientific challenges, developers are in great difficulty when testing the impact of individual changes on the performance of such a large codebase, which will likely slow down the evolution of OpenStack. In the light of the difficulties the OpenStack community is facing, we claim that it is time for our scientific community to join the effort and get involved in the development and the evolution of OpenStack, as it has been once done for Linux. However, diving into complex software such as OpenStack is tedious: reliable tools are necessary to ease the efforts of our community and make science as collaborative as possible.

In this spirit, we developed ENOS, an integrated framework that relies on container technologies for deploying and evaluating OpenStack on any testbed. ENOS allows researchers to easily express different configurations, enabling fine-grained investigations of OpenStack services. ENOS collects performance metrics at runtime and stores them for post-mortem analysis and sharing. The relevance of ENOS approach to reproducible research is illustrated by evaluating different OpenStack scenarios on the Grid'5000 testbed.

I. INTRODUCTION

Although the adoption of Cloud Computing has been largely favored by public offers (Amazon EC2 and Microsoft Azure, to name a few), numerous private and public institutions have been contributing to the development of open-source projects in charge of delivering Cloud Computing management systems [START_REF]Open Source Cloud Computing[END_REF], [START_REF]Open Source Data Center Virtualization[END_REF], [START_REF]The Open Source, Open Standards Cloud[END_REF]. In addition to breaking vendor lock-in, these operating systems of Cloud Computing platforms enable administrators to deploy and operate private cloud offers, avoiding issues such as data-jurisdiction disputes, latency constraints, etc.

After more than six years of intensive effort, the OpenStack software suite has become the de facto open-source solution to operate, supervise and use a Cloud Computing infrastructure [START_REF]The Open Source, Open Standards Cloud[END_REF]. The OpenStack community gathers more than 500 organizations, including large groups such as Google, IBM and Intel. The software stack relies on tens of services with 6-month development-cycles.

Despite the current dynamicity of the whole ecosystem that makes it incredibly hard to keep up with, its adoption is still growing and the stack is now being used in a large variety of areas such as public administrations, e-commerce and science 1 . With the now undeniable success of OpenStack, we argue that it is time for the scientific community to get involved and contribute to the OpenStack software in the same way it has been once done for the Linux ecosystem, in particular in the HPC area. A major involvement of our community will enable OpenStack to cope better with ongoing changes in the Cloud Computing paradigm such as the Fog and Edge Computing proposals and the IoT application requirements. However, diving into the OpenStack ecosystem is a tedious task. The whole software stack represents more than 20 million lines of code including 2 million lines of Python code for core services alone.

To help developers and researchers identify major weaknesses of a complex system such as OpenStack and to facilitate the evaluation of proposed improvements, we designed ENOS 2 . ENOS is a free software framework that leverages container technologies and "off-the-shelf" benchmarks for automating reproducible evaluations of OpenStack in a flexible and extensible way. To the best of our knowledge, ENOS is the first holistic approach for evaluating OpenStack. That is, it has been designed with the Experimentation-as-Code vision: every step of the experimentation workflow, from the configuration to the results gathering and analysis, can be automated [START_REF] Buchert | A survey of general-purpose experiment management tools for distributed systems[END_REF].

Although several performance studies of Open-Stack [START_REF] Karacali | Experiences evaluating openstack network data plane performance and scalability[END_REF], [START_REF] Litvinski | Openstack scheduler evaluation using design of experiment approach[END_REF], [START_REF] Schmidt | Performance analysis of an openstack private cloud[END_REF] have been achieved in the recent years, they all present weaknesses. First, they have been conducted by using ad-hoc approaches that prevent researchers to reproduce them. Second, the underlying complexity and the aforementioned velocity of the Open-Stack ecosystem make these studies deprecated in less than a year. As a consequence, the results presented by these studies cannot determine whether specific revisions/extensions in the code provide significant benefits. Thanks to ENOS, researchers and developers alike can now evaluate the performance of distinct OpenStack deployment scenarios, and compare the collected results to both identify limitations and validate improvement proposals.

ENOS has been built on top of a containerized deployment model of OpenStack, where each OpenStack service is encapsulated in a dedicated container. This allows to easily express, deploy and evaluate different configurations enabling fine-grained investigations of every OpenStack service, including the latest versions available on the OpenStack trunk. Moreover, ENOS has been designed around pluggable mechanisms:

• The extensible deployment engine of ENOS allows to deploy OpenStack on various infrastructures, e.g., testbed platforms such as Grid'5000 [START_REF] Balouek | Adding Virtualization Capabilities to the Grid'5000 Testbed[END_REF] and Chameleon [START_REF] Mambretti | Next generation clouds, the chameleon cloud testbed, and software defined networking (sdn)[END_REF], public or private cloud infrastructures such as Amazon EC2 [START_REF] Walker | Benchmarking amazon ec2 for hig-performance scientific computing[END_REF], on OpenStack itself or on simpler systems such as Vagrant [START_REF] Hashimoto | Vagrant: Up and Running[END_REF]). • ENOS natively supports different types of benchmarking suites such as Rally [START_REF]Rally, the OpenStack benchmarking tool[END_REF] and Shaker [START_REF]Distributed data-plane performance testing tool for OpenStack[END_REF] in addition to allowing the addition of customized ones. This enables ENOS end-users to conduct either control plane or data plane experiments. In the former case, understanding the performance of the controller nodes (i.e., the nodes in charge of supervising the OpenStack infrastructure) is the objective of the experiments. In the latter case, the goal is to evaluate the performance from the application viewpoint (that is, understanding the performance an application can reach when it is executed on top of a particular OpenStack deployment). For both kinds of experiment, the way OpenStack is deployed in terms of configuration parameters and hardware topology has an impact on performance. Finally, ENOS comes with generic visualization tools that provide different views using diagrams, plots and tables of gathered metrics. We claim it is another valu- The remaining of this article is as follows. Section II presents the different technologies we used to build the ENOS framework. The framework itself is discussed in Section III. To illustrate the possibility offered by our framework, we discuss series of experiments that have been conducted thanks to ENOS in Section IV. Related works are presented in Section V. Finally Section VI concludes and discusses future research and development actions.

II. BACKGROUND

In this section, we first give an overview of Open-Stack, then we describe the technologies used to implement the ENOS framework. While the goal of the first part is to illustrate the richness of the OpenStack ecosystem and the different possibilities offered to deploy it, the second part may look rather technical. However, we believe it is more relevant to use already existing technologies than reinventing the wheel. The objective of this second part is to present these technologies we used as building blocks.

A. OpenStack

OpenStack [START_REF]The Open Source, Open Standards Cloud[END_REF] is an open-source project that aims to develop a complete Cloud Computing software stack. Figures 1 and2 Neutron provides "network connectivity as a service" between interface devices (e.g., vNICs) managed by other OpenStack services (e.g., Nova). Glance provides services to discover, register, and retrieve VM images. Finally, Swift is a distributed object/blob store similar to Amazon S3. This architecture is comparable with the generic reference proposed by Moreno [START_REF] Moreno-Vozmediano | IaaS Cloud Architecture: From Virtualized Datacenters to Federated Cloud Infrastructures[END_REF].

From the technical point of view, OpenStack is composed of two kinds of nodes: on the one hand, the compute/storage/network nodes are dedicated to deliver the XaaS capabilities, such as hosting VMs (i.e., data plane); on the other hand, the control nodes are in charge of executing the OpenStack services (i.e., control plane).

OpenStack services are organized following the Shared Nothing principle. Each instance of a service (i.e., service worker) is exposed through an API accessible through a Remote Procedure Call (RPC) system implemented, on top of a messaging queue or via web services (REST). This enables a weak coupling between services and thus a large number of deployment possibilities, according to the size of the infrastructure and the capacity the cloud provider intends to offer. Nevertheless, we highlight that even if this organization of services respects the Shared Nothing principle, most services create and manipulate logical objects that are persisted in shared databases. While this enables service workers to easily collaborate, it also limits the deployment possibilities as each DB represents a single point of failure [START_REF] Lebre | Revising OpenStack to Operate Fog/Edge Computing infrastructures[END_REF]. Figure 3 illustrates four deployment architectures for OpenStack services on a single site (Figures 3(a) and 3(b)) and on multiple sites linked through a WAN connection (Figures 3(c) and 3(d)), only with the essential core services (without Cinder, Ironic, Heat), for the sake of simplicity.

Nova Nova

Figure 3(a) corresponds to a minimal OpenStack deployment: all services have been deployed on a dedicated controller node. Only the agents (nova-compute and neutron-agent) that are mandatory to interact with hypervisors on the compute nodes have been deployed.

In the second scenario, illustrated in Figure 3(b), each service has been deployed on a dedicated control node (there is no change for the compute nodes). While entirely feasible, this second scenario is rarely deployed. In most cases, a control node executes several services (e.g., Keystone and Horizon are often deployed on the same node). Nevertheless, we highlight that for large scale infrastructures, isolating important services such as Nova and Neutron becomes mandatory. This enables the execution of several instances of sub-services such as the nova-scheduler on the node.

The third and fourth scenarios correspond to WANwide architectures. Although those are rarely deployed in production environments yet, industrial actors and telcos, in particular, are investigating to what extent current OpenStack mechanisms can handle Fog and Edge computing infrastructures [START_REF]OpenStack Massively Distributed Clouds Working Group[END_REF]. In Figure 3(c), the control plane is deployed on one site of the infrastructure, and only compute nodes have been deployed to a remote location.

Finally, Figure 3(d) corresponds to the guidelines presented on the OpenStack website to supervise a multisite infrastructure. This illustrates the current agitation around the architecture of OpenStack deployments and justifies an important feature of ENOS, namely the possibility of testing multiple OpenStack deployment topologies transparently.

B. Deployment of OpenStack

Due to the richness and complexity of the OpenStack ecosystem, making its deployment easy has always been an important topic. Among all the deployment solutions that are available, we chose to use Kolla [START_REF]Kolla: production-ready containers and deployment tools for OpenStack[END_REF]. Kolla provides production ready containers and deployment tools for operating OpenStack infrastructures. In Kolla, each OpenStack service is encapsulated with its dependencies in a dedicated container. Container images can be built on demand, stored and used during the deployment. Kolla features many default behaviors, allowing quick prototyping, but they are fully customizable: vanilla or modified versions of OpenStack can be installed, deployment topologies can be adapted to the user's needs and configuration of all the services can be changed. To perform remote actions such as deploying software components, Kolla uses the Ansible deployment engine [START_REF]Ansible is Simple IT Automation[END_REF]. Ansible gathers hosts on groups of machines on which specific tasks are applied. This group mechanism in play is very flexible and thus allows alternative deployment topologies to be specified. By default, Kolla defines two levels of groups. First on a per-service basis, and second on a logical basis. The former allows, for instance, single service to be isolated on a dedicated node. The latter are groups of OpenStack services based on typical deployment practices. An OpenStack deployment is at least composed of four logical groups: i) Control, for hosting the database, messaging middleware, and the various APIs, ii) Storage, for storing block devices, iii) Network, Neutron services and iv) Compute, for hosting the Nova Compute and Neutron agents.

C. Evaluation of OpenStack

Measuring the performance of a cloud infrastructure in a rigorous and comparable way is an important challenge for our community. The Standard Performance Evaluation Corporation has recently proposed the SPEC Cloud benchmark: The benchmark suite's use is targeted at cloud providers, cloud consumers, hardware vendors, virtualization software vendors, application software vendors, and academic researchers. Although the SPEC Cloud benchmark can be easily integrated into ENOS thanks to the pluggable approach (see Section III), the license issue of SPEC benchmark does not enable us to provide ENOS with it by default. Instead, ENOS comes with two open source benchmarks that are Rally and Shaker.

1) Rally: Rally is the official benchmark suite for OpenStack; it allows to test the control plane by requests to running services using their corresponding Python clients. It can test a running cloud or deploy a cloud first, making it an all-in-one solution for both development and production testing. Rally executes scenarios that can be configured by JSON or YAML files. A scenario includes Runner options (e.g., the number of times a request is performed or how many parallel threads are used to perform the requests), Context options (e.g., how many users and tenants must be used for the injection) and scenario-specific configuration, e.g., the number of VMs to boot, or the image file to use when creating an appliance). SLA options can also be provided. In this case, Rally will consider a scenario to fail if the requested SLA is not met. Execution times, failures, and SLA violations are collected and stored in a database. From this database, Rally can also generate HTML and JSON.

2) Shaker: Shaker is a framework for data plane testing of OpenStack. It currently targets synthetic benchmarks execution (for instance iperf3 [START_REF]iPerf -The ultimate speed test tool for TCP, UDP and SCTP[END_REF], flent [START_REF] Flent | The FLExible Network Tester[END_REF]) on top of instances. Shaker supports the definition and the deployment of different instances and network topologies. The possible scenarios include extensive evaluation of network capabilities of an OpenStack cloud.

D. Analysis of OpenStack

Analysis of OpenStack is mostly based on metrics generated during the experiment and relies on three components: metrics agents, metrics collector and metrics visualization. Those components are loosely coupled, allowing for alternatives to be plugged in when necessary. In the current implementation, metric agents are cAdvisor [START_REF]cAdvisor[END_REF] and collectd [START_REF]collectd -The system statistics collection daemon[END_REF]. They are responsible for sending metrics from hosts to the collector. Metrics can be enabled or disabled at will through the metrics agents configuration files. Metrics collector relies on the In-fluxDB timeseries optimized database [START_REF]InfluxDB -Time-series data storage[END_REF]. Visualization is enabled by Grafana [START_REF] Grafana | [END_REF], a dashboard composer. It allows to query multiple data sources and to display them in a Web browser. Dashboards can be saved and shared between users, increasing the reusability of user-made visualizations. Note that Grafana is suitable for both explanatory visualization (with predefined dashboards) and exploratory visualizations, as dashboard can be built interactively.

III. ENOS

Evaluating a complex appliance such as the Open-Stack software suite can be divided into four logical phases. The first phase consists in getting raw resources; the second one deploys and initializes the selected version of OpenStack over these resources; the third phase invokes the benchmarks to be executed; finally, the fourth phase consists in analyzing results of the evaluation. To help engineers/researchers in tackling all these phases, we developed ENOS 3 , a holistic approach for the evaluation of OpenStack. After presenting the resource description language we used to configure ENOS, this section describes how each phase has been implemented, and in particular how by abstracting fundamental principles of each phase, ENOS can address performance evaluations for any infrastructure.

A. ENOS Description Language for Flexible Topologies

The description of the resources to acquire as well as the mapping of the different services on top of those resources is made with a YAML resource description language. In other words, ENOS comes with a dedicated language that describes what OpenStack service will be deployed on which resource. This language offers a very flexible mechanism that lets ENOS end-users specify and evaluate OpenStack performance over a large set of topologies. However, OpenStack is made of numerous services and writing this description is tedious. For this reason, ENOS reuses Kolla service groups (see II-B) to gather many OpenStack services under the same logical name, which drastically reduces the description size. For instance, the small description in Figure 4(a) describes a single-node deployment topology similar to the one in Figure 3(a). This description says: "provide one resource for hosting control services and two others for hosting compute services".

In the context of ENOS, a resource is anything running a Docker daemon and that ENOS can SSH to. This could be a bare-metal machine, a virtual machine, or a container resource according to the testbed used for conducting the experiments.

Moreover, we emphasize that the language is resource provider dependent in order to handle infrastructure 3 Experimental eNvironment for OpenStack -ENOS: https://github.com/BeyondTheClouds/enos specificities. For instance, on Grid'5000 [START_REF] Balouek | Adding Virtualization Capabilities to the Grid'5000 Testbed[END_REF], the language has been extended to specify the name of physical clusters where resources should be acquired, as depicted in Figure 4(b). In this description, the paravance cluster (located in Rennes) will provide resources for control services and the econome cluster (located in Nantes) will provide resources for the compute nodes.

Last but not the least, it is noteworthy that more advanced deployment topologies can be defined by coupling resources with names of OpenStack services.

Isolating a service on a dedicated resource is as simple as adding its name to the description. For instance, adding rabbitmq: 1 at the end of the description on Figure 4(a) tells ENOS to acquire a dedicated resource for the AMQP bus. Henceforth, the bus will no longer be part of the control resource but deployed on a separate resource at the deployment phase. Obviously, it is possible to do the same for the database, nova-api, glance, neutron-server . . . and hence get a multi-nodes topology similar to the one presented in Figure 3(c).

Scaling a service simply requires increasing the number of resources allocated to this service into the description. For instance, increasing the value of rabbitmq: 1 to rabbitmq: 3 tells ENOS to acquire three dedicated resources for the AMQP bus. Henceforth, the deployment phase will deploy a cluster composed of three RabbitMQ.

These two characteristics of the language allow a very flexible mechanism to both isolate and scale services.

B. ENOS Workflow

In the following, we describe the four steps that are achieved by ENOS.

1) enos up: Getting Resources Phase: Calling enos up launches the first phase that acquires the resources necessary for the deployment of OpenStack. To get these resources, ENOS relies on the aforementioned description and the notion of provider. A provider implements how to get resources on a specific infrastructure and thus makes this job abstract to ENOS. With such mechanism, an operator can easily evaluate OpenStack over any kind of infrastructure by implementing the related provider. A provider can also be given by the support team of an infrastructure, independently of any particular OpenStack evaluation project. In other words for each testbed, an extended version of the ENOS DSL and a provider should be available. Currently, ENOS supports two kinds of infrastructures; the first one gets bare-metal resources from the Grid'5000 testbed [START_REF] Balouek | Adding Virtualization Capabilities to the Grid'5000 Testbed[END_REF]; the second one uses a VM based on Vagrant [START_REF] Hashimoto | Vagrant: Up and Running[END_REF]. We motivate these two choices as follow: an ENOS end-user would go with bare-metal providers such as Grid'5000 for performance evaluation at various scales, and prefer the quick Vagrant deployment for testing a particular feature of one service. We emphasize that additional drivers for Amazon or any other system can be easily implemented, as it should correspond to less than 500 lines of Python code. By providing an OpenStack driver, it would also be possible to evaluate new OpenStack features on top of an existing OpenStack deployment.

The output of the first phase is a list of addresses which reference resources, together with the name of OpenStack services to deploy over each resource. This way, ENOS will be able to initiate a SSH connection to these resources during the next phase and deploy the requested OpenStack services.

2) enos init: Deploying and Initializing OpenStack Phase: The init phase deploys and initializes Open-Stack with Kolla. Concretely, ENOS uses the list of resources and services provided by the previous phase and writes them into a file called the inventory file. Kolla then uses this file to deploy, in a containerized manner, OpenStack services onto the correct resources.

The Kolla tool runs each OpenStack service in an isolated container which presents a huge advantage for collecting metrics such as CPU, memory, and network utilization. Indeed, in addition to isolation, container technologies offer fine-grained resource management and monitoring capabilities [START_REF] Xavier | Performance evaluation of containerbased virtualization for high performance computing environments[END_REF]. This means that it is possible to collect the current resource usage and performance information, whatever the container runs through a standard API. This feature lets ENOS implement a generic metrics collection mechanism that stands for every OpenStack service.

Under the hood, ENOS relies on cAdvisor (see II-D) to implement this generic collection mechanism. Actually, ENOS deploys a monitoring stack that includes cAdvisor for CPU/memory/network usage and collectd for some service specific information such as the number and the type of requests performed on the database.

At the end of this phase, OpenStack has been deployed as defined by the ENOS configuration file. The next phase is the one that performs the execution of benchmarks.

3) enos bench: Running Performance Evaluation Phase: The bench phase runs benchmarks to stress the platform. By default, ENOS comes with Rally and Shaker frameworks. However, the ENOS abstractions allow end-users to plug any custom benchmarks.

a) Core benchmarking tools: ENOS unifies the description and execution of the workloads to run against a given OpenStack installation. A workload in ENOS is composed of generic scenarios that will be run in sequence. Given a set of parameters to apply to a generic scenario, the engine will run a concrete scenario. Workloads are described in dedicated files, as shown in Figure 5 where two Rally scenarios are included. In this workload definition, ENOS will run six concrete scenarios: four "boot and delete" which parameters are in the cartesian product of the top level arguments and two "boot and list", because local arguments shadow global ones. Workloads based on Shaker follow the same pattern.

b) Custom benchmarking tools: To facilitate the extension of the benchmark phase with new benchmarking frameworks that won't fit the previous workload definition, ENOS exposes the list of resources together with the name of OpenStack services deployed over each resource. This way, one can easily develop an ah-doc solution that, using the resources list, deploys and runs a another benchmark framework.

4) enos inspect: Analysing the Evaluation Phase: The inspect phase generates all components needed for the analyses of the performance evaluation.

Metrics gathering is twofold. First, ENOS collects general metrics (CPU/memory usage, network utilization, opened sockets . . . ). Second, it is able to store specific statistics offered by the benchmarking suite used. The former relies on a set of agents whose role is to send metrics to a collector. The latter is specific to the benchmarking suite that is executed and occurs during the inspect phase. Similarly to the previous section, integrating custom benchmarking tools may require extending ENOS to retrieve the relevant reports.

ENOS allows general metrics to be observed in realtime during the experiment. Preconfigured dashboards are indeed accessible through a Web interface. ENOS's inspect gather a larger source of information since they include configuration files, logs of OpenStack services, all the collected metrics and all the reports generated by the benchmarking suite used. ENOS can then build a virtual machine image embedding all these data and tools to allow post-mortem exploration.

IV. EXPERIMENTS

As explained in the previous sections, ENOS enables researchers to easily evaluate the performance of distinct OpenStack deployments in a reproducible manner. Thus, it can be used to compare the collected metrics in order to both identify limitations and validate proposals for improvement. We propose here two experiments that depict how ENOS can be used in these directions.

The first experiment compares control plane evaluations while the number of compute nodes scales up to 1,000. This kind of evaluation illustrates how operators and developers can use ENOS to identify limiting services at both coarse and fine grain (e.g., general services such as RabbitMQ as well as sub-services such as nova-conductor) through the exploration of general metrics.

The second experiment shows that ENOS can be used to compare data-plane evaluations in order to help rally:

args: concurrency: -25 -50 times:

-10 -20 scenarios:

-name: boot and delete file: nova-boot-delete.yml -name: boot and list file: nova-boot-list.yml args: times: -20 Both experiments have been executed on the paravance cluster of the Grid'5000 [START_REF] Balouek | Adding Virtualization Capabilities to the Grid'5000 Testbed[END_REF] testbed. This cluster is composed of 72 nodes, each featuring two octa-core Intel Xeon E5-2630v3 CPUs @ 2.4 GHz, 128 GB of RAM, two 600 GB HDD and two Gigabit Ethernet NICs. OpenStack was based on the Mitaka release.

A. Control-plane performance study

In this section, we demonstrate how ENOS can be used to perform control-plane evaluations. This exploratory evaluation studies the effect of the number of compute nodes on the performance of OpenStack services in an idle state, i.e., when there is no request from end-users/administrators. In other words, our goal is to analyze the resources that are consumed by the OpenStack services themselves.

For this experiment, we deploy an OpenStack cloud multiple times on Grid'5000 with ENOS and vary the number of compute nodes from 100 to 1,000 between two deployments. We use the "fake driver" capability of Nova to deploy 50 nova-compute containers per physical node, thus allowing to reach 1,000 fake compute-nodes with 20 physical machines. Note that two other physical machines were used to host Control and Neutron groups respectively. The fake driver is a hypervisor that does not bootstrap VMs but performs the same routine tasks to maintain the state of its localfake-instances; thus, its use has no effect on the control- plane compared to an actual hypervisor. In addition to the compute nodes, the deployed cloud includes a control and a network node. The former hosts most OpenStack components such as the database and the monitoring services while the latter hosts the Neutron services.

Once the deployment is over, metrics are collected for one hour without performing any user or administrative request. ENOS enables us to individually inspect these metrics for each service. Table I and II present respectively the CPU and RAM consumption of representative services during this one-hour idle period. The observed CPU consumption is very small, except for the Nova Conductor service that interfaces all Nova services with the MariaDB database. This information is valuable for the OpenStack community as it clearly shows that there is room for improvement to reduce the consumption of the nova-conductor service (for 1,000 nodes, the current code requires the equivalent of 7 cores while the compute nodes are idle). For the RAM consumption, an important increase is ob- served for RabbitMQ, another supporting service that is heavily used for communications between services like nova-conductor and MariaDB. Table III presents the maximum number of connections for RabbitMQ and MariaDB. It clearly shows that the increased RAM usage is linked to network usage: the number of open connections on the RabbitMQ container grows indeed at the same rate as memory usage. Moreover, the number of connections opened to RabbitMQ can be explained by the fact that, even in idle state, OpenStack is maintaining several permanent connections with each Nova and Neutron agents. This leads to the conclusion that RabbitMQ will be hard to scale beyond this limit without reviewing the communication patterns in use. To further explain this increase in resource usage, we export from ENOS the number of database queries performed each second by MariaDB. The average of SELECT and UPDATE queries are presented in Table IV, while the number of SELECT queries performed each second for the onehour period is plotted on Figure 6. From the table, we observe that the average number of queries increases linearly with the number of nodes. More importantly, from the figure, we observe periodic spikes. These spikes are due to periodic tasks run by Nova services and Neutron agents. They are indeed reporting periodically their states in the database. UPDATE queries follow the same pattern but aren't plotted here. Note that the reporting interval is configurable and may be decreased in the configuration file at the cost of decreasing the consistency of the state stored in the database.

This evaluation demonstrates how OpenStack can be studied with ENOS as a black-box and how complex mechanisms involving multiple services can be explored.

B. Data-plane performance study with shaker

In this section, we illustrate ENOS's ability to conduct data-plane evaluations. This evaluation could have taken place some time ago when a new feature called Dis-tributed Virtual Routing (DVR) 4 was introduced in Neutron. From a high-level perspective, it enables Neutron to distribute routing across a set of hosts instead of being centralized on a specific node. DVR removes a single point of failure and alleviates the load on the network. From the developer point of view, such an experiment is crucial as it validates intricate implementation choices that have a direct impact on application performance.

We deploy OpenStack using 3 physical machines hosting respectively the Control, Network and Compute group. For this experiment, we use the Shaker benchmark named L3 East-West Dense. This benchmark spawns pairs of instances on the same physical host. Paired instances are put in different tenant networks, forcing network traffic to be routed by the infrastructure. OpenStack was deployed using ENOS on Grid'5000 alternatively with and without DVR respectively. The latter uses a single neutron-l3-agent hosted on a dedicated node (the network node) whereas the former takes advantage of the distribution of neutron-l3-agent on all the compute nodes. In both cases, we are interested in the data transfer rate between instances in the same pair while increasing the number of simultaneous active pairs (the concurrency).

Figure 7 depicts the transfer rate observed by individual instances, while the concurrency increases. Inter networks traffic clearly shows better performance when enabling DVR. Figure 8 reports the network traffic observed by the network node. On the one hand, when DVR is disabled, we observe that the network node receives all inter-tenant data transfers since it acts as a gateway for all the tenant-networks. Note that for each concurrency value, three series of data transfer measurements were made, explaining the spikes seen on the graph. On the other hand, when DVR is enabled, no more traffic is seen on this host.

We underline that ENOS greatly eases this kind of study since it collects data from multiple sources. Here data from the application benchmark and general metrics on the network node were automatically gathered.

V. RELATED WORK

The evaluation of OpenStack can be achieved either by the control or the data plane side.

As previously highlighted, several control plane evaluations have been performed [START_REF] Litvinski | Openstack scheduler evaluation using design of experiment approach[END_REF], [START_REF] Schmidt | Performance analysis of an openstack private cloud[END_REF]. However, they have been investigated using ad-hoc frameworks that prevent researchers to reproduce them. For instance, the authors of [START_REF] Litvinski | Openstack scheduler evaluation using design of experiment approach[END_REF] reviewed the behavior of the Nova Scheduler using specific and deprecated tools [START_REF] Krash | Reproducible CPU Load Generation for Linux[END_REF] components were compared in [START_REF] Callegati | Performance of network virtualization in cloud computing infrastructures: The openstack case[END_REF] using deprecated tools named Rude and Crude [START_REF] Crude | Real-time UDP Data Emitter & Collector for RUDE[END_REF]. Additionally, an ad-hoc framework based on perl and bash scripts has been proposed for network evaluation [START_REF] Karacali | Experiences evaluating openstack network data plane performance and scalability[END_REF]. Many studies have investigated the challenges of evaluating complex infrastructures such as distributed systems [START_REF] Buchert | A survey of general-purpose experiment management tools for distributed systems[END_REF] and IaaS clouds [START_REF] Iosup | IaaS Cloud Benchmarking: Approaches, Challenges, and Experience[END_REF]. Four challenges can be extracted from these studies: the ease of experimenting, the replicability (replay an experiment in the same conditions), the reproducibility (experiments can be launched on different infrastructures), the control of the parameter space and the experiment scalability. By embracing the Experimentation-as-Code vision and by choosing a pluggable design, ENOS should be able to offer a sustainable method for evaluating OpenStack by tackling these four challenges. Although the current code base only integrate two benchmark suites, namely Rally [START_REF]Rally, the OpenStack benchmarking tool[END_REF] and Shaker [START_REF]Distributed data-plane performance testing tool for OpenStack[END_REF], attractive tools such as PerfKit [START_REF]PerfKit Benchmarker[END_REF] and CloudBench [START_REF] Silva | Cloudbench: Experiment automation for cloud environments[END_REF] can be easily invoked to provide a large panel of synthetic and real-world workloads.

Among the different actions we know, Browbeat [START_REF]Browbeat -Performance monitoring and testing of OpenStack[END_REF] is the only OpenStack project whose goals closely match those of ENOS. It provides indeed a set of Ansible playbooks to run workloads on OpenStack and to analyze the result metrics. The workloads are generated by Rally, Shaker or PerfKit Benchmarker [START_REF]PerfKit Benchmarker[END_REF] and the metric visualization is done by services such as collectd, grafana or ELK [START_REF] Elasticsearch | [END_REF]. However, compared to ENOS, Browbeat requires that the operator sets a functional OpenStack with TripleO [START_REF] Tripleo | [END_REF] (i.e., OpenStack On OpenStack). TripleO is an OpenStack project to deploy two clouds. The first one is a deployment cloud (named undercloud), and is used to set up tunable workload overclouds on which Browbeat runs its benchmarks. This deployment phase adds a significant difficulty for researchers who desire to evaluate OpenStack releases. Moreover, it constraints the researcher to evaluate OpenStack on top of an OpenStack cloud whereas ENOS is testbed agnostic.

VI. CONCLUSION

With a community that gathers more than 5,000 people twice a year at the single location, the Open-Stack software suite has become the de facto opensource solution to operate, supervise and use Cloud Computing infrastructures. While it has been mainly supported by key companies such as IBM, RedHat and more recently Google, we claim that distributed computing scientists should now join the effort to help the OpenStack consortium address the numerous technical and scientific challenges related to its scalability and reliability. Similarly to what our scientific community has been doing for Linux, the OpenStack software suite should benefit from scientific guidance. However, diving into OpenStack and understanding its intricate internal mechanisms is a tedious and sometimes too expensive task for researchers.

To allow academics, and more generally the Open-Stack consortium to identify issues, propose countermeasures, and validate code improvements, we presented in this paper the ENOS framework. Thanks to container technologies and the use of "off-the-shelf" benchmarks, ENOS is the first holistic approach for evaluating Open-Stack in a controlled and reproducible way. Two experiments illustrated how ENOS can address control-plane and data-plane evaluations. The first one focused on analyzing how an idle OpenStack behaves at different scales. This experiment helps in identifying services which will become bottlenecks (e.g., RabbitMQ, nova-conductor) with a large number of compute nodes. The second evaluation shows how ENOS can be used to validate the performance of a specific feature, namely Distributed Virtual Routing. ENOS proved that enabling this new feature significantly improved the performance of intertenants network communications.

But the value of ENOS can be even bigger when integrated with a Continuous Integration system; in this case, ENOS can automatically execute performance evaluations of individual code contributions in order to prevent code that has a negative impact on performance to be merged into the upstream repository.

We emphasize this work enabled us to exchange with the OpenStack Foundation and take part in different discussions/working groups. As an example, we are using ENOS to conduct several experiments in the context of the OpenStack performance working group [START_REF]OpenStack Performance Documentation[END_REF]. First, we are evaluating different message bus solutions that can replace the current RabbitMQ solution that does not scale well. Second, we are performing several experiments to identify network requirements in the case of WANwide infrastructures. Conducting these experiments is a first interest for telcos that target the deployment of Fog and Edge Computing infrastructures.

As mid-term actions, we plan to extend ENOS with the new OpenStack Profiler tool. This will help researchers investigating in details performance issues by analyzing the execution traces of any OpenStack functionality.

Finally, we would like to highlight that OpenStack is only one example of such large software projects that can benefit from the involvement of our scientific community. The approach pursued by ENOS can be easily extended to other complex software stacks. The only requirement is to get containerized versions of said software. This trend is expected to grow, looking for instance at the Docker Hub repository [START_REF] Docker | [END_REF].

Fig. 1 .

 1 Fig. 1. OpenStack Overview

  shared services. The second figure shows the historical core services of OpenStack. Briefly, Keystone provides service discovery and client authentication. Horizon provides a web-based user interface. Nova provides on-demand access to compute resources (i.e., VMs).

Fig. 2 .

 2 Fig. 2. OpenStack core-services

Fig. 4 .

 4 Fig. 4. ENOS Resources Description Examples

Fig. 5 .

 5 Fig. 5. Benchmark definition example

  are well known from the OpenStack community. The first one presents the general vision of OpenStack with the three expected capabilities of IaaS platforms: Compute, Network and Storage. Applications at the top can request compute, network and storage resources through a high-level API. OpenStack components in the middle layer communicate through

	Horizon		Nova compute Neutron agents Nova	Nova compute Neutron agents Neutron	AMQP Nova compute Neutron agents RDBMS
				Nova
	Nova compute AMQP Neutron agents Nova compute Neutron agents RDBMS		Nova compute Neutron agents Nova	Neutron Nova compute Neutron agents Neutron	AMQP Nova compute Neutron agents RDBMS
	Nova compute Glance Neutron agents	WAN	Nova	Nova
			Neutron	Neutron
		(d) multi regions
	Fig. 3. OpenStack deployment scenarios. Gray squares represent control nodes whereas white squares correspond to the compute nodes (i.e.,
	the nodes that host VMs).

TABLE I AVERAGE

 I CPU USAGE OF OPENSTACK SERVICES WITH VARYING THE NUMBER OF COMPUTE NODES (IN NUMBER OF CORES).

	Nb. of compute nodes	100	200	500	1,000
	Nova Conductor	2.47	2.47	2.45	2.47
	Neutron server	419	420	359	441
	HAProxy	6.27	6.32	7.04	8.71
	RabbitMQ	1,628	2,580	5202	11,520
	MariaDB	502	546	570	594

TABLE II MAXIMUM

 II RAM USAGE OF OPENSTACK SERVICES WITH VARYING THE NUMBER OF COMPUTE NODES IN MEGABYTES.

TABLE IV AVERAGE

 IV NUMBER OF SQL QUERIES PER SECOND ON MARIADB Fig. 6. Number of SQL queries per second executed by MariaDB with varying the number of compute nodes. Horizontal lines show the average for each series.

		500 600						100 200 500 1,000
	SELECT queries	300 400
		200
		100
		0 0	500	1,000	1,500 Time (seconds) 2,000	2,500	3,000	3,500
								DURING THE IDLE PERIOD

  on Nova Computes. Data plane evaluations have suffered from the same problem. For instance, different OpenStack network

	Mbit/s	300 400 500 600 700 800				Transfer rate without DVR	tcp download tcp upload
			200
			100
				0
								Transfer rate with DVR
	8,000 10,000							tcp download tcp upload
	Mbit/s	4,000 6,000
		2,000
				0	0	1	2	3	4 Concurrency 5	6	7	8	9
	Fig. 7. Effect of DVR on the transfer rate observed by instances
		1,000 1,200 1,400			Traffic rate on network node without DVR	eth0 tx eth0 rx
	Mbit/s		600 800
			400
			200
				0 50 60 70			Time (s) Traffic rate on network node with DVR	eth0 tx eth0 rx
		Mbit/s	30 40
				20
				10
				0	0		500		1,000	Time (s)	1,500	2,000	2,500
	Fig. 8. Effect of DVR on the network traffic observed on the network
	node

https://blueprints.launchpad.net/neutron/+spec/neutron-ovs-dvr

Publisher Inria Domaine de Voluceau -Rocquencourt BP 105 -78153 Le Chesnay Cedex inria.fr ISSN 0249-0803

ACKNOWLEDGMENTS

Most of the materials presented in this article are available on the Discovery initiative website. Supported by the Inria Project Lab program, Discovery is an Open-Science Initiative aiming at implementing a fully decentralized IaaS manager: http://beyondtheclouds.github.io. Experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations.

https://github.com/BeyondTheClouds/


Antonin Gilles

Patrick Gioia
email: patrick.gioia@b-com.com.

Rémi Cozot
email: remi.cozot@b-com.com.

Luce Morin
email: luce.morin@b-com.com.

Complex modulation computer-generated hologram by a fast hybrid point-source/wave-field approach

Keywords: Computer-Generated Hologram, Color holography, Real-time holography, Threedimensional imaging

We propose a fast Computer-Generated Hologram (CGH) computation method based on a hybrid pointsource/wave-field approach. Whereas previously proposed methods tried to reduce the computational complexity of the point-source or the wave-field approaches independently, our method uses the two approaches together and therefore takes advantages from both of them. The algorithm consists of three steps. First, the 3D scene is sliced into several depth layers parallel to the hologram plane. Then, for each layer, we compute the complex wave scattered by this layer either using a wave-field or a point-source approach according to a threshold criterion on the number of points within the layer. Finally, we sum up the complex waves scattered by all the depth layers in order to obtain the final CGH. Experimental results reveal that this combination of approaches does not produce any visible artifact and outperforms both the point-source and wave-field approaches.

Introduction

Holography is often considered as the most promising 3D visualization technology, since it can provide the most authentic and natural three-dimensional illusion to the naked eye. Indeed, it provides complete human depth cues without the need for special viewing devices and without causing eye-strain [START_REF] Schnars | Digital Holography: Digital Hologram Recording, Numerical Reconstruction, and Related Techniques[END_REF]. Over the past decades, several methods have been proposed to generate holograms by computer calculation. Using these methods, it is possible to obtain Computer-Generated Holograms (CGH) of synthetic or existing scenes by simulating the propagation of light scattered by the scene towards the hologram plane. CGH computation techniques usually sample 3D scenes by a set of primitives and calculate light propagation as the sum of complex light waves scattered by each of the primitives. Commonly used primitives include points (point-source approach) and planar segments (wave-field approach).

The point-source approach samples 3D scenes by a collection of self-luminous points, and calculates complex wave scattered by each of the points using the monochromatic spherical light wave equation. This approach is very flexible and does not impose any restriction on the scene geometry. However, its complexity is very high since it requires one calculation per point of the scene per pixel of the hologram. Moreover, to produce shapes that appear solid and continuous, the scene needs to be sampled at very high densities, making the CGH computation pro-1 hibitively slow. In order to reduce the computational complexity, several methods have been proposed, including geometric symmetry [START_REF] Juárez-Pérez | Nonredundant calculations for creating digital fresnel holograms[END_REF], look-up tables [START_REF] Lucente | Interactive computation of holograms using a look-up table[END_REF][START_REF] Kim | Effective generation of digital holograms of threedimensional objects using a novel look-up table method[END_REF], interframe and interline redundancy reduction [START_REF] Kim | Fast computation of hologram patterns of a 3d object using run-length encoding and novel look-up table methods[END_REF][START_REF] Kim | Accelerated computation of hologram patterns by use of interline redundancy of 3-d object images[END_REF], difference and recurrence formulas [START_REF] Yoshikawa | Fast computation of fresnel holograms employing difference[END_REF][START_REF] Matsushima | Recurrence formulas for fast creation of synthetic three-dimensional holograms[END_REF], image holograms [START_REF] Yamaguchi | Real-time image plane full-color and full-parallax holographic video display system[END_REF][START_REF] Yoshikawa | Real-time generation of full color image hologram with compact distance look-up table[END_REF], wave-front recording planes [START_REF] Shimobaba | Simple and fast calculation algorithm for computer-generated hologram with wavefront recording plane[END_REF][START_REF] Shimobaba | Rapid calculation algorithm of fresnel computergenerated-hologram using look-up table and wavefront-recording plane methods for three-dimensional display[END_REF][START_REF] Weng | Generation of real-time large computer generated hologram using wavefront recording method[END_REF], using GPU hardware [START_REF] Ahrenberg | Computer generated holography using parallel commodity graphics hardware[END_REF][START_REF] Masuda | Computer generated holography using a graphics processing unit[END_REF], and special purpose hardware [START_REF] Ito | Special-purpose computer HORN-5 for a real-time electroholography[END_REF][START_REF] Ichihashi | HORN-6 special-purpose clustered computing system for electroholography[END_REF].

The wave-field approach samples 3D scenes by a collection of self-luminous planar segments, and computes complex wave scattered by each of the segments using the angular spectrum of plane waves [START_REF] Goodman | Introduction to Fourier Optics[END_REF]. The computation of the angular spectrum of plane waves involves the use of the Fast Fourier Transform (FFT) algorithm twice, and is therefore more time-consuming than the computation of the spherical light wave scattered by a single point. However, complex waves scattered by scene points located within a single planar segment are calculated all at once using the angular spectrum of plane waves. Therefore, this approach is more efficient than the point-source approach when objects in a scene consist of large planar segments containing many points. However, when the scene geometry contains complex shapes, a large number of small planar segments containing only one or a few points are needed to sample it, making the wave-field approach less efficient than the point-source approach. In order to reduce the computation burden, several methods have been proposed, including the use of analytic expression of the angular spectrum [START_REF] Ahrenberg | Computer generated holograms from three dimensional meshes using an analytic light transport model[END_REF][START_REF] Kim | Mathematical modeling of triangle-meshmodeled three-dimensional surface objects for digital holography[END_REF][START_REF] Sakata | Fast computation method for a fresnel hologram using three-dimensional affine transformations in real space[END_REF][START_REF] Liu | A fast analytical algorithm for generating CGH of 3d scene[END_REF][START_REF] Liu | Highspeed full analytical holographic computations for true-life scenes[END_REF], and color space conversion [START_REF] Shimobaba | Acceleration of color computergenerated hologram from three-dimensional scenes with texture and depth information[END_REF][START_REF] Shimobaba | Calculation reduction method for color digital holography and computer-generated hologram using color space conversion[END_REF].

In this paper, we propose a fast CGH computation method based on a hybrid point-source/wavefield approach. Whereas previously proposed methods tried to reduce the computational complexity of the point-source or the wave-field approaches independently, our method uses the two approaches together and therefore takes advantages from both of them. Section 2 gives a detailed description of our method, Section 3 gives experimental results, and Section 4 concludes this paper.

Overview

Figure 1 shows the scene geometry and coordinate system used by the proposed method. The coordinate system is defined by (x, y, z) so that the hologram lies on the (x, y, 0) plane. The 3D scene is treated as a set of N z depth layers parallel to the hologram plane and located between z min and z max . The hologram is sampled on a regular 2D grid of resolution N x × N y with a sampling pitch p. Figure 2 shows the overall block-diagram of the proposed method, which consists of three steps. First, the 3D scene is sliced into N z depth layers parallel to the hologram plane. Then, for each layer d, if the number of points N d within the layer exceeds a maximum value N d,max (selection criterion which will be determined in section 2.4), we compute the complex wave scattered by this layer using a wave-field approach. Otherwise, if N d is smaller than N d,max , the method calculates the complex wave scattered by this layer using a pointsource approach. Finally, the method sums up the complex waves scattered by all the depth layers in order to obtain the final CGH. Afterwards, the scene image can be reconstructed from the computed CGH pattern. layer towards the hologram plane is computed using a wave-field approach. To this end, we use the angular spectrum of plane waves [START_REF] Goodman | Introduction to Fourier Optics[END_REF], which is given by

Wave-field approach

U w d (x, y) = F -1 F A d (x, y)e jφ d (x,y) × e -j2π √ λ -2 -u 2 -v 2 z d , (1)
where A d (x, y) and φ d (x, y) are the amplitude and phase of the (x, y) point in layer d; λ is the wavelength of light, u and v are the spatial frequencies, z d is the depth of layer d, and F and F -1 are respectively the forward and inverse Fourier Transform. These transforms can be computed using the Fast Fourier Transform algorithm (FFT). In order to render a diffusive scene, the phase φ d (x, y) is set to a random value. Finally, the complex wave U w scattered by all the layers whose number of points

N d exceeds N d,max
in the hologram plane is given by

U w (x, y) = Nz d=0 N d >N d,max U w d (x, y). (2)
In order to avoid one FFT per layer and therefore to speed-up the computation, the algorithm sums up the complex waves scattered by each layer directly in the frequency domain, and then inverse Fourier transforms the result to get U w , as proposed in [START_REF] Ahrenberg | Computer generated holograms from three dimensional meshes using an analytic light transport model[END_REF]:

Û w d (u, v) = F A d (x, y)e jφ d (x,y) e -j2π √ λ -2 -u 2 -v 2 z d , U w (x, y) = F -1        Nz d=0 N d >N d,max Û w d (u, v)        . ( 3
)

Point-source approach

When the number of points within layer d ∈ {0..N z } is smaller than N d,max , the complex wave scattered by this layer towards the hologram plane is computed using a point-source approach. The complex wave scattered by a point source i located within layer d is given by the angular spectrum of plane waves [START_REF] Goodman | Introduction to Fourier Optics[END_REF] as

U p d,i (x, y) = A i e jφi F -1 e -j2π √ λ -2 -u 2 -v 2 z d ⊗ δ(x -x i , y -y i ), ( 4
)
where A i and φ i are the amplitude and phase of the point, x i and y i its coordinates within the layer, and ⊗ is the convolution operator. In order to avoid interference between the points, the phase φ i is set to a random value. Convolving a function with a Dirac delta shifts it around the delta impulse. Therefore, if we know the inverse Fourier transform term in Eq. ( 4) beforehand, U p d,i can be computed simply by scaling this term with the point's amplitude and phase factor, followed by a shifting operation. In order to speed up the computation, we use a pre-calculated LUT, as proposed in [START_REF] Kim | Effective generation of digital holograms of threedimensional objects using a novel look-up table method[END_REF]. The LUT T (x, y, z) is pre-computed as

T (x, y, z) = F -1 e -j2π √ λ -2 -u 2 -v 2 z d h(x, y, z). (5)
h being an envelope function used to restrict the region of contribution of a given point source, equal to one within the region of contribution of the point and zero elsewhere. This function limits the spatial frequencies of the complex wave to avoid aliasing in the CGH.

According to the Nyquist Sampling Theorem, the maximum spatial frequency f max which can be represented with a sampling pitch p is given by f max = (2p) -1 . The grating equation [START_REF] Goodman | Introduction to Fourier Optics[END_REF] gives the relation between the maximum spatial frequency f max and the maximum diffraction angle θ as sin(θ) = λf max . Therefore, the region of contribution of a point source at depth z is given by its maximum radius R max by

R max = z tan(θ) = z tan arcsin λ 2p , (6)
as shown in Figure 3. The envelope function h can thus be defined as

h(x, y, z) = 1 if x 2 + y 2 < R max 0 otherwise. (7)
In order to limit its number of pixels, the LUT is pre-computed only within the circumscribing square of the region of contribution defined by the envelope function h. Therefore, the number of pixels N T,d of the LUT for depth z d is given by

N T,d = 2R max p 2 N T,d = 2z d p tan arcsin λ 2p 2 . ( 8
) CGH Point ( )
U p d (x, y) = N d i=1 A i e jφi T (x -x i , y -y i , z d ). (9)
Finally, the complex wave scattered by all the layers whose number of points N d is smaller than N d,max in the hologram plane is given by

U p = Nz d=0 N d <N d,max U p d (x, y). (10)

Determination of the selection criterion

The first step to implement the proposed method is to determine the value of N d,max . We call t p the time needed to compute the complex wave scattered by a layer at depth z d with N d luminous points using the point-source approach presented in Section 2.3, and t w the time needed to compute it using the wavefield approach presented in Section 2.2. Since the wave-field approach involves one complex multiplication per pixel and a Fourier transform, t w is linearly dependent on the number of pixels of the hologram N pix = N x × N y . The point-source approach involves one complex multiplication per pixel of the LUT per point within the layer, so t p is dependent on the number of pixels N T,d of the LUT for depth z d and on the number of points N d within the layer. t w and t p are experimentally found to be expressed by

t w (N pix ) = kN pix t p (N d , N T,d ) = a (bN T,d + c) 1 2 + dN T,d + e N d. (11)
We find the numerical values for the coefficients a, b, c, d, e and k in Eq. ( 11

In order to maximize the efficiency of our method, N d,max must be set such that

t p (N d,max , N T,d ) = t w (N pix ) ( 13
)
⇔N d,max = kN pix a (bN T,d + c) 1 2 + dN T,d + e . ( 14
)
3

Experimental results and discussion

The proposed method was implemented in C++/CUDA on a PC system employing an Intel Core i7-4930K CPU operating at 3.40 GHz, a main memory of 16 GB and an operating system of Microsoft Windows 8 as well as three GPUs NVIDIA GeForce GTX 780Ti.

For the experiments, we used the Middlebury's views and disparity maps datasets [START_REF] Scharstein | Learning conditional random fields for stereo[END_REF] as test scenes (Figures 4a and4b). From each view and disparity map pair, a 3D point cloud is extracted, where each point is given an amplitude proportional to its corresponding pixel value in the view image and a random phase. Since each disparity map is encoded as an 8bits gray level image, the extracted 3D point cloud is naturally sliced as a set of N z = 255 depth layers parallel to the CGH plane1 . The total number of points N scene within the point cloud is given by the number of pixels of the disparity map minus the number of unknown disparity pixels. The 3D point cloud is considered to be located between z min = -d and z max = d in front of the CGH plane, where 2d = 2cm is the depth extent of the scene. Finally, the CGH to be computed has a resolution of 4096 × 4096 with a sampling pitch p = 8, 1µm.

We compare our method with GPU implementations of two other methods: (1) the wave-field method proposed in [START_REF] Bayraktar | Method calculate the far field of three-dimensional objects for computer-generated holography[END_REF], which computes complex wave scattered by each layer using a wave-field approach, and (2) the point-source method proposed in [START_REF] Kim | Effective generation of digital holograms of threedimensional objects using a novel look-up table method[END_REF], which computes complex wave scattered by each layer using Figure (c) shows in blue the scene points whose complex waves are computed by our method using the wave-field approach and in yellow the scene points whose complex waves are computed using the pointsource approach. On the second line, scene images numerically reconstructed from the CGH patterns generated by (d) the wave-field method, (e) the pointsource method, and (f) our method.

a point-source approach. We adapted both methods to produce colorful complex modulation CGH. Figure 4 shows the scene images numerically reconstructed from the CGH patterns of the scene "Moebius" generated by the wave-field method (Figure 4d), the pointsource method (Figure 4e), and our method (Figure 4f). Figure 4c shows in blue the scene points whose complex wave is computed by our method using the wave-field approach and in yellow the scene points whose complex wave is computed using the pointsource approach. As seen in Figure 4, our method does not produce any visible artifact, even at the boundaries between these two categories of points.

In order to evaluate the objective quality of the reconstructed images compared to the original view image, we used the Peak Signal-to-Noise Ratio (PSNR). The PSNR of the reconstructed images of the scene "Moebius" were found to be 21, 20dB, 21, 19dB, and Figure 5: (a) CGH computation time for a synthetic 3D scene using the wave-field method (in blue), the point-source method (in red), and our method (in green) depending on the number of scene points N . (b) CGH computation time reduction using our method depending on the number of scene points N .

21, 20dB for the wave-field method, the point-source method and our method, respectively. These results show that our method does not reduce the quality the reconstructed scene images compared to the conventional point-source and wave-field methods. It must be noted that unlike the original view image, the numerically reconstructed images have a low depth of field due to the reconstruction technique used. As a consequence, the PSNR of the reconstructed images are found to be below 30dB. Additionally, we compared the CGH pattern generated by our method to those generated by the wave-field and point-source methods using the PSNR. The PSNR of the CGH pattern generated by our method was found to be 40, 59dB and 42, 88dB, compared to the wave-field and point-source methods, respectively.

In Figure 5a, we compared the CGH computation time of the wave-field method (in blue), the pointsource method (in red), and our method (in green) depending on the number of scene points N using views and depth maps pairs of a single synthetic 3D scene with different resolutions. As shown on Figure 5a, while the computation time of the pointsource method increases linearly with the number of scene points, the computation time of the wavefield method does not depend on it. Therefore, while the point-source method is faster than the wave-field method for scenes with few points, the wave-field method is still more efficient than the point-source method for scenes with a large number of points. By combining these two approaches, our method takes advantages from both of them and is therefore always the most efficient.

Figure 5b shows the reduction of the CGH computation time using our method depending on the number of scene points N . As seen in Figure 5b, our method allows the CGH computation time to be reduced by a percentage that increases quickly until N passes a threshold N t , and then decreases slowly. This threshold corresponds to the number of scene points for which the computation time of the point-source method reaches the computation time of the wave-field method. The value of N t depends on the number of hologram pixels N pix = N x × N y , on the number of depth layers N z , and on the distance between the scene and the CGH plane. As shown on Figure 5b, the CGH computation time is reduced by 68% using our method when the number of scene points is equal to N t . Moreover, our method outperforms both the point-source and wavefield methods even when the number of scene points is higher than N t . In addition to the results shown here, we have conducted many tests on both real and synthetic scenes with different number of hologram pixels and depth layers. A reduction of over 65% of the computation time has been reached for each test scene when the number of scene points is equal to N t . These experimental results confirm the performance superiority of our method over the conventional point-source and wave-field methods in terms of computation time.

Conclusion

In this paper, we proposed a fast Computer-Generated Hologram (CGH) computation method based on a hybrid point-source/wave-field approach. The algorithm consists of three steps. First, the 3D scene is sliced into several depth layers parallel to the hologram plane. Then, for each layer, if the number of points within the layer exceeds a determined max-imum value, we compute the complex wave scattered by this layer using a wave-field approach. Otherwise, we compute the complex wave scattered by this layer using a point-source approach. Finally, we sum up the complex waves scattered by all the depth layers in order to obtain the final CGH. Experimental results reveal that the CGH computation time has been reduced up to 68% compared to the conventional point-source and wave-field methods without producing any visible artifact. This confirms the performance superiority of our method over the conventional point-source and wave-field methods in terms of computation time.

Our method does not take into account occlusions between objects in the scene, so in future study we plan to improve this method in order to handle scene occlusions properly.

Figure 1 :

 1 Figure 1: Scene geometry and coordinate system used by the proposed method

←←Figure 2 :

 2 Figure 2: Block-diagram of the proposed method

Figure 3 :

 3 Figure 3: Region of contribution of a given point source

  ) using the Gnuplot implementation of the nonlinear least-squares Levenberg-Marquardt algorithm [26]: a = 1, 11.10 -5 b = 0, 59 c = 1, 0 d = 2, 59.10 -8 e = 4, 78.10 -4 k = 5, 54.10 -7

Figure 4 :

 4 Figure 4: (a) Intensity view and (b) disparity map of the test scene "Moebius" from Middlebury's dataset.Figure(c) shows in blue the scene points whose complex waves are computed by our method using the wave-field approach and in yellow the scene points whose complex waves are computed using the pointsource approach. On the second line, scene images numerically reconstructed from the CGH patterns generated by (d) the wave-field method, (e) the pointsource method, and (f) our method.

The 8-bits pixels in the disparity maps can have

different values, but in this dataset, the 0 value is used to encode an unknown disparity. Points with unknown disparity are not extracted from the disparity maps.

* This work has been achieved within the Institute of Research and Technology b<>com, dedicated to digital technologies. It has been funded by the French government through the National Research Agency (ANR) Investment referenced ANR-A0-AIRT-07.


Antonin Gilles

Patrick Gioia
email: patrick.gioia@b-com.com.

Rémi Cozot
email: remi.cozot@b-com.com.

Luce Morin
email: luce.morin@b-com.com.

Computer generated hologram from Multiview-plus-Depth data considering specular reflections

Keywords: Three-dimensional imaging, Holography, Computer-Generated Hologram

Introduction

Holography is often considered as the most promising 3D visualization technology, since it can provide the most realistic and natural depth illusion to the naked eye. Indeed, it provides all the human depth cues without the need for special viewing devices and without causing eye-strain [START_REF] Schnars | Digital Holography: Digital Hologram Recording, Numerical Reconstruction, and Related Techniques[END_REF]. In the last decades, several methods have been proposed to generate holograms by computer calculation. Using these methods, it is possible to obtain Computer-Generated Holograms (CGH) of synthetic or existing scenes by simulating the propagation of light scattered by the scene towards the hologram plane.

Thanks to its attractive visualization properties, CGH may replace the stereoscopic technique in the future. However, in order to be successful, CGH computation techniques should meet several requirements. First, CGH computation methods should provide all the human depth cues in order to produce a strong depth illusion. Second, the displayed 3D scene should be as realistic as possible, which means that occlusions as well as shading should be accurately reproduced. Last but not least, in order to have application in the field of videoconferencing or telepresence systems, computation techniques should be able to produce CGH of real scenes at interactive rates.

Meeting all these requirements is still very challenging for state-of-the-art methods. These methods can be classified depending on the type of input data they use to compute the hologram: synthetic 3D scenes or multiview data.

Hologram computation from synthetic 3D scenes

In order to compute a hologram from a synthetic 3D scene, CGH computation techniques usually sample it by a set of primitives and calculate the hologram as the sum of complex light waves scattered by each primitive. Commonly used primitives include points (point-source approach) and planar segments (polygon-based approach).

In the point-source approach, 3D scenes are populated by a collection of self luminous points, and the hologram is computed as the sum of light waves scattered by the points towards the hologram plane [START_REF] Brown | Complex Spatial Filtering with Binary Masks[END_REF]. The complex wave scattered by each point is computed using the monochromatic spherical light wave equation. This approach is very flexible and does not impose any restriction on the scene geometry. Moreover, Computer Graphics (CG) shading techniques, such as Phong shading [START_REF] Bui Tuong | Illumination for Computer Generated Pictures[END_REF], can be integrated during CGH computation in order to produce photo-realistic effects [START_REF] Ichikawa | CGH calculation with the ray tracing method for the Fourier transform optical system[END_REF][START_REF] Kurihara | Speckle-free, shaded 3d images produced by computer-generated holography[END_REF]. However, to produce shapes that appear solid and continuous, the scene needs to be sampled at very high densities, making the CGH computation prohibitively slow.

In the polygon-based approach, 3D scenes are populated by a collection of self luminous polygons, and the hologram is computed as the sum of light waves scattered by the polygons towards the hologram plane [START_REF] Leseberg | Computergenerated holograms of 3-D objects composed of tilted planar segments[END_REF]. The complex wave scattered by each polygon is computed using propagation formulas such as the Angular Spectrum diffraction [START_REF] Goodman | Introduction to Fourier Optics[END_REF]. The computation of the Angular Spectrum diffraction involves the use of the Fast Fourier Transform (FFT) algorithm twice, and is therefore more time-consuming than the computation of the spherical light wave scattered by a single point. However, complex waves scattered by scene points located within a single planar segment are calculated all at once using the Angular Spectrum diffraction. As for the point-source approach, CG shading techniques can be integrated during CGH computation in order to produce photorealistic effects. These include Blinn and Torrance Sparrow reflection models [START_REF] Yamaguchi | Computer-generated holograms considering background reflection on various object shapes with reflectance distributions[END_REF], Finite-difference timedomain (FDTD) method [START_REF] Ichikawa | Calculation method of reflectance distributions for computer-generated holograms using the finitedifference time-domain method[END_REF] and Phong reflection model [START_REF] Nishi | Rendering of specular surfaces in polygon-based computer-generated holograms[END_REF].

All these methods present the common advantage of being able to produce CGHs of accurately shaded scenes with all the human depth cues. However, they require the input scene to be accurately modeled, with the position of light sources and materials composing each scene object fully described. As a consequence, these methods cannot be easily applied to real existing scenes.

Hologram computation from multiview data

In order to compute CGHs of real existing scenes, it is possible to use multiview data. Two approaches are commonly used for CGH computation from multiview data: the multiviewpoint projection based methods and the holographic stereogram approach. Multi-viewpoint projection based methods generate holograms by first acquiring two-dimensional (2D) projections of a 3D scene using a 2D camera array or by shifting a single camera along the vertical and horizontal axis. The number of acquired images should be equal to the number of hologram pixels. Each hologram pixel is then computed by multiplying its corresponding projection image by a given complex point spread function (PSF) and by summation of the resulting inner product [START_REF] Natan | Review of three-dimensional holographic imaging by multiple-viewpoint-projection based methods[END_REF]. Occlusions as well as shading of the scene are naturally recorded in the acquired projections, and the resulting CGH is able to provide all the human depth cues. However, since each projection image yields a single hologram pixel, these methods require a huge number of images to be acquired, especially for high resolution CGHs. In order to reduce the number of required projections, several methods have been proposed. These include Horizontal Parallax Only (HPO) holograms [START_REF] Li | Computer-Generated Holograms of Three-Dimensional Realistic Objects Recorded Without Wave Interference[END_REF], view synthesis algorithms [START_REF] Katz | Synthesizing computer generated holograms with reduced number of perspective projections[END_REF], in which the computational complexity is increased, and integral imaging capture systems [START_REF] Mishina | Calculation of holograms from elemental images captured by integral photography[END_REF][START_REF] Natan | Integral holography: white-light singleshot hologram acquisition[END_REF], which make a trade-off between the number of projections and their resolution.

The holographic stereogram approach is another class of CGH computation methods which use multiview data as input. In this approach, the hologram is spatially partitioned into several holographic elements, called hogels. The hogels angularly multiplex the 2D projections of the scene towards the corre-sponding viewing directions. This approach is computationally very efficient and does not require as many projection images as for multi-viewpoint projection based methods. Moreover, occlusions and shading of the scene are naturally recorded in the multiplexed 2D views. However, since each hogel only corresponds to a 2D parallax view, the accommodation cue is sacrificed, which strongly limits the naturalism of the perceived 3D effect. Several methods have been proposed to improve the depth illusion provided by the holographic stereogram, including phase-added stereograms [START_REF] Yamaguchi | Phase-added stereogram: calculation of hologram using computer graphics technique[END_REF][START_REF] Kang | Accurate phase-added stereogram to improve the coherent stereogram[END_REF], diffraction-specific coherent panoramagram [START_REF] Barabas | Diffraction specific coherent panoramagrams of real scenes[END_REF], and ray sampling plane techniques [START_REF] Wakunami | Occlusion culling for computer generated hologram based on ray-wavefront conversion[END_REF].

As discussed above, it is still very challenging to compute CGHs of real existing scenes which provide all the human depth cues and accurate shading of the scene with only a few projection images. In order to overcome these limitations, we propose a novel approach for CGH computation from Multiview-plus-Depth (MVD) data. A method for CGH computation from MVD data has been proposed in [START_REF] Senoh | Study of a holographic TV system based on multi-view images and depth maps[END_REF]. However, in this paper, authors did not take into account specular reflections, and the displayed scene was totally diffuse.

A preliminary version of the proposed method which can only render diffuse scenes is presented in Section 2, and the generalization of this method to scenes with specular reflections is described in Section 3. Finally, experimental results are analyzed in Section 4.

Hologram computation for a diffuse scene

In this section we limit ourselves to the rendering of diffuse scenes. The proposed method consists of three steps, which are detailed in the following. First, 2D plus depth projections of the scene are taken from different perspective viewpoints. Then, the 3D scene geometry is reconstructed from MVD data as a layered point-cloud. Finally, light scattered by the scene is numerically propagated towards the hologram plane and encoded into the final CGH.

Step 1: Multiview-plus-Depth capture

The first step of the proposed method is to capture 2D plus depth projections of the scene from different perspective viewpoints. Although other camera layouts may also be used to acquire the MVD data, the proposed method uses (2K x + 1) × (2K y + 1) cameras aligned in parallel on a 2D grid, as shown in Figure 1.

The coordinate system is defined by (x, y, z) so that the camera matrix lies on the (x, y, 0) plane and the 3D scene is located between z min and z max . We call d x and d y the distance between the cameras along the x and y axis, respectively, and f the focal length of the cameras. The acquired intensity views and depth maps pairs have resolution N x × N y and pixel pitch p. Finally, the cameras are numbered by (i, j), with i ∈ {-K x , ..., K x } and j ∈ {-K y , ..., K y }.

The projection matrix of camera (i, j) is therefore given by

P i,j =   α 0 u 0 αx i 0 α v 0 αy j 0 0 1 0   , (1)
where α = f /p is the focal length expressed in pixels, (x i , y j ) is the camera location and (u 0 , v 0 ) corresponds to the principal point pixel's coordinates, given by

x i = id x ; y j = jd y ; u 0 = Nx 2 ; v 0 = Ny 2 .
(2)

Step 2: Scene geometry reconstruction

The second step of the proposed method is to reconstruct the 3D scene geometry from the MVD data. We call I i,j and D i,j the intensity and depth images captured by camera (i, j). Since D i,j is encoded as an 8-bits gray level image, each pixel (u, v) can be projected back to a 3D point whose coordinates are given by

  x i,u y j,v z d   =   z(u-u0) α -x i z(v-v0) α -y j 255-d 255 (z max -z min ) + z min   , (3)
where

d = D i,j (u, v).
The projected 3D points are thus naturally sliced into 256 depth layers separated by a distance d z = zmax-zmin 255

. Furthermore, in order to improve the visual quality, each projected 3D point is attributed a Gaussian distribution, as proposed in [START_REF] Symeonidou | Computergenerated holograms by multiple wavefront recording plane method with occlusion culling[END_REF].

Each depth layer d is therefore considered to operate as a surface source of light which emits a complex wave o d given by

o d (x, y) = i,j,u,v Di,j (u,v)=d I i,j (u, v) exp [jφ(x, y)] × exp - (x -x i,u ) 2 + (y -y j,v ) 2 2s 2 , (4)
where s 2 is the variance of the Gaussian distribution, which controls the surface of each 3D point, and φ(x, y) ∈ [0; 2π[ is the initial phase, set to a uniform random value in order to keep the diffuseness of the scene.

Step 3: Light propagation and hologram encoding

The third step of the proposed method is to compute the propagation of light scattered by the scene towards the hologram plane. The hologram lies in the (x, y, 0) plane, as shown in Figure 2. As stated in Section 2.2, the 3D scene is sliced into 256 depth layers numbered from 0 to 255, from the farthest to the nearest to the hologram plane. Light scattered by the scene is numerically propagated and shielded from one layer to another using the recurrence formula

u 0 (x, y) = o 0 (x, y), u d (x, y) = o d (x, y) + O d {P dz {u d-1 }} (x, y), (5)
where u d is the total complex wave scattered by layer d, operator O d stands for light shielding by layer d, and operator P z stands for the numerical propagation of light between two parallel planes separated by a distance z.

The propagation of light is numerically computed using the Angular Spectrum diffraction [START_REF] Goodman | Introduction to Fourier Optics[END_REF], which expresses light propagation between two parallel planes separated by a distance z as

P z {u} = F -1 F {u} e -j2π √ λ -2 -f 2 x -f 2 y z , ( 6
)
where λ is the wavelength of light, f x and f y are the spatial frequencies, and F and F -1 are respectively the forward and inverse Fourier Transform. These transforms can be computed using the Fast Fourier Transform algorithm (FFT). Light shielding is performed using a binary crosssection mask function, as proposed in [START_REF] Lohmann | Three-dimensional properties of wave-fields[END_REF]. An occluding obstacle is located within layer d. Complex wave u ′ d incident on layer d is given by

u ′ d (x, y) = P dz {u d-1 } (x, y), (7)
where u d-1 is the complex wave scattered by layer d -1. Part of u ′ d is shielded by the occluding scene points, and its amplitude vanishes in the area of the obstacle. This is expressed by multiplying u ′ d by a binary mask function m d that has value zero on the obstacle and one elsewhere. O d is thus given by

O d {u ′ d } (x, y) = m d (x, y)u ′ d (x, y). (8)
Finally, the complex wave scattered by layer N z -1 is numerically propagated to the hologram plane in order to obtain the object wave O, according to O(x, y) = P zmin {u 255 } (x, y).

(

) 9
The final CGH is computed using the bipolar intensity formula [START_REF] Lucente | Interactive computation of holograms using a look-up table[END_REF], such that

H = ℜ {OR * } , ( 10
)
where R is the reference plane wave, and ℜ{C} and C * are the real part and complex conjugate of C, respectively.

Rendering specular reflections

This section describes the generalization of the proposed method to the rendering of specular reflections. Indeed, MVD data do not only capture the geometry of the scene but also its shading, including specular reflections. These variations of brightness are totally lost when using the diffuse-only version of the method presented in Section 2.

Role of the initial phase distribution

As stated in Section 2.2, each depth layer d is considered to operate as a surface source of light which emits a complex wave o d given by Eq. ( 4). In Eq. ( 4), the phase factor h(x, y) = exp [jφ(x, y)] plays the role of a diffuser that spreads light in space. As a consequence, φ(x, y) is important for rendering specular reflections because it determines the diffuseness and direction of light emitted by each scene point.

We call H the Fourier spectrum of h, given by

H(f x , f y ) = F {h} (f x , f y ) = +∞ -∞ h(x, y) exp[-j2π(f x x + f y y)]dxdy, (11)
where f x and f y are the spacial frequencies with respect to the x and y axis, respectively. The Angular Spectrum diffraction [START_REF] Goodman | Introduction to Fourier Optics[END_REF] expresses the propagation of light scattered by h over distance -z as

h(x, y, z) = F -1 He j2πωz (x, y) = +∞ -∞ H(f x , f y )e j2π(fxx+fyy+ωz) df x df y , (12)
where ω = λ -2 -f 2

x -f 2 y . In order to understand the behavior of h, we consider the time-independent complex form of a plane wave, given by p(x, y, z) = A exp(jk • r), [START_REF] Katz | Synthesizing computer generated holograms with reduced number of perspective projections[END_REF] where A is the complex amplitude, k is the wave vector, of magnitude |k| = 2π/λ, and r = (x y z) t is the position vector. According to Eq. ( 12), h can therefore be interpreted as a superposition of plane waves of complex amplitude H(f x , f y ) propagating with wave vector k given by

k = 2π    f x f y λ -2 -f 2 x -f 2 y    . (14)
In the preliminary version of the proposed method, φ(x, y) is set to a uniform random value, and h has therefore a broadband spectrum, as shown in Fig- ure 3a. This means that h is composed by an infinity of plane waves propagating in all the directions in space. As a consequence, each 3D scene point spreads light equally in all the directions, as shown in Fig- ure 3b.

Creation of a new phase distribution

In order to reproduce specular reflections, scene points should be initialized with a new phase distribution able to concentrate the energy of emitted light around a given direction, as shown in Figure 4b. We consider the direction of emitted light given by angles α and β around the vertical and horizontal axes, respectively. We call h ′ α,β the diffuser which scatters light around this direction. Its Fourier spectrum should therefore be band-limited and shifted from the origin by (∆f x , ∆f y ), as shown in Figure 4a. According to the grating equation [START_REF] Goodman | Introduction to Fourier Optics[END_REF], the shift in frequencies is given by

∆f x = sin(α) λ ; ∆f y = sin(β) λ . (15)
We therefore multiply H by a 2D Gaussian window in order to get a band-limited Fourier spectrum given by

H ′ α,β (f x , f y ) = H(f x , f y ) exp - (f x -∆f x ) 2 + (f y -∆f y ) 2 2σ 2 (16)
where σ is an adjustable parameter which controls the bandwidth of H ′ α,β and therefore the diffuseness of each scene point. This parameter depends on the wavelength of light. Since

-1/λ ∆ f x 1/λ f x [Hz] |H' α,β (f x , 0)| (a) (b)
F -1 H ′ α,β
≡ 1, we define the new phase distribution as

φ ′ α,β (x, y) = arg F -1 H ′ α,β (x, y) , (17)
where arg{C} is the phase of C. The new diffuser is therefore given by

h ′ α,β (x, y) = exp jφ ′ α,β (x, y) . (18)
Since a new diffuser has to be computed for every possible values of α and β, this process is time consuming. In order to reduce the computational load, we first compute h ′ 0,0 using Eq. ( 18). Then, the other diffusers h ′ α,β are approximated by simply multiplying h ′ 0,0 by a plane wave propagating in the direction given by angles α and β, such that

h ′ α,β (x, y) = h ′ 0,0 (x, y) exp [j2π(x∆f x + y∆f y )] . (19)

Overall algorithm

In order to generalize it to the rendering of specular reflections, the proposed method is modified as follows.

First, 2D plus depth projections of the scene are taken from different perspective viewpoints, as described in Section 2.1.

Then, the 3D scene geometry is reconstructed from MVD data as a layered point cloud, as described in Section 2.2. However, this time, the complex wave o d emitted by each depth layer d is given by

o d (x, y) = i,j,u,v Di,j (u,v)=d I i,j (u, v)h ′ θu,θv (x, y) × exp - (x -x i,u ) 2 + (y -y j,v ) 2 2s 2 , (20)
where

θ u = arctan (u-u0)p f ; θ v = arctan (v-v0)p f ( 21
) define the direction from which the 3D point intensity has been captured by the camera.

Finally, light scattered by the scene is numerically propagated towards the hologram plane and encoded into the final CGH, as described in Section 2.3.

Experimental results

The proposed method was implemented in C++/CUDA on a PC system employing an Intel Core i7-4930K CPU operating at 3.40 GHz, a main memory of 16 GB and an operating system of Microsoft Windows 8 as well as three GPUs NVIDIA GeForce GTX 780Ti.

Hologram synthesis parameters

For the experiments, we used Unity3D software to capture the MVD projections of a highly reflective car. In this software, we set up 9 × 9 virtual cameras with resolution 1000 × 1000, focal length 2.0mm and pixel pitch 4.0µm. The distance between the cameras was set to 0.3mm along the vertical and horizontal axes and the scene was located between z min = 2.0mm and z max = 5.0mm.

For the 3D scene geometry reconstruction step, we set the standard deviation s of the 3D points' Gaussian distribution to be equal to 2.5µm. The points' diffuseness parameter σ was set to 94, 224Hz, 113, 353Hz and 127, 492Hz for the Red, Green and Blue channels, respectively. Finally, the hologram is sampled on a regular 2D grid of resolution 8192×8192 with sampling pitch 0.36µm. The wavelengths are set to 640nm, 532nm and 473nm for the Red, Green and Blue channels, respectively.

In the following, we compare the proposed method presented in Section 3.3 with its diffuse-only version, presented in Section 2.

Reconstructed scene images

Figure 5 shows the scene images numerically reconstructed from the CGHs patterns generated by the diffuse-only version of the method (first row) and by the proposed method (second row), captured from different viewpoints. For the sake of comparison, Computer Graphics images synthesized from the same viewpoints are shown on the third row. These experimental results show that both the proposed method and its diffuse-only version are able to produce correct motion parallax and occlusion effect, In order to demonstrate the capability of reproducing specular reflections, we compare the numerically reconstructed scene images to their corresponding Computer Graphics images. As shown in the first row, the diffuse-only version of the method is unable to reproduce specular reflections. Indeed, the intensity of light reflected by the car hood remains the same in all the directions, whereas it should vary according to the viewpoint. On the other hand, specular reflections are reproduced without any visible artifact using the proposed method, as shown in the second row.

Finally, Figure 6 shows the scene images numerically reconstructed from the CGH generated by the proposed method, focused on different depths. As shown in this figure, when focusing on the headlights, the rear of the car is blurred, and vice versa. These results show that the proposed method can produce an accurate accommodation cue of the 3D scene with continuous depth change.

Hologram calculation time

In order to evaluate the performance of the proposed method, we generated six holograms of the reflective car scene with different number of MVD projections. Figure 7 shows the CGH calculation time depending on the number of MVD projections using the proposed method and its diffuse-only version.

The third step does not depend on the MVD projections and is exactly the same for the two methods. As a consequence, its calculation time remains constant for both of them. On the contrary, since the number of reconstructed scene points depends on the number of MVD projections, the calculation time of the second step increases linearly with it. Moreover, it is always more important for the proposed method than for its diffuse-only version. This is due to the fact that the computation of h ′ α,β is more timeexpensive than the computation of h. However, in all cases, most of the CGH calculation time is spent during the third step. As a consequence, the CGH calculation time of the proposed method has been increased only by 0.99%, 5.22%, 8.57% and 14.58% when the number of MVD projections is equal to 9, 49, 81 and 169, respectively, compared to the diffuseonly version.

Conclusion

In this paper, we proposed a CGH computation method from MVD data considering specular reflections. The proposed method consists of three steps. First, 2D plus depth projections of the scene are taken from different perspective viewpoints. Then, the 3D scene geometry is reconstructed from the MVD data as a layered point-cloud. In order to take into account specular reflections, each scene point is considered to emit light differently in all the direc-tions. Finally, light scattered by the scene is numerically propagated towards the hologram plane and encoded into the final CGH.

Experimental results reveal that the proposed method is able to produce motion parallax with occlusion effect and accurate accommodation cue even if only a few MVD projections of the scene are used as input data. Furthermore, specular reflections are accurately reproduced with a small computational overhead. The validity of the proposed method was only verified with a fully synthetic scene, so in future study we plan to evaluate its sensitivity to errors in depth data when capturing a natural scene with real cameras.

Figure 1 :

 1 Figure 1: Multiview-plus-Depth capture geometry.

Figure 2 :

 2 Figure 2: Numerical propagation of light scattered by the scene towards the hologram plane, side view.

Figure 3 :

 3 Figure 3: (a) Fourier spectrum of the original diffuser, (b) illustration of diffuse reflection.

Figure 4 :

 4 Figure 4: (a) Fourier spectrum of the specular diffuser, (b) illustration of specular reflection.

Figure 5 :

 5 Figure 5: Scene images numerically reconstructed from the CGHs patterns generated by the diffuseonly version of the method (first row) and by the proposed method (second row), captured from different viewpoints. For the sake of comparison, Computer Graphics images synthesized from the same viewpoints are shown on the third row.

Figure 6 :

 6 Figure 6: Scene images numerically reconstructed from the CGH generated by the proposed method, focused on different depths.

Figure 7 :

 7 Figure 7: CGH calculation time depending on the number of MVD projections using the proposed method and its diffuse-only version.

The authors would like to thank Kenji Yamamoto's team from the National Institute of Information and Communications Technology for their warm welcome, for the very interesting discussions and for giving us access to their holographic display.

* This work has been achieved within the Institute of Research and Technology b<>com, dedicated to digital technologies. It has been funded by the French government through the National Research Agency (ANR) Investment referenced ANR-A0-AIRT-07.