Jean-Michel Dischler

Reinhard Klein

Xavier Granier

Alexander Wilkie

Stefanie Wuhrer D'avoir

Stefanie Wuhrer

Michal Mojzík

Alban Fichet

• Handling Fluorescence in a Uni-directional Spectral Path Tracer

Computer graphics is a valuable tool used from industrial applications to entertainment; it enables creation of images from virtual scenes. One of the aims of computer graphics is to generate photorealistic scenes; an important part of realism is relying on the accuracy of material models. Hence, the study of material models and reflectance is crucial.

Recently, measured materials gained popularity. This works well for uniform materials but spatially varying materials present multiple challenges in acquisition, storage, rendering and edition. This thesis presents methods for processing and editing measured materials.

We make three specific contributions:

• We propose a technique for fast acquisition of anisotropic spatially varying bidirectional reflectance (SVBRDF) with a limited amount of light directions and a fixed point of view.

While not suitable for high fidelity measurements, it gives a starting point to edit a realworld material while other techniques require lengthy measurement time and complex gantry setups.

• Then, we propose a pipeline to fit measured materials (BTF) to analytical materials (BRDF). This technique drastically decreases the memory footprint of a measured material by approximating it with analytical BRDFs parametrised with 2D maps. It allows direct edition of those 2D maps and combination of different materials from various datasets.

• Finally, we introduce a method to mix the reflectance properties of two materials. We can use the shininess of a material with the anisotropy of another while allowing edition of each contribution separately.

Each chapter will have a summary in French, according to the legislation.

III

Je tiens tout d'abord à remercier mon directeur de thèse, Nicolas Holzschuch. Plus que d'offrir un encadrement scientifique, il a su m'encourager dans les moments difficiles de ma thèse. J'ai pu arriver au bout de cette épreuve grâce à sa conviction et son accompagnement.

Résumé

L'informatique graphique a prouvé son utilité dans de nombreux domaines, du prototypage industriel en passant par les loisirs avec le cinéma ou le jeu vidéo ou encore pour l'archivage du patrimoine historique. Elle permet la création d'images à partir de scènes virtuelles. L'un des objectifs de l'informatique graphique est la génération de rendus photo-réalistes. Pour atteindre le photoréalisme, la qualité des données d'entrée et en particulier des matériaux est essentielle.

Les matériaux mesurés ont pris de l'importance en informatique graphique. Ils sont efficaces pour la représentation de matériaux homogènes. Cependant, pour les matériaux qui varient en fonction de la position, leur numérisation, utilisation pour le rendu, stockage et édition révèle de nombreuses difficultés. Cette thèse se propose de présenter des méthodes pour le traitement et l'édition de tels matériaux.

Nous présentons trois contributions :

• Nous proposons une technique pour l'acquisition rapide de matériaux anisotropes avec variation spatiale à partir d'un nombre restreint d'illuminations et avec un point de vue fixe. Cette méthode est adaptée pour des mesures ne requérant pas une fidélité critique. Alors que d'autres techniques nécessitent un système de mesure coûteux et une durée d'acquisition importante, notre technique permet d'obtenir un point de départ pour l'édition à partir de paramètres issus de matériaux réels.

• Ensuite, nous proposons une procédure pour l'approximation de données mesurées par une fonction analytique. Elle permet de réduire significativement l'emprunte mémoire requise par le stockage des données mesurées tout en proposant l'édition aisée du matériau ainsi représenté. Il est aussi possible de modifier la distribution spatiale des matériaux et ainsi remplacer des matériaux d'une surface par ceux d'une autre.

• Enfin, nous proposons une méthode permettant d'utiliser une partie des propriétés de réflectances d'un matériau pour les fusionner avec celles d'un second. Cette méthode permet aussi l'édition des deux caractéristiques indépendamment l'une de l'autre de façon intuitive.

À la fin de chaque chapitre, nous en dressons un résumé en français, conformément à la législation.

CHAPTER 1

Introduction U sing our visual system to scan a room, we can identify accessories and furniture. We can distinguish their colour, and categorise their texture without touching them:

The surface of this bathtub looks smooth. This car paint is old and damaged, its shininess is uneven. All of these characteristics are given by materials' reflectance: the way a material reflects the light conveys to our visual system a lot of information. We first rely on our visual system to name the material that compose an object.

We use computers to generate images of virtual scenes. It has many applications whether for entertainment (special effects, video games) or for more serious applications (virtual prototyping, cultural heritage...).

One key element for these applications is the photorealism of pictures: to improve realism, we need realistic materials. To get these, we rely on material acquisition. It comes at a cost. The procedure is tedious and requires special and expensive equipment. After acquisition, materials are difficult to edit. Finally, acquired materials use a significant amount of memory.

We want to represent real-world reflectance efficiently. Memory is a valuable resource and production scenes rely on many materials at the same time. Using directly measured materials is not an option. Edition is another important aspect for usability: artists need to get intuitive handles to tweak material appearance.

In this thesis, we propose techniques to reduce the memory footprint of measured materials while allowing edition. We also propose a method for fast acquisition of material in a noncritical context.

Local and global effects

Material and geometry Light is scattered by surfaces. The material drives a part of the scattering. The surface geometry accounts for the other part. In this thesis, we use the "microfacet" model which models light scattering by micro mirrors at micro scale. We want to distinguish two scales of geometry:

• The observable geometry which is not part of the material model. It is the surface geometry and is independent of the material. We can observe this scale with bare eyes. • The theoretical geometry. It is the root of the microfacet model later introduced. This is a mathematical construction to model light scattering by a material.

The hard limit we draw between those two geometries is arguable. Material models are already a simplification of light scattering. Is brushed aluminium an aluminium surface with valleys? The scratches being barely noticeable by naked eyes, we may consider it part of the material scattering. It will provide anisotropy property to the material reflectance. In fact, the limit between those two scales is blurry and a range of work is focusing on providing level of details depending on the distance of the observer to the surface.

When we are dealing with measured materials, we consider the sub-pixel effect to be material effects and effects occurring over the footprint of a pixel to be surface effects. The effects of geometry and material influence each other due to multiple scattering. When measuring reflectance, we measure the two effects combined. It is challenging to separate the two when analysing measured reflectance.

Influence of material on their surrounding Materials affect not only the appearance of surfaces but also their surrounding: they reflect light on the neighbouring objects. Surfaces become secondary light sources when we compute global illumination. Figure 1.2 features an example of global illumination. The indirect illumination takes a significant share in the overall appearance of a scene. It is determined by materials populating the scene.

The importance of material in the overall appearance of a scene motivates our need for accurate reflectance. With the same lighting and geometry, appearance of a scene can drastically change depending on materials coating the surfaces. Images courtesy of Cyril Soler.

Desirable material properties

Energy conservative Accurate material reflectance is crucial for realism. One of the main properties a material representation shall verify is energy conservation: it shall not reflect more light than it received. We call the materials verifying this property physically plausible.

A more restrictive group base their representation on physical properties; we call them physically based. This does not mean they are perfectly modelling real-world; it means they are built upon a theoretical foundation following laws of physics.

Physically based materials are highly desirable in computer graphics when targeting photorealism.

Efficiency Simulating light transport is a computationally intensive task. For both real time and offline rendering, materials efficient to evaluate are capital. Efficiency takes two necessary components:

• Fast evaluation: thousands of evaluations are required for a single pixel. When using stochastic methods, the property of guessing in which direction the light is most likely scattered is important as well. This is given by importance sampling. Fast evaluation and accurate importance sampling are both important.

• Reduced memory footprint: a scene uses multiple materials at once. We cannot afford very large materials. We need to fit the whole scene in fast memory for efficient light transport evaluation. Addressing large blocks of memory is also expensive and a potential bottleneck.

Edition Computer graphics is used in a wide range of context. Artists are often involved. A material is at the heart of overall appearance. Editability is a valuable asset. Not all materials offer the same ease for edition.

Material representations

Homogeneous and spatially varying A material in its most general form is spatially varying: wood shows varying grain (see Figure 1.3a), skin has a varying cellular and imperfection distribution. Some materials can be assumed to be homogeneous. A plastic, except potential manufacturing defects, keeps a constant reflectance across its surface.

Isotropic and anisotropic

An anisotropic material has a privileged reflection axis regardless illumination and observation. This is typically the case of brushed metals. It requires 4 dimensions to characterise its reflectance: both viewing and illumination elevations and azimuths.

An isotropic material has no such reflection pattern and can be characterised with 3 dimensions: the viewing and illumination elevations and the azimuths difference between the two.

Measured and analytic

To get realistic materials, acquisition is an appealing choice. The acquisition measure how light is reflected from multiple combination of illuminations and viewing directions. Editability and acquisition costs makes usage of measured data impractical and not always possible. For spatially varying materials, memory footprint is also a major issue.

Analytic models offer another approach to render materials. They are based on mathematical expressions with appearance driven by parameters. Those models are compact and efficient to evaluate. Although the parameters involved in analytic models are not always meaningful for a non-specialist relating to the physical underlying model used, they are accessible for edition.

While captured materials offer realism, physically based models express material appearance in a canonical fashion with a limited number of parameters. We want to represent a measured material by an analytic model to benefit from compactness, efficiency, and editability. This process is difficult: not all the reflectance properties are well modelled and approximations have to be made. These have to be convincing enough to be visually conclusive. In addition, acquisition errors have to be addressed when we want to fit to an analytic model.

Organisation of the manuscript

The manuscript is organised in two parts; the first part reviews the basic physical principles we use in the scope of this thesis for computer graphics.

Chapter 3

We introduce the physical nature of light and the quantities used to measure light radiation. We explain some of the properties of an electromagnetic wave which can have visual consequences when simulating light transport.

Chapter 4 We introduce how our visual system interprets the light wavelengths as colours.

We motivate the utility of spectral rendering for accurate simulation of light transport and colour accuracy.

Chapter 5 We explain how light is reflected on surfaces and give the basis of material scattering models. We explain the basics of the microfacet model.

Chapter 6

We overview and group material acquisition techniques proposed in the literature.

We omit important points for the reader interested in implementing a path tracer: we do not present rendering techniques, specific acceleration structures for scene hierarchy, intersections or filtering methods. We concentrate on the physical properties of materials and their interaction with light.

The second part aggregates my contributions in the direct scope of my thesis.

Chapter 7 We present a method for acquiring spatially varying anisotropic materials. Our setup works with as few as twenty sample and is cheap and simple to build. This work was presented at Graphics Interface 2016 in Victoria, BC, Canada.

Chapter 8 We describe a pipeline for fitting measured spatially varying materials with editability in mind.

Chapter 9 We evaluate the accuracy of our fitting method presented previously. We explain the implication of acquisition sampling on material acquisition.

Chapter 10 We introduce a technique for editing measured materials without computing the underlying analytical model. We can mix material properties from different datasets. For example, we can use the shininess from one material and the anisotropy of another. Then, we can edit each property independently.

Contributions and notions developed in the course of this thesis are summarised at the end of the manuscript. We propose outlooks for material acquisition and edition.

Collaboration

I did a gap year in the Czech Republic at Charles University during my PhD. I was under the supervision of Alexander Wilkie and Jaroslav Křivánek. I have strengthened my knowledge on rendering during this year. I have collaborated on an article for rendering fluorescence in a path tracer, enabling the usage of Hero Wavelength Spectral Sampling (HWSS) and participating media with fluorescence [START_REF] Mojzík | Handling Fluorescence in a Uni-directional Spectral Path Tracer[END_REF]. We use parts of this article in this thesis to introduce fluorescence in the context part. I chose not to include its core contribution: I joined an ongoing project and was not at the initiative of the idea. I have contributed to the project by writing rendering code, debugging evaluating and writing part of the article.

During this year, I took part in the open source release of the long time in house developed physically based rendered ART (The Advanced Rendering Toolkit) which enables spectral path tracing with polarisation, fluorescence, and HWSS support [START_REF]The Advanced Rendering Toolkit[END_REF].

Publications

Here is a list of the published work during my thesis.

• Capturing Spatially Varying Anisotropic Reflectance Parameters using Fourier Analysis Alban Fichet, Imari Sato, Nicolas Holzschuch [START_REF] Fichet | Capturing Spatially Varying Anisotropic Reflectance Parameters using Fourier Analysis[END_REF] Graphics Interface Conference 2016, Jun 2016, Victoria, BC, Canada. pp.65 -73, <10.20380/GI2016.09>

Introduction version française

E

n utilisant notre système visuel pour observer une pièce, nous pouvons identifier les objets et meubles qui s'y trouvent. Nous pouvons en distinguer les couleurs et catégoriser leurs textures sans même les toucher : La surface de cette baignoire semble lisse. La peinture de cette voiture semble usée, sa brillance est irrégulière.

Toutes ces caractéristiques sont données par la réflectance des matériaux : la façon dont un matériau renvoie la lumière fournit à notre système visuel de nombreuses informations. Nous nous basons en premier sur ce sens pour estimer les matériaux qui constituent un objet.

L'informatique nous permet de générer des images de synthèse. De nombreuses applications utilisent cet outil, que ce soit pour des applications récréatives (effets spéciaux, jeux vidéo) ou pour des applications critiques (prototypage virtuel, héritage culturel. . . ).

Un élément clé pour ces applications est le photoréalisme des images générées : afin d'améliorer le réalisme, nous avons besoin de matériaux réalistes. Pour cela, il est possible de se baser sur des matériaux mesurés. Cependant, il y a des inconvénients : le processus d'acquisition est contraignant et nécessite des équipements onéreux. Après l'acquisition, il est difficile d'éditer les matériaux. Enfin, les matériaux mesurés occupent une quantité de mémoire importante.

Nous souhaitons représenter des réflectances réalistes efficacement. La mémoire est une ressource rare et les scènes utilisées en production utilisent de nombreux matériaux à la fois. L'utilisation directe de matériaux mesurés n'est donc pas une option. L'édition est un autre aspect capital : les artistes ont besoin de paramètres intuitifs pour manipuler l'apparence des matériaux.

Dans cette thèse, nous proposons des méthodes pour réduire l'emprunte mémoire des matériaux mesurés tout en permettant leur édition. Nous proposons aussi une méthode pour l'acquisition rapide de matériaux dans un contexte non critique.

Effets locaux et globaux

Les matériaux et la géométrie La lumière est diffusée par les surfaces. Les matériaux sont responsables d'une part de la distribution de cette diffusion, l'autre partie est due à la géométrie de la surface. Nous utilisons au cours de cette thèse le modèle à micro facettes qui modélise la diffusion de la lumière par une distribution de micro miroirs. Cependant, nous distinguons deux échelles de géométrie :

• La géométrie observable. Elle ne fait pas parti du modèle de matériau. Il s'agit de la géométrie de la surface et est indépendante du matériau qui la constitue. Nous pouvons l'observer à l'oeil nu.

• La géométrie théorique. Il s'agit du fondement du modèle à micro facettes expliqué dans cette thèse. C'est une construction mathématique pour modéliser la réflexion de la lumière par un matériau.

La limite entre ces deux géométries est discutable. Les modèles de matériaux sont déjà une simplification de la réflexion de la lumière. Est-ce que l'aluminium poncé est un matériau en soi où s'agit-il de la combinaison de la géométrie des rayures avec de l'aluminium lisse ? Ces aspérités étant difficilement visualisables à l'oeil nu, nous pouvons la considérer comme faisant parti du modèle de matériau. La limite entre ces deux échelles est floue et un ensemble de travaux s'intéressent au niveau de détail adéquat suivant la distance entre l'observateur et la surface.

Lorsque nous traitons des données mesurées, nous considérons les effets sous pixels des effets dus aux modèle de matériau. Les effets au-dessus de la taille d'un pixel sont considérés comme des effets dus à la géométrie de la surface.

Les effets de la géométrie et du matériau s'influencent l'un et l'autre en raison des réflexions multiples de la lumière. Lorsque nous mesurons des réflectances, nous mesurons le résultat de ces deux effets combinés. Il est ainsi difficile de les séparer l'un de l'autre lors de l'analyse des réflectances mesurées. L'influence de la géométrie sur l'environnement Les matériaux affectent non seulement les surfaces mais aussi leur environnement : ils diffusent la lumière sur leur voisinage. Les surfaces deviennent des sources secondaires de lumière lorsque nous calculons l'illumination globale. La couleur des murs d'une pièce influence la couleur des objets la peuplant.

Propriétés souhaitées pour les modèles de matériau

Propriétés souhaitées pour les modèles de matériau

Conservation de l'énergie Des modèles de matériaux corrects sont capitaux pour un rendu réaliste. L'une des propriétés principales qu'un modèle doit vérifier est la conservation de l'énergie : un matériau ne doit pas renvoyer plus de lumière qu'il n'en a reçue. Ces types de matériaux sont dits physiquement plausibles. Un groupe plus restrictif base leur représentation sur des propriétés physiques. Ils sont dits basés physique. Cela ne veut pas dire qu'ils modèlent parfaitement le monde réel. Cela signifie qu'ils sont construits à partir d'une fondation théorique suivant les lois de la physique.

Efficaces La simulation du transport de la lumière est intensive en ressources. Des matériaux efficaces à évaluer sont importants. L'efficacité se base sur deux composants essentiels :

• L'évaluation rapide : des milliers d'évaluations sont nécessaires pour chaque pixel. Lorsque l'on utilise des méthodes stochastiques, le guidage des directions de la lumière est important. Cela est donné par l'échantillonnage préférentiel.

• L'usage mémoire réduit : une scène typique utilise de nombreux matériaux à la fois. Nous ne pouvons pas nous permettre une utilisation importante de la mémoire pour chacun d'entre eux : il est nécessaire de pouvoir stocker l'intégralité de la scène dans une zone mémoire rapide. De plus, l'accès et l'adressage de grandes zones mémoire est couteux en peut devenir un facteur limitant l'exploitation des ressources de calcul.

Éditables L'informatique graphique est utilisée dans un large domaine d'applications où souvent des artistes sont impliqués. Les matériaux sont à la base de l'apparence d'une scène. Leur potentiel d'édition est donc crucial. Tous les matériaux n'offrent pas le même aval concernant le paramétrage de leur apparence. Les modèles analytiques offrent une autre approche pour simuler des matériaux. Ils sont basés sur une équation mathématique dont l'apparence est régie par des paramètres de cette équation. Ces modèles sont compacts et efficaces à évaluer. Même si les paramètres configurant leur aspect ne sont pas nécessairement intuitifs pour des non spécialistes, ils sont directement accessibles.

Types de matériaux

Organisation du manuscrit

D'un côté, les matériaux mesurés garantissent le réalisme, de l'autre, les matériaux analytiques sont compacts et efficaces. Nous souhaitons dans cette thèse représenter des matériaux mesurés par des modèles analytiques afin de bénéficier des avantages de ces derniers : compacité, efficacité et possibilité d'édition. Ce processus est difficile : tous les paramètres de réflectance ne sont pas modélisés correctement par les modèles analytiques et certaines approximations doivent être concédées. La représentation doit rester toutefois visuellement convaincante. De plus, les potentielles erreurs d'acquisition doivent être prises en compte lorsque nous souhaitons effectuer cette transition durant les processus de minimisation que nous utilisons.

Organisation du manuscrit

Le manuscrit est organisé en deux parties.

La première partie passe en revue les propriétés physiques de base utilisées au cours de cette thèse en informatique graphique : Chapitre 3 Nous introduisons la nature physique de la lumière et les quantités radiométriques utilisées pour mesurer la radiation des ondes électromagnétiques. Nous présentons quelques propriétés de ces ondes qui peuvent avoir un impact visuel lors de la simulation du transport de la lumière Chapitre 4 Nous présentons comment notre système visuel interprète les radiations lumineuses comme des couleurs. Nous motivons l'intérêt de la considération spectrale pour la simulation fidèle du transport de la lumière. Nous omettons des éléments importants pour le lecteur intéressé par l'implémentation d'un moteur de lancé de rayons. Nous ne présentons pas les différentes méthodes de rendu, les structures d'accélération pour l'accès hiérarchique à la géométrie de la scène, les méthodes d'intersection ou de filtrage. Nous nous concentrons sur l'aspect physique des propriétés de matériaux et leur interaction avec la lumière.

La seconde partie est dédiée à la présentation de mes contributions directement liées au sujet de ma thèse :

Chapitre 7 Nous présentons une méthode pour l'acquisition de matériaux anisotropes variant spatialement. Notre méthode fonctionne avec seulement vingt échantillons, est facile et peu onéreuse à reproduire. Ce travail a été présenté à Graphics Interface en 2016 à Victoria, Colombie Britannique, Canada.

Chapitre 8 Nous décrivons un processus pour modéliser des réflectances mesurées à variation spatiale par un modèle analytique par une méthode de minimisation. Notre approche permet l'édition des matériaux ainsi représentés.

Chapitre 9 Nous évaluons la précision du processus présenté au chapitre 8. Nous expliquons l'implication du choix de l'échantillonnage sur la mesure de réflectance.

Chapitre 10 Nous présentons une méthode pour mixer les propriétés de matériaux provenant de deux sources différentes. Nous pouvons utiliser la brillance d'un matériau combinée à l'anisotropie d'un second. Ensuite, il nous est possible d'éditer chacune de ces propriétés, de façon simple, indépendamment l'une de l'autre.

Les contributions et les notions utilisés au cours de cette thèse sont résumées à la fin du manuscrit. Nous proposons des perspectives pour l'acquisition et l'édition de matériaux.

Collaboration

J'ai effectué au cours de ma thèse une année de césure en République Tchèque à l'Université Charles. J'ai été sous la supervision d'Alexander Wilkie et de Jaroslav Křivánek. Au cours de cette année, j'ai renforcé mes connaissances sur les méthodes de rendu. J'ai collaboré avec Michal Mojzík et Alexander Wilkie sur un article traitant du rendu de la fluorescence dans des médias participatifs et le support de la fluorescence avec la méthode Hero Wavelength Spectral Sampling (HWSS). Des parties de cet article sont utilisés dans cette thèse pour introduire la fluorescence. J'ai cependant choisi de ne pas inclure la contribution principale : il s'agit d'un travail que j'ai rejoint en cours et je n'ai pas été à l'initiative du projet. J'ai contribué au projet en écrivant certaines sections de l'article et en programmant certaines fonctionnalités.

Durant cette année, j'ai aussi pris part à la diffusion du moteur de rendu physique ART (The Advanced Rendering Toolkit) sous la forme d'un logiciel libre, développé depuis de nombreuses années en interne. Ce moteur permet le rendu spectral avec un support de la polarisation, de la fluorescence et de Hero Wavelength Spectral Sampling.

Publications

Au cours de cette thèse, j'ai participé aux publications suivantes :

L

ight is both an electromagnetic wave and a particle (photon) displacement. For computer graphics, the latter model is more often used; it is based on geometrical optics.

In simple scenes, the effects due to the wave theory of light are barely noticeable. With more complex materials, they may have a significant impact on the appearance. Most surfaces cause polarisation and light interferences. Polarisation influences inter-reflection between specular surfaces. Interferences produce rainbow effects when light penetrate thin layers or hit small irregularities or scratches.

In this chapter, we first introduce the physics used in computer graphics for describing light radiation. We explain the relationships between wave, frequency and wavelength. We expose how electric field and magnetic induction are linked together and described by Maxwell equations. We introduce radiometric units used in computer graphics. We expose some effects the wave theory of light has on appearance.

Finally, we introduce what is a reflectance with its wavelength dependency. In Chapter 5, we further describe reflectance adding the angular dependency with Bidirectional Reflectance Distribution Function (BRDF).

Historical background

Historically, two interpretations of light have been competing. The first popular interpretation of light was the corpuscular theory proposed by René Descartes in 1637 [START_REF] Descartes | Discours de la méthode pour bien conduire sa raison et chercher la vérité dans les sciences, plus La dioptrique, Les météores et La géométrie qui sont des essais de cette méthode[END_REF], and further developed by Isaac Newton. The second interpretation was the wave front theory established by Christiaan Huygens presented in Traité de la Lumière [START_REF] Huygens | Traité de la lumière[END_REF].

Despite the comprehensive explanation of light transport provided by Huygens theory, corpuscular theory remained the consensual interpretation among scientists during the 17th century, mainly due to the fame of Descartes. It took a century before the wave theory appeared again thanks to Augustin Fresnel's work [START_REF] Fresnel | Oeuvres complètes d'Augustin Fresnel[END_REF].

Still, the real nature of this wave was unknown. James Clerk Maxwell provided the answer in 1864 with his work on electromagnetic waves [START_REF] Clerck | VIII. a dynamical theory of the electromagnetic field[END_REF].

The corpuscular theory was getting less attention until the beginning of 20th century with quantum theory. To explain black body radiations, Max Planck [START_REF] Planck | Ueber das gesetz der energieverteilung im normalspectrum[END_REF] suggested light could gain or lose finite quanta of energy depending on wavelength. Albert Einstein used this idea in his first article on quantum physics in 1905 [START_REF] Einstein | Über einen die Erzeugung und Verwandlung des Lichtes betreffenden heuristischen Gesichtspunkt[END_REF][START_REF] Einstein | On a Heuristic Point of View about the Creation and Conversion of Light[END_REF]. In 1924, de Broglie [START_REF] De | Recherches sur la théorie des Quanta. Theses, Migrationuniversité en cours d'affectation[END_REF] proposed the matter-wave theory extending the light corpuscular-wave duality.

The dual interpretation of light is now consensual. Light is both a photon displacement with finite energy states (electron volt1 ) and an electromagnetic wave propagation. It was experimentally shown by Tsuchiya et al. [START_REF] Tsuchiya | Photon-counting imaging and its application[END_REF] with photon-counting imaging device using the Young's dual slits experiment.

Frequency and wavelength

Electromagnetic waves can take various frequencies ν, or wavelengths λ. The wavelength λ characterise the distance between two repetitive patterns of the wave (see Figure 3.1). Light is a subset of electromagnetic waves to which our eyes are sensitive (see Figure 3.2). The wavelength of light characterises its colour. This will be discussed in Chapter 4.

Wavelengths and frequencies are linked by equation 3.1.

λ = v ν [m] (3.1)
Where:

• λ is the wavelength in meters [m].
• v is the speed of the wave in the considered medium in meters per seconds [m.s -1 ].

• ν is the wave frequency in Hertz [Hz] i.e. [s -1 ].

Notice from equation 3.1, a wavelength is dependent of speed of propagation for the medium the wave is travelling in. Intuitively, if a wave travels slower in a medium, the repeating oscillation patterns will be closer to each other, and the wavelength will then be shorter. When designating wavelengths for light, we are referring to their wavelength in a vacuum, i.e. for travelling speed of about 3.10 8 m.s -1 .

Frequency and wavelength

ν [Hz] λ [m]
10 -14 10 -12 10 -11 10 -9 10 -8 10 -7 10 -5 10 -4 10 -3 10 -1 10 0 10 1 10 2 10 3

Mathematical description

All waves, not only electromagnetic waves, are described by the following equation:

Ψ(r, t) = A 0 cos(k • r -ωt + φ 0 ) (3.2)
Or in its complex form:

Ψ(r, t) = A 0 e i(k•r-ωt+φ 0 ) (3.3)
Where:

• r is the position in space of three components (x, y, z), with ∥r∥ in [m].

• t is the time [s].

• A 0 is the amplitude. It describes the maximum value a waveform can take. For an electric field it is expressed in Volt per meter [V • m -1 ].

• k is the wave-number of three components (k x , k y , k z ) with ∥r∥ in [rad•m -1 ]. It describes the number of oscillations per distance unit.

• ω is the wave pulsation, or angular frequency [rad • s -1 ]. It describes the phase shift per time unit.

• φ 0 is the phase shift [rad]. It describes the de-synchronisation of the wave from a reference wave without retardance.

To simplify calculus the complex form is often used. We deduce the physical solution from the real part.

The wave pulsation is proportional to the wave frequency:

ω = 2πν [rad • s -1 ] (3.4)
The wave-number vector is proportional to the wavelength, hence, to the frequency and the speed of the wave in the medium it is propagating in. In an isotropic medium, it is pointing to the same direction as the wave propagation direction. This is not the case for anisotropic media such as asymmetrical crystals.

∥k∥ = 2π λ [rad • m -1 ] (3.5)
Hence, for an isotropic medium with a wave travelling in the z direction:

k = ⎡ ⎣ 0 0 k z ⎤ ⎦ = ⎡ ⎣ 0 0 2π λ ⎤ ⎦ [rad • m -1 ] (3.6)
For an electromagnetic field travelling in the z direction, in an isotropic medium with no charge and no current, the electric field E(r, t) (in volts per meter [V.m -1 ]) and magnetic induction 2 B(r, t) (in teslas [T]) can be deduced from the Maxwell equations 3 :

∇ • E(r, t) = 0 ∇ • B(r, t) = 0 (3.7) ∇ × E(r, t) = - ∂B(r, t) ∂t ∇ × B(r, t) = µ m ϵ m ∂E(r, t) ∂t (3.8)
Where:

• ϵ m is the permittivity of the medium in [F • m -1 ]. • µ m is the permeability of the medium in [H • m -1 ]. E(r, t) = ⎡ ⎣ E 0x cos(k z -ωt + φ x ) E 0y cos(k z -ωt + φ y ) 0 ⎤ ⎦ [V.m -1 ] (3.9) B(r, t) = √ µ m ϵ m ⎡ ⎣ -E 0y cos(k z -ωt + φ y ) E 0x cos(k z -ωt + φ x ) 0 ⎤ ⎦ [T]
(3.10)

Poynting vector

To describe the energy flux of an electromagnetic wave, the Poynting vector S is used. This is expressed in [W.m -2 ] and denoted S.

S(r, t) = E(r, t) × H(r, t) [W.m -2 ] (3.11)
H is the magnetic field intensity:

H(r, t) = 1 µ 0 B(r, t) -M (r, t) [A.m -1 ] (3.12)
Where:

• µ o is the permeability of free space in [H • m -1 ].
• M is the magnetisation vector in [A.m -1 ].

In the following of this manuscript, we are considering wave travelling in an isotropic medium with no charge and no current with M such as B = µ m H 4 .

Name

Symbol Units

radiant flux Φ e [W] irradiance E e [W.m -2 ] radiant intensity I e [W.sr -1 ] radiance L e [W.m -2 .sr -1 ]
Table 3.1.: Commonly used radiometric quantities

Radiometry

To measure physical quantities transported by an electromagnetic wave, we use radiometric quantities (see table 3.1). Those are time average values. They do not describe the waveform of the electromagnetic radiation. The unit described in this section are for a whole spectrum or for a specific bandwidth. They shall be given along with the considered bandwidth. Spectral variant of those quantities gives the value for a specific wavelength (per meters or more conveniently, for light, per manometer), or frequency (per hertz).

Radiant flux

The radiant flux Φ e is the flow of radiant energy Q e passing by unit of time. It is expressed in Joule per seconds [J.s -1 ] i.e. in Watts [W]:

Φ e = ∂Q e ∂t [W] (3.13)

Irradiance

The irradiance is the radiant flux per unit area. This is the density of power for a given surface area A (in square meters m2 ). It is expressed in Watt per square meters [W.m -2 ]:

E e (p) = ∂Φ e ∂A [W.m -2 ] (3.14)
Irradiance is the time average of the Poynting vector:

E e (p) =< ∥S∥ > cos θ [W.m -2 ] (3.15)
Where: θ is the angle between the Poynting vector (i.e. E×H ) and the surface element normal.

Then, given the electric field amplitude, the irradiance in the plane of incidence of the wave is: Where:

E e (p) = 1 2 √ ϵ m µ m ∥E 0 ∥ 2 cos θ [W.m -2 ] (3.16) (
• ϵ m is the electrical permittivity of the medium in [F.m -1 ].

• µ m is the magnetic permeability of the medium in [H.m -1 ].

• E 0 is the amplitude E 0 = (E x , E y , 0) of the electric field, ∥E o ∥ in [V • m -1 ]

Radiant intensity

The radiant intensity I e is the radiant flux per unit of solid angle. A solid angle is a measurement determining the coverage of a spherical cap for a unit sphere (ω unit steradians sr). It is expressed in watt per steradian [W.sr -1 ].

I e (ω) = ∂Φ e ∂ω [W.sr -1 ]
(3.17)

Radiance

The radiance L e is the radiant flux per unit area per unit of solid angle. It is expressed in watt per square meter per steradian [W.m -2 .sr -1 ].

L e (ω, p) = ∂I e (ω)

∂A ⊥ = ∂ 2 Φ e ∂A ⊥ ∂ω = ∂ 2 Φ e cos θ∂A∂ω [W.m -2 .sr -1 ] (3.18)
Reciprocally, we can deduce irradiance from the radiance: it corresponds to the irradiance coming from all directions (see Figure 3.3):

E e (p) = ∫ Ω L e (ω, p) cos θdω [W.m -2 ] (3.19)
Similarly, for the radiant flux:

Φ e = ∫ A ∫ Ω L e (ω, p) cos θdωdA [W] (3.20)
Radiance and irradiance are important in computer graphics. Radiance is the quantity which is perceived by our eyes or by a camera sensor. In Chapter 5, we use the differential ratio between exiting radiance and incident irradiance on a surface to define the bidirection reflectance distribution function (BRDF).

Spectral power distribution

The previously presented photometric quantities are defined for the whole spectrum (a range of frequencies or wavelengths). For computer graphics, it is necessary to characterise the radiometric quantities for a given frequency or wavelength in order to translate those quantities into colours later on. The corresponding photometric units can be derived per unit of wavelength λ or frequency ν.

The spectral radiance:

L e (ω, p, λ) = ∂ 3 Φ e ∂A ⊥ ∂ω∂λ = ∂L e (ω, p) ∂λ [W.m -3 .sr -1 ] (3.21)
The spectral irradiance:

E e (p, λ) = ∂ 2 Φ e ∂A∂λ = ∂E e (p) ∂λ [W.m -3 ] (3.22)
A spectral power distribution (SPD) describes the value of spectral irradiance E e (p, λ) for a range of frequencies. It is usually more convenient to express the wavelength of light in manometer [nm] i.e. [10 -9 m] so the spectral irradiance in [W.m -2 .nm -1 ]. A relative SPD is a normalised SPD relative to the spectral irradiance at a given wavelength, often unity or 100% at λ 0 = 560 nm.

Interferences

In this section and the following, we describe several phenomena related to the wave nature of light. They are not always visible but it is useful to know they exist and keep them in mind.

Wave superposition

The superposition of two waves of the same frequency is a wave of the same frequency: Let A 3 e iφ 3 = A 1 e iφ 1 + A 2 e iφ 2 (3.25) The resulting wave (equation 3.26) have a different amplitude A 3 and phase φ 3 . So, when two waves of the same frequency cross or superpose each other, this causes interferences. The superposition of two waves can yield in the extreme cases of a resulting wave of null amplitude (destructive interference) or a wave with a double amplitude (constructive interference).

Ψ = A 1 cos(ωt + φ 1 ) + A 2 cos(ωt + φ 2 ) (3.24) = A 1 2 [ e i(ωt+φ 1 ) + e -i(ωt+φ 1 ) ] + A 2 2 [ e i(ωt+φ 2 ) + e -i(ωt+φ 2 ) ] = 1 2 [ e iωt ( A 1 e iφ 1 + A 2 e iφ 2 ) + e -iωt ( A 1 e -iφ 1 + A 2 e -iφ 2 )]
Ψ = 1 2 [ e iωt A 3 e iφ 3 + e -iωt A 3 e -iφ 3 ] = A 3 2 [ e i(ωt+φ 3 ) + e -i(ωt+φ 3 ) ] = A 3 cos(ωt + φ 3 ) (3.26) ℜ{z} ℑ{z} ϕ 1 ϕ 2 ϕ 3 A 1 A 2 A 3
The phase difference from one to the other defines how they are going to interfere (see Figure 3.5).

To compute the resulting amplitude A 3 and phase φ 3 , we can use Fresnel vectors. Equation 3.25 shows the resulting phase and amplitude can be deduced from the sum of two complex numbers. This is equivalent to addition of two vector in the complex plane, one of magnitude A 1 and phase φ 1 , the other with magnitude A 2 and phase φ 2 . Those are the Fresnel vectors corresponding to the two superposed waves. The module of resulting vector will be A 3 and its argument φ 3 (see Figure 3.6).

Optical path length Given a wave Ψ of wavelength λ and phase shift φ at point O, travelling at speed c in the considered medium. Its phase shift φ ′ at point T of distance l from O, will be:

Ψ(O) = A cos(2π • c λ • t + φ) Ψ(T ) = A cos(2π • c λ • t + φ ′ ) = A cos(2π • c λ • (t + ∆t) + φ) ∆t = l c Ψ(T ) = A cos(2π • c λ • t + 2π λ • l + φ) φ ′ = 2π λ • l + φ (3.27)
From the optical path length l (OPL) we get the relative shift of a wave: δ = 2π λ • l.

P 1 Interference patterns Figure 3.7 shows two waves Ψ 1 , Ψ 2 of same wavelength λ with phase shifts φ 1 , φ 2 at position P 1 , P 2 . They cross at P 3 at distances l 1 , l 2 respectively from P 1 , P 2 . The resulting wave Ψ 3 at P 3 will be the result of the interference between Ψ 1 and Ψ 2 . To get the resulting wave Ψ 3 , we are interested in knowing respective phases φ ′ 1 , φ ′ 2 of Ψ 1 , Ψ 2 at P 3 . Given the optical path lengths l 1 , l 2 , we can deduce the superposition of Ψ 1 and Ψ 2 at P 3 :

P 2 P 3 Ψ 1 (ϕ 1 ) Ψ 2 (ϕ 2 ) Ψ 3 (ϕ 3 ) = Ψ 1 (ϕ ′ 1 ) + Ψ 2 (ϕ ′ 2 ) l 2 l 1
Ψ 3 = A [ cos(ωt + φ ′ 1 ) + cos(ωt + φ ′ 2 ) ] (3.28) = A [cos(ωt + (δ 1 + φ 1 )) + cos(ωt + (δ 2 + φ 2 ))] (3.29) = 2A cos ( δ 1 -δ 2 2 φ 1 -φ 2 2 ) cos ( ωt + δ 1 + δ 2 2 φ 1 + φ 2 2 ) (3.30)
Notice the amplitude variation in space is dependent of the wavelength. This means, the constructive and destructive interferences occurs at a different spatial frequency depending on the wavelength. The second cosine term is time dependant and not influencing the appearance of the interference pattern: the irradiance is the time-average of the resulting wave. So, the optical path difference l 1l 2 determines the wave amplitude and irradiance at P 3 .

In practice, this leads to the observation of rainbow patterns with interferences and polychromatic light. Fresnel's vectors can be used if having to consider waves of different amplitude or for accumulating results from a path tracer (see Figure 3.8).

While the superposition of waves is linear, the resulting irradiance is proportional to the square of the wave amplitude (see equation 3.16).

Interferences are observable at macro-scale (see Figure 3.9). For instance, surfaces whose thickness is in the order of magnitude of wavelength of light (soap bubble, anti-glare coating, leather conditioners...) create visible rainbow effects due to interferences. Microstructure also cause interferences (small scratches on metals, tiny bumps on some plastics...). Those phenomena are due to light path interfering with itself causing coloured reflection even with incoherent light.

Interferences also occur when light hits a surface with irregularities at the scale of the wavelength. Holzschuch and Pacanowski [START_REF] Holzschuch | A Two-scale Microfacet Reflectance Model Combining Reflection and Diffraction[END_REF] proven the need for taking interferences into account and proposed a model to handle those phenomena (see Figure 3.10). Belcour and Barla [START_REF] Belcour | A Practical Extension to Microfacet Theory for the Modeling of Varying Iridescence[END_REF] proposed a model to handle iridescent effects caused by thin-films.

Polarisation

We have considered the electric field oscillating in one plane in the previous examples. In equation 3.9, the electric field was presented in a more complete form. Its amplitude and shift may differ for x and y axis when propagating in z direction. The plane where the electric field propagates may rotate over time. This defines the polarisation of the wave.

Polarisation states

When the plane where the electric field propagates remains fixed, the light is said linearly polarised. If the plane rotates over time at the same frequency as the wave frequency, the light is said to be circularly polarised. The intermediate states are said elliptically polarised. Finally, if this plane rotates randomly the light is said to be unpolarised. Figure 3.11 illustrates different states of polarisation. The electric field E propagates along the z axis and is projected on x and y. It is defined as:

E(t) = E x (t) • x + E y (t) • y [V.m -1 ] (3.31)
Where E x (t) and E y (t) are the electric field components with respect to x and y directions. We cannot measure the value of E(t) but only its time-averaged value, corresponding to the irradiance E e (see equation 3.16).

While our eyes are almost not sensitive to the polarisation of light5 , light transport is: surfaces can reflect or transmit the light differently depending on the polarisation state of the incoming radiation. The resulting energy received by our eyes is then impacted. It is necessary to consider polarisation for light transport interactions in computer graphics.

Stokes formalism

To represent polarisation states, various mathematical formalisms exists. We use the Stokes vector formalism, adopted in computer graphics:

S = ⎛ ⎜ ⎜ ⎝ S 0 S 1 S 2 S 3 ⎞ ⎟ ⎟ ⎠ = ⎛ ⎜ ⎜ ⎝ I Q U V ⎞ ⎟ ⎟ ⎠ (3.32)
The same wave can be represented as different combination of sinusoidal waves on an orthogonal basis depending on the frame orientation. So, this vector6 is defined within a specific frame.

Linear polarisation

We can measure the irradiance but not directly the variation of electric field over time. With two linear polarisation filters, one aligned in x direction and the other with y, we can measure E ex and E ey , the irradiance components on x and y.

The Stokes' parameter Q is defined as follows:

Q = E ex -E ey [W.m -2 ] (3.33)
When the linearly polarised wave is rotated, the polarisation axis cannot be determined solely with Q: two solutions are possible.

To leverage this indetermination, we use the third Stokes' parameter U . It consists in rotating the (O, x, y) frame by 45 • . We are calling this new frame (O, a, b). Then, the irradiance components are measured in this new frame (see Figure 3.12):

U = E ea -E e b [W.m -2 ] (3.34)

Circular polarisation

Figures 3.11b and 3.11c illustrate the case where the electric field is rotating over time leading to circular or elliptical polarisation.

We get E e l and E er , the irradiance of left and right polarisation with combination of a quarter-wave filter and a linear polarisation filter. A quarter-wave filter introduces a phase shift of π 2 on one axis of the electric field. Depending on the rotation direction of the polarisation plane, it transforms circular into linear polarised wave rotated by 45 • or -45 • (see Figure 3.13).

The fourth Stokes' parameter V is defined as:

V = E er -E e l [W.m -2 ] (3.35)
We have to use and specify a convention for the orientation of the circular polarisation. Here, we are in the frame facing the propagation direction of the wave.

Irradiance and degree of polarisation

The first component I of the Stokes' vector is the irradiance. We can represent any polarisation state by Stoke' vectors: a randomly polarised light will have its Q, U, V components null. But irradiance information will remained characterised by its I component.

The ratio between norm of vector defining polarisation (Q, U, V ), (Q, U ) or V and irradiance I is the degrees of polarisation (see Table 3.2).

Degree of polarisation

Degree of linear polarisation

Degree of circular polarisation

DoP = 100 • √ Q 2 +U 2 +V 2 I DoLP = 100 • √ Q 2 +U 2 I DoCP = 100 • V I Table 3.2.:
Definition of degrees of polarisation in percent.

Reflectance

When light interacts with matter, it is scattered. The energy is not scattered equally depending on the wavelength and the direction of the incoming wave. We will discuss in Chapter 5 the directional dependency.

Reflectance is the effectiveness of a surface to reflect radiant energy. It is a function of wavelength.

Spectral reflectance

In the simpler scenario, when light illuminate a surface, the reflected radiation will be at the same frequency as the incoming one. The reflectance can be described similarly as a spectral Dupuy and Jakob [START_REF] Dupuy | An Adaptive Parameterization for Efficient Material Acquisition and Rendering[END_REF]. This shows the efficiency of the material to reflect incoming radiance for a given set of illumination and observation. power distribution. For a given wavelength, the effectiveness of its reflection is given by a value.

Figure 3.14 shows an example of such reflectance spectrum from a measured material of the database of Dupuy and Jakob [START_REF] Dupuy | An Adaptive Parameterization for Efficient Material Acquisition and Rendering[END_REF].

Bi-spectral reflectance

In addition to straight re-emission with the same wavelength, a material can, for certain exiting wavelengths re-emit at a longer wavelength. This phenomena is called fluorescence.

A molecule re-emits a previously absorbed photon in a lower energy state than the one it was absorbed in: this means the re-emitted light from a collision event with such a molecule has a longer wavelength than the original incoming light. This shift from the absorbed frequency to the re-emitted one is spread around a main frequency as shown in Figure 3.15. This is referred to as Stokes shift.

Such phenomena can be described by a fluorescence response function Φ(λ i , λ o ), which for an excitation wavelength λ i returns the amount of energy relaxed to another re-emission wavelength λ o . For λ i = λ o , this corresponds to the non-fluorescent reflectance introduced in the previous section.

Re-radiation matrix

The fluorescence response Φ(λ i , λ o ) can be represented in a discretised form as a so-called reradiation matrix. An example dataset is shown in Figure 3.17. The incoming wavelength λ i lies on its vertical axis, and the outgoing wavelength λ o on its horizontal. The diagonal of such a re-radiation matrix describes the amount of light that does not undergo any wavelength shift. Practically all fluorophores exhibit similar characteristics: namely, that there is one main band where energy is absorbed, and one where it is re-emitted. Note that one still needs the full re-radiation matrix of the fluorophore in question to generate such a plot, which only shows the cumulative shifting effect of the material. Figure 3.17 shows the entire reradiation matrix of this material. .17. This plot amounts to several vertical cuts approximately halfway through the centre of the wavelength-shifting activity, for different incident wavelengths. These show that Kasha's rule is not an exact rule, so that there is some variation: but the overall appearance of the spectra is quite similar overall.

Shifting probability

Kasha's rule states that the re-emission spectrum distribution should be the same, whatever wavelength within the absorbing area of the spectrum is used for excitation. Only the intensity of this re-emitted spectrum is modified. However, this rule does not always apply -at least not in the strict sense.

While the accumulated energy absorption and re-emission, as show in Figure 3.15, represents, respectively, the sum of the columns and the lines of the re-radiation matrix, i.e. ∫ Λ Φ(λ, λ o ) dλ o and ∫ Λ Φ(λ i , λ) dλ i , the probability for a given wavelength to be the re-emitted may differ if the particle does not exactly follow Kasha's rule (and as figure 3.18 shows, such slight variations are actually the norm). Then, the distribution of possible re-emission wavelengths for an excitation wavelength j is given by the corresponding column j in the re-radiation matrix. On the other hand, from a re-emitted wavelength i, the possible spectrum of excitation wavelengths is given by the corresponding line i in the matrix.

Conclusion

In this chapter, we have presented light as a wave. We introduced radiometric units useful in computer graphics. We insist on the importance of considering light as an electromagnetic wave through its spectral power distribution and its polarisation state. We have exposed some visual effects due to the wave nature of light.

In computer graphics, we mostly use geometrical optics for light transport. We can use specific mathematical frameworks to account for some of wave properties. For instance, Stokes' vectors account for polarisation in geometrical optics. Local models also handle properties of the wave nature of light like interferences.

For global light transport, we generally ignore effects like diffraction, causing wavefront propagation in occluded areas. At the time of writing this thesis, they are hard to evaluate globally and induce a huge memory overhead with an increasing complexity. They generally does not impact significantly the final rendering, with wavelength we are working with.

We prefer to use local models to handle those complicated cases while providing a good approximations.

Résumé du chapitre

Dans ce chapitre, nous avons présenté la lumière sous sa forme d'onde. Nous avons introduit certaines unités radiométriques utiles en informatique graphique. Nous avons insisté sur l'importance de considérer la lumière comme une onde électromagnétique en la caractérisant par sa distribution spectrale et son état de polarisation. Nous avons présenté certains phénomènes liés à la nature ondulatoire de la lumière.

Les phénomènes ondulatoires sont majoritairement ignorés en informatique graphique. Le transport de la lumière suit généralement les règles de l'optique géométrique. Toutefois, certains outils mathématiques permettent la simulation d'une partie des phénomènes liés à la nature ondulatoire de la lumière. Les vecteurs de Stokes permettent, par exemple, la prise en compte de la polarisation de l'onde. Des modèles locaux peuvent aussi permettre d'incorporer certains phénomènes ondulatoires tels que les interférences.

Toutefois, pour le transport global, certains effets tels que la diffraction, qui cause une propagation du front d'onde dans des zones où l'optique géométrique considère que l'énergie est nulle, sont généralement ignorés. Pour l'instant, certains de ces phénomènes sont difficiles à évaluer efficacement car ils induisent une augmentation drastique de la complexité et de la consommation mémoire. Enfin, ils ne participent pas nécessairement de façon significative à l'apparence finale d'une image et peuvent souvent être ignorés.

Des modèles locaux sont alors privilégiés proposant de bonnes approximations.

CHAPTER 4

Spectra and colour perception H uman vision system is sensitive a certain range of frequencies. Radiations out of that range are not visible. In daylight, we perceive different colours depending on received spectral distribution: three types of cells in our retina are exited with different level.

Tristimulus is a mathematical basis for representing a mix of three primaries. It would seem a natural way of characterising reflectance and illumination: we use colour primaries in photography and cinema. We are familiar with red, green and blue, or cyan, magenta and yellow primaries.

But a colour or tristimulus value does not defines a unique spectrum: we can perceive multiple spectra as the same colour. This is called metamerism. This is not a concern for photography: we are interested by the image resulting of the physical transport of light. It reproduces what our eyes would see. It is however a concern for computer graphics where we simulate light transport.

In this chapter, we discuss the importance of spectral consideration for reflectance and illuminant in computer graphics.

The first section introduces photometric units and how spectra are perceived as colours. We show example of metamerism.

In the second section we compare tristimulus rendering with spectral rendering. We discuss the reasons for tristimulus wide adoption in computer graphics and motivate the need for a spectral rendering pipeline transition.

We need to characterise the energy distribution when simulating light transport. Using tristimulus values can lead to serious discrepancies. Light is reflected depending on its spectral distribution not depending on a set of primaries. Tristimulus values, and colour in particular, are perceptual concepts and mathematical framework resulting of physics and biology. They do not characterise a physical wavelength distribution.

Human vision and Tristimulus

Photometry

The retinas at the back of the human eye are composed by two different types of photosensitive cells, rods and cones. Both are used for vision, depending on lighting conditions.

• In bright environments, cone cells are active. This is called photopic vision.

The human eye have three types of cones sensitive to different frequencies; S for smallwavelength-sensitive, M for middle-wavelength-sensitive, L for long-wavelength-sensitive. This allows colour vision.

• In lower brightness conditions, rod cells are active. This is called scotopic vision. Rod cells are more sensitive than cone cells. Rod cells share the same sensitivity range hence we are unable to distinguish colours in a darker environment.

Cone and rod cells having different sensitivity bandwidth, vision varies depending on lighting condition. This is described by the luminosity function (Figure 4.1) and varies depending on the lighting condition.

We use photometric units quantities to describe how much of the light is perceived. It is an equivalent of radiometric units, characterising radiation, for perception (see Table 4.1).

To convert from radiance L e to the luminance L v , we use the following equation for photopic vision (daytime):

L v = K m ∫ Λ L e (λ)V M (λ)dλ [lm.m -2 .sr -1 ] (4.1)
With: K m = 683 lm.W -1 , the maximum luminous efficacy for photopic vision. For scotopic vision (dim-light):

Name Symbol Units Radiometric equivalent luminous flux Φ v lumen [lm] radiant flux illuminance E v lux [lx] i.e. [lm.m -2 ] irradiance luminous intensity I v candela [cd] i.e. [lm.sr -1 ] radiant intensity luninance L v nits [cd.m -2 ] i.e. [lm.m -2 .sr -1 ] radiance
L v = K ′ m ∫ Λ L e (λ)V (λ)dλ [lm.m -2 .sr -1 ] (4.2)
With: K ′ m = 1700 lm.W -1 , the maximum luminous efficacy for scotopic vision.

We have to specify the luminous efficiency function corresponding to the environment when we use a photometric unit.

Determining colour primaries

The theory of colour vision based on primaries started with Newton [START_REF] Newton | Opticks or, a Treatise of the reflexions, refractions, inflexions and colours of light[END_REF]. He stated that seven primaries (red, orange, yellow, green, blue, indigo and violet) can be mixed to reproduce any visible colours. He based his theory on the observation of the sun light dispersed by a prism. Later, mathematical and experimental foundation was refined by Young [START_REF] Young | The bakerian lecture: On the theory of light and colours[END_REF][START_REF] Young | An account of some cases of the production of colours, not hitherto described[END_REF], Grassmann [START_REF] Grassmann | Zur theorie der farbenmischung[END_REF] Helmholtz [START_REF] Helmholtz | Ueber die theorie der zusammengesetzten farben[END_REF][START_REF] Helmholtz | Ueber die zusammensetzung von spectralfarben[END_REF] and Maxwell [START_REF] Clerk | Xviii.-experiments on colour, as perceived by the eye, with remarks on colour-blindness[END_REF][START_REF] Maxwell | On the theory of compound colours, and the relations of the colours of the spectrum[END_REF] with the theory of three primaries.

To determine the visual sensitivity to light, the Commission Internationnale de l'Éclairage (CIE), conducted a series of experiments from 1920 to lead to the CIE 1931 XYZ colour space. The CIE 1931 2 • colour matching functions (CMFs) are based on the CIE 1924 V (λ) photopic luminosity function. This photopic luminosity function underestimate the importance of eye sensitivity below 460 nm. Stiles and Burch re-conduct the experiment to refine the CMFs among 10 observers for 2 • matching field [START_REF] Stiles | Interim Report to the Commission Internationale de l'Eclairage, Zurich, 1955, on the National Physical Laboratory's Investigation of Colour-matching[END_REF] and for 10 • matching field [START_REF] Stiles | Colour-matching Investigation: Final Report[END_REF].

An observer was facing a screen with two patches of light and a series of three knobs. One patch, the reference, is made of a monochromatic light and the observer have to manipulate the knobs controlling the amount of three primaries of almost monochromatic lights to match the colour of the second patch with the reference (r = 648.6 nm, g = 526.4 nm, b = 445.3 nm in the CIE 1931, r = 645.2 nm, g = 526.3 nm, b = 444.4 nm in [START_REF] Stiles | Interim Report to the Commission Internationale de l'Eclairage, Zurich, 1955, on the National Physical Laboratory's Investigation of Colour-matching[END_REF][START_REF] Stiles | Colour-matching Investigation: Final Report[END_REF]). To match some colours, negative values have to be used for the knobs. A negative position on the knob is simulated by adding the corresponding opposite value to the reference patch. Figure 4.2 shows the resulting curves for the colour-matching function.

To overcome the problem of having negative RGB coordinates, the CIE proposed the CIE 1931 2 • XYZ colour space. Similar to Stiles and Burch RGB corrected visibility function for V (λ) below 460nm, Judd [START_REF] Judd | Report of u.s. secretariat committee on colorimetry and artificial daylight[END_REF] and later Vos [START_REF] Vos | Colorimetric and photometric properties of a 2 • fundamental observer[END_REF]

Reflectance and colours

When characterising a reflectance by its colour, it is important to also specify under which illuminant it is observed. For example, Figure 4.4 shows a Macbeth colour checker lit by different illuminants. The illuminant used to lit the scene drastically influences the perceived colours of the patches. Under D65, some patches appear blue while they are looking violet under fluorescent illuminant FL4.

From spectra to tristimulus

To transform a spectra into a perceived colour, we use colour matching functions. Usually, we first transform spectra to XYZ coordinates. Then, we transform XYZ coordinates to a specific colour space depending on the targeted application.

In computer graphics, we typically display an image on a 8bit RGB monitor. We transform XYZ to RGB values, then gamma correct to sRGB.

Radiance

To get XYZ coordinates of a radiance spectrum, we multiply and integrate the spectral radiance with the colour matching functions:

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ X = ∫ Λ L e (λ)x ¯(λ)dλ Y = ∫ Λ L e (λ)y ¯(λ)dλ Z = ∫ Λ L e (λ)z ¯(λ)dλ (4.3)
Notice from equation 4.1, the luminance can be retrieved from the Y component in photopic conditions by multiplying it with the corresponding K m .

Reflectance

We mentioned in section 4.1.3 the perceived colour of reflectance depends on the illumination under which it is observed. Then, we specify XYZ coordinates of a reflectance relative to an illuminant:

⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ X = 1 N ∫ Λ s(λ)L e (λ)x ¯(λ)dλ Y = 1 N ∫ Λ s(λ)L e (λ)y ¯(λ)dλ Z = 1 N ∫ Λ s(λ)L e (λ)z ¯(λ)dλ (4.4) With N = ∫ Λ L e (λ)y ¯(λ)dλ.
Where:

• L e (λ) is the spectral radiance of the illuminant.

• s(λ) is the spectral reflectance.

Metamerism

A spectrum defines a colour, but a colour does not define an unique spectra: multiple spectra can lead to the observation of the same colour due to the way humans perceive colours. In mathematical terms, there is a surjection between spectra and colours.

While some patches have a clear colours difference in Figure 4.4 under D65 (green and yellow), it is much harder to see the difference between the same patches under FL4 or A illuminants. The extreme cases is when two different reflectance spectra appears the same under a given illuminant. They are then called metamers under this illuminant. When changing illuminant, the difference between the reflectance may appear.

In practice, metamers can easily appear when two surfaces are lit with purely monochromatic light. If the surface is not fluorescent, it only reflects the incoming wavelength (see Figure 4.6). This is typically the case with street lighting using low pressure sodium-vapour lamps which are nearly monochromatic.

Figure 4.5 shows a example of two metamers for a given illuminant. One reflectance is taken from one of the Macbeth colour checker. The second reflectance is synthetically generated to produce the same perceived colour under HP1 illuminant (code is provided in Appendix A).

Spectral and tristimulus pipelines

A colour does not properly define an illuminant or a reflectance due to metamerism. Still, many rendering pipelines rely on tristimulus values, whether renderers, or softwares used by artists to edit textures, for multiple reasons: If only two LEDs are used,some colours are impossible to distinguish while being very different. For instance, with red and blue lighting (h), yellow and red (on the fish's tail) look very similar.

Efficiency

In the early days, with limited computational resources, the exclusive usage of tristimulus values enabled producing colour images with reasonable compromise on accuracy. A consequence of this long-term usage is the knowledge acquired on weaknesses and drawbacks of the tristimulus pipeline.

Transition From a software point of view, moving from a tristimulus to a spectral workflow would imply an important codebase adaptation. Years of development resulted in software working correctly in tristimulus with known weaknesses. Transitioning to spectral would potentially break many features.

Intuitiveness For artists, it is no easy task to switch from colours to spectra paradigm, the latter being far less intuitive to work with.

Tristimulus renderers are approximative. In some circumstances, they produce images very different from what a real scene would look like. This is especially problematic for prototyping applications. An architect or a car manufacturer could have misleading information about the appearance of paints depending on lighting conditions.

In the following sections we review some of the approximations caused by tristimulus rendering.

The core approximation of a tristimulus renderer

In a spectral renderer, we treat each wavelength independently. We transform to tristimulus values only at the end of the evaluation. Each reflectance and illuminant have then to be specified as a spectrum.

In a tristimulus renderer, we treat each reflectance and illuminant as tristimulus values and multiply them. The integration transforming the spectra to tristimulus values is applied prior to the evaluation. In practice, the values are directly provided in XYZ or RGB and multiplied with each other. Equation 4.5 shows the core approximation made by a tristimulus renderer1 . In this equation, we apply a normalisation factor to the illuminant. This results in a correct result only in the case of single bounce if the illuminant we use to lit the scene match the one we use for reflectance tristimulus values. In practice, the need for this correction factor is arguable:

• If the illuminant is seen directly, this leads to incorrect colour result.

• While getting accurate results in single bounce and same illuminant scenario, when changing illuminant, it can cause more discrepancies.

• In a multiple bounce scenario, each element reflecting in the scene would act as a secondary light source. The re-normalisation factor is unpredictable and can increase the error.

∫ Λ r(λ)L e (λ)c ¯(λ)dλ ⏞ ⏟⏟ ⏞ Spectral rendering ≈ Tristimulus reflectance ⏟ ⏞⏞ ⏟ ( 1 N m ∫ Λ r(λ)L m e (λ)c ¯(λ)dλ ) • Tristimulus illuminant ⏟ ⏞⏞ ⏟ ( N m ∫ Λ L e (λ)c ¯(λ)dλ ∫ Λ L m e (λ)c ¯(λ)dλ ) ⏞ ⏟⏟ ⏞
Tristimulus rendering (4.5)

N m = ∫ Λ L m e (λ)y ¯(λ)dλ (4.6)
Where:

• r(λ) is the spectral reflectance.

• L e (λ) is the spectral power distribution of the illuminant for rendering.

• L m e (λ) is the spectral power distribution of the illuminant for measuring reflectance.

• c ¯(λ) is one of the colour matching functions (x ¯, y ¯, z ¯).

Comparing spectral and tristimulus rendering

We evaluate the approximation made by a tristimulus rendering pipeline influences the final appearance. We compare both methods:

1. In a single bounce context with variation of reflectance and illuminant. We use the Macbeth colour checker as the base collection of reflectance. Each patch is specified as a reflectance spectra. We show: a) Colour discrepancies when using a different illuminant than the one used for measurement, b) Limitations of tristimulus rendering threw reflectance and illuminant metamers.

Single bounce

Reflectance rendering under different illuminants The illuminant influences the perceived colour of a reflectance. Figure 4.7 shows the impact of the pre-integration made in a tristimulus renderer on the colour accuracy using three different illuminants:

D65 is from the CIE series of daylight illuminants. This illuminant represent the average daylight illumination in Western Europe. Its spectral power distribution is well distributed and provides good ability for distinguishing colours.

FL4 is from the CIE series of fluorescent illuminants. Also called F4, this illuminant corresponds to a fluorescent lighting. The CIE used FL4 for colour rendering index calibration. It has two peaks around 410 nm and 435 nm and a larger emission band from 520 nm to 650 nm.

HP1 illuminant corresponds to a high-pressure vapour lamp. It has a narrow emission band roughly from 550 nm to 620 nm. It is difficult to differentiate colours under this illuminant.

For tristimulus rendering, we convert each patch of the colour checker to XYZ values as they would be measured under a given illuminant (here CIE D65). We convert the illuminant to XYZ values and correct it to match the measurement illuminant. We multiply the two XYZ sets together.

• With D65, the illuminant matches the one used for measuring the reflectance, the tristimulus rendering is exact. This is expected since the integration is made during the measurement and then a simple scaling is applied depending on the relative power of the illuminant used for rendering.

• With FL4, the error increase for the coloured patches. When reflectance spectrum have its most reflectivity in values within the re-radiation band of the FL4 illuminant (520 ∼ 630 nm), it can be well represented with a tristimulus renderer. The fluorescent lamp also have an emission peak around 400 nm and around 440 nm. Those two peaks are going to misrepresented: mostly X and Z values are affected. Those two functions have a much larger bandwidth than the emission peaks. This explains the poor rendering accuracy of the blue part of the spectrum. • This high frequency phenomena is even more pronounced with HP1 illuminant: the emission band is narrow and cannot be well represented by multiplications with XYZ values.

• The grey reflectance, on the bottom line, have good rendering accuracy in a tristimulus renderer. Their reflectance spectrum is almost constant. This limited dependence to wavelength explains the good rendering results regardless the illuminant used.

In conclusion, the error caused by a tristimulus renderer is limited when we have low frequency illuminant or reflectance spectra.

Reflectance metamers A tristimulus renderer will not handle reflectance metamers. When observed under another illuminant, the reflectance will appear identical while they should differ according to their reflectance spectra: the same XYZ values are used in both cases even if reflectance spectra differs. Figure 4.8 shows an example of two colour checkers. One is the original Macbeth colour checker. For the other, we replaced each of its patches by a metamer reflectance under an illuminant with a flat SPD. Only a spectral renderer accurately shows the difference between the two colour checkers depending on the illuminant.

Illuminant metamers

Similarly to metamer reflectance, we can generate metamer illuminant.

If the illuminant is reflected by a perfectly grey surface (constant reflectance regardless the wavelength), the surface will appear the same to our eyes with both illuminants even though their emission spectra differ. Other reflectance are going to look different.

In Figure 4.9 we show a such scenario. We generate a metamer illuminant for D65. We then compare the Macbeth colour checker lit by the original D65 illuminant and its metamer.

• With a spectral rendering, the patches have a different colour from one illuminant to the other. This is expected: the illuminants have different spectral power distributions.

• With a tristimulus rendering, there is no difference when changing illuminant. The two metamer illuminants are transformed to the same XYZ values prior to the rendering.

Only a spectral renderer with spectral reflectance can handle metamer illuminants.

Global illumination

Global illumination increases the discrepancies exposed before. At each new reflection bounce event, a tristimulus renderer accumulates error. The corresponding XYZ values are in the bottom left of each patch.

Even if we specify all reflectance using the illuminant in the scene, each light bounce step is equivalent to a new illuminant radiating the next reflectance. This yields to incorrect results and breaks energy conservation, starting from the second light bounce.

Energy conservation To illustrate the potential loss and creation of energy caused by a tristimulus renderer, we generate two synthetic cases:

• In the first (see Figure 4.10a), we simulate scattering from D65 illuminant to a patch and a second bounce from this first patch to another one having the same reflectance. The reflectance is perfectly reflecting the light between 440 nm and 550 nm. Otherwise, the light is absorbed. We observe a loss of energy in the case of a tristimulus renderer: the XYZ components of the reflectance cannot reflect the nature of the spectra.

• In the second (see Figure 4.10b), we use the first spectrum as the one used before. For the second bounce, we use a complementary spectrum: its reflectance is null at the value where the other spectrum reflects and perfectly reflective where the other absorbs light.

We observe creation of energy in the case of a tristimulus renderer: not null XYZ values are multiplied together.

We do not attenuate the light depending on its angle nor considering the distance: we are only interested in showing the absence of energy conservation caused by the multiplication of XYZ values instead of spectra.

Cornell box example To show a practical example of discrepancies observable with a simple scene, we use the Cornell box rendered with Mitsuba. The spectral variant is using 30 spectral samples while the tristimulus variant is converting all spectra to RGB prior to light transport.

The appearance drastically change if using tristimulus values instead of the full spectrum for rendering (see Figure 4.11).

For the same input spectra, the colour matching functions used in a tristimulus renderer can also drastically change the appearance of the output image (see Figure 4.12)).

The error would significantly increase with multiple light sources having different spectra.

Discussion

Drawbacks of spectral rendering

The major drawback of a spectral renderer is the need to specify a spectrum for each illuminant and reflectance. Most tools provided to artists are designed with tristimulus in mind. Spectral uplifting overcomes this problem by creating plausible spectrum from tristimulus values. Jakob and Hanika [START_REF] Wenzel | A Low-Dimensional Function Space for Efficient Spectral Upsampling[END_REF] recently proposed a method to convert tristimulus textures to plausible spectral reflectance. Another drawback of a spectral renderer is its inefficiency. While a tristimulus renderer evaluates directly a colour, a spectral renderer adds a dimension to the evaluation. In a path tracer, it means, in addition to the ray direction sampling, the wavelength of the ray also have to be sampled. Usually, in a spectral path tracer, a ray carries a single wavelength. This results in colour noise in low sampled images. However, Hero Wavelength Spectral Sampling (HWSS) technique drastically limit this phenomena carrying multiple wavelengths as the same time with almost the same computational time taking advantage of Single Instruction Multiple Data (SIMD) hardware capabilities [WND + 14]. rendering modes. We observe significant differences between the two rendering methods.

Advantages of spectral rendering

Spectral rendering is more accurate compared to its tristimulus counterpart, avoiding energy leaks. It also simulates effects impossible to handle correctly in a tristimulus renderer: light dispersion, fluorescence, and polarisation. Those effects are wavelength dependent and thus require consideration for spectra. Metamerism is another major issue a tristimulus renderer cannot deal with. While two reflectance spectra can be different, their tristimulus counterparts can be the same under a given illuminant. Only a spectral renderer is able to show the difference between those two spectra when changing the illuminant.

Finally, a spectral renderer is agnostic to colour matching functions. The final image can be saved as a spectrum and later converted to a tristimulus display for instance an RGB monitor. The monitor colour response and the observer one can be applied directly to the spectral image to get correct colourimetry. On the other hand, a tristimulus image has to rely on chromatic adaptation.

Conclusion

We presented how light is perceived by the human vision system and experiments at the origin of the colour matching functions used for transforming spectra to tristimulus coordinates. We use XYZ coordinates to avoid negative values. Starting from these coordinates, we convert to different colour spaces depending on the application. In computer graphics, we mostly use RGB primaries. For computer graphics applications, tristimulus representations have a major limitation: different spectra can be represented with the same tristimulus coordinates. This can result in important discrepancies using tristimulus rendering pipelines. Spectral rendering is immune to this problem.

However, moving a well-established tristimulus pipeline to spectral is challenging. It requires important code rewriting. We have to specify each reflectance and illuminant in their spectral form. Editing software used by artists are not yet suited for spectral application and using spectral paradigm instead of colour is less intuitive.

Finally, material acquisition is usually performed using tristimulus coordinates. Recently, efficient spectral acquisition techniques were proposed for non-spatially varying materials. For spatially varying material, is out of reach though at the time of writing this thesis. Storage and acquisition time issues, already problematic with tristimulus representations, would drastically increase.

Tristimulus rendering offers a decent approximation in most of the cases and provides efficient evaluation. For critical applications, it is important to be well aware of its limitations though. We cannot use it with line spectra (low pressure vapour lamps for instance) and it is incapable of handling metamers. Particular wavelength dependent effects such as fluorescence and polarisation are also out of reach for a tristimulus renderer. • The Ocean Optics Web Book Toutefois, passer d'un système trichromatique éprouvé à un système spectral est difficile. Cela nécessite une importante ingénierie logicielle. De plus, toute réflectance et tout illuminant doit être alors spécifié en spectral. Enfin, les systèmes d'édition de texture utilisés par les artistes ne sont pas encore adaptés pour le support du spectral. L'utilisation de données spectrale est aussi moins intuitive. L'acquisition de matériaux se fait aussi généralement en trichromatique. Certaines méthodes ont été proposées afin de mesurer des matériaux efficacement en spectral. Mais, ces méthodes sont pour l'instant restreintes aux matériaux uniformes. Pour les matériaux variants spatialement, l'acquisition spectral est hors de portée pour l'instant : les problèmes de stockage et de temps d'acquisition déjà problématiques pour les matériaux uniformes s'exacerbent avec des matériaux variants spatialement.

Sources & references

Le rendu trichromatique bien qu'approximatif offre des résultats d'une qualité correcte avec une importante efficacité de calcul. Pour ces applications critiques, il reste important d'être conscient de ses limitations : il est incapable de prendre en charge les spectres de raies ou les métamères. Enfin, certains effets dépendants dans la longueur d'onde comme la polarisation ou la fluorescence sont hors de portée des moteurs de rendu trichromatiques.

CHAPTER 5

The microfacet model W hen light hits a surface, it is scattered in different directions depending on the material. For a given incoming direction, the amount of energy scattered in various directions is given by the Bidirectional Reflectance Distribution Function (BRDF), introduced in [Nic65] for opaque materials. A more general term is given by the BSDF, the Bidirectional Scattering Distribution Function. We are going to introduce the microfacet BRDF model.

In this chapter, we use geometrical optics framework and may refer to wave optics interpretation: it convey a good intuition of light transport in some instances.

In the first section, we present basic interaction of light with matter. In a second section, we explain how scattering is modelled in computer graphics. We present some of the early models used, then we introduce the microfacet theory. This physically based model takes its roots on the interaction of light with mirror.

Basic interaction with matter

Light propagates along paths in the geometric optics interpretation. A path materialise the propagation of a photon and is potentially redirected when interacting with matter.

Fermat's principle

A photon propagates from a point to another on trajectories such as the travel time is locally minimal. This describes an optical path is the foundation of the geometrical optics laws.

Consequently, in an isotropic medium, light travels on strait direction and can be modelled as a ray by its origin and propagation direction. The Huygens' principle provides a good insight of the way a wavefront propagates.

Huygens' principle

Following this principle, a wavefront is a superposition of "spherical wavelets". The next wavefront is formed by the sum of those wavelets and propagate following the same principle. In between two wavefronts, the interferences between those wavelets explains the amplitude variation. In summary, a wavefront can be seen as an iso-value of the maximum amplitude at a given time (see Figure 5.1).

Huygens used this principle to explain diffraction and refraction.

Mirror reflection

When light hits a mirror, it is reflected in a single direction. This reflected direction is the symmetric of the incoming ray with respect to the normal of the mirror's surface (see Figure 5.2). The reflected intensity is not equal at every angle. This is later characterised using Fresnel equations.

Media interface and refraction

The direction of light propagation is bent when getting though a boundary of two homogeneous media with different propagation speed. The light is said to be refracted.

The index of refraction of a medium is the relative speed of light in a vacuum with its speed in the medium.

n = c v (5.1)
Where:

• n is the index of refraction.

• c is the speed of light in vacuum.

• v is the speed of light in the medium.

Given the two indices of refraction, the ray hitting the boundary of two homogeneous media is deflected following the Snell-Descartes' law (Figure 5.3a):

n 1 sin θ i = n 2 sin θ o (5.2)
Where:

θ i θ o n 1 , v 1 n 2 , v 2 (a)
Light refraction when a ray get through a boundary of media with different light propagation speed.

λ 1

λ 2 = v 2 v 1 λ 1 n 1 , v 1 n 2 , v 2 (b)
Huygens' principle explanation of the phenomena: in the second medium, the travel speed is slower, hence, the wavelength is shorter. The interferences deviate the wavefront from its original orientation of propagation at the boundary • n 1 , n 2 are the indices of refraction of the two media.

• θ i is the angle of the incident ray with the medium boundary's normal.

• θ o is the angle of the exitant ray with the medium boundary's normal.

An intuitive understanding of this phenomena was given by Huygens [START_REF] Huygens | Traité de la lumière[END_REF]. When entering at the boundary of two medium, the wavefront that can be decomposed as superposition of wavelets following Huygens' principle, those wavelets are travelling with a different speed. This cause a change of wavelength (see equation 3.1). The resulting wavefront in the second medium is obtained by the interferences between those wavelets. This deviate the propagation axis of the wavefronts. See Figure 5.3b.

Fresnel equations

In the previous section, we've described how light paths can be redirected when interacting with matter. We haven't quantise the impact on the energy transported after those interactions. Fresnel equations are needed if wanting to know how much of the energy is reflected or refracted depending on the incoming angle, polarisation state of the light, and index of refraction of the

F ⊥ (θ, η) = ⃓ ⃓ ⃓ ⃓ cos θ -c i cos θ + c i ⃓ ⃓ ⃓ ⃓ 2 tan δ ⊥ = 2 • cos θ cos 2 θ -|c i | 2
(5.3)

F ∥ (θ, η) = ⃓ ⃓ ⃓ ⃓ η 2 cos θ -c i η 2 cos θ + c i ⃓ ⃓ ⃓ ⃓ 2 tan δ ∥ = 1 2 • cos θ • (c i -c i ) • (η 2 • c i -η 2 • c i ) |η| 4 • cos 2 θ -|c i | 2
(5.4)

With:

η = n + ik c i = ( η 2 -sin 2 θ ) 1 2 (5.5)
Where:

• F ⊥ is the reflected intensity for perpendicular polarised light (x component of the electric field).

• F ∥ is the reflected intensity for parallel polarised light (y component of the electric field).

• δ ⊥ is the phase retardance for perpendicular polarised light.

• δ ∥ is the phase retardance for parallel polarised light.

• η is the complex index of refraction.

Unpolarised light

The whole expression of Fresnel equations are needed only if interested in simulating polarised light. While this has some important effects on inter-reflections and scattering, light polarisation is more often ignored in computer graphics. In such case, the Fresnel term can be simplified:

F unpolarised (θ, η) = F ⊥ (θ, η) + F ∥ (θ, η) 2 (5.6)
Schlick approximation A commonly used approximation for Fresnel term was proposed by Schlick [START_REF] Schlick | An Inexpensive BRDF Model for Physically-based Rendering[END_REF].

F (θ, η) = F (0, η) + (1 -F (0, η))(1 -cos θ) 5
(5.7)

Mueller calculus

When light interacts with matter, the polarisation state can change. To support light polarisation, a path tracer can use Stoke vectors. Mueller matrices transforms a Stoke vector to change accordingly its polarisation state. Table 5.1 introduces some of commonly used matrices. A Stoke vector have to be specified within a frame (see Chapter 3). Before multiplication with a Mueller matrix, a Stoke vector may have to be transformed to share the same frame. So, it is important to keep track of reference frames, prior and after the transformation. For in depth details about polarisation computation and path tracing, see [START_REF] Wilkie | How to Write a Polarisation Ray Tracer[END_REF].

Attenuator

Depolariser Linear polarisation filter rotated by an angle φ against x axis

⎡ ⎢ ⎢ ⎣ α 0 0 0 0 α 0 0 0 0 α 0 0 0 0 α ⎤ ⎥ ⎥ ⎦ ⎡ ⎢ ⎢ ⎣ α 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ⎤ ⎥ ⎥ ⎦ 1 2 ⎡ ⎢ ⎢ ⎣ 1 cos 2φ sin 2φ 0 cos 2φ
cos 2 2φ sin 2φ cos 2φ 0 sin 2φ sin 2φ cos 2φ sin 2 2φ 0 0 0 0 0

⎤ ⎥ ⎥ ⎦ Fresnel reflectance ⎡ ⎢ ⎢ ⎢ ⎣ F ⊥ +F ∥ 2 F ⊥ -F ∥ 2 0 0 F ⊥ -F ∥ 2 F ⊥ +F ∥ 2 0 0 0 0 cos(δ ⊥ -δ ∥ ) √ F ⊥ F ∥ sin(δ ⊥ -δ ∥ ) √ F ⊥ F ∥ 0 0 sin(δ ⊥ -δ ∥ ) √ F ⊥ F ∥ -cos(δ ⊥ -δ ∥ ) √ F ⊥ F ∥ ⎤ ⎥ ⎥ ⎥ ⎦
Table 5.1.: Mueller matrices of some common optical elements.

Toward complex material models

In computer graphics, distinction is made between empirical, physically plausible and physically based models for describing material scattering properties.

Empirical models are built upon observation to give a convincing appearance when rendering.

They are not necessary physically plausible.

Physically plausible models are model that satisfies laws of physics in term of positivity, energy conservation and reversibility: the result shall be the same if the observer and the light source are reversed due to Fermat principle. The trajectory followed by the light to go from one point to another doesn't depend on the propagation direction.

Physically based models are based on physical laws to describe light scattering upon a theoretical construction of the matter at a micro-scale. They necessary physically plausible.

BSDF

The most general case to describe how a surface scatter the light is the Bidirectional scattering distribution function (BSDF). In this thesis, we focus on the BRDF. The BRDF was defined in 1965 by Fred Nicodemus [START_REF] Nicodemus | Directional reflectance and emissivity of an opaque surface[END_REF]. It is the differential ratio of exiting radiance by incident irradiance. It is expressed per unit of solid angle [sr

BSSDF

-1 ]. f r (λ, ω i , ω o ) = dL eo (λ, ω o ) dE e i (λ, ω i ) [sr -1 ] (5.8)
The total spectral reflectance of the surface is:

r(λ) = ∫ ∫ Ω 2 f r (λ, ω i , ω o ) < ω i • n > dω i dω o
(5.9)

Early models

The first method used to represent plain surfaces instead of wire frame geometry was the flat shading technique. Each polygon had a constant intensity along its surface. Later, Henri Gouraud introduced a method to interpolate along the surface of the facet to give a more convincing appearance [START_REF] Gouraud | Computer display of curved surfaces[END_REF][START_REF] Gouraud | Continuous Shading of Curved Surfaces[END_REF]. It was later called the Gouraud shading after Bui Tuong Phong presented his technique for shading referred as Phong shading [START_REF] Bui Tuong | Illumination for computer generated pictures[END_REF] and compared it against Gouraud one. Blinn [START_REF] Blinn | Models of light reflection for computer synthesized pictures[END_REF] later improved the Phong shading and is referred in the literature as the Blinn-Phong shading. The Gouraud shading is simple to implement and can be hard wired. It was one of the technique of choice for early specialised hardware.

Blinn-Phong shading is not physically based but cheap to evaluate. Hence, it remained popular until recently in game engines to render specular surfaces. With increase of computational power, physically based shading are preferred to render such surfaces, even for real time rendering.

Rusinkiewicz parametrisation

A convenient parametrisation of BRDF was introduced by Rusinkiewicz [START_REF] Szymon | A new change of variables for efficient brdf representation[END_REF]. In this parametrisation, incoming and outgoing vectors are replaced with half vector ω h and difference vector ω d (see Figure 5.7). This parametrisation makes it convenient for expressing normal distribution function and Fresnel term independently in microfacets models (discussed in section 5.2.4):

The half vector ω h is the normalised sum of the ingoing ω i and outgoing ω o vector. It corresponds to the axis of symmetry between those two vectors. Hence, only the facets having their normal aligned with this half vector contributes to the reflectance.

The difference vector ω d corresponds to the vector ω i expressed in the frame defined with the half vector ω h as the z axis. For facets aligned with the half vector, the corresponding Fresnel coefficient can then be determined solely by this difference vector.

Those vectors can be calculated with:

ω h = ω i + ω o ∥ω i + ω o ∥
(5.10)

ω d = rot b,-θ h rot n,-ϕ h ω i (5.11)
The inverse mapping is given by:

ω i = rot n,ϕ h rot b,θ h ω d (5.12) ω o = 2(ω i • ω h )ω h -ω i
(5.13)

The microfacets model

Cook and Torrance [START_REF] Cook | A reflectance model for computer graphics[END_REF] introduced in 1982 the microfacets model for computer graphics. This model is based on a statistical construction to express how light is scattered after hitting a surface.

f r (ω i , ω o , λ) = ρ d (λ) π + F (θ d , λ) • D(ω h ) • G(ω i , ω o ) 4 • cos(θ i ) • cos(θ o ) [sr -1 ] (5.14)
Where:

Microfacets (micro-scale)

Surface (macro-scale) • D(ω h ) is the Normal Distribution Function (NDF).

• F (θ d , λ) is the Fresnel term.

• G(ω i , ω o ) is the shadowing-masking term.

In the next paragraphs, we review the three terms that appear in Cook-Torrance model.

Micro surface geometry & normal distribution function

At the microscopic scale, a surface is considered as a collection of small facets acting as perfect mirrors (see Figure 5.8). The distribution of those facets defines how shiny or rough as surface is. For instance, if all of those facets are aligned the same way, light is going to be reflected to a single direction making the surface a perfect mirror. On the other hand, if the facets are perfectly distributed on the hemisphere, i.e, for any direction the light can come from, there is the same number of facet aligned with the light, the surface will be perfectly diffuse: no direction of reflection will be privileged.

Given an incoming and outgoing direction, we have an half vector. Only the micro mirrors having their normal aligned with this half vector will reflect the light: we've seen a mirror reflects an incoming ray to a single direction, the symmetric with respect to the normal of the mirror. We are interested in the distribution of normals of those microfacets. This is given by the Normal Distribution Function (NDF) D(ω h ). It is measured in square meters per steradian [m 2 .sr -1 ]. The distribution is isotropic when it does not change when both ω i and ω o are rotating around the surface normal. Otherwise, it is anisotropic.

We can used different normal distribution functions:

• Beckmann:

-Isotropic:

D(ω h ) = χ + (ω h • n l ) e -tan 2 θ h α 2 πα 2 cos 4 θ h (5.15)
-Anisotropic:

D(ω h ) = χ + (ω h • n l ) e -tan 2 θ h ( cos 2 φ h α 2 x + sin 2 φ h α 2 y )
πα x α y cos 4 θ h (5.16)

• GGX [WMLT07]:

-Isotropic:

D(ω h ) = χ + (ω h • n l ) πα 2 cos 4 θ h ( 1 + tan 2 θ h α 2
) 2 (5.17)

-Anisotropic:

D(ω h ) = χ + (ω h • n l ) πα x α y cos 4 θ h ( 1 + tan 2 θ h ( cos 2 φ h α 2 x + sin 2 φ h α 2 y )) 2 (5.18)
Where:

• α is the isotropic roughness parameter.

• α x , α y are the anisotropic roughness parameters.

The NDF is causing most of the BRDF aspect.

Fresnel terms

The amount of light being reflected depends on the incident vector angle with the mirror surface. The Fresnel terms quantise how much of the energy is reflected2 . Fresnel terms are responsible of the specular colour of the material and are influenced by the polarisation state of the light.

The angle between the considered micro mirror normal ω h and the incoming light ω o is θ d . We multiply the NDF by the Fresnel term F (θ d , λ) (see section 5.1.5).

Shadowing Masking & Distribution of heights

In the original microfacet theory, we only consider a single bounce. To approximate multiple bounces, we add a Lambertian term to the direct specular term from the microfacets. While this is a pretty good approximation for specular materials, rougher surfaces may be very much influenced by multiple scattering effects within microfacets. Later works tackle this problem of multiple scattering between microfacets.

The NDF tells us how many micro mirror "may" reflect light. We need to modulate this value by knowing how many of those path are going to exit the surface after the first bounce. This is the role of the shadowing masking term. Another statistic tells about the geometry of the surface giving this shadowing masking function. It is called the Geometric Attenuation Function (GAF). This statistic is derived from the distribution of height.

Multiple masking models were proposed [START_REF] Oren | Generalization of Lambert's Reflectance Model[END_REF][START_REF] Schlick | An Inexpensive BRDF Model for Physically-based Rendering[END_REF]. Heitz [START_REF] Heitz | Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs[END_REF] provides an in depth review of the shadowing-masking term. We use the Smith model [START_REF] Smith | Geometrical shadowing of a random rough surface[END_REF] which is considered the most plausible:

G 1 (ω, ω h ) = χ + (ω • ω h ) 1 + Λ(ω) (5.19)
We get the shadowing-masking term from the correlated bistatic Smith shadowing function [HMD + 14]:

G(ω i , ω o ) = G 1 (ω i ) • G 1 (ω o ) G 1 (ω i ) + G 1 (ω o ) -G 1 (ω i ) • G 1 (ω o )
(5.20)

We get Λ(ω) for the corresponding normal distribution function we use:

• Beckmann:

-Exact form:

Λ(ω) = erf(a) -1 2 + 1 2a √ π e -a 2
(5.21)

-Approximation from Walter et al. [WMLT07] Λ(ω) = ⎧ ⎨ ⎩ 1 -1.259a + 0.396a 2 3.535a + 2.181a 2 if a < 1.6 0 otherwise (5.22)
(5.23)

• GGX:

Λ(ω) = -1 + √ 1 + 1 a 2 2 (5.24)
Where:

a = ⎧ ⎨ ⎩ 1 α tan θ for an isotropic distribution 1 √ α 2
x cos 2 φ+α 2 y sin 2 φ tan θ for an anisotropic distribution (5.25)

Conclusion

In this chapter, we introduced the basic concepts of light interaction with matter. We presented how light interacts with a mirror and how it is refracted when its changing medium of propagation. The relationship between the speed travel and wavelength explains the bent of the ray. This can be interpreted either with Fermat's principle in geometric optics or with Huygens' principle in wave optics. We introduced the Fresnel equations giving the amount of energy reflected or refracted depending on light polarisation and index of refraction. The polarisation effects can be handled using Mueller calculus along with Stokes' vectors. Different scattering functions are used in computer graphics. BSDF is the most generic one. We focus on the BRDF in the course of this thesis. The microfacet model is the dominant analytic BRDF model used in computer graphics at the time of writing this thesis. Light mirror interactions is the foundation of this model. It is based on a distribution of tiny mirrors at micro scale.

Its simplicity, versatility, and physically based approach explain its popularity in computer graphics. It is also the model of choice when wanting to fit a measured material: it is both used in real time and offline rendering.

A large range of work proposed alternative microfacet distributions. Recent works also extended it for handling of wave effects such as diffraction and interferences. Différentes fonctions pour la diffusion de la lumière par une surface sont utilisées en informatique graphique. Les BSDF sont les plus générales. Nous nous intéressons aux BRDF dans cette thèse. Le modèle à microfacettes est le modèle analytique dominant en informatique graphique au moment de l'écriture de cette thèse. Il est basé sur un distribution de micro miroirs à l'échelle microscopique.

Sources & references

Sources

Sa simplicité, sa modularité et sa fondation physique explique sa popularité. Il s'agit du modèle privilégié lorsqu'il s'agit de modéliser des réflectances mesurées : il est aussi bien utilisé pour le rendu temps réel que pour le rendu non temps réel.

Beaucoup de travaux ont proposé des distributions alternatives depuis son introduction. Des travaux récents l'ont aussi étendu pour supporter des phénomènes ondulatoires comme la diffraction ou les interférences.

CHAPTER 6

Taxonomy of material acquisition techniques I n the previous chapter, we introduced analytic functions to represent a material reflectance given an illumination angle and viewing position. Then, an artist has to tweak the formula's parameters to obtain the desired appearance. To represent materials, we can also use measured reflectance. Material acquisition, unlike a photograph, must capture multiple view and illumination directions. While a photograph will provide an accurate representation of a material for a given view orientation and lighting condition, it will not provide any information on the material property such as its shininess or structure: it is not possible to deduce accurately from a single photograph how a material would look like with a novel point of view or different lighting conditions. For material acquisition we need multiple view and illumination directions.

Acquiring measured reflectance data is important for many application domains of computer graphics: for cultural heritage to archive artefacts, for prototyping to get accurate simulations.

We can differentiate different main acquisition strategies:

Brute force acquisition captures the reflectance for all combination of viewing and illumination angles. This is the preferred method for accurate measurements, but a lengthy process.

One shot multiple samples allows to retrieve multiple views and illuminations at once using a special setup and speeds-up the acquisition process by lowering the number of measurements to make.

Sparse sampling makes the acquisition process faster by analysing how a material can be characterised by sampling only few direction. We capture only the most important directions. It limits the amount of directions to capture. At evaluation, we reconstruct the signal from the characteristic captured directions.

Analytic model acquisition of parameters assumes a material is following a specific analytic model. We design an acquisition setup to retrieve the underlying parameters of the model. The resulting materials are not properly speaking measured materials but rather an analytic representation of a real-world material.

Using acquired materials directly is appealing in the quest of accuracy, but the memory overhead make it unsuitable for numerous application needing a wide spectrum of material in the same scene especially for spatially varying materials. Editability is also an issue, measured material are not meant to be edited directly. A range of research target material compression and fitting to analytic BRDF.

In the context of research, measured materials are a valuable source of information to compare them against comparing analytic models. They also can be inspirational for proposing enhanced models that can explain specific appearance not accurately modelled with state of the art models. Holzschuch and Pacanowski [START_REF] Holzschuch | A Two-scale Microfacet Reflectance Model Combining Reflection and Diffraction[END_REF] built their two scaled microfacet model upon the analysis of the MERL Database [START_REF] Matusik | A data-driven reflectance model[END_REF] and proved diffraction can have a significant contribution in the overall appearance of a material.

In this chapter, we present the most popular acquisition techniques used in the literature. We discuss about their strengths and weaknesses.

In the first section, we present techniques for BRDF acquisition. We classify those in the aforementioned categories. In a the second section, we present techniques for acquiring spatially varying materials. Their spatial heterogeneity adds some constraints; most methods for speeding up the acquisition process on BRDF acquisition are not applicable.

BRDF acquisition

The first range of techniques presented targets non spatially varying materials. It range from 3 to 4 dimensions if we consider anisotropy. This spatial homogeneity allows optimisations.

Brute force acquisition

A gonioreflectometer is a device consisting in a controlled arm for moving a sensor and second arm for moving the light source. We place a material sample under those two arms. This allows to take all possible combinations of illumination and viewing to capture a tabulated BRDF.

Discussion

The acquisition time is very long with a brute force approach, taking all possible views and illumination directions. Up to six dimensions are necessary to represent materials (spatially varying anisotropic). Different strategies were proposed to speed-up the acquisition time.

The advantage of this method is its generality. Except for the angular sampling frequency, it does not make any assumption about the material properties.

One shot multiple samples

Acquiring material properties is time consuming, as it requires sampling over all incoming and outgoing directions. For spatially uniform materials, researchers speed-up the acquisition process by exploiting this uniformity: a single photograph samples several directions, assuming the underlying geometry is known.

Ward [START_REF] Ward | Measuring and modeling anisotropic reflection[END_REF] presented an acquisition setup based on a digital camera equipped with a fisheye lens pointing to an half-silvered or hemisphere. This setup was presented along with an energy conservative anisotropic BRDF model. It enables anisotropic measurement of a surface with lower cost and higher speed than the traditional gonioreflectometer method. The main issue with mirror based approach is this add an interaction with another optical element which may decrease the accuracy of the measurements. Ward also mentioned as a limitation the poor ability to get accurate measurements for grazing angles. Marshner et al. [MWLT00, MWL + 99] proposed using an object of a known geometry coated with the material to acquire to speed-up the acquisition process. Their technique relies on the varying surface normal providing from a single photograph varying illumination and viewing angles depending on the pixel capture by a CCD sensor (see Figure 6.1).

BRDF acquisition

Publication

Matusik et al.

[MPBM03a] used a similar approach. They capture images of a sphere coated with the material being measured. To lower acquisition time and concentrate on the capture of high frequency parts of the BRDF, they use Rusinkiewicz parametrisation expecting the higher frequencies distributed around ω h . In addition, they use a non uniform sampling over θ h using a squared root mapping. They acquired 100 materials using this technique. They made this database available and is commonly known as the MERL BRDF database. This database is extensively used in the literature.

Ngan et al. [START_REF] Ngan | Experimental analysis of BRDF models[END_REF] extended the technique for capturing anisotropic materials. By using a rotating cylinder coated with strips of the material with different orientation, they were able to recover the full material.

Filip et al. [START_REF] Filip | Effective acquisition of dense anisotropic BRDF[END_REF] reverted the approach for anisotropic materials: they used a flat material sample, placed the camera close and exploited parallax effects for dense direction sampling.

Hullin et al. [HHA + 10] used a similar setup as Matusik et al. [START_REF] Matusik | A data-driven reflectance model[END_REF] to capture bispectral materials. To retrieve the re-radiation matrix, they added filters in front of a xenon arc lamp and in front of the camera. For a better coverage of the blue and UV range, they used LEDs. A depolarising filter was placed in front of the camera lens: they've shown how polarisation can affect measurements, even for non-coated apparently diffuse surfaces, if not taken into account. Depending on the material type, they also use a cylinder coated with material strips as in Ngan et al. [START_REF] Ngan | Experimental analysis of BRDF models[END_REF] instead of a sphere.

Discussion

In this type of acquisition processes is the optical and electronic aberrations of the camera that occur especially at grazing angles affects the accuracy of the samples. Holzschuch and Pacanowski [START_REF] Holzschuch | The Effects of Digital Cameras Optics and Electronics for Material Acquisition[END_REF] studied this phenomena, and proposed to reconstruct the Modulation Transfer Function prior to the acquisition for better accuracy. Lucat et al. [START_REF] Lucat | Diffraction Prediction in HDR measurements[END_REF][START_REF] Lucat | Diffraction effects detection for HDR image-based measurements[END_REF][START_REF] Lucat | Diffraction Removal in an Image-based BRDF Measurement Setup[END_REF] proposed techniques for diffraction prediction and removal.

Also, those methods require having a specific geometry coated with the material to be acquired. Anisotropic acquisition techniques also are destructive needing to cut strips of material to coat a specific geometry.

They provide a fast way for capturing materials and proven to be versatile for various type of reflectance (isotropic, anisotropic, bi-spectral).

Sparse sampling

Matusik et al. [MPBM03b] investigate on how their dataset ([MPBM03a]

) can be analysed to lower the required amount of measurements. They proposed a common wavelet basis using Haar wavelets and pull-push reconstruction method requiring both 69 000 sparse BRDF samples. They also proposed a linear combination method based on their 100 captured database requiring 6.2. Sampling spatially varying materials [MWL + 99] to capture a BRDF without the need of a gonioreflectometer. An object is coated with a single material and a light source is rotating around. After a calibration phase, the image captured by the camera's sensor can be mapped on points over the surface of the object. This enable capture of multiple views and illuminations for a given material with a single shoot. Matusik et al. [START_REF] Matusik | A data-driven reflectance model[END_REF] used a similar approach but using multiple exposures and Rusinkiewicz parametrisation with more sample concentrated around ω h to improve accuracy of the specular peak. 800 measurements at the expense of storing the whole 100 materials to reconstruct the captured BRDF.

Dupuy and Jakob [DJ18] used a two-pass method to acquire isotropic and anisotropic materials. A first pass identifies the shininess and anisotropy of the materials. The second pass selected the most relevant directions for acquisition based on the information acquired in the first pass. This method enable compact storage and fast evaluation out of the box in a path tracer: the importance sampling used for measuring the material is used when rendering. Furthermore, the database is provided both in RGB and spectral.

Discussion This range of techniques is promising. The materials available in Dupuy and

Jakob database are compact compared to the MERL database and support importance sampling. Currently, it does not support bi-spectral measurements

Sampling spatially varying materials

Spatially varying materials add two dimensions to the acquisition process. This constraint generally makes the speed-up techniques used for non-spatially varying materials not applicable.

For this reason, while non spatially varying materials rarely rely on brute-force acquisition, it is, at the time of writing this thesis, the method of choice to acquire spatially varying materials. Discussion Brute force acquisition for spatially varying materials is even more time consuming than for homogeneous materials. Multiples challenges also comes from the spatially varying nature of those materials. Acquisition need a careful calibration process to ensure perfect alignment of capture. The effective resolution decreases when viewing elevation increases. Samples near grazing angles are unreliable. Many material samples are also non-flat. Their local geometry blends with reflectance. This is one of the challenging issues when we fit BRDF to acquired materials. We discuss this in Chapter 9.

Brute force acquisition

Analytic model acquisition of parameters

Several setup were proposed to simulate a real-world spatially varying material from sparse measurements. Those methods acquire SVBRDFs faster by transforming the problem into acquiring the local parameters of a BRDF model or a mixture of measured BRDF. Those methods base the acquisition process with the later fitting of a reflectance model in mind.

A common technique is to pre-compute material response to illumination, then identify materials on their response to varying incoming light. Gardner et al. [START_REF] Gardner | Linear light source reflectometry[END_REF] a moving linear light source, combined with pre-computed material response to identify spatially varying materials and normal maps. Wang et al. [WZT + 08] identifies similar materials in the different pictures acquired and combines their information for a full BRDF measure. Dong et al. [DWT + 10] extend the approach in a two-pass method: they first acquire the full BRDF data at few sample points, then use this data to evaluate the BRDF over the entire material using high-resolution spatial pictures.

Ren et al. [RWS + 11] begin by acquiring full BRDF information for some base materials. Samples of these materials are placed next to the material to measure, and captured together. By comparing the appearance of the known and unknown materials, they get a fast approximation of spatially varying reflectance. Their method is limited to isotropic reflectance.

Another promising area of research is to record material response to varying incoming light. Holroyd et al. [START_REF] Holroyd | A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance[END_REF] used spatially modulated sinusoidal light sources for simultaneous acquisition of the object geometry and isotropic material. Tunwattanapong et al. [TFG + 13] used a rotating arm with LEDs to illuminate the object with spherical harmonics basis functions, allowing simultaneous recovery of geometry and reflectance properties. Aittala et al. [START_REF] Aittala | Two-shot svbrdf capture for stationary materials[END_REF] combined two different pictures of the same material (one with flash, the other without), then exploited the material local similarity to recover the full BRDF and normal map at each pixel. Their algorithm is limited to isotropic materials.

Recently, Merzbach et al. [START_REF] Merzbach | Learned Fitting of Spatially Varying BRDFs[END_REF] proposed a method to efficiently acquire SVBRDF.

Compression

They improve the fitting efficiency by training a neural network with a set of reference measurements and corresponding fit. They can drastically reduce SVBRDF parameters of new materials using their deep learning based method compared to a traditional optimisation based appraoch.

In chapter 7 we use a fixed viewpoint and a rotating light source to capture anisotropic spatially varying BRDFs, and use Fourier analysis to extract an anisotropic axis and fit an microfacet BRDF. Compared to these methods, we do not make any assumptions about the homogeneity of the material, and recover material response independently at each pixel. Thus, we can acquire reflectance properties even for materials with high-frequency spatial variations.

Discussion

We cannot clearly qualify this range of techniques as material acquisition. They base their approach on measuring few samples to fit an analytic model. The resulting reflectance is then based on the physical assumptions made by the underlying model.

They are fast to acquire and does not require expensive equipment. Relying on analytic models, they benefit from their advantages: editability, compactness and fast evaluation. They provide an efficient way to approximate a real-world reflectance.

Compression

Other methods rely on the captured data to minimise the underlying dimension for efficient storage. There are not targeting extraction of parameters meaningful for edition or post processing but rather a compact way of storing BTF for a rendering context.

Fitting BRDF models to measured data

Lawrence et al. [LBAD + 06] introduced a framework to characterise a SVBRDF by a shade tree. Their technique emphasis on the ability to edit the material. They rely on multiple matrix factorisation to separate components of the SVBRDF into meaningful parameters for edition.

Holzschuch et Pacanowski [START_REF] Holzschuch | A Two-scale Microfacet Reflectance Model Combining Reflection and Diffraction[END_REF] proposed a model based on microfacet theory to account for diffraction by adding a second scale of facets in the order of magnitude of visible wavelength. They validated their model by fitting it against the MERL database. The resulting low fitting error prove their model to be convincing compared to a standard microfacet fitting.

Using power iterations, Dupuy et al. [DHI + 15] proposed a method for retrieving microfacets BRDF parameters from arbitrary materials.

Sun et al. [START_REF] Sun | Connecting Measured BRDFs to Analytic BRDFs by Data-driven Diffuse-specular Separation[END_REF] proposed a method to separate diffuse and specular term with a three step minimisation process. The first step is used to roughly separate each term by fitting the BRDF with a GGX NDF. The second step consists in minimising the error between two tabulated diffuse and specular BRDF with the target BRDF and with the GGX NDF previously fitted. Finally, the last step enables colour restoration, previous steps being done on sum of acquired colour channels. In Chapter 8 we use a similar approach, a multi step minimisation process to split a BTF into a discrete set of BRDFs with a colour restoration step in the end.

Dimension reduction

Another range of techniques are based on PCA [START_REF] Schröder | Non-Local Image Reconstruction for Efficient Computation of Synthetic Bidirectional Texture Functions[END_REF], statistical models [START_REF] Haindl | Extreme compression and modeling of bidirectional texture function[END_REF] or machine learning [START_REF] Rainer | Neural BTF compression and interpolation[END_REF] to store efficiently captured materials. While those methods offer superior accuracy, there are not suitable for material edition.

Conclusion

In this chapter, we have presented commonly used techniques for reflectance acquisition. We categorised techniques in four principal groups along with their respective limitation:

• The brute force approach is the most generic one. It allows acquisition of homogeneous and inhomogeneous materials. But it is time consuming. Its accuracy depends on the sampling frequency.

• One shot multiple samples approach allows faster acquisition. It is restricted to homogeneous materials and requires getting material samples of specific shapes. For anisotropic materials, it is destructive.

• With sparse sampling, we reduce the acquisition time of a brute force approach: we select the most important directions to acquire and reconstruct the material at evaluation. It is currently restricted to homogeneous materials.

• Acquisition based on analytical models assumes we can model the sample with an analytic BRDF. It does not provide per se measured reflectance function. But it is an interesting approach for designing fast acquisition devices.

Material acquisition techniques are interesting for a wide range of applications, from cultural heritage to industrial design.

Unlike analytical models, we cannot easily modify measured reflectance. Spatially varying measured materials have a large memory footprint and are inefficient to evaluate. We may want to fit analytical model to measured reflectance. So, we can edit the material and provide an interactive visualisation. form, and acquire photographs with varying incoming light direction. In a second step, we extract anisotropic direction, shading normal, albedo and reflectance parameters from these photographs and store them in texture maps. We later use these texture maps to render new views of the material.

Sources & references

photographs, we extract a set of maps depicting reflectance parameters such as albedo, shading normal or anisotropic direction. These parameters are then used to generate new pictures of the material.

Using Fourier analysis provides accurate results for phase shift: anisotropy direction is simple to extract in Fourier domain. We assume the effect of the shading normal and anisotropy are separable in a first step. We rely on the property of periodicity of the signal of the anisotropy: we can extract the anisotropy direction from the even rank harmonics.

The reflectance parameters are much more difficult to retrieve. The amplitudes in the Fourier domain are much more affected by the approximation made to separate shading normal effects from anisotropy. We use information provided by amplitudes to feed starting parameters for a final fit with all parameters.

This works was done with Nicolas Holzschuch and Imari Sato and was presented at Graphics Interface 2016.

Acquisition Apparatus

We have set up our own acquisition apparatus (see Figure 7.3):

• A flat material sample is placed on a console.

• The camera is vertically aligned with the sample normal, at a large distance.

• A rotating arm carrying lights provides varying lighting conditions.

The axis of rotation for the arm is equal to the view axis for the camera. A computer controls the arm position, which light sources are lit and the camera shooting, so the acquisition process is done automatically and can be conducted by a novice user.

For each acquisition we sample the incoming light direction regularly, with N samples, separated by 2π/N .

We used a Nikon D7100 camera with a AF-S Nikkor 18-105 mm 1:3.5-5.6G ED lens. Shots were done in NEF format at 14 bits per channel in no compression mode. We took 20 pictures for each material, corresponding to 20 samples in the incoming light direction in the azimuthal direction with constant elevation. Then we estimate BRDF parameters by minimisation: specular reflectance with even harmonics, diffuse reflectance and normal elevation with c 0 and c 1 . In a final step, we perform minimisation on the measured signal for minor adjustments on the parameters. For added accuracy, we can loop over the minimisation process using computed parameters as input values. coming direction, ω o is the outgoing direction, ω h is the half-vector. θ h is the angle between ω h and the shading normal, n s . φ h is the angle between the half-vector and the main anisotropic direction on the tangent plane.

Acquisition Apparatus

Even though the camera and sample are fixed with respect to each other, the movements of the rotating arm induce small displacements in the pictures, of a few pixels between each frame. We align all the pictures before processing, using Enhanced Correlation Coefficient Maximization [START_REF] Evangelidis | Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization[END_REF][START_REF] Evangelidis | Projective Image Alignment by using ECC Maximization[END_REF], as implemented in OpenCV [START_REF] Bradski | The OpenCV Library[END_REF].

After this image registration and alignment, a given pixel in all the pictures corresponds to the same point on the material. We have for each pixel 20 samples corresponding to regularly sampled incoming light directions. We then extract material parameters at each pixel independently, using these samples.

Parameter Extraction

Reflectance Model

We make the hypothesis that the object surface is flat at the macroscopic level, but with small variations that can be encoded using a shading normal n s = (θ n , φ n ). At every point on the surface, we define the main anisotropic direction. Angles in the tangent plane, such as φ n , are measured with respect to this direction.

We assume that our material can be modelled with anisotropic Cook-Torrance model [CT82] and a diffuse component:

ρ(ω i , ω o ) = ρ d π + F (cos θ d )G(ω i , ω o )D(ω h ) 4 cos θ i cos θ o (7.1) D(ω h ) = 1 πα x α y cos 4 θ h f ( tan 2 θ h ( cos 2 φ h α 2 x + sin 2 φ h α 2 y )) (7.2)
Where:

• • θ h is the angle between the shading normal n s and the half-vector

ω h = (ω i + ω o )/||ω i + ω o ||.
• θ d is the angle between ω h and ω i (see Figure 7.4). φ h is measured with respect to the main anisotropic direction.

• F is the Fresnel term.

• G is the shadowing and masking term.

• ρ d is the diffuse response.

• α x , α y are the main BRDF parameters and control the shape of the specular lobe. Isotropic BRDF correspond to α x = α y .

The difficulty is that shading variations can be caused by changes in normal orientation, changes in local geometry and anisotropic direction or change in albedo. We only have access to the outgoing light, which is an average over all factors. Using Fourier analysis allows us to separate the different contributions.

We also have to simplify the model using a constant specular coefficient ρ s in place of F (θ d ).

Main idea

We use a Fourier analysis of the signal for each pixel to extract the reflectance parameters:

• The phase of the Fourier coefficients gives us the direction of anisotropy.

-The phase of the second harmonics gives us the direction of anisotropy for the specular lobe.

-The phase of the first harmonic gives the direction of anisotropy for the diffuse lobe, and thus for the normal.

• The amplitude gives us the reflectance parameters.

The Fourier analysis provides noise reduction to the signal; it is very efficient at identifying the direction of anisotropy.

We conduct the analysis independently for each pixel of the images, resulting in spatially varying information.

Fourier analysis for each pixel

For each pixel in the pictures from the acquisition setup ω o remains fixed and ω i rotates around n. The contributions for all directions define a periodic signal of φ, s(φ), which has been sampled N times, where: φ is the light azimuth in the acquisition setup frame.

We use the Fourier transform of this signal, defined as the set of complex numbers c n :

c n = 1 2π ∫ 2π 0 s(φ)e -inφ dφ (7.3)
We compute the (c n ) using Fast Fourier Transform (FFT) [START_REF] Cooley | An algorithm for the machine calculation of complex Fourier series[END_REF][START_REF] Frigo | The Design and Implementation of FFTW3[END_REF].

If the shading normal is equal to the geometric normal, with our acquisition setting θ h is a constant. Luminance information for a single pixel is only a function of φ h , and more precisely s(φ h ) = s(φ h + π).

In that case, coefficients c n for odd n are null:

c n = 1 2π ∫ 2π 0 s(φ h )e -inφ h dφ h = 1 2π [∫ π 0 s(φ h )e -inφ h dφ h + ∫ 2π π s(φ h )e -inφ h dφ h ] = 1 2π [∫ π 0 s(φ h )e -inφ h dφ h + ∫ π 0 s(φ h + π)e -in(φ h +π) dφ h ] = 1 2π ∫ π 0 s(φ h ) ( e -inφ h + e -in(φ h +π) ) dφ h (7.4) c 2k+1 = 1 2π ∫ π 0 s(φ h ) ( e -i(2k+1)φ h + e -i(2k+1)(φ h +π) ) dφ h = 1 2π ∫ π 0 s(φ h )e -i2kφ h ( e -iφ h + e -i(φ h +π) ) dφ h = 0 (7.5)
When the shading normal is different from the geometric normal, this assumption does not hold. We use this observation to help separating the effects of the anisotropic BRDF and those of the shading normal: we fit the specular BRDF parameters using Fourier coefficients with even rank c 2k , and separately estimate the shading normal angle and diffuse BRDF parameters using the first two Fourier coefficients, (c 0 , c 1 ).

In a final step, we fit all BRDF parameters together, using the entire model, and with the approximated values computed using Fourier coefficients as starting points for parameters.

Local frame azimuth

Anisotropy direction

To estimate the anisotropic direction for the BRDF, we only consider the c n for even numbers n, expressed as c 2k . All these Fourier coefficients are complex numbers, having a modulus d 2k and a phase β 2k :

c 2k = d 2k e iβ 2k (7.6)
The angle φ h in Equation 7.2 is expressed with respect to this local anisotropic direction. In the local frame, the anisotropic direction is expressed as an angle, φ a . We compute the angle directly from the phase of the c 2k coefficients:

φ a ≡ β 2k 2k mod π/k (7.7)
For example, with k = 1:

φ a ≡ β 2 2 mod π (7.8)
Since φ a is defined modulo π, Equation 7.8 defines it uniquely.

Shading normal azimuthal direction

We use the c 1 Fourier coefficient to estimate the shading normal azimuthal direction: the reflected intensity is maximum when the incoming light is aligned with the shading normal. This has a 2π period; the phase of c 1 gives us the direction of this maximum.

Specular reflectance parameters

Once we have the anisotropic direction, we extract the remainder of the specular reflectance parameters from the even-numbered Fourier coefficients. In this step, we neglect the diffuse component.

We need to find values for:

• One RGB colour parameter ρ s We find the optimal set of parameters using a Levenberg-Marquadt optimisation on the Fourier spectrum [START_REF] Levenberg | A method for the solution of certain problems in least squares[END_REF][START_REF] Marquadt | An algorithm for least squares estimation on nonlinear parameters[END_REF], as implemented by Lourakis [START_REF] Lourakis | levmar: Levenberg-Marquardt nonlinear least squares algorithms in C/C++[END_REF].

For each set of parameter values (ρ s ,α x ,α y ,(p)), we compute the corresponding signal for all incoming light directions, then the Fast Fourier Transform of this signal, c ′ n . On this estimated signal, all Fourier coefficients with odd rank are null by construction. We define the minimisation error E in the optimisation process as the L 2 distance between the Fourier spectrum of the measured signal and the Fourier spectrum of the estimated signal over coefficients with even rank:

E = ∑ k ⃓ ⃓ c 2k -c ′ 2k ⃓ ⃓ 2 (7.9)
For the fitting process, we observe that we have n = ω o , from our acquisition setup. Therefore cos θ o = 1. cos θ i is also constant, and is known from the acquisition parameters. θ h is also a constant, equal to θ i /2. This reduces the number of unknowns.

This optimisation converges quickly (less than 50 iterations for most of the pixels), and gives us the reflectance parameters for each pixel (see Figure 7.5).

For the Cook-Torrance BRDF model, the optimisation process gives us the value of the product ρ s F (θ d )G(ω i , ω o ). For our measuring angle, we can assume G(ω i , ω o ) ≈ 1. We will reconstruct the value of G at other angles from α x and α y using Smith's shadowing function [START_REF] Smith | Geometrical shadowing of a random rough surface[END_REF]. With a single sample on θ i , we do not have enough information to separate the Fresnel term from the specular reflectance and approximate it with a constant.

Diffuse component and shading normal

In a separate step, we fit two parameters using only the first two Fourier coefficients c 0 and c 1 :

• The diffuse component of the BRDF ρ d

• The shading normal angle θ s .

As specular effects are functions π periodic, their influence on these Fourier coefficients is theoretically null, leaving only diffuse lighting. This step converges very quickly, because there is a small number of variables.

Control: fitting over the entire model and feedback loop

Our previous two fitting steps made several strong hypothesis, by separating shading normal and specular reflectance coefficients estimation.

In practice, the shading normal and diffuse coefficients influence the specular BRDF coefficients. We considered specular anisotropy, diffuse albedo and shading normal influencing specific harmonics in the Fourier domain. Although this is an approximation, it provides a rough estimate of the geometry and BRDF parameters which will be helpful. In practice, when the normal is not aligned with ω o , s(φ i ) ̸ = s(φ i + π) and the contributions are not separable by odd and even ranks.

In a third step, we fit the entire BRDF model, with shading normal, diffuse and specular component with the full signal, using the values computed in the previous two steps as starting points.

Because the starting points are close to the actual position of the minimum, this step also converges quickly. At the end of this step, we have the full set of material parameters: BRDF values with diffuse component and shading normal.

For added accuracy, we loop over the whole fitting process, using the values we just computed: recompute parameters for the specular and diffuse part of the BRDF using the shading normal, fit the entire signal with the new specular and diffuse parameters. After two loops, our fitting process has converged to acceptable solutions.

Everything is done per pixel and can easily be parallelised.

Storage and retrieval

After the full fitting process, we have for each pixel:

• The anisotropic direction: φ a

• The shading normal: n s

• The BRDF parameters: ρ d , ρ s , α x , α y .

We store these into several textures (see Figure 7.6 and 7.7), to be accessed at run time.

Results and Comparison

We have tested our extraction algorithm with different material samples and BRDF models. We captured reflectance data on four different materials:

Amulet shows sharp variations in anisotropic directions, combined with smooth variations in diffuse reflectance, and is a difficult case for material acquisition.

Carbon fibre combines high frequency and low frequency material variations. It is an example of a woven material, with two principal anisotropic directions.

Powerbank is made of brushed aluminium, and exhibits strong anisotropy in a single direction. Fourier analysis provides a robust answer even with a small number of samples and measurement noise.

Phone case has a repetitive pattern with three different anisotropic directions.

Our experiments show that the algorithm is extremely robust for parameter extraction, and works with a small amount of input photographs. Figure 7.6 shows the extracted parameters for our test materials.

Numerical validation

Anisotropic direction Figure 7.8 shows the angular shift along with reflectance values for a single pixel of measured data. A strong advantage of Fourier analysis is its ability to identify the correct angular shift, even though it does not correspond to one of the sampled directions.

We also conducted tests on synthetic data: we generated data sets with known angular shift, sampled with 5 to 20 samples, and extracted the shift from the Fourier coefficients. Figure 7.9 shows the average difference between the actual shift and the retrieved value, as a function of the number of samples. The error falls below 1 degree for as little as 10 samples.

Analytic BRDFs We tested our algorithm on analytic BRDFs, where the parameter values are known. Figure 7.10 shows an example. As you can see, the reconstructed signal provides a very good fit with the sample points.

BRDF parameters

Our algorithm also identifies BRDF parameters in a robust manner. Figure 7.6 shows the parameters retrieved from all our data sets: the shading normal, diffuse and specular coefficients, anisotropy direction and BRDF parameters. Instead of displaying directly the BRDF parameters α x and α y , we display the anisotropy and shininess of the BRDF. The former corresponds to the ratio α x /α y . It is equal to 1 for isotropic materials; the larger it is, the more pronounced the anisotropy of the material is. The latter corresponds to the average (α x + α y )/2. The smaller it is, the more specular of the material is. Looking at the values for a single pixel (Figure 7.5), we see that the Fourier transform has filtered measure noise. The reconstructed reflectance values are not equal to the measured values, while keeping the main features of the signal.

0 • 0.5 • 1 • 1.5 •
We have tried our algorithm with two different microfacet distributions: the GGX distribution [START_REF] Walter | Microfacet models for refraction through rough surfaces[END_REF] and the e -x p distribution [START_REF] Brady | genBRDF: Discovering new analytic BRDFs with genetic programming[END_REF]. Figure 7.5 shows a comparison between the BRDF values with these two distributions and the measured signal for a given pixel. Both are providing a good fit.

Computation time

Table 7.1 shows the overall computation time for each step in the algorithm, along with the average computation time per pixel. Although the average time per pixel is relatively low (6 ms to 9 ms, depending on the material), the high resolution of the pictures make the computation time several hours per material sample. Almost all this computation time is due to the parameter extraction using Levenberg-Marquadt. This fitting step is required, due to the non-linear nature of the BRDF and the small number of samples per pixel.

To reduce the computation time, we cropped the pictures to focus on the sampled material, removing as much as possible of the background material.

All these timings were recorded on a 2 processors 8-core Intel Xeon E5-2630 v3 at 2.40 GHz, with 32 Gb of RAM running Ubuntu 14.04 64 bit with the kernel 3.19.0-25-generic. The Levenberg-Marquadt fitting library runs on 32 threads.

Rendering

Figure 7.11 shows a side-by-side comparison between one of the pictures from the original dataset and a picture generated using the retrieved BRDF parameters. Of course, we can also generate pictures with different viewing directions and lighting conditions (see Figure 7.12). All these pictures show only local illumination and are rendered in real-time.

Once we have extracted BRDF parameters, we can also transfer the material to other objects and use them in global illumination settings. Figure 7.14 & Figure 7.13 show a virtual scene with materials extracted from the powerbank, phone case, and carbon fibre datasets, rendered using the Mitsuba renderer [START_REF] Wenzel | Mitsuba renderer[END_REF].

Discussion

Spatial coherency

We found extraction per pixel is convincing for extracting the anisotropy axis. We rely on the phase shift of the second Fourier harmonics. Even with normal variation, we get an accurate estimate with few sample and with little computational resources. The tangent directions are spatially coherent (see Figure 7.6) and corresponds to the observable highlight direction under the same conditions (see Figure 7.11).

The BRDF parameters are more challenging to extract. The shading normal makes the separation between diffuse albedo and specular parts difficult. We cannot rely solely on the Fourier coefficients magnitude, the final fit is necessary. The Amulet shows some discontinuities in its specular, diffuse and roughness maps. It is highly specular and 20 samples is too little for this specific sample. Other resulting parameter maps, while being not as consistent as tangent maps, are coherent.

Shading normal

Our acquisition process is currently limited to planar objects. Identifying the shading normal enables use to acquire small variations in the local planarity of the object.

If the object is perfectly flat, with its normal aligned with the point of view (ω o = n), then the reflectance should be symmetric along the direction of anisotropy (see Equation 7.2 and Figure 7.15a). In practice, most of our measurements show a variation of amplitude between the two peaks of the signal (see Figures 7.1 and 7.5). Variations of the shading normal explain this difference of amplitude, but introduce a bias in our retrieval of the direction of anisotropy from the phase shift of the second harmonics, hence the need to conduct a final fitting step on Figure 7.15 illustrates the influence of the different sources of anisotropy. We represent the BRDF intensity in polar coordinates, as a function of φ h , with ω o = n. With a flat surface and anisotropic BRDF, the signal is symmetric along both the direction of anisotropy and the orthogonal direction (Figure 7.15a). With a shading normal and isotropic BRDF, the signal is symmetric along the shading normal (Figure 7.15b). With a combination of shading normal and anisotropic BRDF, there is no symmetry (Figure 7.15c). Signal measured on our test scenes exhibit the same behaviour (Figure 7.15d).

In our analysis, we constrained the angle between shading normal and geometric normal θ n to remain below a certain threshold (20 • ). As θ n increases, it becomes harder for our method to retrieve anisotropy. An extension to curved objects would require a different method to identify the direction of the normal, and more samples over the incoming light elevation.

Limitations

The main advantage of our acquisition process is the robustness and spatial accuracy. We treat each pixel independently, with only a small number of sampled pictures, and still retrieve enough information to reproduce the aspect of the material. We can handle pixel-size features in the measured data: the spatial resolution of the output is equal to the spatial resolution of the input photographs.

The drawbacks are related to the limited amount of information: we cannot retrieve the Fresnel term with only one set of measurements, with constant elevation angle for incoming light. This reduces the accuracy of our reflectance model near grazing angles. This limited amount of samples also limits the ability to deal with highly specular materials. As matter of fact, due to Nyquist-Shannon sampling theorem, highly specular material will have higher frequencies in the Fourier domain which are going to be represented as lower frequency contributions with a limited amount of samples. This explains the error observed in the amulet sample. Although we have limited our capture with twenty samples in this paper, our algorithm can work with higher number of samples.

Finally, our entire acquisition process assumes anisotropic reflectance with a single anisotropic lobe. Reflectance with two anisotropic lobes, as exhibited by Filip et al. [START_REF] Filip | Effective acquisition of dense anisotropic BRDF[END_REF] would require a different approach.

Conclusion

In this chapter, we have presented a complete system for rapid acquisition of spatially varying anisotropic materials. Our system works by taking several pictures of the material sample, then extracting anisotropic direction and material properties at each pixel using Fourier analysis.

Our system is stable and robust, and produces noise-free parameter maps, even with a small number of sampled directions. We extract material properties at a high resolution, at the pixel level. Our system lets designers reuse the sampled material on virtual objects.

Since the camera angle is fixed, unlike other techniques no re-projection is necessary and we can use the full resolution of the sensor to extract the parameter maps. We had to use an image alignment procedure but this step should not be necessary if we strengthened the gantry.

Diffuse and specular albedo are difficult to separate. We used a shading normal to account for local variations. This breaks the separability of specular and diffuse contributions in the Fourier domain. We then can only get an estimate of BRDF parameters through Fourier analysis and rely on a traditional fitting step at the end to account for shading normal effects.

Anisotropy direction extraction is fast and reliable using Fourier analysis. We get convincing results even with specular materials with a limited number of samples. Contrary to the extraction of roughness and diffuse parameters, we rely on the phase shift. It is less prone to be influenced by the shading normal and is a promising method for rapid extraction of local frame.

We exploit the anisotropy properties to extract reflectance parameters. This makes our acquisition setup simple using only azimuthal light sampling. But its strength is also its limitation: it is restricted to anisotropic material acquisition.

Résumé du chapitre

Dans ce chapitre, nous avons présenté un système pour la numérisation rapide de matériaux anisotropes avec variation spatiale. Notre système se base sur l'extraction par pixel de la direction d'anisotropie et des propriétés de réflectance du matériau à partir de plusieurs photographies.

Notre système est stable et robuste : nous obtenons des cartes de paramètres sans bruit même avec le nombre réduit de directions acquises. Les cartes haute résolution extraites par pixel permettent d'être facilement réutilisées et modifiées par un artiste.

Comme l'angle de la caméra reste fixe durant l'acquisition, contrairement à d'autre techniques, aucune re-projection n'est nécessaire et nous pouvons exploiter l'intégralité de la résolution effective du capteur pour l'extraction des cartes de paramètres. Nous avons toutefois dû utiliser une procédure de ré-alignement d'image mais, cette étape pourrait être supprimée en renforçant la stabilité du système d'acquisition.

L'albédo diffus et spéculaire sont difficiles à séparer. Nous avons utilisé une normale locale pour prendre en compte les variations d'orientation surfacique des échantillons.

Cela ne permet plus la séparation simple des contributions diffuse et spéculaire dans le domaine de Fourier. Ainsi, nous ne pouvons qu'obtenir une approximation des paramètres de la BRDF dans le domaine de Fourier. Nous devons procéder à une optimisation plus traditionnelle dans une étape finale sur l'ensemble de la BRDF afin de prendre en compte cette normale locale.

L'extraction de la direction d'anisotropie est quant à elle rapide et fiable en utilisant l'analyse de Fourier. Nous obtenons des résultats convaincants avec un nombre restreint d'échantillons. Contrairement à l'extraction des paramètres de diffus et spéculaire, nous utilisons l'argument des coefficients de Fourier. Ce dernier est moins impacté par l'influence de la normale locale. Cette méthode semble donc intéressante pour l'extraction rapide de l'orientation locale d'anisotropie.

Nous tirons parti des propriétés des matériaux anisotropes pour l'extraction des paramètres de réflectance. Cela permet à notre système d'acquisition d'utiliser seulement un échantillonnage sur les directions azimutales d'illumination. Mais, cet avantage est aussi sa limite. Nous sommes restreints à l'acquisition de matériaux anisotropes.

CHAPTER 8

A pipeline for fitting acquired spatially varying materials W hen we need accuracy, the acquisition technique we have presented in the previous chapter is not suitable. We need to capture multiple view and illumination directions. Dealing with spatially varying materials is still an open research problem. For these, geometry and material properties often interact in complex ways. With a woven fabric, the changes in geometry as threads go above or under each other result in changes in normal direction, but also in changes in appearance due to wear after some time. Simply encoding changes in appearance in a texture is usually not enough for complex materials.

Bidirectional texture functions (BTF) solve the problem by measuring the reflectance over a patch of material, for multiple incoming and outgoing directions. They encode all variations of appearance and can be used to reproduce complex materials in rendering. Their main weakness is the large memory cost, due to the high dimensionality of the measurements: BTFs are varying over space, incoming direction and outgoing direction, a total of 6 dimensions. Another issue is that it is difficult for an artist to edit a BTF, as they lack convenient editing handles. Measured values are not connected to editable concepts such as roughness, albedo or normal direction.

This chapter present a full pipeline to make bidirectional texture functions convenient for both rendering and edition. The key idea is to subdivide the measured material into a set of different materials, each with consistent properties, such as roughness. This has two benefits: first, it reduces the memory footprint for the material; second, it provides artists with a way to edit the material consistently, for example changing the position of each material, or their local properties.

In our pipeline, we first estimate the local frame orientation at each measured location. Then we estimate roughness parameters at each location to segment the measured surface into material groups. We determine a rule to edit consistently roughnesses of a whole group at once. Finally, we reconstruct diffuse and specular colour per location.

After applying our algorithm, the BTF is represented by a series of 2D textures: normal map, anisotropic direction map, roughness map, material map, diffuse colour map, specular

Symbol Description

Geometry n g Surface normal. This is the average BTF normal aligned with the measurement device z axis. n l

Local normal. This defines the true surface orientation at a considered location on the BTF. θ ig , θ og Elevation angles made by ω i , ω o in global frame i.e. acos(< colour map. Each can be edited independently, but the most interesting is the ability to edit the material map and maintain a consistent behaviour, thus changing the surface appearance.

ω i • n g > ), acos(< ω o • n g >). θ i l , θ o l Elevation angles made by ω i , ω o in local frame i.e. acos(< ω i • n l >), acos(< ω o • n l >). φ ig ,

Overview of the pipeline

Our pipeline consists in four steps:

1. Identifying the local frame for each location.

2. Fitting of a microfacet BRDF for each location.

3. Grouping materials based on fitted roughness parameters.

4. Restoring colour with diffuse albedo and Fresnel function for each location.

Finding local surface orientation in the first step enables local correction of the bidirectional texture function for fitting a normal distribution function. We group similar materials based on the roughness parameters fitted. Colour restoration is finally done by retrieving a diffuse a Fresnel function while using the normal distribution function found previously.

In this chapter we present the pipeline through the choice of a specific method at each step. However, our pipeline is not limited to the specific set of methods we have used. Each method can be replaced with a more efficient one given a specific context, as long as input and output data are preserved. on the sphere represented by a sample doesn't have an equal area at each elevation and azimuth. This have to be taken into account when re-normalising the BTF and for fitting.

BTF re-normalisation

To ensure energy conservation, we re-normalise the BTF data. While some steps of the pipeline are not affected, the fitting processes work best with a re-normalisation. We use the normalisation test presented in [START_REF] Heitz | Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs[END_REF] to divide all colour channels by the maximum of the integral found for each colour channel if it exceeds 1.

∫ Ωo b r (ω i , ω o ) < ω o • n g > dω o ≤ 1 (8.1)
We discretise this integral:

L = N θo ∑ n=1 Nφ o [θo n ] ∑ k=1 ∫ θ + on θ - on ∫ φ + o k φ - o k b r (ω i , ω o ) < ω o • n g > dω o (8.2) = N θo ∑ n=1 Nφ o [θo n ] ∑ k=1 b r (ω i , ω o n,k ) ∫ θ + on θ - on ∫ φ + o k φ - o k cos θ o sin θ o dφ o dθ o (8.3) = 1 2 N θo ∑ n=1 Nφ o [θo n ] ∑ k=1 b r (ω i , ω o n,k )(cos 2 θ - on -cos 2 θ + on )∆φ on (8.4)
So, we divide the BTF values by I if I > 1:

I = max ω i 1 2 N θo ∑ n=1 Nφ o [θo n ] ∑ k=1 b r (ω i , ω o n,k )(cos 2 θ - on -cos 2 θ + on )∆φ on (8.5) ∆φ on = 2π N φo [θ on ] θ - on = max ( 0, θ on - ∆θ o 2 ) (8.6) ∆θ o = π/2 N θo θ + on = { θ on + ∆θo 2 if n < N θo π 2 otherwise (8.7)
Where:

• ω o n,k is the measured sample for illumination of angles (θ on , φ o k ).

• N θo is the number of samples on θ o .

• N φo [θ on ] is the number of samples on φ o at a given θ on .

Local frame retrieval

When dealing with bidirectional texture functions, the measured surfaces are most likely non planar. While being considered as globally planar to make the measurements, the light and view direction orientations for a specific location of the BTF is not the same as the measurement orientation set in the global frame. It is affected by the surface irregularity given by a local normal. For instance, on a brushed metal plate, while being globally planar, we expect having ridges and valleys, impacting the local normal orientations. For surfaces like fabric, leather, or embossed wallpaper these effect can be even more prominent. It is occurring on larger scales and cannot be assumed to be sub-texel. When the surface geometry is larger than a pixel, local orientation matters: for a fabric made of coarse fibre, even at low resolution, effects of the independent fibre will affect multiple pixels around the fibre.

In the regions where the local surface normals are not aligned with the global normal, the viewing and lighting directions for measurement do not correspond to the real directions at the specific location. We want to compensate for this effect to more accurately identify materials while fitting a BRDF for each location.

The surface can also be locally anisotropic. The anisotropy axis then has to be retrieved at each location.

Surface normal orientation

Normal estimation

We rely on Helmholtz stereopsis [START_REF] Zickler | Helmholtz stereopsis: Exploiting reciprocity for surface reconstruction[END_REF] to recover the surface normal orientation. This technique is based on Helmholtz reciprocity: when switching the position of illumination and camera, we expect the same reflectance with a factor depending on the distance made by the illumination and the camera with the sample. We consider the acquisition setup calibrated i.e.

the illumination constant at each position and time and with at same distance of the sample.

L e (ω a ) = f r (ω a , ω b , n l ) • E e (ω b )• < ω b • n l > (8.8) = b r (p, ω a , ω b ) • E e (ω b )• < ω b • n g > (8.9)
Where:

• L e (ω a ) is the measured radiance at (p) when ω i = ω a and ω o = ω b .

• E e (ω b ) is the irradiance from direction ω b .

From a pair measurements L e (ω a ); L e (ω b ) respectively with:

{ ω a = ω i , ω b = ω o ω b = ω i , ω a = ω o (8.10)
We have f r (ω a , ω b , n l ) = f r (ω b , ω a , n l ) due to Helmholtz reciprocity. This doesn't hold for a BTF since it encodes the geometry of the material as well. Then, this can be written as:

(L e (ω a ) • ω a -L e (ω b ) • ω b ) • n l = 0 (8.11) (b r (p, ω a , ω b )• < ω b • n g > •ω a -b r (p, ω b , ω a )• < ω a • n g > •ω b ) • n l = 0 (8.12) w(p, ω a , ω b ) • n l = 0 (8.13)
We build a matrix W (p) with each row corresponding to a pair measurement difference for a position p. Each row is represented by w(p, ω a , ω b ). The right singular vector of the singular value decomposition (SVD) gives the best possible local normal n l matching the measurements at this location. We need at least three pairs of measurements to restrict W (p) to null space. We use all possible pairs of measurements with ω a ̸ = ω b .

Projected light angle correction

Once we have the local normal, we re-parametrise the BRDF at each location: the light incidence angle with the global normal does not correspond to the angle made with the local normal. If a correction factor was applied during measurements to compensate for projected angle made with light, we have to cancel it out and use the actual angle instead.

b r corrected (p, ω i , ω o , n l ) = < ω o • n g > < ω o • n l > • b r (p, ω i , ω o ) (8.14)
We will use this corrected BTF in all subsequent steps.

Anisotropy axis

We recover the local anisotropy direction using the technique based on Fourier analysis, described in Chapter 7.

c n = 1 2π ∫ 2π 0 s(φ h )e -inφ h dφ h (8.15)
We build s(φ h ) from the corrected BTF with ω i aligned with the normal and ω o rotating around with a constant angle. With this construction, we have a constant

θ h = θo l 2 for all • F (θ d ) is the Fresnel term. • D(ω h ) is the NDF. • G(ω i , ω o ) is the shadowing-masking term.
We use an anisotropic Beckmann distribution for the NDF and the shadowing-masking term from [START_REF] Heitz | Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs[END_REF] with the approximation for shadowing-masking from [START_REF] Walter | Microfacet models for refraction through rough surfaces[END_REF].

D(ω h ) = χ + (ω h • n l ) e -tan θ h ( cos 2 φ h α 2 x + sin 2 φ h α 2 y ) πα x α y cos 4 (θ h ) (8.17) G 1 (ω , ω h ) = χ + ( ω • ω h ω • n l ) ⎧ ⎨ ⎩ 3.535a + 2.181a 2 1 + 2.276a + 2.577a 2 if a < 1.6 1 otherwise (8.18) a = ( tan θ √ α 2 x cos 2 φ + α 2 y sin 2 φ ) -1 (8.19) G(ω i , ω o ) = G 1 (ω i ) • G 1 (ω o ) G 1 (ω i ) + G 1 (ω o ) -G 1 (ω i ) • G 1 (ω o ) (8.20)
The Fresnel term can be challenging to fit with an analytic model. The grazing angles where the Fresnel term is supposed to be the most prominent are also the less reliable on the measured data. To ease the fit, we use an arbitrary Fresnel term driven by control points. We use nine control points equally distributed over [0; π 2 ]. This allows to increase or decrease the number of control points and their distribution depending on the sampling of the measured data. Those control points are parameters of the optimisation process. An interpolation is done during the evaluation of a given F (θ d ).

The main anisotropy axis drives the peak of reflectance around the local normal, so we expect α y ≤ α x . We fit using β a , β b for driving roughness parameters to limit the space of solution to explore during the the fitting process:

β a = α x + α y 2 β b = α x α y (8.21)
β a defines how specular the material is and β b how anisotropic it is. β b is constrained within [0.8, 10] to allow some cases where α y can be slightly over α y while preferring solutions with α y ≤ α y .

In reverse, α x , α y are defined by:

α x = 2β a β b 1 + β b α y = 2β a 1 + β b (8.22)
The optimisation process is done using Levenberg-Marquardt algorithm [START_REF] Levenberg | A method for the solution of certain problems in least squares[END_REF][START_REF] Marquadt | An algorithm for least squares estimation on nonlinear parameters[END_REF], as implemented by Lourakis [START_REF] Lourakis | levmar: Levenberg-Marquardt nonlinear least squares algorithms in C/C++[END_REF]. The difference is computed using an L 1 distance with a cosMap weighting from [START_REF] Sun | Connecting Measured BRDFs to Analytic BRDFs by Data-driven Diffuse-specular Separation[END_REF].

d 1 (a, b) = cosMap • (a -b) (8.23) cosMap = max (cos θ i l • cos θ o l , ϵ) (8.24)
Azimuthal sampling In the databases we use, the azimuthal sampling of the BTF differs depending on the elevation (see figure 8.2). The error for each sample have to be weighted accordingly to even importance of measurements depending on elevation. In [START_REF] Lourakis | levmar: Levenberg-Marquardt nonlinear least squares algorithms in C/C++[END_REF] implementation, the error is computed as:

E = N ∑ n=1 e 2 n (8.25)
Where:

• E is the total error.

• e n is an error between a given sample and the fit with the same incoming and outgoing directions.

• N is the number of samples.

With d 1 defined in equation 8.23, we compute e n as:

e n = d 1 ( b r corrected (p, ω in , ω on , n l ), f r f it (ω in , ω on ) ) √ ∆ω in ∆ω on 2π (8.26) ∆ω = ( cos θ --cos θ + ) ∆φ ∆φ = 2π N φ [θ] θ -= max ( 0, θ - ∆θ 2 ) ∆θ = π/2 N θ θ + = { θ + ∆θ 2 if n θ < N θ π 2
otherwise Where:

• N θ is the number of samples on θ.

• N φ [θ]
is the number of samples on φ at a given θ.

• n θ is the lookup index for a given θ i.e. n θ = θ ∆θ .

Filtering If the fit does not converge after 500 iterations, we mark the location of the BTF as unreliable and extrapolate parameters by averaging surrounding fitted values which have converged.

After the optimisation and filtering process, for each location, we obtain parameters describing a local BRDF:

• A diffuse intensity (the value for each colour channel will be retrieved latter).

• A Fresnel term F (θ d ) based on control points equally distributed in [0; π/2].

• Roughness parameters α x , α y .

Restricting roughness within a group

A given material can locally have variation of roughness. For example, a fabric can be made of a single thread but with varying wear and tear over the surface. We want to preserve local variation of roughness due to those effects while preserving grouping. This would allow editing a of whole group of materials while still letting it vary slightly.

To allow variations of shininess and anisotropy within a material group, we perform a linear regression for each group on α x , α y parameters.

Each (α x , α y ) is projected on this linear model found for the material group they are corresponding to. This allows a global edition of the roughness of the group while letting local variations occur. The linear model is editable and each roughness parameter is re-projected to the model corresponding to the material group.

α y = a • α x + b (8.27)
Where: a, b are the linear regression coefficients.

Colour restoration

After the projection of fitted roughness parameters on the linear regression for each group, we want to find diffuse albedo and Fresnel term matching the original BTF. As with the roughness fitting step, we use a tabulated Fresnel function for each colour channel.

To focus on the most reliable samples, we use the same cosMap weighting strategy along with the same distance function in the Levenberg-Marquardt algorithm.

The error is calculated similar as in equation 8.26 with a compensation for the number of measurements which is three times higher than for the previous step (one per colour channel instead of the average value):

e n = d 1 ( b r corrected (p, ω in , ω on , n l ), f r f it (ω in , ω on ) ) √ ∆ω in ∆ω on 2π √ 3 (8.28)
The fitted parameters are initialised with the fitted diffuse and Fresnel values found previously. The roughness values for the NDF are the projected (α x , α y ) values and are not parameters of the optimisation process. Filtering As before, we replace the non-converged values using converged neighbours. This time, having segmented the material, we use the average of surrounding values of the same group. We grow the search area until we find at least one converged value from the same material group. Then, we replace the non-converged diffuse and control points for the Fresnel tabulated function by the average of those converged around. At the end of this last step, we have for each colour channel and at each location:

• A diffuse albedo for each colour channel.

• A Fresnel term F (θ d ) based on control points equally distributed in [0; π/2] for each colour channel.

Results

At the end of our pipeline, we obtain a SVBRDF consisting in:

Geometry map the local frame is described by:

• A normal map n l (x, y).

• A tangent map φ a (x, y).

BRDF parameters map at each location:

• Roughness parameters α x (x, y), α y (x, y).

• Coloured parameters :

-A diffuse map ρ d (x, y, λ).

-A tabulated Fresnel term F (x, y, θ d , λ). • A material map m(x, y).

Groups map to help with edition:

• A linear model for each material where the roughness parameters are re-projected to.

This limited amount of data drastically decrease the memory footprint. Starting form n views • n lightings • n channels values per texel, we have, (3 + n channels + 2 + 1 + n Fresnel • n channels ) values per texel. With 81 directions for view and illumination and with 3 colour channels, using 9 Fresnel control points, this corresponds to a compression ratio of 1:182.25.

For rendering, we use Mitsuba renderer [START_REF] Wenzel | Mitsuba renderer[END_REF] with the technique presented by Schüssler et al. [START_REF] Schüssler | Microfacet-based Normal Mapping for Robust Monte Carlo Path Tracing[END_REF] for using normal maps in a path tracer. We use their default model consisting in a purely specular BRDF for the tangent facet. Figure 8.8 & 8.9 shows examples of renderings for some materials of the UBO1 and UTIA2,3 databases.

Our pipeline works well with materials having reasonable surface geometry such as fabrics. It catches correctly the specular part of coarse sandpaper UTIA MAM2014 003 as well. While the anisotropic effects of fabric UTIA fabric02 (Figure 8.7) is not perfectly reproduced, the pipeline offers a good starting point for editing the material.

Measurements goes down to 75 degrees for elevations. When rendering, extrapolation is done for elevations over this value. This can be problematic for materials having strong Fresnel effect. On the other hand, the fit using analytic functions is not affected but errors, especially on this Fresnel term can occur. This can explain some discrepancies between the "reference" and the fit.

Timing Local frame retrieval and material grouping are fast, taking few seconds at maximum. The two fitting process (NDF fitting and colour restoration) are computationally intensive. We treat each pixel independently, so it is parallelised per pixel. We ran our pipeline on materials from UBO and UTIA databases on various machine configurations ranging from Intel Xeon CPU W3520 to Intel Xeon-E5-2620 v4. The average execution time per core per pixel for the NDF fitting is 4.8 seconds. For colour restoration it is 16.7 seconds. Those execution time vary significantly depending on the material. See table 8.2 for details.

Material edition

The separation of a BTF in discrete set of BRDF allows quick edition of a whole material group.

In Figure 8.10, we edit material maps and normal orientations. We can easily reshape the material distribution to change the overall aspect.

In Figure 8.11, we change the regression parameters on each group to give a rougher appearance.

Limitations

Our approach relies on the microfacet BRDF model and suffers the same limitations. We do not account for subsurface scattering, multiple scattering, and macro scale masking effects. Figure 8.12 features some of those pathological cases. The other parameters are taken from the fitted material from the areas corresponding to the same group according to the provided material map.

Material

Sharpness

In our fitted materials, we observe a slight decrease of sharpness at raising angles compared with reference while grazing angles shows an improvement of sharpness: bidirectional texture measurement cannot maintain a constant effective resolution across all viewing angles (see green cloth MAM2014 007 ). To keep a consistent resolution, the images are re-wrapped to the resolution of the captured sample at ω o = z. We used in the fit a weighting to lower impact of samples at grazing angles. Then, the importance for samples at grazing angles is limited. But the combined weights is still higher than the one used when ω o = z where the maximum effective resolution is. This explains the slight decrease of sharpness at raising angles. If more sharpness is desired, one can tweak weights in the fitting process at the expense of less accurate Fresnel coefficients and diffuse albedos.

Masking and multiple-scattering At micro-scale in the microfacet model, shadowing-masking term accounts for loss of energy. Multiple scattering is traditionally modelled by the diffuse albedo. For bidirectional texture functions, the geometry is unknown and potentially at larger scale than a texel. Masking effects occur also at macro-scale. We use normal mapping to account for geometric variations over the surface. But, this cannot handle masking effects. In the measurements, a texel can represent different location: the geometry can mask. At grazing angles, the effect is even more pronounced. This is problematic for the fit. At rendering, multiple scattering also cause discrepancies. It even more pronounced if the material have important surface variations. For crinkle paper MAM2014 015, the creases on the paper scatter light to the facing edge. We get a loss of energy in our fitted material. Multiple scattering effects over texel are also challenging during the fit, a texel can be influenced by its neighbouring.

Subsurface scattering

The use of a BRDF avoid the ability to model subsurface scattering.

For materials having such effect this will result in loss of energy (see corduroy sample from UBO).

Conclusion and future work

In this chapter, we have presented a pipeline to convert measured BTFs into a set of logical components (local frame, reflectance properties). We then separate the measured BTF into a small set of materials, each with similar properties. This drastically reduce the memory footprint of measured BTFs, while allowing users to edit material properties.

Our method works well with materials with reasonable surface geometry, such as fabrics, leather and tiles. We used the microfacet model so we are limited in the range of effects handled by this model. We cannot model subsurface scattering effects.

We could handle multiple scattering by getting the geometry and considering neighbourhood during fitting. It would drastically increase computation time and complexity. Acquisition of geometry along with reflectance shall be considered.

Getting access to the geometry would also allow to account for masking effects: we could perform a pre-processing to determine locations actually acquired and not masked at each observation point. We could use the same pre-processing to compute cast shadows and getting an estimate of the neighbours affecting a given location after multiple scattering.

From an acquisition standpoint, such requirements are challenging to meet. The fitting process would also be much more difficult to implement efficiently. Similarly to the previous chapter, we treated each texel independently making our algorithm easy to parallelise. Considering inter-influence of neighbouring would drastically limit this ability.

In future work, we want to improve material grouping to have a more robust method for gathering materials with similar properties. We also want to take into account inter-reflections inside the measured BTF to increase the accuracy of the fit. We want to provide a more intuitive method for editing material roughness group. Finally, we want to work on matching material properties when editing to keep the local roughness structure.

Résumé du chapitre

Dans ce chapitre, nous avons présenté un processus pour convertir des surfaces mesurées en un ensemble de cartes cohérentes (orientation locale, propriété de réflectances). Nous avons ensuite séparé les données mesurées en des sous-ensembles de matériaux partageant les mêmes propriétés. Cela permet une réduction importante de l'emprunte mémoire comparé aux données mesurées et permet une édition plus aisée par des artistes.

Notre méthode fonctionne bien avec des matériaux présentant une géométrie de surface limitée comme les tissus ou le cuir. Nous utilisons un modèle à microfacettes et nous sommes donc limités par les effets proposés par ce modèle. Ainsi, il n'est pas possible de reproduire les effets de transport sous surfaciques.

La réflexion multiple au sein même de la surface pourrait être gérée en obtenant la géométrie surfacique du matériau et en considérant le voisinage de chaque élément pendant l'étape de minimisation. Cela augmenterait de façon importante le temps de calcul. Pour cela, il faudrait capturer la géométrie en plus de la réflectance.

Avoir accès à la géométrie pourrait aussi permettre de traiter les effets de masquage : une pré traitement pourrait être fait pour déterminer les positions capturées masquées par d'autres éléments suivant l'angle d'observation. De la même façon, ce processus permettrait de déterminer les ombres protées et l'influence du voisinage sur la réflectance capturée en un point.

Cependant, acquérir en plus la géométrie surfacique reste difficile. Concernant le traitement des données pour la compression, une telle information serait utile mais ajouterais beaucoup de complexité. Actuellement, nous traitons chaque texel indépendamment ce qui nous permet de proposer une approche simple et parallélisable. Les considération géométriques et de voisinage complexifieraient significativement l'approche.

Dans des travaux futurs, nous aimerions améliorer le regroupement des matériaux en proposant une méthode plus robuste. Nous voudrions prendre en compte les phénomènes d'inter-réflection pour accroitre la précision de l'étape de minimisation. Nous voudrions proposer une méthode plus simple pour l'édition de la rugosité par groupe. Enfin, nous souhaiterions permettre une détection de la structure de groupe dans les variations locales de rugosité lors du changement de carte de matériaux.

We evaluate our pipeline to the material segmentation step. The last step, colour restoration, is based on the roughness projected after a linear regression. This is an approximation proposed to ease edition. An estimation of this approximation is not meaningful for our evaluation procedure.

Issues when using acquired data for validation

We use synthetic data: we want to compare our algorithm with a ground truth in an ideal case. To evaluate its weakness, we need to remove any potential error imputable to the acquisition process.

When we compare our fit with measured BTF the issues are:

• Material acquisition is based on multiple illumination directions and positions of a 2D sensor. To keep a consistent resolution regardless the captured viewpoint, each BTF slice with not null sensor elevation have to be re-projected: at grazing angles, a given pixel on a slice corresponds to multiple samples of a slice captured with lower viewing elevation (see Figure 9.1).

• There is "macroscopic" shadowing-masking effects. Each pixel of a slice is not representative of a homogeneous material. While the sub-pixel geometry is handled by a BRDF with a shadowing-masking term, over-pixel surface geometry is going to cast shadows around its neighbourhood for illumination at grazing angles while occluding materials for viewing at grazing angles.

• We observed alignment discrepancies and acquisition issue for some BTFs. This is one of the challenges for BTF acquisition. Using a fit per pixel, the alignment is crucial while a reasonable error can remain barely noticeable when used directly in a renderer after interpolation across samples.

The main problems occur at grazing angles, especially for viewing angle. This motivated the weighting using in the fits (see Figure 9.2).

Limitations of the model used

We used a BRDF per texel with a local frame adjustment (normal and tangent orientation) to fit BTFs. This is the major limitation of our method. A BTF is not a spatially varying BRDF: it also encodes sub-surface scattering and geometric effects.

The macroscopic shadowing-masking discussed before is not necessarily a feature of the BTF. It is also one of its limitation. A BTF shall be provided with a height map. Such surface geometry also brings multiple scattering. This is not supported by our method.

Our main goal in this chapter is to evaluate if an "ideal" BTF following the strong assumptions made can be accurately modelled. We are already aware of the limitations of the model and they should not mask other potential limitations in the evaluation.

Limitation of sampling

The angular sampling constrains the ability to accurately represent a measured material depending on its shininess and anisotropy. We cannot accurately reproduce the signal if we use too few angular samples for acquisition: Nyquist-Shannon theorem states that for a given signal, we have to sample it at least twice as much as its highest frequency. The fitting is then impacted and limited to accurately retrieve parameters above a certain threshold.

To illustrate the limit of representable shininess and anisotropy ranges, we compare analytic BRDFs against their tabulated representation. We use the same sampling as the BTF databases we have worked with for tabulating the BRDFs.

We evaluate the reference and tabulated BRDF for 15 4 (ω i , ω o ) sets and compare the error when using nearest and interpolated values for the tabulated BRDF.

Error metric

We use the following error metric to compare the two BRDFs:

E = 1 2π ∫ Ω i ∫ Ωo |f r 1 (ω i , ω o ) -f r 2 (ω i , ω o )| < ω o • n > dω o dω i (9.3)
We discretise this integral:

E = 1 2π N θ i ∑ n=1 Nφ i [θ in ] ∑ j=1 N θo ∑ k=1 Nφ o [θo k ] ∑ l=1 ∫ θ + in θ - in ∫ φ + i j φ - i j ∫ θ + o k θ - o k ∫ φ + o l φ - o l d(ω i , ω o ) < ω o • n > dω o dω i = 1 2π N θ i ∑ n=1 Nφ i [θ in ] ∑ j=1 N θo ∑ k=1 Nφ o [θo k ] ∑ l=1 d(ω i n,j , ω o k,l ) ∫ θ + o k θ - o k ∫ φ + o l φ - o l cos θ o sin θ o dφ o dθ o ∆ω in E = 1 4π N θ i ∑ n=1 Nφ i [θ in ] ∑ j=1 N θo ∑ k=1 Nφ o [θo k ] ∑ l=1 d(ω i n,j , ω o k,l )(cos 2 θ - o k -cos 2 θ + o k )∆φ o k ∆ω in (9.4) d(ω i , ω o ) = |f r 1 (ω i , ω o ) -f r 2 (ω i , ω o )| (9.5) ∆ω in = (cos θ - in -cos θ + in )∆φ in (9.6) ∆φ n = 2π N φ [θ n ] θ - n = max ( 0, θ n - ∆θ 2 ) (9.7) ∆θ = π/2 N θ θ + n = { θ n + ∆θ 2 if n < N θ π 2 otherwise (9.8)
Where:

• ω i n,j is the measured sample for view of angles (θ in , φ i j ).

• ω o k,l is the measured sample for illumination of angles (θ o k , φ o l ).

• N θ i is the number of samples on θ i .

• N φ i [θ in ] is the number of samples on φ i at a given θ in .

• N θo is the number of samples on θ o .

• N φo [θ on ] is the number of samples on φ o at a given θ on .

Results

Figure 9.3 shows the error between the BRDF and synthetic BTF for various roughness and anisotropy values. The first trivial observation is the impossibility to represent accurately BRDFs when the sampling rate is too low for a given shininess. Linear interpolation slightly reduces the error for rougher materials with limited anisotropy (β a > 0.2, β b < 4). However, for rendering applications, linear interpolation shall be used. The visual benefit is significant for any materials (see Figure 9.4). We evaluate the accuracy of the local frame retrieval against a BTF generated with known normal orientation and anisotropy main directions. This step is crucial: we estimate the local frame once. We rely on this estimation for all the following steps. We also evaluate how good the estimation is when other BRDF parameters vary.

Local frame

Normal map

We generate a BTF with a known normal map (see Figure 9.5) and modify the other parameters. The normal map represents all possible directions over an hemisphere.

β a = 0.01 0 • 90 • 180 • 0 • 90 • 180 • 0 • 90 • 180 • β a = 0.02 0 • 90 • 180 • 0 • 90 • 180 • 0 • 90 • 180 • Table 9.
• 90 • 180 • 0 • 90 • 180 • 0 • 90 • 180 • η = 10.00 0 • 90 • 180 • 0 • 90 • 180 • 0 • 90 • 180 • Table 9
.4.: Angle difference between the retrieved normal and the reference with diffuse and index of refraction. The index of refraction have no impact on the accuracy. When the diffuse term slightly increase, the accuracy drastically increase. The low frequency term from the diffuse term overcome the sampling limit allowing use to retrieve accurately the normals.

Error metric We use the angle between the retrieved normal and the ground truth for our error metric:

E = acos(n ref • n f it ) (9.9)
Influence of roughness parameters We vary shininess and anisotropy β a , β b parameters.

Diffuse albedo and index of refraction are fixed: ρ d = 0, η = 8.8. Table 9.3 presents the results. We observe very good accuracy for normal retrieval. The problematic cases are occurring when the material is both highly specular (β a < 0.02) and highly anisotropic (β b > 0.5).

The sampling is the limiting factor for accuracy on highly specular materials.

Influence of diffuse albedo and index of refraction

We change diffuse albedo ρ d and index of refraction η parameters for each test. Roughness parameter are fixed using the worst case found previously: β a = 0.01, β b = 10. Results are presented in table 9.4. A slight increase of diffuse component clears out errors observed previously. The index of refraction has no impact on the quality of the retrieval.

The diffuse albedo is low frequency and the normal retrieval method then rely on this term enhancing the accuracy: sampling is not a limiting factor anymore. The index of refraction acts solely on the specular term (Fresnel coefficient): it does not limit the error introduced by a coarse sampling.

Tangent orientation

The accuracy of the tangent orientation is important in our algorithm. We evaluate it once and it drives the accuracy of the following steps for anisotropic materials. We use fixed tangent and anisotropy maps to evaluate this step. Tangents are varying from 0 to π in x while anisotropy is varying from 1 to 3 in y (see Figure 9.6).

First, we use different shininess and index of refraction values. Then, we vary the normal orientation.

E = √ (α x ref -α x f it ) 2 + (α y ref -α y f it ) 2 (9.11)
Influence of diffuse albedo and index of refraction Table 9.7 shows the error on retrieved roughness values among different diffuse albedo and index refraction values. We observe significant errors:

• At low index of refraction (η = 1.10).

• For very rough materials with low anisotropy (β a > 0.9, β b < 1.5)

In those cases, the fit tends to over-fit the diffuse term to reduce the error. While the parameters are different from the reference, the error is limited. However, this will limit our ability to segment very rough materials. Influence of normal orientation Table 9.8 shows the normal orientation have a very limited impact on the accuracy of the roughness parameter retrieval. The errors observed with high specularity or very rough materials without anisotropy are similar to the one observed when evaluating the influence of diffuse albedo and index of refraction. Again, sampling limits the range of shininess that can be properly modelled and fitted and, very rough materials tends to be fitted with a diffuse lobe. We can expect accurate roughness estimation if the local frame is correctly estimated.

Clustering

We separate the materials based on their roughness parameters. We have seen the accuracy of the roughness retrieval mainly depends on the accuracy of the local frame estimation. If the sampling chosen for acquisition is adapted, we expect good estimate for both local frame and roughness parameters. We generate synthetic BTF using the normal map in Figure 9.5 and tangent map in Figure 9.6. We set ρ d = 0 which is the worst case for normal estimation and η = 8.5.

Our synthetic BTF have three different roughness type distributed as a checker board pattern. We use a base β a and β b parameter and vary either the first, the second or both from section to section. The difference between the base values and the modified ones is changed for each test and denoted by δ βa and δ β b (see Table 9.9).

β a , β b β a + δ βa , β b β a , β b + δ β b β a , β b + δ β b β a , β b β a + δ βa , β b β a + δ βa , β b β a , β b + δ β b β a , β b

Error metric

In each group, we count the number of miscategorised texel. Hence, we get the percentage of miss for a segmentation:

E = 100 • N miss width • height (9.12)
Discussion For δ βa = 0.01, we get poor results for segmentation. The reason is that the normal retrieval is poor. This propagates to errors in the following steps of the pipeline. After a slight increase of δ βa , the segmentation drastically improves. We get good results even with challenging cases where the normal variations cover the entire hemisphere.

Conclusion

We have evaluated our pipeline against synthetic BTFs. This allowed us to separate influence of different parameters on the accuracy.

The main limitations come from the sampling:

• Angular sampling sets a hard limit on the shininess and anisotropy that can be represented and analysed.

• Spatial sampling limits the ability to use a BRDF per location: multiple effects are difficult to separate. Acquisition constrains increases these imprecisions. When sensor elevation increases, the effective resolution is lower: captured samples have to be reprojected.

Normal orientation is also a limiting factor: to determine the anisotropic axis, we need samples around the normal. We cannot construct this signal when the normal orientation is too close to the last captured elevation. Also, azimuthal sampling is not equal at every elevation and we have to rely on interpolation. Finally, inter-reflection effect is more pronounced when normal elevation is higher.

When aware of those limitations, we found our pipeline provides convincing segmentation on both synthetic and measured BTFs. The roughness parameters and the local frame are accurate as long as the sampling used for capturing materials is appropriate. The resulting segmentation depending on accuracy of those parameters is accordingly convincing.

In a future work, we would like to improve the segmentation to automatically detect the number of clusters necessary to accurately represent a BTF. Further investigation has to be done to determine the best segmentation technique to use.

Résumé du chapitre

Nous avons testé notre processus avec des données synthétiques. Cela nous a permis de séparer l'influence de chaque paramètre sur la précision du recouvrement des paramètres pour chaque étape.

Les principales limitations viennent de l'échantillonnage :

• L'échantillonnage angulaire pose une limite sur la brillance et l'anisotropie représentable et analysable.

• L'échantillonnage spatial limite la possibilité d'utiliser une BRDF par position. Plusieurs effets peuvent être mêlés dans un texel et leur séparation est difficile. Les contraintes d'acquisition augmentent cette imprécision : lorsque le capture se trouve à des angles rasants, la résolution effective diminue, les images acquises devant alors être re-dimensionnées.

L'orientation des normales locales et aussi un facteur limitant. Afin de déterminer l'axe d'anisotropie, nous nous basons sur des échantillons distribués autour de la normale. Lorsque cette dernière à une inclinaison proche des derniers échantillons mesurés, il nous est alors impossible d'obtenir des échantillons distribués autour de cette normale.

Lorsque les données traitées prennent en compte ces limitations, notre méthode donne des résultats convaincants, à la fois sur des données synthétiques et sur des données mesurées.

Dans des travaux futurs, nous souhaiterions améliorer la segmentation afin de détecter automatiquement la nombre de groupes nécessaire à la représentation fidèle des données d'entrée. D'autres recherches seraient aussi nécessaires pour déterminer la meilleure méthode de segmentation à utiliser suivant le matériau.

CHAPTER 10

Physically plausible mix and edition of microfacet profiles I ntuitive interactive edition of materials is often challenging due to the large number of parameters. Transferring properties such as the anisotropic aspect from one material to another is especially difficult. This chapter introduce a method to edit measured reflectance. For example, we can combine the anisotropic response from one material with the overall aspect of another. One can create materials combining the shininess of metals and the anisotropy of fabrics (see Figure 10.1).

This method allows the interactive edition of both contributions: one can make the material smoother or rougher, and change its anisotropic response, editing the axes and intensity. The method works with any kind of anisotropic material, including those with two different fibre colours. It allows mixing either measured or analytic materials.

In this chapter, we explain how we extract the isotropic distribution and the anisotropic response for measured materials. Then, we introduce a method to mix isotropy from one material with the anisotropic response from another. Finally, we edit isotropic shininess and anisotropy axis independently.

After extracting isotropic and anisotropic components, the edition can be done in real time. The parameters provided to an artist are intuitive acting directly on the specular lobe shape, orientation, and colour. Distribution re-normalisation is done seamlessly.

Extracting distributions from measured materials

Our method works with measured materials. We used the MERL database as the source for isotropic materials [START_REF] Matusik | A data-driven reflectance model[END_REF] and the UTIA database for anisotropic materials [START_REF] Filip | Effective acquisition of dense anisotropic BRDF[END_REF]. First, we extract material response in a form compatible with combining and editing. For isotropic materials, we extract the Normal Distribution Function (NDF) D(θ h ). For anisotropic materials, we extract the anisotropic response, expressing how the NDF depends on φ h .

We assume for all materials a simplified Cook-Torrance model, where we separate the NDF D and the rest of the response K:

f r (ω i , ω o , λ) = ρ d (λ) π + ρ s (λ) D(ω h )K(ω i , ω o ) 4 cos θ i cos θ o (10.1)

Isotropic materials

For isotropic materials, we extract D iso from the back-scattering data (ω i = ω o ). Under those conditions, the Fresnel term is constant and

G(ω i , ω o ) = 1. f r (ω i = ω o , λ) = ρ d (λ) π + ρ s (λ) D iso (θ h ) 4 cos 2 θ io (10.2)
The azimuthal angle of the half-vector ω h has no influence on the response: D iso is only a function of the elevation angle θ h . Since D iso is supposed to be the same for all channels, we find ρ d , ρ s minimising the differences between each colour channel using Levenberg-Marquadt optimisation [START_REF] Levenberg | A method for the solution of certain problems in least squares[END_REF][START_REF] Marquadt | An algorithm for least squares estimation on nonlinear parameters[END_REF], as implemented by Lourakis [START_REF] Lourakis | levmar: Levenberg-Marquardt nonlinear least squares algorithms in C/C++[END_REF]:

ρ d (λ), ρ s (λ) = min ρ d ,ρs∀λ 1 ,λ 2 ,∀ω i =ωo ⃓ ⃓ ⃓ ⃓ ⃓ f r (ω i , ω o , λ 1 ) -ρ d (λ 1 ) π ρ s (λ 1 ) - f r (ω i , ω o , λ 2 ) -ρ d (λ 2 ) π ρ s (λ 2 ) ⃓ ⃓ ⃓ ⃓ ⃓ (10.3)
The remainder of material response K is tabulated, using the tabulated D iso and Equation 10.1. It represents the shadowing-masking and the remainder of the Fresnel term.

Anisotropic materials

For anisotropic materials, we assume the NDF is of the form:

D aniso (ω h ) = D 0 (θ h ; g(φ h )) (10.4)
where g encodes the anisotropic response of the material.

The key idea to extract the anisotropy component is to use the measured material response for θ h = π/4, so tan θ = 1. We capture as A aniso the anisotropic material response, for each channel, as a function of φ h : A aniso (φ h ) = D(π/4, φ h ). We normalise this response so that:

∫ 2π 0 A aniso (φ h )dφ h = 1 (10.5) (

Combining and editing

After extraction, we multiply the specular lobe of the first material by the anisotropic response of the second material:

f ′ r (ω i , ω o ) = ρ d π + ρ s D iso (θ h )A aniso (φ h )K(ω i , ω o ) 4 cos θ i cos θ o (10.6)
The resulting BRDF is normalised by construction, if both BRDFs were normalised:

∫ Ω D iso (θ h )A aniso (φ h ) cos θ h dω h = ∫ π/2 0 D iso (θ h ) sin θ h cos θ h dθ h ∫ 2π 0 A aniso (φ h )dφ h (10.7)
This new model combines together the respective contributions of both materials; it also separates them in a way that makes edition easier, while preserving normalisation.

Editing the isotropic component

To edit the isotropic component D iso , we assume it corresponds to a shape-invariant probability distribution:

D iso (θ h ) = χ [0,π/2] (θ h )P 22 (tan θ h ) cos 4 θ h (10.8)
We extract P 22 from the tabulated D and edit it with:

P ′ 22 (tan θ h ) = α 2 [P 22 (tan θ h )] α (10.9)
For a Gaussian distribution, this corresponds to dividing its standard deviation by α, making it shinier for α > 1. Figure 10.2 shows the impact on blue-metallic-paint.

Specular and diffuse colours, ρ s and ρ d , can also be edited separately. Note that we do not need to project D onto a specific model to edit its roughness.

Editing the anisotropic component

Anisotropic response from the material is stored as a function of azimuthal angle φ h (see Figure 10.3). For bi-coloured anisotropic materials, it has two separate axis, usually orthogonal. Input material: fabric107. Rotations:

φ u = 20 • , φ v = -35 • .
To edit the material response, we allow rotating each axes, either together (rotating the entire response) or separately (shearing the material response along one axis). We also allow scaling material response along each axis. Specifically, if (φ u , φ v ) are the rotation angle for each axis, and (s u , s v ) are the scaling factors, for each direction φ h we compute a modified direction (c ′ , s ′ ):

[ c ′ s ′ ] = [ s u cos φ u s u sin φ u -s v sin φ v s v cos φ v ] [ cos φ h sin φ h ] (10.10)
We compute the angle φ ′ h = arctan(s ′ /c ′ ) and scaling factor r = √ c ′2 + s ′2 and resample A aniso :

A ′ aniso (φ h ) = r • A aniso (φ ′ h ) (10.11)
We then renormalise A ′ aniso so that:

∫ 2π 0 A ′ aniso (φ h )dφ h = 1.
(10.12)

Conclusion

In this chapter, we have presented a method to edit measured reflectance. We can combine the properties of isotropic and anisotropic materials, allowing the user to create new materials with the shininess of a metal and the anisotropic response of satin.

With our method we can to edit each component individually. We do not need to project the angular dependant components to an analytic model but act directly on the measured data. This allows us to handle a variety of distributions. We can make the new material shinier or to scale the anisotropic response. We can rotate the axes of anisotropy independently. We also can edit the diffuse and specular albedos originating from the measured materials.

Our algorithm has a very small cost after the extraction of material response, allowing realtime response to user inputs, for interactive material editing.

Our algorithm simplicity comes from several approximations: first, we neglected the shadowing and masking term in the Cook-Torrance model; second, the anisotropic contribution should be inside the argument of the NDF. Instead, we multiplied both contributions. These approximations have little impact on the visual aspect of the materials we create, but they contradict the underlying physical model. Removing them while keeping the ability to edit materials in real time is an avenue for future works. Our approximations separate normalisation issues over the parameters, so we only have to compute 1D integrals. Combining contributions inside the argument would require normalisation with 3D integrals.

PART III Conclusion CHAPTER 11

Conclusion H

aving accurate reflectance models is crucial when targeting photorealistic rendering; multiple effects influence the appearance of a surface: the material coating the surface but also the geometry. Specific physical phenomena (diffraction, interferences, fluorescence...) also occur and can significantly influence appearance. To represent reflectance, we explored two solutions: Analytic models Based on physics or observation; analytic models represent light scattering with a mathematical expression. Parameters drive the appearance of the resulting material. They are compact to store (only few parameters), efficient to evaluate (with importance sampling) and editable. They are designed for modelling a specific range of effects.

Measured materials

We get reflectance of a surface from sampling multiple view and illumination directions. We use a lookup table to evaluate reflectance from specific directions. They require lengthy measurement time and expensive devices. This type of materials could theoretically represent arbitrary complicated reflectance; acquisition constraints limits the range of measurable materials. A gantry design is meant to capture a defined range of physical behaviour. Measured materials are interesting for research to validate or develop new analytical models.

Recent advances to efficiently sample homogeneous materials made their usage in production possible. But, for spatially varying materials, we still have to rely on compression methods. Also, limited editability of measured materials make them difficult to handle in a creative context.

In this thesis, we propose techniques to solve the storage issue and the editability at the expense of fidelity. This is a reasonable trade-off if editability is required.

Summary of contributions

We have presented a technique for capturing spatially varying anisotropic BRDF with few illumination directions and a fixed viewpoint. We can quickly capture a material and get parameter maps. While not being as accurate as a full measurement with multiple point of views, we meant this method to set editable plausible analytic BRDF parameter maps starting from a real-world material. It is restricted to anisotropic materials.

Brute force material measurements are not restricted to anisotropic materials and target accuracy. They require a lot of storage and are difficult to edit. We proposed a technique for fitting an analytic BRDF to measured spatially varying materials. We combine different locations of the material to expose an editable material map. Given this map, we can either edit the material structure, mix materials from different datasets or modify the roughness of a whole group. This segmentation also allows us to reuse the raw measured data with another material spatial distribution if accuracy is needed.

Another challenging aspect for edition is to expose intuitive handles with which to edit a material. Physically based analytical models use physical quantities which are not necessarily the most intuitive parameters to tweak. We introduced a technique to directly edit measured reflectance. We can edit the shininess and the anisotropy independently and mix them together. We can mix the anisotropy of a fabric with the shininess of a metal. Then, we can edit axes of anisotropy and change the shininess. It works either with measured or analytical materials. This method provides convincing results and is interactive. This conveys intuitiveness. It lacks a physically based foundation but holds a physically plausible nature being energy conservative.

Future challenges for material acquisition 11.2.1. Better specifications

We only get the post processed data when we get access to measured reflectance. Unprocessed data are valuable for research purposes. It would also be a valuable asset for archiving purpose: when new calibration techniques and post processing are introduced, previously measured data may benefit from them.

We also wish to gain access to more specifications regarding measurement conditions: hardware specifications are hardly specified. Also, different colour spaces are used and not always clearly defined.

An ideal measured database would provide:

• Unprocessed data along with processed data,

• Full specification of the measurement platform,

• Sensor's spectral sensitivity curves,

• Spectral specification of the illuminant in radiometric units,

• Lens spectral response,

• Lens optical aberrations,

• Clear file specification,

• Portable code for accessing file.

Ideally, we could reproduce the measurement setup virtually and simulate it in a path tracer to validate fitting process and better evaluate influence of surface geometry on the overall measured reflectance.

Future challenges for material acquisition

A lot of these information are difficult to get access to. Manufacturers does not necessary provide full hardware specifications. Also, providing unprocessed data is challenging: the storage needed is many orders of magnitude higher than the processed data.

Further physical properties acquisition

Spectral We exposed the importance of spectral rendering in the first part of this thesis.

We then based our approach on tristimulus acquired materials. The techniques we presented also work with spectral data but we could not validate this claim. There is a lack of acquired spectral spatially varying material:

• Tristimulus spatially varying material with a coarse sampling are already barely manageable in term of storage. Adding several spectral bands would multiply accordingly the amount of memory necessary to store such measurements.

• Consumer cameras use a Bayer matrix in front of the sensor. They capture RGB values. Acquisition setup would have to use specific sensors with a multi pass capture process using different filters in front of the lens. This would significantly increase measurement time. Alignment issues would also increase.

We wish camera manufacturers propose more spectral band acquisition at the expense of resolution. This would not only have an impact on material acquisition setup design but also, in photography for more accurate white balance correction and post processing.

Bi-spectral A large range of natural or manufactured materials are fluorescent. There is currently a lack of consideration for those effects in computer graphics. Supporting fluorescence would have important benefits, from industrial design to artistic purposes.

Study of measured materials bootstraps development of new analytical models. Getting access to bi-spectral BRDF would be desirable for popularising fluorescence effects.

Bi-spectral measurement adds a significant overhead to the acquisition process though. Recent advances to importance sample spectral acquisition of homogeneous materials is a promising research area for proposing an efficient bi-spectral measurement process.

Polarisation While being barely noticeable by our eyes, polarisation effects impact light transport. There is a lack of polarisation aware material acquisition devices targeting computer graphics.

Affordable acquisition techniques

In the literature, many setups propose fast acquisition of material assuming the reflectance is following an analytic model. We proposed our own in this thesis for anisotropic spatially varying materials. This range of heuristic based acquisition techniques are valuable: they are potentially broadly accessible and can give a realistic starting point for an artist to design a material. They also are generally simple to setup and usable on site.

These techniques mostly address RGB acquisition. We could use heuristics to target spectral and bi-spectral affordable acquisition process. For instance, illumination using specific spectral basis or using grating filters for spectral acquisition, exploiting Kasha's rule for bi-spectral acquisition.

Edition

Fit and semantic maps

We tackle efficient representation of measured reflectance by fitting BRDF to measured data. Our motivation was to benefit from the compactness and evaluation efficiency of analytic BRDF. Using a physically based microfacet model ensures energy conservation after the fitting process. Editability comes through 2D parameter maps.

Semantic maps are an interesting approach to propose user friendly handles to edit a material: editing wear and tear or oxidation level instead of roughness, and index of refraction guided by physically based spatial distribution. In our fitting pipeline, we made a step in this direction. We proposed to group similar materials and expose an editable material map. For a given material, structural information such as wear and tear distribution would also be meaningful.

We expect further work in this direction to improve editability of spatially varying materials

Spectral edition

A large range of research target intuitive material edition, but there is a lack of consideration for spectral authoring.

The need for full spectral specification partially explains the lack of interest for spectral rendering. Spectral uplifting is a promising tool to overcome this limitation. But an artist shall keep control of the final appearance. We could use spectral uplifting as a starting point to let artists design in RGB and tweak the spectral uplifted curves.

Still, we miss support for designing software supporting spectral images and edition. Designing a user-friendly pipeline for spectral image edition is an avenue for future work.

The same apply for material parameter edition. While widely used microfacet models have no restriction on handling spectral parameters, material design tools target mainly tristimulus pipelines. Spectral rendering is mostly reserved to specific areas or research.

Providing user-friendly spectral aware tools to artist would give another creativity dimension to modify appearance depending on illumination. It would also push spectral rendering forward.

What is an efficient representation?

In the course of this thesis, we focused on BRDF analytic models for representing measured reflectance. We target efficient rendering and compact representation with editability in mind.

The editability was our motivation for using analytical models. It is part of an efficient representation to target efficient authoring.

We briefly mentioned other efficient representations. A range of work rely on PCA or neural based approaches. We did not explore those areas. They provide more convincing visual results than our approach but mostly lack editability. There is potentially a gap to bridge, between analytic BRDF representation approach and dimension reduction. An hybrid approach to guide BRDF fitting seem an interesting area to explore.

CHAPITRE 12

Conclusion version française P our le rendu photoréaliste, il est capital d'avoir des modèles de réflectance de bonne qualité.

Plusieurs effets influencent l'apparence d'une surface : le matériau la constituant mais aussi sa géométrie. Des phénomènes physiques particuliers (polarisation, interférences, fluorescence. . . ) entrent aussi en jeu dans l'apparence.

Pour représenter des réflectances, nous avons exploré deux solutions :

Les modèles analytiques Basés sur la physique ou sur l'observation, les modèles analytiques représentent la réfection de la lumière par une expression mathématique. Les paramètres gouvernent l'apparence du matériau. Ce type de modèle est économe en mémoire (seulement quelques paramètres à stocker) et modifiables. Ils sont conçus pour modéliser certains types d'effets.

Les matériaux mesurés Nous les obtenons en échantillonnant plusieurs points de vue et d'illumination. Une table de correspondance est ensuite utilisée pour déterminer la réflectance dans des directions spécifiques. Ils nécessitent des temps de mesures importants, des équipements couteux et sont difficiles à éditer. Ces types de matériaux peuvent en théorie représenter des réflectances arbitrairement compliquées. Les contraintes d'acquisition limitent toutefois la variété des matériaux pouvant être acquis : un système de mesure est conçu pour numériser un certain nombre de phénomènes physiques. Ils sont intéressants pour la recherche afin de développer ou de valider des modèles analytiques.

Les avancées pour l'échantillonnage efficace des matériaux homogènes ont rendu leur usage possible en production. Mais, lorsqu'il s'agit de matériaux variant spatialement, nous devons toujours utiliser des méthodes de compression. De plus, les possibilités d'édition limitées des matériaux mesurés rendent leur adoption difficile dans des contextes créatifs.

Dans cette thèse, nous proposons des méthodes pour solutionner les problèmes de stockage et d'édition au détriment de la fidélité. Il s'agit d'un compromis raisonnable lorsque la possibilité d'édition est nécessaire.

Résumé des contributions

Nous avons présenté une technique pour l'acquisition de matériaux anisotropes à variance spatiale en utilisant un nombre réduit de directions lumineuses et un point de vue fixe. Nous pouvons capturer rapidement un matériau et en obtenir des cartes de paramètres. Malgré le fait que cette méthode soit moins précise qu'une mesure intégrale avec plusieurs points de vue, nous l'avons mise au point pour permettre d'obtenir un point de départ réaliste pour une fonction de matériau analytique en partant d'un matériau réel. Cette méthode est réservée aux matériaux anisotropes.

Si une acquisition détaillée est souhaitée, on peut se tourner vers les méthodes brute-force. Celles-ci ne se restreignent pas à l'acquisition de surfaces anisotropes et sont plus précises. Cependant, elles nécessitent un espace de stockage important et sont difficiles à éditer. Nous avons présenté une technique permettant de représenter un matériau mesuré par une fonction analytique. Nous avons ensuite combiné différentes positions de la surface présentant des caractéristiques similaires pour en établir une carte de matériaux. Cette carte peut ainsi être facilement éditée afin de modifier l'apparence de la surface.

Nous avons enfin présenté une technique permettant l'édition directe des matériaux mesurés. Nous modifions les paramètres directement à partir des données acquises. Notre méthode permet de combiner les caractéristiques de brillance d'un matériau avec l'anisotropie d'un autre. Chacun de ces composants sont modifiables indépendamment. Bien que notre méthode ne soit pas basée physique, elle est physiquement plausible : elle assure la conservation de l'énergie. De plus, l'édition peut se faire interactivement.

Défis d'avenir pour l'acquisition de matériaux 12.2.1. Meilleures spécifications

Lorsque nous avons accès à des matériaux mesurés, nous n'avons accès qu'aux données finales. Les données brutes seraient intéressantes pour la recherche. Elles seraient aussi d'intérêt pour l'archivage : lorsqu'une nouvelle technique de calibration et de traitement serait introduite, les données déjà mesurées par le passé pourraient en bénéficier.

Nous aimerions aussi avoir accès à plus de spécifications concernant les conditions de mesures : les données matérielles ne sont pas souvent clairement indiquées. De plus, différents espaces de couleurs sont utilisés et ne sont pas toujours clairement identifiés.

Une base de données de mesures idéale contiendrait :

• Les données brutes avec les données traitées,

• Une spécification complète de la plateforme de mesure,

• La réponse spectrale du capteur,

• Les caractéristiques spectrales de l'illuminant en unités radiométriques,

• La réponse spectrale de la lentille,

• Les aberrations optiques de la lentille,

• Une spécification claire du format de fichier,

A. Code for generating a metamer spectrum

We say two spectra metamer when those two spectra are different but perceived the same by our vision system. A reflectance can be a metamer of another reflectance when illuminated using a specific illuminant: the resulting irradiance can be different while our eyes see the same colour.

Our aim is to find two different spectra s(λ) and s ′ (λ) with the same XYZ values. Let (X, Y, Z) be the tristimulus values of the first spectra s(λ). We are fitting a spectrum s ′ (λ) such as its tristrimulus (X ′ , Y ′ , Z ′ ) is minimizing ∥(X, Y, Z) -(X ′ , Y ′ , Z ′ )∥ 2 .

In addition, the CIE ∆E * is a colour metric designed to compare two colours. We also try to minimize this distance. From this cost function, we fit a spectrum s ′ (λ) which will have the same XY Z values as a reference spectrum s(λ).

A.1. Cost function

If the fit is successfull, the two spectra while being different, will appear the same. We want to find a reflectance spectrum r ′ (λ) looking the same as r(λ) when illuminated with the same illuminant i(λ). So, we want to get a spectrum l ′ (λ) which is a metamer of l(λ).

After executing the fit, we retrieve r ′ (λ) by dividing it by i(λ) to cancel out the illuminant.

IV 8 .

 8 A pipeline for fitting acquired spatially varying materials 97 8.1. Overview of the pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 8.2. BTF re-normalisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 8.3. Local frame retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 8.3.1. Surface normal orientation . . . . . . . . . . . . . . . . . . . . . . . . . . 101 8.3.2. Anisotropy axis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 8.4. Material classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 8.4.1. Normal distribution function fitting . . . . . . . . . . . . . . . . . . . . . 103 8.4.2. Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 8.5. Restricting roughness within a group . . . . . . . . . . . . . . . . . . . . . . . . 107 8.6. Colour restoration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 8.7. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 8.8. Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 8.9. Conclusion and future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 8.10. Résumé du chapitre (French) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 9. Evaluation of our material segmentation method 117 9.1. Scope of the evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 9.1.1. Issues when using acquired data for validation . . . . . . . . . .

  (a) A bathroom scene only with diffuse material coating every surfaces. (b) The same scene with different materials.

Figure 1

 1 Figure 1.1.: Materials drastically change the appearance of a scene. This scene is modified from the PBRT renderer example scenes.

1. 2 .

 2 Desirable material properties (a) Direction illumination. (b) Indirect illumination. (c) Global illumination.

Figure 1 . 2 .:

 12 Figure 1.2.: When materials reflect light, they become secondary light sources. Direct illumination (a) does not solely contributes to the overall appearance of the scene (c). The indirect illumination takes an important part to the overall appearance (b). With the same lighting and geometry, appearance of a scene can drastically change depending on materials coating the surfaces. Images courtesy of Cyril Soler.

( a )

 a Wood is a spatially varying material. (b) Wall painting detail.(c) Microfiber towel detail.

Figure 1 . 3 .:

 13 Figure 1.3.: Different types of materials and surfaces: wood (a) is spatially varying while the wall (b) has a uniform paint and the towel (c) is made of the same fibre but their geometry influences the surface appearance.

( a )

 a Une salle de bain où toutes les surfaces sont couvertes d'un matériau diffus. (b) La même scène utilisant differents materiaux.

FIGURE 2

 2 FIGURE 2.1. : Les matériaux changent significativement l'apparence d'une scène. Ces images ont été simulées avec le moteur de rendu PBRT à partir des scènes d'exemple.

( a )

 a Illumination directe. (b) Illumination indirecte. (c) Illumination globale.

FIGURE 2 . 2 .:

 22 FIGURE 2.2. : Lorsque les matériaux réfléchissent la lumière, ils deviennent des sources lumineuses secondaires. L'illumination directe (a) n'est pas la seule à contribuer à l'apparence de la scène (c). L'illumination indirecte a une contribution significative dans l'apparence globale (b). Avec les même conditions d'éclairage et la même géométrie, l'apparence d'une scène peut changer suivant les matériaux constituant les surfaces. Images de Cyril Soler.

( a )

 a Le bois est un matériau variant spatialement. (b) Détail de la surface d'un mur. (c) Détail d'une étoffe de type micro fibres.

FIGURE 2 . 3 .:

 23 FIGURE 2.3. : Different types de matériaux et de surfaces : le bois (a) varie spatialement alors que le mur (b) est recouvert d'une couche de peinture uniforme et que l'étoffe (c) est faite de la même fibre mais leurs géométries influencent leurs apparence.

Chapitre 5 Chapitre 6

 56 Nous expliquons comment la lumière est réfléchie par les surfaces et donnons les bases du modèle à micro facettes. Nous proposons une revue et une catégorisation des méthodes d'acquisition proposées dans la littérature.

Figure 3 .

 3 Figure 3.1.: Illustration of a linearly polarised electromagnetic wave. The magnetic induction and the electric field are perpendicular and in phase with each other.

Figure 3 . 2 .:

 32 Figure 3.2.: Visible light spectrum situated in the range of electromagnetic waves.

  a) Radiance is the radiant flux Φ e over a patch dA projected on in the direction ω i per unit of solid angle dω i . (b) Irradiance is the radiant flux Φ e over a patch dA. It can be seen as the integral of the radiance from all directions.

Figure 3 . 3 .:

 33 Figure 3.3.: Radiance and irradiance definitions.

Figure 3

 3 Figure 3.4.: CIE D65 illuminant relative spectral power distribution. E e (560 nm) = 1.

E

  Figure 3.4 shows an example of the relative SPD of CIE (Commission Internationnale de l'Éclairage) D65 illuminant. An illuminant is a standardised light emission relative SPD.

  When summing two waves of the same frequency of phase difference π, they cancel each other. In this case, the phase difference is 5π 6 , the resulting wave have a lower intensity than each individual wave it is resulting from.

Figure 3

 3 Figure 3.5.: Interference between two waves of the same frequency. The phase and amplitude of the resulting wave depends on the amplitudes and phases of the two waves.

Figure 3

 3 Figure 3.6.: Fresnel's vectors: red and purple vectors represent waves of the same frequency with different amplitudes and phase shifts. The sum of the two gives the amplitude and phase shift of the resulting wave.

Figure 3 . 7 .:

 37 Figure 3.7.: Optical path length. The optical path length determines the amplitude and phase shift when two monochromatic waves cross point.

Figure 3 .

 3 Figure 3.8.: A simulation of interferences caused by two slits when lit with coherent D65 spectrum. We used Fresnel's vectors to accumulate contributions from a stochastic evaluation.

Figure 3 . 9 .:

 39 Figure 3.9.: Interferences observable at macro scale. At the surface of a lens, the thin layer of anti-glare coating creates interferences causing coloured reflection at the surface of the glass (a). The surface of soap bubbles is also showing rainbows effects (b). Other surfaces such as leather show bright coloured highlight (c).

Figure 3 .

 3 Figure 3.10.: Interferences can cause visible hazy reflections. Here, an example for aluminium from Holzschuch and Pacanowski [HP17a]. Material have irregularities at the scale of the wavelength producing interferences.

( a )

 a Linear polarisation. The amplitude projection of the field on x is null. (b) Elliptical polarisation. The projection of the field on x and y planes is 90 • out of phase with higher magnitude for the sinusoid on y axis. (c) Circular polarisation. The projection of the field on x and y planes is 90 • out of phase with equal magnitude.

Figure 3 .

 3 Figure 3.11.: Polarisation states of the electric field of a monochromatic wave. The magnetic induction is perpendicular to the electric field (see Figure 3.1) and is not represented for clarity. The projection on the x plane represents the rotation of the electric field over time. When the wave is projected onto the z plane, the different polarisation patterns are visible.

Figure 3 .

 3 Figure 3.12.: When using both Q and U , the indetermination is solved. Only one solution is valid, we can discard the second (represented by the dashed arrows).

( a )

 a Left circular polarised light. (b) Right circular polarised light.

Figure 3 .

 3 Figure 3.13.: Using quarter-wave filter and linear linear polarisation filters, we can determine the polarisation rotation direction.

Figure 3 .

 3 Figure 3.14.: A measured reflectance, TeckWrap vinyl wrapping film (Ibiza Sunset RD02), from

Figure 3 . 15 .:

 315 Figure 3.15.: An example of the so-called Stokes shift, here shown for the material found in pink fluorescent 3M pink Post-it notes.Practically all fluorophores exhibit similar characteristics: namely, that there is one main band where energy is absorbed, and one where it is re-emitted. Note that one still needs the full re-radiation matrix of the fluorophore in question to generate such a plot, which only shows the cumulative shifting effect of the material. Figure3.17 shows the entire reradiation matrix of this material.

  Rendering of fluorescent objects using ART renderer[START_REF]The Advanced Rendering Toolkit[END_REF]. The inner spheres are fluorescent while the outer ones are not.

Figure 3 .Figure 3 .Figure 3

 333 Figure 3.16.: Example of scenes featuring fluorescent objects. In (a, b), fluorescent objects are lit with a near monochromatic blue LED torch. The objects reflect other colours than the one used to lit the scene. In (c), a rendering shows the same object with and without fluorescence in a daylight environment. Fluorescent objects appear brighter than the environment bringing energy from bandwidth where our eyes are less sensitive to bandwidth where it is more sensitive.

  Figure 4.1.: Depending on the lighting condition, the cones or the rods cells are active, resulting in our eyes having different sensitivity. Judd and Vos corrected photopic CIE 1924 V (λ) known as V M (λ) [Jud51, Vos78] and CIE 1951 scotopic V (λ).

Figure 4 Figure 4

 44 Figure 4.2.: Stiles and Burch RGB colour matching functions for 2 • observer [SB55]. These are not emission spectra. Those curves describe the knob's positions for the three monochromatic illuminants used in the experiment (represented by the arrows) to match a specific monochromatic stimulus.

  The metamer reflectances shown under HP1 and under D65 illuminants.

Figure 4 . 5 .:

 45 Figure 4.5.: We generate a metamer reflectance for HP1 illuminant. While the two spectra are very different, the perceived colour is exactly the same under this illuminant. When using CIE D65 illuminant, we can distinguish the colours.

(a)

  Fluorescent light (b) Red (R) LED (c) Green (G) LED (d) Blue (B) LED (e) R+G+B LEDs (f) R+G LEDs (g) G+B LEDs (h) R+B LEDs

Figure 4 . 6 .:

 46 Figure 4.6.: Painted artwork lit with different lightsources. It is impossible to distinguish colours when lit with almost monochromatic coloured LED (b, c, d), each reflectance reemitting shades of the same colour, the paint not being fluorescent. With the three LEDs combined (e), we can better make the difference. Althought white and yellow are hardly the same while easily separable with the fluorescent lighting (a).If only two LEDs are used,some colours are impossible to distinguish while being very different. For instance, with red and blue lighting (h), yellow and red (on the fish's tail) look very similar.

2.

  In a global illumination context we show: a) Energy conservation problems rising with tristimulus rendering with generated spectra, b) Error accumulation after multiple bounces in a Cornell box, c) Discrepancies depending on the colour matching function used for representing illuminant and reflectance.

  Figure 4.7.: Comparison of spectral and tristimulus rendering. When using spectral rendering, we transform the colours from spectra to RGB at the end. With tristimulus rendering, we transform both reflectance and illuminants into their tristimulus counterparts prior to the evaluation. Then, we compute the final colour by multiplying illuminant and reflectance colour together. This approximation yields to observable differences as shown by the ∆E * 2000 metric.

  Figure 4.8.: For each patch of the Macbeth colour checker, we generate a reflectance metamer for D65, then illuminate with HP1. When using a tristimulus renderer, the XYZ values are the same for both reflectance spectra of a given patch. Only a spectral renderer is able to show the colour difference with a suitable illuminant.

  * 2000 (a ′ , b ′ ) (b) Macbeth colour checker lit by the two illuminants.

Figure 4

 4 Figure 4.9.: We generate a metamer illuminant from D65. If illuminating a perfect grey albedo (reflecting equally all the wavelengths) with the two illuminants our eyes see the same reflected colour. Other reflectance spectra will look different. Contrary to a tristimulus renderer, a spectral renderer can show the colour difference.

Figure 4 .

 4 Figure 4.10.: Comparing scattered light colour after two bounces with a tristimulus and spectral renderers using a CIE D65 illuminant.

  Figure 4.11.: Rendering of a Cornell box with Mitsuba using 30 spectral samples and RGB

( a )

 a Spectral rendering. (b) CIE XYZ 2006 colour matching functions.(c) RGB colour matching functions.

Figure 4 .

 4 Figure 4.12.: Comparison of Cornell box lit with HP1 illuminant using a spectral renderer (a) with tristimulus renderers using XYZ (b) and RGB (c) colour matching functions for light transport. Tristimulus renderers are strongly impacted by the colour matching function used to convert spectra. Those renders were done using Shadertoy 2 .

  [MBR] • Microphysics of light emission (colors of the Universe -part 1)[START_REF] Neyret | Microphysics of light emission (colors of the Universe -part 1)[END_REF] • The Young-(Helmholtz)-Maxwell Theory of Color Vision[START_REF] Heesen | The young-(helmholtz)-maxwell theory of color vision[END_REF] 4.5. Résumé du chapitreDans ce chapitre, nous avons présenté comment la lumière est perçue par notre système visuel. Nous avons expliqué l'origine des fonctions de transfert de couleur permettant la transformation d'un spectre en une couleur. Afin d'éviter les valeurs négatives, la base XYZ est utilisée. À partir de cette base, différents systèmes de représentation des couleurs peuvent être utilisés suivant l'application. En informatique graphique, nous employons généralement la base RVB (Rouge Vert Bleu) pour l'affichage sur un moniteur.Pour l'informatique graphique, ces représentations trichromatiques ont une contrainte majeure : différents spectres peuvent donner les mêmes coordonnées trichromatiques. Cela peut engendrer d'importantes différences dans les systèmes de rendu utilisant un système trichromatique. Les systèmes de rendu spectraux sont épargnés par ce problème.

Figure 5

 5 Figure 5.1.: Huygens' principle: a wavefront is a superposition of wavelets.

  Light reflection with a perfect mirror.

  Notation and vector direction more often used in computer graphics.

Figure 5 .

 5 Figure 5.2.: Reflection of an incoming ray on a perfect mirror (5.2a). In computer graphics, ω i , ω o notations are more often used with ω o pointing toward the light source and ω i toward the observer from the surface hit point (5.2b).

Figure 5 . 3 .:

 53 Figure 5.3.: When entering a medium with a different index of refraction, the light path is deviated following Snell-Descartes' law.

Figure 5 .

 5 Figure 5.4.: Dispersion of light beam by a prism. The speed of light in the prism depends on the wavelength. This is a dispersive medium. This image was simulated using Shadertoy 1 .

Figure 5

 5 Figure 5.5.: Reflection of a mug on a wooden table. At grazing angles, the reflection of the mug on the table increases. Fresnel equations model this angular dependency.

  Bidirectional Subsurface Scattering Distribution Function describes light reflected by a surface. Light can travel inside the surface and exit at another point: this is subsurface scattering. BRDF Bidirectional Reflectance Distribution Function is a approximation of the BSSDF. The ray is reflected at the same position it is hitting the surface. BTDF Bidirectional Transmittance Distribution Function is similar to BRDF for characterising transmittance. It describes the scattering of a ray in the bottom hemisphere of a surface. BRDF BTDF (a) BRDF defines the scattered light at the surface. BTDF defines the scattered light after transmission. Subsurface scattering BSSDF (b) BSSDF takes into account subsurface scattering which BRDF does not. The hit point of the ray is not necessary the same as the exit point.

Figure 5 .

 5 Figure 5.6.: BSDF are the most general functions to characterise scattering distribution.

( a )

 a Regular coordinate system: ω i and ω o are defined in the global frame. (b) Rusinkiewicz parametrisation. θ d and φ d are the spherical coordinates of the ω o direction in the frame with normal ω h shown in blue.

Figure 5 .

 5 Figure 5.7.: Reflection of an incoming ray on a perfect mirror (a). In computer graphics, ω i , ω o notations are more often used with ω o pointing toward the light source and ω i toward the observer from the surface hit point (b).

Figure 5 . 8 .:

 58 Figure 5.8.: Microfacets illustration: at micro-scale, a material is made of tiny perfect mirrors oriented in various directions. This visualisation gives an "intuition" of the model. If we use the Smith shadowing-masking term [Smi67], the microfacets are not correlated.

•

  How to Write a Polarisation Ray Tracer [WW11] • Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs [Hei14] Further readings • Peter Shirley Ray tracing book series 3 : -Ray Tracing in One Weekend [Shi16a] -Ray Tracing: The Next Week [Shi16b] -Ray Tracing: The Rest of Your Life [Shi16c]5.5. Résumé du chapitreDans ce chapitre, nous avons présenté les concepts de base de l'interaction de la lumière avec la matière. Nous avons présenté l'interaction avec un miroir et la réfraction lors d'un changement de milieu. La relation entre la vitesse de la lumière et la longueur d'onde expliquent la trajectoire du rayon lumineux. Cela peut être à la fois interprété par le principe de Fermat en optique géométrique ou par le principe d'Huygens en optique ondulatoire. Nous avons présenté les équations de Fresnel donnant la quantité d'énergie réfléchie ou transmisse lors d'un changement de milieu suivant l'angle et la polarisation de la lumière. Les changements de polarisation peuvent être modélisés par les matrices de Mueller appliquées aux vecteurs de Stokes.

Figure 6

 6 Figure 6.1.: Method proposed by Marschner et al. [MWL + 99] to capture a BRDF without the

Dana

  et al. [DvGNK99] introduced the term BTF (Bidirectional Texture Function). They proposed a database of 60 surfaces with 200 combination of viewing and direction samples. Sattler et al. [SSK03] introduced a database (UBO2003) of six BTF with 81 view and 81 light directions and proposed an algorithm to render these BTFs based on Principal Component Analysis. Schwartz et al. [SSW + 14] provide an extensive description of the acquisition setup used at UBO (University of Bonn). Haindl and Filip [HFR12] captured and released a database of six BTF (UTIA BTF). Filip et al. [FKH + 18] released a database of sixteen BTFs captured from the MAM2014 material set.

  Figure 7.1.: Our acquisition pipeline: first, we place a material sample on our acquisition plat-

SignalFigure 7 . 2 .:

 72 Figure 7.2.: Our algorithm: first, we estimate the azimuthal angle φ n of the shading normal and the anisotropy direction φ a using the phase shifts of the first and second harmonics.Then we estimate BRDF parameters by minimisation: specular reflectance with even harmonics, diffuse reflectance and normal elevation with c 0 and c 1 . In a final step, we perform minimisation on the measured signal for minor adjustments on the parameters. For added accuracy, we can loop over the minimisation process using computed parameters as input values.

Figure 7 .

 7 Figure 7.3.: Our acquisition platform: the sample and camera are static and aligned with each other, the light arm rotates around them. Arm position and picture acquisition are controlled by a computer.

Figure 7 . 4 .:

 74 Figure 7.4.: Notations for BRDFs: ω i is the in-

  D is the microfacet normal probability distribution. It is an open parameter. The original paper used a Gaussian: f (x) = e -x . In this paper, we use two different distributions: GGX, introduced by Walter et al. [WMLT07] and f (x) = e -x p , introduced by Brady et al. [BLPW14].

•Figure 7

 7 Figure 7.5.: Extracting BRDF parameters from the sampled signal: we minimise the distance in Fourier space between the measured signal and the signal predicted by the BRDF models.

Figure 7

 7 Figure 7.6.: After extraction, we store BRDF parameters in textures: anisotropy direction, shading normal, diffuse, specular (ρ d , ρ s ) and roughness (α x , α y , p) parameters.We visualise here the anisotropy α x /α y and shininess, (α x +α y )/2 of the material.

Figure 7 Figure 7 .

 77 Figure7.9.: Difference between anisotropy direction and the value retrieved using the phase shift of c 2 , as a function of the number of samples. Our algorithm retrieves an accurate value for the anisotropy direction even for small number of samples.

( a )

 a Original picture. (b) Reconstructed picture with similar viewing and lighting direction.

Figure 7 .

 7 Figure 7.11.: Comparison between original picture and picture reconstructed from the extracted BRDF. See also Figure 7.6.

Figure 7 .

 7 Figure 7.12.: Picture of our four datasets under different viewing conditions, rendered in realtime.

Figure 7 .

 7 Figure 7.13.: We can transfer extracted material parameters to other objects and use them with global illumination. Left teapot with the carbon material, centre one with the powerbank material and right one with the phone material.

Figure 7 .

 7 Figure 7.14.: Scene rendered with Mitsuba using the extracted maps from the phone case and the powerbank.

  Figure 7.15.: Influence of the shading normal and the anisotropy on a theoretical signal. Red axis shows the anisotropic direction, blue line, the φ n angle. The parametrisation is reflectance on the radius as a function of φ i . (a) theoretical anisotropic BRDF with the shading normal aligned with view and normal. (b) theoretical isotropic BRDF with a tilted shading normal. (c) combination of the tilted normal and the theoretical anisotropic BRDF. (d) signal extracted from a pixel on the phone case sample.

Figure 8

 8 Figure 8.1.: Overview of our pipeline. Starting from a measured BTF, we extract first the local frame for each location. Then, we fit a microfacet BRDF. This provide us with roughness parameters that allows us to segment the material. Finally, each group of similar materials is fit again for colour restoration. The maps can be modified to derive a new material from the original BTF.

Figure 8 .

 8 Figure 8.2.: Measured sample positions in the UTIA and UBO BTF databases. The surface S

( a )

 a Measurement device frame. (b) Local frame for given position p on the BTF having a surface normal n l .

Figure 8 .

 8 Figure 8.3.: While the BTF angles are provided in the frame of the measurement device (a), for a given position on the BTF, the local orientation can be shifted due to local surface irregularities (b).

  Figure 8.6.: Example of segmentation in three sets on UTIA fabric02.

  Figure 8.7.: Colour restoration on UTIA fabric02.

FineFigure 8 Figure 8

 88 Figure 8.8.: Example of resulting maps obtained at the end of the pipeline.

Figure 8 .

 8 Figure 8.10.: Example of material edition by changing the material map and the normal map.

( a )

 a Original material. (b) Fit without modification.(c) Edition of the roughness parameters.

Figure 8 .

 8 Figure 8.11.: Editing roughness per material group to give a rougher appearance.

  Figure8.12.: Some limitations of our pipeline. For MAM2014 007, the sharpness decreases at raising angles while improves at grazing angles. For MAM2014 015 multiple scattering effects occurring in the groves of the paper is not present in the fit showing darker groves. For UBO corduroy, multiple scattering effects and interreflections are not model by the fit resulting in decrease of loss of energy.

9. 1 .•Figure 9

 19 Figure 9.1.: We get sharp samples when the camera axis is aligned with the sample global normal. The whole camera sensor resolution is not used when elevation is increasing. Up-scaling is necessary to keep a consistent resolution across all captured samples.

Figure 9 .

 9 Figure 9.5.: Reference normal map used in the synthetic BTF for evaluating normal retrieval.

  β b = 1.00 β b = 5.50 β b = 10.00

  3.: Influence of roughness parameters: shininess β a and anisotropy β b , on normal retrieval. Diffuse albedo and index of refraction are fixed. The accuracy is poor when both shininess and anisotropy are high. The sampling limits our ability to handle those extreme cases. ρ d = 0.00 ρ d = 0.05 ρ d = 0.10 η = 1.10 0

  β b map.

Figure 9

 9 Figure 9.6.: Reference maps used in the synthetic BTF generated for evaluating accuracy of tangent retrieval.

  Influence of diffuse albedo and index of refraction on the roughness retrieval. With rough materials, accuracy of roughness parameters is poor: the fit focuses on the diffuse term to reduce the error. Error metric We use the euclidean distance between (α x ref , α y ref ) and (α x f it , α y f it ) to define our error:

  Influence of normal orientation on the accuracy of the roughness retrieval. When the local frame is accuratly estimated, we get limited error on roughness estimation.

δ

  βa = 0.010 δ βa = 0.040 δ βa = 0.070 δ βa = 0

Figure 10 . 1 .:

 101 Figure 10.1.: We extract the anisotropy of a material (centre, fabric-107) and combine it with the isotropic roughness of another material (left, alum-bronze), creating a new material (right) with the shininess of a metal and the anisotropy of a fabric.

  a) Original material. (b) Modified roughness.

Figure 10 . 2 .:

 102 Figure 10.2.: Roughness edition on blue-metallic-paint.

  Figure 10.3.: Editing the anisotropic component A by changing the orientation of its axes.

  λ)x ¯(λ)dλ = ∫ Λ s ′ (λ)x ¯(λ)dλ ∫ Λ s(λ)y ¯(λ)dλ = ∫ Λ s ′ (λ)y ¯(λ)dλ ∫ Λ s(λ)z ¯(λ)dλ = ∫ Λ s ′ (λ)z ¯(λ)dλ (1)

For a given

  channel C from (X, Y, Z) tristimulus the corresponding c ¯from the colour matching functions (x ¯, y ¯, z ¯), our cost function is:{ E C = (∫ Λ s ′ (λ)c ¯(λ)dλ -C e c tr u m _ t o_ X Y Z _ c o s t (x , * args , ** kwargs ) : XYZ_ref = args [0] xyz_bar = args [1] wavelength = args [2] s = Spectrum ( wavelength , x ) XYZ_metamer = spectrum_to_XYZ (s , xyz_bar ) de = delta_e_from_XYZ ( XYZ_ref , XYZ_metamer ) return [ ( XYZ_metamer [0] -XYZ_ref [0]) **2 , ( XYZ_metamer [1] -XYZ_ref [1]) **2 , ( XYZ_metamer [2] -XYZ_ref [2]) **2 , de ]

  from scipy . optimize import least_squares import numpy as np def g e n e r a t e _ m e t a m e r _ r e f l e c t a n c e ( spectrum wavelength = [ wl for wl in range ( start_wl , end_wl + 1 , step ) ] init_values = [ illuminant . value ( wl ) for wl in wavelength ] res = least_squares ( spectrum_to_XYZ_cost , init_values , xtol =1 e -15 , ftol =1 e -15 , bounds =( len ( init_values ) * [0.0] , init_values ) , args =[ XYZ , xyz_bar , wavelength ] ) return wavelength , res . xA metamer reflectance is a reflectance which appear the same as a reference reflectance under a given illuminant.The spectrum reflected l(λ) by a surface of a given reflectance r(λ) under the illumination i(λ) is:l(λ) = r(λ)i(λ)The resulting XY Z values are: λ)i(λ)x ¯(λ)dλ Y = ∫ Λ r(λ)i(λ)y ¯(λ)dλ Z = ∫ Λ r(λ)i(λ)z ¯(λ)dλ

  def g e t _ r e f l e c t a n c e _ m e t a m e r ( reflectance , illuminant , xyz_bar , stat_wl , end_wl , step ) : s = reflectance . mul ( illuminant ) XYZ = spectrum_to_XYZ (s , xyz_bar ) wavelength , values = g e n e r a t e _ m e t a m e r _ r e f l e c t a n c e ( XYZ , illuminant , xyz_bar , start_wl , end_wl , step ) # We ' ve multiplied the reflectance by the illuminant , we # need to revert it back to get a reflectance refl_meta = [ v / ( illuminant . value ( wl ) ) for wl , v in zip ( wavelength , values ) ]

10. Physically plausible mix and edition of microfacet profiles 10

  . . . . . 118 9.1.2. Limitations of the model used . . . . . . . . . . . . . . . . . . . . . . . . 118 9.2. Validation of the fitting pipeline . . . . . . . . . . . . . . . . . . . . . . . . Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.3. Limitation of sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.4. Local frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.5. Roughness estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.6. Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4. Résumé du chapitre (French) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1. Extracting distributions from measured materials . . . . . . . . . . . . . . . . . 10.1.1. Isotropic materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1.2. Anisotropic materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2. Combining and editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2.1. Editing the isotropic component . . . . . . . . . . . . . . . . . . . . . . 10.2.2. Editing the anisotropic component . . . . . . . . . . . . . . . . . . . . . 10.3. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4. Résumé du chapitre (French) . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary of contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2. Future challenges for material acquisition . . . . . . . . . . . . . . . . . . . . . What is an efficient representation? . . . . . . . . . . . . . . . . . . . . . . . . .

	III. Conclusion
	11. Conclusion
	11.1.

. . . 120 9.2.1. Note on roughness parameters . . . . . . . . . . . . . . . . . . . . . . . . 120 IX Contents 9.2.2. 11.2.1. Better specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.2. Further physical properties acquisition . . . . . . . . . . . . . . . . . . . 11.2.3. Affordable acquisition techniques . . . . . . . . . . . . . . . . . . . . . . 11.3. Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.1. Fit and semantic maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.2. Spectral edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.

12. Conclusion (French version) 12

  .1. Résumé des contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2. Défis d'avenir pour l'acquisition de matériaux . . . . . . . . . . . . . . . . . . . 12.2.1. Meilleures spécifications . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.2. Avancées sur la capture des phénomènes physiques . . . . . . . . . . . . 12.2.3. Des méthodes d'acquisition accessibles . . . . . . . . . . . . . . . . . . . 12.3. Édition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.1. Cartes sémantiques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.2. Édition spectrale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4. Qu'est-ce qu'une représentation efficace ? . . . . . . . . . . . . . . . . . . . . . .

  Ils ne requièrent ainsi que trois dimensions pour caractériser leur réflectance : l'élévation des directions d'observation et d'illumination et la différence d'azimut entre ces deux directions. Pour obtenir des matériaux réalistes, l'acquisition est un choix intéressant. Le processus d'acquisition capture de multiples directions d'illumination et d'observation. Mais, les difficultés d'édition et le coût d'acquisition rend leur usage rare et parfois impossible. Pour les matériaux variants spatialement, l'espace de stockage est aussi une contrainte majeure.

	Matériaux mesurés et analytiques

Homogènes et variant spatialement Un matériau dans sa forme la plus générale varie spatialement. Le bois est constituée de veines (voir Figure

2

.3a). La peau a une distribution cellulaire variable et des imperfections. Certains matériaux peuvent êtres considérés homogènes. Un plastique, en ignorant des défauts de fabrication potentiels, a une réflectance constante sur sa surface.

Isotropes et anisotropes

Un matériau anisotrope a un axe de réflexion privilégié quelque soit la direction d'illumination et d'observation. C'est typiquement le cas des métaux brossés. Ils requièrent quatre dimensions pour caractériser leur réflectance : l'azimut et l'élévation des directions d'observation et d'illumination.

Un matériau isotrope n'a pas ce type de réflexion privilégiant un axe.

Table 4 .

 4

1.: Commonly used photometric quantities

Table 6 .

 6

	Method	Spectral/ Bi-Spectral	HDR Anisotropic	Spatially varying
	Publicly available (see 6.5)

1.: Summary of material acquisition techniques in 2019.

  Computation time for the different steps of our algorithm.

	Material	Alignment	Parameter Extraction	Resolution	Average per pixel
	Amulet Carbon fibre Power bank Phone case	12.8 s 3 s 13 s 6 s	36599 s 2137 s 6177 s 3870 s	2299 × 1724 px 561 × 421 px 1249 × 937 px 949 × 712 px	9.2 ms 9.0 ms 5.3 ms 5.7 ms
	Table 7.1.:

  φ og Azimuthal angles made by ω i , ω o in global frame. φ i l , φ o l Azimuthal angles made by ω i , ω o in local frame. p Location on the BTF. BRDF with incoming direction ω i and outgoing direction ω o . f r (ω i , ω o , n l ) BRDF of the surface with local frame orientation of normal n l . b r (p, ω i , ω o ) BTF at position p with ω i toward the observer and ω o toward the light. α x , α y Roughness parameters. ρ

	Material properties
	f

r (ω i , ω o ) d Diffuse albedo.

Table 8.1.: Notations used.

  Execution time in seconds per pixel per core.

	Rendering Normals Material map	MAM2014 MAM2014 MAM2014 MAM2014 MAM2014 MAM2014 MAM2014 MAM2014 MAM2014 MAM2014 MAM2014 MAM2014 MAM2014 MAM2014 MAM2014 MAM2014 corduroy01 fabric02 fabric03 wood01 wallpaper corduroy proposte impalla Average Table 8.2.: Original	Modified	UTIA UBO	NDF fitting Colour restoration 7.91s 25.24s 0.42s 4.64s 4.06s 11.57s 2.94s 8.54s 0.86s 6.37s 0.85s 3.06s 2.37s 20.30s 0.49s 4.61s 2.33s 5.34s 5.90s 23.01s 8.50s 30.42s 6.59s 22.69s 10.60s 31.67s 15.92s 36.48s 2.33s 7.68s 2.88s 10.30s 9.38s 25.55s 0.97s 5.52s 0.81s 2.30s 0.81s 3.58s 0.93s 3.68s 14.82s 38.19s 7.75s 41.11s 7.94s 29.94s 4.80s 16.74s Original Modified

Table 9

 9

	.3 Table 9.4	Fig. 9.5	0 0	0 [0; 0.1] [1.10; 10] 8.5	[0.01; 0.02] 0.01	[1; 10] 10.0
		Tangent map retrieval
	Table 9.5 Table 9.6	0 [5 • ; 75 • ] [0 • ; 137.5 • ] 0	Fig. 9.6a	0 0	[1.10; 10] [0.01; 0.20] Fig. 9.6b 8.5 0.2
		Roughness map retrieval
	Table 9.7 Table 9.8	0 [0 • ; 50 • ] [0 • ; 360 • ] 0	0 0	[0; 0.1] [1.1; 7.03] Fig. 9.7a Fig. 9.7b 0 8.5
		Material segmentation
						δ βa	δ β b
	Table 9.10	Fig. 9.5	Fig. 9.6a	0	8.5	[0.01; 0.1]	[1.1; 3]

Table 9 .2.:

 9 Summary of tests performed to evaluate our pipeline.

Table 9 .

 9

9.: Distribution of roughness for segmentation test.

Table 9 .

 9

10.: Segmentation. The error on grouping drastically drops after δ βa reaches 0.04.

1 eV (electron volt) is the amount of energy of a single infrared photon (λ = 1240 nm).

We chose to call the B-field "magnetic induction". It can be called "magnetic field " which is confusing: the H-field (in amperes per meter [A.m -1 ]) being also called "magnetic field " on some textbooks. The B-field may also be designed as "magnetic flux density". The H-field may also be designed as the "magnetic field intensity" or "magnetic field strength".

This is a simplified form of Maxwell equations in the considered conditions.

This often holds for diamagnetic and paramagnetic materials.

The Haidinger's brush is a noticeable example showing our eyes are slightly sensitive to polarisation.

We use S notation which is widely adopted in the literature. It shall not be confused with the previously introduced Poynting vector.

To simplify the notation. we are representing a single bounce. The reflectance spectra or tristimulus values have to be multiplied to each other for each further bounce.

https://www.shadertoy.com/view/wsd3Rj

https://www.shadertoy.com/view/wlSXz3

This is an approximation of the model: Fresnel terms are defined for perfectly smooth surfaces.

http://in1weekend.blogspot.com

http://cg.cs.uni-bonn.de/en/projects/btfdbb/download/ubo2003/

http://btf.utia.cas.cz/?btf_dwn

http://btf.utia.cas.cz/?btf_mam2014

Different BTF databases use different file formats. We first translate each format into our own prior to the fitting. This was done to help managing various BTF formats and to speed-up loading.

Remerciements 10.4. Résumé du chapitre Dans ce chapitre, nous avons présenté une méthode pour combiner les propriétés d'anisotropie d'un matériau avec les propriétés de brillance d'un autre. Cela permet à un utilisateur de créer de nouveaux matériaux en utilisant par exemple la brillance d'un métal avec l'anisotropie d'un satin. Notre méthode permet l'édition de chaque composant individuellement. Nous n'avons pas besoin de projeter les composantes angulaires de la BRDF sur un modèle analytique. Nous travaillons directement sur les données mesurées. Il nous est ainsi possible de gérer une grande variété de distributions. Nous pouvons changer la brillance ou l'anisotropie indépendamment. Nous pouvons tourner les axes d'anisotropie indépendamment. Nous pouvons enfin modifier les albédos diffus et spéculaires venant d'un matériau mesuré. Notre algorithme a un coup très réduit après l'extraction des caractéristiques des matériaux. Cela permet une édition en temps réel.

La simplicité de notre méthode est due à plusieurs approximations. D'une part, nous négligeons les effets de masquage dans le modèle de Cook-Torrance. D'autre part, la contribution anisotrope devrait fait partie intégrante de la distribution de normales ; dans notre méthode, nous multiplions les deux contributions. Ces approximations on peut d'impact sur l'aspect visuel des matériaux que nous générons mais, elles contredisent le modèle de réflectance utilisé. Cela nous permet toutefois de normaliser notre modèle simplement en n'ayant qu'à calculer des intégrales en une dimension.

Dans des travaux futurs, nous voudrions corriger ces approximations. Nous souhaiterions incorporer les effets de masquage et passer de notre modèle conservant l'énergie à un modèle basé physique.

Databases

• Dana et al. [START_REF] Dana | Reflectance and texture of real-world surfaces[END_REF] http://www1.cs.columbia.edu/CAVE/software/curet/html/texmap.html • Matusik et al. [MPBM03a] https://www.merl.com/brdf/ • Ngan et al. [NDM05] https://people.csail.mit.edu/addy/research/brdf/ • Sattler et al. [SSK03] http://cg.cs.uni-bonn.de/en/projects/btfdbb/download/ubo2003/ • Haindl and Filip [HFR12] http://btf.utia.cas.cz/?btf_dwn • Schwartz et al. [SSW + 14] https://cg.cs.uni-bonn.de/en/projects/btfdbb/download/ubo2014/

• Filip et al. [START_REF] Filip | Effective acquisition of dense anisotropic BRDF[END_REF] http://btf.utia.cas.cz/?brdf_den_dwn https://cg.cs.uni-bonn.de/de/projekte/svbrdf-database-bonn/ubofab19/

Résumé du chapitre

Ce chapitre présente les techniques communément utilisées pour l'acquisition de matériaux. Nous avons catégorisé ces techniques en quatre groupes principaux et avons présenté leurs limites respectives :

• L'approche brute force est la plus générique. Elle permet l'acquisition de tout type de matériaux. Cependant, elle nécessite une longue durée d'acquisition.

• L'approche combinant plusieurs vues et illumination par capture permet une acquisition plus rapide. Elle est restreinte aux matériaux homogènes et nécessite d'obtenir des objets d'une certaine géométrie fait du matériau à mesurer. Elle est aussi destructive pour les matériaux anisotropes.

• L'approche par échantillonnage préférentiel permet de limiter les directions d'acquisition aux positions importante. Une reconstruction permet ensuite l'évaluation de la fonction de réflectance. Cette approche est pour l'instant réservée aux matériaux homogènes.

• L'approche basée sur un modèle analytique part de l'hypothèse qu'un matériau à acquérir peut se représenter sous une forme analytique. Il ne s'agit pas, en soi, d'une méthode donnant lieu à une fonction de réflectance mesurée. Mais, il s'agit d'une approche intéressante pour concevoir des systèmes d'acquisition rapides.

Les matériaux mesurés sont d'intérêt dans un large panel de domaines. De l'héritage culturel pour l'archivage d'objets précieux au prototypage industriel.

Ainsi, il parait intéressant de combiner la fidélité des matériaux mesurés avec la compacité des modèles analytiques. À défaut d'une fidélité parfaite, cela permet l'édition aisée des propriétés de réflectance, l'évaluation efficace ainsi que le stockage réduit au moment du rendu.

Pour cela, nous proposons dans la suite de ce manuscrit des méthodes pour relier les réflectances mesurées à des modèles analytiques.

PART

II

Contributions CHAPTER 7

Fast acquisition of spatially varying anisotropic reflectance P ractical acquisition of reflectance parameters is a difficult problem. Accurately sampling and storing a single material can take several hours with current acquisition devices. Anisotropic materials show interesting reflection patterns, varying with the orientation of the material. Because the amount of light reflected depends on both the incoming light and observer orientation, the acquisition process is even more time-consuming for anisotropic materials. Similarly for spatially varying materials, where material properties change with the position on the object.

For this reason, people often use analytic material models for rendering. Analytic BRDFs can also be authored and importance sampled more efficiently. Fitting parameters for these analytic materials to measured data is difficult; it is still an ongoing research project.

In this chapter, we present a method for acquisition of spatially varying anisotropic materials.

We designed an acquisition process that is cost effective and simple to reproduce. We extract material parameters from a small set of pictures (about 20), with different lighting conditions but the same viewing condition. We use Fourier analysis to reconstruct material properties from the sub-sampled signal. We extract diffuse and specular reflectance, direction of anisotropy, shading normal and reflectance parameters in a multi-step process. Our algorithm works with the anisotropic Cook-Torrance BRDF model, with several microfacet distributions. The choice of the microfacet distribution is left to the user, our algorithm automatically extracts the corresponding parameters. We make no assumptions about homogeneity or regularity of the material, and recover parameters for each pixel independently: the spatial resolution of our algorithm is equal to the spatial resolution of the input pictures.

Previous approaches enforced spatial coherence through post-processing. We found that our approach produces coherent maps without any post-processing, making it more robust and straightforward to parallelise.

Our algorithm is described in Figure 7.1: we place a material sample on a rotating gantry, acquire a set of photographs for varying incoming light azimuthal directions. From these samples on ϕ o . For a given (ω i , ω o ), we are most likely not having a measured sample. We linearly interpolate from closest values to limit high frequencies.

The BTF databases we have used have more azimuthal samples on grazing angles. For a specific position on the BTF, the angle made by the local normal with the global normal prevents using samples close to the horizon, since they are under the horizon after the correction. Furthermore, those measurements are less reliable due to potential masking effects and resolution reduction. There are also more prone to influence of neighbouring materials. This limits our ability to deal with normals close to the lasts sampled elevations. This is discussed in Chapter 9.

Considering anisotropy symmetric around φ, we are interested in the second harmonic (s(φ) = s(φ + π)). We extract c 2 using Fast Fourier Transform [START_REF] Cooley | An algorithm for the machine calculation of complex Fourier series[END_REF][START_REF] Frigo | The Design and Implementation of FFTW3[END_REF]. Then, the phase of the second harmonic gives the anisotropy main axis.

Material classification

Once we have local frame orientation for each location, we then fit locally a BRDF using the corrected values of reflectance from the BTF using equation 8.14. Based on the roughness parameters found through this optimisation process, we will gather the different locations sharing a similar normal distribution function shape and segment the material into groups of matching roughness.

Normal distribution function fitting

We first extract the roughness parameter per location: those are going to be used later for grouping similar materials. In microfacet theory, a normal distribution function is independent of the wavelength. Hence, to speed up the fitting process, we use the average of captured colour channels.

Classification

The first fitting process is meant to identify similar materials at different locations on the bidirectional texture function. It provides information per pixel about the parameters of an analytic microfacets BRDF. We now select which of those are relevant for making the decision on the material categorisation.

Criteria Various criteria could be used to separate different appearances of the surface. While diffuse and specular albedo parameters have been used so far, there are not the most physically suited for this separation: an imprinted wallpaper can be made of the same material while having varying albedos. Another example where the separation would be wrongly made on texture rather on material intrinsic characteristics is wood grain.

Fresnel function would be an appealing choice for a physically based separation. Due to its very nature, the Fresnel term, driven by the index of refraction (IOR) have an increasing influence at grazing angles. Those are also the less reliable in the measurements. We have replaced the analytic Fresnel term by a tabulated one. It is one of the main limitation of the algorithm. Hence, we could not use Fresnel function to separate materials.

We chose to group similar material based on their roughness parameters α x , α y . Those are driving the microfacet profile, they should give an accurate estimate of the material types found over the bidirectional texture function. Figure 8.6 shows an example of various combination of roughness parameters found for UTIA fabric 02. While the woven pattern are blurry, our method is still able to give a consistent segmentation. Figure 8.8 & 8.9 show segmentation for other materials. It gives convincing segmentation for UTIA MAM2014 009 (basket ball) being able to separate rougher pattern from the shinier ones. For UTIA MAM2014 010 (flocked paper), even if the background is slightly noisy, it correctly identify the hairy surface from the rest of the material.

Grouping similar roughness

CHAPTER 9

Evaluation of our material segmentation method W e presented a method to fit analytic BRDFs to BTFs. We relied on a segmentation technique for grouping sparse measurements from different locations.

We used fitted roughness parameters to perform this segmentation. The coloured specular and diffuse albedos are later retrieved based on those steps. The roughness parameters fit rely itself on the correctness of a local frame estimation procedure.

We need to evaluate the accuracy of each component of our pipeline. We focus on the first steps as they are crucial for the last steps of our pipeline.

Using measured BTFs for validation is not necessarily meaningful: we observed measurement errors and alignment discrepancies in the databases we have used. We do not want those to affect the evaluation. BTFs also pack geometry and reflectance together. This results in interreflection and masking effects encoded in the BTF. These effects are beyond the scope of our model.

We instead use synthetic BTFs. They allow to study retrieval accuracy for each parameter independently and evaluate the inter-influence of parameters on retrieval accuracy.

We estimate the accuracy of the local frame retrieval, the analytic fit, and the material grouping. At each step, we study the influence of other parameters on the retrieval accuracy of the parameters. Finally, we discuss the strengths and weaknesses of our approach.

Scope of the evaluation

In the algorithm described in Chapter 8, we retrieve:

• A local frame consisting in a normal and a tangent. Each is estimated in a separate step.

The rest of the pipeline rely on those estimates.

• Roughness parameters: in the microfacets model used, α x and α y drive the roughness in the direction of the tangent and in the orthogonal direction.

• A segmentation of the BTF made of a discrete set of material groups.

• Coloured specular and diffuse albedos.

Table 9.1.: Sampled values of θ l , θ v and number of samples on φ l , φ v for captured and synthetic BTFs used.

Validation of the fitting pipeline

Note on roughness parameters

In the course of the tests, we use β a , β b in place of α x , α y to set roughness parameters:

• β a defines how specular the material is. An high value corresponds to a rough material. Lower the value is, shinier the material is.

• β b defines how anisotropic the material is. For isotropic material, α x = α y i.e. β b = 1. Further from unity β b is, the more anisotropic the material is. For convenience, we are using

x and α y parameters and rotating the anisotropy axis φ a by 90 • .

We get α x , α y from β a , β b with:

We used previously this parametrisation to constrain the fit. Here, this choice lets us evaluate independently the impact of anisotropy and shininess on the robustness of the algorithm.

Procedure

We generate a synthetic BTF from parameter maps and execute our pipeline. We keep same viewing and illumination samples as in the measured BTF used (see Table 9.1). At each location, we evaluate a BRDF based on generated parameter maps. We pack those slices using the same file format as used during the fitting process 1 .

To evaluate our algorithm, we use a specific set of parameter maps at each step. We compare the retrieved value with ground truth using a specific error metric at each step. Thanks to this process, we can evaluate the influence of other parameters on the accuracy.

If a parameter estimated by a previous step is necessary, we use the estimated parameter found by the previous step: we are not relying on the reference parameter. Hence, we evaluate the accumulation of errors as well.

Table 9.2 presents the different tests performed with range of parameters used. β a = 0.01 β a = 0.07 β a = 0.14 β a = 0.20

Table 9.5.: Influence of shininess and index of refraction on the accuracy of tangent retrieval.

When the shininess is too important (β a low), the low number of samples limits our ability to retrieve tangent orientation.

θ n = 5.00 • θ n = 28.33 • θ n = 51.67 • θ n = 75.00 • φ n = 0.00

Table 9.6.: Influence of normal orientation on the accuracy of tangent retrieval. When normal is at grazing angle, we cannot retrieve the tangent orientation: we need samples around the normal. Errors are also introduced by different sampling scheme on φ depending on elevation.

Error metric We define our error with the angle difference between the recovered tangent and the ground truth:

Influence of shininess and index of refraction We evaluate the impact of shininess and index of refraction on the accuracy of the tangent retrieval. We fix the value for diffuse albedo and normal orientation (ρ d = 0, n l = z). We change the shininess β a and index of refraction η for each test. The results are presented in Table 9.5. When the material is highly specular (β a low), we are not able to retrieve the anisotropy axis. From β a = 0.07, the error decreases. Increase of η also helps slightly the algorithm accuracy.

Errors observed with highly specular materials are due to sampling not suited for such kind of materials. If we want to acquire highly specular materials, we need a higher number of samples.

Influence of normal orientation

We evaluate the impact of normal orientation on the accuracy of the tangent retrieval. We fix the value for diffuse, specular and index of refraction (ρ d = 0, η = 8.5, β a = 0.2). We change the normal elevation and azimuth for each test. The results are presented in Table 9.6.

We cannot retrieve the tangent direction when the normal is too close to the maximum acquired elevation: we rely on samples distributed around the normal to retrieve the tangent orientation. But acquired samples stops at elevation 75 • .

The normal azimuth also influences the accuracy of retrieved tangent. The azimuthal sampling is not equal at every elevation. We have to interpolate between samples to feed to the Fourier transform an equally distributed signal on φ h around the normal.

Roughness estimation

We fit a microfacet BRDF at each location in the second step of our pipeline. We rely on the roughness parameters to later segment the material. So, we are primary interested in evaluating the accuracy of roughness parameters at this stage. The Fresnel and the diffuse component have no influence on the segmentation.

We generate BTFs with varying shininess β a on x and anisotropy β b on y (see Figure 9.7).

A.2. Generating metamers

We generate a metamer reflectance under HP1 of one of the Macbeth colour checker's patch. The metamer looks as expected the same under D65. But when lit by another illuminant, we can distinguish the two resulting colours.