Mr Bachar

Keywords:

Face recognition is a task so common to humans that the individual does not even notice the extensive number of times it is performed every day. Nowadays, face recognition has been studied as a specific case of object recognition. It has received special attention in recent years due to a great variety of applications such as robothuman interaction, control by gesture, surveillance, security, and people tracking.

A lot of face modeling techniques and classification methods have appeared and have progressed in the last three decades. The very first section of this chapter is dedicated to introducing some features of visible face recognition with its advantages and problematics. Some representative methods of visible face recognition are also mentioned and compared with each other inside a brief overview. The second section analyzes the potential of applying infrared technology in face recognition to solve at the same time two major problems in visible imagery linked to illumination conditions and face spoofing attacks.

Visible Face Recognition

State of the art

. FIGURE 2.6: Facial expression changes the face model.

In our first study, we concentrate on the research of a new method that can detect the face spoofing attack using a single uncalibrated normal camera such as smartphone. The concept of this type of attack will be introduced in the next section.

Face Spoofing Attack

Context and problematic

Human faces are the most characteristic features which can be used to distinguish one person from the others. Recognizing the parents and the family is the very first lesson for each human being. Since the development of imagery technology, human brains are charged with another task: face recognition from a photo. This task is now the most critical identification solution in our society when our face appears in many papers such as passports, ID card, driver license, student card ... At this era of digital technology, the task of face recognition is more and more entrusted to automatic systems. The powerful computer is able to accomplish many complex tasks including people detection and authentication, movement tracking and predicting, illness detection and classification... Face recognition is now applied in a wide range of use-cases with different levels and constraints of security. There are passive systems using by the authority to inspect the activity in sensitive regions.

There are active calibrated systems for access control in airports, companies ... and many other facilities. There are also many distributed uncalibrated systems like laptop unlock program, smartphone identification application which gives users more initiatives to decide the environmental conditions of their attempt. However, visible face recognition is theoretically and practically vulnerable to face spoofing attack. A well-performing authentication system can be easily bypassed using a photo of a genius user's face presented in front of the system's camera. The threat is especially dangerous since many people let their pictures be public on the internet, in particular, on social networks. An intruder can find many highquality photos without needing anything more than the user's name and exploits them to operate the attack. In order to strengthen the authentication process, the administration can add to the system a new security layer which can reduce this vulnerability as known as the liveliness detector or face spoofing detector. The existent solutions for such layer are wide large in terms of technology but almost all the precise methods require a complex system with two or more cameras and even other types of sensor. The more complex the system, the less applicable the solution, these methods are useless outside its use-cases. Therefore, in this study, we propose a new way for liveliness detection dedicated to simple mono-camera systems like Chapter 1. Introduction smartphones and tablets. This novel method exploits their movability to make use of not only one picture but a whole video of user's face in many poses in order to rebuild the face in 3D coordination which is used later to distinguish a genius face from the attack attempts.

Another crucial problem of visible face recognition is linked to the fact that all the light and color that can be observed from human faces is only a reflection of the light from other sources such as the sun, the lamps. Obviously, visible imagery is highly dependent on the illumination conditions. Some research has proposed methods that can function correctly across the change of light intensity, but there is always a decrease in terms of precision when the modification is so brutal. No visible face recognition method can be processed in the lack of light, but the necessity of authentication in darkness is not overrated. In this context, infrared and particularly thermal imagery has become a promising alternative and complement method for face recognition. However, until this day, thermal face recognition does not achieve the required mature level to be applied widely and distributively. In fact, the application of infrared images in face recognition process is challenged by the lack of distinguishable feature in these images. IR spectrum has its own problems which can affect the precision of the identification program. In order to deal with this problem, we aim to introduce a new method of thermal face recognition solution employing 3D models of the head which contains information of vascular network measured from a thermal video. The process is dedicated to functioning in different use-cases where the only required equipment is a single thermal camera.

Outline

This first chapter of the Thesis introduces, in general, the context and the problems that lead to this study. Il also describes the structure of this Thesis. The second chapter is dedicated to present an overview of the automatic face recognition domain which, in this study, is divided into two principal sections: Visible face recognition and thermal face recognition. In the first section of this chapter, the Thesis introduces the advantage and problematic of visible imagery. It also provides some representative methods from the beginning of computer vision to this day. Infrared technology is presented in the next section where we emphasize how it can bypass the problematic of visible imagery. But, as having been stated above, infrared technology is not void of its challenges which is also described in the second section.

The third chapter introduces a novel approach to model the head of users by its 3D features. The chapter starts by reviewing the advantage of representing users' face data using its 3D model. Then, the next section is dedicated to giving an overview of the 3D reconstruction method using different sensors. The last part of the chapter presents the details process to make this 3D model from a single video of users' face which is applied in two principal directions of this Thesis.

Outline

The fourth chapter constructs and examines a new method for the detection of one of the major problems in the visible face recognition domain: face-spoofing attack. It starts with a recall of face-spoofing attack and how it can affect the authentication process. The next section describes the detailed scheme to detect this attack.

In the last part, we provide the performance of this method using the result of our experiment.

In the fifth chapter, we propose a novel face recognition solution using a 3D model of the head computed from a thermal video which contains information of vascular network. By its nature, the thermal imagery can obviously detect the facespoofing attack and stay invariant to illuminant conditions. However, there is less distinguishable information in a thermal image than a visible image which reduces the precision of the face recognition process. The 3D reconstruction provides geometric data of the face which can be mixed with vascular network information from thermal imagery to improve its performance.

Introduction

Authentication by biometric characteristics is the most important demand which makes evolute face recognition. This technique has become more and more popular and plays a key role in many security systems such as passport identification for some airports, access control for a lot of companies and even unblocking application for some smartphone.

The reason of this phenomenon can be explained by several advantages that face recognition holds over other biometric technique. Face recognition is above all, a very natural, very human-friendly process which scores the highest percentage of compatibility with machine-readable travel documents among the 6 biometric techniques (face, finger, hand, voice, eye, signature).

However, facial features are not considered as the most reliable biometric technique because of its limits in terms of performance in unconstrained environment [START_REF] Singh | An overview of face recognition in an unconstrained environment[END_REF]. Facial expression, various illumination conditions, face spoofing attack, twin The following part is dedicated to highlighting some typical solutions for facerecognition challenge from the early state of this domain to now. However, visible imagery is always vulnerable to face spoofing attacks as we asserted this problem in the next subsection. A study of state of the art about face spoofing detection can be found in the third part of this section.

Methods

Eigenfaces Identifying the different face from an image is the primary purpose of face recognition. Little noise (unconstrained environmental conditions) exists in every photo, but the presence of these noises does not make the image totally random. There are some patterns which help in recognizing the different features of the image. A lot of patterns which can be seen in face recognition are in the neighborhood of the nose, eye, and mouth or the distance between the facial features. In facial recognition field, these characteristics are known as eigenfaces [START_REF] Turk | Face recognition using eigenfaces[END_REF]. Most of these experiments are processed on the frontal view of the face. Principle component analysis (PCA) is a method for extracting these eigenfaces from an image. It is a vice-versa, that is if a system is including a set of eigenfaces then the original image of the face can be restored. This process is efficient and practical compared to others techniques in constrained conditions [START_REF] Martinez | PCA versus LDA[END_REF][START_REF] Sirovich | Low-Dimensional Procedure for the Characterization of Human Faces[END_REF][START_REF] Turk | Face recognition using eigenfaces[END_REF].

But this approach has a few weaknesses over unconstrained environment and behavior such as facial expressions in which there are some changes in facial feature and shape. Besides, the variety in pose heads to the distortion of the distance of the elements. Thirdly, the changes in the illumination conditions, for example, the bright light will make image saturate. But the illumination conditions can be overcome using the Fisher face method which is an enhancement of eigenfaces, but it uses the Fisher's linear discriminant analysis (LDA) [START_REF] Etemad | Discriminant analysis for recognition of human face images[END_REF][START_REF] Belhumeur | Eigenfaces vs. Fisherfaces: recognition using class specific linear projection[END_REF]. There are multiple techniques of face recognition which employs the neural network approach for face authentication. PDBNN is a probabilistic decision-based neural network and it applies the idea of decision based neural network (DBNN) [START_REF] Lin | Face recognition/detection by probabilistic decision-based neural network[END_REF][START_REF] Kung | Decision-based neural networks with signal/image classification applications[END_REF]. The network approach is not entirely connected with this method. The network is split into K subnets; each subnet identifies one individual from the set. Its neurons use Gaussian activation function. The "face-subnet" is the summation of neuron outputs.. This method primarily consists of two stages. Firstly, the subnets are trained by their own face models, and after that, the subnet features are trained by some other appropriate samples from other face class, this stage is called as decisionbased learning scheme. Only misclassified features are utilized by the decisionbased learning scheme and not the whole of the training samples for the training.

If the samples are classified to the wrong subnet, then the parameters of the legitimate subnet will be attuned, so that its decision region will be shifted closer to the misclassified sample. PDBNN classification has the benefits of both statistical methods and neural network techniques [START_REF] Lin | Face recognition/detection by probabilistic decision-based neural network[END_REF]. It is simple to implement its distributed computing process on parallel machines.

Geometrical feature matching

The overall geometrical feature of a face is enough for identifying a person. The facial form is represented by the location and the dimension of the facial elements like nose, mouth, eyebrows, eye, and the face outline [START_REF] Goldstein | Identification of human faces[END_REF][START_REF] Kaya | A basic study on human face recognition[END_REF][START_REF] Cox | Feature-based face recognition using mixture-distance[END_REF]. Mixture-distance-based method was proposed by I.J.Cox et.al [START_REF] Cox | Feature-based face recognition using mixture-distance[END_REF],

Template Matching

In the template matching approach, images are described by the 2D matrix of intensity, and these values are compared by metrics methods such as Euclidean distance using a single template which describes the whole face. There are multiple techniques based on the template matching, in which the entire face is represented by more than one model [START_REF] Baron | Mechanisms of human facial recognition[END_REF].

Bruneli and Poggio chose four feature regions (eyes, mouth, nose and the whole face) [START_REF] Brunelli | Face recognition: features versus templates[END_REF]. While comparing the performance between the template matching solution and the geometrical matching solution on the same face database, they observed that the geometrical matching solution is less accurate than the template matching algorithm. However, the requirement of high capacity in computation is the most crucial limit of template matching and, furthermore, the difficulty also situates in the template representation. There are a few discrepancies between the test image and the template. This tolerance might influence the achievement of the face recognition method.

Thermal image based technique

The thought of a thermal image was introduced to overcome the distortions and the illumination variation. It takes the subsurface characteristics of the face which may be admitted as a biometric feature. In this method the obtained thermal images are given to morphological processes and filtration, selecting the features of the face which can be employed for face recognition.

Principally the outlines of the face are taken into account.

A study was conducted by Chen in which they compared the visible image methods and infrared methods [START_REF] Chen | IR and visible light face recognition[END_REF]. In the experiments, it was shown that the visible image methods were outperformed by the infrared technique when they were done in an environment where the illumination condition was not constrained.

Selinger and Socolinsky affirmed that the combination of the two (infrared and visible imagery techniques) improved the performance when the experiments were taken outdoor whereas the thermal image based techniques have a few drawbacks such as the temperature of the skin [START_REF] Socolinsky | Face recognition with visible and thermal infrared imagery[END_REF][START_REF] Socolinsky | A comparative analysis of face recognition performance with visible and thermal infrared imagery[END_REF]. Although, it has been remarked that unlike thermal imagery, the hyperspectral information of the face is least influenced by the temperature than the thermal radiance. Before it, the spectroscopy has also been studied broadly in the remote sensing applications and biomedicine, assessing that various people show a high variation on the hyperspectral features of the facial texture, but these characteristics do not vary for the same person under different brightness condition and over the time.

Problematic

Illumination A face is a 3D object, thanks to which various light source on the face can cause different obscurations and various brightness [START_REF] Chellappa | Robust Face Recognition Using Symmetric Shapefrom-Shading[END_REF]. The variation in face images of different individuals can be less notable than the variation of the image of the same individual in different illumination conditions. There have been multiple studies to develop facial features which are invariant against lighting variations [START_REF] Chen | Total variation models for variable lighting face recognition[END_REF].

Another problem associated to the absence or the lack of illumination which makes visible imagery useless. In this dark environment, even face detection cannot be accomplished. The presence of another camera type such as thermal sensor is necessary in this case.

Face spoofing attack Spoofing attack is trying to get a false acceptance of the authentication system using fake evidences. In the case of face spoofing, the attacker can use a photo, a video or a 3D mask of legitimate user as fake proof. In this digital era, a photo of normal person can be easily found in social network which makes the whole system become vulnerable.

Twin Faces Due to security purposes, the issue of twin faces was introduced. Even the human eyes get a lot of difficulty in recognizing the twins. There have been multiple studies performed on twin faces, but those are under calibrated environmental conditions i [START_REF] Klare | Analysis of facial features in identical twins[END_REF]. To recognize identical twins these methods either uses the entire face or different facial elements such as eyes, nose, and mouth.

Disguise Disguise is the most significant security menace of a system, or we can assume that disguise is still a significant problem for face-recognition methods to recognize a person when he or she seeks to cover its own identity to imitate someone else. There are few proposed solutions by researchers in which the difficulty of disguise can be resolved [START_REF] Singh | Face recognition with disguise and single gallery images[END_REF].

However, this problem is not necessarily considered in an authentication system which can demand the person to take off his or her disguise when it detects one. It is the make-up, a type of unintentional disguise, which is the real problem for the identification process.

Introduction

Face spoofing is an active attack against the authentication system by face recognition [START_REF] Kollreider | Evaluating liveness by face images and the structure tensor[END_REF][START_REF] Galbally | Biometric Antispoofing Methods: A Survey in Face Recognition[END_REF][START_REF] Bagga | Spoofing detection in face recognition: A review[END_REF][START_REF] Patel | Secure Face Unlock: Spoof Detection on Smartphones[END_REF][START_REF] Galbally | Three-dimensional and two-and-a-half-dimensional face recognition spoofing using three-dimensional printed models[END_REF][START_REF] Pinto | Using Visual Rhythms for Detecting Video-Based Facial Spoof Attacks[END_REF]. The notion active emphasizes the real intention of the attacker instead of a normal user's mistake like the case of makeup or natural problems such as illumination conditions of facial expressions. In this case, the attacker is supposed to have sufficient knowledge of the system mechanism.

Attackers have many ways to attack a facial recognition system [START_REF] Kumar | A comparative study on face spoofing attacks[END_REF][START_REF] Chingovska | On the Use of Client Identity Information for Face Antispoofing[END_REF][START_REF] Gragnaniello | An Investigation of Local Descriptors for Biometric Spoofing Detection[END_REF]. They can utilize a photo of legitimate user printed on a piece of paper or displayed on an LCD screen and present it in front of the camera in operation. They can also replay a video which filmed the victim previously or evenly use a 3D mask to mislead the face detection process. Face spoofing can be classified into two types of attack by their false proof of authentication: 2D attack and 3D attack. The 2D attack can be further divided into photo attack of video attack (replay attack). Since the attackers are supposed to know the authentication method, they can choose and customize their attack for each system.

Photo attack

In the early age, face spoofing could be accomplished using a single printed photo of a legitimate user. The attacker shows this photo to the biometric sensor (in our case, the biometric modality is usually a camera) which considers this one as a proof of authentication and grants access to the owner. This type of simple process bears the name of its material: photo attack. It is the easiest and also the most widespread method especially as users photo can be simply retrieved from the social networks such as Facebook, Twitter, and Instagram [START_REF] Yang | Person-Specific Face Antispoofing With Subject Domain Adaptation[END_REF] .

An advanced type of photo-attack in which high-resolution prints of eyes and mouth are morphed is developed under the name photographic masks. During the attack, the impostor placed himself behind, so that certain facial movements like eyes blinking or random face expression is reproduced [START_REF] Anjos | Motion-based counter-measures to photo attacks in face recognition[END_REF]. Photographic masks become a crucial threat to a lot of face liveliness detection methods which uses these facial movements as the ultimate information for their classification.

Video attack

Video attack is another advanced version of photo attack in which the attacker replays a video of the genuine user in front of the camera [START_REF] Wen | Face Spoof Detection With Image Distortion Analysis[END_REF]. The video can be taken by a smartphone, a tablet, a surveillance camera, with or without the cooperation of the user. The screen that displays that video can also be diverse in terms of size and resolution. The main purpose of replacing a photo with a video is also to reproduce some facial movement in order to deceive the authentication system [START_REF] Poh | An Evaluation of Video-to-Video Face Verification[END_REF][START_REF] Evans | Guest Editorial: Special Issue on Biometric Spoofing and Countermeasures[END_REF].

For the same idea to mimic certain face movement video attack is closer to the original version than photographic masks. However, the video replayed is a fixed sequence which cannot interact with the system. In some case of an active system which demands the user to generate an unexpected expression, the video attack may be totally inefficient. Furthermore, a video which focuses the most on the users' face is more difficult to retrieve than a single photo.

3D mask attack

The 3D mask attack is the most advanced attack due to the depth elements in the facial features [START_REF] Sun | Eyeblink-based Anti-Spoofing in Face Recognition from a Generic Webcamera[END_REF]. In 3D mask attack, attackers have to focus on their target and do firstly manage to construct a 3D mask or maybe a sculpture of the target. The 3D masks are usually made of different materials and sizes, i.e., paper, plastics and silicon. If the mask is constructed perfectly, there is less chance to detect it. However, the achievement of this type of attack is quite difficult and expensive.

Face Spoofing Detection

The issued research studies through the former 20 years have shown that important protection against the spoofing of biometric proof has taken place making biometric method safer and more robust thanks to the intense efforts of researchers [START_REF] Hadid | Biometrics Systems Under Spoofing Attack: An evaluation methodology and lessons learned[END_REF][START_REF] Ratha | Enhancing security and privacy in biometrics-based authentication systems[END_REF].

Many approaches have been proposed in the literature to deal with face spoofing attacks using different features like texture, liveliness, structure, etc. These types of method of detection can be introduced as follows:

Textural information

Textural information, which manages to be different between real-face images and fake ones, can be exploited for face spoofing detection. From a single image, Matta et al. [START_REF] Maatta | Face spoofing detection from single images using micro-texture analysis[END_REF] propose to analyze the texture of facial images using multi-scale Local Binary Pattern (LBP). In the same spirit, Kim et al. [START_REF] Kim | Face liveness detection based on texture and frequency analyses[END_REF] , also utilized LBP, but in fusion with frequency analyses by using the power spectrum. Other researchers exploited the Local Graph Structure (LGS) [START_REF] Bashier | Face Spoofing Detection Using Local Graph Structure[END_REF] or its improved versions (ILGS, SLGS) as texture descriptors to conceptualize their face spoofing detection method. Another method proposes to exploit the statistic behavior of the distribution of noises local variances to detect face spoofing attacks [START_REF] Nguyen | Face spoofing attack detection based on the behavior of noises[END_REF].

Zhang et al. [START_REF] Zhang | Face liveness detection by learning multispectral reflectance distributions[END_REF] proposed a texture based technique in which Lambertian model was employed to recognize the human skin and SVM (Support Vector Machine) was applied to classify the real person and imposter. This liveness detection method has some limit dueing with real-time spoofing attempt in unconstrained conditions.

The low-resolution webcam ( 320x240 pixel frames at 25 fps) used in this experiment gave an accuracy of 7% error rate.

Zinelabidine Boulkenafet et al. [START_REF] Boulkenafet | Face Spoofing Detection Using Colour Texture Analysis[END_REF] described a face spoofing detection technique using color texture analysis. In this algorithm, the researcher concentrated on luminance data of the face photo and the chrominance data was rejected so that this method could recognize the legitimate users. This novel approach is tested on CASIA-FASD database, Replay Attack database, MSU mobile face spoof database.

The results showed by experimentation is n accuracy of 0.4% error rate. Shervin Rahimzadeh et al. [START_REF] Arashloo | Face Spoofing Detection Based on Multiple Descriptor Fusion Using Multiscale Dynamic Binarized Statistical Image Features[END_REF] introduced various descriptor mixing method.

This kernel mixing method was constructed based on a fast kernel discriminant analysis (KDA). The test was executed on another publically accessible database. In this method, multi-scale, dynamics binarized statistical image patents were utilized. However, the technique demonstrated less effective performance on photo databases and 3D databases.

Liveliness Detection

Some approaches manage to distinguish real faces from spoofed faces by seeking proofs of liveness from a sequence of images or from a video capturing the face.

Kollreider et al. [START_REF] Kollreider | Non-intrusive liveness detection by face images[END_REF] proposed an approach in which lip movements are exploited for face spoofing detection while the user is asked to speak some numerical digits.

Huyng-Keun Jee et al. [START_REF] Hyung-Keun | Liveness Detection for Embedded Face Recognition System[END_REF] , in their approach, proposed to study uncontrollable movements of eyes regions, such as the eye blinking or pupil movement. Lin Sun et al. [START_REF] Sun | Eyeblink-based Anti-Spoofing in Face Recognition from a Generic Webcamera[END_REF][START_REF] Sun | Blinking-based Live Face Detection Using Conditional Random Fields[END_REF] described a real-time liveness detection method against photo attack in 2006. This method detects involuntary eye blinking. This method demands no additional device besides a webcam. Adaboost classifier and HMM methods are applied for eye blinking detection which yields high accuracy results with 3% error rate. In these studies, the researchers exploit eyes movements by modeling and detecting the two principal states of the eyes: opened-state and closed-state.

Mihai Gavrilescu et al. [START_REF] Gavrilescu | Study on using individual differences in facial expressions for a face recognition system immune to spoofing attacks[END_REF] proposed soft biometric methods using the neural network and principal component analysis. In this video-based face recognition scheme, the researchers employed several facial expressions on people in various frames. The results showed an error rate of 5.5%.

Structure and motion study

Some other methods exploit the differences between 2D objects and a 3D face in their structure, their moving features or the depth information that they provide. For instance, Kim et al. [START_REF] Kim | Face liveness detection using variable focusing[END_REF] proposed to compare images captured in different focusing.

For a 3D object, due to depth information, the difference between images of different focusing will be clearer than the one in the case of 2D objects. The approach permits to identify efficiently spoofing attacks using a 2D display support. Studying the difference in the behavior of optical flow generated by 2D spoofed face and real face have been also envisaged.

K.Kollreider et al. [START_REF] Kollreider | Real-Time Face Detection and Motion Analysis With Application in "Liveness" Assessment[END_REF] Infrared technology is also known as a robust solution against face spoofing attack. In the next section, we focus on thermal face recognition methods in recent years and assert its capacity to compete with visible imagery in some particular cases.

Thermal Face Recognition

For the last few years, Infrared imagery has attracted particular attention, principally thanks to its robustness against the changes in illumination by visible light and its capacity of liveliness detection. This section is divided into three parts. The first subsection analyzes the thermal spectrum with its features, its advantages, and its limits. The second part gives a brief overview of thermal face recognition solutions. A very potential solution using the vascular network is described in the last subsection.

Introduction Thermal Spectrum

Infrared radiation (IR) is a type of electromagnetic radiation (EMR) of which wavelengths are longer than wavelengths of visible light. Infrared radiation is ordinarily invisible to the human eye and hardly distinguishable by human perception. A detailed report of its physical characteristics, which is outside the border of this thesis, can be found in [START_REF] Maldague | Theory and practice of infrared technology for nondestructive testing[END_REF]. Infrared imagery can be used in many applications as follows: For instance, thermal data of the faces can be acquired under any illumination condition, even in the absence of any lighting source or absolute dark environments, and there is some study which proves that infrared face image may exhibit a higher degree of robustness to facial expression change [START_REF] Friedrich | Seeing People in the Dark: Face Recognition in Infrared Images[END_REF].

Infra-red radiation is also less affected by scattering and absorption by dust or smoke than reflected visible light [START_REF] Chang | Improving Face Recognition via Narrowband Spectral Range Selection Using Jeffrey Divergence[END_REF][START_REF] Nicolo | A Method for Robust Multispectral Face Recognition[END_REF]. Another advantage of infrared imagery is based on its capacity to detect any disguise in the face of which the material may not radiate the same way as human skin. In contrast to visible spectrum imagery, infrared imagery can be used to extract not only exterior features but also useful subcutaneous anatomical information, such as the blood perfusion or the vessels This classification of the IR spectrum is also observed in the manufacturing of infrared cameras, which are usually made with sensors that correspond to energy radiation constrained to a specific sub-band.

Chapter 2. Literature Review

It should be highlighted that this classification of the IR spectrum is not arbitrary.

Instead, different sub-bands correspond to continuous frequency chunks of the solar spectrum which are divided by absorption lines of different atmospheric gasses [START_REF] Maldague | Theory and practice of infrared technology for nondestructive testing[END_REF].

In the scope of face recognition, one of the most significant differences between the 4 IR sub-bands emerges as a result of the human skin's heat emission spectrum which is, in ideal condition, shown in Figure 2.15. Usually, the NIR and SWIR bands can be referred to as "the reflected infrared radiation." The human's body does not emit a significant amount of electromagnetic energy in these sub-bands. In fact, the NIR and SWIR bands are dominated by the reflected radiation. Just like the visible imagery, these two sub-bands require an additional energy source ( which can be in the form of light or heat.

The MWIR and LWIR bands are often labeled as "thermal infrared radiation".

Unlike NIR and SWIR bands, they do not require any additional source of infrared energy since the human's body emits strong electromagnetic radiation in these spectra. Since IR sensors in these bands depend mostly on the amount of emitted energy of a recorded object, they are, in the contrast of visible light camera, invariant to the change of lighting conditions, robust against a lot of problems like weather conditions and can operate in complete darkness.

Especially, since most of the heat radiation is emitted in LWIR sub-band, MWIR sub-band plays a lower role in the thermal spectrum. However, heat energy emitted in MWIR sub-band is usually strong enough to overcome any other type of radiation.

Both of these sub-bands can be used in a passive thermal system. That is one of the reasons why thermal sub-band like LWIR or MWIR have received the most attention in the context of face recognition. Unlike them, body heat emission in the SWIR and In general, infrared sensor does not distinguish various wavelengths inside a sub-band. Color sensors require a highly complex structure to classify wavelength which is very difficult to be applied into infrared modality. The infrared spectrum is much larger than the visible spectrum and cannot map uniformly into the human color system. Outside the visible spectrum, color does not have a natural interpretation.

Infrared images are typically displayed in the form of grayscale images which describe the intensity of infrared energy captured by each pixel. Lighter area represents The use of infrared imagery for automatic face recognition has its own problems and challenges. For example, thermal images are sensitive to the environmental heating condition, as well as the emotional, physical and health condition of the person. LWIR images are even affected by alcohol intake. Another problem source is the eyeglasses which are totally opaque to many of the IR spectrum (LWIR, MWIR and SWIR) [START_REF] Jaeger | Duane's Ophthalmology[END_REF][START_REF] Arandjelovic | Thermal and Reflectance Based Personal Identification Methodology under Variable Illumination[END_REF]. This means that a large area of the face wearing eyeglasses may be occluded, causing the loss of important discriminative features around the eyes. Unsurprisingly, each of the problem has begun a new research direction. Some researchers have proposed fusing the information from IR and visible modalities as a hybrid solution to the problem of eyeglasses opaqueness. Others have suggested methods which use infrared images to extract a map of invariant features such as facial blood perfusion data[ [START_REF] Seal | Minutiae based thermal face recognition using blood perfusion data[END_REF] or vascular network [START_REF] Ghiass | Illuminationinvariant face recognition from a single image across extreme pose using a dual dimension AAM ensemble in the thermal infrared spectrum[END_REF] in order to overtake the temperature dependency of thermal "appearance".

Thermal Sensors

Cooled infrared detectors: The conventional cooled infrared sensors detect and convert electromagnetic energy in the same way as standard visible-light camera (indeed, they are made of different materials). Without any cooling system, the infrared radiation of the object is mixed with the energy emitted by the sensors themselves.

The cooling system can keep the sensor's temperature at a prefixed level so that the energy captured by sensors does not vary by detector status. The disadvantage of cooled infrared cameras is their expensive cost to produce and to operate. Cooling is not only energy consuming but also take a lot of time.

These cameras may need several minutes to cool down before they can begin to run. Although the cooling apparatus is comparatively bulky and expensive, cooled infrared sensors provide images in higher quality compared to uncooled detector thanks to their superior capture rate. Uncooled infrared sensors do not require any bulky, expensive, energy consuming cryogenic coolers. They also need to be stabilized to a fixed operating temperature to reduce image noise, but this temperature is not as low as the one required by a cooled detector. These advantages make infrared cameras smaller and cheaper than cooling technology so that an uncooled camera can be added into any individual machine such as smartphone or drone.

However, in uncooled detectors, the temperature variance at the sensor pixels are miniature; a 1 o K variation at the object produces merely a 0.03 o K difference at the detector. The pixel response rate is also much lower than a cooling system, at the level of tens of milliseconds. This makes their resolution and image quality more moderate than a cooled camera. This is because of differences in their fabrication material which is limited by currently available technology.

Uncooled sensors are principally based on pyroelectric and ferroelectric materials or microbolometer technology. These materials are used to form pixels with a high level of temperature-dependency in some electrical properties.

Advantages

A significant part of the early work on the capacity of infrared images as identity proof was studied by Prokoski et al. [START_REF] Prokoski | Identification of individuals by means of facial thermography[END_REF]. They were the first to propose the idea that thermal "appearance" of a face could be used to retrieve distinct biometric features which contain a high level of repeatability and uniqueness.

The invariance to complex Ilumination condition

The invariance to complex Ilumination condition is the most essential advantage of using thermal face recognition compared to using standard visible imagery. The lightning has very little affectation upon the infrared signature. The longer the wavelength, the weaker the variance.

Under LWIR sub-band, thermal radiation does not depend on any illumination factor. This is also the main purpose of using thermal imagery in face recognition [START_REF] Wolff | Quantitative measurement of illumination invariance for face recognition using thermal infrared imagery[END_REF]. Facial expression and pose positions are two challenges that a face-recognition system must overcome in order to be useful in most real case application. Using the image space differences between infrared and visible images, Friedrich et al. [START_REF] Friedrich | Seeing People in the Dark: Face Recognition in Infrared Images[END_REF] shown that infrared images are more robust against changes in facial expression or head pose than their standard visible imagery.

Face spoofing and disguise are also crucial risks that attack the authentication system by face recognition. The property of thermal imagery also opens the possibility of non-invasive extraction of superficial anatomical features for recognition such as blood perfusion and vessel patterns. Naturally, the blood vessel which transport circulating blood continuously , are warmer than the surrounding area. This property can be captured by the thermal camera and be extracted by processing technique in order to isolate the blood vessel from the face image. An essential characteristic of these patterns which makes them particularly interested in face identification is that the blood vessels are defined by young day and form a realative network which remains very little affected by ageing factors such as ageing. Furthermore, it seems that the human vessel feature can also respond for another key challenge: the scalability in large populations. Prokoski et al. assume that about 175 blood vessel based minor features can be retrieved from a complete facial image [START_REF] Prokoski | Infrared Identification of Faces and Body Parts[END_REF] which, they considered, can represent a far greater amount of possible setting than the number of the maximum human population. However, the authors did not propose a particular method to obtain the minutiae in question.

In the very same work, Prokoski et al. also indicated that spoofing attempts and disguises can both be detected naturally by infrared imagery. The critical proof is that the temperature signature of artificial hair or other facial mask differs from the heat distribution of natural skin and hair, allowing them to be distinguable one from another. This fact also provides thermal imagery immunity against the face spoofing attack. Making a mask which can match the vessel pattern of someone else is almost impossible for now.

Monozygotic twins:

An interesting issue first proposed by Prokoski et al. [START_REF] Prokoski | Infrared Identification of Faces and Body Parts[END_REF] involves the thermal signature of monozygotic twins. The image of monozygotic twins is almost indistinguishable in the visible spectrum. Using a little number of infrared image of monozygotic twins which were evaluated for similarity, Prokoski et al. observed that the variation in appearance was significantly higher in the infrared imagery than in the standard visible representation, and provides sufficient proof to automatically distingue these twins.

Limits

In the scope of automatic face recognition and identification, the main problematic specific to the LWIR sub-band images, the only sub-band of the infrared spectrum can provide absolute invariance to illumination, arises from the fact that the heat model emitted by the object is affected by a lot of mixing variables, such as environmental temperature, atmosphere flow conditions, postprandial metabolism, exercise, sickness, alcohol and drugs [START_REF] Wolf | Learning over Sets using Kernel Principal Angles[END_REF]. Some of these variables create local, other global infrared appearance changes. Wearing make-up, enduring stress, blushing, having an infected tooth or even a headache are examples of issues which can affect the thermal appearance. In the last decade, many studies in thermal face recognition have been realized using the same methodology as visible imagery. However, the result of these methods is far lower than the original one. A few other methods which devote only to thermal imaging are also developed by exploiting some unique feature of infrared images. A lot of methods use more than one step of feature extraction before the classification that makes categorizing thermal face recognition methods more complex.

Methods

In this study, we use the categorization of Ghissa el al in [START_REF] Ghiass | Infrared face recognition: A literature review[END_REF][START_REF] Ghiass | Infrared face recognition: A comprehensive review of methodologies and databases[END_REF], a comprehensive survey on infrared face recognition methods. These methods are regrouped according to their core feature extraction descriptor in three categories: Appearance-based method, Feature-based method, and Hybrid method.

Appearance-based approach

Inside the Appearance-based category, there are two types of approach: global appearance approach and local appearance method.

Global appearance-based approaches are the first developed methods for thermal face recognition. These approaches reuse essentially early technologies of visible face recognition like PCA (Principal Component Analysis) or LDA (Linear Discriminant Analysis) to project the thermal image into a high-dimensional vector space.

Most of the early methods follow the work of Prokoski et al. and Socolinsky et al.

[75] The approach was enhanced by Hermosilla et al. [START_REF] Hermosilla | An enhanced representation of thermal faces for improving local appearance-based face recognition[END_REF] and by Desa et al. [START_REF] Desa | IR and visible face recognition using fusion of kernel based features[END_REF] who applied KPCA (Kernel PCA) and KLDA (Kernel LDA) methods to thermal image respectively. One drawback of these approaches is that they require a huge number of samplings to maintain the accuracy of the covariance matrix.

Cutler described in his study an application of eigenfaces in infrared face recognition using a database of 24 persons at three viewpoints (frontal, left and right profiles) and two facial expressions (thus 288 images in total) [START_REF] Cutler | Face Recognition Using Infrared Images and Eigenfaces[END_REF]. This database is taken under SWIR and MWIR sub-bands. Moulay et al. [START_REF] Akhloufi | Infrared face recognition using texture descriptors[END_REF] also proposed a face recognition framework using probabilistic Bayesian and SVM on Equinox and Laval University multispectral face databases. However, in the same study, the author reported that the best result is obtained by LDA. This study has the most extensive use case among all the thermal face recognition approach. The authentication system can match a thermal signature an object to its visible image database which is usually already available. Features), WLD(Weber Linear Descriptor), GJD (Gabor Jet Descriptor) to describe the information of thermal images [START_REF] Hermosilla | An enhanced representation of thermal faces for improving local appearance-based face recognition[END_REF].

Li et al. [START_REF] Li | Illumination Invariant Face Recognition Using Near-Infrared Images[END_REF] proposed an infrared face recognition method based on LBP under NIR sub-bands. In this study, the author trait NIR image as a pseudo-visible image and reuse standard face recognition technology to operate the classification. In order to deal with the illumination issue, the author proposed a schema of NIR image detector device which reduces the influence of lighting condition on face images.

In another study, Mendez et al. [START_REF] Méndez | Face Recognition with LWIR Imagery Using Local Binary Patterns[END_REF] also used LBP representation for LWIR images. They also indicated that LBP is robust against fixed-pattern noise so that not only no noise suppressing process is necessary, but also non-uniformity correction is not needed. Xie et al. [START_REF] Xie | Joint Encoding of Multi-scale LBP for Infrared Face Recognition[END_REF] enhanced the method of applied joint encoding of multiscale LBP. This approach considers the correlation in divers microstructures using a co-occurrence matrix of multiscale LBP. This method can achieve 91.2% in accuracy under standard heating conditions which outperform classic LBP-based methods.

Other studies introduced by Wang [START_REF] Wang | An enhanced thermal face recognition method based on multiscale complex fusion for Gabor coefficients[END_REF] and by Majumder [START_REF] Majumder | Gabor-Fast ICA Feature Extraction for Thermal Face Recognition Using Linear Kernel Support Vector Machine[END_REF] used Gabor transformation as a feature generator. However, these methods are limited by the fact that thermal imaged have fewer details than visible ones and therefore it's hard to recognize the face.

Feature-Based Method

Feature-Based Method studies some unique literal features of thermal images which do not appear in visible images. The blood perfusion model developed is a very example of the Feature-based approach category. Seal et al. stated that thermal face recognition could exploit local temperature changes in the face image [START_REF] Seal | Automated Thermal Face recognition based on Minutiae Extraction[END_REF]. The heat imbalance region represents anatomical information because of the heating effect caused by the blood flow under the skin. The author indicated that this imbalance could be observed as texture features and could be extracted by Haar wavelet transform.

In another recent study [START_REF] Xie | Blood perfusion construction for infrared face recognition based on bio-heat transfer[END_REF], Xie et al. indicated that veins structure induces a unique thermal signature of the face which is similar to a fingerprint. A feed-forward back propagation neural network with five layers was used in classification phases to obtain 95.24% in terms of accuracy. The author also highlights that segmentation preprocessing such as DAD (directional anisotropic diffusion) or region growing is required to extract blood perfusion features.

The blood perfusion model is appreciated for its simplicity in implementation and its robustness against various changes such as aging or illness. This thermal signature is not only independent of face geometric but also impossible to be spoofed.

The main drawback of this model is that the resolution required of the image has a minimum, under this limit, the process cannot operate normally. The infrared camera has to be closed to user's face that makes this technique more appropriate for authentication than passive identification. The vascular network studied by Buddharaju [START_REF] Buddharaju | Physiology-Based Face Recognition in the Thermal Infrared Spectrum[END_REF] and later by Ghiass et al. [START_REF] Ghiass | Illuminationinvariant face recognition from a single image across extreme pose using a dual dimension AAM ensemble in the thermal infrared spectrum[END_REF] extracts blood vessels from an image using morphological filters. These approaches are proven to be effective and robust as a recognition method but they suffer from a high sensibility to normalization process.

Hybrid Method

Despite a lot of advantages, the accuracy rate of infra-red face recognition is far lower than the performance of standard visible imagery. In order to enhance the reliability of biometrics systems, research community tends to combine thermal imaging with other technologies and form hybrid solutions. In this category, many thermal/visible fusion methods, multi-spectral methods, multimodal methods are mentioned as follows.

In their work [START_REF] Bourlai | Multi-spectral face recognition: Identification of people in difficult environments[END_REF]

Vascular Network Introduction

The vascular network is the product of anatomical observations in thermal imagery.

The key idea of this feature is the higher temperature of the blood vessel in comparing with neighbor region. The method is proposed by [START_REF] Buddharaju | Physiology-Based Face Recognition in the Thermal Infrared Spectrum[END_REF] and is enhanced by Reza Shoja Ghiass [START_REF] Ghiass | Illuminationinvariant face recognition from a single image across extreme pose using a dual dimension AAM ensemble in the thermal infrared spectrum[END_REF] . The so-called vascular network is a map of tubular structures extracted from a thermal image. This type of feature is proven to be a effective transformation in thermal face representation.

Method Details

For each frame F i (i = 1, ..., n) consider the two eigenvalues λ 1 and λ 2 of the Hessian matrix computed at a certain image locus and at a particular scale s. Without loss of generality let us also assume that

|λ 1 | |λ 2 |
The two key values used to quantify how tubular the local structure at this scale is are R A and S:

R A = |λ 1 | |λ 2 | , S = λ 2 1 + λ 2 2 (2.1)
The former of these measures the degree of local "blobiness".If the local appearance is blob-like, the Hessian is approximately isotropic and |λ 1 | ≈ |λ 2 | making R A close to 1. For a tubular structure R A should be small. On the other hand, S ensures that there is sufficient local information content at all: in nearly uniform regions, both eigenvalues of the corresponding Hessian will have small values. For a particular scale of image analysis s, the two measures, R A and S, are then unified into a single vesselness measure: where β and c are the parameters that control the sensitivity of the filter to R A and S.

V(s) =    0 i f λ 2 > 0, (1 -e - R A 2β 2 ) × (1 -e -S 2c
In fact, the "vessel value" of a pixel is represented by the measure V(s) of the (6s + 1) × (6s + 1) block centered at this pixel. Finally, if an image is analyzed across scales from s min to s max , the vesselness of particular image locus can be computed as the maximal vesselness across the range:

V 0 = max s min ≤s≤s max V(s) (2.3)
In the end, each vertex is associated with a value V 0 which presents the vessel probability at this point. Another column V 0 can be added to matrix M:

M =       x 1 x 2 ... x N y 1 y 2 ... y N z 1 z 2 ... z N V 0 (1) V 0 (2) ... V 0 (N)       (2.4)
For each intensity image, the poses, positions and contributions to the 3D model is computed under a texture map. By using this texture map, these vascular networks can be projected to the 3D model in order to form a 3D vessel model which represents the 3D coordinates of vessel features (Fig. 5.5).

Advantage

In our studie, Vascular Network is highly appreciated as a feature extraction approach thanks to its advantages compared to other methods.

The vessel features are robust against the change of image scale. The size of the user's face inside an input image or video cannot be predicted. Almost all the global and local appearance-based methods are highly dependent on image's resolution. A lot of features disappear when there are not enough pixels describing them.

The vessel features are also less affected by ageing or illness. There is not any significant change in a user's vascular network except the case of plastic surgery.

But even in such extreme case, it is practically impossible to mimic the vessel map of someone else.

The Vascular Network is lately mentioned in the fifth chapter of this thesis as the primary feature extraction of the thermal image. The next chapter looks at the 3D model of user's face and its reconstruction by various techniques. A methodology using minimal equipment to obtain 3D data will be described at the end of that chapter. This little restraint allows a broad range of applications for this methodology.

Chapter 3

3D Reconstruction

Introduction

3D reconstruction is the domain appearing to respond for the need of capturing and recognizing the 3D geometric form of a subject. 3D reconstruction has numerous applications in various areas such as:

• Computer vision : object description for augmented reality, motion capture for body tracking, robotics mapping ...

• Medicine : organ scanning and modeling.

• Entertainment: filming and gaming.

• Archaeology: visualizing constructions and objects.

• Security: face recognition, fingerprint recognition, human tracking, video surveillance.

In this chapter, we will examine the capacity of representing users' face data by its 3D model. The following section synthesizes the state of the art of 3D reconstruction method using various sensor and technology. The last section introduces our scheme to obtain this model from a single video of user's face.

Existing Methods

3D reconstruction is the process of capturing the shape and appearance of real objects. This process can be accomplished either by active or passive methods. In passive methods, the number of cameras and images used in the process divides this type into: Monocular Cues Methods, Binocular Stereo Vision and Structure From Motion.

Active Method

Active methods, i.e. range data methods, using the depth map, rebuild the 3D surface by digital approximation method and reconstruct the object in scenario based on the model . These techniques actively interfere with the rebuild object, either mechanically or radiometrically using rangefinders, to obtain the depth map, e.g. The laser range finder only estimates the distance of one object in its direction of view. Thus, the scanner examines point by point its entire field of view by turning the range finder's direction of view to examine various points. The view orientation of the laser range finder can be modified either by pivoting the range finder itself or by employing a set of turning mirrors. The latter method is usually applied since mirrors are much lighter and can thus be rotated much quicker and with higher precision. Standard time-of-flight 3D laser scanners are able to estimate the distance of 10,000 100,000 points per second.

Triangulation based 3D laser scanners are also active scanners that employ laser radiation to examine the environment. Concerning time-of-flight 3D laser scanner, the triangulation laser irradiates a laser on the object and utilizes a camera to localize the laser dot. Depending on the distance between the object's surface and the camera, the laser mark rises at various places in the camera's range of view. This method is named triangulation because of the triangle created by the laser emitter, the laser dot, and the camera. The length of one side of the triangle, the one between the laser emitter and the camera can be determined. The direction of the laser emitter corner is also defined. The angle of the camera corner can be defined by examining the position of the laser dot in the camera's range of view. These three sets of information entirely limit the contour and dimension of the triangle and provide the place of the laser dot angle of the triangle. In most circumstances, a laser line, rather than a single laser dot, is swept over the object to accelerate the scan process.

Time-of-flight and triangulation range finders each possess advantages and disadvantages which make them favorable for different circumstances. The strength of At a speed of 10,000 points per second, low-resolution scans can use less than one second, but high-resolution scans, demanding millions of samples, can use several minutes for some TOF Machine. The difficulty that produces is distortion from the movement. Since each point is examined at a different time, any movement in the object or the machine will distort the obtained information. Thus, it is regularly required to fix both the object and the device on stable stands and reduce vibration.

Utilizing these machines to scan objects in movement is pretty tough. Hand-held laser scanners : Hand-held laser scanners generate a 3D model using the triangulation technique introduced before: a laser dot or line is projected onto an object from a portable machine, and a sensor (typically a charge-coupled sensor or position sensitive sensor) estimates the gap to the surface [START_REF] Goel | A Motion Correction Technique for Laser Scanning of Moving Objects[END_REF][START_REF] Strobl | The self-referenced DLR 3d-modeler[END_REF][START_REF] Strobl | Image-based pose estimation for 3-D modeling in rapid, hand-held motion[END_REF]. Information is retrieved using an inner coordinate system and therefore to obtain information when the device is in motion the position of the device must be defined. The As with all optical methods, reflective or transparent surfaces raise difficulties.

Reflections cause light to be reflected either away from the camera or right into its optics. In both cases, the dynamic range of the camera can be exceeded. Transparent or semi-transparent surfaces also cause major difficulties. In these cases, coating the surfaces with a thin opaque lacquer just for measuring purposes is a common practice. A recent method handles highly reflective and specular objects by inserting a 1-dimensional diffuser between the light source (e.g., projector) and the object to be scanned. Alternative optical techniques have been proposed for handling perfectly transparent and specular objects.

Monocular Cues Methods

The monocular cues systems indicate to use images (one, two, three or more) from one viewpoint (camera) to proceed 3D reconstruction. It makes use of 2D characteristics(e.g. Silhouettes, shading, and texture) to measure 3D form, and that is the reason, for which it is also entitled Shape-From-X, where X can be silhouettes, motion, contour, shading, texture, etc. 3D reconstruction by monocular cues is quick and straightforward, and only one suitable numerical image is required thus only one camera is sufficient. Technically, it eludes stereo correspondence, which is moderately complicated. the researchers concentrate on the computational section of the problem, attempting to calculate directly analytical solutions. Topics about the existence and uniqueness of such solutions were completely not even appeared at that moment with the critical exception of the studies of Bruss [START_REF] Bruss | Shape from Shading[END_REF] and Brooks [START_REF] Brooks | Two Results Concerning Ambiguity in Shape From Shading[END_REF]. Because of the lousy quality of the issues, these problems, as well as those related to the convergence of digital schemes for computing solutions, became principal in the last decade of the 20th century.

Now, the Shape From Shading approach is considered as an ill-posed problem.

For example, many articles prove that the existing solution is not unique. Such concave/convex ambiguities have usually represented the encountered problems like the one presented in Figure 3.4. In this figure, the ambiguity is due to a variation in the calculation of the parameters of the illumination. In fact, this sort of ambiguity can be usually generalized. Belhumeur et al. [START_REF] Belhumeur | The bas-relief ambiguity[END_REF] show that when the lighting orientation and the Lambertian reflectance of the object are undefined, then the same image can be captured by a connected family of surfaces (relying linearly on three parameters). In other words, they prove that neither shading nor shadowing of a subject, observed from a single image shows its correct 3D structure.

Photometric stereo is a method in computer vision for evaluating the surface normals of a subject by observing that subject under varying illumination conditions.

It is built on the basis that the quantity of light reflected from a surface is reliant on the direction of the surface concerning the light source and the observer system. By estimating the quantity of light bounced into a camera, the set of possible surface directions is restrained. Given sufficient light sources from various angles, the surface direction may be limited to a single direction.

Woodham originally proposed the method in 1980. Photometric stereo has since been generalized to multiple other circumstances, including extended radiation sources and non-Lambertian subject. Latter research tries to get the technique work in the appearance of projected shadows, highlights, complex and non-uniform lighting.

The first stage in the evaluation of the normal map is to calibrate the light source by estimating the light orientation. One method to do this is to utilize a chrome ball on which the brightest spot is employed to recognize the orientation of the light.

Shape-from-texture Regarding a pattern with some sort of regularity, or texture, converging on a receding surface, humans can readily recognize the 3D depth of the scene. This fact has long since intrigued many researchers, and studies have been made to reproduce, by a computer, this apparently highly intelligent human capacity. This topic is now generally remembered as 3D recovery of shape from texture. Typically, 3D reconstruction from texture is potential if we have some prior information about the right texture; if the observed surface has characteristics separate from those of the true texture, the 3D shape is calculated in such a way that the inconsistency is accounted for. For example, if the true texture is given to be an arrangement of components with a given shape, say circular, the patent gradient can be deduced from the perceived distorted shape, say elliptical, of the components. If the true texture components are given to be periodically distributed at periods of the same interval, the patent gradient can be evaluated from the rate of the converging period lengths. If the true texture components are aligned on parallel lines, or if individual texture components have parallel line sections, the surface gradient is induced from the fading points determined by pairs of such lines. Similar logic is reasonable if the true texture elements or their alignments are given to hold orthogonality or symmetry of some sort. One significant problem about these methods is that we must first identify the structure of the real texture regularity, periodicity, collinearity, parallelism, orthogonality or symmetry. This is in overall very challenging to automate by a computer as the perceived texture does not show the expected regularity,

Binocular Stereo Vision

Binocular Stereo Vision receives the 3-dimensional geometric data of a subject from multiple images based on the study of the human visual system. The results are displayed in the form of depth images. Images of a subject obtained by two cameras concurrently in separate viewing angles, or by one single camera at separate times in various viewing angles, are used to reconstruct its 3D geometric data and restore its 3D form and position. This is more direct than Monocular techniques like shapefrom-shading. Binocular stereo vision system needs two similar cameras with parallel optical axis to perceive one same subject, obtaining two photos from different positions of view. In words of trigonometry connections, depth data can be determined from the variation. Binocular stereo vision technique is well matured and contributes to beneficial 3D reconstruction, heading to a more significant performance when compared to other 3D construction techniques. Regrettably, it is computationally expensive, besides it works rather inadequately when baseline distance is considerable.

2D digital image acquisition is the data source of 3D reconstruction. Usually used 3D reconstruction is based on two or more images, although it may use only one image in some circumstances. There are different types of techniques for image acquisition that depends on the circumstances and objectives of the specific system.

Not only must the specifications of the system be met, but also the visual variation, brilliance, production of camera and the feature of the scene should be considered.

Camera calibration in Binocular Stereo

Vision relates to the measurement of the mapping relationship between the image points P 1 and P 2 , and space coordinate P in the 3D acquisition. Camera calibration is a fundamental and necessary part in 3D restoration using Binocular Stereo Vision.

Feature extraction is the method which intends to obtain the properties of the photos, through which the stereo correspondence performs. As a consequence, the properties of the photos approximately connect to the selection of matching techniques.

There is no such globally appropriate theory of features descriptor, heading to a large diversity of stereo correspondence in Binocular Stereo Vision study.

Stereo correspondence this is to build the correspondence between fundamental elements in photos, such as to match P 1 and P 2 from two photos. Some interference elements in the background should be remarked, like light, noise, surface physical property, etc.

Structure From Motion

Structure from motion (SfM) is an imaging method for determining three-dimensional object from two-dimensional photo arrays that may be joined with local movement signals. Humans recognize a lot of data about the three-dimensional object in their surroundings by moving over it. When the person moves and the objects around the person move, data is received from images seen over time. Obtaining structure from motion represents a similar question to obtaining structure from stereo vision.

In both cases, the correspondence between photos and the reconstruction of the 3D structure must be discovered.

The structure of an image is a projection from a 3D scene onto a 2D plane, during which the depth data is dropped. The 3D point corresponding to a particular image pixel is constrained to be on the line of view. From a single photo, it is improbable to decide which point on the line corresponds to the image pixel. If two images are accessible, then the position of a 3D point can be located at the intersection of the two projection line. This method is related to as triangulation. The core of this method is the connections between multiple views which conduct the data that corresponding sets of points must include some structure and that this structure is linked to the position and the orientation of the camera.

Scheme

In our study, the input thermal video is supposed to contain many frames from various poses of one head. An algorithm of 3D reconstruction is used to compute a 3D point cloud which describes the head filmed in the video. In this scenario, we use VisualSFM -Structure from motion [START_REF] Wu | Towards Linear-Time Incremental Structure from Motion[END_REF], developed by Changchang Wu , a robust and stable reconstruction algorithm which uses not only the shape but also the color of each pixel to compute the coordination of the face. Therefore, instead of using the gray-scale version of intensity, the experiment observes another color representation of thermal image: the Iron-palette which can be computed from the gray-scale one (Fig. 3.9).

These features(common edges and points) are tracked from one frame to next so the position and orientation of each frame can be estimated by geometric calculation.

Different views of one point which can be obtained from many consecutive frames are extracted to estimate its deep coordination and then its 3D position.

From the original video, a set of frame F i (i = 1, ..., n where n is the number of frames) can be extracted. Each frame is an image (p × q pixels) which contains a view of the face:

F i =       x i 1,1 x i 1,2 ... x i 1,q x i 2,1 x i 2,2 ... x i 2,q ... ... ... ... x i p,1 x i p,2 ... x i p,q       (3.1)
In fact, each frame is compared to all other frames by method SIFT (scale-invariant feature transform). Two frames (F c 1 , F c 2 ) which maximize the similarity index are chosen to form the base of 3D object. These common points of F c 1 and F c 2 will form (c

1 , c 2 ) = argmax i =j SIFT(F i , F j ) (3.2)
Then, the process sorts all images in the decreased order of SIFT(F c 1 , F i ) which make a complete sequence c 3 , c 4 , ..., c n . For each frame F c i (i = 3, ...n), the process will firstly try to find the common points with ST i-1 which will be called STF i (STF i ⊆ ST i-1 ). So we have a certain points STF i in 3D coordination and its project in the plan of camera: F c i . Thus the process will try to estimate the orientation and the position of the camera by maximizing the similarity index between the two sets:

(R c i , Pc i ) = argmax R,P SIFT(F c i , T P (PROJ R (STF i )) (3.3)
where T P is the translation operator by vector P and PROJ R is the projection operator by rotation matrix R.

P c i =    x y z    (3.4) R c i =    r 1,1 r 1,2 r 1,3 r 2,1 r 2,2 r 2,3 r 3,1 r 3,2 r 3,3    (3.5)
The geometric information of F c i is represented in position matrix P c i and rotation matrix R c i (which can be transformed to orientation matrix `i)

`i =    θ x θ y θ z    (3.6)
(x,y,z) is the coordination of the center pixel of frame F c i , (θ x , θ y , θ z ) is its orientation respectively in the view of Ox, Oy and Oz.

The process will, then, studies the extra part of frame F c i which is not exist inside STF i and try to match it's with some other nearest frame to find additional common

points STP i ST i = ST i-1 ∪ STP i (3.7)
The last one (ST i ) becomes the 3D model formed by {F c 1 , ..., F c i }.

The set ST n definite the 3D model of all frame. This 3D model form a matrix which represents a cluster of featured points in a Cartesian coordinate system. Notes M ini the 3D model , it can be represented as: where N is the number of feature points and (x k , y k , z k ) (k = 1, ..., N) are the coordinates of the k-th point in the space Oxyz. (r k , b k , g k ) (k = 1, ..., N) represent the pseudo-color (which is computed from intensity) of this point. The obtained point cloud, which is neither dense nor periodic, must be improved using the Patch-based Multi-view Stereo (PMVS) developed by Yasutaka Furukawa [START_REF] Furukawa | Accurate, Dense, and Robust Multiview Stereopsis[END_REF] and using the Poisson Surface Reconstruction studied by Michael Kazhdan [START_REF] Kazhdan | Poisson surface reconstruction[END_REF]. By this supplement process, the set of point is transformed into a dense collection of small oriented rectangular. The algorithm of PMVS can be decomposed in • Filtering: Intensity constraints (and a weak form of regularization) are used to eliminate incorrect matches. These 3 steps are repeated for several iterations until the set of points (the mesh) is dense enough. This mesh can be further refined by a mesh-based MVS algorithm that enforces the photometric consistency with regularization constraints like Poisson Surface Reconstruction . The resolution of the mesh model is adaptive, and the size of a triangle depends on the density of the nearby oriented points: The denser the points are, the finer the triangles become. The PSR software outputs a closed mesh model even when patches are only reconstructed for a part of a scene. In order to remove extraneous portions of the mesh, we discard triangles whose average edge length is greater than six times the average edge length of the whole mesh since triangles are large where there are no points. So, in fact, after the PMVS process, a matrix M (having the same structure as M ini but having much more points, here we always use N as the number of points ) is obtained. This rebuild 3D model includes a huge number of vertex and triangular surfaces which can be considered as a dense facial surface (Fig. 3.11). This scheme of 3D reconstruction which is applied both in visible imagery and thermal imagery is one of the core features of our study. Geometric data from the 3D model provide robust proof to detect face spoofing attack which is described in the next chapter. On the other hand, depth information extracted from this model is exploited in the fifth chapter to improve the performance of our method of face recognition.

M ini =    x 1 x 2 ... x N y 1 y 2 ... y N z 1 z 2 ... z N    (3.8)

Chapter 4

Face Spoofing Detection Using 3D Model

Introduction

Problematic and Objectives

Authentication by facial recognition can be exploited as an additional solution to reinforce the security level of our information systems. However, it is proven that this solution is vulnerable. Facial recognition is easily compromised by face spoofing attacks. Therefore, photos and video widely shared on social networks may become a weapon against their owner's security. Attackers have many ways to attack a facial recognition system. They can utilize a photo of a legitimate user printed on a piece of paper or displayed on an LCD screen and present it in front of the camera in operation. They can also replay a video which filmed the victim previously or just use a 3D mask to mislead the face detection process. In 3D-mask attack, attackers have to focus on their target and do firstly manage to construct a 3D mask or maybe a sculpture of the target. If the mask is constructed perfectly, there is less chance to detect it. However, the achievement of this type of attack is quite difficult and expensive. In this paper, the proposed method seeks to detect basically the photo and video-replay attacks.

Contributions

In this study, we construct and investigate a new face-spoofing detection using a 3D model of the head computed from a video captured by the user's smartphone.

• The novel method is designed for one of the most challenging use cases: face recognition using a smartphone which is highly dependent on the user's behavior. However, the study explores some convenient features of smartphone such as the movement capacity and motion sensor to construct an adaptive method.

• The 3D model is super-effective against photo-attack as different in geometric features between a real object and an image is pretty significant.

• The process also compares the prior-motion of the camera and the capturedmotion estimated from the input video to justify the credibility of the user. This phase can detect a large portion of video-attack.

Method Details

In the last few years, we can remark a constant evolution of mobile technology and of the smartphone market. More and more people use smartphones to ease their daily life as well as their professional activities. Myriad mobile applications require or have access to personal or private information of users. Therefore, they need a high level of security. Authentication by facial recognition is proposed as a solution to reinforce the security of mobile systems. However, the problem of face spoofing is always unavoidable. Actual solutions are quite relevant and optimized to settle this problem, but just in some provided cases study. Thus, an efficient solution dedicated to smartphone system is indispensable.

In the case of a smartphone system, which is mobile, images or videos could be captured under different conditions of lighting, under different orientations and with an uncontrollable background. The quality of acquisition could also be affected by the movement of the camera and the movement relative of the acquisition system (e.g. when a user authenticates while he is traveling in a train). In addition, the diversity and the constant evolution of smartphone models, as well as the difficulty in calibrating their cameras, are also among the big barriers for an efficient face spoofing detection solution.

However, the presence of different sensors integrated into a smartphone may be an advantage which allows us to develop a novel dedicated solution to face spoofing detection. Indeed, with the help of the movement sensors and the multitasking ability of smartphones, we can simultaneously capture the device's movement information while filming the user's face by our Android Application. (Notice that all smartphones in our day include at least the gyroscope sensor.) At the end of this phase, the output will include a video of the head and sensors raw data. In the case of legit authentication, information given by movement sensors is a priori coherent with information estimated from the camera's outputs, but it is generally not the same case when a spoofing attack happens. Therefore, it is a good idea to exploit the coherence between these two sources of information as features for face spoofing detection. Our proposed solution relies mainly on this idea.

The proposed solution consists of three major steps. Figure 4.1 shows the detection flowchart of the solution. Firstly, a 3D model of the face is estimated thanks to a 3D reconstruction process. Then, a Photo Attack Detection (PAD) classifier exploiting the 3D shape is employed to retrieve photo attacks (which use a static image of legitimate user, e.g. photo printed on paper or displayed on an LCD screen). The construction of PAD classifier is described in the section 4.2.2. For the ones which pass through the PAD classifier, they will be after that classified thanks to the Video Apart from the necessary movement of smartphone (Figure 4.2) which requires the collaboration of user, all other processes can be automatic. In this study, the video and sensor data collector, 3D reconstructor and classifier are regrouped inside an unique android application. However, the 3D reconstruction is not real-time yet that slows down the detection. In a real scenario, it is recommended to offshore 3D reconstruction and final classification to a dedicated server.

The next section gives details about our solution to face spoofing detection starting with an overview of the method. Step by step, we will explain how our scheme can distinguish photo and video attacks from valid attempts of authentication. The result will be given in the last section where we discuss more about the performance and the perspectives of the method.

Preprocessing: Facial 3D reconstruction

In the process of proposed method, a three-dimensional model of the object (e.g. real face or fake face) is constructed from a video captured during the authentication process. For a better quality of the 3D model, the user is asked to move the phone's camera around their face in such a way as various head poses can be captured in the video. Two simple camera movements are considered in our proposed approach: in 60 Chapter 4. Face Spoofing Detection Using 3D Model the vertical direction (i.e. upwards or downwards) and in the horizontal direction (i.e. to the left or to the right) (Figure 4.2). These proposed basic movements allow applications to easily communicate with users during authentication. They also permit to simplify the measure of coherence mentioned above.

In our study, 3D reconstruction process is assured by VisualSFM, a 3D reconstruction application developed by C. Wu [START_REF] Wu | Towards Linear-Time Incremental Structure from Motion[END_REF] using Structure From Motion (SFM).

The method requires a sequence of images in input. It gives as outputs the 3D reconstruction of the object captured as well as information related to the camera poses.

Other recent solutions can replace VisualSFM in this step such as a faster 3D reconstruction proposed by Maninchedda et al . The choice of technology here doesn't affect the final outcome severely but highly depends on the computation capacity of smartphone .

In fact, each frame F i in the video (i = 1, ..., n where n is the number of frames) is compared to all other frames by method SIFT (scale-invariant feature transform).

Two frames (F c 1 , F c 2 ) which maximize the similarity index are chosen to form the base of 3D object. After that, an extraction of common edges and interest points (corners) is applied for all pairs of two frames:(F i , F c j ) where i = 1, ..., n and j = 1, 2 (this extraction based on the derivation of intensity and color of pixels). These features(common edges and points) are tracked from one frame to next so the position and orientation of each frame can be estimated by geometric calculation. Different views of one point which can be obtained from many consecutive frames are extracted to estimate its deep coordination and then its 3D position. The next step is the filter of relevant features points which appears in many frames to form the 3D model.

The 3D model is a matrix which represents a cluster of featured points in a Cartesian coordinate system. Notes M the 3D model , it can be represented as:

M =       x 1 y 1 z 1 x 2 y 2 z 2 ... ... ... x N y N z N       (4.1)
With color video a RGB matrix M c can be associated with

M M c =       r 1 b 1 g 1 r 2 b 2 g 2 ... ... ... r N b N g N       (4.2)
where N is the number of feature points and (x k , y k , z k ) (k = 1, ..., N) are the coordinates of the k-th point in the space Oxyz. (r k , b k , g k ) (k = 1, ..., N) represent the color of this point. For each frame F i (i = 1, ..., n), all the appeared features are saved in a matrix M i :

M i =       k 1 px 1 py 1 k 2 px 2 py 2 ... ... ... k N i px N i py N i       (4.3)
where N i is the number of appeared feature points of frame F i , (px j , py j ) (j = 1, ...N i ), is the pixel that represents the j-th point in F i , k j is the coordination of this point in

M.

The geometric information of F i is also calculated and represents in position matrix P i and rotation matrix R i (which can be transformed to orientation matrix `i)

P i = x y z (4.4) R i =    r 1,1 r 1,2 r 1,3 r 2,1 r 2,2 r 2,3 r 3,1 r 3,2 r 3,3    (4.5) `i = θ x θ y θ z (4.6)
(x,y,z) is the coordination of the camera when it captured frame F i , (θ x , θ y , θ z ) is its orientation respectively in the view of Ox, Oy and Oz.

Photo Attack Detection

2D Photo Attack

In the case of photo attacks, the 3D reconstruction given by SFM method is clearly different from the one given in the case of a real face. it accounts for as much of the variability in the data as possible), and each succeeding component, in turn, has the highest variance possible under the constraint that it is orthogonal to the preceding components. In that way, the variance of point cloud projected in the last component (w (3) ) is the minimum among all vectors of space that can be used to represent the "thickness".

w (3) = arg min w =1 n ∑ i=1 (m i .w) 2 (4.7)
where n is the number of points of the cloud, m i = (x i , y i , z i ) with i = 1, .., n is the coordination vector of i-th point.

For a simplicity of the PCA transformation, the columns of the matrix M are firstly shifted to have a zero-mean. Without ambiguity, we use the same term M as the matrix shifted for the following development. The principal component matrix P is defined as an orthogonal linear transformation of the matrix M:

P = MW (4.8)
where the matrix W is a 3-by-3 matrix whose columns are the eigenvectors of M T M.

Denote v j the variance of the i-th column of P (j = 1, 2, 3). The order of magnitude of each column, denotes d i , is given as follows:

d i = v i v 1 + v 2 + v 3 (4.9)
For a face spoofing attack, the points of the 3D reconstruction is in a plan. Therefore, there is almost no information in the third component that makes d 3 very tiny.

Meanwhile, for a real face, the thickness play a significant part in the total information. The different is so net that a simple SVM classifier fed by the order of magnitude d i can be employed as a PAD classifier without any other processing.

Advanced photo attack

However, in a more sophisticated situation where the attacker uses some deformed photos to create a model which a significant depth dimension, the PCA is not performance enough for the detection. Here, we propose a method to extract the depth image of the face from the 3D model to produce another proof of liveliness.

Plan estimation and coordination

In this stage of preprocessing, the depth image can be trivially extracted by fixing a plan which is perpendicular to the normal vector of view and calculating the distance of each vertex to this plan. Ensemble of these distances forms a map which can be called a depth image. (Fig. 4.6) For each pixel in this depth image, one vertex is linked and also its value of vessel intensity. However, 3D reconstruction process is not always stable, it gives relative measure rather than absolute one. This is the reason why depth value must

D =       u 1 v 1 d 1 u 2 v 2 d 2 ... ... ... u N v N d N       (4.10)
where u i , v i (i ∈ 1, ..., N) are the projected coordination of i-th vertex in the plan. d i is depth value which corresponds to this vertex. However, the point cloud is not uniformly distributed. There are regions that contain much higher density of point than the others. There are also some parts of the face representing almost all the distinguishable features of this user that makes studying other parts is wastes.

These problems are fixed in the normalization phase.

Normalization There are two normalizations in this preprocessing: the crop of effective region of the face and the pixelization of depth and vessel image. In the scenario of observation of a front view, the crop of effective region is simply the application of the elliptic mask on the face using the nose tip detection. The location of the nose tip can be easily determined by the depth image and the width of the elliptic mask is calculated by the localization of information region. This crop is applied on both images to eliminate unnecessary points.

The pixelization is, in fact, the transformation of the point cloud to an image (in this case an image of 320 × 256-pixels). This is very similar to a scaling process apart from the fact that the point cloud is not equi-distributed. Our solution is using an adapted version of bilinear interpolation which can be summarized as follows. For p 1 , p 2 ...p h ∈ 1, ..., N are the points inside a pixel (h ∈ 0, .., N). Assume that the depth measure follows a linear relation to u-axis and v-axis, it can be represented by a local function D(u, v):

D(u, v) = c 0 + c 1 u + c 2 v + c 3 uv (4.11)
where c 0 , c 1 , c 2 , c 3 are 4 coefficients to be determined.

In the case of h ≥ 4, the problem becomes a linear least-square problem estimating C=(c 0 , c 1 , c 2 , c 3 ) T that minimizes the sum S (for the case h = 4, the equation becomes a standard bilinear interpolation and the minimized sum must be zero):

S = h ∑ j=1 (d p j -(c 0 + c 1 u p j + c 2 v p j + c 3 u p j v p j )) 2 (4.12)
This equation could be represented in matrix form as follows:

S = h ∑ j=1 (d p j -D(u p j , v p j )) 2 (4.13)
where

      D(u p 1 , v p 1 ) D(u p 2 , v p 2 ) ... D(u p h , v p h )       =       1 u p 1 v p 1 u p 1 v p 1 1 u p 2 v p 2 u p 2 v p 2 .... 1 u p h v p h u p h v p h       ×       c 0 c 1 c 2 c 3       (4.14)
The result of this minimization problem can be directly obtained by matrix equation:

Ĉ = (Q T Q) -1 Q T × d (4.15) where d = [D(u p 1 , v p 1 ), D(u p 2 , v p 2 )...D(u p h , v p h )] T and Q =       1 u p 1 v p 1 u p 1 v p 1 1 u p 2 v p 2 u p 2 v p 2 .... 1 u p h v p h u p h v p h       (4.16)
The depth value of the pixel can be calculated by D(u 0 , v 0 ) with (u 0 , v 0 ) is the center of the pixel. In the rare event when h < 4, values of neighbor pixels can be used to feed the bilinear interpolation solution. At the end of this phase, a depth image of 160 × 128 pixels is obtained .

Gabor transformation In this study, 2D Gabor filters are applied to all depth images in order to characterize each video. The Gabor wavelets contain information of spatial localization, orientation selectivity and spatial frequency selectivity . A lot of robust 2D face recognition algorithms use Gabor wavelet as the principal representation of face which places great emphasis in both spatial frequency and spatial relations. The Gabor kernel can be described as follows:

Ψ(Z) = k 2 µ,ν σ 2 exp( -k 2 µ,ν Z 2 2σ 2 )[exp(ik µ,ν Z) -exp(- σ 2 2 )] (4.17)
where µ and ν represent the orientation and scale of the Gabor wavelets. Ψ(Z) is the value of Gabor wavelet at Z = (t u , t v ). t u , t v are the centered coordination of any point in the plan. The coefficient k µ,ν is defined by k µ,ν = k ν e iφ µ with k ν = k max / f ν and φ µ = πµ/8 so that e iφ µ determines the orientation of the kernel and k ν places it into a scale. In this study, we use five scales ν ∈ {0, 1, ..., 4} and eight orientations µ ∈ {0, 1, ..., 7} which make 40 Gabor kernels with the other parameters as following:

σ = 2π, k max = π/2 and f = √ 2.
The representation of an image by Gabor wavelets, so-called the Gabor image, is the convolution of the image with a Gabor kernel. However, the convolution gives each pixel a complex value with two Gabor parts: the real part and the imaginary part. These two parts can be transformed to two types of information: Gabor magnitude features and Gabor phase features. In this study, only Gabor magnitude features are used to describe the face. For 40 Gabor kernel, 40 Gabor image can be computed/ Each image is an ellipse of size 256x320 which includes about 63600 features, so in total 2,544,000 features to feed into the classification algorithm.

Feature Selection and final classifier

The richness of Gabor transformation in terms of quantity of features improves significantly the result of classification. However, the complexity of this algorithm increases with the number of features. Therefore, a scheme proposed by Chenghua Xu is applied to divide the whole system into small ones which can work in parallel. This hierarchical selection includes two stages:

LDA sub-sampling: for each Gabor vessel image, the optimal LDA sub-sampling extrudes massively non-efficient or redundant features by minimizing the withinclass distance when maximizing the between-class distance.

Unlike the usual sub-sampling method where the sub-windows is uniformly distributed in the image, this optimal method aims for rich-information regions where the features could provide more proof of recognition. Gabor images under different orientation and scale may not share the same sub-sampling pixels. Therefore, 40 sets of sub-sampling positions are constructed correspondents to 40 Gabor depth images.

To minimize the within-class distance (explain by scatter matrix S W ) and maximize the between-class distance (S B ), the optimal discriminant vectors constructing the LDA subspace is computed by solving the following criterion in the standard LDA algorithm:

W * = argmax(J(W)) = W T S B W W T S W W (4.18)
where

W =                  w 1,1 w 2,1 ... w p,1             w 1,2 w 2,2 ... w p,2             ... ... ... ...             w 1,v max w 2,v max ... w p,v max                  (4.19)
Here, p = 15900 is the number of pixels in one image and v max is the amount of discrimination vectors (each vector is one column of W). The summation vector V can be computed as follows:

V = k=v max ∑ k=1 |w k,1 |, k=v max ∑ k=1 |w k,2 |, ..., k=v max ∑ k=1 |w k,p | (4.20)
The magnitude of V at a particular position represents the corresponding variations among the training set, which also reflects the corresponding importance in distinguishing the faces. After this stage, only 1278 features in each image are chosen for AdaBoost selection.

AdaBoost learning: a supervisor learning which applies a weak and tiny classifier on each feature of the sample in order to:

• Select the less redundant group of effective features which can discriminate the two hypotheses,

• Construct weak classifiers using these features,

• Build a strong cascaded classifier .

The algorithm of Adaboost learning for feature selection can be introduced as below:

Given example couple images (I 1 , J 1 , y 1 ),(I 2 , J 2 , y 2 )...(I n , J n , y n ) where y i =1 when I i and J i are all images of genius faces or all images of the same type of attack (photo or video) and y i =0 in other case (negative examples). Initialize weights w 1,i =1/2m or 1/2l, for y i =0,1, respectively, where m and l are the number of negatives and positives examples. For t=1,...,T (T is the maximum number of chosen features)

• 1. Normalize the weights: w t,i := w t,i ∑ n j=1 w t,j

• 2. For each feature, j, train a LDA classifier h j which using only this feature (which has 2 values). The error is evaluated by: j = ∑ i w i |h j (x i )y i |.

• 3. choose the classifier h j which minimizes the error j . j is the feature chosen in this step.

• 4.Update the weights w t+1,1 = w t,i β 1-e i t where e i = 0 if example x i is classified correctly, e i = 1 otherwise and β t = j /(1j )

Each iteration, the algorithm searches for a feature that minimize l'error of clas- In this stage, the AdaBoost selection is used twice as following:

• Individual learning: apply AdaBoost method to each Gabor depth image to select the effective feature for each image (about 30-38 per image) and group all these features into one set.

• Total learning: apply AdaBoost to this set of features to reduce one more time the number of features (about 127 features in this case).

The final step of training phase is the construction of a cascaded strong classifier from these 127 features. That cascaded classifier contains many layers, each layer is also building by the efficient features in features learning stage. In fact, instead of constructing a big classifier of a lot of features in order to achieve a detection rate D and limit the false positive rate under F, the method aims to build some small independent classifiers that provide a higher detection rate d l with a huge false positive rate f l . When these classifiers are used as layers for a bigger one we can choose the layer so that:

F = l=L ∏ l=1 f l (4.21)
D = l=L ∏ l=1 d l (4.22)
where L is the number of layers. In this way, from 12 classifiers with high false positive rate (by 50%), cascaded classifier can be building which limits F at 0.5 10 ≈ 10 -3 . The algorithm of Adaboost learning for strong classifier can be introduced as below:

• 1. Parameters initialization: selecting the value of f (the maximum acceptable false positive rate per layer) and d (the minimum acceptable detection rate per layer). This step depends essentially on the efficiency of features

• 2.Target determination: selecting the value overall false positive rate (F target ).

This step depends on the result we aim to • 5. Initialization:

F 0 = 1.0; D 0 = 1.0; l = 0 • 6. While F l > F target : -l ← l + 1 -n l = 0; F l = F l-1 -While F l > f × F i-1 * n l ← n l +

Video Attack Detection

In the scenario of a video attack, a clip of authentic user's head is displayed in a LCD screen in front of the camera. The video is edited so that the head moves in the same way to mimic the process. This attack can pass the PAD classifier since the moving head can provide different views of user's face to construct a genuine 3D model of the head. The form of the 3D reconstruction doesn't give us enough information for spoofing detection. Therefore, we proposed to study, in addition, the camera poses. In fact, the movement of the camera can be observed in two ways. The first is the positions and the poses of camera corresponding to each image. These positions which represent the movement of the camera according to the user's face can be located by 3D reconstruction. The 3D reconstruction describes the movement by position vectors P i for the camera's position and by orientation vectors `i for the camera poses with i = 1, .., n is the index of the frame. Since the movement is mono-direction (the camera move by x-axis, y-axis but not both at the same time), the position and orientation can be represented separately axis by axis in the form of sequence:

X i = P i,x (4.23)
θ x i = θ i,x (4.24)
This trajectory observed by the 3D model is, in fact, the relative movement of the camera in view of the head. These motions come from 2 sources: the absolute movement of this camera and head trajectory.

The second observation is the trajectory of camera captured by movement sensors (e.g. accelerometer, gyroscope). These sensors observe the acceleration of translation and rotation of camera continuously by the time that allows to describe that movement independently. The gyroscope captures rotation acceleration and the accelerometer captures linear acceleration of the device affected by the force of gravity.

a(t) = a x , a y , a z (4.25)
a `(t) = g x , g y , g z (

Where t ∈ R + is an index of time, g x , g y , g z is gyroscope data. Since the δt (time between two consecutive measures) is very small, the acceleration data can be considered as continuous.By the same reason of mono-direction, we can observe the movement only by x-axis or y-axis.

d 2 x(t) dx 2 = a x (t) (4.27)
d 2 θ x (t) dx 2 = g x (t) (4.28)
By using integration method, the position and orientation of the camera can be calculated from the acceleration as the initial speed and initial position are both zeros. Let's t max is the length of video, on frame is taken each ∆t = t max n-1 second. Another sequence of the camera's state can be obtained from sensor data:

X i = x(t i ) (4.29) θ x i = θ x (t i ) (4.30)
where i = 1, .., n, t 1 = 0, t n = t max , t i = ∆t.(i -1)
The camera poses are compared to the information captured from movement sensors (e.g. accelerometer, gyroscope) to obtain some similarity index. For a legit authentication case, the process does not demand any head's movement. The head's motion becomes insignificant that makes a high coherence between absolute and relative trajectory. Meanwhile, when attackers use a video to authenticate, in order to mimic a 3D object, they must display some motion in this video (without motion, it becomes a normal 2D photo). Therefore, the similarity between the two sources would not be assured. Here, the dissimilarity index is, in fact, a factor of head's motion. To estimate this dissimilarity, a simple correlation can be applied for each couple of data as: (θ x i , θ x i ), (θ

y i , θ y i ), (X i , X i ) and (Y i , Y i ).
All these features (correlation results) are fed to an SVM classifier to form the VAD classifier mentioned previously.

Result and Evaluation

Result

The proposed video-replay attack detection process requires the data of motion sensors integrated within the smartphone. Actually, there are no public face spoofing datasets responding to this requirement. Therefore, we tested the proposed method in a specific database constructed in our laboratory. The database includes 1001 videos of 3 people including sensor data, therein: 451 cases of legitimate authentication, 362 cases of video-replay attack and 188 cases of photo attack. The videos are captured in different light conditions and movement speed by three devices: 2 instances of Samsung Galaxy Alpha and a Samsung Galaxy Tab. The database is divided into 2 sets. Each set has all three types of videos (legitimate, photo-attack and video-replay attack authentication). One set is used for training the PAD and VAD classifiers and the other set is used for testing. From the training set, we used all videos to train the PAD classifier, but we used just legitimately and video-replay attack authentication videos to train the VAD one. We also implemented the multiscale Local Binary Pattern method, which applies for a single image. Therefore, frames from the videos are used as input to implement this method. Figure 4.11 shows the Receiver Operating Characteristics (ROC) curves of the proposed method and the LBP one. Our method performs much better than the LBP one especially for a small rate of false positive. In fact, a large portion of false alarms links to the bad behavior in capturing phase. A lot of problems such as the blurry frame due to the high speed of movement or the escape of user's face from the input video make

Evaluation

Our solution for photo attack detection is, in fact, a classification of 3D objects based on their depth images. The different is quite large between the form of a genius face and the form of a material that displays a photo. It is the reason why we can obtain 100% in detection rate. Because of the maximum percentage obtained by different methods of feature extraction, we present only Gabor transformation here to emphasis that Gabor wavelet outperform other methods in case of depth image.

An experimental proof of this statement can be found in the next chapter.

In the process of Video attack detection, we assume the worst case of authentication where users have total control of proof capturing phase: authentication using smartphone. We do not have any idea of what they place in the face of the camera, what material they can use to display the video and what type of movement in the attacking video. We do not control which type of smartphone used in the process or any supplemental calibration information. The unstable smartphone's camera (due to vibration of manual manipulation) is the principal source of false detection. The error is generated by the manipulation itself but not by the user which explains the fact that a large percentage of false detection linked to first or second videos of each scenario. In the real application, a demand for "retry" for another capturing phase can reduce significantly the false detection rate.

Chapter 4. Face Spoofing Detection Using 3D Model

In other solution where the situation is more controlled, maybe, with a mechanically moving smartphone or a calibrated camera, the performance can be significantly ameliorated. However, 3D mask attack is theoretically immune against our detection because all of our proof is based on the hypotheses that the attacker use a 2D material for authentication. In general, 3D mask attacks are very efficient in case of visible face recognition. A perfect mask can overcome any system using an only normal camera. Nevertheless, this type of attack can be easily detected by other types of sensor such as IR detector. In the next chapter, we will present a novel method of face recognition using an uncooled thermal camera which solves not only face spoofing attack but also illumination problem.

Chapter 5

Face Recognition by Thermal Video Using Vesselness Features in Multiview 3D Projections

Introduction

In this chapter, we present a novel method for face recognition by a movable thermal uncalibrated camera. The first section mentions some problematic of face recognition in general which leads to the necessity of our method. We will also point out the advantages and limits of our solution in comparing with others. The general process is divided into 2 phases: preprocessing and main-processing which are correspondingly described inside the second and the third section. The last section proposes an ameliorated scheme of the solution which alternates the main-processing with only one frontal pose by a multi-pose method.

Problematic and Objectives

Promoted by impressive growth of computation capacity in recent years, face recognition has conquered a large portion of biometric security domain and has appeared in many applications such as authentication, identification, human tracking and surveillance. Inherit from numerous studies in computer vision over the last there decades, face recognition is now broadly used in a lot of information systems, from giant immigration control in airports to tiny user unlock solution for smartphone, from extremely strict authentication for Internet banking transaction to popular identification for Facebook photo... However, visible face recognition always suffers from crucial limits due to complex illumination conditions. Another persistent problem in visible face recognition authentication comes from the face spoofing attack using images, video or 3D mask with the face of the victim. These problems have opened a road for thermal imagery to join in the competition. This last one is more robust not only against environmental variations but also against facial expressions. Thermal images can solve many impossible challenges in face recognition like complete dark environment or Chapter 5. Face Recognition by Thermal Video Using Vesselness Features in Multiview 3D Projections monozygotic twin problems. Despite all these advantages, the average accuracy of thermal image classification is always lower than the visible one that requires new solution to improve its performance. • Preprocessing: The 3D model of the head is reconstructed using a video which includes various poses and positions of the head . A blood vessel transformation is applied to each frame of this video to obtain a vessel map . These blood vessel maps are projected on the 3D model in order to obtain a 3D vesselness representation. Thanks to this new model, we can obtain a depth image and a vessel map for each pose. Our first experiment had used only frontal view which had neglected a large portion of information in other poses. Later, the process has been tested for different combinations of views to choose the most relevant result in terms of accuracy and computation.

Proposed approach

• Feature representation: A lot of transformations widely used in visible imagery (like LBP, Weber, ...) are tested to combine the two informations. We finally chose Gabor filters as the primary transformation in feature representation.

These filters with multiple scales and multiple orientations are applied to the depth image and vessel map to extract a lot of Gabor features. Each feature includes two values, one from vessel network, another from depth information.

• Feature selection: As described above, many Gabor features are extracted from the depth image and vessel map, but not all of these informations are effective to the recognition system. In this phase, a hierarchical scheme for effective feature selection is proposed using linear discriminant analysis (LDA) and Ad-aBoost method .

The testing phase uses the same preprocessing scheme as training phase. The feature templates constructed in training phase are now applied to select the feature in

Contributions

In this study, we propose and examine a new face recognition solution using a 3D model of the head computed from a thermal video which contains information of vascular network. Its contributions are as follows:

• The reconstruction of a 3D model from a thermal video and the projection of the map of blood vessels on this model give a new 3D representation of vascular network.

• The depth and blood vessels intensity aren't treated like unbound features but are jointed to form a single feature with two values. In this way, the face recognition system bases essentially on the 3D location of vascular network.

• The 3D model is represented in the form of a combination of depth images in different views. This process make possible the feature selection and the classification of a complex object without any loss of information.

Preprocessing

In this paper, we propose a four-step process (Fig. 5.2): rebuilding of 3D model from every frames of the input thermal video, vessels extraction for each video frame and the projection of these vessels on the 3D model, estimation of the depth image and finally a step of the normalization.

Reconstruction of 3D model

In our study, the input thermal video is supposed to contain many frames from various poses of one head. An algorithm of 3D reconstruction is used to compute a 3D point cloud which describes the head filmed in the video. In this scenario, we use VisualSFM -Structure from motion, developed by Changchang Wu , a robust These features(common edges and points) are tracked from one frame to next so the position and orientation of each frame can be estimated by geometric calculation.

Different views of one point which can be obtained from many consecutive frames are extracted to estimate its deep coordination and then its 3D position.

From the original video, a set of frame F i (i = 1, ..., n where n is the number of frames) can be extracted. Each frame is an image (p × q pixels) which contains a view of the face:

F i =       x i 1,1 x i 1,2 ... x i 1,q x i 2,1 x i 2,2 ... x i 2,q ... ... ... ... x i p,1 x i p,2 ... x i p,q       (5.1)
In fact, each frame is compared to all other frames by method SIFT (scale-invariant feature transform). Two frames (F c 1 , F c 2 ) which maximize the similarity index are chosen to form the base of 3D object. These common points of F c 1 and F c 2 will form a first model which is called ST 2 (ST 1 do not exist).

(c 1 , c 2 ) = argmax i =j SIFT(F i , F j ) (5.2)
Then, the process sorts all images in the decreased order of SIFT(F c 1 , F i ) which makes a complete sequence c 3 , c 4 , ..., c n . For each frame F c i (i = 3, ...n), the process will firstly try to find the common points with ST i-1 which will be called STF i (STF i ⊆ ST i-1 ). So we have a certain points STF i in 3D coordination and its project 5.2. Preprocessing 79 in the plan of the camera: F c i . Thus the process will try to estimate the orientation and the position of the camera by maximizing the similarity index between the two sets:

(R c i , Pc i ) = argmax R,P SIFT(F c i , T P (PROJ R (STF i )) (5.3)
where T P is the translation operator by vector P and PROJ R is the projection operator by rotation matrix R.

P c i =    x y z    (5.4) R c i =    r 1,1 r 1,2 r 1,3 r 2,1 r 2,2 r 2,3 r 3,1 r 3,2 r 3,3    (5.5)
The geometric information of F c i is represented in position matrix P c i and rotation matrix R c i (which can be transformed to orientation matrix `i)

`i =    θ x θ y θ z    (5.6)
(x,y,z) is the coordination of the center pixel of frame F c i , (θ x , θ y , θ z ) is its orientation respectively in the view of Ox, Oy and Oz.

The process will, then, studies the extra part of frame F c i which does not exist inside STF i and try to match its with some other nearest frame to find additional common points

STP i ST i = ST i-1 ∪ STP i (5.7)
The last one (ST i ) becomes the 3D model formed by {F c 1 , ..., F c i }.

The set ST n definite the 3D model of all frame. This 3D model form a matrix which represents a cluster of featured points in a Cartesian coordinate system. Notes M ini the 3D model , it can be represented as: • Filtering: Intensity constraints (and a weak form of regularization) are used to eliminate incorrect matches.

M ini =    x 1 x 2 ... x N y 1 y 2 ... y N z 1 z 2 ... z N    (5.
These 3 steps are repeated for several iterations until the set of points (the mesh) is dense enough. This mesh can be further refined by a mesh-based MVS algorithm that enforces the photometric consistency with regularization constraints like Poisson Surface Reconstruction. The resolution of the mesh model is adaptive, and the size of a triangle depends on the density of the nearby oriented points: The denser the points are, the finer the triangles become. The PSR software outputs a closed mesh model even when patches are only reconstructed for a part of a scene. In order to remove extraneous portions of the mesh, we discard triangles whose average edge length is greater than six times the average edge length of the whole mesh since triangles are large where there are no points.

So, in fact, after the PMVS process, a matrix M (having the same structure as M ini but having much more points, here we always use N as the number of points ) is obtained. This rebuild 3D model includes a huge number of vertex and triangular surfaces which can be considered as a dense facial surface (Fig. 5.4).

Vessel extraction and projection

The vascular network is the product of anatomical observations in thermal imagery.

The key idea of this feature is the higher temperature of the blood vessel in comparing with neighbor region. The method is proposed by Buddharaju and is enhanced by Reza Shoja Ghiass. The so-called vascular network is a map of tubular structures extracted from a thermal image.

This type of feature is proven to be a effective transformation in thermal face representation.

For each frame F i (i = 1, ..., n) consider the two eigenvalues λ 1 and λ 2 of the Hessian matrix computed at a certain image locus and at a particular scale s. Without

V(s) =    0 i f λ 2 > 0, (1 -e
- R A 2β
2 ) × (1 -e -S 2c 2 ) otherwise, (5.11)
where β and c are the parameters that control the sensitivity of the filter to R A and S.

In fact, the "vessel value" of a pixel is represented by the measure V(s) of the (6s + 1) × (6s + 1) block centered at this pixel. Finally, if an image is analyzed across scales from s min to s max , the vesselness of particular image locus can be computed as the maximal vesselness across the range:

V 0 = max s min ≤s≤s max V(s)
(5.12) In the end, each vertex is associated with a value V 0 which presents the vessel probability at this point. Another column V 0 can be added to matrix M:

M =       x 1 x 2 ... x N y 1 y 2 ... y N z 1 z 2 ... z N V 0 (1) V 0 (2) ... V 0 (N)       (5.13)
For each intensity image, its poses, positions and contributions to the 3D model is computed under a texture map. By using this texture map, these vascular networks can be projected to the 3D model in order to form a 3D vessel model which represents the 3D coordinates of vessel features (Fig. 5.5).

Plan estimation and coordination

In this stage of preprocessing, the study aims to obtain a couple of depth and vessel images corresponding to a certain view. The depth image can be trivially extracted by fixing a plan which is perpendicular to the normal vector of view and calculating the distance of each vertex to this plan. Ensemble of these distances forms a map which can be called a depth image. (Fig. 5.6) For each pixel in this depth image, one vertex is linked and also its value of vessel intensity. However, 3D reconstruction process is not always stable, it gives relative measure rather than absolute one. This is the reason why depth value must

D =       u 1 u 2 ... u N v 1 v 2 ... v N d 1 d 2 ... d N V 0 (1) V 0 (2) ... V 0 (N)       (5.14)
where u i , v i (i ∈ 1, ..., N) are the projected coordinations of i-th vertex in the plan. d i and V i are depth value and vessel measure which corresponds to this vertex. However, the point cloud is not uniformly distributed. There are regions that contain much higher density of points than the others. There are also some parts of the face representing almost all the distinguishable features of this user that makes studying other parts is wastes. These problems are fixed in the normalization phase.

Normalization

There are two normalizations in this preprocessing: the crop of effective region of the face and the pixelization of depth and vessel image. In the scenario of observation of a front view, the crop of effective region is simply the application of the elliptic mask on the face using the nose tip detection. The location of the nose tip can be easily determined by the depth image and the width of elliptic mask is calculated by the localization of information region (Fig. 5.7). This crop is applied on both images to eliminate unnecessary points

The pixelization is, in fact, the transformation of the point cloud to an image (in this case an image of 160 × 128-pixels). This is very similar to a scaling process apart from the fact that the point cloud is not equi-distributed. Our solution is using an adapted version of bilinear interpolation which can be summarized as follows. For p 1 , p 2 ... In the case of h ≥ 4, the problem becomes a linear least-square problem estimating C=(c 0 , c 1 , c 2 , c 3 ) T that minimizes the sum (for the case h = 4, the equation becomes a standard bilinear interpolation and the minimized sum must be zero): 2 (5.16)

S = h ∑ j=1 (d p j -(c 0 + c 1 u p j + c 2 v p j + c 3 u p j v p j ))
This equation could be represented in matrix form as follows:

S = h ∑ j=1 (d p j -D(u p j , v p j )) 2 (5.17)
where

      D(u p 1 , v p 1 ) D(u p 2 , v p 2 ) ... D(u p h , v p h )       =       1 u p 1 v p 1 u p 1 v p 1 1 u p 2 v p 2 u p 2 v p 2 .... 1 u p h v p h u p h v p h       ×       c 0 c 1 c 2 c 3       (5.18)
This equation becomes:

d = Q × C (5.19) where d = [D(u p 1 , v p 1 ), D(u p 2 , v p 2 )...D(u p h , v p h )] T and Q =       1 u p 1 v p 1 u p 1 v p 1 1 u p 2 v p 2 u p 2 v p 2 .... 1 u p h v p h u p h v p h       (5.20)
The result of this famous problem can be directly obtained by matrix equation:

Ĉ = (Q T Q) -1 Q T × d (5.21)
The depth value of the pixel can be calculated by D(u 0 , v 0 ) with (u 0 , v 0 ) is the center of the pixel. This process also works for vessel measure. In the rare event when h < 4, values of neighbor pixels can be used to feed the bilinear interpolation solution. At the end of this phase, a double image (I d and I V ) of 160 × 128 pixels is obtained where each pixel has a couple of value (d, V) correspond to depth information and vessel measure.

Feature learning

As mentioned above, the training phase (Fig. 5.8) includes three steps which are: first, the data representation by Gabor transformation, then the hierarchical feature

Gabor Transformation

In this study, 2D Gabor filters are applied to all double images in order to characterize each video. The Gabor wavelets contain information of spatial localization, orientation selectivity and spatial frequency selectivity . A lot of robust 2D face recognition algorithms use Gabor wavelet as the principal representation of face which places great emphasis in both spatial frequency and spatial relations. The Gabor kernel can be described as follows:

Ψ(Z) = k 2 µ,ν σ 2 exp( -k 2 µ,ν Z 2 2σ 2 )[exp(ik µ,ν Z) -exp(- σ 2 2 )] (5.22)
where µ and ν represent the orientation and scale of the Gabor wavelets. Ψ(Z) is the value of Gabor wavelet at Z = (t u , t v ). t u , t v are the centered coordination of any point in the plan. The coefficient k µ,ν is defined by k µ,ν = k ν e iφ µ with k ν = k max / f ν and φ µ = πµ/8 so that e iφ µ determines the orientation of the kernel and k ν places it into a scale. In this study, we use five scales ν ∈ {0, 1, ..., 4} and eight orientations µ ∈ {0, 1, ..., 7} which make 40 Gabor kernels with the other parameters as following:

σ = 2π, k max = π/2 and f = √ 2.
The representation of an image by Gabor wavelets, so-called the Gabor image, is the convolution of the image with a Gabor kernel. However, the convolution gives each pixel a complex value with two Gabor parts: the real part and the imaginary part. These two parts can be transformed to two types of information: Gabor for the vessel intensity), so in total 1,272,000 features to feed into classification algorithm.

Feature Selection and final classifier

The richness of Gabor transformation in terms of quantity of features improves significantly the result of classification. However, the complexity of this algorithm increases with the number of features. Therefore, a scheme proposed by Chenghua Xu is applied to divide the whole system into small ones which can work in parallel.

This hierarchical selection includes two stages:

LDA sub-sampling: for each Gabor vessel image, the optimal LDA sub-sampling extrudes massively non-efficient or redundant features by minimizing the withinclass distance when maximizing the between-class distance .

Unlike the usual sub-sampling method where the sub-windows is uniformly distributed in the image, this optimal method aims for rich-information regions where the features could provide more proof of recognition. Gabor images under different orientation and scale may not share the same sub-sampling pixels. Therefore, 40 sets of sub-sampling positions are constructed correspondents to 40 Gabor double images. To minimize the within-class distance (explain by scatter matrix S W ) and maximize the between-class distance (S B ), The optimal discriminant vectors constructing the LDA subspace are computed by solving the following criterion in the standard LDA algorithm:

W * = argmax(J(W)) = W T S B W W T S W W (5.23)
where

W =                  w 1,1 w 2,1 ... w p,1             w 1,2 w 2,2 ... w p,2             ... ... ... ...             w 1,v max w 2,v max ... w p,v max                  (5.24)
Here, p = 15900 is the number of pixels in one image and v max is the amount of discrimination vectors (each vector is one column of W). The summation vector V can be computed as follows: AdaBoost learning: a supervisor learning which applies a weak and tiny classifier on each feature of the sample in order to:

V = k=v max ∑ k=1 |w k,1 |, k=v max ∑ k=1 |w k,2 |, ...,
• Select the less redundant group of effective features which can discriminate the two hypotheses ,

• Construct weak classifiers using these features,

• Build a strong cascaded classifier .

The algorithm of Adaboost learning for feature selection can be introduced as below:

Given example couple images (I 1 , J 1 , y 1 ),(I 2 , J 2 , y 2 )...(I n , J n , y n ) where y i =1 when I i and J i are images of a same person (positive examples) and y i =0 in other case (negative examples). Initialize weights w 1,i =1/2m or 1/2l, for y i =0,1, respectively, where m and l are the number of negatives and positives examples. For t=1,...,T (T is the maximum number of chosen features)

• 1. Normalize the weights: w t,i := w t,i ∑ n j=1 w t,j

• 2. For each feature, j, train a LDA classifier h j which using only this feature (which has 2 values). The error is evaluated by: j = ∑ i w i |h j (x i )y i |.

• 3. choose the classifier h j which minimizes the error j . j is the feature chosen in this step.

• 4.Update the weights w t+1,1 = w t,i β 1-e i t where e i = 0 if example x i is classified correctly, e i = 1 otherwise and β t = j /(1j )

• 2.Target determination: selecting the value overall false positive rate (F target ).

This step depends on the result we aim to

• 3. P = set of positive examples.

• 4. N = set of negative examples.

• 5. Initialization:

F 0 = 1.0; D 0 = 1.0; l = 0 • 6. While F l > F target : -l ← l + 1 -n l = 0; F l = F l-1 -While F l > f × F i-1 * n l ← n l +

Testing phase

In testing phase, the process of preprocessing and feature extraction (Gabor transformation) is the same as in training phase. However, the features are extracted directly

S(V 0 , V k ) = L ∑ l=1 n l ∑ p=1 α l,p h l,p (V 0 , V k ) (5.28)
where L = 10 is the number of layers and n l is the number of features in l-th layer. h l,p is a weak classifier which is based on one efficient double-feature and α l,p is its weight. V 0 is the testing video and V k where k ∈ 1, 2, ...20 are training video.

Among 20 scores of similarity, the k -NN classifier chooses k training video which are the most similar to the input video to decide the result of the test (Fig. 5.8).

The number of reference sample (k) depends on the number of training samples for As none of the existing thermal database provides video with the movement required in this study, we have decided to test our method in our own thermal video database. This database contains 161 videos of 4 subjects which are captured using a Therm-app camera.

In experiment phase, the database is randomly divided into two subsets: the training set and the testing set. The training set contains 5 videos of each subject (20 in total) and the testing set contains 141 other videos. In order to prove the improvement of this method compared to local matching approach using Gabor transformation, 3 tests are evaluated at the same time: one uses only intensity information (Intensity only), another one mixes the intensity and depth data at the decision level (Decision mix), and the last one mixes the two types of information at feature level (our method: Feature mix).

The experimental results (Table 5. set of 40 videos, 10 for each subject, the accuracy of these tests is all around 99%).

These results which came from the frontal pose of the head can be enhanced by mixing with other poses from the 3D model to achieve a complete process.

By comparing the proposed method with another process based on LBP [? ], we observe that LBP is far better than Gabor in description of intensity data. However, by mixing these intensity data with deep information, we obtain a great amelioration in classification using Gabor features (9%) which help this last one surpass the method using LBP-features. This result becomes a solid evidence to our choice of using Gabor-description for geometry data.

Multi-pose recognition

The result of last experiments proves the advantage of mixing depth and intensity data at feature level. However,by using only frontal view, the process neglects a large source of information in profile views. In fact, the profile views are proven to be more effective than frontal image in automatic face recognition. At this stage, the study aims to get some projected images of 3D model in profile views in order to ameliorate the strong classifier.

Profile views images

The profile views are normally taken at 90 • . However, the process of 3D reconstruction is based on a couple of frontal photos at the first step. The farther this process goes from the original couple, the less precise the performance of 3D reconstruction.

M right = θ right × M (5.30)
Where θ le f t and θ right are respectively the matrix of 60 • rotation and -60 • rotation.

(Notice that the intensity value does not depend on the rotation)

`left =       -sin π 3 0 cos π 3 0 0 1 0 0 cos π 3 0 sin π 3 0 0 0 0 1       (5.31) `right =       -sin -π 3 0 cos -π 3 0 0 1 0 0 cos -π 3 0 sin -π 3 0 0 0 0 1       (5.32)
The projection of these two models (M le f t and M right ) by z-axis will be processed in the same way of frontal views to make 2 double-images: D le f t from M le f t and D right from M right .

D left =       u l 1 u l 2 ... u l N v l 1 v l 2 ... v l N d l 1 d l 2 ... d l N V 0 (1) V 0 (2) ... V 0 (N)       (5.33)
D right =       u r 1 u r 2 ... u r N v r 1 v r 2 ... v r N d r 1 d r 2 ... d r N V 0 (1) V 0 (2) ... V 0 (N)       (5.34)
where (u l i , v l i ) and (u r i , v r i ) (i ∈ 1, ..., N) are respectively the projected coordination of i-th vertex in the left-plan and the right-plan. d l i and d r i are depth values which corresponds to this vertex.

The normalization phase is now more complicated as the oval model centered on the nose is no longer adapted. The oval model is determined by the nose at the extreme pixel of the left and the chin is at a fixed point of the oval

Experiments and results

The feature extraction of multi-view is similar to the frontal view. The complexity of the training process is estimated as: training-time, the study is limited at 3 views which can provide information about most part of the face.

T = O(h 2 k) + O(k 2 ) ( 5
In experiment phase, the training-set contains 20 videos (5 for each subject). The test-set contains 100 videos among the rest. All the sets are chosen randomly. The classification is processed for mono-view and multi-views at the same time in order to provide a comparison between them.

The result ameliorate significantly from using only frontal view of using 3 principal views. This fact proves that a combination of views can represent a 3D model.

The result can be ameliorated a little more by augmenting the number of views using in the combination. However, the increase in the computation cost reduces the performance of the method.

Chapter 6

Conclusions and Perpectives

Conclusion

The development of imaging technology and computing capacity lead us to an era where user's face can be considered as their proof of authentification toward an automatic system. The most convenient and natural method is trying to mimic the human's vision using computer vision. Because of this reason, visible imagery is the first option for every system authentication by facial recognition.

However, visible imagery technology is not robust enough to be used as the only source of identification information. This method has two major limits that make the authentication systems vulnerable. The first limit is its sensitivity to the illumination condition. The very nature of visible image that is captured inside trivial cameras is a reflection of the lighting which hit the object placed in front of the lens. This type of photo depends not only on the color of the object but also on the nature and intensity of light source. The second problem is an active attack where the attacker uses a signature of genius user's face in order to bypass the authentication system.

The signature may be a photo, a video found in social network or even a 3D mask of this user.

In the first part of this study, we aim to construct a solution against the facespoofing attack with minimum equipment required. Our hardest use-case associates to the face recognition method for smartphones with a visible camera. This use-case is complicated since the system is based on a single uncalibrated camera and the scene of authentication depends entirely on users. We adjust our general solution for this case by exploring the capacity of movement and some motion sensors inside the smartphone. From a set of video's frames, the method uses a 3D reconstruction process to build a 3D model of the head which is highly effective against photoattack as differences in geometric features between a real object and an image is truly large. The video attack can be detected by observing the synchronization between the prior motion of the smartphone (explored by motion sensors) and the capturedmotion calculated by the 3D reconstruction process. The limit of our first study is that it can only justify if the object has a real 3D form of user's faces, but it is not able to detect a mask attack.

A.4.2 État de l'art

Des méthodes de reconnaissance faciale dans l'infrarouge peuvent se regrouper en 4 groupes:

• Apparence globale : utiliser la totalité de l'image infrarouge de l'apparence d'un visage pour la reconnaissance.

• Caractéristique: utiliser des caractéristiques extraites de l'image infrarouge tels que la géométrie du visage, son réseau vasculaire ou la figure de perfusion sanguine.

• Multi-spectral: modéliser le processus de formation d'une image infrarouge pour décomposer des images de visages. Certaines approches utilisent directement les données de capteurs d'imagerie multi-spectrale ou hyperspectrale pour obtenir des images faciales à travers différentes sous-bandes de fréquences.

• Multi-modal: combiner des informations contenues dans les images infrarouges avec celles obtenues avec d'autres types de modalités, telles que les données du spectre visible, afin d'exploiter leurs complémentarités.

A.4.3 Solution proposée

Notre méthode de reconnaissance du visage s'effectue en deux étapes: l'extraction d'informations caractéristiques et leur classification pour identifier la personne. The improvement of imaging technology leads us to an era in which user's faces can be acknowledged as a biometric proof of authentication toward an automatic system. Visible imagery is naturally the first option for every facial recognition system. However, visible imagery has two major drawbacks that make the identification systems vulnerable: its dependency on the light source and its incompetence toward face-spoofing attacks. The first part of this study aims to construct a solution against the face-spoofing attack with minimum equipment required. The face recognition solution for smartphones is our hardest use-case because of the uncalibrated camera and unpredictable behaviors of users. From a set of video's frames, the method builds a 3D model of the head using a dedicated reconstruction scheme. This model is highly effective against photo-attack as differences between a real object and an image is truly large. The video attack can be detected by examining the synchronization between the prior motion of the smartphone (explored by motion sensors) and the captured-motion calculated by the 3D reconstruction process. In thermal imagery where the emission source of the spectrum is human's face, the detection of all types of face-spoofing attack is trivial, and the illumination conditions do not affect thermal images. Though, in general, thermal images present less information than visible images. In our second study, we aim to improve the performance of thermal face-recognition method using a 3D model of the vascular network computed from an infrared video.

4. 8

 8 Example of the camera's positions estimated from the 3D reconstruction. Direction of the camera's move is marked form 1 to 8. . . . . . . . 4.9 Orientation of camera described by gyroscope sensor (blue for θ x i , red for θ y i and green for θ z i , . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.10 Correlations between θ x i (black)and θ x i (red) and between θ y i (black) and θ y i (red). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.11 ROC curve for the proposed detection method (Red) in comparison with the one of LBP method(blue). The proposed method with recapturing option is displayed in yellow. . . . . . . . . . . . . . . . . . . . . 5.1 Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Gray-scale and iron-palette version . . . . . . . . . . . . . . . . . . . . . 5.4 point cloud (left), dense vertex (middle), surface (right) . . . . . . . . . 5.5 3D vascular network model . . . . . . . . . . . . . . . . . . . . . . . . . 5.6 Depth image of a face . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7 Elliptic mask (left), cropped intensity image(middle) and cropped vessel image(right) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.8 Learning and testing phase of classifier . . . . . . . . . . . . . . . . . . 5.9 LDA template for Gabor image with u = 4 and v = 8 . . . . . . . . . . 5.10 Roc curve of the strong classifier. . . . . . . . . . . . . . . . . . . . . . . 5.11 Profile view. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.12 Normalized profile view. . . . . . . . . . . . . . . . . . . . . . . . . . . 5.13 Roc curve of the strong classifier for mono-view and multi views. . . . ix A.1 Différentes types de l'attaque de l'usurpation de visage. . . . . . . . . . A.2 Processus de détection de l'attaque de l'usurpation de visage. . . . . . A.3 Nuage de points du modèle 3D d'un visage réel . . . . . . . . . . . . . A.4 Différentes vues de la modèle 3D d'une attaque par photo. . . . . . . . A.5 PCA de la modèle 3D d'un utilisateur (a) et d'une attaque (b). . . . . . A.6 Image de profondeur d'un visage . . . . . . . . . . . . . . . . . . . . . . A.7 Positions de l'appareil photo estimées à partir de la reconstruction 3D. A.8 Orientation de l'appareil photo à partir des données du capteur gyroscopique (θ x i en bleu, θ y i en rouge, θ z i en vert) . . . . . . . . . . . . . . . A.9 Corrélation entre θ x i (noir) et θ x i (rouge) et entre θ y i (black) et θ y i (red). . A.10 Courbe ROC de la méthode proposée (rouge) en comparaison avec la méthode LBP (bleu). La méthode proposée avec une seconde prise (jaune) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.11 Les images thermiques ne sont pas sensibles à la condition d'illumination109 A.12 Différentes applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . A.13 Réseau vasculaire d'une image thermique . . . . . . . . . . . . . . . . A.14 La reconstruction 3D de visage en utilisant des images thermiques. . . A.15 Projection du réseau vasculaire dans le modèle 3D pour personne A. . A.16 Projection du réseau vasculaire dans le modèle 3D pour personne B. . A.17 La courbe de Roc du classificateur final. . . . . . . . . . . . . . . . . . . A.18 Caméra ThermApp couplée avec un smartphone . . . . . . . . . . . . . xi List of Tables 2.1 Existing methods comparing. . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Existing methods comparing. . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Infrared sub-bands comparing. . . . . . . . . . . . . . . . . . . . . . . . 5.1 Average results after repeating 20 times the process of experimentation. 5.2 Average results for 20 test sets. . . . . . . . . . . . . . . . . . . . . . . . A.1 Résultats moyens après ayant répété 20 fois le processus d'expérimentation.114

FIGURE 2 . 1 :

 21 FIGURE 2.1: Face recognition example

FIGURE 2 . 2 :

 22 FIGURE 2.2: Face description using Eigenface and Fisherface

FIGURE 2 . 3 :

 23 FIGURE 2.3: Geometrical features detected in the face

FIGURE 2 . 4 :

 24 FIGURE 2.4: Regions for template matching

FIGURE 2 . 5 :

 25 FIGURE 2.5: Face recognition's problematics.

FIGURE 2 . 7 :

 27 FIGURE 2.7: Different types of face spoofing attack.

FIGURE 2 . 8 :

 28 FIGURE 2.8: Photo attack illustration.

FIGURE 2 . 9 :

 29 FIGURE 2.9: 3D-mask attack enhanced by painting.

Ivana

  Chingovska et al.[48] suggested a texture feature technique in which the Local Binary Pattern (LBP) algorithm was applied. Tests were made various sorts of databases: NUAA Photograph Imposter Database, Replay Attack Database, Public Database, and CASIA Face Antispoofing Database. This strategy presented 15% total error rate of different databases. Jukka Komulainen et al. [49] presented a context-based face anti-spoofing method in the year of 2013. The experiment was tested with two publically accessible databases,

FIGURE 2 . 10 :

 210 FIGURE 2.10: Face spoofing detection algorithm based on color texture analysis. [47]

FIGURE 2 . 11 :

 211 FIGURE 2.11: Score the eye closity to detect eye blinking.

FIGURE 2 . 12 :

 212 FIGURE 2.12: Infrared imagery application

FIGURE 2 . 13 :

 213 FIGURE 2.13: Infrared image for face recognition

FIGURE 2 . 14 :

 214 FIGURE 2.14: Transmitting rate in atmosphere of IR spectrum

FIGURE 2 .

 2 FIGURE 2.15: Infrared spectrum

FIGURE 2 . 16 :

 216 FIGURE 2.16: SWIR image is almost as details as visible image

FIGURE 2 . 17 :

 217 FIGURE 2.17: Face image in the visible spectrum (left), SWIR, MWIR, LWIR(right)

FIGURE 2 . 18 :

 218 FIGURE 2.18: Infrared image in pseudo-color

FIGURE 2 . 19 :

 219 FIGURE 2.19: Image taken by cooled Infrared camera(left) in high capture rate compared to one taken by uncooled camera(right)

FIGURE 2 . 20 :

 220 FIGURE 2.20: Microbolometer used in Uncooled infrared detectors

FIGURE 2 . 21 :FIGURE 2 . 22 :

 221222 FIGURE 2.21: Thermal image is more robust against various illumination conditions than visible one

  The great sensibility of the facial infrared image in no small number of external factors challenges the thermal face recognition community in finding persistent and discriminative features. It also provides proof for the ideas first proposed by Prokoski et al.[START_REF] Prokoski | Infrared Identification of Faces and Body Parts[END_REF] who discussed against the use of appearance-based methods for thermal face recognition in favor of superficial anatomical feature based approaches which is robust against many aforementioned factors.

FIGURE 2 . 23 :

 223 FIGURE 2.23: Thermal image perturbed by eyeglasses(They are not sun eyeglasses)

FIGURE 2 . 24 :

 224 FIGURE 2.24: Categorization of thermal face recognition

  Wu et al. proposed in[START_REF] Wu | Thermal face recognition using convolutional neural network[END_REF] an architecture using CNN (convolutional neural network) for face recognition. The author wants to replace traditional methods using hand-crafted features that need a great work for feature selection and extraction by an auto process. CNN method is also applied experimentally on RGB-D-T face database . Simon et al. state that CNN architecture can achieve higher accuracy in face recognition than other traditional features like LBP or HOG (Histograms of Oriented Gradients)[START_REF] Simón | Improved RGB-D-T based face recognition[END_REF]. Using CNN method for NIR image, NIRFaceNet[START_REF] Peng | NIRFaceNet: A Convolutional Neural Network for Near-Infrared Face Identification[END_REF] proposed by Peng et al. reached 98,48% in terms of recognition rate. Another work studied by Orji et al. [83] combined CNN and FWT (Fast Wavelet Transform) in order to form a deep neural network of 6 layers (one input, two convolution layers, two subsampling layers and an output). The author also compared the result between with and without the preprocessing using PCA and LDA. In their work, Kwasniewska et al. [84] used deep neural network Inception v3 for face detection and human tracking with a low-resolution thermal image acquired by a portable camera. Also based on deep learning, Sarfraz et al. present in their work [85] a thermal/visible cross face recognition methods using DPM (Deep Perceptual Mapping).

FIGURE 2 . 25 :

 225 FIGURE 2.25: CNN structure for thermal face recognition

FIGURE 2 . 27 :

 227 FIGURE 2.27: Blood perfusion model

,

  Bourlai et al. investigate the advantages and limitations of cross-matching from SWIR, MWIR or NIR to visible imaging. The author state that long distant cast a severe consequence upon thermal face recognition rate. Their experiments indicated that cross-spectral matching is a tough challenge which demands further investigation.Zhang et al. considered in their work[START_REF] Zhang | TV-GAN: Generative Adversarial Network Based Thermal to Visible Face Recognition[END_REF] that direct application of visible face recognition model into thermal spectrum do not reach a satisfactory performance. Therefore, they proposed the TV-GAN (Thermal-to-Visible Generative Adversarial Network ), a transformation technique allows obtaining a pseudo-visible image corresponding to the original thermal image. The transformation is said to be able to conserve enough of identification features to operate a visible face authentication. Saxena et al. evaluated the possibility to used features from a CNN pre-trained on standard visible images in heterogeneous face recognition[START_REF] Saxena | Heterogeneous Face Recognition with CNNs[END_REF]. After having explored various learning strategies with different modalities, the author state that Near Infrared (NIR) image can recognize using the CNN pre-trained features of visible spectrum images.

FIGURE 2 .

 2 FIGURE 2.28: TV-GAN generates Visible image from IR image

FIGURE 2 .

 2 FIGURE 2.29: 3D multi-spectrum sensor system

  structured radiation, laser range finder, and other active sensing methods. A simple example of a mechanical technique would employ a depth gauge to estimate a gap to a rotating object placed on a turntable. More relevant active scanners release some radiation, beam or light and catch its reflection or diffraction passing through the object to probe an object or environment. Examples cover from colored visible light, time-of-flight lasers, moving light source to ultrasound and microwaves[98,[START_REF] Schuon | High-quality scanning using time-of-flight depth superresolution[END_REF][START_REF] Blais | Accurate 3d acquisition of freely moving objects[END_REF].Time-of-flight :The time-of-flight laser scanner is an active scanner that employs laser beams to examine the subject. At the core of this kind of sensor, a time-of-flight laser range finder resides. The laser range finder spots the distance of a facade by measuring the round-trip period of pulsation of light. A laser is utilized to release a pulse of light and the amount of time before a detector recognizes the reflected light is measured. Because the speed of light c is constant, the round-trip interval is enough to calculate the travel distance of the light, which is twice the gap between the surface, the scanner. If t is the round-trip interval, then the distance can be calculated by c × t/2 . The exactitude of a time-of-flight 3D laser scanner relies on how accurate is the measured time t : 3.3 picoseconds (approx.) is the time needed for light to move 1 millimeter.

FIGURE 3 . 1 :

 31 FIGURE 3.1: Function of Time-of-flight 3D laser scanner

  Lately, there have been studies on counterbalancing for distortion from tiny amounts of vibration and distortions due to movement and rotation . When examining in one area for any length of time slight change can happen in the machine position due to variations in temperature. If the machine is arranged on a tripod and there is strong daylight on one view of the device, then that side of the tripod will expand and slowly distort the obtained information from one view to another. Some laser machines have scale compensators included inside them to offset any motion of the device during the scan time.

FIGURE 3 . 2 :

 32 FIGURE 3.2: Principle of a laser triangulation sensor.

  location can be determined by the device employing indicating points on the surface or by utilizing an external tracking technique. External tracking usually takes the appearance of a laser tracker (to give the device location) with integrated camera (to determine the direction of the device) or a photogrammetric method employing 3 or more cameras implementing the entire six degrees of freedom of the device.Both methods conduce to utilize infrared light-emitting diodes associated with the machine which are observed by the camera(s) through filters giving resilience to ambient lighting.Information is retrieved by a machine and registered as points cloud in threedimensional coordination, with processing this can be reformed into a triangulated mesh and then a computer design model, usually as non-uniform rational B-spline surfaces. Hand-held laser devices can join this information with passive, visiblelight cameras which capture an object's textures and colors to reconstruct a complete 3D model.Structured-light 3D scanner : emit a pattern of light beams on the object and study the deformation of this pattern. The pattern is projected onto the object employing either an LCD projector or another constant light source. A camera, offset slightly from the projector device, studies the appearance of the pattern and measures the distance of every dots in the range of view.Structured-light scanning is still a very active area of research with many research papers published each year. Perfect maps have also been proven useful as structured light patterns that solve the correspondence problem and allow for error detection and error correction.The advantage of structured-light 3D scanners is speed and precision. Instead of scanning one point at a time, structured light scanners scan multiple points or the entire field of view at once. Scanning an entire field of view in a fraction of a second reduces or eliminates the problem of distortion from the motion. Some existing systems are capable of scanning moving objects in real-time. VisionMaster creates a 3D scanning system with a 5-megapixel camera so 5 million data points are acquired in every frame. A real-time scanner utilizing digital fringe projection and phase-shifting method (certain types of structured light techniques) was developed, to catch, build, and render high-density features of a dynamically deformable subject (like facial expressions) at the speed of 40 frames every second. Lately, another device has been developed. Various patterns can be used to this method, and the frame rate for capturing and data processing reaches 120 frames every second. It is also able to examine isolated surfaces, for example, two swaying hands. By using the binary defocusing method, speed breakthroughs could achieve millions of Chapter 3. 3D Reconstruction frames every tens second.

FIGURE 3 . 3 :

 33 FIGURE 3.3: Fringe pattern recording system with 2 cameras (avoiding obstructions)

Chapter 3 .

 3 3D ReconstructionDouble reflections and inter-reflections can cause the stripe pattern to be overlaid with unwanted light, entirely eliminating the chance for proper detection. Reflective cavities and concave objects are therefore difficult to handle. It is also hard to handle translucent materials, such as skin, marble, wax, plants and human tissue because of the phenomenon of the subsurface scattering. Recently, there has been an effort in the computer vision community to handle such optically complex scenes by redesigning the illumination patterns. These methods have shown promising 3D scanning results for traditionally difficult objects, such as highly specular metal concavities and translucent wax candles.

FIGURE 3 . 4 :

 34 FIGURE 3.4: Example of Shape From Shading ambiguities

FIGURE 3 . 5 :FIGURE 3 . 6 :

 3536 FIGURE 3.5: Synthetic data generated using OpenGL to verify light calibration

FIGURE 3 . 7 :

 37 FIGURE 3.7: Photometric stereo's schema

FIGURE 3 . 8 :

 38 FIGURE 3.8: Structure from motion's principles

FIGURE 3 . 9 :

 39 FIGURE 3.9: Gray-scale and iron-palette version

3. 3 . Scheme 53 a

 353 first model which is called ST 2 (ST 1 do not exist).

FIGURE 3 . 10 :

 310 FIGURE 3.10: 3D mesh and camera's position

•

  Matching: Pixel-level correspondences of point cloud is computed to enhance a portion of features points. Features found by Harris and difference-of-Gaussians operators are first matched across multiple pictures, yielding a sparse set of patches associated with salient image regions • Expande: Initial matches is spread to nearby pixels to obtain a denser cloud of points.

3. 3 . Scheme 55 FIGURE 3 .

 3553 FIGURE 3.11: point cloud (left), dense vertex (middle), surface (right)

FIGURE 4 . 1 :

 41 FIGURE 4.1: Flowchart of the whole proposed detection process

FIGURE 4 . 2 :

 42 FIGURE 4.2: Camera movements during authentication process

Figure 4 .

 4 Figure 4.3 gives an example of a real face reconstructed in the form of point cloud.

FIGURE 4 . 3 :

 43 FIGURE 4.3: Point cloud of a real face 3D model

Figure 4 .

 4 [START_REF] Sirovich | Low-Dimensional Procedure for the Characterization of Human Faces[END_REF] shows different views of the 3D reconstruction of a printed face. It is easy to realize that the form of the 3D reconstruction is flattering in the case of photo attack. It can be explained by the fact that a real face is a real 3D object which contains much more depth information than a face printed on a piece of paper.

FIGURE 4 . 4 :

 44 FIGURE 4.4: Different views of a printed face 3D model

Figure 4 .

 4 5 (a) and (b) give an illustration of the three principal components obtained respectively from fake and real face 3D models.

FIGURE 4 . 5 :

 45 FIGURE 4.5: PCA of real (a) and fake (b) face 3D reconstruction

FIGURE 4 . 6 :

 46 FIGURE 4.6: Depth image of a face

  sifier which is pondered by the weight of each sample in training set. By this way, each time the algorithm choses a feature as efficient feature, it will update the weight of all the samples so that the incorrectly classified samples become more important for next iterations. Instead of one image, our examples become any couple of images possible. The database includes 1001 videos of 3 people including sensors data, therein: 451 cases of legitimate authentication, 362 cases of video-replay attack and 188 cases of photo attack. However, in the training set we used only 100 cases of legitimate authentication, 50 cases of video-replay attacks and 50 cases of photo attack which makes 7400 intra-case couples and 12500 extra-case couples.

• 3 .

 3 P = set of positive examples.

• 4 .

 4 N = set of negative examples.

1 *

 1 Use P and N to train a classifier with n l features using AdaBoost * Evaluate current cascaded classifier on validation set to determine F l and D l . * Decrease threshold for the l-th classifier until the current cascaded classifier has a detection rate of at least d × D l-1 (this also affects F l ) -N ← ∅ -If F i > F target then evaluate the current cascade detector on the set of nonface images and put any false detections into the set N The output of our experiment is a cascaded strong classifier with 12 layers and 108 features.

FIGURE 4 . 7 :

 47 FIGURE 4.7: General schema of video attack detection.

Figure 4 .

 4 [START_REF] Lin | Face recognition/detection by probabilistic decision-based neural network[END_REF] shows an example of the camera's positions estimated from the 3D reconstruction.

FIGURE 4 . 8 :

 48 FIGURE 4.8: Example of the camera's positions estimated from the 3D reconstruction. Direction of the camera's move is marked form 1 to 8.

FIGURE 4 . 9 :

 49 FIGURE 4.9: Orientation of camera described by gyroscope sensor (blue for θ x i , red for θ y i and green for θ z i ,

FIGURE 4 . 10 :

 410 FIGURE 4.10: Correlations between θ x i (black)and θ x i (red) and between θ y i (black) and θ y i (red).

4. 3 .

 3 Result and Evaluationthe 3D reconstruction phase unstable. So in a real use case, the process asks naturally for another capturing phase. This option ameliorates the process performance significantly 4.11.

FIGURE 4 .

 4 FIGURE 4.11: ROC curve for the proposed detection method (Red) in comparison with the one of LBP method(blue). The proposed method with recapturing option is displayed in yellow.

FIGURE 5 .

 5 FIGURE 5.1: Framework

FIGURE 5 . 2 :

 52 FIGURE 5.2: Preprocessing

FIGURE 5 . 3 :

 53 FIGURE 5.3: Gray-scale and iron-palette version

8 )

 8 With color video a RGB matrix M c can be associated withM M c =    r 1 r 2 ... r N g 1 g 2 ... g N b 1 b 2 ... b N    (5.9)where N is the number of feature points and (x k , y k , z k ) (k = 1, ..., N) are the Chapter 5. Face Recognition by Thermal Video Using Vesselness Features in Multiview 3D Projections coordinates of the k-th point in the space Oxyz. (r k , b k , g k ) (k = 1, ..., N) represent the pseudo-color (which is computed from intensity) of this point. The obtained point cloud, which is neither dense nor periodic, must be improved using the Patch-based Multi-view Stereo (PMVS) developed by Yasutaka Furukawa and using the Poisson Surface Reconstruction studied by Michael Kazhdan . By this supplement process, the set of point is transformed into a dense collection of small oriented rectangular. The algorithm of PMVS can be decomposed in 3 steps: • Matching: Pixel-level correspondences of point cloud is computed to enhance a portion of features points. Features found by Harris and difference-of-Gaussians operators are first matched across multiple pictures, yielding a sparse set of patches associated with salient image regions • Expande: Initial matches is spread to nearby pixels to obtain a denser cloud of points.

FIGURE 5 . 4 :S = λ 2 1 + λ 2 2 ( 5 . 10 )

 5422510 FIGURE 5.4: point cloud (left), dense vertex (middle), surface (right)

FIGURE 5 . 5 :

 55 FIGURE 5.5: 3D vascular network model

FIGURE 5 . 6 :

 56 FIGURE 5.6: Depth image of a face

FIGURE 5 . 7 :

 57 FIGURE 5.7: Elliptic mask (left), cropped intensity image(middle) and cropped vessel image(right)

  p h ∈ 1, ..., N are the points inside a pixel (h ∈ 0, .., N). Assume that the depth measure follows a linear relation in u-axis and v-axis, it can be represented by a local function D(u, v): D(u, v) = c 0 + c 1 u + c 2 v + c 3 uv (5.15) Chapter 5. Face Recognition by Thermal Video Using Vesselness Features in Multiview 3D Projections where c 0 , c 1 , c 2 , c 3 are 4 coefficients to be determined.

FIGURE 5 . 8 :

 58 FIGURE 5.8: Learning and testing phase of classifier

Chapter 5 .

 5 Face Recognition by Thermal Video Using Vesselness Features in Multiview 3D Projections magnitude features and Gabor phase features. In this study, only Gabor magnitude features are used to describe the face. For 40 Gabor kernel, 40 Gabor image can be computed/ Each image is an ellipse of size 128x160 which includes about 15900 features, each feature has two dimensions (one dimension for the depth and another

  k=v max ∑ k=1 |w k,p | (5.25)

FIGURE 5 . 9 :

 59 FIGURE 5.9: LDA template for Gabor image with u = 4 and v = 8

1 *

 1 Use P and N to train a classifier with n l features using AdaBoost * Evaluate current cascaded classifier on validation set to determine F l and D l . * Decrease threshold for the l-th classifier until the current cascaded classifier has a detection rate of at least d × D l-1 (this also affects F l ) -N ← ∅ -If F i > F target then evaluate the current cascade detector on the set of nonface images and put any false detections into the set N The output of our experiment is a cascaded strong classifier with 10 layers and 96 features.

  by using the selected template from training phase. The testing video is paired with each training video and generate 20 examples of couples. These 20 examples are classified into two classes: intra-personal and extra personal with a score of the similarity (S(V 0 , V k ) where k ∈ 1, 2, ...20) for two videos in each couple:

FIGURE 5 . 10 :

 510 FIGURE 5.10: Roc curve of the strong classifier.

  1) prove the advantage of mixing intensity and depth information in feature level. The same concept can be used with any other descriptor like LBP or Weber local descriptors. The different between the performance of three tests is reduced when the number or training videos increases (for training 5.4. Multi-pose recognition 91

  FIGURE 5.12: Normalized profile view.

FIGURE A. 6 :

 6 FIGURE A.6: Image de profondeur d'un visage

FIGURE A. 8 :A. 3 . 3

 833 FIGURE A.8: Orientation de l'appareil photo à partir des données du capteur gyroscopique (θ x i en bleu, θ y i en rouge, θ z i en vert)

FIGURE A. 12 :

 12 FIGURE A.12: Différentes applications.

  Extraction d'informations caractéristiques: La comparaison directe de deux photographies numériques est fortement sensible aux petits changements des conditions d'acquisition. C'est pourquoi il est nécessaire d'extraire des informations utiles qui permettent de différencier les images. En raison de leur nature différente, les caractéristiques extraites dans chacune de modalités d'imagerie (visible ou infrarouge) sont différentes. Par exemple, des détails du visage peuvent ne pas apparaitre dans l'image thermique si la température de surface reste la même.

FIGURE A. 13 :

 13 FIGURE A.13: Réseau vasculaire d'une image thermique

FIGURE A. 16 :A. 5 Conclusion

 165 FIGURE A.16: Projection du réseau vasculaire dans le modèle 3D pour personne B.

FIGURE A. 18 :A. 6 Perspectives

 186 FIGURE A.18: Caméra ThermApp couplée avec un smartphone

  Chapter 2. Literature Review on support vector machine. Mid-level feature extractors were used rather than lowlevel feature extractors. This suggested method was realized on databases of all types such as photos, video, and 3D mask database and the achieved results were a 0.6% error rate on different databases.

	Akshay Agarwal et al. [54] introduced a novel strategy using hard lick method
	on Face antispoofing. The technique yielded an error rate of 1.1% on video database.
	The researcher assessed the method on Replay Attack database, CASIA database,
	NUAA Photograph Imposter and Cross database which generated an error rate of
	1.67%. The obtained results display an improvement in recognizing the different
	spoof attacks as compared to other methods.

Allan Pinto et al.

[START_REF] Pinto | Face Spoofing Detection Through Visual Codebooks of Spectral Temporal Cubes[END_REF]

suggested a face spoofing technique based on visual codebooks of spectral-temporal cube methods. This texture-based technique was based

TABLE 2 .

 2

			1: Existing methods comparing.
	Author	Methods		Attacks		Database		Accuracy
	Pan et al.	Eyeblink Detection using	Photo		Public	Database	Error Rate= 2%
	(2007)	conditional random field			and NUAA Pho-
		(CRP)				tograph Imposter
						Database
	Kollreider et	Motion Detection	Photo and	Proprietary	Error Rate= 3.5%
	al. (2008)			Video
	Kollreider et	Face motion Detection us-	Photo		Proprietary	Error Rate= 0.5%
	al. (2009)	ing optical flow of lines
	Tan et al.	Face texture using the	Photo		Public	Database	Error Rate= 15%
	(2010)	Lambertian model			and NUAA Pho-
						tograph Imposter
						Database
	Anjos et al.	Context-Based using cor-	Photo		Public	Database	Error Rate= 10%
	(2011)	relation between face mo-			and Print Attack
		tion v/s background mo-			Database
		tion
	Zhang et al.	Reflectance using multi-	Photo, Video	Public	Database	Error Rate= 7%
	(2011)	spectral lighting in 2D im-	and	3D	and Print Attack
		ages		Masks		Database
	Chingovska	Face Texture using Local	Photo and	Public Database,	Error Rate= 15%
	et al. (2012)	Binary Pattern (LBP)	Video		Replay	Attack
						Database, NUAA
						Photograph	Im-
						poster	Database
						and CASIA Face
						Antispoofing
						Database
	Komulainen	Context based using up-	Photo and	Public Database,	Error Rate= 3%
	et al. (2013)	per body & spoof support	Video		NUAA	Photo-
		Detection				graph	Imposter
						Database		and
						CASIA Face Anti-
						spoofing Database
	Nesli Erdog-	Modified Local Binary	Photo and	Morpho Database	Error Rate= 3%
	mus et al.	Pattern (MLBP) + Linear	3D Mask		and 3D Mask At-
	(2014)	Discriminant	Analysis			tack Database
		(LDA) + Support Vector
		Machine (SVM)
	Santosh	Principal	Component	Photo and	Print	Attack	Error Rate= 9.5%
	Tirunagari	Analysis (PCA) + Local	Video		Database, Replay
	et al. (2015)	Binary Pattern (LBP) +			Attack	Database
		Support Vector Machine			and CASIA-FASD
		(SVM)				Database

TABLE 2 .

 2

		2: Existing methods comparing.
	Author	Methods	Attacks		Database	Accuracy
	Shervin	Spectral Regression ker-	Photo and	Replay	Attack	Error Rate= 1.67%
	Rahimzadeh	nel discriminant analysis	Video		Database, CASIA
	Arashloo et	(SR-KDA)			Face	Antispoof-
	al. (2015)				ing	Database,
					NUAA	Photo-
					graph	Imposter
					Database	and
					Cross-Database
					Evaluation
	Mihai	Neural Network (NN)	Photo and	Honda/UCSD	Error Rate= 5.5%
	Gavrilescu	+ Principal Component	Video		Video	Database,
	et al. (2015)	Analysis (PCA)			Youtube	Faces
					Database, Replay
					Attack Database,
					NUAA	Photo-
					graph	Imposter
					Database	and
					CASIA Face Anti-
					spoofing Database
	Allan Pinto	Partial Least Square (PLS)	Photo, Video	Replay	Attack	Error Rate= 0.6%
	et al. (2015)	and Support Vector Ma-	and	3D	Database, CASIA
		chine (SVM)	Masks		Face Antispoofing
					Database, UVAD
					Database, 3DMAD
					Database
	Zinelabidine	YCbCR + HSV	Photo and	Replay	Attack	Error Rate= 0.4%
	Boulkenafet		Video		Database, CASIA-
	et al. (2016)				FASD	Database
					and MSU mo-
					bile face spoof
					Database
	Akshay Ag-	Haralick Texture Features	Photo and	3DMAD Database,	Error Rate= 1.1%
	garwal et al.	+ Discrete Wavelet Trans-	Video		CASIA-FASD
	(2016)	formed + Principal Com-			Database	and
		ponent Analysis + Sup-			MSU-MFSD
		port Vector Machine			Database

TABLE 2 .

 2

		3: Infrared sub-bands comparing.
		NIR	SWIR	MWIR	LWIR
	Wavelength	0.75 -1.4 µm 1.4 -3 µm 3 -8 µm 8 -15 µm
	Illuminator	+	+	-	-
	Details	++	+	-	-
	Lighting Invariance	-	-	+	++
	Type	Reflected	Reflected Thermal Thermal
	higher temperature (eyes, lips) However, in some particular case, these monochro-
	matic infrared images can be shown in pseudo-color where different colors represent
	the change in intensity. This technique favors the human vision of which color con-
	ception can be easier to distinguish different intensities. The pseudo-color is not
	absolute but relative, that means in different images, one temperature may be repre-
	sented by different colors depending on the temperature range of the object and the
	background.

TABLE 5 .

 5 1: Average results after repeating 20 times the process of experimentation.

		Accuracy Precision Recall
	Gabor-Intensity only	79.43%	77.62%	81.76%
	Gabor-Decision mix	82.64%	80.12%	85.21%
	Gabor-Feature mix	88.43%	87.05%	90.83%
	LBP-Intensity only [? ]	86.11%	85.05%	87.70%
	LBP-Mix	87.36%	88.55%	88.79%

  90• views are normally the ultimate views this process can get, but they are also the most imprecise. To bias between getting more information and resting accurate, the profile views are projected at 60 • . Firstly, the 3D model will be rotated by 60 • and -60 • .

	Chapter 5. Face Recognition by Thermal Video Using Vesselness Features in
		Multiview 3D Projections
	M left = θ le f t × M	(5.29)

  Chapter 5. Face Recognition by Thermal Video Using Vesselness Features in

									Multiview 3D Projections
					Multi-view vs frontal-view
	1
										Multi-views
	0.9									Mono-view
	0.8
	0.7
	0.6
	0.5
	0.4
	0.3
	0.2
	0.1
	0
	0	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1
	FIGURE 5.13: Roc curve of the strong classifier for mono-view and
					multi views.
		TABLE 5.2: Average results for 20 test sets.
				Accuracy Precision		Recall
		Mono-view		88.43%		87.05%		90.83%
		Multi-views		92.27%		89.25%		93.231%

Acknowledgements

This work has been carried out within the laboratory of Systems Modeling and Dependability (M2S) at University of Technology of Troyes (UTT). It is funded by Troyes Champagne Métropole through the Graduate School of Engineering of EPF. This work has been accomplished under the supervision of Mr Florent RETRAINT and Miss Cathel ZITZMANN. I would like to express my deepest appreciation to them for their highly professional guidance and incessant support. Florent has accompanied me from my master's intership at UTT and encouraged me to continue the research in the field.

Chapter 5. Face Recognition by Thermal Video Using Vesselness Features in Multiview 3D Projections Each iteration, the algorithm searches for a feature that minimize l'error of classifier which is pondered by the weight of each sample in training set. By this way, each time the algorithm choses a feature as efficient feature, it will update the weight of all the samples so that the incorrectly classified samples become more important for next iterations. In this stage, the AdaBoost selection is used twice as following:

• Individual learning: apply AdaBoost method to each Gabor double image to select the effective feature for each image (about 30-38 per image) and group all these features into one set.

• Total learning: apply AdaBoost to this set of features to reduce one more time the number of features (about 127 features in this case).

The final step of training phase is the construction of a cascaded strong classifier from these 127 features. That cascaded classifier contains many layers, each layer is also building by the efficient features in features learning stage. In fact, instead of constructing a big classifier of a lot of features in order to achieve a detection rate D and limit the false positive rate under F, the method aims to build some small independent classifiers that provide a higher detection rate d l with a huge false positive rate f l . When these classifiers are use as layers for a bigger one we can choose the layer so that:

where L is the number of layers. In this way, from 10 classifiers with high false positive rate (by 50%), cascaded classifier can be building which limits F at 0.5 10 ≈ 10 -3 . The algorithm of Adaboost learning for strong classifier can be introduced as below:

• 1. Parameters initialization: selecting the value of f (the maximum acceptable false positive rate per layer) and d (the minimum acceptable detection rate per layer). This step depends essentially on the efficiency of features

Chapter 6. Conclusions and Perpectives

The delicate mask attack is not readily revealed by visible imagery technology because its imprint is very close to the genius face. However, in thermal imagery where the emission source of the spectrum is human's face, the detection of all types of face-spoofing attack is trivial. The thermal imagery technology can solve the other major problem of visible imagery concerning illumination conditions. Though, in general, thermal images provide less information than visible images. In our second study, we aim to improve the performance of infrared face-recognition method by using a 3D model of the head computed from a thermal video. Vascular network build from the thermal video is now observed in intensity level and geometric feature. The depth information and blood vessels data aren't handled like unchained marks but are associated to form a single feature with two values. By this way, the face recognition method bases mainly on the 3D position of the vascular network.

Perpectives

For the face-spoofing detection method which is introduced in chapter 4, there is always a type of attack which is not considered in the study: the 3D mask attack.

Until this step, our face spoofing detection is independent of the face recognition process. We do not use the same database as authentication system but a database dedicated to our solution. In fact, the geometric information can be used further in face recognition phase which makes 3D mask attacks detectable in this layer. However, this type of solution requires enough images of each user in different positions in the database to compare with the attempt. Another solution is to study the nature of the object's material to distingue the genius face from the attacks. In fact, each type of material generates a special imprint of noise which can be used to detect if the object is made by human's skin.

The thermal face recognition method comes all the way to construct a 3D model of the vascular network. However, we do not make direct use of this 3D model but its projection into a set of depth images. The process with multi-pose depth-images augment our method's performance but cannot achieve an entire comparison of 3D model. In some future works, we aim to construct a method that can study directly these 3D model in order to make uses of all the provided information.

Appendix A

Résumé en français

où n est le nombre de points du nuage, m i = (x i , y i , z i ) avec i = 1, .., n est le vecteur de coordonnées du i-ème point.

Pour simplifier l'ACP, les colonnes de la matrice M sont centrées pour avoir une moyenne nulle. La matrice de composantes principales P est définie comme une transformation linéaire orthogonale de la matrice M : P = MW.

Appendix A. Résumé en français Soit v j (j = 1, 2, 3) la variance de la i-ème colonne de P. L'ordre de grandeur de chaque colonne, noté d i , est donné par :

A.4.1 Problématique