Biometrics for face skin analysis using machine learning based approaches

Keywords: Skin disease classification, Face analysis, Deep learning, Convolutional neural network, VGG-16, EfficientNet b0, Inception V3, Medical biometrics, Dermatology Classification des maladies de la peau, Analyse du visage, Apprentissage profond, Réseau de neurones convolutifs, VGG-16, EfficientNet b0, Inception V3, Biométrie médicale, Dermatologie. Acne images Predicted class .1791% Image 2 Acne: 98.9879% Rosacea: 63.3976% Image 3 Acne: 99.7595% Angioedema: 99.1347% Image 4 Acne 0000% Acne: 99.7977% Class: Actinic keratosis Actinic images Predicted class (FSDNet) Predicted class Actinic: 99.9976% Actinic: 96.9296% Image 2 Actinic: 99.8859% Rosacea: 99.4904% Image 3 Rosacea: 99.9820% Actinic: 99.5998% Image 4 Acne: 48.0390% Actinic: 67.7229% Image 5 Actinic: 70.1654% Rosacea: 56.9207% Image 6 Actinic: 100.0000% Rosacea: 99.0967% Image 7 Actinic: 98.6569% Actinic: 54.1970% Image 8 Actinic: 88.5610% Actinic: 99.7013% Image 9 Actinic: 99.6006% Rosacea: 77.0670% Image 10 Actinic: 99.9916% Actinic: 94.7248% Class: Angioedema Angioedema images Predicted class(FSDNet) Predicted class(Pretrained VGG16) 5447% Image 3 Angioedema: 99.2950% Angioedema: 73.3742% Image 4 Angioedema: 99.9999% Normal: 51.7840% Image 5 0000% Melasma: 76.0969% Image 7 Angioedema: 99.9995% Angioedema: 94.6091% Image 8 Angioedema: 99.9820% Melasma: 76.5969% Image 9 Angioedema: 100.0000% Angioedema: 74.4274% Image 10 Angioedema: 100.0000% Melasma: 73.3384% Class: Eczema Eczema images Predicted class(FSDNet) Predicted class(Pretrained VGG16) Image Eczema: 99.1136% Eczema: 78.9951% Image Eczema: 95.0109% Eczema: 92.2045% Image Eczema: 93.7058% Vitiligo: 54.9187% Image Eczema: 94.3913% Acne: 44.0073% Image Eczema: 99.9118% Acne: 40.5535% Image Eczema: 100.0000% Eczema: 77.2535% Image Eczema: 98.6569% Rosacea: 52.0459% Image Eczema: 98.5610% Acne: 86.4012% Image Acne: 25.6425% Eczema: 77.5904% Image 10 Acne: 44.2101% Eczema: 75.4175% Class: Melasma Melasma images Predicted class(FSDNet) Predicted class(Pretrained VGG16) Image 1 Melasma: 91.6315% Melasma: 94.9185% Image 2 Melasma: 100.0000% Rosacea: 89.8135% Image 3 Melasma: 100.0000% Melasma: 98.3580% Image 4 Melasma: 100.0000% Melasma: 99.8119% Image 5 Melasma: 100.0000% Acne: 33.4502% Image 6 Melasma:99.9371% Acne: 45.8840% Image 7 Melasma: 83.7741% Normal: 53.7066% Image 8 Melasma: 100.0000% Melasma: 96.8072% Image 9 Melasma: 97.7482% Melasma: 75.6706% Image Melasma: 99.8925% Melasma: 75.6920% Class: Actinic keratosis Actinic images Predicted class with fine-tuned EfficientNet b0 Actinic: 96.9876% Image 2 Rosacea: 70.5007% Image 3 Rosacea: 98.6781% Image 4 Actinic: 99.9941% Image 5 Actinic:74.1771% Image 6 Actinic: 95.6463% Image 7 Rosacea: 99.4783% Image 8 Acne: 44.4715% Image 9 Actinic:99.7310% Image 10 Actinic: 97.9344% Class: Angioedema Angioedema images Predicted class with fine-tuned EfficientNet b0 .2552% Image 3 Angioedema: 88.1840% Image 4 Angioedema: 98.0572% Image 5 Angioedema:97.2835% Image 6 Angioedema: 75.4522% Image 7 Angioedema: 99.8112% Image 8 Angioedema: 98.8096% Image 9 Angioedema:76.8689% Image 10 Angioedema: 75.5810%

The emergence of artificial intelligence (AI), access to large databases, and the availability of supercomputers have undoubtedly revolutionized the medical treatment in various fields. In fact, taking into consideration the development of "Machine-learning" (ML) algorithms and in particular "Deep-learning" (DL), has greatly benefited the biomedical field and added to its efficiency. In the context of dermatology, much research has been carried out to automatically analyze images of the skin in order to predict diseases and to follow their evolution over time.

This thesis continues this present research and proposes a computer-aided diagnostic system based on "Deep-learning" (DL) approaches that analyzes facial images and identifies potential facial diseases using only facial phenotypes, without region of interest extraction. This medical, facial biometrics is based on the use of pretrained convolutional neural networks, VGG-16, EfficientNet b0 and Inception v3, which are fine-tuned to create new models adapted to classify facial skin images into eight distinct pathologies namely: acne, actinic keratosis, angioedema, blepharitis, eczema, melasma, rosacea and vitiligo. Thus, a transfer learning method has been used. Specifically, the original architectures of the three models have been changed by adding new layers to the top. In this study, the proposed algorithms are trained and validated on a database that we have created for this purpose. The models are tested and evaluated considering different acquisition conditions (facial pose, brightness, image resolution, etc...). Very promising results have thus been obtained and will be discussed in detail in this dissertation.

Résumé substantiel

L'intelligence artificielle (IA), l'apprentissage automatique et l'apprentissage profond sont des termes interconnectés et très répandu. L'intelligence artificielle est la capacité des machines à effectuer des tâches comme les êtres humains. L'apprentissage automatique est un sous-domaine de l'intelligence artificielle qui automatise l'apprentissage des ordinateurs à partir de données et améliore leurs performances grâce à l'expérience et en considérant la moindre intervention humaine. Par ailleurs, l'apprentissage profond est un sous-domaine de l'apprentissage automatique, principalement basé sur des réseaux de neurones artificiels qui apprennent à partir d'énorme quantité de données et imitant ainsi le fonctionnement du cerveau humain en termes de traitement d'informations. L'intérêt est évidemment de prendre des décisions ou reconnaître des modèles. Les applications de l'apprentissage profond ont révolutionné la technologie utilisée dans les différents domaines tels que : les affaires, l'éducation, la fabrication, la banque, l'armée, la santé et bien d'autres.

En particulier, d'énormes investissements ont été entrepris dans le domaine médical afin d'apporter des solutions adaptées à des techniques de soin. Ainsi, l'intégration de l'apprentissage profond dans les algorithmes d'aide au diagnostic est devenue une pratique courante. Les différentes modalités sont ainsi considérées pour le traitement de bio-signaux (e.g. EEG, ECG, EMG, etc), ainsi que pour le traitement d'images médicales (e.g. échographiques, RMN, etc). Pour ces dernières, la caractérisation, la segmentation et la classification automatique ont permis d'améliorer considérablement les performances des logiciels. En particulier, la classification des images est l'une des tâches les plus importantes de la vision par ordinateur. Elle Dans le contexte de notre sujet de recherche, l'apprentissage profond a fait l'objet d'exploration dans des applications liées à la dermatologie, en allant du diagnostic au traitement. Ainsi, de nombreuses études ont été menées pour développer des approches automatisées d'analyse d'images de la peau humaine afin de prédire et de classer les pathologies, d'évaluer et de suivre leur évolution dans le temps.

Dans ce contexte, l'objectif de cette recherche est de développer un système de diagnostic assisté, fondé sur l'IA afin d'identifier les lésions cutanées du visage en utilisant différentes approches d'apprentissage profond, et en particulier les modèles CNN. Huit pathologies cutanées du visage sont traitées dans cette étude. Il s'agit, en effet, de pathologies les plus courantes affectant les personnes à tout âge et tout genre. En particulier, nous nous sommes intéressés aux pathologies suivantes : l'acné, la kératose actinique, l'angioedème, la blépharite, l'eczéma, le mélasme, la rosacée et le vitiligo. Le protocole considéré utilise des techniques de pré-traitement issues de la biométrie faciale. L'analyse est effectuée indépendamment des conditions d'acquisition telles que la résolution de l'image, l'éclairage, l'occultation ainsi que la pose du visage. Nous avons donc construit un nouveau modèle basé sur l'apprentissage par transfert en utilisant le VGG-16, adapté à l'identification des maladies de la peau du visage, que nous avons appelé "Facial Skin Diseases Network" (FSDNet). Notre système peut également prédire une classe de "Peau saine" et une classe de "visage-absent", notamment si l'image présentée ne représente pas un visage. L'efficacité deFSDNet est évaluée en comparant ses performances avec celles obtenues par d'autres architectures CNN telles que le VGG-16 pré-entraîné, et l'apprentissage transfert utilisant Inception V3, et l'apprentissage par transfert utilisant EfficientNet b0. L'apprentissage par transfert est appliqué avec la même architecture proposée par FSDNet afin de rendre la comparaison significative. Les principales contributions de notre étude se résument comme suit : (1) identification des pathologies sans extraction locale de la région d'intérêt relative à la texture de la peau (modèle entraîné indépendamment de la résolution de l'image, de la pose et de l'illumination, etc). ( 2) Un plus grand nombre de pathologies est identifié par rapport aux techniques les plus récentes, publiées jusqu'ici.

En raison de l'absence d'un ensemble de données publiques standard, nous avons créé une base de données contenant 20 000 images étiquetées contenant : des visages présentant huit maladies cutanées, des visages à la peau normale, et enfin des images présentant des objets et des scènes variantes (e.g. images de nourriture, de tasses, d'arbres, etc.) collectées de différentes sources et utilisées pour entraîner et valider nos réseaux.

Ce rapport est divisé en quatre chapitres. Le premier chapitre se concentre sur l'état de l'art des méthodes d'identification des maladies cutanées du visage. Nous décrirons également les différentes maladies qui seront classées par l'approche que l'on a développée. Nous introduisons ainsi le fondement de notre approche d'identification basée sur l'apprentissage profond ayant pour objectif de classer plusieurs pathologies cutanées du visage (i.e. acné, kératose actinique, angioedème, blépharite, eczéma, mélasma, rosacée et vitiligo).

Dans le deuxième chapitre, nous passons en revue, le principe de l'apprentissage profond. Ainsi, nous présentons le concept de " hiérarchie des caractéristiques " qui fait de l'apprentissage profond un outil performant et adapté aux applications de vision par ordinateur. Nous présentons ensuite les différentes architectures des réseaux de neurones profonds, en mettant en évidence les réseaux de Dans le quatrième chapitre, nous allons élaborer une méthode complète de bout en bout qui permet d'identifier les pathologies cutanées du visage dans les images RVB en utilisant l'apprentissage profond et les techniques de vision par ordinateur. Afin d'accomplir cette tâche, nous utiliserons l'apprentissage par transfert avec les trois modèles pré-entraînés VGG-16, Inception v3 et EfficientNet b0 pour créer nos modèles adaptés à ce contexte. Dans la suite du chapitre, nous détaillerons l'approche que nous proposons tout en mettant en évidence les résultats que nous avons obtenus pour chaque modèle en termes de précision, d'exactitude et d'autres métriques. Nous démontrons ainsi que l'apprentissage par transfert peut améliorer considérablement la précision de l'identification des lésions cutanées du visage. Enfin, une conclusion générale et des perspectives seront présentées à la fin de ce rapport.

Acknowledgement

I would like to express my deep and sincere gratitude to my research supervisor, Professor Amine NAIT-ALI, for giving me the opportunity to do research and providing invaluable guidance throughout this research, for his patience, motivation, and immense knowledge. He has taught me the methodology to carry out the research and to present the research works as clearly as possible. It was a great privilege and honor to work and study under his guidance.

I would also like to give special thanks to Dr Sambit BAKSHI for his contribution in this thesis.

My sincere thanks go to all members of the jury for accepting to participate in the defense of this dissertation.

My appreciations also go to my family for their love, prayers, caring, sacrifices and support all through my studies. I am very thankful to my husband, my daughter, and my son for their love, continuing support, tremendous understanding, and encouragement.

List of publications

International journal

Rola EL SALEH, Samer CHANTAF and Amine NAIT-ALI. Identification of facial skin diseases from face phenotypes using FSDNet in uncontrolled environment. Machine Vision and Applications, 2021 (accepted for publication).

General introduction

Artificial intelligence, machine learning, and deep learning are the buzzword of this era. These popular and trendy terms are interconnected. Artificial intelligence is the ability of machines to perform tasks like humans. Machine learning is a subfield of artificial intelligence that automates the learning of computers from data and improves their performance through experience while considering the slightest human intervention. On the other hand, deep learning is a subfield of machine learning, mainly based on artificial neural networks which learn from huge amounts of data and thus mimics the functioning of the human brain in terms of processing information. The interest is obviously to make decisions or recognize models. Deep learning applications have revolutionized the technology used in different fields such as: business, education, manufacturing, banking, military, healthcare and many more.

In particular, huge investments have been made in the medical field in order to provide solutions adapted to care techniques. Thus, the integration of deep learning into diagnostic aid algorithms has become a common practice. The different modalities are thus considered for the processing of bio-signals (e.g. EEG, ECG, EMG, etc.), as well as for the processing of medical images (e.g. ultrasound, NMR, etc.). For the latter, characterization, segmentation and automatic classification have significantly improved software performance. In particular, the classification of images is one of the most important tasks of computer vision. It consists of analyzing images and classifying them in a category among a predefined set. Unlike traditional images classification techniques, deep learning algorithms combine the tasks of feature extraction and classification. In this context, many algorithms have been proposed. In particular, convolutional neural networks (CNNs) are the most popular deep learning algorithms used for image classification. These are networks inspired by the way the human brain recognizes information. CNNs eliminate the manual extraction of low and mid-level features. In the literature, several CNN models have been proposed to achieve high classification accuracy and in some cases can even exceed human recognition.

In the context of our research topic, deep learning has been explored in applications related to dermatology, from diagnosis to treatment. Thus, numerous studies have been carried out to develop automated approaches for analyzing images of human skin in order to predict and classify pathologies, and to evaluate and monitor their evolution over time.

Within this context, the purpose of this research is to develop an AI based aided diagnostic system to identify facial skin diseases using different deep learning approaches, and especially CNN models. Eight skin pathologies of the face are handled in this study. These are, in fact, the most common pathologies affecting people of all ages and genders. In particular, we were interested in the following pathologies: acne, actinic keratosis, angioedema, blepharitis, eczema, melasma, rosacea and vitiligo. The protocol considered uses pre-treatment techniques derived from facial biometrics. Analysis is performed regardless of acquisition conditions such as image resolution, lighting, shadowing, and face pose. So we built a new model based on transfer learning using VGG-16, suitable for identifying facial skin diseases, which we called "Facial Skin Diseases Network" (FSDNet). Our system can also predict a class of "normal skin" and a class of "no-face", especially if the image presented does not represent a face. The efficiency of FSDNet is evaluated by comparing its performance with that obtained by other CNN architectures such as the pre-trained VGG-16, and transfer learning using Inception V3, and transfer learning using Ef-ficientNet b0. Transfer learning is applied with the same architecture offered by FSDNet in order to make the comparison meaningful. The main contributions of our study are summarized as follows: (1) identification of pathologies without extraction of the region of interest relating to the texture of the skin (model trained independently of image resolution, pose and illumination, etc.). ( 2) A greater number of pathologies are identified compared to the most recent techniques published so far.

Due to the absence of any standard public dataset for the same, we established a database that contains 20000 labelled images belonging to the eight facial skin diseases and faces with normal skin in addition to no face images (images of food, cups, trees,etc..) collected from different sources, which is used to train and validate our networks.

This report is divided into four chapters. The first chapter focuses on the state of the art in methods of identifying skin diseases of the face. We will also describe the different diseases that will be classified by the approach that we have developed. We thus introduce the basis of our identification approach based on deep learning with the objective of classifying several skin pathologies of the face (i.e. acne, actinic keratosis, angioedema, blepharitis, eczema, melasma, rosacea and vitiligo).

In the second chapter, we review the principle of deep learning. Thus, we present the concept of "hierarchy of characteristics" which makes deep learning a powerful tool suitable for computer vision applications. We then present the different architectures of deep neural networks, by highlighting convolutional neural networks (CNN), by explaining the blocks and layers constituting the network, its different architectures, as well as its learning mode. Finally, we proceed to the use of CNNs for image classification.

In the third chapter, we discuss the architecture of the different VGG, Inception v3, and EfficientNet b0 networks, which we used as a basis to build our models adapted to the classification of facial skin pathologies. VGG is considered a general architecture and a benchmark for many tasks and datasets outside of ImageNet, the database on which the network is initially trained. Inception v3 is the third edition of CNN Inception from Google. While most popular CNNs are working on increasing network depth, Inception models use other techniques to improve network performance in terms of speed and accuracy. Many versions have been created, each version being an improvement on the previous one. The EfficientNet models, recently developed by Google, present a new concept of scaling networks to improve performance, called "Compound Scaling". Previous CNN models follow the classic method of arbitrarily scaling a single dimension and adding additional layers.

EfficientNet models uniformly scale all dimensions (i.e. depth, width and resolution) using a compound coefficient. They improve the accuracy and efficiency of the model. We will also explain how we can apply these pre-trained networks on ImageNet to classify other images into categories other than ImageNet.

In the fourth chapter, we will develop a complete end-to-end method that identifies facial skin pathologies in RGB images using deep learning and computer vision techniques. In order to accomplish this task, we will use transfer learning with the three pre-trained models VGG-16, Inception v3 and EfficientNet b0 to create our models suitable for this context. In the rest of the chapter, we will detail the approach we propose while highlighting the results we obtained for each model in terms of precision, accuracy and other metrics. We thus demonstrate that transfer learning can significantly improve the accuracy of identifying facial skin lesions.

Finally, a general conclusion and perspectives will be presented at the end of this report.

Chapter 1

Facial skin diseases: generalities

Introduction

Skin is the largest organ in the human body, covering the body's surface from head to toes. It protects the body against diseases, UV radiation, chemicals and microbes. It controls our body temperature, prevents loss of water, and transmits sensory stimuli such as heat, cold, and touch. The skin is composed of three layers: the epidermis, the dermis, and the hypodermis [START_REF] Youssef | Skin (Integument)[END_REF]. The epidermis is the outer, waterproof layer forming the body's first line defense against bacteria, germs, UV radiation and environmental factors. It provides the skin tone and is composed mainly from various types of cells. The dermis is the middle layer containing connective tissues which produce the proteins collagen and elastin that determine the structure, shape, and elasticity of the skin, hair follicles, sweat and oil glands, and blood vessels. The hypodermis is the deepest layer made up of fats, connective tissues, blood vessels and nerves. Actually, the thickness of the epidermis differs depending on what area of the body it is located. Facial skin is thinner than the skin on other parts of the body. Blood vessels, hair follicles, sweat and oil glands are concentrated in the face to protect and moisturize it and also to heal wounds and

Chapter 1

Facial skin diseases: generalities scars [2]. Like skin on the rest of the body, facial skin is affected by inflammatory, bacterial and viral diseases and also cancer. Facial skin diseases affect persons at all ages and from different geographies. They differ in symptoms, severity, and pain, and vary from rashes to severe infections happening due to, allergens, auto-immune system disorder, medications, infections or genetics. They can have physical and psychological impacts on the affected persons. Skin disorders are usually diagnosed by visual inspection. However, such naked-eye observation may result in a wrong diagnosis especially in diseases with similar symptoms, and thus the treatment is prolonged, and therefore the cost of treatment increases. To reduce the problem of missed diagnoses, computer-aided diagnosis systems were proposed to help dermatologists overcome the limits of human knowledge. Besides prevention of missed diagnoses, automated identification of diseases also ensures touchless diagnosis that protects physicians in case of contagious diseases, saves time because they can be used anytime and anywhere, solves the problem of lack of dermatologists in rural regions and even more reduces the cost where people can't bear the high costs of consultations.

In the remainder of this chapter, we will focus on state of the art facial skin diseases.

Image processing techniques previously proposed for facial skin diseases detection and classification will be reviewed, as well as our proposed approach. Then, we will describe different diseases that will be classified according to our method. Finally, the chapter will end in a conclusion.

State of the art facial skin diseases identification methods

In the past few years, facial skin diseases have been worthwhile as their procuration and complexities have increased. With the breakthrough in medical technology, the concept of computer-aided diagnostic systems for facial skin diseases have C.Y Chang et al. have proposed an automated method to detect facial spots and acne using support-vector-machine based classifier. Their method consists of detecting the face region from the input images using a skin color detection method, and the facial view (front view or profile) to extract the ROI that includes the eyes, the eyebrows, the mouth, and the nose. Once the ROI is extracted, they apply an image segment method to detect potential defects. The classification of the defects includes three stages. First, they compute 14 texture features from co-occurrence matrix: Contrast, Homogeneity, Mean, Variance, Energy, Entropy, Angular Second Moment, Correlation, Sum Average, Sum Entropy, Difference Average, Difference Variance and Difference Entropy. Then, they keep the most significant features that will be used in the classification. Finally, a support-vector-machine (SVM) classifier is used to classify the defects. It consists of two SVMs. The first SVM classifies the images as defects or normal skin, while the second one classifies the defects into acnes or spots. The system is evaluated using 147 images in 3 views: front, left profile, and right profile. The proposed approach detects the defects and recognizes acnes and spots with an average accuracy of 98.95% and 95.2% respectively [3].

Chantharaphaichit et al. have suggested a blob detection based method to detect

acne. It consists of detecting regions that have a circular shape called blobs, corresponding to acne. Then, they extract the features that will be used to detect acne. They also use Bayesian classifier following features extraction to minimize the misclassification. This method can detect acne with an accuracy of 70.6%, but is

Chapter 1

Facial skin diseases: generalities affected by many conditions such as the form of acne and lighting [4].

M. Amini et al. have developed a mobile application to detect acne and classify it

into papules and pustules using smartphone images. The application requires that the images should be acquired from the front. The entire face should be included in the frame and the eyes should be open. The images are first calibrated, and then the features of the face images are identified to detect the human face in the image using facial recognition algorithm. The face images are then normalized to guarantee that the acne is identified on the same region, even with distance and face position changes toward the camera, and the ROI is extracted to detect acne. Actually, acne causes skin redness with comparison to the skin around it. This change of color is used to detect acne. Once identified, acne lesions are classified into papules or pustules. To assess their approach, 10 real human images, and 60 digital face images where lesions are placed virtually are used. The system could detect acne lesions with an accuracy of 92%, and predicts the type of lesion with an accuracy of 98% [START_REF] Amini | Automated facial acne assessment from smartphone images[END_REF].

An automatic skin disease prediction method based on image processing and machine learning has been proposed by L. Bajaj et al. The proposed approach comprises two phases. In the first phase, color face skin images undergo a series of image processing techniques to detect and extract the affected region such as RGB extraction, grey scale, sharpening filter to improve the accurate details of the grey scale image, median filter to remove all noise from the image, smooth filter, and binary filter which is the basis of extraction of the affected region. The extracted diseased region is then converted to a feature vector that is fed to an artificial neural network which is a machine learning algorithm. The network is evaluated using a dataset of 813 images referring to the 5 diseases to predict (eczema, psoriasis, impetigo, melanoma, and scleroderma), split into training, validation, and test sets with 70:15:15 ratio.

The method achieves a prediction accuracy of 90% [START_REF] Bajaj | Automated System for Prediction of Skin Disease using Image Processing and Machine Learning[END_REF].

X. Shen et al. have presented a CNN based diagnosis method of facial acne vul-

Chapter 1

Facial skin diseases: generalities garis. They locate first the ROI using a binary classifier with CNN that detects skin and non-skin areas from the images. Then they use a seven-classifier with CNN to classify the images into one of seven classes: six types of acne, and healthy skin.

They use pre-trained VGG16 and also construct their own CNN network as binary and seven classifiers. Two datasets are gathered for this purpose. The first one, for binary classification, includes 3000 skin images and 3000 non-skin images, where 80% of the dataset is used for training, 10% for validation and 10% for test. For the seven-classifier model, they augment the dataset to 6000 images in each of the seven classes: blackhead, white-head, papule, pustule, cyst, nodule, and normal skin. The experiments show that the pre-trained VGG16 performs better in both binary and seven classifiers, achieving average accuracies of 87% and 92% respectively [START_REF] Shen | An Automatic Diagnosis Method of Facial Acne Vulgaris Based on Convolutional Neural Network[END_REF]. The segmentation aims to extract the region of interest (ROI). They extracted 36 features referring to color and texture categories to classify the images and store them in a feature vector. The classification is achieved by an artificial neural network composed of three layers: an input layer, a hidden layer, and an output layer.

The images are classified using two algorithms: Multi-Class Multi-Level (MCML) algorithm and Multi-Class Single-Level (MCSL) algorithm. In MCSL classification, the feature vector is presented to the ANN that classifies the images into healthy, benign, malignant, and eczema. While in MCML method, the classification is done in many levels. The images are first classified into healthy and unhealthy. Then, the unhealthy images are categorized into melanoma and eczema. Finally, in the last Facial skin diseases: generalities cinoma, and squamous cell carcinoma. They create a dataset containing 2656 face images referring to the six diseases to identify. For all models, the images are first randomly reversed and cropped and then presented to the networks. To remedy the problem of classes imbalance, various weights in the cost function for different lesions are used. The networks are given images of size 300 x 300 as inputs and trained with facial images. The results show that the Inception-ResNet-v2 has the best performance with an average precision of 63.7% and an average recall of 67.2%.

Then, the different models are evaluated with a new dataset used including images from different body parts of the same diseases mentioned previously. Inception-ResNet-v2 also procures the best performance with an average precision of 70.8% and an average recall of 77% [START_REF] Wu | Studies on Different CNN Algorithms for Face Skin Disease Classification based on Clinical Images[END_REF].

H. Wu et al. have developed an artificial intelligence dermatology diagnosis assis-

tant (AIDDA) based on CNN model, EfficientNet B4, that classifies skin images into 3 classes: psoriasis, eczema and atopic dermatitis, and healthy skin. They have built a dataset including 4740 skin diseases images grouped in sets of images belonging to the same disease but from different angles, or various images of identical skin disorders for the same person. Before being stored in the dataset, the images undergo a data cleaning where duplicate case and blur images are removed from the dataset, and then undergo data structuring and standardizing. After cleaning, images are managed as cases instead of individual images. These images are divided into training and validation sets. Five-fold cross-validation is used to evaluate the algorithm. The validation set is randomly picked and contains 20% of the cases.

The proposed system achieves an accuracy of 95.8% [START_REF] Wu | A deep learning, image based approach for automated diagnosis for inflammatory skin diseases. Annals of Translational Medicine[END_REF]. The units are separated by a block containing a batch normalization layer, a leaky ReLU activation function, a convolutional layer where the filter size is 1 x 1, and an average pooling layer of size 2 x 2. The model is fed with labelled dermoscopic images referring to the eight diseases to classify, acquired from ISIC-17, ISIC-18, and ISIC-19 dataset, and of different resolutions. The classes are imbalanced. The images undergo some pre-processing techniques such as cropping to transform them into square images with centered lesion, and rescaling to 64 x 64. The artifacts like hair, gel bubbles, ruler markers, ink markers, patches, and dark borders, are kept in the images to balance the classes. They perform data augmentation by rotation, translation and flipping. The proposed method can classify the skin lesions with a 94% precision, 93% sensitivity, and 91% specificity [START_REF] Iqbal | Automated multi-class classification of skin lesions through deep convolutional neural network with dermoscopic images[END_REF].

The majority of the methods mentioned in literature have focused on the detection of acne or a definite number of diseases, maximum eight classes. Some require features extraction and others don't. In this thesis, we have worked on developing a computer-aided diagnostic and classification system based on deep convolutional neural networks, classifying facial skin images into ten classes, including 8 face skin pathologies (Acne, Actinic keratosis, Angioedema, Blepharitis, Eczema, Melasma,

Chapter 1

Facial skin diseases: generalities Rosacea, Vitiligo), normal skin, and no-face class. The classification process doesn't require the extraction of the ROI from face images since the system is trained regardless of face pose, illumination, image resolution, etc.

Facial skin diseases

Skin lesions are defined as the variation of a region of the skin by comparison to the surrounding tissue. They can originate in the different layers of the skin, and occur in childhood, adulthood or even born with. Facial skin lesions are caused by an infection, allergens, medications or system disorders. These lesions could be transitory or permanent, benign or malignant, hereditary or acquired. They have different locations, colors, sizes, and textures that can help to diagnose the disease and the underlying cause. Facial skin lesions are classified into primary or secondary. Primary lesions appear at birth or over a lifetime as a reaction to the internal or external environment (such as acne, rosacea, etc.). Secondary lesions are developed from primary lesions or result from an infection or scratching (like atopic dermatitis, etc.). Hundreds of facial skin disorders affect humans. In our work, we have chosen to classify eight diseases that are mostly spread ad happen to all age groups. These disorders are acne, actinic keratosis, angioedema, blepharitis, eczema, melasma, rosacea, and vitiligo. In the remainder of this section, we will define these diseases, their symptoms and causes.

Acne

Acne is a skin disorder, affecting 3 in every 4 persons, males and females in the age group 11 to 30 years. It is a chronic, inflammatory, but not dangerous skin condition where the pores of the skin are clogged by hair, sebum, dead skin cells and bacteria.

It is caused by the elevation of androgen hormone, an active hormone that increases during puberty. High androgen levels cause the growth of oil glands under the Chapter 1

Facial skin diseases: generalities skin, that secrete more sebum. The excessive production of sebum destroys cellular walls in the pores and thus bacteria grows. Hormones increase, surface bacteria, and sebum combined together cause acne. Other factors can cause or trigger acne such as oily or greasy cosmetics, stress, some medications, hormonal variations, menstruation, genetics, humidity [START_REF] Bhate | Epidemiology of acne vulgaris[END_REF]. Acne occurs mainly on the face, forehead, chest, and back. It has numerous forms [START_REF]Acne vulgaris[END_REF]:

• Blackheads are open and dark pores at the surface of the skin.

• Whiteheads are close pores at the surface of the skin, due to oil and dead skin.

• Papules are red inflamed bumps with no visible fluid.

• Pustules are large bumps filled with yellow or white pus with red base.

• Nodules are large, hard and painful pimples under the skin.

Actinic keratosis

Actinic keratosis or AK, known as solar keratosis too, is a skin disorder characterized by a scaly, rough spots on the skin, due to chronic exposure to ultraviolet rays of sunlight or indoor tanning. It is a type of precancer, and if not treated, 5 to 10% of AKs could turn into a type of skin cancer named squamous cell carcinoma. It affects people most often after age 40 and younger persons who live in mild year round climate areas, who don't protect their skin from sun exposure especially persons having light skin, red or blond hair, green or blue eyes, weak immune system, and history of sunburns. The lesions appear on sun exposed regions such as the face, neck, lips, ears, scalp and hands. They are of various colors red, white, light or dark tan, pink, or the same color as the skin, and gray, but all have a yellow or brown crust on top (Figure 1.2). AKs have different symptoms: pain or tenderness, bleeding, itch, burn, and dry, scaly and colorless lips [START_REF] Rigel | Actinic keratosis, basal cell carcinoma, and squamous cell carcinoma[END_REF].

Angioedema

Angioedema is a swelling under the skin or mucosa. It is mainly due to a severe allergic reaction, that leads to histamine release by the body, that causes dilation of blood vessels and fluid leakage. The fluid accumulates and causes swelling. Angioedema is triggered by different allergens as insect bites, intolerance to food, medication, animal dander, and pollen. It also can be caused by an infection or disease like leukemia or lupus, or simply can be hereditary. The swelling can affect especially loose regions of tissue as the lips, tongue, eyes, as well as hands, feet, genitals, and respiratory and gastrointestinal mucosa (Figure 1.3). It has multiple symptoms: welts, itching, pain and warmth in swollen areas, swelling and redness around the eyes and lips. In severe cases, it can cause breathing problems, dizziness and abdominal pain [START_REF] Bernstein | Angioedema in the emergency department a practical guide to differential diagnosis and management[END_REF].

Blepharitis

Eczema

Eczema is a skin disease characterized by red, inflamed, and itchy skin rash [20]. It can appear in any part of the body including the face. Facial eczema is classified into three groups(Figure 1.5):

• Atopic dermatitis is the most prevalent form of eczema affecting mainly children. It appears around the cheeks and chin. It also affects adults around the eyes, on the eyelids and around the lips [START_REF] Stanway | Atopic dermatitis[END_REF].

• Seborrheic dermatitis is the most widespread form of facial eczema in adults.

It happens around the ears, in the eyebrows and on the extremities of the nose [START_REF] Oakley | Seborrheic dermatitis[END_REF].

• Contact dermatitis exists in two forms: Irritant contact dermatitis caused by cosmetics, toiletries detergents, solvents and friction. It leads to dryness and

Chapter 1

Facial skin diseases: generalities pain of the skin around the eyes and around the hairline. The second form is allergic contact dermatitis provoked by the contact of the skin with substances usually non allergenic such as perfumes, jewelry, hair dye, rubber. . . it is found on the neck, and face [START_REF] Oakley | Contact dermatitis[END_REF].

All facial eczema types have almost identical symptoms including red spots, itching, burning, inflammation, dryness, swelling, rashes, and pain. Many factors can contribute to the development of eczema such as family history, allergies, autoimmune diseases and the age. Environmental irritants, hormone variations, food allergens, stress, and temperature trigger eczema.

Melasma

Melasma is a skin pigmentation disorder characterized by brown or gray patches on the skin. It affects women more than man. 90% of persons developing melasma

Chapter 1

Facial skin diseases: generalities are women. The patches appear on the cheeks, chin, forehead and bridge of the nose (Figure 1.6). It can also occur on other parts of the body such as the neck and forearms. The pigmentation is caused by the overproduction of melanin by the melanocytes, the color making cells. Sun exposure, skin care products, family history, hormone fluctuations, and pregnancy trigger melasma. When it occurs in pregnant women, melasma is called "mask of pregnancy", and in this case, it disappears on its own [START_REF] Oakley | [END_REF].

Rosacea

Rosacea is a chronic disease that mainly affects the face. It can be mistaken for eczema, acne or allergic skin reaction. Rosacea affects mainly more middle aged women with light skin than men. It causes redness across the cheeks, chin, nose and forehead. It also causes alsopimples, dry skin, swelling, burning when using water or cosmetics, visible blood vessels, and thick skin in advanced phases (Figure 1.7). It may also cause eye problems. Rosacea could be a hereditary disease or due to the dilation of blood vessels leading to flushing and redness. It is triggered by spicy foods, alcohol, hot drinks, environmental factors, emotions, medications and cosmetic products [START_REF] Oakley | [END_REF].

Vitiligo

Vitiligo is a chronic disease caused by the deficiency of melanin, the pigment in skin, due to melanocytes death or stop functioning. The depigmentation is due to an autoimmune disease, stress, trauma or simply a family history. It mainly occurs on the face, neck and hands. Facial vitiligo is characterized by white patches developed on the skin, lips, and in the mouth. It affects all races and genders and mainly in persons between 10 and 30 years. Besides white patches, the most common symptoms of vitiligo include whiteness of eyelashes, eyebrows and beard hair, change of color of the retina, lighting of the tissues in the nose and mouth [START_REF] Oakley | [END_REF]. In some cases, persons may have pain, and itching (Figure 1.8).

Conclusion

Computer-aided diagnosis systems are computer-based techniques to assist doctors to interpret medical images and thus detect and diagnose the abnormalities in the images. They are widely used in different medical fields including dermatology.

Computer-aided diagnosis systems used for facial skin diseases process images, extract features and then pass these features to a classifier to assign the skin lesion to the most likely disease. In this chapter, we have showed some automated techniques that have been developed by researchers to classify facial skin diseases images. Some methods are based on image processing techniques and pattern recognition. With the advances in technologies, high performance computers, and advances in machine learning and deep learning, artificial intelligence (AI)-based approaches have been suggested to classify skin lesions. We have proposed a deep learning based identification method capable of classifying several facial skin diseases. We have also presented the different diseases to be classified in our method. In the next chapter, we will provide an overview of artificial intelligence, machine learning and deep learning. In this section, we will describe deep neural networks, discover the concept "feature hierarchy" that makes deep learning a powerful tool in computer vision, and present the different architectures of deep neural networks. In the rest of the chapter, convo-

Chapter 2

Deep learning based skin disease classification lutional neural networks (CNN) will be explored: the blocks and layers constituting the network, its different architectures, as well as its training. Finally, we move to image classification using CNN. An overview of image classification definition will be provided, along with the challenges the network may encounter. The chapter is ends by explaining the steps of training a CNN for image classification and how the classification is performed.

Artificial intelligence

Artificial intelligence (AI) is the ability of machines to mimic the work of the human brain in learning and processing data to perform recognition, identification tasks and make decisions without human intervention. The term AI was brainstormed in the 1950s, but has become very popular nowadays due to huge data volumes, advanced algorithms and enhancement of computers potential and storage.

It has wide applications and in different fields such as image recognition, chatbots, speech recognition, sentiment analysis, chess playing computers, self-driving cars. . . .

Machine learning

Machine learning is a sub-field of AI enabling systems to automatically learn and recognize patterns on the basis of existing algorithms and datasets: learn from experience, and thus solve problems adequately. It is considered as the brain of AI systems. Machine learning systems are trained rather than programmed using structured and semi-structured data. These systems require a manual feature extraction,and then these features are used to create the model performing the desired task. Machine learning has numerous applications: automation, natural language processing, computer vision and image processing. In addition, machine learning algorithms are divided into two large classes: supervised and unsupervised learning. Supervised learning learns a function that maps an input (X) to an output (Y). Su-pervised machine learning algorithms are given a known labeled set of inputs called training data and the desired output values. The algorithm learns automatically the patterns to make the correct prediction of the new data. This type of learning uses classification and regression techniques [START_REF]Artificial Intelligence: A Modern Approach[END_REF]. Unsupervised learning learns from unlabeled data and without predefined target values. The algorithm tries to identify discriminating characteristics in the input data. Clustering is the most common method used in unsupervised learning [START_REF]Unsupervised Machine Learning: Clustering Analysis[END_REF].

Deep learning

Deep Learning is a type of machine learning inspired by the architecture of the human brain, enabling the automatic learning [START_REF] Deng | Deep Learning: Methods and Applications[END_REF]. It can work and learn from unstructured and unlabeled data. It trains the system by huge amounts of data (images, videos or texts) to learn the features from them using many layers of processing. Deep learning has found applications in the fields of computer vision: audio/speech recognition, automated driving, aerospace and defense, medical research, industrial automation, electronics. . . . Deep learning algorithms use a hierarchical level of artificial neural networks, comprising tens or hundreds of layers stacked on top of each other, to extract automatically the features from the data and improve the identification and the classification without human supervision. Deep learning models are capable of unsupervised learning.

Deep learning versus machine learning

Despite being a subfield of machine learning, deep learning differentiates itself from traditional machine learning algorithms by the type of data that it deals with and the learning method. Deep learning cancels some pre-processing of data that is commonly associated with machine learning. These methods can deal and process unstructured data such as audio, video, images. . . and extract automatically the

Comparison between machine learning and deep learning

Comparison between classic machine learning algorithms and deep learning algorithms (Figure 2

How artificial neural networks work?

The network receives an input signal (image, audio, text. . . ) in the form of vector defined by x(n). Each input is multiplied by a corresponding weight representing the strength of the signal of the neuron, connected to a neuron. All the weights are added in the computing unit. In case the weighted sum is zero, we add a bias to adjust the system's response. The weighted sum passes through an activation function f and then the output y is generated (Figure 2.7), and given by equation (2.1).

x 1 w 1

x 2 w 2

x n . . . . . .

y = f ( n i=1 x i w i + bias) (2.1)

Activation function

Activation functions are mathematical models that define the output of a neural network. They decide if the neuron should be fired or not. They are a non-linear

1-Step function

It is the simplest activation function and defined by:

U (n) =        1 n ≥ 0 0 otherwise (2.2) n U (n) 0 1 Figure 2.8:
Step function.

As seen from the equation above, we can see that this function makes the output 1 when the weighted sum is greater or equal to zero and 0 otherwise (Figure 2.8). It is mostly used in binary classification problems. The step function is non differentiable which causes a problem during the training process and thus the weights can't be updated.

2-Sigmoid function

It is a non-linear, real and differentiable activation function characterized by:

f (x) = 1 1 + e -x
(2.

3)

The output varies between 0 and 1 (Figure 2.9). The main disadvantage of the Chapter 2

Deep learning based skin disease classification sigmoid function is that it is not zero centered and the neurons saturate which kills the gradient and thus the training can not be achieved.

-6 -4 -2 0 2 4 6 0.5

1 x f (x) f (x)
Figure 2.9: Sigmoid function.

3-Hyperbolic tangent [31]

It is a continuous zero-centered function ranging between -1 and 1 (Figure 2.10). It is represented by:

tanh(x) = e x -e -x e x + e -x (2.4) -2 -1 1 2 -1 1 x tanh(x)

4-Rectified Linear Unit (ReLU)

Chapter 2

Deep learning based skin disease classification

ReLU is the most generally used function in neural networks because it speeds up the learning process and improves the performance. Despite its linear form, ReLU is not linear and is differentiable [START_REF] Hahnloser R | Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit[END_REF]. It gives an output 0 if the input is negative and x for positive values (Figure 2.11). It is designated by [START_REF] Nair | Rectified linear units improve restricted Boltzmann machines[END_REF]:

f (x) =        x if x > 0 0 if x ≤ 0 (2.5) -4 -2 2 4 1 2 3 4 5 x f (x)
Figure 2.11: Rectified Linear Unit (ReLU) [START_REF] Nair | Rectified linear units improve restricted Boltzmann machines[END_REF].

5-Cost function

The cost function is an important component of a neural network. It determines how badly the network is performing. It measures the error between the predicted output and the actual output. This error is back propagated in the network and minimized during the training process by adjusting the weights and biases repetitively.

Deep neural networks

Deep neural networks are large artificial neural networks composed of several hidden layers between the input and output layers [START_REF] Schmidhuber | Deep Learning in Neural Networks: An Overview[END_REF]. The layers in a deep neural

Architecture of deep neural networks

Deep learning architectures are divided into supervised and unsupervised learning.

1-Unsupervised learning: In unsupervised learning, the deep neural network learns patterns from unlabeled data. The most common unsupervised deep learning architectures are autoencoders [START_REF] Baldi | Unsupervised Learning, and Deep Architectures[END_REF] and deep belief networks [START_REF] Hinton | Deep belief networks[END_REF].

2-Supervised learning:

In supervised learning, the deep neural networks are trained using labelled data. The most used supervised deep learning architectures are recurrent neural networks [START_REF] Amidi | VIP Cheatsheet: Recurrent Neural Networks[END_REF], and convolutional neural networks [START_REF] Goodfellow | Deep Learning[END_REF].

In our work, we use convolutional neural networks that will be described in details in the next section.

Convolutional neural networks

Convolutional neural networks, also known as CNN, are a class of feed-forward deep neural networks mostly used in image analysis. They take images as input.

They are widely used in different computer vision tasks and find interest across various domains. Unlike traditional feed-forward networks, CNNs use fully connected layers only in the very last layer. The fully connected layer is substituted by a convolutional layer for at least one of the layers of the network [START_REF] Goodfellow | Deep Learning[END_REF]. The convolution layers are followed by a nonlinear activation function, such as ReLU, till the end of the network where we can add one or two fully connected layers to obtain the final output classifications. The core of a CNN is the convolutional layer that gives its name to the network. The term "convolution" refers to the mathematical function of convolution. In mathematics, the convolution is a linear operation on two functions that provides a third function that indicates how the shape of one is changed by the other. Whereas in deep learning, it is a dot product of two matrices followed by a sum. In terms of CNN, it is the multiplication of the input with a series of weights.

In case of a 2D input, the multiplication is done between an array of input data and a 2D array of weights, denoted filter or kernel. The convolution between 2D input image I and 2D kernel K is:

S(i, j) = (I * K)(i, j) = m n K(i -m, j -n)I(m, n))) (2.6)
Since its size is smaller, this filter is multiplied many times by the input array at different points. Hence, the filter is applied systematically from left to right and from top to bottom to cover the whole image. Actually, the filter is conceived to detect a specific type of feature in the input and the systematic application across In the remainder of this section, each of these layer types will be reviewed in detail and the parameters associated with each one and how to set them, and how to train CNNs will be discussed as well.

1-Convolutional layer

The convolutional layer, or CONV layer, is the core of a CNN and is used to extract features from the input. The CONV layer has a set of learnable filters or kernels, each having a width and a height. These filters are small but applied through the full depth of the volume. The convolution is done between the input image and K kernels of size MxM. Each kernel is slided across the input image and convolved with it. The result of each convolution is a 2D output called feature map giving specific information about the image. Once the K kernels are applied to the input, we obtain K, 2D feature maps, stacked along the depth size of the array to form the final output volume that is fed to the next layer in the network. Each feature map represents specific characteristics of the input (Figure 2.17). The kernels are thus considered as feature extractors.

Figure 2.17: K kernels are applied to the input at each convolutional layer. Each one is convolved with the input to produce a feature map. After application of all kernels, we obtain K feature maps [START_REF] Karpathy | Convolutional Networks[END_REF].

Every map in the output is an output of a neuron looking at a small zone of the input. in this way, CNN learns filters that trigger when they detect a specific feature at a spatial region in the input. In low layers, filters are triggered when they detect low-level features such as edges or corners. While in deeper layers, they are activated when seeing high-level features like object parts. In CNNs, each neuron is connected It is given by integer of:

(W + 2P -F + 1) (2.7)
If the result is not an integer, this means that the stride is incorrect and thus the neurons don't fit efficiently and uniformly across the input. In summary, the convolutional layer takes an input of dimension W1 x H1 x D1 and necessitates four hyperparameters: the number of kernels K, the receptive field F, the stride S and the zero padding P.

It provides an output volume of size W2 x H2 x D2, where [START_REF] Karpathy | Convolutional Networks[END_REF]:

W 2 = ( W 1 -F + 2P S ) + 1; H 2 = ( H 1 -F + 2P S ) + 1; D 2 = D1 1 = K (2.8)

2-Activation layer

The convolutional layer is usually followed by a nonlinear activation function. It determines which info of the model should fire at the end of the network and which ones should not. There are many activation functions as seen previously, but recently the most commonly used one is ReLU since it almost has a better performance than others and speeds the training of the network.

3-Pooling layer

The pooling layer is embedded between two consecutive CONV layers. It decreases the size of the input of width and height to lessen the number of parameters and the computational costs, and also controls overfitting. It works separately on each feature map. There are two types of pooling: max pooling and average pooling.

Usually, the max pooling is performed in the middle of the network to decrease spatial dimension, while average pooling is done in the final layer in case we don't

Chapter 2

Deep learning based skin disease classification want to use fully connected layers entirely. Max pooling is the most common type of pooling. Average pooling calculates the average of all the values from the section of the image covered by the filter. Whereas in max pooling, the largest value is taken.

Pooling layer takes an input of size W 1 x H 1 x D 1 . It requires two hyperparameters:

the pool size F, and the stride S.

It provides an output of size W 2 x H 2 x D 2 where:

W 2 = W 1 -F S + 1 ; H 2 = H 1 -F S + 1 ; D 2 = D 1 (2.9)
Usually, we use a pool size of 2x2 or 3 x 3 in networks using larger input images and a stride S =1 or S =2. When we increase the stride, the size of the output decreases (Figure 2.20). The neurons in the FC layer are fully connected to all activations in the flatten layer. FC layer is followed by an output layer that computes the output.

5-Output layer

It is the last layer of the network and contains the label in one-hot encoded form.

In practice, other layers could be added to control the network. The most common ones are batch normalization and dropout layers. The same network after dropout of probability p = 0.5. 50% of the neurons is randomly dropped out from the model [START_REF] Hinton | Improving neural networks by preventing co-adaptation of feature detectors[END_REF].

There are several CNN models differing by their architecture and the number of layers in the network. The most common ones are: LeNet [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF], AlexNet [START_REF] Krizhevsky | ImageNet Classification with Deep Convolutional Neural Networks[END_REF],

GoogleNet (also called Inception) [START_REF] Szegedy | Going Deeper with Convolutions[END_REF], and VGGNet [START_REF] Simonyan | Very Deep Convolutional Networks for Large-Scale Image Recognition[END_REF].

LeNet

CNN training

CNN training consists of finding kernels in CONV layers and weights in FC layers that reduce changes between output predictions and real labels on a training set.

The training comprises two stages: a forward propagation and a backpropagation (Figure 2.23). In the forward phase, the input is propagated through the network to produce an output. In this stage, each layer will store all variables computed and used that will be needed in the backward phase. In the backward phase, the gradients are back propagated and the weights are updated. Backpropagation algorithm is the approach generally used to train and optimize neural networks. The weight matrix and the bias vector are the learnable parameters of the classifier that should be optimized to minimize the loss function and thus increase the performance of the network. The most prevalent optimization algorithm used is the gradient descent. The kernels and weights are updated by backpropagation using gradient descent optimization algorithm [START_REF] Yamashita R | Convolutional neural networks: an overview and application in radiology[END_REF].

1-Loss function

The loss function or cost function measures the similarity between predicted class labels through forward propagation and ground truth labels. The greater the compatibility between the two sets of labels, the less the loss function. During training our network, we aim to minimize the loss function, so the accuracy of classification increases.

2-Gradient descent

Gradient descent is the most common optimization algorithm used to train deep neural networks. It updates iteratively the parameters of the network, such as kernels and weights in CNN, to minimize the loss function by iteratively moving in the direction of steepest descent. Hence, each parameter is updated in the negative direction of the gradient with a random step called learning rate. Mathematically, it is the partial derivative of the loss with respect to the learnable parameter. The single update of a parameter is given by: Numerous improvements have been applied to the gradient descent algorithm and are widely used such as momentum, Nesterov acceleration, RMSprop [START_REF] Ruder | An overview of gradient descent optimization algorithms[END_REF], and Adam [START_REF] Kingma | Adam: a method for stochastic optimization[END_REF]. They are given by:

w -α * ∂L ∂w (
m t = βm t-1 + (1 -β)[ δL δw t ] (2.11) v t = βv t-1 + (1 -β) * [ δL δw t ] 2 (2.12)
Where δL is the derivative of Loss Function, δw t is the derivative of weights at time t, and β is the moving average parameter of value 0.9. This bias is fixed by calculating bias corrected. This also helps to reach global minimum efficiently with minimal oscillations. The formulas are given by:

m t = m t 1 -β t 1 (2.13) v t = v t 1 -β t 2 (2.14)
Where m t and v t are the bias corrected m t and v t , β t 1 and β t 2 are the decay rates of average of gradients at time t. The gradient descent is adapted after each iteration on a current mini-batch to be controlled and unbiased during the process, from where the naming Adam comes. Finally, the weight update is performed following the equation:

w t+1 = w t -m t ( α √ v t + ε ) (2.15)
Where w t is the model weight at time t, α is the learning rate, and ε is a small positive constant.

CNN and Image Classification

Image classification is a very interesting and large field of study, involving an enormous variety of approaches, including deep learning. It is a sub-field of computer vision permitting computers to see the world as we do. The invention of deep learning made image classification more widespread. Image classification has various applications in healthcare, industry, social networks, marketing and others.

What is image classification?

Image classification is the labelling of images into a predefined set of classes. Previous

Image classification based on Deep learning

To construct a deep learning based image classifier, four steps are required. the detected features previously to detect higher-level ones such as shapes in the second layer and objects in the highest layers. The last layer utilizes the higher level features to make the predictions. In case of a misclassification, it learns from this error and enhances itself using gradient descent algorithms. Last, we evaluate our trained model using the test set. The network predicts the class of each image in the set. These predictions are compared to the ground truth labels, representing the correct labels, from the test set. The performance is evaluated by computing the number of correct predicted labels, and classification metrics such as precision, recall, accuracy and F1-score.

Convolutional Neural Networks for image classification

Conclusion

In are generally made up of numerous CONV layers, Maxpooling layers, followed by one or many fully connected layers and finally a softmax layer. The most common networks are LeNet [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF] ,AlexNet [START_REF] Krizhevsky | ImageNet Classification with Deep Convolutional Neural Networks[END_REF],VGG [START_REF] Simonyan | Very Deep Convolutional Networks for Large-Scale Image Recognition[END_REF], and later more advanced networks have been suggested such as GoogLeNet or Inception [START_REF] Szegedy | Going Deeper with Convolutions[END_REF], ResNet [START_REF] He | Deep Residual Learning for Image Recognition[END_REF], DenseNet [START_REF] Huang | Densely Connected Convolutional Networks[END_REF], EfficientNet [START_REF] Szegedy | Rethinking the inception architecture for computer vision[END_REF], and many others. In all these networks, the basic components, convolution and pooling, are almost the same, but the difference lies in their topology. VGG, Inception v3, and EfficientNet B0 are some of the most popular architectures widely used for image recognition and classification due to their good performance and availability of use on different applications for object identification tasks.

Chapter 3

Convolutional neural networks architectures VGG is composed of several sets of 3 x 3 convolutional layers stacked on top of each other of different depth in various architectures, of stride 1 and padding 1, and ReLU activation function followed by a Maxpooling layer of size 2 x 2 with a stride 2. The number of layers in each set increases as we go deeper in the network.

The CONV layers are followed by 3 fully connected layers where the first two layers contain 4096 nodes, and the third one contains 1000 nodes referring to 1000 classes.

The network is ended by a softmax layer for classification (Figure 3.1).

The VGG family includes four architectures: VGG- [START_REF] Krizhevsky | ImageNet Classification with Deep Convolutional Neural Networks[END_REF]. Also, in one of the models, they use 1× 1 convolution kernels, that act as a linear transformation of the input, followed by ReLU. The architectures of the different models are presented in (Table 3.1).

We will focus on VGG-16 network that is the backbone of the network we have proposed in our work to classify facial skin diseases. 3.1: VGG configurations. Each column represents a configuration. The depth increases from left to right from A with 11 weight layers to E with 19 weight layers. The added layers are represented in bold. The CONV layer parameters are designated as conv "receptive field" "width of CONV layers". Each CONV layer is followed by a ReLU activation function [START_REF] Simonyan | Very Deep Convolutional Networks for Large-Scale Image Recognition[END_REF]. VGG-16 is trained by optimizing the multinomial logistic regression using mini batch SGD based on back propagation with momentum. They use a batch size of 256, and a momentum of 0.9. The training is regularized by L2 weight decay and dropout for the first two FC layers of 0.5. The learning rate is initially set to 0.01, and when the validation accuracy stops enhancing, it is dropped by factor of 10.

VGG-16

VGG-16 has a larger number of parameters and depth than previous deep neural networks, but requires less epochs for loss function to converge and this is due to the use of small convolutional filter sizes and regularization by large depth, and pre-initialization of some layers [START_REF] Simonyan | Very Deep Convolutional Networks for Large-Scale Image Recognition[END_REF]. The network is trained with images of size 224

x 224. The network has learned plenty of feature representations for a large variety of images. The pre-trained VGG-16 is an open-source model, so it can be generally used out of the box for different applications.

Inception-v3

Inception-v3 is a CNN architecture from the inception family [START_REF] Szegedy | Rethinking the inception architecture for computer vision[END_REF]. It is widely used in image identification and classification, and pre-trained on ImageNet dataset.

The Inception module is based on the idea of running several operations of pooling, and convolution with different filter sizes as 3 x 3, and 5 X 5 in parallel.The inception family includes 4 versions. The first version is Inception-v1, known also as GoogleNet [START_REF] Szegedy | Going Deeper with Convolutions[END_REF]. Later, this architecture has been modified and improved by introducing batch normalization resulting in a new model called Inception-v2 [START_REF] Ioffe | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[END_REF]. The third generation is Inception-v3 which especially uses factorization ideas. The last version is Inception-v4 which is a simplified architecture involving more inception modules than Inception-v3 [START_REF] Szegedy | Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning[END_REF]. Inception-v3 is built on factorization ideas. Its architecture is built as follows.

Factorized convolutions

Factorizing convolutions aim to diminish the number of parameters in the network without deteriorating the efficiency. It consists of: factorization into smaller convolutions, and factorization into asymmetric convolutions.

a-Factorization into Smaller Convolutions

The benefit of smaller convolutions is to fasten the training. The authors have developed three inception modules based on the idea of factorization, that is a method that allows the reduction of the number of parameters for the entire network; which makes overfitting less likely, and thus the depth of the network can be augmented.

Auxiliary Classifier

Auxiliary Classifiers are small CNNs placed between layers and have different purposes. In Inception-v1, two auxiliary classifiers are used to deepen the network.

Whereas, in inception-v3, one classifier is used that act as a regularizer (Figure 3.5).

Efficient grid size reduction

The size of the feature maps is usually reduced by max pooling. This method has a computational cost. Hence, an efficient grid size reduction is suggested. 640 feature maps are obtained by concatenation of 320 feature maps that are obtained by convolution with stride 2, and 320 feature maps that are got by max pooling. The concatenated feature maps are presented to the next level of inception module. This method is cheap and cancels the bottleneck representation. It is presented in Figure The general architecture is shown in Figure 3.7. Inception v3 is trained on ImageNet dataset with stochastic gradient using a batch size of 32 for 100 epochs. They use initially momentum with a decay of 0.9, and then used RMSProp with a decay of 0.9, and = 1 which achieve th best model. The learning rate was set to 0.94, and decayed after each two epochs with an exponential rate of 0.94.

EfficientNet

EfficientNet is a CNN model proposed by Mingxing Tan and Quoc V in 2019 and scaling method that evenly scales all dimensions of the network, depth, width, and resolution, using a compound coefficient [START_REF] Tan | EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks[END_REF]. EfficientNet models achieve better accuracy and efficiency than previous CNNs. Scaling up CNNs is usually used to improve the accuracy. It consists of increasing the model's depth, or width, or image resolution for training and test. Increasing the depth means adding more CONV layers to the network; which allows the extract of more complex features, but the model becomes hard to train. Width scaling; increasing the number of channels in a CONV layer, results in more feature maps. The model captures accurate features but saturates fast. Resolution scaling simply increases the image resolution that is being fed to the CNN. Generally, depth scaling is the most used amongst all.

These manual scaling methods improve the accuracy of the model, but they are tedious and also after a few level, the performance saturates or even degrades. A new scaling method is suggested, called compound scaling, based on scaling up the three dimensions simultaneously with a fixed ratio (Figure 3.8). It is justified by the fact that if the input image is bigger, the model requires more layers to increase the receptive field, and more channels to detect fine-grained features in the bigger image. The compound scaling best improves the performance than the individual scaling. The relation between the different scaling dimensions of the baseline network is determined using grid search algorithm [START_REF] Lavalle | On the relationship between classical grid search and probabilistic roadmaps[END_REF]. It computes the coefficient of each dimension, and then these coefficients are applied to scale up the baseline network.

The three dimensions are scaled uniformly as follows: 3.2: EfficientNet B0 baseline network architecture. Each row presents the type of layer in the block, its input resolution, its output channels, and the number of layers in each block [START_REF] Sandler | MobileNetV2: Inverted Residuals and Linear Bottlenecks[END_REF].

(Depth : d = α φ ); (W idth : w = β φ ); (Resolution : r = γ φ ) (3.

Swish activation is an activation function proposed by Google Brain team for

deep networks [START_REF] Ramachandran | Searching for Activation Functions[END_REF]. It is the multiplication of the input x with the sigmoid activation.

The form of the Swish function and its derivative are shown in (Figure 3.9).

Swish(x) = x * sigmoid(x) = x * ( 1 1 + e -x ) (3.2)
It is a smooth function that looks like ReLU, but differs in the domain around 0. ReLU changes direction near x = 0. It cancels all negative values, and thus all their derivatives are zero, while swish activation twists from 0 toward negative values and then upwards. It is non-monotic. The authors show that swish surpasses ReLU in deep neural networks. For example, when replaced with ReLU on Incep-tionResNetV2, the performance of the network has improved by 0.6% on ImageNet dataset.

Squeeze and Excitation Block

In CNNS, the channels of the feature maps created from a CONV layer have equal weights. Squeeze and excitation (SE) block is an approach that assigns a specific weight to each channel. It focuses on only the significant features. It procures an output of size 1 x 1 x channels. This weightage is determined by the network as other parameters [START_REF] Hu | Squeeze-and-Excitation Networks[END_REF].

SE block, fed with a convolutional block, is composed of a global average pooling layer that squeezes each channel into a single value, an FC layer with C/r (where C is the number of channels, and r is a hyper parameter set to 16) and neurons followed by ReLU activation to reduce the complexity of the output channel. An FC layer with C neurons followed by a sigmoid activation provides a smooth gating function to each channel and scaling where we give a weight to each feature map in the convolutional block depending on the side network. This is called excitation (Figure 3.11).

Figure 3.11: SE block structure: Each feature map resulting from CONV layer is squeezed, excited, and then scaled to produce a weighted feature map [START_REF] Hu | Squeeze-and-Excitation Networks[END_REF].

Chapter 3

Convolutional neural networks architectures

Being the baseline network, EfficientNet B0 is scaled up using a compound scaling method at two stages. In the first stage, the authors fix the value of φ to 1, and determine the values of α , β, and γ using a small grid method that is set to 1.2, 1.1, and 1.15 respectively for EfficientNet B0. In the second stage, α , β, and γ are fixed as constatnts, and the base network is scaled up with a different φ, to generate EfficientNet B1 to B7. All EfficientNet models include 7 blocks, but with a different number of sub-blocks that increases as we go from EfficientNet B0 to B7. They are all trained on ImageNet database using similar settings, an RMSProp optimizer with decay 0.9 and a momentum 0.9, a batch norm momentum 0.99, a weight decay 1e-5, and an initial learning rate 0.256 decayed by 0.97 every 2.4 epochs. The dropout is linearly increased from 0.2 to 0.5 for EfficientNet B0 to B7 respectively, since bigger models require more regularization.

Transfer learning with CNN

Transfer learning

Training Deep CNNs on large datasets from scratch can take days or weeks, and is also computationally expensive due to the depth and number of fully connected nodes in the network. To minimize these huge time and compute resources, and thus accelerate the process of image classification, the model weights of pre-trained networks on benchmark datasets are used as the starting point. These models can be used directly, or implemented in a new model for other classification tasks. This is called transfer learning. Thus, transfer learning is the most common technique used in deep learning, where a model trained on a task is reused as an initialization for the task of interest. It aims to decrease the training time and enhance the performance of the model. Transfer learning is also beneficial when there is not enough training data to train a network from scratch. In Figure 3.12, we show how transfer learning enhances the performance of the network during training.

T raining P erf ormance

With transfer Without transfer Figure 3.12: Benefits of transfer learning for deep learning with CNN: The y axis represents the performance, and the x axis the number of training samples. When using transfer learning, the performance has a higher start, higher slope, and higher asymptote which leads to a better performance.

In deep learning, there are two types of transfer learning (Figure 3.13):

• CNN is considered as feature extractor. In this case, we remove the output of the pre-trained model, and then use the rest of the network as a fixed feature extractor for the new dataset. The initial layers of the pre-trained model are frozen to retain the contained information during the retraining. A new classifier is added on top of the frozen layers, that will learn to convert old features into predictions on a new dataset. The classifier is retrained on the new dataset.

• Fine-tuning consists of removing the fully-connected layers of the pre-trained network, replacing them by a new FC layer set, retraining them, and also finetuning the weights of the pre-trained network. We can fine-tune all the layers of the network, or freeze the weights of initial layers for overfitting concerns and then fine-tune those of higher layers. This is justified by the fact that early layers extract low level features (edges, corners, blobs. . . ) that are useful to many tasks, while later layers extract task specific features. The number of frozen and fine-tuned layers can be determined by testing. The model is retrained on the new dataset with a very low learning rate.

Transfer learning scenarios

Transfer learning methods depend on several factors, especially the size of the new dataset, and its resemblance to the prototype dataset. There are 4 scenarios that help us to decide which transfer learning method will be used (Figure 3.

14).

Scenario 1: Size of the new dataset is small, with a high similarity to the original dataset. In this case, the pre-trained network is used as a feature extractor. We only replace the output layers with a new classifier and retrain it. The convolutional base is frozen. In this case, the fine-tuning is not a good choice because it may lead to overfitting. Scenario 2: Size of the new dataset is small, with a low similarity to the original dataset. In this case, we use the fine-tuning method. A new classifier is added on top of the pre-trained network. Some convolutional layers are retrained in addition to the classifier, and what remains is frozen. Scenario 3: Size of the new dataset is large, with a low similarity to the original dataset. The best idea is to train the network from scratch since we have a large dataset, and the new data is different.

Scenario 4: Size of the new dataset is large, with a high similarity to the original dataset. This case requires fine-tuning. The whole model is retrained, using the initial convolutional layer weights as a starting point. Figure 3.14: Transfer learning methods: There are four scenarios: (a) the new dataset is small, with a high similarity to the original dataset: feature extraction method, (b) the new dataset is small, with a low similarity to the original dataset: fine-tuning method, (c) the new dataset is large, with a low similarity to the original dataset: training from scratch, (d)the new dataset is large, with a high similarity to the original dataset: fine-tuning method.

Conclusion

In this chapter, we presented an overview of the three CNN models that have been used in our method: VGG, Inception v3, and EfficientNet B0. In this chapter, we will be building a complete end-to-end method that can detect facial skin diseases in RGB images using deep learning along with computer vision techniques. To accomplish this task, we will be using transfer learning with the three pre-trained models VGG-16, Inception v3, and EfficientNet b0 and training them on a dataset of images containing faces of people affected with a skin disease, normal face skin, and no face images. In the remainder of the chapter, we will explain in details our proposed approach. We will also present the results obtained in each model.

Facial skin diseases identification method

In this study, we proposed an approach that can identify probable dermatological We will be implementing our models network using the Keras framework with Tensorflow as backend. The system ran on an Intel® core™ i7 7700HQ, 2.8 GHz CPU, 1060 GPU GTX.

Facial skin disease dataset

Facial skin disease dataset consists of labelled RGB images of faces affected by one of the eight skin diseases used in our approach, normal faces, and images of various objects such as animals, cups, cans, trees, cars, food, etc. . . collected from different dermatological sites. Initially, the dataset is formed of 2000 labeled images belonging to ten classes that are 8 facial skin diseases (Acne, Actinic keratosis, Angioedema, Blepharitis, Eczema, Melasma, Rosacea, and Vitiligo), normal skin class, and no-face class. All the classes are balanced, and contain 200 images of different resolutions and brightness, for males and females of different age groups, and in several face poses. Deep neural networks have a lot of parameters, so in order to have good performance we should present a proportional amount of data.

Since the performance of networks enhances with the amount of data available, we perform data augmentation to increase the size and the diversity of the available data, without having to gather new data. Data augmentation can also help reduce overfitting and enhance the generalization of the model due to the increase in the diversity of our dataset. In our work, the augmentation is performed before feeding

Facial skin diseases identification using VGG-16

To classify facial skin diseases, we use pre-trained VGG-16 based models. We have applied two models of VGG-16; VGG-16 in its original architecture, and transfer learning with VGG-16 which generates a new adapted architecture to facial skin diseases identification to evaluate performances on Facial Skin Disease Dataset (FSDD). Before diving into our approaches, we will introduce some terminologies that will be seen in the next sections.

1-Overfitting

Overfitting occurs when a network models well the training data, but can't generalize on unseen data. It can be noticed when the validation or testing error is greater than the training error. To prevent overfitting, we can reduce the complexity of the network, or apply regularization techniques such as weight decay, dropout, data augmentation, early stopping, etc.., or train with more data.

Underfitting happens when the network has not learned the basic features, and thus can neither model the training nor generalize to unseen data. Underfitting is remedied by simply using another model. Actually, when we train a network we aim to lessen the training loss as much as we can, and keep the gap between the training and testing loss small.

2-Batch

The batch size is a hyper-parameter of gradient descent, the learning algorithm that updates the network using the training dataset, which defines the number of training samples used in one forward and backward pass. At the end of the batch, the error is computed by comparison of the predictions with the ground truth labels.

Consequently, the update algorithm enhances the model.

There are three types of batches:

• Batch mode: the batch size is equal to the size of the dataset.

• Mini-batch mode: 1 < batch size < size of the dataset, and is a number that can be divided into the total dataset size.

• Stochastic mode: the batch size = 1.

3-Epochs

An epoch is the one time forward and backward passage of the whole dataset through the neural network. The number of epochs is a hyper-parameter of gradient descent that determines the number of complete passes of the learning algorithm through the training dataset. This number of epochs is related to the diversity of data, and thus is determined by test. The best number is determined when the model error has been sufficiently minimized. A small number of epochs can lead to underfitting, while a huge number may cause overfitting.

Classification using pre-trained VGG-16

The first step in building our facial skin diseases model is to train the pre-trained VGG-16 on our dataset. The convolutional base and the FC layers are kept the same; For the test, we give the network images from outside the FSDD. For this purpose, we have created a small dataset containing 10 images for each class. By experiment, we have found that the best model to use for test is the split case 90:10.

Classification using transfer learning with VGG-16

The second method we have applied for facial skin diseases classification is transfer learning with VGG-16. We fine-tuned the model to adapt our image classification tasks. The backbone, the CONV base of the pre-trained model, is kept in its original form, while the classifier, consisting of Flatten, 2 FC and softmax layers, is removed.

Our classifier is built using a global average pooling layer, dropout layer of 0.5, and a Softmax classifier of dimension 10. This network is called Facial Skin Diseases Network, FSDNet. The architecture of FSDNet is summarized in Figure 4.9. The convolutional base is used to extract the features that will be fed to the classifier that we want to train to identify to which class the facial skin images belong. We choose the global average pooling as classifier for many reasons. The global average pooling (GAP) is a method that computes the average output of each feature map from the preceding layer. This operation tends to minimize the data considerably, and presents the model for the classification layer. The GAP has no trainable parameters. As seen in the architecture in (Figure 4.9), there are 512 averaging of dimension 7 x 7. The GAP layer converts the dimensions from [START_REF] Shen | An Automatic Diagnosis Method of Facial Acne Vulgaris Based on Convolutional Neural Network[END_REF][START_REF] Shen | An Automatic Diagnosis Method of Facial Acne Vulgaris Based on Convolutional Neural Network[END_REF]512) to (1, 1, 512) by doing averaging across the 7 x 7 channels. By doing this, a large number of trainable parameters is eliminated from the model which speeds up the training.

Also the removal of these parameters diminishes the probability of overfitting.

On the other hand, the removal of FC layers forces the feature maps to be more related to the identification classes. Lastly, GAP strengthens the model regarding spatial translations [START_REF] Lin | Network In Network[END_REF]. By experiments, we found that adding a dropout layer after the GAP layer improves the performance of the network. The best results are obtained for a dropout of 0. We have trained our model for 30 epochs using different batch sizes and compared their performances to decide the best configuration. Adam optimizer with a learning rate of 0.0001 is utilized. We have found that the model fits well in just 10 epochs for all batch sizes (Figure 4.10), so we have decided to train our model for 10 epochs to gain time. We have tried different batch sizes, 16, 32, and 64 and compare their performance to determine which value to be adopted in our model. The training and validation loss curves in each case are shown in Figure 4.11. Analyzing these curves, we have found that the training and validation loss in case of batch size 16 starts with the minimum loss value. Also, the training loss decreases the faster and this case provides the best validation accuracy. Hence,in our approach we have adopted the batch size 16. The weights from block 5 are trained for 10 epochs because we found that the model could fit very well in 10 epochs with a batch size of 16, compiled with a categorical crossentropy loss function, and Adam optimizer with a learning rate of 0.0001. We chose a small learning rate to preserve former knowledge, and avoid distorting the weights too early and too considerably.

By experiment, we have found that Adam optimizer works better than SGD especially when we test our model. Adam was more efficient and robust and could predict more correct labels. with an accuracy of 99.99%, 99.99%, 99.87%, and 99.77%, respectively.

2-Face pose effect

We have evaluated the effect of the face pose on the identification accuracy in three classes: vitiligo, angioedema, and melasma. We have taken a test image, and we have applied two rotations of of 30

Facial skin disease identification using EfficientNet b0

The second approach used to perform facial skin disease identification is transfer learning with efficientNet b0. The architecture of EfficientNet b0 has been modified.

We have added the same layers we have used to build the FSDNet. We have added

Comparative study

In Table 4.8, we compare the test prediction results of each model used in our approach, in terms of the number of correct labels and accuracy of prediction. which is characterized by the stack of a small sized kernel 3 x 3. Using this stack is better than using a large sized kernel because numerous nonlinear layers augment the depth of the model, and thus it can learn more complex features at a low cost.

Moreover, small size kernels help to maintain the finer level properties of the image.

This explains the superiority and the advantage of FSDNet in facial skin diseases identification among the other models.

Conclusion

In this chapter, we have presented a complete end-to-end computer vision and a deep learning method to perform facial skin disease identification using RGB images. To do so, first we built our facial skin disease dataset (FSDD) including As perspectives, we can implement FSDNet, seeing that it is the most accurate model adapted to facial skin diseases identification, into a real time acquisition system that acquires face images and predicts directly the corresponding class. On the other hand, the model can be improved to identify more diseases and to measure the severity of the disease. Other pre-trained architectures can also be explored using transfer learning. Also, we can work on increasing the size of FSDD by adding more classes and more images. We can use Generative adversarial networks, GAN, to generate new synthetic images referring to the different classes identified in our approach and study their effect on the performance of the models.

  consiste à analyser des images et à les classer dans une catégorie parmi un ensemble prédéfinies. Contrairement aux techniques traditionnelles de classification d'images, les algorithmes d'apprentissage profond combinent les tâches d'extraction de caractéristiques et de classification. Dans ce contexte, de nombreux algorithmes ont été proposés. En particulier, les réseaux neuronaux convolutifs (CNN) sont des algo-rithmes d'apprentissage profond les plus populaires utilisés pour la classification des images. Il s'agit de réseaux inspirés de la manière dont le cerveau humain reconnaît l'information. Les CNN éliminent l'extraction manuelle de caractéristiques de bas et moyen niveaux. Dans la littérature, plusieurs modèles CNN ont été proposés permettant d'atteindre une grande précision de classification et dans certains cas, peuvent dépasser même la reconnaissance humaine.

  neurones convolutifs (CNN), en explicitant les blocs et couches constituant le réseau, ses différentes architectures, ainsi que son mode d'apprentissage. Enfin, nous procédons à l'utilisation des CNN pour la classification d'images. Dans le troisième chapitre, nous abordons l'architecture des différents réseaux VGG, Inception v3, et EfficientNet b0, que nous avons utilisés comme base pour construire nos modèles adaptés à la classification des pathologies cutanées du visage. Le VGG est considéré comme une architecture générale et comme une référence pour de nombreuses tâches et ensembles de données en dehors d'ImageNet, la base de données sur laquelle le réseau est entraîné initialement. Inception v3 est la troisième édition du CNN Inception de Google. Alors que la plupart des CNN populaires travaillent sur l'augmentation de la profondeur du réseau, les modèles Inception utilisent d'autres techniques pour améliorer les performances du réseau en termes de vitesse et de précision. De nombreuses versions ont été créées, chaque version étant une amélioration de la précédente. Les modèles EfficientNet, récemment développés par Google, présentent un nouveau concept de mise à l'échelle des réseaux pour améliorer les performances, appelées "Compound Scaling". Les modèles CNN précédents suivent la méthode classique de mise à l'échelle arbitraire d'une seule dimension et d'ajout de couches supplémentaires. Les modèles EfficientNet mettent uniformément à l'échelle toutes les dimensions (i.e. profondeur, largeur et résolution) en utilisant un coefficient composé. Ils améliorent la précision et l'efficacité du modèle. Nous expliquerons également comment nous pouvons appliquer ces réseaux pré-entraînés sur ImageNet pour classer d'autres images de catégories différentes de celles d'ImageNet.

International conference 1 .

 1 Rola EL SALEH, Sambit BAKHSHI and Amine NAIT-ALI. Deep convolutional neural network for face skin diseases identification. 2019 Fifth International Conference on Advances in Biomedical Engineering (ICABME), 2019, pp. 1-4, doi: 10.1109/ICABME47164.2019.8940336.

2 .

 2 Hazem ZEIN, Samer CHANTAF, Rola EL SALEH and Amine NAIT-ALI. Generative Adversarial Networks Based Approach for Artificial Face Dataset Generation in Acne Disease Cases. 2021 4th International Conference on Bio-Engineering for Smart Technologies (BioSMART). Book chapter Rola EL SALEH, Hazem ZEIN, Samer CHANTAF, Amine NAIT-ALI. Artificial Intelligence in Dermatology: A Case Study for Facial Skin Diseases. In: Advances in Artificial Intelligence, Computation, and Data Science. Springer International Publishing, 2021, pp. 163-178. CONTENTS CONTENTS 3.4.2 Inverted Residual Block . . . . . . . . . . . . . . . . . . . . . 3.4.3 Squeeze and Excitation Block . . . . . . . . . . . . . . . . . . 3.5 Transfer learning with CNN . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 What is a pre-trained model? . . . . . . . . . . . . . . . . . . 3.5.2 Transfer learning . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.3 Transfer learning scenarios . . . . . . . . . . . . . . . . . . . . 3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Facial Skin Disease Identification AI Based Approach 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Facial skin diseases identification method . . . . . . . . . . . . . . . . 4.3 Facial skin disease dataset . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Facial skin diseases identification using VGG-16 . . . . . . . . . . . . 4.4.1 Classification using pre-trained VGG-16 . . . . . . . . . . . . 4.4.2 Classification using transfer learning with VGG-16 . . . . . . 4.4.3 Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.4 Performance evaluation of FSDNet . . . . . . . . . . . . . . . 4.5 Facial skin disease identification using EfficientNet b0 . . . . . . . . . 4.6 Facial skin disease identification using transfer learning with Inception V3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Comparative study . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . General conclusion and perspectives Bibliography List of Figures 1.1 Different types of acne . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Examples of actinic keratosis on the face . . . . . . . . . . . . . . . . 1.3 Angioedema in face . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Blepharitis in face . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Eczema images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 Examples of Melasma . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7 Rosacea images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.8 Vitiligo Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 AI vs Machine learning vs Deep learning . . . . . . . . . . . . . . . . 2.2 Machine learning and Deep learning technique . . . . . . . . . . . . . 2.3 ML vs DL algorithms performance . . . . . . . . . . . . . . . . . . . 2.4 Comparison between ML and DL . . . . . . . . . . . . . . . . . . . . 2.5 Hierarchical feature learning . . . . . . . . . . . . . . . . . . . . . . . List of Tables 3.1 VGG configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 EfficientNet B0 baseline network architecture . . . . . . . . . . . . . . 4.1 Training and Validation times . . . . . . . . . . . . . . . . . . . . . . 4.2 Training accuracy and loss . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Validation accuracy and loss . . . . . . . . . . . . . . . . . . . . . . . 4.4 Training and validation times . . . . . . . . . . . . . . . . . . . . . . 4.5 Training and validation accuracies and loss . . . . . . . . . . . . . . . 4.6 Number of correct and incorrect predicted labels . . . . . . . . . . . . 4.7 Training and validation accuracy loss of fine-tuned EfficientNet . . . . 4.8 Summary of results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.9 Summary of test images classification . . . . . . . . . . . . . . . . . .

Chapter 1 Facial

 1 skin diseases: generalities evolved. These systems simplify the detection and improves the accuracy of prediction and classification of facial skin diseases and minimizes the missed diagnoses. They are based on artificial intelligence using machine learning. These systems interpret, process and predict or classify. Many machine learning based methods have been suggested related to facial skin diseases classification. They use different techniques and algorithms. These approaches are based on image processing techniques, machine learning and deep learning for feature extraction and classification.

A

  Multi-Class Multi-Level classification algorithm has been suggested by N. Hameed et al. to improve the accuracy of the classification of multiple diseases. The skin lesion images are classified using two methods: a traditional machine learning based algorithm, and an advanced deep learning based algorithm. The first method based on machine learning consists of many phases: preprocessing, segmentation, feature extraction and finally classification. The preprocessing stage includes image resizing, and artifacts (hair, black frame and circle) removal when capturing images.

1

 1 E. Goceri has worked on an automated deep learning based approach to classify Chapter Facial skin diseases: generalities facial dermatological diseases using color digital images. The method comprises two stages. In the first stage, lesions are detected and extracted with a fully automated updated extension of the Automated Detection of Facial Disorders (ADFD) technique, named F-ADFD. This technique consists of several steps. First, the type of the noise is determined by computing the statistical features of noise signals: kurtosis and skewness. That shows that the noise in the images is Gaussian. This noise is reduced using pixel based linear filtering, specifically Weiner filtering. Then the images undergo a contrast enhancement. The images are converted from RGB space to La × b*, and then the double-type data is scaled to the range [0 1] by dividing the values in the luminosity channel by 100 since they vary between 0 and 100. The pixels in the 1% area from the top and bottom part are saturated to apply contrast enhancement only to the values in the luminosity channel by retaining the values in a x and b x channels identical. At last, the images are converted from La × b* to RGB. To set automatically the active contours, an image with the variance of the values in red and green color is generated. After the intensity normalization step, the regions with lesions are of higher intensity than the other pixels. With a k-means based clustering, a new binary image is produced and used to set the active contour. In the second stage, a deep learning based classification of the segmented lesions is performed using a pre-trained DenseNet 201. The images are classified into five facial skin diseases classes: acne vulgaris, psoriasis, hemangioma, seborrheic dermatitis and rosacea. The network is trained with 80 images from each class and tested with 21 images from each class. The lesions are classified with an accuracy of 95.4% [10]. Researchers have also developed some techniques based on deep convolutional neural networks for classification purposes, and not requiring features extraction. Z. WU et al. have used five pertained CNN models: ResNet-50, Inception-v3, DenseNet121, Xception, and Inception-ResNet-v2 to classify six facial skin diseases: seborrheic keratosis, actinic keratosis, rosacea, lupus erythematosus, basal cell car-Chapter 1

1

 1 Shanti et al. have proposed an automatic skin disease classification approach using Convolutional Neural Network. They have used AlexNet architecture. The method can detect four skin diseases: acne, keratosis, eczema herpeticum and utricaria. The network is trained with 105 images and tested with 69 images of different skin tones, location of the disease and different acquisition systems, from DermNet dataset refer-Chapter Facial skin diseases: generalities ring to the diagnosed diseases. The model achieves an accuracy of 98.6% to 99.04% [13]. A deep learning based classification method has been proposed by I. Iqbal et al. They have developed a deep Convolutional Neural Network model, being called Classification of Skin Lesions Network (CSLNet), to classify eight skin diseases: melanoma, melanocytic nevus, basal cell carcinoma, actinic keratosis, benign keratosis lesion, dermatofibroma, vascular lesion, and squamous cell carcinoma. CSLNet contains less filters and parameters to enhance efficacy and performance. The proposed network is formed of many layers and numerous filter sizes gathered into four units. Each unit comprises a different number of convolutional layers of various filter sizes, preceded by a batch normalization layer, and leaky ReLU activation function.

•

  Cysts are large, painful pimples filled with pus. They can leave scars on the skin (Figure 1.1).

Figure 1 . 1 :

 11 Figure 1.1: Different types of acne: (a) whitehead, (b) blackhead, (c) pustule, (d) papule, (e) cyst, (f) nodule [58].

Figure 1 . 2 :

 12 Figure 1.2: Examples of actinic keratosis on the face [58].

Figure 1 . 3 :

 13 Figure 1.3: Angioedema in face: (a) lips, (b) tongue, (c) eyes [58].

1

 1 Blepharitis is the inflammation of the eyelid margin. It exists in two types of blepharitis: anterior blepharitis occurring at the external eyelid where the eyelashes attach, and posterior blepharitis affecting the inner eyelid in contact with the eyeball (Figure 1.4). Blepharitis happens when oil glands in the eyelids are blocked, provoking irritation and redness. It also may be caused by bacteria settling in the eyelashes, skin condition (dandruff of the scalp), rosacea, contact allergy to substances such as Chapter Facial skin diseases: generalities cosmetic products and contact lens solutions. Persons with blepharitis suffer from different symptoms in the eyes including burning feeling, redness, dryness, and excessive tearing, and in the eyelids such as inflammation, itching, swelling and crusting.They may also experience blurry vision, sensitivity to light and inflammation of the cornea[19].

Figure 1 . 4 :

 14 Figure 1.4: Blepharitis in face: (a) Posterior blepharitis, (b) Anterior blepharitis [58].

Figure 1 . 5 :

 15 Figure 1.5: Eczema images: (a) Atopic dermatitis in children; (b) Atopic dermatitis in adults; (c) Seborrheic dermatitis; (d) Allergic contact dermatitis; (e) Irritant contact dermatitis [58].

Figure 1 . 6 :

 16 Figure 1.6: Examples of Melasma: (a) on the cheeks; (b) on the forehead [58].

Figure 1 . 7 :

 17 Figure 1.7: Rosacea images: (a) acne-like rosacea; (b) rosacea with visible blood vessel [58].

Figure 1 . 8 :

 18 Figure 1.8: Vitiligo Images [58].

  Deep learning is a subset of machine learning, which in turn, is a subset of artificial intelligence. The relationship between the three terms is represented in (Figure 2.1). The deep learning revolution started with the necessity of high accuracy predictive models for unstructured data like images, audio and natural language. Breakthroughs in artificial neural networks lead to the explosion of deep learning. Deep learning algorithms have ameliorated the capability to classify, identify, and detect. They can process huge number of features which makes them very efficient when working with unstructured data. Deep learning is currently used in several fields such as customer experience like chatbots, aerospace and military, industry, text generation, healthcare and medical research, and also in computer vision as image classification, speech recognition, object detection, and natural language processing. Big data, new machine learning methods, algorithmic ameliorations, novel neural networks models, powerful computing systems and also human-to-machine development are all factors advancing deep learning.

Figure 2 . 1 :

 21 Figure 2.1: AI vs Machine learning vs Deep learning: A diagram describing the relation between artificial intelligence, machine learning and deep learning [27].

Chapter 2 Figure 2 . 2 :Figure 2 . 3 :

 22223 Figure 2.2: Machine learning and Deep learning technique: In machine learning technique, the features are manually extracted from the input, and then fed to a machine learning classifier to predict the output. While in deep learning, the network extracts automatically the features and learn them to perform the classification and predict the output.

Figure 2 . 5 :

 25 Figure 2.5: Deep neural network consisting of stacking layers learning more complex and distinct features as we go deeper in the network. Given an input image, the lower level layers detect the edges. The next layers detect the corners and contours.Combining the features extracted in the previous layers lead to learn more complex features. The output layer classifies the input.

2. 5 2 Figure 2 . 6 :

 5226 Figure 2.6: Artificial neural network architecture: the network consists of an input layer, hidden layer and an output layer. Each layer contains several neurons receiving inputs from previous neurons.

Figure 2 . 7 :

 27 Figure 2.7: Structure of ANN: the inputs are multiplied by the weights and then summed. A bias can be added and then the weighted sum is activated by an activation function to generate the output. The activation function determines if the neuron should be activated or not.

Chapter 2

 2 Deep learning based skin disease classification transformation used to prevent the convergence of the network, normalize the output between [0,1] or [1,-1] and reduce the computation time. There are many types of activation types. The most common ones are: step function, sigmoid function, hyperbolic tangent and rectified linear unit.

Figure 2 . 10 :

 210 Figure 2.10: Hyperbolic tangent [31].

Figure 2 . 12 :Figure 2 . 13 :

 212213 Figure 2.12: Deep neural network architecture: the network is composed of an input layer, n hidden layers and an output layer.

Chapter 2

 2 Deep learning based skin disease classification the whole image and this enables it to detect that feature anywhere in the image. This is known as translation invariance. The result of multiplication between the filter and the input one time gives a single value. As the filter is applied numerous times to the input, we obtain a 2D output array called "feature map" (Figure2.14).

Figure 2 . 14 :Figure 2 . 15 :

 214215 Figure 2.14: Example of a filter applied to an input to produce a feature map

Figure 2 . 16 :

 216 Figure 2.16: Basic CNN architecture. The network is mainly composed of an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer [38].

  to a spatial region of the input. the size of this spatial region is called the receptive field F of the neuron. The size of the output is controlled by three parameters: the depth, stride and zero padding. The depth of the output is a hyperparameter, equal to the number of filters applied in the current layer. The stride S refers to the distance between two consecutive slides of the filters. It affects the size of the output volumes. Large strides contribute to less interfered receptive fields and small output volumes, while smaller strides result in an interfered receptive field and large output volumes (Figure2.18).

Figure 2 . 18 :

 218 Figure 2.18: Example of stride. Given an input image 5x5 along with a 3x3 kernel. (a) When the stride is 1, the filter jumps one pixel at a time from left to right and top to bottom producing an output 3x3. (b) When the stride is 2, the filter slides two pixels at a time along the whole image providing a smaller output of size 2x2 [38].

Figure 2 . 19 :

 219 Figure 2.19: Example of zero padding. Given an input image 4x4 convolved with a 3x3 kernel. (a) Without padding, the convolution results in an output of size (4-3+1) =2, 2x2. (b) with padding P = 1, the spatial size of the output volume increases to (4+2-3+1)= 4, 4x4. The input and the output have the same size [38].

Figure 2 . 20 : 4 - 2

 22042 Figure 2.20: Example of max pooling. Given an input image 4x4. (a) we apply a 2x2 max pooling with a stride of 1. (b) we apply a 2x2 max pooling with a stride of 2. The spatial size of the input is reduced [38].

Figure 2 . 21 :

 221 Figure 2.21: Example of fully connected layer. The FC layer is preceded by a flatten layer that converts the output of the convolutional layers into 1 dimensional array.The neurons in the FC layer are fully connected to all activations in the flatten layer. FC layer is followed by an output layer that computes the output.

6 - 2

 62 Batch normalization layer Batch normalization (BN) layers are used to stabilize and accelerate learning of the network by diminishing the number of training epochs required to train the network. They can be inserted before or after the activation. This can be determined by experiment. BN layers normalize the input values of the next layer for each mini batch. Furthermore, they can prevent overfitting and help attain high classification Chapter Deep learning based skin disease classification accuracy in less epochs compared to the same model without batch normalization layer [39]. 7-Dropout Dropout is a regularization technique used in CNN to prevent overfitting caused by the connection of all the features to the FC layer. Overfitting happens when a model performs well on the training set provoking a negative effect in the model's performance when applied on a new data. The dropout layer randomly selects neurons and sets them to zero during the training (Figure 2.22). Therefore the size of the model is reduced [40].

Figure 2 . 22 :

 222 Figure 2.22: Example of dropout: (a) A standard network with one hidden layer without dropout. (b) The same network after dropout of probability p = 0.5. 50% of the neurons is randomly dropped out from the model [40].

Figure 2 . 23 :

 223 Figure 2.23: Training process of CNN.The performance of the model, with specific kernels and weights, is computed with a loss function, by forward propagation, quantifying the agreement between the predicted class labels and the ground truth ones. The kernels and weights are updated by backpropagation using gradient descent optimization algorithm[START_REF] Yamashita R | Convolutional neural networks: an overview and application in radiology[END_REF].

2 . 10 )

 210 Where w is the learnable parameter, α the learning rate and L the loss function. The learning parameter is a hyper parameter that should be determined before starting the training. Actually, the gradient descent is slow to run on big datasets. Instead, we use Stochastic Gradient Descent (SGD), a modification of the gradient descent algorithm, that calculates the gradients and updates the parameters on mini batches of the training dataset, rather than the whole training set. The mini-batch size is a hyper parameter. The typical batch sizes are 32, 64, 128, and 256. SGD is the most commonly used algorithm to train deep neural networks[START_REF] Ruder | An overview of gradient descent optimization algorithms[END_REF] (Figure2.24).

Figure 2 . 24 :

 224 Figure 2.24: Gradient descent algorithm: it consists of iteratively updating the learnable parameters to minimize the loss, which calculates the distance between a predicted class label and a ground truth label. The gradient of the loss function determines the direction in which the function has the steepest descent. The parameters are updated in the negative direction of the gradient with a definite learning rate [47].

  image classification techniques relied on raw pixel data. The problem is that images of the same thing could be different. Many factors vary such as: backgrounds, sizes, viewpoint, deformation, illumination, intra-classes variation, and poses. This makes it challenging for a computer to correctly see and classify images. This problem is solved with deep learning. Deep learning approaches have a robust learning ability, that incorporates the feature extraction and classification process to perform the image classification test, that enhances the accuracy of classification. Image classification in deep learning is a data driven approach. It is a supervised learning, consisting of giving examples of each class and teaching our model to identify the difference between the classes based on these examples. These examples are called training dataset, where each example in the dataset is an image with a label. The labels are used by the algorithm to teach itself how to identify each class.

  First, we have to gather our dataset. The dataset comprises labelled images. The labels define a finite set of classes (cat, dog, cup, car. . . .). The number of images within each class should be identical. Unbalanced classes lead the classifier to overfitover fit. Second, the dataset is created, we split it into two parts: a training set, and a test set. The training set is used to train the classifier to learn how each class looks. It predicts the input data and then adjusts itself when the predictions are incorrect. After being trained, the performance of the classifier is evaluated on the testing set. The training set and testing set should be different and don't overlap to evaluate correctly the performance of the classifier. If the testing set is a part of the training set, the classifier would have already seen the example and learned from it. We have many split cases. Actually a third set could be added, the validation set. It is a part of the training set and used as a fake test set. Generally, it forms 10-20% the training set. The third step is training the network using the training set. The model learns how to identify each class in the labeled data. CNN detects low level characteristics like edges in the first layer, and the following layers combine

  CNNs show a huge advance in image recognition. They are used to analyze images and are widely used in image classification due to many reasons. CNNs are endto-end models which allows the user to skip the hand-engineered feature extraction stage. Instead, the network learns the features. The network is fed by raw input data and then learns filters that can differentiate between classes in the hidden layers. CNNs reduce the number of parameters without losing the features. The number of parameters increases with the number of layers which can slow down the training process and thus makes parameters tuning a massive task. CNNs decrease the tuning time. Layers of a CNN have numerous convolutional filters that work and scan the entire feature matrix and perform dimensionality reduction. This makes CNN a suitable and appropriate network for image classification.

  this chapter, we have explained what deep learning is, showed how it is correlated to the terms AI and machine learning and how it works. Deep learning is implemented using neural network architectures, hence the nomenclature of deep neural networks (DNN). They comprise several layers stacked on top of each other, and learn in a hierarchical mode. It exists comes in different architectures of DNN divided into supervised and unsupervised learning. We also discussed CNNs, belonging to supervised learning, that are mainly used in image classification. CNNs are composed of a several layers where each one has a specific task.. CONV layers, the core of CNNs, learn a set of K kernels, each of size F x F. They are followed by activation layers to get a non-linear transformation. The spatial dimensions of the input are reduced by pooling layers. FC layers end the network, in which the neurons are fully connected to all activations in the preceding layer. They are followed by a softmax classifier to predict the output. Batch normalization and dropout layers could be added to the network to normalize inputs before passing it to the next layer, and to prevent overfitting, respectively. Backpropagation algorithm is mainly used for training CNNs. During training, we compute the loss which is the difference between the predicted labels and the ground truth labels, and the gradient. The weights are thus updated using the gradients to minimize the loss through forward propagation on a training set. Finally, we learned what image classification is and how it is performed by deep learning. Deep learning classification consists of gathering the dataset, splitting data into training, testing and validation sets, training the network, and at last evaluating the model. CNNs, end-to-end models, are ideal for image classification. They eliminate manual feature extraction. Instead, the network learns the features that can differentiate amongst image classes. In the next chapter, we will present the different CNNs architectures that we have used to classify facial skin diseases images and the results obtained from each one. There are various popular CNN architectures. Generally, most of deep CNNs are composed mainly of basic layers such as a convolutional layer, pooling layer, fully connected or dense layer, and softmax classifier. The different deep networks

Figure 3 . 1 :

 31 Figure 3.1: VGG architecture. The model includes an input, several stacked CONV layers of different depths, Maxpooling layers, three fully connected layers, and softmax layer performing the classification [44].

VGG- 16

 16 is considered one of the most important pretrained models for image classification. It has 16 weight layers; 13 CONV layers and 3 FC layers. It includes only CONV layers of fixed kernel size 3 x 3 and a stride of 1 and 2 x 2 Maxpooling layers of stride 2 inserted between CONV layers to down sample the input size. The network has 138 million parameters and trained on 1.2 million images from Imagenet dataset to classify objects into 1000 different classes. The VGG-16 could classify images with an accuracy of 92.7%. The architecture of VGG-16 is presented in Figure 3.2. The VGG-16 takes as input an image of size 224 x 224 x 3 referring to height x width x depth where depth is the number of RGB channels. As mentioned previously, VGG-16 is composed of several blocks of CONV followed by ReLU activation functions and Maxpooling layers. The first block has 2 CONV layers of size 224 x 224 x 64 each one, and a Maxpooling layer reducing the spatial dimension of the images to 112 x 112 x 64. The second is formed of 2 CONV layers of size 112 x 112 x 128 each, and a Maxpooling layer that reduces the volume size to 56 x 56 x 128. The third constitutes of 3 CONV layers; each of size 56 x 56 x 256, followed by a Maxpooling layer that reduces the image size to 28 x 28 x 256. In the fourth block, there are also 3 CONV layers of size 28 x 28 x 512 each and a Maxpooling layer reducing the image to 14 x 14 x 512. The fifth block has 3 CONV layers too of size 14 x 14 x 512 each one, and a Maxpooling layer with 7 x 7 x 512. These sets are followed by two FC layers with 4096 nodes each one, and one FC layer with 1000 nodes and a softmax classifier (Figure 3.2).

Figure 3 . 2 :

 32 Figure 3.2: VGG16 architecture.The network is composed of 13 convolutional layers with filter size 3 x 3, 5 Maxpooling layers embedded between CONV layers, 3 fully connected layers,and softmax layer arranged in a specific architecture. The input is an RGB image of size 224 x 224 [44].

Figure 3 . 3 : 3 Convolutional neural networks architectures 3 x 3

 3333 Figure 3.3: Factorized convolutions: (a) original inception module including 5 x 5 convolutions; (b) new factorized module, called Module A, where 5 x 5 filter in the original module is replaced by two 3 x 3 filters. Pool denotes pooling layer, and m x n denotes a CONV layer, where m and n determine the size of the convolution.

Figure 3 . 4 :

 34 Figure 3.4: Asymmetric Convolutions: (a) Inception Module B where 7 x 7 filter is replaced by 1 x 7 followed by 7 x 1 filters; (b) Inception Module C where 3 x 3 filter is replaced by 1 x 3 followed by 3 x 1 filters. Pool denotes pooling layer, and m x n denotes a CONV layer, where m and n determine the size of the convolution.

Figure 3 . 5 :

 35 Figure 3.5: Auxiliary Classifier in Inception-v3 acting as regularizer that consists of average pooling, convolution, FC, and softmax layers.

3. 6 .

 6 All these concepts presented above are integrated into the final architecture of Inception-v3, consisting of 42 layers. It includes several layers, such as convolution, average pooling, max pooling, concat, dropout, fully connected, and softmax. Batch normalization and ReLU are used after CONV layers. It has 11 inception modules.

Figure 3 . 6 :

 36 Figure 3.6: Efficient grid size reduction: (a) Detailed architecture of efficient grid size reduction that reduces the grid size while augments the filter banks; (b) Efficient grid size reduction diagram representing the feature maps sizes resulting from each operation.

Figure 3 . 7 :

 37 Figure 3.7: Architecture of Inception-v3: Inception-v3 is made up of different type of layers with a specific topology and gathered in blocks, each performing a specific task. It includes 11 Inception modules: 5 Inception modules A, 4 Inception Modules B, and 2 Inception Modules C, 2 Grid Size Reduction blocks, and 1 Auxiliary Classifier. It takes in input an image of size 229 x 229 x 3 [52].

Figure 3 . 9 :

 39 Figure 3.9: The Swish activation function [55].

3. 4 . 2

 42 Inverted Residual BlockInverted Residual Block, or MBConv Block, is a class of residual block using an inverted architecture for efficiency purposes. It was initially used in MobileNetV2 network[START_REF] Sandler | MobileNetV2: Inverted Residuals and Linear Bottlenecks[END_REF], and reused in numerous mobile extension networks. Residual blocks create a shortcut connection between the start and end of a convolutional block.They connect the wide layers having the high number of channels. They follow a wide → narrow (bottleneck) → wide approach. Inverted Residual blocks are the opposite. They have a narrow → wide → narrow structure, inspired by the fact that bottlenecks include all the significant information. They have less parameters (Figure3.10).

Figure 3 .

 3 Figure 3.10: A comparison between a residual block and an inverted residual block: (a) A residual block connects wide layers that have large number of channels. Layers in between are bottleneck; (b)An inverted residual block connects bottleneck layers, while layers in the center are wide [54].

3. 5 . 1

 51 What is a pre-trained model?A pre-trained model is a trained model on a large dataset that has learned to extract efficient features from images and are then used as the starting point to perform a new task. The pre-trained network can be downloaded and used as-is to classify new images. There exists a standard benchmark dataset for image classification in the computer vision and machine learning literature that are used to train the models.The most famous ones are:• MNIST (Modified National Institute of Standards and Technology dataset) is a dataset constituted of labelled gray scale images of handwritten single digits between 0 and 9. It includes 60,000 training images and 10,000 testing images of size 28×28.

Figure 3 . 13 :

 313 Figure 3.13: Transfer learning scenarios: (a) train the entire model, (b) fine-tuning method: some layers (convolutional, and the classifier) to be retrained, while some to be kept frozen, (c) Feature extractor: applying a new classifier on top of the pre-trained network and retraining it, while freezing the convolutional base.

4 Facial

 4 VGG is a deep pre-trained ConvNet widely used for image classification and identification. It forms a baseline network for many tasks and datasets beside ImageNet.The main contribution is to show in depth the benefits on the performance of the network. It is characterized by the use of a small receptive field 3 x 3 of stride 1. VGG has different configurations: VGG-11, VGG-13, VGG-16, and VGG-19. All have the same architecture but differ in the number of layers. VGG-16 is composed of 16 weight layers; 13 CONV layers and 3 FC layers. This model includes CONV layers followed by a ReLU activation function, max pooling layers, and FC layers. It takes as input an RGB image of size 224x224x3. It is pre-trained on ImageNet ILSVRC dataset and achieves a top-5 test accuracy of 92.7%. Inception v3 is an optimized version of GoogleNet belonging to the Inception family. Inception models rely on the Inception modules. The contribution is the use of factorization ideas to build the network. It has been developed using different techniques including factorizing convolutions, dimension reduction, and one auxiliary classifier. It has 11 inception modules of 3 kinds; Inception Module A, Inception Module B, and Inception Module C. Inception v3 is made up of 42 layers of different types; CONV, average pooling, max pooling, concat, dropout, FC, and softmax layers forming several blocks having different functions. It takes an image of size 229 x 229 x 3 as input. Inception v3 is pre-trained on ImageNet ILSVRC and reaches a classification accuracy greater than 78.1%. It achieves the lowest error rates with comparison to the state-of-the-art. EfficientNet models presented a new technique to scale up CNNs to enhance the performance of the network and denoted compound scaling method. It consists of balancing the three dimensions: depth, width, and resolution at one time by scaling Chapter

  facial pathologies from a single face image. Only the phenotypes of the face are used in different face poses, illumination and image resolution. Our approach does not require extraction of Region of Interests (ROI). Our method could classify the images into ten classes that are 8 facial skin diseases, normal face class, and no face class. The classification is achieved through three based deep CNN architectures: VGG-16, Inception v3, and EfficientNet b0 while using transfer learning and fine-tuning and has yielded different results. These models were trained and validated using a dataset, containing 20000 facial skin images gathered from various sources, that we have created due to the unavailability of any public database including the identified diseases in our work. The proposed approach is composed of four phases which are gathering and labeling facial skin disease images, image preprocessing including resizing of images and data augmentation, network training and validation, and identification of the lesions. The general block diagram of the method is illustrated in Figure 4.1.

Figure 4 . 1 :

 41 Figure 4.1: Bloc diagram of facial skin diseases identification. Facial skin images are gathered and labelled. They undergo some preprocessing such as resizing, and data augmentation. Then, these images are used to train the network to finally identify the probable diseases.

9 (

 9 the data to the network. It is called offline augmentation. New samples generated by horizontal flip and rotation are used. Images are rotated at different angles (5°, -5°, 10°, -10°, 30°, -30°, 45°, and -45°) (Figure 4.2). Thus, our dataset has in total 20000 images and 2000 images in each class divided randomly into training and validation sets where each image is associated with one of ten labels from 0 to Figure 4.3). The training dataset is used to train our classifiers to learn what every class looks like while. While the validation set is used to evaluate the model while tuning the model hyper-parameters. We also created a small dataset that includes 20 images in each of the ten classes in our facial skin diseases dataset and is used to assess the performance of our fully trained classifiers. These images are collected from Dermweb [58]. These images are of different resolutions. Since our classifiers take an RGB image of size 224 x 224 x 3, we rescale all the images in our dataset to the appropriate size before feeding them to the network.

Figure 4 . 2 :

 42 Figure 4.2: Sample images from our database: (a) original image,(b) horizontal flip image, rotated images with angles (c) -5°, (d) 5°, (e) -10°, (f) 10°, (g) -30°, (h) 30°, (i) -45°, (j) 45°.

Figure 4 . 3 :

 43 Figure 4.3: Images from our dataset referring to the ten classes: 8 facial skin diseases (Acne, Actinic Keratosis, Angioedema, Blepharitis, Eczema, Melasma, Rosacea and Vitiligo), normal skin class, and no face class including different images such as animals, cups, coins, phones...

  we replace the original Softmax classifier by a 10-dimension output vector to fit the number of predicted classes. The original architecture of the pre-trained VGG-16 and the modified architecture for facial skin diseases identification are presented in (Figure 4.4) and (Figure 4.5) . All the layers are frozen during training, and only the new classifier is trained. Hence, among 134 million parameters, we train only 40970 parameters. The pre-trained VGG-16 takes an RGB image of size 224 x224 x 3 as input. First, the data is loaded into the model and the classes are mapped to an integer label between 0 and 9 (we have 10 classes) and then converted to one-hot encoding. To train the network, the FSDD data is split into training and validation sets. We assess the performance of the network using 5 split cases 90:10, 80:20, 70:30, 60:40, and 50:50, for training versus validation respectively. The split of data helps to see if the network overfits and if we have to regulate some hyper-parameters such as to lower the learning rate, increase the number of epochs if the validation accuracy is higher than the training accuracy or stop the overtraining if the training accuracy varies higher than the validation. We trained the model for 30 epochs with batch size of 16, compiled with categorical crossentropy loss function, and Adam optimizer with a learning rate of 0.0001. The five blocks including CONV2D and Maxpooling2D layers, Flatten layers, and FC layers are kept the same in both architectures. The Dense layer of 1000 nodes is replaced with a 10 dimension one to fit the number of predicted classes in our method.

Figure 4 . 4 :

 44 Figure 4.4: Model summary of the original architecture of pre-trained VGG-16.

Figure 4 . 5 :

 45 Figure 4.5: Model summary of the modified architecture adapted to facial skin diseases identification.

2 )

 2 We plot the variation of training and validation accuracies through the 10 epochs as well as the training and validation loss, in each data split case in Figure 4.6. In all cases, we observe that the training and validation accuracy are both increasing, and the training and validation loss are decreasing together. Now that we have trained the classifier, we will compute the metric values such as precision, recall, and F-1 score, to evaluate the model performance. The precision evaluates the accuracy of a model to classify a positive sample. It is the ratio of correct positive samples to the total number of positive predicted samples. The recall determines the ability of a model to detect positive samples. The number of detected positive samples increases proportionally with the recall. It is given by: Recall = T P T P + F N (4.3) The F1-score measures the accuracy of a model on a dataset. It assesses the strength and precision of the classifier. A good classifier with a good F1-score gives low false positives and low false negatives. It is comprised between 0 and 1. True Positives, TN = True Negatives, FP = False Positives, and FN = False Negatives.We calculate these metrics to assess the performance of the classifier in each class for the different split cases shown in Figure4.7. To see how well our model is performing, and what types of error it is making, we compute the confusion matrix that provides a summary of prediction results. Hence, the confusion matrix is a N x N matrix, where N is the number of predicted classes that reveal how the model is confused while making predictions. We also compute the confusion matrices in the five split cases shown in Figure4.8. As we can see in the confusion matrices, the correct predicted values are way greater than the wrong predicted ones.

Figure 4 . 6 :

 46 Figure 4.6: Plots of training and validation accuracy, and training and validation loss in the five split cases (a) 90:10,(b) 80:20,(c) 70:30,(d) 60:40,(e) 50:50. We observe that the training and validation accuracy are both increasing, and the training and validation loss are decreasing together. 106

Figure 4 . 7 :

 47 Figure 4.7: Evaluation metrics in different split cases for each class (a) 90:10, (b) 80:20, (c) 70:30, (d) 60:40, (e) 50:50, where we compute for each class the precision, recall, and F1-score. The column support represents the number of test samples available for validation, while macro avg stands for macro average that calculates the average without taking into account the proportion of each class from the total number of samples, and weighted average calculates the average with consideration of the proportion. The performance of the model varies between the classes. Some classes are classified better than the others.

Figure 4 . 8 :

 48 Figure 4.8: Confusion matrices in (a) 90:10, (b) 80:20, (c) 70:30, (d) 60:40, and (e) 50:50 for training versus validation respectively. The vertical axis of the confusion matrix represents the true label belonging to the ten classes (Acne, Actinic keratosis, Angioedema, Blepharitis, Eczema, Melasma, Rosacea, Vitiligo, Normal skin, and no face), and the horizontal axis is the predicted label. The diagonal of the matrix in dark color shows the number of correctly predicted samples while all the other numbers present the wrong predicted samples in each class.

5 .

 5 The FSDNet also takes in input as RGB image of dimension 224 x 224 x 3. Data is initially loaded into the model, and the classes are mapped (from 0 to 9) and converted to one-hot encoding. As the aim of transfer learning, CONV weights are frozen which, means that they are set and are not updated during training. The features learned by the pre-trained network are preserved and can be useful for several computer vision issues, even when dealing with totally different categories than those of the initial task. You can keep all of them frozen or freeze some and train the others.

Figure 4 . 9 :

 49 Figure 4.9: FSDNet architecture. The convolutional base is the same as in the original architecture of the pre-trained VGG-16. The classifier consists of a global average pooling, a dropout, and a 10-diemnsion softmax denoted by dense 1.

Figure 4 . 10 :

 410 Figure 4.10: Training and validation accuracies and loss for 30 epochs. The training accuracy and loss are almost stable after 10 epochs. They do no't vary significantly.

Figure 4 .

 4 Figure 4.11: (a) Training loss, and (b) validation loss in different batch sizes 16, 32, and 64.

Figure 4 . 12 :

 412 Figure 4.12: Training and validation accuracy, and training and validation loss curves in the five split cases (a) 90:10, (b) 80:20, (c) 70:30, (d) 60:40, (e) 50:50. In all cases, we observe that the training and validation accuracy are both increasing, and the training and validation loss are decreasing together. 115

Figure 4 . 13 :

 413 Figure 4.13: Classification report in different split cases for each class (a) 90:10, (b) 80:20, (c) 70:30, (d) 60:40, (e) 50:50, where the precision, recall, and F1-score are calculated to assess the model. The column support represents the number of test samples available for validation, macro avg stands for macro average that calculates the average without taking into account the proportion of each class from the total number of samples, and weighted average calculates the average with consideration of the proportion. The classifier could identify some classes with higher accuracy than others.

Figure 4 . 14 :

 414 Figure 4.14: Confusion matrices of FSDNet in (a) 90:10, (b) 80:20, (c) 70:30, (d) 60:40, and (e) 50:50 for training versus validation respectively. The vertical axis of the confusion matrix represents the true label belonging to the ten classes (Acne, Actinic keratosis, Angioedema, Blepharitis, Eczema, Melasma, Rosacea, Vitiligo, Normal skin, and no face), and the horizontal axis is the predicted label. The diagonal of the matrix in dark color shows the number of correctly predicted samples and all the other numbers present the wrong predicted samples in each class.

Figure 4 . 15 :

 415 Figure 4.15: Study of the effect of brightness on the performance in class Eczema. (a) The original image is identified as Eczema with an accuracy of 99.9%. (b) The brightness is modified by a factor of 0.8, and Eczema is predicted with an accuracy of 99.1%. (c) The brightness is changed by a factor of 0.6, and the accuracy of identification of Eczema is 98.5%. (d) Finally, with a modification by a factor of 0.5, the image is classified as Eczema with an accuracy of 95.1%.

Figure 4 . 16 :

 416 Figure 4.16: Study of the effect of brightness on the performance in class Acne. (a)The original image is identified as acne with an accuracy of 99.99%. (b) The brightness is modified by a factor of 0.8, and the identification accuracy of acne is 99.99%. (c) Acne is identified with an accuracy of 99.96% when the brightness is varied by a factor of 0.6. (d) The image becomes darker with a brightness modification by a factor of 0.5, and the image is predicted as acne with an accuracy of 99.93%.

Figure 4 . 17 :

 417 Figure 4.17: Study of the effect of brightness on the performance in class Normal. (a) The original image and images with brightness modified by a factor of (b) 0.8, (c) 0.6, and (d) 0.5. All images are correctly predicted as belonging to normal class with an accuracy of 99.99%, 99.99%, 99.87%, and 99.77%, respectively.

  • and -30 • on the image to change the face pose. All images are correctly predicted with accuracies varying according to the class as presented in Figure 4.18, Figure 4.19, Figure 4.20.

Figure 4 .

 4 Figure 4.18: (a) The original image is identified as vitiligo with an accuracy of 100%.(b) When rotated 30 • the image is identified also as having vitiligo with an accuracy of 99.99%. (c) The image is rotated with an angle -30 • , and it is predicted as having vitiligo with an accuracy of 99.96%.

Figure 4 .

 4 Figure 4.19: Angioedema: The 3 images are classified as angioedema with accuracies (a) 100% for the original image, (b) 98.81% when rotated 30 • , and (c) 99.96% when rotated -30 • .

Figure 4 .

 4 Figure 4.20: (a) the original image is classified as melasma with an accuracy of 100%. (b) the image is rotated with an angle of 30 • , and the accuracy of melasma prediction is 84.91%. (c) With a rotation of -30 • , the image is classified as melasma with an accuracy of 99.89%.

  to the top of the model a global average pooling layer, a dropout of value 0.5, and a Softmax classifier of dimension 10. We have used the same architecture to make the comparison between the models meaningful. The architecture of the fine-tuned version of efficientNet b0 is presented in Figure4.21.

Figure 4 . 21 :

 421 Figure 4.21: Fine-tuned EfficientNet b0:The general architecture of EfficientNet b0 has been kept the same and 3 layers have been added to the top: a global average pooling, a dropout denoted as dropout 1, and a softmax of dimension 10 denoted dense 1. The total trainable parameters in the network is 4020358.

Figure 4 . 22 :

 422 Figure 4.22: Curves of training and validation (a) accuracy, and (b) loss: We have observed the accuracy and loss of training and validation converge around the tenth epoch and their value is almost stable.

4 Facial

 4 Since the training and validation accuracies and loss vary slightly with the number of trainable parameters, and according to the results obtained when testing the model with the test images that we have used in FSDNet, we have chosen to train our model from block 7 in addition to the adding layers at the top, which reduces the number of trainable parameters and thus the training time. To summarize the hyper-parameters of the fine-tuned EfficientNet b0, the model is used in split case 80:20 for training versus validation. It is trained for 12 epochs with batch size 16 using Adam optimizer with learning rate 0.0001. The curves of training and validation accuracy and loss are demonstrated in Figure 4.23. Experimental analysis and Chapter Skin Disease Identification AI Based Approach performance evaluations have been carried out in terms of precision, recall, and F1score which are widely preferred for the evaluation of machine learning techniques (Figure 4.24), as well as the confusion matrix (Figure 4.25).

Figure 4 .

 4 Figure 4.23: (a) Training and validation accuracy, and (b) training and validation loss curves of fine-tuned EfficientNet b0. The training and validation accuracy are both increasing, and the training and validation loss are decreasing together. There is no gap between them.

Figure 4 . 24 :

 424 Figure 4.24: Classification report of fine-tuned EfficientNet b0: The metrics precision, recall, and F1-score have been calculated for each class to evaluate the model. The column support represents the number of test samples available for validation. Macro avg stands for macro average that calculates the average without taking into account the proportion of each class from the total number of samples, and weighted average calculates the average with consideration of the proportion. The classifier has very high metrics varying between 0.99 and 1.

Figure 4 . 25 : 4 Facial

 4254 Figure 4.25: Confusion matrices of fine-tuned EfficientNet b0: The vertical axis of the confusion matrix represents the true label belonging to the ten classes (Acne, Actinic keratosis, Angioedema, Blepharitis, Eczema, Melasma, Rosacea, Vitiligo, Normal skin, and no face), and the horizontal axis is the predicted label. The diagonal of the matrix in dark color shows the number of correctly predicted samples and all the other numbers present the wrong predicted samples in each class. The model has predicted all the images correctly except for three images. One vitiligo image has been predicted as angioedema and two rosacea images have been classified as eczema.

Figure 4 .

 4 Figure 4.26: A study of the effect of brightness of the image on the performance of fine-tuned EfficientNet b0 model in 3 classes: eczema, acne, and normal skin: All the images are correctly identified but with a lower accuracy as the image gets darker.

Figure 4 . 27 :

 427 Figure 4.27: Face pose effect on the performance of fine-tuned EfficientNet b0 evaluated on 3 classes: vitiligo, angioedema, and melasma. The classes of the images are predicted correctly, but the accuracy decreases with the rotation.

4. 6

 6 Facial skin disease identification using transfer learning with Inception V3The third model for facial skin disease identification is built using transfer learning with Inception V3. We have added at the top of the original model a global average pooling layer, dropout of value 0.5, and a softmax classifier of dimension 10.We have used the same architecture used at the top of FSDNet to make the comparison between the models significant. The architecture of the fine-tuned version Inception V3 is presented in Figure4.28.

Figure 4 .

 4 Figure 4.28: Fine-tuned Inception V3: We have conserved the general architecture of Inception V3, and we have added 3 layers at the top: global average pooling, dropout denoted as dropout 1, and 10-dimension softmax classifier denoted dense 1. The total trainable parameters in the network is 21788842.

Figure 4 . 29 :

 429 Figure 4.29: Fine-tuned Inception V3: We have conserved the general architecture of Inception V3, and we have added 3 layers at the top: global average pooling, dropout denoted as dropout 1, and 10-dimension softmax classifier denoted dense 1. The total trainable parameters in the network is 21788842.

Figure 4 .

 4 Figure 4.30: (a) Training and validation accuracy, and (b) training and validation loss curves of fine-tuned Inception V3: The training and validation accuracy are both increasing, and also the training and validation loss are decreasing together.

Figure 4 . 31 :

 431 Figure 4.31: Classification report of fine-tuned Inception V3: The metrics precision, recall, and F1-score are computed for each class to assess the model. The column support represents the number of test samples available for validation while, macro avg stands for macro average that calculates the average without taking into account the proportion of each class from the total number of samples, and weighted average calculates the average with consideration of the proportion. The classifier presents high metrics varying between 0.97 and 1.

Figure 4 . 32 : 4 Facial

 4324 Figure 4.32: Confusion matrix of fine-tuned Inception V3: The vertical axis of the confusion matrix represents the true label belonging to the ten classes (Acne, Actinic keratosis, Angioedema, Blepharitis, Eczema, Melasma, Rosacea, Vitiligo, Normal skin, and no face), and the horizontal axis is the predicted label. The diagonal of the matrix in dark color shows the number of correctly predicted samples while all the other numbers present the wrong predicted samples in each class. Generally, the model has predicted correctly a great number of images. The number of incorrect labels in each class is very small.

Figure 4 . 33 :

 433 Figure 4.33: Evaluation of the effect of brightness of the image on the performance of fine-tuned Inception V3 model in 3 classes: eczema, acne, and normal skin. All the images are correctly identified but with a lower accuracy as the image gets darker.

Figure 4 . 34 :

 434 Figure 4.34: Face pose effect on the performance of fine-tuned Inception V3:. It is evaluated on 3 classes: vitiligo, angioedema, and melasma. The images are classified correctly in the correct classes. The accuracy decreases with the rotation.

  20000 labelled images referring to the eight facial skin diseases to be identified, normal skin, and no-face categories. The ten classes are balanced. This dataset is used to train and validate our models regardless of face pose, illumination, and image resolutions. We have tested four models: a pre-trained model, and three fine-tuned models adapted to facial skin diseases classification. We have adopted the same architecture of the classifier in three of the proposed models to make the comparison meaningful. The first model was the pre-trained VGG-16. After training and validation, we have evaluated the model on our testing set and found that the network has achieved an 86.19% identification accuracy. The second model is a identified in our work.We have introduced a deep learning based computer aided diagnosis system that could identify more diverse diseases than the ones handled in the literature. Our system identifies 8 facial skin diseases such as acne, actinic keratosis, angioedema, blepharitis, eczema, melasma, rosacea, and vitiligo. It can also determine if the skin is "normal" and can indicate "no face" if the image is not a facial image. For this purpose, we have built three convolutional neural network (CNN) models to predict the class of the images using a transfer learning method. CNN is a commonly used shift invariant method of extracting learnable features. CNNs have played a major role in the development and popularity of deep learning and neural networks, and are mainly used in image classification. As a first step, we have used the pre-trained VGG-16 model without changing its architecture to classify facial skin diseases. We have only replaced the softmax classifier of dimension 1000 by a new one of dimension 10 to fit the number of classes handled in our method. The aim was to see this pre-trained model on another dataset for a specific task and observe how it will perform on a new task that is facial skin diseases identification and compare it with the models that we will build using transfer learning. The model was trained and validated using a dataset that we have created and we called Facial Skin Diseases Dataset (FSDD). Actually, there is not any public dataset handling these diseases, so we had to build our own dataset composed of 20000 images referring to the ten classes that are 8 facial skin diseases, normal skin, and no face. The classes are balanced; each class has 2000 labelled images for different persons, of different genders, group ages, and races. They are also of different resolutions, illumination, and face poses. We have trained only the new softmax classifier that we have added. We trained the model for 30 epochs with batch size of 16, compiled with categorical crossentropy loss function, and Adam optimizer with a learning rate of 0.0001 with different split cases. The model performs the best when it is trained with 90% of the data and validated on 10%. Pre-trained VGG-16 achieves an accuracy of 97% and can predict the classes of unseen images with an average accuracy of 86.19%. The first model is a VGG-16 based model adapted to facial skin diseases identification.We have kept the convolutional base of VGG-16 , and replaced the top composed of a flatten layer, 2 fully connected layers, and a softmax classifier of dimension 1000 with a global average pooling layer, a dropout of 0.5, and a 10-dimension softmax classifier corresponding to the number of predicted classes. The model is trained and validated using our dataset FSDD. We have tried many split cases of data into training and validation to see which case achieves a better performance. After many trials to set the best hyper-parameters, and the best optimizer to be used, FSDNet has been trained from block 5, for 10 epochs with batch size 16, using Adam optimizer with a learning rate 0.0001. Experimental results of performance evaluation metrics have shown that FSDNet in split case 80:20 for training versus validation has the best performance, achieving an accuracy of 98.57%. It could predict the classes of test images that are not included in FSDD with a high accuracy of 98%.We have built the second model using transfer learning with EfficientNet b0. We have added to the pre-trained efficientnet b0 model the same layers added to FS-DNet at the top of the network to make the comparison and the evaluation of the classifier meaningful. The model is trained and validated on FSDD data from block 6, for 12 epochs with batch size 16 using Adam with learning rate 0.0001. Also the performance has been evaluated in different split cases, and the best performance is obtained with split case 80:20 for training versus validation, with an accuracy of 99.92%. The fine-tuned EFFicientNet model adapted to facial skin diseases identification has been tested with the same test images used in FSDNet. The built model predicts the classes with an accuracy of 89.5%. The third model we proposed is created using transfer learning with Inception V3. The same architecture adopted in the previous models is used at the top of Incepion V3. We have trained the new added layers and some previous layers as well, for 15 epochs with a batch 16 using Adam optimizer with a learning rate of 0.0001. The best metrics are obtained in the split case 80:20 for training versus validation, with an accuracy of 99%. Despite this high accuracy, when tested on new unseen images, the model has achieved an average classification accuracy of 82.5%.

  .4). for consecutive hidden layers of representations. Deep neural networks (DNN) are formed of numerous layers, tens or even hundreds, stacked on top of each other's, and they are all learned automatically during the training process.The hierarchical structure of deep networks allows data processing with a nonlinear approach. Each layer in the network utilizes the output of preceding layers to learn more complex concepts. First layers detect low level features as edges and the deeper you go in the DNN, the nodes identify more complex features by combining features from the preceding layer. This hierarchical learning eliminates the hand-crafted feature extraction in the network (Figure2.5).

		Comparison
		between
		ML
		and DL
	Machine Learning	Deep Learning
		Structured data	Structured and Un-structured data
	Input	Hand-crafted extrac-tion of features from images Simple features (edges) Intermediate features contours) (corners and	Automatic extrac-tion of features from images Complex (object part) features Output
		Create a model that	Give the network
		categorizes the ob-	raw data and a task
		jects in the image	to perform, such as
			classification, and it
			learns how to do this
		Performance plateau	automatically
		at a certain level
		when you add more examples and train-ing data to the net-work	Performance contin-ues to improve as the size of your data in-creases
	Figure 2.4: Comparison between classic machine learning algorithms and deep learn-
	ing algorithms.
	2.4.3 How deep learning works?

Deep learning methods are usually based on artificial neural networks. That's why we refer to deep learning methods as deep neural networks. They use a hierarchical level of neural networks mimicking the human brain. The term "deep" in deep learning stands

  is the first successful model of CNN developed by Yann LeCun in 1998. It It consists of 3 convolution layers, 2 average pooling layers, and 2 fully connected layers with a softmax classifier [41]. Alexnet is proposed in 2012 by Alex Krizhevsky and his colleagues. It has the same architecture as LeNet but deeper with 8 layers with learnable parameters. The input to the network is RBG images. This model is composed of 5 convolution layers, max-pooling layers, and 3 fully connected layers with a Softmax classifier. They use ReLU as activation function. The network includes also two Dropout layers [42]. ResNet or Residual Network is developed by Shaoqing Ren, Kaiming He, Jian Sun, and Xiangyu Zhang in 2015. It is one of the most used and successful models. ResNet is based on the concept of skip connections and the use of batch normalization. Its architecture is inspired by the VGG-19 but doesn't include FC layers at the end of the network. They are replaced by an average pooling layer. All ResNet models have only one max pooling layer after the first one. It There are several

	Chapter 2	Deep learning based skin disease classification
	architectures of ResNet differing by the number of layers that vary from 18 to 152
	[45].

is a straightforward and small network composed of 5 layers with learnable parameters. This architecture is used for image classification and especially recognition of handwritten and machine printed characters. It takes a grayscale image in input. GoogleNet is a network proposed by from Szegedy et al. from Google in 2014. It uses inception modules, which permit to choose between several filter sizes in each convolution block. It is formed of 22 layers but with reduced number of parameters compared to previous models [43]. VGGNet is introduced in 2014 by K. Simonyan and A. Zisserman (Very Deep Convolutional Networks for Large-Scale Image Recognition, conference paper at ICLR 2015). It is one of the most popular networks in deep learning applications. VGGNets are fed with RGB images of fixed size 224x224. They replace large size filters (11 x11 and 5 x 5) in Alexnet with smaller ones of size 3x3 which results in reducing the number of parameters. They show that the performance of the network increases with depth. There is a different architecture of VGGNet such as: VGG-11, VGG-13, VGG-16 and VGG-19 where the number within each name refers to the number of deep layers in the network [44].

Table

  1) EfficientNet B0 is mainly composed of mobile inverted bottleneck MBConv used in MobileNetV2 [54] besides squeeze and excitation block with swish activation function. It has 7 blocks of different settings and 11 million trainable parameters. It is developed to be used in image classification.

	Stage	Operator	Resolution #Channels #layers
	1	Conv3x3	224x224	32	1
	2	MB Conv1,k3x3	112x112	16	1
	3	MB Conv6,k3x3	112x112	24	2
	4	MB Conv6,k5x5	56x56	40	2
	5	MB Conv6,k3x3	28x28	80	3
	6	MB Conv6,k5x5	14x14	112	3
	7	MB Conv6,k5x5	14x14	192	4
	8	MB Conv6,k3x3	7x7	320	1
	9	MB Conv1x1/Pooling/FC	7x7	1280	1

•

  CIFAR-10 (Canadian Institute For Advanced Research) dataset consists of 60000 labelled colored images (RGB) of size 32 x 32 referring to 10 classes, including: airplanes, automobiles, birds, cats, deer, dogs, frogs, horses, ships, and trucks. Each class contains 6000 images; 5000 training images and 1000 test images. • ImageNet is a large dataset of annotated images for computer vision research. It includes more than 14 million images organized in approximately 22000 categories. It is useful for object recognition and localization, and image classification tasks. This dataset is used in ImageNet Large Scale Visual Recognition Challenge (ILSVRC) [57] that aims to train a deep neural network to classify an image into 1000 object classes that we meet in daily life. It uses ap-

proximately 1.2 million training images, 50,000 validation images and 100,000 testing images.

Table 4 .

 4 1. After training the model in each split case, we obtain the following training accuracy and loss, as well as validation accuracy and loss presented in Table 4.2 and Table 4.3 respectively.

	Split Case Training Time Validation Time
	90%-10%	4171 s	14 s
	80%-20%	4124 s	27 s
	70%-30%	4297 s	41 s
	60%-40%	4361 s	55 s
	50%-50%	4196 s	70 s

Table 4 .

 4 1: Training and validation times in the five data split cases. The training time in the different cases is approximately stable, while the validation time increases with the number of the validation images.The accuracy is a metric that characterizes the performance of the model in all classes. It is the ratio of correct predictions to the total number of samples.

	Accuracy =	T P + T N T P + T N + F P + F N	(4.1)
	Where TP = True Positives, TN = True Negatives, FP = False Positives, and FN
	= False Negatives.
	Split Case Training accuracy Training loss
		90%-10%	99.91%	0.0219
		80%-20%	99.89%	0.0255
		70%-30%	99.81%	0.0289
		60%-40%	99.85%	0.0328
		50%-50%	99.85%	0.0374

Table 4 .

 4 2: Training accuracy and loss in the five split cases. The accuracy and loss vary slightly between the split cases. The highest accuracy is 99.91% and the lower loss is 0.0219 in data split case 90:10 for training versus validation.

	Split Case Validation accuracy Validation loss
	90%-10%	96.9%	0.0987
	80%-20%	95.77%	0.1198
	70%-30%	95.05%	0.1381
	60%-40%	94.85%	0.1559
	50%-50%	93.06%	0.2003

Table 4 .

 4 3: Validation accuracy and loss in the five split cases. The validation accuracy varies between 96.9% and 93.06% and the loss between 0.0987 and 0.2003. The highest accuracy and lowest loss are achieved in the split case 90:10 for training versus validation.

  In split case 90% for training and 10% for validation, we found 1928 correct labels and 72 incorrect labels, so 96.4% of the samples are correctly predicted. In case 80:20, the classifier predicts 95.2% of the samples correctly; 3811 correct labels and 189 incorrect labels. In case 70:30, 5673 labels are appropriately predicted equivalent to 94.55% of the samples, while 327 are wrongly predicted. For the split case 60:40, we obtained 7532 correct predictions, and 468 wrong ones, so 94.15% of the predictions are well classified. Finally for the case of 50:50, the model could classify 92.15% of the samples correctly; we have 9215 correct labels, and 785 wrong labels. From all the evaluation metrics presented above, we can observe that the pre-trained VGG-16 model has the best performance when splitting data into 90%for training and 10% for validation, and this is because the model is trained with more data. Once the model is trained and validated, we save the model to test it.

Table 4

 4 .4. The FSDNet is trained and achieves the following training and validation accuracies and loss as shown in Table4.5.

	Split Case Training Time Validation Time
	90%-10%	1486 s	13 s
	80%-20%	1559 s	26 s
	70%-30%	1519 s	40 s
	60%-40%	1484 s	52 s
	50%-50%	1454 s	66 s

Table 4 .

 4 4: Training and validation times in the five data split cases. The time required to train and validate the weights of block 5 and the new added layers added in the top of the network varies between the split cases.

	Split	Training Accuracy Training Loss Validation Accuracy Validation Loss
	90%-10%	99.04%	0.0451	98.95%	0.0582
	80%-20%	98.58%	0.0611	98.57%	0.0567
	70%-30%	98.09%	0.0868	98.56%	0.0557
	60%-40%	98.99%	0.0431	98.087%	0.0955
	50%-50%	98.62%	0.0489	96.2%	0.1377

Table 4 .

 4 5: Training and validation accuracies and loss in the five split cases. The accuracy and loss vary slightly between the split cases. The training accuracy varies between 98.09% and 99.04%, while the lowest loss is 0.0431 obtained in split case 60:40. The validation accuracies are smaller and are between 96.2% and 98.95% but the loss values are higher than in training, and the lowest loss is 0.0557. For training and validation, the highest accuracies are obtained in case of splitting 90% of data for training and 10% for validation.One observes that the FSDNet achieves better performance than the pre-trained VGG-16 with an accuracy of 99.04% for 96.95%. We have plotted the learning curves (accuracy and loss) for the five split cases to see the changes in performance of the model over time and thus determine if the model overfit, underfit or well fit to data. The curves are presented in Figure4.12. Comparing to the curves of the pretrained VGG-16 model, we observe that the learning curves in FSDNet are improved considerably. The gap between training and validation loss has almost disappeared.Even the training and validation accuracy are very close to each other. We can say the model is well fitted to the data. To make sure that the model performs well, we have calculated the evaluation metrics, precision, recall, and F1-score. The results are presented in Figure4.13. Also with comparison to the pre-trained VGG-16, the metrics have considerably improved. The averages have risen by approximately 8%.To evaluate the performance of the network, and find the errors it makes, we have calculated the confusion matrices presented in Figure4.14. These matrices show that FSDNet has high number of correct predicted labels. The confusion with other classes is very low and in some cases it is null. The number of correct predicted labels has increased by 9%. The number of correct and incorrect predicted labels is presented in Table4.6.

	Split	Correct labels Incorrect labels Total number of classified images
	90%-10%	1979	21	2000
	80%-20%	3939	61	4000
	70%-30%	5898	102	6000
	60%-40%	7830	170	8000
	50%-50%	9580	420	10000

Table 4 .

 4 6: Number of correct and incorrect predicted labels in the different split cases of FSDNet. The number of correct labels forms approximately 98% of the total images to be classified in all cases.

	The next step is to test the FSDNet using the test dataset, composed of 100

images with 10 images in each class, used in testing the pre-trained VGG-16. According to the carried experiments, we will use the FSDNet in split case 80:20 to predict the class of the different test images.

Table 4 .

 4 8: Summary of results: The number of correct and incorrect predicted labels for each class are shown in the table below. The FSDNet could predict more correct labels and with higher accuracy than the pre-trained VGG-16 model.

		Total number of correct labels Average accuracy of prediction
	Pre-trained VGG-16	70	86.19%
	FSDNet	92	98%
	Fine-tuned EfficientNet b0	80	89.5%
	Fine-tuned Inception V3	77	82.5%

Table 4 .

 4 9: Summary of test images classification: We mention in the table the average accuracy of classification of pre-trained VGG-16, FSDNet, fine-tuned Effi-cientNet b0, and fine-tuned Inception V3, as well as the total number of correctly predicted images among 100 test images.

Table 4 .

 4 8 and Table4.9 show that the models that we have built with transfer learning adapted to facial skin disease classification; FSDNet, fine-tuned Efficient-Net b0, and fine-tuned Inception V3, are more accurate than the pre-trained VGG-16 model. Transfer learning has boosted the performance of the network. Since the model has been trained previously and learned over large sets, transfer learning provides it with a higher starting accuracy, a faster convergence, and a higher asymptotic accuracy when used on a new task. The number of trainable parameters differs between the models. We have 40970 trainable parameters in pre-trained VGG-16, 7084554 in FSDNet, 3168550 in fine-tuned Efficientet b0, and 6094026 in fine-tuned Inception V3. Among all, FSDNet is the most accurate model. It has achieveds the highest average accuracy of classification of 98%, and has been able to predict the highest number of test images, which is 92 correct labels among 100. It is followed by fine-tuned EfficientNet b0 with 80 correct labels predicted with an average accuracy of 89.5%. Then comes fine-tuned Inception V3 with 77 correct labels classified with an average accuracy of 82.5%, and finally pre-trained VGG-16 with 70 correct labels and an average accuracy of 86.19%. FSDNet is based on VGG-16

Class: No-Face No-Face images Predicted class(FSDNet) Predicted class(Pretrained VGG16) Image No-Face: 100.0000% Angioedema: 68.5342% Image No-Face: 100.0000% No-Face: 99.9086% Image No-Face: 100.0000% No-Face: 99.9995% Image No-Face: 100.0000% No-Face: 91.1643% Image No-Face: 100.0000% No-Face: 99.9999% Image No-Face: 100.0000% No-Face: 100.0000% Image No-Face: 100.0000% No-Face: 99.9999% Image No-Face: 100.0000% No-Face: 100.0000% Image No-Face: 100.0000% No-Face: 100.0000% Image 10 No-Face: 100.0000% No-Face: 100.0000% Acne images Predicted class with fine-tuned EfficientNet b0 Image 1 Acne: 65.0025% Image 2 Acne: 99.7379% Image 3 Acne: 96.3461% Image 4 Acne: 71.1650% Image 5 Melasma:59.0859% Image 6 Normal: 72.7821% Image 7 Acne: 86.9358% Image 8 Acne: 77.6664% Image 9 Acne:97.3565% Image 10 Acne: 99.9390% Acne images Predicted class with inception V3 Image 1 Eczema: 71.5601% Image 2 Acne: 89.8751% Image 3 Acne: 89.6239% Image 4 Acne: 83.7845% Image 5 Acne:76.7306% Image 6 Melasma: 72.7248% Image 7 Normal: 56.0885% Image 8 Acne: 78.1119% Image 9 Acne:77.8795% Image 10 Acne: 70.0996% Class: Actinic keratosis Actinic images Predicted class with inception V3 Image 1 Actinic: 94.8712% Image 2 Melasma: 77.1518% Image 3 Melasma: 61.3543% Image 4 Actinic:99.9806% Image 5 Actinic:71.5958% Image 6 Actinic: 90.5395% Image 7 Melasma: 83.0333% Image 8 Acne: 33.6184% Image 9 Actinic:99.9722% Image 10 Actinic: 99.8913% Class: Angioedema Angioedema images Predicted class with inception V3 Image 1 Angioedema: 88.2401% Image 2 Angioedema: 89.6239% Image 3 Angioedema: 88.1840% Image 4 Angioedema: 76.7306% Image 5 Angioedema:75.9901% Image 6 Angioedema: 85.5522% Image 7 Angioedema: 79.5112% Image 8 Angioedema: 93.0375% Image 9 Acne:76.8689% Image 10 Angioedema: 70.5702% Class: Blepharitis Blepharitis images Predicted class with inception V3 Image 1 Blepharitis: 80.4607% Image 2 Blepharitis: 75.9901% Image 3 No-Face: 68.7601% Image 4 Blepharitis: 83.0333% Image 5 Blepharitis:71.5958% Image 6 Melasma: 61.3543% Image 7 Blepharitis: 99.9926% Image 8 Blepharitis: 99.9973% Image 9 Blepharitis:99.9997% Image 10 Acne: 73.6184%

Chapter 3

Convolutional neural networks architectures VGG is seen as a general architecture and considered a baseline for many tasks and datasets outside of ImageNet, the dataset on which the network is trained initially, and widely used for image identification and classification. Inception v3 is the third edition of Google's Inception CNN. While most popular CNNs work on increasing the depth of the network, Inception models used other techniques to enhance the performance of the network in terms of speed and accuracy. Many versions have been created, where each version is an improvement of the previous.

EfficientNet models are recently developed from Google presenting a new concept in networks scaling to improve the performance, called Compound Scaling. Previous CNN models follow the classic method of arbitrary single dimension scaling and adding more layers. EfficientNet models scale uniformly all dimensions (depth, width, and resolution) using a compound coefficient. They improve the accuracy and the efficiency of the model. In this chapter, we will review the architecture of the different networks that we have used in our thesis.

We will also explain how we can apply these pre-trained networks on ImageNet to classify other images of different categories than those of ImageNet. The chapter ends in a conclusion.

Visual Geometry Group (VGG)

Visual Geometry Group, or VGG, is a convolutional neural network introduced by K. Simonyan and A. Zisserman [START_REF] Simonyan | Very Deep Convolutional Networks for Large-Scale Image Recognition[END_REF]. The main contribution of this model is to show that the depth is a significant factor to enhance classification accuracy. They have been widely used for object recognition. The VGG family is distinguished from other CNN architectures previously proposed by the use of small receptive fields of size 3 x 3 and stride 1 in all CONV layers, and many stacked CONV with ReLU layer sets followed by a pooling operation.

Chapter 3

Convolutional neural networks architectures

φ is a coefficient determined by the user that controls the number of available resources for model scaling, whereas α, β, and γ are constants determined by grid search that define how to designate these resources to the depth, width, and resolution of the network. Based on the compound scaling method and automated machine learning, the researchers have generated a family of EfficientNet models, smaller and faster than state-of-the-art CNN models, achieving up to ten times better accuracy and efficiency.

EfficientNet architecture

Scaling improves accuracy but doesn't change the function of the layers. The first step was finding the baseline network and then scaling it up using compound scaling.

The authors have developed a new mobile-size baseline, called EfficientNet B0 that optimizes the accuracy and FLOPS (floating point operations per second) that measure the number of operations needed to run the network model. The architecture of the baseline network is presented in Table 3.2.

Chapter 3

Convolutional neural networks architectures each of them by a constant ratio. EfficientNet B0 is the baseline network of this method on which the introduced scaling method is applied. EfficientNet B0 is scaled from B1 to B7. It is principally made up of MBConv of different settings, as well as a squeeze and excitation block, and a swish activation function gathered in 7 blocks. It is pre-trained on ImageNet dataset and achieves a classification accuracy of 84.4%.

These pre-trained networks can be used to perform image classification on different tasks and datasets than ImageNet. This is called transfer learning. This technique is commonly used in deep learning because it speeds the training time, enhances the performance of the model, and doesn't require a lot of data. The pre-trained model is trained on new data but initialized with pre-trained weights. Also, these pre-trained models can be fine-tuned by making some modifications to the model and then retrain it on the new data using a small learning rate. Using transfer learning or fine-tuning depends on the size of the new dataset, and the similarity with the original dataset.

In the next chapter, we will present the results obtained when applying transfer learning on these three pre-trained networks to classify facial skin diseases images into ten classes, that is 8 facial skin pathologies, normal class, and no-face class.

Test

The pre-trained VGG-16 and the FSDNet have been tested using the same images.

As described previously, 10 images from each class not included in the FSDD (not used for training nor validation), were used for test. In the following tables, we will present the results and the accuracy of prediction of images in each class.

Class: Acne

Class: Blepharitis From the results obtained above, one can observe the superiority of FSDNet in predicting all classes with high accuracy of 98%. This can be explained by the fact that FSDNet is more trained on our data. The number of trainable parameters in FSDNet is greater than in pre-trained VGG-16. In pre-trained VGG-16 only the new softmax classifier is trained, while in FSDNet the training starts from block 5 that includes 3 CONV layers, as well as the classifier so the network learn more complex features specific to facial skin diseases.

Performance evaluation of FSDNet

We assess the performance of FSDNet in different brightness conditions and face poses to see how they can affect the accuracy of the prediction.

1-Brightness effect

We have studied the effect of brightness on three classes: eczema, acne, and normal.

We have chosen a test image from each class and applied the brightness variations on it. We have changed the brightness with different factors, 0.8, 0.6, and 0.5. The image gets darker. We have found that the images are still predicted correctly but with a slightly lower accuracy. The results are presented in

Class: Blepharitis

Blepharitis images Predicted class with fine-tuned EfficientNet b0

General conclusion and perspectives

The