Besides its contribution to computer-aided diagnosis, Digital Pathology has opened up a new dimension in the investigation of complex diseases, by adopting innovative image analysis tools able to provide new features exposing different facets of the disease, impossible to analyze using classical microscopy by the human eye. One of the most challenging problems in histological image analysis is the evaluation of the spatial organizations of histological structures in the tissue. In fact, histological sections may contain a very large number of cells of different types and irregularly distributed, which makes their spatial content indescribable in a simple manner.

Graph-based methods have been widely explored in this direction, as they are effective representation tools having the expressive ability to describe spatial characteristics and neighborhood relationships that are visually interpreted by the pathologist. We can distinguish three main families of graph-based methods used for this purpose: syntactic structure analysis, network analysis and spectral analysis. However, another distinctive set of methods based on mathematical morphology on graphs can be developed for this issue. The main goal of this dissertation is the development of a framework able to provide quantitative evaluation of the spatial arrangements of histological structures using graph-based mathematical morphology. The main contributions of this work are five-fold.

First, we propose a theoretical framework dedicated to the evaluation of the spatial arrangement of points based on features derived from mathematical morphology on graphs. Two morphological transforms are established and they have shown the ability to describe the spatial distribution of a point set and how distinct point sets are located relative to each other. The framework suggests the integration of graph-based mathematical morphology in spatial point pattern analysis. The new methodology would find an effective use in a wide range of practical problems.

Second, we present a comprehensive review on graph-based methods explored in histopathological image analysis. We propose a classification of the state-of-the-art methodologies based on the type of the graph, the category of the feature extraction technique, and the histological structures considered.

Third, we develop a method for nuclei detection and classification from histopathological images of HES (Haematoxylin, Eosin, Saffron)-stained breast cancer tissues, using supervised machine learning algorithms based on color and texture information, and methods for collagen and adipose tissue segmentation.

Fourth, we study the spatial distribution of the different tissue components and the spatial interactions between them using the proposed graph-based mathematical morphology approach. The analysis is conducted on a dataset composed of 55 whole slide images (WSI) of tissue sections obtained from surgical resections of invasive breast carcinomas from distinct patients. The approach offers a comprehensive visual interpretation of different spatial aspects encountered in histopathology, and has shown the ability to distinguish between different tumor architectures and tumor microenvironment configurations.

Fifth, we propose an original tissue simulation approach that gives access to a large spectrum of tumor-like architectures based on tools of classical mathematical morphology and statistics. The method can be used to generate large synthetic datasets in order to evaluate the effectiveness of graph-based methods in the quantification of the spatial organization of histological structures. vii I would like to express my thankfulness to the master students that I have co-supervised/supervised during their internships, Rui Venâncio, Taibou Birgui Sekou and Kaiwen Chang, for their help in the accomplishment of several tasks related to this project, and also to my colleagues, Maxime Doury, Oumeima Laifa and Lamine Traoré, for their help and support. I am also thankful to all our collaborators in FlexMIm, FRM and PAPHOS projects, and all co-authors of the papers published at conferences and journals during

Drawing up a PhD project and achieving its objectives by working alone is a difficult and enduring challenge that comes with a mixture of frustration and enthusiasm. However, that would not be possible without the participation and support of many people. Therefore, I would like to thank all people who were involved or contributed in one way or another to the accomplishment of that goal.

Firstly, I would like to thank my PhD supervisor Prof. Daniel Racoceanu (Pontificia Universidad Católica del Perú, San Miguel, Peru) for offering me the opportunity to discover and explore the modern field of Digital Pathology, and to meet and collaborate with great scientists and researchers in the area. I would also like to thank Dr. Lori Bridal (Université Pierre et Marie Curie, Paris, France) for all the significant administrative and organizational efforts she made to help see the project through to completion.

I would like to express my gratitude to Dr. Catherine Bor (Centre Jean Perrin, Clermont-Ferrand, France) and Dr. Philippe Bertheau (Hôpital Saint-Louis, Paris, France), who provided me medical expertise and knowledge about breast and colon cancers. I'm also very grateful to Dr. Nicolas Elie (Université de Caen Normandie, France), Dr. Benoit Plancoulaine (Université de Caen Normandie, France) and Dr. Alain Coron (Université Pierre et Marie Curie, Paris, France) for sharing ideas, experiences and insights on topics related to medical image analysis.

List of Abbreviations

Histopathology

In clinical medicine, histopathology refers to the microscopic examination of tissue specimen in order to study the manifestations of disease. The word "histopathology" is derived from a combination of three Greek words: "histos" tissue, "pathos" disease, and "logos" which refers to the study in this context. Generally, a histopathological examination is prescribed for the patient after observing physiological symptoms. The tissue sample is then removed from the suspected organ by means of biopsy or surgical resection, and prepared for viewing under a microscope. The pathologist observes the architecture of the tissue, the distribution and the morphology of the cells in very fine details, and determines the level of the severity of the disease. The diagnosis requires highly skilled expertise, and is often a tedious and time consuming task due to various complex tissue structures and information related to different biological conditions of the organism.

Histopathology plays an important role in disease detection, interpretation, and diagnosis, which are central in determining the treatment strategy for the patient. In fact, histological slides provide a comprehensive view of a disease and its effect on the tissue of a particular organ, since they deliver rich visual content at the cellular level featuring disease characteristics that are not observable in other medical imaging modalities. This richness as the high routine-available resolutions, make the histopathological examination viewed as the gold standard for the diagnosis of several diseases, and a corner stone in cancer grading and staging.

Specimen preparation and staining

To observe the tissue under a microscope, a sequence of preparation procedures is first applied. After being removed from the organ, the specimen is first sectioned into thin cross sections with a microtome. Then, fixation is used to stop the metabolic activities and stabilize the tissue to prevent decay and preserve the histological structure. The fixation is made either with a chemical fixative or frozen section procedure. The most commonly used chemical fixative in histopathology is formaldehyde, also known as formalin or methanal, which is an organic compound produced principally by the vapor phase oxidation of methanol. Whereas, in the frozen section procedure, also called cryosection, the tissue sample is cut frozen with the microtome into thin slices. The latter technique of preparation is much more rapid than the formalin fixation procedure (around 10 minutes vs 16 hours) [START_REF] Wright | William Osler Medal Essay": The Development of the Frozen Section Technique, The Evolution of Surgical Biopsy, and the Origins of Surgical Pathology[END_REF], which makes it principally used in the examination of tissue while surgery is taking place (most often in oncological surgery). However, the histological slides produced by the frozen section procedure are of lower quality.

The thin tissue sections are then mounted on glass slides and stained with one or more pigments in order to enhance the contrast and highlight specific cellular structures under the microscope. The most widely used stain in medical diagnosis is the Haematoxylin and Eosin stain (H&E stain) [START_REF] Bancroft | Theory and practice of histological techniques[END_REF]. Hematoxylin stains the cell nuclei in blue-purple shade, while Eosin Chapter 1. Role of Digital Pathology and Image Analysis in Oncology stains cytoplasm and stroma in various shades from reddish to pinkish color, and collagen in pale pink shade (see figure 1.1.(a-b)). To better distinguish collagen fibers from stroma, a trichromatic stain, known as HES can be used [START_REF] Kiernan | Histological and histochemical methods: theory and practice[END_REF]. It is obtained by adding an extra staining step with Saffron to the H&E stain. Saffron accentuates collagen fibers with color ranges from yellow to orange (see figure 1.1.(c-d)). However, the best known staining method for collagen uses a stain known as Picro-sirius red, which colors collagen fibers in red and other tissue components in yellow [START_REF] Dapson | Certification procedures for sirius red F3B (CI 35780, Direct red 80)[END_REF]. In order to identify the localization of differentially expressed proteins in the tissue, methods based on immunohistochemical (IHC) staining can also be applied. Immunohistochemistry uses specific molecular biomarkers that are characteristic of particular cellular events such as cell proliferation, cell division (mitosis) or cell death (apoptosis). The Ki-67 for example is a cellular biomarker for proliferation, and is commonly used to determine the growth fraction of a given cell population. In clinical applications, the fraction of Ki-67-positive tumor cells, referred to as Ki-67 labelling index, is used in the assessment of the tumor proliferative activity [START_REF] Scholzen | The Ki-67 protein: from the known and the unknown[END_REF]. The phospho-histone-H3 biomarker (also known as pHH3, see figure 1.1.(e-f)) is a specific marker for cells undergoing mitosis, and is used in the assessment of the mitotic index, which is an important prognostic factor in most types of cancer [START_REF] Tapia | Two mitosis-specific antibodies, MPM-2 and phospho-histone H3 (Ser28), allow rapid and precise determination of mitotic activity[END_REF]. The CD31 biomarker is used to demonstrate the presence of endothelial cells in histological tissue sections, which can help to evaluate the degree of tumor angiogenesis (formation of new blood vessels, which can imply a rapidly growing tumor [START_REF] Delisser | Involvement of endothelial PECAM-1/CD31 in angiogenesis[END_REF]). Whereas, apoptotic cell conditions can be determined with the Caspase-3 (CASP3) protein [START_REF] Abu-Qare | Biomarkers of apoptosis: release of cytochrome c, activation of caspase-3, induction of 8-hydroxy-2'-deoxyguanosine, increased 3nitrotyrosine, and alteration of p53 gene[END_REF]. Some other commonly used stains in histopathology are Diaminobenzidine (DAB), Feulgen, Schiff, Wright, and Toluidine blue.

Whole Slide Imaging

In routine clinical practice, after fixation, preparation and staining, the glass slides are observed by the pathologist under a microscope. The histopathological examination of the tissue specimen is then assessed only qualitatively in regard to highly complex information, which involves a degree of subjectivity and leads to the requirement for a quantitative information based on digital systems. Digital imaging of histological specimen was first established by mounting a digital camera on the microscope in order to produce static digital images. These static images were found to be of limited clinical utility because they capture only specific regions of a glass slide.

With the recent advent of whole slide digital scanners, tissue slides can now be digitized and stored as high-resolution digital images, allowing them to be viewed, managed, and analyzed on a computer monitor in a similar manner as under an optical microscope. Digital slide scanners are rapid and high resolution scanners that convert the histopathological glass slides to digital slides (also called virtual slides, or Whole Slide Images WSI). Certain sophisticated scanners are designed to accommodate hundreds of glass slides. An example of a digital slide scanner, ScanScope R CS, from Aperio Technologies is given in figure 1.2. A list of common WSI systems, their respective technical characteristics and vendors is provided in [START_REF] Farahani | Whole slide imaging in pathology: advantages, limitations, and emerging perspectives[END_REF]. WSI files are significantly larger than common digital image files used by other medical imaging modalities (from a few megabytes to several gigabytes). Therefore, they are stored as thousands of compressed (e.g. JPEG2000) image files spanning multiple directories based on the Tagged Image File Format (TIFF), which can be made up of one or more image file directories. These image files (called tiles) are then constructed into a multilayered pyramid, where each layer represent a given digital magnification (e.g. 5X, 20X, 40X), see figure 1.2. The pyramidal encoding model allows the digital slide to be easily manipulated when retrieving data from any location at any magnification, since lower computational cost is required to traverse the large WSI file. To display the content of these enormous digital files, a Chapter 1. Role of Digital Pathology and Image Analysis in Oncology specialized software called virtual slide viewer is used, enabling the user to navigate virtual slides at various magnifications. The viewing software accurately replicates the manner of slide viewing that conventional microscopy offers (pan and zoom, etc). Some image viewers allow in addition simultaneous view of many slides with synchronized navigation. Commonly used virtual slide viewers include Aperio ImageScope from Leica Biosystems; and NDP.view2 from Hamamatsu Technology.

Advantages of Digital Pathology

The image-based information environment that allows the management of information generated from Whole Slide Images has now become termed Digital Pathology. Digital Pathology has benefited over recent years from the rapid progress of image digitizing technology, and it has become one of the very important evolutions in modern medicine. Beside the image acquisition, the practice of digital pathology provides several benefits involving clinical medicine, medical research, and education.

In fact, virtual slide viewers offer numerous functions not offered by conventional microscopy. These additional functions range from, better slide viewing (e.g. parallel display of different stains), practical annotation tools (e.g. measurements or comments), to more sophisticated features such as automatic image analysis. Examining a large number of highcontent tissue slides represents a labor-intensive and time-consuming work for the pathologist. With the development of digital slides, the integration of computer-aided diagnosis (CAD) systems, to complement the opinion of the pathologist and provide him valuable assistance in his daily practice, has now become feasible. Computer-aided diagnosis, based on the application of computerized image analysis and machine learning techniques, can be useful to derive valuable information from virtual slides, like automatic recognition or counting of histological structures (e.g. cell nuclei, glands).

Digital pathology systems can be used, not only for visualization, interpretation and analysis, but also for sending and sharing virtual slides at distant locations for diagnosis, or for a second opinion in a doubtful case for example. This practice, called telepathology (or teleconsultation), is aimed to enhance collaborative capabilities between pathologists and it consists of: sharing static images captured from the slide viewer; or allowing remote handling of the virtual slides. Digital pathology is also very useful for saving and storing of virtual slides, enabling a significant reduction in the volume of slide archives, and avoiding problems related to glass slide deterioration, e.g. stain attenuation, breakage or loss. In addition, saving data accrued from virtual slides allows easy retrieval and management of clinical information, and better traceability and reproducibility, which is very important for laboratory quality control in clinical trials.

In addition to its promising contribution to clinical medicine, digital pathology has opened up a new dimension to investigate complex and perplexing diseases that are in the focus of the modern medical research, by adopting innovative image analysis tools exposing new features and different facets of the disease, impossible to conclude by the human eye. One of the examples is the evaluation of the spatial organization of different cells in the tissue, which constitutes one of the most challenging problems of visual interpretation in histopathology.

Digital pathology has also created tremendous opportunities in medical education, by transforming pathological trainings from microscope-based to fully-digitized lessons that can be learned more accurately from inside or outside the classroom. In addition, with the availability of digital slides and consistent annotations, students can benefit from improved educational methods regulated by standardization of learning.

The current status of digital pathology applications and its impact on the field of pathology were widely discussed by the authors of [START_REF] Al-Janabi | Digital pathology: current status and future perspectives[END_REF]. According to some authors, the expanding field of digital pathology will not only improve the pathology services, but will also revolutionize research and healthcare [START_REF] Saini | Digital pathology: an overview[END_REF]. Despite the numerous advantages of digital pathology in helping in improving disease diagnosis, sharing information, novel ideas and knowledge, the implementation in routine clinical practice remains slow. In fact, digital pathology has been approved by the Food and Drug Administration (FDA) for primary diagnosis only in April 2017 [START_REF]FDA allows marketing of first whole slide imaging system for digital pathology April 12[END_REF].

Breast Cancer Histology

Breast cancer is the most common cancer in women worldwide, with nearly 1.7 million new cases diagnosed in 2012. This represents about 12% of all new cancer cases and 25% of all cancers in women. It is the fifth most common cause of death from cancer in women [START_REF] Ferlay | Cancer incidence and mortality worldwide: sources, methods and major patterns in GLOBOCAN 2012[END_REF]. The most common type of breast cancer is known as breast carcinoma. Breast carcinomas are cancers that arise from the epithelial cells of the breast, which are the cells that line the lobules and the lactiferous ducts, see figure 1.3. Cancers developing from the ducts are called ductal carcinomas (fig. 1.3), while those developing from lobules are called lobular carcinomas. Histopathology plays an important part in determining the treatment strategy for patients with breast cancer. An example of whole slide image (WSI) of a breast cancer tissue section stained with H&E is presented in figure 1.4. The histopathological examination is based on the visual evaluation of the shape and size of nuclei, size of nucleoli, regularity of nuclear contour, and the distribution of populations of nuclei. Successful treatments for breast cancer depend largely on the cancer subtype and stage (grade).

Grading

Breast cancer grading refers to the evaluation of the aggressive potential of the tumor ("low grade" cancers tend to be less aggressive than "high grade" cancers). Determining the grade is thus very important information to help guide the treatment strategy for patients. There are different grading systems available for determining the grade of breast cancer. One of the most known and used systems is the Nottingham Histologic Score system (also known as Scarff-Bloom-Richardson grading system) [START_REF] Bloom | Histological grading and prognosis in breast cancer: a study of 1409 cases of which 359 have been followed for 15 years[END_REF]. In this scoring system, there are three factors that the pathologists take into consideration: i) the amount of tubule formation, which describes how well the tumor cells try to recreate normal mammary glands. ii) the nuclear pleomorphism, it describes variability in the size, shape and staining of cell nuclei. iii) the mitotic index, it measures the cellular proliferation and how much the tumor cells are dividing (multiplying). Each of these features is scored from 1 to 3, and then each score is added to give a final total score ranging from 3 to 9.

Breast Cancer Histology

Glandular/Tubular Differentiation:

-Score 1: > 75% of tumor area forming glandular/tubular structures.

-Score 2: 10% to 75% of tumor area forming glandular/tubular structures.

-Score 3: < 10% of tumor area forming glandular/tubular structures.

Nuclear Pleomorphism:

-Score 1: Nuclei small with little increase in size in comparison with normal breast epithelial cells, regular outlines, uniform nuclear chromatin, little variation in size.

-Score 2: Cells larger than normal with open vesicular nuclei, visible nucleoli, and moderate variability in both size and shape.

-Score 3: Vesicular nuclei, often with prominent nucleoli, exhibiting marked variation in size and shape, occasionally with very large and bizarre forms.

Mitotic Index:

The mitotic count score criteria vary depending on the field diameter of the microscope used by the pathologist. The pathologist will count how many mitotic figures are seen in 10 high power fields. Using a high power field diameter of 0.50 mm, the criteria are as follows:

-Score 1: less than or equal to 7 mitoses per 10 high power fields.

-Score 2: 8-14 mitoses per 10 high power fields.

-Score 3: equal to or greater than 15 mitoses per 10 high power fields.

The final total score is used to determine the grades such as; in grade 1 tumors have a score of 3-5, in grade 2 tumors have a score of 6-7, and in grade 3 tumors have a score of 8-9.

Histological classification

Breast carcinomas can be divided into several subtypes based on their histopathological appearance [START_REF] Band | Histological, molecular and functional subtypes of breast cancers[END_REF]. The histological classification of breast carcinoma is extremely important considering the significant implications of the subtypes in the prognosis and treatment of the disease. The three most common histopathological types, that together represent nearly three-quarters of breast carcinomas are: ductal carcinoma in situ (DCIS), invasive ductal carcinomas (IDC) and invasive lobular carcinoma (ILC), with an estimated prevalence of 13%, 55%, and 5% of all breast cancers respectively [START_REF] Eheman | The changing incidence of in situ and invasive ductal and lobular breast carcinomas: United States, 1999-2004[END_REF].

IDC, also known as infiltrating ductal carcinoma, is cancer that has invaded the connective tissue of the breast outside the lactiferous duct. While in DCIS, tumor cells are found inside, and not moved out of, the lactiferous duct 1.3. ILC is a type of invasive cancer, where tumor cells tend to infiltrate as individual rows known as 'Indian files' (one of its key histological features). The sub-classification of invasive breast carcinomas includes largely carcinoma of no special type (NST). This group of breast cancers comprises all tumors without the specific differentiating features that characterize the other categories of breast cancers (e.g. DCIS, IDC, ILC, Lobular carcinoma in situ, Medullary carcinoma, Mucinous (colloid) carcinoma, Tubular carcinoma, Papillary carcinoma) [START_REF] Sinn | A brief overview of the WHO classification of breast tumors[END_REF]. Examples of histopathological images showing different architectures of subtypes of breast carcinomas are given in figure 1.5. When they have specialized patterns in at least 50% of the tumor and non-specialized patterns between 10% and 49%, breast carcinomas are then designated as mixed invasive NST [START_REF] Sinn | A brief overview of the WHO classification of breast tumors[END_REF].

Tumor microenvironment

From an ecological point of view, tumor inhabits a cellular environment composed of parts of different kinds, such as immune cells, blood vessels, collagen, adipose tissue (fat) and many other cell types (see figure 1.6). The whole of these components is referred to as the tumor microenvironment (TME). The interactions between tumor and TME are recognized as playing an important role in the progression of the disease. By understanding these interactions, tumor may be treated more precisely and effectively. Several research studies have been carried out in order to improve knowledge about the relationships existing between the tumor behavior and the characteristics of its microenvironment.

Being composed of immune cells, the inflammatory microenvironment (iTME) is known to have a great impact on the tumor behavior [START_REF] Bhoola | Immune responses in cancer[END_REF]. With the progress being made in the development of cancer immunotherapy, a considerable number of efforts have focused on the understanding of the immune system-cancer interactions. Links have been shown to exist between clinical outcome and immune cell presence, relative abundance, as well as spatial proximity of immune cells to invasive cancer cells [START_REF] Gajewski | Innate and adaptive immune cells in the tumor microenvironment[END_REF].

It has been known for more than a century that tumors have the ability to establish their own blood supply and that the tumor vasculature is actually superior to that of normal tissues. In fact, tumors induce their neovasculature by secreting angiogenic factors. Being executive of angiogenesis and vascular function, endothelial cells are then playing a central role in the development of the tumor, as they form tumor-associated blood vessels [START_REF] Chouaib | Endothelial cells as key determinants of the tumor microenvironment: interaction with tumor cells, extracellular matrix and immune killer cells[END_REF].

Histopathological Image Analysis

9

Recently, it has been shown that invasion and metastasis of breast tumors are influenced by collagen organization at the tumor-stromal interface [START_REF] Provenzano | Collagen reorganization at the tumor-stromal interface facilitates local invasion[END_REF]. Three different Tumor-Associated-Collagen-Signatures (TACS) were defined: (i) TACS-1, which is a limited collagen density localized around small tumor foci, (ii) TACS-2 characterized by stretched collagen fibers tangentially oriented along a smooth tumor boundary, and (iii) TACS-3 characterized by collagen fibers aligned perpendicularly to an irregular tumor boundary and oriented in the direction of cell invasion. The later was correlated with a poor disease-specific survival [START_REF] Matthew | Aligned Collagen Is a Prognostic Signature for Survival in Human Breast Carcinoma[END_REF].

Obesity is widely accepted as an important factor in cancer pathogenesis [START_REF] Hursting | Obesity, metabolic dysregulation, and cancer: a growing concern and an inflammatory (and microenvironmental)[END_REF], but the mechanisms linking the excess of adiposity and cancer are to date unclear. Recently, it has been shown that tumor-associated adipocytes contribute in the development of malignant cells, and that was explained by the fact that adipose tissue fuels the growth of blood vessels by acting as a proximate source of energy [START_REF] Wagner | Inflamed tumor-associated adipose tissue is a depot for macrophages that stimulate tumor growth and angiogenesis[END_REF]. ing the major constituents mentioned above [START_REF] Pattabiraman | Tackling the cancer stem cells-what challenges do they pose?[END_REF].

The interactions between cells in the tumor ecosystem were compared to the interactions between species in their natural habitats in [START_REF] Nawaz | Computational pathology: Exploring the spatial dimension of tumor ecology[END_REF]. The authors proposed four classes of relationships, adopted from ecological concepts: (i) predation, where one species benefits by consuming another, (ii) mutualism, where two species both derive benefit by interacting with each other, (iii) commensalism, where one species benefits from the other without affecting it, and (iv) parasitism, where one species benefits at the expense of the other. The authors have also listed some spatial statistics methods that can be used for the analysis of ecological relationships in the tumor microenvironment, and they were discussed in [START_REF] Heindl | Mapping spatial heterogeneity in the tumor microenvironment: a new era for digital pathology[END_REF].

Histopathological Image Analysis

Overview

Computer-based analysis of histopathological images is currently an expanding field of research. Image analysis methods are being explored with the aim of automating the extraction of useful information from histopathological images, in order to offer several benefits for both routine clinical practice and medical research. These creative methods are being discovered into a growing interdisciplinary field, combining medical knowledge with elements of machine learning, artificial intelligence and image processing. A considerable amount of work has been put into developing automatic techniques for the recognition of histological structures (e.g. nuclei, glandular structures) and for quantitative assessment and characterization of pathological tissue in order to understand the biological mechanisms of disease from high-content images.

Histopathological images have special characteristics which enable them to be distinguished from other medical images (e.g. X-ray radiography, ultrasound, MRI), including Chapter 1. Role of Digital Pathology and Image Analysis in Oncology high resolutions, diverse magnifications, different types of tissues, various stains and complex appearances associated to divergent semantic interpretations. The challenges for histopathological image analysis were very well discussed in [START_REF] Buhmann | Computational pathology: Challenges and promises for tissue analysis[END_REF][START_REF] Madabhushi | Image analysis and machine learning in digital pathology: challenges and opportunities[END_REF]. In addition, the acquisition procedure is rarely accomplished without imperfections (e.g. blur, tears, voids, folds, stain spots) [START_REF] Chatterjee | Artefacts in histopathology[END_REF][START_REF] Samar | Artifacts in Histopathology: A Potential Cause of Misinterpretation[END_REF]. Hence the quality of the acquired histopathological images is also currently the subject of discussion among researches in the field.

However, that did not necessarily prevent the emergence of several progressive works dedicated to the application of image analysis in digital pathology. Literature reviews and summaries have been carried out on this issue, and they can be found in [START_REF] Gurcan | Histopathological image analysis: A review[END_REF][START_REF] Belsare | Histopathological image analysis using image processing techniques: An overview[END_REF][START_REF] Stokes | Pathology imaging informatics for quantitative analysis of whole-slide images[END_REF]. However, the accessibility and usability of these frameworks are still limited in practice, mainly due to complex computational problems and restricted effectiveness of the approaches being developed. These issues make the use of the image analysis software developed to date considered less reliable compared to the pathologist assessment.

Cancer diagnosis

In the clinical practice guidelines for cancer diagnosis, pathologists examine the tissue sample to provide diagnostic assessment relevant to the disease state largely based on cell morphology and distribution. Image analysis techniques have been explored towards developing tools for extracting such diagnosis-related information, enabling objective assessment based on quantitative measures. Several works in this direction have been aimed at developing classification methods in order to confirm the presence or the absence of disease, or to classify tissue regions into healthy or cancerous, benign or malignant or into several grades or types. A systematic survey on the computational diagnostic tools in automated cancer diagnosis based on histopathological image analysis is presented in [START_REF] Yener | Automated cancer diagnosis based on histopathological images: a systematic survey[END_REF]. There are different approaches in the literature to quantify the characteristics of tissues. C. Demir [START_REF] Yener | Automated cancer diagnosis based on histopathological images: a systematic survey[END_REF] has divided them into five categories; intensity based, morphological, textural, fractal, and topological. In [START_REF] Sharma | A review of graph-based methods for image analysis in digital histopathology[END_REF], a sixth category based on frequency was stated. A seventh category of methods that are based on spatial statistics was recently identified in [START_REF] Heindl | Mapping spatial heterogeneity in the tumor microenvironment: a new era for digital pathology[END_REF]. Furthermore, a new set of methods based on Deep Learning have been developed for quantitative image analysis in histopathology [START_REF] Litjens | Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis[END_REF], which makes it the eighth category.

Color or intensity-based features are commonly extracted from the gray-level histograms, or the color histograms, of the tissue image. They are computed based on the pixel values in a single color channel (e.g. R, G, B), in a relationship between the color values in different channels (e.g. blue ratio), or in a single color channel obtained by transformation of the image (e.g. color deconvolution). The weakness of this type of features lies in the fact that they do not provide any information about the spatial distribution of the pixels.

Shape-based or morphological features have been extensively used in image analysis methods for cancer diagnosis, and they are usually expressed by parameters describing: the geometry of cell nuclei, such as area, perimeter, roundness and eccentricity [START_REF] Street | Nuclear feature extraction for breast tumor diagnosis[END_REF], or the morphology of glandular structures such as luminal area and ductal area [START_REF] Anderson | Computerized scene segmentation for the discrimination of architectural features in ductal proliferative lesions of the breast[END_REF]. The features are then used to distinguish between malignant, benign and normal cells/glands. This type of methods necessarily relies on the development of efficient nuclei/gland segmentation algorithms, which remains questionable to date.

The textural approaches make use of spatial inter-relationships of the image pixels based on their intensity (or each single color channel) to extract features. These features describe the variation in the intensity of pixels in a tissue surface by quantifying properties such as smoothness, regularity and coarseness. According to the authors of [START_REF] Yener | Automated cancer diagnosis based on histopathological images: a systematic survey[END_REF], there are two main families of texture methods that have been widely explored to characterize histopathological images: methods based on co-occurrence matrices (GLCM) and methods based on run-length matrices (GRLM). The GLCM-based methods quantify the image texture using statistical measures, such as correlation, contrast and heterogeneity, by making use of the spatial dependency between the gray-level pixel values [START_REF] Haralick | Textural features for image classification[END_REF]. Whereas, GLRLM-based methods look at runs of pixels rather than their spatial dependency. In other words, they analyze how many pixels of a given grey value occur in a sequence in a given direction [START_REF] Galloway | Texture analysis using gray level run lengths[END_REF].

Fractal analysis have been used to understand different phenomena in several biomedical applications, including cancer diagnosis [START_REF] Cross | Fractals in pathology[END_REF]. Fractal geometry is useful for describing tumor architecture and is closely related to mechanisms of tumor growth [START_REF] Jain | Fractals and cancer[END_REF]. In fact, cancer is often characterized by chaotic and unregulated growth which are from the concepts of fractals. By quantifying those irregularities using fractal geometry, morphometric measures of irregular structures typical of tumor growth can be extracted.

Features derived from frequency analysis take into consideration information encoded in the frequency domain, which constitutes what it is called spectral image analysis. Such methods make use of techniques such as Fourier and multiwavelet transforms, which convert the information from the spatial domain defined by the image pixels, into an information in the frequency domain. Wavelet-based features have also been proved as powerful features for breast cancer and prostate cancer classification [START_REF] Sharma | A review of graph-based methods for image analysis in digital histopathology[END_REF].

Methods based on spatial statistics include the study of tissue entities using their geographic properties by mean of statistical analysis. This family of techniques is commonly used in geographic information science (GIS). They use statistical tools that incorporate space directly into their mathematical formulations based on fundamental concepts defining a collection of data composed of random variables, associated to the locations of the objects under study. In cancer diagnosis, these features measure spatial heterogeneity in cell locations, cell density, or co-localization of different cell types. However spatial statistics have not been largely explored in histopathological images, the authors of [START_REF] Heindl | Mapping spatial heterogeneity in the tumor microenvironment: a new era for digital pathology[END_REF] described them as powerful approaches to explore the spatial dimension of the tumor ecosystem.

Deep Learning can be described as the application of multi-layered Artificial Neural Networks (ANN) for learning tasks. ANN are computing systems inspired by the biological neural networks that constitute the human brain. These systems operate by learning progressively relevant features and improving performances jointly to do tasks, by considering examples from huge image databases (typically millions of images). Which is in contrast with traditional approaches for pattern recognition, where features are manually identified and developed. A specific neural network subtype, known as Convolutional Neural Networks (CNN) [START_REF] Sutskever | Imagenet classification with deep convolutional neural networks[END_REF], has become the de facto standard in image recognition [START_REF] Litjens | Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis[END_REF]. According to the authors [START_REF] Janowczyk | Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases[END_REF], deep learning is a learning approach ideally suited for image analysis challenges in digital pathology, and they have presented a large comprehensive study of deep learning approaches in digital pathology to date [START_REF] Janowczyk | Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases[END_REF], including nuclei segmentation, tubule segmentation, lymphocyte detection, mitosis detection, invasive ductal carcinoma detection, and lymphoma classification. They stated that in many cases, the results are superior to the state of the art hand crafted feature based classification approaches. In another work, deep learning was used for the identification of metastatic breast cancer [START_REF] Janowczyk | Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases[END_REF].

The techniques mentioned above do not capture the structure-function relationship presumed by the spatial distribution of cells and the organization of histological structures in the tissue sample. The studies in this direction are based on topological approaches that make use of graphs which are mathematical structures used to model pairwise relations between objects. Graphs are flexible representation tools that have been widely used in the fields of computer vision and image analysis due to their expressive ability to model topological and relational information between image components. The earliest works in this direction were introduced by Prewitt et al. [START_REF] Prewitt | Graphs and grammars for histology: An introduction[END_REF] in 1979. Today, they are gaining a large popularity in histopathological image analysis, as they describe spatial characteristics and neighborhood relationships that are visually interpreted by the pathologist during the examination of a tissue specimen. A review on graph-based methods used in histopathological image analysis is presented in [START_REF] Sharma | A review of graph-based methods for image analysis in digital histopathology[END_REF]. Graph-based methods for histopathological image analysis were explored mainly in three directions: syntactic structure analysis, network analysis and spectral analysis. A summary and discussion of the categorization of these methods is presented in chapter 2.

Nuclei detection, segmentation and classification

One of the most difficult challenges in quantitative image analysis in histopathology, is represented by the identification of the elementary, but crucial, tissue components, that are cell nuclei. The recognition of cell nuclei in histopathological images is a primordial objective in many applications related to the study of tissue at the cellular level. The objective of nuclei detection is to determine for each cell nucleus, a point in the image plane that defines its location (generally its geometric centroid). The objective of nuclei segmentation is to define for each cell nucleus, a set of pixels that delineate the nucleus surface. The objective of nuclei classification is to assign different labels to different types of nuclei (e.g. cancer cell, immune cells, fibroblasts, mitosis), see figure 1.7. Nuclei detection and classification can be used for the purpose of conducting several studies including cell counting and spatial organization. Whereas, nuclei segmentation is commonly used to study the morphology of cells in the tissue (e.g. size, geometry, shape, pleomorphism). The detection, segmentation and classification of nuclei are relatively difficult tasks, since most of cell nuclei are part of complex histological structures presenting irregular visual aspects. Most existing methods for nuclei detection and classification rely on the extraction of color, texture and morphological features from the image, that are then used in supervised learning models with learning algorithms that analyze the features for classification, such as support vector machines (SVM) [START_REF] Cortes | Support vector machine[END_REF], random decision forests (RDF) [START_REF] Ho | Random decision forests[END_REF]. The basic philosophy of these machine learning approach is that the user provides examples of the desired results, and leaves the optimization and parameter tuning tasks to the learning algorithm. Recently, Convolutional Neural Networks and Deep Learning approaches [START_REF] Sutskever | Imagenet classification with deep convolutional neural networks[END_REF], have also been explored in this direction and they have shown to produce encouraging results [START_REF] Sirinukunwattana | Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images[END_REF]. Methods that have been developed for nuclei detection, segmentation and classification in histopathological images were discussed and described in details in [START_REF] Irshad | Methods for nuclei detection, segmentation, and classification in digital histopathology: a review-current status and future potential[END_REF].

Glandular structure segmentation

Glandular structures, or glands, are a type of tissue that performs a specific function, usually by secreting substances. Glands are important histological structures that are present in most organ systems and they can control the different processes that occur in the body with their different secretions (see figure 1 1.4. Background and Motivation

13

Most of the studies so far in this direction were explored in the segmentation of glandular structures from intestinal tissues. This was the main reason for organizing a challenge contest for intestinal gland segmentation held at MICCAI'2015 1 . The Gland Segmentation in Colon Histology Images (GlaS) challenge brought together computer vision and medical image computing researchers to solve the problem of gland segmentation in digitized images of Hematoxylin and Eosin (H&E) stained tissue slides. Participants developed gland segmentation algorithms, which were applied to benign tissue and to colonic carcinomas. A training dataset was provided, together with ground truth annotations by an expert pathologist. The participants developed and optimized their algorithms on this dataset. The results were judged on the performance of the algorithms on test datasets. According to the organizers success was measured by how closely the automated segmentation matched the pathologist's [START_REF] Sirinukunwattana | Gland segmentation in colon histology images: The glas challenge contest[END_REF].

Background and Motivation

The expanding field of digital pathology, by adopting innovative image analysis technologies, has now opened up a new dimension to investigate complex diseases that require a microscopic examination of tissue. Remarkable progress has been made to investigate new ways unraveling the complexity of tissues by providing automatic analysis tools for segmentation and classification of histological structures, as well as quantitative and reproducible tissue metrics for grading and subtyping. As medical research evolves, there is an increased demand of new tissue-derived parameters. Computerized image analysis techniques can also be employed in order to expand medical knowledge by exposing important disease facets that are impossible to carry out visually. Cancer for instance is a heterogeneous cellular disease well-known by its diverse morphological and phenotypic profiles. The current taxonomies in cancer histopathology, rooted in morphological interpretation, are divergent and they have been emphasized with molecular profiling [START_REF] Liotta | Molecular profiling of human cancer[END_REF]. For example, the investigation of ductal carcinoma of no special type (NST) uncovers a large number of tumor subtypes whose distinctive features are to date not well recognized. However, it is recognized that the majority of malignant tumors have the hallmark of heterogeneous architecture [START_REF] Eheman | The changing incidence of in situ and invasive ductal and lobular breast carcinomas: United States, 1999-2004[END_REF]. In addition, cancer is influenced by a combination of multiple microenvironmental factors regulated by different cell types having different biological roles. To date, the heterogeneity of the tumor microenvironment and the spatial interactions between the different entities are not well understood. However, this microenvironmental heterogeneity was also found to have an influence on therapeutic response [START_REF] De Sauvage | Influence of tumour micro-environment heterogeneity on therapeutic response[END_REF]. Quantifying this heterogeneity and exploring these interactions using digital image analysis can contribute to a more comprehensive understanding of cancer. The image analysis techniques that have been adopted in the fields of machine learning and artificial intelligence so far, are not entirely fitting the problem of analyzing complex spatial data from histopathological images. In fact, tissue sections may contain a tremendous number of histological structures of different types, irregularly distributed in the tissue, making their their spatial content indescribable in a simple manner.

Very recently, few studies have been dedicated to the quantitative spatial analysis of the tumor microenvironment using image processing tools. Most of them, have focused on the spatial interactions between the tumor and the immune system. In [START_REF] Ali | Computational pathology of pre-treatment biopsies identifies lymphocyte density as a predictor of response to neoadjuvant chemotherapy in breast cancer[END_REF], machine-learning methods were used to classify cells as cancer, immune cells, and stromal. Then, fifteen image metrics were computed including the number of cells of each type (counts), relative cell type proportion (ratio) and cell densities (minimum, median and maximum). The study have been conducted among 768 H&E-stained surgical samples of breast tumors. From the set of the metrics, median lymphocyte density was associated with one of the clinical variables, the predictor of pathological complete response (pCR) which is an important histological indicator of chemotherapy response. In [START_REF] Yuan | Modelling the spatial heterogeneity and molecular correlates of lymphocytic infiltration in triple-negative breast cancer[END_REF], a cancer cell density map was first generated using kernel density estimation (KDE) (statistics). Then, the spatial proximity of each lymphocyte from cancer is deduced directly from its location in the cancer density landscape. Three classes of lymphocytes based on this measure were derived: intra-tumor lymphocyte (ITL), adjacent-tumor lymphocyte (ATL) and distal-tumor lymphocyte (DTL). The ratio between the number of ITLs and the number of cancer cells was found to be significantly associated 14 Chapter 1. Role of Digital Pathology and Image Analysis in Oncology with disease-specific survival based on statistical modelling conducted on 181 H&E-stained breast cancer tissues. Previously, the same authors have shown that a measure of the proportion (ratio) of lymphocytes among all cell types is predictive of disease-specific survival in a subtype of breast tumors (ER-) [START_REF] Yuan | Quantitative image analysis of cellular heterogeneity in breast tumors complements genomic profiling[END_REF]. Furthermore, the same authors have developed a quantitative measure of microenvironmental heterogeneity, named the tumor ecosystem diversity index (EDI) [START_REF] Natrajan | Microenvironmental heterogeneity parallels breast cancer progression: a histology-genomic integration analysis[END_REF]. A study conducted on 1026 breast cancer samples has shown a significant link between high EDI and a poor prognosis that, according to the authors, cannot be explained by tumor size, genomics or any other data types. In another study [START_REF] Nawaz | Beyond immune density: critical role of spatial heterogeneity in estrogen receptor-negative breast cancer[END_REF], they have found that the amount of co-localized cancer and immune hotspots, weighted by tumor area, correlates with a better prognosis in univariate and multivariate analysis. The co-localization is determined by the fractional area of cancer-immune hotspot. The study was carried out on H&E-stained tumor section images from 245 breast cancer patients. A different approach, in [START_REF] Chang | Spatial organization of dendritic cells within tumor draining lymph nodes impacts clinical outcome in breast cancer patients[END_REF], has revealed that the degree of clustering of dendritic cells (a special type of immune cells) in tumor-positive lymph nodes correlates with the duration of disease-free survival in breast cancer patients. The degree of clustering in this approach was determined using clustering algorithm based on the Euclidean distance between dendritic cells. The study was conducted on 59 tissue sections using different immunohistochemistry stains that highlight different immune cell types. To investigate the interactions between dendritic cells and T cells, the contact between two cells was defined as co-localization within a radius of 100 pixels. In [START_REF] Kruger | Combat or surveillance? Evaluation of the heterogeneous inflammatory breast cancer microenvironment[END_REF], three classes of interactions between tumor and immune cells were defined: surveillance (S), indicating a low chance, combat and surveillance (CS), indicating intermediate chance of interaction, and combat (C), indicating high chance of interaction. Immune cell clusters were classified using a supervised learning algorithm based on the abundance, the distance to tumor cells and clustering behavior. In [START_REF] Setiadi | Quantitative, architectural analysis of immune cell subsets in tumordraining lymph nodes from breast cancer patients and healthy lymph nodes[END_REF], based on spatial statistics, spatial grouping patterns of T and B cells were found different between healthy and breast cancer lymph nodes. The cell density was estimated using the Gaussian kernel density estimation method. The study was conducted on 25 patients using different immunohistochemistry staining techniques.

The mentioned approaches are based mainly on spatial statistics to quantify the spatial heterogeneity of immune cell infiltration in tumors. Another family of techniques can be adopted for this purpose based on graph theory. As explained above, nowadays, graphbased methods are gaining a large popularity in histopathological image analysis. However, they were mainly adopted for the study of architectural characteristics of tumors, but they have not been abundantly used in the study of the interactions with the tumor microenvironment. In addition, in studying the architecture of tissue, these methods were explored "only" in three directions: syntactic structure analysis, network analysis and spectral analysis. These different techniques are explained and discussed in chapter 2. Another distinctive set of methods can additionally be developed to extract spatial information from graphs using mathematical morphology [START_REF] Vincent | Graphs and Mathematical Morphology[END_REF], and be applied to spatial analysis in the tumor microenvironement. Therefore, we contribute to this issue, by providing innovative spatial analysis tools able to reveal various metrics describing spatial relationships in the cancer ecosystem using graph-based mathematical morphology, where the spatial information is given straight by the morphology of the tissue.

Frameworks Related to Histopathological Image Analysis

In regard to a relatively new and growing field of research, the applications of advanced image analysis technologies for specific histopathology-related problems are numerous. Give its research study a clear direction and target a specific research topic, requires a minimum knowledge about the needs in that area. Our understanding of the discipline has been enriched by the time, and sharing novel ideas and knowledge with many individuals has led to define a clear research objective. While there are still gaps in our knowledge, our work efforts have been applied to several issues related to histopathological image analysis, and we present them in figure 1.9.

Our research work has been initiated in the direction of intestinal gland segmentation [START_REF] Ben Cheikh | A structure-based approach for colon gland segmentation in digital pathology[END_REF][START_REF] Sirinukunwattana | Gland segmentation in colon histology images: The glas challenge contest[END_REF]. The initial idea was to conduct a study on the characterization of the architecture 1.5. Frameworks Related to Histopathological Image Analysis of intestinal glands, and thus, an accurate segmentation of these histological structures is primordial. Initial experiments designed to study the spatial distribution of glands have suggested that more sophisticated segmentation tools are needed and that the study has to be driven on WSIs of tissues obtained with surgical resection rather than biopsy, which was not the case of the available dataset. In fact, biopsy slides do not preserve the architecture of glands in the tissue due to major folding occurred during the specimen preparation, in contrast to slides obtained from surgical resection.

An algorithm for nuclei detection and classification from IHC (pHH3)-stained breast cancer tissue images was developed using a machine learning technique [START_REF] Ben Cheikh | Nuclei classification in Immunohistochemical stainings for tumour microenvironement analysis in Digital Pathology (Abstract)[END_REF]. The goal behind developing this algorithm was to study the spatial distribution of different cells in the tissue (i.e. cancer cells, immune cells and fibroblasts) from a large IHC-image dataset. The algorithm gave promising results and the study will be continued in the near future, by consolidating a reliable IHC-image dataset. [START_REF] Ben Cheikh | A structure-based approach for colon gland segmentation in digital pathology[END_REF][START_REF] Sirinukunwattana | Gland segmentation in colon histology images: The glas challenge contest[END_REF] [66]

[67] [START_REF] Ben Cheikh | Graph-Based approach for spatial heterogenity analysis in tumour microenvironement[END_REF][START_REF] Cheikh | Spatial interaction analysis with graph based mathematical morphology for histopathology[END_REF] [70] FIGURE 1.9: Our frameworks related to histopathological image analysis elaborated in parallel with this study.

In order to palliate the absence of a suitable data-collection allowing spatial analysis of tissue, we have developed a technique for generating synthetic histopathological images based on a spatial model of tumor architecture and spatial interactions with the tumor microenvironment [START_REF] Ben Cheikh | A Model of Tumor Architecture and Spatial Interactions with the Microenvironement in Breast Carcinoma[END_REF], which can contribute to: i) simulation and modeling of histopathological data, ii) validating and comparing analytical methods used in digital pathology.

However, thereafter, another dataset composed of 55 whole slide images of HES-stained breast cancer tissue become available. We have benefit from our expertise acquired during the development of the nuclei detection and classification algorithm from IHC-stained tissue images [START_REF] Ben Cheikh | Nuclei classification in Immunohistochemical stainings for tumour microenvironement analysis in Digital Pathology (Abstract)[END_REF], in the improvement/development of a new algorithm adapted to HES-stained tissue images.

Chapter 1. Role of Digital Pathology and Image Analysis in Oncology

Having a more complete picture in mind, and knowing more closely the reality about the spatial dimension of histopathological data, a theoretical framework has been established for spatial point pattern analysis with the focus on spatial aspects that are found in histopathological images [START_REF] Ben Cheikh | Graph-Based approach for spatial heterogenity analysis in tumour microenvironement[END_REF]. The methodology developed has then been applied to the dataset of HES breast tissue images in order to study the spatial arrangement of histological structures [START_REF] Cheikh | Spatial interaction analysis with graph based mathematical morphology for histopathology[END_REF].

Work efforts had been made in the development of automatic algorithms for the detection of micro metastasis from colorectal lymph nodes [START_REF] Venancio | Micrometastasis detection guidance by whole-slide image texture analysis in colorectal lymph nodes[END_REF], and for the improvement of a mitosis detection algorithm developed by third parties [START_REF] Irshad | Automated mitosis detection using texture, SIFT features and HMAX biologically inspired approach[END_REF], with the aim of integrating the technology in an image analysis software dedicated for research purposes. Furthermore, the methodology developed within the framework of spatial point pattern analysis has also been explored in the investigation of tumor growth from mouse xenograft models.

Different research issues linked to histopathological image analysis were explored, enabling us to take a larger view of the problems encountered in Digital Pathology and meet the challenges they are currently facing. As we can see, there are still a number of challenging problems to be tackled, and this domain is in a particular need for further research, enabled by the establishment of well-defined projects and identification of research priorities.

Thesis Structure

This thesis is structured in six chapters (see figure 1.10). Chapter 2 gives a comprehensive review on graph-based methods explored in histopathological image analysis. We propose a classification of the state-of-the-art methodologies based on the type of the graph, the category of the feature extraction technique, and the histological structures considered.

Chapter 3 presents a theoretical framework dedicated to the evaluation of the spatial arrangement of points using graph-based mathematical morphology. Morphological features are proposed to describe the distribution of spatial point patterns and how distinct point sets are located relative to each other.

Chapter 4 describes a method for nuclei detection and classification from histopathological images of HES (Haematoxylin, Eosin, Saffron)-stained breast cancer tissues, using supervised machine learning algorithms based on color and texture information, and methods for collagen and adipose tissue segmentation.

Chapter 5 combines the framework established in chapter 3 and the output of the method described in chapter 4, in a study for the analysis of the spatial distribution of different tissue components and the spatial interactions between them. The study is conducted on a dataset composed of 55 whole slide images of tissue sections obtained from surgical resections of invasive breast carcinomas from distinct patients. Finally, Chapter 6 describes a novel tissue simulation approach that gives access to a large spectrum of tumor-like architectures based on tools of classical mathematical morphology and statistics. The simulation method presents a valuable tool to generate a large synthetic data for validating and comparing the methodology developed in chapter 3 with other methods such as spatial statistics.

Review of Graph-based Methods in Histopathological Image Analysis 2.1 Introduction

As mentioned in chapter 1, section 1.3.2, graph-based methods have been widely applied in histopathological image analysis. In this chapter, we first present a short introduction of the fundamental concepts in graph theory, enabling us to better understand some basic notions and definitions that will help to follow the discussion given in the rest of the document. Then, we provide a state-of-the-art summary of the various architectural methods developed using graph-theoretic descriptions in histopathological image analysis based on an assessment of the existing literature. A very good review on these methods is presented in [START_REF] Sharma | A review of graph-based methods for image analysis in digital histopathology[END_REF], where the authors have have categorized the methods based on the type of the graph. In our study, we focus mainly on the methodology used to extract information from graphs, regardless their types, which are discussed separately.

Basic Concepts and Notions in Graph Theory

Overview

Graph theory is a branch of mathematics that deals with mathematical structures called graphs. The theory was born in 1736, when Leonhard Euler investigated the notable mathematical problem of the seven bridges of Königsberg [START_REF] Euler | Solutio problematic ad Geometrian situs pertinentis[END_REF]. From there, graphs have been used to model relations between mathematical objects. There are many excellent references on the mathematics of graph theory that can be found in the literature [START_REF] Diestel | Graph Theory[END_REF][START_REF] Bondy | Graph Theory[END_REF][START_REF] Gross | Handbook of Graph Theory[END_REF][START_REF] Biggs | Algebraic Graph Theory[END_REF][START_REF] Harary | Graph Theory[END_REF][START_REF] Gibbons | Algorithmic Graph Theory[END_REF]. Graph theory has then been applied in almost every scientific and engineering field, such as chemistry, geography, astrophysics, electrical and mechanical engineering, architecture, management, operational research, optimization [START_REF] Watts | Six Degrees: The Science of a Connected Age[END_REF][START_REF] Barabasi | Linked: How Everything Is Connected to Everything Else and What It Means[END_REF][START_REF] Bondy | Graph theory and related topics[END_REF], and in particular, during the last half of the century, in computer science [START_REF] Shirinivas | Applications of graph theory in computer science an overview[END_REF][START_REF] Deo | Graph theory with applications to engineering and computer science[END_REF]. Indeed, for many applications in this field, the notion of structure is important, such as in cluster analysis, pattern recognition [START_REF] Jolion | Graph-Based Representations in Pattern Recognition[END_REF], computer vision, and image processing and analysis [START_REF] Lezoray | Image Processing and Analysis with Graphs: Theory and Practice[END_REF]. In the latter, a certain number of objects may be studied in 2D or 3D space, and it may be of great interest to consider these image components (objects), not only composed of pixels, but also composed of relations between these objects, enabling the extraction of relevant structural information about how they are interacting with each other. These relationships may be modeled by graphs, as they are flexible representation tools, having an expressive ability to model topological and relational information between geometric objects. In addition, in numerous problems in digital image analysis, each object can be characterized by various numerical or symbolic parameters, which allows us to study relationships existing between different classes of objects within the same structure. For instance, in histopathology images, each cell in a given tissue can be characterized by its type, function, antibody/antigen expression, size, or shape. In satellite imagery, image components can be associated to different earth observations, such Chapter 2. Review of Graph-based Methods in Histopathological Image Analysis as habitations, vegetation, regions or aerosols, which can be characterized by various parameters. In astronomical images, celestial bodies can be characterized by their size, brightness, color, temperature, or composition. In all cases, we are interested in studying the spatial organization of objects parametrized by numerical values.

To deal with this problem using graph theory, the first step consists in defining relationships on a population of objects by constructing a graph. These initial objects are usually derived from connected components of a binary image, that was obtained from a prior segmentation step. This modeling issue is generally best resolved by proximity graphs, especially, graphs that are derived from the well-known Voronoi diagram, such as Delaunay triangulation, Gabriel graph and relative neighborhood graph. The main reason behind the popularity of these graphs lies in the fact they are independent of any scaling, which makes them suitable for various kinds of applications. In fact, these graphs do not depend on any parameter like maximal distance between objects or minimal number of neighbors, which makes them useful tools in practice. In addition, these graphs are connected and planar, which ensures correlation between adjacency and proximity. Another very particular property lies in the fact that they are included one into another, allowing modeling of neighborhood relationships with increasing degree of connectedness.

Once the relationships between objects are defined, using one of the graphs mentioned above or other neighborhood relationships, the next step consists in extracting appropriate measures from the graph which had been constructed. There are many graph-based features that can be used to extract quantitative structural information from a graph. They can be derived from statistical analysis of the geometrical properties of the structures composing the graph, e.g. area, perimeter, angles of the triangles of a Delaunay triangulation, as well as from topological measurements that describe the way the objects are connected to each other, e.g. the number of connections that have each object (degree), the greatest distance between an object and any other object (eccentricity). Some definitions of the most commonly used measures can be found in [START_REF] Wallis | A beginner's guide to graph theory[END_REF]. Several graph-based features are derived from spectral analysis based on spectral graph theory, which is the study of properties of graphs relative to the characteristic polynomial, eigenvalues and eigenvectors of matrices associated with graphs, such as Laplacian matrix or adjacency matrix [START_REF] Andries | Spectra of Graphs[END_REF]. These approaches were successfully used in a wide range of applications [START_REF] Daniel | Spectral graph theory and its applications[END_REF], but they are inadequate when we deal with a very large number of objects due to the complexity of calculations, as in the case of whole slide images.

Definition

A graph is a representation of a set of objects and a set of pairwise relationships between those objects. The objects are called nodes or vertices, and the relationships are called links or edges. Formally, a graph G is an ordered pair (V, E) comprising a set of vertices V = {v i } 1≤i≤n and a set of edges E ⊆ V × V , which are 2-element subsets of V . The number of vertices |V | and the number of edges |E| are called the order and the size of the graph, respectively.

We assimilate a binary graph G with its indicator function G : V → {0, 1}, and a decimal graph with its indicator function G : V → R. This allows us to deal with objects parameterized by numerical values. The numerical values associated to a binary or decimal graph can be assimilated to weights associated to vertices. The graph is then called a vertex-weighted graph. However, there are also edge-weighted graphs, where weights are associated to edges, but we do not use them in our study.

From now on, we will consider only simple graphs, which are graphs that satisfy the following assumptions: (i) undirected, i.e. the edges have no orientation, edge (u, v) is identical to edge (v, u). (ii) containing no multiple edges (two vertices cannot be connected by more than one edge). (iii) containing no loops (edge that joins a vertex to itself).

In addition, most graphs used in this study are planar. A planar graph is a graph that can be drawn on the plane in such a way that its edges intersect only at their endpoints, which means that it can be drawn in such a way that no edges cross each other.

Adjacency and distance

For every vertex v ∈ V , we call the neighborhood, and we denote N E (v), the set of all vertices adjacent to vertex v:

∀v ∈ V, N E (v) = {u ∈ V | (u, v) ∈ E} (2.1)
An example of a neighborhood of a vertex is shown in figure 2.2.a. For any pair of vertices (u, v) ∈ V 2 , we define the path of length k from u to v by the (k + 1)-uplet (p 0 , p 1 , ..., p k ) ∈ V k+1 such that:

∀i ∈ [1, k], (p i-1 , p i ) ∈ E and p 0 = u, p k = v
So, for any pair of vertices (u, v) ∈ V 2 , we can define a distance d E (u, v) as the shortest path between them: d E (u, v) is the minimal length of paths from u to v. If there is no path between these two vertices, the distance is taken as equal to infinity. An example of the shortest path between two vertices is shown in figure 2

Labeling

We call connected component in a binary graph G, the sub-graph of G in which any two positive vertices, i.e. G(v) = 1, are connected to each other by at least one path (see example in 2.3).

Connected component labeling of a binary graph consists of assigning a unique number to each of its connected components. Hence, the vertices that belong to the same connected component will have the same number (identifier of their connected component).

Let G be a binary graph and C 1 , C 2 , ..., C k , its different connected components. The connected component labeling of G can be defined by a decimal graph L(G) such that: The labeled graph L(G) is not unique, there are k! possibilities, as any connected component can be labeled with any number between 1 and k. To compute the connected components of a graph in linear time, we can use either breadth-first search (BFS) or depth-first search (DFS) algorithms [START_REF] Hopcroft | Algorithm 447: efficient algorithms for graph manipulation[END_REF]. Connected component labeling is useful to detect connected regions in a binary graph.

∀v ∈ V, L(G)(v) = 0 if G(v) = 0 k if v ∈ C k (2.2) (a) (b)

Proximity graphs

Given a set of objects, a linking rule has to be defined between them in order to produce edges for the graph to be processed/analyzed. There exist various ways to define relationships within a set of objects, and some of them define neighborhood relationships between geometric points in the Euclidean space, called proximity graphs. A proximity graph is a graph in which two vertices are adjacent if they satisfy a specific geometric rule. In what follows, we will present some of the most commonly used graphs, with particular attention to those which present interesting characteristics for our study. In the following, we assume that the initial objects are points on a 2-D Euclidean plane.

k-nearest neighbor graph

The k-nearest neighbor graph (k-NNG) is a graph in which every object is connected to its k nearest objects. Two vertices (u, v) ∈ V 2 are connected by an edge iff the Euclidean distance between u and v is among the k smallest distances from v to all other objects of V . Next figure (2.4) presents some examples of k-nearest neighbor graphs. The k-nearest neighbor graph is one of the simplest and the most classical techniques for generating neighborhood relationships between spatial points. However, the graph obtained is not necessarily connected and not defined in some specific configurations. The topology of the graph relies heavily on the parameter k, which regulates the intensity of connecteness.

Epsilon-Ball graph

The Epsilon-Ball Graph (EBG E ) is a graph in which two vertices are connected if the Euclidean distance between them is not greater than E, with E ∈ R * . Here again, the graph is not necessarily connected or planar. The connectedness of this graph depends largely on the choice of the distance maximal distance E (see fig.

Voronoi diagram

Voronoi diagram [START_REF] Georges | Nouvelles applications des paramètres continus à la thèorie des formes quadratiques. Deuxième mémoire. Recherches sur les parallélloèdres primitifs[END_REF] also called Thiessen polygons or Dirichlet tessellation [START_REF] Dirichlet | Über die Reduction der positiven quadratischen Formen mit drei unbestimmten ganzen Zahlen[END_REF], is the partitioning of the plane with a set of points into convex polygons. Let V be a finite set of points in the plane. For any v ∈ V , the Voronoi polygon Γ(v), associated with v, is the set of all points in the plane that are closer to v than to any other point of V :

∀v ∈ V, Γ(v) = {p ∈ R 2 |dist(p, v) < dist(p, u), ∀u ∈ V \ {v}} (2.3)
where dist(p, q) is the Euclidean distance between points p and q. The polygon Γ(v) is also called the "influence zone" of point v. These regions partition the plane into a net which is called Voronoi diagram of V , denoted V or(V ). An example of a Voronoi diagram and its dual graph ( In the following, we will focus on graphs derived from Voronoi diagram, as they are the most interesting for several reasons that are presented below. More details about the properties of Voronoi diagrams are also given in [START_REF] Aurenhammer | Voronoi diagrams[END_REF].

Delaunay triangulation

The Delaunay triangulation (DT) of a set of points V in the plane is a triangulation based on the "empty circle" property, where no point in V falls in the interior of the circumcircle of any triangle of the triangulation [START_REF] Delaunay | Sur la sphère vide. A la mémoire de Georges Voronoi[END_REF]. An example of Delaunay triangulation is shown in figue 2.7.a.

Delaunay triangulation corresponds to the dual graph of Voronoi diagram and its edges are defined by all pairs of points (u, v) ∈ V 2 , whose Voronoi Polygons Γ(u) and Γ(v) are adjacent (polygons that share one side). The Delaunay triangulation is a graph (V, E DT ) such that:

E DT = {(u, v) ∈ V 2 |Γ(u) ∩ Γ(v) = ∅} (2.4)

Gabriel graph

Gabriel graph was introduced in [START_REF] Gabriel | A new statistical approach to geographic variation analysis[END_REF], and it is a graph that is based on least square adjacency between points in the plane. The edges of Gabriel graph (GG) are defined by all pairs of points (u, v) ∈ V 2 such that no other point in V falls in the closed disc admitting segment [uv] as one of its diameters:

E GG = {(u, v) ∈ V 2 |dist(u, v) 2 < dist(u, w) 2 + dist(w, v) 2 , ∀w ∈ V \ {u, v}} (2.5)
where dist(u, v) is the Euclidean distance between points u and v.

Gabriel graph is a sub-graph of Delaunay triangulation and can be easily obtained by removing edges from Delaunay graph when a point in V falls in the closed disc admitting the edge as one of its diameters. The properties of Gabriel graph were studied by Matula and Sokal in [START_REF] Matula | Properties of Gabriel graphs relevant to geographic variation research and clustering of points in the plane[END_REF]. An example of Gabriel graph is also shown in figure 2.7.b.

Relative neighborhood graph

First introduced by Lankford [START_REF] Lankford | Regionalization: theory and alternative algorithms[END_REF], the relative neighbor graph (RNG) is a graph (V, E RN G ) in which two points (u, v) ∈ V 2 are connected whenever there does not exist a third point w ∈ V that is closer to both u and v than they are to each other. The edges of RNG are then defined by all pairs of points (u, v) ∈ V 2 , such that there is no other point of V in the intersection of open discs having segment [uv] as radius and centered in u and v respectively:

E RN G = {(u, v) ∈ V 2 |dist(u, v) max{dist(u, w), dist(w, v)}, ∀w ∈ V \ {u, v}} (2.6)
This graph has been studied by Toussaint [START_REF] Toussaint | The Relative Neighborhood Graph of a finite planar set[END_REF] and Supowit [START_REF] Supowit | The Relative Neighborhood Graph with an Application to Minimum Spanning Trees[END_REF]. It can be derived from Gabriel graph by retaining only edges (u, v) for which the intersection of the two circles of radius [uv], and centered in u and v, is empty (see 2.7.c).

Euclidean minimum spanning tree

A tree is a graph that contains no cycle (a path where a vertex is reachable from itself). In a tree, any two vertices are connected by exactly one path. A spanning tree of a graph is a tree that connects all vertices of the graph.

The Euclidean minimum spanning tree (EMST) [START_REF] Czumaj | Testing euclidean minimum spanning trees in the plane[END_REF] is a spanning tree of the graph that connects by an edge each pair of points (complete graph), such that the total length of all edges is minimized [START_REF] Graham | On the history of the minimum spanning tree problem[END_REF]. EMST can be obtained from the RNG, such that an edge is retained if it is not the longest edge of a cycle in the RNG (see 2. The neighborhood graphs derived from Voronoi diagram, DT, GG, RNG and EMST, are very useful to study proximity problems as they describe objects based on their relative spatial distribution and having the following remarkable properties: -They are connected (there is a path from any point to any other point).

-They are planar (can be drawn in such a way that no edges cross each other).

-They are included one into another:

EM ST ⊂ RN G ⊂ GG ⊂ DT (2.7)
The inclusions here prescribe the neighborhood relationships induced by the graph on V , which have increasing intensity. Thus, the morphological analysis of a point set can be studied with different connectedness intensity levels by the mean of these graphs, describing that way different characteristics.

Alpha-shape and Alpha-complex

α-shapes were introduced by H. Edelsbrunner [START_REF] Edelsbrunner | On the shape of set of points in the plane[END_REF] to give generalization of the concept of the convex hull representation [START_REF] Brown | Voronoi diagrams from convex hulls[END_REF]. For α > 0, the α-shape S α of a point set V ∈ R 2 is a polygon such that, when α decreases, it gradually develops cavities. S ∞ (V ) is the convex hull of V , and S 0 = V (see figure 2.8).

α-complexes are closely related to α-shapes. For α > 0, the α-complex C α of a point set V ∈ R 2 can be defined by the triangles of the Delaunay triangulation of V , that are in the interior of the envelop defined by the corresponding α-shape of V [START_REF] Loménie | Morphological Mesh Filtering and α-objects[END_REF]. It should be noted that C ∞ (V ) is the Delaunay triangulation of V and C 0 (V ) is the null graph (edgeless graph where all nodes are isolated).

α-complexes can be easily obtained by sculpting Delaunay triangulation. Each triangle of Delaunay triangulation can be associated with a characteristic radius, i.e. the radius of the circumscribed circle of the triangle. For each real number α, the triangles with circumradii bigger than α are removed. α-complexes are actually considered as simplicial complexes [START_REF] Munkres | Simplicial Complexes and Simplicial Maps, New York: Perseus Books Pub[END_REF], a combinatorial description of points, line segments, triangles, tetrahedron, etc. But, one can comfortably adopt the notion of graphs by defining the points as vertices and the line segments as edges (see figure 2

.8). (a) (b) (c) (d) (e) (f) (g) FIGURE 2.8: (a) Set of points V = (null graph = S0(V ) = C0(V )). (b) Con- vex hull (= S∞(V )). (c) S300(V ). (d) S100(V ). (e) Delaunay triangulation (C∞(V )). (f) C300(V ). (g) C100(V ).
One of the main advantages of using α-complexes is that they allow us to remove connections from Delaunay graph that may exist between points that hardly seem to be neighbors, such as the case of the outermost points.

Graph transformations

Graph transformation (or graph rewriting) is the technique of creating a new graph from an original graph algorithmically. In this section, we present two main techniques of graph transformation.

Line graph

The line graph of a graph G is another graph L(G) that represents the adjacencies between the edges of G.

Face graph

The face graph of a planar graph G is another graph that represents the adjacencies between the face of G.

The adjacency between two faces can be defined in two manners. We define the face graph of order 1 and we denote F 1 (G), the graph that has a vertex for each face of G and an edge whenever two faces of G are separated from each other by an edge (see examples in figures 2.10.b and 2.10.e). We define the face graph of order 2 and we denote F 2 (G), the graph that has a vertex for each face of G and an edge whenever two faces of G are sharing a common point (see examples in figures 2.10.c and 2.10.f).

(a) (b) (c) (d) (e) (f)

Summary

In this section, we have presented the basic defintions and notions of in graph theory and a non-exhaustive list of graphs that can be used to study the spatial distribution of a spatial point set. However, there is a very large variety of graphs that can be used to extract relevant information about the spatial distribution of points, such as barycentric neighbors graph [START_REF] Lankford | Regionalization: theory and alternative algorithms[END_REF], beta-skeleton [START_REF] Kirkpatrick | A framework for computational morphology[END_REF], Urquhart graph [START_REF] Urquhart | Algorithms for computation of relative neighborhood graph[END_REF], sphere of influence graph [START_REF] Toussaint | Proceedings Fifth International Conference on Pattern Recognition[END_REF], O'Callaghan [START_REF] O'callaghan | An alternative definition for neighborhood of a point[END_REF], Johnson-Mehl tessellation [START_REF] Moller | Random Johnson-Mehl tessellations[END_REF], Vietoris-Rips complex [START_REF] Hausmann | On the Vietoris-Rips complexes and a cohomology theory for metric spaces[END_REF], Waxman graph [START_REF] Waxman | Routing of multipoint connections[END_REF], attributed relational graph [START_REF] Sanfeliu | A distance measure between attributed relational graphs for pattern recognition[END_REF], complete graph [START_REF] Skiena | Complete Graphs[END_REF], cactus graph [START_REF] Harary | On the number of Husimi trees[END_REF], Ulam tree [START_REF] Ulam | Patterns of Growth of Figures: Mathematical Aspects[END_REF], and the list goes on. There are various types of graphs, each with its own definition. Algorithm for reconstructing commonly used graphs and freely available tools for developing graph-based applications can be found in [START_REF] Sharma | A review of graph-based methods for image analysis in digital histopathology[END_REF]. The choice of the graph and its reconstruction parameters depends on the application and the structural properties being sought. One can still adopt new linking rules to connect a set of points and define specific graphs, but in our study we are more interested in the methodology adopted to extract useful information from a graph, that has already been defined.

Node Identification Techniques

Quantitative architectural analysis in histopathological images may concern the study of the spatial organization of objects representing diverse histological entities perceived at different tissue levels. These objects may be representative of simple entities, like cell nuclei, or more complex histological structures, like glands. To study the spatial organization of these histological structures using graphs, a prior node identification step is needed. The Chapter 2. Review of Graph-based Methods in Histopathological Image Analysis nodes are commonly defined as the geometric centroids of image objects representing these structures. The identification may be accomplished by manually labeling these structures on the tissue image or by using an automatic image segmentation technique.

Most of the studies have focused on the characterization of the spatial organization of cell nuclei (see example in figure 2.11.a). The nodes are either defined on all cell nuclei, of any type, without distinction [START_REF] Landini | Architectural analysis of oral cancer, dysplastic, and normal epithelia[END_REF][START_REF] Guillaud | Exploratory analysis of quantitative histopathology of cervical intraepithelial neoplasia: Objectivity, reproducibility, malignancy-associated changes, and human papillomavirus[END_REF][START_REF] Doyle | Automated Grading of Prostate Cancer using architectural and textural image Features[END_REF][START_REF] Doyle | Cascaded discrimination of normal, abnormal, and confounder classes in histopathology: Gleason grading of prostate cancer[END_REF] or defined on a particular type of cells, e.g. tumor cells [START_REF] Kayser | Localized fibrous tumors (LFTs) of the pleura: Clinical data, asbestos burden, and syntactic structure analysis applied to newly defined angiogenic/growthregulatory effectors[END_REF][START_REF] Basavanhally | Multi-field-of-view framework for distinguishing tumor grade in ER+ breast cancer from entire histopathology slides[END_REF], or immune cells [START_REF] Basavanhally | Manifold learning with graph-based features for identifying extent of lymphocytic infiltration from high grade, her2+ breast cancer histology[END_REF][START_REF] Schaadt | A Graph-Based Digital Pathology Approach To Describe Lymphocyte Clustering Patterns After Renal Transplantation[END_REF]. To characterize the spatial distribution of glandular structures, the majority of the studies have initiated research on the topic by exploring colorectal tissue images [START_REF] Cohen | Colon Biopsy Classification Using Crypt Architecture[END_REF][START_REF] Kayser | Neighborhood Analysis of Low Magnification Structures (Glands) in Healthy, Adenomatous, and Carcinomatous Colon Mucosa[END_REF][START_REF] Kayser | Analysis of adenomatous structures in histopathology". In: Analytical and quantitative cytology and histology/the International Academy of Cytology [and[END_REF][START_REF] Kayser | Application of structural pattern recognition in histopathology[END_REF][START_REF] Zaitoun | Quantitative assessment of gastric atrophy using the syntactic structure analysis[END_REF] (see example in figure. 2.11.b), and exceptionally, in prostate tissue images [START_REF] Lee | Co-Occurring Gland Angularity in Localized Subgraphs: Predicting Biochemical Recurrence in Intermediate-Risk Prostate Cancer Patients[END_REF]. It should be noted that graphs themselves have been used for gland segmentation purposes [START_REF] Gunduz-Demir | Automatic segmentation of colon glands using object-graphs[END_REF]. Which in addition indicates the usefulness of graphs in histopathological image segmentation. However, automatic nuclei detection and gland segmentation are both still challenging problems in the image processing field (section 1.2). Due to the complexity of the detection/segmentation task, some architectural analysis approaches limited themselves to the study of the spatial organization of image objects that represent clusters of cells rather than individual cells [START_REF] Bilgin | Cell-graph mining for breast tissue modeling and classification[END_REF][START_REF] Ali | Cell cluster graph for prediction of biochemical recurrence in prostate cancer patients from tissue microarrays[END_REF][START_REF]A quantitative histomorphometric classifier (QuHbIC) identifies aggressive versus indolent p16-positive oropharyngeal squamous cell carcinoma[END_REF][START_REF] Demir | Spectral analysis of cell-graphs for automated cancer diagnosis[END_REF][START_REF] Demir | Augmented cell-graphs for automated cancer diagnosis[END_REF][START_REF] Demir | Learning the topological properties of brain tumors[END_REF] (see example in figure 2.11.c). In the same logic, some approaches study the spatial distribution of tissue compoennts (regions) instead of 2.4. Relationship Establishment Techniques 29 significant histological objects like glands or cell nuclei. These tissue regions are commonly obtained by classification of the image pixels based on intensity, color and/or texture features [START_REF] Gulden | Graph walks for classification of histopathological images[END_REF][START_REF] Sharma | Determining similarity in histological images using graph-theoretic description and matching methods for content-based image retrieval in medical diagnostics[END_REF][START_REF] Altunbay | Color graphs for automated cancer diagnosis and grading[END_REF] (see example in figure 2.11.d).

The last two approaches are more accessible compared to techniques that require the identification of every single nucleus/gland in the image, and they offer architectural features in a higher tissue level than nuclei-attributed nodes' techniques. In addition, as it has been stated in [START_REF] Gurcan | Histopathological image analysis: A review[END_REF], the histological objects may not need to be perfectly segmented to be properly evaluated when a list of comprehensive features is adopted in the framework. However, using any of these node identification techniques, enables us to deal with histological objects represented by a set of points identified by their coordinates in the image plan. And the main objective is to extract information about how they are distributed relative to each other. In addition to their coordinates, numerical values may be attributed to the nodes that have been defined. For examples, nodes representative of nuclei may be characterized by the cell type, function, or antigen expression. Whereas, nodes associated to glands may be characterized by their size or shape, cell clusters by their density, and regions by their area, perimeter, or color information. In Table 2.2, we have listed the biological objects whose spatial organization is studied in several graph-based methods found in the literature.

Relationship Establishment Techniques

The main idea behind defining a set of histological objects by their spatial coordinates (nodes), is to be able at a later stage to set up links (graphs) between them, based on their locations. Various linking rules have been used in the literature to quantify tissue architecture. Some approaches have adopted graphs that have been traditionally used to respond some graph theoretical problems, whereas, in some other studies, authors have chosen to define linking rules and neighborhood relationships in their own ways.

Earliest works have explored O'Callaghan graph [START_REF] O'callaghan | An alternative definition for neighborhood of a point[END_REF] to differentiate histological subtypes of lung cancer by studying the spatial arrangement of epithelial cells [START_REF] Kayser | TNM stage, immunohistology, syntactic structure analysis and survival in patients with small cell anaplastic carcinoma of the lung[END_REF][START_REF] Kayser | Syntactic structure analysis of bronchus carcinomas first results[END_REF]. Then, O'Callaghan graph has been employed in automatic grading of colorectal cancers by evaluating the spatial organization of intestinal glands [START_REF] Kayser | Application of structural pattern recognition in histopathology[END_REF][START_REF] Kayser | Analysis of adenomatous structures in histopathology". In: Analytical and quantitative cytology and histology/the International Academy of Cytology [and[END_REF][START_REF] Kayser | Neighborhood Analysis of Low Magnification Structures (Glands) in Healthy, Adenomatous, and Carcinomatous Colon Mucosa[END_REF]. The advantages of O'Callaghan graphs have been discussed in detail in [START_REF] Kayser | Neighborhood condition and application of syntactic structure analysis in histopathology[END_REF]. According to the authors, the motivation behind using this graph is that two histological objects are considered to interact with each other only if they are located within a certain distance (distance constraint), and not hidden behind any other object considered as neighbor (direction constraint).

The minimum spanning tree (MST) [START_REF] Graham | On the history of the minimum spanning tree problem[END_REF] has been abundantly used in quantitative architectural analysis of histological tissue of the breast [START_REF] Van Diest | Syntactic structure analysis in invasive breast cancer: analysis of reproducibility, biologic background, and prognostic value[END_REF][START_REF] Basavanhally | Manifold learning with graph-based features for identifying extent of lymphocytic infiltration from high grade, her2+ breast cancer histology[END_REF][START_REF] Doyle | Automated grading of breast cancer histopathology using spectral clustering with textural and architectural image features[END_REF][START_REF] Madabhushi | Review: Integrated diagnostics: a conceptual framework with examples[END_REF][START_REF] Basavanhally | Computerized image-based detection and grading of lymphocytic infiltration in her2+ breast cancer histopathology[END_REF][START_REF] Basavanhally | Multi-field-of-view framework for distinguishing tumor grade in ER+ breast cancer from entire histopathology slides[END_REF], prostate [START_REF] Doyle | Automated Grading of Prostate Cancer using architectural and textural image Features[END_REF][START_REF] Doyle | Using manifold learning for content-based image retrieval of prostate histopathology[END_REF][START_REF] Doyle | Cascaded discrimination of normal, abnormal, and confounder classes in histopathology: Gleason grading of prostate cancer[END_REF], lung [START_REF] Kayser | Localized fibrous tumors (LFTs) of the pleura: Clinical data, asbestos burden, and syntactic structure analysis applied to newly defined angiogenic/growthregulatory effectors[END_REF][START_REF] Kayser | Combined morphometrical and syntactic structure analysis as tools for histomorphological insight into human lung carcinoma growth[END_REF][START_REF] Weyn | Value of morphometry, texture analysis, densitometry, and histometry in the differential diagnosis and prognosis of malignant mesothelioma[END_REF][START_REF] Guillaud | Quantitative architectural analysis of bronchial intraepithelial neoplasia[END_REF], colon [START_REF] Meijer | Syntactic structure analysis of the arrangement of nuclei in dysplastic epithelium of colorectal adenomatous polyps[END_REF], stomach [START_REF] Zaitoun | Quantitative assessment of gastric atrophy using the syntactic structure analysis[END_REF], bladder [START_REF] Choi | Image analysis based grading of bladder carcinoma. Comparison of object, texture and graph based methods and their reproducibility[END_REF], and cervix [START_REF] Chaudhuri | Characterization and featuring of histological section images[END_REF][START_REF] Weyn | Determination of tumour prognosis based on angiogenesis-related vascular patterns measured by fractal and syntactic structure analysis[END_REF]. MST can be derived from different graphs, the most commonly the Voronoi diagram. Features derived MST are usually combined with features derived from other graphs from the same Voronoi family (e.g. Delaunay, Gabriel) (see Table 2.

2).

The Voronoi diagram [START_REF] Georges | Nouvelles applications des paramètres continus à la thèorie des formes quadratiques. Deuxième mémoire. Recherches sur les parallélloèdres primitifs[END_REF] has been largely used. According to the authors of [START_REF] Kayser | Neighborhood condition and application of syntactic structure analysis in histopathology[END_REF], the main advantages of using Voronoi diagram in the representation of a population of cells lay in the possibility of defining a unique area for each cell (Voronoi polygon), visualizing its boundaries, and introducing metrics according to its geometry. The same authors [START_REF] Kayser | Neighborhood condition and application of syntactic structure analysis in histopathology[END_REF], stated that Voronoi neighborhood condition is not suitable for analysis of glandular structures as it is independent of the size of structures and the polygon defined for each gland may not match with the real glandular surface.

Although, Delaunay graph [START_REF] Delaunay | Sur la sphère vide. A la mémoire de Georges Voronoi[END_REF] has been abundantly applied in the field. Some of the advantages of Delaunay graph include an easy extraction of geometric features from the triangulation by exploring the triangle surfaces, circumradii, angles, etc. The authors of [START_REF] Raymond | Germinal center analysis with the tools of mathematical morphology on graphs[END_REF] have rejected the Delaunay triangulation, because the outmost points may be linked even if they barely seem to be neighbors. They declared that Gabriel graph [START_REF] Gabriel | A new statistical approach to geographic variation analysis[END_REF] seems to be more interesting to minimize those artefacts created at the periphery of the graph. The authors have also explained that Gabriel graph seems to better express neighborhood relationships that may have a biological significance, as two cells are considered as neighbors if no other interfering cell is placed between them. In addition, they have stated that the Chapter 2. Review of Graph-based Methods in Histopathological Image Analysis relative neighborhood graph is too poor -in term of connectedness -to be used in their approach.

In several works, a combination employing more than one of the graphs from this family is used [START_REF] Basavanhally | Multi-field-of-view framework for distinguishing tumor grade in ER+ breast cancer from entire histopathology slides[END_REF][START_REF] Doyle | Cascaded discrimination of normal, abnormal, and confounder classes in histopathology: Gleason grading of prostate cancer[END_REF][START_REF] Basavanhally | Computerized image-based detection and grading of lymphocytic infiltration in her2+ breast cancer histopathology[END_REF][START_REF] Madabhushi | Review: Integrated diagnostics: a conceptual framework with examples[END_REF][START_REF] Basavanhally | Manifold learning with graph-based features for identifying extent of lymphocytic infiltration from high grade, her2+ breast cancer histology[END_REF][START_REF] Doyle | Automated grading of breast cancer histopathology using spectral clustering with textural and architectural image features[END_REF][START_REF] Doyle | Using manifold learning for content-based image retrieval of prostate histopathology[END_REF][START_REF] Doyle | Automated Grading of Prostate Cancer using architectural and textural image Features[END_REF][START_REF] Guillaud | Exploratory analysis of quantitative histopathology of cervical intraepithelial neoplasia: Objectivity, reproducibility, malignancy-associated changes, and human papillomavirus[END_REF][START_REF] Landini | Architectural analysis of oral cancer, dysplastic, and normal epithelia[END_REF][START_REF] Weyn | Determination of tumour prognosis based on angiogenesis-related vascular patterns measured by fractal and syntactic structure analysis[END_REF][START_REF] Guillaud | Quantitative architectural analysis of bronchial intraepithelial neoplasia[END_REF][START_REF] Weyn | Value of morphometry, texture analysis, densitometry, and histometry in the differential diagnosis and prognosis of malignant mesothelioma[END_REF][START_REF] Weyn | Computer-assisted differential diagnosis of malignant mesothelioma based on syntactic structure analysis[END_REF]. Another neighborhood relationships defined by Ulam tree [START_REF] Ulam | Patterns of Growth of Figures: Mathematical Aspects[END_REF] has been applied along with the Voronoi diagram and the derived Delaunay graph family.

More recently, a graph technique, named cell-graph, has been extensively used [START_REF] Ali | Cell cluster graph for prediction of biochemical recurrence in prostate cancer patients from tissue microarrays[END_REF][START_REF] Bilgin | Cell-graph mining for breast tissue modeling and classification[END_REF][START_REF] Gunduz | The cell graphs of cancer[END_REF][START_REF] Demir | Augmented cell-graphs for automated cancer diagnosis[END_REF][START_REF] Bilgin | ECM-aware cell-graph mining for bone tissue modeling and classification[END_REF][START_REF] Demir | Spectral analysis of cell-graphs of cancer[END_REF][START_REF] Schäfer | CD30 cell graphs of Hodgkin lymphoma are not scale-free-An image analysis approach[END_REF]. Several authors have used the term cell-graph because the graph nodes represent cells or clusters of cells in the tissue. But from a theoretical point of view, based on neighborhood conditions and linking rules, these graphs have already been defined in the literature. In fact, the authors in [START_REF] Sharma | A review of graph-based methods for image analysis in digital histopathology[END_REF], have presented three subtypes of cellgraphs; i) Simple cell-graph, where two vertices are connected if the Euclidean distance between them is less than a threshold, and thus, this graph is nothing else than the Epsilon-Ball Graph defined in 2.2.5. ii) Probabilistic cell graph, where the connection between two vertices is defined by a probability function that takes into account the Euclidean distance between them. This linking rule has already be defined in the model of Waxman graph [START_REF] Waxman | Routing of multipoint connections[END_REF]. iii) Hierarchical cell-graph, where vertices represent cell clusters, rather than cells, and links are defined using the Epsilon-Ball-Graph linking rules. Thus, the later technique does not use a new linking rule but a different node identification technique. In [START_REF] Bilgin | ECM-aware cell-graph mining for bone tissue modeling and classification[END_REF], the authors have pointed out some advantages of the "cell graph" approach over Delaunay family graphs, especially in the fact of considering spatial relationships between histological objects within a small neighborhood, allowing formation of connected components which can be used for representing sparse tissues. This comes from the fact that the node-adjacency is regulated by the parameter E, equal to the distance between them, in the graph Epsilon-Ball.

Some studies have adopted the attributed relational graphs (ARG), where both vertices and edges have attributes (numerical values or weights), representing their properties. The linking rules used for these graphs may be defined by one of the graphs presented above. In [START_REF] Waxman | Routing of multipoint connections[END_REF], it is mentioned that ARGs represent an interest for researchers in this field, and they have many advantages as they take into account, both spatial distribution and object/edge properties. In fact, ARG can be employed to study relationships between histological structures of different types when each node is parameterized according to the type of the structure, for example to study the tumor microenvironment.

Some authors have developed other graph reconstruction techniques used less frequently for the quantitative spatial analysis in histological images. Most of them are presented in [START_REF] Waxman | Routing of multipoint connections[END_REF]. One still can conceive more graphs defined by various neighborhood conditions and can imagine numerous linking rules in order to extract significant information about the architecture of the tissue.

Feature Extraction Techniques

In computerized image analysis, feature extraction refers to the determination of characteristics that are discriminative in a given context, in form of input variables or attributes derived from the image pixels (e.g. using intensity, color, texture, morphology). In graph theory, feature extraction refers to the calculation of a set of parameters (metrics) that describe the graph structure in a view of a particular aspect. These parameters can be descriptive of the entire graph structure (global metrics), or associated to graph nodes (local metrics), where each node has its own descriptive features. Local metrics can be used for the segmentation of a graph/tissue or to deduce global metrics allowing classification of graphs/tissues.

Although a considerable number of graphs, defining various linking rules between objects, have been used in histopathological image analysis, only few feature extraction techniques can be identified. In fact, we can distinguish five different techniques (i.e. syntactic structure analysis, network analysis, spectral analysis, persistent homology and mathematical morphology). Only three of these five techniques have been amply explored. In the following, we give a brief description of these methods.

Syntactic structure analysis

The earliest graph-theoretic methods employed to quantitatively analyze tissue architecture were explored in regard to syntactic structure analysis (SSA). SSA is a technique for computing topological features from a graph defined in a 2-or 3-dimensional space, by measuring the geometrical properties of the structures composing the graph, and by analyzing their statistical distributions in order to derive spatial interpretations. The theoretical basis of syntactic structure analysis has been extensively discussed in [START_REF] Marcelpoil | Methods for the study of cellular sociology: Voronoi diagrams and parametrization of the spatial relationship[END_REF][START_REF] Van Diest | An introduction to syntactic structure analysis[END_REF][START_REF] Van Diest | Syntactic structure analysis[END_REF][START_REF] Kayser | Neighborhood condition and application of syntactic structure analysis in histopathology[END_REF]. The geometrical structures whose properties are studied may have different dimensions, i.e. nodes, edges, or faces. The latter (faces), is available only if the graph is planar. For example, the graph structures whose geometry is analyzed, can be triangles of Delaunay triangulation, polygons on Voronoi diagram, or branches of the minimum spanning tree or Ulam tree. SSA-based properties are usually derived from geometrical properties like area, perimeter, length, angle, etc. Statistical measures, i.e. mean, standard deviation, minimum to maximum ratio, disorder, and Haralick descriptors are commonly calculated from these properties to provide global graph features. A summary of graph-based features calculated using syntactic structure analysis in histopathology is given in Table 2.1.

Syntactic structure analysis has been used in many applications in histopathology. It has been shown that SSA is useful to differentiate mesothelioma from metastatic carcinoma of the pleura [START_REF] Kayser | Differential diagnosis of mesothelioma and metastatic adenocarcinoma of pleura by means of indirect immunoperoxidase technique combined with syntactic structure analysis[END_REF] and that the technique is able to provide clinically relevant information in carcinoma of the lung, [START_REF] Kayser | Alteration of integrated optical density and intercellular structure after induction chemotherapy and survival in lung carcinoma patients treated surgically[END_REF][START_REF] Kayser | TNM stage, immunohistology, syntactic structure analysis and survival in patients with small cell anaplastic carcinoma of the lung[END_REF][START_REF] Kayser | Combined morphometrical and syntactic structure analysis as tools for histomorphological insight into human lung carcinoma growth[END_REF], prostate carcinoma [S16], colonic dysplasia, adenoma and carcinoma [START_REF] Kayser | Neighborhood Analysis of Low Magnification Structures (Glands) in Healthy, Adenomatous, and Carcinomatous Colon Mucosa[END_REF][START_REF] Meijer | Syntactic structure analysis of the arrangement of nuclei in dysplastic epithelium of colorectal adenomatous polyps[END_REF][START_REF] Kayser | Analysis of adenomatous structures in histopathology". In: Analytical and quantitative cytology and histology/the International Academy of Cytology [and[END_REF], gastric atrophy [START_REF] Zaitoun | Quantitative assessment of gastric atrophy using the syntactic structure analysis[END_REF], carcinoma of the breast [START_REF] Van Diest | Syntactic structure analysis in invasive breast cancer: analysis of reproducibility, biologic background, and prognostic value[END_REF], endometrial carcinoma [S18], bone and soft tissue [START_REF] Kayser | Analysis of soft tissue tumors by an attributed minimum spanning tree[END_REF]. More explanation of these approaches can be found in [START_REF] Sharma | A review of graph-based methods for image analysis in digital histopathology[END_REF], however the authors declared that though SSA is theoretically powerful, it has limited usability in practical applications due to its extensive computational requirements [START_REF] Sharma | A review of graph-based methods for image analysis in digital histopathology[END_REF]. Indeed, SSA technique offers architectural features able to efficiently describe the spatial distribution of a set of points based on the geometry of the structures derived from the topology of the graph, however, for an application that deals with a very large number of objects (e.g. all cell nuclei in a WSI) the computation of the geometric properties of all the structures could be very costly.

Network analysis

The spatial distribution of histological objects have been studied using features formally related to the communication and the connections existing between them. A graph in this context is treated literally as a network, and features are describing the way the nodes are connected to each others within this network. Which makes them largely used in network theory [START_REF] Dorogovtsev | Evolution of networks[END_REF]. These features are, in general, expressed by path lengths and geodesic distances, rather than metrics in the Euclidean space (e.g. length, surface, angle, shuch the case in SSA). In fact, the topological measurements used in this context are commonly derived from the number of connections that have each node (degree), the greatest distance between a node v and any other node u (eccentricity), or the proportion of links between the vertices within the neighborhood of a vertex divided by the number of links that could possibly exist between them (clustering coefficient). A long list of graph-based features earned from this methodology is given in Table 2.1. Detailed definitions and explanations of these features can also be found in [START_REF] Wallis | A beginner's guide to graph theory[END_REF][START_REF] Dorogovtsev | Evolution of networks[END_REF][START_REF] West | Introduction to Graph Theory[END_REF][START_REF] Faloutsos | On power-law relationships of the Internet topology[END_REF].

In practice, features calculated using network analysis and features calculated using syntactic structure analysis are commonly combined together in order to study the architecture of a tissue image. In Table 2.2, we have reported several studies that use these two approaches, and we have sorted them according the category of features that is the most abundant or more highlighted by the authors in their study.

Spectral analysis

In some studies, it has been proposed to quantitatively evaluate the architecture of tissue based on spectral graph theory [START_REF] Andries | Spectra of Graphs[END_REF][START_REF] Daniel | Spectral graph theory and its applications[END_REF][START_REF] Chung | Spectral graph theory[END_REF][START_REF] Chung | Spectral graph theory[END_REF]. Which is the study of properties of graphs relative to the characteristic polynomial, eigenvalues, and eigenvectors of matrices associated to the graph, such as its adjacency matrix or Laplacian matrix. Spectral graph features Chapter 2. Review of Graph-based Methods in Histopathological Image Analysis have the ability to indicate various fundamental properties of the structure of the graph, and they are commonly extracted using some algebraic operations applied to the associated matrices. Thus, spectral analysis of graphs allow modeling and interpretation to be performed in vector space rather than graph space.

Spectral graph theory was introduced in quantitative analysis of histopathological images by C. Gunduz-Demir & al. In [START_REF] Demir | Learning the topological properties of brain tumors[END_REF][START_REF] Demir | Augmented cell-graphs for automated cancer diagnosis[END_REF][START_REF] Demir | Spectral analysis of cell-graphs for automated cancer diagnosis[END_REF][START_REF] Demir | Spectral analysis of cell-graphs of cancer[END_REF], they extracted spectral and topological features from graph representations of brain tissues to distinguish cancerous tissues (malignant glioma) from healthy, benign and reactive/inflammatory tissues. Spectral features used in their study include spectral radius, sum of eigenvalues, eigen exponent, slope, etc. Some other frequently used features are given in Table 2.1. The authors have reported that they achieved 100% accuracy in the classification of cancerous and healthy tissues. The study was conducted on 646 brain biopsy samples from 60 patients. They also declared that spectral features represent efficiently the structure of a graph and are very quick to compute. Another study has adopted more spectral features in order to classify bone tissues to normal, fractured or cancerous, and the classification accuracy was reported higher than 90% [START_REF] Bilgin | ECM-aware cell-graph mining for bone tissue modeling and classification[END_REF]. The latter also uses a combination of spectral features and network analysis based features.

However, it should be noted that even though the introduction of spectral analysis of graphs in histopathology seems to be suitable in characterizing different tissue architectures, this approach has limited usability in practical applications that deal with a very large number of objects. In fact, the computational complexity of the algebraic operations needed to the calculation of the spectral features increases exponentially with the size of the associated matrices, which is the equal to the number of nodes in the graph (e.g. all cell nuclei in a WSI).

Persistent homology

A relatively new study has introduced a new concept in extracting features from graphs applied to histopathological objects [START_REF] Singh | Topological descriptors of histology images[END_REF]. The approach uses ideas from algebraic topology to construct topological descriptors that describe the spatial distribution of objects based on persistent homology. The concept of homology is a modern mathematical tool [START_REF] Mischaikow | Morse theory for filtrations and efficient computation of persistent homology[END_REF][START_REF] Herzog | Monomial Ideals[END_REF][START_REF] Alexandrov | Combinatorial Topology[END_REF][START_REF] Hibi | Algebraic combinatorics on convex polytopes[END_REF], that offers methods for computing topological features of a space at different spatial resolutions. The approach presented in [START_REF] Singh | Topological descriptors of histology images[END_REF] applies Vietoris-Rips complex [START_REF] Hausmann | On the Vietoris-Rips complexes and a cohomology theory for metric spaces[END_REF] to cell nuclei in order to discriminate breast cancer subtypes such as Basal, Luminal A, Luminal B and HER2 by studying a sequence of Betti numbers [START_REF] Mischaikow | Morse theory for filtrations and efficient computation of persistent homology[END_REF]. However, as the method was developed based on simplicial complexes, it can be easily extended to a subset of graphs, when a link between simplicial complexes and the graph structures can be established . For instance, Delaunay triangulation can be considered as a graph composed of a set of nodes and a set of vertices, and at the same time, as a simplicial complex composed of a set of points, line segments, and triangles.

To the best of our knowledge, the work presented in [START_REF] Singh | Topological descriptors of histology images[END_REF] is, to date, the only study that uses persistent homology to extract topological information from graphs in histopathological images. Some other studies that use the concept of homology in histopathology image analysis were also developed recently [START_REF] Nakane | Homology-based method for detecting regions of interest in colonic digital images[END_REF][START_REF] Nakane | A simple mathematical model utilizing topological invariants for automatic detection of tumor areas in digital tissue images[END_REF], but they do not use the notion of graph in their application. In fact, in these works, persistent homology is employed to study the topology of binary objects in the image, rather than graph nodes. However, the authors have announced that using the concept of the Betti numbers (homology), it is possible to evaluate quantitatively the contact degree between two histological objects.

Mathematical morphology

One of the first studies dedicated to the investigation of architectural characteristics in histopathology was based exclusively on mathematical morphology on graphs [START_REF] Raymond | Germinal center analysis with the tools of mathematical morphology on graphs[END_REF]. The method was developed in a framework demonstrating the practical usability of a new technique that has emerged just a few years earlier that applies mathematical morphology operations on graphs [START_REF] Vincent | Graphs and Mathematical Morphology[END_REF][START_REF] Vincent | Mathematical Morphology on Graphs[END_REF]. In [START_REF] Raymond | Germinal center analysis with the tools of mathematical morphology on graphs[END_REF], Gabriel graph was constructed on points representing cell nuclei of different types. Using a series of morphological opening operations on the graph, Granulometry, aggregates of small and large cells (benign and malignant cells) were detected. The size and the number of the aggregates were used as morphological features, together with other features derived from the distance transforms calculated from subgraphs of different cell types. According to the authors, the approach is efficient to differentiate between follicular lymphoma and follicular hyperplasia tissue images. However, only 29 images, of size ranging between 512 × 512 and 1024 × 1024, were used in the study, and the identification of cells was achieved by manually labeling on a video-screen.

In practice, this approach requires a large number of nodes to show its usefulness. Due to the lack of adequate image processing techniques and limited computational facilities, allowing the detection and classification of large number of histological structures, the approach had to be left behind. However, recently, a formalism for spatial relation modeling of unorganized point sets using mathematical morphology on graphs, with an application to histopathological images, has been introduced in [START_REF] Lomenie | Visual point set processing with lattice structures: Application to parsimonious representations of digital histopathology images[END_REF][START_REF] Lomenie | Point set morphological filtering and semantic spatial configuration modeling: Application to microscopic image and bio-structure analysis[END_REF]. The morphological operators proposed in the approach are slightly different from the original ones proposed in [START_REF] Raymond | Germinal center analysis with the tools of mathematical morphology on graphs[END_REF], as they perform on 2D simplicial complexes (i.e. triangular faces of Delaunay triangulation [START_REF] Loménie | Morphological operations on Delaunay triangulations[END_REF]) rather than graph vertices. The authors have suggested that the approach is convenient for structural representations of large image data by the means of interest point sets and their morphological analysis.

Summary

Graph-based methods have been employed in several applications in histopathological image analysis in order to study the spatial distribution of different histological objects (e.g. cell nuclei, glandular structures, tissue components) in various pathological tissues (e.g. breast, colon, prostate, lung, brain). These methods have explored a wide range of graphs defining different relationships and linking rules between histological objects (e.g. Voronoi diagram, Delaunay triangulation, Gabriel graph, minimum spanning tree) by extracting features using mainly three different techniques (i.e. syntactic structure analysis, network analysis and spectral analysis). Two other methods were explored only once each (i.e. persistent homology and mathematical morphology). In table 2.2 we give a summary of the state-of-the-art graph-based methods by indicating the histological objects studied (Vertices), the tissue organ to which they are belonging and the stain used, and the graphs applied. [START_REF] Sudbo | New algorithms based on the Voronoi Diagram applied in a pilot study on normal mucosa and carcinomas[END_REF] -cyclomatic number -eigen exponent . roundness factor [START_REF] Marcelpoil | Methods for the study of cellular sociology: Voronoi diagrams and parametrization of the spatial relationship[END_REF] -persistence -energy . compacity [START_REF] Sudbo | New algorithms based on the Voronoi Diagram applied in a pilot study on normal mucosa and carcinomas[END_REF] -girth -number of the eigenvalues . polygonal form [START_REF] Weyn | Determination of tumour prognosis based on angiogenesis-related vascular patterns measured by fractal and syntactic structure analysis[END_REF] -expansion -lower slope . polygonal shape: ratio of the -distortion -nbr. of eigenvalues equal to 1 minimal to the maximal -joint degree distribution -nbr. of eigenvalues equal to 2 side length [START_REF] Weyn | Determination of tumour prognosis based on angiogenesis-related vascular patterns measured by fractal and syntactic structure analysis[END_REF] -assortativity -upper-slope -reliability -trace -Delaunay edge probability -coreness -eigenmode perimeter disorder [START_REF] Sudbo | New algorithms based on the Voronoi Diagram applied in a pilot study on normal mucosa and carcinomas[END_REF] -chromatic number -eigenmode volume -average Delaunay edge -core number probability [START_REF] Sudbo | New algorithms based on the Voronoi Diagram applied in a pilot study on normal mucosa and carcinomas[END_REF] -clique number Persistent homology -divergence from the regular -isoperimetric number tree [START_REF] Sudbo | New algorithms based on the Voronoi Diagram applied in a pilot study on normal mucosa and carcinomas[END_REF] -crossing number -Betti numbers -radius of the maximum -Colin de Verdière's number percolating ball [START_REF] Sudbo | New algorithms based on the Voronoi Diagram applied in a pilot study on normal mucosa and carcinomas[END_REF] -slope number Mathematical Morphology -weighted compacity [START_REF] Sudbo | New algorithms based on the Voronoi Diagram applied in a pilot study on normal mucosa and carcinomas[END_REF] -vertex connectivity -polygon density:

Discussion and Conclusion

Remarkable efforts have been made to investigate new methods to automate the process of analyzing digital histopathological images using a wide range of analysis techniques (see figure 2.12). Given the need for exploring new tissue-derived parameters, image analysis techniques have been applied to provide quantitative tissue metrics exposing different aspects that are impossible to carry out visually. Graph-based methods represent a significant part of these techniques, and they have been employed mainly within frameworks for the characterization of tissue architectures.

In this chapter, we have presented a comprehensive account of the various graph-based methods developed for spatial analysis in histopathological images. We have discussed these methods starting from the histological objects whose spatial distribution is studied, to the methodologies used in order to extract descriptive features from the graphs. Which has not been the case in the review presented in [START_REF] Sharma | A review of graph-based methods for image analysis in digital histopathology[END_REF], where the discussion has focused on the graph type (e.g. Voronoi, Delaunay, Gabriel. . . ), and the paradigm shift; from the "constrained" methods based on proximity graphs to more "flexible" application-specific graphs (e.g. cell graphs and attributed relational graphs) was emphasized. Moreover, in this chapter we provide a summary table of these studies by indicating their key elements and categories 2.2, and another table where a large list of graph-based features is available 2.1, which could be informative for researchers in the field. To the best of our knowledge, the graph-based methods presented in table 2.2 have been applied locally on individual regions of interests of tissue, where image frames are extracted from whole slide images (WSI) of tissue, which does not provide a global characterization of the tissue architecture. This is due to two factors: i) state-of-the-art methods proposed to date involve too high computational cost to be applied on large whole slide images, ii) lack of efficient image processing tools for automatic detection, segmentation and classification of histological structures (e.g. nuclei/glands) applicable to large WSIs. In fact, in most of the studies, the histological objects have been interactively marked (labeled) to identify nodes, which is manually unfeasible on a large WSI.

The Digital Pathology community has expressed the need for developing analysis techniques applicable to whole slide images and not only to some specific regions [START_REF] Belsare | Histopathological image analysis using image processing techniques: An overview[END_REF][START_REF] Stokes | Pathology imaging informatics for quantitative analysis of whole-slide images[END_REF][START_REF] Buhmann | Computational pathology: Challenges and promises for tissue analysis[END_REF][START_REF] Sharma | A review of graph-based methods for image analysis in digital histopathology[END_REF]. Hence, there is a need for developing efficient and completely automated algorithms for the recognition of the different tissue components in histopathological images, and for developing more optimized methods of quantitative analysis of tissues. The authors of [START_REF] Sharma | A review of graph-based methods for image analysis in digital histopathology[END_REF] have suggested that the current graph-based algorithms should be modified for analysis of WSIs by using advanced parallel processing techniques. In our view, there is a need for exploring new approaches in the extraction of parameters from graphs to derive information about the architecture of tissues, and that the new approaches need to take into account the heterotypic interactions between the histological structures of different kinds, which actually constitutes the main objectives of this thesis.

A wide range of techniques have been used in histopathological image analysis. From these techniques, graph-based methods have proved their ability as potentially very powerful tools for characterizing tissue architectures, and need to be explored further. From the graph analysis tools explored in this direction, mathematical morphology has shown its suitability to the problem, and can constitute a distinctive set of methods to study graphs. In next chapter, we propose a new approach for quantitative evaluation of the spatial arrangements of point-sets based on mathematical morphology on graphs.

Chapter 3

Spatial Point Pattern Analysis using Graph-based Mathematical Morphology

Introduction

In 1988, Luc Vincent [START_REF] Vincent | Mathematical Morphology on Graphs[END_REF] proposed an original approach to study neighborhood relationships within populations of objects. The idea was to process the graph via morphological transformation in order to extract useful information about the organization of objects. These morphological operations are derived from mathematical morphology (MM) [START_REF] Serra | Image Analysis and Mathematical Morphology[END_REF], which is originally a theory and technique for the analysis of geometrical structures, commonly applied for morphological image processing. MM consists of a set of operators that transform images according to topological concepts based on a geometric object called structuring element. MM was originally developed to be applied to binary images and was later extended to the case of grayscale and color images, and it has been successfully used in various applications [START_REF]Mathematical Morphology and Its Applications to Image Processing[END_REF]. MM offers a wide range of advantageous techniques employed in image processing based on a few simple mathematical operators. Common usages of these operations, include noise removal, edge detection, image enhancement and image segmentation. Later, the theory has been extended to the more general case of graphs, where the morphological operators perform on the graph nodes rather than the image pixels [START_REF] Vincent | Graphs and Mathematical Morphology[END_REF][START_REF] Vincent | Mathematical Morphology on Graphs[END_REF]. This theory provides a great number of powerful tools for studying graphs that can be defined on a given set of objects depending on a desirable neighborhood relationship.

An overview of the advantages of mathematical morphology on graphs are given in [START_REF] Laurent | A graph-based mathematical morphology reader[END_REF]. Example of different applications were also presented in [START_REF] Stawiaski | Mathematical Morphology and Graphs : Application to Interactive Medical Image Segmentation[END_REF][START_REF] Heijmans | Graph morphology in image analysis[END_REF][START_REF] Vincent | Mathematical Morphology for Graphs Applied to Image Description and Segmentation[END_REF]. In the mentionned studies, graph-based MM was proposed mainly as a tool for image segmentation, image description or image filtering, where the graph nodes represent the image pixels, offering in that way spatial information, additional to their intensity. The graphs obtained with pixel adjacency relations are usually regular, as these pixels are often structured in grid fashion, i.g. 4-or 8-adjacency. It may also be very interesting to use MM to describe the spatial organization of objects irrugulary distibuted, derived from image components. In fact, graph-based morphological operations have already been successfully used in histopathology image analysis to study neighborhood relationships between cells of germinal centers from lymph nodes [START_REF] Raymond | Germinal center analysis with the tools of mathematical morphology on graphs[END_REF]. The same techniques can be employed in diverse problems where quantitative description of spatial relationships between objects is investigated [START_REF] Barrera | Morphological operators characterized by neighborhood graphs[END_REF][START_REF] Heijmans | Graph morphology in image analysis[END_REF].

Continuing these efforts in integrating graph-based mathematical morphology in spatial analysis and modeling, we have developed a new framework that will help us to evaluate the spatial arrangement of points based on features derived from two morphological functions that we have established during our study. These functions are composed of combinations of basic morphological operations, and they have shown the ability to describe how a point set is distributed and how distinct point sets are located with respect to each other. The formalism makes it possible to handle concepts like "clutering", "dispersion", "proximity", "betweenness", and "enclosness" over spatial point sets. Such issues may be, and have already been, studied with spatial statistics [START_REF] Gelfand | Handbook of spatial statistics[END_REF]209,[START_REF] Fortin | Spatial analysis: a guide for ecologists[END_REF][START_REF]Spatial Statistics and Digital Image Analysis[END_REF][START_REF] Paez | Spatial statistics for urban analysis: a review of techniques with examples[END_REF], spatial pointpattern analysis [START_REF] Wiegand | Handbook of spatial point pattern analysis in ecology[END_REF][START_REF] Upton | Spatial data analysis by example. Volume 1: Point pattern and quantitative data[END_REF][START_REF] Illian | Statistical analysis and modelling of spatial point patterns[END_REF][START_REF] Diggle | Statistical analysis of spatial and spatio-temporal point patterns[END_REF][START_REF] Gatrell | Spatial point pattern analysis and its application in geographical epidemiology[END_REF][START_REF] Li | Comparison of point pattern analysis methods for classifying the spatial distributions of spruce-fir stands in the north-east USA[END_REF], and fuzzy set theory [START_REF] James | Learning spatial relationships in computer vision[END_REF][START_REF] Bloch | Fuzzy relative position between objects in images: a morphological approach[END_REF][START_REF] Bloch | Fuzzy Relative Position Between Objects in Image Processing: A Morphological Approach[END_REF][START_REF] Bloch | Fuzzy spatial relationships for image processing and interpretation: a review[END_REF]. Each approach has its advantages and drawbacks, and works well for particular applications. However, they are not very suitable when the phenomenon under study require analysis based on the neighborhood relationships (links) existing between the spatial entities. Therefore, the modeling of these relationships by a graph and studying them using mathematical morphology seems to be plausible.

This chapter provides background and context on the modeling of the spatial configuration of objects using mathematical morphology on graphs. In the following, we first introduce a short description of the basic notions of graph-based mathematical morphology on graphs. The main morphological operations are presented, as well as two new morphological functions that we have additionally developed in this study, i.e. openness and closeness functions. Then, we present a new approach that uses these morphological functions in order to study the spatial distribution of point-sets irregularly distributed within other point-sets, and the spatial interactions/relations existing between them.

Mathematical Morphology Operations on Graphs

Overview

It has been demonstrated by Luc Vincent [START_REF] Vincent | Mathematical Morphology on Graphs[END_REF] that it is possible to process graphs by Mathematical Morphology and he proposed a series of morphological operators that perform on graphs, and he described them in details in [START_REF] Vincent | Graphs and Mathematical Morphology[END_REF]. Thereafter, graph morphology has became part of the research interests of several authors, and it has been defined in many different ways. Although, all definitions are actually adopting the same reasoning.

At the beginning, in [START_REF] Vincent | Graphs and Mathematical Morphology[END_REF][START_REF] Vincent | Mathematical Morphology on Graphs[END_REF], the morphological operators were introduced as operations applied to vertices. Where, the numerical value of a vertex varies depending on the numerical values of the vertices in its neighborhood. Then, the authors of [START_REF] Cousty | Some Morphological Operators in Graph Spaces[END_REF] proposed to apply the morphological operations on edges rather than vertices. In this case, numerical values are associated to the graph edges rather than vertices, and they are transformed based on the edge-adjacencies. However, this approach is correlative to the previous one when we apply the morphological operators proposed in [START_REF] Vincent | Mathematical Morphology on Graphs[END_REF] to the line-graph of the graph (The line-graph of a graph is another graph where each vertex represents an edge and two vertices are adjacent if their corresponding edges share a common endpoint (refer to section 2.2.7). Later, it was proposed to apply morphological operations on 2D simplicial complexes [START_REF] Munkres | Simplicial Complexes and Simplicial Maps, New York: Perseus Books Pub[END_REF]. The authors of [START_REF] Loménie | Morphological operations on Delaunay triangulations[END_REF] proposed to apply the morphological operations on the triangular faces of Delaunay triangulation [START_REF] Loménie | Morphological operations on Delaunay triangulations[END_REF], where a numerical value is associated to each triangle and two triangles are considered to be adjacent if they share an edge. The numerical values of the triangles are then modified by the morphological operations. This approach is also correlative to the one proposed in [START_REF] Vincent | Mathematical Morphology on Graphs[END_REF] when we consider the face graph of the Delaunay triangulation (another graph that has a vertex for each triangular face, refer to section 2.2.7). In other works, it was proposed to perform morphological operations on hypergraphs [START_REF] Sebastian | Mathematical Morphology on Hypergraphs Using Vertex-Hyperedge Correspondence[END_REF][START_REF] Bloch | Mathematical morphology on hypergraphs, application to similarity and positive kernel[END_REF][START_REF] Bloch | Mathematical Morphology on Hypergraphs: Preliminary Definitions and Results[END_REF] rather than graphs (a hypergraph is a generalization of a graph in which an edge can join any number of vertices [START_REF] Sebastian | Mathematical Morphology on Hypergraphs Using Vertex-Hyperedge Correspondence[END_REF]).

In our study, we consider the morphological operations which perform on vertices [START_REF] Vincent | Mathematical Morphology on Graphs[END_REF], because at the end of the day, our goal is to characterize the spatial distribution of a pointset, based on measurements associated to nodes representing their locations. In this section, we first present the basic morphological operations as well as the morphological functions proposed in [START_REF] Vincent | Mathematical Morphology on Graphs[END_REF]. Then, we expose two new morphological functions that we propose for quantifying spatial properties of patterns in a spatial point set, such as clustering and relative positioning. In the following, only morphological operations applied to binary graphs, are illustrated, as they are more intuitive than decimal graphs, and more adequate to understand the concept of mathematical morphology on graphs. Given a binary or decimal graph G defined on (V, E), G(v) will stand for the value of G at vertex v (refer to section 2.2.2).

Algebraic operations

Before presenting the morphological operations that can be applied on graphs, we have first to introduce the algebraic operations that can be performed between two or more graphs.

Given two decimal graphs G 1 and G 2 , we define their addition, the graph

G 1 + G 2 , by ∀v ∈ V, (G 1 + G 2 )(v) = G 1 (v) + G 2 (v) (3.1)
The subtraction (-), multiplication ( * ) and division (/) of two decimal graphs are defined in the same way. These algebraic operations can be useful to design functions that combine different morphological operations (more details are given in the following).

Basic morphological operations

Dilation and erosion

Let G be a binary or decimal graph (on (V, E)), the dilated graph δ(G) and the eroded graph ε(G) are defined respectively by:

∀v ∈ V δ(G)(v) = max{G(u), u ∈ N E (v) ∪ {v}} (3.2)
and

ε(G)(v) = min{G(u), u ∈ N E (v) ∪ {v}} (3.3)
where N E (v) is the set of all vertices adjacent to vertex v (refer to section 2.2.3).

The operations of dilation and erosion are dual and increasing. Dilation is extensive, while erosion is anti-extensive. These two fundamental operations of mathematical morphology serve as a basis for great number of more elaborated transforms. They are easily implemented using a sequential scanning of the vertices of V . These morphological operations imply "structural" changes in the numerical values of a point set, which results in transformations in the point patterns given by the numerical values of the vertices (foreground nodes in this case). These changes depend on the connectedness of the graph adopted. For example, in the case of a binary graph, the more connections the graph contains, the more the dilation propagates foreground vertices and the more the erosion affects foreground vertices, and vice versa.

In order to include larger neighborhood in the process, we can define morphological operations of size n. The dilation and erosion of size n of a graph are defined as follows:

∀v ∈ V δ n (G)(v) = max{G(u), d E (u, v) ≤ n} (3.4) and ε n (G)(v) = min{G(u), d E (u, v) ≤ n} (3.5)
where d E (u, v) is the geodesic distance between vertices u and v.

The dilation and erosion of size n of a graph G can be easily computed by performing n successive dilations or n successive erosions of G.

δ n (G) = δ • δ • • • • • δ(G), n times (3.6) and ε n (G) = ε • ε • • • • • ε(G), n times (3.7)
The size n of a dilation or erosion allows us to consider a small or large neighborhood when processing vertices. The reasoning is equivalent to the size of a structuring element in classical mathematical morphology [START_REF] Serra | Image Analysis and Mathematical Morphology[END_REF], e.g. line, disk, square, etc. Roughly speaking, the set of vertices defining the neighborhood plays the role of the image pixels defining the structuring element in classical morphology, whereas the set of edges E defines the neighborhood relationships rather than the image grid. In this context, it should be noted that in the same way as it is done in classical morphology, we can think about performing dilations and erosions on graphs based on other neighborhood definitions. For example, for a pointset in the 2D space, we can define the anisotropic dilation of size n and angle θ, where the neighborhood of a vertex is defined by the adjacent vertices only located at an angle θ, by:

∀v ∈ V, δ (θ) n (G)(v) = max{G(u), d E (u, v) ≤ n, ( uv, θ) < } (3.8)
with a small angle that regulates the precision of the orientation. For a decimal graph, we can define the parameterized dilation δ (h) , h ∈ N by:

∀v ∈ V, δ (h) (G)(v) = max(max{G(u), u ∈ N E (v)}, G(v) + h) (3.9)
Additional neighborhood functions relating individual vertices using small structures, called structuring graphs, were also proposed in [START_REF] Heijmans | Graph Morphology[END_REF].

The operators of dilation and erosion are already useful when they are employed by themselves, but they become much more interesting when they are combined together. And that's what we will present in the following.

Closing and opening

Given a binary or decimal graph G, the closed graph ϕ(G) and the open graph γ(G) are defined as follows:

ϕ(G) = ε • δ(G) and γ(G) = δ • ε(G) (3.10)
The closing and the opening of size n are defined in the same way:

ϕ n (G) = ε n • δ n (G) and γ n (G) = δ n • ε n (G) (3.11)
The closing and opening have two similar properties: they are increasing and idempotent. The closing transformation is extensive whereas the opening is anti-extensive. The application of a binary closing and opening on a graph can result in a spatial reorganization of the patterns given by the foreground vertices, i.e. G(v) = 1. For example, closing can be used to join connected components and fill holes. A hole in this context is a set of background vertices that cannot be reached from other background vertices of V . Whereas, opening can be useful to separate connected components and detect foreground vertices that are forming significant aggregates. Closing and opening transformations are at the basis of many morphological filtering operations, such as the Alternating Sequential Filters that we will present in the next section.

Compound morphological operations

The basic morphological transforms presented above are very convenient to conceive interesting functions by using algebraic combinations of them. This section presents some morphological functions introduced in [START_REF] Vincent | Mathematical Morphology on Graphs[END_REF] and two morphological functions that we have established in this study.

Alternating Sequential Filter

The increasing and idempotency properties of the closing and opening transforms allow us to study a class of operations that have the same characteristics, called morphological filters. The theory of morphological filters is very large and complex, here, we limit ourselves to show few examples of elementary filters.

We can introduce the alternating sequential filter (ASF) as the composition of closing and opening operations of increasing size, forming a granulometry or an anti-granulometry. For example, the alternating sequential filter beginning with a closing operation can be defined by:

Φ n = γ n • ϕ n • γ n-1 • ϕ n-1 • • • • γ 1 • ϕ 1 (3.12)
Similarly, we can define the alternating sequential filter beginning with an opening operation by: Alternate sequential filters can serve to reduce "noise" originated from incorrect numerical values associated to the graph nodes, facilitating this way a subsequent analysis. For example, the noise induced from the identification of nuclei in a histopathology image, due to the errors generated from nuclei classification algorithms. In such application, ASF can also be useful to detect significant clusters and hotspots of cells of the same type, while rejecting those who are dispersed (less significant).

Ψ n = ϕ n • γ n • ϕ n-1 • γ n-1 • • • • ϕ 1 • γ 1 (3.

Geodesic Reconstruction

We call geodesic operator any morphological transformation, which applies to a binary or decimal graph, that is no longer defined on (V, E), but on a subgraph of (V, E). We can define the geodesic dilation of a binary graph as a conditional dilation, where the result of the dilation of the graph is combined with one of its subgraphs using a logical operation. Let G and G be two binary graphs on (V, E), such that the set of foreground vertices of G is a subset of the set of the foreground vertices of G.

Formally, {v ∈ V |G (v) = 1} ⊆ {v ∈ V |G(v) = 1}.
Here, G is admitted to be a subgraph of G in the sense of the numerical values associated to the vertices. The geodesic dilation of the graph G given G can be defined by:

δ |G (G ) = δ(G ) * G (3.14)
The principle of geodesic reconstruction is to repeat conditional dilations until idempotence, i.e. until no more modification occurs. The graph R G (G) called geodesic reconstruction of G from G is given by:

R G (G) = δ k |G (G ) = δ |G • δ |G • • • • • δ |G (G ), k times (3.15) with δ k |G (G ) = δ k |G (G ), ∀k ≥ k.

Distance function

The distance function of a binary graph G is the decimal graph D(G), where every foreground vertex, i.e. G(v) = 1, is parameterized by a numerical value equal to its distance from the nearest background vertex. The distance function D(G) is defined by:

∀v ∈ V, D(G)(v) = min{d E (v, u), G(u) = 0} (3.16)
Analytically, the distance function can be easily calculated from the operation of erosion as follows: The distance function of a binary graph has positive values at foreground vertices, it decreases in value as a foreground vertex approaches a background vertex, and it takes zero values at background vertices. The distance function can be useful to determine the relative distance between subsets of points. Furthermore, the distance function is practical for computing the skeleton of a binary graph very quickly.

D(G) = k n=0 ε n (G) (3.17) with ε k (G) = ε k (G), ∀k ≥ k.

Skeleton

The skeleton S(G) of a binary graph G is a binary graph, such that v is a positive vertex of the skeleton iff vertex v is the center of a maximal ball included in G. That is, S(G)(v) = 1 iff D(G)(v) is a local maximum of the distance function D(G). Formally:

S(G)(v) = 1 ⇐⇒ D(G)(v) ≥ D(G)(u), ∀u ∈ N E (v) (3.18)
The local maxima of the distance function D(G) can be easily determined using basic morphological operators as follows: This skeleton does not preserve the connectivity of the original graph G (see Fig 3.8). The skeleton provides a simple and compact representation of a pattern by preserving some of its topological characteristics.

S(G) = k n=0 ε n (G) -γ • ε n (G) (3.19) with ε k (G) = ε k (G),

Openness function

We define the openness function of a binary graph G by the decimal graph Ω(G), where every vertex v is parameterized by the value of the minimal size of the opening operation that produces G(v) = 0. Hence, each foreground vertex is parametrized by a value proportional to the size of the opening operation that transforms it to a background vertex. While, background vertices will keep a numerical value equal to zero. Formally, the openness function Ω(G) can be defined by:

∀v ∈ V, Ω(G)(v) = min{n ∈ N| γ n (G)(v) = 0} (3.21)
The openness function can be easily calculated from opening operations as follows:

Ω(G) = k n=0 γ n (G) (3.22) with γ k (G) = γ k (G), ∀k ≥ k
The openness function computes in every foreground vertex v of a binary graph, the size of the largest pattern (a subset of vertices of G composed of foreground vertices) in which v is included. An example is shown in figure 3.9. The values of vertices of Ω(G) increase with the consistency of the pattern in which they are included. One of the main advantages of the openness function comparing to the foreground distance function is that it provides groups of vertices that are adjacent and having the same openness value, which is convenient in dealing with segmentation problems based on topological properties. In fact, this function can be used to quantify the expanse and consistency of a subset of points relative to another and employed in segmentation of aggregates of objects based on their degree of clustering or to estimate the density in a particular domain of a graph.

Closeness function

We define the closeness function of a binary graph G by the decimal graph ζ(G), where every vertex v is parameterized by a numerical value expressing the minimal size of the closing operation that produces G(v) = 1. Each background vertex is then parametrized by a numerical value inversely proportional to the size of the closing that transforms it to a foreground vertex. While, foreground vertices will always have a numerical value equal to the maximal value of the function, which is the value of idempotency of the closing operation. The closeness function ζ(G) can be defined by:

∀v ∈ V, ζ(G)(v) = k -min{n ∈ N| ϕ n (G)(v) > 0} (3.23) with ϕ k (G) = ϕ k (G), ∀k ≥ k
The closeness function can be easily calculated from cumulative sum of closing operations as follows:

ζ(G) = k n=0 ϕ n (G) (3.24) with ϕ k (G) = ϕ k (G), ∀k ≥ k
This transform computes in every background vertex v of a binary graph, the degree to which the area that includes u is enclosed by foreground vertices. An example is shown in figure 3 The closeness function is useful to quantify the sparsity and the dispersion of a set of points (e.g. foreground) relative to another point set (e.g. background), by studying the properties extracted from the second one (e.g. background). One of the main advantages of the closeness function comparing to the background distance function (refer to section 3.2.4) is that it provides groups of background vertices that are adjacent and having the same closeness value (local homogeneity), which is also convenient in dealing with segmentation problems based on topological properties.

Overview

Data in the form of a set of points, irregularly distributed within a region of the space, arise in many different contexts, including natural phenomena studied in different scientific fields such as astronomy, geographical information science (GIS), or histopathology (refer to section 1.4). In such fields, the observed objects can be assimilated to spatial points and their locations can be considered as events that have been generated by random mechanisms [START_REF] Wiegand | Handbook of spatial point pattern analysis in ecology[END_REF]. The variations of these mechanisms, together with environmental factors, may create patches (patterns) of spatial points having common properties.

For example, the invasion of tumor cells in an organ is influenced by metabolic changes in the tumor microenvironment, composed of immune cells, blood vessels, adipose tissue (fat), and supportive tissue (collagen) (see figure 3.11(a-b)), which lead to the emergence of tumor structures having different spatial patterns (refer to section 1.2.3). Furthermore, studying the spatial organization of tumor cells relative to their microenvironment components can reveal useful information about biological interactions that may have a clinical implication [START_REF] Nawaz | Computational pathology: Exploring the spatial dimension of tumor ecology[END_REF].

In a forest stand, spatial patterns are important indicators on a variety of physical and ecological processes. The variations in the micro-environment create patches of trees of similar type and size (see figure 3.11(c-e)). These patches have a strong influence on the properties of eco system function. Thus, the quantification of the spatial patterns in a forest stand has a decisive impact on future stand development and dynamics [START_REF] Li | Comparison of point pattern analysis methods for classifying the spatial distributions of spruce-fir stands in the north-east USA[END_REF].

The gravitational clustering process of cosmic structures in the Universe leads to the emergence of complex patterns (see figure 3.11(f-g)). Galaxies are organized into groups and clusters, often within larger superclusters, that are strung along great filaments between nearly empty voids, forming a cosmic web-like patterns, arising from gravitational instability. The characterization of the arrangements of cosmic spatial patterns may lead to new interpretations and better understanding of the Universe [START_REF] Lee | The life of the cosmos[END_REF].

Quantitative spatial analysis of such data include the investigation of three main spatial aspects: i) localization of spatial point patterns, ii) evaluation of their spatial distribution, and iii) evaluation of the spatial interactions between distinct patterns. Researchers have developed many methods of spatial point pattern analysis. These methods can be classified into different categories based on various criteria. The most commonly known methods use statistical approaches such as kernel density estimators, spatial regression models, quadrat methods, Ripley's K function, pair correlation function [START_REF] Gelfand | Handbook of spatial statistics[END_REF]. While, other methods focus on graph-based approaches using structural, topological or spectral analysis [START_REF] Wallis | A beginner's guide to graph theory[END_REF][START_REF] Andries | Spectra of Graphs[END_REF]. In this framework, we propose a new approach based on the morphology of graphs. We use the openness and closeness functions that we have defined in the last section, in order to study the spatial distribution of point-sets irregularly distributed within other point-sets, and the spatial interactions/relations existing between the them. f) A view of thousands of distant galaxies, in a patch of sky called the Lockman Hole [START_REF]HerMES: The Lockman Hole[END_REF]. Almost every point of light is an entire galaxy. The colours represent the far-infrared wavelengths measured by Herschel [START_REF] Hasinger | A deep x-ray survey in the lockman-hole and the soft x-ray N-Log[END_REF]. g) Classification of points of significant galaxies: red, blue, and green points are associated to orangish, yellowish, and green galaxies, respectively.

Formalism

Figure 3.12 presents an example of three distinct sets of points, defined by their colors, irregularly distributed in the same spatial domain and forming spatial patterns, similar to what can be observed in some situations (see applications in figure 3.11). The example that we have chosen presents various spatial aspects. On the one hand, it shows groups of points (also called aggregates or "clusters") having different shapes and sizes, that we will call point patterns in what follows. On the other hand, it reveals different configurations of relative spatial positioning of these patterns. In the following, we denote the set of red points by A, blue points by B, and green ones by C. We denote by S the set of all the points of including the three sets together, S = A ∪ B ∪ C. These notions will allow us to describe the spatial aspects presented in this example based on visual interpretations, which include various configurations including: i. Pattern (A7) is very large and consistent comparing to the other red patterns, like (A1), (A2) or (A3) which are small and thin.

ii. Pattern (A5) is small, and enclosed (surrounded) by the large pattern (B3).

iii. Pattern (A2), respectively (A4), is located between two blue patterns (B1) and (B2), respectively (B1) and (B3). It should be noted that from a practical point of view, using the term "pattern" is more efficient than using the term "connected component" in this context. For example, the patterns (A7) and (A8) belong both to the same connected components in the Delaunay graph (zoom on figure 3.13), but it is more reasonable to consider two subsets of points forming distinct clusters. Therefore, we call point pattern any subset of points belonging exclusively to one of the sets A or B. The aim of our study is to extract features describing these spatial aspects using mathematical morphology on graphs. Therefore, the Delaunay graph is first reconstructed on the set S of all points. Here, we have chosen the Delaunay triangulation because its neighborhood conditions ensure good connectedness between neighbor nodes. It should be noted that from a theoretical point of view, the same study can be established using other proximity graphs (refer to section 2.2.5). In the following, we will expose our approach through illustrations showing the morphological operations applied to the Delaunay graph defined on the point-set S.

First, we will focus on the analysis of the spatial distribution of the point-subset A, regardless the interactions with the other point-subsets (i.e. B, C). This analysis is centered around two main questions: (i) How the nodes of A are clustered (expanse) within the patterns of A. (ii) How the point patterns of A are distributed relative to each other (dispersion/sparseness).

Then, we will focus on the evaluation of the spatial positioning of patterns of A relative to patterns of B. Here also two main questions to be taken into account: (i) Are patterns of A, enclosed in, between, or free from, patterns of B. (ii) Are patterns of A close or far from patterns of B.

In the following, we denote by G the Delaunay graph defined on the point-set S, and we assume that The adoption of the openness function in the analysis of spatial point patterns gives access to different practices in this context. In fact, the values of Ω can be used as features or descriptors attributed to the graph nodes, allowing detection of aggregates, segmentation and classification of point patterns based on a new morphological criteria. However, this criteria, solely, is not sufficient to describe the spatial arrangement of point patterns, since it does not provide information about how the patterns are distributed relative to each other. To answer this question, we introduce another criteria, also based on mathematical morphology transforms, that we explain in details in the next paragraph.

G is labeled such that, G(v ∈ A) = 1, G(v ∈ B) = 2, and G(v ∈ C) = 3. We define the binary graph G A such that G A (v) = 1 iff v ∈ A,

Sparsity

In order to extract information that characterizes how point patterns of

Spatial relations between two subsets of points

Until now, we have discussed only the spatial characteristics of the point patterns defined by the set subset of points A. It may also be very interesting to examine the spatial relations existing between the point patterns of A and other point patterns, such as patterns formed by the distribution of B.

Encloseness

As explained in section 3. This technique may introduce a slight ambiguity when dealing with large patterns, as the pattern (A7), which include a wide range of values of the function D(G B ). To circumvent this problem, we can consider only the minimum value of D(G B ) in each pattern of A, to determine whether the pattern is close to or far from B, as the minimum value represents the distance of the closest node in that pattern to B.

Discussion

In this framework, we have proposed spatial features that can be useful to describe pointsubsets forming spatial patterns and irregularly distributed within other point-subsets. These features include: 1) clustering, 2) dispersion or sparsity (which also evaluates the proximity or closeness of the point-subset to itself), 3) the encloseness by another point-subset, and 4) the distance from another point-subset. These spatial aspects are summarized in table 3.1 for each of the patterns of the set of points A, based on the values given in figures 3.13, 3.16.d, 3.18, and 3.20. We use the symbols +/-to label the spatial characteristic queried for each pattern. The proposed morphological functions, openness and closeness, have shown good discriminative capacities about the spatial aspects discussed in this section, by offering features able to distinguish between different spatial configurations of a point-set. It should be noted that the numerical values obtained by the proposed morphological functions may vary widely from one application to another and they may be contingent on the neighborhood relationships defined by the graph adopted in the application.

Conclusion and perspectives

Conclusion and perspectives

In this chapter, we have exposed the basic morphological operations performing on graphs (erosion and dilation, opening and closing), and few composite morphological operations such as geodesic reconstruction, distance function, and alternate sequential filter. Then, we have defined two new composite morphological operations, namely the openness function and the closeness function, and we have studied their properties. We have presented a consistent methodology for studying populations of objects, modeled by a set of points, based on few morphological functions applied to the Delaunay graph. These functions offer relevant spatial features that describe efficiently the spatial arrangements of point-sets independently from any parameter. This framework allows to introduce Mathematical Morphology on graphs in spatial point pattern analysis. The methodology would find its effective usefulness in a wide range of the practical problems.

There is a large variety of graphs that can be defined on a given set of objects. Future works should include comparative analysis between different graphs in order to investigate the variation of the morphological features discussed in this framework depending on the connectedness of the graph. Such analysis would give deeper knowledge about the usefulness of the proposed morphological functions and their adaptation to practical problems. The morphological features should also be compared to features used in other approaches, such as spatial statistics. In addition, in the majority of the applications related to this problem, the initial objects are obtained using an automatic segmentation algorithm, which may introduce errors. The robustness of this approach against noise should be assessed. The noise in this context is defined as nodes that were incorrectly labeled, such as in the case of nuclei classification in histopathology. Such studies can be conducted on simulated datasets, involving different spatial distributions of point-sets. Finally, it has to be mentioned that the proposed methodology is completely adaptable to the study of the spatial arrangements of objects in 2D (or 3D) images using classical mathematical morphology, when the image pixels are defined as nodes.

Chapter 4

A Method for Tissue Segmentation and Node Identification

Introduction

Quantitative architectural analysis in histopathology images may concern the study of the spatial organization of different histological structures, perceived at different tissue levels. Most of the graph-based methods have focused on quantifying the spatial distribution of primitive biological structures like cell nuclei, or composite structures like glands. Our study is dedicated to quantitative analysis of the spatial arrangements of tissue structures of different categories, including simple and complex biological structures: cell nuclei, collagen, and adipose tissue, using graph based mathematical morphology.

For such a study, a proximity graph can be defined on a set of nodes, where each node represent partially or entirely a histological component; A cell nucleus can be identified entirely by a node at its geometric centroid, while collagen and adipose tissue can be first segmented into small fragments, then the centroids of these fragments can be used to identify partially the whole histological structure. However, it should be noted that in the case of adipose tissue modeling, each adipocyte can be entirely identified by a node at its centroid, but the segmentation of every single adipocyte in the tissue can be an arduous task when the thin boundaries of adipocytes are not well defined or when the image presents artifacts that alter their morphology. In addition, the dissimilarity in size between a nucleus and an adipocyte is considerable, which would imply unfair comparison between structures of different histological levels. Therefore, we promote the use of a partial node identification method for the adipose tissue.

Nuclei in the tissue can be of different cell types having different biological roles. It would be interesting to classify the cell nuclei in the tissue image according to their function, and treat each class as a separate entity. In this study, we consider three classes of cells, i.e. cancer cells, immune cells and fibroblasts, which relies on an accurate cell identification technique. This chapter presents our method for the identification of sets of nodes defining the different histological structures mentioned above. First, we give a short presentation of the dataset used in this framework and a brief overview of the method. Then, we give a detailed description of the proposed tissue segmentation and node extraction method.

Dataset

We conduct our experiments on a dataset consisting of 55 whole slide images (WSI) of tissue sections obtained from surgical resections of invasive breast carcinomas from 55 different patients, randomly chosen from the François Baclesse Cancer Center, Caen, France. The patients were treated between 1991 and 1995, without neoadjuvant therapy and with a follow-up of more than 15 years. The histological sections have been stained with the Hematoxylin-Eosin-Saffron (HES), a trichromatic staining obtained by adding an extra staining step with Saffron to the commonly used H&E staining procedure. Saffron accentuates collagen fibers of connective tissues with color ranges from yellow to orange depending on the stain concentration 1.1.2. However, in our dataset collagen fibers appear orangishreddish-colored, probably due to a combination with a high concentration of Eosin. Histological slides have been scanned with a high resolution slide scanner, ScanScope R CS from Aperio Technologies (20X NA 0.7 objective) to obtain virtual slides with a final resolution of 0.5µm/pixel. The obtained color (RGB) images have been saved in the tiled pyramidal TIFF file format using JPEG compression with a compression quality of 30.

For each virtual slide, a region of interest (ROI) that includes the invasive tumor region was drawn by the pathologist using the Aperio ImageScope R software. Regions within the ROIs containing noticeable artifacts, such as tears, voids, folds or stain spots [START_REF] Samar | Artifacts in Histopathology: A Potential Cause of Misinterpretation[END_REF][START_REF] Chatterjee | Artefacts in histopathology[END_REF], were also marked and removed from the ROI's masks in order to facilitate the further automatic image analysis. An example of WSI labeling using ImageScope R software is shown in figure 4.1. Each ROI is than extracted and saved in another tiled pyramidal TIFF file, where the tile size is set to 1024 × 1024 pixels ( 500 × 500 µm). For a lower computational cost, only tiles located within the ROI are processed.

Method overview

Our final objective in this study is to quantify the spatial organization of different tissue components that have distinct histological appearances and significances. These components include: three types of cell nuclei, i.e. cancer cells, immune cells, and fibroblasts; and two composite biological structures, which are collagen fibers and adipose tissues. In order to represent these histological objects by a point-set and study their spatial distribution using graphs, a prior node identification stage is needed. In our approach, cell nuclei are identified by their centroids. While, collagen fibers and adipose tissues are first decomposed into small fragments, then nodes are defined by the centroids of these fragments. The decomposition of the tissue is accomplished based on a superpixel segmentation technique. Then, the centroids of the superpixels associated to the tissue components are defined as nodes. To determine superpixels that belong to nuclei of a certain class of cells, a nuclei detection and classification algorithm is applied. Superpixels that belong to collagen and adipose tissues are determined using two different segmentation algorithms. In the following, we give a detailed description of the proposed node identification technique and the algorithms applied for detection, segmentation, and classification of the tissue components considered under this study.

Color normalization

Color variability in histopathological images is one the most common problems encountered when using automated image processing algorithms, due to a considerable divergence in the methods and the materials applied to provide virtual slides. In fact, stain preparation is dependent on concentration, chemical reactivity, storage conditions, and timing; image acquisition equipment are designed by different manufacturers; and light transmission is a function of the tissue section thickness. These inconstancies in color and illumination may reduce the performance of automated segmentation algorithms, as they directly affect color-based features. Beside the efforts being made toward the standardization of reagents and procedures in histological practice [START_REF] Lyon | Standardization of reagents and methods usid in cytological and histological practice with emphasis on dyes, stains and chromogenic reagents[END_REF], several works, as part of the image processing field, have focused on developing color normalization algorithms that adjust the color variations in virtual slides. Color normalization then became one of the first steps essential and commonly applied in histopathological image analysis, with the aim of getting efficient and stable algorithms. Different color normalization methods have been proposed in the literature [START_REF] Khan | A nonlinear mapping approach to stain normalisation in digital histopathology images using image-specific colour deconvolution[END_REF][START_REF] Macenko | A Method for Normalizing Histology Slides for Quantitative Analysis[END_REF][START_REF]Color transfer between images[END_REF]. Some of these methods are discussed in [START_REF] Magee | Colour normalisation in digital histopathology images[END_REF], and a comparative study is also given in [START_REF] Hoffman | Comparison of normalization algorithms for cross-batch color segmentation of histopathological images[END_REF].

For our application, we have adopted the method proposed by Reinhard & al [START_REF]Color transfer between images[END_REF]. This approach matches the mean and the standard deviation of each channel of an over/under stained image to that of a target image by means of a set of linear transforms in the Lab color space [START_REF] Ruderman | Statistics of cone responses to natural images[END_REF]. The process to normalize a source image based on the colors of a target image using this method is given by the following equation:

I (i) norm = (I (i) source -µ (i) source ) σ (i) target σ (i) source + µ (i) target , i ∈ {L, a, b} (4.1)
Where µ (i) and σ (i) are the mean and the standard deviation of Lab channel i, calculated over all the image pixels. An example of color normalization of an under stained image with a well stained image from our dataset is given in figure 4.3. The target image shown in figure 4.3.a was chosen as a reference image to normalize all the whole slide images of the dataset before beginning any processing. This image was chosen because it covers approximately the same amount of two important histological components in which we are interested, i.e. collagen and nuclei, and few whitish regions (background, lumen, etc). These characteristics provide extended RGB histograms, avoiding this way the saturation of certain image tiles that may not include one of the tissue components when normalization is applied. In addition, the tissue components shown in this image appear to be well stained and separable.

In fact, the RGB histograms in figure 4.3.d, show distinguishable color distributions allowing an easy separation of the two tissue components: the red and the green peaks, together, represent the orangish colors of collagen fibers, while the blue and the red peaks, together, represent the purplish colors of cell nuclei.

Superpixel segmentation

Superpixel segmentation is a technique that groups the image pixels based on their color similarity and proximity. Formally, it clusters the image pixels in the combined fivedimensional color and image plane space to generate a desired number of regular, compact, and nearly uniform image regions, called superpixels. In our approach, nodes representing the histological objects are defined by the geometric centers of these superpixels. Here, we are interested solely in the subset of superpixels that determine the selected histological objects. For this purpose, we apply the simple linear iterative clustering (SLIC) algorithm, proposed in [START_REF] Achanta | SLIC Superpixels[END_REF]. This algorithm performs a local clustering of pixels in the 5-D space defined by the L, a, b values of the CIE LAB color space [START_REF] Ruderman | Statistics of cone responses to natural images[END_REF] and the x, y pixel coordinates based on k-means clustering algorithm. SLIC algorithm produces better segmentations than other superpixel segmentation methods [START_REF] Achanta | SLIC Superpixels Compared to State-of-the-art Superpixel Methods[END_REF], as SLIC superpixels are more regular, compact, and tend to be more discriminative for histological structures. In addition, the good adherence to image boundaries exhibited by SLIC superpixels results in smoother and more accurate segmentations. Moreover, for the same image, SLIC algorithm gives access to various segmentations based on few input parameters, with a low computational cost. The input parameters are the number of desired superpixels and the compact factor. For our study, they are set to 3500 and 35, respectively. Experimentally, these values have shown an adequate decomposition of the image into significant tissue components, and especially, a good separation of cell nuclei. In fact, lower compact factor values imply that the superpixels may have very random shapes enclosing each other. While, greater values imply that superpixels have very regular shapes similar to the image grid, making a loss of the object boundary information. A low number of superpixels may result in under-segmentation where multiple nuclei may stand within the same superpixel. While, a large number may result in an over-segmentation where one nucleus may be divided to multiple superpixels. An example of SLIC segmentation algorithm on an image from our dataset is shown in figure 4

Cell nuclei identification 4.6.1 Nuclei detection Overview

The detection of cell nuclei in each image tile is established by selecting a subset of superpixels based on color information, thresholding and morphological operations. The flowchart given in figure 4.5 illustrates the different steps of our algorithm for nuclei detection. After color normalization, color deconvolution is applied to the normalized HES image in order to obtain the haematoxylin image that highlights cell nuclei. Then, the haematoxylin image is thresholded based on its intensity to give a first approximation of the locations of cell nuclei. Thereafter, a binary morphological opening is applied to the thresholded image in order to remove insignificant small objects and keep objects that better identify the locations of the nuclei. The remaining objects are then used as markers to perform a morphological reconstruction under the mask of the superpixel image. The optimal parameters were chosen based on fine tuning and optimization of performance criterions conducted on a set of annotated data (ground truth). The proposed nuclei detection algorithm, the choice of the parameters, and the quantitative evaluation method, are described in details in the following.

Color deconvolution

Color deconvolution is a technique of separation of histochemical staining by means of color image analysis. It provides intensity (grayscale) images representing the contribution from each stain. For our study, we have adopted the color deconvolution algorithm proposed by Ruifrok & al. [241]. This algorithm produces, from an (R, G, B) image of a H&E-DAB-stained tissue, three channels (H, E, DAB) corresponding to the contribution of haematoxylin (H), eosin (E), and Diaminobenzidine (DAB), in each image pixel, respectively. This is accomplished using a fixed optical density matrix determined by measuring relative absorption for red, green and blue on slides stained with a single stain. The normalized optical density matrix proposed in [START_REF] Ruifrok | Quantification of histochemical staining by color deconvolution[END_REF] for the combination of hematoxylin, eosin and DAB stains is defined as follows: The amounts of the three stains (H x , E x , DAB x ) in a particular pixel x can then be determined from its three color components (R x , G x , B x ) using this formula:

  H x E x DAB x   = OD -1 ×   -log( Rx+1 256 ) -log( Gx+1 256 ) -log( Bx+1 256 )   (4.3)
However, calculating the optical density matrix for each of the haematoxylin, eosin, and saffron (H, E, S) from images of our dataset was not possible, because slides stained with only one of these stains were not available. Hence, we have applied the optical density matrix proposed by Ruifrok & al. [START_REF] Ruifrok | Quantification of histochemical staining by color deconvolution[END_REF]. The obtained results remain comparable to the ones obtained in their study, especially, for the separation of haematoxylin and eosin. An example of color deconvolution using this approach is given in figure 4.6. In the following, the output image corresponding to the DAB color (like in figure 4.6.d) is considered as a background image, and it will not be used in our procedures.

Superpixel classification

The haematoxylin image obtained using the color deconvolution algorithm discriminates effectively nuclear regions in the image by their high intensity (see figure 4.7.b). This intensity image may have low contrasts when the haematoxylin concentration is very low, and when the color normalization step was not sufficient to correct this issue. Therefore, we perform a histogram stretching technique in order to enhance the image contrast, followed by an average filtering with a window of size 5 × 5 pixels. Assuming that the enhanced intensity image has real values between 0 and 1, we apply a thresholding to create a binary image, where pixels are set to 1 if their intensity is higher than T h , and 0 otherwise. The obtained image contains binary objects that may represent the cell nuclei (see figure 4.7.c). In order to remove undesired small objects that may not match with nuclei localities, we apply morphological opening [START_REF] Serra | Image Analysis and Mathematical Morphology[END_REF] with a disk of radius R d . For a better approximation of nuclei objects, we perform a morphological reconstruction [START_REF] Serra | Image Analysis and Mathematical Morphology[END_REF] of the obtained image under the mask image of the superpixels. This operation allows to build up the superpixels of the SLIC segmentation mask that are marked by the binary objects of the image obtained after the morphological opening. The result of the reconstruction is shown in figure 4.7.d. Formally, the nuclei segmentation process can be defined by the following equation:

I nuclei = R S mask (O R d (J T h (H)) (4.4)
Where, J T h is the thresholding operator that applies to the image haematoxylin H a binarization, such that for every pixel (i, j), The proposed algorithm is controlled by two parameters: T h , the threshold applied to the haematoxylin image; and R d , the radius of the binary opening. The optimal parameters were chosen by varying their values and evaluating the detection performance for each set of parameters. The annotated dataset that was used for the evaluation is presented in the next subsection.

(J T h (H)) i,j = 1 iff H i,j ≥ T h ; O R d is

Ground truth

The evaluation of the performances of image processing algorithms like detection, segmentation or classification of histological structures, such as nuclei or glands, is usually accomplished based on a comparative analysis between the data manually annotated by an expert pathologist (ground truth) and the results obtained by the algorithm. Due to the wide heterogeneity in histopathological data, an effective evaluation of such algorithms relies on a large number of annotated data.

Since it was tedious and time consuming to have an expert pathologist to manually annotate nuclei in our dataset, we have used the ground truth provided by a recent study dedicated to the detection of nuclei from histopathological images [START_REF] Xu | Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images[END_REF]. A set of 512 images were extracted from Whole Slide Images of H&E stained breast cancer tissues from 49 patients from Case Western Reserve University Hospitals, Ohio, USA. The histological slides have been scanned with a high resolution slide scanner, Aperio ScanScope R , at 40X optical magnification. The size of each extracted image is about 2200 × 2200 pixels, having a resolution of 0.25µm/pixel1 . The expert was asked to randomly select a region of interest (ROI) of 400 × 400 pixels in each image where cell nuclei were visible (see figure 4.8.a). The expert then placed a dot in the center of each nucleus within the ROI (see figure 4.8.b). Consequently, quantitative evaluation of the different models was limited to these 512 ROIs across the 512 images.

Images from the dataset described above do not have the same size, nor the same resolution as the image tiles considered in our dataset. In addition, the staining technique applied is slightly different. Therefore, the images from the dataset provided in [START_REF] Xu | Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images[END_REF], were first cropped (from 2200 × 2200 pixels to 2048 × 2048 pixels), then resized using bicubic interpolation (from to 2048 × 2048 pixels to 1024 × 1024 pixels), which provides images that have exactly the same size and resolution (0.5µm/pixel) as our dataset image tiles.

The staining technique adopted in the preparation of our dataset (HES) uses the Saffron stain in addition comparing to the dataset provided in [START_REF] Xu | Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images[END_REF], where the tissue sections were stained only with H&E. However, the Saffron stain does not affect the coloration of nuclei, as it highlights mainly collagen tissue. To avoid any uncertainty, we have applied color normalization to all images from the dataset in [START_REF] Xu | Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images[END_REF], by using the reference image provided in figure 4 As explained above, our node identification approach is based on the superpixels provided by the SLIC segmentation (section 4.5). Therefore, the annotated dots provided by the Chapter 4. A Method for Tissue Segmentation and Node Identification expert were translated to superpixels. Which consists in selecting superpixels that contain dots marked in the ground truth of [START_REF] Xu | Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images[END_REF] (see figures 4.8.(b-c)). It should be noted that one superpixel may cover one or more dots, but this is not problematic for our study since we are interested in defining fragments that represent cell nuclei, rather than each single nucleus, which may be impossible in some cases, due to many eventual artifacts. Finally, our evaluation method consists in comparing the superpixels provided by our nuclei detection algorithm and the superpixels derived from the ground truth of [START_REF] Xu | Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images[END_REF]. The ground truth contains 17294 annotated nuclei (dots) in total, which corresponds to 14830 superpixels that represent nuclei based on the proposed SLIC segmentation algorithm. There are 71191 superpixels within the validation ROIs (boxes) that are not nuclei. In the following, we will present the evaluation of the performance, and the choice of the optimal parameters, of the proposed nuclei detection algorithm based on this ground-truth dataset.

Detection performance evaluation

The evaluation of the performance of detection, segmentation or classification algorithms in histopathology images, is carried out generally in two steps; a first step that consists in learning the parameters of the algorithm, called training; and a second step that consists in validating the algorithm, called testing. The second step is the one that actually evaluates the success of results. In order to quantitatively assess the performance of such algorithms, evaluation metrics are typically used. These metrics are numeric scores that provide interpretation of the quality of the results, and allow comparison between the performances of different approaches. In addition, some evaluation metrics are specifically designed for discriminating the optimal solution during the learning step. Thus, the selection of suitable evaluation metrics is an important key for obtaining the optimal parameters. Evaluation metrics that are commonly used in machine learning classification tasks (binary, multi-class, multi-labelled, and hierarchical) are described and well explained in [START_REF] Sokolova | A systematic analysis of performance measures for classification tasks[END_REF]. Metrics that are specifically designed for discriminating the optimal solution are also discussed in [START_REF] Hossin | A review on evaluation metrics for data classification evaluations[END_REF]. The nuclei detection approach proposed in our study can be seen as a binary classification problem, where the input superpixel is to be classified into one of the two classes; nucleus (positive) or not nucleus (negative). The correctness of the classification can be evaluated by computing the number of superpixels correctly identified as nuclei (true positives, T P ), the number of superpixels correctly identified as not nuclei (true negatives, T N ), the number of superpixels incorrectly identified as nuclei (false positives, F P ), and the number of superpixels incorrectly identified as not nuclei (false negatives, F N ). Figure 4.9 shows an example that presents these four different situations.

Evaluation metrics derived from these measures include; accuracy, precision (the positive predictive value, P P V ), recall or sensitivity (the true positive rate, T P R), specificity (the true negative rate, T N R), and F-score or F-measure. Table 4.1 presents these evaluation metrics for the case of binary classification.

F β -score (β 2 +1)tp (β 2 +1)tp+β 2 f n+f p
Relations between actual positives and predicted positives TABLE 4.1: Evaluation metrics for binary classification. tp, tn, f p, and f n stand for the number of T P , T N , F P , and F N respectively.

The F β -score can also be interpreted as a weighted average of the precision and recall, where it reaches its best value at 1 and worst at 0. The general formula of the F β -score can be written as:

F β = (β 2 + 1) • P recision • Recall (β 2 • P recision) + Recall (4.5)
In order to find the optimal solution of our binary classification problem, based on the evaluation metrics presented above, we can use a graphical plot that illustrates the performance of the classification, when the discriminating threshold is varied. The most commonly used plot for this purpose is the Reception Operating Characteristic (ROC) curve. This curve is created by plotting the Sensitivity against (1-Specificity), also known as the fall-out or false positive rate (FPR), at various threshold settings. ROC analysis provides a tool to select possibly optimal models based on a concept closely related to cost/benefit analysis. Using this tool, the best possible prediction would yield a point in the upper left corner of coordinate (0, 1) of the ROC space, representing 100% of sensitivity (no false negatives) and 100% specificity (no false positives).

ROC analysis in our nuclei detection problem concerns the determination of the optimal parameters: T h the threshold of the binarization of the Haematoxylin image and R d the radius of the binary morphological opening. Figure 4.10.a presents different ROC curves, obtained when T h is varied between between 0.1 and 0.9, and R d is varied between 1 and 10.

The plot shows that the performance of the model decreases when the value of R d increases. The optimal solution is found at the point of coordinates ( 0.20, 0.88), which is located on the curve R d = 1, when T h = 0.6. The same experiment conducted on the Haematoxylin image, was also conducted on the blue ratio and the Eosin images. The blue ratio image is also an intensity image that highlights nuclei and it is given by this transform:

I BR = 255 × B (1 + R + G)(1 + R + G + B) (4.6)
where R, G and B are the red, green and blue channels, respectively. More details about this transform are given in the next section. The experiment shows that the detection performance given by the blue ratio image are slightly lower than the performance given by the Haematoxylin image (see figure 4 The amount of data in training and testing has critical impact on the evaluation of the system performance. In fact, more data in training lead to better system designs, whereas more data in testing lead to more reliable evaluation of the system. Therefore, we have divided our data into two equal training and testing sets, containing the same amount of positive and negative data, after being randomly mixed. The training set was used to determine the optimal parameters T h and R d based on ROC analysis. While, the testing set was used to evaluate the performance of the model when the parameters are optimal. The evaluation metrics calculated using the testing set with the optimal parameters are given in table 4.2. Evaluation of nuclei detection using other approaches can be found in [START_REF] Irshad | Methods for nuclei detection, segmentation, and classification in digital histopathology: a review-current status and future potential[END_REF]. An example of qualitative results of the proposed nuclei detection algorithm on an image tile from our dataset is presented in figure 4.11. The example displays the superpixels identified as nuclei overlaid in cyan. In what follows, the term detected nuclei will refer to the superpixels identified as nuclei by our algorithm.

Nuclei classification Overview

Once the superpixels that identify cell nuclei are determined, the next step consists in classifying the positive superpixels into three classes: 1) cancer cells, 2) immune cells and 3) fibroblasts (see figure 4.12). This can be accomplished by finding characteristics that are able to discriminate between each of the three classes. Characteristics (or features) that may be attributed to cell nuclei include shape, size, color and texture. In fact, generally, immune cells are small, round and stain deep blue; fibroblasts are lighter-colored, flat and elongated; while cancerous nuclei are most often larger and may have different size and shapes. Next figure shows samples of the three types of cell nuclei from our dataset. The classification approach proposed in our work is based on supervised feature learning, which consists in; extracting features from annotated data, evaluating them using a feature selection technique, and classifying the observations based on the significant features being selected. Therefore, we extract color and texture features from nuclei that were detected during the previous step and have been manually labeled as belonging to one the three classes. Here, we exclude geometric features, that describe size and shapes of nuclei, because they are not effective at the resolution of 0.5µm/. Then, we select relevant features using Fischer scoring. And finally, we use the Random Forests classifier in order to determine the class of each nucleus in the dataset. The flowchart in figure 4.13 presents the different steps of our approach for nuclei classification that we will describe in details in the following.

Ground truth generation

In supervised feature learning techniques, such as the present one, features are learned with labeled input data (ground truth). Therefore, we have manually annotated 2533 regions in 1005 image tiles extracted from 17 whole slide images randomly chosen from our dataset presented in section 4.2. Each region contains one or more nuclei having the same type, such as described in figure 4.14. The total number of nuclei detected within the annotated regions is 112125 including: 77676 cancer nuclei, 31037 immune cell nuclei, and 3412 fibroblasts. Cancer cells are delineated in red, immune cells in blue and fibroblasts in green contours.

Cell type Nbr. of detected nuclei within the annotated areas

Cancerous 77676

Immune cell 31037

Fibroblast 3412

Total 112125 In the proposed labeling method, we took advantage of the fact that cells of the same type are frequently located in the same region of the tissue, to collect the largest possible number of annotated nuclei in a shortest possible time. In addition, labeling only few samples per image tile, leaves as the option to consider a large number of tiles located at different regions of the tissue, which allows the labeled data to be heterogeneous.

Color feature extraction

For each nucleus, color features are calculated within the surface inside the nucleus, and within three surfaces around it. The surface inside the nucleus is given in a natural way by the superpixel surface. Whereas, the surfaces around the nucleus are obtained by morphological dilations with disks of three different radii: 5, 10 and 20 pixels respectively (see figure 4.15.a). The idea behind using the colors in the surface around a nucleus as features, is to introduce a contextual information describing the region in which the nucleus is located. For example, fibroblasts are most often located within collagen, which has particular orangish colors in the case of HES-staining. The advantages of the use of contextual information to classify nuclei in histopathology images are discussed in [START_REF] Nguyen | Using contextual information to classify nuclei in histology images[END_REF]. The color features are calculated from the three RGB channels of every image tile, and other intensity images derived from them, including: the haematoxylin (H) and eosin (E) images obtained using color deconvolution (refer to section 4.6.1), the blue ratio image (I BR ) (equation 4.6), and the red ratio image (I RR ) defined as follows:

I RR = R 1 + G + B + (G-B) 2 1+G+B + R (4.7)
It should be noted that the term (G-B) 2 1+G+B was adopted in order to highlight orangish colors that are found in collagen. Examples of blue ratio and red ratio images are given in figure 4.16. We also use the grayscale intensity image, which is obtained using the following equation:

Ig = 0.2989 × R + 0.5870 × G + 0.1140 × B (4.8)
Color features are then calculated using statistical metrics over the outlined surface of the intensity image. These statistics are: median, mean, standard deviation, skewness and kurtosis. Using this scheme, there are 160 color features per nucleus in total, calculated using 5 statistical metrics in 4 different regions from 8 intensity images.

Texture feature extraction

For each nucleus, texture features are calculated from image frames of different sizes, and centered at the centroid of the corresponding superpixel. The adopted frame sizes are: (20 × 20), (40 × 40), (70 × 70), and (100 × 100), (see figure 4.15.b). The texture features are calculated from intensity images that emphasize cell nuclei, such as the blue ratio image (I BR ), the haematoxylin image (H) and the grayscale image (Ig), using two different texture approaches. Several texture analysis methods have been proposed in the literature. A detailed description of the most commonly used texture methods in digital image analysis can be found in [START_REF] Materka | Texture analysis methods -a review[END_REF][START_REF] Zhang | Brief review of invariant texture analysis methods[END_REF][START_REF] Bharati | Image texture analysis: methods and comparisons[END_REF][START_REF] Nielsen | Statistical nuclear texture analysis in cancer research: a review of methods and applications[END_REF]. In our work, we adopt two texture methods known as co-occurrence matrix method [START_REF] Haralick | Textural features for image classification[END_REF][START_REF] Haralick | Statistical and structural approaches to texture[END_REF], and Laws texture energy measures [START_REF] Laws | Texture energy measures[END_REF][START_REF] Laws | Textured image segmentation[END_REF].

a) Co-occurrence matrix based features:

This method is one of the most used methods to extract second-order statistical texture features. It is also known as Haralick's texture features [START_REF] Haralick | Statistical and structural approaches to texture[END_REF], or gray-level co-occurrence matrix (GLCM) features, and it analyzes the spatial distribution of the image intensity values based on statistical metrics calculated over cooccurrence matrices. A co-occurrence matrix is a square matrix with dimension n g , where n g is the number of intensity levels in the image. Each element (i, j) of the matrix is generated by counting the number of times a pixel with value i is adjacent to a pixel with value j. Since the pixel adjacency can be defined to occur in four different directions in a 2D image (horizontal, vertical, left diagonal and right diagonal, see figure 4.18.c), four co-occurrence matrices can be calculated (one for each direction). Each co-occurrence matrix is then normalized such that the sum of its elements is equal to 1, by dividing the entire matrix by the total number of co-occurrences found. Each element p i,j of the normalized matrix is then considered to be the probability that a pixel with value i is adjacent to a pixel with value j. The texture features are then calculated from the normalized co-occurrence matrices using statistical measures. Haralick have described 14 second order statistical measures with the intent of describing image textures in [START_REF] Haralick | Statistical and structural approaches to texture[END_REF]. However, in this work, we limit ourselves to 4 statistics; Contrast, Correlation, Energy, and Homogeneity, which are presented in table 4.4.

Statistical metric

Formula Description Contrast i j (i -j) 2 pi,j

indicates the difference between adjacent pixels Correlation i j (i-µx)(j-µy ) σxσy pi,j describes the dependency of the gray levels with µ x ;µ y and σ x ;σ y are the means and standard deviations of the partial probability density function, and they are expressed as follows:

µ x = i j ip i,j and σ x = i j (i -µ x ) 2 p i,j (4.9
)

µ y = i j jp i,j and σ y = i j (j -µ y ) 2 p i,j (4.10)
The number of gray-levels n g determines the size of the co-occurrence matrix. Which means that a wide range of intensity levels will induce matrices of large dimensions and a high computational cost. Hence, the intensity values of the images are scaled such that they are integers between 1 and 8. Which means that we use 8 gray-levels and co-occurrence matrices of dimension 8 × 8. Using this technique, there are 16 texture features that can be computed from one image frame; 4 statistical measures of 4 co-occurrence matrices (defining 4 directions). Since we use 4 frames of different sizes and 3 different intensity images (I BR , H, Ig), for each detected nucleus, there are 192 co-occurrence matrix-based features calculated in total. Figure 4.19 presents our approach for texture feature extraction. b) Laws texture energy measures: This approach was developed by Laws [START_REF] Laws | Texture energy measures[END_REF] to detect various types of texture by measuring the amount of variation within a fixed-size window. It consists of a set of 5 × 5 convolution masks, used to compute texture energy features. The 2D convolution masks are generated by computing products of pairs of vectors, which are defined as follows:

L5(Level) = [ 1 4 6 4 1 ] E5(Edge) = [ -1 -2 0 2 1 ] S5(Spot) = [ -1 0 2 0 -1 ] W 5(W ave) = [ -1 2 0 -2 1 ] R5(Ripple) = [ 1 -4 6 -4 1 ] (4.11)
The names of the vectors stand for the information they are expressing; the L5 vector gives a center-weighted local average, the E5 vector detects edges, the S5 vector detects spots, and the W 5 and R5 vectors capture waves and ripples. These vectors are then multiplied to produce 5 × 5 convolution masks. For example, the mask E5L5 is computed as the product of E5 and L5 as follows:

      -1 -2 0 2 1       × [ 1 4 6 4 1 ] =       -1 -4 -6 -4 -1 -2 -8 -12 -8 -1 0 0 0 0 0 2 8 12 8 2 1 4 6 4 1       (4.12)
A total of 25 convolution masks is generated, and applied to the input intensity image, producing 25 filtered images. Thereafter, the texture energy maps are calculated as follows:

E k = i j |F k I(i, j)| (4.13)
where is the convolution operator and E k is the energy map of the input image I, filtered with mask F k , k ∈ {1, ..., 25}. Hence, each texture energy map E k is a full image, representing the application of the kth mask to the input image I. Certain symmetric pairs of energy maps are combined to produce final maps, by replacing each pair with its average. For example, E5L5 detects horizontal edges, and L5E5 detects vertical edges. Thus, the average of these two maps is used to measures the total edge content. The result of all the processing gives a total of 15 final energy maps. From each energy map, 5 statistical metrics are calculated: mean, standard deviation, skewness, kurtosis, and entropy, refer to table 4.5. Hence, this technique computes 75 texture features from an input image. Since we use 4 image frames and 3 intensity images, there are 900 Laws texture features calculated per nucleus in total.

Statistical metric Formula Description

Mean

1 N f i j E k (i, j)
indicates the average texture energy, denoted µ k Standard deviation

1 N f i j (E k (i, j) -µ k ) 2
represents the energy variation in comparison with the average, denoted σ k Skewness

1 σ 3 k i j (E k (i, j) -µ k ) 3
describes the degree of histogram asymmetry around the mean Kurtosis

1 σ 4 k i j (E k (i, j) -µ k ) 4
describes the sharpness of the energy level histogram Entropy -

i j E k (i, j)log2(E k (i, j))
describes the randomness of the texture energy values N f is the number of pixels in the energy map. A total of 1092 texture features using the both methods is computed for each nucleus.

Feature selection

The proposed method for feature extraction provides a total of 1252 descriptive features per nucleus. Indeed, due to the complexity of the task, a large number of features is highly desirable to effectively classify a superpixel as cancer cell, immune cell, or fibroblast. However, the use of all the extracted features for classification, implies a high computational cost and may reduce the classification performance. In fact, some features are irrelevant for classification and does not provide class discriminatory information, degrading the classification performance. And some features are redundant, representing duplicate information, useless for the classification. Therefore, a feature selection step is required. There are several feature selection techniques that can be found in the literature [START_REF] Dash | Feature selection for classification[END_REF][START_REF] Kwak | Input feature selection for classification problems[END_REF][START_REF] Tang | Feature selection for classification: A review[END_REF].

In our study, we use a univariate feature selection method, known as Fisher Score [START_REF] Gu | Generalized fisher score for feature selection[END_REF], where each feature is ranked independently of the feature space. In fact, Fisher Score method evaluates each feature individually, using the following measure:

F s = K k=1 n k (µ -µ k ) 2 K k=1 n k σ 2 k (4.14)
where µ is the mean of the feature values over all instances, n k is the number of instances in the k-th class, and µ k and σ k are the mean and standard deviation of the feature values over the instances in the k-th class. Assuming that high quality features should assign similar values to instances within the same class and different values to instances from different classes, the Fisher score F s of a particular feature will then increase with its discriminative power.

Since this technique evaluates features individually, it cannot handle feature redundancy. However, in the proposed features, there are obvious redundancy between the features that provide the same information from frames (or regions) of different sizes. For example, between the two features providing the same statistical measure calculated from a surface of radius 10 and a surface of radius 20. Our aim is to select the best feature between its counterparts and reject the others. Finally, using this feature selection method, there are 80 color features (5 statistics × 8 images × 2 surfaces, inside and around nuclei) and 257 texture features (75 Laws features × 3 images + 16 GLCM features × 2 images). Which makes a set of 337 features in total. In the following, we will present the method used to classify the nuclei based on these features.

Feature normalization

In most cases, the features have different dynamic ranges. These variations affect the performances of the majority of classifiers. In fact, if one of the features has a broad range of values, the decision will be mastered by this particular feature. Therefore, dynamic ranges of features are normalized so that each feature contributes proportionately to the final decision. We solve this problem by normalizing the features values so that they lie within similar dynamic ranges using the following formula:

f = f -f min f max -f min (4.15)
where f is original feature value, f is the normalized feature value, f min is the minimum feature values and f max is the maximum feature value.

Random forest classification

In order to find optimal feature values, allowing us to recognize one class from another, we promote the use of ensemble learning methods, which use multiple learning algorithms to obtain better predictive performance [START_REF] Rokach | Ensemble-based classifiers[END_REF]. Thus, we have adopted a supervised learning technique known as Random Forests, or random decision forests, which fits a number of decision tree classifiers on various sub-samples of the dataset and use averaging to improve the predictive accuracy [START_REF] Ho | Random decision forests[END_REF]. A decision tree classifier organizes a series of test questions and conditions in a tree structure, where nodes contain attribute test conditions to separate observations that have different features. A comprehensive scheme of a decision tree classifier is given in figure 4.22. In our approach we use Random Forest classifier, composed of 20 decision trees, that selects a random subset of predictors to use at each decision tree. Next subsection presents the evaluation of the performance of this classifier based on the 337 features extracted for each nucleus.

Detected nucleus

Haematoxylin (in)

Classification performance evaluation

In section 4.6.1, we have presented evaluation measures used to evaluate the performance of a binary classifier, employed for the detection of nuclei. However, the evaluation of the performance of nuclei classification is slightly different as it uses a multi-class classifier, since there are three different classes of nuclei (cancer cells, immune cells and fibroblasts). In a multi-class classification the input is to be classified into one, and only one, of K nonoverlapping classes (K = 3). The correctness of a multi-class classification can be evaluated by computing the number of correctly recognized class samples (true positives), the number of correctly recognized samples that do not belong to the class (true negatives), and samples that either were incorrectly assigned to the class (false positives) or that were not recognized as class samples (false negatives). The assessment can be defined by calculating the number of true positives/negatives and false positives/negatives for each individual class C k , denoted tp k , tn k , f p k and f n k , respectively. The quality of the classification is usually assessed in two ways: i) micro-averaging, where the performance measure is calculated from the cumulative sum of counts obtained by tp k , tn k , f p k , f n k , ii) macro-averaging, where the performance measure is the average of the same measures calculated for each class C k individually. The difference between micro-averaging and macro-averaging, is that the first one favors bigger classes, while the second one treats all classes equally. In order to avoid a possible inequity between cell nuclei classes, we promote the use of macro-averaging measures, which are presented in table 4.7.

Evaluation metric

Formula Description AccuracyM

1 K K k=1 tp k +tn k tp k +tn k +f p k +f n k
The average per-class effectiveness of the classifier

F β -scoreM (β 2 +1)• P recision M •Recall M (β 2 •P recision M )+Recall M
Relations between data's positive labels and those given by the classifier based on sums of per-text decisions In order to evaluate the effectiveness of our nuclei classification approach, we use the evaluation metrics provided in table 4.7 computed by a 10-fold cross-validation technique. Where, the system is evaluated on a separate testing datasets, which are not used in the training dataset. Hence, the data is first partitioned into 10 equally (or nearly equally) sized segments (or folds). Then, 10 iterations of training and testing are performed, such that, in each iteration, one fold of the data is used for training while the remaining K -1 folds are used for testing. The advantage of this method is that all observations are used for both training and validation, and each observation is used for training exactly once.

The evaluation metrics of our nuclei classification algorithm computed by 10-fold crossvalidation, are given in figure 4.23. Since our ground truth dataset for nuclei classification is imbalanced (different amounts of data in each class, refer to table 4.3), it would be important to validate the algorithm not only on the entire imbalanced dataset (as shown in figure 4.23.a), but also on a random balanced subset of the dataset where the numbers of samples in all classes are equal. This is done by randomly selecting N f instances from the cancer cell and immune cell classes, where the N f is the number of fibroblasts (the smallest subset) in the dataset. The average evaluation metrics over the 10 iterations of the 10-fold cross-validation using the imbalanced dataset are given in table In order to evaluate how the algorithm confuses two classes of nuclei, we use the four counts tp k , tn k , f p k and f n k to constitute a confusion matrix (see table 4.9. Each column of the matrix represents the instances in a predicted class while each row represents the instances in an actual class. The confusion matrix shows that cancer cell nuclei are the best recognized by the algorithm, and when they are misclassified, they are more likely to be classified to immune cells than fibroblasts. The table shows also that the algorithm misclassifies fibroblasts at a high percentage: 25% as cancer cells and 6% as immune cells. Qualitative results can also be found in figure 4.24.

Collagen node identification

In order to establish a set of nodes that identifies the location of collagen fibers, we use the centroids of superpixels that may represent collagen based on color information. As described above, the red ratio image, obtained using the formula in equation 4.7, gives prominence to collagen fibers and presents them with significant intensity values comparing to other tissue components. Figure 4.25.b shows an example of a red ratio image derived from a an image tile that presents collagenous tissue. Therefore, the red ratio image is thresholded to give a first approximation of the location of significant collagen fibers. The applied threshold is set to 0.5, where pixels with intensity lower than 0.5 are removed. Then, a binary morphological opening with a disk of radius 2 is applied to the binary image in order to filter the first output of the thresholding and remove insignificant small objects. The obtained binary image represents significant collagenous structures, and it is then multiplied with the binary mask that defines the regions of the superpixels, see figure 4.25.c. This operation eliminates the superpixels that do not represent collagen. Finally, the centroids of the remaining superpixels are used as nodes, to symbolize the collagenous structures of the tissue that can be used for further studies. Due to unavailability of annotated data, a quantitative evaluation of the proposed collagen segmentation approach was not possible. However, the evaluation was performed qualitatively by inspecting several collagen segmentation results obtained with different thresholds on whole slide images of different tissue samples.

Adipose tissue node identification

Adipose nodes are established using a semi-automatic approach, where tissue regions that include adipose tissue were annotated using Aperio ImageScope R software, then, a segmentation method is applied in order to extract adipose structures. In the regions of interest of the breast tissue sections provided in our dataset, adipose tissue is not very abundant and adipocytes have tendency to form large clusters, with a limited number of clusters, which makes it easy to annotate them in a quick manner. In fact, a number of 1362 annotations (annotated regions) was sufficient to annotate the totality of adipose tissues found within the regions of interest in our dataset composed of 55 WSI. Which is only a fraction of the effort made to provide a nuclei classification ground truth. Figure 4.26.a shows an example of annotated adipose tissue regions. The goal behind annotating these regions is to avoid confusion between adipose tissue and other white image components, such lumen of lymphatic/blood vessels or glands, or background, during the segmentation step.

Therefore, the input image is first thresholded in order to extract whitish pixels, such that only pixels with (R,G,B) values, simultaneously, greater than 220 are retained (see figure 4.26.b). The obtained binary image is then filtered by morphological opening operation with a disk of radius 10. The result is then multiplied with the annotated regions' mask in order to eliminated possible whitish components, and with the superpixel mask in order to determine superpixels that define the adipose tissue structure. The final result is shown in figure 4.26.c. The centroids of the remaining superpixels are used as nodes that identify the locations of the adipose tissue structures as shown in figure 4 However the adipose tissue segmentation method proposed here is not automatic, the output that it provides can be used to generated a consistent ground truth which can be used as training data by supervised learning algorithms for adipose tissue segmentation, that may be applied to large datasets.

Conclusion and perspectives

In this chapter, we have presented our node identification technique for the identification of five histological structures constituting an important part of the breast tumor ecosystem. A nuclei detection algorithm from H&E/HES-stained tissue images has been developed, and qualitatively evaluated on our dataset and quantitatively evaluated on a third-party dataset. A nuclei classification algorithm has been established and quantitatively evaluated on a large subset of our dataset, which was annotated by our efforts in order to provide a ground-truth, and it should be made public. The final result obtained by the combination of the two algorithms, is reasonable for studying the spatial arrangements of cell nuclei in HESstained breast tissues based on three histological classes of cells (i.e. cancer cells, immune cells, fibroblasts).

An algorithm for the segmentation of collagenous structures from HES tissue images has also been developed and qualitatively evaluated on our dataset. The preliminary approach gives promising results and should be quantitatively evaluated on annotated dataset. A semi-automatic algorithm for the segmentation of adipose structures has also been developed and qualitatively evaluated on our dataset. The results are satisfying and they can be used in generating annotated dataset and validating new adipose tissue segmentation approaches. An example of the overall node identification result is given in figure 4.27.

The proposed method for node identification has shown the possibility of obtaining adequate results for the identification of cell nuclei and other histological structures using handcrafted features based on color and texture information, however, in future works, we will consider deep learning methods for tissue segmentation and node identificaiton, such as deep convolutional neural networks (refer to section 1.3.2), as they are efficient and fast tools for these purposes, and they are gaining a large popularity in histopathological image analysis during the last decade [START_REF] Janowczyk | Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases[END_REF][START_REF] Sirinukunwattana | Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images[END_REF].

Introduction

In chapter 4, we have presented a node identification method that extracts from a whole slide image (WSI) of breast tumor tissue stained with HES, a set of points identified by their coordinates in the WSI-plan and labeled according to their biological functions/histological types. The node extraction algorithm was applied to the 55 WSIs available in the dataset described in 4.2. In this chapter, we propose a framework for the characterization of the spatial arrangement of histological structures in the tissue based on spatial configuration modeling of the extracted point-sets using the graph-based mathematical morphology approach presented in chapter 3.

From now on, our current input data to be analyzed is composed exclusively of the pointsets extracted from these WSIs, whereas the pixel-based information contained in the image is set apart. In the following sections, we first present the current dataset being composed of these point-sets and we study their statistical distributions, before we perform spatial analysis dedicated to the characterization of their spatial distributions and interactions.

However, the available dataset is composed of WSIs and information about the corresponding regions of interests (ROIs), but it does not include annotations or variables associated to each case, like clinical parameters or histopathological attributes identified by the pathologist. Hence, the analysis presented in this chapter are based mainly on unsupervised learning, where we try to find some intrinsic class labels directly from the unlabeled data.

Histological structure counting analysis

The node identification approach proposed in chapter 4, identifies 5 distinct histological structures by their locations in the plan of the tissue image: 3 cell types (i.e. cancer cells, immune cells and fibroblasts), collagen fibers and adipose tissue fragments. From a functional and spatial point of view, fibroblasts and collagen can be considered as one histological component, since fibroblasts are a type of cells that synthesize collagen and they are found in area within or near collagenous tissue in general. Hence, in our study, we consider fibroblasts and collagen as belonging to the same class of tissue components. In the following, nodes associated to fibroblasts and nodes associated to collagen are dealt with without distinction, and we denote them F CN . Whereas, cancer cell nodes, immune cell nodes, and adipose tissue nodes are denoted CCN , ICN and AT N , respectively.

The total number of nodes extracted for each tissue component within the ROI of each whole slide image of the dataset, are given in figure 5.1. The dataset is sorted according to the area of the ROI, which is also given in the following figure. At first glance, the diagrams given by the number of nodes of the tissue components share some similarities together with the diagram given by the ROI area, which suggests some correlations. In order to determine the degree to which the number of nodes defined within an ROI is dependent on the ROI area, and see if there is any dependency between the amounts of nodes of different histological types, we calculate the matrix of correlation coefficients, which is a matrix where each element represents a measure of the linear dependence between two random variables based on the Pearson correlation coefficient [START_REF] Pearson | Notes on regression and inheritance in the case of two parents[END_REF], which is defined as follows:

ρ(X, Y ) = 1 n -1 n i=1 X i -µ X σ X Y i -µ Y σ Y (5.1)
where µ X ;µ Y and σ X ;σ Y are the means and standard deviations of the variables X and Y , respectively. And n is the number of observations (cases in the dataset). ρ(X, Y ) has a value between +1 and -1, where 1 is total positive correlation, 0 is no correlation, and -1 is total negative correlation (see table 5.1). We calculate also the matrix of p-values for testing the hypothesis that there is no relationship between two variables (null hypothesis). If an element of the matrix is smaller than the significance level p < 0.05, then the corresponding correlation between X and Y is considered significant (see table 5 The correlation and p-value matrices suggest that the amounts of cancer cells and collagen are highly correlated with the size of the invasive area (ROI) compared to the amounts of immune cells and adipose tissue (table 5.1). Furthermore, table 5.2 shows that the correlation between the amounts of cancer cells and immune cells cannot be rejected. Which would mean that a high number of tumor cells will induce a high number of immune cells or vice-versa. However, it is now too early to draw any conclusions, it may be interesting to study the relative abundance by calculating the ratio between the amounts of two components of the tissue, particularly the number of tumor microenvironment (TME) components relative to the number of tumor cells. Figure 5.2 shows the ratios ICN/CCN , F CN/CCN , AT N/CCN , and the percentage of each component found in each case of the dataset. The data is sorted according to the percentage of cancer cells. Several interpretations from the diagrams can be made. One of the most important is the ratio ICN/CCN which can be interpreted as the number of immune cells willing potentially to fight cancer cells, and thus, as one of the indicators of immune response against cancer. For example, we can distinguish a unique case where the number of immune cells exceeds the number of cancer cells, in case N • 52. In fact, as shown in figure 5.3, the tissue presents a large number of immune cells that are widespread within the ROI and mixed (co-located) with cancer cells. We can also distinguish one case where the ratio F CN/CCN is relatively high (case N • 37), shown in figure 5.4, and it corresponds to a mucinous carcinoma where cancer cells are scattered throughout pools of mucin, even though this is not related to the fact that collagen is abundant, it reflects the fact that there are only few cancerous cells in the tissue. Another case that can catch our attention where the ratio AT N/CCN is relatively The purpose of this counting analysis is to get a clearer understanding of the available data more than any other reason. The percentage of a tissue component in conjunction with the size of the ROI give some idea about the amount of space occupied by the component (global density); for example, a low percentage of cancer cells found within a large ROI area, means that cancer cells are not very abundant relative to the size of the considered ROI. This information is valuable, but not sufficient to determine whether these particular cancer cells are clustered within a small area or uniformly distributed in the tissue, which would allow us to know if the tumor is sparse or dense. Similarly, the ratio of a TME component relative to the tumor describes the relative abundance of that component comparing to the tumor; for example high ICN/F CN ratio may explain the presence of a immune presence. This information is also valuable, but it does not describe whether immune cells are located close or far from cancer cells, which would indicate if there is a spatial interactions between both entities. Hence, further analysis is needed. In the following, we conduct a deeper study in order to answer these questions based on a techniques of spatial point pattern analysis using graph-based mathematical morphology.

Neighborhood relationship establishment

Defining relationships between the different histological structures in the tissue based on nodes identifying their locations, may be accomplished using different linking rules (refer to section 2.2.5). In order to keep pace with the point pattern analysis approach proposed in 3, we use the linking rules defined by the Delaunay triangulation [START_REF] Delaunay | Sur la sphère vide. A la mémoire de Georges Voronoi[END_REF].

In addition to the nodes that identify the 5 histological structures considered in our tissue analysis, we include an additional set of nodes derived from the boundary of the ROI. This set of nodes is useful in further computational tasks, that are explained later in this chapter, and it is obtained from the centroids of distinct random superpixels lying on the edge of the ROI mask (see figure 5.6, boundary nodes are shown in cyan). In the following, we denote this set of nodes by BDN . Thereafter, the Delaunay graph is computed on the whole population of nodes (V = CCN ∪ ICN ∪ F CN ∪ AT N ∪ BDN ) without distinction (see figure 5.6.a). The disadvantage of the linking rule defined by the Delaunay graph lies in the fact that links may exist between points that seem to be far from each other, especially in the case of the outermost points (see figure 5.6.a). To circumvent this limitation, we make use of α-shape filtering (refer to section 2.2.6), where triangles with circumradius larger than 50 pixels (25µm) are removed. This operation eliminates connections derived from sharp triangles that can be found mostly on the borders of the Delaunay triangulation of a point set (see figure 5.6.b).

In the following, we denote by G the graph obtained from the α-shape filtering (α = 25µm) of the Delaunay triangulation of the whole population of nodes V of any whole slide image. For a simple visual presentation, the edges of the graph will not be traced in every illustration, but we simply display the graph nodes by colors associated to their numerical values.

Morphological filtering of histological structure nodes

Despite the efforts made in recent years in the development of methodologies and techniques for tissue segmentation and nuclei classification, the output results obtained from the employed image processing algorithms are hardly completely free from error. Using the node identification approach proposed in chapter 4 several errors may occur, and they can be divided into two main categories: position errors and labeling errors. Position errors are related to errors of over/under detection that can be generated from the nuclei detection algorithm, and from the collagen and adipose tissue segmentation algorithms. While labeling errors are related to misclassification, where a histological class (label) is incorrectly attributed to an already defined point, which comes mainly from the nuclei classification algorithm (fibroblasts that are wrongly classified as cancer cells for example). Some of these errors can be noticed on the results of our node identification algorithm (see figure 5.7.b). Therefore, this first output of the node identification step have to be improved in order to conduct a proper analysis of the spatial arrangement of histological structures.

As explained above, when the graph nodes are parametrized with numerical values, they can be processed by morphological filters created from combinations of morphological operations, such as the Alternating Sequential Filter 3.2.4. Tissue nodes in our case can be parametrized with binary values in order to distinguish one tissue component from others, or with class labels where a unique value is associated to nodes representing each tissue component. In the following, we suppose that the graph G is labeled such that CCN = 1, ICN = 2, F CN = 3, AT N = 4 and BDN = 0.

Figure 5.7.c shows the result of morphological closing of size 2 applied to the graph G defined on nodes with this parametrization. The morphological closing, in this particular case, associates to the nodes that were labeled as CCN or ICN , and that are dispersed within collagenous tissue, the F CN class (since F CN nodes are labeled with a higher value (F CN = 3)). This operation can be useful in improving the quality of the spatial information by smoothing the class labels according to their locations. For example, fibroblast nodes that were incorrectly classified as CCN or ICN , may regain their true value. It should be noted that the result obtained for F CN using this decimal configuration would be the same if we have applied a morphological closing on a binary graph G F , where G F (v) = 1 if v ∈ F CN , and zero otherwise.

Figure 5.7.d shows the result of an Alternating Sequential Filter of order 5 begging with closing (Φ 5 (G)) applied to the graph G. This operation allows us to determine nodes located in an area significantly occupied by cancer cells for example. It should be noted here that the result obtained for CCN using this operation would be the same if we have applied the Alternating Sequential Filter of order 5 begging with opening (Ψ 5 ), to the binary graph The aim behind exposing these morphological operations at this stage is to get a better comprehension of the modifications and transformations that can be produced based on the opening and closing operations in the tumor microenvironment. Morphological operations of this kind can be useful in improving the results of node identification and nuclei classification from a holistic point of view. The performance of such approach can be evaluated on simulated data in the form of spatial point sets imitating histological structure distribution, where we can introduce errors in the class labels of nodes and evaluate the output after filtering.

G C defined by CCN nodes (G C (v) = 1 if v ∈ CCN ,
In the following, we use morphological functions derived from combinations of these operations, in order to extract parameters that describe the spatial arrangement of the histological structures.

Quantitative analysis of the spatial distribution of histological structures

Quantifying the spatial arrangement of histological structures in the tissue may be led in the determination of several spatial aspects. In this study, we are interested in describing two central spatial aspects that have complementary visual interpretations, based on quantitative morphological features.

First, we propose to evaluate for each histological structure its expanse in the tissue by estimating the number of nodes found in the widest continuous areas including only its nodes. The expanse of cancer cell nodes for example may describe the size of the aggregates that are formed by populations of cancer cells. The expanse in this case can be interpreted as a measure that is related to the density or degree of clustering of cells in the agglomerations. While when calculated over collagen nodes, the expanse can be considered as expressing the thickness and the extent of the collagenous tissue.

Second, we evaluate the sparsity of each tissue component based on the number of nodes found in the widest continuous area enclosed by, but not including, its nodes. Sparsity in this context describes the dispersion of the cell agglomerations in the tissue in terms of space intervals existing between them. For example, in the case of cancer cell nodes, it provides an idea on whether cancer cell agglomerations are occurring and growing at widely or tiny spaced intervals.

Expanse of histological structures

Evaluating the expanse of a tissue component may be established by evaluating the number of nodes (order) of its connected components in the graph G. However, connected component labeling in this particular application would not be adequate because of computational limitations due to the size of the graph, and even if that were possible, the approach would not be efficient because of the errors that may occur during the node identification step. In fact, connected component labeling algorithms are very sensitive to this type of errors: sometimes it is sufficient that one cell nucleus is incorrectly classified as cancerous between two "touching" tumor aggregates, to make the labeling algorithm consider them as only one connected component of cancer cells, which does not meet our expectations. Another way to evaluate the expanse of a tissue component is to calculate the number (or ratio) of its nodes within fixed-size local regions (commonly grid-type frames) which would express literally the local density. The disadvantages of this technique lie in the fact that it introduces a degree of subjectivity in the choice of the frame size, and that when an agglomeration of cells is larger than the window frame, the measurements do not express its content entirely (globally), since they consider individual parts of the same agglomeration.

Therefore, we promote the adoption of a more holistic approach based on mathematical morphology on graphs in order to derive information about the expanse of histological structures in the tissue. The approach is derived from the method described in section 3.3.3, which we have proposed as a tool for spatial point pattern analysis. In fact, this method integrates a global approach in quantifying the expanse of agglomerations of nodes, independently from any scaling and input parameters, based on a morphological openness function that is defined as a cumulative sum of morphological opening operations with different sizes applied to a binary graph (refer to section 3.2.4). A binary graph in our case can be defined by setting the nodes of one histological structure as foreground nodes, and other nodes to background. In the following, we denote by G C , G I , G F and G A , the binary graphs defined by CCN , ICN , F CN and AT N nodes, respectively (for example, in the case of cancer cells, G C (v) = 1 if v ∈ CCN , and zero otherwise).

Hence, for any of the four histological structures, the openness function Ω can be applied to the binary graph defined by its corresponding nodes, in order to associate a numerical descriptor to each of them. However, prior to that, an improvement of the quality of the output of the node identification step is advisable. The openness function is more likely to be affected by inappropriate background nodes that are located inside agglomerations of foreground nodes (nodes of the histological structure under study), since it is derived from opening operations which tend to reduce the solidity of these agglomerations. To respond to this, we apply a morphological closing of size r c to the binary graph of any histological Graph-based Mathematical Morphology structure before applying the function Ω. Morphological features are then derived from these operations with the aim to define classes of expanse for each histological structure. The flowchart of our method for the characterization of tissue components based on their expanse is shown in figure 5.8.

Morphological openness function-based features

For each tissue component (e.g. CCN , ICN , F CN , AT N ) the size of the morphological closing r c is varied between 0 and 5. As can be seen, the output of the openness function changes considerably depending on the size of the prior closing operation. Since it is not possible to quantitatively determine the best value of r c , that better improves the result of the node identification, we calculate for each node the average of the values of Ω • ϕ rc over the six configurations, i.e. r c ∈ {0, 5}. We call the result of the averaging the expanse function and it can be formulated by the following equation:

Σ(G X ) = 1 6 5 rc=0 Ω • ϕ rc (G X ) (5.2)
where G X is a binary graph defined by one of the four histological structures (equal to G C , G I , G F or G A ), and ϕ rc is the morphological closing of size r c . An example and the associated histogram of the expanse function of G C are given in figures 5.9.(i-j). The histogram of the expanse function of cancer cell nodes over the whole dataset is given in figure 5.10.g. Histograms of the expanse functions of the other tissue component can be found in figures A.1, A.

Comparative analysis with population density

In order to better understand the significance reflected by the expanse function, we conduct a brief comparative analysis with population density measurement. Therefore, we calculate the density of each tissue component (e.g. CCN , ICN , F CN and AT N ) as the ratio of its number of nodes within a frame (image tile) to the frame size (1024 × 1024). Then, we associate to each node the density-value of the frame in which it is included. In figure 5.11, we give the bivariate histograms of Σ-values and density-values of nodes of each tissue component over the whole dataset. The histograms suggest that some correlations exist between the two measurements, enabling us to assume that the expanse function expresses some sort of spatial density, even though it is not directly derived from measures of the metric space. Qualitative results of population density are also given in figure 5.16.(g-i), and they are discussed later in this section.

Intra-tumor heterogeneity analysis

Cell agglomerations of the same type, and within the same tissue sample, may have various spatial arrangements and distributions. For example, we can find large dense tumor patterns in one part of the tissue, and small thin (e.g. Indian file patterns) in the other. The characteristics offered by the expanse function can be useful in describing such diversity. In fact, the Σ-values of nodes representing one histological structure in the same image can be used in calculating statistical measures, such as the coefficient of variation or entropy, that may reflect the spatial heterogeneity within the same tissue. However these two cases do not have comparable ROI sizes (refer figure 5.1), here we are interested in the architectural aspect rather than the amount of cells. Thus, we pick n s random CCN nodes from each case and we calculate the entropy of their Σ-values using the following formula: 106

Chapter 5. Characterization of the Spatial Organization of Histological Structures using Graph-based Mathematical Morphology

Ξ Σ(G C ) = - ns v∈CCN Σ(G C )(v) • log 2 (Σ(G C )(v)) (5.3)
where Σ(G C )(v) is the value of the expanse function in node v and n s = 10 4 is the number of random nodes.

Although these two particular cases have comparable percentages of cancer cells (50% for case N • 2, and 52% for case N • 53), the sizes of the two random samples considered in the evaluation are equal (n s = 10 4 ), and the values of Σ are normalized relative to the maximal value in each case, the entropy obtained in case N • 2 is significantly higher (figure 5.12.b, Ξ = 6.8)) than the entropy obtained in case N • 53 (figure 5.12.a, Ξ = 3.7). In fact, in case N • 2, the tumor patterns have varied sizes and densities, whereas in case N • 53, the tumor patterns have almost the same dimensions. This is also confirmed qualitatively from figures 5.12.(c-h), where random frames (tiles 1024 × 1024) from each ROI are presented.

In the same manner, more statistical measures, such as median, mean, standard deviation, skewness and kurtosis, can be derived from the values of the expanse function of histological structures within the same tissue in order to provide features describing the architecture of the tissue, enabling evaluation and comparison between different cases in a framework of inter-tumor heterogeneity analysis.

Inter-tumor heterogeneity analysis

The features proposed above are derived from analysis and comparison of the expanse measure of nodes within the same tissue, independently from all other cases. In order to understand what the values associated to nodes in a case A reveal comparing to the values associated to nodes in a case B, an overall comparison is needed. Therefore, we propose to determine classes of nodes in the whole dataset based on their expanse measure, which will allow us to objectively evaluate the expanse of cell agglomerations in a given tissue. Therefore, we use an unsupervised machine learning technique known as k-means clustering algorithm [START_REF] Hartigan | Algorithm AS 136: A k-means clustering algorithm[END_REF] with the aim of partitioning all nodes representing histological structures in the dataset into k clusters, in which each observation belongs to the cluster with the nearest mean.

Since the choice of the number of clusters k for k-means algorithm is somehow subjective, we apply the Elbow method which is a technique of interpretation and validation designed to help finding the appropriate number of clusters in a dataset. First of all, we compute the sum of squared error (SSE) for different numbers of clusters k ∈ 2, 3, ..., 10. The SSE is defined as the sum of the squared distance between each member of the cluster and its centroid using this formula:

SSE = k i=2 x∈Ci dist(x, µ ci ) 2 (5.4)
with C i is the set of observations associated to cluster number i and µ ci is the mean (centroid) of cluster C i .

Then, when we plot k against the SSE, we will find that the error decreases as k gets larger; this is because when the number of clusters increases, the distances members to centroids get smaller. The idea of the Elbow method is to choose the k from which the SSE decreases slowly, known as "elbow effect". The important number of nodes to be classified allows us to conduct a multi-fold analysis, where the set of nodes of the whole dataset is randomly mixed and partitioned into 100 groups. The k-means classification algorithm and the Elbow method are then applied to each group separately. The plot of the number of clusters k against the average SSE over the 100 groups is given in figure 5. For each case from the dataset, the percentages of the expanse classes of each of the tissue components are then determined in order to make a qualitative comparative analysis between different cases. These percentages are given in figure 5.15. In order to evaluate the points of similarity and difference between the proposed approach and the typical density method (refer to section 5.5.1), we make a qualitative comparative analysis between both approaches. Qualitative results given by our approach can be found in figures 5.16.(d-f) for the three cases. While results obtained by the density method are given in figures 5.16.(g-i). As can be seen, the density method provides local features that describe the size of the population of cancer cell nodes within a window frame, independently from the actual size of the agglomeration formed by these cells. While, in contrast, our approach, based on the morphological expanse function, provides a global description of cell agglomerations in a continuous manner, which in addition offers a better comprehensive visual presentation allowing finer interpretations. The features presented until now have the ability to characterize the expanse of cell agglomerations based on morphological parameters. However these features do not describe how these agglomerations are distributed relative to each other in terms of sparsity and dispersion. In the following we propose new features able to describe such characteristics.

Sparsity

Cell agglomerations, regardless their size and density, they may have different relative spatial positioning. For example, cancer cell aggregates, large or small, they can be found far or close from each other. In addition, the aggregates of cells can be enclosing or surrounding other cells of another type, like immune cells enclosing cancer cells for example. Studying the relative positioning of cell agglomerations can be performed by first identifying these agglomerations as connected components (using graph-based connected component labeling), and then evaluating the relative pairwise distance between the centroids of these regions. Such an approach is subject to a number of errors related to connected component labeling (as explained in section 5.5.1). In addition, the centroid of an agglomeration does not define effectively the geographic location of the agglomeration regarding the various shapes and sizes that it can take. Graph-based Mathematical Morphology

In this study, we propose a different approach based on a morphological function that characterizes the relative positioning of a tissue component X based on parameterization of nodes of the opposite tissue components, which is the complement of the set X, denoted X or V \ X, where V is the set of all nodes in the tissue. In fact, the information relative to sparsity, dispersion or closeness of a set of nodes X is provided by nodes that are not in X, since they define the space unoccupied by X. Therefore, for a given tissue component X, we apply the closeness function ζ proposed in 3.2.4 to the binary graph G X defined by nodes of X, in order to study the values of ζ in the set X.

Similarly, before applying the closeness function, a prior improvement of the output of the node identification algorithm is needed. The closeness function is more likely to be affected by improper foreground nodes that are dispersed within background nodes, since it is derived from closing operations which tend to strengthen the connectedness between foreground agglomerations. Hence, we apply a morphological opening of size r o to the binary graph before applying the closeness function ζ. Morphological features are then derived from these operations with the aim of defining classes of sparsity for each histological structure. The flowchart of our method for the characterization of the sparsity of the tissue components under consideration, using our morphological approach is shown in figure 5.17. As can be seen, given a fixed size of opening r o , the closeness function take high values, within, around and between cancer cell aggregates. In addition, nodes of the set CCN located between close (near) CCN aggregates have higher values than that found between distant aggregates. Moreover, the larger the size of the prior opening operation γ ro , the higher is the maximal value of the subsequent closeness function ζ. This is due to the fact that the opening operations reduce the amounts of dispersed foreground nodes (CCN nodes in this case) between the actual significant aggregates, which implies higher distances between the significant aggregates. Thus, the convergence of the closeness function is achieved in a higher number of iterations, which implies a higher maximal value of the function ζ. It should be noted that a very large r o value, may completely remove the cell aggregates, and thus, the processing will not be adequate

Inter-tumor heterogeneity analysis

To allow comparison between different cases based on the sparsity of the tissue components (e.g. CCN , ICN , F CN or AT N ), we propose to determine classes of nodes over the whole dataset using their Γ-values. For each tissue component X, we perform a classification of the nodes of the complement set X, based on the values of Γ(G X ) using the k-means algorithm and the Elbow method, in the same manner as described in section 5. For each case in the dataset, the percentages of the sparsity classes of each of the tissue components are then determined in order to make a comparative analysis between different cases. These percentages are given in figure 5.22. In the following we will explain links between these characteristics and visual interpretations from the tissue based on some examples. In figure 5.23, we give qualitative results of three cases from the dataset and we compare them with the result in figure 5.22. These cases are N • 8, N • 32 and N • 39. As can be seen in case N • 8, the tumor patterns are widespread almost every where in the inner area of the ROI, which is correlative with the presence of a significant percentage of CCN nodes of class 5 (orange). Whereas, in the cases N • 32 and N • 39, the tumor patterns are mostly distributed towards the boundary ROI, living the inner are for other histological structures (collagen and immune cells in these cases). This can be described by the fact that there is high percentage of CCN nodes of class 1 and 2 together (blue and cyan).

Quantitative analysis of the spatial interactions between histological structures

Until now we have presented quantitative features that characterize the spatial distribution, in terms of expanse and sparsity, of each tissue component individually. As explained above, it is of interest to study the spatial interactions between tissue components of different types. In this section, we will focus on the spatial interactions between cancer cells and the other tissue components (i.e. immune cells, collagen, adipose tissue) based on the expanse properties of cancer cells and the sparsity properties of the other tissue components. The main goal of this part of our study is to characterize cancer cells by two parameters simultaneously: one that describe the size of the aggregate that they form, and another that describe the degree of encloseness by other tissue components. The degree of encloseness of cancer cells by another tissue component X (i.e. ICN , F CN , AT N ) is obtained by the sparsity function Γ(X) applied to the binary graph defined by nodes of X, since the values of Γ(G X ) describe also how nodes of X enclose nodes of X, in addition of describing the sparsity of X. Therefore, each node v ∈ CCN is parameterized by two numerical values: the value of the expanse function applied to the binary graph defined by cancer cells Σ(G C )(v),

Cancer-immune cell interactions

The clusters provided by k-means algorithm in figure 5.26.a define 6 classes of cancer cells based on their clustering and their positioning relative to immune cells: 1) in deep blue, are CCN nodes that are small, having low expanse and highly enclosed by ICN nodes. 2) in light blue, are CCN nodes that are small and found between immune cell aggregates. 3) in cyan, are CCN nodes that are small but having low spatial interactions with ICN nodes. 4) in green, are CCN nodes having moderate size and found in area of interaction with ICN nodes (they can be close or between distant aggregates of immune cells). 5) in yellow, are CCN nodes having medium size and low spatial interactions with immune cells. 6) in orange, are CCN nodes that are large, having high expanse and low spatial interactions with immune cells.

For qualitative evaluation, few examples that exhibit these different spatial configurations are given in figure 5

Cancer-collagen interactions

The 4 classes of cancer cells defined based on their properties of clustering and positioning relative to collagen tissue, are presented through an example in figure 5.26.b, and they indicate:

1) in deep blue, cancer cells with low expanse and highly enveloped by collagen fibers. 2) in cyan, cancer cells that have low-medium expanse and barely surrounded by collagenous tissue. 3) in yellow, cancer cells with low-medium expanse and found between or surrounded by collageneous tissue. 4) in red, cancer cells with large expanse and having low spatial interaction with collagen.

Cancer-adipose tissue interactions

Similarly, the 6 classes defined for cancer cells based on their degree of clustering and spatial interactions with adipose tissue (in figure 5.26.c) represent: 1) in deep blue, CCN nodes that are small and growing within adipose tissue.

2) in light blue, CCN with low expanse and found between groups of adipocytes.

3) in cyan, CCN with very low expanse and very low spatial interaction with adipose tissue. 4) in green, CCN with medium-high expanse and found between clusters of adipocytes. 5) in yellow, CCN having medium size and low interaction with adipose tissue. 6) in orange, large CCN aggregates that have low interaction with adipose tissue.

Qualitative result of the classification can be found in figure5.29.

Inter-tumor heterogeneity analysis

In order to characterize breast cancer tissue sample based on the spatial distribution of cancer cells, and their interactions with other components form the microenvironment, we propose to calculate the percentage of cancer cells in each of the classes of distribution/interactions defined above. These percentages are given for each case in our dataset in figure 5.30. The idea behind calculating, for each case, the percentage of cancer cells belonging to each distribution/interactions class, is to use these percentages as descriptors in order to perform a discriminative analysis based on these class-percentages. Each case is then described by 16 (6 + 4 + 6) features expressing the spatial interactions of the tumor with its microenvironment (refer to figure 5.30). Therefore, we apply a statistical procedure called principal component analysis (PCA) [START_REF] Jolliffe | Principal component analysis[END_REF], which uses an orthogonal transformation to the data to reduce the number of random variables under consideration (16 features in our case), by converting the possibly correlated variables into a set of linearly uncorrelated variables, called principal components. PCA converts the data in such a way that the variance of the new variables is maximized; the first principal component has the largest possible variance, and each succeeding component has the following highest possible variance. In our study, we limit ourselves to the first 3 principal components provided by the PCA applied to the 16 features proposed above. A presentation of the data (55 patients) in the feature space defined by the first 3 principal components is given in figure 5. However, the dataset is not large enough (55 cases) to make any conclusion about the number of clusters that can be found in histopathological data based on the spatial interaction features. Graph-based Mathematical Morphology

Conclusion and Perspectives

It has long been acknowledged that cancer is a heterogeneous cellular disease marked by its diverse morphological and phenotypic profiles. More recently it has become widely accepted that the development of cancer is influenced by a combination of multiple microenvironmental factors regulated by different cell types having different biological roles. The whole of these elements is referred to as the tumor microenvironment (TME). The spatial interactions between cancer and the different entities in the TME, are to date not well understood. Cancer histopathology slides are composed of a tremendous number of histological structures of different types irregularly distributed in the tissue which make their spatial content indescribable in a simple manner. Computerized image analysis techniques, combined with medical knowledge, can contribute to a more comprehensive understanding of cancer by exposing important tissue aspects that are impossible to carry out visually. From the image analysis techniques that have been used for this purpose, graph-based methods

Conclusion and Perspectives

123 have proved their ability as potentially very powerful tools for characterizing tissue architectures. However, there is a need for exploring new approaches in the extraction of parameters from graphs to derive such information, while taking into consideration of the heterotypic spatial interactions in the TME. In this chapter, we have explored a new approach based on mathematical morphology on graphs in order to extract new tissue-derived parameters describing the spatial distributions of histological structures and their spatial interactions. Two morphological graph transforms have been established and their characteristics were studied on a dataset composed of a large number of histological structures. One of the two morphological transforms, namely the expanse (openness) function, was compared to spatial density, and they were found to be correlated, even though the expanse function is not directly derived from measurements in the metric space (e.g. distance, surface). The combination of the two morphological functions applied to different histological structures provides a powerful tool for characterizing their spatial interactions/relative positioning in terms of encloseness, envelopment and betweenness. Furthermore, the morphological distance function can be applied to study the relative distance between histological structures [START_REF] Cheikh | Spatial interaction analysis with graph based mathematical morphology for histopathology[END_REF]. It has to be said that the proposed approach was applied to study the morphology of tissues from whole slide images (WSIs), which has not been the case in previous studies, and which also makes this approach to be in line with the recommendations in the review on the topic [START_REF] Sharma | A review of graph-based methods for image analysis in digital histopathology[END_REF], where the need for developing analysis techniques applicable to WSIs and not only to some specific regions was expressed.

The exploration of a new research topic like the heterogeneity of the tumor microenvironment by the application of a new image processing methodology on unlabeled data, accompanied by complex information and inappropriate artefacts, represent the main challenges that meet this framework. In fact, the effectiveness of the proposed morphological analysis relies on the robustness of the automatic algorithms employed for the identification of histological structures (e.g. nuclei detection/classification, collagen/adipose tissue segmentation) from tissue images, which is still limited. The proposed theoretical framework for spatial point pattern analysis using graph-based mathematical morphology has not been widely explored in its full context. The morphological transforms have been applied exclusively on the Delaunay graph and have not been tested on other graphs defined by other linking rules. And in addition, the morphological features extracted from the graph have not been compared to a ground truth provided by a medical expert. The current analysis is based mainly on unsupervised learning and derivation of class labels from unlabeled data, guided by visual and narrative interpretations rooted in morphological aspects.

For future works in this direction, we suggest that the current morphological approach should be explored in greater detail and validated on synthetic datasets simulating the spatial organization of histological structures in tissue. A comparative analysis of the respective outputs of the morphological transforms applied to different graphs and the associated characteristics is needed in order to develop a better knowledge about their relevance. The histopathological significance and the clinical implication of the proposed morphological features of tissue should be studied within a framework including a large annotated dataset. More spatial aspects have to be discussed further, such as the orientation of collagen fibers relative to the tumor cells (TACS) (refer to section 1.2.3). Such issue may be addressed by adopting anisotropic morphological operations that take into account directional dependencies in their formula (equation 3.8). The dynamic of the morphological operations like erosion and dilation, suggests to study the dynamic of tumor growth with tools based on morphological features analogous to the ones proposed in this work. The morphological features that can be extracted from graphs under a longitudinal study may have the potential to describe and characterize cell population growth by considering their neighborhood relationships.

Chapter 6

A simulation model of the Spatial Organization of Histological Structures

Introduction

Developing a spatial model of tumor architecture and spatial interactions with TME can advance our understanding of tumor heterogeneity. Furthermore, generating histopathological synthetic datasets can contribute in validating and comparing analytical methods that are used in digital pathology. In this work, we propose a modeling method that applies to different breast carcinoma subtypes and TME spatial distributions based on combination of classical mathematical morphology and statistics. The model is based on a few morphological parameters that give access to a large spectrum of breast tumor architectures, and they are able to provide architectures like in-situ ductal carcinomas (DCIS) and histological subtypes of invasive carcinomas such as ductal (IDC) and lobular carcinoma (ILC). In addition, a part of the parameters of the model controls the spatial distribution of TME components relative to the tumor. The validation of the model has been performed by comparing morphological features between real and simulated images.

Few frameworks for synthetic histopathology image generation have been proposed in the literature. A model for the spatial microenvironment of healthy and cancerous colon tissue was proposed in [START_REF] Kovacheva | A model of the spatial tumour heterogeneity in colorectal adenocarcinoma tissue[END_REF]. The model is designed to generate synthetic Haematoxylin and Eosin (H&E) images with parameters that allow control over cancer grades based on crypt sizes, cellularity, cell overlap ratio, and lumen texture. The model was first introduced for simulating healthy colonic crypt microenvironments in immunofluorescence images [START_REF] Kovacheva | A model of the spatial microenvironment of the colonic crypt[END_REF]. Another set of methods based on the Cellular Potts Model (CPM) has been used for tumor modeling [START_REF] Szabó | Cellular potts modeling of tumor growth, tumor invasion, and tumor evolution[END_REF]. In this model, a cell is analogous to a spin in Potts models in physics [START_REF] Graner | Simulation of biological cell sorting using a two-dimensional extended Potts model[END_REF]. The CPM works on a rectangular Euclidean lattice where it represents each cell as a subset of lattice sites. The model then uses Metropolis-Hastings algorithm, which iteratively attempts to copy a random cell in a random adjacent lattice site. Eventhough this model allows precise simulation of many phenomena, such as cell migration, clustering and growth, by taking into account adhesive forces, environment sensing and volume/surface-area constraints, it ignores complex structural features found in real tissue, such as tubule-shaped glandular structures, Indian-file cell migration, or the likely effects of competitive interactions between different components in the tumor microenvironment, such as tumor-associated collagen.

To the best of our knowledge, our model is the first to simulate breast tumor architecture at the tissue level by taking into account microenvironmental factors. The model is capable of simulating a large range of tissue types, close enough to reality (according to partner pathologist), with few parameters. The next section provides a description of the proposed approach for modeling breast tumor tissues. Then, a description of the experiments and the results of this study are given. Finally, a conclusion summarizes our main contribution and future works.

126 Chapter 6. A simulation model of the Spatial Organization of Histological Structures

Method

Overview

Our approach for simulating breast tumor tissue images is based on compound mathematical morphology tools, since these tools have the ability to design objects and spatial interactions according to topological concepts. The model can be divided into two main modules (see figure 6.1). The first module deals with the architecture of the tumor regardless the TME components and it generates tumor growth patterns, that are defined by aggregates of epithelial cells in a real tissue. The second module concerns the spatial arrangement of TME components relative to the tumor patterns that were generated during the first step. Even though TME is composed of a wide range of biological components, in this study we restrict the second module to model collagen fibers' organization and the spatial distribution of immune cells. In order to simulate tissue images that cover significant histological structures, the size of the simulated images is set to 2048 × 2048 at 0.5µm/pixel resolution.

In the following, given a binary image I, we denote D b (I) the morphological dilation (respectively, E b (I) the morphological erosion) of I with a structuring element b [START_REF] Serra | Image Analysis and Mathematical Morphology[END_REF][START_REF] Cousty | Some Morphological Operators in Graph Spaces[END_REF]. Let {x i } 1≤i≤n be the set of the binary objects that are in I, we denote δ B (I) the morphological dilation (respectively, B (I) the morphological erosion) of the n binary objects {x i } with the set of structuring elements B = {b i } 1≤i≤n . In other words, each binary object x i is dilated (or eroded) individually with the structuring element b i , then the results are added to a unique binary image. A disk-shaped structuring element (SE) of radius r is denoted Φ r . And a line-shaped structuring element of length l and orientation θ is denoted by ∆ l,θ .

We denote U n (a, b) a set of n random values generated from a discrete uniform distribution on a, b , and N n (µ, σ) a set of n random values generated from a normal distribution with mean µ and standard deviation σ [START_REF] Evans | Statistical distributions[END_REF]. We define the function P n,dmin : I → J that generates a binary image J with at most n random white pixels, such that the pixels' coordinates are inside the white surfaces of the binary image I and with a minimum distance d min from each other.

Tumor pattern generation

The first characteristics that a pathologist inspects during the examination of a breast tissue specimen are the shape and the spatial distribution of the tumor patterns (TP), that are defined by clusters of cancer cells having various spatial arrangements. The histopathological classification of breast cancer is based mainly on the spatial appearance of these growth 6.2. Method 127 patterns (refer to section 1.2.2). In this work, we expose our model through four types of breast carcinomas: Ductal Carcinoma In Situ (DCIS), where tumor cells are found inside, and not moved out of, the mammary duct. Two subtypes of invasive ductal carcinomas; 1) medullary carcinoma (MC), where the tumor pushes against the surrounding stroma but stays relatively circumscribed, and 2) tubular carcinoma (TC), which is made up of tubeshaped structures. And Invasive Lobular Carcinoma (ILC), where cancer cells tend to infiltrate as individual rows known as 'Indian files'.

In order to design tumor patterns, we generate in a blank image I 0 , n 0 random white pixels with a minimum distance d 0 from each other, using the function P n0,d0 . These white pixels are the initial binary objects representing the tumor seeds, from which tumor patterns will be created (grown). Each white pixel x i is dilated by a disk Φ ri of a random radius r i , then by a line ∆ li,θi of a random length l i and a random orientation θ i . The overlap between the growing seeds during the dilation process is controlled by a Boolean parameter ω. When ω = 0, the overlap is not allowed, the intersection pixels are associated to one of the touching objects, chosen randomly, and the objects are then separated. While when ω = 1, the dilation operation allows merging objects. In the following, we denote δ ω B the morphological dilation with the set of structuring elements B, with overlapping control ω (see figure 6.2). In a real tumor tissue, tumor patterns may show empty surfaces inside that do not contain tumor cells, like lumen of tubules or glands, or necroses. Therefore, holes are created inside the tumor pattern objects by the mean of morphological erosions and subtractions: A morphological erosion is applied to each object with a disk Φ hi of random radius h i , and the result is then subtracted from the previous image. It should be notet that if h i = 0, then the erosion is not applied and the tumor pattern x i is not hallow. Analytically, the binary mask M T P of the tumor patterns is given by the following operations:

M = δ ω ∆ L 0 ,Θ 0 δ ω R0 (P n0,d0 (I 0 )) (6.1)
M T P = M -Φ H 0 (M ) (6.2)
where I 0 is a blank image. R 0 , L 0 , Θ 0 and H 0 are sets of random variables corresponding to the structering elements properties. An interpretation of the parameters is given in figure 6. Finally, the boundaries of TPs are refined by generating random white pixels around the objects, followed by alternating sequential filtering in order to make the boundaries "less linear" and more realistic.

Tumor Microenvironment Generation

Collagen fibers

The second aspect we are interested in, is the density and the orientation of collagen fibers relative to the tumor growth patterns. The motivation comes from studies that have shown that the invasion of breast tumors is influenced by collagen organization at the tumorstromal interface. In fact, the authors of [START_REF] Provenzano | Collagen reorganization at the tumor-stromal interface facilitates local invasion[END_REF][START_REF] Matthew | Aligned Collagen Is a Prognostic Signature for Survival in Human Breast Carcinoma[END_REF], have defined three Tumor-Associated-Collagen-Signatures (TACS): i. TACS-1, which is a limited collagen density localized around small tumor foci.

ii. TACS-2 characterized by stretched collagen fibers tangentially oriented along a smooth tumor boundary.

iii. TACS-3 characterized by collagen fibers aligned perpendicularly to an irregular tumor boundary and oriented in the direction of cell invasion.

More explanations are given in section 1.2.3. Some examples of different TACS configurations are given in figure 6.4. Hence, we incorporate in our approach parameters adapted to these exposed configurations. In order to introduce in our model an aspect of interactions between tumor patterns (TP) and collagen fibers (CF), the curves of the latter are generated from the reshaping of the objects of the first and their associated distance maps. The reshaping is made in two different fashions: i) To generate waved lines tangentially oriented along with the boundary of a binary object, we first generate in the image M T P , m×N random white pixels around the objects at a distance γ from the TP boundaries, where N is the total number of pixels at the distance γ. Then, we apply a morphological closing E • D Φγ . This operation produces deformations and reshaping of the TP objects. The lines corresponding to the collagen fibers are obtained from the subsampling ↓ d C of the distance map of the reshaped objects. The parameter d C controls the number and the distance between fibers, and thus the density. While the parameters m and γ control the stretchiness of the fibers. The whole process is repeated iteratively, such that at each iteration i, M T P ← D Φi * γ (M T P ), until the whole image becomes white.

ii) To generate aligned collagen fibers oriented in the direction of the cell invasion, we use the same technique but we apply a different reshaping method. Instead of generating random segments radial to the object boundary, we generate a straight line passing through the object center and oriented in the same direction as the object. An illustration of two collagen fiber configurations is given in figure 6.5. Some examples of simulated collagen fibers are also presented in figure 6.6. Therafter, fibroblasts coordinates were chosen randomly from pixels of the generated curves: P F = P n F ,d F (C i ), where C i is the collagen curve obtained at iteration i.

Immune cells

It has become obvious that the inflammatory microenvironment (iTME) has a great impact on the tumor progression and behavior. Many studies have shown links between the clinical outcome and the immune cell presence, the relative abundance, as well as the spatial proximity to cancer cells (refer to section 1.2.3). Some samples showing different spatial organizations of immune cells relative to the tumor patterns are given in figure 6.7. To that end, we introduce in our model parameters that control the density and the proximity of immune cells to the tumor growth patterns. Let k be the number of layers in the discrete distance map of the binary image M T P that contains the tumor patterns as defined in equation 6.2, we calculate k coefficients λ 1 , λ 2 , . . . , λ k , such that λ i = i -α k i=1 i -α , ∀i ∈ 1, k .

The coefficient λ i is the ponderation coefficient associated to the number of immune cells to generate in the layer i of the distance map (an example is shown in figure 6.8). In each layer i we generate λ i × n L random immune cells with a minimum distance d L from each other, where n L is the desired number of immune cells. If α > 0 then λ i < λ i-1 , ∀i ∈ 1, k , which means that immune cells are mostly generated in the first layers of the distance map, and thus, collected around the tumor patterns (see example in figure 6.9.a). If α < 0 then λ i > λ i-1 , and consequently immune cells are mostly generated in the last layers of the distance map, away from the tumor-stromal interface (see example in figure 6.9.b). And finally, if α = 0 then λ i = λ j = 1 k , ,∀i, j ∈ 1, k and the spatial distribution is uniform (see example in figure 6.9.c).

Refinement

Although this study is dedicated to the architectural modeling of breast tumors and tumor microenvironment spatial configurations, we are also interested in the esthetic aspect of the simulated images. Therefore, we have added a third module to draw the simulated tissue components based on ingredients extracted from real images of HES-stained breast tumor tissue. Patches of nuclei of different cell types, i.e. epithelial cell nuclei, lymphocyte nuclei and fibroblasts, were extracted, randomly rotated, and then pasted into the simulated image at their corresponding locations. In particularly, fibroblast patches are oriented in the same direction as their corresponding collagen fibers. Patches containing collagen and tumor patterns were also extracted. Then, in each simulation, pixels are randomly selected from the patches, and their colors are set to the corresponding component in the simulated image. Tiny white surfaces were also randomly generated to simulate the tearing that commonly appears in real tissue images. And finally, gaussian noise and median filter are applied consecutively (see results in figure 6.10). In addition in figure 6.10.f shows an example of simulated medullary carcinoma, where a thin mucinous layer was added around the tumor patterns, using mathematical morphology operations (another dilation of TP). The approach may provide other options such as simulating necrosis or artifacts.

Experiments and Results

In order to assess the effectiveness of our model, we compare our simulation results to real data based on morphological features. The number of tumor patterns (TP), their area, their major axis and minor axis lengths, their solidity and the coefficient of variance (CoV) of their orientations were calculated from the four types of breast carcinoma and compared to the modeled structures. A comparison between results of 80 images (20 of each type) and 32 images from real data is shown in table 6.2. The calculated feature values are relatively close between the simulated data and the real data within the same group type. Qualitative results, obtained with parameters in table 6.1, are shown in figure 6.10.

Conclusion

In this chapter, we have presented a model for simulating different breast carcinoma architectures and various tumor microenvironment spatial configurations based on compound mathematical morphology. The validation of the model has been performed by comparing morphological features between real and simulated images (more examples of simulated images are given in appendix B). The model gives promising results showing spatial aspects that are closely related to reality of tissues based on few parameters.

In the future, we will extend the model to generate synthetic whole slide images, explore more parameters of the model to simulate other subtypes of breast tumor architectures, but also tumors from other organs (e.g. colon, prostate, lung) and consider more microenvironmental components (e.g. adipose tissue and blood vessels). Moreover, the simulation model presents a valuable tool to generate a large synthetic data for validating and comparing new methods for histopathological image analysis. In future works, we will validate our spatial point pattern analysis method proposed in chapter 3 on synthetic dataset of whole slide images and compare it to other methods such as spatial statistics.

Chapter 7

Overall Conclusion and Future Perspectives

Conclusion

The introduction of digitized whole slide images (WSI) has opened up new interesting prospects in the fields of histopathology and cancer diagnosis. Considerable efforts have been made to explore new approaches to automate the process of analyzing digital histopathological images using a wide range of image processing techniques. Several studies have been carried out in order to provide new quantitative tissue-derived parameters exposing different facets of cancerous tissues that are impossible to carry out visually. However, there is still a lack of tools that allow the description of the spatial arrangements of different cell types. Since it was recognized that the tumor microenvironment (TME) configuration has a valid impact on tumor behavior and possible treatments, the question of characterization of the TME with quantitative features remains a subject of discussion among researches in the field. In fact, to date, there is no comprehensive approach to model the TME and to determine relevant feature measurements that enable accurate differentiation between various forms of TME. Roughly speaking, there is a problem in measuring, interpreting and modelling of pathological tissues, which may contain a very large number of cells of different types with irregular spatial distributions indescribable in a simple manner. In this thesis, we focused attention on such issue by studying and discussing different paths in applying quantitative evaluation of the spatial arrangements of histological structures, and by exploring a new approach based on mathematical morphology on graphs.

In the first chapter, we have presented an introduction to the field of digital pathology, as well as the important role of image analysis in histopathology by focusing on breast cancer as a pathology. A description of glass slide preparation procedures and Whole Slide Imaging is given, and underlined by the variety of tissue staining protocols and the technical characteristics of histopathological virtual slides. The main notions in breast cancer histology were presented, and emphasized with concepts of staging, grading and subtyping, as well as perplexing questions about the heterogeneity of the tumor microenvironment. Then, a brief overview on histopathological image analysis applications (i.e. cancer diagnosis, nuclei or structures detection, segmentation and classification) is provided, and it divides cancer diagnosis techniques into nine categories. The chapter concludes by introducing the motivations of the thesis, which are to explore the potential of graph-based methods to quantify the architectural characteristics of tumors and to study their interactions with the TME, with a priori favorable to the exploitation of methods derived from mathematical morphology.

The second chapter reviews the methods of the state-of-the-art that exploit neighborhood graphs for the treatment and analysis of histological images. The basic concepts and notions of graphs are first recalled and the neighborhood graph reconstruction is detailed. Then, a comprehensive account of the various graph-based methods developed for spatial analysis in histopathology is given. In these methods, we have first discussed their inference, by focusing on the nodes that represent the histological objects, whose spatial distribution is studied, and the edges which depend on the type of the neighborhood graph considered. Then, we have concentrated our analysis on the methodologies used in order to extract features from graphs, and we have divided these feature extraction methods into five categories (i.e. Syntactic Structure Analysis, Network Analysis, Spectral Analysis, Persistent Homology and Mathematical Morphology). Subsequently, we have summarized these studies by indicating their key elements (e.g. tissue type, histological objects, graph type, features and objectives), and concluded that there is a lack of feature extraction methods that allow the characterization of the spatial interactions in the TME.

The third chapter presents the theoretical angle of our contribution to resolving the problem by exploiting signal processing on graphs using mathematical morphology for the analysis of the spatial arrangement of points in two-dimensional space. An overview on graphbased mathematical morphology is presented and we restricted ourselves to those considering graphs with labeled nodes. The basic morphological operations performing on graphs (erosion, dilation, opening and closing) are exposed, and two new operators are proposed for binary graphs and named openness and closeness. They assign to a node the minimum size of the structuring element of an opening (or of a closing) to activate (or deactivate) the node. These two morphological transformations are then exploited to propose an analysis of the spatial organization of groups of nodes of the same label and the spatial relations between two groups of nodes of different labels. The analysis have focused on concepts of "expanse" and "dispersion" to characterize the spatial distribution of a set of population of objects modeled by Delaunay graph, and "encloseness", "betweeness" and "freedom" to characterize the spatial interactions between sub-populations of objects. With respect to the fact that image pixels may be interpreted as a special subtype of a graph, all the functions and concepts may be used for arbitrary images as well. The framework presented in this chapter suggests to introduce Mathematical Morphology on graphs in Spatial Point Pattern Analysis.

The fourth chapter concerns the identification of nodes that represent different histological structures: cell nuclei (i.e. cancerous, immune and fibroblast) and biological structures (i.e. collagen fibers, adipose tissue), in whole slide images of breast cancer tissues. For this purpose, pre-processing is first performed on the slides with the definition of a region of interest, color normalization and superpixel segmentation. The potential nuclei are identified at the superpixel level by a morphological reconstruction of a binary image obtained by fixed thresholding of the hematoxylin channel resulting from a color deconvolution of the HES image. Using a reference database, the parameters are optimized by ROC analysis and evaluated performance have reached a correct F1-score but with a low precision, which suggests a high rate of false positives. The identified nuclei are then characterized by multiscale statistical measurements based on color and texture from different image channels. Since the number of extracted features is important, a feature selection is carried out based on Fisher criterion upstream of a feature learning. A random forest classifier is then used to classify nuclei into the three types. Finally, collagen fibers and adipose tissues are identified by simple ad-hoc thresholding methods at the superpixel level, the precision of which is not evaluated. However, the results presented in this chapter are based on classical methods, their quality is adequate to identify nodes enabling the study of the spatial distribution of histological structures using graphs.

The fifth chapter concerns the characterization of the spatial organization of histological structures with the graph-based mathematical morphology approach presented in chapter three. A neighborhood graph is constructed from the nodes identified in the previous chapter by Delaunay triangulation and followed by alpha-shape filtering. To remove the residual noise from the labeling errors, a morphological filtering by sequential alternating filter is carried out on the labels. In order to evaluate for each histological structure its expanse in the tissue, we have applied the openness function to filtered versions by closing of increasing sizes of binary graphs of the type of the structure and calculated the average, called expanse function. The intra-tumor heterogeneity, of cells of the same type in the same tissue, is then measured by the entropy of the expanse function and the inter-tumor heterogeneity where each node is divided into 5 classes according to its expanse-value by k-means. The same type of analysis is performed with the closeness function to obtain a sparsity function. Then, the analysis focused on quantifying the interactions between cancer cells and other histological structures on the basis of expanse and sparsity measurements. The goal is to know whether the cancer cells are in aggregates and how close they are to other structures of the TME. Again, a k-means is used to subdivide the cells and analyze their interactions. The proposed analysis are interesting, but their contribution is difficult to evaluate since the analysis is purely qualitative and prospective and there is a lack of validation to verify the relevance of the derived interpretations.

The sixth chapter deals with the generation of synthetic histopathological images. Based on mathematical morphology tools, we have developed a model for simulating different breast carcinoma architectures and various tumor microenvironment spatial configurations. The method is based on three steps: first generate tumor patterns, then spatially arrange TME patterns relative to the tumor patterns, and finally generate points within the corresponding patterns. Since automatic learning methods require a lot of data, generating histopathological synthetic datasets can contribute in validating and comparing analytical methods that are used in digital pathology. In addition, the proposed model gives promising results showing spatial aspects that are closely related to reality of tissues based on few parameters. The validation of the model has been performed by comparing few morphologic and geometric features of tumor patterns from real and simulated images, but the spatial distribution of simulated points has not been compared to real data.

In conclusion, through this work, we contribute towards solving the problem of measuring, interpreting and modelling of diseased altered histological tissue, by providing new tools able to describe the spatial organization of histological structures and that take into account the heterotypic interactions in the tumor microenvironment. The proposed tools have been explored on a dataset composed of breast cancer tissue slides, and they have been applied to analyze histological structures from "complete" whole slide images, which presents one of the primary limitations of state-of-the-art methods. However, the proposed theoretical framework for resolving the problem has not been examined in its full context. The morphological transforms have been applied exclusively on an alpha-shape-filtered version of Delaunay graph and have not been tested on other graphs. And the morphological features extracted from the graph have not been compared to ground truth annotations about the spatial aspects. The current analysis is still based mainly on unsupervised feature learning for the sake of finding some intrinsic class labels from the unlabeled data.

Perspectives

The study and analysis of the spatial arrangement of points in the space arise in several contexts and applications. Future works should give deeper knowledge and understanding about the proposed morphological approach for spatial point pattern analysis. The current morphological functions and features should be explored in greater detail and validated on simulated data of spatial point-sets with various spatial configurations, and more spatial aspects than relative distance, encloseness and expanse have to be considered and modeled. The proposed functions should be evaluated on other graphs than Delaunay (see figure 7.1), and the features extracted from the different graphs should be compared to each other in order to develop a better apprehension of their applicability and validity. The validation and assessment should be carried out by comparison with features derived from other spatial analysis approaches such as syntactic structure, network or spectral analysis, and spatial statistics. Such analysis would give deeper knowledge about the usefulness of the proposed morphological functions and their adaptation to practical problems. As in the majority of the applications related to this problem inevitable errors may occur due to imprecise node detection and/or misclassification, the effectiveness of the approach and the stability of the functions against these errors should be assessed. To carry this out in an adequate way, the variations of the features should be studied relative to artificial alterations of the locations and/or labels of spatial points in simulated data. Probabilistic methods related to node identification (about locations and labels) have to be explored and integrated into the model, and hence, the adaptation of the morphological functions to probabilistic information in graphs have to be studied. Deep learning techniques have to be exploited for learning of multiple levels of features from the representation of the spatial data given by the proposed morphological functions. Furthermore, the spatial analysis techniques studied in this work have to be explored in three dimensional space, and the spatial point simulation model have to be extended to higher dimensional spaces, including time-evolution of spatial point patterns. Quantifying the spatial distribution of histological structures and the heterotypic interactions in the tumor microenvironment using Spatial Point Pattern Analysis techniques is a crucial step towards a better understanding of tumor heterogeneity. In future works, the histopathological significance and the clinical implication of the proposed morphological features of tissues should be studied within frameworks including large annotated datasets. New quantitative and reproducible tissue metrics should be explored and their relationships should be examined, not only relative to grading and subtyping, but also, molecular profiling and genomic data. In addition, the histological spatial features should be compared with other image-based features derived from other medical image modalities such as quantitative ultrasound or elastography. Furthermore, the morphological approach, by taking advantage of the dynamic of the incorporated morphological operations, should be explored in studying the dynamic of tumor growth in longitudinal analysis, following cohorts over an extended period, and by characterization of cell population growth by considering their neighborhood relationships on simulated data. The spatial analysis should be carried out on histopathological image modalities other than HES-stained tissue images to study concepts of tumor heterogeneity other than the tumor microenvironment. Such studies could be targeted to the spatial arrangements of positive, relative to negative, cell nuclei in immunohistochemistry IHC-staining (cf. fig.

A.1.2 Expanse of collagen fibers and fibroblasts

A.1.3 Expanse of adipose tissue

3. 1

 1 Morphological dilation of a graph . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Morphological erosion of a graph . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Morphological closing of a graph . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Morphological opening of a graph . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Alternate Sequential Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Geodesic reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Distance function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Skeleton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.9 Openness function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.10 Closeness function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.11 Example of image processing problems related to point-set spatial configuration modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.12 Example of spatial point patterns . . . . . . . . . . . . . . . . . . . . . . . . . . 3.13 Aggregate and clustering analysis using mathematical morphology . . . . . . 3.14 Aggregate detection using mathematical morphology . . . . . . . . . . . . . . 3.15 Sparsity analysis using mathematical morphology . . . . . . . . . . . . . . . . 3.16 Quantification of the dispersion of point patterns . . . . . . . . . . . . . . . . . 3.17 Encloseness analysis using mathematical morphology . . . . . . . . . . . . . . 3.18 Encloseness evaluation based on closeness function . . . . . . . . . . . . . . . 3.19 Relative distance evaluation using mathematical morphology . . . . . . . . . 3.20 Quantification of relative distance based on the distance function . . . . . . . 4.1 Region of interest drawing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Node identification flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Color normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xiv 4 . 4 164 xvii

 44164 Superpixel (SLIC) segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Nuclei detection flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Color deconvolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Nuclei detection process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8 Nuclei detection ground truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.9 Nuclei detection evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.10 Nuclei detection ROC curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.11 Qualitative evaluation of nuclei detection . . . . . . . . . . . . . . . . . . . . . 4.12 Samples of cell nuclei . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.13 Nuclei classification flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.14 Annotation of cell nuclei according to their class. . . . . . . . . . . . . . . . . . 4.15 Regions of analysis for feature extraction . . . . . . . . . . . . . . . . . . . . . 4.16 Blue and Red ratio images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.17 Color feature extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.18 Co-occurrence matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.19 Texture feature extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.20 Fisher scores of color features . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.21 Fisher scores of texture features . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.22 Decision tree model for nuclei classification . . . . . . . . . . . . . . . . . . . . 4.23 10-fold cross-validation of the nuclei classification algorithm . . . . . . . . . . 4.24 Qualitative evaluation of nuclei classification . . . . . . . . . . . . . . . . . . . 4.25 Collagen node identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.26 Adipose tissue node identification . . . . . . . . . . . . . . . . . . . . . . . . . 4.27 Result of the proposed node identification approach . . . . . . . . . . . . . . . 5.1 ROI area and number of nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Percentage and ratio of nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Example of tissue with high immune/cancer ratio . . . . . . . . . . . . . . . . 5.4 Example of tissue with high (fibroblasts+collagen)/cancer ratio . . . . . . . . 5.5 Example of tissue with high adipose/cancer cell ratio . . . . . . . . . . . . . . 5.6 Neighborhood relationship establishment . . . . . . . . . . . . . . . . . . . . . 5.7 Morphological filtering of nodes of histological structures . . . . . . . . . . . . 5.8 Flowchart of the characterization of tissue components based on their expanse 5.9 Quantification of the expanse of tumor patterns . . . . . . . . . . . . . . . . . 5.10 Histograms of the values of the openness function of CCN nodes . . . . . . . 5.11 Comparative analysis with population density . . . . . . . . . . . . . . . . . . 5.12 Intra-tumor heterogeneity quantification based on the expanse function . . . 5.13 Elbow method for determining the optimal number of clusters according to expanse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.14 Examples of tumor cell nodes classification based on the expanse function values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.15 Percentages of the expanse classes of each tissue component . . . . . . . . . . 5.16 Examples of expanse classes of cancer cell nodes . . . . . . . . . . . . . . . . . 5.17 Flowchart of the characterization of tissue components based on their sparsity 5.18 Quantification of the sparsity of cancer cell aggregates . . . . . . . . . . . . . . 5.19 Histograms of the values of the closenness function of CCN in the set CCN . 5.20 Elbow method for determining the optimal number of clusters according to sparsity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.21 Examples of tissue nodes classification based on the values of the closeness function applied to cancer cells . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.22 Percentages of the sparsity classes of each tissue component . . . . . . . . . . 5.23 Examples of sparsity classes of cancer cell nodes . . . . . . . . . . . . . . . . . 5.24 Bivariate histograms of expanse of cancer cells and sparsity of other tissue components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.25 Elbow method and silhouette method for determining the optimal number of clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.26 Clustering of the interaction parameters using k-means algorithm . . . . . . . 5.27 Examples of cancer-immune cell spatial interactions . . . . . . . . . . . . . . . xv 5.28 Examples of cancer-collagen spatial interactions . . . . . . . . . . . . . . . . . 5.29 Examples of cancer-adipose tissue spatial interactions . . . . . . . . . . . . . . 5.30 Percentages of the interaction classes of cancer cells with other tissue component120 5.31 Principal component analysis based on tumor-microenvironment interaction features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.32 Classification of tissues based on the interactions of the tumor with its microenvironment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Flowchart of the tissue simulation model . . . . . . . . . . . . . . . . . . . . . 6.2 Interpretation of the parameters regulating the tumor patterns . . . . . . . . . 6.3 Examples of generation of tumor patterns . . . . . . . . . . . . . . . . . . . . . 6.4 Examples of different Tumor-Associated-Collagen-Signatures (TACS) . . . . . 6.5 Example of different configuration modeling of collagen fiber curves . . . . . 6.6 Simulation of the organization of collagen fibers relative to the tumor patterns 6.7 Examples of different spatial organizations of immune cells relative to the tumor patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.8 Example of a distance map for the configuration of spatial distribution of immune cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.9 Simulation of the spatial distributions of immune cells relative to the tumor patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.10 Examples of simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Potential framework for Spatial Point Pattern Analysis . . . . . . . . . . . . . 7.2 Examples of applications where neighboring cells may have different properties138 A.1 Histograms of the values of the openness function of ICN nodes . . . . . . . A.2 Examples of immune cell classification based on the expanse function values A.3 Histograms of the values of the openness function of F CN nodes . . . . . . . A.4 Examples of collagen nodes classification based on the expanse function values A.5 Examples of expanse classes of collagen nodes . . . . . . . . . . . . . . . . . . A.6 Histograms of the values of the openness function of AT N nodes . . . . . . . A.7 Example of adipose tissue nodes classification based on the expanse function values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.8 Examples of expanse classes of adipose tissue nodes . . . . . . . . . . . . . . . A.9 Histograms of the values of the closenness function of ICN in the set ICN . A.10 Histograms of the values of the closenness function of F CN in the set F CN . A.11 Histograms of the values of the closenness function of AT N in the set AT N . B.1 Examples of simulated images of DCIS architectures . . . . . . . . . . . . . . . B.2 Examples of simulated images of IDC architectures . . . . . . . . . . . . . . . B.3 Examples of simulated images of TC architectures . . . . . . . . . . . . . . . . B.4 Examples of simulated images of ILC architectures . . . . . . . . . . . . . . . . A.1 Histograms of the values of the openness function of ICN nodes . . . . . . . A.2 Examples of immune cell classification based on the expanse function values A.3 Histograms of the values of the openness function of F CN nodes . . . . . . . A.4 Examples of collagen nodes classification based on the expanse function values A.5 Examples of expanse classes of collagen nodes . . . . . . . . . . . . . . . . . . A.6 Histograms of the values of the openness function of AT N nodes . . . . . . . A.7 Example of adipose tissue nodes classification based on the expanse function values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.8 Examples of expanse classes of adipose tissue nodes . . . . . . . . . . . . . . . A.9 Histograms of the values of the closenness function of ICN in the set ICN . A.10 Histograms of the values of the closenness function of F CN in the set F CN . A.11 Histograms of the values of the closenness function of AT N in the set AT N . B.1 Examples of simulated images of DCIS architectures . . . . . . . . . . . . . . . B.2 Examples of simulated images of IDC architectures . . . . . . . . . . . . . . . B.3 Examples of simulated images of TC architectures . . . . . . . . . . . . . . . . xvi B.4 Examples of simulated images of ILC architectures . . . . . . . . . . . . . . . .List of Tables2.1 A summary of graph-based features used in histopathology. . . . . . . . . . . 2.2 A summary of state-of-the-art graph-based methods in histopathology. . . . . 3.1 Summary of the spatial features extracted using mathematical morphology . 4.1 Evaluation metrics for binary classification . . . . . . . . . . . . . . . . . . . . 4.2 Evaluation metrics of the nuclei detection algorithm . . . . . . . . . . . . . . . 4.3 Total number of detected nuclei within the annotated areas . . . . . . . . . . . 4.4 Second-order statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Statistical metric from Laws energy maps . . . . . . . . . . . . . . . . . . . . . 4.6 Example of training data for nuclei classification . . . . . . . . . . . . . . . . . 4.7 Evaluation metrics for multi-class classification . . . . . . . . . . . . . . . . . . 4.8 Evaluation metrics of the nuclei classification algorithm . . . . . . . . . . . . . 4.9 Confusion matrix of the nuclei classification algorithm . . . . . . . . . . . . . 5.1 Matrix of correlation coefficients for the number of nodes . . . . . . . . . . . . 5.2 Matrix of p-values for the number of nodes . . . . . . . . . . . . . . . . . . . . 5.3 Matrix of correlation coefficients for the ratio of the number of nodes . . . . . 5.4 Matrix of p-values for the ratio of the number of nodes . . . . . . . . . . . . . 6.1 Parameters of the results shown in figure 6.10 . . . . . . . . . . . . . . . . . . . 6.2 Mean and standard deviation (in brackets) of the evaluation parameters. . . . xix

FIGURE 1 . 1 :

 11 FIGURE 1.1: Examples of microscopic images of breast tissue samples. (ab) H&E-stained tissue. (c-d) HES-stained tissue. (e-f) Immunohistochemical staining with pHH3, positive cells are colored in brown while negative cells are colored blue.

FIGURE 1 . 2 :

 12 FIGURE 1.2: Illustration of Whole Slide Imaging.

FIGURE 1 . 3 :

 13 FIGURE 1.3: Cross-section scheme of the breast and different subtypes of breast ductal carcinoma [14].

Chapter 1 .FIGURE 1 . 4 :

 114 FIGURE 1.4: (a) Whole Slide Image of breast cancer tissue (0.5X). (b) tissue region at 10X. (c) tissue region at 40X.

FIGURE 1 . 5 :

 15 FIGURE 1.5: Examples of histopathological images showing different architectures of subtypes of breast carcinomas: (A) Tubular carcinoma, (B) cribriform carcinoma, (C) classic ILC, (D) pleomorphic ILC, (E) mucinous carcinoma, (F) neuroendocrine carcinoma, (G) micropapillary carcinoma, (H) papillary carcinoma, (I) low grade IDC [19].

FIGURE 1 . 6 :

 16 FIGURE 1.6: Illustration depicts the tumour microenvironment (TME), showing the major constituents mentioned above [27].

FIGURE 1 . 7 :

 17 FIGURE 1.7: Examples of different cell types in H&E staining: (a) Lymphocytes. (b) Fibroblast. (c) Epithelial cells. (d) Cancers cell nuclei

FIGURE 1 . 8 :

 18 FIGURE 1.8: Examples of glandular tissues in H&E staining: (a) Intestinal glands (Crypt of Lieberkühn). (b) Mammary glands (lactiferous ducts). (c) Prostate glands.

Figure 2 .FIGURE 2 . 1 :

 221 FIGURE 2.1: (a) Binary graph: the black vertices have value 1, whereas the white ones have value 0. (b) Decimal graph: the gray tones associated with the vertices stand for the numerical values.

FIGURE 2 . 2 :

 22 FIGURE 2.2: (a) Vertex neighborhood: gray vertices are neighbors of the black vertex. (b) The shortest path between the two black vertices is highlighted with large black edges (dE(u, v) = 4). Another possible path (of length 6) is shown in dashed lines.

FIGURE 2 . 3 :

 23 FIGURE 2.3: (a) Binary graph G, including 5 connected components (in black). (b) Connected component labeling of G, decimal graph L(G).

FIGURE 2 . 4 :

 24 FIGURE 2.4: (a) 1-NNG. (b) 2-NNG. (c) 3-NNG. (d) 4-NNG.

FIGURE 2 . 5 :

 25 FIGURE 2.5: (a) EBG200; (b) EBG300; (c) EBG400; (c) EBG500.

FIGURE 2 . 6 :

 26 FIGURE 2.6: (a) Voronoi diagram. (b) Delaunay triangulation (dual graph).

FIGURE 2 . 7 :

 27 FIGURE 2.7: (a) Delaunay triangulation. (b) Gabriel graph. (c) Relative neighborhood graph. (d) Euclidean minimum spanning tree.

  The line graph L(G) is a graph such that each vertex of L(G) represents an edge of G, and two vertices of L(G) are adjacent iff their corresponding edges share a common endpoint in G [105].

FIGURE 2 . 9 :

 29 FIGURE 2.9: (a) Graph G. (b) Line graph of G, denoted L(G).

FIGURE 2 .

 2 FIGURE 2.10: (a) Delaunay graph DT . (b) The face graph of order 1 of DT , denoted F1(DT ). (c) The face graph of order 2 of DT , denoted F2(DT ). (d) Gabriel graph of a regular point set GG. (e) The face graph of order 1 of GG, denoted F2(GG). (c) The face graph of order 2 of GG, denoted F2(GG).

FIGURE 2 . 11 :

 211 FIGURE 2.11: Different node identification techniques used in graph-based methods in histopathological image analysis: (a) nodes attributed to cell nuclei. (b) nodes attributed to gland centroids. (c) nodes attributed clusters of cells (the clusters are determined in a grid-fashion). (d) nodes attributed to image regions (tissue components).

FIGURE 2 . 12 :

 212 FIGURE 2.12: Feature extraction techniques used in histopathological image analysis.

  FIGURE 3.1: (a) Binary point set on which morphological operations are applied. (b) Dilation using Delaunay graph. (c) Dilation using Gabriel graph. (d) Dilation using relative neighborhood graph.

Figure 3 .FIGURE 3 . 3 :FIGURE 3 . 4 :

 33334 FIGURE 3.3: (a) Point set with binary values. (b) closing using Delaunay graph. (c) closing using Gabriel graph. (d) closing using relative neighborhood graph.

13 )Figure 3 .FIGURE 3 . 5 :

 13335 Figure 3.5 shows examples of Alternating sequential filter beginning with a closing operation.

Figure 3 .

 3 Figure 3.6 shows an example of geodesic reconstruction by dilation of a binary graph. The foreground vertices of the graph G (also called markers) propagate within the connected components of G until they fill the patterns including them. The reconstructed graph is then the graph obtained by building up those connected components of G, that are marked by G .

FIGURE 3 . 6 :

 36 FIGURE 3.6: (a) A binary graph G. (b) The binary graph G is such that G (v) = 1 if v is black and zero otherwise. The foreground vertices of G are shown in gray and black. (c) R G (G) geodesic reconstruction of G from G .

Figure 3 .FIGURE 3 . 7 :

 337 FIGURE 3.7: (a) Point set with binary values. (b) Distance function calculated from the set of foreground nodes. (c) Distance function calculated from the set of background nodes. The color bar indicates the color code of the distance function values. The white nodes have value equal to zero.

FIGURE 3 . 8 :

 38 FIGURE 3.8: (a) Point set, binary graph. (b) Foreground distance function. (c)The skeleton graph is such that S(v) = 1 if v is black and zero otherwise.

FIGURE 3 . 9 :

 39 FIGURE 3.9: (a) Set of points linked using Delaunay graph (DG) and parameterized by binary values. (b) Openness function Ω(DG). Color bar indicates the color code of the Ω(DG)-values. The white nodes have value equal to zero.

  FIGURE 3.10: (a) Set of points linked using Delaunay graph (DG) and parameterized by binary values. (b) Closeness function ζ(G). Color bar indicates the color code of the ζ(G)-values. The white nodes have value equal to zero.

3. 3 . 51 3. 3

 3513 Point-set Spatial Configuration Modeling using Mathematical Morphology Point-set Spatial Configuration Modeling using Mathematical Morphology

52 Chapter 3 .FIGURE 3 . 11 :

 3311 FIGURE 3.11: Example of image processing problems related to point-set spatial configuration modeling: (a-b) histopathology, (c-d-e) geographical information science, and (f-g) astronomy. (a) Sample of histological section of breast cancer. (b) Detection and classification of histological stuctures: cancer cells, immune cells and adipocytes, are shown with red, blue and yellow points respectively. Blood vessels are shown with magenta-colored triangles. Collagen fibers are presented with green arrows expressing their orientation. Each arrow can be defined as a point parametrized by the corresponding orientation. c) Sample of a satellite image showing evidence of deforestation in the Brazilian state of Pará [229]. d) Zoom on the region delimited by the yellow square in figure c. (e) Blue, respectively red, points are associated to the locations of alive trees, respectively trees that were cut down (tree stumps). f) A view of thousands of distant galaxies, in a patch of sky called the Lockman Hole[START_REF]HerMES: The Lockman Hole[END_REF]. Almost every point of light is an entire galaxy. The colours represent the far-infrared wavelengths measured by Herschel[START_REF] Hasinger | A deep x-ray survey in the lockman-hole and the soft x-ray N-Log[END_REF]. g) Classification of points of significant galaxies: red, blue, and green points are associated to orangish, yellowish, and green galaxies, respectively.

  iv. Pattern (A9) is very distant (far) from any point of the subset B. v. Pattern (A1), respectively (A3), is close to (or near) pattern (B1), respectively (B2), but they are both non-enveloped by patterns of B (we can also use the terms "open" or "free" to describe a situation where a point-set is non-enveloped by another point-set).

FIGURE 3 . 12 :

 312 FIGURE 3.12: Example of spatial point patterns: we denote the set of red points by A, the set of blue points by B, and the set of green points by C.

  and zero otherwise. Similarly, we define the binary graph G B such that G B (v) = 1 iff v ∈ B, and zero otherwise.

3. 3 . 3 FIGURE 3 .FIGURE 3 .

 3333 FIGURE 3.13: (a) Ω(GA): Result of the openness function of the binary graph GA, such that GA(v) = 1 iff v ∈ A, and zero otherwise. The color bar indicates the color code of the Ω(GA)-values. The white nodes have value equal to zero. (b) Histogram of positive values of Ω(GA).

  FIGURE 3.15: (a) (a) ζ(GA): Result of the closeness function of the binary graph GA, such that GA(v) = 1 iff v ∈ A, and zero otherwise. The color bar indicates the color code of the ζ(GA)-values. The white nodes have value equal to zero. (b) Histogram of all values of ζ(GA) of all nodes.

  (ab-c) show results obtained with different thresholds. Furthermore, we can use the output of the thresholding operation to label patterns of A based on this criteria. In fact, we can use the values of ζ(G A ) in S \ A, to differentiate patterns of A that are close to each other, from patterns that are separate or isolated (see figure 3.16.d). In this example, we have used the set of red points in 3.16.c to build up the patterns of A that are close to each other. The result is obtained by morphological reconstruction of G A from its intersection with the dilation of the set of the red points in 3.16.c, analytically: R δ(ζ(G A )≥20) * G A (G A ).

FIGURE 3 . 16 :

 316 FIGURE 3.16: Quantification of the dispersion of point patterns: Black nodes represent the subset A. (a) Nodes in cyan are such ζ(GA) < 10, (b) nodes in green are such 10 ≤ ζ(GA) < 20, and (c) nodes in red are such ζ(GA) ≥ 20. (d) Red patterns are patterns that are close to each other, obtained by morphological reconstruction of GA from the intersection of A with the dilation of the red nodes in 3.16.c, R δ(ζ(G A )≥20) * G A (GA). While, blue patterns are isolated patterns, that were not found after reconstruction, .

  2.4, the closeness function offers significant information about the envelopment of a point set by another. Here, we will investigate how the patterns of A are located relative to the patterns of B in terms of betweenness, encloseness and freedom based on this transform. First, we apply the closeness function ζ to the binary graph G B . The result, denoted by ζ(G B ) is shown in figure 3.17.

FIGURE 3 . 17 :

 317 FIGURE 3.17: The closeness function of the binary graph defined by the set B, denoted ζ(GB). The color bar indicates the color code of the ζ(GB)-values.The white nodes have value equal to zero.

Figure 3 .FIGURE 3 .

 33 FIGURE 3.18: (a) ζ(GB) in nodes A. The result is displayed as (ζ(GB)+1) * GA for a better interpretation of the values. The color bar indicates the color code of the values of (ζ(GB) + 1) * GA. The white nodes have value equal to zero. (b) Histogram of the values of (ζ(GB) + 1) * GA.

FIGURE 3 . 19 :

 319 FIGURE 3.19: The distance function D(GB) of the binary graph GB. The color bar indicates the color code of the D(GB)-values. The white nodes have value equal to zero.

Figure 3 .FIGURE 3 .

 33 FIGURE 3.20: (a) D(GB) * GA D(GB) showing the values of the distance function of D(GB) in nodes of A. The color bar indicates the color code of the values of D(GB). * GA. The white nodes have value equal to zero. (b) Histogram of the values of D(GB). * GA.

1 :

 1 Summary of the spatial features of the point patterns of the subset A, extracted using morphological operations on Delaunay graph.

FIGURE 4 . 1 :

 41 FIGURE 4.1: Region of interest drawing with Aperio ImageScope R software on a whole slide image (WSI) from the dataset. The green line delineates the ROI. The cyan dashed lines delineate the artifacts.

Figure 4 .

 4 2 presents the flowchart showing the steps of the proposed method.

FIGURE 4 . 2 :

 42 FIGURE 4.2: Node identification flowchart.

FIGURE 4 . 3 :

 43 FIGURE 4.3: Color normalization: a) reference (target) image. b) under stained (source) image. c) Reinhard color normalization of the source image (normalized image). d) RGB histograms of the reference image. e) RGB histograms of the source image. f) RGB histograms of the normalized image, showing more separable RGB distributions than in (e).

FIGURE 4 . 4 :

 44 FIGURE 4.4: Example of superpixel segmentation using SLIC algorithm [239]. (a) Quarter of an image tile from our dataset (512 × 512). (b) Corresponding SLIC segmentation; Superpixel boundaries are drawn in black.

FIGURE 4 . 5 :

 45 FIGURE 4.5: Flowchart of the nuclei detection algorithm.

FIGURE 4 . 6 :

 46 FIGURE 4.6: Color deconvolution of a HES image from our dataset using the method proposed in [241]: (a) Original HES image. (b) Haematoxylin image. (c) Eosin image. (d) Background. The three intensity images (b), (c), and (d) are actually intensity images, but here they are displayed, using pseudocolor-maps to allow a better visualization of the shades of the stains, [241].

  the morphological opening with a disk of radius R d ; R X (Y ) is the morphological reconstruction of the marker image Y under the mask image X; and I nuclei is the binary image, result of nuclei segmentation.

FIGURE 4 . 7 :

 47 FIGURE 4.7: Nuclei detection steps: (a) HES image from our dataset. (b) Haematoxylin image obtained from color deconvolution. (c) Thresholded haematoxylin image, such that for every pixel (i, j), (JT h (H))i,j = 1 iff Hi,j ≥ T h . (d) Result of the morphological reconstruction of the thresholded haematoxylin image under the mask of the superpixels, after morphological opening with a disk of radius R d .

FIGURE 4 . 8 :

 48 FIGURE 4.8: Ground truth generation: (a) Cropped and resized image from the dataset provided in [242]. The ROI is within the yellow box. (b) Manual annotation of cell nuclei by an expert. (c) Superpixels corresponding to locations of annotated nuclei are shown in blue.

FIGURE 4 . 9 :

 49 FIGURE 4.9: Nuclei detection evaluation. (a) Nuclei annotated by the pathologist in green points. Example of detection with our algorithm in red (with non-optimal parameters). Red dots are associated to the centroids of the superpixels, delineated in white. (b) True positives (TP) are in green, false positives (FP) are in yellow, true negatives (TN) are in purple, and false negatives (FN) are in red.

  Proportion of negative predictions from actual negatives

FIGURE 4 . 10 :

 410 FIGURE 4.10: ROC curves obtained with different parameter sets on the training set. (a) Evaluation using the Haematoxylin image. (b) Evaluation using the blue ratio image. (c) Evaluation using the Eosin image.

FIGURE 4 . 11 :

 411 FIGURE 4.11: Qualitative evaluation of nuclei detection. (a) Original HES image tile. (b) Results of nuclei detection with our approach. Detected nuclei are highlighted in cyan.

FIGURE 4 . 12 :

 412 FIGURE 4.12: Samples of cell nuclei: (a) cancer cell nuclei, (b) immune cell nuclei, (c) fibroblasts

FIGURE 4 . 13 :

 413 FIGURE 4.13: Flowchart of the nuclei classification algorithm.

FIGURE 4 . 14 :

 414 FIGURE 4.14: Annotation of cell nuclei according to their class.

FIGURE 4 .

 4 FIGURE 4.15: (a) Regions of analysis for color feature extraction: nucleus regions are highlighted in green. Regions around nuclei are obtained with morphological dilation and they are delineated in yellow. (b) Regions of analysis for texture feature extraction: image frames of different sizes are drawn in yellow.

Figure 4 .FIGURE 4 .

 44 FIGURE 4.16: (a) Original HES image tile. (b) Blue ratio image IBR. (c) Red ratio image IRR. (d) Grayscale image Ig. (e) Haematoxylin channel H obtained with color deconvolution. (f) Eosin channel E obtained with color deconvolution. (g) Red channel R. (g) Green channel G. (g) Blue channel B.

FIGURE 4 . 17 :

 417 FIGURE 4.17: Color feature calculation: Statistical metrics are calculated from region of analysis of different sizes from different color channels.

Figure 4 .

 4 [START_REF] Sinn | A brief overview of the WHO classification of breast tumors[END_REF].d presents four co-occurrence matrices calculated from the sample presented in figure4.18.b, in the four directions.

FIGURE 4 .

 4 FIGURE 4.18: Co-occurrence matrix calculation: (a) intensity image, (b) intensity values of a very small sample, presenting 4 gray-levels, (c) four possible directions of pixel adjacency. (d) Co-occurrence matrices generated using each of the four directions of adjacency from the sample in (b).

  -j) 2 pi,j measures the closeness of the distribution of the gray levels

Figure 4 .

 4 [START_REF] Reis-Filho | Histological types of breast cancer: How special are they?[END_REF] presents our approach for texture feature extraction.

FIGURE 4 . 19 :

 419 FIGURE 4.19: Texture feature calculation: statistical metrics calculated from regions of analysis (frames) of different sizes (see figure 4.15.b), using two different texture methods applied three different intensity images.

  Figures 4.20

  and 4.21 present the Fisher scores of the color and texture features computed during the feature extraction.The diagram in figure4.20 presents, for each intensity image, the maximal Fisher scores over the 5 statistical measures calculated in different surfaces (their average is shown in a horizontal line). The diagram shows that features derived from the R, B, I BR , I RR , H and Ig images are more discriminative when the surface around the nucleus is large (radius=20). While, features derived from the G and E images are more discriminative when the surface around the nucleus is small (radius=5). Hence, these features are chosen as contextual color features and the others are eliminated.

FIGURE 4 . 20 :

 420 FIGURE 4.20: Fisher scores of color features.

FIGURE 4 . 21 :

 421 FIGURE 4.21: Fisher scores of texture features.

  k +f p k An average per-class agreement of the data class labels with those of the classifiers RecallM k +f p k An average per-class proportion of negative predictions from actual negatives

Figure 4 .

 4 [START_REF] Provenzano | Collagen reorganization at the tumor-stromal interface facilitates local invasion[END_REF].b presents the 10-fold cross-validation on a random balanced subset. The results show that the evaluation measures remain in the same range for the both experiments. which reduces the risk of an over-optimistic view relative to the performance of the classification.

FIGURE 4 .

 4 FIGURE 4.23: 10-fold cross-validation of the nuclei classification algorithm based on the evaluation metrics in 4.7. (a) 10-fold cross-validation using the entire imbalanced dataset. (b) 10-fold cross-validation using a random balanced subset of the dataset.

Chapter 4 .

 4 A Method for Tissue Segmentation and Node Identification (a) (b)

FIGURE 4 . 24 :

 424 FIGURE 4.24: Qualitative evaluation of nuclei classification. (a) Original HES image tile (presented in fig 4.11. (b) Results of nuclei classification with our approach. red nodes represent cancer cells, blue nodes represent immune cells, and green nodes represent fibroblasts.

FIGURE 4 . 25 :

 425 FIGURE 4.25: Collagen node identification: (a) original HES image tile. (b) red ratio image. (c) binary image obtained from the thresholded red ratio image multiplied with the binary mask of the superpixels. (d) identified collagen nodes shown in green.

FIGURE 4 . 26 :

 426 FIGURE 4.26: Adipose tissue node identification: (a) original HES image, with annotated regions overlayed in yellow. (b) binary image obtained from thresholding the original image, such that only pixels with (R,G,B) values simultaneously greater than 220 are retained. (c) Multiplication of the binary image in (b) with the annotated region mask and the superpixel mask simultaneously, after morphological filtering (opening with a disk of radius 10). (d) nodes corresponding to the centroids of the remaining superpixels are associated to the adipose tissue (show in yellow).

Chapter 4 .

 4 A Method for Tissue Segmentation and Node Identification (a) (b)

FIGURE 4 .

 4 FIGURE 4.27: (a) Region of interest extracted from a whole slide image from our dataset. (b) Result of the proposed node identification approach: red nodes represent cancer cell nuclei, blue nodes represent immune cell nuclei, while green nodes represent both fibroblasts and collagen-associated structures. Yellow nodes represent adipose tissue nodes

FIGURE 5 . 1 :

 51 FIGURE 5.1: The ROI area of each of the 55 cases in the dataset, and the number of nodes extracted for each tissue component using our node identification approach. Note: The number of fibroblast nodes is overlayed in dark green on the bars of the total number of F CN nodes.

FIGURE 5 . 2 :

 52 FIGURE 5.2: Percentage of nodes associated to each tissue component, and the ratios of tumor microenvironment components relative to the tumor.

Chapter 5 .FIGURE 5 . 3 :FIGURE 5 . 4 :

 55354 FIGURE 5.3: Example of tissue with high immune/cancer ratio (case N • 52). CCN , ICN , F CN , AT N are shown in red, blue, green and yellow respectively. Immune cells are widespread within the ROI and mixed (co-located) with cancer cells.

FIGURE 5 . 6 :

 56 FIGURE 5.6: Neighborhood relationship establishment. (a) Delaunay graph of the whole population of nodes V . (b) α-shape filtering of the Delaunay graph, where triangles with circumradius larger than 50 pixels (25µm) are removed. CCN , ICN , F CN , AT N and BDN are shown in red, blue, green, yellow and cyien respectively.

FIGURE 5 . 7 :

 57 FIGURE 5.7: (a) Original HES image. (b) Result of our node identification algorithm, CCN , ICN , F CN , AT N are shown in red, blue, green and yellow respectively. (c) Result of morphological closing: (ϕ2(G)). (d) Result of Alternating Sequential Filter of order 5 begging with closing: (Φ5(G)).

FIGURE 5 . 8 :

 58 FIGURE 5.8: Flowchart of our method for the characterization of tissue components based on their expanse.

  Figures 5.9.(c-h) show examples of outputs of the openness function Ω applied to the binary graph G C defined by CCN nodes after being processed with different morphological closing ϕ rc , with r c ∈ {0, 5}. It is clear that, given a fixed size of closing r c , the values of the function Ω • ϕ rc (G C ) increase with the expanse of the agglomerations of cancer cells. And, in addition, the larger the size of the prior closing operation ϕ rc , the higher are the values of the function Ω • ϕ rc (G C ), this is due to the fact that the closing operations strengthen the connectedness between foreground nodes (CCN nodes in this case). This is also demonstrated by the distributions of the values of Ω•ϕ rc (G C ) calculated over the whole dataset (55 whole slide images), see figure 5.10.(a-f). It should be noted that this is also true for the distributions of the values of Ω • ϕ rc applied to G I , G F and G A (see figures A.1, A.3 and A.6 in appendix).

FIGURE 5 . 9 :

 59 FIGURE 5.9: Quantification of the expanse of tumor patterns: (a) sample from a HES image. (b) result of node identification (CCN , ICN and F CN are shown in red, blue and green respectively). (c-h) outputs of the openness function after morphological closings of sizes ranging from 0 to 5 respectively. The operations Ω • ϕr c , rc ∈ {0, 5}, are applied to the binary graph GC defined by cancer cell nodes (GC (v) = 1 if v ∈ CCN , and zero otherwise). (i) Expanse function Γ(GC ): average over the 6 outputs in (c-h). (j) histogram of the values of Σ(GC ) over the set CCN .

FIGURE 5 . 10 :

 510 FIGURE 5.10: Histograms of the values of the openness function of CCN nodes over the whole dataset: (a-f) variations of the histogram of Ω•ϕr c (GC ) depending on the size of morphological closing rc ∈ {0, 5}. (h) histogram of the expanse function over the six configurations, values of Σ(GC ).

FIGURE 5 . 11 :

 511 FIGURE 5.11: Bivariate histograms of density-values (x-axis) and Σ-values (y-axis) of nodes of each histological structure over the whole dataset: from (a) to (b) correspond to CCN , ICN , F CN and AT N respectively.

  Figures 5.12.(a-b) show the average values of the expanse function of CCN nodes calculated from two tissue samples where the percentages of cancer cells identified within the corresponding ROIs are comparable. These samples are case N • 2 and case N • 53, refer to figure 5.2.

FIGURE 5 . 12 :

 512 FIGURE 5.12: Intra-tumor heterogeneity quantification based on expanse: (a) Σ function of case N • 53, entropy Ξ = 3.7. (b) Σ function of case N • 2, entropy Ξ = 6.8. (c-e) random frames from N • 53 showing similar spatial distributions of tumor patterns. (f-h) random frames from N • 2 showing various (heterogeneous) spatial distributions of tumor patterns.

FIGURE 5 .

 5 FIGURE 5.13: (a) Elbow method for determining the optimal number of clusters according to expanse. (b) Cluster means obtained with k-means classification for each tissue component (k = 5).

FIGURE 5 . 14 :

 514 FIGURE 5.14: Examples of tumor cell nodes classification based on the expanse function values: (a-b) original HES images. (c-d) results of k-means classification, where CCN nodes belonging to classes 1, 2, 3, 4 and 5 are shown in blue, cyan, green, yellow and orange, respectively. The classes are sorted in ascending order from low to high expanse.

FIGURE 5 . 15 :

 515 FIGURE 5.15: Percentages of the expanse classes of each tissue component. Classes 1, 2, 3, 4 and 5 are shown in blue, cyan, green, yellow and orange, respectively.

FIGURE 5 . 16 :

 516 FIGURE 5.16: Examples of expanse classes of cancer cell nodes and comparison with population density approach: (a), (b) and (c) present cases N • 23, N • 24 and N • 46, respectively. (d-f) results of CCN nodes classification according to the values of Σ(GC ). (g-i) results of CCN density calculated within frames of size 1024 × 1024.

FIGURE 5 . 17 :

 517 FIGURE 5.17: Flowchart of our method for the characterization of tissue components based on their sparsity.

  . The distributions of the values of ζ • γ ro (G C ) calculated over the set V \ CCN of the whole dataset (55 whole slide images) are given in figure 5.19.(a-f).

FIGURE 5 . 18 : 5 ro=0ζ

 5185 FIGURE 5.18: Quantification of the sparsity of cancer cell aggregates: (a) sample from a HES image. (b) result of node identification (CCN , ICN and F CN are shown in red, blue and green respectively). (c) to (h) outputs of the closeness function after morphological openings of sizes ranging from 0 to 5 respectively. The operations ζ • γr o , ro ∈ {0, 5}, are applied to the binary graph GC defined by CCN nodes (GC (v) = 1 if v ∈ CCN , and zero otherwise). (i) Sparsity function Γ(GC ): average over the 6 outputs in (c-h). (j) histogram of the values of Γ(GC ) over the set V \ CCN .

FIGURE 5 . 19 :

 519 FIGURE 5.19: Histograms of the values of the closenness function of CCN in the set CCN over the whole dataset: (a-f) variations of the histogram of ζ • γr o (GC ) depending on the size of morphological opening ro ∈ {0, 5}. (h) histogram of the sparsity function Γ(GC ) over the set CCN .

FIGURE 5 .FIGURE 5 . 21 :

 5521 FIGURE 5.20: (a) Elbow method for determining the optimal number of clusters according to sparsity. (b) Cluster means obtained with k-means classification (k = 5) for each of the tissue component CCN , ICN , F CN or AT N .

FIGURE 5 . 22 :

 522 FIGURE 5.22: Percentages of the sparsity classes of each tissue component. Classes 1, 2, 3, 4 and 5 are shown in blue, cyan, green, yellow and orange, respectively.

FIGURE 5 . 23 :

 523 FIGURE 5.23: Examples of sparsity classes of cancer cell nodes: (a), (b) and (c) present cases N • 8, N • 32 and N • 39, respectively. (d-f) Outputs of the node identification step, CCN , ICN , F CN , AT N nodes are shown in red, blue, green and yellow respectively. (g-i) results of CCN nodes classification according to the values of Γ(GC ).

FIGURE 5 . 26 :

 526 FIGURE 5.26: Clustering of the interaction parameters using k-means algorithm. (a) CCN vs ICN, (b) CCN vs FCN and (c) FCN vs ATN.

. 27 .FIGURE 5 . 27 :

 27527 FIGURE 5.27: Examples of cancer-immune cell spatial interactions. (a-b) Outputs of the node identification step, CCN , ICN , F CN , AT N nodes are shown in red, blue, green and yellow respectively. (c-d) Results of cancer cell classification based on their interactions with immune cells. Cancer cells are highlighted with colors representing their class in figure 5.26.a.

Figure 5 .FIGURE 5 . 28 :

 5528 Figure 5.28 shows an example of such classification.

FIGURE 5 . 29 :

 529 FIGURE 5.29: Examples of cancer-adipose tissue spatial interactions. (a) Output of the node identification step, CCN , ICN , F CN , AT N nodes are shown in red, blue, green and yellow respectively. (b) Result of cancer cell classification based on their positioning relative to adipose tissue. Cancer cells are highlighted with colors representing their class in figure 5.26.c.

FIGURE 5 . 30 :

 530 FIGURE 5.30: Percentages of the interaction classes of cancer cells with other tissue component. The colors representing the interaction classes are defined in figure 5.26.

FIGURE 5 .. 2 )

 52 FIGURE 5.31: (a) Principal components provided by PCA based on the 16 tumor-microenvironment interaction features. (b) k-means clustering of the data based on the 3 principal components (each case of the dataset is presented by its identifier number).

FIGURE 5 . 32 :

 532 FIGURE 5.32: Classification of tissues based on the interactions of the tumor with its microenvironment. (a), (b), (c) and (d) show cases from tumormicroenvironment interaction classes 1, 2, 3 and 4, respectively. The images are miniatures of the results of the node identification, where CCN , ICN , F CN , AT N nodes are shown in red, blue, green and yellow respectively.

FIGURE 6 . 1 :

 61 FIGURE 6.1: Flowchart of the tissue simulation model. Each step is regulated by parameters that control particular characteristics of the tissue.

FIGURE 6 . 2 :

 62 FIGURE 6.2: Interpretation of the morphological parameters that regulate the architecture of the tumor growth patterns.

2 .FIGURE 6 . 3 :

 263 FIGURE 6.3: Examples of generation of tumor patterns: yellow nodes are the point generated with P n 0 ,d 0 , whereas the green objects correspond to their dilations, the binary masks MT P .

FIGURE 6 . 4 :

 64 FIGURE 6.4: Examples of different Tumor-Associated-Collagen-Signatures (TACS) in real data. (a) TACS-2. (b) TACS-3. (c) TACS-1.

FIGURE 6 . 5 :FIGURE 6 . 6 :

 6566 FIGURE 6.5: Example of different configuration modeling of collagen fiber curves: TP objects that are output of the first module are shown in red, TP reshaping is shown in cyan, and the generated collagen fiber curves are highlighted in light blue.

FIGURE 6 . 7 :

 67 FIGURE 6.7: different spatial organizations of immune cells relative to the tumor patterns.

FIGURE 6 . 8 :

 68 FIGURE 6.8: Example of a distance map for the configuration of spatial distribution of immune cells relative to the tumor patterns: TP objects that are output of the first module are shown in grey, the distance map values increase from the coldest-color to the hottest-color layers.

FIGURE 6 . 9 :

 69 FIGURE 6.9: Simulation of the spatial distributions of immune cells relative to the tumor patterns: In all the configurations, the total number of immune cells is nL = 3000 and the minimum relative distance is dL = 10 pixels.The parameter α is set to -1, 0 and 3 respectively in (a), (b) and (c).

TABLE 6 . 1 :DCIS N 1 1 TC N 1 1 ILC N 1 1 6FIGURE 6 .

 611111116 FIGURE 6.10: (a-d) tumor pattern masks, MT P . (e-f) simulated images, associated parameters are given in table 1: (e) DCIS, (f) MC shown immune cells around the TPs, (g) TC, and (h) ILC. (i-l) real tissue images.

FIGURE 7 . 1 :FIGURE 7 . 2 :

 7172 FIGURE 7.1: Further potential framework for Spatial Point Pattern Analysis

FIGURE A. 2 :

 2 FIGURE A.2: Examples of immune cell classification based on the expanse function values of Σ(GI ): (a-b) results of node identification (CCN , ICN , F CN and AT N nodes are shown in red, blue, green and yellow respectively). (c-d) results of k-means classification, where ICN nodes belonging to classes 1, 2, 3, 4 and 5 are shown in blue, cyan, green, yellow and orange, respectively. The classes are sorted in ascending order from low to high expanse.

FIGURE A. 3 :

 3 FIGURE A.3: Histograms of the values of the openness function of F CN nodes over the whole dataset: (a-f) variations of the histogram of Ω•ϕr c (GF ) depending on the size of morphological closing rc ∈ {0, 5}. (h) histogram of the expanse function over the six configurations, values of Σ(GF ).

FIGURE A. 4 :FIGURE A. 5 :

 45 FIGURE A.4: Examples of collagen and fibroblasts' nodes classification based on the expanse function values of Σ(GF ): (a-b) original HES images. (c-d) results of k-means classification, where F CN nodes belonging to classes 1, 2, 3, 4 and 5 are shown in blue, cyan, green, yellow and orange, respectively.The classes are sorted in ascending order from low to high expanse.

FIGURE A. 6 :

 6 FIGURE A.6: Histograms of the values of the openness function of AT N nodes over the whole dataset: (a-f) variations of the histogram of Ω • ϕr c (GA) depending on the size of morphological closing rc ∈ {0, 5}. (h) histogram of the expanse function over the six configurations, values of Σ(GA).

FIGURE A. 7 :FIGURE A. 8 :

 78 FIGURE A.7: Example of adipose tissue nodes classification based on the values of Σ(GA): (a) HES image. (b) and (d) results of k-means classification, where AT N nodes belonging to classes 1, 2, 3, 4 and 5 are shown in blue, cyan, green, yellow and orange, respectively. The classes are sorted in ascending order from low to high expanse.

TABLE 2 .

 2 1: A summary of graph-based features used in histopathology.

	Syntactic structure analysis	-closeness	-linear arboricity
		-nbr. of conn. components	-fractional arboricity
	-Average, min, max, standard	-Giant conn. component	-pseudo-arboricity
	deviation, disorder, skewness,	-nbr. of isolated nodes	-Wiener index
	kurtosis, and heterogeneity of:	-nbr. of nodes with k	-boxicity
	. polygon (or triangle)	neighbors	-Randic connectivity index
	. polygon (or triangle) perimeter	-nbr. of central vertices	-Balaban centric index
	. edge length	-nbr. of spanning trees
	. chord length	-nbr. of nodes belonging to	Spectral analysis
	. minimum angle	the spanning trees
	. maximum angle	-nbr. of components with	-spectral radius
	. nbr. of sides of polygons	equal nbr. of nodes & edges	-Fiedler vector
	. nearest neighbor distance	-nbr. of nodes belongin to	-Cheeger constant
	. distance to the k-nearest	components with equal	-algebraic connectivity
	neighbors	number of nodes and edges	-principal eigenvector
	. nbr. of neighbors within a disk -betweenness	-the second largest
	of radius r	-rich club coefficient	absolute value of eigenvalues
	. area of holes

TABLE 2 .

 2 2: A summary of state-of-the-art graph-based methods in histopathology.

	nbr.polygon totalarea	-edge connectivity	-histogram of the distance
		-toughness	transform
	Network analysis	-thickness	-granulometry
		-degeneracy
	-order (nbr. of nodes)	-strength
	-size (nbr. of edges)	-graph genus
	-average degree	-book thickness
	-average eccentricity	-Hosoya index
	-radius (min. eccentricity)	-Estrada index
	-diameter (max. eccentricity)	-arboricity
	-clustering coefficient	-star arboricity

H&E

Chapter 2. Review of Graph-based Methods in Histopathological Image Analysis

TABLE 4 .

 4

	Accuracy Precision	Recall	Specificity F1-score
	0.81065	0.46406	0.87843	0.7971	0.60729

2: Evaluation metrics of the nuclei detection algorithm applied to the testing set with optimal parameters T h = 0.6 and R d = 1.

TABLE 4 .

 4

3: Total number of detected nuclei within the annotated areas.

TABLE 4 .

 4

4: Second-order statistical metrics used in calculating texture features from a co-occurrence matrix.

TABLE 4 . 5 :

 45 Statistical metrics used in calculating texture features from Laws energy maps.

TABLE 4 .

 4

	Red-ratio	Class
	(out)

6: Training data FIGURE 4.22: Example of a decision tree model for nuclei classification derived from features in table 4.6.

TABLE 4 .

 4 7: Evaluation metrics for multi-class classification. tp k , tn k , f p k and f n k are the numbers of true positives/negatives and false positives/negatives for each individual class C k , receptively.

TABLE 4 .

 4 4.8.

	AccuracyM	PrecisionM	RecallM	SpecificityM	F1-scoreM
	0.9784	0.9383	0.8745	0.9737	0.9053

8: Average values of the evaluation metrics of the nuclei classification algorithm based on 10-fold cross validation.

Predicted class Cancer cells Immune cells Fibroblasts Actual class Cancer cells 68858 (98.50%) 901

			(1.29%)	149 (0.21%)
	Immune cells 1087 (3.89%)	26705 (95.60%)	142 (0.51%)
	Fibroblasts	771 (25.11%)	187 (6.09%)	2113 (68.80%)

TABLE 4 .

 4 9: Average confusion matrix of the nuclei classification algorithm based on 10-fold cross validation.

TABLE 5 .

 5 .2). 1: Matrix of correlation coefficients for the number of nodes.

		ROI area	CCN	ICN	FCN	ATN
	ROI area	1.0000	0.8893 0.4896 0.7719 0.4171
	CCN	0.8893	1.0000 0.3116 0.5726 0.2684
	ICN	0.4896	0.3116 1.0000 0.1966 0.1047
	FCN	0.7719	0.5726 0.1966 1.0000 0.2487
	ATN	0.4171	0.2684 0.1047 0.2487 1.0000

TABLE 6 .

 6 2: Mean and standard deviation (in brackets) of the evaluation parameters.

			Nbr. of TP	Area (10 4 )	Major axis	Minor axis	Solidity	CoV of Orient.
	DCIS	Real	5.14(1.95)	25.5(21)	660(280)	432(195)	0.97(0.03)	0.63(0.43)
		Synthetic	5.67(2.34)	27.6(17)	713(243)	464(126)	0.98(0.02)	0.43(0.12)
	MC	Real	22.6(10.7)	5.89(4.3)	375(263)	186(184)	0.89(0.10)	0.61(0.14)
		Synthetic	17.7(12.9)	7.34(5.6)	412(287)	211(191)	0.78(0.23)	0.63(0.22)
	TC	Real	40.6(7.63)	2.74(2.3)	238(114)	137(51.5)	0.96(0.04)	0.71(0.11)
		Synthetic	63.8(10.2)	1.97(3.2)	217(129)	120(62.7)	0.94(0.10)	0.83(0.13)
	ILC	Real	95.7(23.5)	0.76(0.5)	198(166)	46.7(27.9)	0.89(0.07)	0.32(0.15)
		Synthetic	89.3(35.1)	0.98(0.3)	230(183)	39.2(19.1)	0.84(0.12)	0.29(0.09)

http://www.warwick.ac.uk/bialab/GlaScontest

FIGURE 1.10: Thesis structure.

The complete access to the full dataset is provided with this link: http://gleason.case.edu/webdata/ TMI2015/

Acknowledgements

Acknowledgements vii

Acknowledgments

This study has been done with the support of the FUI Project FlexMIm: Collaborative Digital Pathology, the European Institute of Innovation & Technology (EIT) Health -PAPHOS project, and the French Foundation for Medical Research (FRM) -convention DBS20131128436.

In order to evaluate the dependence between the ratios of the tissue components, we have also calculated the correlation and p-value matrices of the above variables, and they are given in tables 5.3 and 5.4, respectively. Here, we are only interested in correlations between variables that do not involve the same entity simultaneously, like the percentage of ICN and the ratio ICN/CCN for example. These values are shown in light gray in the following tables. The result reveals some negative correlations between the percentages of CCN and F CN , and between the percentages of CCN and AT N . This can be explained by the fact that if the number of cancer cells is relatively low compared to the size of the ROI, then the remaining space is occupied mostly by collagen or adipose tissue. In order to determine classes of cancer cells based on these two features, we apply kmeans algorithm and Elbow method. The plot of the Elbow method is given in figure 5.25.a. Elbow method is sometimes ambiguous when the data seem hardly to be separable as is in this case, and this can be interpreted from the low convexity of the Elbow curve. Therefore, we apply another technique called silhouette [START_REF] Rousseeuw | Silhouettes: a graphical aid to the interpretation and validation of cluster analysis[END_REF], which is a method of interpretation and validation of how well each point (data) lies within its cluster. The silhouette value for each point is a measure of how similar that point is to points in its own cluster, when compared to points in other clusters. The silhouette value for a point x in the dataset is defined as:

where a x is the average distance from point x to the other points in the same cluster as x, and b x is the minimum average distance from point x to points in a different cluster. The silhouette values range from -1 to 1, where a high value indicates that the point is well matched to its own cluster and poorly matched to neighboring clusters.

A.2 Sparsity of histological structures A.2.1 Sparsity of immune cells

A.2.2 Sparsity of collagen fibers and fibroblasts

A.2.3 Sparsity of adipose tissue