Jesús Angulo

Camille Couprie

Benjamin Rayanal

Laszlo Marak

John Chaussard

Nadine Dommanget

Anthony Giroud

Ngo Phuc

Christophe Guentleur

Olivia Miraucourt

Yongchao Xu

Rolland Levillain

Imen Melki

Ania Jezierska

Fiona Zolyniak

Hiroko Mitarai

Ali Kanj

Mathieu Lagarde

Kacper Pluta

Michel Revollo

Michel Couprie

Gilles Bertrand

Laurent Najman

Jean Cousty

Benjamin Perret

Nabil Mustafa

Yukiko Kenmochi

Laurent Tsang

Vicent Nozick

Venceslas Biri

Mireille El Abboud

Manar Gheche

Audrey Qamhieh

Mai Quyen Rappeti

Pham

Ajith Sivadasan

Phuc Magda Norbert

Francesco And Matheusz

Sonja Dolce

Feriel Hiltunen

Sergiu Abboud

Matija Ivanov

Jacopo Hustić

Jacopo Corbetta

Pavel Mancin

Kacper Heller

Pluta

Chi-Wei Chen

Katia Martemianova

Ravi Kiran

Jean Serra

Articles in Preparation/Submission 9. Optimization on Braids of Partitions, B

Keywords: Hierarchical segmentation, Lagrangian Multipliers, Lattice optimization, Mathematical Morphology

Hierarchical segmentation has been a model which both identifies with the construct of extracting a tree structured model of the image, while also interpreting it as an optimization problem of the optimal scale selection. Hierarchical processing is an emerging field of problems in computer vision and hyperspectral image processing community, on account of its ability to structure high-dimensional data.

Chapter 1 discusses two important concepts of Braids and Energetic lattices. A braid of partitions is a richer hierarchical partition model that provides multiple locally nonnested partitioning, while being globally a hierarchical partitioning of the space. The problem of optimization on hierarchies and further braids are non-tractable due the combinatorial nature of the problem. We provide conditions, of h-increasingness, scaleincreasingness on the energy defined on partitions, to extract unique and monotonically ordered minimal partitions.

Furthermore these conditions are found to be coherent with the Braid structure to perform constrained optimization on hierarchies, and more generally Braids. Chapter 2 demonstrates the Energetic lattice, and how it generalizes the Lagrangian formulation of the constrained optimization problem on hierarchies.

Finally in Chapter 3 we apply the method of optimization using energetic lattices to the problem of extraction of segmentations from a hierarchy, that are proximal to a ground truth set.

Chapter 4 we show how one moves from the energetic lattice on hierarchies and braids, to a numerical lattice of Jordan Curves which define a continuous model of hierarchical segmentation. This model enables also to compose different functions and hierarchies.

Chapter 5 compiles the scale-climbing algorithms by Guigues and Salembier-Garrido, over the hierarchies of partitions, and provides the new dynamic program for the Braids of partitions. Further it discusses a perspective on using intersection graphs to solve the optimal cut problem, and identities "Partition Graphs" to be one of the good graph structures to model partition selection. It finally concludes by formulating the optimal cut problem on hierarchies as a flow-maximization on a tree structure, the case of braids are also discussed.

Résumé

La segmentation hiérarchique est une méthode pour produire des partitions qui représentent une même image de manière de moins en moins fine. En même temps, elle sert d'entrée à la recherche d'une partition optimale, qui combine des extraits des diverses partitions en divers endroits. Le traitement hiérarchique des images est un domaine émergent en vision par ordinateur, et en particulier dans la communauté qui étudie les images hyperspectrales et les SIG, du fait de son capacité à structurer des données hyper-dimensionnelles.

Le chapitre 1 porte sur les deux concepts fondamentaux de tresse et de treillis energétique. La tresse est une notion plus riche que celle de hierarchie de partitions, en ce qu'elle incorpore, en plus, des partitions qui ne sont pas emboîtées les unes dans les autres, tout en s'appuyant glolalement sur une hiérarchie. Le treillis énergétique est une structure mixte qui regroupe une tresse avec une énergie, et permet d'y défninr des éléments maximaux et minimaux. Lorsqu'on se donne une énergie, trouver la partition formée de classes de la tresse (ou de la hiérarchie) qui minimise cette énergie est un problème insoluble, de par sa complexité combinatoriale. Nous donnons les deux conditions de h-croissance et de croissance d'échelle, qui garantissent l'existence, l'unicité et la monotonie des solutions, et conduisent à un algorithme qui les détermine en deux passes de lecture des données.

Le chapitre 2 reste dans le cadre précédent, mais étudie plus spécifiquement l'optimisation sous contrainte. Il débouche sur trois généralisations du modèle Lagrangien.

Le chapitre 3 applique l'optimisation par treillis énergétique au cas de figure où l'energie est introduite par une "vérité terrrain", c'est à dire par un jeu de dessins mauel, que les partitions optimales doivent serrer au plus près.

Enfin, le chapitre 4 passe des treills énergétiques à ceux des courbes de Jordan dans le plan euclidien, qui définissent un modèle continu de segmentations hierarchiques. Il permet entre autres de composer les hiérarchies avec diverses fonctions numériques.

Chapitre 5 compile les algorithmes d'escalade par Guigues, sur les hiérarchies de partitions, et en plus fournit le nouveau programme dynamique pour les tresses de partitions. En outre, il décrit une perspective sur des graphes d'intersection qui aide à résoudre le problème de la coupure optimale, et identifie les "Partition Graphs" comme l'une des bonnes structures de graphes pour modéliser la sélection de la partition a partir d'un famille hierarchique. Il conclut enfin par la formulation du problème de coupe optimal sur les hiérarchies, comme un problème de maximisation de flux, sur une structure d'arbre, le cas de tresses est également discuté.

Mot Clé: Segmentation hiérarchique, Multiplicateurs de Lagrange, Optimisation dans les treillis, Morphologie mathématique. Chapter 0

Motivation and Thesis Overview

Hierarchical Clustering has been used extract clusters from data which are assumed to have an underlying tree structure. This has been evident in the field of gene expression array analsysis, phylogenetics, phylolinguistics. It has also been in the domain of image segmentation, with perceptual organization trees [70] where one the segmentation problem was motivated by a inclusion tree of regions/objects.

The basic utlities of hierarchical clustering and the following segmentation/classification is to represent images, summarize non-hierarchical data like images into hierarchical classes for various tasks like objection detection recognition, recover underlying structures of the data, for example in texture images.

Hierarchical clustering is an classical method to group set of datapoints in nested classes by introducing a metric between pairs of points each time. The grouping occurs iteratively with the recalculation of pairwise distances.

There are two approaches to hierarchical clustering: we can go from the "bottom up"(Agglomerative), grouping small clusters into larger ones, or from the "top down" (Bifurcative), splitting big clusters into small ones. These are called agglomerative and divisive clusterings, respectively. We consider Agglomerative clustering since the bottom-up approach bounds the complexity to polynomial at worst, while divisive clustering requires a combinatorial search of every possible division of cluster, leading to exponential complexity.

Hierarchical segmentations methods have been studied to be able to parameterize the number of segments, and later chose an optimal scale of partitions for the purpose of segmentation. The methods in literature study split-merge heuristics, the feature dissimilarities, to produce segmentations. This thesis will study how to introduce optimality 1 0.1. MOTIVATION when the solution space is the hierarchy of partitions, while using its inherent lattice structure, and avoid the combinatorial nature of the problem. 1 .

Motivation

This thesis studies the optimality on constrained and unconstrained problems on hierarchies of partitions. The thesis continues on the lines of the work by Salembier et al.,

Guiges et al., Casselles et al., [START_REF] Ballester | Level lines selection with variational models for segmentation and encoding[END_REF]47,[START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF], mainly. They pose the problem of constrained minimization of an energy on the space of the hierarchy of partitions. The goal in these studies and also one of the goals of the thesis is to define conditions on the energy that obtains: unique solutions and in case of parmetrized energy, conditions for achieving refinement ordered solutions with increasing parameters. In said studies, parametrized energies are of the Lagrangian form ω ϕ (π) + λ • ω δ (πS), where ω ϕ (π), ω δ (π) represent a objective and a constraint term on partitions π(E, H) in the hierarchy.

There are many methods to obtain a hierarchy of segmentations, and this thesis assumes its construction to be an independent step. We will thus consider the hierarchies themselves to be the input of our optimization problem.

Convex vs. Lattice based optimization

Most of conventional mathematical optimization depends on conditions ascertaining local optimality that implies global optimality. Consequently the theory has evolved into subject of study of convex sets. In contrast, the thesis uses lattice-theoretic approach and follows in the lines of lattice programming methods such as [11,[START_REF] Topkis | Supermodularity and Complementarity[END_REF] which are concerned with the order of optimal solutions and so are led to a development based on lattices.

The important difference between convex analysis and lattice programming methods are seen when one removes one of the following restrictions: differentiability, convexity(or concavity), continuity, local analysis, and adds the following qualities: using order-based

properties. This has been demonstrated in the area of pricing in economical applications by the Topkis Theorem [START_REF] Topkis | Supermodularity and Complementarity[END_REF], and in the area of sub-modular optimization [12].

As an example if one considers integer constraints, e.g., one must order in multiples of a given batch size like a case, a box, etc. This destroys convexity, but preserves sublattices. Thus the presence of integral constraints enormously complicates the results of convex programming. By contrast, the monotonicity results of lattice programming carry over without change to their integer counterparts [11].

A Morphological Approach

In earlier work which performed constrained optimization over the HOP in, Guigues et al. [47], the optima were ordered based on the choice of the scale parameter λ. This that was left open as choice, thus calculating all the ordered set of partitions corresponding for each feasible value of multiplier λ. [START_REF] Garrido | Hierarchical Region Based Processing of Images and Video Sequences: Application to Filtering, Segmentation and Information Retrieval[END_REF][START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF] provided a rate-distortion interpretation to the optimization problem following from the work on tree structured vector quantizers [START_REF] Gersho | Vector Quantization and Signal Compression[END_REF], and calculate the λ that got closest to the constraint.

These methods do not use the inherent lattice structure. On a HOP with partition dependent functions, like perimeter, there is no guarantees of continuity and topological constraints of these functions. For example, one cannot describe the convex composition of partitions, and must accept its discrete combinatorial nature. Furthermore we will see how even when the perimeter is rational or even real valued there might be instances when we cannot achieve the desired constraint value.

The area of Mathematical Morphology [101] is well known for its use of lattice structure in image processing. Transformations, like the dilation, erosion, in morphology are defined on complete lattices which are partially ordered sets, containing a unique infimum and unique supremum. Morphological operators on such lattices frequently lead to scales spaces, these are classical examples of non-linear scale-spaces. In this thesis we will introduce a numerical lattice structure on partial partitions from the HOP. The Lagrangian approach to the problem, in the thesis is substituted by a generalized energy based ordering of partial partitions and a subsequent lattice to define the optima. While operating on different energetic lattices corresponding to the objective, constraint and Lagrangian, it generalizes Guigue et al's [47] method to obtain monotonically ordered optimal cuts, for a constrained optimization problem on the HOP. Further the dynamic program first used in classification and regression trees, by Breiman et al. [21] and later generalized for the HOP in Selmbier- Garrido & Guigues [47,[START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF], are further generalized in this thesis for non-linear energies.

Further the hierarchy of partitions were a result of a multi-scale segmentation step that produced either nested or disjoint classes. This is usually a result of a greedy aggregative step from the application of HAC algorithms. We present in the thesis a more relaxed and richer structure, which is still hierarchical, while allowing non-nested partitions. This is the Braids of partitions, and the advantage is that we have a larger structured 3 0.2. THESIS ORGANIZATION space of solutions where the energetic lattice structure works and hope for a better infimum on the energy.

Thesis Organization

The organization of the thesis is as follows. While one can find the contributions summarized in the conclusion section.

Chapter 1

Definitions: To begin with, this chapter provides the basic definitions including the partial partition, hierarchies of partitions, energies, classes, and a basic recall of lattices.

Review on Optimization on Hierarchies: Following this a review of the constrained optimization problem hierarchies is presented, with a recapitulation of the basic problems.

Braids and Energetic Lattices: This is a crucial section and contribution of this chapter introduces the new structure of Braids of partitions(BOP).

And also the key definitions of energetic ordering, and energetic lattices, followed by properties of multi-scale energies, namely h-increasingness, scaleincreasingness and singularity. Examples of different compositions and their optimal cuts are demonstrated with simple examples.

Chpater 2

Rate-distortion Interpretation: This bibliographic section 2.1 recapitulates the Rate-Distortion theory and its applications in tree based source coding problems. Following which, we lead to the work of [47,[START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF].

Lagrangian Relaxation: Firstly we discuss Lagrangian of the constrained optimization problem on the HOP, already established in [47,[START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF]. We establish how this is a Lagrangian relaxation, and give important implications with respect to the Lagrangian dual problem. Further we demonstrate how one can use penalty based methods to obtain a better upper bound on the minimization.

Three models: We provide three models for the constrained problem on a general Braid of partitions (and thus HOP as well), namely the Lagrangian relaxation using the energetic lattice, A partition or Cut constraint based Lagrangian relaxation, and finally a class based local constraint model.

Chapter 3

Experiments and Demonstrations: The first part, demonstrates the applications of the energetic lattice, in the problems of, calculating the optimal cuts for different image segmentation models, mainly Mumford-Shah. We introduce here the problem of extraction of partition from a hierarchy that is proximal to a ground truth partition, and its resolution with the energetic lattice based on local Hausdorff distances. We introduce further evaluation measures for hierarchies of segmentations w.r.t ground truth partition.

Chapter 4 studies the saliency function and how it can be defined by considering the numerical lattice over a finite family of Jordan curves partitioning the Euclidean plane. This is an extension from Chapter 3, where the energetic lattice is no more defined on the partial partitions, but purely on contours of an initial fine partition. Demonstrations include, transforming a hierarchy by performing openings on the lattice of Jordan curves, creating a braid of partition by operating on intersection of Jordan nets.

Chapter 5

Algorithms: This section compiles the important algorithms, including the dynamic program on hierarchies by Guigues, the generation of braids, and finally the extension of the dynamic program to the braids.

Partition Graphs: This section reviews maximal independent sets on intersection graphs for a family partitions and the formulation of image segmentation problem as the calculation of a Maximally weighted independent set of the intersection graph first proposed by Brendel-Todorovic [START_REF] Brendel | Segmentation as maximum-weight independent set[END_REF]. Further we recall the definition of a partition graph which is an intersection graph over a family of subsets, that form a covering of the image domain E, where the maximal independent sets of this graph, are partitions of E. This basically is mapping between the MIS set and the partitions or cuts from the hierarchy.

This correspond to a address partitions from the covering, and helps repair few pathological cases in case of Todorovic et al. [START_REF] Brendel | Segmentation as maximum-weight independent set[END_REF]. The chapter ends with a Max-flow formulation on trees, which reformulates the optimal cut problem.

The case of braids of partitions are studied for both the Partition Graph and Max-flow formulation.

Chapter 6 provides a conclusion to the thesis. It discusses the main contributions of the thesis and its future perspectives.

Chapter 1

Braids and Energetic Lattices

Publications Associated with Chapter

• [59] Global-local optimizations by hierarchical cuts and climbing energies, Pattern Recognition(PR) 2014.

• [START_REF] Serra | Optima on hierarchies of partitions[END_REF] Optima on Hierarchies of Partitions ISMM 2013

• [62] Climbing: A unified approach for global constraints on hierarchical segmentation, ECCV 2012 Workshop.

• [START_REF] Serra | Hierarchies and climbing energies[END_REF] Hierarchies and climbing energies, CIARP 2012

After a brief reminder on lattices, partitions and hierarchies, this chapter develops theoretical tools necessary for the rest of the thesis, namely the braid of partitions and energetic lattice. Though several notions are defined to analyze constrained optimization, we will actually deal with the problem only in the second chapter.

Basic notions and Notations

This section presents the basic constructs of the hierarchies of partitions (HOP). The space E under study is arbitrary, continuous or discrete, finite or not, topological or not. P(E) represents the power set of set E. The elements x, y of E are called points, or leaves. where S : E → P(E), and for each point x ∈ E, we have x ∈ S(x), and

x, y ∈ E ⇒ S(x) = S(y) or S(x) ∩ S(y) = ∅ (1.2)

These S are called the classes of the partition π.

Intuitively, a partition is a division of image domain E into classes which are pair-wise disjoint and whose union restores E in its entirety. We also refer to the partition as π(E) in some cases.

First introduced by Ronse in [START_REF] Ronse | Partial partitions, partial connections and connective segmentation[END_REF], a partial partition is a local partitioning of a subset S ⊆ E of the input space.

Definition 1.2. A partial partition π(S) of support S ∈ P(E) is a set,

π(S) = {A i |A i ⊆ S, A i ∩ A j = ∅} (1.3)
where S = ∪A i , is called the support of partial partition π(S).

An example is demonstrated in figure 1.1. The partial partition of S into the single class S is denoted by {S}.

The set of all partitions π of E forms a complete lattice Π(E) for the partial ordering of the refinement, where π i ≤ π j when each class S i (x) of π i is included in the class S j (x) of π j at the same point x ∈ E:

π i ≤ π j ⇔ S i (x) ⊆ S j (x). (1.4)
This refinement lattice is denoted by Π(E). The refinement infimum of a family {π i , i ∈ I ⊆ R} in Π(E) is the partition π whose class at point x is ∩S i (x), and the refinement supremum is the finest partition π such that S i (x) ⊆ S (x) for all i ∈ I and x ∈ E. This is demonstrated in figure 1.1.

Hierarchy

Hierarchies of partitions are the matter of an abundant literature (see for example [10], [START_REF]Connected operators based on tree pruning strategies[END_REF], [78]). The definition that we propose here is based on two axioms: Definition 1.3. (Hierarchy of Partitions(HOP)) A family {π i , i ∈ I ⊆ Z} of partitions of E defines a hierarchy when, (i) The partitions π i are nested, i.e. they form a chain for the refinement ordering:

H = {π i , i ∈ I} with i ≤ k ⇒ π i ≤ π k , I ⊆ Z, (1.5)
where the finest partition π 0 is called the leaves, and the coarsest one, is the root;

(ii) The number of leaves is finite in any class of the hierarchy, except possibly, in the class {E}.

One often takes the whole space {E} for the root. A toy example is given by the following hierarchy of nested partitions of Z where the central class enlarges: In earth sciences, most of the phenomena are not studied in a finite domain. For example Air-borne and satellites images are of this type. Optimal segmentation of such phenomena must be reached by local, or regional information, and not via a global energy, which would involve the whole space, which needs intricate treatment. Here Axiom (ii), opens the door to a regional approach, a door which leads to the energetic lattices.

Classes

A hierarchy can be described from its classes, or nodes. At each point x ∈ E the family of all classes S i (x) containing x forms a closed chain of nested elements in P(E), from the leave S 0 (x) to E. This chain is called the cone at point x. Let S ={S i (x), x ∈ E, i ∈ I} be the family of all classes of H. One directly extends to S the characterization (1.2) of a partition by its classes containing any points x, y ∈ E, i ≤ j ⇒ S i (x) ⊆ S j (y), or S i (x) ⊇ S j (y), or S i (x) ∩ S j (y) = ∅.

(1.6)

The classes of the partition π i-1 included in the class S i of the partition π i are the sons of S i . The symbol refers to the disjoint union of classes, i.e. S = S 1 S 2 ⇔ S 1 ∪ S 2 = S and S 1 ∩ S 2 = ∅.

A cut of H is a partition of the space E into classes taken in S. The symbol Π(E, H) stands for the set of all cuts of H. Clearly, Π(E, H) is a sub-lattice of Π(E), the lattice of all partitions of E. If S ∈ S(H), then Π(S, H) denotes the family of all partial partitions of S whose classes are in S(H).

Energy

An energy ω is a real valued function over the family of partial partitions D(E) of space

E: ω : D(E) → R (1.7)
When the energy ω of a p.p. a is the sum of the energies of its classes, then ω is linear, or separable in terms of Guigues [49,[START_REF]Connected operators based on tree pruning strategies[END_REF][START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF], and can be written now using the general definition from equation 1.7 as:

ω(π(S)) = A i ∈π(S) ω(A i ) (1.8)
Furthermore in equation 1.8 when ω(π(S)) ≥ ω(S), i.e. the sum of energies of classes in the p.p of support S is greater than or equal to the energy on the single class {S}, then ω is sub-additive. Both assumptions of linearity and sub-additivity will be generalized in our theory, already visible in the definition of the energy in equation (1.7). One can find work on the valuation of partial partitions by Ronse in [98, 99].

Recall on Orderings and lattices

A set is (partially) ordered when a binary relation ≤ is defined on its elements, with a ≤ a (reflexivity), a ≤ b and b ≤ a ⇒ a = b (anti-symmetry), and a ≤ b , b ≤ c ⇒ a ≤ c (transitivity). The ordering is total when all pairs of elements of E are ordered. The sets of Z 2 are ordered for the inclusion, the real numbers are totally ordered for the numerical inequality.

A lattice is an ordered set L in which any two elements have a greatest lower bound (g.l.b.) and a least upper bound (l.u.b.). They are denoted by ∧ and ∨ respectively.

When this property extends to any family of elements, possibly infinite, the lattice is said to be complete. A finite lattice is thus always complete. A lattice contains always two extreme, or universal, elements, namely the least and greatest of all. A basic example of a lattice is provided by the elements P(E) of an arbitrary E. They are ordered by inclusion, the two bounds are the intersection ∩ and the union ∪, and the universal elements are (∅, E}. An ordering relation does not systematically produce a lattice, for example when a connection is defined on E, the connected sets of E do not form a lattice, unlike P(E), though they are ordered by inclusion.

A sub-lattice of L is a family L ⊆ L closed under ∧ and ∨ (a notion sometimes called pseudo sub-lattice in literature). For example, the family of all sets contained in a zone Z ⊆ R 2 is a sub lattice of P(R 2 ). Finally, a totally ordered sub-lattice is called a chain.

The lattice is flexible, because a same family of mathematical objects may be the matter of several different lattices. For example, the braid cuts will be provided with two lattices in this chapter, and with four lattices for constrained optimization, considered in chapter 2. Furthermore the lattice structure assures us that any family has a minimal and a maximal elements, both unique, and which belong to the lattice. A set Π of partitions of E (e.g. the braid cuts) cannot be significantly modeled by a topological vector space, so that one cannot speak of the continuity, or the zero gradient, or the convexity, of a real valued function over Π. Minimization questions must be addressed in another manner, what precisely does the lattice approach. 11

Optimization on Hierarchies of Partitions

In this section we review the problem of optimization on HOP in literature.

Classification and Regression Trees

Classification and Regression Trees (CART) were introduced in the 80's by Breiman et al [21], which creates powerful and simple binary tree based models for classification and regression problems, in statistical learning theory [START_REF] Hastie | The Elements of Statistical Learning (2nd edition): Data Mining, Inference, and Prediction, Second Edition[END_REF]. In both cases, the method consisted in creating rectangular partition of a feature space (high dimensional R n ), either fit a model over each of these rectangles in case of regression, produce a classification.

These trees (now called decision trees) described then the estimator for the regression function, or a linear separator for classification tasks. The trees we consider here only address recursive binary partitions as show in in the figure 1.2.

To avoid over-fitting the data, such trees are dealt with commonly in two broad ways:

Firstly, prune them (which is what we will consider for the rest of our study). Secondly one can use aggregations of different subset of points to average over many such trees, which also includes the classical Random Forests which consists in using a random subspace of features to partition a node and aggregate them.

Pruning CART trees

First the tree is grown until a minimal number of samples per class is reached. The growth is done by splitting pre-existing classes such that at each step the split point picked minimizes the quadratic deviation from the mean value, in each of the split produced. This is the case for regression, while for classification we have the node impurity and entropy based measures [37].

Optimal tree pruning

Deciding the tree size is important parameter since this describes the model's complexity as well the solution space being spanned. Breiman et al. proposes to grow a initial large tree T 0 , until a minimal number of samples per rectangular class is reached (a minimum number of points of population), following which a pruning is performed to reduce the complexity of the tree for the purpose of classification or fitting a regression model for function.

R 1 R 2 R 3 R 4 t 1 t 2 t 3 X 1 → ↑ X 2 t 1 ≤ 2 t 2 ≤ 1 R 2 R 1 t 3 ≤ 3 R 3 R 4 Figure 1
.2: A two dimensional feature space is recursively partitioned using a binary tree where the partitions consist of purely rectangles. The corresponding binary tree is shown. One continues growing the tree until each rectangular encloses a minimal number sample points.

Tree Terminology: Considering Breiman et al. 's [21] notation for the trees just for this section, we denote by T 0 the complete non-trivial binary tree that is grown by recursive binary partitioning of the feature space. T 1 is said to be a subtree of T 0 of the triple T 1 , left 1 (•), right 1 (•) forms a tree. If T 1 is a subtree of T and T 2 a subtree of T 1 then T 2 is subtree of T 0 . Given any node t ∈ T 0 , the subset T t consisting of t and all its descendants is called the branch of t stemming from t. This is a subtree of t. Let T denote the terminal nodes of the tree. A tree T is trivial if the cardinality of set is empty, i.e. |T | = 1, or also when the set T -T is empty. A subtree T 1 of T 0 is called a pruned subtree of T 0 if root(T 1 ) = root(T 0 ); this is denoted by T 1 T 0 . is transitive, and so is ≺. This provides a top-down definition of a pruned subtree, while Breiman also provides a bottom up definition.

Any arbitrary subset T 1 of T 0 is a subtree of T 0 having a root t, if an only if it is a pruned subtree of T 0 rooted at t, iff, it is a pruned subtree of T t . Given now a real number λ (α in Breimans notation), lets define, R λ (t) = R(t) + λ for t ∈ T 0 . Given a subtree T of T 0 , let, R(T ) = T R(t) and

R λ (T ) = T R λ (t) + λ T .
Cost complexity Pruning: If T T 0 be a subtree obtained by pruning T 0 (Pruning a branch T t from a tree T consists of deleting from T all descendants of t), and R m represents the m different regions representing the terminal nodes of tree T .

The optimal pruned subtree is one which minimizes:

13

R λ (T ) = |T | m=1 x i ∈Rm (y i -µ Rm ) 2 + λ T . (1.9)
over each node which results in an optimal pruned subtree T λ T 0 for a given λ.

Following [START_REF] Hastie | The Elements of Statistical Learning (2nd edition): Data Mining, Inference, and Prediction, Second Edition[END_REF], here µ Rm is the mean value of observed variable y in the region R m . λ is a parameter that governs the trade-off between tree size and fidelity to data.

The expected training error is non-monotonic with the subtrees of decreasing size. This

shows that as the trees initially decrease in size, the error rate decreases. The error reaches a minimum at a particular subtree size and begins to climb again as the trees get too small. This behavior can be attributed to a trade-off between bias and variance.

The optimal λ is adaptively determined by cross-validation. For each λ the weakest link pruning, i.e. successively collapse each internal node, that produces the smallest per node increase in the in an optimal pruned subtree's quadratic term in 1.9, is performed and continued till the root. λ minimizes the cross-validated sum of squares.

Uniqueness and Monotonicity

In defining a Cost-Complexity measure that is a linear combination of misclassification error and tree size, Breiman et al. [21] analyses the following questions:

• Uniqueness Is there a unique subtree T ≺ T 0 which minimizes R λ (T )?

• Monotonicity In the minimizing sequence T 1 , T 2 , ... is each subtree gotten by upward pruning from the previous subtree, i.e. does the following nesting relation hold: T 1 T 2 , ..., T 0 .

Optimally pruned subtree: A pruned subtree T 1 of T 0 , is said to be one of the optimally pruned subtree of T 0 , with respect to λ if:

R λ (T 1 ) = min T ≺T 0 R λ (T ) (1.10)
Since there are only finitely many pruned subtrees of T , there is clearly an optimal one, but not necessarily a unique one. An optimally pruned subtree T * is said to be the smallest optimally pruned subtree of T 0 if T * T for every optimally pruned subtree T of T . There is clearly at most one smallest optimally pruned subtree of T (with respect to λ); when it exists, it is denoted by T (λ).

To avoid the combinatorial problem of choosing an optimal subtree from the exponentially growing number of choices,

• R λ (T (λ)) = min T ≺T 0 R λ (T ), • If R λ (T ) = R λ (T (λ)), then T (λ) ≤ T
R(t) > R(T t ) (1.11)
where R can be for example the error term in 1.9.

This gives us the core idea of the cost-complexity pruning, in that it starts with a pruned subtree, eliminates the weakest sub-branch, to produce the next optimal pruned subtree. This is called weakest link cutting.

For a singleton node {t}, where t ∈ T 0 let consider,

R λ ({t}) = R(t) + λ(1).
For any branch T t rooted in t, lets define

R λ (T t ) = R(T t ) + λ Tt .
The value of λ at which T t (child) becomes equally optimal in complexity w.r.t the subtree root node (parent) t, we solve the inequality R λ (T t ) < R λ ({t}), giving us Breiman also defines a mapping that spans all nodes and associating the critical parameter value in equation 1.12.

λ < R(t) -R(T t ) |T t | -1 (1.12)
Λ 1 (t) =      R(t)-R(Tt) | Tt|-1 , t / ∈ T1 +∞, t ∈ T1 (1.13)
The weakest link t 1 in T 1 is a node such that

Λ 1 (t 1 ) = min Λ(t), t ∈ T 1
and for the next new complexity parameter value put

λ 2 = Λ 1 (t 1 )
After this step we define a new tree

T 2 = T 1 -(T t 1 )
We continue performing a the weakest link cutting on this new tree, by recalculating a new function Λ 2 . This done until we reach the root. This yields a monotonically ordered set of optimally pruned subtrees, T 1 > T 2 > T 3 > ... > t 1 .

Departing from Binary Trees

One can observe that the conditions for optimal pruning are not restricted to binary trees. Guigues [47] provides a generalization for the weakest link pruning on a general hierarchy of partitions. Though it is also notable that any general tree can be rewritten as an equivalent binary tree, and such reorganization of tree topology don't actually change the optimal cut, i.e. the leaves set of the optimally pruned subtree.

We shall demonstrate this with a quick numerical example in figure 1.3. As seen in this figure, one can consider any general tree with a monotonically increasing function R in equation 1.11 on the nodes of the tree in the place of the variance term, while just adding a complexity cost of 1. This penalizes subtrees which have too many leaves. We can see that the cut progressively moves up the tree to fewer number of classes as the value of λ increases. This is a very simple version of the constrained optimization problem on the tree. From now on we will simply refer to the optimal subtree as the optimal cut, both of which refer to the same result as demonstrated.

Salembier-Garido's Optimal Pruning & Guigue's Scale-set

Salembier-Garrido and Guigues [47,[START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF] generalized the CART framework for the constrained optimization problem. Salembier-Garrido study binary partition trees, which represent a hierarchy of partitions created by using the max-tree representation, while Guigues considers a hierarchy created from complete linkage on regions of an oversegmentation [START_REF] Guigues | The hierarchy of the cocoons of a graph and its application to image segmentation[END_REF], titled as Cocoons. In both studies the cost-complexity pruning, a greedy strategy is used to find a constrained minimum, while replacing the constraint of the size of the subtree T by a more general function which is the perimeter of the partition. See Equation (1.14).

One must note now that the difference in these methods w.r.t CART is the interpretation of the constrained minimum. Salembier-Garido and Guigues provide a Lagrangian multiplier interpretation of the optimization [START_REF] Everett | Generalized lagrange multiplier method for solving problems of optimum allocation of resources[END_REF]. This is discussed later in section 2.1.

Constraints are now based on the number of regions and length of perimeter, thus providing a larger range of constraint values, while not requiring a large initial tree to start pruning like in the case of CART. Salembier-Garrido [START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF] provided a Rate-Distortion interpretation of the constraint problem of choosing a segmentation from a hierarchy with the constrained perimeter. We interject here to note that the pruning of trees has been a rich area of study, with recent theses on using them in a variety of problems. In the domain of morphological filtering Yongchao Xu's thesis [123], studies pruning strategies on attribute(of gray scale components) trees, instead of directly on the max-tree. One can also cite the optimal pruning on Binary Partition Tree(BPT), for region based classification map generation on hyper-spectral images, by Valero et al. [119] as well as in Valero's thesis [START_REF] Valero | Hyperspectral image representation and Processing with Binary Partition Trees[END_REF].

Problem review on hierarchies

Now we are in a position to formally state the problem of constrained optimization on the HOP as formulated by Guigues [47]. This generalizes of the complexity of the model in CART as formalized in equation (1.9). where the objective and constraint functions ω ϕ (S), ω ∂ (S) is defined over every partial partition following the separability condition3 . The problem above is solved using a dynamic program which follows from the CART weakest link pruning algorithm. The value C is a constant on bounding the constraint function. Here ω ϕ (S) is identified by

Guigues as the fidelity or data term, while, ω ∂ (S) is identified as the model complexity term. Further he states, that for two models with equal data terms, the simpler of the two should be preferred. This is to choose the parent w.r.t the children, when both have the same energies.

Guigues states the following to be a dual problem, which is, between two equally complex models, the closer to the data of the two should be preferred, giving the following constrained optimization problem:

minimize π∈Π(E,H) S∈π ω ∂ (S) subject to S∈π ω ϕ (S) ≤ K (1.15)
This second constraint problem in equation (1.15) has an interpretation in the ratedistortion interpretation introduce by Salembier-Garrido [START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF]. This will be again discussed in section 2.1.

The constrained optimization problem in equation (1.14) with no conditions on the energies are NP-hard, with an exponentially growing solution space of the lattice of partitions belonging to a HOP.

Guigues does not directly solve the constrained optimization problem in equations (1.14) and (1.15). Instead he provides conditions on ω ϕ , ω ∂ to achieve monotonicity. This is termed as the scale-set which is a family of partitions π λ , which are monotonic mapping of cuts or partitions Π(E, H) of hierarchy H, to the levels of an optimal hierarchy. These monotone cuts of the hierarchy H are a result from a total ordering. We will discuss later why we will not need this additive monotonicity condition and will be replaced by the less strict condition of scale-increasingness 1.22.

Unconstrained optimization:

The problem in equation (1.14) were solved by Salembier-Garrido and Guigues, by performing unconstrained minimization on the Lagrangian function, to solve the constrained optimization problem in equation (1.14). We will now distinguish two types of optimization problems, firstly performing unconstrained optimization for any general energy ω, discussed subsequently in this chapter, while constrained optimization using the Lagrangian will be detailed in chapter 2.

Dynamic program for minimal cut

The algorithm achieving the optimally pruned subtree in [21], also known as the BFOS (after the authors, Breiman, Friedman, Olshen, Stone) algorithm in literature, consists of a dynamic program, that performs a bottom-up scan, while pruning off a branch or the node its rooted in.

Dynamic program consists in solving a structured complex problem by decomposing it into smaller simpler subproblems. These subproblems require to be overlapping in nature, such that a solution to a subproblem is calculated only once and serves to solve a larger subproblem. Furthermore this partial solution when aggregated with others, produces a global solution. The aggregation is quite often has a successive approximation interpretation. Following Guigues, in view of the Bellmenian characteristic of any dynamic program, in order to find the optimal cut, while one performs a bottom up search starting at the leaves level, the algorithm will implicitly have to solve subproblems appearing at higher scales before some lower scales. This means, some nodes appearing early lower down in the hierarchy might be part of a global solution.

In the current case the subproblem consists in comparing the energies of parents w.r.t

to their children. The subproblem can be brought down to over a partial support of the image domain, where one needs to check whether to keep the children or prune it and keep the parent node.

In our terms, it is whether a parent or a child partial partition is retained. This has been used in a more general setting of general partitions from a hierarchy by Salembier-Garrido and Guigues to calculate provisional optimal cut over a given support of the image domain.The optimal structure of the dynamic program is:

ω * (π(S)) = min ω({S}), a∈π(S) ω(a) (1.16) π * (S) =    {S}, if ω(S) ≤ a∈π(S) ω(a) π(S), otherwise (1.17)
Here we see in equation (1.17) that one either chooses the parent class {S} or its child partial partition π(S). The advantage of the partial partition structure can be especially seen here. It helps encode any partitioning of a local support S, and represent the provisional partition in the dynamic program. Further in subsection 1.6.4 on h-increasingness we will see the generalization of this dynamic program with non-linear compositions of energies of classes of child partial partitions, as well as any change in the multi-scale energy which can start out linear at lower scale and turn non-linear at higher scales, as long as it abides by the h-increasingness.

Going from Scale-sets To Energetic Lattices

With the following, we connect the work on HOP in literature with our approach: 4. Uniqueness in the above studies have not been explicated, while leaning on Breiman's condition of the smallest optimal subtree. We achieve uniqueness, by introducing a lattice structure, i.e. energetic lattice.

In the following sections we will introduce the Braids, energetic ordering and energetic lattices, h-increasingness, scale-increasingness, and finally inf-modularity, respectively.

Braids of Partitions

The Braid of partitions, provides a hierarchical structure richer than the hierarchy while lending itself to the constrained optimization problem.

In the lattice of all partitions of a set, the hierarchies form chains, i.e. totally ordered sub-lattices. Can it be possible to construct other sub-lattices, which no longer form chains, while they share hierarchical properties, and help improve the search space for the constrained optimization and the inherent dynamic programming structure? The need for such models arises in several situations, in frequency domain analysis, multivariate segmentations, where partition contours are not totally ordered. One can also The set B 1 = {π 1 , π 2 , π 3 } forms a braid whose pairwise supremum is indicated at the right. One can note now that π 1 (X), π 2 (X) have a common parent X, but π 2 (Q), π 3 (Q) a common grand parent Q. The partition π x cannot be added to B 1 . There does not exist a supremum class, in the monitor hierarchy H, other than from the whole space E, thus not producing a braid structure.

consider situations where hierarchies have to be combined, or enlarged to more adapted structures. One can view such structures as chains of segmentations of an input image where some levels are uncertain, and yield several partially ordered variants. In such cases, it should be preferable to maintain all of variants and to choose among them later, in a dynamic program step.

In figure 1.4 we demonstrate a simple example of a braid with its dendrogram structure.

A formal definition is provided in equation 1.18. As we can see the partitions π 1 , π 2 are not nested nor disjoint, and basically correspond to different segmentation hypotheses that exist in the stack of segmentations.

Definition

To define the braids, we start from the lattice Π(E) of the partitions of where the refinement supremum of any pair π 1 , π 2 in B is a cut of H, other than {E}, and belongs to Π(E, H) \ {E}:

∀π 1 , π 2 ∈ B ⇒ π 1 ∨ π 2 ∈ Π(E, H) \ {E} (1.18)
In other words, the classes of supremums π 1 ∨ π 2 are classes of monitor hierarchy H, and the monitor by itself is not uniquely defined. Though a unique monitor can be imposed by extracting the largest hierarchy (in terms of refinement) whose classes contain the classes of the partitions in the braid. One thus still has a scale selection to perform in the context of chosing a monitor hierarchy for a given application. Just as the cuts of H, which were denoted by Π(E, H), we now define the cuts of B as the partitions whose classes are taken in B, and denote the class of all these cuts by Π(E, B).The hierarchy H may itself belong to the braid, or not. On the other hand, any hierarchy is a braid with itself as monitor. When H ⊆ B, we have Π(E, H) ⊆ Π(E, B) ⊆ Π(E), i.e. the braid cuts Π(E, B) are in between the cuts of the hierarchy H and the set of all partitions of E. A braid cannot be represented by a saliency, except when it reduces to a hierarchy whose classes are connected sets.

Remark:

The partition with one class {E} is not considered in definition 1.18, since this would imply that any family of arbitrary partitions would form a braid with {E} as supremum, thus losing any useful structure. We also assume a locally finite number of classes in such cases, like in the case of hierarchies.

The braids of partitions (BOP) provide an alternative hierarchical structure. In case of a hierarchy the cone or family of classes containing a point x ∈ E, can only be nested or disjoint. While the cone of classes in the BOP, that contain a single point, are not necessarily nested, though their suprema are.

The braid structure's definition is a general and provides multiple ways of creating the braid. We state one direction here, which is a composition law on tuples of hierarchies which produces a braid.

Lack of discriminatory ultrametric:

A classical result about hierarchy tells that the set S of the classes of H is a metric space, where the distance is the absolute difference between the levels of the classes. It is defined as an ultrametric, where the triangle inequality by addition, is replaced by the maximum, which is stricter [START_REF] Leclerc | Description combinatoire des ultramétriques[END_REF]. This ensures a characterization of each hierarchy, by its ultrametric function. In the case of a braid B, because two different classes S and S of π and π respectively may be inserted at 1.3. BRAIDS OF PARTITIONS the same level of the monitor hierarchy H (hence have a zero distance), the ultrametric distances between such classes lose characterization, and the ultrametric may not be very discriminative. However, the classes of B can always be described by other distances (e.g. measure of the intersection, Hausdorff metric in case of tessellation of R n , etc.).

Particular versions of braids have appeared in classification problems, for example Diday [START_REF] Diday | Spatial pyramidal clustering based on a tessellation[END_REF], demonstrates pseudo-hierarchies, called pyramids, where a child may have two parents.

1-D Example:

Given a hierarchy H = {π i }, and when the level index i is odd, let us associate with π i a second partition (residue) π i ≤ π i+1 . This generates the following sequence:

π 0 , {π 1 , π 1 }, π 2 , ..., π 2i , {π 2i+1 , π 2i+1 }, π 2i+2 , ..., {E}, (1.19)
which resemble a braid of hair, by successive enlargements and shrinkages. One demonstrates a more flexible braid using equation (1.19). To do this, let the number of supplementary partitions at the odd levels be random. One can also let the hierarchy H monitor the residue, once in every three levels, at level i say, and introduce supplementary partitions at levels i + 1 and i + 2 with the condition that all these supplementary partitions are smaller than π i+3 . Note that the partitions π k ∈ R may not correspond to some level of H. It is the case for example when π 1 , π 2 , π 3 ∈ R, with π 1 ∪ π 2 = π i and π 2 ∪ π 3 = π i+10 , where π i , π i+10 ∈ H.

An example of the use of braid is depicted in Figure 1.5 Voronoi partitioning are utilized to remove parasitic grains of small sizes. One later decides during the optimization phase from the constructed braid, which of these partitions have minimal energy. This example also shows that the braid based minimization works also when the input is a single partition, the monitor hierarchy is part of the construction step of the braid.

In figure 1.6 we demonstrate a braid created by composing two hierarchies, which are both created from flooding area and volume attributes of a gray scale image. The partitions of the braid consists of partitions from the two hierarchies. Choosing the attribute and gradient function enables one to create a family of non-trivial braids using the watershed transformation. We demonstrate more examples on color and depth as well as an algorithm to generate braids in chapter 5. 5) is removed and replaced with a Voronoi partitioning giving ( 6). ( 7) indicates the intersections of the contours of the initial segmentation (2) and the Voronoi partition in (4), and ( 8) is the net opening (refer to operator in chapter 4) of ( 7) and similarly for ( 9) and (10) with the second parasitic class in (5). (11) depicts the supremum of the two partitions ( 8) and (10). The three partitions (1,[START_REF] Arbelaez | Boundary extraction in natural images using ultrametric contour maps[END_REF]11), with the whole space form the monitor hierarchy H of the braid made by (1,[START_REF] Ravi | Optima on Hierarchies of Partitions[END_REF]5,[START_REF] Arbelaez | Boundary extraction in natural images using ultrametric contour maps[END_REF]10,11), where the (8) = sup (1,[START_REF] Ravi | Optima on Hierarchies of Partitions[END_REF] and ( 11) is the smallest element of H larger than the sup (1,[START_REF] Angulo | Multiscale stochastic watershed for unsupervised hyperspectral image segmentation[END_REF] and also (11) = sup(8,10).

25

Classes of Braids

We now study the classes of any cut π from a braid. The family cuts are denoted by Π(E, B). Here are few properties of these classes.

• ∀π 1 , π 2 ∈ B, π 1 ∨ π 2 contains a finite number of classes of π 1 and π 2 . This is a direct consequence of the second axiom of a hierarchy, in definition 1.3.

• The definition of a braid is transitive. Indeed, Π(E, B) is a sub-lattice for the refinement, so that if π 1 ∨ π 2 and π 2 ∨ π 3 belong to Π(E, B), then π 1 ∨ π 2 ∨ π 3 ∈

Π(E, B). Moreover, the supremum in relation (1.18) extends to infinite families.

Let {S j (x)} be set of all classes at point x of a possibly infinite family {π j , , j ∈ J} of cuts of the braid B. The finite unions of these classes forms a cone of classes in the monitor hierarchy H, since they all contain the point x. By definition of a hierarchy their union S(x) = ∪S j (x) also belongs to H. As point x spans E, the S(x) generate the classes of a cut π of H, which turns out to be the lowest upper bound (l.u.b.), in H, of the family {π j , j ∈ J}.

• The next two properties are the concern of minimal covering of braid cuts by hierarchy cuts. Proof. Let V (x) be the class at point x of the braid cut π ∈ Π(E, B). Given another braid cut π j consider the class S j (x) of the hierarchy cut π ∨ π j . Make π j span all braid cuts Π(E, B)\π. The intersection

S min (x) = ∩ j {S j (x)} ⊇ V (x) (1.20)
is a class of H because the S j (x) are nested, thus are classes of partitions of the chain H as well as their intersection. Therefore, when the point x describes E then the l.u.b.

π min of π in H is generated. The inequality π ≤ π min shows that the class S min (x) of π min at point x is the support of a p.p. of π.

As a consequence, if B is a braid of monitor hierarchy H, then B ∪ H is a braid of same monitor.

For the braid of Figure 1.7 for example, the cut π min (π 1 ) = π min (π 2 ) = π 1 ∨π 2 . Moreover, the case when two S min (x) are ordered is instructive: Proof. The proposition is obviously true when S min 1 (x) = S min 2 (x). Suppose that S min 1 (x) = S min 2 (x). Let y ∈ S min 2 (x)\ S min 1 (x) and S min 1 (y) be l.u.b. class of π 1 at point y. As y ∈ S min 1 (y) ∩ S min 2 (x), which are both classes of the hierarchy H, we have either S min 1 (y) ⊃ S min 2 (x) or S min 1 (y) ⊆ S min 2 (x). The first case is impossible for S min 1 (x) is not empty, and S min (x) and S min (y) are disjoint, thus S min 1 (y) ⊆ S min 2 (x).

Proposition 1.8. Let π 1 , π 2 ∈ Π(E, B)
As this inclusion is satisfied for all points y ∈ S min 2 (x) \ S min 1 (x), we finally obtain that π 1 {S min 2 (x)} is a partition of S min 2 (x).

Motivations

The braid structure by itself can be seen in problems of segmentations and evaluation.

We state briefly the interest of the braid of partitions.

• This type of nested structure is already well known in the area of super-pixel merging, and the generation of "segmentation-soup" in Malisiewicz et al. [69],

where the use of segmentation merging is proven emperically to improve detection support. Furthermore this improvement in detection due to the merging operation is shown to be independent of the segmentation algorithm producing the family of partitions.

• • Multivariate segmentations operate on independent functions, like the case of hyper-spectral segmentations or RGB color images. Partition contours here could have a braid structure given certain compositions of functions. Furthermore, one can envisage a composition of segmentations resulting from independent channels, provided they have a non-trivial braid structure.

• The primary motivation is the use of the braid in Breiman's dynamic program that calculated optimum on hierarchies. In this thesis we show that the dynamic program works for a larger class of partitions which is the braids and further more it ensures better optimum than hierarchies, when operating on braids with non-trivial monitors. This is further discussion in detail in section 1.5, and demonstrated in figure 1.14. No single Ground truth segmentation is a refinement of the mean-shift segmentation, but their suprema are. This is a result in the inconsistent partition contours in case of both the human and machine based mean shift segmentation. Image reproduced from Unnikrishnan et al. [START_REF] Unnikrishnan | A measure for objective evaluation of image segmentation algorithms[END_REF]115]. This helps handle boundary ambiguity.

Energetic lattices of braids

First of all, what does "minimal cut" mean? A cut of minimal energy? For a finite set of only 25 leaves partitioning E generates 0.5×10 18 different potential partitions, following the bell's number. The set of all energies, which is in practice an interval in the positive integers, risks to be too poor for the purpose, and we may try and act directly on some lattice of cuts, which should of course involve the energy ω by some modalities. Then the existence and uniqueness of minimal cuts will be ensured by this lattice structure itself.

In figure 1.9 we demonstrate how one can obtain multiple optimal cuts. One requires singular energies to ensure the existence of a unique optimum.

Definition 1.9. Let ω be an energy on the partial partitions D(E), and B be a braid B of monitor hierarchy H. Energy ω is singular when (i) the energy ω({S}) of every class S of H is either strictly smaller, or strictly greater, than the energies of all partial partitions of B of support S:

∀ π(S) ∈ Π(S), ω({S}) < ω(π(S))} or ω({S}) > ω(π(S))}, (1.21) (ii) if ∀π 1 , π 2 ∈ B and π 1 ∨ π 2 = {S} ∈ Π(E, H), then ω(π 1 ) = ω(π 2 ).
This is demonstrated in figure 1.9. Definition 1.9 extends the cone structure at point x defined for hierarchies (i.e. i ≤ j ⇒ S i (x) ≤ S j (x)). In a braid several different classes

S 1 i (x), S 2 i (x) etc.
, may coexist at the same level.

Energetic ordering

Consider two partitions π 1 , π 2 of E, and the class S of the refinement supremum π 1 ∨ π 2 at point x ∈ E. S is the support of two p.p. a 1 of π 1 and a 2 of π 2 (see Figure 1.10).

Intuitively, one may assess that, in some sense, π 1 is less energetic than π 2 for an energy

ω when ω[π 1 {S}] ≤ ω[π 2 {S}] in each class of π 1 ∨ π 2 .
This intuition is true and has the meaning of an ordering relation when ω is singular and B is a braid.

Theorem 1.10. Let Π be a family of partitions of E, and let π 1 , π 2 ∈ Π. Given an energy ω, the partition π 1 is said to be less energetic than π 2 , and one writes π 1 ω π 2 when in each class of π 1 ∨ π 2 the energy of the partial partition of π 1 is smaller or equal to that of π 2: :

π 1 ω π 2 ⇔ {S ∈ π 1 ∨ π 2 ⇒ ω(π 1 {S}) ≤ ω(π 2 {S})} (1.22)
The relation ω is an ordering relation for all singular energies ω, if and only if the family Π(E) is the set of cuts of a braid.

Proof. The reflexivity is obvious. For the anti-symmetry, we observe that π 1 ω π 2 and π 2 ω π 1 involve the same refinement supremum π 1 ∨ π 2 . Therefore in any S ∈ π 1 ∨ π 2 the energies of π 1 {S} and π 2 {S} are the same (Rel. (1.22)). Then the singularity imposes that these two partial partitions are identical.

As for transitivity, consider three elements π 1 , π 2 , π 3 ∈ B, with

π 1 ω π 2 (a) and π 2 ω π 3 (b) (1.23)
We must prove that π 1 ω π 3 . Let V 1 (resp. V 2 , resp. V 3 ) be the class of π 1 at point x,

and For the "only if" statement, we have to prove that the ordering vanishes either when ω is not singular, or when B is not a braid. Consider first an ordering ω whose energy is not singular, and two cuts π and π identical everywhere except in the class S (x) of π , where π is locally the p.p. a. Suppose that ω(a) = ω(S (x)). This implies π ω π and also π ω π. However we do not have π = π since a = S (x). Thus singularity is needed. Suppose now ω singular, and applied to the three partitions π 1 , π 2 , and π 3 as indicated in Figure 1.11. These three partition do not belong to a braid, because the three classes of π 1 ∨ π 2 , π 2 ∨ π 3 , and π 3 ∨ π 1 at point x are not nested. We have π 1 ω π 2 , π 2 ω π 3 but not π 1 ω π 3 , which achieves the proof.

S 1 = S min (V
The "only if" part of the theorem means that the braids, and their cuts, have the exact level of generality to work. As a consequence, all the downstream results which involve the theorem, including energetic lattices and constraint minimizations, are valid for braid

1 1 1 1 π 1 1 1 2 1 1 π 2 3 4 1 π 3 1 1 1 π 1 ∨ π 2 3 4 1 π 2 ∨ π 3 2 π 3 ∨ π 1 x Figure 1
.11: The three partitions π 1 , π 2 ,and π 3 cannot come from a braid, because the three classes of π 1 ∨ π 2 , π 2 ∨ π 3 and π 3 ∨ π 1 at point x are not nested. The values of energy ω for the classes are indicated above them, and the energy of a p.p. is the sum of its classes. The transitivity of the relation ω is not satisfied.

cuts only. If one wants to build up energetic orderings on other families of partitions, the energy ω must be more specified.

Energetic lattice

Given the energy ω, the cuts of any braid B form a complete lattice structure w.r.t.

their energetic ordering ω . We will prove it in two steps, by beginning with the finite families of cuts Lemma 1.11. Any finite family of cuts {π j , j ∈ J} in Π(E, B) admits a greatest lower bound ω π j and a lowest upper bound ω π j .

Proof. As the family is finite, it suffices to prove the results for the pairs of partitions

π 1 , π 2 ∈ {π j , j ∈ J}.
In each class S of the refinement supremum π 1 ∨ π 2 the three p.p. Proof. If B is not a braid, the relation ω does not define an ordering. Suppose B to be a braid, and let {S j (x)} be set of all classes at point x for a family {π j , j ∈ J ⊆ I} of cuts of B. We saw that these classes form a cone, and that their union S M (x) = ∪S j (x), which belongs to H, has a finite number of leaves. Therefore the number of possible partitions of these leaves is finite, as well as the number of different partitions π j {S M (x). Lemma

* = ω {π, π ∈ Π(E, B)}.
It is the unique cut of B smaller than all the other cuts of Π(E, H) for the ordering ω . Remarkably, the theorem was established without assuming any linearity, or hincreasingness, or sub-modularity, of the energy ω. The theorem is prior to these notions, though they will be useful later. Figure 1.12 depicts a toy example of the energetic infimum ω for a HOP.

The three lattices

The assumption of singularity is crucial. If we drop it, we lose all theorems of this paper which involve energetic lattices: the scale increasingness structure is no longer valid, the Lagrange model is undefined, etc.. Fortunately, the singularity hypothesis is not very restricting in practice, where most of the energies admit a singular version, up to minor changes.

In the notation, the three symbols ≤, ∨, and ∧ (without ω subscript) are allocated to the refinement lattice Π(E, B), and ω , ω and ω to the energetic lattice Π(ω, E, B). The expression "minimal cuts" always refers to energy infima ω , the only ones for which the expression makes sense (the refinement universal infimum is the leaves partition).

Moreover, the meaning of the energy infimum is twofold: Proposition 1.13. The minimal cut π * of Π(ω, E, B) is not only the ω infimum of the family of all cuts, but it is also less energetic than every cut π in each class of π ∨ π * .

It thus turns out to be both local and global.

Three lattices (and orders) interact on the family of cuts of a braid B:

1. Numerical lattice ( ≤, ∨, and ∧) for the energies ω, 2. Refinement lattice ( ≤, ∨, and ∧) for the cuts of B, 3. Energetic lattice Π(ω, E, B), again for the cuts ( ω , ω , ω ) h-increasingness in section 1.5, studies the relations between the energetic and numerical order of energies as will be demonstrated in relation (1.25).

Remark 1.14. (Finite energies) Energetic lattices allow for infinite image domain on account of its local nature. Consider a finite window Z centered at the origin, and Z h translated by h. Suppose now that all classes encountered within Z h have finite energies, as do their partial partitions. When not the energies these classes and are infinite. Then, for any point in the space, the cone of classes containing the point, given that the energies of these classes in the cone are finite, of course except E, one can determine now the class forming the minimal cut, containing said point.

h-increasing energies

This section is devoted to the links between an energetic ordering ω on the cuts Π(E, B) and the numerical ordering of the energies of these cuts. The theorem 1.12 says nothing about the energy of a minimal cut, and does not tell whether the energetic ordering π ω π between two cuts implies the same sense of variation for the energies themselves, i.e. ω(π) ≤ ω(π ). Indeed, one easily sees that it is not always the case. For example, take for singular energy ω(π) = 0 (resp. 1) when the number of classes of the p.p. π is odd (resp. even), and greater than 1, the energy of the one class partitions being 2. Then, in Figure 1.10, π ω π whereas ω(π) = 1 and ω(π ) = 0. A new condition is needed, namely that of h-increasingness: Definition 1.15. (h-increasingness) Let (a i , a i ) be elements of two different p.p. of the same support S i , and {S i , S i ∈ E, i ∈ I} a family of disjoint supports. A finite singular energy ω on the partial partitions D(E) is h-increasing when for every triplet {a i , a i ,

S i ∈ E, i ∈ I} one has, ∀i ∈ I: ω(a i ) ≤ ω(a i ) ⇒ ω( a i ) ≤ ω( a i ) (1.24)
When in addition one has ω(a i ) < ω(a i ) for one i at least, and when this leads to ω( a i ) < ω( a i ), then the energy ω is strictly h-increasing.

For example, a linear energy, i.e. an energy where ω( a i ) is the sum of the ω(a i ) is h-increasing, an even strictly h-increasing since

ω(a i ) < ω(a i ) for all i ∈ I ⇒ ω( a i ) < ω( a i ).
Unlike, the h-increasing energy ω( a i ) = ω(a i ) when ω(a i ) < K and = K when not, is not strictly h-increasing. Figure 1.13 shows the geometrical meaning of the h-increasingness.

Here we demonstrate in figure 1.13 the h-increasingness condition generalized to the BOP(bottom), along with the HOP where we necessarily have an ordering of partial partitions on a given support as seen in figure 1.27.

When operating on parametrized energies, across various parameter values, please refer to scale increasingness.

The two orderings and ≤

h-increasingness bridges the gap between the energetic ordering ω for partitions and the numerical ordering of their energies. Consider two cuts π and π of a braid B, and denote by {S i , i ∈ I } the set of all classes of π ∨ π . If a i and a i stand for the p.p.

of support S i of π and π respectively, and ω for a h-increasing energy, then the left member of (1.24) means that π ω π and the right one that ω(π) ≤ ω(π ), hence:

π ω π ⇒ ω(π) ≤ ω(π ).
(1.25) with in particular

π * = ω {π ∈ Π(E, B)} ⇒ ω(π * ) = ∧{ω(π), π ∈ Π(E, B)} (1.26)
The converse is false for a general case, since several cuts can share the same energy, as demonstrated in figure 1.9. However in case of strict h-increasingness the minimal cut is unique, and is infimum of the energetic lattice.

ω(a 1 ) ≤ ω(a 1 ) ⇒ ω(a 1 a 2 ) ≤ ω(a 1 a 2 ) ω(a 2 ) ≤ ω(a 2 ) ⇒ ω(a 1 a 2 ) ≤ ω(a 1 a 2 )
hence ω(a 1 a 2 ) < ω(a 1 a 2 ). Under iteration, this inequality extends to any finite family {ω(a i ), ω(a i ), i ∈ I}, i.e. to Relation (1.24).

Minimal cut and h-increasingness

The finite definition ( As this is true for all partial partitions of E, the energy ω is therefore h-increasing.

There is an obvious extension to the family of Braids. 37

H-INCREASING ENERGIES

To compare the energy of the one class partition {S} to the energies of all its descendants, it suffices to compare S to its sons. The lower descendants do not intervene. Moreover, if ω is not singular, one can always decide to choose ω({S}) when ω({S}) = ω(π), π ∈ Π(S, B). This choice makes ω singular and preserves its h-increasingness (Proposition

of [59]).

What happens if we drop the singularity axiom in proposition 1.18? We risk to meet a node S which has the same energy as the p.p. of its sons. This event introduces two solutions which are then carried over the whole induction. And since such a doublet can occur regarding any node S ∈ H, the number of minimal cuts may become huge.

However at each node S, there is always, among the solutions, a larger partition (for the refinement). By ordering the solutions, we thus structure them in a complete lattice. uniqueness reappears, but the question "find the cut that minimizes the energy" has been replaced by "find the largest (or the smallest) cut that minimizes the energy".

Instead of using the refinement, we can, alternatively, introduce a second optimization.

For example, for color images, ω can hold on the luminance, and the criterion for choosing between the optimal cuts can derive from the product saturation× hue.

Dynamic Program over Braid:

As demonstrated in figure 1.14, the dynamic program substructure would now consist in making a choice between the parent supremum (if it is a class of the braid), and the partial partitions that it monitors. We consider in the figure a braid composed of two hierarchies (this is to be able to index the partial partitions.). We can now write the dynamic program step first shown for HOP in equations 1.16 and 1.17, now for the BOP:

ω * (π(S)) = min ω({S}), a∈π 1 (S) ω(a), b∈π 2 (S) ω(b) (1.29) π * (S) =          {S}, if ω({S}) ≤ min( a∈π 1 (S) ω(a), b∈π 2 (S) ω(a)) π 1 (S), if ω(π 1 (S)) < min(ω({S}), b∈π 2 (S) ω(a))
π 2 (S), if ω(π 2 (S)) < min(ω({S}), b∈π 2 (S) ω(a))

(1.30)

Equation (1.30) demonstrates a subtructure very similar to the hierarchies except now they are applied to the classes of the BOP B. When the energies ω(π a (S)) = ω(π b (S)),

and ω(π a (S)) < ω({S})), we can either pick randomly, as long as we pick one of the partial partitions, so that in a strict sense to keep the energies remain singular.

When a composition of multiple hierarchies H i , i ∈ {1, 2, ..., n}, leads to a braid of partitions B with a monitor H , the dynamic program on a braid consists of either:

• the (n+1)-ary choice between the partial partitions from the n hierarchies, and the monitoring supremum (if considered part of the braid), or

• Independent dynamic programs in the n hierarchies, when the monitoring hierarchies is trivial, i.e. H consists of the complete space as a single class {E}, for all compositions of partitions from the n-hierarchies. In such a case the braid structure does not improve the minimum energy of the optimal cut achieved globally.

Simple example for a non h-increasing energy

ω(π 0 (E)) = 0 ω(π 0 (S)) = 2 ω(π 0 (c)) = 1 ω(π 1 (E)) = 2 ω(π 2 (E)) = 1 E S c a b c
Figure 1.16: Figure demonstrating, initial hierarchy with cuts and their energies(left), the minimal cut by dynamic program(center) is π 2 (E), and the true minimal cut by observing the minimum directly is π 0 (E), the leaves(in white). The dynamic program fails to extract the minimal cut, and produces E as the minimal cut (in gray). This as well implies that we can not use the global-local property of the energy's optimum, even if the energy is singular.

h-increasing compositions and Minkowski norms

h-increasingness is a property of energies, which preserves the optimal substructure in extracting the minimal cut problem so that one can use a dynamic program to solve it.

As one can see, linear compositions is not the only way to ensure that the optimal cut remains, in the provisional optima of bottom up scan in the hierarchy.

The two common modes of composition are by addition and supremum. The additive mode was studied by Guigues under the name of separable energies [47, 49] a context in which he established the Rel. (1.44) below. Denote by {T u , 1 ≤ u ≤ q} the q sons which partition the node S, i.e. π(S) = T 1 ..T u .. T q . Provide the simply connected sets of P(E) with an arbitrary energy ω, and extend it from P(E) to the set D(E) of all partial partitions by using the sums

ω(π(S)) = ω(T 1 ..T u .. T q ) = q 1 ω(T u ). (1.31)
Just as the sum-generated ones, the ∨-generated energies on the partial partitions are defined from an energy ω on P(E) followed by a law of composition, which is now the supremum.

ω(π) = ω(T 1 ... T n ) = ∨{ω(T i )}. (1.32)
Both laws are indeed particular cases of the classical Minkowski expression

ω(π(S)) = u∈[1,q] ω(T u ) α 1 α (1.33)
which is a norm in R n for α > 0. Even though over partial partitions D(E), it is no longer a norm, it yields h-increasing energies for all α ∈ [-∞, +∞]:

Proposition 1. [START_REF] Beucher | A topological approach to hierarchical segmentation using mean shift[END_REF]. Let E ∈ P(E), let ω : P (E) → R + be a positive or negative energy defined on P(E). Then the extension of ω to the partial partitions D(E) by means of

Relation (1.33) is h-increasing.
Proof. We have to prove the relation (1.27), for any two partial partitions π and π of S. When α ≥ 0, the mapping α √ * on R + is increasing and, according to Relation (1.33), the inequality ω(π) ≤ ω(π ) implies

q 1 [ω(T u )] α ≤ q 1 [ω(T u )] α (1.34)
which in turn implies, for the same reason

q 1 [ω(T u )] α + ω(π 0 ) ≤ q 1 [ω(T u )] α + ω(π 0 ) (1.35) hence ω(π 1 π 0 ) ≤ ω(π 2 π 0 ).
When α ≤ 0, the sense of the inequality changes in relations (1.34) [START_REF] Valero | Hyperspectral image representation and Processing with Binary Partition Trees[END_REF][START_REF] Veganzones | Hyperspectral image segmentation using a new spectral unmixing-based binary partition tree representation[END_REF] which all provide h-increasing energies.

Corollary 1.20. If {α j , j ∈ J} stands for a family of non negative weights, then the weighed sum α j ω j and supremum α j ω j of h-increasing energies ω j turn out to be h-increasing.

Unconstrained optimization of the Lagrangian function corresponding to a constrained optimization problem on the BOP is a case corresponding to this corollary demonstrated in chapter 2. A number of other laws are compatible with h-increasingness, such as multiplication.

Minkowski's norm or power mean: Minkowski's norm was used by Allene et al.

[3], to relate the Maximum Spanning Forest(MaxSF), to the graph-cut on edge weighted graphs, with source and sink labels . Cousty et al. related the Minimum spanning forests and Watershed-Cuts [START_REF] Cousty | Watershed cuts: Minimum spanning forests and the drop of water principle[END_REF] again on edge weighted graphs. This transition between various algorithms are done by using the q-th power on the gradient weighted edges of the graph. This preserved the ordering required for the MaxSF and the watershed-cut, for the limiting value of q → ∞, while producing the graph-cut case for q = 1. This In our case though, we use the Minkowski-norm to generalize the h-increasing composition laws, and provide a way to parametrize this choice, and explore this a bit further in the next chapter, for penalty based constrained optimization.

Significance of exponential parameter α:

The value α basically provides a way to determine a scale of a partition by fixing the energy values of the parents and child classes. This parameter α has nothing to do with singularity, while it acts in a complementary way to choose the child or parent classes based on their energies.

A number of other laws are compatible with h-increasingness, outside the Minkowski norm generalization, such as the alternating sum-sup composition laws, demonstrated further down the chapter. Application involving the inf and sum compositions are demonstrated for in Section 3.

Soille's Constrained Connectivity and Hierarchies

In this subsection we demonstrate the different ways of enforcing uniqueness while using different non-linear compositions especially the case of α-flat zones by Soille et al.

[108] and Akcay-Aksoy [2]. Both of these methods choose an optimal cut which ensure uniqueness and monotonicity conditions.

We provide a quick recall of α-connected components or the quasi flat zones [107, 108].

Following the minimum dissimilarity metric to define a single linkage, Soille et al. define the α-connected component to be connected sets within which there is at least one path with a difference in function bounded by α along each pair of points along the path.

Such a path based gradient definition can be defined in R 2 while one defines this on a more accessible pixel-graph with 4-adjacency. This min-metric dissimilarity yields an ultrametric distance and thus a hierarchy of partitions.

α-CC(x) = {x} ∪ {y|∃P (x → y) : ∀x i ∈ P (x → y)} ∧ x i = y, d(x i , x i+1 ) ≤ α. (1.36)
The α-connected component for a hierarchy with increasing α. In other words,

α-CC(x) ⊆ α -CC(X), ∀α ≤ α
The problem with such connected components, like with single linkage is the chaining effect which produces very long chains with small path-wise differences, even when the global contrast (max-min) maybe we large. For such cases Soille et al defines the

(α, ω-connected component containing a point x is the largest α i -connceted component containing x with its global range (sup f (y) -inf f (y), y ∈ α i -CC(x)) ≤ ω. (α, ω)-CC(x) ⊆ (α , ω )-CC(X), ∀α ≤ α , ω ≤ ω 1.6.1.1 (α, ω)-

components composed by supremum

The simplest ∨-composition of energies consists like comparing the supremum of the energies of the classes in the child w.r.t the parent energy. Here we will see that in the (α, ω)-connected component hierarchy, the maximal (α, ω)-component is a choice between parent component and child components that binarized global range parameter ω, and picks the largest. We are lucky here that the ω corresponding to our energy over partial partitions and the ω for the global contrast term in α, ω-components are the same! Consider a binary ∨-energy ω such that for all π, π 0 , π 1 , π 2 ∈ D(E) we have

ω(π(S)) = 1 ⇒ ω(π(S) π x ) = 1, (1.37) ω(π 1 (S)) = ω(π 2 (S)) = 0 ⇒ ω(π 1 (S) π x ) = ω(π 2 π x ). (1.38)
This binary ∨-energy is obviously h-increasing. A numerical function f is now associated with hierarchy H. Consider the range of variation δ(S) = max{f (x), x ∈ S} -min{f (x), x ∈ S} of f inside set S, and the h-increasing binary energy ω k ( S ) = 0 when δ(S) ≤ k, and ω k ( S ) = 1 when not. Compose ω according the law of the supre-

mum, i.e. π = S i ⇒ ω k (π) = i ω k ( S i ).
Then the class of the optimal cut at point

x ∈ E is the larger class of H whose range of variation is ≤ j. When the energy ω k of a father equals that of its sons, one keeps the father when ω k = 0, and the sons when not. As k varies a climbing family is generated.

Dominant ancestor by supremum

Here is an example of ordered energy due to H.G.Akcay and S. Aksoy [2] who study airborne multi-bands images and introduce (up to a small change) µ(S) =Area (S) ×

(mean of all standard deviations of all bands in S). They work with energy maximization.

Allocate a non negative measure µ(S) to each node of a hierarchy H, where µ takes its values in a partially ordered set M , such as a color space. The energy ω is ordered by the two conditions

ω(S) ≤ ω(S ) ⇔ S ⊇ S & µ(S) ≥ µ(S ) S , S ∈ P(E), µ ∈ M . (1.39)
The node S * of the optimal cut at point x is the highest more energetic than all its descendants. The optimal cut π * is obtained in one pass, by Guigues' algorithm [49].

The dominant ancestor pruning is also seen in the maximum decision rule by Valero et al. [START_REF] Valero | Hyperspectral image representation and Processing with Binary Partition Trees[END_REF]. The pruning decision consists of a maximum decision rule which considers that a node is removed if and only if all its descendant nodes can be removed.

Composition of ∨-generated energies

Though the weighted supremum of ∨-generated energies is h-increasing (Prop. ??), the infimum is not. In practice, this half-result is nevertheless useful, since the ∨, paradoxically, expresses the intersection of criteria. For example, when the function f to optimize is color, one can take for energies:

• ω 1 (S) = 0 when range of luminance in S < k 1 , and ω 1 (S) = 1 when not,

• ω 2 (S) = 0 when range of saturation in S < k 2 , and ω 2 (S) = 1 when not.

Then the h-increasing energy ω 1 (S) ∨ ω 2 (S) = 0 when S is constant enough for both luminance and saturation. Figure 1.17: Optimal cuts for composition laws: addition, (α, ω)-component supremum and Akcay's refinement ordering. For the composition by addition, one compares the energies of parent and sum of energies of child nodes. For the binary energy based supremum, in the current example each node consists of α i -components and the value withing each component is the global contrast parameter ω. Here we demonstrate a cut for which the ω ≤ 20. Though this is not a direct composition by supremum. Finally in the Akcay's refinement ordering example the optimal cut picks nodes whose descendants are all smaller that itself. This requires a two pass algorithm. According to the application other laws may be used e.g. both supremum and infimum for the proximity of ground truth with Hausdorff distances [61]. It is interesting to note that the uniqueness conditions in all the 3 cases have been assured by choosing the smallest/largest amongst the optimal cuts in the different cases.

Figure 1.17 shows optimal cuts for three different laws of composition. In a) the additive mode chooses the father S, when ω(S) ≤ ω(T j ). In b) the mode by supremum chooses the S, when ω(S) ≤ ∨ ω(T j ). Finally, in c) one takes the largest node which is more energetic than all its descendants (maximization of ω).

h-increasingness and generalizing DP

Now we describe how the h-increasingness from equation (1.24) provides a non-linear and multi-scale generalization of the dynamic program (DP) in equation (1.17).

The separability condition defined by Guigues is generalized for non-linear compositions while still maintaining the monotonicity of cuts. h-increasingness preserves the order of energies across partial partitions during the intermediary steps DP. This enables one to interleave different composition of energies, thus creating a richer way to describe multi-scale energies.

For example consider the following multi-scale energies:

ω(π(S)) =    a∈π(S) ω(a), if Level(π) ≤ N 2 sup a∈π(S) ω(a), otherwise. (1.40)
This energy of partial partition π(S) is linearly composed for all partial partitions that are contained in partitions whose level in the hierarchy is below N/2, where N corresponds to the total number of levels. Above N/2 all partial partitions energies are composed supremum.

Composition by alternating sum-supremum

We see in figure 1.18 that the optimal cut in such a case is obtained using h-increasingness condition even when we alternate with compositions of addition and supremum. This is very similar to max-pooling in neural networks. One important observation is that the sum-supremum optimal cut is lower-bounded by the optimal cut by supremum, while upper-bounded by the optimal cut by addition. This is the case since

T i ∈π(S) ω(T i ) ≥ T i ∈π(S) ω(T i ) (1.41)
One method to be able to obtain set of cuts between the those of addition and supremum is to alternate the composition rules, but this would still have the problem of choosing parents when supremum is smaller but the sum is not, and vice versa. To this effect we can also consider the h-increasing α-cuts from equation 1.33, that has no effect on the supremum, but produces cuts that are finer than the composition by sum. By alternating the composition laws one can span finer cuts between the composition by sum and supremum. One can refer to [START_REF] Masci | A learning framework for morphological operators using counter-harmonic mean[END_REF] where the This demonstration shows an alternating composition: at each odd level we compose the energies by addition while at each even level we compose the energies by supremum. It's notable that the supremum of child energies is always smaller than or equal to their parent energies.

network structure that learns the morphological structuring element and the composition of operators. The pooling step in such operations are by supremum.

In summary h-increasingness condition applies to composition by addition, supremum as well as infimum. It helps generalize the optimal substructure in the dynamic program that chooses between parent or child based on their energies. h-increasingness applies in case of braids too, which requires a choice now between the monitoring supremum class and the set of child partial partitions being monitored. When the monitor hierarchy does not belong to the braid, the choice is just between the child partial partitions. One can see an example of an alternating composition in Veganzones-Channusot et als.

[121] work on the obtaining an optimally pruned binary partition tree. Here they propose multiple models to perform spectral-unmixing on the BPT, using the variants on the Lagrangian model. One can also find work on optimal pruning on the BPT for region based hyper-spectral image segmentation, by Valero [118]. This thesis demonstrate how one can extend compositions to a multidimensional setting. We shall see an example of infimum in the applications involving partition extraction based on local Hausdorff measures.

Scale increasing families of energies

We first begin by comparing the energetic ordering ω with that ≤ of the refinement, when both apply on the partitions Π(E, B). At a first glance, combining these two structures is not very intuitive. This is because they do not hold on the same features of the partitions Π(E, B). But we can enlarge the scope and consider a family {ω(λ), λ ∈ R} of singular energies which act on Π(E, B). Each energy ω(λ) induces a minimal cut π * (λ).

Remark 1.21. How are the minimal cuts structured as scale parameter λ varies?

In other words what conditions of monotonicity can we establish in the given energetic lattice structure.

Scale increasingness

We will define now a generalized monotonicity property, namely scale increasingness of the family {ω λ }:

S min (x | λ) of each class S * λ (x) of π * (λ) increases with λ at all points x ∈ E i.e. λ ≤ µ ⇒ S min (x | λ) ⊆ S min (x | µ) x ∈ E λ, µ ∈ R. (1.43)
and the partitions

π min (λ) = {{S min (x | λ)}, x ∈ E} form the hierarchy H min = {π min (λ), λ ∈ R} Proof. Put S min (x | λ) = S λ (x) and S min (x | µ) = S µ (x). As H is a hierarchy, we have either S λ (x) ⊆ S µ (x), or S µ (x) ⊂ S µ (x). If S µ (x) ⊂ S µ (x) Proposition 1.8 shows that
there exists a partial partition a µ of π * µ of support S λ (x). The ω(λ)-energy of a µ is ≥ ω({S λ (x), λ}), because S λ (x) is a class of the minimal cut π * (λ) and a µ {S λ (x)}. Then we have by scale increasingness ω({S λ (x)}, µ) ≤ ω(a µ , µ). On the other hand, as π * µ is the minimal cut for the ω(µ)-energetic lattice Π(ω(µ), B), the p.p. a µ is lessω(µ)-energetic than its support {S λ }, i.e. ω(a µ , µ) ≤ ω({S λ (x), µ}), thus ω(a µ , µ) = ω({S λ (x), µ}). But such an equality contradicts the singularity of ω(µ). Therefore we have S λ (x) ⊆ S µ (x).

As λ increases, this inclusion characterizes the cone at point x of a hierarchy, namely that of the partitions π min (λ) = {{S λ (x)}, x ∈ E}, which achieves the proof.

The if part of the theorems already appears in [49] for linear energies. Note that we also have π * (λ) ω(λ) π * µ by scale increasingness (1.42).

Corollary 1.24. The family B = H min ∪ {π * (λ), λ ∈ R} is a braid.

Corollary 1.25. When a braid reduces to a hierarchy H, then the minimal cuts generated on H by the family {ω(λ), λ ∈ R} form a hierarchy, i.e.

λ ≤ µ ⇒ π * (λ) ≤ π * µ λ, µ ≥ 0. (1.44)
if and only if the energies {ω(λ), λ ∈ R} are singular and scale increasing.

Proof. For the if part we observe that Rel.

(1.43) becomes S(x | λ) ⊆ S(x | µ), thus π * (λ) ≤ π * µ .
For the only if part, suppose that there exists a set S ⊆ E for which ω({S}, λ) ≤ ω(π, λ), π ∈ Π(S, H) does not imply ω({S}, µ) ≤ ω(π, µ), hence implies ω({S}, µ) > ω(π, µ). It means that {S}, which is a class of π * (λ), is replaced by π in π * µ , so that π * (λ) π * µ , which achieves the proof.

The theorem 1.23 was stated for a scalar parameter λ. It extends, however, to the vector case. Let λ (in bold) be a positive vector in R p , and {λ 1 , λ 2 , ...λ p } its p positive coordinates. The relation

λ ≤ λ ⇔ λ i ≤ λ i 1, ...i...p
defines a partial ordering. We can go from λ to λ in p "scalar" steps, by firstly changing only λ 1 into λ 1 , then λ 2 into λ 2 ,, so on. Between two steps of the sequence we have, Usual energies, like ω(λ

{λ 1 , λ 2 , λ 3 ...λ p } ≤ {λ 1 , λ 2 , λ 3 , ..., λ p } ≤ {λ 1 , λ 2 , λ 3 , ..., λ p } ≤ {λ 1 , λ 2 , λ 3 , ...λ p }... ≤ {λ 1 , λ 2 , λ 3 , ..., λ p },
) = ω ϕ + λω ∂ , ω(λ) = ω ϕ ∨ λω ∂ , or ω(λ) = ω ϕ ∧ λω ∂ lead thus
to hierarchies of minimal cuts. This nice property can be used to compress a hierarchy by reducing the number levels in a significant manner.

Scale space properties

The theorem 1.23 indicates, indirectly, that the scale increasing families might build up causal scale-spaces, in Alvarez et Al. sense [START_REF] Ravi | Optima on Hierarchies of Partitions[END_REF], i.e. induce semi-groups of operators. Now a semi-group can only be defined when the starting and arrival spaces are the same.

It is not the case with the mapping B 0 → π * (B 0 ). For generating a convenient space, we have to start from the classes S of B 0 , and to introduce the set B(S) = B of all braids whose classes belong to S. Next, we need to provide B with an ordering relation which extends to braids the set-wise refinement ordering. We can state that

B 1 = (H 1 , R 1 ) is smaller than B 2 = (H 2 , R 2 ) when H 1 ≤ H 2 and R 1 ⊆ R 2 .
The first inequality means that at each level i ∈ I the partition π 1 (i) ∈ H 1 is smaller than its homologue π 2 (i) ∈ H 2 ; the second inequality means that there are less partitions in the supplement family

R 1 than in R 2 .
This relation is clearly the matter of an ordering which generates a lattice on B, where the supremum of B 1 and

B 2 is B = [(H 1 ∨ H 2 ), (R 1 ∪ R 2 )], (dual relation for the infimum).
Let ω be a singular energy acting on B 0 , and giving the ω-energetic minimal cut π * .

We observe that π * is the same for all braids B, as it depends only on the classes S of Proof. The map ϕ ω is obviously extensive. It is also idempotent because all braids of B admit the same minimal cut π * , so that

ϕ ω (B) = B ⊗ B * ⊗ B * = B ⊗ B * .
ϕ λ ϕ µ = ϕ µ ϕ λ = ϕ max(λ,µ) . (1.45) Proof. The semi-group from equation (1.45) is equivalent to the implication λ ≤ µ ⇒ ϕ λ ≤ ϕ µ , which is a direct consequence of Rel.(1.44).
The meaning of the semi group is the following: ϕ λ replaces the lower sections of H by a cylinder of section π * λ . As the parameter λ increases, the cylindric part develops upwards, and one can start form any "half cylindric" transform ϕ λ (B) to get the transform ϕ µ (B), as soon as λ ≤ µ.

The Scale Function Λ

We call Scale function the scale of appearance of each class S in a singular and scale increasing family {ω λ , λ ≥ 0}. As the parameter λ increases, there happens a first λ

such that ω λ ({S}) < ω λ (π), π ∈ Π(S, B). Following Guigues [49], we denote by λ + (S)
this first scale of appearance of S. The leaves, which have no descendants, are given λ(S) = 0. Now each point x ∈ E labels nested sets {S i (x)} in the hierarchy. The axiom of scale increasingness shows that the class S i (x) is a candidate to participate in a minimal cut π * µ if its scale of appearance is λ + (S i (x)) ≤ µ and if the scale of appearance of each S j (x), j > i is ν + > µ. Then S i (x) will effectively belong to the minimal cuts for all scales [λ + , ∧ν[ since in this interval it is not covered by a larger class S j (x), j > i.

Inf-modularity

In this intermediary section we describe a property of energies defined on the family of partial partitions akin to sub-modularity for set based functions, which have the property of diminishing returns. Inf-Modularity will be later useful in describing the constraint functions. We will later continue with the resolution of the Lagrangian constrained optimization problem resolution with energetic lattices in section 2.7.

Here we detail the property of the energies on any family of p.p., and does not necessary imply those of braids or hierarchies. We saw the interest in defining scale increasing The inf-modularity resembles the singularity axiom a lot (1.9), but at the same time is different. Firstly, the inequality in equation (1.46) is not strict, and secondly it is a monotonic property all supports S ∈ P(E), unlike singularity which just requires distinct energies. Note that if we replace the infimum of (1.46) by a sum, the condition becomes less severe, which is that of sub-additivity.

Inf-modularity and scale increasingness

For the Lagrange type energies given by equation (2.22), the two notions of scale increasingness and of inf-modularity coincide, but the latter applies to ω ∂ only. More precisely, we can state:

Proposition 1.31. The family

{ω λ = ω ϕ + λω ∂ , λ ≥ 0} is scale increasing if and only if ω ∂ is inf-modular. Proof. If ω ∂ is inf-modular, and µ > λ, we have (µ -λ)ω ∂ ({S}) ≤ (µ -λ)ω ∂ (π), If in addition ω λ ({S}) ≤ ω λ (π)
, then by summing the two inequalities, we obtain ω µ ({S}) ≤ ω µ (π), and the scale increasingness is satisfied. Conversely, if the implication (1.42) holds, then by taking the difference between ω ϕ ({S}) + µω ∂ ({S}) < ω ϕ (π) + µω ∂ (π) and

ω ϕ ({S}) + λω ∂ ({S}) < ω ϕ (π) + λω ∂ (π), we find (µ -λ)ω ∂ ({S}) ≤ (µ -λ)ω ∂ (π), i.e.
Rel. (1.46), which achieves the proof.

The "only if" part of Proposition 1.31 is specific to the ω ϕ (π) + λω ∂ (pi), type energies.

In short, inf-modularity ⇔ Causality (or in our words, additive scale-increasingness), while inf-modularity =⇒ scale-increasingness. For a family such as

{ω λ = ω ϕ ∨ λω ∂ }
for example, the inf-modularity of ω ∂ implies the scale increasingness of the {ω λ }, but the converse is false.

An example of inf-modularity: The example refers to the α -ω-hierarchy by Soille

Inf-modularity Vs Sub-additivity

The concept of inf-modularity we just introduced is to be comparable with that of subadditivity. Remember that an energy ω is sub-additive when, energies over parent class is always smaller than or equal to additive composition of energies on classes of child partial partition. This inequality when ω is not restricted to positive functions, gives sub-modularity [12] which serves as substitute for the convexity when dealing with the subsets of E. As inf-modularity is applied to the partial partitions of E, we firstly need to introduce some energy ω ∂ on sets that corresponds to ω ∂ , by putting

ω ∂ (S) = ω ∂ ({S}), S ∈ P(E), {S} ∈ D(E) 1.8. INF-MODULARITY
with ω ∂ (∅) = 0. Then we must match sets and partial partitions in some sense. To do this, lets consider the comparison of a p.p. π with its classes T j , 1 ≤ j ≤ p. If we take

ω ∂ (π) ≤ p j=1 ω ∂ ({T j }) = p j=1 ω ∂ (T j ), (1.47)
then the inf-modularity of ω ∂ yields inequality

ω ∂ (S) = ω ∂ ({S}) ≤ ω ∂ (π) ≤ p j=1 ω ∂ ({T j }) = p j=1 ω ∂ (T j ),
with π ∈ Π(S, B), which is nothing but the sub-additivity of ω ∂ (i.e. the relation For example, in a partition of the plane R 2 the perimeters ω ∂ of the classes generate an inf-modular energy ω ∂ on the partial partitions. We passed from partial partitions to sets, and vice versa, by the relation (1.47) which restricts the set approach. This formulation, defined over partial partitions (and no longer on sets) using inf-modularity, frees ourselves from this limitation.

ω ∂ (A ∪ B) + ω ∂ (A ∩ B) ≤ ω ∂ (A) + ω ∂ (B)

Summary

Chapter contribution summary

We introduced a new hierarchical structure, namely, the braid of partitions (BOP), that is any general family of partitions whose, where the supremum of any two partial partitions pairs are disjoint or nested. The corresponding hierarchy is called the monitor hierarchy of the braid.

We introduced the Energetic ordering on partial partitions, and the associated Energetic Lattice, which helps define an infimum in the space of partitions.

h-increasingness was used to formally generalize the dynamic program to a very wide variety of linear and non-linear compositions of energies of partial partitions. Furthermore dynamic program for the new BOP structure, remains the same, while providing a larger search space.

Unconstrained optimization over the space of partitions using the energetic lattice was the main structure one studies in this chapter, which generalizes various optimization criteria on HOP [2, 49, 100, 108], while being generally applicable to linear and non-linear energies. Chapter 2 will discuss constrained optimization using the energetic lattice.

The scale-increasing family of energies provide a general multi-scale monotonicity conditions, which generalize Guigue's multi-scale conditions [49]. The energies are defined on the larger family of partial partitions belonging to a braid.

The four axioms of singularity, h-increasingness, scale-increasingness, and infmodularity are conditions on energies only, and not on the image domain E.

This space E maybe topological space, vector space, discrete, or neither of these. Therefore, the notions of convexity, connectivity, are not needed for the main results of the chapter.

The axioms introduced in this chapter serve the purposes of: Singularity → existence of energetic lattices and thus uniqueness, h-increasingness → localglobal solution by dynamic programming, scale-increasingness → monotonically ordered optimal cuts, and finally inf-modularity is non-linear version of sub-additivity property, which will be used to describe constraint functions in constrained optimization problems.

trees while, information theory use them to solve variable-rate source coding problems.

Further, they explain that classification trees can be put into a data compression framework by thinking of the unknown class as a clean probabilistic source corrupted by observation noise and modifying the distortion measure accordingly. The use of tree based models for source coding has been critical in demonstrating how one can achieve rate constraints by alternating across multiple optimal subtrees, even though such a constraint might not exist statically for given distortion level or bandwidth constraint [START_REF] Ramchandran | Best wavelet packet bases in a rate-distortion sense[END_REF][START_REF] Shoham | Efficient bit allocation for an arbitrary set of quantizers [speech coding[END_REF].

The goal of this section is to study the Lagrangian optimization framework used in the different papers, and clarify the importance of the optima. Further we will see the connection with the Lagrangian dual and its importance as in the context of the rate-distortion theoretic [START_REF] Ramchandran | Best wavelet packet bases in a rate-distortion sense[END_REF] framework and the Scale-set framework [47].

Rate Distortion Theory

To give a quick introduction to rate-distortion theory, we will aim at defining the distortion-rate function, which is the distortion incurred for a given bandwidth or rate constraint on the communication channel. prunes a complete code tree T to achieve pruned subtrees S such that we have:

D(R) = inf P Y |X {E[ρ(X, Y )]|I(X, Y ) ≤ R} (2.
D(R) = min S⊂T {δ(S)|l(S) ≤ R} (2.2)
gives rate-distortion function for some pruned subtree of a given tree T .

Image segmentation formulation using Rate Distortion

The rate-distortion function was reformulated in the context of image segmentation where the constraint optimization problem now was defined on the hierarchy of partitions. This was studied first by [START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF] and in more detail in the thesis [START_REF] Garrido | Hierarchical Region Based Processing of Images and Video Sequences: Application to Filtering, Segmentation and Information Retrieval[END_REF] over a binary partition tree structure. Further the thesis of Guigues [47] provides different conditions additivity and separability on energies that can be minimized in the above constraint optimization problem, the solution of which can be achieved by solving a dynamic program. This study has later been extended to the minimization of the Mumford-Shah energy on hierarchy of partitions generated from tree of shapes in [START_REF] Ballester | Level lines selection with variational models for segmentation and encoding[END_REF]. We will review the optimization problem in the new formulation to understand their components:

Caselles et al. [START_REF] Ballester | Level lines selection with variational models for segmentation and encoding[END_REF] view the distortion measure as mean square error between the original image and recovered image,

D = E[(f (x) -g(x)) 2 ].
Here for a maximum allowed distortion D * one can achieve a lower bound on the bit-rate given by R(D * ). Conversely when bounding the rate we have achievable distortion given by D(R * ). This gives us the following two constraint problems which are equivalent:

min R(D), subject to D ≤ D * (2.3)
and while minimizing the distortion min

B∈S b D(B), subject to R(B) ≤ R(B * (λ)) (2.4)
where the rate R(B) is measured in terms of the cost of encoding the curves, plus, the cost of encoding gray level values of the regions.

Caselles et al. [START_REF] Ballester | Level lines selection with variational models for segmentation and encoding[END_REF] following Shoham-Gersho [START_REF] Shoham | Efficient bit allocation for an arbitrary set of quantizers [speech coding[END_REF] use the Everett's theorem [START_REF] Everett | Generalized lagrange multiplier method for solving problems of optimum allocation of resources[END_REF] to obtain the optimal solution B * (λ) by solving the unconstrained problem:

min B∈S b (D(B) + λR(B)) (2.5)
Shoham-Gersho [START_REF] Shoham | Efficient bit allocation for an arbitrary set of quantizers [speech coding[END_REF] clearly suggest that if one can calculate the global minimum to unconstrained Lagrangian in equation 2.5 one has a solution for the constrained problem in 2.4. We will discuss this further showing that this is called the Everett's main theorem [START_REF] Shoham | Efficient bit allocation for an arbitrary set of quantizers [speech coding[END_REF], and is used to solve constrained optimization problems where the objective and constraint functions are not necessarily continuous or derivable.

59

Shoham-Gersho and Garrido [START_REF] Garrido | Hierarchical Region Based Processing of Images and Video Sequences: Application to Filtering, Segmentation and Information Retrieval[END_REF][START_REF] Shoham | Efficient bit allocation for an arbitrary set of quantizers [speech coding[END_REF] suggest the use of successive approximation methods. Garrido repeats a bottom up scan analysis while searching the Lagrangian parameter values in a secant iterative search to approximate the rate function R. While they achieve the desired rate with the smallest distortion possible, they in reality are achieving perturbed and thus approximate solution of the primal problem. In such cases the optimum corresponding to optimal λ corresponds to one that achieves nearly exact rate.

For an understanding of the dual and primal problems and their relations in channel capacity-distortion optimization problems, we refer the reader to 34].

One can note here that duality is not always present inherently and can be overridden by achieving a mapping i.e by some simple mappings of signs, variables, constant parameters, and mathematical operations.

Tree structured Vector Quantization(TSVQ)

The BFOS algorithm [21] has been used to optimally prune tree structured vector quantizers(TSVQ). The reader is highly consulted to read the book on the subject by Gersho-Gray [START_REF] Gersho | Vector Quantization and Signal Compression[END_REF]. This formulation is one of the important motivations for the rate-distortion theoretical framework for constrained optimization and its application in image analysis, with a whole domain developing different ways to quantize signals using the tree structure. The resulting sequence of pruned subtrees would lie on the convex hull of the operational rate-distortion points.

In this section we remark that the tree functionals or energy being minimized are the expected distortion and expected length of codeword. That is the energy to minimize is

given by [START_REF] Gersho | Vector Quantization and Signal Compression[END_REF]:

d = E(d(X, X) + λlen(i( X)) = E(ρ(X, X)) (2.6)
where len() represents the length of the code word, d represents the distortion measure defined on the random variables X and X.

The expected error has a quadratic form, which consequently is also a sub-additive function. The crucial difference to be noted here is that the energy functionals are defined on trees, pertain to a distribution of input signal/image values. The tree structure encodes the signal up to a required constraint, but over a distribution of values, thus allowing a tractable assumption of convexity the subsequent validity of application of the Lagrangian multipliers method.

The existence of a multiplier value for a given constraint function value, has been a problem in the domain of continuous optimization. But in case of TSVQ's applied using wavelet bases one achieves the convex hull of the Rate-distortion Curve by time sharing across two valid constraints to achieve the expected or average constraint value on the R-D curve. In most cases the multiplier method assures only an upper bound.

The purpose of this section on rate-distortion theory was to remind the reader that the constraint and objective functions are defined for distribution X, X. One needs to be careful when one evaluates minimum using the multiplier method for a constraint function such as perimeter or number of classes, since these is no probabilistic model, and furthermore no assurance of achieving a constraint exactly.

Lagrangian Multipliers and Everett's Theorem

The Lagrange multipliers are associated with an optimization problem which is referred to as the Lagrangian dual2 , or simply dual, problem. The role of the dual problem is to define a largest lower bound on the primal value d * of the primal (original) problem.

The important property of the dual is that its concave.

Reminder on Lagrange Multipliers

In continuous constrained optimization, with the space of solutions are n-dimensional points x ∈ R n one poses the constrained minimization problem as:

minimize x∈R n f 0 (x) subject to f i (x) ≤ 0, i = 1, . . . , m. h i (x) = 0, i = 1, . . . , p.
(2.7)

where, f 0 is called the objective function, while f i , h i are constraint functions. The Lagrangian multipliers method consists in obtaining the unconstrained minima of a Lagrangian function. In the processing describing the minima of such a function, one has to have the gradient of the objective function, and the gradient constraint functions, scaled differently by a scalar multiplier. Following [20] Lagrangian is classically written as:

L(x, λ, ν) = f 0 (x) + m i=1 λ i f i (x) + ν i h i (x)
The solution of the the problem in equation (2.7) requires derivatives

∇f 0 (x) + m i=1 λ i ∇f i (x) + m i=p ν i ∇h i (x) = 0 (2.8)
We obtain the classical Lagrangian case when we have purely equality constraints, giving us ∇f 0 (x)+ m i=p ν i ∇h i (x) = 0. This is important to remark in the case of the scale-sets framework, since there can exist a constraint for which one cannot find a multiplier, but one can only find a lower bound.

In our expression for the energy we will use ω ϕ + λω ∂ + νω to represent the Lagrangian, with λ the Lagrange multiplier, where ω ϕ is objective function, to be minimized, and ω ∂ and ω are constraint functions.

The Relaxation Theorem

Given a constraint optimization problem of the form

f * := inf x f (x), subject to x ∈ S, (2.9)
where f : R n → R and S ⊆ R n . One defines relaxation by the following:

f * R := inf x f R (x), subject to x ∈ S R , (2.10) where f R : R n → R such that f R (x) ≤ f (x)
, ∀x ∈ S, and S R ⊇ S. For problem pair in equations (2.9, 2.10) we state the relaxation theorem [80] as:

Theorem 2.1. (Relaxation Theorem) (i) Relaxation: f R (x) ≤ f (x)
(ii) In-feasibility: If the relaxed problem is infeasible so is the original problem.

(iii) Optimal relaxation: If 2.10 has an optimal solution x * R , then

x * R ∈ S, and f R (x * R ) = f (x * R ), (2.11) then x *
R is an optimal solution to original problem in equation (2.9).

Definition 2.2. We call a vector λ * ∈ R m a Lagrange multiplier if it is non-negative and if f * = inf x∈X L(x, λ * ), for a given Lagrangian function L(x, λ).

We can thus state the conditions on the existence of Lagrange multipliers from [80]:

(i) If there is no duality gap, then the set of Lagrange multipliers equals the set of optimal dual solutions (which however may be empty).

(ii) If there is a duality gap, then there are no Lagrange multipliers.

The relaxation theorem is stated here to understand the Everett's theorem. Lagrangian relaxation is employed in optimization problems to find approximate solutions, by decomposing the constraints to produce "easier" subproblems.

Everett's Main, λ and Theorems

Everett's seminal paper studies resource allocation problem as evident from its use by Shoham-Gersho [START_REF] Shoham | Efficient bit allocation for an arbitrary set of quantizers [speech coding[END_REF] to study variable rate set quantizers. For the proofs of the λ, theorems refer the reader to the original paper by Everett [START_REF] Everett | Generalized lagrange multiplier method for solving problems of optimum allocation of resources[END_REF].

Theorem 2.3. (Everett's Main Theorem) Given the multiplier vector λ ∈ R m × R p , and the lagrangian function,

min x∈X {f (x) + µ T g(x) + λ T h(x)}
The solution x(λ) to this unconstrained minimization problem, is also an optimal solution to perturbed primal problem given by minimize

x∈X f (x) subject to g i (x) ≤ g i (x(λ)); i = 1, . . . , m. h j (x) = h j (x(λ)); j = 1, . . . , p.
The Everett's main theorem states the following: for any non-negative λ if an unconstrained minimum of the Lagrangian function can be found, with solution x(λ), then this solution is also the solution to the constrained problem whose constraints are, in fact, the amount of each resource expended in achieving the unconstrained solution. This 63 implies that the constraints are set by choosing the λ vector. Any arbitrary set of nonnegative λ's works here, notably causing the original constraint optimization problem to be unknown, and is only to be defined once the Lagrangian's solutions is determined.

Its important to recall here that there are no conditions of continuity nor derivability on the objective and constraint functions, other than being real-valued.

Theorem 2.4. (λ-Theorem) Given λ k 1 , λ k 2 be two multiplier values that produce solution x * 1 , x * 2 , respectively, and only one of the constraint values are not met, i.e.

g k (x * 1 ) = g k (x *
2 ), k = j and that g j (x * 1 ) > g j (x * 2 ), then,

λ j 2 ≥ f (x * 1 ) -f (x * 2 ) g i (x * 1 ) -g i (x * 2 ) ≥ λ j 1 (2.12)
This theorem states that, given that we have two optimal solutions produced by two Lagrange multipliers corresponding to two different constraint functions, for which only one of the constraints are met, the ratio of the change in optimal objective value to that of the optimal constraint is bounded between the two multiplier values that correspond to the value change. In the case where the objective function values are derivable w.r.t those of the constraints, the partial derivative w.r.t single constraints, gives the corresponding multiplier λ i .

λ i = df (x * ) dg j (x * ) g k constant
The λ-theorem is useful in multiple constraint case, to demonstrate that the multiplier values corresponding to a particular constraint, produce changes in the constraint, where the multiplier measures how far one goes from the objective's optimal value. When applying Lagrangian multiplier methods it is of interest to know the distance from the approximate upper bound to the global minimum, and its variation with the constraint function. This helps in identifying if one is lower bound whose resource-payoff variation is not drastically varying. This following theorem is for case where one can only guarantee an objective value which is away from the true optimum possible.

Theorem 2.5. ( -Theorem)

1.

x comes within of maximizing the Lagrangian, i.e. for all feasible points,

f (x) - λ k g i (x) > f (x) - λ k g i (x) - 2.
x is a solution of the constrained problem with constraint function value g i (x) that itself being within of the maximum for those constraints.

Lagrangian Dual and KKT conditions

We state the KKT conditions to view both the primal and dual problems on the HOP and helps discuss, the feasibility in both domains. The Karush-Kuhn-Tucker(KKT) conditions generalizes the method of Lagrange multipliers to the situation where one has inequalities. We review shortly here, following the [20], the primal problem and its Lagrangian from equation (2.8), without the equality conditions, since we are in such a case:

Problem 1. Lagrange Primal problem:

minimize x∈R n f 0 (x) subject to f i (x) ≤ 0, i = 1, . . . , m.
(2.13)

The Lagrangian for the primal problem is given by

L(x, λ i ) = f 0 (x) + p 1 λ i f i (x).
The unconstrained minimization of the Lagrangian L replaces the initial constrained minimization of f 0 . The λ i are the Lagrange multipliers. Interpreted as the coordinates of a vector λ in R p , they yield the dual Lagrange function g(λ) by the relation:

g(λ) = inf f 0 (x) + p 1 λ i f i (x), x ∈ dom-f 0 (2.14)
Problem 2. Lagrange dual problem: find multiplier λ 03 which maximizes inf-

Lagrangian function: maximize g(λ)
subject to λ 0

(2.15)

The solution to the dual problem in 2.15 for a feasible dual parameter, is only a lower bound to the solution to the primal in general, and are only equal in case of strong duality. This is written as:

g(λ) ≤ f 0 (x * ). (2.16)
which is termed as the lower bound property.

In the domains dom-f 0 and dom-g, of possible solutions in f 0 and g respectively, we always have weak duality when,

g(λ * ) ≤ f 0 (x * ).
(2.17)

We have strong duality when there is no gap, that is the primal's optimum value equals the dual's optimum value,

sup λ g(λ) = inf x f 0 (x), (2.18)
which corresponds to local extrema of both x * and λ * . To summarize, for any optimization problem with differentiable objective and constraint functions for which strong duality obtains, any pair of primal and dual optimal points must satisfy the KKT con-

ditions 4 [20].
Theorem 2.6. (KKT conditions) Let f 0 and {f i , 1 ≤ i ≤ p} be p + 1 continuously differentiable functions: R n → R, and let

(i) f i (x) ≤ 0 1 ≤ i ≤ p (primal feasibility) (ii) λ i ≥ 0, 1 ≤ i ≤ p (dual feasibility), (iii) λ i f i (x) = 0, 1 ≤ i ≤ p (complementary slackness),
(iv) ∇f 0 (x) + p 1 λ i ∇f i (x) = 0, (minima condition), be the four so called KKT conditions. If f 0 and {f i , 1 ≤ i ≤ p} are convex, and x * and λ * satisfy the four KKT conditions, then they are optimal. Conversely, if strong duality holds, and if x * and λ * are optimal, then they satisfy the four KKT conditions .

The first two conditions are the constraints set by the two problems 2.13 and 2.15. The third condition of complementary slackness is obtained when we assume Strong duality.

The condition of complementary slackness basically requires that, at the optimal solution, every constraint that is does not apply, have a zero valued multiplier. This is because, if this constraint does not bind, then we could just as well have solved the problem without that constraint, and setting the corresponding multiplier equal to 0 effectively deletes the constraint from the Lagrangian. In a result similar to Theorem 2.6 for constrained minimization on braids, we present conditions in the following section.

Reviewing constrained optimization on hierarchies

The block diagram 2.1 demonstrates the two chains of ideas, converging on a third one. We will see with a simple counter-example, how the λ-cuts do not correspond to the global minimum under the conditions for sub-additive constraint and super-additive objective, for a particular cost or constraint function value.

Counter-example

The following counter-example considers the Guigues framework of sub-additive and As ω(λ) is scale-increasing, the minimal cuts π * (λ) span bottom-up the hierarchy H.

But this does not mean that they meet all classes of H. A class S is met iff it belongs to an ascending path in the λ-tree. This is not the case, for example, for class j show in Figure 2.2. Following Guigues, we say that these classes are anti-causal. They do not disturb the computation of the λ-cuts or π * (λ). Though the parameter λ ∈ R varies continuously, both ω ϕ and ω ∂ are piecewise constant functions of λ. This is due to the finite nature the hierarchy. One observes that ω ϕ increases with λ, while ω ∂ decreases with it.

Let us consider now consider constrain function value, say C = 7.5 and the minimal cut it may correspond to. In the range of the values around 7.5, the only change which may occur in the minimal cut π * (λ) is the replacement of (a, b, c, d) by (g, h, i) as λ increases.

More precisely,

ω(λ)(g, h) ≤ ω(λ)(a, b, c, d) ⇔ 10 + 4λ ≤ 4 + 6λ
i.e. λ ≥ 3. For λ = 3 the minimal cut is (g, h, i) by singularity, thus:

ω(λ)(a, b, c, d, i) = 8 + 8λ ⇒ (a, b, c, d, i) = π * (λ) for 2 ≤ λ < 3 ω(λ)(g, h, i) = 14 + 6λ ⇒ (g, h, i) = π * (λ) for 3 ≤ λ < 6
For the smallest λ such that ω ∂ (π * (λ)) ≤ 7. What happened is as we will see further, is that we do not have a feasible λ for the dual problem. We have

λ * = inf{λ | ω ∂ (π * (λ)) ≤ 7.5} = 3, but ω ∂ (π *
3 ) = 6 and not 7.5. The is more formally seen in the energetic lattice formulation in Theorem 2.11 cannot give constrained minimal cuts.

Lessons from the Counter-Example

We discuss a few important implications of the counter example:

• Lack of Cost→Multiplier mapping: For a given cost ω ∂ ≤ C one is not assured a corresponding multipler λ, the collection {π * (λ), λ ∈ R} is not informative enough.

A cut that minimizes ω ϕ can perfectly not belong to the {π * (λ)}.

• The dual problem is still a combinatorial problem.

• Uniqueness is lost, even when ω ϕ is strictly h-increasing.

• π * (λ * ) is only the upper-bound of the constrained minimal cuts.

• The values of λ are discrete due to the discrete nature of the cuts of the hierarchy, furthermore these values of λ, can be integral or rational, while still lacking a C → λ map.

• One can always reach any one of these minimal cuts (as there may be many).

It suffices to begin from any arbitrary class of π * (λ * ) and to replace it by its descendants until we reach ω ∂ > C. If one still has ω ∂ (π) ≤ C when one arrives to the leaves, one repeats the descent from another class of π * (λ * ). This helps improve the upper-bound on the λ-cuts, for singular h-increasing functions.

• The error | ω ∂ (π * (λ * )) -C | gives no information about the error | ω ϕ (π * (λ * )) - ω ϕ (π) | where π is a constrained minimal cut.
• One important structure of the constraint problem we see is, for a given cost on the ω ∂ -tree the structure of the solution space forms a lattice. The choice of the partial partition structure, and the energetic lattice to solve the constraint problem, can be seen more clearly in a simple example. The constrained optimization problem will be formulated in a more general framework of the energetic lattice in section 2.7.

• Finally, the counter-example suggests a way to advocate costs independent of λ.

For example it is sufficient to allocate a ω ∂ to each class and not, globally, to the cuts, which is demonstrated in section 2.10 on class based constraints. This is no more Lagrangian, and depends on a class based constraint.

Improving the upper-bound λ-cuts

Let us consider again the constrained optimization problem presented earlier in equation 1.14:

minimize π∈Π(E,H) S∈π ω ϕ (S) subject to S∈π ω ∂ (S) ≤ C
Guigues and Salembier-Garrido's optimization perspective consisted in achieving a λ value that provides a cut with minimal objective and at the same time approaches an given constraint function value over the cut is ω ∂ (π) ≤ C. This basically due to Everett's theorem 2.3. In this section we discuss two approaches which try to improve the upper bounding λ-cuts by perturbing the original problem, so as to achieve a better cut. 71

Perturbing the Scale Function

Guigues's λ-cuts have addressed the problems of monotonicity and constraint satisfaction in just one sense: multiplier-constraint. The constraint function values, correspond to the result of a solution to a perturbed problem where the constraint value is set by the choice of the multiplier. Here we will now perturb the values of the constraint function(or equivalently the objective) so as to achieve a better solution.

Given that we have any general cost ω ∂ (π * (λ i )) < C < ω ∂ (π * (λ j )) bounded by between any two causal λ-cuts, λ i , λ j , more generally between the level sets of the scale function Λ.

Let us consider now the perturbed scale function Λ(S) for each parent class S in the hierarchy or Braid:

Λ(S, ) = ω ϕ (S) -a∈π(S) ω ϕ (a) a∈π(S) ω ∂ (a) + -ω ∂ (S) = ∆ω ϕ ∆ω ∂ + (2.19)
The perturbation corresponds to a small slack variable that is added to the perimeter function ω ∂ . One should note here that the perimeter constraint function has a higher Given λ-cuts we may have nodes for example g, h in figure 2.2 with the same value of λ which are parents of nodes with same λ, here we have set them to be 1 for nodes a, b, c, d. In such a circumstance we have a way to reach a partial partition and thus a cut with a constraint function value closer to C, resulting in a cut which is not a λ-cut.

The basic idea is borrowed from Penalty methods associated with Augmented Lagrangian methods [20]. These methods have two multipliers, one that penalizes deviation from a constraint and the other the regular Lagrangian multiplier. The advantage of these methods are a low span of search values for the penalty multiplier, since the Lagrange multipliers bound the optimal solution. Now we present two approaches for the problem of obtaining a better λ-cut.

λ-Perturbation: For partial partition in the λ-cut, we perturb the values of one of the partial partitions by a very small value λ = λ + ∆λ. The effect is that we have no introduced a new family of λ -cuts that reaches cost values between the initial ω ∂ (π * (λ i )), ω ∂ (π * (λ j )). For example in figure 2.2 between π 2 , π 3 we have π or π as a result of moving the λ values of either g or h by ∆λ.

Penalty Methods

Lets consider the cost C = 5, which corresponds to a cut (j, i) which in the scale-set framework is avoided, since this produces does not result in monotonically ordered cuts, is the Lagrangian like in equation (2.8). For convenience, we begin with a single constraint. We have the Lagrangian ω λ ,

ω λ (π) = ω ϕ (π) + λω ∂ (π) π ∈ Π(E, B). (2.22)
The formalism (2.22) is classical in image segmentation, as well as the Lagrangian in [START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF], [49], when the braid reduces to a hierarchy, and when the energies ω ϕ and ω ∂ are linear, i.e. when the energy of a p.p. is the sum of the energies of its classes. These results strongly rest on this linearity assumption, which is in fact a particular case, one can find in the literature various energies which involve other operations, like suprema or infima, in the α -ω-trees [107], as well as a refinement based ordering in [2]. We will now present a more comprehensive approach, which works across these cases.

Discussion on Everett's theorem 2.6.1 Gap's and lower bounds

The Lagrangian relaxation used by Everett generates a mapping from the multiplier vectors λ into the space of constraint vectors. There is an a priori guarantee that this mapping is onto for a given problem, and one may have inaccessible "gaps" consisting of constraint vectors, for whom λ vectors don't exist always (examples can be easily constructed). Optimum in such cases are not guaranteed by the Lagrangian relaxation method and we would need other means to achieve them. The basic cause of the gap's are the non-concavity in the objective function w.r.t the constraints, i.e convexities in the envelope of the set of achievable objective points in the space of feasible set. [START_REF] Everett | Generalized lagrange multiplier method for solving problems of optimum allocation of resources[END_REF] The method by itself does not guarantee solutions for a general constraint problem, but ensures a lower bound. It is interesting to note that with the λ-mapping we ensure to find a cut that is at least far from the optimal cut.

Another important point to note is that the existence of a global minima for a given function on a general set X using the Weierstrass theorem 5 , which provides the existence of a minimum by necessitating a continuity of the said function [17].

Discontinuous objective and Constraints: For function's defined on partial partitions, such as ω ∂ and ωϕ, the continuity is mostly never true or even defined, these functions are akin to abstract set value functions.

Lagrangian and KKT multiplier methods depend on the ability to describe the local nature of a minima (or extrema), such as the first order or second order conditions, leading to strict definitions of convexity. In case of non-convex functions with global minima ensure, we find the work that uses the "convexified" epigraph of such functions to obtain the said global minima. In the set valued case the possibility of achieving saddle points are also studied in literature, but we restrict ourselves here to the case of partitions.

Retrospective on Optimization by Pruning: In this section we briefly remark in the table 2.1 the development of the pruning based optimization on trees starting with Breiman et al.

[21] while comparing with the energetic lattices. We also study the different characteristics of monotonicity, uniqueness, and the fact that if there has been a Lagrangian interpretation used.

Minimal λ-cuts and energetic lattices

We develop in this section an energetic-lattice based approach to constrained optimization. Three different models of constraints are proposed, firstly numerical constraint on constraint function on partitions, secondly partition based constraint, and finally a class based constraint. The last two reinforce the model of constrained optimization where the energetic lattice is an optimal framework. From now on, ω(λ, π)

= ω ϕ (π) + λω ∂ (π)
is used to denote scalar Lagrangian energy, and ω(λ) its vectorial version (2.22). The Lagrangian energy of cut π is thus

ω(π, λ) = ω ϕ (π) + λω ∂ (π)
A number of lattices are involved in the formalism, we delineate between them, while keeping our notation minimal. The first trivial lattice is of course R, which serves in comparing energies. Moreover the same set Π of all cuts of B, is the matter of the four different partitions lattices:

• Π of the refinement ordering, with the leaves as minimal cut and E as maximal one (the symbol E of the previous notation Π(E) is dropped for the convenience);

• Π(ω(λ)) of the energetic ordering ω(λ) w.r.t the Lagrangian ω(λ), of order ω(λ) .

The minimal cut for this energetic lattice is π * (λ) = ω(λ) {π, π ∈ Π}. The value of ω(λ) for a cut π ∈ Π is denoted by ω(π, λ), and that for the minimal cut by ω(π * (λ));

• Π(ω ϕ ) (resp. Π(ω ∂ )) of the energetic ordering w.r.t. ω ϕ , of order ϕ (resp. the energetic ordering w.r.t. ω ∂ , of order ∂ ). If necessary, one makes the notation more precise, and indicates the hierarchy or the braid under study (e.g. Π(ω ϕ , B);

In the vector case, the vector λ = {λ i , 1 ≤ i ≤ p} replaces λ in the notation, i.e. Π(ω(λ))

→ Π(ω(λ)), π * (λ) → π * (λ), ω(π, λ) → ω(π, λ),
ω ϕ (a λ ) + µω ∂ (a λ ) ≥ ω ϕ (a µ ) + µω ∂ (a µ ).
(2.24)

On the other hand, as π * (λ) is minimal cut for the energetic ordering ω(λ) , we have also ω(λ, a µ ) ≥ ω(λ, a λ ), i.e.

ω ϕ (a µ ) + λω ∂ (a µ ) ≥ ω ϕ (a λ ) + λω ∂ (a λ ) (2.25)
By adding the two inequalities (2.24) and (2.25) we obtain, as λ, µ > 0

ω ∂ (a λ ) ≥ ω ∂ (a µ ).
(2.26)

The inequality (2.26) is true for all supports S max (x | µ), x ∈ E, which results in π * (λ) ∂ π * (µ). Similarly, by subtracting the inequality (2.24) from (2.25) we obtain

2[ω ϕ (a µ ) -ω ϕ (a λ )] ≥ (λ + µ)[ω ∂ (a λ ) -ω ∂ (a µ )] ≥ 0,
which leads to π * (λ) ωϕ π * (µ), and achieves the proof.

Corollary 2.9. If in addition ω ∂ and ω ϕ are h-increasing, then

λ ≤ µ ⇒ ω ∂ (π * (λ)) ≥ ω ∂ (π * (µ))
, and ω ϕ (π * (λ)) ≤ ω ϕ (π * (µ)).

(2.27)

The h-increasingness allows us to apply relation (1.25) to both implications (2.23). Then the two energies ω ∂ and ω ϕ vary in opposite senses over the minimal cuts. The important corollary 2.9 generalizes the λ-cuts by Salembier-Garrido and Guigues [47, 100]: Firstly to braids from hierarchies, and secondly from linear Lagrange families of energies ω ϕ and ω ∂ to various non-linear compositions. Note that ω ∂ does not need to be h-increasing

for obtaining λ ≤ µ ⇒ ω ϕ (π * (λ)) ≤ ω ϕ (π * (µ))
, and vice versa.

Corollary 2.10. The proposition 2.8 extends to vector Lagrange families, and the implication (2.23) still holds when the vector inequality λ ≤ µ replaces the scalar one in (2.23).

Proof. The proof is similar to that of Corollary 1.26. The vector variation from λ to µ can be decomposed into a succession of scalar variations of each coordinate. As the proposition 2.8 applies for each of these scalar steps, we finally get

0 < λ ≤ µ ⇒ π * (λ) ω ∂ π * (µ) and π * (λ) ωϕ π * (µ).
i.e. the vector version of Rel.(2.23).

Lagrangian Minimization by Energy (LME)

This first type of minimization focuses on the energies and performs unconstrained minimization of the Lagrangian to obtain a relaxation of the constraint optimization problem. However, as the set Π of partitions replaces the Euclidean space R n , the notions of continuity, derivability, gradient, and convexity vanish and we work on in the space of lattices. For the sake of pedagogy, we view the case of one constraint first.

The primal and dual problems are re-stated over the cuts of B, and within an energetic lattice: maximize ω(π * (λ))

LME Primal problem: minimize π∈Π(E,B) ω ϕ (π) subject to ω ∂ (π) ≤ C, (
subject to λ > 0

(2.30)

The two problems, (LME) 2.28 and 2.30 will be solved jointly by introducing

λ * = inf{λ | ω ∂ (π * (λ)) ≤ 0}.
(2.31)

The constraint function ω ∂ being inf-modular and h-increasing, corollary 2.9 applies, and

0 ≤ λ * ≤ λ ⇒ π * (λ * ) ∂ π * (λ) ⇒ 0 ≥ ω ∂ (π * (λ * )) ≥ ω ∂ (π * (λ)).
The domain of the feasible λ is therefore λ ≥ λ * .

One must notice an immediate difference between the exact dual of the combinatorial problem and the one proposed here. The space of solutions, in the latter are the λ-cuts π * (λ). We can now set the minimization problem more precisely. In case of the dual, the idea is to span the multplier space.

Three conditions are needed:

1. Primal constraint qualification: the set Π is not empty, 2. Dual constraint qualification: λ * exists and is ≥ 0,

3. Relaxation: ω ∂ (π * (λ * )) = 0.
4. First order condition for minima replaced by lattice: inf Π ω (E, B)

We observe that the two functionals ω(π * (λ)) and ω ϕ (π) are ordered. Indeed, the h-

increasingness of ω(λ) implies π * (λ) ω(λ) π ⇒ ω(π * (λ)) ≤ ω(π, λ) π ∈ Π,
On the other hand, for every doublet π ∈ Π and λ ≥ λ * the Lagrangian ω(π, λ) is smaller or equal to ω ϕ , since its term λ ω ∂ (π) is ≤ 0 (condition 2 ):

ω(π, λ) ≤ ω ϕ (π) π ∈ Π , λ ≥ λ * .
(2.32)

Hence ω(π * (λ)) ≤ ω ϕ (π) and

λ≥λ * ω(π * (λ)) ≤ π∈Π ω ϕ (π), (2.33)
which is nothing but a transposed version of the weak duality inequality (2.17). As π * (λ * ) satisfies condition 3, the Lagrangian ω(π * (λ * )) is reduced to its term in ω ϕ , i.e.

ω(π * (λ * )) = ω ϕ (π * (λ * )) (2.34)
and the inequality (2.32), applied to the doublet {π * (λ * )}, gives

π ∈ Π ⇒ ω ϕ (π * (λ * )) = ω(π * (λ * )) ≤ ω(π, λ * ) ≤ ω ϕ (π). (2.35) This results in ω ϕ (π * (λ * )) ≤ ∧{ω ϕ (π), π ∈ Π }. But π * (λ *
) is an element of Π , hence it belongs to the infimum, and

ω ϕ (π * (λ * )) = {ω ϕ (π), π ∈ Π }, ( 2
ω(π * (λ)) ≤ ω ϕ (π * (λ * )) = ω(π * (λ * ))
. The reverse inequality also holds because the right member is an element of the supremum, and finally

{ω(π * (λ)), λ ≥ λ * } = ω(π * (λ * )) = ω ϕ (π * (λ * )) = {ω ϕ (π), π ∈ Π }.
(2.37)

The weak duality of Rel.(2.33) becomes strong duality for the doublet (π * (λ * ), λ * ) of arguments. This pair solves both primal and dual problems 2.28 and 2.30. At this stage, 81 λ * is unique, but not necessarily π * (λ * ). However, the uniqueness of the solution π * (λ * ) is attained when ω ϕ is strictly h-increasing, by application of Proposition 1.16. This happens, for example, when the energy ω ϕ is linear. In conclusion, we can state:

(iii) ω ∂ (π * (λ * )) = 0,
then π * (λ * ) and λ * are solutions of the problems 2.28 and 2.30 respectively. When ω ϕ is strictly h-increasing, then the solution π * (λ * ) is unique.

Conversely, if Π is empty, or if there is no λ such that ω ∂ (π * (λ)) ≤ 0, then there is no solution. If these two conditions are satisfied, but not the third one, the dual problem is still solved by λ = λ * , but not the primal one.

Vector case

Theorem 2.11 easily extends to the vector case, of Lagrangian

ω(π, λ) = ω ϕ (π) + p 1 λ i ω ∂i (π).
(2.38)

The previous set Π becomes the family Π i = {π, π ∈ Π, ω ∂i (π) ≤ 0}. Applying the corollary 2.10 in the previous proof leads to the Theorem 2.12. (Braid vector constraint minimization). Given the vector

λ = λ 1 , λ 2 ...λ p Let ω(π, λ) = ω ϕ (π) + p 1 λ i ω ∂i (π)
be a vector Lagrange family of energies. Put

λ * i = inf{λ | ω ∂i (π * (λ)) ≤ 0}, 1 ≤ i ≤ p, and vector λ * = λ * 1 , λ 2 ...λ * p . If (i) the set intersection Π = ∩{Π i , 1 ≤ i ≤ p} is not empty, (ii) there exist p values λ * i ≥ 0, (iii) ω ∂i (π * (λ * i )) = 0, 1 ≤ i ≤ p,
then π * (λ * ) and λ * are respectively solutions of the problems 2.28 and 2.30. When ω ϕ is strictly h-increasing, then the solution π * (λ * ) is unique.

Conversely, if Π is empty, or if there is one i at least such that ω ∂i (π * (λ) is always > 0, then there is no solution. If these two conditions are satisfied, but not the third one, the dual problem is still solved by λ = λ * , but not the primal one.

Costs

In the scalar case, one can interpret the constraint in terms of a cost C, by letting

ω ∂ (π) = ω ∂ (π) -C, hence ω ∂ ≤ 0 ⇔ ω ∂ ≤ C. Applied to cut π, this gives ω ∂ (π) = ω ∂ (π) -C.
This amounts to a change of origin on the axis of the energies. If ω ∂ is inf-modular, of h-increasing, then ω ∂ also is, and Theorem 2.11 still applies. Figure 2.3 depicts a situation where the constraint is compared to a cost C.

In the vector case, similarly, p cost constants C i , 1, 2, ...i, ..., p can complete the inputs set, and be interpreted as the coordinates of a vector C in R p . The vector Lagrangian is now written

ω(π, λ) = ω ϕ (π) + p 1 λ i [ω ∂i (π) -C i ],
(2.39)

The ω ∂i are inf-modular, and the theorem 2.12 is valid for them. In the vector case, the solutions of equation (2.39) are the doublets {π * (λ * ), λ * -C}, and in the scalar case the doublet {π * (λ * ), λ * -C}.

Discussion

We shall describe some of the similarities we have seen with the KKT conditions. The feasibility assumptions in the KKT theorem 2.6 have counterparts in 2.11 and 2.12.

It is not surprising to find them again. Furthermore, the complementary slackness, leading to strong duality, reappears in braid optimization via the condition 3 of Theorem 2.11. This condition 3 appears because we want that the minimal cut π * (λ * ) in the Lagrangian energetic lattice Π(ω(λ)), such that ω(π * (λ * )) ≤ 0, be also minimal cut in the energetic lattice Π(ω ∂ ). The counter example of section 2. [START_REF] Allène | Some links between min-cuts, optimal spanning forests and watersheds[END_REF] shows what happens when this condition is dropped. 83 2.9. LAGRANGE MINIMIZATION BY CUT-CONSTRAINTS (LMCC) Four major differences separate KKT conditions and their counterpart conditions in the optimization problem on braid:

1. The counterpart of the points x of R n is now the cuts of Π(B) and these new "points" are structured by several orderings and lattices.

2. The assumptions of "continuously derivable" functions f 0 , f i have no counterpart in braid optimization. A topology on the set Π(B) would just be cumbersome.

The concept which replaces the first order minima condition in 2.6 is the infimum of the energetic lattice Π(ω(λ)).

3. First order minima conditions are replaced inf-modularity.

4. The KKT conditions (2.6) when interpreted on a BOP, works by ordering the cuts themselves (e.g. the energetic lattices of cuts versus the numerical one of energies), and goes to energies in a second step, via h-increasingness.

Lagrange minimization by Cut-Constraints (LMCC)

In the two theorems 2.11 and 2.12, h-increasingness seems to be an artificial construction residual of the dynamic program. We now will reformulate the minimization problems directly in the refinement of partitions or cuts, getting closer to a lattice based approach, in such a way that h-increasingness will no longer be required. We begin, by looking at cuts Π(ω ∂ ) by a given cut itself π C ∈ Π(E).

ω ϕ (π) subject to π ω ∂ π C (2.40)
The set of feasible solutions for π ϕ is clearly

Π C = {π | π ∈ Π(E, B), π ω ∂ π C } and π ϕ = ϕ {π | π ∈ Π C }.
Minimal cut using the energetic-lattice: Equation 2.40 refers here to a minimization using two energetic lattice structures, and not just the numerical order of energies. This is a critical change in the method and approach to the minimization problem, as the energetic lattice helps order the solution space and thus also the final optimum reached by enforcing the singularity condition. This consists in finding the infimum in Π(ω ϕ ) constrained by ω ∂ -energetically ordered cuts. For clarity, one must note here that this requires interaction between two energetic-lattices.

As before, we introduce the Lagrangian ω(λ) = ω ϕ +λω ∂ , λ ∈ R, which induces the energetic lattice Π(ω(λ)) of minimal cut π * (λ). To obtain the optimal Lagrangian parameter λ * , we proceed as in classical Lagrangian dual, and obtain The comparison between the two LME and LMCC approaches is instructive. It shows how h-increasingness is not really essential, and that Lagrangians still work for lattices of partitions Π C , and not only for the numerical lattice of the energies Π ω 's. But the most interesting feature is that theorem 2.13 applies to family of partitions of possibly infinite space E, as long as the number of classes is locally finite. This situation occurs quite often in the "remote sensing", where the area under imaging is incomparably smaller than the actual span of study. The LMCC provides such an independence (while not the LME) on account of the association an optimum with each leaf.

λ * = inf{λ | π * (λ) ω ∂ π C } Problem

Class constrained minimization (CCM)

We now study a stricter constraint model of Class-local constraint. This restricts the constraint function to be defined now the classes and no more on the partitions. Further we see how this becomes purely a energetic lattice based solution to the solve the constrained optimization problem.

Single constraint

This section treats firstly the case of hierarchies and then that of braids, and develops an alternative method for constrained optimization, which does not resort to Lagrangians.

The hierarchy under study here is considered to be finite, and energies ω ∂ , ω ϕ : S → R + are defined on classes. Consider the following optimization problem:

CCM problem minimize π∈Π(ωϕ) ω ϕ (π) subject to ω ∂ (S) ≤ C S , ∀S ∈ π (2.42)
The method consists in generating a new hierarchy H where the minimization of ω ϕ is no longer conditioned. Let Π(C S ) stand for the family of the cuts π of a hierarchy H,

where constraint function values for each class S is bounded to C S .

π ∈ Π(C S ) ⇔ {S π ⇒ ω ∂ (S) ≤ C S }. (2.43)
Obviously, the problem is feasible if and only if Π(C S ) is not empty.

Proposition 2.14. If the family Π(C S ) is not empty, it is closed for the refinement infimum and supremum.

Proof. As the number of levels of H is finite, it is sufficient to prove that {π 1 ;

π 2 } ∈ Π(C S ) implies that {π 1 ∧ π 2 ; π 1 ∨ π 2 } ∈ Π(C S ). Consider π = π 1 ∧ π 2 . At leaf x ∈ E let
the class of π 1 be S 1 and let that of π 2 be S 2 . If S 2 ⊆ S 1 , the p.p. a 2 of π 2 of support S 1 is the minimum of π 1 ∧ π 2 in S 1 , and vice versa. In both cases the two infima have energies bounded by C S . This property remains true as point x spans E, the infimum

π 1 ∧ π 2 belongs to Π(S C ). One continues similarly for π 1 ∨ π 2 .
Since the family Π(C S ) is closed under the refinement infimum, it admits a smallest element π 0

π 0 = ∧{π, π ∈ Π(C S )} (2.44)
The classes of π 0 can be interpreted as the set of leaves of a new hierarchy H , identical to H above and on π 0 , but where all classes below π 0 are removed (see Figure 2.5).

The cuts π of new hierarchy H are exactly those of H that satisfy the class constraint

ω ∂ (π) ≤ C S .
The problem now reduces to find the non-conditional minimal cut of H w.r.t. ω ϕ , a question that we already know how to treat. If the minimization is considered in the ω ϕ -energetic lattice Π(ω ϕ , H ) relative to H , we just have to suppose ω ϕ is a singular energy. If we want that the minimal cut π * ϕ induces a minimal energy, then, according Rel.(1.26), we must take ω ϕ h-increasing (in addition π * ϕ is then found in one bottom-up pass). Now we can state: Proposition 2.15. When ω ϕ is a singular and h-increasing energy, then the minimal cut π * ϕ in the ω ϕ -energetic lattice Π(ω ϕ , H ) is also a cut of smallest ω ϕ energy in Π(ω ϕ , H) whose all classes S * satisfy the cost constraint ω ∂ (S * ) ≤ C.

The result is important. It grants the existence and the uniqueness of the minimal cut π * ϕ under very large conditions: no prerequisite is needed for ω ∂ , and uniquely singularity and h-increasingness for ω ϕ . Note that the cost C has not to stay constant. Equivalence (2.43) holds on each class separately. C may vary through the space, or according to the level i in the hierarchy.

When the energy ω ϕ is also increasing w.r.t. the refinement of the cuts (e.g. the quadratic deviation term Mumford-Shah energy), i.e. when:

π 1 ≤ π 2 ⇒ ω ϕ (π 1 ) ≤ ω ϕ (π 2 ),
(2. [START_REF] Grady | Weights and topology: A study of the effects of graph construction on 3d image segmentation[END_REF] then the minimal cut π * ϕ coincides with π 0 , since 87 Fast implementation is then obtained by the following greedy top-down algorithm :

π 0 ≤ π ⇒ ω ϕ (π 0 ) = ∧{ω ϕ (π), π ∈ Π(C S )} = ω ϕ (π * ϕ ). ( 2
-index the classes of H by a lexicographic ordering from the root E to the leaves;

-starting from E, go down;

-when class S has all its sons T such that ω ∂ (T ) ≤ C S , then replace S by its sons;

-otherwise don't perform the replacement (this because, according to Rel.

Class constraint versus Lagrange minimization by energies

What is the relation between the above results and those we can get by using ω ∂ as the constraint term in a Lagrange family? Suppose ω(λ), ω ϕ , and ω ∂ singular, and extend ω ∂ to partitions by ∨-composition:

ω ∂ (π) = ∨{ω ∂ (T ), T π}.
(2.48)

The energy ω ∂ inf-modular, since equality (2.48) remains valid when π is reduced to the single class {S}. Therefore, according to Definition 2.7, the family {ω(λ) = ω ϕ + λω ∂ , λ ∈ R} is Lagrange. As above, π * (λ) stands for the minimal cut of the Lagrangian energetic lattice Π(ω(λ), H}. We saw (Proposition 2.8) that the function λ → π * (λ) is increasing for the refinement, and π * (λ) → ω ∂ (π * (λ)) decreasing, so that there is a maximal ω ∂ (π * (λ 0 )) ≤ C. Suppose ω ϕ strictly increasing for the refinement ordering;

it is thus h-increasing. If π 0 < π * (λ 0 ) then ω ϕ (π 0 ) < ω ϕ (π * (λ 0 )) which contradicts the meaning of a minimum of ω ϕ (π * (λ 0 )) (Rel.(1.26)). On the other hand, we also have Eq.(2.46), which leads to π 0 = π * (λ 0 ), and we can state:

Proposition 2.17. Let {ω(λ) = ω ϕ + λω ∂ , λ ∈ R} be a Lagrange family where ω ϕ is strictly increasing and ω ∂ is obtained by ∨-composition (Eq.(2.48)). Then the minimal cut π * (λ) equals the minimum π 0 given by Eq.(??).

uses a numerical constraint on composition of energies of partial partitions constituting any cut, secondly LLCM, where the constraint is cut belonging to solution space and thus using lattice structure. Finally and thirdly, CCM, which provides a numerical constraint for each class.

CCM vs LME: CCM solves the problem of finding the cut of a hierarchy with least energy ω ϕ under the constraint ω ∂ ; while LME provides an upper bound π * (λ 0 ); One should also note that CCM is valid for hierarchies, LME for the larger class of braids.

In CCM, the mapping π → ω ϕ (π) holds globally on the cuts π, it is just supposed to be increasing, which is not very demanding; and ω ∂ is inf-modular; in LME ω(λ) is singular, ω ϕ is h-increasing, and ω ∂ is inf-modular and h-increasing.

Another difference is that the CCM model is local, in the sense that the constraint ω ∂ is allocated to every class of the cut under study, and it becomes also global when one take the supremum for law of composition ( i.e. when the energy ω ∂ of a p.p. equals the supremum of the energies of its classes); for the implementation it is assumed that ω ∂ is inf-modular. CCM can use non-local costs C which may vary over the space E, or remain constant.

Finally one can note that in CCM, the extension to the multi-constrained cases is straightforward, and concerns both logical and and or of constraints. This latter mode is out of the scope of LME.

Primal Vs Dual: C-cuts Vs λ-cuts

In this section we shortly discuss the scale-sets descriptor of Guigues [47], and multipler approximation by Salembier-Garrido [START_REF] Garrido | Hierarchical Region Based Processing of Images and Video Sequences: Application to Filtering, Segmentation and Information Retrieval[END_REF], and provide interpret a primal version of the climbing algorithm, and why it is cumbersome. Both Guigues and Salembier-Garrido search values of the Lagrangian multiplier λ, so as to obtain a cut π which reaches or is closest to an input or given constraint function value ω ∂ (π) ≤ C. We know that the solution to the dual, lower bounds the solution to the primal problem. Furthermore we have also seen that the non-existence of multipliers refers also to the lack of existence a strong duality. Thus spanning the λ-cuts at best provides us, in this case, with an upper-bounding cut. This is one of the important results in this chapter.

Dual Domain Composition: Furthermore a composition by addition of child energies ω ϕ (T i ), ω ∂ (T i ) associates with each braid and thus hierarchy, a Λ, i.e. scale function.

In this dual domain, one can see that the scale-climbing corresponds to a "composition by supremum" of the Lagrangian parameters. This basically correponds to the infimum over the multiplier values in guigues are our result. The λ-cuts also have the same problem, except that the implicit condition of having the largest partition by Guigues, Breiman and various other authors, provides a unique solution. This has been explicited in two forms, first as scale-increasingness condition on the energies and, second as singularity condition, which ensures a unique minimum.

Summary

Chapter contribution summary

We have shown that the λ-cuts correspond to an upper-bound and is not a global solution to a constrained optimization problem on HOP and BOP.

The solutions in fact are shown to be a Lagrangian relaxation of the original constrained optimization problem, who solutions are global only at the cost endowed by the multiplier value.

We demonstrate three locally constrained optimization models, that use the partition itself as constraints, enforcing further the energetic lattice structure.

We also demonstrate basic perturbation and penalty methods to obtain a better bounding λ-cut. An interesting prospect would be to use a energetic lattice based perturbation for quicker convergence rates. A further analysis of the problem would be to consider the global convergence analysis of the constrained optimization problem on HOPs and BOPs. In such a context we propose three different models.

In using the energetic lattice for constrained optimization, we have three different classes of constrained optimization problems that correspond to the three different ways of enforcing a constraint:

-Lagrangian Minimization by Energy(LME): (By numerical constraint on engery) Energetic lattice based generalization of the Lagrangian, when one works in the space of partitions from a Braid, instead of R n .

-Lagrange minimization by Cut-Constraints (LMCC): (By a partition constraint) We introduce a partition based constraint optimization model, which does not involve any numerical constraint function, but one that is driven by the energetic lattice.

-Class constrained minimization (CCM): (By numerical constraint on engery on a class) The third model demonstrates how the energetic lattice handles local constraints. There is no Lagrangian formulation here.

Chapter 3

Applications of Energetic Lattice

Publications Associated with Chapter given a hierarchy of segmentations and a ground truth partition corresponding to an image, that is closest to the ground truth or marker partition.

A few useful h-increasing energies

Following the description of h-incresingness and the different compositions of energies possible we briefly describe a few common segmentation models that can be formulated as h-increasing energies.

Mumford and Shah energy

One of popular, notably non-convex image segmentation model is the Mumford-Shah functional [85]. One can find an exhaustive study of this functional in Morel et al.'s book [START_REF] Morel | Variational methods in image segmentation[END_REF], where it is formulated in the Euclidean plane using edges which are composed of rectifiable simple arcs.

95

ω(π(S), λ) = ω ϕ (π(S)) + λω ∂ (π(S)) = 1≤u≤q x∈Tu f (x) -µ(T u ) 2 +λ 1≤u≤q (∂T u ) (3.1)
This can be generally written as an affine energy of the form

ω(S, λ) = ω ϕ (S) + λω ∂ (S) S ∈ S. (3.2)
The first term, ω ϕ , is the additive fidelity term which sums up the quadratic deviations from the mean value µ(T u ) over the class T u over different u producing a partial partitioning of class S, and the second term ω ∂ = ∂T u , the lengths ∂T u of the frontiers of all

T u .
Both increasingness relations, h-increasingness and scale-increasingness (1.27) and (1.42), are satisfied by the family of energies ω + ϕ, ω ∂ in eqn (3.1). For short these energies are called climbing.

When the energy ω ∂ is sub-additive, i.e.

ω ∂ ( 1≤u≤q T u ) ≤ 1≤u≤q ω ∂ (T u ), (3.3)
then the family is obviously scale increasing, since

ω ∂ (S) = ω ∂ ( 1≤u≤q T u ) ≤ 1≤u≤q ω ∂ (T u ) = ω ∂ (π(S)).
Conversely, L. Guigues has shown that the condition (3.3) is necessary for scale increasingness [47].

Additive energy by convexity

The arc length function is not the only choice. One can also think about another ω ∂ (S), which reflects the convexity of the class S. Consider in R 2 a connected set S without holes and with a non zero curvature everywhere on ∂S. Let dα be the elementary rotation of its outward normal along the element du of the frontier ∂S. As the curvature c(u) equals dα du , and as the total rotation of the normal around ∂S equals 2π, we have

2π = c≥0 c(u)du - c<0 |c(u)| du.
When dealing with partitions, the distinction between outward and inward vanishes, but the parameter

ω ∂ (X) = 1 2π ∂S |c(u)| du (3.4)
still makes sense. It reaches the minimum value 1 when set S is convex, and increases with the degree of concavity. For a starfish with five pseudo-podes, it values around 5.

Now ω ∂ (S) is sub-additive for the open parts of contours, therefore it can participate as a regularity term in an additive energy. In digital implementation, the angles between contour arcs must be treated separately (since sub-additivity applies on the open parts).

Additive energies by active contours

The active contours aim to match regular closed curves with the zones of maximum The simplest comparative energy is given by the difference of a given energetic function f on the two crowns:

ω ϕ (S) =| (S⊕B\S) f (x)dx - (S\S B) f (x)dx |, S ∈ P(E). (3.5)
It can be expressed in a dimensionless form by putting:

ω ϕ (S) =| (S⊕B\S) f (x)dx -(S\S B) f (x)dx a(S) |, S ∈ P(E),
where a(S) denotes the area of S. When the absolute value bars are removed, the both energies ω ϕ and ω ϕ become sub-additive. Alternatively, the energy ω ϕ proposed in [124] is the sum of the variances of f in the two crowns, divided by the variance of f in the union of these two crowns.

For the regularity term ω ∂ of the energy (3.2), one classically takes the above function ν of Rel. (3.4), which is scale increasing and generates the climbing family {ω ϕ + λω ∂ }.

Mumford-Shah Energy for Color image segmentation

We aim to find an optimal cut which provides the simplified version of a colour image f , constrained by compression rate. A hierarchy H has been obtained on input image figure 3.1, by segmentations of the scalar luminance l = (r + g + b)/3 based on flooded watersheds [START_REF] Cousty | Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts[END_REF]. In each class S ∈ H, the simplification consists in replacing the function f by its mean value of colors, i.e. the means of the three channels over S. Note that this colour mean does not intervene directly in the three energies (3.6) to (3.8), but rather in the display of the optimal cut. We use the energy ω(S), as defined in equation (3.2) to demonstrate different optimal cuts.

For the first experiment, as the data fidelity term ω ϕ (S) we take the variance of the luminance for each class S of hierarchy H (first term of equation (3.6)). The regularization term ω ∂ (π), is equal to the contour length | ∂S |, plus 24 bits for the average color of S. This gives the energy ω lum (S), whose result is shown in figure 3.1 (left).

ω lum (S) = x∈S l(x) -l(S) 2 +λ(24+ | ∂S |), (3.6)
In a second experiment depicted in figure 3.1(right), we separate each colour vector

ω chrom (S) = x∈S,1≤i≤3 c i (x) -c i (S) 2 +λ(24+ | ∂S |), (3.7)
The principle idea demonstrated by this experiment is the independence between the function generating the hierarchy and the energy creating the energetic ordering or lattice, and the imminent minimization by a dynamic program. We observe in figure 3.1 (right) that the plant in front of the female duck are now correctly segmented, and that the water in the background has lesser detail.

ω Texture (S) = ω chrom (S) + S ∈siblings(S) µ σ 2 (Area(S )) , (3.8)
This leads to a third experiment, depicted in figure 3.2, based on energy in equation (3.8).

This experiment is to extract textures parametrized by regularity. The energy (3.8) in is similar to (3.7), except for the term in which is inversely proportional to the quadratic deviation of sizes (area) of the children from the mean size of the children, which is a trivial h-increasing energy. Furthermore the fidely term is a quadratic deviation of the chrominance vector being minimized on the partitions of hierarchy produced from the luminance vector l. This experiement basically shows that the optimal partitions that are extracted by using minimal norm of a vector does not remain the same when one obtains when using their components.

Intuitively, texture features are formulated into this multi-scale framework where obtains a two level filtering scale parameter, which combines the effect of minimal deviation of chrominance and structure of texture into one global energy function. λ controls the perimeter of the segmentation, while µ constrols the regularity of classes in the segmentation. The monotonicity of the segmentations in the optimal hierarchy becomes a bit more complicated and is subject to more detailed study, since the pair π(λ i , µ k ), π(λ j , µ l ), for i < j, k < l are cuts that are not predestined to be ordered, and would require a more 99 general law of scale-increasingness. This demonstration basically shows the flexibility of the multiscale-energy formulation.

We conclude this section by quoting Salembier and L. Garrido [START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF], and L. Guigues [49],

who demonstrate constrained optimization on HOP, by replacing the gradient descent by climbings dynamic program for family of h-increasing energies.

Hierarchical structure based energies

In this experiment we demonstrate how one can use the hierarchical structure itself as a constraint. This is possible by using the integral of the saliency function on support of the contours of the partial partitions (and not purely the length of partial partition).

That is a numerical measure g(x) = 1 -s(x) is introduced on the perimeter ∂S

ω g (S) = x∈S,1≤i≤3 c i (x) -c i (S) 2 +λ ∂(S) g(x)dx, (3.9)
This constraint function requires that the saliency value on the partial partitions be the highest, while having a minimal ω ϕ . A high saliency function value implies higher level in the hierarchy. This enables the use of saliency function itself as a multi-scale constraint function. In our example we use saliency function's value which corresponds to the connection value of watershed floodings by volume [START_REF] Cousty | Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts[END_REF]. This is demonstrated with a set of optimal cuts at different values of λ.

One can also use a value which is not dependent on the gradient function but a proximity function to a ground truth. This will be demonstrated in the following section.

Ground truth Proximal Energies

In this section we concentrate on a particular application which is supervised image segmentation evaluation. More specifically we will look at ground truth set based segmentation evaluation. For this purpose we work on the Berkeley Dataset [10]. This chapter focusses on the case when the segmentations are part of a hierarchy and different ground truth sets provide different qualitative information on the segmentation.

The motivation in ground truth based evaluation of hierarchicy of segmentations lies in the key fact that the ground truth partition can be found at a finite scale or cut in the hierarchy of segmentations, above and below which the cuts become non-optimal, resulting in sub-optimal F-measures. This is refered to in [START_REF] Pont-Tuset | Supervised assessment of segmentation hierarchies[END_REF] as the upper-bound partition selection, and poses it as a combinatorial optimization problem. Here we will motivate the requirement of an energetic lattice and the h-increasing energy that can be formulated using Haussdorf distance to solve the same problem. This provides a way to climb up to the partition closest to the ground truth partition.

Ground truth evaluation

Here we introduce the problem of ground truth evaluation of hierarchies of segmentations a bit more strictly. First let us determine our inputs and outputs we desire from the problem. Given a hierarchy of segmentations H and a ground truth partition G we would like to determine:

1. The cut π * in hierarchy H that is closest to ground truth partition G. Here we will use the localised Haussdorf distance to define a maximal distance between two a cut in the hierarchy and a ground truth set. This gives two energies associated to the two sense's of proximities:

• Hierarchy to Ground truth partition: π → G, π ∈ H.

• Ground truth partition to Hierarchy: G → π, π ∈ H.

2. Compare any hierarchy H with multiple ground truth partitions of the same image.

3. Compare any two hierarchies with respect to a common ground truth partition.

We demonstrate in figure 3.3 how the above problem is a scale selection problem by a toy example.

Segmentation Versus GT Partitions: Refinements and Overlaps

In this section we consider the different topological possibilites between the classes of the ground truth partition and segmentation being evaluated. This has been first addressed The problem now consists in extracting a proximal partition in the hierarchy that is closest to one of these ground truth partitions. Further how do we compose when we have multiple ground truth partitions, and how do we compare hierarchies w.r.t a single ground truth.

1. Oversegmentation: When the union of a finite set of classes in the segmentation is a class of the ground truth partition. In other words, the segmentation is locally a refinement of the ground truth partition.

2. Undersegmentation: When the union of a finite set of classes in the ground truth partition is a class of the segmentation. In other words, the ground truth partition is locally a refinement of the segmentation.

3. Overlaps: When the classes of the segmentation and ground truth overlap but do not produce refinements. This termed as noise in case of [START_REF] Pont-Tuset | Image Segmentation Evaluation and Its Application to Object Detection[END_REF]. To be more precise, the cases 1,2,4 are all in the general structure of a Braid of partition. Now a single or a set of ground truths can demonstrate either one or all of these characteristics. Here we demonstrate how the refinements are handled using the inf-composition.

For a similar study on the saliency function, please see 4.7.5.

Segmentation Evaluation Measures

The comparison of two segmentations of the same image is not clear, as image segmentation is inherently ill-defined: there is no single ground truth (partition, since when we compare segmentations it necessitates that the ground truth is a partition too!) that can capture the faulty and correct labellings in the two segmentatations [115]. Once this is lack of definition is noted, one can find heuristic assumptions on the correspondence between ground truth set and segmentation, providing us with a variety of evaluation measures in literature. We cite few which are important for our discussion later. The measures evaluating hierarchy of partitions has been studied extensively in the thesis by Tuset et al. [START_REF] Pont-Tuset | Image Segmentation Evaluation and Its Application to Object Detection[END_REF].

1. Many region based measures have the Jaccard distance between sets as a begining point. The Jaccard index between finite sets S, S is given by:

J(S, S ) = S ∩ S S ∪ S (3.10)
The Jaccard distance is given by: allows for refinements to occur in either direction at different locations in the segmentation.

d J (S, S ) = 1 -J(S, S ) = 1 - S ∩ S S ∪ S ( 3

Segmentation Covering [10]:

The overlap between two regions (classes of segmentation S or S ) R and R , is defined as:

O(R, R ) = R ∩ R R ∪ R
and the segmentation covering is defined as: This is an example of contour/boundary based measure.

C(S → S ) = 1 N R∈S |R| • max R ∈S O(R, R ) (3
It would be interesting now to observe that the contour based measures are sensitive to the placement of class contours w.r.t each other unlike the region based measures.

It is important to note that a single human annotated ground truth rarely provides an objective segmentation containing all objects of interest. Figure 3.5 demonstrates the problem across different Ground truth partitions of the same image.

Haussdorf Distance

The Haussdorf distance is a tool used in the image processing community to compare images (cites needed). It is well known that the Hausdorff distance is a metric over the set of all closed, bounded sets, while satisfying properties of identity, symmetry and triangle inequality.

Let E, d be a metric space of distace d and let A, B ⊂ E be non-empty subsets of space E. The Haussdorf distance can be defined now as: The Haussdorf distance in equation (3.13) can be calculated by the supremum of minimal radii of dilation by a ball of one set to cover the other set [START_REF] Serra | Hausdorff distance and interpolations[END_REF]. If δ r (X) represents a dilation of set X by a compact ball of radius r, then:

d H (A, B) = inf{r : B ⊆ δ r (A); A ⊆ δ r (B)} (3.14)
equivalently for an erosion operation r (X), the partitions. This is a well known problem with the Haussdorf distance. This is demonstrated in figure 3.6.

We have to address thus two questions:

• How to define the Haussdorf distance for partial partitions or classes of partition?

• How to handle asymmetric shapes by formulating a composition of energies over the partial partitions ?

Felzenszwalb et al. propose the use of a dynamic program approach using curves of shapes, ordered in a tree [START_REF] Felzenszwalb | Hierarchical matching of deformable shapes[END_REF] for deformable shape matching.

Hausdorff distance

Most of the supervised evaluations of hierarchies, including Arbelaez et al., Tuest et al. [START_REF] Arbelaez | Une approche metrique pour la segmentation d'images[END_REF]10,[START_REF] Pont-Tuset | Supervised assessment of segmentation hierarchies[END_REF], derive from the intuition of the Hausdorff distance, in various critical manners. Let us briefly recall this background.

In a metric space E of distance d we aim to match the support S(π) of a bounded partial partition π with a set G of points and lines, considered as a GT drawing. The smallest isotropic dilation of G that covers the contour S(π) has a radius

ρ G = inf{ρ | G ⊕ ρB ⊇ S(π)}, (3.16)
where ρB is the disc of radius ρ centred at the origin. One can interpret ρ G as the "energy" required for reaching ∂S from the GT G. In the same way, the counterpart covering is given by the radius ρ A :

ρ A = inf{ρ | S(π) ⊕ ρB ⊇ G}. (3.17)
By introducing the so called distance function d(x, Z) from point x to the fixed set Z, i.e.

d(x, Z) = inf{d(x, z), z ∈ Z} x ∈ E (3.18)
we see that class T of π, and no longer with the whole S(π) itself, then the regions with a good fit would be treated separately from the others. And in addition, if this equivalent was hincreasing, then it would provide an energy for calculating easily the associated optimal cut [59], i.e. the smallest upper bound of all cuts of the hierarchy, in the wording of [START_REF] Pont-Tuset | Supervised assessment of segmentation hierarchies[END_REF]).

ρ G = sup{d(x, G), x ∈ S(π)} and ρ A = sup{d(x, S(π)), x ∈ G}, ( 3

Half Haussdorf distances

Composition of Ground truths for Segmentation Evaluation: Peng et al. [START_REF] Peng | Evaluation of image segmentation quality by adaptive ground truth composition[END_REF] evaluate segmentations with multiple ground truths as human-labeled ground truths are only a small fraction of all the possible interpretations of an image. Furthermore they describe that the labeled ground truths set by itself might not be optimal to compare with the input segmentation, and conclude that such evaluation leads to a certain biassed evaluation. Their key observation lies in the local structural similarities between the groundtruhths and the segmentation. They create a composite ground truth, which remains similar across all ground truths in the set, created by a labeling minimizing a potts prior. We will use this structural similarity feature in defining our composition of energies across multiple ground truths.

107

Region based and Contour Based Measures

The ground truth set can be interpretted as a boundary or a set of segments based on which there can be many measures defined. Movhedi et al. [START_REF] Movahedi | Design and perceptual validation of performance measures for salient object segmentation[END_REF] results show that a Contour Mapping measure based upon contour bimorphisms between the boundaries of the object segmentations under comparison were most consistent with psychovisual studies involving human evulation. Further more this also suggests that the ground truth set need not always contain partitions locally.

Based on the above motivations we have constructed the half(complementary) haussdorf local energies ω G , θ G as shown in figure 3.7. These energies are called precision and recall energies, by corruption of classical usage of terms precision and recall refering to the type I and II of errors.

Precision energy

We now focus on the classes {T i } whose concatenation T i T 2 ... T k generates the partition π. The {T i } are said to be the children of parent S. Consider the class T i of the partition π. The smallest dilate G ⊕ ρB that covers T i has a radius: Therefore, in all cases, we arrive to an optimal cut which can only be at the lowest level of hierarchy H, i.e. the leaves, or at the highest one, i.e. the space E itself.

ω G (T i ) = inf{ρ | G ⊕ ρB ⊇ T i }. ( 3
For being more informative, we can introduce a trade off based on mutual comparisons of the energies of the sons. An easy way consists in adding a quantizer λ in the composition by infimum, so that

ω G (π) = ω G (T i T 2 ... T k ) = inf{ω G (T i )} + λ. (3.22)
As this new energy is h-increasing, the optimal cut is reached in one pass by comparing the respective energies of sons and fathers [59]. As ω G (S) = sup{ω G (T i )}, we have

ω G (π) < ω G (S) iff λ < sup{ω G (T i )} -inf{ω G (T i )}.
The parent replaces its children when the latter are sufficiently "identical" , parameterized by complexity parameter λ. For each value of λ we thus obtain the cut which minimizes the distances to the ground truth G, i.e. the smallest upper bound of all cuts, as posed by Tuset et al. [START_REF] Pont-Tuset | Supervised assessment of segmentation hierarchies[END_REF]. To give an idea of the distribution of the energies ω G (S), Θ G (S) shown in Fig 3.8 For two different ground truths, over different partitions from a hierarchy. As seens there are cases where the parent is as proximal as the child.

Recall energy

The number ω G (S) informs us about those points of ∂S close enough to G, but not on those of G close to ∂S. We cannot take, here, the dual form of the ω G (S) of Rel. a quantity which risks to be extremely large, for the drawing G may spread over the whole space, whereas class S is locally implanted. Fortunately, when dealing with hincreasing energies, one is less interested in the actual values of the energies than by their increments between fathers and sons. Now, when a point of G is outside class S, then its distance to S is the same as the max of the distances to the sons T i of S: Proof. Let π(S 1 ) and π (S 1 ) be two partial partitions of set S 1 , with

x ∈ G ∩ S c ⇒ d(x, S) = d(x, ∂S) = d(x, T i ) = d(x, ∂T i ), ( 3
θ G (π(S 1 )) = {θ G (T i ), T i S} ≤ θ G (π (S 1 )) = {θ G (T i ), T i S 1 } (3.28)
Consider a partial partition π(S 2 ), where S 2 ⊆ S c 1 . By taking the supremum of each member of inequality (3.28) with {θ G (X j ), X j S 2 } one does not change the sense of the inequality, which becomes

θ G (π(S 1 ) π(S 2 )) ≤ θ G (π (S 1 ) π(S 2 )), ( 3

Composition of ω G (S) and θ G (S).

The composition of the energies happens with respect to a single ground truth, or to several ones. In the first case, one can wonder if preferable not to combine ω G and θ G so that they can provide two separated maps for the precision and for the recall. The two associated overall values may be presented in a 2-D graphic as proposed in [START_REF] Arbelaez | Constrained image segmentation from hierarchical boundaries[END_REF]. We can also take for final energy either max(ω G , θ G ), or sum ω G +θ G , they are both h-increasing.

On the example of the "peppers", and for two different ground truths, one obtains the results depicted in Fig. 3.9

Composing multiple ground truth sets

In case of multiple ground truths, the usual techniques proposed in literature are additive [START_REF] Arbelaez | Constrained image segmentation from hierarchical boundaries[END_REF]. Formally speaking, why not? Putting ω G = ω G i yields an h-increasing energy, hence a best cut (which is, of course different from the sum of the best cuts of the various

G i ).
The implicit assumption here is that all ground truths are more or less similar.

But one can also encounter drawings G i that focus on different regions of the scene. Then if we take the sum, each part of the space risks to be penalized because if is far from one drawing, at least. This was the case in case of adaptive ground truth compoisition in Peng et al. [START_REF] Peng | Evaluation of image segmentation quality by adaptive ground truth composition[END_REF].

For the situation depicted in Fig. 3.10, the energies first two best cuts are given by sup{ω G , θ G } and the third one by taking inf{sup{{ω

G 1 , θ G 1 }, sup{{ω G 2 , θ G 2 }}. When point x ∈ E is farther from G 1 than from G 2 then the G 1 energy is not taken into account.

Number of Classes Constraint

The two energies ω G (S) and θ G (S) of Rel. (3.25) and (3.22) have been chosen because of their geometrical meanings, but they are far for being the only possible ones. It is iindeed easy to build an energy which fits the features one wants to emphazise. Suppose for example that we decide that the number of classes n of the ground truth is a cruxial feature. Then when applying energy ω G we can condition the ascending pass which generates the best cut to stop as soon as the number n of classes is reached. Fig. 3.11 depicts the best cuts w.r.t. ω G . when the parameter λ of Eq.(3.22) equals 0, 10, and 80, and when the ground truth is GT7, which has 87 classes. For λ = 0, we do not obtain 113 the leaves partition, because the classes with an equal energy have been clustered, as pointed out previously. In Fig. 3.11c) and d), but not in Fig. 3.11b), one arrives to 87 classes before the end of the climbing algorithm. This explains why the two partitions are not comparable.

h-increasing Coverage Measures

The measures for evaluating segmentations with ground truths categorize into two types:

Region based and Boundary based. The global purpose of these measures, is to be able to evaluate the image segmentation algorithms w.r.t certain metrics. Here we use one of the region based measures to extract an best possible segmentation from a hierarchy of segmentations. One of the common measures is the coverage measure [10]. The coverage criterion is not h-increasing due the division by the union of regions. This changes the optimality and we can't ensure a local optimum that is part of the global optima. One can now rewrite it (eqn. (3.12)) in an h-increasing energy form as follows:

ω(S) = N - j∈[1,n] |S ∩ G j | + λn (3.30)
where N is number of pixels in the image or partition, G j are the different connected components of the ground truth partition G, and n is the total number of connected components in the ground truth partial partition of support S.

ω(S) = (N - j∈[1,n] |S ∩ G j |) + λn
The sum term in the functional refers to the number of pixels that correspond between the ground truth and the class. This is should be maximum, while the difference N -

j |S ∩ G j | should be minimum.
The second term corresponds to the number of connected components of ground truth in each class S. Minimizing this value gives the largest partition that fits G.

This equation can be replaced by

ω(S) = N - j∈[1,n] |S ∩ G j | + λ j∈[1,n] |∂G(x)| x∈G j ∩S

Local linear dissimilarity

Another variant consists in replacing the supremum that appears in Rel. (3.19) by a L p sum, which gives less importance to the farthest zones. A similar approach has been successfully used by L. Gorelick et Al. [START_REF] Gorelick | Segmentation with non-linear regional constraints via line-search cuts[END_REF] in regional line-search cuts. Among the L p integrals, the one which weakens the most the weights of the farthest zones is obtained for p = 1. Therefore we take for precision energy ω G (S) the integral of distance function g(x) of G along the contour ∂S and for recall energy θ G (S) the integral of the distance

function g(x, ∂S) of S on G ∩ S: ω G (S) = 1 ∂S ∂S g(x)dx θ G (S) = 1 G ∩ S G∩S g(x, ∂S)dx (3.31)
The two functionals ω G and θ G are extended from classes to partial partitions by addition, since they both involve integrals, and one easily checks that the two energies are h-increasing. The higher ω G (S), (resp. θ G (S)), the farther S is from G (resp. G is from S). In case of a ground truth given by k drawings, one just sums up the k energies ω G and θ G .

Global Precision-Recall similarity integrals

P = 1 i=0 i N x∈ (S i ) (1 -g(x)).S i (x)dx |S i | R = 1 i=0 i N x∈G (1 -g S i (x))dx |G| (3.32)
The integral calculates the similarity between partition S i produced by thresholding the saliency s at i and the ground truth partition G by integrating the inverse distance function 1 -g under the binary function S i . Also the sense of the hierarchy is such that s i+1 ⊂ s i which represents that partition at a higher level in the hierarchy has fewer contours than the one below to respect the inclusion order. Each integral is weighted by the relative rank of the partition within the hierarchy H. This is done by weight it by ratio of threshold index i and the total number of levels in the hierarchy N as shown in equation (3.32).

Similarly a global precision value for the contours of the partitions in the hierarchy can be calculated by integrating the distance functions g S i of partition S i under the ground truth partition G. These integrals are normalized with respect to each image support by dividing by the size of the image.

Proximity between hierarchies

The integrals in equation (3.32) is between a partition G (ground truth) and a hierarchy H. The same can be extended to measure the proximities between two hierarchies of partitions. Given two hierarchies of partitions, H 1 , H 2 , with N and M number of levels, and partitions indexed by i and j respectively,

φ 12 = j∈[1,M ] i∈[1,N ] x∈ (π i ) (1 -g π j (x)).π i (x)dx |π i | (3.33a) φ 21 = i∈[1,N ] j∈[1,M ] x∈ (π j ) (1 -g π i (x)).π j (x)dx |π j | (3.33b)
where g π i is the distance function of the partition π i .

The measure lacks the similarity measures across partitions which are not horizontal cuts, but generally cuts from the two hierarchies. This becomes again a combinatorial problem. The refinement of cuts π i from an input hierarchy H 1 would have a value of the distance function g π j which decreases on average till the point where the two partitions nearly fit and the integral starts increasing again.

Ground truth energies to Saliency functions

Here we shortly describes the motivation to leave the energetic lattice and moves to a lattice defined purely on the finest partition of the space. We have seen in the formulation of the local Hausdorff energies that they consisted in measuring the proximity between the contours of the class of the hierarchy and the ground truth partition.

This requirement of local energies, when changed to an global assignment of a proximity measure over all the contours of the segmentations in HOP, results in a way to transform a HOP, by reordering its contours based on its proximity to a ground truth. This was the starting point for the chapter 4 on saliency functions. Furthermore one can examine such a proximity measure in the two Hausdorff senses, that is, distance from partition in HOP to ground truth, and vice versa. Interestingly here, we deviated from the classical saliency function, [START_REF] Cousty | Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts[END_REF][START_REF] Najman | Geodesic saliency of watershed contours and hierarchical segmentation[END_REF], which consists in weighting of contours of the minima by their extinction function [START_REF] Vachier | The viscous watershed transform[END_REF], in other words the result of defining a flooding.

We will see later that a proximity measure, along with a leaves partition is sufficient to define a lattice, and thus produce a saliency function. For a deeper understand we redirect the reader to the chapter 4.

Summary

In summary, one must note that we are demonstrating a framework to perform constrained optimization on hierarchies and braids, using various energies. The image segmentation problem by itself requires another step, which is to determine the optimal scale parameter. This can be done in a variety of methods.

Chapter contribution summary

Demonstration of various h-increasing energies to minimize Mumford-Shah functional, texture energies, which enforce different constraints.

Half Hausdorff energies to calculate proximal cut from hierarchy w.r.t Ground truth, and formulation of the proximity in a constrained optimization framework.

A global measure which distinguishes between a set of hierarchies of segmentations given a set of GT partitions.

Chapter 4

Hierarchies and Saliency function

Publications Associated with Chapter

• [58] Fusions of ground truths and of hierarchies of segmentations (PRL 2014).

• [START_REF] Kiran | Scale space operators on hierarchies of segmentations[END_REF] Scale space operators on hierarchies of segmentations (SSVM 2013).

In the previous chapters, the cuts from a BOP B have been our solution space, and various types of increasing energies were optimized to produce an optimal cuts, and further an optimal hierarchy whose cuts are ordered based on increasing energies. We leave the space braids of partition, and concentrate now on a particular subfamily of the hierarchies, which is the represented by saliency functions. Here one can renounce the lattice over the support of classes and purely work on the contours of partitions in the hierarchy. Following the decision to work on contours of the partitions, we will restrict ourselves to hierarchies represented by a saliency functions in R 2 .

A saliency function is a numerical representation of a hierarchy, which was first introduced by [START_REF] Najman | Geodesic saliency of watershed contours and hierarchical segmentation[END_REF] to represent the scale of disappearance of watershed contours. More generally saliency function is a positive function defined on the frontiers of the classes of a partition π 0 .The function value on a contour represents the altitude at up to which it separates two components, beyond this value, the components fuse into a single component. Thus the saliency characterizes a hierarchy which has the partition π 0 for leaves.

In the Euclidean plane a simple method to model π 0 is to consider its frontiers as Jordan curves.

With this numerical representation of the hierarchy, we formalize the work in [START_REF] Kiran | Scale space operators on hierarchies of segmentations[END_REF], by introducing an order on the contours of the partitions thus avoiding the combinatorial problem of choosing an optimal closest cut. We define the partition of input space to

Ground truth Evaluation of Segmentation Hierarchies

In section 3.2.1 we have seen hierarchies of segmentations have been evaluated by expert annotated ground truths by various local, regional and global measures, over classes regions and partitions [10,[START_REF] Pont-Tuset | Supervised assessment of segmentation hierarchies[END_REF]. This was formulated as an energy using the Hausdorff distance between contours of ground truth and classes in the hierarchy [61], to extracts partitions from a given hierarchy, closest to the ground truth. Until here we have used the energetic lattice from the lattice based optimization framework [59] in chapter 1.

Contour proximity

There has been substantial work on comparing the contours of a ground truth set and image segmentation, for the purpose of evaluating the distance between them. One can find the use of mainly three types measures between the classes of ground truth and segmentation: Region measures (Region Intersection Measure/Coverage measure) [START_REF] Movahedi | Design and perceptual validation of performance measures for salient object segmentation[END_REF],

Boundary-based Measures [START_REF] Huttenlocher | Comparing images using the hausdorff distance[END_REF] which use the Hausdorff distance between the contours. There are also certain mixed regional measures, which weight the false positives and negatives differently. The Hausdorff distance calculates the supremum of distance distribution between two classes, thus for contours of any pair of classes, having the same worst case distance, will be evaluated irrespective of other distances.

In such a context we will consider only contour proximity information. This can be sufficient to extract a set of partitions from a given hierarchy of segmentations. No explicit measure or energy on the classes is required to do so. In this line of work [START_REF] Kiran | Scale space operators on hierarchies of segmentations[END_REF] studies how the distance function of the ground truth can be used to order contours of the hierarchy of partitions, so as to be able to pick the partition closest to the ground truth, and the next closest until one completely transforms the hierarchy.

Jordan Curves

We can adopt either the framework of the Euclidean plane R 2 , or that of an abstract graph, or that again of a planar graph. We will work on the Euclidean plane in this paper, and further discuss its significance after establishing the main results. The critical feature we need to model is, the distinction between inside and outside of a contour.

This is exactly what is achieved by Jordan curves defined in the Euclidean plane R 2 equipped with the topology of Euclidean distance.

Here we present the assumptions of our model:

1. The working space is Euclidean plane R 2 , 2. R 2 is partitioned into faces and contours by Jordan curves, 3. There exists a finest partition with a finite number of faces, called leaves.

This last axiom permits to construct a lattice structure and thus hierarchies.

We now define what a Jordan Curve and how it has been used to describe image segmentation or partitions.

Normal Segmentations

Here we must interject that the class of hierarchies of segmentations produced here are a subclass of the general hierarchy of partitions with no topological constraint. The constraint requires that two classes in the partition can't be merged if they are not connected an arc is removed from the partition. Thus hierarchies of segmentations which label the classes are a more general class of hierarchies. Here we will use the definition of a Normal Segmentation introduced by Morel et al. in [START_REF] Morel | Variational methods in image segmentation[END_REF] to formalize this:

Definition 4.2. A segmentation π is called normal if every sub-segmentation π of π verifies ω(π ) > ω(π)
Here the normality of a segmentation refers to the simplification of a segmentation such that:

• the merging of any two classes necessarily produces a suppression of atleast one Jordan Curve, thus causing a reduction in the energy associated with the partition (in the Mumford-Shah sense)

• the suppression of a Jordan curve produces a merging of 2 classes.

Morel et al. [START_REF] Koepfler | A multiscale algorithm for image segmentation by variational method[END_REF] further states to check computationally if the segmentation produced is normal, aside the necessity for finite number of classes, one requires the classes have no internal boundaries. That is if each Jordan Curve separates two different regions.

This property ensures finite classes remain at the end of the operation. This is termed by Morel et al as an 1-normal segmentation. An example in figure 4.3 from Morel et al [START_REF] Morel | Variational methods in image segmentation[END_REF] is reproduced. 123

Ordering and lattice of J-nets

Though the space of all J-nets of R 2 is ordered by inclusion, it does not generate a lattice because it is not upper-bounded, and anyway it is broad for our goal: in a circular crown of radius r > 0, one can draw an uncountable number of Jordan curves. We will restrict this space by considering only the J-nets included in a finite net N 0 , whose associated faces define the leaves. An example is given by the faces of the tessellation in figure 4.5.

Two families derive from N 0 : Firstly, the power set P(N 0 ), constituted by all sets whose points belong to N 0 . Secondly, the family N (N 0 ) of all J-nets included in N 0 . We have N (N 0 ) ⊆ P(N 0 ). Both sets P(N 0 ) and N (N 0 ) comprise the empty set ∅ and are ordered by inclusion. P(N 0 ), as a power set, is a Boolean lattice. But unlike P(N 0 ), the family N (N 0 ) is not complemented: if N ∈ N (N 0 ) the complement N 0 \N may have Jordan arcs which are not looped. However, the following property holds: Proof. N (N 0 ) admits a greatest element, namely N 0 . Let N, N ∈ N (N 0 ). The union N ∪ N , composed of Jordan curves belonging to either N or N , is therefore an element of N (N 0 ). Concerning the infimum, the largest lower bound of N and N is obtained by the union of all Jordan curves common to N and N . This family exists, i.e. is not empty, since it contains the empty set.

This approach focuses on contours, but the duality in R 2 provides the inverse ordering for the faces. If N ⊆ N , then The faces increase when contours are removed. Proposition 4.4 suggests to associate an opening with the N 0 -infimum.

∪ i {F i } ∪ S = R 2 \N ⊇ R 2 \N = ∪ i {F i } ∪ S
Corollary 4.5. Given X ∈ P(N 0 ) the union γ(X) of Jordan curves C contained in X γ(X) = ∪{C ⊆ X, C ∈ N (N 0 )} (4.1)
is an opening on P(N 0 ), called net opening, whose set of invariants is N (N 0 ). γ(X)

provides the largest J-net included in X. By duality in R 2 , if Y = R 2 \ X, the closing ϕ(Y ) = R 2 \ γ(X)
provides the largest classes having γ(X) as J-net contours.

In particular the N 0 -infimum between N, N ∈ N (N 0 ) is γ(N ∩ N ). The opening γ(X) simplifies X by suppressing all points which are not involved in a Jordan contour included in X [START_REF] Kiran | Scale space operators on hierarchies of segmentations[END_REF]. Figure 4.4 illustrates these changes.

Net Opening in Literature

The net opening operator γ is not a new notion. It appears already in 1982 in [101] under the name of pruning, where it serves indeed to cut branches in digital skeletons.

One can also find it in [START_REF] Leclerc | Description combinatoire des ultramétriques[END_REF] for discrete classifications by ultrametric. A similar line of thoughts is developed for characterizing the types of edges in topological watersheds [16] by Cousty et al. [START_REF] Cousty | Watershed cuts: Minimum spanning forests and the drop of water principle[END_REF], which produces watershed cuts, that partition the vertex set. In [START_REF] Cousty | Raising in watershed lattices[END_REF] a local variant is proposed to model flooding in digital watersheds. In the same context, it reappears in [START_REF] Cousty | Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts[END_REF], where each pixel is assigned as nodes set, and the edges is obtained as a function of the gradient between two pixels. The dynamic, surface, volume based saliencies are calculated using such edge weight values. The Minimum spanning tree is applied to the edge set to obtain the hierarchies of segmentations on the vertex set.

Further in the framework of graphs, Haximusa et al. [START_REF] Haxhimusa | Segmentation graph hierarchies[END_REF] perform hierarchical image segmentation by obtaining the infimum of costs over edges separating two components with the costs defined as the largest internal contrast in the fused component. This operator is posed as edge contraction on the Region adjacency graph(RAG), which removes an edge from a graph while merging the two vertices it previously connected.

There has also been work in the domain of edge and vertex based labeling studied in the well known Multi-cut problem [START_REF] Kappes | Globally optimal image partitioning by multicuts[END_REF]. The multi-cut is a NP hard combinatorial problem that determines the edges such that the sum of weights of cut-set is minimal. The paper [START_REF] Kappes | Globally optimal image partitioning by multicuts[END_REF] then proposes to optimize over the set of all separating boundaries while minimizing particular energy functions. Applications here are seen in the domain of closed boundary segmentation problems [5]. While the multi-cuts framework is flexible we are looking here for a simpler framework utilizing lattice (sup-inf) based optimization to extract closed contours from a predetermined hierarchy of segmentations.

Formulating γ as an Opening: By working on Jordan Curves, we can interpret this as an indifference in operating on edges and vertices on graphs, is one of the key motivations which these cited approaches miss. Furthermore the by defining the underlying lattice N (N 0 ), whose net opening operator γ expresses the infimum, one reaches a unique infimum by nature of the lattice, and is also reason why the net-opening γ works so well. Consequently this facilitates the ability to state theorems 4.6 and 4.9, and more importantly to compose several inputs, to introduce the filter γϕ, to provide closest bounds, etc. The opening γ results in the largest saliency function under g, which corresponds to an optimum. This is in contrast to the multi-cut criterion, where one minimizes the sum of the costs of the edge cutset.

Watershed transformation and Saliency function

This section which is mainly bibliographic revisits the saliency function from its development in the transformation.

Watershed

The watershed transformation has been a subject of intense study in literature. The transformation was introduced in the continuous domain by Beucher and Lantuejoul [START_REF] Beucher | A topological approach to hierarchical segmentation using mean shift[END_REF], followed by others, including Najman et al. [START_REF] Najman | Watershed of a continuous function[END_REF], and also in the discrete domain by Vicent et al. [16,[START_REF] Couprie | Topological gray-scale watershed transformation[END_REF]122]. In the continuous domain there have been problems in describing the watershed of a continuous function, since the watershed line produced may be thick, with nonzero area. It may also have so-called barbs which are branches of zero area with an end point [START_REF] Najman | Watershed of a continuous function[END_REF]. There has been an energy minimization, Water-snakes, based calculation of the continuous watershed for functions that are derivable [START_REF] Nguyen | Watersnakes: energy-driven watershed segmentation[END_REF].

The watershed transform in its approach, treats the input functions as a relief. The function used to determine the presence of edges, usually considered is the morphological gradient. When the function is seen as a mountain landscape, and one floods topographically the valleys, region boundaries are determined as watershed lines [START_REF] Meyer | Topographic distance and watershed lines[END_REF].

This analogy is also well described by the drop of water principle: a drop of water falling on a topographic surface follows a descending path and eventually reaches a minimum.

The watershed may be thought of as the separating lines of the domain of attraction of drops of water [START_REF] Cousty | Watershed cuts: Minimum spanning forests and the drop of water principle[END_REF]. The above discussion demonstrates that the watershed transformation has many ways of being defined. For example, based on the catchement basins of its minima [START_REF] Meyer | Topographic distance and watershed lines[END_REF], the watershed contour separating components [START_REF] Cousty | Watershed cuts: Minimum spanning forests and the drop of water principle[END_REF]. In addition, one finds continuous or discrete definitions. There are also many ways of handling cases when one finds a drop on a saddle point (plateaus).

In such cases one requires a distance function, which has been addressed in many ways, in literature, but can be categorized into the following:

1. General Non-Smooth Geodesic Distance: [START_REF] Meyer | Topographic distance and watershed lines[END_REF], Surfacial mean curvature based watersheds [START_REF] Romstad | Mean-curvature watersheds: A simple method for segmentation of a digital elevation model into terrain units[END_REF].

2. Smooth Geodesic Distance: [START_REF] Nguyen | Watersnakes: energy-driven watershed segmentation[END_REF], which uses a geodesic distance weighted by the norm of the gradient of the function to formulate the watershed calculation as a minmization.

Morphological gradient weighted distance functions:

The waterfall uses a simplified gradient resulting from the iterative mosaicking of the image. [START_REF] Beucher | hierarchical segmentation and waterfall algorithm[END_REF][START_REF] Meyer | Morphological operators for flooding, leveling and filtering images using graphs[END_REF], A topological watershed of a function, preserves the gradient between its regional minima.

The gradient across the regional minima is given by the minimal altitude at which the minima basins recombine [16,[START_REF] Cousty | Watershed cuts: Minimum spanning forests and the drop of water principle[END_REF]. Operations of extending the basins/minima by "raisings" has been suggested in [START_REF] Cousty | Raising in watershed lattices[END_REF] using again the gradient function. One can also find work on using the area, volume and dynamic of connected components of level sets of the function [START_REF] Cousty | Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts[END_REF].

4. Regularized gradient function: [START_REF] Vachier | The viscous watershed transform[END_REF] uses flooding by viscous liquid to simulate the regularization of the gradient to avoid over-segmentation problems. [START_REF] Vanhamel | Multiscale gradient watersheds of color images[END_REF] uses a diffusion scheme on the RGB gradient vector field, to simplify the gradient.

In all these cases its important to note that one assumes a finite set of local minima of the input function.

Saliency functions

Saliency functions have been firstly used to represent the hierarchy of catchment basins, produced when one employs hierarchical watershed methods. This was first introduced in [START_REF] Najman | Geodesic saliency of watershed contours and hierarchical segmentation[END_REF] where Najman et al. used the geodesic distance to produce sequence pairwise floodings that recombined minima based on the geodesic distance between pair of points belonging to these minima. The minima of the function are flooded as a function of their extinction values [START_REF] Vachier | Extinction value: a new measurement of persistence[END_REF], which orders the attributes [START_REF] Cousty | Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts[END_REF] of the components of the levels sets of a function, for example the area, volume and dynamic, to produce a hierarchy of partitions, and the corresponding saliency function.

Further Najman et al. [START_REF] Najman | On the equivalence between hierarchical segmentations and ultrametric watersheds[END_REF] suggests an equivalence between the set of all saliencies possible and an equivalent ultrametric watershed. It is of interest to note that the watershed depends on the gradient function which is calculated over the original image.

The Watershed clearly combines the two processes: calculating a dissimilarity function (usually a gradient), and calculating the partition separating the minima of the function using its gradient. We distinguish the two processes and define in our axioms:

• A finite set of Jordan nets provides model of segmentation [START_REF] Morel | Variational methods in image segmentation[END_REF]. This states that our starting point is the partition itself. The watershed is a more complicated problem, which involves various questions of thick contours and barbs, that are avoided by having a Jordan curve. The goal in using the Jordan nets is simply to define a hierarchy of partitions bounded under a numerical function. • In our study, the numerical function that decides the recombination/flooding of the components of the partition is no more dependent on the gradient of a function. This broadens the choice of the gradient based ordering required to generate watershed based saliency functions. It can be any function. In our case, it corresponds to the proximity of segmentation contours to a ground truth set. This is demonstrated in figure 4.

A hierarchical partition of the space, one can be determined in multiple ways based on the problem at hand (region merging, graph based hierarchies, waterfall strategies).

The critical difference in the formulation of a saliency function as numerical function on the Jordan net, is the use of a function which is independent of the function being partitioned (like Luminance of the image). In this case it depends on an external constraint: the proximity between the contours of a ground truth set and a partition. This is disjunct from the properties/attributes of the components or classes in the hierarchy of partitions.

Finally, the two methods watersheds, net-opening, are close in the type of operations they perform on arcs, but are operations really for two different cases. Watershed are more involved operations that involve gradient calculation which result in a partition of the space. While the net-opening simply tries to define a family of saliency functions that can be defined using a primitive partition under a given function.

Fusion of hierarchies and functions

We now get to the heart of the matter. We would like to distribute weights g : R 2 → R on the points of the basic J-net N 0 so that one could obtain larger and larger tessellations as the threshold increases. The cross section X t (g) at level t is given by

X t (g) = {x | x ∈ N 0 , g(x) ≥ t} (4.2)
has no special reason to be a J-net. Breaches can appear in some curves of N 0 . But γ(X t ) has no breach and is the largest J-net smaller than X t . Moreover, as γ is increasing, the successive J-nets transforms are nested:

t ≥ t ⇒ X t ⊆ X t ⇒ γ(X t ) ⊆ γ(X t )
This orients us towards the unique numerical version of the binary opening γ acting on level sets X t , denoted by the same symbol γ (γ(X) for sets and γ(g) for functions). It is given by

γ(g)(x) = {t > 0 | x ∈ γ[X t (g)]}
or, equivalently by the level sets

X(γ(g), t) = s<t γ[X s (g)] (4.3)
The successive thresholds of γ(g) delineate increasing tessellations of R 2 , i.e. by definition, a hierarchy denoted by H[γ(g)]. Though t is defined on R + , the number of level sets of γ(g) which are different is finite, just as the number of Jordan curves in N 0 is.

The Jordan net opening γ(g) generates a saliency on N 0 , in L. Najman and M. Schmitt's sense [START_REF] Najman | Geodesic saliency of watershed contours and hierarchical segmentation[END_REF], i.e. a numerical function whose thresholds are always closed contours. More precisely we can state the following:

Theorem 4.6. Let F be the family of the positive bounded numerical functions on R 2 , let N 0 be a basic J-net, and let γ be the associated numerical net opening . Given g ∈ F, γ(g) is piecewise constant on N 0 and provides the greatest saliency smaller than g on N 0 .

The pair [N 0 , γ(g)] characterizes a finite hierarchy H[N 0 , γ(g)]. The image I = γ(F) of F under the opening γ is exactly the family of all positive and bounded saliencies on N 0 .

Proof. Consider a Jordan curve C i ⊆ N 0 . Denote by t i the supremum of the t such that the contour C i is not damaged, i.e.

t i = sup{t | C i = C i ∩ X t (g)} (4.4)
For t < t i the opening γ[X t (g)] preserves integrally the internal face F i of C i , which is thus also preserved for t = t i , according equation (4.3). For t > t i a breach is made in C i ∩ X t (g) towards some face F j adjacent to F i and γ[X t (g)] makes both faces merge in a larger one, whose contour is included in N 0 . The set difference N 0 (t i ) =

C i \ ∪ t>t i γ[X t (g)
]indicates the points of C i that vanish above level t, or equivalently, the points of N 0 where γ(g) = t i . The net opening γ(g) is thus piecewise constant at values t i whose cardinal is finite, and its thresholds are closed contours. It is thus a saliency, and the two data of N 0 and γ(g) characterize the hierarchy H[N 0 , γ(g)]. As each saliency γ(g) ∈ F, the image I = γ(F) provides all saliencies. Besides, the transform γ(g) is the greatest saliency smaller than g since, for any saliency s 1 ≤ g ∈ F, we have by increasingness of γ, that s 1 = γ(s 1 ) ≤ γ(g) ≤ g.

Corollary 4.7. The family S of all saliency functions s on N 0 is a lattice for the pointwise numerical ordering. The supremum of a finite family {s i , 1 ≤ i ≤ I} is ∨s i , and the infimum is γ(∧s i ).

A saliency function s on Jordan net N 0 is equivalent to a parametrized hierarchy H(N 0 , s)

since the threshold

N 0 (t) = {C, C ⊆ N 0 , s(C) ≥ t}
generates a Jordan net at level t and since

t ≥ t ⇒ N 0 (t ) ⊆ N 0 (t)
Moreover this hierarchy H(N 0 , s) is parametrized in the sense that the sequence {N 0 (t), t ≥ 0}, of successive contours is not only ordered, but labeled by the level t of appearance of each N 0 (t). Conversely it is clear that the successive thresholds of a parametrized hierarchy H(N 0 , s) induce a saliency function on N 0 by Eq(4.4).

The equivalence between saliency functions and parametrized hierarchies leads to the following result:

Corollary 4.8. The lattice S of the saliency functions provides a lattice structure on the set H of all parametrized hierarchies on N 0 , where for H(s 1 ), H(s 2 ) ∈ H :

s 1 ≤ s 2 ⇔ H(s 1 ) ≤ H(s 2 ); H(s 1 ∨ s 2 ) = H(s 1 ) ∨ H(s 2 ); H(γ(s 1 ∧ s 2 )) = H(s 1 ) ∧ H(s 2 )

Composing hierarchies and numerical functions

The lattice structure of N (N 0 ) allows us to combine various bounds by suprema and infima. We can ask the following questions on the compositions of a saliency and any general family of functions, and in what type of problems would different compositions make sense.

1. Given that the base Jordan net N 0 is already the support of a predetermined finite hierarchy H with saliency s. When a non-negative function g 1 on R 2 is introduced, how to compose it with s?

2. When in turn a second function, g 2 , acts on the saliency s 1 resulting of g 1 , how the two effects are composed?

We will successively take up these questions by openings, closings and thickenings. Theorem 4.9. Let H be a finite hierarchy on N 0 , of saliency s, and let g 1 and g 2 be two non negative functions on R 2 , then:

Lower bounds by opening

(i) γ(s ∨ g 1 ) (resp. γ(s ∧ g 1 ))
is the largest saliency smaller than s ∨ g 1 (resp. s ∧ g 1 ),

(ii) if g 1 is itself a saliency on N 0 , then the supremum

g 1 ∨ s = γ(g 1 ∨ s) is a saliency, (iii) γ(g 1 ) ∨ γ(g 2 )
is the smallest saliency larger than or equal to γ(g 1 ) and γ(g 2 ),

(iv) if g 1 g 2 denotes an operation from F × F → F, such as +, -, ×, ÷, ∨,or ∧, then γ(g 1 g 2 ) is the largest saliency smaller than g 1 g 2 , and γ(g

1 ∨ g 2 ) ≤ γ(g 1 + g 2 ).
In all cases the resulting saliency is unique.

Lack of upper bounds by closing

Is it possible to reach similar bounds, but from above, by means of closings? We saw that the lattice N (N 0 ) is not preserved under complement. Thus it is direct but interesting observation that the complement saliency function g given by g = max(g) + 1 -g need not be a saliency function by itself.

Consequently, the closing ϕ, dual of the binary net opening γ for the complement on N 0 , maps P(N 0 ) on itself, but not on N (N 0 ), as shown by Figure 4.10. The asymmetry extends, of course, to the numerical case, and there is no closest closing because the infimum of saliencies may not be a saliency. The closing ϕ(g) in figure 4.9 does not define a hierarchy, but composition product γϕ(g) in figure 4.12 does.

The saliency γϕ(g), greater than the largest saliency lower-bound [γ(g) ≤ γϕ(g)] and smaller than the upper-bound ϕ(g) is indeed a good interpolation of saliency function from g.

Upper bound by Geodesic Reconstruction

The net opening γ(g) can be interpreted as a geodesic reconstruction of function g under the marker N 0 , J-net. The dual operation, which consist in a geodesic reconstruction from above(the complement), provides a saliency function which upper bounds g, called netting τ . For constructing it, we start from level set X t (g) of N 0 weighted by a function g as described in equation 4.4:

• if X t (g), as defined by Eq (4.2), is not a J-net, replace it by the leaves N 0 , which is a J-net,

• if not, leave X t (g) unchanged.

In both cases, continue with X 2 (g), which is now compared to the transform of X 1 (g).

When the top of the hierarchy is reached, thus in one pass, then the tessellations at all levels are larger than initially and increase with the level, i.e. τ (g) is a saliency, and τ (g) ≥ g, and this operation is idempotent: τ [τ (g)] = τ (g). However the operation τ is not increasing (see Figure 4.11), and thus belongs to the class of thickening [101]. Figure 4.12 shows the thickening of function g in figure 4.8. Though τ (g) is an upper-bound of g, but not necessarily the closest one, which may or may not exist.

Discussion on Graph based methods

We would like to conclude, firstly, by coming back to three arguments reasons that motivated the Euclidean framework:

1. The only operands used in both theory and experiments are points, Jordan loops, and their interiors, but never edges. The principle of parsimony urged to avoid 135 the useless distinction between vertices and edges, which would have automatically been introduced by a graph approach. However, our results could also be obtained, probably, in a discrete framework which distinguishes vertices and edges.

2. Jordan nets were weighted with functions g (e.g. distance functions), so that each point, in each arc of the net was given a weight. In case of a finite graph the resolution of function g is restricted to a single value for each edge. But there is no particular reason to impose this quantization to the function g, whose resolution can be much finer.

3. Distances in the physical world are isotropic, and we need a mathematical representation to preserve it. Now, an abstract graph ignores isotropy, except one that imposes it with its metric. The planar graphs are better: embedded in R 2 , they allow us to build distance functions by emulating the discs of R 2 . It is why we used this formulation in the preliminary version of the present study. [START_REF] Grady | Weights and topology: A study of the effects of graph construction on 3d image segmentation[END_REF] studies how image segmentation algorithms on graph often have to decide optimal connectivity and topology. With respect to the up-sampling in the discrete spaces, the discretized image fits correctly with its euclidean version, if we admit two assumptions. Firstly no features is smaller than the elementary square grid, and secondly, there is at most one feature change between two neighbouring pixels. The double resolution is nothing but a mean One important different w.r.t [START_REF] Najman | On the equivalence between hierarchical segmentations and ultrametric watersheds[END_REF] is that here we have boundary operator that simply calculates the non-zero gradient contours, since the labeling of the components does not produce an ordering dependent on the gradient of the image, and furthermore can be arbitrary. This double resolution to separate cells in discrete topology is the well known Khalimsky topology [START_REF] Melin | Digital khalimsky manifolds[END_REF], further one can find the Khalimsky's digital jordan curve theorem in by Kiselman [START_REF] Kiselman | Digital jordan curve theorems[END_REF]. for exploiting these assumptions. It fits well with the Euclidean Jordan net approach, which divide the space into faces and J-net, since it localizes the J-net in the union of the even rows and columns of the up-sampled image.

Algorithm and Experiments

Fusion of ground truth and hierarchy

Conventionally the ground truth information is intended to assess the quality of a segmentation, here a hierarchy H of segmentations. Here in the place of evaluating the hierarchy, we analyse it with respect to the given ground truth. The saliency transformation by a ground truth is an amelioration of the partitions in the hierarchy by reordering them. They generate new partitions with the same edges ordered by combined effect of proximity to the ground truth, and high value of saliency function(note that this is optional). More clearly, how do we combine a ground truth and a hierarchy?

The

Fusions of two hierarchies

In this second example, we will demonstrate a fusion of two hierarchical structures. In To achieve this fusion one needs a spatial proximity information of the different scales of partitions across the two saliency functions s 1 and s 2 . Thus we convert a saliency function s into an ordered distance function d (s) by summing the distance functions of the different partitions in the thresholds of the saliency function as in equation (4.5).

One of them, s 1 say, serves to generate the function g 1 by taking the sum d Σ (s 1 ) of the distance functions of its level sets,

d Σ (s 1 ) = d(X t ), X t = {x | s 1 (x) ≥ t}, (4.5)
where 0 ≤ t ≤ t max . The information about s 1 is considered as summarized by d Σ (s 1 ).

Thus we use the distance function representing one hierarchy over the leaves (Jordan- This is a bit more tougher to calculate since in the earlier case the distance function associated with set G gave a point-wise function that could be used on each Jordan net.

In this case for each point in the Jordan net, we can now define the complementary distance as the radius dilation at each point so as to cover the ground-truth partition completely. The distance function can now be writing as the infimum of the distance function of a point x on the ground truth set G:

d G (x) = ∧ x∈G d(x) (4.6)
This basically corresponds to the supremum of the distance function for each point, on the ground truth set.

Thus now one can write the reordering of the Jordan anew now using this new distance function as follows:

s G = γ(π 0 + (1 -d G )) (4.7)
Now we can produce two saliencies: Going from G → H and other from H → G. We are now in a position to write the Hausdorff distance ordered saliency:

s G↔H = γ(N 0 + 1 d ) ∨ γ(N 0 + 1 d G ) (4.8)
There is also another way of writing this saliency function:

s G↔H = γ(N 0 + 1 d ∧ d G ) (4.9)
The two methods of writing the Hausdorff distance ordered saliency suggest that it is composed of the supremum of two net openings, which is also an opening and the resulting function is also a saliency. The inverse of the distance is taken to produce a function which is highly salient when the classes of ground truth and the segmentation are symmetrically placed.

s ∧ = ∧ n i=1 γ(N 0 + (1 -d i )) (4.11)
which can be rewritten using a inf-composition over the functions followed by the net opening:

s ∧ = γ(N 0 + (1 -∧ n i=1 d i )) = γ(N 0 + g ∧ ) (4.12)
Let's consider the case when we have multiple ground truths G i drawn by multiple experts, and the problem to now consider is to produce a saliency function weighted by the relative frequency with which the experts draw the same contours as ground truth. In this case the inf-composition of distance functions d ∧ = ∧ i d i , it would produce minima's at all points of the ground truth, including the locations where they intersect.

In the current problem one would like weighting that produces saliency for contours proportional in value to the number of ground truth contours. This gives us:

s + = γ(N 0 + (1 - n i=1 )d i )) = γ(N 0 + g ) (4.13)

Measuring structural changes after transformations

In this section we provide a way to analyse the structural changes in a hierarchy of partitions, resulting from a net opening using a ground truth distance function. This provides one of the possible measures to evaluate the hierarchy, while various studies are already available on the subject [10, [START_REF] Haxhimusa | Evaluating hierarchical graph-based segmentation[END_REF][START_REF] Pont-Tuset | Supervised assessment of segmentation hierarchies[END_REF]. We introduce an evaluation measure:

M = π i L π i+1 L = #labels in childLevel #labels in parentLevel (4.14)
where, the quantity π i L refers to the number of different labels at a particular level of the hierarchy. The number of labels in the case of saliency function is the number of labeled connected components, in the partition π i . The measure is not new and has been widely used in studying dendrograms and hierarchies. Classically in computer science, the branching factor for tree data structures, is the number of children at each node.

Measure M is plotted for different distance functions corresponding to different ground truths and a point distance function, in figure 4.23. The x-axis here corresponds not only to the threshold of the saliency function,i.e. level in hierarchy, but also to the proximity of contours at this level to the respective ground truths. Though this measure M without 147 Note: The union of finite countable set of Jordan curves gives a finite set of points, over which a further net opening gives a finite Jordan net. Though this may not be the case with the union of Jordan nets, which might produce Jordan curves with infinite number of faces. Thoght it should be noted that on the base Jordan net representing the pixel grid, the union will always, provide a finite Jordan net, and numerical net opening consists of a union of openings.

Braids over multiple hierarchies

In this section we use the elementary binary net opening described in the corollary 4.12, to create the monitor hierarchy. Demonstrated in the figure 4.25, the net opening is used to extract the largest partition whose classes contain the classes of two input partitions.

By iterating this binary operation top-down on both input hierarchies, over every pair of partitions extracted for the threshold of the saliency function, one generates the monitor saliency function, and thus hierarchy. Please refer to algorithm 5 in the next chapter, for a better idea of how this is done in a top-down manner. This composition of hierarchies enables a multi-variable hierarchies to be governed by a monitor hierarchy, enabling us now to calculate optimal cuts on the braids. One needs to note that the monitor hierarchy can turn out to be trivial or of less importance if it very few monitoring supremum classes.

There are a variety of experiments we can perform in case of multi-variable hierarchies: 151

• Hierarchies for color images, for example over the R,G,B images.

• Hierarchies for Hyperspectral images, similar to the RGB case, except larger in dimension

• Hierarchies for Stereo images, to characterize hierarchical structures that remain invariant across the stereo-pair iamges.

In these different applications one has different interpretations for the monitor hierarchy. 4.9. SUMMARY

Summary

Chapter contribution summary

We worked purely on the contours of a finest Jordan net and defined a netopening which either removes or preserves the arcs in the net, producing a new hierarchy.

The net opening was used to find the hierarchy whose saliency is the closest to a given ground truth, represented by its distance function g. This is the first time the operator has been interpreted as an opening γ producing the largest Jordan net that includes any input contour set X. The dual closing does not exist since the lattice N 0 is not complemented.

Geometrical net openings were discussed which help simplify regular hierarchies like Quad-trees and mesh hierarchies. We distinguish here the difference between reordering the Jordan nets based on an external function and based on the intrinsic properties of the partition created by the Jordan net (curvature, junctions, symmetry).

We demonstrate rules of composition of functions, yielding the largest saliency function, bounded by the composition.

In Arbelaez et al, Cousty et al. [10,[START_REF] Cousty | Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts[END_REF], the saliency function is used solely as a representation of the hierarchy of partitions. They provide a visual interpretation of the segmentations at different scales of attributes(dynamic, area and volume) of the gray scale components. In contrast, our study the saliency is used an input numerical function to manipulate the hierarchical structure.

Braids can be created by composing multiple hierarchies, by performing net opening over the tuples of partitions from the different hierarchies. Braids can also be created by composing net openings using multiple external functions on the same Jordan net.

First we will present the original dynamic program used by Guigues, over the HOP, and later continue with version for the Braids. Algorithm 1 is the same as BottomUpAnalysis from Garrido's thesis [START_REF] Garrido | Hierarchical Region Based Processing of Images and Video Sequences: Application to Filtering, Segmentation and Information Retrieval[END_REF] and calculates the optimal cut given a scale parameter λ. It also calculates the R, D functions in [START_REF] Garrido | Hierarchical Region Based Processing of Images and Video Sequences: Application to Filtering, Segmentation and Information Retrieval[END_REF] which here are ω ϕ (π * ), ω ∂ (π * ). The energies ω ϕ , ω ∂ are are available for all classes in the hierarchy S ∈ S. The energy in case of Salembier-Garrido and Guigues correspond to the Lagrangian ω ϕ (π)+λω ∂ (π). While the composition of energies of child classes can be obtained by a general function denoted here by ComposeF unc(ω φ (π(S)), ω ∂ (π(S)), λ)

Optimal Cut DP on HOP

or ComposeF unc(ω(S)), which represent:

• Addition: T i ∈π(S) ω φ (T i ) + λω ∂ (T i ) • Supremum: T i ∈π(S) ω(T i ) • Infimum: T i ∈π(S) ω(T i ) • Also u∈[1,q] ω(T u ) α 1 α
Minkowski-functional from equation (1.33).

all of which are h-increasing energies with compositions that are h-increasing. The algorithm 1 can also be used for a single energy, while not involving a constrained optimization problem.

Algorithm 2: Λ-Func(H, ω φ (S), ω ∂ (S)) Data: H, ω φ (S), ω ∂ (S).

Result: Scale function Λ(S). The optimal value of λ in case of Garrido has been obtained by a gradient search which starts with a upper and lower bounding value of λ, giving ω h ϕ (π), ω h ∂ (π) high data and constraint terms, and ω l ϕ (π), ω l ∂ (π) and the lower terms, such that the constraint is bounded between the two bounds ω l ∂ (π), ω h ∂ (π). The new λ iteratively recalculated by setting it to

λ = ω l ϕ (π) -ω h ϕ (π) ω h ∂ (π) -ω l ∂ (π) (5.1)
And in each iteration one recalculates the optimal cut for each new λ .

Algorithm 3 is not an efficient implementation, we present it for the sake of pedagogy to demonstrate how scale-increasingness works. One has linear complexity according to L.Guigues implementation, to extract all the λ-cuts. One should note here that the Guigues calculates the whole hierarchy of optimal cuts π * (λ), which is not the same as

(λ)| > 1 do i ←-i + 1 π * (λ) ←-OptimalCut(H, Λ(i), ω φ (S), ω ∂ (S)) H * (i) ←-π * (λ) end end
Salembier and Garrido, who calculate an optimal λ * given a constraint function value

ω ∂ ≤ C.

Optimal Cut DP on BOP

In this section we first present the algorithm to create a Braid of partitions by composing a family of HOPs.

The algorithm 4 extracts an optimal cut given q-hierarchies and a λ. The Monitor hierarchy to compose the q-hierarchies is generated by algorithm 5, which uses the net opening operator. Though this might not be the only way to generate braids. This demonstrates that given q-hierarchies one might encounter two possibilities:

1. Monitor Hierarchy with, single(full space) class, as unique class.

2. Non-trivial monitor hierarchy and thus Braid.

In case (1) one can now calculate the q-optimal cuts independently from the q-hierarchies, after which the optimal cut with the least energy is picked, in case of equal energies, one picks any of the equivalent optimal cuts. In case (2), the braid structure now ensures an partial optimal cut, that composes partial partitions from the q-hierarchies, and compares it with the monitoring hierarchy's parent, in the dynamic programming stage. The partial optimum is then either, the partial partition that is minimal in energy, or any of the equivalent partial partitions with equal energies (enforcing singularity here), or the parent. This would ensure a minimal λ-cut that could reach at least equal or better infimum w.r.t the original q-hierarchies, This betterment also impinges constraints upon the generation of the braid structure.

Algorithm 4: DP on BOP: Optimal-Cut-BOP({H i , i ∈ [1, q]}, λ, ω ϕ (S), ω ∂ (S)) Data: q-Hierarchies {H 1 , H 2 , ...H q }, λ, ω φ (S), ω ∂ (S). Result: Optimal cut π * (λ), ω ϕ (π * (λ)), ω ∂ (π

, i ∈ [1, q]} do [π * i (λ), ω ϕ (π * i (λ), ω ϕ (π * i (λ)] = Optimal-Cut(H, λ, ω ϕ (S), ω ∂ (S)) ω i (λ) ←-ω ϕ (π * i (λ) + λ • ω ϕ (π * i (λ)) end i * ←-arg min i ω i (λ) π * (λ) = π * i * (λ) ω ϕ (π * (λ)) = ω ϕ (π * i * (λ)) ω ∂ (π * (λ)) = ω ∂ (π * i * (λ)) return else \\Perform DP with monitor hierarchy F (S) ←-0, S ∈ H Set all flags to zero for level ∈ [1, N monitor ] do for S ∈ H monitor (level)] do ω(S) ←-ω ϕ (S) + λω ∂ (S) for π i (S) ∈ H i do ω i (π(S)) ←-ComposeF unc(ω ϕ (π i (S)), ω ∂ (π i (S)), λ) if (ω(S) ≤ ∧ i ω i (π(S))) then F (S) = 1 Optimal Parent else π * (S) = arg min i ω i (π(S)) ∀T i ∈ π * (S), F (T i ) ←-1 Optimal Child end end end end end for S ∈ {S} do if F (S) then π * ←-π * S add optimal classes to cut. ω ϕ (π * ) = ω ϕ (π * ) + ω ϕ (S) ω ∂ (π * ) = ω ∂ (π * ) + ω ∂ (S) end end end
The scale-function and hierarchy of λ-cuts algorithms do not change very much in their structure for the BOP otherwise.

Algorithm 5: Generate Monitor Composing q-hierarchies. Data:

{H i }, i ∈ [1, q]. Result: Monitor Hierarchy H monitor begin ∀H ∈ {H i }, i ∈ [1, n], N i ←-|H i | N min = min i N i \\Top-Down scan on q-hierarchies for level = 1 : N min do ∀H ∈ {H i }, i ∈ [1, n], π i ←-H(N i -level)
\\Intersection set of all partitions I = i π i \\Net opening on intersection set.

H monitor (N min -level) ←-γ(I) end end

Intersection Graphs for Partition selection

The goal in this section is to identify that the Maximaly independent sets of intersection graphs corresponding to a family of partitions, enables one to extract partitions of the space, other than the ones in the family. Further on we re-introduce the family of "Partition Graphs" which explicity ensure this property for general coverings of the space.

First off, to being this section we follow the work in Brendel-Todorovic [START_REF] Brendel | Segmentation as maximum-weight independent set[END_REF] which provides the formulation of image segmentation segmentation problem as a maximally weighted independent set(MWIS) problem on the intersection graph of segmentation stack generated by low level segmenters.

We note that the MWIS problem on the intersection graph corresponding to a HOP, consists in calculating a family of optimal cuts in the hierarchy. We also discuss a family of partition graphs modeling partitions of the input space given any general covering set, that has useful properties.

Definitions

We briefly review the definitions of a maximal independent set on a graph, and the Maximally weighted independent set on a weighted graph, in this section. Let the pair G = (V, E) be an undirected finite graph where V denotes the set of vertices and E denotes the set of edges. If G is connected and acyclic, then it is called a tree.

A subset I ⊆ V is said to be an independent set of G if no two vertices are adjacent in G, or there is no pair of nodes in I linked by an edge in E. Assume also that a positive value is associated to every node in the graph ω : V → R + . Now we define a maximal independent set (MIS).

Definition 5.1. (Maximal Independent Set) Given an graph, an independent set consist of vertices in a graph, where no two vertices are adjacent. A maximal independent set is an independent set such that no more vertices can be added to the set without forcing the set to contain an edge. Now, maximally weighted independent set(MWIS) is an independent set with maximal weight.

In case of identical weights, the maximal property corresponds to the ordering relation, that there are no other set I that contains I. Finding a MWIS in a general graph is NP-hard. For particular cases of the graph structure, we can calculate the MWIS in polynomial time.

Definition 5.2. An undirected graph G = (V, E), where for any covering {S 1 , S 2 , ...S n } of the space E, on associates a node v i ∈ V of the graph for each set S i , and connecting any two vertices v i , v j when their corresponding sets have a non-empty intersection, i.e.

S i ∩ S j = ∅.

One can distinguish different types of graphs by the nature of the intersection between two sets in the family covering the space:

• Intersection Graph: S i ∩ S j = ∅ • Overlap Graph: S i ∩ S j = ∅

MIS and hierarchies:

There has been some notable work in using the maximal independent sets, on the one hand to create the hierarchies, while on the other hand to perform partition selection or in other words extract an optimal cut.

To create stochastic hierarchies of partitions with minimal height, Haxhimusa et al. [START_REF] Haxhimusa | The structurally Optimal Dual Graph Pyramid and its Application in Image Partitioning[END_REF][START_REF] Haxhimusa | Constructing stochastic pyramids by mides: Maximal independent directed edge set[END_REF], create a graph pyramid from maximal independent vertex set of a base weighted pixel graph, while associating a uniform distribution of values to the vertex set. This is purely to generate a HOP, and not partition selection. The paper also studies the Maximal Independent Directed Edge Set(MIDES), to create the stochastic pyramid.

From experiments [START_REF] Brendel | Segmentation as maximum-weight independent set[END_REF], it is notable that MWIS algorithm select meaningful segments from a provided set of low-level segmentations of the image. This of course depends on the features producing, based on the weights or energies assigned to each segment and to each neighboring segment pair the MWIS outputs a single, unique partition of the image. Brendel-Todorovic [START_REF] Brendel | Segmentation as maximum-weight independent set[END_REF] applies MWIS onto the already calculated UCM hierarchy, segmentation performance is noted to go up by selecting a cut traversing several levels and not just a horizontal cut.

Partition Graphs

In this section we define a graph structure that would enumerate and extract the partitions in a HOP. This corresponds basically to the MIS of an Intersection graph corresponding to a HOP.

First defined in [START_REF] Mcavaney | A characterization and hereditary properties for partition graphs[END_REF], where for given set or space E, the Maximum Independent Sets of a Partition Graph would correspond to partition of space E. Thus for any MIS M ⊂ V yields a partition,

E = v∈M S v
Furthermore this results gives a bijection between the partitions from a HOP and the family of MIS of the partition graph corresponding to the classes from a HOP. Consider M := {M } to be the set of all MIS of a partition graph corresponding to a HOP H, then we can write:

M → Π(H, E) (5.2)
One of the important properties from [START_REF] Mcavaney | A characterization and hereditary properties for partition graphs[END_REF], we will be using in the case of hierarchies and later for braids, is that the subgraph of any partition graph will also be a partition graph for the subset of vertices considered in the subgraph. This property is simple to prove, though is vital to understand the nested structure that recursively is produced in each class in the hierarchy/braid. We will return to this point shortly. We also see later why the intersection graph for a braid does not form a partition graph, but needs some transformation so as to have the MIS-partition bijection as in equation 5.2. For further development on the properties of the graph one can find a good summary in [START_REF] Mckee | Topics in Intersection Graph Theory[END_REF].

Maximally weighted Independent Set on HOP Intersection Graph

Given a hierarchy of partitions(HOP) H of space E, one can construct an intersection graph, where each class family S = S i ∈ H is represented by node v i ∈ V , and an edge between any two classes is added when they have non-zero intersection S i ∩ S j = ∅, ∀S i , S j ∈ H. In case of hierarchies the intersection is necessarily implies an inclusion relation, though in case of braids this will not be the case. The intersection graph corresponding corresponding to the classes of H, is in fact a containment graph, where an edge exists between any two vertices V (S i ), V (S j ) is S i ⊂ S j or S i ⊃ S j [START_REF] Čenek | Maximum independent set and maximum clique algorithms for overlap graphs[END_REF]. This produces an edge between any child and all of its ancestors and vice versa. One can note that the whole space E as a single class, if present, is linked to all the other nodes in the graph since its intersection with any class is not empty. Note for brevity we reiterate here that, the maximal nature of a Independent set does not refer to the refinement or coarseness of the classes in the partition, and the impending refinement ordering of partitions. One can now add the following observation:

MWIS for HOP: Now, given a weight ω : V → R + to each node, then the Maximally weighted Independent Set (MWIS) is the MIS with the largest weight, denoted by the family M * . This basically refers to the fact that there are more than one, MWIS that partition the space. When we have the singular energy ω 1.9, the exists only one MWIS, i.e. M * = {M * }, which corresponds to the unique optimal cut.

An example of MWIS on the intersection graph formed for a given hierarchy of partitions is shown in figure 5.2.

Intersection Graph for Braids

Using low-level segmenters like in [START_REF] Brendel | Segmentation as maximum-weight independent set[END_REF], or super-pixel segmentations algorithms like SLIC [1] with increasing scale, one can create "stack" of partitions/segmentations Π stack . and its intersection graph was calculated. For the purpose of robustness [START_REF] Brendel | Segmentation as maximum-weight independent set[END_REF] to avoid thin or parasitic intersections, Brendel-Todorovic adds a constraint that the overlap between segments needs to be greater than a filtering threshold chosen, i.e., S i ∩ S j ≥ t. This also the reason why [START_REF] Brendel | Segmentation as maximum-weight independent set[END_REF] needs morphological post-processing to remove soft-overlaps.

One also needs to take into account that this threshold needs to be chosen smaller than size of classes in the stack, i.e. t << |S i |.

Intersection graphs on Braids: Given a braid of partitions as in figure 5.3, one can find a situation where the soft overlaps are not respected anymore. The family of partitions from a braid consists of classes which may have multiple fathers with large overlaps, which may not produce partition graphs, and would require further operations to have a correspondence between the cuts of a braid and the Maximal independent sets of its intersection graph. To repair this situation when one does not have a covering, we basically reject this MIS by forcibly connecting disjoint classes, and recursing this process for the tuplets of classes from the disjoint class cones. This operation basically enforces the extraction of partitions and enforces them to be a covering of E.

We observe though that for a general stack of partitions, where small overlaps are possible between large classes, one would lose approximate good solutions by performing this recursive correction of the intersection graph, though this is still up for debate since the class size are controlled by parameters in low-level segmentation algorithms. We leave this problem of repairing the intersection graphs "open" for further study.

Another point to note is the similary in the dynamic program substructure between the Breiman's BFOS algorithm [21] and that of the of extraction of the MIS of an intersection graph. Both have a recursive structure.

Max-Flow Min-Cut Analysis

In this section we briefly formulate the optimization problem over a graph, which is an undirected forest/tree, corresponding to first the HOP, Further we also show what could be the equivalent graph topology to model the optimal cut by min-cut in case of the Braid of Partitions.

Graph Structure for HOP

The definition of a flow through G requires the data of a source and a sink. The particular shape of a pyramid leads us to take for source the family A of all leaves, and for sink the whole space E. In flows, capacities are often allocated to the edges, and sometimes to the vertices. For the sake of comparison, in case of a hierarchy H, we will take the nodes. Now, in the graph case, one wants to maximize the flow, whereas above, both additive and sup-generated energies were the matter of minimizations. We must choose, and from now on we decide to maximize the hierarchical energies, i.e. to invert the ordering relations (e.g. in comparisons father/sons of the h-increasing case).

In a hierarchy, each leaf a is connected to the root E by a unique path [a, .., E], strictly increasing, and different for each leave. For example, in optimal cut are those which are labelled 0, as depicted in Figure 5.4. Finally, we exactly obtain a min-cut in the graph-cut sense, but presented in another formalism, and we can state:

Proposition 5.5. Given an additive energy ω the optimal cut of a hierarchy H is exactly the min-cut for sources located at all leaves and a sink located at the root, over the weighted tree representing H.

A final point to note is that the energies in case of HOP, are additive in case of calculating the max-flow. We will now extend this to a braid with a convenient graph decomposition.

Graph Structure for BOP

Let us consider a braid of partitions B with a monitor H included. For the sake of demonstration we will consider a braid created by composing two hierarchies H 1 , H 2 .

Each class S ∈ B is associated with a node, while the inclusion relation S ⊆ a i , a i ∈ π(S) led to the edges, representing the parent-child relationships.

In figure 5.5 we demonstrate an elementary example of a braid composed of two hierarchies: H 1 = {{E}{a 1 a 2 }} and H 2 = {{E}{b 1 b 2 b 3 }}, where the monitoring class is the {E}. As seen in the demonstration from having a source node S followed by ∞ nodes, provides us a graph that has a tree structure, over which the calculation for min-cut corresponds exactly to the min-cut on the HOP.

In the case of BOP, aside the binary decision between a parent class and child classes, one also needs a binary decision between the different child classes, in this case in figure 5.5

Summary

This chapter has basically provides a compilation of the different algorithms and graph structures used in the domain of the hierarchical cuts problem.

Chapter contribution summary

Algorithms to calculate a braid from a family of hierarchies, and extract the optimal cut for given energy.

Review on super-pixel and review of MWIS formulation of image segmentation problem.

Recalling the definition of "Partition Graphs", which enumerates the partitions from a HOP, transforming an intersection graph for BOP, into a Partition Graph.

Max-flow min-cut formulation over a tree corresponding to a HOP and over a series of trees.

Chapter 6 Conclusion

Thesis Contributions

A brief overview of contributions are provided below.

• The thesis introduces the concept of an energetic lattice which is a pair (H, ω), of a hierarchy of partitions H, and energy ω. This structure enables the use of axioms of a lattice, especially the in defining the existence and uniqueness of the minima.

• We introduce the Braid of Partitions, that is richer structure in between, the all possible sets of partitions and the hierarchy of partitions. This enables one to have a larger search space than the HOP, and possibly better minimums on the energetic lattice. Furthermore the braid allows for a multiscale overlapping segmentation hypothesis over the image domain, thus providing its possible use in optical flow and achieving better segmentations by structuring segmentation stacks from low level super-pixel segmenters. The structure of energetic lattice, which is valid for braids and uniquely for them, leads to various results of optimality on braids. • We define a local Haussdorf based proximity measure between any two partitions. This is introduced to solve the problem of extracting a partition from a hierarchy H, that is closest to a ground truth partition G, where G, h correspond to an input image. This local-global problem enables the application of the energetic lattice as well as the dynamic program with infimum composition. We also study the laws of composition for composition of ground truth partitions.

• In an apparently disjunct contribution, we show how to construct a hierarchy of partitions using a finite set of Jordan curves partitioning the image domain, and introducing a lattice structure over it. The method derives but delineates from the study on ground truth energies, where the lattice is moved from partial partitions to the family of jordan curves. With the lattice structure we introduce the net opening that extracts the smallest set of jordan curves, and the numerical opening.

In the problem of extracting a proximal partition from a hierarchy w.r.t the ground truth, we calculate a new hierarchy, including the proximal partition, by ordering the set of Jordan curves, by the distance function of the ground truth.

To have a global picture of the progression of ideas with the energetic lattice and the lagrangian approach refer to Figure 6.1, and Figure 6.2.

Applications and Future perspectives

We shall briefly provide perspectives on how Braids can be used in the domain of hyperspectral image segmentation, when working on spatial-spectral methods [START_REF] Valero | Hyperspectral image representation and Processing with Binary Partition Trees[END_REF] and

arg min π∈Π(E,H) ω ϕ (π) subject to ω ∂ (π) ≤ C Lagrangian Multiplier [47, 100] min ω ϕ (π) + λ(ω ∂ (π) -C) Energetic Lattice Π ω (E)
Sub/Super-additivity of ω ∂ /ω ϕ Singularity Inf-Modularity Family of Minimal Cuts Π * (E) Figure 6.2: Two models to formulate the constrained optimization problem. The Lagrangian multipler's method minimizes the Lagrangian of the constrained optimization problem for each feasible λ, while the energetic lattice defines an order on the lattice structure of partitions. The infimum of the energetic lattice, in this case for the Lagrangian function ω(π, λ) = ω ϕ (π) + λω ∂ (π), leads to the ordered set of optimal cuts. The conditions on the energy are presented in rounded boxes, so as to obtain a minimum on the HOP for both methods. One can see the counterpart conditions of the Lagrangian Multipliers method, are properties of functions needed to achieve a ordering of energies on the HOP partition strucure. Though its important to note that the energetic lattice can be formulated for the lagrangian linear case, while the axioms of h-increasingness and scale-incresingness, generalize subadditivity and superaddivity.

with a common base of leaves, and a same number of levels. How to synthesize these hierarchies, and to extract from the series of H k a unique (constrained or not) optimal cut ? Introduce the refinement supremum H = ∨H k of the H k , i.e. the hierarchy H whose section π at each level is the refinement supremum of the π k at the same level.

The family {H k } of all sections of all H k turns out to be a braid of monitor H. At point x the minimization processing will result in the class among the S k (x) which is locally the less energetic. We have already seen the work of Chanussot, Angulo, Santiago and Valero et al. [START_REF] Angulo | Multiscale stochastic watershed for unsupervised hyperspectral image segmentation[END_REF][START_REF] Gueguen | Local mutual information for dissimilarity-based image segmentation[END_REF][START_REF] Valero | Hyperspectral image representation and Processing with Binary Partition Trees[END_REF] on how one enters the multidimensional case.

Thematic partitions

We now consider a partition of the Euclidean plane into thematic classes, like the GIS classification in the P categories of water, forest, cultures, habitations, soil, etc... Each class is given one "color" taken among the P categories. As R 2 is topological, one can speak of frontiers and of adjacent classes. A matrix M (p, q) = [m(p, q)] of similarities between adjacent classes is given. Two classes S p and S q merge if they are adjacent and if their similarity is higher or equal to [m(p, q)]. The similarity of S p and S q is an energy which can include physical parameters (nature of the soil, slope, etc.) and geometrical ones (length of the frontier, simplicity of the union S p ∪ S q , distance to a highway, etc.).

The merging of each p → q of the P categories leads to 2 P 2 partitions. They form a braid B because one can construct a monitor hierarchy H by means of the Jordan net opening developed in Chapter 5. The maximal cut of this braid is a partition which optimizes the similarities, a processing which thus results in a clustering.

Thematic prediction

We continue which the previous situation, by adding the time dimension. We start from the partition of a region Z in P categories, and we want to estimate the partition of the same region at time t . The classes are the same, but their categories may change between t and t . One can urbanize a class initially devoted to agriculture, or extend former soils to forest, etc. A transition matrix A = [a(p, q)] has been estimated from other sources, and is given. This matrix is a priori different from the previous one M .

A class S p of category p is changed into S q according to an energy associated with the categories of the neighbors of S p and with matrix A. It lead to P 2 partitions which form a braid B, as previously. The maximal cut of B is then the best estimator w.r.t. the energetic choices and the matrix A.

Similar situations occur in binocular vision, or in motion estimation. In binocular vision, for example, an energy is associated with the lateral shifts, and the classes of the optimal cut provides zones of the space considered as located at the same depth.

Combination of earth images

When they focus on a same geographical zone Z, different sensors produce maps which indicate various informations, and with different accuracies. The same as for situation c) (thematic hierarchies) may be said here. The optimal cut of the braid provides here the optimal synthesis of the maps, w.r.t. the chosen criteria. 5) is removed and replaced with a Voronoi partitioning giving [START_REF] Angulo | Multiscale stochastic watershed for unsupervised hyperspectral image segmentation[END_REF]. [START_REF] Arbelaez | Une approche metrique pour la segmentation d'images[END_REF] indicates the intersections of the contours of the initial segmentation (2) and the Voronoi partition in (4), and ( 8) is the net opening (refer to operator in chapter 4) of ( 7) and similarly for ( 9) and (10) with the second parasitic class in (5). (11) depicts the supremum of the two partitions ( 8) and (10). The three partitions (1,[START_REF] Arbelaez | Boundary extraction in natural images using ultrametric contour maps[END_REF]11), with the whole space form the monitor hierarchy H of the braid made by (1,[START_REF] Ravi | Optima on Hierarchies of Partitions[END_REF]5,[START_REF] Arbelaez | Boundary extraction in natural images using ultrametric contour maps[END_REF]10,11), where the (8) = sup (1,[START_REF] Ravi | Optima on Hierarchies of Partitions[END_REF] 1.17 Optimal cuts for composition laws: addition, (α, ω)-component supremum and Akcay's refinement ordering. For the composition by addition, one compares the energies of parent and sum of energies of child nodes. For the binary energy based supremum, in the current example each node consists of α i -components and the value withing each component is the global contrast parameter ω. Here we demonstrate a cut for which the ω ≤ 20. Though this is not a direct composition by supremum. Finally in the Akcay's refinement ordering example the optimal cut picks nodes whose descendants are all smaller that itself. This requires a two pass algorithm. According to the application other laws may be used e. two level saliency functions. We have τ (g) ≥ τ (g ) though g ≤ g . . . . . 4.12 a) filter γϕ(g), i.e. the closest saliency below ϕ(g), b) thickening τ (g). . . 4.13 Toy example showing down-sampling and the different stages of operation to achieve the net opening. One important different w.r.t [START_REF] Najman | On the equivalence between hierarchical segmentations and ultrametric watersheds[END_REF] is that here we have boundary operator that simply calculates the non-zero gradient contours, since the labeling of the components does not produce an ordering dependent on the gradient of the image, and furthermore can be arbitrary. This double resolution to separate cells in discrete topology is the well known Khalimsky topology [START_REF] Melin | Digital khalimsky manifolds[END_REF], further one can find the Khalimsky's digital jordan curve theorem in by Kiselman [START_REF] Kiselman | Digital jordan curve theorems[END_REF]. . . . . . . . Lagrangian multipler's method minimizes the Lagrangian of the constrained optimization problem for each feasible λ, while the energetic lattice defines an order on the lattice structure of partitions. The infimum of the energetic lattice, in this case for the Lagrangian function ω(π, λ) = ω ϕ (π) + λω ∂ (π), leads to the ordered set of optimal cuts. The conditions on the energy are presented in rounded boxes, so as to obtain a minimum on the HOP for both methods. One can see the counterpart conditions of the Lagrangian Multipliers method, are properties of functions needed to achieve a ordering of energies on the HOP partition strucure.

Though its important to note that the energetic lattice can be formulated for the lagrangian linear case, while the axioms of h-increasingness and scale-incresingness, generalize subadditivity and superaddivity. . . . . . .

List of Tables

2.1 This table summarizes the different components of the algorithms of different authors work, related to the study of extract of optimal cuts, the monotonicity of ordering of the cuts based on λ, the uniqueness of these solutions of a given constraint, as well as whether the multiplier is chosen or not. (a) First formulation of Dynamic Program for optimal pruning on trees (b) First formulation of multiplier dependent perturbed primal problem for any arbitrary set. (c) Guigues interprets scale-space semigroup structure as in [START_REF] Ravi | Optima on Hierarchies of Partitions[END_REF] by introducing the causality condition in the scale-sets framework. (d) The energetic lattice can be used to solve the linear Lagrangian case, while also non-linear energies which do not entail a Lagrangian multiplier method and application of Everett's theorem. . . 76

  under study R 2 or Z 2 x, y -Points of E P(E) -Set of the all subsets of E A, S, T i -Classes of E π -Partition of E, sometimes written as π(E) Π or Π(E) -Set of all partitions of E π(S) : S ∈ P(E) -Partial partition of E of support S {S} -Partial partition with unique class S D(E) -Set of all partial partitions of E -Disjoint union: S = S 1 S 2 ⇔ S 1 ∪ S 2 = S and S 1 ∩ S 2 = ∅ -Disjoint intersection: π S := π(S), π is a partition, S ⊆ E H = {π i , i ∈ I} -Hierarchy, i.e. family of increasing partitions B -Braid of partitions, monitor is characterize by partitions in B Π(E, H) -Set of all cuts of Hierarchy H that partitions E Π(E, B) -Set of all cuts of Braid B that partitions E S 0 (x) -Classes of the leaves partition π 0 (E) containing point x S i (x) -Classes of the partition π i (E) at level i, containing point x S -Family of all classes of H or B Cut π(S) -Partition of E into classes taken in S ω : D → R + -energy, i.e.non negative function on D(E) ω , ω , ω -energetic ordering, infimum, and supremum, w.r.t. energy ω Π(ω, H) -ω-energetic lattices on cuts of Hierarchy H Π(ω, B) -ω-energetic lattices on cuts of Braid B π * -Minimal cut in an ω-energetic lattice λ, µ : -Scalar parameters (e.g. Lagrangian multipliers) λ, µ : -vector of parameters (or multipliers) π * (λ) -minimal cut in the energetic lattice of energy ω(λ) π * ϕ , π * ∂ -minimal cut in the energetic lattice of energy ω ϕ , and ω ∂ ).

7 1. 1 . 1

 711 Partitions and Partial Partitions Definition 1.1. A partition π of the image domain E is a family of sets S: π = {S ⊆ E} (1.1)

Figure 1 . 1 :

 11 Figure 1.1: Left: Partial partition with support S highlighted in red, with local partitioning shown in dotted lines. Right: Partial partition refinement ordering.

i = 0 π 0 = all points of Z i = 1 π 1 =

 1 {-∞}...{-3}; {-2}; [-1, +1]; {+2}; {+3}...{+∞}. i = 2 π 2 = {-∞}...{-4}; {-3}; [-2, +2]; {+3}; {+4}...{+∞}. i = 3 π 3 = {-∞}...{-5}; {-4}; [-3, +3]; {+4}; {+5}...{+∞} ............................................... Though |I| = ∞, the number of leaves at any class S i (x), x ∈ Z remains finite as soon as the label i < ∞. Similar situations also occur in the Euclidean spaces. One can think of hierarchies where the leaves are Voronoi polygons, or polyhedra, or more generally of stationary random partitions. 10 1.1. BASIC NOTIONS AND NOTATIONS

Figure 1 . 3 :

 13 Figure 1.3: Pruning example demonstrating Cost-Complexity pruning. Figure demonstrates a tree with classification the cost function given for each node given by equation(1.9) where values for different values of λ = 0(left), 0.5(center), 1(right), are shown with their corresponding pruned optimal subtrees. The pruned nodes are presented in gray. One can also see the representation of the the cuts in blue. They represent the leaves T of the optimally pruned subtree. These cuts represent not only the pruned subtree, but also the partition of the space. It is sufficient to refer to the cut and not the whole subtree.

1

  and thus the family of subtrees T ⊆ T0 in terms of CART 2 which in case of Breiman is implicitly set by the size of a subtree constraint T .

π 2 ∨ π 3 Figure 1 . 4 :

 314 Figure 1.4: Toy example of a braid. E is partitioned by leaf nodes {a, b, c, d, e, f }.The set B 1 = {π 1 , π 2 , π 3 } forms a braid whose pairwise supremum is indicated at the right. One can note now that π 1 (X), π 2 (X) have a common parent X, but π 2 (Q), π 3 (Q) a common grand parent Q. The partition π x cannot be added to B 1 . There does not exist a supremum class, in the monitor hierarchy H, other than from the whole space E, thus not producing a braid structure.

Figure 1 . 5 :

 15 Figure 1.5: The intial color image (1) was segmented, giving the partition (2). Two zones are perhaps not correctly segmented demonstrated in (3), that we filled up by a Voronoi partitioning producing partition (4). Further a second parasitic class in (5) is removed and replaced with a Voronoi partitioning giving (6). (7) indicates the intersections of the contours of the initial segmentation (2) and the Voronoi partition in (4), and (8) is the net opening (refer to operator in chapter 4) of (7) and similarly for (9) and (10) with the second parasitic class in(5). (11) depicts the supremum of the two partitions (8) and (10). The three partitions(1,[START_REF] Arbelaez | Boundary extraction in natural images using ultrametric contour maps[END_REF] 11), with the whole space form the monitor hierarchy H of the braid made by(1,[START_REF] Ravi | Optima on Hierarchies of Partitions[END_REF] 5,[START_REF] Arbelaez | Boundary extraction in natural images using ultrametric contour maps[END_REF] 10, 11), where the (8) = sup(1, 4) and (11) is the smallest element of H larger than the sup(1,[START_REF] Angulo | Multiscale stochastic watershed for unsupervised hyperspectral image segmentation[END_REF] and also (11) = sup(8,10).

Figure 1 . 6 :

 16 Figure 1.6: From left to right: Initial gray-scale image, watershed hierarchy saliency by area attribute flooding, Watershed hierarchy saliency by volume attribute flooding, the monitor hierarchy saliency obtained by net opening over the intersection of Jordan nets, see subsection 4.8.2.

Proposition 1 . 7 .

 17 Every braid cut π ∈ Π(E, B) admits a lowest upper bound π min among the cuts Π(E, H) of the monitor hierarchy H. The classes S min (x) of π min are supports of p.p. of π.

Figure 1 . 7 :

 17 Figure 1.7: Left: and middle: details of the partitions π 1 and π 2 of the braid B, with the two classes S 1 (x) and S 2 (x) at point x. Right: S min 1 (x) and S min 2 (x) are the two lowest upper bounds of S 1 (x) and S 2 (x) in the monitor hierarchy. S min 2 (x) is the support of a partial partition of π 1 because S min 1 (x) ⊆ S min 2 (x).

  be two cuts of a braid B of monitor H. Let S min 1 (x) and S min 2 (x), the classes at x of the two associated l.u.b.. If S min 1 (x) ⊆ S min 2 (x), then S min 2 (x) is the support of a partial partition of π 1 .

Figure 1 .

 1 Figure 1.7 illustrates the lower bounding S min 1 , S min 2 classes in proposition 1.8.

Figure 1 . 8 :

 18 Figure 1.8: No single Ground truth segmentation is a refinement of the mean-shift segmentation, but their suprema are. This is a result in the inconsistent partition contours in case of both the human and machine based mean shift segmentation. Image reproduced from Unnikrishnan et al.[START_REF] Unnikrishnan | A measure for objective evaluation of image segmentation algorithms[END_REF] 115]. This helps handle boundary ambiguity.

Figure 1 . 9 : 35 πFigure 1 . 10 :

 1935110 Figure1.9: A hierarchy with multiple optimal cuts with the same energy. One requires the general condition of singularity to ensure the existence of a unique optimum. In a hierarchy, an arbitrary energy ω becomes singular when the parent picked w.r.t its children in case they have equal energies.

Figure 1 . 13 :

 113 Figure 1.13: An example of hierarchical increasingness on HOP(top) and BOP(bottom). We see that the condition of h-increasingness holds generally for any family of partial partitions.

Figure 1 . 14 :

 114 Figure 1.14: An elementary step of the dynamic program in a braid structure over a support S. The partial optimal cut in each sub-branch is shown. The final step is to compares energies ω(S), ω(π * 1 (S)), ω(π * 2 (S)), where one picks the partial partition with the least energy. Furthermore one needs to implement a consistent rule to obtain a unique solution, in other words, one needs to implement a singular energy.

Figure 1 . 15 :

 115 Figure 1.15: For the example energy demonstrated in the table, the energy of a partial partition depends on the number of its classes, by a non h-increasing rule.

  was further generalized by the seminal work of C.Couprie et al. by introducing the Powerwatershed framework [28], further regrouping the Random walker algorithm by Sinop-Grady[106], into a compact energy minimization framework, parameterized by exponents on the weights on the edges, and its coefficients. In particular when q = 2, the power watershed leads to a multi-label, scale and contrast invariant, unique global optimum obtained in practice in quasi-linear time [29].

Figure 1 .

 1 Figure 1.18: This demonstration shows an alternating composition: at each odd level we compose the energies by addition while at each even level we compose the energies by supremum. It's notable that the supremum of child energies is always smaller than or equal to their parent energies.

Definition 1 . 22 .

 122 A family {ω(λ), λ ∈ R} of energies on the partial partitions D(E) of E is scale increasing when λ ≤ µ and ω({S}, λ) ≤ ω(a, λ) ⇒ ω({S}, µ) ≤ ω(a, µ), S ∈ P(E), a {S} (1.42) These inequalities become strict when the scale increasing ω(λ) are singular energies. Theorem 1.23. Let {ω(λ), λ ∈ R} be a family of singular energies acting on a braid B = H ∪ R, and let {π * (λ), λ ∈ R} the minimal cuts of the energetic lattices {Π(ω(λ), B), λ ∈ R}. When the family {ω(λ), λ ∈ R} is scale increasing, then the least upper bound

  the vector variation reduces to a scalar one and the theorem 1.23 applies, so that we we can state: µ 49 Corollary 1.26. The relations(1.43) and(1.44) extend to the case when λ = {λ 1 , λ 2 , ...λ p } and µ = {µ 1 , µ 2 , ...µ p } are positive vectors to the Euclidean space R p .The following proposition shows how to easily construct scale increasing families: Proposition 1.27. When the map λ → ω(λ) is increasing, then the family {ω(λ)} is scale increasing.Proof. For λ ≤ µ and a ∈ Π(S, B), we have ω(λ, S) ≤ ω(µ, S) and ω(λ, a) ≤ ω(µ, a). By difference, it comes ω(λ, a)-ω(λ, S) ≤ ω(µ, a)-ω(µ, S). Hence, when ω(λ, a)-ω(µ, S) ≥ 0, then ω(µ, a) -ω(µ, S) ≥ 0, i.e. the axiom(1.42).

B 0 .

 0 Transform the braid B ∈ B, of monitor H, in the following manner: if at level i the section π(i) of the monitor H is ≤ π * (for the refinement) replace it by π * ; if π(i) π * (λ) don't change it, and in all cases keep unchanged the additional partitions R of B. We denote this operation by ϕ ω (B) = B ⊗ B * . Proposition 1.28. When the energy ω is singular, the operation ϕ ω (B) = B ⊗ B * from B into itself is a closing.

3 Figure 1 . 19 :

 3119 Figure 1.19: An example of a scale function. For each λ the classes which are hold are just above the dotted line. Note that the class on the right branch with value 1, repeated twice correspond the same same class, and a cut having one or the other are the same cuts.

familiesDefinition 1 . 30 .

 130 {ω λ }. Can we define a monotonicity property on a more general class of partial partitions, akin to the discrete concavity sense of sub-modularity [12]? An energy ω ∂ : D(E) → R + is said inf-modular when for each p.p. π of support S ∈ P(E) we have ω ∂ ({S}) ≤ {ω ∂ (a), a {S}}.(1.46)

  et al. [107, 108]. They have indicated several variants, which all rest on a same idea. A family of previous segmentations of a 2 -D function f led to hierarchy H. One requires the largest classes where function's global contrast is bounded, i.e. at each node S we have the energy ω(S) = sup{f (S)} -inf{f (S)}. The values of f (S) obviously increase monotonically on the hierarchy. A node S is kept when ω(S) ≤ 20). The minimal cut is then the union of the largest remaining nodes. see figure 1.17 where this supremum composition over ω ≤ 20, is demonstrated. One stops, climbing the hierarchy locally when the global contrast reaches the constraint 20.

  with here A ∩ B = ∅). Now we can state: Proposition 1.32. Let ω ∂ : P(E) → R + be a sub-additive energy on the sets of E. Any extension ω ∂ of ω ∂ to the partial partitions of E which satisfies the inequality (1.47) is inf-modular. Conversely, the restriction ω ∂ to sets of an inf-modular energy ω ∂ is sub-additive.

  1) where X and Y are random variables, where Y represents the coded signal and X the input signal. Here ρ(X, Y ) gives error measure, and P Y |X is the conditional probability source distribution. Then for a given channel one can define the rate I(X, Y ) and distortion E[ρ(X, Y )] which are achieved by some coding scheme (not necessarily tree structured)[26]. The rate-distortion function is the solution to the constrained minimization problem in equation 2.1. In a tree structured coding scheme, Gray et al.[26]

  separable energies. A dendrogram is depicted in Figure 2.2. The two trees shown ω ϕtree and the ω ∂ -tree represent the energies (ω ϕ (S), ω ∂ (S)), associated with each node S in the tree. The dendrogram with node is shown separately, at bottom left of the figure.To recall, by separability condition ω ϕ (and resp. ω ∂ ) of a partial partition is the sum of the ω ϕ (resp. ω ∂ ) of its classes, and when parent and child have the same energy, one chooses the parent. Guigues considers ω ϕ to be super-additive, i.e ω ϕ (S) ≤ a∈π(S) ω ϕ (a), while the constraint function ω ∂ to be sub-additive, i.e. ω ∂ (S) ≥ a∈π(S) ω ∂ (a).

Figure 2

 2 Figure 2.2 (bottom right) indicates the lambda function, which gives the value λ for which the energy ω(λ)(S) = ω ϕ (S) + λω ∂ (S) of the class S equals the energy of the partial partition of the children of S. The minimal cuts of H w.r.t. ω(λ) are thus the level sets of the lambda function. The three minimal cuts π * (λ), for λ = 1, 2, 3 are shown and denoted by π 1 , π 2 , π 3 for quick reading.

3 Figure 2

 32 Figure 2.2: Bottom Left, a hierarchy H with classes. the pairs of trees in the top row, indicate the two energies (ω ϕ , ω ∂ ) associated with the corresponding classes. π and π are two cuts of H. Bottom right, in the nodes, we depict the lambda values by equating parent and child energies, whose level sets give the minimal cuts w.r.t. the ω λ . They are depicted in the λ-tree for λ = 2, 3, 4 as π 2 , π 3 , π 4 . The λ values for the leaves are assumed to be 0, though in case of Breiman et al. [21] λ for the leaf classes are set to ∞ to avoid over-fitting.

Figure 2 . 3 :

 23 Figure 2.3: For 2 < λ < 3 the minimal cut is (a, b,c, d, i) and ω ∂ = 8, for λ ≥ 3 the minimal cut is (g, h, i) and ω ∂ = 6, i.e. ω ∂ is never equal to the cost C = 7.5 at any time.

  range of values possible compared to the number of classes as constraint, i.e. 1, ...N , N being the number of leaves. The perturbed scale function Λ (S) in such a case improves the constraint value → Multiplier mapping (C → λ), by finding cuts that are between a feasible upper-bounding λ-cut and a limiting infeasible λ-cut. The algorithm and results will presented in detail in the applications chapter 3.

Figure 2 . 4 :

 24 Figure 2.4: Demonstrating the feasible space in the hierarchy with the family of Ccuts which are the set of partitions that satisfy a constraint of ω ∂ (π) ≤ C, which here for demonstration are cuts with number of classes no greater than 6.

and 73 2. 5 .

 735 Guigues removes them by scale-climbing, which uses a similar dynamic program but on the scale function. Classes with non-monotonically ordered scale function values are termed by Guigues as Anti-causal Classes. Though this problem purely depends on the relation between the objective and constraint function. One can now resolve this ordering problem by changing the constraint function to a penalty function, such that: ω(π(S), λ) = a∈π(S) (ω ϕ (a)) α + λ a∈π(S) ω ∂ (a) (2.20)where α is a global parameter for all classes. For the toy example in figure2.2 it produces the following λ's for a range of values of α, which are calculated by the scale function equation:Λ(S, α) = ω ϕ (S) -( a∈π(S) ω ϕ (a) α ) 1 α a∈π(S) ω ∂ (a) -ω ∂ (S) (2.21) λ-values for classes (E, j, g/h, i)Causality on the refinement of λ-cuts is the parameter α in equation (2.20). As already seen in the study of composition of h-increasing energies in section 1.6. This α parameterized THE ENERGIES ω λ = ω ϕ + λω ∂ penalty energy is h-increasing as already shown in proposition 1.19, to enable one to control the choice of parent/children more finely, than just by supremum, infimum, sum or number of classes.Increasing Scale-Function range: In overview from the two methods above we can conclude that the greater the range of values of perturbed λ, the higher the chance of converging at a better optimal cut meeting constraining ω ∂ (π) ≤ C. The λ-perturbation and the penalty function are ways of inspecting λ-cuts that are finer. While one can note here that the penalty function method provides cuts that are in between the ones address by scale-openings and closings in subsection 1.7.3.2.5 The energies ωλ = ω ϕ + λω ∂Here we present the critical result of constraint minimization over braids by their characteristic energetic lattices. The first chapter dealt with unconstrained minimization of any model of energy on partial partitions. We now will focus on the Lagrangian energy ω λ = ω ϕ + λω ∂ , whose unconstrained minimization leads to the solution of the original constrained minimization.Lagrange Multipliers for constrained optimization on HOP: When dealing with the cuts Π(E, B) of a braid B, can we find a cut which minimizes ω ϕ on Π(E, B) under some energy constraint ω ∂ ? Since Lagrange's starting points seems unrealistic, let's consider his arrival point, which

Proposition 2 . 8 .

 28 and the set of constraint ω ∂i , 1 ≤ i ≤ p replaces the constraint ω ∂ . Note that the π * (λ) are the only minimal cuts used below (the minimal cuts in the lattices Π(ω ϕ ) and Π(ω ∂ ) play no role). Definition 2.7. One calls "scalar Lagrange family" of energies any family {ω(λ) = ω ϕ + λω ∂ , λ ∈ R} where ω(λ), ω ϕ , and ω ∂ are singular, and where ω ∂ is inf-modular.Similarly a "vector Lagrange family" of energies is a family {ω(λ) = ω ϕ + λ i ω ∂i }where the λ i are scalar, and where ω(λ) ω ϕ , and ω ∂i are singular, and the ω ∂i are inf-modular. 2.7. MINIMAL λ-CUTS AND ENERGETIC LATTICES Given a braid B, we already know that a scalar Lagrange family provides a unique minimal cut π * (λ) of B with each λ, since ω(λ) is singular. Moreover the inf-modularity of ω ∂ shows that the lower upper bound of these minimal cuts π * (λ) enlarge as λ increases (Proposition 1.31 and Theorem 1.23). These first results are improved by the following proposition: Let {ω(λ) = ω ϕ +λω ∂ }, be a scalar Lagrange family of energies on the partial partitions of a space E, and suppose λ > 0. Given a braid B on E, let Π(ω(λ)), Π(ω ϕ ) and Π(ω ∂ ) be the energetic lattices of the cuts π of B w.r.t. the energies ω(λ), ω ϕ , and ω ∂ respectively. The minimal element of Π(ω(λ)) is denoted by π * (λ). For all λ, µ ≥ 0 the two implications 0 ≤ λ ≤ µ ⇒ π * (λ) ω ∂ π * (µ) and π * (λ) ωϕ π * (µ) (2.23) are true, i.e. as λ increases, the sequence {π * (λ), λ > 0} of the minimal cuts w.r.t. the Π(ω(λ)) decreases in the energetic lattice Π(ω ∂ ) and increases in the energetic lattice Π(ω ϕ ). Proof. If λ = µ, the relation 2.23 is obviously true. Suppose λ < µ. Consider the class S µ of the minimal cut π * (µ) at point x and the associated l.u.b. S max (x | µ). The restriction π * (µ) S max (x | µ) of π * (µ) to S max (x | µ) is the p.p. a µ . As ω ∂ is inf-modular, the energy ω(λ) is scale increasing. Then, according to Proposition 1.8, S min (x | µ) is the support of a p.p. a λ of π * (λ). As π * (µ) is the minimal cut for the energetic ordering ωµ , we have ω µ (a λ ) ≥ ω µ (a µ ), i.e.

79 Π

 79 2.28) Now the domain of the feasible cuts is the subset Π of Π = {π, π ∈ Π, ω ∂ (π) ≤ C} (2.29) In the Lagrangian energetic lattice Π ω (λ) corresponding to Lagrangian ω(λ) = ω ϕ +λω ∂ , the minimal cut π * (λ) has an energy ω(π * (λ)) which is itself minimal (h-increasingness of ω(λ)). The energy ω(π * (λ)) turns out to be the dual Lagrangian, g(λ). The energy ω(π * (λ)) is a function of λ, ω ϕ and ω ∂ , but not of the cuts π ∈ Π(E, B). The dual problem can now be stated: LME Dual problem: Given a braid B find the parameter λ which maximizes ω(π * (λ)), subject to the constraint λ > 0.

Theorem 2 .

 2 11. (Braid minimization by energy, scalar case). Given a braid B on E, let {ω(λ) = ω ϕ + λω ∂ , λ ∈ R} be a scalar Lagrange family of energies where in addition ω ϕ , ω ∂ and ω(λ) are h-increasing. And let λ * = inf{λ | ω ∂ (π * (λ)) ≤ 0}. Now If we have, (i) the set Π is not empty, (ii) λ * exists and is ≥ 0,

Problem 3 .

 3 LMCC Primal problem: Find the minimal cut π ϕ in the Energetic lattice Π(ω ϕ ) which is constrained by a cut π C minimize π∈Π(ωϕ)

4 .

 4 LMCC Dual problem: Find the parameter λ which maximizes the cut π * (λ) in the energetic lattice Π(ω ∂ ):maximize π * (λ) ∈ Π(ω ∂ ) subject to λ ≥ 0 (2.41)To solve both problems (2.40, 2.41) jointly, the following three conditions are required: Theorem 2.13. Given a braid B on E, let {ω(λ) = ω ϕ + λω ∂ , λ ∈ R} be a scalar Lagrange family of energies where in addition ω ϕ , ω ∂ , and ω(λ) are h-increasing. Letλ * = inf{λ | π * (λ) ω ∂ π C }. If (i) Primal feasibility: the set Π C = {π | π ∈ Π(E, B), π ω ∂ π C } is non-empty,(ii) Dual feasibility: λ ≥ 0 (iii) Lattice Assumption: π ϕ ϕ π * (λ * ) then the set of feasible multipliers are λ ≥ λ * , and π * (λ * ) and λ * are the unique solutions to the problems 2.40 and 2.41 respectively. Proof. We first prove that the feasible set of multipliers are λ ≥ λ * . If λ < λ * , then π * (λ) does not belong to the space Π C of solutions π ϕ . The class S(λ) of π * (λ) at leaf x, contains one or more classes of π * (λ * ) which form a partial partition a(λ * ). By proposition 2.8 ω ∂ (a(λ * )) ≤ ω ∂ [S]. Moreover, as λ varies there is a finite number of different sets S (axiom (ii) of the braid definition 1.18), so that ω ∂ [a(λ)] is necessarily one of the ω ∂ (S), hence π * (λ * ) ∂ π C . Therefore π * (λ * ) ∈ Π C which implies π ϕ ϕ π * (λ * ), and by assumption (iii) π ϕ = π * (λ * ). The minimal cut (in the Energetic-Lattice Π(ω(λ * )) is the solution of the primal LMCC problem 2.40, and the solution is unique since π ϕ must be the minimal element of a lattice. In the ω ∂ -energetic lattice Π(ω ∂ ) we have λ ≥ λ * implies π * (λ) ∂ π * (λ * ) = π ϕ , hence ∂ {π * (λ), λ ≥ λ * } ∂ π * (λ * ), and finally π * (λ * ) = ∂ {π * (λ), λ ≥ λ * }, which solves the LMCC dual problem 2.41.

Figure 2 . 5 :

 25 Figure 2.5: Minimal CCO cut. The + classes have ω ∂ > C, the -classes have ω ∂ ≤ C.

91 2

 91 .11. PRIMAL VS DUAL: C-CUTS VS λ-CUTSWe have up until now, seen the following types of optimization on BOP/HOP: 1. unconstrained optimization of ω(π), π ∈ Π(B, E) using dynamic program 2. constrained optimization of ω ϕ (π) subject to ω ∂ (π), for π ∈ Π(B, E), using dynamic program on the space multipliers λ. (DP on Dual Problem) both using the energetic lattice structure to ensure an optimum. Now we will consider the problem (2) above in the primal domain, and establish why it is easier consider the dual domain.These are the family of cuts in the braid whose constraint function has a value smaller than C. One thing that can be directly noted about these partitions are that they are in any sense monotonically ordered w.r.t C. Furthermore we can see the nature of the energetic lattice unravel itself with this simple parametrized family of cuts. This is demonstrated in figure 2.4.

•

  [59] Global-local optimizations by hierarchical cuts and climbing energies, Pattern Recognition(PR) 2014. • [103] Optima on Hierarchies of Partitions, ISMM 2013. • [61] Ground truth energies for hierarchies of segmentations, ISMM 2013. This chapter demonstrates the applications of dynamic program for energetic lattices, involving different compositions of energies. A specfic contribution and application is the formulation of proximal partition extraction. This consists in extraction of a partition,

  variation in an image, example the Snakes or Chan et al.s Active contours [24, 57]. The energies we view are particular cases of active contours adapted to hierarchies, and derive from the approach proposed by Y. Xu at Al. [124]. The main idea is the following: each node S ∈ H is dilated and eroded by a disc B, and the two crowns S ⊕ B\S, and S\ S B are compared. This comparison stands for the fidelity term ω ϕ in Rel. (3.2), and a function of the curvature (e.g. Rel. (3.4)) stands for the regularity term. One goes from sets to partial partitions by additivity, according to the relation (1.31).

  (r, g, b) into two components: the vector luminance -→ l = (l, l, l) which gives the gray scale, plus the orthogonal chrominance vector -→ c = (r -m, g -m, b -m) = (c 1 , c 2 , c 3 ) whose module is the saturation.The fidelity term of the energy is now the sum of the variances of the components of -→ c over S as shown in (3.7).

Figure 3 . 1 :

 31 Figure 3.1: First row: Initial image and Saliency function corresponding to hierarchical watershed flooding [32]. Second Row: Optimal cuts by using variance of luminance(left), chrominance(right).

Figure 3 . 2 :

 32 Figure 3.2: Optimal Cuts for texture using variance of chrominance for scale λ = 100: Left, input Image, middle and right, cuts for parameters at µ = 10 12 (low uniformity) and µ = 10 14 (high uniformity), in Eq. 3.8.

Figure 3 . 3 :

 33 Figure 3.3: The first row shows an input image, with two ground truth partitions corresponding to the image, from the Berkeley dataset. The bottom row consists of a sequence of thresholds of the Ultrametric Contour Map(UCM) segmentation hierarchy.The problem now consists in extracting a proximal partition in the hierarchy that is closest to one of these ground truth partitions. Further how do we compose when we have multiple ground truth partitions, and how do we compare hierarchies w.r.t a single ground truth.

4 .

 4 Braids: When the same support is segmented in a non-nested structure. The supremum of the two partial partitions have the same support, in other words the same monitor. This difference in local segmentation can result, when we have textured regions or smooth zones which are differently segmented by human experts. This has been observed in the evaluation of segmentation algorithms byUnnikrishnan et al. [115]

Figure 3 . 4 :

 34 Figure 3.4: Example demonstrating two partitions π 1 and π 2 , where one of them could be a ground truth segmentation while the other being a machine segmentation. Figure demonstrates the different possible configurations of refinement, braid and overlaps of classes.

.12) 4 .

 4 Boundary based measures: D.Martin thesis [70] provides a variety of Ground truth based evaluation measures. Using the boundary of segmentations, the unitary element: Edgel, where he performs edgel correspondence between the segmentations.

Figure 3 . 5 :

 35 Figure3.5: GT's corresponding to an input image. This demonstrates how the human experts in this case have drawn different scales of details in the scene. All scales are not present in no single GT partition. This as described well across literature is due to the ill-posedness of the segmentation problem. The OIS averages the results of choosing the right scale of partition from the UCM across various GTs to evaluate the segmentation hierarchy. We will use instead an inf-composition to extract the proximal partition. We remark the inherent braid structure in such cases.

d. 15 ) 105 Figure 3 . 6 :

 1510536 Figure 3.6: Haussdorff distance Assymetry: Smallest disc dilation of X that contains Y is drastically larger than theat of X to contain Y, thanks to a the difference in convexities of the shape. The same situations occur when dealing with classes of a partition.

  .20) By taking the supremum of all ω G (S) we find the above value ρ G of Rel.(3.16):ρ G = {ω G (S), S π A }. (3.21)This shows the soundness of ω G . But a problem arises when we want to extend it from sets to the partial partitions D(E) of E by some law of composition between the T i . When the chosen energy is h-increasing, which will always be the case here, finding optimal cuts in hierarchies amounts to compare the partition energies of parents and children [59]. If we compose the energies of the children by supremum, then we trivially always find ω G (π) = ω G (S), the parent. If we compose by infimum, we have ω G (π) = ω G (S) when the ω G (T i ) all identical, and ω G (π) < ω G (S) when not. And if we compose the energies of the sons by averaging, we obtain again ω G (π) < ω G (S).

Figure 3 . 7 :

 37 Figure 3.7: Energies ω G (S), θ G (S) for each class S in a segmentation, defined w.r.t class from a ground truth partitionin red. The composition of these energies decid the local distance measure introduced and minimized.

( 3 .

 3 20), as we did before with the global Hausdorff distance. Such a dual energy would beω G (S) = inf{ρ | S ⊕ ρB ⊇ G},(3.23)

Figure 3 . 8 :

 38 Figure 3.8: Row 1: ω GT 2 (S), Row 2: θ GT 2 (S), Row 3: ω GT 7 (S), Row 4: θ GT 7 (S).Figure shows the different half haussdorf proximity functions ω(S) and θ(S) for each class from different partitions in a hierarchy. The two ground truhs chose are of different scales. The gray scale value 0 corresponds to closest while 255 corresponds to farthest. Ground truth and associated distance function on left, energy values over 6 different partitions from the hierarchy on its right. One can already get a quick idea of what the dynamic program would extract an a minimal cut looking at the individual values. One can see that the scale of the ground truth partition affects the energy associated with classes of the hierarchy of segmentatations. What's left is to obtain a good composition.

Proposition 3 . 1 .

 31 .24) so that the part of G exterior to S is not significant. For the sake of comparison, it thus suffices to focus only on the distances involved in the covering of G ∩ S by dilations of ∂S on the one hand, and on those of ∂T i on the other hand. Then the energy ω G of Rel.(3.23) has to be replaced by the more appropriate oneθ G (S) = inf{ρ | S ⊕ ρB ⊇ G ∩ S}.(3.25)When S spans all classes of a partition π A , then the supremum of all θ G (S) gives the value ρ A of Rel.(3.17)ρ A = {θ G (S), S π}, (3.26) and the (global) Hausdorff distance ρ between π and G turns out to be the double supremum, ρ = {{ω G (S) {θ G (S)}, S π}. (3.27) It remains to verify that θ G is h-increasing. Given a ground truth set G, the extension of the energy θ G of Rel.(3.25) to partial partitions by ∨ composition is h-increasing.

  .29) which achieves the proof.Note that when G ∩ S = ∅, then θ G (S) = K max , which is a large penalty set as a factor of the number of pixels in the input image segmentation.The energies are demonstrated in figure 3.7. to summarize ω G (S) gives the largest radius of dilation of the ground truth set, so as to cover the contours of class S, from any set of contours of the ground truth, while the energy θ G (S) gives the largest radius of dilation of contour of class S so as to cover contours of the ground truth parition covered by the class S.

Figure 3 . 9 :

 39 Figure 3.9: Ground truth partitions, and corresponding optimal cuts, for energies ω G , θ G and for composition by sum ω G + θ G . The input hierarchy is the UCM from the Berkeley dataset, consisting of 800 level binary partition tree.

Figure 3 .

 3 Figure 3.10: Two ground truths and their union are shown, with their corresponding optimal cuts, for the energy θ G + ω G . The composition over different ground truth sets is achieved by infimum as shown.

Figure 3 .

 3 Figure 3.11: a) Leaves partition b), c) and d) Conditional λ-cuts for λ = 0, 10, & 80.

Following from the local

  measures in(3.31) which are integrals of the distance function associated with each class, we propose here a global similarity measures for a hierarchy.Two global measures of precision and recall for a given hierarchy H of segmentations with respect to an input ground truth partition G. The measure now is not between 2 partitions any more and deals with the global similarity between hierarchies of partitions H and a single partition G. The representative functions we are going to use for the global measures are: s the saliency and g the distance function, the set S i saliency map 115 threshold at an index i.

  be a finite set of Jordan curves partitioning R 2 . Further define a lattice of such Jordan 119 curves which when associated with a numerical function produces a new ordering and thus a hierarchy. We treat the problem of ground truth evaluation of hierarchies of segmentations. More precisely we use the ground truth distance function to order contours in the hierarchy reflecting the proximity to ground truth. We further demonstrate with other point-wise function to reorder contours, resulting in a new transformed saliency function.First in section 4.3 we describe the necessary topological framework, and explicit the lattice of Jordan curves. Section 4.3 gives the theory, Section 4.5 and 4.6 describes a fusion between the saliency and external functions, which reorders the initial hierarchy. Section 4.7 demonstrates such a reordering based on the proximity to a ground truth partition. Then the composition of two hierarchies is obtained by using a distance between them. Following which one studies a measure that demonstrates how the distance function produces structural changes in the transformed or reordered hierarchy. Saliency function: a numerical representation of partition hierarchy: We revisit the saliency function, first introduced as a representation of the hierarchy. They have been popularized as Ultrametric Contour Map(UCM) [10] The hierarchy and thus its saliency function, can be generated in many ways, most frequently by watershed of floodings [10, 32, 79], or again they can be regular hierarchies such as, quad-trees, oct-trees, all of which can be represented by the saliency function.

Figure 4 . 1 :

 41 Figure 4.1: Saliency Function example: First image is an input image from Berkeley data base. Second image is the corresponding Ultrametric Contour Map(UCM) which represents a hierarchy of partitions. Third image shows the corresponding altitude map for the UCM. Here we see that each arc in the saliency function separates different components at different altitudes. This third image is oriented 180 degrees off to better render the details.

Figure 4 . 2 :

 42 Figure 4.2: Examples of Jordan Curves. The Jordan curve tessellates the space, with one bounded interior component or face F and an unbounded exterior component S

Theorem 4 . 1 .

 41 A Jordan curve C in R 2 is the image of an injective (i.e. without selfintersection points) continuous map of the unit circle into the plane. According to a famous theorem due to C. Jordan, the complement R 2 \ C consists of exactly two open connected components, the first one, called face F is bounded, and the other, called background S, which is not. They are homeomorphic to the inside and the outside of a disk respectively. One classically calls a tessellation any partitioning of a topological space into open classes, and classes formed by their frontiers [101]. The partition of R 2 into three classes {C, F, S} by a Jordan curve is thus a tessellation of R 2 .

Figure 4 . 3 :

 43 Figure 4.3: One can suppress the contour in the interior of region F 1 , while no increasing the energy ω(π) of the partition by F 1 , F 2 .

Proposition 4 . 4 .

 44 The set N (N 0 ) of all J-nets included in the base N 0 forms a lattice with N 0 and ∅ as universal bounds. The supremum of N, N ∈ N (N 0 ) is the union N ∪ N , and the infimum is the union of all Jordan curves common to N and N (empty set included).

Figure 4 . 4 :Figure 4 . 5 :

 4445 Figure 4.4: An elementary Jordan net N 0 , A set X of arcs and Jordan curves, and their net openings γ(X). It is important to note that the number of connected components of white pixels don't change after a net opening |γ(X)| CC = |X| CC . γ(X) removes two types of arcs: open arcs and arcs which are not normal [64].

Figure 4 . 6 :

 46 Figure 4.6: Initial Image 239096 from Berkeley database, Saliency function s: Ultrametric contour map(UCM), Ground truth partition G, Inverted distance function g. The inputs here we consider are the saliency function s and the inverted distance function g.

127 4 . 4 .

 44 WATERSHED TRANSFORMATION AND SALIENCY FUNCTION

Figure 4 . 7 :

 47 Figure 4.7: The figure demonstrates how the image I produces a meaningful gradient function in the xy directions ∇ x I ∪ ∇ x I, using the image values. The label image L on the other hand does not depend on the actual values in the gradient (∇ x L depends on the labeling L which can be arbitary) but just the existence of the non-zero gradient value. This can be seen as a indicator function of the gradient function 1(∇L).

Figure 4 . 8 :

 48 Figure 4.8: Input function g on a simple toy Jordan net, The net opening γ(g) and their level sets.

Figure 4 . 9 :

 49 Figure 4.9: Complement of input function g = max(g) + 1 -g, Intermediate resultshowing the opening of inverted function γ(g ), and finally for the pseudo closing we use another iteration of the net opening: φ(g) = γ(max(γ(g ) + 1 -γ(g ), with level sets of g and φ(g). The opening of the complement is a closing in the space of arcs but not that of the Jordan nets.

Theorem 4 .

 4 [START_REF] Angulo | Multiscale stochastic watershed for unsupervised hyperspectral image segmentation[END_REF] allows us to combine a saliency s with a function g. The following nice properties are indeed direct consequences of this theorem: 133

Figure 4 .

 4 Figure 4.10: Lack of upper bounds by closing.

Figure 4 .

 4 Figure 4.11: Dual Closing: The thickening τ is not increasing. g and g are binaryor two level saliency functions. We have τ (g) ≥ τ (g ) though g ≤ g .

Figure 4 .

 4 Figure 4.12: a) filter γϕ(g), i.e. the closest saliency below ϕ(g), b) thickening τ (g).

4. 7 . 1

 71 Net Opening by up-sampling and down-sampling

Figure 4 .

 4 Figure 4.13a) represents the leaves partition whose components are labeled 2,3, and 4. This partition is up-sampled by 2 according a dilation by a 2 × 2 square. This results in Figure 4.13b). The basic J-net separating the leaves is obtained by the support of the gradient of image 4.13b) (still with the same origin), and is depicted in Figure 4.13c).This appears only in the even rows and columns. Introduce now a weighting function g. Figure4.13d) depicts the g weights on the gradient support. The cross section X t (g) for t = 2 is indicated in Figure4.13e). It opens a breach between the label 2 and 4 which merge in a unique component, of label 4 in Figure4.13f). By down-sampling, i.e. by removing even rows and columns in Fig.4.13f), we return to initial space in Figure4.13g), where the two labels 4 and 2 are clustered into a single label, which is exactly the level t = 2 of the hierarchy. This results in the closing of classes as defined in corollary 4.5.

Figure 4 . 13 :

 413 Figure 4.13: Toy example showing down-sampling and the different stages of operation to achieve the net opening. One important different w.r.t[START_REF] Najman | On the equivalence between hierarchical segmentations and ultrametric watersheds[END_REF] is that here we have boundary operator that simply calculates the non-zero gradient contours, since the labeling of the components does not produce an ordering dependent on the gradient of the image, and furthermore can be arbitrary. This double resolution to separate cells in discrete topology is the well known Khalimsky topology[START_REF] Melin | Digital khalimsky manifolds[END_REF], further one can find the Khalimsky's digital jordan curve theorem in by Kiselman[START_REF] Kiselman | Digital jordan curve theorems[END_REF].

Figure 4 . 14 :

 414 Figure 4.14: Inverse distance function g GT 4 = 1 -d 4 (GT 4), Transformed saliency γ(s + g GT 4 ), Point distance function g point , Transformed saliency γ(s + g point ).

  inputs given to us are the saliency function s representing the initial hierarchy H and the ground truth set of edges G. They are shown in figure 4.15. Here we use the 4-connected distance function of ground truth d 4 , to define the inverse distance function 137

Figure 4 . 15 :

 415 Figure 4.15: Hierarchy fusion: Here we fuse two hierarchical structures by introducing a distance function d which is unique(given its leaves) for every saliency function s.

Figure 4 .

 4 Figure 4.14 summarizes the input inverse distance function g and resulting saliency functions. The input saliency is shown for input image 239096 from the Berkeley database, already shown in figure 4.6. The ground truth G is more or less representative of the image structure in the saliency s, and thus the resulting transformed saliency s G = γ(s+g) is not too different, except that in general edges very far from the ground truth are reduced or weakened, while the ones in close proximity are reinforced (see Fig 4.14). An additional example with a point ground truth is used to demonstrate with a function which has nothing to do with the image structure. The choice of the distance function affects the partitions selected as seen in the toy example in Figure 4.16, the diamond is extracted before the circle in case of city-block distance, while the Euclidean distance extracts both at the same scale.

  this specific example we show how to combine hierarchy of segmentations from images having two very different image structures. In figure 4.15 we present the images of a flower in Image 1 and Corn in Image 2 portraying different textures. The hierarchies are represented by their saliencies s 1 and s 2 .

Figure 4 .

 4 Figure 4.16: Scaling space with different distance functions: Here we demonstrate how the inner diamond and the circle get separated when using the (4-connection d 4 ) city-block distance function, while the euclidean (d ball ) distance function reaches them at the same scale. This produces two different saliency functions and thus hierarchies: γ(d 4 + π 0 ) (with 2 levels) and γ(d ball + π 0 ) (with single level).

  net) of another hierarchy to obtain a composition of the two hierarchies. It enters as operand in the net opening γ, which gives the modified saliency γ(d Σ (s 1 ) + s 2 ) of the Image 4.15. By inverting the roles of s 1 and s 2 , we find similarly the modified saliency γ(d Σ (s 2 ) + s 1 ) Both are depicted in Figure 4.15. Thus here we are able to see in s 12 the closest set of partitions in the flower saliency function s 1 to the texture partitions from the corn saliency s 2 . We can scan the different scales by weighting by a constant K the distance function d (s), i.e. s K 12 = γ(K • d (s 2 ) + s 1 ). Here we have demonstrated the crossed saliencies for K = 10, though to find interesting scalings is another problem. 139 4.7.4 Composition by ∧: Hausdorff distance Ordered saliencies Until this section we have described how the inverted distance function g = 1 -d of Ground truth partition G was used to reorder the Jordan curves in a hierarchy of segmentations H to produce a new hierarchy of partitions ordered by the proximity to the ground truth. Though this operation in the Hausdorff sense has only measured the distance from the ground truth to the partition contour. In this section we briefly demonstrate an example on how to calculate the Hausdorff distance reordering the hierarchy of partitions by going from the hierarchy H to the ground truth partition G.

Figure 4 .

 4 Figure 4.17: Toy example demonstrating Hausdorff distance ordering. First case with blue circle demonstrates a symmetrically aligned pair circular contours(black, blue), the second case demonstrates an asymmetrically aligned pair of circular contours(black, red). Aside the arrows we calculate the infimum of radius of dilation for one contour to cover completely, the other, for example, the radius of dilation of set in blue to cover set in black is 1, while radius of dilation for set red to cover set in black is 2. The third figure demonstrates how the two circles are reordered, by associating them with the inverse of the Hausdorff distance between the circles.

4. 7 . 4 . 1

 741 Partition Asymmetry and Hausdorff distanceWhen the ground truth partition G and the base Jordan Net N 0 , are refinements of each other, i.e. G ⊆ N 0 or, N 0 ⊆ G, we demonstrate here that the Hausdorff distance between their contours, in this case provide a measure of asymmetry between the partitions.Consider the simple example of concentric and asymmetrically placed circles, figure4.17. The Hausdorff distance in case of concentric circles case is the supremum of two equal distances, that correspond to the difference in radii. This is demonstrated by the blue circle. While in case of an asymmetrically placed red circle, the distance from the black→red = 0.3 red→black = 2, is shown. The Hausdorff distance based saliency is calculated using equation 4.9.The Hausdorff distance function ordered saliency function thus basically produces high saliency value for partitions which are symmetrically placed w.r.t the contours of the ground truth set. Symmetry of object contours itself can be used to extract salient object contours[START_REF] Stahl | Globally optimal grouping for symmetric closed boundaries by combining boundary and region information[END_REF]. But here the Hausdorff distance ordering provides different partition contours in base Jordan net N 0 that are symmetrically placed w.r.t the contours of the ground truth partition contours. We demonstrate with different refinements of ground truth in figure4.[START_REF] Beucher | hierarchical segmentation and waterfall algorithm[END_REF]

Figure 4 . 18 :

 418 Figure 4.18: This example demonstrates 3 scales of ground truths and the corresponding, Hausdorff distance ordered saliency functions. The base Jordan net is extracted from the leaves/finest level of the input saliency function. Partitions corresponding for each ground truth at a threshold(level) of 350 is shown.

Figure 4 . 19 :

 419 Figure 4.19:This toy example demonstrates a partition N 0 with three ground truths partitions. Two of the three ground truth lines overlap and is represented in red, while the single ground truth in blue. In such a case one produces the saliency function weights as seen. This is due the composition by addition that weights a partition contour if its proximal to larger number of ground truths (here overlapping, but in general can span space at different locations) than if it is simply close to a single ground truth.This also in contrast produces a different ordering compared to the ∧-compositions.

Figure 4 . 20 :

 420 Figure 4.20: Initial image with (a,b,c,d,e) representing five different ground truths, with images (g,h,i,j,k) corresponding inverted distance functions of ground truths. While (f) shows the sum of ground truths, and (l) its inverted distance function. We see different contours of the lizard in the image that are reinforced. Further more the ground truth partitions in this case are not simple refinements, and thus validating our use of a composition by addition. Corresponding net openings are demonstrated in 4.21

Figure 4 . 21 :

 421 Figure 4.21: Figure shows initial leaves partition N 0 with three ground truths partitions, and the different net openings possible.The last saliency demonstrates the composition by addition that weights, where higher weight is given to a partition contour, if its proximal to larger number of ground truths (here overlapping, but in general can span space at different locations) than if it is simply close to a single ground truth. Composition by addition (eqn 4.13) also in contrast produces a different ordering compared to the ∧-compositions (eqn) 4.12). Please refer to figure4.22 to view the different scales that can be extracted using the M-measure from equation(4.14).

Figure 4 . 22 :

 422 Figure 4.22: The plot demonstrates the M measure for the saliencies in figure 4.21 generated with different inverse distance functions of ground truth(in figure 4.20), saliency by Inf-composition and composition by addition. As we can see the inf and sum composition form the bounds of variation of the M -measure which not only provides a structural measure of how many children are regrouped by the parent level in each hierarchy but also the number of levels in the hierarchy.

Figure 4 . 23 :

 423 Figure 4.23: Evaluating with M : The first row shows 3 different distance functions, d GT 2 , d GT 4 , d pt with their corresponding ground truths on the top right corner of the images. π 0 is the leaves partition on which the distance functions will reorder the arcs. In the bottom row we have the transformations γ(π + d GT 2 ), γ(π + d GT 4 ), γ(π + d pt ) representing new the hierarchies(saliency functions). The plot on the bottom right displays the M measure at different levels of the hierarchy. The maximum number of levels in these hierarchies is bounded by the maximum value of the distance function producing a partition.

Figure 4 .

 4 Figure 4.24: Net opening over the intersection of two finite Jordan nets N, M . This example demonstrates the use of the net opening operator to extract from the intersection of two Jordan nets (which is not necessarily a set of Jordan curves) the largest Jordan net in the base net N 0 . Since we work on pixel domains, one can consider the finest net N 0 to be the one separating all pixels. This operator will be used to create braids while recomposing partitions from different hierarchies. This a very simple operator, and it helps in combining partitions which don't share a common leaf partition or Jordan net.

Figure 4 .

 4 26, shows the finest partition in the monitoring hierarchy. When we stagger two hierarchies of different number of levels, and perform the top-down algorith 5 we can obtain different scales of π min partitions, and thus different monitor hierarchies, but the braid generated by starting top down on all hierarchies produces the largest monitor.

Figure 4 .

 4 Figure 4.25: Calcuating the monitor partition of two partitions using the net opening. The partitions (a), (b) are extracted from watershed flooding by attributes of area and volume respectively [32]. One can note that the watershed by different attributes are not hierarchical. (c) is the intersection of contours between the two partitions, (d) gives the net opening of the intersection, resulting in the monitor partition. (e) shows a magnified view of the contour in the intersection set, where the area and volume floodings have small difference, resulting in a fissure in the intersection set. This leads to a loss of a large class, in the corresponding net opening. We calcuate the monitor of the braid formed in such an event demonstrated in figure 1.6.

Figure 4 .

 4 Figure 4.26: Figure shows, Leaves partition π min 1 in watershed saliency by area, π min 2 , corresponding partition from watershed saliency by volume. And finally π min as described in figure 1.7 in chapter 1, is the net opening of the intersection π min 1 ∩ π min 2 . The π min of the braid and finest partition picked from watershed saliency π min 2 by volume are the same, thought this might not be the case generally.

Algorithm 1 :

 1 DP on HOP: Optimal-Cut(H, λ, ω ϕ (S), ω ∂ (S) Data: H, λ, ω φ (S), ω ∂ (S).Result: Optimal cut π * , ω ϕ (π * ), ω ∂ (π * ) begin N ←-|H| # levels F (S) ←-0, S ∈ H Set all flags to zero for level ∈ [2, N ] do for S ∈ H(level)] do ω(S) ←-ω ϕ (S) + λω ∂ (S) ω(π(S)) ←-ComposeF unc(ω ϕ (π(S)), ω ∂ (π(S)), λ) if (ω(S) ≤ ω(π(S))) then F (S) = 1 Optimal Parent else ∀T i ∈ π(S), F (T i ) ←-1 Optimal Child end end end end for level ∈ [2, N ] do for S ∈ H(level)] do if F (S)then π * ←-π * ∪ S add optimal classes to cut. ω ϕ (π * ) = ω ϕ (π * ) + ω ϕ (S) ω ∂ (π * ) = ω ∂ (π * ) + ω ∂ (S) end end end end

beginN

  ←-|H| # Levels∀S ∈ H, AC(S) ←-0 Set all Anti-causal flags to 0 ∀S ∈ H(1), Λ(S), ←-0 Set Λ(S) to 0 for leaves levelfor level ∈ [2, N ] do for S ∈ H(level) do if |π(S)| > 1 then Λ(S) ←-T ∈π(S) ωϕ(T )-ωϕ(S) ω ∂ (S)-T ∈π(S) ω ∂ (T ) AC(S) = Λ(S) < {∨Λ(T i ), T i ∈ π(S)}?1, 0 else Λ(S) ←-Λ(π(S)) repeat Λ when parent has single child AC(S) = AC(π(S)) Repeat flag end end end end

Algorithm 3 :

 3 Hierarchy of λ-cuts H * . Data: H, ω ϕ (S), ω ∂ (S). Result: Optimal Cut Hierarchy H * begin i ←-1 indexing starts at 1 Λ(S) ←-Λ-Func(H, ω ϕ (S), ω ∂ (S)) N ←-# Parents in H π * (λ) ←-OptimalCut(H, Λ(1), ω ϕ (S), ω ∂ (S) π * (λ) is Finest Optimal Cut while |π *

Figure 5 . 1 :

 51 Figure 5.1: For the given graph, the maximal independent set of vertices are {{a, c}, {b, e}, {d, f }}. Addition of any other vertex would lead to an inclusion of an edge between the MIS vertex set. This characterizes its maximal nature.

Figure 5 .

 5 Figure 5.1 shows an example of a MIS. Now one can further define a weighted counterpart of the MIS: Maximally weighted independent set(MWIS). The weight of the independent set I is defined to be the sum of the weights of vertices in the set: ω(I) = S∈I ω(S).

Definition 5 . 3 . 1 . 3 .

 5313 (Partition Graph) Let S = {S 1 , S 2 , ..., S n } be a family of distinct, nonempty subsets of some universal set U . Then intersection graph G for the family S, if For any two vertices x, y are adjacent iff S x ∩ S y = ∅, 2. If the family covers E, i.e. E = S∈S S, If every maximal independent set M on the vertices in the family S x : x ∈ M partitios of E, Then G is partition graph of S [73, 74].

5 Figure 5 . 2 :

 552 Figure 5.2: Typical Intersection Graph for a hierarchy of partitions H. The independent sets of such an intersection graph enumerates all the cuts in the hierarchy H:{π 1 (E) = l 1 , l 2 , l 3 , l 4 , l 5 , π 2 (E) = a 1 , l 4 , l 5 , π 3 (E) = l 1 , l 2 , l 3 , a 2 , π 4 (E) = a 1 , a 2 , π(E) = E}.This graph codes the intersection of classes in the hierarchal structure shown beside it.

Proposition 5 . 4 .

 54 The intersection graph for a family of classes S = {S|S ∈ H} from any hierarchy of partitions, is a partition graph, and consequently the Maximum Independent Sets(MIS) of this Partition graph has a unique correspondence with the cuts in the hierarchy, ∀x ∈ M , ∃π(E) ∈ Π H (E).

Figure 5 . 3 :

 53 Figure 5.3: The MIS of the intersection graph corresponding to family of partitions from a Braid are shown in figure. They are {(a 1 , a 2 ), (b 1 , b 2 ), (a 1 , b 2 ), (E)}. Node pair (a 1 , b 2 ) form a MIS but does not form a cover of the space, since if A Y = E. Consequently the intersection graph is not a partition graph for the class family {a 1 , a 2 , b 1 , b 2 , E}. This is due to the missing edge shown in red, which when added transforms the graph in to a partition graph.

Figure 5 .

 5 4 we demonstrate a toy example with sample energies shown on a dendogram. Each node is given a capacity, which appears within it, as shown in figure. As long as two paths in this graph(tree) have no common node, the flows they carry are independent, and upper bounded by the lowest capacity along the portion where they are disjoint. When two such lines meet at some node, e.g. the two paths [a 1 , .., S] and [a 1 , .., S] which meet in S in then one must adopt some law for composing them, which is exactly what the optimal cut algorithm performs in the dynamic program.Consider for example the additive energies, which are the most similar to the flows over a directed graph. In this additive case, the capacities ω(T i } of the sons {T i } of S are added, and compared to ω(S). The min of ω(T i } and ω(S) gives the provisional capacity of the flow in S, and one pursues the climbing. At the end, the nodes of the

Figure 5 . 4 :

 54 Figure 5.4: Flow on Hierarchy: The diagram shows the min-cut for a planar graph(a tree) representing the hierarchy. The source S is connected to all the leaves by an infinite weight to force the flow through all leaves, while the sink is fixed at the root of the hierarchy. The iterations of the augmenting flow method are shown, where the minimum value on each path is subtracted from each node in the path, up till the point where we obtain a cut that separates S and T . Each augmenting flow step saturates necessarily one edge of the tree. The max-flow optimization is equivalent to the climbing optimization.

Figure 5 . 5 :

 55 Figure 5.5: Series arrangement of of partial partitions in the formulation of max-flow on a braid of partitions.

List of Figures 1 . 1

 11 Left: Partial partition with support S highlighted in red, with local partitioning shown in dotted lines. Right: Partial partition refinement ordering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 A two dimensional feature space is recursively partitioned using a binary tree where the partitions consist of purely rectangles. The corresponding binary tree is shown. One continues growing the tree until each rectangular encloses a minimal number sample points. . . . . . . . . . . . . . . . 1.3 Pruning example demonstrating Cost-Complexity pruning. Figure demonstrates a tree with classification the cost function given for each node given by equation (1.9) where values for different values of λ = 0(left), 0.5(center), 1(right), are shown with their corresponding pruned optimal subtrees. The pruned nodes are presented in gray. One can also see the representation of the the cuts in blue. They represent the leaves T of the optimally pruned subtree. These cuts represent not only the pruned subtree, but also the partition of the space. It is sufficient to refer to the cut and not the whole subtree. . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Toy example of a braid. E is partitioned by leaf nodes {a, b, c, d, e, f }. The set B 1 = {π 1 , π 2 , π 3 } forms a braid whose pairwise supremum is indicated at the right. One can note now that π 1 (X), π 2 (X) have a common parent X, but π 2 (Q), π 3 (Q) a common grand parent Q. The partition π x cannot be added to B 1 . There does not exist a supremum class, in the monitor hierarchy H, other than from the whole space E, thus not producing a braid structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 The intial color image (1) was segmented, giving the partition (2). Two zones are perhaps not correctly segmented demonstrated in (3), that we filled up by a Voronoi partitioning producing partition (4). Further a second parasitic class in (

  and (11) is the smallest element of H larger than the sup(1,[START_REF] Angulo | Multiscale stochastic watershed for unsupervised hyperspectral image segmentation[END_REF] and also (11) = sup[START_REF] Arbelaez | Boundary extraction in natural images using ultrametric contour maps[END_REF]10). . . . . . . . . . 1.6 From left to right: Initial gray-scale image, watershed hierarchy saliency by area attribute flooding, Watershed hierarchy saliency by volume attribute flooding, the monitor hierarchy saliency obtained by net opening over the intersection of Jordan nets, see subsection 4.8.2. . . . . . . . . . . 1.7 Left: and middle: details of the partitions π 1 and π 2 of the braid B, with the two classes S 1 (x) and S 2 (x) at point x. Right: S min 1 (x) and S min 2 (x) are the two lowest upper bounds of S 1 (x) and S 2 (x) in the monitor hierarchy. S min 2 (x) is the support of a partial partition of π 1 because S min 1 (x) ⊆ S min 2 (x). . . . . . . . . . . . . . . . . . . . . . . . . 1.8 No single Ground truth segmentation is a refinement of the mean-shift segmentation, but their suprema are. This is a result in the inconsistent partition contours in case of both the human and machine based mean shift segmentation. Image reproduced from Unnikrishnan et al. [114, 115]. This helps handle boundary ambiguity. . . . . . . . . . . . . . . . . . . . . 1.9 A hierarchy with multiple optimal cuts with the same energy. One requires the general condition of singularity to ensure the existence of a unique optimum. In a hierarchy, an arbitrary energy ω becomes singular when the parent picked w.r.t its children in case they have equal energies. 1.10 An example of energetic ordering: We have π ω π since in each class of π ∨ π , the energy ω of π is lesser than or equal to that of π . . . . . . . . 1.11 The three partitions π 1 , π 2 ,and π 3 cannot come from a braid, because the three classes of π 1 ∨ π 2 , π 2 ∨ π 3 and π 3 ∨ π 1 at point x are not nested. The values of energy ω for the classes are indicated above them, and the energy of a p.p. is the sum of its classes. The transitivity of the relation ω is not satisfied. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.12 Energetic infimum of three partitions: At point x we look for the largest class to be less energetic than the constituting internal p.p., this is π 2 and at point y it is the class of π 1 . The energetic infimum ω is the partition drawn at the bottom. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.13 An example of hierarchical increasingness on HOP(top) and BOP(bottom). We see that the condition of h-increasingness holds generally for any family of partial partitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.14 An elementary step of the dynamic program in a braid structure over a support S. The partial optimal cut in each sub-branch is shown. The final step is to compares energies ω(S), ω(π * 1 (S)), ω(π * 2 (S)), where one picks the partial partition with the least energy. Furthermore one needs to implement a consistent rule to obtain a unique solution, in other words, one needs to implement a singular energy. . . . . . . . . . . . . . . . . . . 1.15 For the example energy demonstrated in the table, the energy of a partial partition depends on the number of its classes, by a non h-increasing rule. 1.16 Figure demonstrating, initial hierarchy with cuts and their energies(left), the minimal cut by dynamic program(center) is π 2 (E), and the true minimal cut by observing the minimum directly is π 0 (E), the leaves(in white). The dynamic program fails to extract the minimal cut, and produces E as the minimal cut (in gray). This as well implies that we can not use the global-local property of the energy's optimum, even if the energy is singular. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

  g. both supremum and infimum for the proximity of ground truth with Hausdorff distances[61]. It is interesting to note that the uniqueness conditions in all the 3 cases have been assured by choosing the smallest/largest amongst the optimal cuts in the different cases. . . . . . . . . . . . . . . . . . . . .1.18 This demonstration shows an alternating composition: at each odd level we compose the energies by addition while at each even level we compose the energies by supremum. It's notable that the supremum of child energies is always smaller than or equal to their parent energies. . . . . . 1.19 An example of a scale function. For each λ the classes which are hold are just above the dotted line. Note that the class on the right branch with value 1, repeated twice correspond the same same class, and a cut having one or the other are the same cuts. . . . . . . . . . . . . . . . . . . . . . . 2.1 Blue: Breiman Chain, Green: Everett Chain, Pink: Salembier Chain . . . 2.2 Bottom Left, a hierarchy H with classes. the pairs of trees in the top row, indicate the two energies (ω ϕ , ω ∂ ) associated with the corresponding classes. π and π are two cuts of H. Bottom right, in the nodes, we depict the lambda values by equating parent and child energies, whose level sets give the minimal cuts w.r.t. the ω λ . They are depicted in the λ-tree for λ = 2, 3, 4 as π 2 , π 3 , π 4 . The λ values for the leaves are assumed to be 0, though in case of Breiman et al. [21] λ for the leaf classes are set to ∞ to avoid over-fitting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 For 2 < λ < 3 the minimal cut is (a, b, c, d, i) and ω ∂ = 8, for λ ≥ 3 the minimal cut is (g, h, i) and ω ∂ = 6, i.e. ω ∂ is never equal to the cost C = 7.5 at any time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Demonstrating the feasible space in the hierarchy with the family of Ccuts which are the set of partitions that satisfy a constraint of ω ∂ (π) ≤ C, which here for demonstration are cuts with number of classes no greater than 6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Minimal CCO cut. The + classes have ω ∂ > C, the -classes have ω ∂ ≤ C.

3. 1

 1 First row: Initial image and Saliency function corresponding to hierarchical watershed flooding[START_REF] Cousty | Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts[END_REF]. Second Row: Optimal cuts by using variance of luminance(left), chrominance(right). . . . . . . . . . . . . . . . . . . . . 3.2 Optimal Cuts for texture using variance of chrominance for scale λ = 100:Left, input Image, middle and right, cuts for parameters at µ = 10 12 (low uniformity) and µ = 10 14 (high uniformity), in Eq. 3.8. . . . . . . . . . . 3.3 The first row shows an input image, with two ground truth partitions corresponding to the image, from the Berkeley dataset. The bottom row consists of a sequence of thresholds of the Ultrametric Contour Map(UCM) segmentation hierarchy. The problem now consists in extracting a proximal partition in the hierarchy that is closest to one of these ground truth partitions. Further how do we compose when we have multiple ground truth partitions, and how do we compare hierarchies w.r.t a single ground truth. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Example demonstrating two partitions π 1 and π 2 , where one of them could be a ground truth segmentation while the other being a machine segmentation. Figure demonstrates the different possible configurations of refinement, braid and overlaps of classes. . . . . . . . . . . . . . . . . . 3.5 GT's corresponding to an input image. This demonstrates how the human experts in this case have drawn different scales of details in the scene. All scales are not present in no single GT partition. This as described well across literature is due to the ill-posedness of the segmentation problem. The OIS averages the results of choosing the right scale of partition from the UCM across various GTs to evaluate the segmentation hierarchy. We will use instead an inf-composition to extract the proximal partition. We remark the inherent braid structure in such cases. . . . . . . . . . . . . . . 3.6 Haussdorff distance Assymetry: Smallest disc dilation of X that contains Y is drastically larger than theat of X to contain Y, thanks to a the difference in convexities of the shape. The same situations occur when dealing with classes of a partition. . . . . . . . . . . . . . . . . . . . . . . 3.7 Energiesω G (S), θ G (S) for each class S in a segmentation, defined w.r.t class from a ground truth partitionin red. The composition of these energies decid the local distance measure introduced and minimized. . . . . 3.8 Row 1: ω GT 2 (S), Row 2: θ GT 2 (S), Row 3: ω GT 7 (S), Row 4: θ GT 7 (S). Figure shows the different half haussdorf proximity functions ω(S) and θ(S) for each class from different partitions in a hierarchy. The two ground truhs chose are of different scales. The gray scale value 0 corresponds to closest while 255 corresponds to farthest. Ground truth and associated distance function on left, energy values over 6 different partitions from the hierarchy on its right. One can already get a quick idea of what the dynamic program would extract an a minimal cut looking at the individual values. One can see that the scale of the ground truth partition affects the energy associated with classes of the hierarchy of segmentatations. What's left is to obtain a good composition. . . . . . . . . . . . . . . . . . 3.9 Ground truth partitions, and corresponding optimal cuts, for energies ω G , θ G and for composition by sum ω G + θ G . The input hierarchy is the UCM from the Berkeley dataset, consisting of 800 level binary partition tree. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.10 Two ground truths and their union are shown, with their corresponding optimal cuts, for the energy θ G + ω G . The composition over different ground truth sets is achieved by infimum as shown. . . . . . . . . . . . . . 3.11 a) Leaves partition b), c) and d) Conditional λ-cuts for λ = 0, 10, & 80. .

4. 1

 1 Saliency Function example: First image is an input image from Berkeley data base. Second image is the corresponding Ultrametric Contour Map(UCM) which represents a hierarchy of partitions. Third image shows the corresponding altitude map for the UCM. Here we see that each arc in the saliency function separates different components at different altitudes. This third image is oriented 180 degrees off to better render the details. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Examples of Jordan Curves. The Jordan curve tessellates the space, with one bounded interior component or face F and an unbounded exterior component S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 One can suppress the contour in the interior of region F 1 , while no increasing the energy ω(π) of the partition by F 1 , F 2 . . . . . . . . . . . . . . 4.4 An elementary Jordan net N 0 , A set X of arcs and Jordan curves, and their net openings γ(X). It is important to note that the number of connected components of white pixels don't change after a net opening |γ(X)| CC = |X| CC . γ(X) removes two types of arcs: open arcs and arcs which are not normal [64]. . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Initial Image 25098 from Berkeley database, leaves given by lowest(finest partition) threshold of Ultrametric contour map(UCM). The leaves here represents the initial finite net N 0 . . . . . . . . . . . . . . . . . . . . . . . 4.6 Initial Image 239096 from Berkeley database, Saliency function s: Ultrametric contour map(UCM), Ground truth partition G, Inverted distance function g. The inputs here we consider are the saliency function s and the inverted distance function g. . . . . . . . . . . . . . . . . . . . . . . . 4.7 The figure demonstrates how the image I produces a meaningful gradient function in the xy directions ∇ x I ∪ ∇ x I, using the image values. The label image L on the other hand does not depend on the actual values in the gradient (∇ x L depends on the labeling L which can be arbitary) but just the existence of the non-zero gradient value. This can be seen as a indicator function of the gradient function 1(∇L). . . . . . . . . . . . . . 4.8 Input function g on a simple toy Jordan net, The net opening γ(g) and their level sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.9 Complement of input function g = max(g) + 1 -g, Intermediate result showing the opening of inverted function γ(g ), and finally for the pseudo closing we use another iteration of the net opening: φ(g) = γ(max(γ(g )+ 1 -γ(g ), with level sets of g and φ(g). The opening of the complement is a closing in the space of arcs but not that of the Jordan nets. . . . . . . 4.10 Lack of upper bounds by closing. . . . . . . . . . . . . . . . . . . . . . . 4.11 Dual Closing: The thickening τ is not increasing. g and g are binary or

4. 14

 14 Inverse distance function g GT 4 = 1-d 4 (GT 4), Transformed saliency γ(s+ g GT 4 ), Point distance function g point , Transformed saliency γ(s + g point ). .[START_REF] Ravi | Optima on Hierarchies of Partitions[END_REF].15 Hierarchy fusion: Here we fuse two hierarchical structures by introducing a distance function d which is unique(given its leaves) for every saliency function s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.16 Scaling space with different distance functions: Here we demonstrate how the inner diamond and the circle get separated when using the (4connection d 4 ) city-block distance function, while the euclidean (d ball ) distance function reaches them at the same scale. This produces two different saliency functions and thus hierarchies: γ(d 4 + π 0 ) (with 2 levels) and γ(d ball + π 0 ) (with single level). . . . . . . . . . . . . . . . . . . . . . 4.17 Toy example demonstrating Hausdorff distance ordering. First case with blue circle demonstrates a symmetrically aligned pair circular contours(black, blue), the second case demonstrates an asymmetrically aligned pair of circular contours(black, red). Aside the arrows we calculate the infimum of radius of dilation for one contour to cover completely, the other, for example, the radius of dilation of set in blue to cover set in black is 1, while radius of dilation for set red to cover set in black is 2. The third figure demonstrates how the two circles are reordered, by associating them with the inverse of the Hausdorff distance between the circles. . . . . . . . . . . 4.18 This example demonstrates 3 scales of ground truths and the corresponding, Hausdorff distance ordered saliency functions. The base Jordan net is extracted from the leaves/finest level of the input saliency function. Partitions corresponding for each ground truth at a threshold(level) of 350 is shown. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.19 This toy example demonstrates a partition N 0 with three ground truths partitions. Two of the three ground truth lines overlap and is represented in red, while the single ground truth in blue. In such a case one produces the saliency function weights as seen. This is due the composition by addition that weights a partition contour if its proximal to larger number of ground truths (here overlapping, but in general can span space at different locations) than if it is simply close to a single ground truth. This also in contrast produces a different ordering compared to the ∧compositions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.20 Initial image with (a,b,c,d,e) representing five different ground truths, with images (g,h,i,j,k) corresponding inverted distance functions of ground truths. While (f) shows the sum of ground truths, and (l) its inverted distance function. We see different contours of the lizard in the image that are reinforced. Further more the ground truth partitions in this case are not simple refinements, and thus validating our use of a composition by addition. Corresponding net openings are demonstrated in 4.21 . . . . 4.21 Figure shows initial leaves partition N 0 with three ground truths partitions, and the different net openings possible. The last saliency demonstrates the composition by addition that weights, where higher weight is given to a partition contour, if its proximal to larger number of ground truths (here overlapping, but in general can span space at different locations) than if it is simply close to a single ground truth. Composition by addition (eqn 4.13) also in contrast produces a different ordering compared to the ∧-compositions (eqn) 4.12). Please refer to figure 4.22 to view the different scales that can be extracted using the M-measure from equation(4.14). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.22 The plot demonstrates the M measure for the saliencies in figure 4.21 generated with different inverse distance functions of ground truth(in figure 4.20), saliency by Inf-composition and composition by addition. As we can see the inf and sum composition form the bounds of variation of the M -measure which not only provides a structural measure of how many children are regrouped by the parent level in each hierarchy but also the number of levels in the hierarchy. . . . . . . . . . . . . . . . . . . . . . . 4.23 Evaluating with M : The first row shows 3 different distance functions, d GT 2 , d GT 4 , d pt with their corresponding ground truths on the top right corner of the images. π 0 is the leaves partition on which the distance functions will reorder the arcs. In the bottom row we have the transformations γ(π + d GT 2 ), γ(π + d GT 4 ), γ(π + d pt ) representing new the hierarchies(saliency functions). The plot on the bottom right displays the M measure at different levels of the hierarchy. The maximum number of levels in these hierarchies is bounded by the maximum value of the distance function producing a partition. . . . . . . . . . . . . . . . . . . . 4.24 Net opening over the intersection of two finite Jordan nets N, M . This example demonstrates the use of the net opening operator to extract from the intersection of two Jordan nets (which is not necessarily a set of Jordan curves) the largest Jordan net in the base net N 0 . Since we work on pixel domains, one can consider the finest net N 0 to be the one separating all pixels. This operator will be used to create braids while recomposing partitions from different hierarchies. This a very simple operator, and it helps in combining partitions which don't share a common leaf partition or Jordan net. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4.25 Calcuating the monitor partition of two partitions using the net opening. The partitions (a), (b) are extracted from watershed flooding by attributes of area and volume respectively [32]. One can note that the watershed by different attributes are not hierarchical. (c) is the intersection of contours between the two partitions, (d) gives the net opening of the intersection, resulting in the monitor partition. (e) shows a magnified view of the contour in the intersection set, where the area and volume floodings have small difference, resulting in a fissure in the intersection set. This leads to a loss of a large class, in the corresponding net opening. We calcuate the monitor of the braid formed in such an event demonstrated in figure 1.6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.26 Figure shows, Leaves partition π min 1 in watershed saliency by area, π min 2 , corresponding partition from watershed saliency by volume. And finally π min as described in figure 1.7 in chapter 1, is the net opening of the intersection π min 1 ∩ π min 2 . The π min of the braid and finest partition picked from watershed saliency π min 2 by volume are the same, thought this might not be the case generally. . . . . . . . . . . . . . . . . . . . . . 5.1 For the given graph, the maximal independent set of vertices are {{a, c}, {b, e}, {d, f }}. Addition of any other vertex would lead to an inclusion of an edge between the MIS vertex set. This characterizes its maximal nature. . . . . . . . . . 5.2 Typical Intersection Graph for a hierarchy of partitions H. The independent sets of such an intersection graph enumerates all the cuts in the hierarchy H: {π 1 (E) = l 1 , l 2 , l 3 , l 4 , l 5 , π 2 (E) = a 1 , l 4 , l 5 , π 3 (E) = l 1 , l 2 , l 3 , a 2 , π 4 (E) = a 1 , a 2 , π(E) = E}. This graph codes the intersection of classes in the hierarchal structure shown beside it. . . . . . . . . . 5.3 The MIS of the intersection graph corresponding to family of partitions from a Braid are shown in figure. They are {(a 1 , a 2 ), (b 1 , b 2 ), (a 1 , b 2 ), (E)}. Node pair (a 1 , b 2 ) form a MIS but does not form a cover of the space, since if A Y = E. Consequently the intersection graph is not a partition graph for the class family {a 1 , a 2 , b 1 , b 2 , E}. This is due to the missing edge shown in red, which when added transforms the graph in to a partition graph. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Flow on Hierarchy: The diagram shows the min-cut for a planar graph(a tree) representing the hierarchy. The source S is connected to all the leaves by an infinite weight to force the flow through all leaves, while the sink is fixed at the root of the hierarchy. The iterations of the augmenting flow method are shown, where the minimum value on each path is subtracted from each node in the path, up till the point where we obtain a cut that separates S and T . Each augmenting flow step saturates necessarily one edge of the tree. The max-flow optimization is equivalent to the climbing optimization. . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Series arrangement of of partial partitions in the formulation of max-flow on a braid of partitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Chain of ideas for yielding a minimal cut in a hierarchy or a braid. (axioms in rounded boxes). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Two models to formulate the constrained optimization problem. The

  1.2. OPTIMIZATION ON HIERARCHIES OF PARTITIONSInGuigues thesis [47] he maps the family of cuts in an input hierarchy H 1 to an ordered set of cuts forming a new hierarchy. He views this step as a precursor to image segmentation, where the all choices of scale parameter values λ are indexes of the new hierarchy of segmentations H λ . The optimal λ for image segmentation is not really considered. He refers to the monotonicity in λ, as causality using terminology of scale-spaces, since the scale-sets are image descriptors defined by a hierarchy, energy and a numerical function generating the energy.Furthermore Guigues provides conditions of monotonicity akin to Breiman et al, by describing multi-scale energies on the nodes in the hierarchy of partitions. The constraint function (the number of leaves in a sub-branch in case of CART) is defined to be sub-

additive 2 , i.e. f (A ∪ B) ≤ f (A) + f (B). While the objective function(variance in the case of CART) was defined to be super-additive, i.e. f (A ∪ B) ≥ f (A) + f (B). One must note that both conditions of super-additivity and sub-additivity are sufficient for monotonicity, but not uniqueness. As we have already seen there can be many optimal cuts, but there can be only one smallest/largest (in terms of refinement) optimal cut. This is how uniqueness is assured. Please refer to the summary in table 2.1 for the different conditions on uniqueness, monotonicity and constraint sense.

  1. The sequence of studies by Breiman et al., Salembier-Garrido, and Guigues work on the space of HOP. Here it is critical to note that we will work on the expanded space of Braids, introduced further on.

2. All energies in above mentioned studies are always linear. Though there exists non-linear energies on HOP, such as area filtering, Soille's constrained connectivity

[108], Ackay-Ackcoy filtering on multi-spectral images

[2]

. We propose necessary and sufficient conditions where the dynamic program of weakest link pruning still holds, namely h-increasingness.

3. Monotonicity of cuts have up until now been assured by sub/super additivity of

the constraint function, in conjunction with said linear energies. For the classes of both linear and non-linear energies, we introduce a necessary and sufficient, monotonicity condition, namely scale-increasingness.

  1 ) (resp. S 2 , resp. S 3 ). The classes S 1 , S 2 , S 3 of H, which contain the point x, are thus ordered in one of the possible three mannersS 1 ∪S 3 ⊆ S 2 , S 1 ∪S 2 ⊆ S 3 ,or S 1 ∪ S 2 ⊆ S 3 . Suppose firstly that in the first case S 2 = S 3 thus S 1 ⊆ S 3 . The proposition 1.8 tells us that S 3 is the support of a p.p. of π 1 , and according to Rel.(1.23) the restrictions of π 1 , π 2 , and π 3 to this support satisfy ω(π 1 S 3 ) ≤ ω(π 2 S 3 ) and ω(π 2 S 3 ) ≤ ω(π 3 S 3 ), so that the transitivity is satisfied at point x. The same proof still applies for S 1 = S 3 , and extends to the two other cases by circular permutation.If S 1 ∪S 3 ⊂ S 2 , then there exists a partial partition a 3 of π 3 such that {S 3 } a 3 has S 2 for support, and Rel.(1.23b) implies that ω({S 2 }) ≤ ω({S 3 } a 3 ), thus by singularity thatω({S 2 }) < ω({S 3 } a 3 ). Similarly, the inclusion S 1 ⊆ S 2 leads to ω({S 1 } a 1 ) < ω({S 2 })(by applying Rel. (1.23a)), which contradicts the singularity axiom. Therefore, the two possible orders are S 1 ∪ S 2 ⊆ S 3 and S 2 ∪ S 3 ⊆ S 1 .If S 1 ∪S 2 ⊂ S 3 , there exist two p.p. a 1 and a 2 with {S 1 } a 1 = {S 3 } and {S 2 } a 2 = {S 3 }. If S 2 ∪ S 3 ⊆ S 1 , the same proof yields the same conclusion. Since local transitivity is true for all point x ∈ E, the relation π 1 ω π 2 itself is transitive.

	By Rel. (1.23b), we find ω({S 2 } a 2 ) ≤ ω({S 3 }). Then by singularity, all p.p. of S 3 , thus
	{S 1 } a 1 , have an energy ≤ ω({S 3 }), so that ω({S 1 } a 1 ) ≤ ω({S 3 }), which shows the
	transitivity at point x.

  {S}, {S} π 1 , {S} π 2 have three energies which are different by singularity. One can always choose the less (resp. most) energetic one. By doing the same for all classes of π 1 ∨ π 2 we obtain the unique largest lower-bound π 1 ω π 2 (resp. smallest upper bound π 1 ω π 2 ) of π 1 and π 2 , which achieves the proof. Energetic infimum of three partitions: At point x we look for the largest class to be less energetic than the constituting internal p.p., this is π 2 and at point y it is the class of π 1 . The energetic infimum ω is the partition drawn at the bottom.if and only if B is a braid. Given a family {π j , 1 ≤ j ≤ p} of cuts in Π(ω, B), the infimum ω π j (resp. supremum ω π j ) is obtained by taking the p.p. of lowest energy (resp. highest energy) in each class of the refinement supremum ∨π j .

				π 1
		11		6
				π 2
	50	1	8	20
	10		8
		x	y
				π 1 ∧ ω π 2
	Figure 1.12:
	Theorem 1.12. Let B be a family of partitions of E, and ω be a singular energy. The
	set of all cuts of B forms a complete lattice Π(ω, E, B) for the energetic ordering ω

  Finally, it is increasing. Let i 1 and i 2 be the first levels of H 1 and H 2 where the section is not replaced by π * . If B 1 ⊆ B 2 , then we have i 1 ≥ i 2 . Above i 1 , the sections are those of H 1 and H 2 ; between i 1 and i 2 the sections of H 1 are π * and those of H 2 are ≥ π * , and below i 2 the sections of both H 1 and H 2 are π

*

. In all cases ϕ(B 1 ) ⊆ ϕ(B 2 ), which achieves the proof. Corollary 1.29. Let {ω(λ), λ ∈ R} be a scale increasing family of singular energies acting on the braids B ∈ B . The closings {ϕ λ , λ ∈ R} of the proposition form the Matheron semi-group 4 , i.e.λ,

  In the Breiman chain one uses scale-increasingness w.r.t the multiplier to have a monotonic pruning of HOP, or increasing λ-cuts, while in the Everett chain one is trying to approximate the constraint by choosing a "good" multiplier.2.3 Guigue's λ-cuts are upper boundsWe demonstrate how the λ-cuts calculated byGuigues et al. in [47] are upper bounds the optimal objective function value, achievable in constrained optimization problem on the HOP. One quick intuition to begin with is what happens when we vary the constraint function instead of the Lagrangian multiplier λ ?

	• Second chain develops the lagrange multiplier based constraint, starting with Ev-
	erett's theorem, its use by Shoham-Gersho in calculating a optimal source coding
	schemes, leading to Salembier et al.

• Firstly in blue:

Breiman

's dynamic program to prune classification and regression trees(CART), CART's usage in information theory starting with Grey et al. for source coding, further on CART's DP applied to wavelet tree pruning by Ramchandran-Vetterelli, finally ending in Salembier-Garrido for pruning Binary partition trees. • The third chain consists of Salembier-Garrido try to solve the rate-distortion minimization problem on binary partition trees by approximating the constraint rate by searching for an near optimal multiplier, while Guigues established the scale-sets image descriptor for a given HOP and parameterized energy ω(π, ω). Caselles et al develops Salembier-Garridos model for level line selection in the tree of shapes. 67 Multiplier's Role:

  .36) which solves the primal problem 2.28. Concerning the dual problem, we draw from (2.33), (2.34), and (2.36) that λ≥λ *

  It remains to build up the hierarchy H i.e. to find the leaves π 0 . The search being combinatorial, the complexity reason drives us to call for inf-modularity for ω ∂ . Putω ∂ (S) ≤ ∧{ω ∂ (T ), T son of S},(2.47)i.e. the energy ω ∂ of class S is smaller or equal to the smallest energy of the sons of S.

	.46)
	2.10.2 Implementation for inf-modular ω ∂

Such class inf-modularity acts on classes and no longer on p.p. as Rel.(1.46), but both are equivalent. The inequality (2.47) is preserved indeed when any son T is replaced by its own sons, which allows us to progressively obtain any p.p of the right member of (1.46). Conversely, it suffices to particularize Rel.(1.46) to the sons of S for finding Rel.(2.47).

  .[START_REF] Beucher | A topological approach to hierarchical segmentation using mean shift[END_REF] an interpretation which connects the distance function with the partial order on sets by inclusion. In Rel.(3.19) the value ρ G (resp. ρ A ) is the maximal distance from a point of

∂S to G (resp. of G to ∂S). The first one, ρ G , indicates how precise is S w.r.t. the GT, the second one, ρ A , how representative is this GT. In indexation, these two numbers are respectively named precision and recall. The symmetric expression ρ = max{ρ G , ρ A } is the well known Hausdorff distance Hausdorff distance is lacking of finesse because it is a global notion, and of robustness because it uses suprema. If we could define a local equivalent, associated with each

  (λ)) begin H monitor ←-generateBraid({H 1 , H 2 , ...H q }) monitor hierarchy ∀H ∈ {H 1 , H 2 , ...H q }, N i ←-|H i | N monitor ←-|H monitor | #levels in H monitor T rivialBraidf lag ←-|H monitor (1)| # classes in leaves of H monitor if TrivialBraidflag then \\Perform q-DPs independently, without the braid structure for H ∈ {H i

*

  and S i S j and S j S i • Containment Graph: S i ⊆ S j and S j ⊆ S i • Disjointedness graph: S i ∩ S j = ∅ Following which various combinatorial problems are posed based on the characterization, such as intersection number, containment numbers, (minimum size of model in question) etc. Intersection graphs in general have been used in the area of computational geometry, with various types based on the object under study, e.g. intervals, polygons, lines or other geometric objects[START_REF] Mckee | Topics in Intersection Graph Theory[END_REF], as well as in abstract graph theory.

•

  The thesis generalizes the dynamic program used first in pruning decision trees byBreiman et al. [21], and later by Guigues et al.[START_REF] Ballester | Level lines selection with variational models for segmentation and encoding[END_REF] 47,[START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF] for non-linear energies with the h-increasingness property. We study the meaning of optimality in the Lagrangian Multiplier methods used byGuigues et al., Salembier et al., [47,[START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF], and demonstrate how it is a Lagrangian Relaxation, of the original hard constrained optimization problem on the HOP, and study its implications.

			Constraint
			ω δ ≤ K
	Hierarchy H
	of segmentations
	Energetic Lattice	Ordered set of	Minimal
		Π ω of cuts	Minimal Cut	Cut π *
	Energy ω(π, λ) =
	ω ϕ (π) + λω ∂ (π)
	Singularity	Scale increasingness	h-increasingness
	Figure 6.1: Chain of ideas for yielding a minimal cut in a hierarchy or a braid. (axioms
		in rounded boxes).
	• We provide the scale-incresaingness property energy ω λ in the energetic lattice,
	and extend Guigues et al.'s [47] causality condition of monotonicity to non-linear
	energies.

• The thesis introduces for the problem of constrained optimization on partitions, the notion of inf-modularity. This condition plays a role similar to that of subadditivity, for set valued functions

[12]

, while operating on partial partitions.

•

  These problems have not been studied by braids yet they all can be approached by the braid theory, which of course does not excludes other approaches.

	Index
	α-flat zones, 43	MWIS, 162
	h-increasingness, 35
		net opening, 127
	Braid, 21	Normal Segmentation, 123
	Breiman's Conditions, 15
		Overlap Graph, 163
	CART, 12
	CCM, 86	partition graph, 164
	complementary slackness, 66	perturbation, 72
	composition, 40	polytree, 169
	Containment Graph, 163	pruning, 13
	convex hull, 60 Cost-Complexity pruning, 17	rate-distortion, 58 relaxation, 62
	dominant ancestor, 45 dual problem, 65 dynamic program, 38	Saliency, 129 scalar Lagrange family, 77 Scale function, 51
	energetic ordering, 30	scale increasingness, 48
	Everett's Main Theorem, 63	scale-set, 19
		semi-groups, 50
	ground truth, 100	separability, 19
		singular, 29
	Hierarchies, 9	strictly h-increasing, 35
	inf-modular, 52	sub-additive, 18
	Intersection Graph, 162	tree functionals, 60
	Jordan curves, 122	TSVQ, 60
	Jordan net, 124	Uniqueness, 14
	Khalimsky, 137	watershed, 128
	KKT, 65
	Lagrange multipliers, 61
	Lagrangian energetic lattice, 80
	lattice, 11
	LMCC, 84
	LME, 80
	max-flow, 167
	max-pooling, 46
	Minkowski, 41
	MIS, 162
	Monotonicity, 14

A small square image of 5 by 5 pixels can partitioned in 10 18 of different manners, which is the Bell's Number[START_REF] Bell | Exponential numbers[END_REF] to count the number of different partition of

pixels. This number grows exponentially large with the number of elements.

Though one must note that in further development we will not require additivity condition and we can compose these energies by non-linear operators like the supremum or infimum.

1.3. BRAIDS OF PARTITIONS

There are two main types of scale spaces semi-groups used in scale space studies. First is the linear semi-group, based on a vector space. Second is the semi-group of Matheron's granulometries[72] which uses an underlying lattice for analysis, and where the most active transformation imposes its law.

The dual problem was discovered during the 1920s by John Von Neumann in matrix games, but had for a long time implicitly been used also for nonlinear optimization problems before it was properly stated and studied by Arrow, Hurwicz, Uzawa, Everett[START_REF] Everett | Generalized lagrange multiplier method for solving problems of optimum allocation of resources[END_REF], Falk, Rockafellar, others, starting in earnest in the 1950s. The original problem referred to as the primal problem, was a name given by George Dantzig's father, a Greek scholar.[80]

here refers to the positive semi-definiteness of matrix λ.

The KKT theory is more general. It may involve additional constraints hj = 0 (as in Lagrange' initial work) and extends to the pseudo-convex case (a function f on R n is pseudo-convex if ∇f (x).(y -x) ≥ 0 implies f (y) ≥ (f (x)); a convex function is pseudo-convex). Besides, the continuous derivability can be replaced by a weaker Lipschitz condition. Moreover, this is very wide field of convex analysis. The demonstrations chosen here highlight the comparison with the braid framework required for the next section.

Weierstrass Theorem states that if K ⊂ R n is compact and f : K → R is a continuous function, then f has a maximum and a minimum on K, i.e. there exists k , k * such that f (k ) ≥ f (k) ≥ f (k * ), ∀k ∈ K

Mot Clé: Segmentation hiérarchique, Multiplicateurs de Lagrange, Optimisation dans les treillis, Morphologie mathématique.

Acknowledgments

and Santiago-Velasco for their corrections and detailed comments on the thesis. I am also grateful to ESIEE and Université Paris-Est to have provided me with the PhD Grant and the opportunity to pursue an interesting research problem. The thesis was part of the KIDICO research program ANR-2010-BLAN-0205-0. It was a pleasure, to work with Christine Voiron, and Pierre-Alain Mannonni, at UMR

Abbreviations

PP Partial Partitions

Chapter 2

Constrained optimization

This chapter provides two different contributions 1 , to the problem of constrained optimization on hierarchies:

Chapter contribution summary

• Firstly in section 2.2, we quickly recap Lagrangian multiplier methods, and the conditions of for optimality for the primal and dual Lagrangian. We show that the constrained optimization problem on HOP in literature are solved by applying Everett's theorem. We discuss how one can improve the lower bound of globally optimal solutions of λ-cuts by Guigues and Salembier-Garrido [START_REF] Ballester | Level lines selection with variational models for segmentation and encoding[END_REF]47,[START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF], by perturbing the Lagrangian. Further we demonstrate the use of a different h-increasing penalty function for the constrained optimization problem.

• From section 1.8 onwards, we generalize the constrained optimization problem, at two levels, first by introducing the energetic lattice, and second by enlarging the feasible set by introducing the braids of partitions.

Review on Constrained optimization on Trees

As we have already remarked in the previous chapter, in this section 2.1 we study classification and regression trees(CART) by [21] as applied to the problem of constrained optimization on a generalized hierarchy of partitions (and not just rectangular partitions) [START_REF] Ballester | Level lines selection with variational models for segmentation and encoding[END_REF][START_REF] Garrido | Hierarchical Region Based Processing of Images and Video Sequences: Application to Filtering, Segmentation and Information Retrieval[END_REF]47,[START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF]. There has been a sequence of studies following CART in the domain of information theory to solve constrained optimization problem of rate-distortion [26]. As Gray et. al [26] explain, pattern recognition used distortion-rate bounds on classification

Yes

Scale-increasingness [47,[START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF] Table 2.1: This table summarizes the different components of the algorithms of different authors work, related to the study of extract of optimal cuts, the monotonicity of ordering of the cuts based on λ, the uniqueness of these solutions of a given constraint, as well as whether the multiplier is chosen or not. (a) First formulation of Dynamic Program for optimal pruning on trees (b) First formulation of multiplier dependent perturbed primal problem for any arbitrary set. (c) Guigues interprets scale-space semi-group structure as in [START_REF] Ravi | Optima on Hierarchies of Partitions[END_REF] by introducing the causality condition in the scale-sets framework. (d) The energetic lattice can be used to solve the linear Lagrangian case, while also non-linear energies which do not entail a Lagrangian multiplier method and application of Everett's theorem.

Vector case (multi constraints)

The multi-constraints situations are matter of the same approach. The minimization of ω ϕ is now subject to p constraints ω ∂i (S) ≤ C i . Denote by C the cost vector C = {C i , 1 ≤ i ≤ p ≤} The family A(C) is that of the cuts whose classes S satisfy the p constraints ω ∂i (S) ≤ C i . The proposition 2.14 extends to this vector cases:

Proposition 2.18. The family A(C) is closed for the refinement infimum and supremum.

Proof. We keep the notation used in the proof of proposition 2.14. Let π 1 and π 2 be two elements of A(C). If S 2 ⊆ S 1 , the p.p. a 2 of π 2 of support S 1 is the minimum π 1 ∧ π 2 in S 1 , and all classes S of a 2 satisfy the p conditions ω ∂i (S) ≤ C i . The same occurs if S 1 ⊆ S 2 , which results in that π 1 ∧ π 2 belongs to A(C), and achieves the proof We can also view the situation when one constraint at least is satisfied (a case impossible to treat with Lagrangians). The family A(C) is replaced by B(C) such that (Proof similar to the previous one). Like in case of a single constraint, both A(C) and B(C) lead to hierarchies H with a unique π 0 . Proposition 2.15 and the greedy algorithm still apply, with minor changes.

Extension to braids:

The class constrained minimization that we just developed for hierarchies extends to braids under one more condition. As Proposition 28 involves the refinement minimum of two partitions, i.e. of horizontal of cuts of H, we must also assume that the refinement minimum of two partitions of a braid is a partition of the hierarchy. It is clear that when this condition is fulfilled, then all results of the current section remain valid for braids.

Overview of the three models

The three models for constrained optimization on Braids and thus also hierarchies, basically introduces three different ways of introducing the constraint: Firstly LME, which

Describing Segmentation with Jordan Curves

The image segmentation process results in a segmentation which is formally defined as a partition of the input space. Many authors have used the concept of Jordan Curves to represent the segmentation of images. To represent the Ultrametric Contour Maps (UCM) [START_REF] Arbelaez | Boundary extraction in natural images using ultrametric contour maps[END_REF], Arbelaez refers to a segmentation K as a finite set of rectifiable Jordan curves, which are called the contours of K. A finite set of rectifiable Jordan Curves is said form a normal segmentation when the removal of any number of curves increases the energy defined the segmentation. One similarly finds the definition of 1 -N ormal, 2 -N ormal segmentations by Morel et al. [START_REF] Koepfler | A multiscale algorithm for image segmentation by variational method[END_REF][START_REF] Morel | Variational methods in image segmentation[END_REF]. Furthermore Jordan curves can be used to describe the contours of components(level sets) of a continuous functions [START_REF] Monasse | Fast computation of a contrast-invariant image representation[END_REF].

Jordan curves may be extremely irregular (e.g. the fractal Von Koch snowflake), and even have of non measurable lengths. Measurable Jordan curves have been used in image processing by [START_REF] Monasse | Fast computation of a contrast-invariant image representation[END_REF][START_REF] Morel | Variational methods in image segmentation[END_REF] for functionals whose computations involve length and perimeter as seen in the case of Mumford Shah Functional. But this restriction is not pertinent here, since we do not measure lengths.

Jordan Nets

Here we construct a lattice structure that uses a finite set of Jordan curves to create a partition of R 2 .

Definitions

Definition 4.3. A Jordan net, or J-net N , is defined as a set of Jordan curves, which delineate a finite number of open insides. In addition, the empty set ∅ is also,by definition, a J-net.

N is thus a set of contours C i that delimits the bounded faces F 1 , F 2 , ..., F p , p < ∞, plus the unbounded background S. Both faces and background are open. J-nets may comprise several connected components and faces included in each other. Note that the number of primitive Jordan curves one can extract from N does not reduce to the C i .

In Figure 4.4 for example, one can take as primitive J-net curves, the two half circles which share a diameter, while one can also take, just one of them plus the complete disc.

The faces we consider are the complement of the Jordan net N . One can observe of course that the definition of the Jordan net basically resembles that of a segmentation [START_REF] Morel | Variational methods in image segmentation[END_REF]. According to Jordan theorem, the plane R 2 is partitioned by the union of N, S, and {F i }.

Geometric and Intrinsic Net openings

The net opening until now involved the combination of an external function and an elementary finite Jordan net. In this section we will show how we not only use an external function, but also the intrinsic properties of partition itself to perform the opening. Intrinsic properties here in this particular case refer to curvature of the partition contours.

We use a combination of a corner opening and net opening to basically transform an input weighted quad-tree into a weighted K-d tree saliency function. Such geometrical features have been studied to extract corner points [START_REF] Laganiére | A morphological operator for corner detection[END_REF]. For a study of geometric corner point openings, or L-openings please refer to our paper [58].

When the geometry of the faces of the Jordan net are well defined as in cases of regular polygons, one can also use this in applications involving polygonal mesh simplifications [START_REF] Delest | Hierarchical mesh segmentation using waterfall and dynamics[END_REF]. In extending the study and relaxing geometric constraints, one can think of performing curvature based net openings, which envisage the use a global point-wise curvature function like in [27]. In the place of corner based L-opening, one can transform the saliency to produce directional net openings that remove contours by ordering them based on the supremum of local radius of curvature. In literature we also find global contour extraction methods that use symmetry of object shape as a prior. This helps extract certain object classes having a certain axis of mirror symmetry, which is analytically measured [START_REF] Stahl | Globally optimal grouping for symmetric closed boundaries by combining boundary and region information[END_REF], thus providing us a way to perform a symmetry based net opening.

Braids from net opening 4.8.1 Braids from multiple functions on single J-net

In this section we show that given two numerical functions g 1 , g 2 : R 2 → R, and a input Jordan net N 0 , we have: Proposition 4.10. The union any two hierarchies obtained from the individual net opening with any two numerical functions, forms a braid of partitions, where the monitor hierarchy is given by the hierarchy obtained from the net opening using the supremum of the two input functions. That is:

This is a direct application of theorem 4.9. We can see that given any two numerical functions g 1 , g 2 the numerical net openings γ(g 1 ), γ(g 2 ) on a common net N 0 , are the highest lower bounds of g 1 , g 2 . Furthermore γ(g 1 ∨ g 2 ) is the bounding saliency function for g 1 ∨ g 2 . Also we have, γ(g 1 ) ≤ γ(g 1 ∨ g 2 ) and γ(g 2 ) ≤ γ(g 1 ∨ g 2 ). It is now direct that the braid structure results when we are assured of the existence of γ(g 1 ∨ g 2 ).

Furthermore one can observe that the any transformed saliency functions γ(s + g) of an input saliency function s, by any external function g always produces functions that will include the original saliency function s forming a braid.

Intersection of multiple Jordan nets

We recall here briefly the proposition 4.11, first proposed and proved in [58] , to further study the net-opening operator in the context of calcuating supremums of partitions which are not hierarchical. Further the same operators helps create braids of partitions. Consider all J-nets included in a basic one N 0 , whose associated faces are called the leaves. Two families derive from N 0 . Firstly, the power set P(N 0 ), constituted by all sets whose points belong to N 0 . Secondly, the family N (N 0 ) of all J-nets included in N 0 . We have N (N 0 ) ⊆ P(N 0 ). Both sets P(N 0 ) and N (N 0 ) comprise the empty set ∅ and are ordered by inclusion. P(N 0 ), as a power set, is a Boolean lattice. But unlike P(N 0 ), the family N (N 0 ) is not complemented: if N ∈ N (N 0 ) the complement N 0 \N may have Jordan arcs which are not looped. However, the following property holds:

is an opening on P(N 0 ), called net opening, whose set of invariants is N (N 0 ). In

To conclude this discussion on Jordan net openings operator, in figure 4.24, we demonstrate the net opening while composing over the intersection of two elementary finite Jordan nets.

Chapter 5

Algorithms and Graphs

In this short chapter we provide a quick review of super-pixel segmentation methods, demonstrating the different problems encountered in region merging methods, a posteriori. Further we also continue from the work by Brendel-Todorovic [START_REF] Brendel | Segmentation as maximum-weight independent set[END_REF] who poses the image segmentation problem as an extraction of maximally weighted independent set (MWIS), from the intersection graph over a family of segments generated from any low level segmentation algorithm. To handle them more formally we re-introduce the definition of partition graphs. This class of graphs enables a bijection between the MWIS of the intersection graph and the subsets that provide a partitions of the space. We end the chapter with a max-flow optimization problem on a tree and a graph structure adapted for BOPs.

Region merging methods review

Here we briefly analyse the emergence of super-pixel/region merging/hierarchical segmentation methods. In this bibliographic section we will overview a small family of hierarchical segmentation methods. Before this we will shortly provide a motivation on evolution of clustering methods in the domain of image segmentation.

There has been over few decades of study on the problem of image segmentation, and on using region based methods. For computationally tractable segmentation algorithms one often operates on pre-segmentation images. Thus the area of study is now the choice of pre-segmentation to produce a good leaves partition or super-pixel segmentation, the set of super-pixel features, to choose a discriminatory dissimilarity metric, as well as statistical measures and others, to recombine the regions. These properties of a good super-pixel algorithm in a way reflect the distinguishing properties of hierarchical agglomerative clustering(HAC) algorithms: Metric and Linkage.

Here we provide brief over of studies which compare now how different segmentation algorithms generating super-pixel and the subsequent merging orders or linkage criteria. The dissimilarity is usually represented as the weight on the edge set of a Region Adjacency Graph(RAG) associated with the super-pixel segmentation.

To begin with the super-pixel methods are generated not just for reduced complexity but also to the ends of using grouping super-pixels to produce a good segmentation of the image. One classical example one can site is the Normalized Cut based super-pixel segmentation approach to build limb and torso detectors [START_REF] Mori | Recovering human body configurations: Combining segmentation and recognition[END_REF], the outputs of which are assembled to classify human posture. One can find a good but brief summary of image segmentation by region merging methods in [START_REF] Szeliski | Computer Vision: Algorithms and Applications[END_REF].

Image segmentation has been formulated as a undersegmentation-agglomerative merge step. [START_REF] Hanbury | How do superpixels affect image segmentation?[END_REF] studies Volume based watershed, Mean-shift, and FH algorithm (Felzenszwalb and Huttenlocher) [START_REF] Felzenszwalb | Efficient graph-based image segmentation[END_REF] pre-segmentations, using a hierararhical agglomerative merging based on the euclidean distance between the coordinates of colors in CIELAB space.

The difficulty evident from the study are the different parameterizations to extract a presegmentation from each of the 3 methods. [START_REF] Guigues | The hierarchy of the cocoons of a graph and its application to image segmentation[END_REF] uses Priority queue based optimized agglomerative clustering algorithm, using the Hotelling T 2 statistics on region, again in the CIELAB color space. [START_REF] Calderero | Region merging techniques using information theory statistical measures[END_REF] again study statistical measures of the regions, the Kullback-Leibler Merging Criterion, that maximizes the probability of regions being generated from the same distribution. An interesting scale based measure is the image size dependent scale threshold given by T = α • I /n, where n is the number of regions, and I of the number of pixels in the image. This threshold is used with the KL merging criterion to control the scale of the partition produced, by varying parameter α. One also notes the work in producing hierarchical floodings based on the area, volume and dynamic of gray scale components, producing a hierarchical segmentations by Cousty et al. [START_REF] Cousty | Incremental algorithm for hierarchical minimum spanning forests and saliency of watershed cuts[END_REF].

The watershed-cuts as already remarked in [START_REF] Najman | On the equivalence between hierarchical segmentations and ultrametric watersheds[END_REF]109] with Soille's constrained connectivity are calculated rapidly thanks their use of single linkage clustering (SLC), in the sense of linkages in clustering algorithms, while that of Guigues Cocoons [START_REF] Guigues | The hierarchy of the cocoons of a graph and its application to image segmentation[END_REF] provides a complete linkage clustering (CLC).

Algorithms

In this section we compile the algorithms for calculating the minimal λ-cuts given a HOP or BOP, a Lagrangian parameter λ, the scale function Λ for each parent in a HOP or BOP. Finally given the Λ-function, one can calculate the family of optimal λ-cuts.

GIS(Geographic Information System

) based spatial segmentation problems, for example in Kurtz et al. [START_REF] Kurtz | Approche collaborative segmentation-classification pour l'analyse descendante d'images multirésolutions[END_REF].

Project with UMR ESPACE: The methods studied in this thesis have already been applied to problems in (Geographic information systems) GIS, in particular to spatio-temporal and morphological analysis population the region of Provence-Alpes-Côte d'Azur (PACA) south of France. This work is in collaboration with Christine

Voiron and the team at UMR ESPACE, Nice Sophia Antipolis. Problems purview include population density based hierarchical segmentation [START_REF] Baro | Hierarchical and multiscale Mean Shift segmentation of population grid[END_REF], multivariable hierarchical segmentations, autoroute proximity based segmentation and other problems involving hierarhical spatial structure and different types of constraints.

Multi-segmentations

Satellites and planes provide sequences of images in different wave bands. If there are many, like in airborne images, we suppose that they have been reduced to their first principal components. Each of these significant images is segmented individually. This results in the hierarchies

Abstract

Hierarchical segmentation has been a model which both identifies with the cosntruct of extracting a tree structured model of the image, while also interpretting it as an optimization problem of the optimal scale selection. Hierarchical processing is an emerging field of problems in computer vision and hyperspectral image processing community, on account of its ability to structure high-dimensional data.

Chapter 1 discusses two important concepts of Braids and Energetic lattices. Braids of partitions is a richer hierarchical partition model that provides mutliple locally nonnested partitioning, while being globally a hierarchical partitioning of the space. The problem of optimization on hierarchies and further braids are non-tractable due the combinatorial nature of the problem. We provide conditions, of h-increasingness, scaleincreasingness on the energy defined on partitions, to extract unique and monotonically ordered minimal partitions. Furthermore these conditions are found to be coherent with the Braid structure to perform constrained optimization on hierarchies, and more generally Braids. Chapter 2 demonstrates the Energetic lattice, and how it generalizes the Lagrangian formulation of the constrained optimization problem on hierarchies.

Finally in Chapter 3 we apply the method of optimization using energetic lattices to the problem of extraction of segmentations from a hierarchy, that are proximal to a ground truth set. Chapter 4 we show how one moves from the energetic lattice on hierarchies and braids, to a numerical lattice of Jordan Curves which define a continous model of hierarhical segmentation. This model enables also to compose different functions and hierarchies.

Keywords: Hierarchical segmentation, Lagrangian Multipliers, Lattice optimiztaion, Mathemtical Morphology.

Résumé

La segmentation hiérarchique est une méthode pour produire des partitions qui représentent une même image de manière de moins en moins fine. En même temps, elle sert d'entrée à la recherche d'une partition optimale, qui combine des extraits des diverses partitions en divers endroits. Le traitement hiérarchique des images est un domaine émergent en vision par ordinateur, et en particulier dans la communauté qui étudie les images hyperspectrales et les SIG, du fait de son capacité à structurer des données hyper-dimensionnelles.

Le chapitre 1 porte sur les deux concepts fondamentaux de tresse et de treillis energétique. La tresse est une notion plus riche que celle de hierarchie de partitions, en ce qu'elle incorpore, en plus, des partitions qui ne sont pas emboîtées les unes dans les autres, tout en s'appuyant glolalement sur une hiérarchie. Le treillis énergétique est une structure mixte qui regroupe une tresse avec une énergie, et permet d'y défninr des éléments maximaux et minimaux. Lorsqu'on se donne une énergie, trouver la partition formée de classes de la tresse (ou de la hiérarchie) qui minimise cette énergie est un problème insoluble, de par sa complexité combinatoriale. Nous donnons les deux conditions de h-croissance et de croissance d'échelle, qui garantissent l'existence, l'unicité et la monotonie des solutions, et conduisent à un algorithme qui les détermine en deux passes de lecture des données.

Le chapitre 2 reste dans le cadre précédent, mais étudie plus spécifiquement l'optimisation sous contrainte. Il débouche sur trois généralisations du modèle Lagrangien. Le chapitre 3 applique l'optimisation par treillis énergétique au cas de figure où l'energie est introduite par une "vérité terrrain", c'est à dire par un jeu de dessins mauel, que les partitions optimales doivent serrer au plus près.

Enfin, le chapitre 4 passe des treills énergétiques à ceux des courbes de Jordan dans le plan euclidien, qui définissent un modèle continu de segmentations hierarchiques. Il permet entre autres de composer les hiérarchies avec diverses fonctions numériques.