Nicolas Maillot
email: nicolas.maillot@sophia.inria.fr

Monique Thonnat
email: monique.thonnat@sophia.inria.fr

Ontology Based Complex Object Recognition

Keywords: Ontology, Machine Learning, Categorization, Cognitive Vision

This paper presents a new approach for object categorization involving the following aspects of cognitive vision: learning, recognition and knowledge representation. A major element of our approach is a visual concept ontology composed of several types of concepts (spatial concepts and relations, color concepts and texture concepts). Visual concepts contained in this ontology can be seen as an intermediate layer between domain knowledge and image processing procedures. Machine learning techniques are used to solve the symbol grounding problem (i.e. linking meaningfully symbols to sensory information). This paper shows how a new object categorization system is set up by a knowledge acquisition and learning phase and then used by an object categorization phase.

Introduction

This paper presents a cognitive vision approach designed for complex object categorization. Both knowledge representation and machine learning techniques are involved in the categorization process. The proposed approach is designed for semantic interpretation of isolated objects of interest. Related work on scene analysis issues (i.e. involving non isolated objects) can be found in [START_REF] Hudelot | A cognitive vision platform for automatic recognition of natural complex objects[END_REF].

A long experience in complex object categorization [START_REF] Thonnat | Knowledge-based galaxy classification systems, Knowledge-based systems in astronomy[END_REF] (3) has shown that experts often use a well defined vocabulary for describing the object of their domain. Based on that statement, results coming from the knowledge engineering community can be applied to acquire expert knowledge. This expert knowledge is used to guide object recognition.

Our approach consists of using a conceptualization of the visual perception domain (a visual concept ontology) to define object recognition strategies. Moreover, our approach can be brought close to the procedural semantics [START_REF] Miller | Language and Perception[END_REF] theory where visual concepts and their labels are associated in the sense that they are alternative ways of gaining access to same underlying procedures (segmentation, feature extraction and learning).

Section 2 gives an overview of key issues and existing approaches in high-level image interpretation. Section 3 gives a global point of view on the proposed approach. Section 4 details the structure of a priori knowledge (i.e. domain knowledge and visual concept ontology) involved in the recognition process. Section 5 is dedicated to image processing techniques used for achieving categorization. Section 6 explains how visual concepts are learnt by learning techniques. Section 7 presents the proposed object learning and categorization algorithms. A discussion on our approach is given in section 8. We conclude in section 9.

Related Work

In [START_REF] Mardia | Shape in images[END_REF], an introduction to high-level Bayesian image interpretation techniques can be found. The author explains that Bayesian analysis techniques are more widely applicable and reliable that ad hoc algorithms. Such statistical models allow to evaluate confidence about conclusions. The difficult task in the construction of Bayesian models is to define prior distribution. In particular, context (e.g. point of view, scale, acquisition conditions) is hard to manage. That is why care is required in using statistical knowledge effectively for a given specific problem. A solution to invariance problems are local invariant image feature detectors [START_REF] Lowe | Object Recognition From Local Scale-Invariant Features[END_REF]. In [START_REF] Csurka | Visual categorization with bags of keypoints[END_REF], local features are used to achieve class recognition by using machine learning techniques. These kinds of approaches rarely take into account expert knowledge in an explicit way.

Knowledge based vision systems have proven to be effective for complex object recognition [START_REF] Thonnat | Knowledge-based galaxy classification systems, Knowledge-based systems in astronomy[END_REF] and for scene understanding [START_REF] Matsuyama | SIGMA -A Knowledge-Based Aerial Image Understanding System[END_REF]. They offer a great capacity of reusability and extendability. Moreover, in knowledge based systems, domain knowledge is clearly separated from image processing knowledge. This implies a better tractability of the different sub-problems (i.e. image processing and interpretation) encountered in image understanding. The major negative point of these systems is that they rely on knowledge bases which are difficult to produce and manage.

To achieve complex object recognition, we propose an hybrid approach mixing knowledge-based and learning techniques. The role of the knowledge-based techniques is to make explicit domain knowledge in a symbolic knowledge base.

The role of the learning techniques is to enrich this symbolic knowledge base in order to bridge the gap between high-level symbols and numerical image features. Our objective is both to reduce the effort of knowledge acquisition and to enable semantic object categorization.

Proposed Approach

Our approach is composed of two main phases. Phase one consists of the set up of a new object categorization system by knowledge acquisition and learning. Phase two consists of using the new system in an object categorization phase. This section gives an overview on the whole approach.

Knowledge Acquisition and Learning Phase

Knowledge Acquisition

First comes knowledge acquisition issues which have been discussed in [START_REF] Maillot | Towards ontology based cognitive vision[END_REF]. This knowledge acquisition phase is done by interaction with an expert of the application domain. Usually, experts are not skilled in computer vision but are able to produce an accurate visual description of the objects of their domain.

A dedicated knowledge acquisition tool has been implemented and allows to perform the following tasks:

• domain taxonomy acquisition. This domain knowledge contains both the specialisation and part-whole relations between the domain classes. This knowledge which is shared by the specialists of the domain is easy to acquire. This part is independent of vision (visual appearance and image acquisition context). • Ontology driven visual description of domain object classes which leads to a more detailed symbolic knowledge base. This knowledge is dependant of the image acquisition context (e.g. camera, lightning conditions). • Image sample management (i.e. annotation and manual segmentation of samples of object classes of interest)

As seen in fig. 1, the knowledge acquisition process leads to a knowledge base in which a set of domain classes are described by visual concepts. Manually segmented and annotated image samples of domain objects are also obtained. Sample annotation consists of labeling a set of manually segmented images (fig. 4) by visual concept names. This means that each manually segmented region is associated with one or several visual concepts.

Learning

During the knowledge acquisition process, both expert knowledge and image samples have been provided. The role of learning is to fill the gap between symbols used during knowledge acquisition and manually segmented and annotated sample images. As seen in fig. 2, three tasks are involved in the object learning process.

(1) Object learning is initiated by a learning request which contains a list of classes of the domain taxonomy. For specific applications, some classes are not relevant. Therefore, it may be needed to restrict learning process to a subpart of the whole domain knowledge. Object learning needs to retrieve manually segmented image samples (i.e. a set of regions of interest annotated by visual concepts). ( 2

Categorization Phase

Fig. 3 gives an overview of the proposed object categorization phase. This phase is based on three tasks:

(

A Priori Knowledge

A Visual Concept Ontology

As defined in [START_REF] Gruber | Towards Principles for the Design of Ontologies Used for Knowledge Sharing[END_REF], an ontology is a formalization of a conceptualization. An ontology is composed of:

• a set of concepts (C) (e.g. geometric concepts)

• a set of relations(R) (e.g. spatial relations)

• a set of axioms (e.g. transitivity, reflexivity, symmetry of relations)

Two partial orders C and R define the concept hierarchy and the relation hierarchy, respectively.

In this section, we propose a visual concept ontology. The role of this ontology is to provide a shared vocabulary and semantics between domain knowledge and vision.

This ontology is a guide which provides a vocabulary for the visual description of domain classes. It is important to note that the proposed ontology is not application-dependent and is an extendable basis. We have structured this ontology in three main parts. The first one contains texture concepts, the second one contains color concepts and the last one is made of spatial concepts.

The proposed ontology is used to describe the visual appearance of the objects of interest in a given context. For a new context (e.g. different imaging conditions), visual description has to be changed but domain knowledge remains the same.

Each part of this ontology is detailed in the next subsections.

Texture Concepts

This part of the ontology has been inspired by results from the cognitive science community.

The experiment conducted in [START_REF] Rao | Towards a texture naming system: Identifying relevent dimensions of texture[END_REF] identifies three main dimensions in the texture perception cognitive process. Each perceptual dimension constitutes an important element in texture perception. Each perceptual dimension is seen as an abstraction of a set of visual concepts (Fig. 5). From this study, we have built an ontology of texture concepts. Note that quantifiers (e.g. N on, Average, Strong) are also integrated in this ontology and can be used to give a finer visual description.

Color Concepts

This part of the ontology is derived from the ISCC-NBS (Inter-Society Color Council-National Bureau of Standards) color dictionary. An interesting reflexion on the validity of this dictionary is given in (4). Three kinds of notions are included: hue, brightness and saturation concepts. There are 28 hue concepts (Table 1) which can be combined with five brightness concepts (Very Dark, Dark, Medium, Light, Very Light) and four saturation concepts (Grayish, Moderate, Strong, Vivid ). Certain combinations of brightness and saturation concepts have a perceptual meaning. For instance, the concept Brillant is an association of the Light and Strong concepts. Axioms are contained in the ontology so as to express those kinds of associations.

These visual concepts (e.g. Brightness concept) have no absolute definition. The experts use these terms to discriminate object classes of interest. We will see in Section 6 how these values are computed.

Spatial Concepts

This part of the ontology is used for describing domain objects from a spatial point of view. A part of the hierarchy is composed of geometric concepts that can be used to describe the shape of domain objects (fig. 6). Three other parts of the ontology contain position, orientation and size concepts. A formalization of a similar approach based on a combination of geometric shapes can be found 1 Set of hue concepts in [START_REF] Sciascio | Structured knowledge representation for image retrieval[END_REF]. The size of an object can also be described and quantified with a set of quantifiers. Note that quantification can be done in an absolute way or relatively to another concept. This means that the size of object A can be described as being important relatively to object B. The notion of elongation is also present and can be quantified. We have also added a set of spatial relations based on the RCC-8 model ( 13) that can be used to define relations between each objects and its subparts. These relations are enumerated in Table 2 and graphically represented in fig. 7. Table 3 shows an example of spatial relations between one object class (Poaceae) and its subpart (Pori1).

Image Acquisition Context

Experts often observe the objects of their domain in precise observation conditions. For example, when using a microscope, magnification or lighting conditions are controled. Providing image acquisition contextual information is absolutely necessary. Image acquisition context information is the link between domain knowledge and image samples resulting from the acquisition process. It conditions the resulting acquired images. This implies a relation between the visual description of image samples and the context of acquisition. This knowledge avoids building incoherent sets of image samples. For instance, it would not make sense to gather images of a similar object acquired with different types of sensors. Image acquisition context depends on the application

DC(X,Y) EC(X,Y) TPP(X,Y) TPP-1(X,Y) PO(X,Y) EQ(X,Y) NTPP(X,Y) NTPP-1(X,Y) Y X X Y X Y X Y Y X X Y X Y Y X

Domain Knowledge

During the knowledge acquisition phase (section 3.1), the expert of the domain (e.g. biologists, astronomers) produces a knowledge base containing the domain taxonomy and the visual description of the classes.

The domain taxonomy contains both the specialisation and part-whole relations between the domain classes. The domain classes can be structured in one or several hierarchies in function of the needs. This knowledge which is shared by the specialists of the domain is easy to acquire. This part is independent of vision (visual appearance and image acquisition context) and can be reused for other purposes. An example is given in fig. 9.

The visual description of the domain classes is driven by the ontology presented in the previous section (4.1). This knowledge is dependant of the appearance of the objects and thus of the image acquisition context (e.g. camera, lightning conditions). Visual concepts feature attributes are computed by feature extraction. For instance, numerical features associated with the concept TextureConcept are statistical moments and Gabor Features. Color characterization is done with histograms and color coherence vectors [START_REF] Pass | Comparing images using color coherence vectors[END_REF]. These features are used to learn and recognized visual concepts, this learning process is described in the next section.

6 Visual Concept Learning

Overview

Although visual concept semantics is provided by the visual concept ontology, symbol grounding still has to be performed. Visual concept learning is used ...

}

Table 4 An example of the visual concept GeometricConcept. Some geometric features are given. These features are used during the visual concept learning process. Restrictions on the domain of the features are also defined.

to fill the gap between ontological concepts and image level. Visual concept learning consists of learning a set of classifiers for the recognition of each visual concept. This learning is done thanks to a set of training vectors computed by feature extraction on manually segmented and annotated regions of interest. These features vectors are included in the visual concept learning request. The visual concept ontology is used in this process because the learning process is done in a hierarchical way by using the ontological tree structure. Visual concept learning is focused on specific problems. Therefore, a visual concept is also included in the request so as to define the subpart of the ontology which has to be learnt. For instance, if the visual concept GeometricConcept is inside the request, only this concept and its children are involved in the learning process.

Problem Statement

The proposed architecture is designed to learn a set of visual concepts used during knowledge acquisition. A training set T i is associated with each visual concept C i ∈ C. A training set is a set of N labeled vectors x i ∈ R n computed during a feature extraction step described in section 3. Vectors are labeled by y i ∈ {-1, 1}. y i = 1 means that x i is a representative sample of C i . y i = -1 means that x i is a negative sample of C i . More precisely, negative samples of C i are positive samples of brothers of C i . This approach is a one-versus-rest classification scheme. This means that we use the hierarchical structure of the ontology to obtain simpler and focused classification problems.

A classifier d i is associated with each concept C i (see table 5). P (C i |x) is the posterior probability of C i . P (¬C i |x) is the posterior probability of ¬C i . We also introduce a reject distance class R i so that P (C i ) + P (¬C i ) + P (R i ) = 1. This distance reject class allows to take into account vectors observed in unexpected regions of R n . The probability law of x is defined as p(x) = p(x|C i )P (C i ) + p(x|¬C i )P (¬C i ) + p(x|R i )P (R i ).

We define two thresholds α amb ∈]0.5, 1[ and α dist ∈]0, 1[. α amb is the ambiguity reject threshold and defines the degree of confidence needed to take the decision of recognizing a concept. α dist is the distance reject threshold. Distance reject is inferred from p(x) and α dist and classifies x into R i . Distance reject means that x is unlikely to belong to both C k and ¬C k and might belong to a concept that has not been learnt yet. For more details about the distance reject notion, see [START_REF] Dubuisson | A statistical decision rule with incomplete knowledge about classes[END_REF].

d i (x) Definition C i recognized P (C i |x) ≥ α amb ¬C i recognized P (¬C i |x) ≥ α amb Ambiguity reject max{P (C i |x), P (¬C i |x)} < α amb Distance reject p(x) < α dist Table 5 Decision types
The following subsections aim at presenting a methodology designed to build each classifier d i . Visual concept learning consists of: training set building, feature selection and training (Fig. 10). Each of these steps is detailed in the next subsections. The main algorithm is given in algorithm 1. This algorithm is executed by a visual concept learning request. It sequentially performs training set building, feature selection and training. This algorithm takes two parameters. The first one is a set of feature vectors labeled by visual concepts. The second one is a visual concept C i (e.g. GeometricConcept, HueConcept) which defines a subpart of the ontology. The result of algorithm 1 is a set of classifiers trained to the recognition of C i and its children. or -1) for each C i . This implies that the hierarchical structure of the ontology has to be used to compute each T i .

Algorithm 1 V isualConceptLearning(X, C i ) T rainingSetBuilding(X, C i , T ) F eatureSelection(C i , T, T ) T raining(C i , T ,
             P i = j {(x j , +1) | C j C C i } N i = j {(x j , -1) | C j C (C k ∈ brothers(C i )) ∧ (x j , +1) / ∈ P i } T i = P i ∪ N i 6.4 Feature Selection
Feature selection is hierarchically performed by Algorithm 2. FeatureSelection-Algorithm function applies a feature selection algorithm to each T i to obtain each T i . We currently use a Sequential Forward Floating Selection (SFFS) Algorithm [START_REF] Pudil | Floating search methods in featureselection[END_REF]. This method iteratively adds or removes features until some termination criterion is met. The Bhattacharyya distance [START_REF] Choi | Feature extraction based on the bhattacharyya distance[END_REF] between classes is used as a separability criterion. This implies that features used to recognize a visual concept may be different from features used for the recognition of another concept.

Algorithm 2 F eatureSelection(C i , T, T ) children ← getChildren(C i ) for all C j ∈ children do T j ← F eatureSelectionAlgorithm(T j ) T ← T ∪ T j if hasChildren(C j ) = true then F eatureSelection(C j , T, T ) end if end for

Training

As detailed in Algorithm 3, training is guided by the hierarchical structure of the ontology. Algorithm 3 is initially called with a visual concept C i such as GeometricConcept and T = {T i }. The set of classifiers D = {d i } is built recursively. Learning of each descendant of C i is performed hierarchically. The trainClassifier function first loads T i , then creates and trains a binary classifier d i to the recognition of C i . We currently use multi layer perceptrons and k nearest neighbors as binary classifiers. Next section shows how {di} is used to learn and recognize domain classes.

Algorithm 3 T raining(C i , T , D) children ← getChildren(C i ) for all C j ∈ children do d j ← T rainClassif ier(T j ) D ← D ∪ d j if hasChildren(C j ) = true then T raining(C j , T , D) end if end for

Results

This subsection illustrates an application of the classifiers trained by visual concept learning. A cognitive experiment has been performed in [START_REF] Rao | Towards a texture naming system: Identifying relevent dimensions of texture[END_REF]: a subset of 56 Brodatz texture images has been given to 20 persons who were asked to classify them in different clusters. A few samples of the Brodatz texture set are given in fig. 11. The clusters were formed by evaluating the following symbols (between 1 and 9): contrast, repetitveness, granularity, randomness, roughness, density, directionality, complexity, coarseness, regularity, orientation.

To perform the learning process described in this section, we have used texture related visual concepts associated with each Brodatz image. Each image is split in 16 pieces so as to obtain 128x128 images. Available texture image analysis algorithms (i.e. Gabor filters, auto-correlation matrices and cooccurence matrices

Object Learning and Categorization

Object Learning Algorithm

As seen in fig. 12, the learning process of domain classes is composed of two main steps. This algorithm is executed during object learning (section 3) and learns the visual concepts used as values of domain classes attributes (e.g. Size, Hue, P attern).

(1) The first step consists of extracting features associated with attribute values (e.g. C i =CircularSurface). For each attribute value, a set of positive feature vectors is obtained. The set of negative training vectors is labeled by not(C i ). A recursive call is needed in order to learn the description of subparts of the current class. (2) The second step is visual concept learning which has been described in the previous section. Algorithm 1 is called for each category of visual concept (i.e. spatial attributes: Geometry, Size, Orientation, P osition; color attributes: Hue, Brightness, Saturation; texture attributes: Repartition, Contrast, P attern).

Object Categorization Algorithm

The object categorization algorithm structure is given in fig. 13. This algorithm is divided in five steps. It tries to match an unknown object to be categorized with one class of the domain. Matching is first performed at a local level. This local matching consists of comparing expected visual concepts and visual concepts computed on the unknown object to be classified. The global matching consists of combining the local matchings performed at the local level. For an unknown object, the categorization results contain the name of the class which matches the best the object, the confidence degree of the matching, and the values of the recognized visual concepts. For instance , Class = P oaceae, conf idenceV alue = 0.9, Hue = P ink, Size = ImportantSize, P attern = GranulatedT exture.

Discussion

The proposed approach enables semantic and explicit object categorization. The global architecture does not act as a black box and is able to explain categorization results. The explicit nature of categorization is useful to achieve semantic image indexing and retrieval. In [START_REF] Maillot | A weakly supervised approach for semantic image indexing and retrieval[END_REF], we present how this approach has been applied to the domain of transport vehicles (i.e. motorbikes, aircrafts, cars) in their environment. The results obtained show that our methodology leads to efficient indexing combined with semantic richness: for a recall of 0.5, precision is between 0.75 and 0.78 for the domain classes Aircraf t, M otorBike and Car and of 0.90 for class Sky. Precision is defined as the ratio between the number of relevant retrieved images and the number of retrieved images.

Recall is defined as the ratio between the number of relevant retrieved images and the number of relevant images in the image database. 1850 images have been indexed for this experiment.

One strong point is the modularity of the approach. New algorithms can be integrated for improving segmentation and feature extraction. The same goes for visual concept learning which currently uses k-nn, multi layer perceptrons and support vector machines. Changes in the low-level part of the architecture has no consequence on the high-level part. One of the main advantages of our approach is that the visual concept ontology acts as a user-friendly intermediate between the image processing layer and the expert. Another interesting aspect is that the ontology can be used to describe objects of different domain of expertise. It is up to the learning layer to ground symbolic concepts in a different way.

At the segmentation level, a major remaining challenge is to define precisely the feedback to the segmentation level when object categorization fails. We are also planning to use program supervision techniques (see ( 14)) combined with learning techniques to improve segmentation quality. We have explained that visual concepts are included in segmentation requests. Trained classifiers associated with these visual concepts should be used to validate segmentation.

The proposed approach leans on a knowledge acquisition phase which has to be as complete as possible in order to make the learning process and the cat-egorization phase efficient. This raises the question of the balance that has to be found between knowledge which has to be provided by the expert and knowlege deduced from images samples. In the current implementation (see [START_REF] Maillot | Towards ontology based cognitive vision[END_REF]), the expert has to provide an important amount of knowledge. An interesting future direction is the use of unsupervised learning techniques which could guide the expert by automatically filling class attributes. Indeed, similarity between samples could be used to infer similiar attribute values between classes.

Conclusion

This paper presents an original approach to complex object categorization. We propose an hybrid approach mixing knowledge-based and learning techniques. The role of the knowledge-based techniques is to make explicit domain knowledge in a symbolic knowledge base. The role of the learning techniques is to enrich this symbolic knowledge base in order to bridge the gap between high-level symbols and numerical image features. Our objective is both to reduce the effort of knowledge acquisition and to enable semantic object categorization.

Our approach is structured in two main phases. Phase one consists of the set up of a new object categorization system by knowledge acquisition and learning. During knowledge acquisition, a human expert of the application domain describes a set of domain classes and provides a set of annotated image samples. This knowledge is then completed by the visual description of the classes in terms of concepts belonging to a pre-defined visual concept ontology. This ontology is composed of the following types of visual concepts: spatial concepts and relations, color concept and texture concepts. The result is a domain knowledge base. Visual concept learning follows the knowledge acquisition process in order to obtain a knowledge base augmented by a set of classifiers trained on the image samples to the recognition of the visual concepts used for the description of each classes.

Phase two consists of using the new system in an object categorization phase. The categorization phase matches an unknown object with one domain class. The matching is done between visual concepts computed on the unknown object and visual concepts used for the description of domain classes.

There are several remaining issues. It is important to mention that the visual description depends on the appearance of the objects captured in a given image acquisition context. For a new image acquisition context, visual description has to be changed but domain knowledge remains the same.

A good object segmentation as well as good subpart segmentations are needed in this approach.

For some specific applications, this hypothesis is reasonable. In general, segmentation remains a major issue. We plan to use visual concepts, program supervision and learning techniques to deal with this problem. The visual concept ontology provides an efficient guide for knowledge acquisition but unsupervised learning techniques could make this task easier.

Fig. 1 .

 1 Fig. 1. Knowledge Acquisition Phase Overview

  ) Feature extraction is initiated by feature extraction requests sent during object learning and computes features on segmented samples. (3) Visual concept learning consists of training a set of classifiers by using features extracted by feature extraction. These classifiers are trained to the recognition of visual concepts used for the description of domain classes. The output of object learning is a knowledge base augmented with the trained classifiers.

Fig. 4 .

 4 Fig.4. Image of an isolated pollen grain on the left. On the right, manual segmentation associated with the image. Three different regions are considered: the background in black, the pollen grain in white and one of its subparts in grey.

Fig. 5 .

 5 Fig. 5. Texture concept hierarchy

Fig. 7 .Fig. 8 .

 78 Fig. 7. RCC-8 graphical representation

Fig. 9 .

 9 Fig. 9. Domain knowledge structure: on the left, a taxomony, on the right a partonomy An example of a domain class formalized with the frame formalism can be

  Fig. 11. Brodatz texture samples

Fig. 12 .

 12 Fig. 12. Simplified version of object learning algorithm

( 1 )

 1 The categorization process is initiated by a categorization request which contains an image of the object to be classified.(2) The object of interest has to be segmented from background. If the algorithm tries to classify a subpart, the segmentation task consists of ex-tracting the subpart from the main object. In both cases, a segmentation request has to be sent to perform segmentation. (3) Then comes local matching between current class attribute values (e.g. CircularSurf ace for attribute Geometry) and visual concepts recognized by the classifiers trained during the learning process. Features used for visual concept recognition are computed by feature extraction. The result of local matching is a set of probabilities associated with each attribute value. If the attribute is a subpart, a recursive call has to be made so as to categorize it. (4) This step consists of evaluating if current class matches the object to be recognized. This matching is done by combining probabilities computed during local matching. (5) If the object matched the current class, the classification algorithm tries to go deeper in the domain class hierarchy. If matching fails, the current class is dropped.

Fig. 13 .

 13 Fig. 13. Simplified Version of object categorization algorithm

Table 3

 3 High level description of domain class poaceae. Visual concepts provided by the ontology are in italic. Attribute names are in bold face. Knowledge provided by the expert is in small caps.Segmentation needs have been introduced in section 3. It is not the goal of this paper to give many details on this specific problem. Let us assume that the segmentation consists of performing two different types of segmentation tasks. The first segmentation task is isolation of the object of interest from the background. The second segmentation task is the extraction of object subparts.A segmentation request is composed of two elements: a list of visual concepts and the region of interest where segmentation is performed. Segmentation is guided by visual concepts included in the segmentation request. The reader can refer to (2) and (14) to understand how program supervision techniques are used to control and reuse image processing algorithms.5.2 Feature Management and ExtractionSection 4 has introduced the structure of the proposed visual concept ontology. During the classification of a given object, numerical features are computed. To be interpreted as visual concepts, a link must be established between computed numerical features and symbolic visual concepts. As explained in subsection 4.2, several categories of visual concepts are managed: Geometry; Size;

	5 Image Processing
	5.1 Segmentation

found in table

3

. The SuperClass attribute allows the definition of specialization relations. The SubParts attribute defines part-whole relations. There are three categories of attributes related to visual description:

(1) spatial attributes: Geometry; Size; Orientation; P osition (2) color attributes: Hue; Brightness; Saturation (3) texture attributes: Repartition; Contrast; P attern Each attribute has a defined type and a set of possible values. For instance, attribute Hue is of type HueConcept and has for value Pink. The relation between possible values and attribute types is a specialization relation. Orientation; P osition; Hue; Brightness; Saturation; Repartition; Contrast; P attern. We propose to create a sets of numerical features associated with these visual concepts.

Numerical features depend on available feature extraction algorithms. It is up to the image processing expert to make relevant association between available feature extraction algorithms and visual concepts. As seen in table 4, the concept GeometricConcept can be represented in a frame formalism. Attributes of a given visual concept are inherited by its subconcepts (e.g CircularSurface).

  P i is the set of representative training vectors of a visual concept C i . N i is the set of training vectors computed on negative samples of a visual concept C i . The training set associated with C i is noted T i . Training set building aims at computing T i (feature vectors labeled by +1

			Visual
			Concept
			Learning
			Request
			X={xi,Ci}
		Visual Concept Ontology	Training Set Building
			T={Ti}
			Feature
			Selection
			T'={Ti'}
			Training
			Trained
			Detectors
			D={di}
		Fig. 10. Visual concept learning
	6.3 Training Set Building
	return D	D)

As explained in previous subsections, each visual concept used during knowledge acquisition has an attached classifier. The first step is to build training sets to train classifiers associated with visual concepts. For each visual concept used during knowledge acquisition, image processing algorithms compute a set of feature vectors. Regions of interest are used as an input of feature extraction.

As shown in Fig.

10

, visual concept learning is designed to process a visual concept learning request. The set of training vectors contained in a visual concept learning request (X = {x i , C i }) is computed by feature extraction (see Fig.

2

). Vector labels are provided by the ontology driven description of domain classes. Note that two similar vectors can be labeled by several visual concepts: we are in a multi-label case. The labeled vectors are then processed during training set building.