Alexandre Guimond
email: guimond@iro.umontreal.ca

Gérard Subsol

Jean Meunier
email: meunier@iro.umontreal.ca

Jean-Philippe Thirion
email: thirion@sophia.inria.fr

G ?rard Subsol

Automatic Exploration and Morphometry/Morphology Assessment of Medical Image Databases

Keywords: image database, exploration, volume of interest (VOI), average patient, medical atlas, classi cation, registration, magnetic resonance imaging (MRI)

The design of representative models of the human body is of great interest to medical doctors. Qualitative information about the characteristics of the brain is widely available, but due to the volume of information that needs to be analyzed and the complexity of its structure, rarely is there quanti cation according to a standard model. To address this problem, we propose in this paper an automatic method to retrieve corresponding structures from a database of medical images. This procedure being local and fast, will permit navigation through large databases in a practical amount of time. We present as examples of applications the building of an average volume of interest and preliminary results of classi cation according to morphology.

Neurological disorders like schizophrenia or Alzheimer's disease have for long been the subject of serious studies. It is believed that such conditions could be coupled with abnormal congurations of di erent brain structures, but strong quantitative evidence has still to be reported. Although anatomical brain atlases 1{4 provide information to analyze and compare brain structures, their characteristics, such as precise shape and variance among healthy subjects, are not yet clearly de ned. Moreover, paper representations are not su cient to answer today's problems in this eld [START_REF] Mazziotta | A probabilistic atlas of the human brain: Theory and rationales for its development[END_REF][START_REF] Subsol | Construction automatique d'atlas anatomiques morphom?triques ? partir d'images m?dicales tridimensionnelles[END_REF] ; such atlases are two dimensional, static and based on one or few studies. Computer guided diagnosis, multimedia medical teaching or surgical simulation robotics require more adaptable and complete sources of information. Over recent years, the use of magnetic resonance imaging (MRI) has produced huge medical brain image databases. Study of these data could provide the information needed to build numerical atlases with quanti cation of brain structure characteristics. This article addresses the problem of exploring such databases to retrieve information about a speci c part of the brain (see Figure 1). The idea is to provide tools for practitioners to de ne a volume of interest (VOI) within a patient's MRI and extract corresponding VOIs from the database. These VOIs can be viewed as either control subjects or representative elements of di erent classes of pathology. In the rst case, contrast images would facilitate the identi cation of signi cant abnormalities in the patient's VOI. In the second, comparisons would reveal which pathologies the patient is most related to, with associated probabilities. For example, to study temporal lobe epilepsy, we could use a database composed of representatives of a normal subject, of an epileptic with an atrophied left hippocampus and of an epileptic with an atrophied right hippocampus. Following the extraction of the hippocampus and comparisons, a diagnostic would be automatically produced putting forward a nities between the patient and the database elements.

The heart of such an extraction procedure is a matching method that nds correspondences between the patient's brain and the database elements. Once the VOIs extracted, we can apply di erent kinds of processing: automatic measure of features, shape extraction and comparison, factor analysis, statistical ordering, etc. . . Thus, we present a new method to evaluate di erences between VOIs and obtain quantitative information from the database.

The rst part of this paper reports di erent techniques used to compare brains. The second part deals with brain structure di erences between individuals and the type of information they convey. We present our method in a third section, and explain how to identify important di erences between images depending on the kind of information that we seek. The fourth part is an overview of possible applications including the construction of an average patient and some preliminary results of classi cation according to morphology. We conclude the paper with a brief discussion of future work.

MATCHING METHODS

Matching a model image I m with a scene image I s , using a transformation class T and a similarity measure S, can be formalized as the process of nding the function M that minimizes S(M(I m ); I s ). The application of this function to the model image M(I m ) will be called a mapping, or equivalently a warping, of I m onto I s . The class of transformation T used is also of importance. Various approaches exist. To name a few, registration methods have been developed using linear, 16{18 piecewise linear, [START_REF] Lemoine | An anatomical-based 3D registration of multimodality and atlas data in surgery[END_REF] quadratic [START_REF] Greitz | A computerized brain atlas: Construction, anatomical content, and some applications[END_REF] and free-form [START_REF] Thirion | Fast non-rigid matching of 3D medical images[END_REF][START_REF] Collins | An automated 3D non-linear deformation procedure for determination of gross morphometric variability in human brain[END_REF] classes (see Figure 2 for a rough classi cation).

Previous work

Rigid

Quadratic

Permissive

Affine Talairach

Free-form

Restrained

A very common approach is to use the Talairach reference space, also known as the three-dimensional proportional grid system. [START_REF] Talairach | Co-Planar Stereotaxic Atlas of the Human Brain[END_REF] This system uses the intercommissural line and some other anatomical landmarks to partition the brain into twelve subspaces and is an attempt to model non-linear di erences between data sets.

We can generalize the idea of Talairach by nding more landmarks, which leads to additional subdivisions of the brain volume and make the transformations more local. The di culty lies in the automatic detection of the landmarks. A promising solution relying on di erential geometry criteria can be found in Ref. 21.

D r a f t

N o v e m b e r 2 0 , 1 9 9 7 ( 0 9 : 5 9 )

Our matching method

The matching method we developed di ers from previous work by con ning itself to a VOI. This has several advantages. It corresponds to doctors' need of studying particular brain structures. It also re ects the assumption that gross anatomical shape is only in uenced by neighboring bodies. Finally, the database investigation must be fast. Using sub-images will help us achieve this end.

We propose a new method to quantify di erences between subjects in a localized volume of the brain using a three step approach that progressively re nes the matching:

1. A global matching between the entire data sets. 2. A regional matching between corresponding VOIs. 3. A local matching in a voxel's neighborhood.

The matching procedure we use works as follows: given two images and a transformation class T, the algorithm delivers a mapping function M to warp a model image I m onto a scene image I s . In order to be less sensitive to the intensity values, a global bias and gain between the intensities of the two images can also be estimated by the algorithm. This gives us results qualitatively similar to Refs. 22,23 but with an implementation one or two orders of magnitude faster. More details can be found in Refs. 12,24. Variations between corresponding structures of di erent subjects can be separated in two categories:

Morphological These are di erences due to the presence or the absence of a certain feature. For example, Figure 3 shows two di erent patients with morphological variations. The subject on the left has two gyri and the one on the right only one. These are non-trivial di erences since they require to deduce the location of absent features.

Morphometrical These are variations due to the di erent properties of existing features. For example, the length of a gyri or the volume of the ventricles (Figure 3 presents two patients with di erent ventricle volumes). For the purpose of this article, morphometrical variations will be divided in three categories:

Global Brain scale morphometrical variations.

Regional VOI scale morphometrical variations.

Local Voxel scale morphometrical variations.

Irrelevant variations

As previously mentioned, our interest lies in the study of particular brain structures. In that respect, it is generally agreed that for the vast majority of subjects, global morphometrical di erences, such as whole brain width or length, are not really signi cant. Furthermore, this idea can hold for di erences between su ciently large VOIs. Our registration method will re ect those assumptions by correcting for a ne di erences at global and regional scales. This class of transformation will correct translation, rotation, scale and skew di erences between VOIs.

Choosing this transformation class has some advantages over the Talairach alternative. The proportional grid system is a reference space that is frequently used by medical doctors, which is an important asset since they are the end-users. But it shows poor accuracy when matching structures away from the anterior and posterior commissures and limits the extraction procedure to the brain. It is worth noting that correcting for a ne di erences between VOIs is in agreement with Talairach's philosophy of not considering structures away from the ones of concern. Since they are both based on linear transformations, their di erences can be estimated by looking at the number of free parameters they hold. A Talairach conversion is de ned by 1 3D point, 1 distance (from AC to PC), 1 rotation around each axis and 6 scaling factors for a total of 13 unknowns. An a ne transformation accounts for 12 parameters: 1 3D translation and for each axis, 1 rotation, 1 skew and 1 scaling. So there is only 1 degree of freedom di erence between them. As mentioned earlier, our method is divided into three steps: (1) global, (2) regional and (3) local matching to evaluate global, regional and local morphometrical di erences respectively. These steps are detailed below and displayed in Figure 4.

EXTRACTION OF VOI

Global morphometrical corrections

The global correction procedure consists in estimating an a ne transformation to map the database entry's dimensions to the ones of the reference image. This step is necessary to extract roughly corresponding VOIs needed for regional comparisons in the second step. To reduce computation, the matching is done on sub-sampled images, a justi able approach since we are only looking for a rough correspondence.

Before registration, we apply to each of the two images the following transformations:

1. A transformation A to a standard axis view and voxel size.

In our case these are coronal views and 1 1 1 mm 3 per voxel. This transformation assures images coming from different sources are comparable. 2. A resampling using the scale parameter S g to the desired resolution for global registration. In practice, this is a useful parameter that in uences the precision of the transformation obtained and the computation time.

After matching, we obtain the global a ne transformation G. correction matrix G. As a second di erence, the scaling factor for global registration S g is replaced by it's local equivalent S l and a transformation E accounting for the VOI extraction. This last transformation is a cropping at identical locations for both images. In our work, it is implemented using a translation, to make it representable using matrices, and a size speci cation. This matching provides the regional a ne transformation R.

Regional morphometrical corrections

Local morphometrical corrections Evaluation of local di erences is done using the same procedure as before, with a correction matrix R updated with regional information, and a matching allowing a transformation class T of the free-form type.

Images after the regional correction are called corresponding volumes of interest. They di er only by morphological and local morphometrical variations. Hence, when considering structures of the same morphology, those images can be used to obtain information concerning morphometrical dissimilarities such as size di erences.

Morphological variations can be obtained by applying the third step that corrects for local morphometrical di erences. From this, the detection of particular features could be obtained. An interesting aspect of our work is the combination of those transformations by way of 4 4 matrices in homogeneous coordinates, and optionally a displacement eld derived from local variations, to end up with a single transformation and therefore a single resampling of the raw images.

It is also possible to completely eliminate the rst part of the procedure since, for a given reference image, these transformations do not depend on the VOI de nition. They can be precomputed to save time. Moreover, when the reference image changes, instead of recomputing the whole set of global transformations, it is su cient to combine them with the transformation that brings the old reference image onto the new one (see Figure 5).

Data sets

APPLICATIONS

Applications resulting from the correction of morphometrical di erences could be divided in two categories: (1) studies of local morphometrical di erences or (2) studies of morphological variations.

Morphometrical measures

This class of application relies on measures of brain structures. For example, the comparisons of the basal ganglia's components volumes with a standard model to identify schizophrenia. [START_REF] Hokama | Caudate, putamen, and globus pallidus volume in schizophrenia: A quantitative MRI study[END_REF] The construction of such a model relies on measures taken from a population of normal subjects. We have implemented this last idea.

Construction method

The idea behind the construction procedure is derived from Refs. 8,26,27,16 and consists of extracting from the database the volumes corresponding to a VOI de ned on a reference image, and applying to the average of those VOIs the average of the local morphometrical di erences. This process is depicted in Figure 8.

Using the procedure described in section 4.1. on all the entries of the database, we can eliminate global and regional morphometrical di erences to extract corresponding VOIs of our database. For each of those entries, we can compute the displacement eld that maps the entry VOI onto the reference VOI (forward) and the reference VOI onto the entry VOI (backward). These displacements account for local morphometrical di erences. Applying to every region the forward displacement D r a f t N o v e m b e r 2 0 , 1 9 9 7 ( 0 9 : 5 9 )

eld produces images with the same morphometry as the reference VOI but with di erent intensities. Arithmetic averaging of corresponding voxel's intensities between VOIs produces a mean intensity image with the morphometry of the reference image. Furthermore, by applying the average backward displacement eld to this last image, we create a mean intensity and morphometry representation of our database, or the average VOI. An advantage of our technique is that the average images it produces are of quality similar to the MR images of the database. Hence the same inter-patient matching method can be applied to compare a new patient to the average, which is an alternative to whole database exploration.

Resulting average patients

We have applied the same scheme to extract from the database the volumes that contain the ventricles. Averages have been built using the method previously explained and are shown in Figure 10(a). The careful observer will notice that, although they were obtained using di erent methods, the average VOI (last image of the second row) resembles a smoothed version of the average VOI after a ne correction (last image of the rst row). This is especially true for the ventricles and strengthens the theory behind our method.

In this case, we have also applied to the segmented ventricles, the displacement eld corresponding to the average local morphometrical di erences. This permitted to construct ventricles that represent the average morphometry of our database. They are shown in Figure 10(b). To the left, the original ventricles and to it's right, a display of their average morphometry. An obvious di erence is the back-end of the ventricles which are thinner for the average patient than for the original ventricles.

Morphological measures

This class of applications is concerned with the presence or absence of features. Following this, we have just begun experiments to classify similar patients according to their morphology. This work requires methods related to sorting subjects, or more generally, the development of similarity criteria. Their is a large literature on this topic [START_REF] Fukunaga | Introduction to Statistical Pattern Recognition[END_REF][START_REF] Pavlidis | Structural Pattern Recognition[END_REF] but our goal here is neither to nd the best measure nor to analyze its relationship with our registration procedure. We wish to show practical applications of VOI extraction and evaluate qualitatively di erent similarity criteria. Hence, the following experiments are to be considered as a Figure 10.

rst step of a feasibility study. Furthermore, the images at our disposal are of normal subjects. Consequently, we are not trying to evaluate pathologies or nd anomalies. Among available choices, we have chosen two: (1) the stochastic sign-change (SSC), which is based on zero-crossings, and (2) correlation. The method consists of extracting morphometrically corresponding VOIs automatically from the database using the method previously described. We then de ne a working space in which comparisons are done (see Figure 11). We present the criteria used and show preliminary results of VOI classi cation according to their resemblance with I 10 .

The stochastic sign-change criterion

This criterion was introduced by Venot et al. [START_REF] Venot | An automated method for the normalization of scintigraphic images[END_REF] for the comparison of scintigraphic images. It is a measure of similitude between two images and is based on zero-crossings (see Figure 12).

Let I 1 and I 2 be similar images that contain noise and S = I 1 ? I 2 . Originally, I 1 and I 2 are identical. Then, S is not zero but contains sign uctuations because of noise. However, if the two images di er from one another in some region R of S, R will contain either mostly positive or mostly negative values and the number of sign changes in S is reduced. This shows that a high SSC count in S is a good indication of resemblance between two images.

To count the number of sign changes, we go through the subtraction image three times. Once comparing values in the X direction, once in the Y direction and a last time for the Z direction. This criterion can be normalized by considering the maximal value of SSC in the working space. Hence, values close to 1 shows good similitudes, and those close to 0, poor correspondence.

Correlation

The correlation between two working spaces X and Y each containing N voxels is computed using the following formula:

Corr(X;

; Y ) = 1 N N X i=1 X i Y i ? 1 N 2 N X i=1 X i N X i=1 Y i Var(X) = Cov

Results

The SSC values and correlation coe cients are shown sorted in Table 1. We also present, using this ordering, the working spaces used for computations in Figures 13(a) and 13(b). Each of these gures contains two non-adjacent slices of the VOIs to get a better understanding of the tridimensional structure. They show the importance of our three dimensional approach since one would probably change this ordering by only looking at one slice.

The ordering obtained with the SSC criterion and with correlation are similar up to minor permutations. The only large di erence has to do with I 1 which is identi ed as having the most di erent morphology using the SSC criterion and is place fth using correlation. By looking at both slices of this VOI, we can see that it is quite similar to I 10 . Hence, preliminary results seem to indicate that correlation would be more appropriate to evaluate morphological di erences than the SSC criterion. In light of those results, comparisons based on mutual information techniques are expected to give good classi cations.

Although in this case we used an elastic registration procedure before classi cation, when assuming negligible morphological variations, this classi cation method can be used on corresponding VOIs thus using only a ne transformation for registration purposes. In this case, an order on local morphometrical di erences would be obtained.

Image SSC value (normalized) The main bene t of this method is that it is fast and fairly robust. The time to obtain the corresponding VOIs varies between 5 and 10 seconds, and local warping doubles this time. This opens a new door on medical image database exploration. It is now possible to retrieve a large amount of corresponding volumes in a reasonable amount of time. Once those volumes extracted, further processing like classi cation and similarity matches can be operated to result into extrapolation of information contained in the database to other images. For example, this technology is presently being used for the study morphometrical hippocampal variations in epileptic patients. We have only started to explore this eld but preliminary results are encouraging.

CONCLUSION

We have presented a new method to obtain corresponding volumes of interest from medical image databases. This procedure has the advantage of restraining itself to local volumes of the brain and thus is not in uenced by misleading information from other regions. It has been applied to the quanti cation of morphometrical and morphological variations. We believe that better results could be obtained by dividing subjects into subclasses of normal patients based on morphological similarities. This would facilitate the analysis of only morphometrical di erences, to provide a better understanding of dissimilarities between a patient and the group of normal subjects with corresponding morphology.

Figure 1 .

 1 Figure 1. Volume of interest extraction and comparison are the principal ideas behind medical image database exploration.

Figure 2 .

 2 Figure 2. Classi cation of di erent transformation classes.

Figure 3 .

 3 Figure 3. Morphometrical and morphological di erences.

Figure 4 .

 4 Figure 4. VOI extraction method using the reference image and a database entry.

  This procedure is similar to the previous one. Two steps di er. The rst modi cation concerns the database entry for which we substitute the change of referential by the global

Figure 5 .

 5 Figure 5. Matrix combination to change the reference image.

Figure 6 . 1 1

 61 Figure 6. Coronal slice of each image composing the database. Images are numbered from 1 to 10 when counting from left to right, top to bottom.

Figure 7 Figure 7

 77 Figure 7. VOI Extraction

Figure 8 .

 8 Figure 8. Procedure to build an average VOI from a set of images.

Figure 9

 9 Figure9shows di erent kinds of average VOIs of the right temporal lobe. The rst image is the reference VOI and is shown only for comparison purposes. To its right is a simple average of the VOIs taken from each subject previous to any warping, putting forward the di erent patient positions during the acquisition. The third image presents the average of corresponding volumes of the database. Structures which are morphologically stable appear well contrasted while unstable ones are fuzzy. The rst image of the second line is the average of the VOI intensities after elimination of morphometrical di erences. Deforming this image with the average backward displacement eld creates an image that represents the mean intensity and morphometry of the database, or the average VOI. It has the remarkable property of being an average image while not su ering from severe smoothing. Finally, we present the variance of the backward deformation elds' amplitude obtained from our database as a simple display of the variability of the di erent structures positions.

Figure 9 .

 9 Figure 9. Di erent types of average patients. First line: Original reference VOI; average of the 10 VOIs before any processing; average VOI after exclusion of global and regional morphometrical di erences. Second line: average VOI without morphometrical di erences (mean intensity); average backward deformation applied to the average VOI without morphometrical di erences (mean intensity and morphometry); variance of the average backward deformation. See the text for explications.

  Average VOI obtained from free-form registration (see Figure9for explanations. The positions of the second and third images of the second line have been interchanged for comparison purposes.).(b) Left: original ventricles. Right: morphometrical average ventricles of the database.

Figure 11

 11 Figure 11. A slice of the VOI and working space. Only the working space is considered for comparisons.

Figure 12 .

 12 Figure 12. The stochastic sign-change criterion.

  Classi cation using the SSC criterion. Two volume slices are shown.Second slice First slice (b) Classi cation using correlation. Two volume slices are shown.

Figure 13 .

 13 Figure 13. Classi cations.

  Future applications and research on classication and extraction of similar patients for epidemiology statistics or computer aided diagnosis are envisioned. Such applications are a rst step toward the extrapolation of information from an image database to other images.

Table 1 .

 1 Classi cation using the SSC criterion (left) and correlation (right).

			Image Correlation
	10 4 7 6 9 5 3 2 8 1	1.000 0.325 0.315 0.305 0.285 0.284 0.282 0.279 0.271 0.268	10 4 6 3 1 7 5 9 8 2	1.000 0.917 0.906 0.906 0.888 0.879 0.858 0.845 0.844 0.836

ACKNOWLEDGEMENTS Thanks to Dr. Ron Kikinis of Brigham and Women's Hospital, Boston, for the MR images. Part of this work was funded by the Natural Sciences and Engineering and Research Council of Canada (NSERC), the Fonds pour la Formation de Chercheurs et l'Aide ? la Recherche du Qu?bec (FCAR), the Association des Directeurs de Recherche Industrielle du Qu?bec (ADRIQ) and the D?partement d'Informatique et de Recherche Op?rationnelle de l'Universit? de Montr?al (DIRO).