Anh-Phuong Ta
email: anh-phuong.ta@inria.fr

Mathieu Ben
email: mathieu.ben@powedia.com

Guillaume Gravier
email: guillaume.gravier@irisa.fr

Improving Cluster Selection and Event Modeling in Unsupervised Mining for Automatic Audiovisual Video Structuring

Keywords: Video mining, Video structuring, Multimodality, Mutual Information, Event discovery, Structural event, Audiovisual consistency

Can we discover audio-visually consistent events from videos in a totally unsupervised manner? And, how to mine videos with different genres? In this paper we present our new results in automatically discovering audio-visual events. A new measure is proposed to select audio-visually consistent elements from the two dendrograms respectively representing hierarchical clustering results for the audio and visual modalities. Each selected element corresponds to a candidate event. In order to construct a model for each event, each candidate event is represented as a group of clusters, and a voting mechanism is applied to select training examples for discriminative classifiers. Finally, the trained model is tested on the entire video to select video segments that belong to the event discovered. Experimental results on different and challenging genres of videos, show the effectiveness of our approach.

Introduction

The purpose of video structuring is to automatically find structural events in video sequences. Obviously, a structural element is represented as a key content of videos. Examples of such structural events are jingles in news videos, anchorpersons or participants in TV program videos, choruses from karaoke music videos, etc. Due to its potential applications in various fields, such as video summarization, video indexing and browsing, and content based video retrieval, video structuring can be considered as a crucial step in content-based video analysis. Existing methods can be broadly classified into two groups: (i) dense segmentation of the entire video, in which the video is mapped to a predefined structure [START_REF] Clarkson | Unsupervised clustering of ambulatory audio and video[END_REF] [START_REF] Xie | Sun. Unsupervised Mining of Statistical Temporal Structures[END_REF], and (ii) detection of a specific event like goals in sport videos [START_REF] Petkovic | Multi-Modal Extraction of Highlights from TV Formula 1 Programs[END_REF] [START_REF] Wang | A Generic Framework for Semantic Sports Video Analysis Using Dynamic Bayesian Networks[END_REF], advertisements, or anchorpersons [START_REF] Hauptmann | Analyzing and searching broadcast news video[END_REF] [START_REF] Tat-Seng | Story boundary detection in large broadcast news video archives: techniques, experience and trends[END_REF]. Despite their good results, most of these methods have two shortcomings: (a) they require manually annotated data for training models, thus these methods lack the generality to cope with diverse video sources; (b) they use clustering techniques to group similar video segments, which form structural events. However, the problems that many clustering algorithms encounter are the choice of the optimal number of clusters, and how to deal with outliers.

It should be noted that there are very few research results available on discovering audio-visual events in an unsupervised way. Previous methods that are closely related to our work are those concerning video mining (video structuring). Video mining approaches [START_REF] Covell | Detecting Ads in Video Streams Using Acoustic and Visual Cues[END_REF] [10] [START_REF] Jacobs | Using Self-similarity Matrices for Structure Mining on News video[END_REF] [START_REF] Yang | Efficient Short Video Repeat Identification With Application to News Video Structure Analysis[END_REF], which aim to detect regularly repeating patterns, have focused on the discovery of near-duplicate repetitions. However, these methods cannot deal with the structural events exhibiting content and temporal variations, i.e., the repetitions are not exact. There have been several works in unsupervised mining from videos [START_REF] Naphade | Discovering Recurrent Events in Multichannel Data Streams Using Unsupervised Methods[END_REF] [5] [START_REF] Xie | Sun. Unsupervised Mining of Statistical Temporal Structures[END_REF]. The main idea of these methods is to exploit structural elements through mapping the entire video to predefined models. Because they are based on the assumption of dense segmentation, these approaches cannot handle the discovery of sporadic structural events.

In order to overcome the limitations mentioned above, in our previous work [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF] we have proposed an unsupervised method to detect structural events from audio-visual documents without prior knowledge of the genre of the video. As presented in [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF], two hierarchical clustering trees (called dendrogram) are first constructed for both audio segments and video shots (keyframes). We then measure the consistency between all pairs of audio-visual clusters by using mutual information (MI). Finally, several heuristics are used to select the best (i.e., the most relevant) audio-visual pair that represents a structural event. In this paper, we propose an extension to this method that makes it more robust to deal with variability and easier to extract multiple structural events. Compared to our previous work [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF], the main contributions of this paper are summarized as follows:

• We present a new measure for selecting audio-visually consistent elements. The proposed measure is slightly different from the one used in [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF], in that instead of using the original mutual information measure, which has 4 possible states (correlations) for two input binary variables, we use only two positive correlations (see section 3.1 for more details). • We take into account the structure of the two given dendrograms to represent events, ie., each event is represented by a group of audio-visual cluster pairs, as opposed to only one pair used in [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF], which may result in partial events (i.e. events whose occurrences are incompletely detected). • In order to construct a model for each event, we propose a voting mechanism to select positive and negative examples as input vectors for Support Vector Machines (SVM). Note that, in [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF], positive and negative examples are selected entirely based on the audiovisual segments of the best pair of clusters. Due to errors in clustering, however, not all these segments belong to the same structural event. In this work, we propose a more accurate method, in which each audiovisual segment will first cast its votes for negative or positive. Then, thresholds are used to select positive and negative examples (see section 3.3 for more details).

The rest of this paper is organized as follows. After briefly summing up the early work [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF] for discovering structural events in section 2, we introduce our extensions of this method in section 3. Section 4 describes experimental results. Finally, we conclude and give some perspectives of this work.

Discovering Audio-visually Consistent Events

In this section, we briefly summarize our early work on unsupervised video structuring [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF], on which our approach is based. Figure 1 illustrates the main components of our approach, as well as the original approach4 in [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF]. Recall that our main objective was to design a generic approach to find events of interest that share common characteristics, including audio and visual presentations. First, the audio and video streams are respectively segmented into audio and video (shot) segments. We then extract commonly used audio and visual features (Gaussian components for audio, and RGB histograms for video) for segmentation and clustering. Classical bottom-up clustering algorithms are then applied for each modality to provide two different dendrograms representing video and audio clustering results. Given these two dendrograms, the work in [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF] consists of 3 steps:

1. The first step measures mutual information (MI) between all pairs of audio and video clusters. These pairs of clusters are then ranked according to their MI values, and a list of n-best top pairs of clusters (i.e., the ones having highest MI values), called N-best list, is extracted. 2. The second one selects the best consistency pair among different pairs in the N-best list using several heuristics. This selected pair is considered to be representative of the most relevant event. 3. In the last step, based on the found event (the pair of clusters selected from the above step), positive and negative samples are extracted (see section 3.3 for more details). Then, a binary SVM classifier is applied to construct the event model.

In this paper, we aim at improving the cluster selection (step 1), and the event modeling techniques (step 3). We do not focus on using heuristics (step 2), which depend on the mining objectives, but we propose a new way to represent candidate events. The details of our approach is described in the next section.

Improving Cluster Selection and Event Modeling Techniques

Despite the promising initial results, the method introduced in [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF] has several limitations:

a) It only allows to discover a single audio-visually consistent event.

b) It relies entirely on the consistent power of a single pair of clusters, i.e., a structural event is represented by a pair of audio-visual clusters only. Thus this method may result in partial structural events.

Audiovisual consistency Event mining Event modeling

Visual clustering

Audio clustering

Fig. 1: Our general scheme for mining structural events from videos (this figure is adapted from [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF]). In this figure, we show a simple example illustrating the detection of an anchor-person.

c) The original mutual information measure used in [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF] to compute the consistency between two binary variables, which represent a pair of audio-visual clusters, is

symmetric. That is, M I(A, V ) = M I( Ā, V ) = M I(A, V ) = M I( Ā, V )
, where A, V are two binary variables denote the existence of audio, and video clusters, respectively; and M I(•, •) is a function that measures mutual information of two given variables. This measure can give good results for detecting events that appear quite regularly, or for events, for which the corresponding audio and video segments from the selected clusters are quite strongly consistent. However, it cannot distinguish between relevant and irrelevant events (an irrelevant event appears when one of the two binary variables, or both is absent, i.e., for the cases of M I( Ā, V ), M I(A, V ), and M I( Ā, V )).

In this work we overcome these limitations by exploiting the N-best list based on the structure of the trees (i.e., dendrograms).

Estimating The Consistency of Audio-visual Clusters

After the hierarchical clustering of audio and video segments extracted from an input video, we obtain two corresponding dendrograms, where each node has a set of segments from the corresponding modality. Now we measure audio-visual consistency between an audio cluster and a visual cluster by using mutual information (MI) for two random variables. A large MI value indicates that the two corresponding clusters are closely consistent with each other and share more mutual information. In our case, for finding a repeating event, we aim at finding the consistency of audio-visual segments from a pair of clusters, which can be represented by the two positive correlations of the original mutual information. Let (C A i , C V j ) be the i-th and the j-th nodes of the audio and video dendrograms, respectively. The mutual information between C A i and C V j is given as follows:

M I(C A i , C V j ) = (a,v)∈{(0,0),(1,1)} p(a, v) ln( p(a, v) p(a)p(v) ) (1)
where a and v are binary random variables which respectively denote membership in C A i and C V j . The probabilities p(a, v), p(a), and p(v) are estimated from the temporal distribution of segments. For instance, the join probability p(a = 1, v = 1) is measured as the sum of the amount of time each segment of C A i co-occurs with a segments of C V j , normalized by the total duration of the video. Equation 1 is applied for measuring MI between all audio and visual cluster pairs. Then, a list of pairs of clusters is established, and sorted in descending order according to its MI value. The objective of sorting is to help discover candidate events from more consistent to less consistent, and to filter out the irrelevant pairs (inconsistent pairs), i.e., the ones having low MI values. After removing inconsistent pairs, the final list remains n best pairs, called N-best list. In the next sub-section, we will analyze this list to discover structural events.

Event Mining

From the N-best list obtained in the above step, we extract candidate events. Due to the nature properties of hierarchical clustering algorithms, there are redundancies (in terms of structural relationships and similar characteristics) in the N-best list. Obviously, the pairs of clusters in the N-best list, which share redundant contents, should be part of an event (i.e., they should belong to the same structural event). More precisely, using a pair of audio-visual clusters (a, v) as example, if (a, v) represents a structural event, all other pairs of clusters from the two sub-trees constructed from (a, v) will represent (sub)instances (or other instances) of this event. Therefore, this event should be represented by a group of cluster pairs, including (a, v) and its neighboring pairs. See figure 2 for an explanation of this principle. Based on the analysis above, we extract multiple structural events in the following way: starting from the first pair in the N-best list (i.e., the one having highest MI value), we search from the N-best list to find all successor pairs from the two sub-trees constructed from this pair. The found successor pairs are grouped together with the current pair to establish a representative group for this event. This process is applied for each of the remaining pairs in the N-best list. The result is a list of group containing consistent audio-visual clusters, in which each group represents a structural event. The overall algorithm is presented in Figure 3, where getSubtree(•, •) is a function that returns a subtree from a given node (cluster); remove(•, •) function that removes a pair of clusters from a list; and getF irstAV (•) function returns the first element (ie., the pair of audio-visual clusters having highest MI value) from a list. Please note that we define a subtree for a given node as a list of nodes created from all paths that pass through it from a leaf to the root of the tree. An illustration example is shown in figure 2a, where the subtree for the red node is a list of nodes including all green nodes and the red node.

In our context, a consistent audio-visual cluster belongs to only one structural event. In other words, there are no structural events that share the same cluster pairs. Thus, after having constructed the list of groups, we remove for each group from this list, all common elements of any two groups. Consequently, the obtained list contains only non-intersecting groups, each of which represents a structural event. Note that although the algorithm presented in figure 3 returns all possible structural events from an input video, quantitative evaluations for multiple structural events require the knowledge of a Then, the list of audiovisual candidate pairs contains: {(a,1) (a,2), (a,3),(a,5), (a,6), (a,7), (b,1), (b,2), (b,3), (b,5), (b,6), (b,7), (c,1), (c,2), (c,3), (c,5), (c,6), (c,7), (f,1), (f,2), (f,3), (f,5), (f,6), (f,7), (h,1), (h,2), (h,3), (h,5), (h,6), (h,7), (i,1), (i,2), (i,3), (i,5), (i,6), (i,7)}. From this list, a representative group for the candidate event is constructed by grouping together pairs that are present in the N-best list. Note that the yellow clusters from both dendrograms are not considered to be part of the representative group, because they may belong to another event. human expert, and therefore out of the scope of this paper. In the next sub-section, we present a new method to model an event from a given group of audio-visual clusters.

Event Modeling and Recognition

After the event mining step, a candidate event is characterized by a group of audio and video clusters E = {e 1 , e 2 , ..., e m }, where e i represents a pair of audio and video clusters (C A , C V ) with the corresponding temporal segments (S A , S V ). Note that the audio/video segmentation and clustering steps are performed independently for each modality. Thus S A and S V may be different from each other, and we need to combine them for building representative segments (i.e., audiovisual segments) of the pair in consideration e i . For this end, an audiovisual segment of the video is constructed by merging the boundaries of an audio and a visual segments, and the corresponding feature vector (audiovisual feature vector) is the concatenation of the two component feature vectors. Recall that our goal here is to build a model based on audiovisual feature vectors, which can be used to predict a structural event from the entire video. Our early experiments [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF] show that taking audiovisual segments belonging to the intersections of audio and video segments as positive examples 5 , and those corresponding to neither their intersections nor their unions as negative examples, gives good results.

Particularly, for the above pair e i characterized by the two clusters (C A , C V ), two sets of positive and negative training samples are determined as follows:

AV sk ∈ +1l if s k ⊂ S A ∩ S V AV sk ∈ -1l if s k ⊂ S A ∪ S V
where s k is an audiovisual segment, and AV sk is its corresponding audiovisual feature vector. Now we extend this technique for the selection of training samples for the group of clusters E. The idea is as follows: for each element e i (a pair of clusters) in E, we cast its votes for negative and positive samples (i.e., each audiovisual segment from e i casts vote for positive or negative), and the voting results are accumulated for all elements in the group. This voting algorithm is described in figure 4, where negV ote(•, •) is a function that casts votes for negative examples for the audiovisual segments from a given pair of clusters. Similarly, posV ote(•, •) casts votes for positive examples. Note that, Fig. 5: Illustration of typical structural events in our dataset. From left to right: anchor person in news; a separator screen in flash news; a separator, two participants, and a presenter in games; a guest in a talk show; and magazine anchor person.

SN and SP are used to keep voting results accumulated over all elements in the input group. T p , T n are thresholds, which will be set experimentally, used to filter out the outliers. And doubtSet is a set of audiovisual segments, where we do not have enough information to choose between negative and positive samples. Thus these segments should not belong to both negative and positive sets. Once the positive and negative samples are selected, discriminative classifiers can be used to train a binary model. In our work, we use a binary SVM with 5-fold cross-validation procedure on the training set to optimize Hyper-parameters. We then apply the trained model to all audiovisual segments in the input video. These segments are classified as corresponding to either the event under consideration or not.

Experimental Results

Discovering structural events based on audiovisual consistency should be evaluated on datasets, in which such events are present. Unfortunately, up to our knowledge no standard database is available for such difficult tasks. To evaluate our method, we captured TV programs with different genres from various French television channels (see fig. 5 for several examples of structural events), including: flash news, news, magazines, investigation reports, talk shows, and games. These videos are then annotated by a human expert. There are 18 video sequences (over 16 hours), in which the longest annotated event has about 120 occurrences and the shortest one has 9 occurrences. We used the classical recall, precision, and F-measure to evaluate the performance of our method. Note that, given the occurrences of an discovered event and its corresponding annotated event, recall is defined as the amount of time of correctly detected occurrences with respect to the total amount of time of all the occurrences from the corresponding annotated event in the ground truth, whereas precision is the amount of time of correctly detected occurrences with respect to the total amount of time of the detected occurrences. We performed two different experiments: the first was designed to evaluate the performance of the proposed measure for cluster selection. The second experiment aimed to evaluate the effectiveness of the proposed event modeling method. These two experiments are described below.

Evaluation of The Proposed Measure. In this evaluation, we compare the performance of the proposed measure for cluster selection with the baseline measure presented in [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF]. For more convenience, we denote our measure by MI2 (i.e., it has two possible correlations, cf. eq. 1), and the method in [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF] by MI4. The testing protocol is as follows: for each video, both MI2 and MI4 are first applied to establish two corresponding N-best lists (ranked lists). Then SVM models are trained on samples extracted from the first element for each list. Note that ranking results in the N-best list provided by these two measures are not always in the same order. Therefore, for a fair comparison, for each genre of programs, we selected to report only results from the videos, for which both methods give exactly the same structural event corresponding to the first element in the N-best list. In the case that these two methods return the same event with different orders, we will discuss qualitative results later. of these methods, where the last column indicates the number of videos for which the two methods detect the same event (in terms of the first element in the two N-best lists) with respect to the total number of videos tested. For flash news, these two methods give exactly the same pair of audio and visual clusters, yielding the same results from SVM classification. This is due to the fact that such kinds of videos contain only one structural event and have little variations. From this table, we can observe that for the more challenging videos (eg., investigation videos), MI2 gives much better results. It should be noted that, except for flash news and magazines, MI2 always returns a pair comprising either the same audio cluster as MI4 and associated with a higher (i.e., higher level in the dendrograms) video cluster, or vice-versa. This indicates that MI2 is more robust to variability. Please note that, we cannot directly compare our results with those from the previous method [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF], because the experimental set-ups are different, and we used more challenging videos for the tests in this work.

Evaluation of Event Mining and Modeling Techniques. In this experiment, we test the performance of the proposed event modeling method, i.e., evaluating the voting mechanism based on a representative group for a candidate event. To this end, we compare the performance for both MI2 and MI4 with and without using group (i.e., with or without using the proposed event mining and modeling techniques). For the case of without using group, the experimental set-up is the same as in experiment 1 above. For the case of using group, the testing protocol is given as follows: given the N-best list (obtained by MI2 or MI4), we apply the algorithm presented in Fig. 3 to obtain a list of representative groups of candidate events. The first group is then extracted from this list (i.e.., the group corresponding to the first element in the N-best list). For each group of audiovisual cluster pairs, we keep only 5 elements that have highest MI values. In all our experiments, thresholds 6 T p is set to 1, and T n is 4 for MI4, and T p is set to 2, and T n is 4 for MI2 (i.e., T p and T n are respectively set to be of 40% and 80% with respect to the group size). Finally, the algorithm presented in Fig. 4 is used to select training samples and SVM is applied to train the event model. Table 2 and table 3 give the average results (in terms of recall, precision, and F1measure) by using the measure introduced in [1] (MI4) and our measure (MI2), respectively. Where, the column, namely "Single pair", shows the obtained results from the best pair of clusters; and the column "group" shows the obtained results applying the proposed event modeling technique (i.e., the voting mechanism). From these tables, it can be easily seen that applying the proposed event modeling method (using a group of pairs) to represent events significantly outperforms the case of using only one pair of clusters (the most consistent pair in the N-best list). Taking the investigation videos for example, the performance of using the event modeling technique is increased by more than 10% with respect to that of using only one single pair. The results in terms of the average of F1-measure for MI2 using the voting method moderately decrease by roughly 1% for talkshows and news, however the corresponding recalls are higher. To evaluate the stability of our voting method, we also performed the tests for MI2 by varying the size of the group and setting T p to 40%, and T n to 80% of the group size, respectively, the performance changes by less than 1.5%.

Qualitative Analysis. Although quantitative analysis for multiple structural events is beyond the scope of this paper, we still performed the tests for this task, and observed qualitative trends of both MI4 and MI2 with and without applying the proposed event modeling technique. This allows us to point out some particular points: (a) using the proposed voting method gives much better results in general; (b) MI4 seems to be suited for sparse events, and events having little variations, and provides quite limited potential for discovery of multiple events; (c) if both measures detect the same events but with different orders (the detected order from the N-best list), MI2 often returns a more complete event.

Conclusion and Discussion

In this paper, we have presented an improvement on cluster selection and event modeling in unsupervised mining for automatic audiovisual video structuring. The experimental results, using different genres of videos, demonstrate that the proposed method gives significant improvement compared to our previous work [START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF]. Our current work can be extended in several ways: first, we plan to evaluate multiple structural events, and to automatically determine the thresholds used for selecting training features. Second, we will explore how to automatically select n best pairs in the N-best list, and the relevant events among the different candidate groups. Finally, other features would be useful for the discovery of events in specific domains, eg., optical flows could be interesting for event discovery in sport videos.

Fig. 2 :

 2 Fig.2: Partial views of the audio (a) and visual (b) dendrograms, which illustrate the principle of the construction of a representative group for a given candidate event represented by a pair of clusters (see section 3): assuming that the pair of audio and video clusters in the N-best list (h,6) represents a candidate event. Then, the list of audiovisual candidate pairs contains: {(a,1) (a,2), (a,3),(a,5), (a,6), (a,7), (b,1), (b,2), (b,3), (b,5), (b,6), (b,7), (c,1), (c,2), (c,3), (c,5), (c,6), (c,7), (f,1), (f,2), (f,3), (f,5), (f,6), (f,7), (h,1), (h,2), (h,3), (h,5), (h,6), (h,7), (i,1), (i,2), (i,3), (i,5), (i,6), (i,7)}. From this list, a representative group for the candidate event is constructed by grouping together pairs that are present in the N-best list. Note that the yellow clusters from both dendrograms are not considered to be part of the representative group, because they may belong to another event.

Fig. 3 :

 3 Fig. 3: Algorithm for extracting groups of consistent audio-visual clusters. Data: A ranked list of n pairs of clusters: nbestList; Two dendrograms corresponding to the two hierarchical clustering results: A, V; Result: A list of groups containing consistent audio-visual clusters: E k = 0; while not empty(nbestList) do E(k) = {}; a, v ← getF irstAV (nbestList); suba = getSubtree(A, a); subv = getSubtree(V, v); for ∀(a, b) |(a ∈ suba, b ∈ sub b ) do if (a, b) ∈ nbestList then E(k) = E(k) ∪ (a, b); remove(nbestList,(a,b)); end end k = k + 1; end

Fig. 4 :

 4 Fig. 4: Feature selection for SVM. Data: A group of clusters E; thresholds Tp,Tn; set of audiovisual segments SEG Result: Set of input vectors (including positive samples and the negative ones) for SVM: vp, vn forall s ∈ SEG do SN(s) ← 0 ; SP(s) ← 0 ; end foreach e ∈ E do negVote(e, SN); posVote(e, SP); end vp ← (AVs i | si ∈ SEG and SP (si) > Tp); doubtSet ← (si ∈ SEG | (0 < SP (si) ≤ Tp and SN (si) > 0)); vn ← (AVs i | si ∈ SEG and (SN (si) > Tn and si / ∈ doubtSet));

Table 1

 1

	presents a comparison

Table 1 :

 1 Comparison of the performance between MI4 and MI2.

Table 2 :

 2 Comparison of the performance for MI4 with and without applying the proposed event modeling technique.

	Genre	R	Single pair P	F1	R	Group P	F1
	Flash News	0.78	0.88	0.83	0.88	0.82	0.85
	News	0.81	0.85	0.82	0.82	0.84	0.83
	Magazines	0.78	0.87	0.82	0.79	0.87	0.83
	Investigation	0.35	0.75	0.44	0.52	0.76	0.57
	Talk shows	0.50	0.93	0.65	0.68	0.90	0.72
	Games	0.63	0.81	0.68	0.75	0.76	0.75

Table 3 :

 3 Comparison of the performance for MI2 with and without applying the proposed event modeling technique.

	Genre	R	Single pair P	F1	R	Group P	F1
	Flash News	0.78	0.88	0.83	0.84	0.88	0.86
	News	0.83	0.85	0.84	0.85	0.83	0.83
	Magazine	0.65	0.77	0.71	0.67	0.76	0.71
	Investigation	0.45	0.65	0.49	0.61	0.66	0.61
	Talk shows	0.65	0.85	0.74	0.67	0.80	0.73
	Games	0.82	0.81	0.81	0.90	0.79	0.83

This scheme, which illustrates the different components of our work, is slightly different from the original one presented in[START_REF] Ben | Unsupervised mining of audiovisually consistent segments in videos with application to structure analysis[END_REF], but both operate on the same principle.

Note that, in our case we use a binary classifier to distinguish between an candidate event, i.e., positive class (+1), and non-candidate event, i.e., negative class (-1).

The choice of the optimal thresholds Tp and Tn is beyond the scope of this paper.

http://pim.gforge.inria.fr/pimpy/

Acknowledgments. This work was partly funded by OSEO, French State agency for innovation, in the framework of the Quaero research program. Many thanks to Monica Corlay for doing the ground truth annotation work, and to Sébastien Campion for providing Python code for key frame clustering, as part of the PimPy library 7 .


M.D Marie-Laure Paillère Martinot

M.D Jean-Luc Martinot

Damien Ringuenet

André Galinowski

Thierry Gallarda

M.D Jean-Pascal Lefaucheur

Hervé Lemaitre

M.D Eric Artiges

Baseline Brain Metabolism in Resistant Depression and Response to Transcranial Magnetic Stimulation

Keywords: Depression, Fluorodeoxyglucose, Magnetic Resonance Imaging, Positron Emission Tomography, Transcranial Magnetic Stimulation, Treatment response

Neuroimaging studies of patients with treatment-resistant depression (TRD) have reported abnormalities in frontal and temporal regions. We sought to determine whether metabolism in those regions might be related to response to repetitive transcranial magnetic stimulation (TMS) in patients with TRD. Magnetic resonance images and baseline resting-state cerebral glucose uptake index (gluMI) obtained using 18 F-fluorodeoxyglucose positron emission tomography were analyzed in TRD patients who had participated in a double-blind, randomized, sham-controlled trial of prefrontal 10Hz-TMS. Among the patients randomized to active TMS, 17 responders, defined as having 50% depression score decrease, and 14 nonresponders were investigated for pre-stimulation glucose metabolism and compared with 39 healthy subjects using a voxel-based analysis. In nonresponders relative to responders, gluMI was lower in left lateral orbitofrontal cortex (OFC), and higher in left amygdala and uncinate fasciculus. OFC and amygdala gluMI negatively correlated in nonresponders, positively correlated in responders, and did not correlate in healthy subjects. Relative to healthy subjects, both responders and nonresponders displayed lower gluMI in right dorsolateral prefrontal (DLPFC), right anterior cingulate (ACC), and left ventrolateral prefrontal, cortices. Additionally, nonresponders had lower gluMI in left DLPFC, ACC, left and right insula, and higher gluMI in left amygdala and uncus. Hypometabolisms were partly explained by gray matter reductions, while hypermetabolisms were unrelated to structural changes. The findings suggest that different patterns of frontal-temporal limbic abnormalities may distinguish responders and nonresponders to prefrontal magnetic stimulation. Both preserved OFC volume and amygdala metabolism might precondition response to TMS.

INTRODUCTION

Treatment-resistant depression (TRD) occurs in 20 to 30% of depressed patients and leads to severe disability [START_REF] Greden | The burden of disease for treatment-resistant depression[END_REF]. Neuroimaging studies focused on TRD patients are rare, and have reported decreased blood flow or glucose metabolism in prefrontal regions such as the anterior cingulate cortex (ACC) [START_REF] Mayberg | Paralimbic hypoperfusion in unipolar depression[END_REF]1997a;[START_REF] Drevets | Subgenual prefrontal cortex abnormalities in mood disorders[END_REF][START_REF] Ketter | Effects of mood and subtype on cerebral glucose metabolism in treatment-resistant bipolar disorder[END_REF][START_REF] Konarski | Relationship between regional brain metabolism, illness severity and age in depressed subjects[END_REF], and in anterior temporal cortices [START_REF] Mayberg | Paralimbic hypoperfusion in unipolar depression[END_REF].

Differential frontal-temporal changes have been reported in medication responder and nonresponder subgroups of depressed patients. Responders to antidepressants were found to have lower pre-treatment glucose metabolism in both left amygdala region and temporal cortex, and in bilateral frontal cortices [START_REF] Little | Bupropion and venlafaxine responders differ in pretreatment regional cerebral metabolism in unipolar depression[END_REF]. In line with these findings, higher resting-state hippocampus-amygdala blood flow was found in a group of TRD compared with nonresistant patients and healthy controls [START_REF] Hornig | HMPAO SPECT brain imaging in treatmentresistant depression[END_REF]. Cortical-limbic balance involving orbitofrontal cortex (OFC), ACC, hippocampus and DLPFC might differentiate responders from nonresponders to antidepressant medications [START_REF] Seminowicz | Limbic-frontal circuitry in major depression: a path modeling metanalysis[END_REF].

Repetitive transcranial magnetic stimulation (TMS) has been proposed as a potential treatment for TRD [START_REF] George | Daily repetitive transcranial magnetic stimulation (rTMS) improves mood in depression[END_REF][START_REF] Pascual-Leone | Rapid-rate transcranial magnetic stimulation of left dorsolateral prefrontal cortex in drug-resistant depression[END_REF] that can easily be applied in clinical settings and is well tolerated. This procedure aims at stimulating the left dorsolateral prefrontal cortex (DLPFC) in order to increase brain metabolism in prefrontal areas using high stimulation frequencies, above 5Hz. However, although a number of studies have now demonstrated the clinical effectiveness of TMS in TRD patients [START_REF] Lam | Repetitive transcranial magnetic stimulation for treatment-resistant depression: a systematic review and metaanalysis[END_REF], the effects remain variable and factors predictive of response remain poorly understood [START_REF] Loo | A review of the efficacy of transcranial magnetic stimulation (TMS) treatment for depression, and current and future strategies to optimize efficacy[END_REF]Herrman and Ebmeier, 2006;[START_REF] Lisanby | Daily left prefrontal repetitive transcranial magnetic stimulation in the acute treatment of major depression: clinical predictors of outcome in a multisite, randomized controlled clinical trial[END_REF]. Since this therapy is time-consuming and costly, it is necessary to determine whether brain factors could influence or predict outcome. High-frequency TMS of prefrontal regions, as used to treat depression, has been shown to modulate ACC and caudate nucleus activity in healthy subjects [START_REF] Barrett | Mechanisms of action underlying the effect of repetitive transcranial magnetic stimulation on mood: behavioral and brain imaging studies[END_REF]. In addition, blood flow decreases in OFC and ACC have been reported after treatment with high-frequency TMS [START_REF] Nadeau | Cerebral blood flow changes in depressed patients after treatment with repetitive transcranial magnetic stimulation: evidence of individual variability[END_REF] in responders to TMS. So far, however, it is not clear whether pre-stimulation functional anatomy could influence TMS efficacy and predict response.

We a priori hypothesized that TRD patients who were nonresponders to TMS would have specific alterations in those frontal-temporal-limbic regions involved in TRD and in TMS effects, and also in the uncinate fasciculus a frontal-temporal fiber tract that we have reported to be altered in patients with affective disorders [START_REF] Houenou | Increased white matter connectivity in euthymic bipolar patients: diffusion tensor tractography between the subgenual cingulate and the amygdalohippocampal complex[END_REF]. We used [ 18 F]fluorodeoxyglucose positron emission tomography ([ 18 F]-FDG-PET) to determine whether resting functional brain characteristics at baseline could differentiate responders from nonresponders to 10Hz-TMS in a group of TRD patients. Additionally, we searched for a potential influence of structural anatomy on metabolism, and for functional correlations within the regions involved in response to TMS.

SUBJECTS AND METHODS

The investigation was performed in accordance with the Declaration of Helsinki. The study was approved by the ethics committee Ile-de-France 6, Paris. Written informed consent was obtained from all subjects after full description of the study.

Participants

Pre-trial PET and MRI data from the 34 patients who were initially randomized to an active arm in a double-blind, sham-controlled trial of 10Hz-TMS including 48 patients (Paillère [START_REF] Martinot | Influence of prefrontal target region on the efficacy of repetitive transcranial magnetic stimulation in patients with medication-resistant depression: a [18F]-fluorodeoxyglucose PET and MRI study[END_REF] were considered for the present imaging study. The patients had a DSM-IV-TR diagnosis of Major Depressive Disorder established by clinical interview using the Mini-International Neuropsychiatric Interview (MINI) [START_REF] Sheehan | The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10[END_REF], with criteria for treatment resistance to at least two trials of antidepressants of different classes given at adequate doses (above 150 mg/day in an equivalent dose of imipramine) and duration (at least four weeks for each drug).

Exclusion criteria included age over 65, pregnancy, alcohol or substance dependence in the past 6 months, electroconvulsive therapy (ECT) treatment in the past 6 months, any present medical condition, history of epileptic seizures, history of neurological disorders or substantial brain damage, and contraindication to magnetic fields. On examination of the patient medical charts, three patients were excluded, due to past early-onset alcoholism in 2 patients, and to a possible diagnosis of fibromyalgia in one patient. Indeed, alcoholism with onset in adolescence might have modified brain structure [START_REF] Chanraud | Brain morphometry and cognitive performance in detoxified alcoholdependents with preserved psychosocial functioning[END_REF], and fibromyalgia has been reported to alter brain activity in fronto-cingulate areas [START_REF] Burgmer | Altered brain activity during pain processing in fibromyalgia[END_REF]. Thus, data from 31 patients were analyzed in the present study (mean (SD) age=47.7 (7.3); range: 30.5-59.5, 20 women). They had on average 7.4 (4.4) years of education after primary school, and were not paid for their participation.

Thirty-nine paid healthy comparison subjects with no personal or family history of psychiatric or neurologic disorder, as assessed by a medical examination, were recruited by word of mouth from community volunteers during the same time period (age: mean(SD) =45.2(11.8); range: 25-62, 25 women; years of education after primary school: mean (SD)=9.10 (4.8) ).

Patients and healthy subjects did not differ in age (t=0.99, df=68, p=0.32), gender (χ 2 test=0.000, p=0.98), years of education (t=1.53, df=67, p =0.13), or [START_REF] Annett | A classification of hand preference by association analysis[END_REF] laterality score (patients: mean (SD) =85.7(49.3), healthy subjects: 89.4 (26.5), t=0.40, df=67, p =0.69).

TMS protocol

Design. The stimulation procedure and determination of PET-derived targets for TMS have been described elsewhere (Paillère [START_REF] Martinot | Influence of prefrontal target region on the efficacy of repetitive transcranial magnetic stimulation in patients with medication-resistant depression: a [18F]-fluorodeoxyglucose PET and MRI study[END_REF]. Briefly, after scanning, the patients were randomized to treatment with PET-guided, active-standard or sham-standard TMS and subsequently underwent 10 sessions of 10 Hz-TMS delivered at 90% motor threshold with 1600 pulses/session, using a Magstim super-rapid® device with active and sham air-cooled figure-of-eight coils (Magstim Co., Withland, Dyfed, UK). Guided TMS was on a prefrontal target determined with FDG-PET. Standard stimulation was as usual left pre-frontal, 5cm anterior to the hotspot of the hand motor cortical region. Patients and raters were blind to TMS modality.

Treatment allocation.The 31 patients in this imaging study had been randomly allocated to standard TMS (n=9 in responder group, n=6 in nonresponder group), or to PET-guided TMS (n=8 in responders, n=8 in nonresponders (Table 1). No difference in allocation was found across groups (χ 2 = 0.31, df=1, p=0.56). No patient randomized to sham TMS was analyzed in the present study, as response to sham TMS might be related to different mechanisms than response to active treatment, and as there were only 3 responders out of 14 sham-treated patients in that group.

Clinical assessment

Baseline assessment was performed on the day before scanning using the Montgomery-Åsberg Depression Rating Scale (MADRS) [START_REF] Montgomery | A new depression scale designed to be sensitive to change[END_REF], the 21-item Hamilton Depression Rating Scale (Ham-D) [START_REF] Hamilton | A rating scale for depression[END_REF], and the Tyrer anxiety scale [START_REF] Tyrer | The brief scale for anxiety: a subdivision of the comprehensive psychopathological rating scale[END_REF].

Response to TMS, defined as at least 50% decrease from baseline MADRS score, as in another TMS study of depressed patients [START_REF] O'reardon | Efficacy and safety of transcranial magnetic stimulation in the acute treatment of major depression: a multisite randomized controlled trial[END_REF], was assessed using the last clinical assessment performed just after the final TMS session. Among the actively-treated patients, seventeen were responders, and 14 nonresponders to TMS.

Treatments

Previous medication was titrated down to a minimal dose that did not lead to significant clinical worsening for at least two weeks before scanning. Five responders and 4 nonresponders had a stage II resistance (failure to at least two adequate trials of antidepressants), according to Thase and Rush criteria [START_REF] Thase | When at first you don't succeed: sequential strategies for antidepressant nonresponders[END_REF], 9 responders and 5 nonresponders a stage III (stage II and failure to tricyclics), 2 responders and 5 nonresponders a stage IV (stage III and failure to monoamine oxidase inhibitors (MAOI)), and one patient had a stage V resistance (stage IV and ECT resistance) in the responder to TMS group (Table S1).

Comorbidities

Seven patients met criteria for a comorbid diagnosis of anxiety disorder (panic disorder with or without agoraphobia, or generalized anxiety disorder); eleven patients had resistant depression in the course of bipolar disorder. Among the latter patients, 3 had a bipolar II type disorder, all of which were in the nonresponder group. Those comorbid diagnoses were equally distributed in both groups (Table 1).

Scanning protocols

All participants were investigated at rest. They were instructed to lie and relax in the PET camera with eyes closed, in a quiet room with low dimmed light. Head movement was restricted with an individually molded thermoplastic mask.

[ 18 F]-FDG-PET 3D-images were obtained following a transmission scan for attenuation correction, from a Siemens-ECAT-EXACT-HR+ tomograph which collects 63 simultaneous slices (intrinsic in-plane resolution: 4.3mm; voxel size: 2.42×2.42×2.43 mm 3 ). A summed image corresponding to the attenuation-and decay-corrected uptake of the [ 18 F]-FDG, expressed in activity concentration (Bq/cc), was obtained from two 3D-time-frames (ten minutes each) collected 30 to 50 min after intraveinous injection of the radioligand. The mean (SD) injected 18 F-fluorodeoxyglucose radioactivity was 155.6 (9.25) MBq in the patients (see 3D Structural Magnetic resonance images (MRI) were acquired on a 1.5 Tesla GE Signa scanner (General Electrics Medical Systems, Milwaukee, WI, USA) using a T1-weighted spoiled gradient recalled sequence (124 contiguous slices; field of view 24cm; 256x256 matrix; voxel size: 0.94×0.94×1.3 mm 3 ).

Each [ 18 F]-FDG-PET and T1-weighted images were visually inspected for artifacts. Structural pre-processing. To take into account a possible effect of structure on [ 18 F]-FDG images, voxel-based morphometry (VBM) was performed using the unified segmentation implemented in SPM5 [START_REF] Ashburner | Unified segmentation[END_REF]. T1-weighted images were spatially normalized, segmented into gray and white matter and modulated before being smoothed with a 10 mm full-width half-maximum (FWHM) isotropic gaussian kernel.

Image processing

All of the

[ 18 F]-FDG-PET pre-processing. For the healthy subject group, each [ 18 F]-FDG image was coregistered to its corresponding T1-weighted image using a rigid-body model.

Transformation matrices obtained during the spatial normalization of the T1-weighted images to the MNI template were then applied to the resulting [ 18 F]-FDG images. The mean image of these [ 18 F]-FDG images was computed, providing a [ 18 F]-FDG template that was then smoothed with an 8 mm FWHM isotropic gaussian kernel. Finally, all subjects [ 18 F]-FDG images were spatially normalized to this template and smoothed with an 8 mm FWHM isotropic gaussian kernel.

Statistical analysis

Demographic and Clinical data. Analyses were performed using JMP ® 8 software from SAS (SAS institute inc., SAS campus drive, Cary, North Carolina, USA). Unpaired t-tests and χ 2 tests were used for comparing continuous and categorical variables, respectively, in betweengroup comparisons. Statistical significance was defined as p<0.05, two-tailed.

Image analysis. Voxel-wise comparisons were performed using SPM5 within the framework of the General Linear Model (GLM). All results were obtained within the mask of the hypothesized regions of interest, drawn using Wake Forest University (WFU) PickAtlas [START_REF] Maldjian | An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets[END_REF], including inferior frontal, middle frontal, superior frontal, medial frontal, and orbital gyri, ACC, inferior temporal and parahippocampal gyri, insula, amygdala, caudate, and "extra-nuclear" fiber tracts. Brain locations were reported as x, y, and z coordinates in Montreal Neurologic Institute (MNI) space and WFU PickAtlas was used to convert MNI coordinates into Talairach coordinates. The Talairach and Tournoux atlas [START_REF] Talairach | Co-planar stereotaxic atlas of the human brain: an approach to medical cerebral imaging[END_REF] was used to identify brain regions and approximate Brodmann areas (BA). MRI image analysis. Group comparisons of structural images were performed on gray matter and white matter tissues using an analysis of covariance (ANCOVA) with group (nonresponders, responders, healthy subjects) as between-subject factor and age, gender and total intracranial volume as confounding covariates. An F-test was used to compare the 3 groups. Significance threshold was set at p<0.05, False Discovery Rate (FDR) corrected for multiple comparisons.

PET image analysis. [ 18 F]-FDG 3D-images were entered into the General Linear Model within SPM5; in order to control for global glucose uptake effects we used proportional scaling global normalization, yielding an index of regional-relative to global-glucose uptake (gluMI). Baseline gluMI were compared between responder, nonresponder patients, and healthy subjects using an ANCOVA with group as between-subject factor and age and gender as confounding covariates. An F-test was used to compare the 3 groups. Significance threshold was set at p<0.05, False Discovery Rate (FDR) corrected for multiple comparisons.

Thereafter, post-hoc t-tests were performed within the regions where the F-test was significant. The "healthy group versus responders", "healthy group versus nonresponders", and the "responders versus nonresponders" contrasts were examined. Significance was set at p<0.001, uncorrected. Cluster significance thresholds (extent threshold) were set at 10 contiguous voxels (voxel size = 8 mm 3 ) to reduce type I errors introduced by potential noise.

PET-MRI analysis. Relations between gluMI and structure were assessed using the Biological Parametric Mapping (BPM) toolbox [START_REF] Casanova | Biological parametric mapping: A statistical toolbox for multimodality brain image analysis[END_REF]. BPM combines information from different imaging modalities on a voxel-wise basis using the GLM. Imaging variables are integrated on a voxel-wise basis, so each voxel has a unique regression design that includes the value of each imaging modality for that voxel. Group comparisons of regional cerebral gluMI after accounting for volumetric changes were performed using an ANCOVA with group as between-subject factor, age and gender as basic confounding covariates, and structural images as voxel-dependent confounding covariate. t-tests were performed within the same regions where the F-test of the PET analysis was significant. The "healthy group versus responders", "healthy group versus nonresponders", and the "responders versus nonresponders" contrasts were examined. Significance was set at p<0.001, uncorrected.

Cluster significance thresholds (extent threshold) were set at 10 contiguous voxels (voxel size = 8 mm 3 ).

Correlation analyses. In order to explore the relationships between gluMI in different regions (PET regional correlations) or between gluMI and clinical scores (PET-clinical symptoms correlations), individual gluMI values at significant peak-voxels were extracted from the SPM F-map. Correlation analyses were performed with JMP ® 8. All correlations were examined using ANCOVAs with group as between-subject factor.

As our aim was to examine intergroup differences, we report herein the interaction effects between groups and correlations. Significance was set at p<0.05 Bonferroni corrected, for all ANCOVAs. Post-hoc correlations were performed in each group using Pearson's r test, when an interaction effect was significant.

RESULTS

Demographic, clinical and treatment characteristics of the patients

Responders to TMS did not differ from nonresponders with regard to clinical symptoms at baseline, disease history, comorbidities, family history of depression, TMS parameters, type of ongoing and past treatments including antidepressants, mood stabilizers, antipsychotics, benzodiazepines or ECT prescriptions. No difference was found between subgroups regarding the resistance stage (χ 2 = 1.48, df=2, p=0.48) (Table 1 andtable S1).

Additionally, bipolar (BP) and unipolar (UP) TRD patients did not differ regarding age (BP: mean age(SD)= 47(5.9), UP: mean age(SD)= 47.8(8.1), Student's t test score=0.27, df=29, p=0.79), gender (BP:6 women, 5 men, UP: 14 women,6 men, χ 2 =0.73, df=1, p=0.39), age at onset (BP: 26.3(7.7), UP: 29.3(10.8), t score=0.82, df=28, p=0.42), duration of illness (BP: 20.7(7.3)years, UP: 18.2(10.0)y, t score= 0.73, df=28, p=0.47), duration of depressive episode (BP: 3.4 (3.3)y, UP: 2.7(1.8)y, t score=0.77, df=28, p=0.44), number of depressive episodes (BP: 4.2(1.9), UP: 3.4(1.8), t score=1.07, df=27, p=0.29), or family history (BP: yes=4, UP: yes=10, χ 2 =0.54, df=1, p=0.46). UP and BP did not differ in depression scores improvement rate (BP: 55.6 (9.0) %, UP: 44.8 (6.7)%, t score= 0.96, df=29, p=0.34).

PET findings

There was a main effect of group on the [ 18 F]-FDG-PET images before accounting for structural differences, as evidenced by the results of the F-test, which revealed a pattern of several regions (Table 2). Post-hoc t-tests performed in those regions showed that both responders and nonresponders to TMS had, in comparison with healthy subjects, lower prefrontal gluMI in right ACC, right DLPFC, and left ventrolateral prefrontal (VLPFC) cortex. Additionally, nonresponders had lower gluMI in left ACC, left DLPFC, right and left anterior insula, and left OFC. They also displayed higher gluMI in left inferior temporallimbic regions including uncus, left amygdala, and left uncinate fasciculus. No significantly higher gluMI values were found in responders in comparison with healthy subjects. In nonresponders relative to responders, gluMI was higher in the left amygdala, uncinate fasciculus and anterior commissure, and lower in the left OFC (Table 2, Figure 1).

MRI corrected PET findings

When accounting for volume differences, regions that survived the volume correction included right ACC, right and left insula, left DLPFC, left uncus at BA20, and left amygdala (Table 2). Volume fully accounted for lower gluMI in the left OFC, left VLPFC (BA9/47), left ACC and right DLPFC. Higher gluMI was not related to white matter volume in uncinate fasciculus or in anterior commissure.

MRI Findings

Voxel-based comparison of the GM maps revealed no main effect of group at the p<0.05 FDR corrected for multiple comparisons. However, at a more permissive threshold GM volume reductions were observed in some regions where gluMI was low, including the left OFC (BA10, t=3.52, p<.001 unc, k=36 voxels) in nonresponders relative to responders and relative to healthy subjects (t =2.87, p <.002, k=312 voxels), and in the left ACC (BA 32, t =3.40, p <.001 unc, k=46) in nonresponders relative to healthy subjects. No regional difference was found in white matter maps between responders, nonresponders and healthy subjects.

FDG -PET Regional Correlations

Responders, nonresponders, and healthy subjects significantly differed in several PET regional correlations.

Interactions (F-tests).

Significant interactions (gluMI X group) were found between left OFC and several regions in which metabolic level was independent of structure. Regions included left amygdala (F=8.20, df=2), left DLPFC at BA9 (F=11.79, df=2), left VLPFC at BA9 (F=14.34, df=2), right ACC (F=8.18, df=2), and left uncus (BA20) (F=8.31, df=2). 3). In the nonresponders, left amygdala gluMI was negatively correlated with gluMI in the left OFC, while a positive correlation in the responders and no correlation in the healthy subjects was observed (Figure 2). A negative correlation was also found in nonresponders between gluMI in left uncus and left OFC, but no correlation was found in responders or in healthy subjects. Positive correlations were observed in nonresponders, but not in other groups, between OFC and 1/ prefrontal regions including left DLPFC at BA9; 2/ left VLPFC at BA9; and 3/ right ACC.

Post-hocs tests (Table

Clinical correlations

No significant interaction (gluMI X group) was found between gluMI and baseline scores of depression, anxiety scores or between gluMI and change in depression scores after TMS.

DISCUSSION

Pre-treatment brain glucose metabolism at rest and regional volumes were investigated in patients with resistant depression to assess whether these characteristics could differentiate responders and nonresponders to prefrontal TMS. Nonresponders had a pattern of widespread low prefrontal metabolism associated with high temporal-limbic resting-state metabolism.

Low metabolism was related to variations in GM volume in several regions, notably in left OFC and ACC where lower GM volume was observed in nonresponders. Uniquely, different correlations between left amygdala and left OFC metabolism distinguished the nonresponders, responders and healthy subject groups.

Functional correlates of response to TMS

Few studies have examined brain correlates of antidepressant response to high-frequency TMS and most of those studies have used single photon emission computerized tomography (SPECT) in smaller groups of patients, pooling both medication-resistant and nonresistant depressed patients, with response defined as a 30% decrease in depression scores (50% in the present study). Responders to TMS have been found to have higher blood flow in inferior frontal lobe before stimulation [START_REF] Teneback | Changes in prefrontal cortex and paralimbic activity in depression following two weeks of daily left prefrontal TMS[END_REF], or lower blood flow in amygdala [START_REF] Nadeau | Cerebral blood flow changes in depressed patients after treatment with repetitive transcranial magnetic stimulation: evidence of individual variability[END_REF] than nonresponders, consistently with our results. Improvement in depression scores with TMS has also been found to positively correlate with blood flow in the anterior cingulate [START_REF] Langguth | Pre-treatment anterior cingulate activity as a predictor of antidepressant response to repetitive transcranial magnetic stimulation (rTMS)[END_REF] and in the right peri-insular region [START_REF] Mottaghy | Correlation of cerebral blood flow and treatment effects of repetitive transcranial magnetic stimulation in depressed patients[END_REF].

Consistently with other studies of TRD patients [START_REF] Videbech | PET measurements of brain glucose metabolism and blood flow in major depressive disorder: a critical review[END_REF][START_REF] Konarski | Relationship between regional brain metabolism, illness severity and age in depressed subjects[END_REF], the present findings point to a pattern of decreased prefrontal glucose metabolism, particularly in regions involved in TMS response, including the insula, the ACC and the OFC. In nonresponders to TMS relative to responders, low glucose metabolism was observed in those regions, and related to lower GM volumes in the left ACC and in the rostral part of the left OFC. In line with this finding, hypometabolism in OFC has been hypothesized to be related to GM reduction in more severely-ill patients such as the nonresponders to TMS in our study [START_REF] Drevets | Prefrontal cortical-amygdalar metabolism in major depression[END_REF], while normal glucose metabolism in OFC has been reported in medicationresistant depressives with no GM reduction [START_REF] Drevets | Neuroimaging studies of mood disorders[END_REF][START_REF] Drevets | Functional anatomical correlates of antidepressant drug treatment assessed using PET measures of regional glucose metabolism[END_REF]. Thus, the present results support that OFC gray matter volume reduction might precondition a negative response to prefrontal TMS.

A functional pattern of prefrontal hypometabolism associated with temporal limbic hypermetabolism has previously been reported in various groups of depressed patients regardless of resistance to treatment [START_REF] Konarski | Relationship between regional brain metabolism, illness severity and age in depressed subjects[END_REF][START_REF] Brooks | Corticolimbic metabolic dysregulation in euthymic older adults with bipolar disorder[END_REF][START_REF] Savitz | Bipolar and major depressive disorder: neuroimaging the developmental-degenerative divide[END_REF]. In the present study, a functional amygdala hypermetabolism was only observed in the TRD nonresponders to TMS, regardless of GM volume. Thus, the present findings raise the hypothesis that high metabolic levels of the temporal-limbic regions, particularly the amygdala, might precondition non response to prefrontal TMS.

Frontal-limbic correlations in resistant depressives

Distinct correlations were found between OFC and the amygdala in nonresponders, responders, and healthy subjects, indicating different frontal-limbic relations depending on the group. The OFC is involved in coding of the identity of the sensory stimuli and their rewarding properties (Schultz et al, 1998), and is connected with the lateral nuclei of the amygdala, which activity appears closely related to the context and level of aversiveness of the stimuli (Zald, 2003). Through inhibitory projections to the amygdala, the lateral OFC is consequently involved in controlling information processing, particularly representation of rewards and punishments, and regulates behavior expression and emotional responses [START_REF] Hooker | The role of the lateral orbitofrontal cortex in the inhibitory control of emotion[END_REF]. Thus, GM reductions in this region may disturb interactions between orbital cortex and projections to the amygdala, and to other connected regions such as the cingulate cortex or the striatum. Our findings are consistent with a frontal-limbic dysregulation in the subgroup of nonresponders leading to decreased frontal cortical regulation of temporal limbic activation in response to negative stimuli, as has previously been hypothesized in TRD (Mayberg, 1997b). In those patients, and as prefrontal TMS seems to modulate OFC activity (Nadeau et al , 2002), GM reduction of the OFC might partly account for the observed specific resistance to TMS. Such resistance to TMS in those patients may additionally be related to altered white matter tracts in the ventral frontal-temporal limbic network. Indeed, higher glucose metabolism was detected in left uncinate fasciculus and anterior commissure in nonresponders, independently of white matter volumes. As pointed out by Buschbaum and coworkers (2007), the increased metabolism might reflect increased energy need due to defects in white matter, which in turn, may lead to inefficiency in brain circuitry. The uncinate fasciculus is a bidirectional pathway that links the anterior temporal lobe and amygdala with the medial and orbital prefrontal cortices, and the anterior commissure is a fiber bundle that connects the inferior temporal with sites including the amygdala and orbitofrontal cortex [START_REF] Schmahmann | Uncinate fasciculus[END_REF]. Thus, it can be speculated that, due to changes in the OFC/VLPFC, and connecting white matter tracts such as the uncinate fasciculus, TMS would fail to modulate the prefrontal cortex and temporallimbic structures activity through the ventral-limbic pathway.

Limitations

First, while there is no larger FDG-PET study associated with a controlled TMS trial, the short treatment duration might be seen as a limitation. Indeed, TMS has been found to be more effective with longer treatment durations (O'reardon et al, 2007). Thus, some nonresponder patients in this study would perhaps have been responders had they been treated over a longer period of time. However, despite the small number of subjects and short treatment duration, the results of the clinical trial (Paillère [START_REF] Martinot | Influence of prefrontal target region on the efficacy of repetitive transcranial magnetic stimulation in patients with medication-resistant depression: a [18F]-fluorodeoxyglucose PET and MRI study[END_REF] showed that the chosen protocol accelerated the effect of TMS, with a strong effect size (Cohen's d=0.78;[START_REF] Cohen | Statistical power analysis for the behavioral sciences[END_REF] over sham stimulation, probably accounting for the high response rate in the study.

A second limitation relates to previous medication. It was not possible to withdraw all treatments in those difficult patients, and it was not possible to assess all the dosages prescribed along their illness as they had very long illness durations. However, antidepressant and mood stabilizers were prescribed in usual standard dosages in all the patients. Effects of medication on FDG measures have been studied in a few studies, with inconsistent results.

Apparently, brain metabolic responses to antidepressants vary according to the underlying physiopathology of the patient and the degree of symptomatic improvement [START_REF] Saxena | Differential cerebral metabolic changes with paroxetine treatment of obsessive-compulsive disorder vs major depression[END_REF], and chronic antidepressant drug treatment might reduce metabolism in the amygdala and ventral ACC in depressed subjects showing a positive treatment response only [START_REF] Drevets | Functional anatomical correlates of antidepressant drug treatment assessed using PET measures of regional glucose metabolism[END_REF]. Here, the patients were all similarly resistant to medication according to Thase and Rush criteria, and the number of patients on antidepressants and duration of illness or episode were comparable across groups. Also, the number of patients on antipsychotics and the dosages were very small and similar in both patient groups, as was the number of patients on mood stabilizers; thus, it is unlikely that their effect would have confounded the metabolic findings. Finally, most patients were on low doses of benzodiazepines, which have been reported to induce decreases in glucose metabolic rates in regions such as the basal ganglia, thalamus or visual cortex [START_REF] Martinot | Drug washout issues in studies of cerebral metabolism by positron emission tomography in psychiatric patients[END_REF]. These effects might have blurred some differences with the comparison subjects, but as the patients' prescriptions did not differ across subgroups, it is unlikely that these effects would have confounded the results of direct comparison of responders with nonresponders. Moreover, the metabolic findings are consistent with findings of other studies of untreated depressed patients [START_REF] Teneback | Changes in prefrontal cortex and paralimbic activity in depression following two weeks of daily left prefrontal TMS[END_REF]. Third, in a few studies, resistance to TMS has been associated with previous medication resistance, or anxiety comorbidity [START_REF] Fregni | Predictors of antidepressant response in clinical trials of transcranial magnetic stimulation[END_REF][START_REF] Brakemeier | Patterns of response to repetitive transcranial magnetic stimulation (rTMS) in major depression: replication study in drug-free patients[END_REF][START_REF] Lisanby | Daily left prefrontal repetitive transcranial magnetic stimulation in the acute treatment of major depression: clinical predictors of outcome in a multisite, randomized controlled clinical trial[END_REF]. In this study, medication resistance was comparable across responders and nonresponders to TMS. Regarding comorbid anxiety, the number of patients with a comorbid anxiety disorder was similar in both groups, and anxiety scores before scanning were similar across groups, and did not correlate with PET measures, which suggests that nonresponse parameters did not depend on anxiety comorbidity in this group.

Fourth, the pooling of patients with unipolar resistant depression and with resistant depression in the context of history of bipolar disorder may be considered as a limitation. At variance with most previous reports, the present study aimed at assessing [ 18 F]-FDG in highly treatment-resistant patients notwithstanding the UP/BP dichotomy, rather than to compare UP or BP subgroups. In addition, many imaging studies indicate increased resting state metabolism in the amygdala in Major Depressive Disorder as well as in BP depression [START_REF] Savitz | Bipolar and major depressive disorder: neuroimaging the developmental-degenerative divide[END_REF]. Regarding orbitofrontal cortex, resting state activity has generally been found similarly increased both in UP and BP depressives [START_REF] Drevets | Neuroimaging studies of mood disorders[END_REF], though in more severely ill or TRD patients, studies have found no change or decreased function [START_REF] Mayberg | Paralimbic hypoperfusion in unipolar depression[END_REF][START_REF] Savitz | Bipolar and major depressive disorder: neuroimaging the developmental-degenerative divide[END_REF]. Thus, putative differences in functional pattern reported in the literature between BP and UP patients are not clear-cut.

In addition in the present study, the UP/BP ratios were similar in both responders and nonresponders subgroups, and no difference was found between UP and BP patients regarding improvement rates or depression history, or when comparing UP and BP groups using a similar image analysis, within the same regions of interest and at the same statistical threshold as in the responder/nonresponder analysis. Regarding the responder/nonresponder comparisons, and although the groups were much smaller, the same left amygdala function and left OFC gray matter pattern still distinguished responders from non responders within each subgroup. Both in the BP and in UP the subgroups nonresponders had significantly higher amygdala gluMI (Wilcoxon test, BP: responders, mean(SD)=-1.36 (1.86), nonresponders mean(SD)=9.49 (2.04), χ 2 =5.63, p=0.02; UP: responders, mean(SD)=-2.19

(1.65), nonresponders mean(SD)=5.21 (1.82), χ 2 =6.48, p=0.01). Similarly, BP and UP nonresponders had significantly lower GM in OFC than their responder counterparts (Wilcoxon test, BP: responders, mean(SD)=0.04 (0.02), nonresponders mean(SD)=-0.03 (0.02), χ 2 =5.63, p=0.02; UP: responders, mean(SD)=0.01 (0.01), nonresponders mean(SD)=-0.04 (0.02), χ 2 = 5.73, p=0.02). In addition, the differential relation between amygdala and OFC gluMI according to treatment response (responders, nonresponders) was maintained in both the UP and BP diagnostic subgroups (gluMI x group interactions, UP: t = 2.13, p= 0.05; BP: t= 2.40; p= 0.05).

In summary, the results suggest that alterations in brain metabolism and structure influence the response to TMS. Response to prefrontal TMS might depend on orbitofrontal cortex volume and amygdala functioning. Further research is needed to determine the predictive value of such a functional pattern determined at an individual level in patients with resistant depression referred to TMS therapy.

Zald DH (2003). The human amygdala and the emotional evaluation of sensory stimuli. Brain Res Brain Res Rev 41(1): 88-123.

TITLES AND LEGENDS TO FIGURES

  [ 18 F]-FDG-PET and structural images were processed with Statistical Parametric Mapping software package (SPM5, Wellcome Department of Cognitive Neurology, University College, London, UK; http://www.fil.ion.ucl.ac.uk/spm), implemented on Matlab® (The Mathworks, Natik, Massachusetts; http://www.mathworks.com).

Figure 1 :

 1 Figure 1: Top, left frontal view: Main comparison (F-test) between responders, nonresponders and healthy subjects. Height threshold: p<0.05, false discovery rate corrected, extent threshold: 10 voxels (A). Bottom, lateral views: T-maps of baseline regional cerebral glucose metabolism index (gluMI) in 17 responders to rTMS (B, left) and in 14 non responders (C, right) compared with 39 healthy subjects, respectively (Height threshold: p<0.001, extent threshold: 10 voxels); Red color indicates higher gluMI, blue indicates lower gluMI. Statistical maps are projected on a normalized brain mesh.

Figure 2 :

 2 Figure 2: Regional correlations between regional cerebral glucose metabolism index in left frontal BA10 at MNI coordinates (x=-32, y=50, z=-4) and in left amygdala at (x=-24, y=-2, z=-28). Interaction: F=8.20, df=2, p<0.001. Red: negative correlation in nonresponders, Pearson's r=-0.55, p=0.042. Blue: positive correlation in responders, Pearson's r=0.52, p=0.031. Green: no correlation in healthy subjects, Pearson's r=0.07, p=0.68.

Table 1

 1 for doses in groups), and 155.78 (19.65) MBq in healthy subjects (t =0.04, df=67, p =0.96).

Table 1 . Demographic, clinical and stimulation characteristics of responders and nonresponders to rTMS

 1 χ 2 tests, Fisher exact tests, and t tests were used when appropriate. All statistics were two-tailed 2 years of education after primary school

	Variable	Responders	Nonresponders	p Value 1
		(n=17)	(n=14)
	Gender (M/F), n	6/11	5/9	.98
	Age, y, mean(SD)	48.7(6.2)	46.4(8.6)	.40
	Education, y 2 , mean(SD)	7.1(4.6)	7.8(4.3)	.69
	Annett scale Laterality score, mean(SD)	75.9(64.5)	98.5(3.2)	.22
	Age at onset, y, mean(SD)	30.1(10.9)	26.1(8.2)	.27
	Duration of illness, y, mean(SD)	18.2(10.9)	20.1(6.6)	.58
	Duration of episode, y, mean(SD)	3.6(2.7)	2.1(1.7)	.10
	Family depression, n(%)	9 (53)	5 (36)	.34
	Bipolar resistant depression, n(%)	6 (35)	5 (36)	.98
	Number of depressive episodes, mean(SD)	3.3(1.6)	4.07(2.09)	.30
	Number of manic episodes, mean(SD)	0.6(1.2)	0.4(0.8)	.44
	Comorbid Anxiety Disorder, n(%)	3 (18)	4 (29)	.47
	Tyrer anxiety scale score, mean(SD)	20.2(1.8)	20.7(2.1)	.87
	Left guided/ Right guided/Left standard rTMS, n	6/2/9	3/5/6	.26
	18FDG dose, Mbq, mean(SD)	154.1 (7.6)	157.3 (10.8)	.36
	MADRS scores, mean(SD)
	Baseline	32.1(7.7)	35.1(6.3)	.26
	End-point	8.9(4.0)	27.6(9.0)	<.001
	%improvement	71.0(12.8)	21.5(20.2)	<.001
	HDRS scores, mean(SD)
	Baseline	25.2(4.5)	26.7(5.7)	.41
	End-point	9.0(3.3)	21.6(6.8)	<.001
	%improvement	63.7(12.2)	19.0(16.1)	<.001
	Ongoing medications, n(%)
	Antidepressants	8(47)	6(43)	.81
	Mood stabilizers	5 (29)	6 (43)	.48
	Antipsychotics	7 (41)	7 (50)	.37
	Benzodiazepines	7 (41)	8 (57)	.28
	Past medications, n(%)
	Antidepressants	17(100)	14(100)	>.99
	Mood stabilizers	9 (53)	9 (64)	.72
	Antipsychotics	8 (47)	10 (71)	.27
	Benzodiazepines	11 (65)	11 (78)	.46
	ECT	6(35)	7 (50)	.92
	Abbreviations: rTMS, repetitive transcranial magnetic stimulation; MADRS, Montgomery & Äsberg Depression
	Rating Scale; HDRS, Hamilton depression rating scale, 21 items;18FDG, 18 fluorodeoxyglucose; ECT,
	electroconvulsive therapy.

1

Table 2 . Differences in regional cerebral glucose metabolism index between 17 responders and 14 nonresponders to rTMS compared with 39 healthy subjects Main effect of group

 2 NR, nonresponders; HS, healthy subjects; BA, Brodmann's area, ACC; anterior cingulate cortex; MFG, middle frontal gyrus; IFG, inferior frontal gyrus; ITG, inferior temporal gyrus; UF, uncinate fasciculus; AC, anterior commissure; ellipses, no cluster retrieved by comparison, or not applicable.Statistics at voxel-level (F-test, df=2, 65), corresponding to a minimum corrected threshold of p< .05 False Discovery Rate (FDR) corrected, with Height threshold F=6.97, and Extent threshold k=10 voxels 2 Post-hoc test, Height threshold was set at p<0.001, uncorrected, and extent threshold was set at 10 voxels 3 Empty cells indicate that the region is included in the same cluster as the region immediately above 4 Cluster size is expressed in number of voxels, with voxel size=8mm3 5 Montreal Neurological Institute, in millimeters, for significant peak-voxels

				1				Group Comparison, t value	2		Volume corr.	6 , t value	2
	Brain Region	BA Cluster Size 3,4	x, y, z MNI Coordinates 5		F-test 1	R<HS NR<HS NR>HS R>NR R<NR R<HS NR<HS NR>HS R<NR
	Right MFG	9	317	34	12	38	11.42	3.55	4.14	…	…	…	…	…	…	…
	Right Insula	45	191	36	22	12	9.75	…	3.97	…	…	…	…	3.25	…	…
	Right ACC	32	850	12	34	22	13.67	3.37	4.85	…	…	…	3.56	4.22	…	…
	Left ACC	32	…	-2	38	28	9.03	…	4.07	…	…	…	…	….	…	…
	Left MFG	9	26	-40	28	42	7.86	…	3.93	…	…	…	…	3.53	…	…
	Left IFG	9	630	-40	8	38	9.86	3.49	3.84	…	…	…	3.32	…	…	…
	Left IFG	47	…	-48	14	24	9.60	4.10	3.46	…	…	…	3.57	…	…	…
	Left MFG	46	…	-44	26	26	9.52	…	4.18	…	…	…	3.32	3.43	…	…
	Left MFG	10	43	-32	50	-4	8.17	…	3.94	…	3.79	…	…	…	…	…
	(Orbital)
	Left insula	45	60	-32	26	8	8.28	…	3.80	…	…	…	3.77	3.22	…	…
	Left Amygdala	…	439	-24	-2	-28	13.10	…	…	4.84	…	4.60	…	…	4.79	4.57
	Left ITG (Uncus) 20	…	-32	-10	-38	8.82	…	…	4.20	…	…	…	….	4.33	…
	Left UF	…	93	-32	4	-10	8.41	…	…	3.60	…	3.87	…	…	3.51	3.80
	Left AC	…	…	-32	-8	-10	9.05	…	…	…	…	4.23	…	…	…	4.08
	Abbreviations: R, responders;

1

Volume Corr., Volume corrected analysis of PET images with T1 anatomic images as voxel-dependent confounding covariate

The authors declare no conflict of interest and no financial interests in relation with the subject of this work. Over the past three years, Dr. Paillère Martinot has received compensation from Bristol-Meyers-Squibb, Lilly and Sanofi companies. Dr. Artiges has received compensation from Janssen laboratory; and Dr Galinowski from Ardix, Lilly, and Sanofi companies. ACKNOWLEDGMENTS This study was supported by grant PHRC/AOM-98099 from the Assistance Publique-Hôpitaux de Paris (AP-HP) and the French Health Ministry; grant INSERM-PROGRES A99013LS from the French Institute for Health and Medical Research (INSERM), and an AP-HP/INSERM interface grant (M-L Paillère-Martinot); Dr. D. Ringuenet was supported by the "Fondation pour la Recherche Médicale" (FRM) and the Atomic Energy Commission (CEA).

The authors are grateful to Vincent Brulon, AAS-T, for image acquisition, Thierry Delzescaux PhD, and Claude Comtat PhD, for help with image processing, and to Edouard Duchesnay, PhD, for help with Brainvisa.