Barbara André

Tom Vercauteren

Anna M Buchner

Muhammad Waseem Shahid

Michael B Wallace

Nicholas Ayache

An image retrieval approach to setup difficulty levels in training systems for endomicroscopy diagnosis

Learning medical image interpretation is an evolutive process that requires modular training systems, from non-expert to expert users. Our study aims at developing such a system for endomicroscopy diagnosis. It uses a difficulty predictor to try and shorten the physician learning curve. As the understanding of video diagnosis is driven by visual similarities, we propose a content-based video retrieval approach to estimate the level of interpretation difficulty. The performance of our retrieval method is compared with several state of the art methods, and its genericity is demonstrated with two different clinical databases, on the Barrett's Esophagus and on colonic polyps. From our retrieval results, we learn a difficulty predictor against a ground truth given by the percentage of false diagnoses among several physicians. Our experiments show that, although our datasets are not large enough to test for statistical significance, there is a noticeable relationship between our retrieval-based difficulty estimation and the difficulty experienced by the physicians.

Introduction

Objective The understanding of pathologies through the analysis of image sequences is a subjective learning experience which may be supported by modular training systems. Particularly, the early diagnosis of epithelial cancers from in vivo endomicroscopy is a challenging task for many non-expert endoscopists. Our objective is to develop a modular training system for endomicroscopy diagnosis, by adapting the difficulty level according to the expertise of the physician.

The training simulator, illustrated in Fig. 1 on the top right, consists in a quiz. Given a level of difficulty, a pool of endomicroscopic videos whose average difficulty matches the current level is randomly chosen from the set of the training videos. By iterating this process with increasing levels of interpretation difficulty, the physician may be able to learn faster.

State of the art in estimating interpretation difficulty Typical studies on query difficulty estimation consider textual queries, and not image queries. Besides, they usually do not predict the difficulty of the query interpretation but rather the performance of the query in order to estimate the quality of its retrieval results. However, given the tight analogy between text retrieval and image retrieval, the difficulty criteria used by these methods, most of which were presented in a survey by Hauff et al. [START_REF] Hauff | A survey of pre-retrieval query performance predictors[END_REF], may also be useful for our study. In particular, Zhao et al. [START_REF] Zhao | Effective pre-retrieval query performance prediction using similarity and variability evidence[END_REF] estimated the performance of a textual query from similarity scores, but also from term frequency -inverse document frequency (TF-IDF) weights [START_REF] Salton | Term-weighting approaches in automatic text retrieval[END_REF] extracted during the indexing time. In all these studies, the predictor validation process takes as ground truth an indicator of the performance of the retrieval system, such as the Average Precision (AP). Nevertheless, Scholer and Garcia [START_REF] Scholer | A case for improved evaluation of query difficulty prediction[END_REF] demonstrated that the correlation between the estimated difficulty and the measured retrieval performance highly depends on the chosen retrieval system. Considering human performance in rating x-ray images as a ground truth, Schwaninger et al. [START_REF] Schwaninger | A statistical approach for image difficulty estimation in X-ray screening using image measurements[END_REF] proposed a statistical approach to estimate the image query difficulty solely from image measurements. Turpin and Scholer [START_REF] Turpin | User performance versus precision measures for simple search tasks[END_REF] highlighted the fact that it is not easy to establish, for simple tasks like instance recall or question answering, a significant relationship between human performance and the performance of a retrieval system that uses precision-based measures to predict the query difficulty.

For our study, we consider videos as queries. We propose to learn a query difficulty predictor using relevant attributes from a Content-Based Video Retrieval (CBVR) method. We have two types of ground truths. For video retrieval, a diagnosis ground truth is the set of histological diagnoses of the biopsies associated to all the videos of the database. For interpretation difficulty, a difficulty ground truth is given by the percentage of false video-based diagnoses among several physicians on a subset of the video database. Histological diagnosis and video-based diagnosis both consist in differentiating benign from neoplastic (i.e. pathological) lesions. In these conditions, we aim at establishing a relationship between the physicians performance and our predictor.

Materials Probe-based confocal laser endomicroscopy (pCLE) allows the endoscopist to image the epithelial surface in vivo, at microscopic level with a miniprobe, and in real-time (12 frames per second) during an ongoing endoscopy.

The first pCLE database is of colonic polyps videos acquired by physicians at the Mayo Clinic in Jacksonville, Florida, USA. 68 patients underwent a surveillance colonoscopy with pCLE for fluorescein-aided imaging of suspicious colonic polyps before their removal. For each patient, pCLE was performed of each detected polyp with one video corresponding to each particular polyp. Our resulting Colon database is composed of 121 videos (36 benign, 85 neoplastic) split into 499 stable video sub-sequences (231 benign, 268 neoplastic). 11 endoscopists, among whose 3 experts and 8 non-experts, individually established a pCLE diagnosis on 63 videos (18 benign, 45 neoplastic) of the database. On the non-expert diagnosis database, interobserver agreement was assessed in the study of Buchner et al. [START_REF] Buchner | The learning curve for in vivo probe based Confocal Laser Endomicroscopy (pCLE) for prediction of colorectal neoplasia[END_REF], with an average accuracy of 72% (sensitivity 82%, specificity 53%). On the expert diagnosis database, Gomez et al. [START_REF] Gomez | Interobserver agreement and accuracy among international experts of probe-based confocal laser microscopy (pCLE) in predicting colorectal neoplasia[END_REF] showed an interobserver agreement with an average accuracy of 75% (sensitivity 76%, specificity 72%). Thus, although pCLE is relatively new to many physicians, the learning curve pattern of pCLE in predicting neoplastic lesions was demonstrated with improved accuracies in time as observers' experience increased.

The second pCLE database is related to a different clinical application, namely the Barrett's Esophagus, and was provided by the multicentric "DONT BIOPCE" [START_REF] Biopce | [END_REF] For all these training videos, the pCLE diagnosis, either benign or neoplastic, is the same as the gold standard established by a pathologist after the histological review of biopsies acquired on the imaging spots.

Estimating the interpretation difficulty

For difficulty estimation, our ground truth is given by the percentage, for each query video, of false diagnoses among the physicians.As the understanding of video diagnosis by the physicians is driven by the observation of visual similarities between the query video and training videos, it makes sense to predict the query difficulty based on similarity results of video retrieval.

To learn a difficulty predictor, our idea is to exploit, as relevant attributes, the results of our video retrieval method applied to the training database. Potential relevant attributes are the class c q ∈ {-1, +1} of the video query q, the classes c i∈{1,k} ∈ {-1, +1} of its k nearest neighbors and the similarity distances δ i∈{1,k} to them. Given the small number of videos tested by the involved physicians, too many attributes for difficulty learning may lead to over-fitting. For this reason, we decided to extract one efficient and intuitive difficulty attribute α from the retrieval results. For each query video, we considered the retrieval error between the average of the neighbors' votes and the class of the query: α q = 1 -c q ( i∈{1,k} c i w c i )/( i∈{1,k} w c i ) , where w -1 = 1 and w +1 is a constant weight applied to the neoplastic votes. Introducing w +1 allows us to take into account the possible emphasis of neoplastic votes with respect to the benign votes. Our query difficulty predictor P is thus defined as P (q) = α q for each query video q. Its relevance can be evaluated by a simple correlation measure between the estimated difficulties of all tested videos and their ground truth values. In this case, as there is no learning process, cross-validation is not necessary.

Video retrieval method

As one of the most popular method for Content-Based Image Retrieval (CBIR), the Bag-of-Visual-Words (BoW) method presented by Zhang et al. [START_REF] Zhang | Local features and kernels for classification of texture and object categories: a comprehensive study[END_REF] aims at extracting a local image description that is both efficient to use and invariant with respect to viewpoint changes and illumination changes. Its methodology consists in first finding and describing salient local features, then in quantizing them into K clusters named visual words, and in representing the image by its signature which is the histogram of visual words.

As the field-of-view (FOV) of single images may sometimes be insufficient to establish a diagnosis, we revisited in [START_REF] André | Endomicroscopic video retrieval using mosaicing and visual words[END_REF] the standard BoW method to retrieve videos, and not only single images. We consider each video as a set of stable subsequences corresponding to a relatively smooth movement of the pCLE probe along the tissue surface. We then use a video-mosaicing technique to project the temporal dimension of each sub-sequence onto one mosaic image with a larger FOV and of higher resolution. Thus, each video is a set of mosaic images.

We adapt the BoW method to retrieve endomicroscopic mosaic images, some of which are shown in Fig. 1 on the top left. In colonic polyps, a mesoscopic crypt and a microscopic goblet cell both have a rounded shape, but are different objects characterized by their different sizes. Our description should thus not be invariant with respect to scaling. Noticing that discriminative information is densely distributed in pCLE mosaic images, we decided to apply a dense detector made of overlapping disks of constant radius on a regular grid. For the description step, we used the Scale Invariant Feature Transform (SIFT) descriptor, whose combination with our dense detector keeps all the BoW related invariants except scale invariance. After mosaic image description, we define the signature of a video as the normalized sum of the visual word histograms associated to each of the constitutive mosaic images. Fig. 1 on the bottom shows a CBVR example, where the retrieved videos are not only similar but also unblinded, i.e. displayed along with their contextual and diagnostic information.

Evaluation of the relevance of the retrieval method

Evaluating the relevance of retrieval results is a difficult problem. Because of the subjective appreciation of visual similarities, it is quite difficult to have a groundtruth. A simple evaluation method is to perform a classification based on the retrieval results and to estimate its accuracy. As our focus is on retrieval and not on classification, we chose one of the most straightforward classification method, the k-NN method, even though any other method could be easily plugged in. We consider two classes, benign (vote = -1) and neoplastic (vote = +1). When considering k neighbors for a query, we compute the value of the weighted sum of their votes according to their similarity distance to the query, and we compare this value with an absolute threshold θ to classify the query as benign or neoplastic. θ can be used to arbitrate between sensitivity and specificity.

In the current work, we apply for the first time our video retrieval method to two different pCLE databases, Colon and Barrett. Given their relatively small sizes, we use for each of them the whole database both for training and testing. If we only perform a leave-one-out cross-validation, the independence assumption is not respected because there are several videos acquired from the same patient. Since this may cause bias, we chose to perform a leave-one-patient-out (LOPO) cross-validation: all videos from a given patient are excluded from the training set in order to be then tested as queries of our retrieval and classification methods.

For method comparison, we will take as references the following CBIR methods, which we extended to CBVR by applying our signature summation technique: the HH-SIFT method presented by Zhang et al. [START_REF] Zhang | Local features and kernels for classification of texture and object categories: a comprehensive study[END_REF] a sparse detector, the standard approach of Haralick features, the texture retrieval Textons method of Leung and Malik [START_REF] Leung | Representing and recognizing the visual appearance of materials using three-dimensional textons[END_REF], and an efficient image classification method presented by Boiman et al. [START_REF] Boiman | In defense of nearest-neighbor based image classification[END_REF], referred as "BruteForce", that uses no clustering.

For our retrieval method, we considered disk regions of radius 60 pixels for the Colon database, and 20 pixels for the Barrett database whose discriminative patterns appear at a finer scale. We then chose 20 pixels of grid spacing to get a reasonable overlap between adjacent regions and thus be nearly invariant with respect to translation. For the number K of visual words provided by the K-Means clustering, among the values from 10 to 30000 in the literature, we chose the value K = 100 whose performance appeared to be sufficient for our needs.

The accuracy results of video classification on Barrett are presented in Fig. 2 on the left. In agreement with the presented ROC curves, the accuracy results obtained on Colon are even better. Our retrieval method outperforms all the compared methods with a gain of accuracy greater than 12 percentage points (pp.) on Colon, and greater than 9 pp. on Barrett. McNemar's tests show that, when the number k of neighbors is fixed, the improvement of our method with respect to all others is statistically significant: p-value < 0.011 for k ∈ [START_REF] Hauff | A survey of pre-retrieval query performance predictors[END_REF][START_REF] Zhang | Local features and kernels for classification of texture and object categories: a comprehensive study[END_REF] on Colon and p-value < 0.043 for k ∈ [1, 2] ∪ [START_REF] Scholer | A case for improved evaluation of query difficulty prediction[END_REF][START_REF] Gomez | Interobserver agreement and accuracy among international experts of probe-based confocal laser microscopy (pCLE) in predicting colorectal neoplasia[END_REF] on Barrett. This shows the genericity of our retrieval method, which is successfully applied to two different clinical application, with: 93.4% of accuracy (sensitivity 95.3%, specificity 88.9%) at k = 3 neighbors on the Colon database, and 85.4% of accuracy (sensitivity 90.2%, specificity 80.7%) at k = 7 neighbors on the Barrett database.

Results of the difficulty estimation method

Results on the Barrett database. We experimented our difficulty predictor presented in Section 2 on Barrett. The best correlation results were obtained with k = 10 neighbors and a neoplastic weight w +1 = 0.4. The correlation values reach 0.78 when learning from the subset of videos diagnosed by all the physicians, 0.63 (resp. 0.80) when learning only from the experts (resp. the nonexperts). The corresponding joint histogram is presented in Fig. 3 on the top, along with the histogram of the difficulty ground truth values. We observe a noticeable relationship between ground truth and our proposed difficulty estimation, which confirms the efficiency of our retrieval-based attribute for intuitive difficulty estimation. Perspectives for the Colon database. On Colon the difficulty estimation results are not as good as on Barrett. With k = 10 neighbors and a neoplastic weight w +1 = 6, the correlation values reach 0.45 when learning from the subset of videos diagnosed by all the physicians, 0.30 (resp. 0.45) when learning from experts (resp. non-experts). In order to improve these results, we propose to investigate a machine learning-based approach, which will need more relevant attributes. As the video dataset for which we have the difficulty ground truth is relatively small, we decided to add one discriminative power attribute β and to learn the difficulty predictor from the two attributes α and β by using a robust linear regression model. Our discriminative power attribute β reflects the deviation of the "signed" discriminative power of the query signature, with respect to the benign and the neoplastic classes: β = std( i∈{1,K} f i d(i)). In this formula, the visual word i has a frequency f i in the video query and its "signed" discriminative power d(i) is given by the adapted Fisher criterion:

d(i) = (µ -1 -µ +1 )|µ -1 -µ +1 |/(0.5 (σ 2 -1 + σ 2 +1
)), where µ c and σ c are respectively the mean and the variance of the frequency distribution of the visual word i in the videos belonging to class c. The correlation values obtained by the robust linear regression model with cross-validation reach 0.48 when learning from the subset of videos diagnosed by all the physicians, 0.33 when learning only from the experts and 0.47 when learning only from the non-experts. Even if these correlation results are less convincing than those obtained on Barrett, the correlation tendency can be qualitatively appreciated. The corresponding joint histogram is presented in Fig. 3 on the bottom. To automate the optimal attributes selection and to explore more potentially relevant attributes for difficulty estimation, further experiments based on model selection need to be investigated, for example using the Akaike information criterion. Besides, selection criteria commonly used in active learning [START_REF] Hoi | Semi-supervised SVM batch mode active learning for image retrieval[END_REF] may help to provide a better difficulty estimation.

Conclusion

To our knowledge this study proposes the first approach to estimate interpretation difficulty for endomicroscopy training, based on an original method of Content-Based Video Retrieval. Our experiments have demonstrated that there is a noticeable relationship between our retrieval-based difficulty estimation and the difficulty experienced by the physicians. Moreover, we showed the promising genericity of our difficulty estimation method by applying it on two different clinical databases, one on the Barrett's Esophagus and the other on colonic polyps. Our method could also be potentially applied to other imaging applications.

On one hand we have the diagnosis ground truth for all the videos belonging to our two large databases, on the other hand we have the difficulty ground truth on a small subset of each database. The method proposed in this work can then be used to estimate the interpretation difficulty on the remaining videos. The complete databases could thus be used in a training simulator that features difficulty level selection. This should make endomicroscopy training more relevant. Finally, a clinical validation would be required to see whether such a structured training simulator could help shorten the physician learning curve.

Fig. 1 :

 1 Fig. 1: Top left: 6 mosaic images of Barrett (B: Benign, N: Neoplastic). Top right: Screenshot of www.cellvizio.net Self-Learning Tool, with the added difficulty level information. Bottom: Retrieval example of our CBVR method on Colon with the blinded query video (left) and its 5 most similar videos represented by single frames.

  study (Detection Of Neoplastic Tissue in Barrett's esophagus with In vivO Probe-based Confocal Endomicroscopy). Our resulting Barrett database includes 76 patients and contains 123 videos (62 benign, 61 neoplastic) split into 862 stable video sub-sequences (417 benign, 445 neoplastic). 21 endoscopists, among whose 9 experts and 12 non-experts, individually established a pCLE diagnosis on 20 videos (9 benign, 11 neoplastic) of the database.

Fig. 2 :

 2 Fig. 2: Left: Method comparison for the LOPO classification of pCLE videos with θ = 0 on Barrett. Middle and right: Corresponding ROC curves at k = 5 neighbors, on Barrett (middle) and on Colon (right).

Fig. 3 :

 3 Fig. 3: Left: Difficulty ground truth histograms on Barrett (top) and on Colon (bottom). Right: Joint histograms on Barrett (top) and on Colon (bottom); x-axis is the difficulty of all the physicians and y-axis is our estimated difficulty. On Barrett, 21 physicians, 9 expert and 12 non expert, individually diagnosed 20 videos. On Colon, 11 physicians, 3 expert and 8 non expert, individually diagnosed 63 videos.


J Relan

P Chinchapatnam

M Sermesant

K Rhode

H Delingette

R Razavi

N Ayache

Coupled Personalisation of Electrophysiology Models for Simulation of Induced Ischemic Ventricular Tachycardia

Despite recent efforts in cardiac electrophysiology modelling, there is still a strong need to make macroscopic models usable in planning and assistance of the clinical procedures. This requires model personalisation i.e. estimation of patient-specific model parameters and computations compatible with clinical constraints. Fast macroscopic models allow a quick estimation of the tissue conductivity, but are often unreliable in prediction of arrhythmias. On the other side, complex biophysical models are quite expensive for the tissue conductivity estimation, but are well suited for arrhythmia predictions. Here we present a coupled personalisation framework, which combines the benefits of the two models. A fast Eikonal (EK) model is used to estimate the conductivity parameters, which are then used to set the parameters of a biophysical model, the Mitchell-Schaeffer (MS) model. Additional parameters related to Action Potential Duration (APD) and APD restitution curves for the tissue are estimated for the MS model. This framework is applied to a clinical dataset provided with an hybrid X-Ray/MR imaging on an ischemic patient. This personalised MS Model is then used for in silico simulation of clinical Ventricular Tachycardia (VT) stimulation protocol to predict the induction of VT. This proof of concept opens up possibilities of using VT induction modelling directly in the intervention room, in order to plan the radio-frequency ablation lines.

Introduction

Cardiac arrhythmias are increasingly being treated by Radio-Frequency (RF) ablation procedures. These procedures still have unsatisfactory success rates of only 30-60% for VT, due to non availability of clinical consensus on optimum RF ablation patterns [START_REF] Aliot | Ehra/hrs expert consensus on catheter ablation of ventricular arrhythmias[END_REF]. There is still a need for substantial guidance in locating the optimum ablation lines. This guidance can be provided by personalised in silico cardiac electrophysiology models. Personalisation means estimation of the patient-specific model parameters which best fit the clinical data. It is required to reveal hidden properties of the tissue and to develop predictive models that can be used to improve therapy planning and guidance. There are a variety of cardiac EP models developed at various scales. These models can be broadly categorised into three main categories: Biophysical Models (BM), Phenomenological Models (PM) and Eikonal Models (EM). BM [START_REF] Noble | Modeling the heart[END_REF] model ionic currents and are the most accurate and complex but hardly suitable for parameter estimation from clinical data. PM [START_REF] Fitzhugh | Impulses and physiological states in theoretical models of nerve membrane[END_REF] are based on PDEs and mimic only the shape of action potential and are on intermediate level. EM [START_REF] Keener | Mathematical Physiology[END_REF] describes only the time at which a depolarisation wave reaches a given point. They can be very fast in computation [START_REF] Chinchapatnam | Model-based imaging of cardiac apparent conductivity and local conduction velocity for diagnosis and planning of therapy[END_REF], but less reliable in arrhythmia predictions due to the complexity of both refractoriness and curvature. To introduce models directly in the intervention room, the requirements are a low computational complexity, fast estimation of parameters and reliable predictions. These attributes cannot be found in one single model, thus here we present a new approach, where we combine two models to obtain these attributes and show an application to a clinical dataset. We also show how such personalised model can then be used to simulate in silico a clinical VT Stimulation protocol and can be potentially used to plan optimum RF ablation lines. In this paper, we present a coupled personalisation framework, which is fast and combines the benefits of an Eikonal (EK) model with those of a simplified biophysical model, the Mitchell-Schaeffer (MS) model. The fast 3D EK model is used to estimate the tissue conductivity parameter over the ventricles from the non-contact mapping of endocardial surface potential, using an adaptive iterative algorithm. This is then used to set the conductivity parameter of the 3D MS model. Additional parameters related to APD and APD restitution property of the tissue are then estimated locally using directly the 3D MS model. This framework is applied to a clinical data of an ischemic patient, containing of MR data for geometry and scar detection and electrophysiological data obtained from non-contact mapping. This data is obtained using Hybrid X-ray/magnetic resonance (XMR) suites [START_REF] Rhode | A system for real-time XMR guided cardiovascular intervention[END_REF]. The personalised 3D MS model is then used to simulate a clinical VT-Stim procedure to show a potential application of VT induction modelling.

Simulation of Electrophysiology Models

Anisotropic Fast Marching Model (EK Model) : The EK model simulates the propagation of the depolarization wave in quiescent tissue, ignoring repolarisation phase. The EK model is governed by eikonal-diffusion (ED) equation and is based on Fast Marching Method (FMM), it can be written as

c 0 D(x) ∇T (x) t M∇T (x) -∇ • (D(x)M∇T (x)) = τ (x) (1)
where the superscript t denotes transpose, c 0 is a dimensionless constant, and τ (x) is the cell membrane time constant. D(x) is the square of the tissue space constant along the fiber and is related to the specific conductivity of the tissue in the fiber direction. The anisotropicity is incorporated in the Diffusion tensor and is given by M. The nonlinear term (Eq 1) is solved using a fixed point iterative method combined with a very fast eikonal solver as explained in [START_REF] Sermesant | An anisotropic multi-front fast marching method for real-time simulation of cardiac electrophysiology[END_REF]. Biophysical Model (MS Model) : The MS model [START_REF] Mitchell | A two current model for the dynamics of cardiac membrane[END_REF] is a 2-variable simplified biophysical model derived from the 3-variable Fenton Karma (FK) ionic model [START_REF] Fenton | Vortex dynamics in three-dimensional continuous myocardium with fiber rotation: Filament instability and fibrillation[END_REF]. It models the transmembrane potential as the sum of a passive diffusive current and several active reactive currents including a sodium ion (influx) current and a potassium ion (outflux) current. Unlike FK model, it does not model the Calcium ion current. The MS model is described by the following system of Ordinary Differential Equations (ODE),

         ∂ t u = div(D∇u) + zu 2 (1 -u) τ in -u τ out + J stim (t) ∂ t z =    (1 -z) τ open if z < z gate -z τ close if z > z gate (2)
where, u is a normalised transmembrane potential variable, and z is a gating variable for sodium ion influx which makes the gate open and close, thus depicting the depolarisation and repolarisation phase. J in = (zu 2 (1 -u))/τ in represents the inward sodium ion current which raises the action potential voltage and J out = -u/τ out represents the outward potassium ion current that decreases the action potential voltage describing repolarisation. J stim is the stimulation current, at the pacing location. The diffusion term in the model is controlled by the diffusion tensor D. This spatial diffusion can be related to a pseudoconductivity. In the longitudinal direction of the fibre, this pseudo-conductivity is set to d which is one of the parameters we adjust, and to d/2.5 2 in the transverse directions. The electrophysiology model is solved spatially using P1 Finite Element Method (FEM), and in time using an semi-implicit scheme as Modified Crank-Nicolson/Adams-Bashforth (MCNAB) scheme, which is evaluated in terms of accuracy, stability and computational time [START_REF] Relan | Quantitative comparison of two cardiac electrophysiology models using personalisation to optical and mr data[END_REF].

Coupled Personalisation Method

Apparent Conductivity Parameter Estimation : Cardiac tissue conductivity is a crucial feature for the detection of conduction pathologies. The Apparent Conductivity (AC) of the tissue can be measured by the parameter D(x) in the EK model. It is initially estimated on the endocardial surface as a global value using a simple bisection method which matches the average conduction velocity of the measured Depolarisation Time (DT) isochrones to the simulated ones. Using it as an initial guess, an adaptive multi-level zonal decomposition algorithm is used, which minimizes the mean-squared difference of the simulated and measured DT isochrones at each level using a Brent's Optimisation Algorithm presented in [START_REF] Chinchapatnam | Model-based imaging of cardiac apparent conductivity and local conduction velocity for diagnosis and planning of therapy[END_REF]. Due to the absence of transmural electrical propagation information, we assume no variation across the left ventricle myocardium (excluding LV endocardium and scars) and hence we prescribe a single value for the myocardial tissue across the LV wall. The AC values for RV endocardium and RV myocardial mass are set at 5.0 mm and 0.64 mm (from literature [START_REF] Keener | Mathematical Physiology[END_REF]). The LV myocardial AC value is estimated by one-dimensional minimisation of the following cost function (mean-squared difference of simulated and measured isochrones at endocardium + squared difference of simulated and measured QRS duration). The simulated QRS duration is calculated as the difference between the maximum and the minimum depolarisation times in the biventricular mesh and the measured QRS duration is estimated from the surface ECG.

Coupling of EK and MS Model Parameters : The AC parameter for EK model d EK (D(x) in Eq 1) is a scale for the diffusion speed of the depolarisation wavefront in the tissue. The diffusion tensor used in Eq 1 is M = A DA t , where A is the matrix defining the fiber directions in the global coordinate system and D = d EK diag(1, λ 2 , λ 2 ). λ is the anisotropic ratio of space constants transverse and along the fiber direction and is 0.4 in human myocardium [START_REF] Keener | Mathematical Physiology[END_REF]. The model Conduction Velocity (CV) is related to d EK as,

c EK = c 0 d EK τ in 1D and c EK = α EK d EK + β EK in 3D (3)
where the constants α EK and β EK are introduced to take into account the curvature effect and numerical diffusion and discretization errors in 3D. The corresponding conductivity parameter for MS model, d M S is also a scale for the wave diffusion speed in the tissue. The diffusion tensor D used in Eq 2 is Parameter Estimation for APD : APD for a single heart cycle is defined by the model as, AP D max = τ close ln (1/h min ) where h min = 4 (τ in /τ out ) As we have only one measured APD dependent on three parameters, We chose to estimate τ close , while keeping the other parameter values from the literature [START_REF] Mitchell | A two current model for the dynamics of cardiac membrane[END_REF]. The reason is that τ close has no sensitivity towards the conductivity parameter estimation [START_REF] Relan | Quantitative comparison of two cardiac electrophysiology models using personalisation to optical and mr data[END_REF], whereas τ in and τ out do have. This defined relationship remains valid also in 3D thus allowing us to directly estimate locally at each vertex, the parameter τ close without model simulations.To have a smooth gradation of APD , where f is the heart rate, detected from the ECG waveforms. The parameter optimisation method used here is a non-linear constrained Active-Set Algorithm, with constraints on τ open to be in the range of literature values [START_REF] Mitchell | A two current model for the dynamics of cardiac membrane[END_REF].

D = A DA t ,

Application to clinical data

The coupled personalisation framework is applied on a clinical data obtained during an electrophysiology study in the hybrid X-ray/MR environment. The electrical measurements obtained using the Ensite system were registered to

Conclusion

This novel approach of coupling models for fast estimation of hidden parameters related to the cardiac tissue such as conductivity, APD and APD restitution could enable the clinical use of cardiac electrophysiology models. The parameter estimation algorithm is used on a real interventional data and the obtained results are very encouraging. The estimated conductivity, APD and APD restitution parameters are able to identify the healthy areas from the pathological ones (scar and isthmus). The personalised MS model was able to simulate a clinical VT-Stim protocol in order to assess the risk of VT and fibrillation. This opens up possibilities of introducing patient-specific models in clinics to provide aid in treatment and planning of RF ablation procedures. In future, we need to evaluate the prediction ability of the personalised models for arrhythmias.

  where A is the same as in EK model, but D = d M S diag(1, r, r) with r as conductivity anisotropy ratio in the transverse plane and is set to λ 2 as in EK model. The model CV here is related to d M S as,c M S ∝ d τ in in 1D and c M S = α M S d M S + β M S in 3D(4)where the constants α M S and β M S are introduced for the same reasons as of EK model, while τ in is kept as a constant. The estimated AC parameter d EK can then be used to estimate the parameter d M S . The parameter d EK gives model CV c EK , which is similar to the actual measured data CV (c msd ) after the parameter estimation step. Thus to have MS model CV (c M S ) similar to the measured data, it has to be similar to EK model CV (c EK ). The constants α EK and β EK represent numerical errors for EK model based on FMM. They are different from the constants α M S and β M S , which is based on FEM. These constants are determined in 3D for a mesh representing the slab of a tissue ([0 10] × [0 10] × [0 10]) (with a mean edge length of tetrahedra same as the ventricular mesh). We performed several simulations with various d EK and d M S values and noted the corresponding c EK and c M S values. Then, we fit the analytical curves given in Eq 3 & 4 in least square sense and determine the constants. The constants estimated are α EK = 5.21, β EK = 0.07, α M S = 3.12, β M S = 0.31. Then, the personalised d M S values are computed from corresponding estimated d EK values using the condition, that c msd = c EK = c M S after personalisation.

Fig. 1 .

 1 Fig. 1. (a) shows the MR data, segmented mesh with scars (in red), (b) shows XMR registration of Ensite LV surface with MR data mesh, with values projected from Ensite to MR LV surface,(c) shows the fibre orientation used, (d) shows the unipolar electrograms for detection (black dots) of depolarisation time (upper) and repolarisation time (lower) from positive(red), negative(blue) and biphasic(green) T waves

!Fig. 2 .

 2 Fig. 2. Upper row shows the comparison of the measured Depolarisation Time (DT) isochrones on the LV surface only with model simulated DT isochrones on the whole heart, lower row shows the same for measured (LV surface only) and model simulated (whole heart) APD maps

Fig. 3 .

 3 Fig. 3. (a) shows the conduction velocity estimated from AC maps, (b) shows the parameter τ close estimated for APD, lower τ close values has low measured APD (white contour) and vice versa, (c) shows the parameter τopen estimated for APD restitution and the heterogeneity of the restitution curves for the isthmus (black contour), low τopen values (red) have steep slopes & high (blue) have flat slopes for restitution curves. ular stimulation is a clinical protocol and consists of a number of extra stimuli introduced at two ventricular sites (RV-Apex & RV-Outflow tract), using various Cycle Lengths (CL), with varying coupling interval. This protocol is tested directly on the patient, without any planning, to collect information about the VT and to plan the RF ablation lines. It may be time consuming or fail, when VT is not inducible and recurrent. We use the personalised 3D MS model of the ischemic patient data to simulate in silico this protocol. Here we follow a conventional VT-Stim protocol with RV-Apex pacing site, 2 extrastimuli and a shortest coupling interval of 100 ms at 600 ms pacing cycle length. The results on inducibility are shown in Fig 4 and the causes of rentry were pacing location, restitution heterogeneity in isthmus compared to healthy Fig 3(c) and slow conductivity near the scars.

Fig. 4 .

 4 Fig. 4. (a) shows unipolar electrograms recorded for a clinical VT-Stim protocol, (b) shows the simulated protocol for two extrastimuli, with coupling interval of 100 ms. (c) show DT isochrones(in s) for S1 stimulus and (d) shows for S2, we have a unidirectional block created in the isthmus. (e) shows DT isochrones for induced monomorphic VT

corresponding author. (maxime.sermesant@sophia.inria.fr)

Acknowledgements The research leading to these results was partially supported by a Prize from Microsoft Research Cambridge and by the EUHeart project (FP7/2007-2013 under grant agreement n 224495).


B André

T Vercauteren

A M Buchner

M B Wallace

N Ayache

ENDOMICROSCOPIC VIDEO RETRIEVAL USING MOSAICING AND VISUAL WORDS

Keywords: Endomicroscopy, Bag of Visual Words (BVW), Leave-One-Patient-Out (LOPO), Mosaicing

In vivo pathology from endomicroscopy videos can be a challenge for many physicians. To ease this task, we propose a content-based video retrieval method providing, given a query video, relevant similar videos from an expert-annotated database. Our main contribution consists in revisiting the Bag of Visual Words method by weighting the contributions of the dense local regions according to the registration results of mosaicing. We perform a leave-one-patient-out k-nearest neighbors classification and show a significantly better accuracy (e.g. around 94% for 9 neighbors) when compared to using the video images independently. Less neighbors are needed to classify the queries and our signature summation technique reduces retrieval runtime.

INTRODUCTION AND CLINICAL CONTEXT

Probe-based confocal laser endomicroscopy (pCLE) enables dynamic imaging of tissue in vivo, at microscopic level with a miniprobe, and in real time during ongoing endoscopy. The still evolving semantic of the acquired pCLE videos remains complex for the physicians, for instance to differentiate benign and neoplastic (i.e. pathological) tissues of colonic polyps. Indeed, in these image sequences, there is a large variability of the appearance of polyp tissues having a given pathology. Besides, a video of a neoplastic polyp in the colon may also contain benign tissue. Expert physicians pointed out that the field-of-view (FOV) of single still images may not be large enough to make a robust diagnosis. The temporal dimension of videos is thus needed by the endoscopist to relate each image to its spatial context for the interpretation of partially visible macroscopic features. To aid the endoscopist in establishing a diagnosis, we propose to provide several videos that have a similar appearance to the video of interest but that have been previously annotated by expert physicians with a textual diagnosis, and potentially validated against ground truth information such as a real biopsy. In [START_REF] André | Endomicroscopic image retrieval and classification using invariant visual features[END_REF], we presented a content-based image retrieval (CBIR) method and successfully applied it for a pCLE database of colonic polyps. For the images of this database, we obtained better classification results than those reached by the state-of-the-art methods reviewed by [START_REF] Zhang | Local features and kernels for classification of texture and object categories: a comprehensive study[END_REF]. In this study, the same database has been enriched. We now aim at retrieving videos and not isolated images, by revisiting the Bag of Visual Words (BVW) method explained in Section 2 and by analyzing the impact of including spatial overlap between time-related images. Our content-based video retrieval (CBVR) contributions are presented in Section 2. We first exploit the registration results of the mosaicing technique of [START_REF] Vercauteren | Robust mosaicing with correction of motion distortions and tissue deformation for in vivo fibered microscopy[END_REF] to weight the contribution of each image region to its visual word, according to the overlap of this region with the other regions densely distributed in the images. Then, we compute the video signature with a signature summation technique. This reduces both retrieval runtime and training memory. To quantify the relevance of video retrieval, we perform a k-nearest neighbors (k-NN) classification with leave-one-patient-out (LOPO) cross-validation and compare our method with other methods. In Section 4, the classification results show that our approach achieves a better accuracy on the video database. Using video data improves the results in a statistically significant manner when compared to using the images independently. Moreover, fewer nearest neighbors are necessary to classify the query at a given accuracy, which is clinically relevant for the endoscopist, who will typically examine a reasonably small number of videos.

VISUAL WORDS FOR IMAGE RETRIEVAL

Let us first focus on the BVW method presented in [START_REF] André | Endomicroscopic image retrieval and classification using invariant visual features[END_REF] to retrieve images. As the probe is translating and rotating along the tissue surface, we aim at describing pCLE images in an invariant manner with respect to translation and rotation. As the rate of fluorescein which is administrated before imaging procedure is decreasing through time, we want this description to be also invariant by any affine transformation of intensity. To this purpose, the standard BVW [START_REF] Zhang | Local features and kernels for classification of texture and object categories: a comprehensive study[END_REF] method appeared to be the most appropriate since it extracts a local image description invariant with respect to illumination changes and some viewpoint changes, e.g., translations, rotations and scaling.

The BVW retrieval process can be decomposed into four steps: region detection, description, clustering and similarity measuring. In pCLE images, discriminative information is densely distributed over their entire field. To extract all the information, we decided to apply a dense detector made of overlapping disks. These disk regions are localized on a dense regular grid. At the description step, the SIFT [START_REF] Lowe | Distinctive image features from scaleinvariant keypoints[END_REF] descriptor computes, for each salient region, its gradient histogram vector, the gradient orientations being normalized with respect to the principal orientation of the salient region. Note that with the dense detection, the SIFT vectors are scale-dependent so the image description is not invariant by scaling. In fact, this is beneficial for our application because in colonic polyps, mesoscopic crypts and microscopic goblet cells both have a rounded shape, but are different objects characterized by their different size. To reduce the dimension of the description space, a standard K-Means clustering step builds K clusters, i.e. K visual words, from the union of the description vector sets gathered across all the images of the training database. Each description vector counts for one visual word, and an image is represented by a signature of size K which is its histogram of visual words, normalized by the number of its local regions. Given these image signatures, the distance between two images is typically defined as the χ 2 distance between their signature and the most similar training images to the image of interest are the closest ones.

CONTRIBUTIONS FOR VIDEO RETRIEVAL

On a still pCLE image, some discriminative patterns, e.g. an elongated crypt, may be too partially visible to characterize the pathology. Indeed, in CBIR results, we often observe still images with a very similar appearance but attached to contradictory diagnoses. For this reason, we investigated CBVR methods to retrieve similar videos instead of single images.

To address this problem, the temporal dimension of pCLE videos needs to be exploited, by including in the retrieval process the possible spatial overlap between the images from the same video sequence. In our pCLE video database, the dynamic motion within the tissue can be neglected when compared to the global motion of the probe sliding along the tissue surface. Successive images from the same video are only related by viewpoint changes, our approach can thus take advantage of the mosaicing technique of [START_REF] Vercauteren | Robust mosaicing with correction of motion distortions and tissue deformation for in vivo fibered microscopy[END_REF] to project the temporal dimension of a video sequence onto one mosaic image with a larger FOV and of higher resolution. For an efficient video retrieval, our objective is to build a single signature for each video. Indeed, having one short signature per video not only enables a reasonable memory space to store training data, but also considerably reduces the retrieval run-time.

In a first attempt, to take advantage of the super-resolution which may be present in the fused mosaic image after nonrigid registration, we considered the mosaic images as objects of interest for the retrieval. All videos of the database were splitted into stable sub-sequences identified by expert physicians. We built mosaics on these sub-sequences and applied the dense BVW method directly on the mosaic images, see the colored visual words in the middle of Fig. 1 for an illustration. Although this method provided us quite satisfying retrieval results, it takes a long runtime for the non-rigid reg- istration of the mosaicing process. For this reason, our second attempt investigated a more efficient method which could only use the coarse registration results of mosaicing, i.e. the translation results, computed in real time.

To this purpose, we first compute independently the signatures of all the images belonging to the database of video subsequences. Then, for each sub-sequence, we use the translation results of mosaicing to build a map of the overlap rates of all local regions belonging to the images of the sequence, see Fig. 1 on the right for an illustration. To define the signature H of a video sub-sequence S, we propose to take, for each image I of the sequence, the overlap rate ρ of each region r in the image and to weight the contribution of r to the frequency of its visual word w(r) by 1 ρ . The visual word histogram of the video sub-sequence is given by : H S (w) =

1 Z I∈S r∈I δ(w(r), w) ρ(r)
, where δ is the Kronecker notation and Z is the normalization factor. Thus, with our method called "Bag of Overlap-Weighted Visual Words" (BOWVW), we are now able to describe and retrieve stable sub-sequences of pCLE videos with a method similar to the one described in Section 2. We also define a signature for a video by considering the normalized sum of the signatures of its constitutive sub-sequences. Thanks to the BOWVW method and this histogram summation technique, the size of a video signature remains equal to the number of visual words, which reduces both retrieval runtime and training memory.

RESULTS AND DISCUSSION

At the Mayo Clinic in Jacksonville, the Cellvizio ® system was used to image colonic polyps during surveillance colonoscopies in 68 patients. For each patient were performed one or more acquisitions of pCLE videos, one video corresponding to one particular polyp of the patient. In each of these videos, stable sub-sequences were identified by clinical experts to attach a diagnosis to the video : they differentiate pathological patterns from benign ones, according to the presence or not of neoplastic tissue which is characterized by some irregularities in the cellular and vascular architectures. The resulting database is composed of 121 videos (36 benign, 85 neoplas- tic) and 499 video sub-sequences (231 benign, 268 neoplastic), leading to 4449 endomicroscopic images with a FOV of 240 µm (2292 benign, 2157 neoplastic). For all the videos of the database, the pCLE diagnosis, either benign or neoplastic, is the same as the "gold standard" established by a pathologist after histological review. On this database, we tested several methods to retrieve image sequences. All these methods share the same BVW methodology for the image description part, with the following parameters : we considered disk regions of radius 60 pixels to cover groups of cells; we then chose 20 pixels of grid spacing to get a reasonable overlap between adjacent regions and to be nearly invariant by translation; among the values found in the literature, we took K = 100 visual words, which yields satisfying retrieval results. Note that the number of visual words could be reduced or their frequency could be weighted by their discriminative power w.r.t. the two classes, benign and neoplastic. However, in this study we merely aim at analyzing the impact, on the retrieval performance, of weighting the visual word frequencies. Retrieval results of our BOWVW method applied on pCLE sub-sequences can be qualitatively appreciated in Fig. 2 for a benign query and a neoplastic query. In fact, it is very difficult to have a groundtruth for content-based data retrieval. This is the reason why we chose classification as a means to quantify the relevance of the similarity results, by considering two classes, the benign class and the neoplastic class. This will also allow us to compare the performance of several retrieval methods. More precisely, we used a nearest neighbors classification procedure that weights the votes of the k-nearest neighbors by the inverse of their χ 2 distance to the signature of the queried image. Thus the closest images are the most determinant. To ensure an unbiased classification, we performed a leave-onepatient-out (LOPO) cross-validation : all videos from a given patient are excluded from the training set before the training in order to be tested as queries of our retrieval and classification methods.

For the classification of video sub-sequences, we call : "Weighted-ImOfMos" the method using the BOWVW technique; "ImOfMos" the same method without overlap weight-ing (ρ = 1); "Mos" the method describing the single fused mosaic image obtained with non-rigid registration; "AverageVote-Im" the method describing all the images independently and averaging their individual votes. For the classification of the whole videos, the prefix "Sum-" means that we extended the methods with the signature summation technique to retrieve videos as entities; "Sum-Im" is the method summing all the individual image signatures of the video.

In the classification accuracies shown in Fig. 3 we observe that, for the classification of videos and from k = 3 neighbors, the "Sum-Weighted-ImOfMos" method has an accuracy which is better than the one of "Sum-Im", and equal or better than the one of 'Sum-ImOfMos" or "Sum-Mos" methods. Moreover, when comparing the methods for the classification of video sub-sequences, the number of classified data is sufficient to perform a McNemar's test which shows that, from k = 3, the accuracy of the "Weighted-ImOfMos" method is statistically better than the one of "AverageVote-Im", e.g. for k = 3 neighbors the p-value is equal to 0.021. The best video classification results that is observed before 10 neighbors is achieved by "Sum-Weighted-ImOfMos" for k = 9, with an accuracy of 94.2%, a sensitivity of 97.7% and a specificity of 86.1%. For a smaller number of neighbors, "Sum-Weighted-ImOfMos" already achieves a quite satisfying accuracy, e.g. 93.4% for 3 neighbors.

When considering k nearest neighbors for a query, we computed the value of the weighted sum of their votes (-1 for benign class, +1 for neoplastic class) according to their proximity to the query, and we compared this value with an absolute threshold θ to classify the query as benign or neoplastic. The accuracy results were obtained with θ = 0. In Fig. 3, we also draw ROC curves corresponding to the video subsequence classification results reached by the tested methods at a fixed number of neighbors k = 5, but with several values of the threshold θ ∈ [-1, 1]. The closer θ is to -1 (resp. +1), the more weight we give to the neoplastic votes (resp. the benign votes) and the larger the sensitivity (resp. the specificity) is. We notice that, for each method, there exists a clear accuracy peak at a negative value of θ, e.g. at θ = -0.6 for the "Weighted-ImOfMos" method. This illustrates the fact that global neoplastic features are more discriminative than the benign ones.

One may reproach our methodology for using an arbitrary number of visual words used for clustering and thus being dependent on the clustering results. This is the reason why we decided to compare it with a simple and efficient image classification method presented in [START_REF] Boiman | In defense of nearest-neighbor based image classification[END_REF], which uses no clustering. This method, referred as "BruteForce", consists in computing in the description space, for each local region of the query, its distances respectively to the closest region of the benign and pathological training data sets. If the sum of the benign distances D B is smaller than the sum of the neoplastic distances D N , the query is classified as benign, otherwise as neoplastic. We can easily extend the "Brute-Force" method to a "Weighted-BruteForce" method for video classification, by weighting the closest distance computed for each region by the inverse of its overlap rate. Besides, a ROC curve can be obtained by introducing a multiplicative threshold θ BF ≥ 1, and by classifying the query as neoplastic if and only if D N < θ BF D B . The ROC curve of "Weighted-BruteForce" method shows statistically worse results in comparison to the other methods, with p-values less than 0.05 with the McNemar's test. Besides, the best classification accuracies of "Weighted-BruteForce" are reached for θ BF = 1.017 > 1, which is confirming that local neoplastic features are also more discriminative than the benign ones. In fact, putting more weight on neoplastic patterns leads to increase the classification sensitivity, which is clinically important since it reduces the rate of false negatives.

CONCLUSION

Using the registration results of a mosaicing technique on endomicroscopic video data allowed us to provide the physicians with relevant annotated videos, similar to the video of interest to potentially support diagnostic decision. When compared to learning and retrieving images independently, our video retrieval method called "Bag of Overlap-Weighted Visual Words" improves the results of video classification in a statistically significant manner. Besides, it is based on histogram summations that considerably reduce both retrieval runtime and training memory. For future work, improvements could be made by taking into account the spatial relationship between local features in order to extract the spatial organization of cells. A more complete approach would describe the local 2D + t volumes contained in the videos to work on more accurate visual words and better combine spatial and temporal information.

Fig. 1 :

 1 Fig. 1: From left to right: Neoplastic pCLE mosaic obtained with non-rigid registration; Colored visual words mapped to the disk regions of the mosaic image; Overlap rates of the local regions in the mosaic space, according the translation results of mosaicing.

Fig. 2 :Fig. 3 :

 23 Fig. 2: Two rows of similar pCLE video sub-sequences, represented as mosaics and retrieved with the LOPO Weighted-ImOfMos method. B. indicates Benign and N. Neoplastic. From left to right on each row: the queried video sub-sequence, and its first, second, third and fourth most similar video sub-sequences, along with their similarity distance D.