Arseniy Gorin
email: arseniy.gorin@inria.fr

Denis Jouvet
email: denis.jouvet@inria.fr

Emmanuel Vincent

Dung Tran

Investigating Stranded GMM for Improving Automatic Speech Recognition

Keywords: dynamic Bayesian network, hidden Markov model, trajectory modeling, robust speech recognition

This paper investigates recently proposed Stranded Gaussian Mixture acoustic Model (SGMM) for Automatic Speech Recognition (ASR). This model extends conventional hidden Markov model (HMM-GMM) by explicitly introducing dependencies between components of the observation Gaussian mixture densities.

The main objective of the paper is to experimentally study, how useful SGMM can be for dealing with data, which contains different sources of acoustic variability. First studied sources of variability are age and gender in quiet environment (TIdigits task including child speech). Second, the SGMM modeling is applied on data produced by different speakers and corrupted by non-stationary noise (CHiME 2013 challenge data). Finally, SGMM is applied on the same noisy data, but after performing speech enhancement (i.e., the remaining variability mostly comes from residual noise and different speakers).

Although SGMM was originally proposed for robust speech recognition of noisy data, in this work it was found, that the model is more efficient for handling speaker variability in quiet environment.

Introduction

Hidden Markov models (HMMs) play an important role in statistical acoustic modeling for ASR. However, they are frequently criticized for inability to capture long temporal dependencies and variability of the speech signal because of strong conditional independence assumptions.

In the straightforward formulation it is assumed, that the state of an HMM is conditioned only on the previous state and that the observation is dependent only on the state which generates it. To better capture the contextual variability, contextdependent triphones are used. To capture speaker variability, the observation densities are built in the form of mixture of Gaussians. One problem, associated with conventional HMM is known as trajectory folding. This problem arises because of the fact, that the component of the GMM, trained from one source of variability, can dominate in likelihood computation for the observation from another source of variability [START_REF] Illina | Elimination of trajectory folding phenomenon: HMM, trajectory mixture HMM and mixture stochastic trajectory model[END_REF].

Various techniques were introduced to relax the conditional independence assumptions and to achieve more accurate modeling of temporal dependencies. The classification of these techniques is done depending on where such dependencies are applied (in the feature, or in the model space) and on the type of such dependencies (linear or non-linear, with or without recursion) [START_REF] Deng | Dynamic Speech Models: Theory, Algorithms, and Applications[END_REF].

Most popular early approaches include Stochastic Segmental Models [START_REF] Ostendorf | A stochastic segment model for phoneme-based continuous speech recognition[END_REF] and Stochastic Trajectory Models [START_REF] Gong | Stochastic trajectory modeling for speech recognition[END_REF][START_REF] Goldenthal | Statistical trajectory models for phonetic recognition[END_REF]. Various approaches deal with additional dependencies in model space. These include Buried HMMs [START_REF] Bilmes | Buried Markov models for speech recognition[END_REF] with conditioned Gaussian observation densities and multi-path HMMs [START_REF] Korkmazskiy | Generalized mixture of HMMs for continuous speech recognition[END_REF], where each separate path represents a separate source of variability. Recently, separately from HMM-GMM framework Deep Neural Networks-based systems have been successfully applied to better model both long temporal context and speech signal variability [START_REF] Hinton | Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups[END_REF].

This work focuses on recently proposed Stranded Gaussian Mixture Model (SGMM) [START_REF] Zhao | Stranded Gaussian mixture hidden Markov models for robust speech recognition[END_REF]. SGMM is an extension of HMM, which adds dependencies between components of HMM-GMM by expanding the observation density and replacing state-conditioned mixture weights by Mixture Transition Matrices (MTMs). MTM models transition probabilities between Gaussian components of the adjacent states.

In the original paper [START_REF] Zhao | Stranded Gaussian mixture hidden Markov models for robust speech recognition[END_REF] SGMM was investigated on Aurora 2 connected digits task [START_REF] Hirsch | The aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions[END_REF]. The authors used multi-condition training, i.e., including noised utterances in the training set. With standard MFCC front-end (12 cepstra + log energy, plus first and second derivatives) and Cepstral Mean Normalization (CMN) they demonstrated the improvement of average WER from 8.96% to 8.07% if GMM is replaced by SGMM with the same number of components. Such improvements encourage to further investigate the model for other sources of variability and on more difficult tasks.

Speaker age and gender are important sources of variability for ASR. Child speech is typically hard to recognize, because the acoustic variability comes from both vocal tract differences and specific articulation of children [START_REF] Benzeghiba | Automatic speech recognition and speech variability: A review[END_REF][START_REF] Stern | Hearing is believing: Biologically inspired methods for robust automatic speech recognition[END_REF]. If train and test sets are produced by both child and adult speakers, using all data to learn speaker-independent HMM-GMM leads to low performance. The variability can be partially reduced by using model and feature adaptation techniques, like MLLR, fM-LLR [START_REF] Gales | Maximum likelihood linear transformations for HMM-based speech recognition[END_REF] or VTLN [START_REF] Zhan | Vocal tract length normalization for large vocabulary continuous speech recognition[END_REF]. However, adaptation requires prior knowledge about speaker classes (gender or age) and/or needs an additional pass in decoding for estimating such information.

Other sources of variability are the recording conditions (microphone, room reverberation) and noise. From this point of view, CHiME 2013 challenge [START_REF] Vincent | The second'CHiME'Speech Separation and Recognition Challenge: Datasets, tasks and baselines[END_REF] provides a hard task. Nonstationary noise in CHiME is added in random places of the utterances. In addition, the microphone movements are modeled. If only clean data is used for training, the accuracy of the model is very low. Better results are achieved when noised data is also used in training (multi-condition training). Speech en-hancement [START_REF] Tran | Extension of uncertainty propagation to dynamic MFCCs for noise robust ASR[END_REF] of both train and test set significantly improves the recognition accuracy, although the residual noise still makes the recognition task far from being considered as solved.

The experiments in this work are conducted using Sphinx3 toolkit [17], which was modified to handle SGMM. Speech enhancement is done using Flexible Audio Source Separation Toolbox (FASST) [START_REF] Ozerov | A General Flexible Framework for the Handling of Prior Information in Audio Source Separation[END_REF][START_REF] Ozerov | Using the FASST source separation toolbox for noise robust speech recognition[END_REF].

The paper is organized as follows. Section 2 recaps the SGMM formulation. Section 3 describes the experiments with TIdigits task. Section 4 explains CHiME data and the corresponding experiments. The paper ends with a discussion and a conclusion.

Stranded GMM

The conventional SGMM consists of the state sequence Q = {q1, ..., qT }, the observation sequence O = {o1, ..., oT }, and the sequence of components of the observation density M = {m1, ..., mT }, where every mt ∈ {1, ..., M } is the component of the observation density at the time t, and M denotes the number of such components in the mixture. The difference of SGMM from HMM-GMM is that an additional dependency between the components of the mixture in the current frame mt and at the previous frame mt-1 is introduced (Figure 1). The joint likelihood of the observation, state and component sequences is defined by:

p(O, Q, M|λ) = p(O|M, Q, λ)p(M|Q, λ)p(Q|λ) = T t=1 p(ot|mt, qt)p(mt|mt-1, qt, qt-1)p(qt|qt-1) (1)
The complete set of parameters is the following:

• P (qt = j|qt-1 = i) = aij -state transition probability;

• P (ot|mt = l, qt = j) = b jl (ot) -probability of the observation ot with respect to the single density component mt = l in the state qt = j;

• P (mt = l|mt-1 = k, qt = j, qt-1 = i) = c
(ij) = {c (ij) kl }, where M l=1 c (ij) kl = 1, ∀i, j, k.
For training and decoding the Baum-Welch and Viterbi algorithms are expanded to include MTMs. The derivations can be found in [START_REF] Zhao | Stranded Gaussian mixture hidden Markov models for robust speech recognition[END_REF].

SGMM for age and gender variability

To compare the SGMM behavior with different sources of variability, this section compares conventional HMM-GMM and Stranded HMM when dealing with clean data, which contains speakers of different age and gender.

TIdigits Baseline system and problem formulation

The experiments in this section are conducted on the TIdigits connected digits task [START_REF] Leonard | Tidigits speech corpus[END_REF]. The full training data set consists of 41224 digits (28329 for adult and 12895 for child speech). The test set consists of 41087 digits (28554 for adult and 12533 for child).

The digits are modeled as sequences of word-dependent phones. Each phone is modeled by a 3-state HMM without skips. Each state density is modeled by 32 Gaussian components. The front-end consists of standard 39 MFCC features (12 cepstra + log-energy, plus first and second order derivatives) with CMN.

Similar to other work with TIdigits [START_REF] Burnett | Rapid unsupervised adaptation to children's speech on a connected-digit task[END_REF], the signal is down sampled to 8 kHz. Word Error Rates (WERs) of the baseline systems are reported in Table 1. Two Speaker-Independent (SI) models are trained from the adult subset only and from the full training set. For the last two lines, the Age and Gender-Age dependent models are achieved with MLLR+MAP adaptation. Training on adult data provides the best results for adult speakers, but shows a weak performance on child speech. When child data is included in the training set, the conventional HMM-GMM improves on the child, but degrades on the adult subset. Using class-adapted models further improves the baseline performance. In further experiments only full training set will be considered with no class information (i.e., age and gender) available.

Experimental results on TIdigits data

To train SGMM, MTM rows are initialized from the mixture weights of convention HMM-GMM, and the model parameters are re-estimated with MLE. In addition, to reduce the number of parameters, only 2 MTMs are used for each state (i.e., crossphone MTMs are shared). Two SGMMs were built: with only MTMs and with all parameters re-estimated. The corresponding WERs are shown in Compared to the conventional HMM-GMM, SGMM improves from 1.66% to 1.11% on adult and from 1.88% to 1.27% on child speech. Both improvements are statistically significant with respect to 95% confidence interval. The SGMM performance is even better than the one achieved with the Gender-Age adapted baseline.

Notice, that the largest improvement is associated with only MTM re-estimation, whereas re-estimating means and variances only slightly improves on the child subset.

SGMM for robust speech recognition

This section analyzes the behavior of SGMM for noise-robust speech recognition on the 1 st track (small-vocabulary) of CHiME 2013 challenge.

CHiME task and baseline system

The task is to recognize digit and letter tokens in 6-word utterances. Overall there are 10 possible keywords for the digits and 25 keywords for the letters. The training set consists of 17000 utterances, which come from 34 different speakers (500 utterances per speaker). The development set is used for performance evaluation. The set contains 3600 utterances (600 for each SNR level). The utterances are corrupted by various types of non-stationary background noise with SNR from -6 to 9 dB. In the baseline system each phone is modeled by an HMM with 3 states. However, the phones are not shared across different words (hence, word-dependent phones). Overall the model has 128 context-independent 3-state HMMs without skips. Each state is modeled by 32 Gaussian mixtures.

The performance is evaluated with 2 different types of acoustic features. In the first set of experiments standard 39 MFCC features (12 cepstra + log-energy, plus first and second order derivatives) with CMN are directly derived from noisy speech. In the second case, the same features are extracted after speech enhancement with uncertainty information, included in the feature computation (the approach is described in [START_REF] Tran | Extension of uncertainty propagation to dynamic MFCCs for noise robust ASR[END_REF]). Speech enhancement allows to significantly reduce error rates compared to standard multi-condition training from noisy data.

In all experiments with SGMM, two MTMs for each model state are used, as in TIdigits experiments: one MTM for interand another for intra-state transition. The inter-state transitions between phone models are shared across different contexts. So, the total number of MTMs is equivalent to 2 times the number of states.

Mixture Transition Matrix analysis

It was observed, that inter-and intra-state MTMs are distributed differently. For example, the averaged values of MTMs for inter-and intra-state transitions computed with noisy data on CHiME dataset are presented in Figure 2. The averaged MTMs for TIdigits set has a similar appearance.

Notice, that the intra-state MTM tends to be close to diagonal (in average). This means, that when staying in a given state of an HMM, the same component of the density is likely to be The average value of the diagonal elements for intra-state MTM is 0.67 ± 0.02 for noisy CHiME and 0.58 ± 0.03 for TIdigits data. As it was noticed in [START_REF] Zhao | Stranded Gaussian mixture hidden Markov models for robust speech recognition[END_REF], the distribution is also sparse.

Both sparsity and such sharp distribution for intra-sate transitions make training and decoding problems challenging. In fact, it is further shown, that sharp intra-state transitions constrain the trajectory and can lead to accuracy degradation, when the data contains non-stationary noise.

Experimental results on CHiME data

The results of the first set of experiments with MFCC features, derived from noised CHiME data, are summarized in table 3. The SGMM is initialized from the baseline SI system and reestimated. For this set of experiments it was observed, that sharp diagonal distribution of intra-state MTMs (loops) significantly hurts the performance of the recognizer (this corresponds to "Loop MTM -trained" row in table 3). A simple "work around" approach is to keep MTMs for state loops uniform (not re-estimate them). When intra-state MTM is forced to be uniform, SGMM outperforms GMM with larger gains in the noisy part (corresponds to the rows "Loop MTM -fixed"). Further improvement is achieved for this dataset when means and variances are jointly re-estimated with MTM.

Finally, the same experiment is reproduced with enhanced features, calculated with FASST [START_REF] Ozerov | Using the FASST source separation toolbox for noise robust speech recognition[END_REF]. The training is done in the same way, as for the SGMM trained from noisy data. The performances of the baseline GMM, of the SGMM with and without intra-state MTMs re-estimation and of the SGMM with means and variances re-estimated are summarized in Table 4 [START_REF] Gillick | Some statistical issues in the comparison of speech recognition algorithms[END_REF]. The test consists in analyzing the errors, produced by two systems and computing the probability P of how likely the improvement was done by chance. Comparing GMM and SGMM in the experiments with noisy features (Table 3) gives P = 0.017 and with enhanced features (Table 4) P = 0.040, which means that the results are statistically significant (with respect to 95% confidence interval).

Discussion

Tables 2, 3 and 4 summarize the improvements, which are achieved by SGMM compared to HMM-GMM.

Decoding clean read speech (as in TIdigits) imposes troubles for conventional HMM if the data comes from the speakers of different age and gender. SGMM shows significant improvements in such conditions. Re-estimating only MTM (row "SGMM: MTM" in Table 2) improves the WER by about 30% relative for both child and adult data. This leads to the conclusion, that temporal dependencies (explicit trajectories) introduced by SGMM are useful for modeling speaker variability.

Different observations are drawn from the experiments with noisy data. Training with noisy (Table 3), or enhanced (Table 4) data demonstrates similar behavior.

First, for CHiME data intra-state transitions hurt the model accuracy (rows "Loop MTM-trained" in tables 3 and 4). Interestingly, that at the same time the likelihood of the training data is much higher, when both inter-and intra-state MTMs are reestimated. This leads to the idea, that SGMMs tend to over-fit, which leads to the degradation if the acoustic mismatch between training and testing data is significant.

Second, after fixing (not re-estimating) intra-state MTMs the improvement is not as large, as for TIdigits, but still statistically significant. Overall, about 2% relative improvement is achieved for the full CHiME data set. Up to 5% is also achieved for -6 and -3 dB subsets with noisy MFCC.

Conclusion

This work has described an experimental study of Stranded GMM, which is an extension of HMM-GMM model with additional temporal dependencies between components of the observation densities.

Three types of signal variability were studied. First, gender and age in clean speech were investigated. Then, the same experiments were carried on speech corrupted by non-stationary noise and on enhanced speech with residual noise.

Although SGMM was originally proposed for robust speech recognition of noisy data, in this paper it was demonstrated, that it provides the largest improvement for clean speech with adult and children data. When the signal is corrupted by non-stationary noise, SGMM improves the accuracy not as greatly and only if intra-state MTMs are not re-estimated.

Although they require increased computation power, such extended models, as SGMM, are certainly interesting for the future research. One research direction is the introduction of speaker, or noise information in the SGMM by defining some SGMM parameters from pre-clustered data, or by involving speaker adaptation. Finally, the issue with dominating diagonal probabilities of intra-state MTMs in CHiME data opens new questions, which should lead to improved re-estimation algorithm for MTM involving sparsity constraints, or smoothing.

Figure 1 :

 1 Figure 1: Stranded GMM for triphone HMM model

  probability of moving from the component mt-1 = k of the state qt-1 = i to the component mt = l of the state qt = j (mixture transition probability) The set of component transition probabilities forms the Mixture Transition Matrix (MTM) C

Figure 2 :

 2 Inter-state(a) and intra-state(b) mixture transition matrices, averaged over states for CHiME data dominating for the adjacent frames.

Table 1 :

 1 Baseline word error rates for SI, Age and Gender-Age adapted models with known speaker classes.

		Adult Man Wom Child Boy Girl
	Training on	0.64 0.79 0.49 9.92 6.51 13.33
	adult data
	Training on	1.66 1.86 1.46 1.88 1.69 2.08
	adult+child data
	+Age adaptation	1.42 1.56 1.28 1.56 1.52 1.54
	(classes known in decoding)
	+Gender-Age adaptation	1.31 1.57 1.04 1.31 1.14 1.49
	(classes known in decoding)

Table 2 .

 2 Model Adult Man Woman Child Boy Girl GMM 1.66 1.86 1.46 1.88 1.69 2.08 SGMM: MTM 1.09 1.22 0.96 1.35 1.30 1.41 SGMM: MTM+µ+σ 1.11 1.26 0.96 1.27 1.19 1.36

Table 2 :

 2 Word error rates of stranded GMM vs conventional GMM on TIdigits task

Table 3 :

 3 Keyword recognition accuracy (%) for Dev set of CHiME 2013. Features from noisy data

	Model	-6dB -3dB 0dB 3dB 6dB 9dB AVG
	GMM baseline	55.75 60.08 69.58 77.67 80.08 84.25 71.24
	SGMM	Loop
	training	MTM
	MTM	trained 53.75 58.92 67.50 75.26 79.75 84.17 69.89
	MTM	fixed 57.08 61.08 69.17 77.25 80.00 85.17 71.63
	MTM+µ+σ fixed 57.83 62.08 69.58 77.33 80.17 85.17 72.03

Table 4 :

 4 . Keyword recognition accuracy (%) for Dev set of CHiME 2013. Features from enhanced data Overall, the relative improvements, achieved with SGMM on noisy and enhanced data, are similar. To verify the statistical significance of the results, McNemar test was done

	Model	-6dB -3dB 0dB 3dB 6dB 9dB AVG
	GMM baseline	73.00 78.00 82.92 85.83 89.17 90.33 83.21
	SGMM	Loop
	training	MTM
	MTM	trained 72.67 76.83 81.00 86.33 88.33 90.33 82.58
	MTM	fixed 73.17 78.33 82.58 86.50 89.25 90.67 83.42
	MTM+µ+σ fixed 73.50 79.00 82.83 86.58 89.67 90.92 83.75


Nazim Haouchine

Jeremie Dequidt

Igor Peterlik

Erwan Kerrien

Marie-Odile Berger

Stephane Cotin

Towards an Accurate Tracking of Liver Tumors for Augmented Reality in Robotic Assisted Surgery

This article introduces a method for tracking the internal structures of the liver during robot-assisted procedures. Vascular network, tumors and cut planes, computed from preoperative data, can be overlaid onto the laparoscopic view for image-guidance, even in the case of large motion or deformation of the organ. Compared to current methods, our method is able to precisely propagate surface motion to the internal structures. This is made possible by relying on a fast yet accurate biomechanical model of the liver combined with a robust visual tracking approach designed to properly constrain the model. Augmentation results are demonstrated on in-vivo sequences of a human liver during robotic surgery, while quantitative validation is performed on an ex-vivo porcine liver experimentation. Validation results show that our approach gives an accurate surface registration with an error of less than 6mm on the position of the tumor.

I. INTRODUCTION

Considerable recent advances in medicine have made Minimally Invasive Surgery (MIS) a well-established procedure for hepatic surgery, with strong benefits for patients. However, the reduced endoscopic field of view, the lack of depth and tactile perception make hepatic resection and tumor removal a major challenge that requires advanced surgical skills. Augmented Reality (AR) techniques have the potential to counteract these regressions and provide an enriched visual feedback by the seamless integration of intraoperative images and any targets or risk structures, such as tumors or vessels, visible in 3D pre-operative imaging (cf Figure 1).

Developing a full AR system for MIS implies solving a number of difficult problems, ranging from instrument tracking to intuitive augmented visualization [START_REF] Nicolau | Augmented reality in laparoscopic surgical oncology[END_REF]. Among the most difficult and still unsolved issues is the capacity to tackle, in real time, large elastic deformations on the targeted organ. Though the problem of elastic registration or tracking has been largely addressed in the medical community, elastic deformation in AR systems is only addressed, to our knowledge, in a very limited number of papers, and mainly applied to 3 organs: kidney, heart and liver.

Kidneys do not actually undergo elastic deformations during surgery but rather move independently from their 1 Nazim Haouchine, Jeremie Dequidt and Stephane Cotin are with INRIA and Lille University, France nazim.haouchine@inria.fr, jeremie.dequidt@inria.fr, stephane.cotin@inria.fr 2 Igor Peterlik is with Image-guided Minimally Invasive Surgery Institute, IHU Strasbourg, France peterlik@gmail.com 3 Erwan Kerrien and Marie-Odile Berger are with INRIA and Lorraine University, Nancy, France erwan.kerrien@inria.fr, marie-odile.berger@inria.fr surrounding tissues. As such, related works resort to a rigid 3D-3D registration e.g. based on implanted markers [START_REF] Teber | Augmented reality : a new tool to improve surgical accuracy during laparoscopic partial nephrectomy Preliminary in vitro and in vivo results[END_REF] or an Iterative Closest Point (ICP) surface matching [START_REF] Su | Augmented Reality During Robot-assisted Laparoscopic Partial Nephrectomy: Toward Real-Time 3D-CT to Stereoscopic Video Registration[END_REF]. Recently Puerto-Souza et al. [START_REF] Puerto-Souza | An augmented-reality system for laparoscopic surgery robust to complete occlusions and fast camera motions[END_REF] proposed a Hierarchical Multi-Affine matching algorithm capable of long-term tracking for augmentation during partial nephrectomy. This approach estimates a set of affine transformations from clusters of features in order to relocate occluded or missing features to obtain a coherent spatio-temporal mesh registration. Figl et al. [START_REF] Figl | Image guidance for robotic minimally invasive coronary artery bypass[END_REF] also express the non-rigid registration problem as a rigid registration between stereoscopic views and a 4D model of the heart built from 4D scan data, thus leveraging the cyclicity of the organ motion. A 4D scan of the heart is also used in [START_REF] Pratt | Dynamic Guidance for Robotic Surgery using Image-Constrained Biomechanical Models[END_REF], but is coupled with a biomechanical model to relate the surface motion to internal forces. A local tuning of the deformation is therefore possible and the surface deformation is correctly propagated to in-depth invisible structures. No periodic or rigid pattern can be relied upon in the liver case. Unpredictable, possibly large, deformations can be observed. Furthermore, volume tracking is required since we are interested in in-depth augmentation whereas many approaches are dedicated to the tracking of surfaces. In a similar context, Kim et al. [START_REF] Kim | Tracking by Detection for Interactive Image Augmentation in Laparoscopy[END_REF] propose a solution that is both applicable in a monoscopic setup and integrates a robust image feature reconstruction approach. But their method only retrieves surface deformation and cannot ensure an accurate augmentation of inner structures.

Mechanical-based deformable models have proven to be relevant for volume deformation since they allow to define elastic properties of the shape and thus to infer in-depth structure motion [START_REF] Speidel | Intraoperative surface reconstructionand biomechanical modeling for soft tissue registration[END_REF], [START_REF] Shen | Active Volume Models for Medical Image Segmentation[END_REF], [START_REF] Sermesant | Deformable biomechanical models: Application to 4D cardiac image analysis[END_REF]. Registration is done either by solving mechanical equations given image constraints provided by reconstructed 3D points [START_REF] Speidel | Intraoperative surface reconstructionand biomechanical modeling for soft tissue registration[END_REF] or using the concept of active models where an energy is minimized which takes into account the internal behavior of the model -through the biomechanical constraints-and external image constraints that measure the adequation between the model and the recovered 3D data [START_REF] Shen | Active Volume Models for Medical Image Segmentation[END_REF], [START_REF] Sermesant | Deformable biomechanical models: Application to 4D cardiac image analysis[END_REF].

The work presented here is an improvement of the former work on AR for deformable organs [START_REF] Haouchine | Image-guided Simulation of Heterogeneous Tissue Deformation for Augmented Reality during Hepatic Surgery[END_REF], [START_REF] Haouchine | Deformationbased Augmented Reality for Hepatic Surgery[END_REF]. In these works, a method employing a biomechanical model guided by a stereo-based tracking algorithm which is able to interpolate in the volume surface deformations was proposed. The physical model used, provably adapted to the liver [START_REF] Peterlik | Modeling and Real-Time Simulation of a Vascularized Liver Tissue[END_REF], is superimposed in real-time onto the laparoscopic images within tumors. In this paper we mainly focus on the reconstruction and the selection of 3D points which are used to guide the physical model. In particular, techniques based on moving least-squares are used to improve the set of recovered 3D points. We also restrict the set of points which are used to guide the mechanical model using clustering techniques with significant impact on the accuracy and the robustness of the method. With respect to existing works, we improve upon [START_REF] Speidel | Intraoperative surface reconstructionand biomechanical modeling for soft tissue registration[END_REF] by proposing a method that does not demand a computationally expensive stereo surface recovery and upon [START_REF] Puerto-Souza | An augmented-reality system for laparoscopic surgery robust to complete occlusions and fast camera motions[END_REF] by performing a robust and long-term tracking which is able to handle elastic deformations of the organ. The validation of our approach is presented on both in vivo and ex vivo data where we report visually accurate registration on images of an actual laparoscopic procedure and show that the error between the estimated and actual positions of an artificial tumor implanted in a porcine liver is less than 6mm.

II. STEREOSCOPIC FLOW ESTIMATION

The use of a biomechanical model to propagate the surface motion to the internal structures not visible on the endoscopic images, , such as tumors, requires a stable tracking sufficiently robust with respect to the image noise. For that purpose, we propose a new tracking method designed to guide the physically-based model. Our tracking method follows these steps:

• Sparse three-dimensional recovery from stereoscopic images.

• Moving Least-Squares-based outliers filtering and surface approximation. • Control point selection with clustering and temporal flow estimation.

A. Sparse three-dimensional recovery

Intra-abdominal surface reconstruction has been under investigation in several research groups (for a complete survey, please refer to [START_REF] Maier-Hein | Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery[END_REF]). The methods range from passive techniques [START_REF] Stoyanov | Stereoscopic scene flow for robotic assisted minimally invasive surgery[END_REF], [START_REF] Richa | Robust 3d visual tracking for robotic-assisted cardiac interventions[END_REF] based only on endoscopic image informations, to active techniques [START_REF] Maurice | Real-time structured light coding for adaptive patterns[END_REF], [START_REF] Herrera | Shape-from-Polarization in laparoscopy[END_REF] that involve hardware modifications. Our method relies on a passive approach using a stereo endoscope, calibrated according to Zhang's [START_REF] Zhang | A flexible new technique for camera calibration[END_REF] approach. Features are detected in each image MLS-based surface recovery. Left: Three-dimensional sparse point cloud recovery. Right: outliers filtering and surface reconstruction using MLS, the control points selected after clustering are registered to the biomechanical model of the liver pair (rectified for distortion) using the Speeded-Up Robust Features (SURF) descriptor and are tracked over time thanks to the Lucas-Kanade optical flow method, this coupling has proven to be optimal for laparoscopic surgery [START_REF] Elhawary | Robust feature tracking on the beating heart for a robotic-guided endoscope[END_REF]. Feature correspondences between stereo images are obtained with a nearest neighbour criterion on feature descriptors coupled with the epipolar constraint. A sparse set of m 3D points, denoted by 3 × m coordinate vector z, is reconstructed by triangulation.

Examples of 2D matched features and reconstructed 3D points from laparoscopic images are shown in Figure 3.

B. Moving Least-Squares-based surface reconstruction

The sparse set z of recovered 3D points may contain erroneous points. In order to remove these points and to obtain a smoother point set z ′ we take benefits of the MLS surface to approximate a surface and to detect the noisy points.

Moving Least-Squares surface (MLS) is a mesh-free, nonparametric method to approximate a surface from a set of unstructured scattered point introduced by [START_REF] Levin | Mesh-independent surface interpolation[END_REF].

To determine the MLS surface given a point set P = {p i ∈ R 3 }, a projection procedure which projects any points near the point set onto the surface using a local approximation is conducted. Given a point r ∈ R 3 , the process follows two steps. First the computation of a local patch, then the projection of the query point r onto the surface using a polynomial function :

• Compute a local reference plane in the neighborhood of r : Find a plane H with normal vector a ∈ R 3 passing through some point q = r + ta (for some t ∈ R) such that a = 1 and H minimizes the least-squares error:

∑ p i ∈P < a, p i -q > 2 θ ( p i -q ) (1)
Here, < a, p iq > 2 is the squared distance from each point {p i } to the plane H, and θ ( p iq ) is the weighting function, usually a Gaussian.

• Project r on the surface : We consider the points x i to be the orthogonal projections of the points p i on the plane H with projected distance f i =< a, p iq > oriented with normal a and origin q. Thus, we apply the MLS function approximation which consists of construction of a bivariate polynomial g ∈ R 3 of degree m (here we take m = 3) which locally minimizes the least-squares error around q:

∑ p i (g(x i ) -f i ) 2 θ (||p i -q||) (2)
That amounts to project r along the normal with distance g(0) (local approximation of the surface at q).

C. Clustering-based Filtering

The clustering-based filter (CbF) permits to create a set of n 3D control points, denoted by 3 × n coordinate vector y that guides the biomechanical model. Instead of constraining the model directly using the forces generated from feature displacements, the clustering calculates a displacement field more suitable to handle large deformations (see section IV for more details).

First a set of control point is built by downsampling a set z ′ of 3D points resulting from MLS step. Each control point defines a region of interest to group the surrounded features. The nearest features to the control point are assigned to the cluster by combining the Hessian responses of the SURF detector and the Shepard's Inverse Distance Weighting (IDW) following the formula D(p) = q p ∑ k i=1 W i d i where D is the weighted displacement of the control point p, k the total number of neighbors and d i the displacement of the feature i. Assuming the quality is isotropic at each feature, we define a normalized measure of quality q p for each control point as the average of the SURF Hessian responses of its neighbors. W i is the weight assigned to each feature and is as follows:

W i = ( R c -h i R c h i ) 2 ∑ k j=1 ( R c -h j R c h j ) 2 (3)
R c is the radius of the cluster in 3D space, h i the distance of the feature from the control point and k the total number of neighbors.

The Shepard's IDW allows the nearby feature to the control point to have the largest weight while the Hessian response allows to prevent from poor measurement during the tracking.

III. BIOMECHANICAL MODEL AND NON-RIGID REGISTRATION

A. Biomechanical model

We employ a biomechanical model of liver composed of parenchyma and vascular network which is an important source of heterogeneity and anisotropy. From a mechanical standpoint, the parenchyma model relies on a linear tetrahedral finite element modeling using a co-rotational approach [START_REF] Müller | Stable real-time deformations[END_REF]. This model is very well suited to handle large displacements and has been proven to be relevant in the context of real-time laparoscopic surgery [START_REF] Peterlik | Modeling and Real-Time Simulation of a Vascularized Liver Tissue[END_REF]. The vascular system is modeled with serially linked beam elements previously used in modeling catheters and guide wires model proposed by Duriez et al. [START_REF] Duriez | New approaches to catheter navigation for interventional radiology simulation[END_REF]. Both models share very similar properties which permit to build the composite model. A more detailed description of both models and the coupling can be found in [START_REF] Haouchine | Image-guided Simulation of Heterogeneous Tissue Deformation for Augmented Reality during Hepatic Surgery[END_REF]. Elastic forces are then defined as in [START_REF] Müller | Stable real-time deformations[END_REF]:

f i (x) = R e K(R ⊤ e x -x 0 ) (4)
where K represents the stiffness matrix, R e the corotational matrix and x, x 0 are vectors of size 3n representing the position of the n degrees of freedom of the mechanical model, respectively at any time t and time t = 0. It is worth mentioning that our framework was built using flexible and modular software design principles allowing to easily replace the physics-based model.

B. Initial registration

The initial registration is an important step for AR systems. In this work this step is done manually through a 3D interface as in similar systems [START_REF] Puerto-Souza | An augmented-reality system for laparoscopic surgery robust to complete occlusions and fast camera motions[END_REF], [START_REF] Speidel | Intraoperative surface reconstructionand biomechanical modeling for soft tissue registration[END_REF], [START_REF] Figl | Image guidance for robotic minimally invasive coronary artery bypass[END_REF], [START_REF] Pratt | Dynamic Guidance for Robotic Surgery using Image-Constrained Biomechanical Models[END_REF], [START_REF] Haouchine | Deformationbased Augmented Reality for Hepatic Surgery[END_REF]. First, the biomechanical model is aligned on the first pair of laparoscopic stereo images. Then, only the reconstructed points that reproject onto the model in the images are kept to filter in points belonging to the liver.

C. Temporal non-rigid registration

We propose to handle the non-rigid registration by adding external stretching forces introduced via tracking of 3D control points. The tracked 3D control point set y represents how visible parts of the liver are moving.

External forces are defined by pairing the m 3D control points y to the n degrees of freedom x of the bio-mechanical model. At initialization (t = 0), the tracked points y 0 are expressed in barycentric coordinates of the adjacent degrees of freedom leading to the linear relation: y 0 = Jx 0 , where J is a rectangular (3m, 3n) matrix. We assume this linear relation remains valid during the deformation.

At any later time t, the stretching forces induced by the tracking are defined as f t (y) = k(yy 0 ) where k can be seen as the stiffness.

Finally, the stretching forces can be expressed with respect of the degrees of freedom as

f t (x) = J ⊤ f t (y) = f t (x) = J ⊤ k • J(x -x 0 ) (5)
the parameter k is set to be close to the Young's modulus in order to provide an equilibrium between the stretching forces and the elastic forces (a higher value of k favors stretching forces).

IV. EXPERIMENTATIONS AND RESULTS

In this section, we present the experimentations conducted to validate our approach. The validation in context of MIS is usually a challenging task. In our case, we propose two validation protocols. The first one is performed in the context of in-vivo human liver surgery, the other involves experiments on ex-vivo porcine liver. In latter case, the results are evaluated within a known ground truth. We also compare our method CbF (tracking with Clusters-based Filtering) with the related work DT (a direct tracking used in [START_REF] Haouchine | Image-guided Simulation of Heterogeneous Tissue Deformation for Augmented Reality during Hepatic Surgery[END_REF]).

A. Results on in vivo Human Liver

The purpose of conducting an in vivo experimentation is to employ our technique in a real-life situation during an abdominal hepatic surgery. Several important disturbing factors are present: the illumination produced by the laparoscopic light, motion of instruments during the intervention and motion of the organ induced by respiration and cardiac cycle. For this study we used a heterogeneous biomechanical model, that takes into account the hepatic and portal vein to describe the liver soft-tissue behaviour, with the following elastic parameters (according to [START_REF] Peterlik | Modeling and Real-Time Simulation of a Vascularized Liver Tissue[END_REF]): Young's modulus of parenchyma E p = 7 kPa and Poisson's ratio ν = 0.45, Young's modulus of vessel wall: E v = 0.62 MPa and Poisson's ratio ν = 0.45, thickness of the vessel wall: t V = 200 µm. The parenchyma model was composed of 3391 linear tetrahedral elements and the vascular model of 120 beam elements. The augmentation is achieved in real-time at around 25 fps. It's worth mentioning that the real-time achievement is directly related to the number of elements (a larger number of elements will increase the computation time).

Following [START_REF] Elhawary | Robust feature tracking on the beating heart for a robotic-guided endoscope[END_REF], the parameters of the optical flow consist of 51 × 51 window and an interframe motion threshold of 20 pixels. The camera used is the Da Vinci Robot Endoscopic Camera (Intuitive Surgical) with a baseline of about 6 mm, acquiring image of 960 × 540 at 29.97 fps. On the set of 264 extracted features 26 control points are built when using a radius R c = 0.5mm. The Figure 4 illustrates four selected frames of the augmentation performed on a sequence of 812 frames showing the long-term robustness of the tracking.

We can notice in the comparison illustrated in Figure 5 that the DT approach may fail to accurately register the model onto the image, mainly because of noisy 3D features that locally constraints the model. The table I summarizes the projection errors of the mesh on the endoscopic images (manually extracted contour) and shows that the CbF tracking permits to reduce considerably the error w.r.t the DT approach.

B. Ex vivo quantitative validation on a porcine liver

The purpose of our tracking method is to estimate the location of tumors that are not visible in the endoscopic images. To validate such an approach we propose to perform CT scans of the liver before and after a deformation while acquiring the motion with a stereoscopic camera illustrated in Figure 7. Using a wire attached to one lobe of a porcine liver, we move the tissue in order to induce a deformation which is captured with an endoscopic camera ensuring a CTscanning without producing artefact.

We introduce two artificial tumors in two different locations (cf Figure 6), near from the tension zone (tumor (a) with an average size of 8 mm) and far from the tension zone (tumor (b) with an average size of 5 mm).

In Figure 8 our CbF tracking is compared with the DT approach using the data acquired during the manipulation of ex-vivo liver. We measure the following quantities: The tumor displacement due to the deformation for the Tumor (a) is 88.6 mm and Tumor (b) is 26.9 mm. The measured distance between the centers of mass of the ground truth It represents a significant improvement over the DT method and over current laparoscopic procedures where a safety margin of 10 mm to 25 mm is usually considered around the location of the tumor (planned on pre-operative data) [START_REF] Mala | A comparative study of the short-term outcome following open and laparoscopic liver resection of colorectal metastases[END_REF]. V. CONCLUSION One of the current deadlocks to break before AR systems can find their place in the clinical routine of MIS procedures is the ability to capture organs deformations during surgery. Large and unpredictable liver deformation that occur during surgery highly impacts the location of internal structures such as tumors and vessels. We have presented in this paper, an accurate tracking algorithm designed to constrain properly a biomechanical model that is able to capture such deformations. The model deformation is guided by both internal forces, generated by a biomechanically grounded model of the liver, and external forces, that emanate from 3D control points. These 3D control points are built form stereo images through a clusters-based filtering that restrict the noisy image measurement. Such characteristics make our approach able to accurately register the organ surface and inner the deformation motion to the internal structures. Promising results were obtained through in-vivo experimentation on a human liver and ex-vivo validation on a porcine liver where errors well below the current surgical margins and current methods were measured. Future work will address the issue of initialisation that is currently done manually, and the robustness of our approach against actual surgical events such as smoke, bleeding, or sudden instrument occlusions that may hinder the tracking. More validation is obviously required though it is worth mentioning that validation implying actual organs are seldom reported in previous works.

Fig. 1 .

 1 Fig. 1. Augmented Liver Tumor during Minimally Invasive Surgery. Top: The registration of the liver mesh including the tumor onto endoscopic images. Bottom: Computational flow of the method.

Fig. 2 .

 2 Fig. 2. MLS-based surface recovery. Left: Three-dimensional sparse point cloud recovery. Right: outliers filtering and surface reconstruction using MLS, the control points selected after clustering are registered to the biomechanical model of the liver

Fig. 3 .

 3 Fig. 3. Stereoscopic flow estimation on laparoscopic images of the abdominal cavity showing a part of the liver in two different cases with (top) high textured liver and (bottom) low textured liver: (a) Original input images, (b) the resulting sparse 3D point set z from stereo matching, (c) the resulting sparse 3D point set z ′ from MLS outliers filtering, (d) the projected image on the reconstructed surface.

Fig. 4 .

 4 Fig. 4. Selected frames of a superimposition of the biomechanical model onto the human liver during Minimally Invasive Surgery. Top: Heterogeneous model including the parenchyma (white), the portal vein (red), the hepatic vein (blue) and a tumor (sphere). Bottom: A different visualisation of the superimposition.

Fig. 5 .

 5 Fig. 5. Comparison between (right) our CbF tracking and (left) a the DT method. Top: A top view showing the registration error that may occur using a direct method. The biomechanical model (in white wireframe) is locally constrained by a 3D feature in the case of a direct tracking while it is constrained by set of features grouped around a control point. Middle and Bottom: A front view showing the visual accuracy of our method. The correct shape is well recovered even if the boundary (manually segmented and represented here in yellow) is not correct everywhere.

Fig. 6 .

 6 Fig. 6. Computed meshes form scanned porcine liver with the tumors (left) and the corresponding endoscopic images (right).

Fig. 7 .

 7 Fig. 7. Experimental setup for the quantitative validation: (a) insertion of an artificial tumor into one lobe of the liver, (b) a view of the the Da Vinci Robot Camera capturing liver motion, (c) the wire hooked to the lobe is pulled and maintained to perform a CT scan, (d) front view of the CT-scan and the liver.

Fig. 8 .

 8 Fig. 8. A comparison study between the CbF, the DT approach and a computer-generated force, on the tumor (a) and tumor (b). The clustering reduces considerably the error between the predicted tumor and the ground truth.

ACKNOWLEDGEMENT

The authors highly acknowledge the help provided by Mario Sanz Lopez during the experimentations and advisory provided by Dr. Patrick Pessaux.


Jean-Baptiste Barreau

Ronan Gaugne

Yann Bernard

Gaétan Le Cloirec

Inrap / Creaah

Valérie Gouranton

Virtual reality tools for the West Digital Conservatory of Archaeological Heritage

Keywords: I.3.7 [Three-Dimensional Graphics and Realism]: Virtual reality; J.2 [Physical Sciences and Engineering]: Archaeology Virtual Reality, Archaeology, Digital Heritage

In the continuation of the 3D data production work made by the WDCAH, the use of virtual reality tools allows archaeologists to carry out analysis and understanding research about their sites. In this paper, we focus on the virtual reality services proposed to archaeologists in the WDCAH, through the example of two archaeological sites, the Temple de Mars in Corseul and the Cairn of Carn Island.

INTRODUCTION

Virtual reality is often considered solely as a field of computer science in relation to interactive digital 3D worlds. It actually holds a special position in the usual scientific scheme by coupling humanities sciences with engineering. According to Fuchs et al. [START_REF] Fuchs | Virtual Reality: Concepts and Technologies[END_REF] « The purpose of virtual reality is to make possible a sensorimotor and cognitive activity for a person (or persons) in a digitally created artificial world, which can be imaginary, symbolic or a simulation of certain aspects of the real world. » This proposition positions the man and its activity in the centre of virtual reality.

In the context of the West Digital Conservatory of Archaeological Heritage [START_REF] Barreau | The West Digital Conservatory of Archaeological Heritage project[END_REF], we propose techniques issued from virtual reality domain to help archaeologists in their scientific analysis, either by providing them dedicated tools to work more efficiently in numeric worlds [START_REF] Gaugne | Immersia, an open immersive infrastructure: doing archaeology in virtual reality[END_REF] or by easing the understanding of past human activities such as building techniques or site exploitations. In this paper, we present two case studies illustrating different virtual reality techniques use in the WDCAH project. We focus on the tools proposed to archaeologists in these two studies, and present the purposes of these tools, nevertheless our goal is not here to present the conclusions made by archaeologists from their analysis.

VIRTUAL REALITY AND ARCHAEOLOGY

The ability of the RV to immerse a user in historical sites, which can be completely missing or partially ruined, makes it an indispensable tool for cultural heritage and archaeology, with different purposes according to these two domains. Generally, the use of 3D in the field of cultural heritage is limited to the often spectacular view of sites and monuments, with less concern for the scientific coherence. Since recent years, however, we observe the emergence of a trend to streamline the production of these 3D models, use, storage and distribution such as in Koller et al. [START_REF] Koller | Research challenges for digital archives of 3D cultural heritage models[END_REF] and London Charter (www.londoncharter.org).

On the archaeology side, more emphasis is set on scientific methodology. Virtual archaeology was introduced by Reilly in 1990 [START_REF] Reilly | Towards a virtual archaeology[END_REF], and was initially presented for excavation recording and virtual re-excavation using multimedia technologies. In a similar way, Krasniewicz [START_REF] Krasniewicz | Immersive imaging technologies for archeological research[END_REF] proposed a 360° visualization infrastructure to help archaeologists in their research work. In this case, virtual archaeology was not used to restore knowledge, but to acquire new knowledge. More recently, Pujol Tost et al. [START_REF] Tost | Vers une Réalité Virtuelle véritablement interactive[END_REF], contributed to the interaction, perception and simulation in VR for archaeology, and not only for the visualization of 3D models. Vergnieux [START_REF] Vergnieux | Archaeological research and 3D models (Restitution, validation and simulation)[END_REF], considered the simulation for the movement and gesture validation, the physical coherency and the technical feasibility of constructions. According to new trends in the domain, Forte [START_REF] Forte | Cyber-Archaeology: Notes on the simulation of the past[END_REF] suggests to replace the terminology related to a "reconstitution of the past", by the expression "Cyber-Archaeology" relying on a "simulation of the past".

Several works illustrate clearly the interest of virtual reality for archaeology. Christou et al. [START_REF] Christou | A versatile large-scale multimodal vr system for cultural heritage visualization[END_REF] used an immersive CAVElike structure combined with haptic devices and 3D sound for pedagogic purposes in a museum exhibition, but also as a tool for research. The Archave project [START_REF] Vote | Discovering petra: Archaeological analysis in vr[END_REF] integrated also a CAVE-like structure and proposed tools for archaeologists to study historical sites. Drap et al. in [START_REF] Drap | Towards a Photogrammetry and Virtual Reality Based Heritage Information System: A Case Study of Shawbak Castle Int[END_REF] describe the use of an immersive virtual reality interface for Shawback Castle. The immersive environment consists of a large stereoscopic projected wall, with infrared tracking. The work focuses on semantic annotations of blocks to ensure consistency with a knowledge database associated to the model.

The WDCAH Approach in RV usage

The West Digital Conservatory of Archaeological Heritage aims at being a convenience tool, easy to reach and free for archaeologists in the West of France. It provides a technical and technological accompaniment in 3D data production (see Figure 1), digital storage, and data and metadata access. Various access Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Laval Virtual VRIC '14, April 9-11, 2014, Laval, France.

Copyright 2014 978-1-4503-2626-1 ...$10.00 modes to data and metadata, associated to a set of tools based on virtual reality come in support to archaeologists in their scientific work. Scientific process coherency remains directly handled by archaeologists. Access to virtual reality environments is proposed as a complementary tool for archaeologists to help their research work, from simple desktop VR equipments to CAVE-like facilities, via head-mounted displays (see Figure 2).

A first case study : the Temple of Mars

Ancient Gallo-Roman capital of the Coriosolites, which were one of the five tribes of Brittany, Corseul is a Breton town located in the southwest of St Malo. Its Gallo-Roman sanctuary of the "Haut-Bécherel", [START_REF] Provost | Le monument romain du Haut-Bécherel -Sanctuaire public des Coriosolites[END_REF], otherwise called Temple of Mars was built from 9O AD in Tuscan style and is among the most notable examples of Roman civilization in mainland Britain. It allowed disseminating the Roman religion, to drain pilgrims from all over Brittany and to highlight the power and greatness of Rome.

As the site is almost completely destroyed, a digitalization by photogrammetry or laser scan was not possible. Based on plans provided by archaeologists, a graphic designer produced a 3D model of the site in the CAD tool AutoCAD 3D Studio MAX. Immersive visualization: the difference between a simple first-person visualization on a standard computer and an immersive visualization is highly significant. During the first experiments of immersive visit (Figure 4) of the virtual reconstitution of the site, many modeling defects, hard to detect in the CAD tool, were revealed. The 1:1 scale visualization allows to strongly feel present in the virtual universe. The user realistically perceives volumes and perspectives of the buildings. In the Temple of Mars, the architecture of the monument is centralized around the imposing pediment of the cella. The immersive visualization enables the perception of this effect sought by the architect.

Navigation: We wanted to provide archaeologists a simple and natural way of exploring the monument. We proposed two navigation modes, one based on a standard two-axes navigation used with joysticks, and one based on points of interests (POI). In the first mode, called "free navigation", camera moves in user's gaze direction, backward or forward following joystick or 3D mouse tilt. In the Flystick case, the right/left axis entails a camera spin. In the 3D mouse case, we have more navigation possibilities. The right/left axis drives a translational orthogonal to the user's gaze. Camera spin is associated to 3D mouse rotation. Others 3D mouse degrees of freedom are not used.

In the "POI navigation" mode, a list of points of interest is pre-defined in the application. When clicking on a button of the Flystick or 3D mouse, a path to the next POI is calculated from the current position by the Unity3D path finding functionality. The user automatically travels along the calculated path, at a walking speed. The calculated path is visible to the user and dynamically adapted to his position (Figure 4). When the POI is reached, the travel stops. During the move, the user can shift to the free navigation mode by activating the joystick or the 3D mouse and come back to POI navigation at any time.

A second case study: The Cairn of carn

The monument of Carn (Figure 5) is located in the western part of Brittany. It has been excavated during the 60´s by Pierre-Roland Giot and published in 1987 [START_REF] Giot | Travaux du Laboratoire "Anthropologie, Prehistoire, Protohistoire et Quaternaire armoricains[END_REF]. The dating performed for this monument was the first radiocarbon dating in Bretagne, and results show that the monument belongs to the Vth millennium BC. The cairn central chamber has been digitized by photogrammetry and integrated in an immersive platform in order to enable archaeologists to work in a 1:1 reconstitution of the monument [START_REF] Gaugne | Large-scale immersive reconstitution of a Neolithic corbel dome[END_REF]. Two implementations based on two different software were produced. The first one was based on the collaborative virtual reality engine Collaviz [START_REF] Dupont | Collaborative Scientific Visualization: The COLLAVIZ Framework[END_REF], with a visual rendering by OpenSG [START_REF] Voss | A Multi-thread Safe Foundation for Scene Graphs and its Extension to Clusters[END_REF]. The second one was based on Unity3D combined with MiddleVR. In addition to the immersive vizualisation in a large CAVE-like structure, several interaction metaphors has been implemented in order to help the archaeologist analysis work:

Navigation: the size of the immersive infrastructure used in this work enables natural walking inside the virtual central chamber. We added a flying mode (Figure 6) in order to ease the exploration of the corbel from bottom to top. The user is flying forward or backward following the direction pointed by a flystick, a tracked peripheral with a little joystick. Left and right directions of the joystick perform a rotation of the camera. This navigation metaphor is very natural for inexperienced users. Mobile lightening: the user has the ability to switch on a virtual lamp (Figure 7) linked to the flystick peripheral. This torch allows the user to focus on a restricted part of the chamber wall. This also enables the study of shadows rendering on the stones, which can be useful in presence of graves. Annotations: in order to assist the archaeologists in his analysis work, textual annotations and graphical lines can be added in the 3D universe and saved at the end of a working session to be reused in a following session. Textual annotations are labels stored in a separate editable configuration file. To pin an annotation on the wall, the user point the target with a ray attached to a wand and press a button once the right label is chosen (Figure 6). Labels are moving according to the user position in order to be always visible. A text file storing the 3D coordinates of the labels and drawings provides persistence of the annotations.

Specific display:

We were asked by archaeologists to provide the possibility to hide and display specific parts of the central chamber to visualize the different phases of the building (Figure 8). The purpose is to analyze the human postures and the potential use of scaffolding at each step. For this work, an archaeologist worked with the graphic designer of the WDCAH to identify the different layers corresponding to the building steps. The model was cut according to these layers and integrated in the immersive implementation so that the user was able to successively display the different building phases at 1:1 scale. The Virtual Cairn application has also been deployed on an head-mounted display, and a VR desktop equipment called Leonar3DO (Figure 9). These systems, far lighter and cheaper than CAVE-like structures, enable the use of interactions within virtual reality applications Fig. 9. The virtual cairn on portable VR equipment.

DISCUSSION

An obstacle in the use of virtual reality tools in the archaeology community is technology itself. CAVE-like infrastructures are expensive and uncommon facilities. Interactive peripherals require hardware and software expertise to integrate in VR applications. There has been a great effort of mutualisation of visualization and 3D technologies, with European projects such as 3D-COFORM (http://www.3d-coform.eu/) or Visionair (http://www.infra-visionair.eu) which both aim at promoting and funding the access to digital technologies for a wide range of European researchers. In a similar but less ambitious purpose, the WDCAH is designed as a meeting place where tools and skills join together to better support archaeological works, increase their value and better preserve and disseminate them. Such a space is definitely federative for researchers with different backgrounds. This article is the written proof of this federation through the gathering of different research institutes of both computer science and archaeology. By placing itself in a perspective other regions have long been committed to, western France certainly gives itself the means to boost its archaeological research.

An interesting aspect of the collaboration between computer scientists and archaeologists in the WDCAH is the raising of new problematic for virtual reality. Until now, virtual reality has been essentially focused on industrial applications. In this context, the user interacts with recent, smooth and clean, manufactured objects modeled by well-known CAD tools. In the context of archaeology, manipulated objects may be closer from nature, less steady, thus represented by more complex geometric models. The management of this new kind of data requires the study of new methodologies. As in the Extreme Programming software development methodology, the presentation of these virtual reality tools to archaeologists has helped to initiate a real iterative process in which they will do many frequent, positive and negative feedbacks, to obtain the best tools corresponding to their needs.

PERSPECTIVES

In addition to the fourteen current projects, we are working on the organization of seven other reconstruction projects combining modeling and digitization (photogrammetry and / or laser scanning). Some of these projects already identified the need of virtual reality tools in their analysis process. Particularly, the underwater sites represent very promising fields of application for virtual reality, from exploration training to reconstitution analysis.

Our first experiments with path finding in the Temple of Mars were also encouraging and led us to explore the application of this functionality to the study of complex building such as inside of a castel. Another important aspect to develop is collaboration in virtual environments. We have the software technology to set collaborative sessions, but we still have to identify the suitable project in the WDCAH context.

CONCLUSION

We presented the virtual reality tools currently set for the WDCAH. The West Digital Conservatory for Archaeological Heritage aims at experimenting the integration of 3D reconstruction and virtual reality techniques in collaboration with a community of archaeologists, who might or might not be aware of such technology but who clearly have many needs. Use of immersive interactive virtual reality as illustrated in this paper improved the work of involved archaeologists. Although the WDCAH is only a few months old, the first feedbacks seem very positive and reconstitution requests come regularly enough to let us believe in the real value of this project.

Fig. 1 .

 1 Fig. 1. 3D models of the keep of Sainte-Suzanne. © Y. Bernard

Fig. 2 .

 2 Fig. 2. Large and small immersive platforms.

Fig. 3 .

 3 Fig. 3. Temple of Mars -Corseul. ©M. Gautier / Y. Bernard Virtual reality application implementation: we chose to implement our application in Unity3D combined with MiddleVR

Fig. 4 .

 4 Fig. 4. Immersive visualization of Temple and a travel path.

Fig. 5 .

 5 Fig. 5. The cairn of Carn island.

Fig. 6 .

 6 Fig. 6. Flying in the virtual cairn / Annotations in the cairn

Fig. 7 .

 7 Fig. 7. Virtual torch.

Fig. 8 .

 8 Fig. 8. Visualization of the building steps : archaeologists' description on left, implementation right Remote collaboration: We successfully experimented a distributed deployment of the immersive implementation of the Cairn between our CAVE-like platform and another CAVE-like facility located in the south of France. This deployment was based on the software Collaviz for the collaborative and interaction part, and OpenSG for the visualization part. The actual version allows to share the same visual universe, with simple avatars representing the remote user's head and right hand.

ACKNOWLEDGMENTS

The authors wish to thank Paul Desiles and Cédric Le Cam for their work on the implementation. We'd also like to thank the archaeologists Florian Cousseau, Luc Laporte and Pascal Tregan who performed the scientific supervision of these reconstructions. This work was partially funded by the Université Européenne de Bretagne.