Sensitivity of the electrocardiography inverse solution to the torso conductivity uncertainties

Contact Information:

Carmen project Inria Bordeaux Sud-Ouest 200 ru de la vieille tour, 33405 Talence, France Phone: +33 (0) 5 2457 4180 Email: nejib.zemzemi@inria.fr Abstract Electrocardiography imaging (ECGI) is a new non invasive technology used for heart diagnosis. It allows to construct the electrical potential on the heart surface only from measurement on the body surface and some geometrical informations of the torso. The purpose of this work is twofold: First, we propose a new formulation to calculate the distribution of the electric potential on the heart, from measurements on the torso surface. Second, we study the influence of the errors and uncertainties on the conductivity parameters, on the ECGI solution. We use an optimal control formulation for the mathematical formulation of the problem with a stochastic diffusion equation as a constraint. The descretization is done using stochastic Galerkin method allowing to separate random and deterministic variables. The optimal control problem is solved using a conjugate gradient method where the gradient of the cost function is computed with an adjoint technique . The efficiency of this approach to solve the inverse problem and the usability to quantify the effect of conductivity uncertainties in the torso are demonstrated through a number of numerical simulations on a 2D geometrical model.

Main Objectives

Methods

Stochstic forward problem of electrocardiography

We denote by D the space domain and ⌦ the probability space. 8 > < > :

5.( (x, ⇠) 5 u(x, ⇠)) = 0 in D ⇥ ⌦, u(x, ⇠) = u 0 on int ⇥ ⌦, (x, ⇠) @u(x,⇠) @n = 0 on ext ⇥ ⌦, (1)
where, int and ext are the epicardial and torso boundaries respectively, ⇠ 2 ⌦ is the stochastic variable (it could also be a vector) and u 0 is the potential at the epicardial boundary.

Numerical descretization of the stochastic forward problem

We use the stochastic Galerkin method to solve equation ( 1). The stochastic conductivity and solution are projected on the probability density functions

{ k (⇠)} p k=1 (x, ⇠) = p X i=0 ˆ i (x) i (⇠, u(x, ⇠) = p X j=0 ûj (x) i (⇠)
The elliptic equation ( 1) projected in the stochastic basis could be solved

p X i=0 p X j=0 T ijk r.(ˆ i (x)r)û j (x)) =0 in D, û0 (x) =u 0 (x) on int , ûj (x) =0 on int 8j = 1, ...p, ˆ i (x) @ ûj (x) @n =0 on ext 8 i, j = 0, ...p, (2)
where

T ijk = E[ i (⇠), j (⇠), k (⇠)].
Anatomical data and computational mesh

Forward problem results

Exact deterministic solution Mean value for conductivity ±50%

Stdev Lung conductivity ±50% Stdev Fat conductivity ±50%

Main Remarks

1. The mean value of the stochastic solution matches with the exact forward solution. This comes from the linearity of the forward problem. 2. For each organ, the uncertainty on the conductivity is reflected by a high uncertainty of the solution at its boundary 3. The direction of the standard deviation iso-values are are modified when they cross the the organ for which we introduce the uncertainty. 4. The magnitude of the uncertainty does not exceed ±2% of the magnitude of the forward solution

Stochastic ECGI Inverse Problem Mathematical formulatin

We look for the current density and the value of the potential on the epicardial boundary

(⌘, ⌧ ) 2 L 1 2 ( int ) ⇥ L 1 2 ( int ) by minimizing the following cost function under a stochastic constraint on v 8 > > > > > > > < > > > > > > > : J(⌘, ⌧ ) = 1 2 E ⇣ kv(x, ⇠) f k 2 L 2 ( ext ) + 1 2 (x, ⇠) @v(x,⇠) @n ⌘ 2 L 2 ( int ) ⌘ with v(x, ⇠) solution of : 5.( (x, ⇠) 5 v(x, ⇠)) = 0 in D ⇥ ⌦, v(x, ⇠) = ⌧ on int ⇥ ⌦, (x, ⇠) @v(x,⇠)
@n = 0 on ext ⇥ ⌦.

(3) In order to solve this minimization problem, we use a conjugate gradient method as used in [START_REF] Aboula◆ch | Missing boundary data reconstruction via an approximate optimal control[END_REF] where the components of the gradient of the cost function are computed using an adjoint method. The gradient of the functional J is given by:

8 > > > > > > > > > < > > > > > > > > > : < @J(⌘,⌧ ) @⌘ . >= E[ R int ( @v @n ⌘) d int ] 8 2 L 2 ( int ), < @J(⌘,⌧ ) @⌧ .h >= E[ R int @ @n hd int ] 8 h 2 L 2 ( int ), with solution of : r.( (x, ⇠)r (x, ⇠)) = 0 on D ⇥ ⌦, (x, ⇠) = (x, ⇠) @v(x,⇠) @n ⌘ on int ⇥ ⌦, (x, ⇠) @ (x,⇠) @n = (v f ) on ext ⇥ ⌦. (4)
We use the conjugate gradient method to minimize the energy function J.

Inverse problem results

Exact deterministic solution

Mean value, lung conductivity ±50%

Stdev Lung conductivity ±50% RE lung conductivity ±50% Organ % uncertainties 0% ±10% ±20% ±30% ±50%

Main Remarks

1. The relative error between the mean value of the stochastic solution and exact forward solution reaches 50%. 2. Like for the forward problem, the direction of the standard deviation iso-values are are modified when they cross the the organ for which we introduce the uncertainty. 3. The magnitude of the uncertainty reaches its maximum at the edge of the considered orrgan

Conclusions

• The main contribution of this work was to introduce a new method for solving the ECGI inverse problem. This method is based on stochastic Galerkin approche. And the optimal control problem that we proposed allowed us to incorporate the uncertainties on the conductivity values as a constraint. The conjugate gradient method allow to take into account the conductivity uncertainties during the optimization procedure.

• The results show a low effect of conductivity uncertainties on the forward problem. On the contrary, their effect on the inverse solution is very important.

• For both inverse and forward solution the standard deviation of the stochastic solution achieves its maximum at the boundary of the organ for which the uncertainty was considered.

  A simulation study R. Aboulaich, N. Fikal, E. El Guarmah & N. Zemzemi INRIA Bordeaux Sud-Ouest, LIRYC, LERMA Mohammadia Engineering School, Royal Air Force school.

1 .

 1 Propose a new method for solving the ECGI problem.2. Introduce the uncertainty of the conductivity in the ECGI problem 3. Evaluate the effect of uncertainties on the forward and inverse solutions.

Figure 1 :

 1 Figure 1: MRI 2D slice of the torso (left), 2D computational mesh of the torso geometry showing the different regions of the torso considered in this study: fat, lungs and torso cavity, (right).

Figure 2 :

 2 Figure 2: Stochastic solution of the forward problem: Exact solution (top, left), Mean value of the Stochastic solution for conductivity ±50% (top, right). Standard deviation of the electrical potential for lung conductivity ±50% (bottom, left) and fat conductivity ±50% (bottom, right).

Figure 3 :

 3 Figure 3: Stochastic solution of the inverse problem: Exact solution (top, left), Mean value of the Stochastic inverse solution for lung conductivity ±50% (top, right). Standard deviation of the electrical potential for lung conductivity ±50% (bottom, left) and relative error (RE) between the mean value and the exact solution (bottom, right).

Table 1 :

 1 Relative error and correlation coefficient of the stochastic inverse solution for different levels of uncertainty on the fat and lungs conductivities

	Lungs relative error	0.1245 0.1439 0.2208 0.3333 0.485
		Corr coeff	0.9930 0.9899 0.9767 0.9660 0.885
	Fat	relative error	0.1245 0.1248 0.1248 0.1251 0.127
		Corr coeff	0.9930 0.9945 0.9943 0.9980 0.991

Acknowledgements

We would like to thank the LIRIMA Laboratory which financially supported the teams ANO and EPICARD to perform this work. This work was partially supported by an ANR grant part of "Investissements d'Avenir" program with reference ANR-10-IAHU-04


N Zemzemi

M Potse

L Bear

Y Coudière

Rémi Dubois

J Henry

C Dallet

J Duchateau

O Bernus

M Haïssaguerre

Major

New Mathematical approaches in Electrocardiography Imaging inverse problem

Keywords:

Improve ECGI inverse problem reconstruction

Introduce new mathematical approches to the field of the ECGI inverse problem Compare the performance of the new mathematical approaches to the state-of-the-art methods, mainly the MFS method used in commercial devices.

In silico validation of the new approches.

Assessment of some simplification hypothesis: Torso inhomogeneity Propose some uncertainty quantification apronches to deal with measurements errors

Context and objectives

Optimal control approach

Mathematical model

In silico gold standard

Results

Torso Heterogeneity effect

Conclusions

Forward model

If we know the heart potential we can compute the electrical potential

Inverse problem

If we know the electrical potential and the current density at the outer boundary of the torso and we look for the electrical potential at the heart surface

Computational heart and torso anatomical models + electrodes position

Computational torso meshes: 250 nodes mesh (blue). More accurate FE mesh with 6400 nodes (green)

Remarks

Introducing the torso heterogeneity is natural with FEM. also anisotropy could be introduced

The error is more important in the left ventricle

Main results and perspectives

New mathematical approches for solving the inverse problem in electrocardiography imaging based on optimal control Over all the 20 cases used in this study the optimal control method performs better than the MFS both in terms of relative error and correlation coefficient:

T ru T • n T = 0, on ext .

(

) 2
where T stands for the torso conductivity tensor and n T is the outward unit normal to the torso external boundary ext . The heart-torso interface ⌃ is supposed to be a perfect conductor. Then we have a continuity of current and potential between the extra-cellular myocardial region and the torso region.

( u e = u T , on ⌃, ( e + i )ru e • n T = T ru T • n T , on ext .

(3)

Other works [9], [10], [11] consider that the electrical current does not flow from the heart to the torso by assuming that the heart is isolated from torso. This approximation is appealing in terms of computational cost because it uncouples the Laplace equation ( 2) in the torso from the bidomain equations in the heart (1), which allows to reduce the size of the linear system to solve. It is even more appealing when the interest is only on the ECG computation, in that case the ECG solution could be an "off line" matrix vector multiplication after solving the bidomain equation, details about computing the transfer matrix could be found in [12]. Although this approach is very appealing in terms of computational cost, numerical evidence has shown that it can compromise the accuracy of the ECG signals (see e.g.

[11], [4], [13]). Thus, in order to accurately compute ECGs we consider the the state-of-the-art heart-torso full coupled electrophysiological problem (1)-( 3) representing the cardiac electrical activity from the cell to the human body surface.

III. INVERSE PROBLEM

The inverse problem in electrocadiography imaging (ECGI) is a technique that allows to construct the electrical potential on the heart surface ⌃ from data measured on the body surface ext . We assume that the electrical potential is governed by the diffusion equation in the torso as shown in the previous paragraph. For a given potential data T measured on the body surface ext , the goal is to find u e on ⌃ such that the potential data in the torso domain satisfies (5)

u N satisfies u N (x 0 ) = u D (x 0 ) and 8 > < > :

div( T ru N ( )) = 0, in ⌦ T ,

T ru N ( ).n = 0, on ext , u N ( ) = , on ⌃.

(6)

Let's define the cost function as follows

J( ) = 1 2 Z ⌦ T (ru D ( ) ru N ( )) 2 . ( 7
)
This means that if J( ) = 0, we have ru D ( ) = ru N ( ) and then ru D ( ).n = ru N ( ).n on the boundary and in particular on ext . Therefore, since u D = u N = on ⌃ and by uniqueness of the Laplace solution, we obtain that u D ( ) = u N ( ) in all the domain ⌦ T . Which means that both of solutions u D ( ) and u N ( ), satisfy both Neumann and Dirichlet boundary conditions on ext at the same time.

We need to minimize the cost function J. Since we are able to calculate the gradient of the cost function, we can use a gradient descent method. For any test function , the gradient of J reads: For a given initial guess , we compute the incomplete boundary condition following this algorithm:

Algorithm 1 Algorithm for i = f irst time step to last time step do load the known boundary data u itions of the monodomain problem. The precise defons of g and I ion depend on the electrophysiological membrane ionic model. In the present work we make of one of the human myocyte model Beeler Reuter H stands for the heart domain which is in our case tria. In order to simulate the body surface potential rst need to compute the extracellular potential in the t. Supposing the the extracellular conductivity tensor i , where i is the intracellular conductivity tensor 2 R. The extracellular potential satisfies ( + 1) div( i ru e ) = div( i rv m ).

T (t i ) / ext compute u D ( ) compute u N ( ) compte r J( ) = (ru D ru N ).n while (k r J( ) k> tolerence) do ⇤ r J( ) compute u D ( ) compute u N ( ) compte r J( ) = (ru D ru N ).n end while save u T (t i ) /⌃ =
(2) can easily check that

u e = 1 (1 + ) (< v m > v m ) (3)
solution of equation ( 2). Morover, it is the unique son satisfying

✓ Z ⌦ H u e ◆ = 0. , < v m >= 1 |⌦ H | Z ⌦ H v m is the mean value of v m in e.
In order to compute the body surface potential we to solve a Laplace equation on the torso with a Dirichoundary condition on the heart-torso interface [4].

8 > < > : div( T ru T ) = 0, in ⌦ T , T ru T .n = 0, on ext , u T = u e , on ⌃. (4)
forward problem algorithm is: (a) compute the transbrane potential by solving equation ( 1), (b) compute xtracellular potential using the formula (3), (c) comthe torso potential by solving (4). As it was explained e introduction we build a synthetic data base of BSPs their correspondant EGMs. Each sample of BSPs and s corresponds to a stimulation location. We simulated art beats, each one corresponds to a given I stim . We finite element method in order to solve equations (1) (4), a space discretisation of the heart and torso dos is then needed. Since we are interested in targeting pic beats in the atria, we only consider the electrical activation in the atria. The finite element geometry of atria is given in Figure 2.1 (left). it was embedded in a torso geometry given in Figure 2.1 (right) [6].

2.2.

Inverse problem: a regression method

As explained in the previous paragraph, we have n samples of BSP and EGMs.

The sequence (BSP i , EGM i ) i=1...n 2 R p⇥m ⇥ R q⇥m is the data set of our metamodel. Here p (respectively, q) is the number of potential measurement locations on the body surface (respectively, on the heart surface), and m is the number of time steps. We denote by (x k ) k=1...p (respectively, (y k ) k=1...q ) the positions of the BSP measurements (respectively, positions of EGMs measurements) and (t l ) l=0...m 1 the times of the recordings. For the i th element of our data set we have, BSP i 2 R p⇥m , where BSP i (l ⇥ p + k) = u T (x k , t l ), for k = 1, ..., p and l = 0, ..., m 1, and EGM i 2 R q⇥m where, EGM i (l ⇥ q + k) = u e (y k , t l ), for k = 1, ..., q and l = 0, ..., m 1.

The main goal is to build a function f able to accurately map a BSP to an EGM. We use a kernel ridge regression method based on the gaussian kernel

K(x, y) = e |x y| 2 2 2 , 8x, y 2 R m⇥p . We look for f in a Reproducing Kernel Hilbert Space (RKHS) (H, h•, •i H ) characterized by the following prop- erty, 8f 2 H, 8x 2 R m⇥p , f(x) = hf (•), K(•, x)i H .
(5) where h•, •i H is the inner product in H. The use of the Gaussian kernel can be motivated by the following property (see [7]): given a compact subset K of R m⇥p , the set of the restriction to K of functions from H is dense in the set of continuous functions from K to R. In other word, tween the extra-cellular myocardial region and the torso gion.

MFS approach

( u e = u T , on ⌃,

( e + i )ru e • n T = T ru T • n T , on ⌃. ( 3
)
ther works [9], [10], [11] consider that the electrical current es not flow from the heart to the torso by assuming at the heart is isolated from torso. This approximation appealing in terms of computational cost because it couples the Laplace equation (2) in the torso from the domain equations in the heart (1), which allows to reduce e size of the linear system to solve. It is even more pealing when the interest is only on the ECG computation, that case the ECG solution could be an "off line" matrix ctor multiplication after solving the bidomain equation, tails about computing the transfer matrix could be found [12]. Although this approach is very appealing in terms computational cost, numerical evidence has shown that it n compromise the accuracy of the ECG signals (see e.g. 1], [5], [13]). Thus, in order to accurately compute ECGs e consider the the state-of-the-art heart-torso full coupled ectrophysiological problem (1)-( 3) representing the cardiac ectrical activity from the cell to the human body surface.

.

Inverse problem

The inverse problem in electrocadiography imaging CGI) is a technique that allows to construct the electrical tential on the heart surface ⌃ from data measured on the dy surface ext . We assume that the electrical potential is verned by the diffusion equation in the torso as shown the previous paragraph. For a given potential data T easured on the body surface ext , the goal is to find u e on such that the potential data in the torso domain satisfies (5)

8 > < > : div( T ru N ( )) = 0, in ⌦ T , T ru N ( ).n = 0, on ext , u N ( ) = , on ⌃. (6)
Let's define the cost function as follows

J( ) = 1 2 Z ⌦ T (ru D ( ) ru N ( )) 2 . ( 7
)
This means that if J( ) = 0, we have ru D ( ) = ru N ( ) in the whole domain ⌦ T and then ru D ( ).n = ru N ( ).n on the boundary and in particular on ext . Therefore, since u D = u N = on ⌃ and by uniqueness of the Laplace solution, we obtain that u D ( ) = u N ( ) in all the domain ⌦ T . Which means that both of solutions u D ( ) and u N ( ), satisfy both Neumann and Dirichlet boundary conditions on ext at the same time. This means that is the solution of the inverse problem.

Our goal is to minimize the cost function J. Since we are able to calculate the gradient of the cost function, we can use a gradient descent method. For any test function , the gradient of J reads:

hr J( ), i = ⌧ Z ⌦ T T (ru D ru N ) r T (ru D ru N ), = ⌧ Z ⌦ T T (ru D ru N ) r ( T ru D ), ⌧ Z ⌦ T T (ru D ru N ) r ( T ru N ), = ⌧ Z @⌦ T T (ru D ru N ).n u ⇤ D ( ) ⌧ Z @⌦ T (u D ( ) u N ( )) T ru ⇤ N ( ).n (8) where u ⇤ D ( ) is solution of the following problem 8 > < > : div( T ru ⇤ D ( )) = 0, in ⌦ T , u ⇤ D ( ) = 0, on ext , u ⇤ D ( ) = , on ⌃, (9) and u ⇤ N ( ) is solution of the following problem 8 > < > : div( T ru ⇤ N ( )) = 0, in ⌦ T , T ru ⇤ N ( ).n = 0, on ext , u ⇤ N ( ) = , on ⌃. (10) Since, u ⇤ D ( ) = 0, on ext , u D ( ) = u N ( ) = on ⌃ and T ru ⇤ N ( ).n = 0 on ext , we then obtain r J( ) = T (ru D ( ) ru N ( )).n /⌃ (11)
1) Iterative solver: We can solve the problem iteratively as in [] For a given initial guess , we compute the incomplete boundary condition following the algorithm 1.

In practice, putting = 0 out of the time loop reduces the number of iterations in the "for" loop. This could be explained by the fact that the solution at time t + t is closer to the solution at time t than it is to zero. (3)

8 > < > : div( T ru N ( )) = 0, in ⌦ T , T ru N ( ).n = 0, on ext , u N ( ) = , on ⌃. (6)
Let's define the cost function as follows

J( ) = 1 2 Z ⌦ T (ru D ( ) ru N ( )) 2 . ( 7
)
This means that if J( ) = 0, we have ru

div( T ru N ( )) = 0, in ⌦ T , T ru N ( ).n = 0, on ext , u N ( ) = , on ⌃. (6)
Let's define the cost function as follows

J( ) = 1 2 Z ⌦ T (ru D ( ) ru N ( )) 2 . ( 7
)
This means that if J( ) = 0, we have ru Our goal is to minimize the cost function J. Since we are able to calculate the gradient of the cost function, we can use a gradient descent method. For any test function , the gradient of J reads: (6)

hr J( ), i = ⌧ Z ⌦ T T (ru D ru N ) r T (ru D ru N ), = ⌧ Z T (ru D ru N ) r ( T ru D ),
Let's define the cost function as follows

J( ) = 1 2 Z ⌦ T (ru D ( ) ru N ( )) 2 . ( 7
)
This means that if J( ) = 0, we have ru D ( ) = ru N ( ) in the whole domain ⌦ T and then ru D ( ).n = ru N ( ).n on the boundary and in particular on ext . Therefore, since u D = u N = on ⌃ and by uniqueness of the Laplace solution, we obtain that u D ( ) = u N ( ) in all the domain ⌦ T . Which means that both of solutions u D ( ) and u N ( ), satisfy both Neumann and Dirichlet boundary conditions on ext at the same time. This means that is the solution of the inverse problem.

Our goal is to minimize the cost function J. Since we are able to calculate the gradient of the cost function, we can use a gradient descent method. For any test function , the gradient of J reads:

hr J( ), i = ⌧ Z ⌦ T T (ru D ru N ) r T (ru D ru N ), = ⌧ Z ⌦ T T (ru D ru N ) r ( T ru D ), ⌧ Z ⌦ T T (ru D ru N ) r ( T ru N ), = ⌧ Z @⌦ T T (ru D ru N ).n u ⇤ D ( ) ⌧ Z @⌦ T (u D ( ) u N ( )) T ru ⇤ N ( ).n (8) where u ⇤ D ( ) is solution of the following problem 8 > < > : div( T ru ⇤ D ( )) = 0, in ⌦ T , u ⇤ D ( ) = 0, on ext , u ⇤ D ( ) = , on ⌃, (9) and u ⇤ N ( ) is solution of the following problem 8 > < > : div( T ru ⇤ N ( )) = 0, in ⌦ T , T ru ⇤ N ( ).n = 0, on ext , u ⇤ N ( ) = , on ⌃. (10) Since, u ⇤ D ( ) = 0, on ext , u D ( ) = u N ( ) = on ⌃ and T ru ⇤ N ( ).n = 0 on ext , we then obtain r J( ) = T (ru D ( ) ru N ( )).n /⌃ (11)
1) Iterative solver: We can solve the problem iteratively as in [] For a given initial guess , we compute the incomplete boundary condition following the algorithm 1.

In practice, putting = 0 out of the time loop reduces the number of iterations in the "for" loop. This could be explained by the fact that the solution at time t + t is closer NG AND Y. RUDY flating the heart surface by a factor of 0.8 relative to the geometrical center of the heart. The geometrical center of the heart can be found by computing the average coordinate value of all the heart surface nodes. For the torso surface, the fictitious boundary ˆ T is obtained by inflating the torso surface by a factor of 1.2 relative to the geometrical center of the heart.

Expressing Eqs. ( 2) and (3) in matrix form gives:

Â⃗ a = ⃗ b (4) where, Â = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ 1 f (∥x 1 -y 1 ∥) • • • f (∥x 1 -y M ∥) . . . . . . • • • . . . 1 f (∥x N -y 1 ∥) . . . f (∥x N -y M ∥) 0 ∂ f (∥x 1 -y 1 ∥) ∂n • • • ∂ f (∥x 1 -y M ∥) ∂n . . . . . . • • • . . . 0 ∂ f (∥x N -y 1 ∥) ∂n • • • ∂ f (∥x N -y M ∥) ∂n ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ , ⃗ a = ⎛ ⎜ ⎜ ⎜ ⎝ a 0 a 1 . . . a M ⎞ ⎟ ⎟ ⎟ ⎠ , ⃗ b = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ u T (x 1 ) . . . u T (x N ) 0 . . . 0 ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠
Matrix  is of dimension 2N × (M + 1); ⃗ a and ⃗ b are vectors of dimensions M + 1 and 2N respectively. This matrix equation can not be solved for ⃗ a without regularizaiton, 76 because the matrix  is ill-conditioned and the measured body surface potential contains measurement error. The Tikhonov regularization method 76 with CRESO-determined regularization parameter 24 is used to stabilize the inverse procedure and obtain ⃗ a, similar to our previous ECGI inverse computations using mesh-based BEM. 15,16,32,33,47,56,61,62,65-71 Once the coefficient vector ⃗ a is obtained, u(x) can be computed at any location in the domain using:

u(x) = a 0 + M j=1 a j f (∥x -y j ∥), x ∈ , y j ∈ ˆ (5)
The epicardial potential can then be calculated using:

u E (x) = a 0 + M j=1 a j f (∥x -y j ∥), x ∈ E , y j ∈ ˆ (6)
Epicardial potentials are calculated using (6) on many epicardial nodes; numbers are provided for each dataset in the Results section. An epicardial potential map, reflecting the spatial distribution of potentials on the epicardial eart surface by a factor of 0.8 relative to the center of the heart. The geometrical center of be found by computing the average coordinate the heart surface nodes. For the torso surface, boundary ˆ T is obtained by inflating the torso factor of 1.2 relative to the geometrical center g Eqs. ( 2) and (3) in matrix form gives:

Â⃗ a = ⃗ b (4) 1 f (∥x 1 -y 1 ∥) • • • f (∥x 1 -y M ∥) . . . . . . • • • . . . 1 f (∥x N -y 1 ∥) . . . f (∥x N -y M ∥) 0 ∂ f (∥x 1 -y 1 ∥) ∂n • • • ∂ f (∥x 1 -y M ∥) ∂n . . . . . . • • • . . . 0 ∂ f (∥x N -y 1 ∥) ∂n • • • ∂ f (∥x N -y M ∥) ∂n ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ , a 0 a 1 . . . a M ⎞ ⎟ ⎟ ⎟ ⎠ , ⃗ b = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ u T (x 1 ) . . . u T (x N ) 0 . . . 0 ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠
of dimension 2N × (M + 1); ⃗ a and ⃗ b are vecnsions M + 1 and 2N respectively. rix equation can not be solved for ⃗ a without n, 76 because the matrix  is ill-conditioned ured body surface potential contains measure-The Tikhonov regularization method 76 with rmined regularization parameter 24 is used to inverse procedure and obtain ⃗ a, similar to ECGI inverse computations using mesh-based

ition : a 0 + M j=1 a j f (∥x k -y j ∥) = u T (x k ), 1 ≤ k ≤ N , x k ∈ T , y j ∈ ˆ (2) dition : M j=1 a j ∂ f (∥x k -y j ∥) ∂n = c T (x k ) = 0, 1 ≤ k ≤ N , x k ∈ T , y j ∈ ˆ (3) 1 4πr
is the fundamental solution of Laplace's , r = ∥x -y∥ is the 3D Euclidean distance x and point y, n is normal to the torso surconstant component of u T (x) and a j is the a virtual source at location y j . Note that a 0 ifferent units in this formulation. The cone volume is reflected in the coefficient a j ; it ar explicitly in the ECGI formulation when interest is homogenous. M is the number of s. N is the number of torso surface points. is f domain , and ˆ is the auxiliary boundary y domain ˆ , which contains the domain as 1. conditions are satisfied on N torso surface q.

(2) u T (x k ) is the measured body surface ctrode position x k . In Eq. ( 3), c T (x k ) = 0 beo is in air, an insulating medium that does urrent flow. The locations of the fictitious configured based on the particular domain ich in ECGI is a multi-connected surface sed of the body surface and heart surface ). Using a static configuration scheme (see e fictitious sources are placed on two aux-( ˆ = ˆ T ∪ ˆ E ) which are determined by tion of the true surfaces (torso surface and Figure 1 shows the configuration of the fictia 2D representation. The fictitious boundary to the heart surface ˆ E is obtained by de-flating the heart surface by a factor of 0.8 relative to the geometrical center of the heart. The geometrical center of the heart can be found by computing the average coordinate value of all the heart surface nodes. For the torso surface, the fictitious boundary ˆ T is obtained by inflating the torso surface by a factor of 1.2 relative to the geometrical center of the heart.

Expressing Eqs. ( 2) and (3) in matrix form gives:

Â⃗ a = ⃗ b (4) where, Â = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ 1 f (∥x 1 -y 1 ∥) • • • f (∥x 1 -y M ∥) . . . . . . • • • . . . 1 f (∥x N -y 1 ∥) . . . f (∥x N -y M ∥) 0 ∂ f (∥x 1 -y 1 ∥) ∂n • • • ∂ f (∥x 1 -y M ∥) ∂n . . . . . . • • • . . . 0 ∂ f (∥x N -y 1 ∥) ∂n • • • ∂ f (∥x N -y M ∥) ∂n ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ , ⃗ a = ⎛ ⎜ ⎜ ⎜ ⎝ a 0 a 1 . . . a M ⎞ ⎟ ⎟ ⎟ ⎠ , ⃗ b = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ u T (x 1 ) . . . u T (x N ) 0 . . . 0 ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠
Matrix  is of dimension 2N × (M + 1); ⃗ a and ⃗ b are vectors of dimensions M + 1 and 2N respectively. This matrix equation can not be solved for ⃗ a without regularizaiton, 76 because the matrix  is ill-conditioned and the measured body surface potential contains measurement error. The Tikhonov regularization method 76 with CRESO-determined regularization parameter 24 is used to stabilize the inverse procedure and obtain ⃗ a, similar to our previous ECGI inverse computations using mesh-based BEM. 15,16,32,33,47,56,61,62,65-71 Once the coefficient vector ⃗ a is obtained, u(x) can be computed at any location in the domain using:

u(x) = a 0 + M j=1 a j f (∥x -y j ∥), x ∈ , y j ∈ ˆ (5)
The epicardial potential can then be calculated using:

u E (x) = a 0 + M j=1 a j f (∥x -y j ∥), x ∈ E , y j ∈ ˆ (6)
Epicardial potentials are calculated using (6) on many epicardial nodes; numbers are provided for each dataset in the Results section. An epicardial potential map, reflecting the spatial distribution of potentials on the epicardial Y. WANG AND Y. RUDY e auxiliary domain ˆ , which contains the as shown in Fig. 1). As the fundamental y Laplace's equation everywhere except at this representation satisfies Laplace's equaain . In addition, the specified boundary imposed at a set of boundary points (colloon the domain boundary . Since the funions do not have singularities at points on , standard quadrature rules can be used to e surface potential and its normal gradient on the boundary. 37 the Appendix (Eqs. (a21) and (a22)), MFS to discretize the Dirichlet and Neumann itions in Eq. (1) as:

tion : a 0 + M j=1 a j f (∥x k -y j ∥) = u T (x k ), 1 ≤ k ≤ N , x k ∈ T , y j ∈ ˆ (2) ition : M j=1 a j ∂ f (∥x k -y j ∥) ∂n = c T (x k ) = 0, 1 ≤ k ≤ N , x k ∈ T , y j ∈ ˆ (3) 1 4πr
is the fundamental solution of Laplace's , r = ∥x -y∥ is the 3D Euclidean distance x and point y, n is normal to the torso surconstant component of u T (x) and a j is the virtual source at location y j . Note that a 0 fferent units in this formulation. The convolume is reflected in the coefficient a j ; it r explicitly in the ECGI formulation when nterest is homogenous. M is the number of . N is the number of torso surface points. is f domain , and ˆ is the auxiliary boundary domain ˆ , which contains the domain as . onditions are satisfied on N torso surface q.

(2) u T (x k ) is the measured body surface trode position x k . In Eq. ( 3), c T (x k ) = 0 beis in air, an insulating medium that does rrent flow. The locations of the fictitious onfigured based on the particular domain ch in ECGI is a multi-connected surface ed of the body surface and heart surface ). Using a static configuration scheme (see fictitious sources are placed on two aux-( ˆ = ˆ T ∪ ˆ E ) which are determined by on of the true surfaces (torso surface and igure 1 shows the configuration of the ficti-2D representation. The fictitious boundary to the heart surface ˆ E is obtained by de-flating the heart surface by a factor of 0.8 relative to the geometrical center of the heart. The geometrical center of the heart can be found by computing the average coordinate value of all the heart surface nodes. For the torso surface, the fictitious boundary ˆ T is obtained by inflating the torso surface by a factor of 1.2 relative to the geometrical center of the heart.

Expressing Eqs. ( 2) and (3) in matrix form gives: This matrix equation can not be solved for ⃗ a without regularizaiton, 76 because the matrix  is ill-conditioned and the measured body surface potential contains measurement error. The Tikhonov regularization method 76 with CRESO-determined regularization parameter 24 is used to stabilize the inverse procedure and obtain ⃗ a, similar to our previous ECGI inverse computations using mesh-based BEM. 15,16,32,33,47,56,61,62,65-71 Once the coefficient vector ⃗ a is obtained, u(x) can be computed at any location in the domain using:

Â⃗ a = ⃗ b (4) where,  = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ 1 f (∥x 1 -y 1 ∥) • • • f (∥x 1 -y M ∥) . . . . . . • • • . . . 1 f (∥x N -y 1 ∥) . . . f (∥x N -y M ∥) 0 ∂ f (∥x 1 -y 1 ∥) ∂n • • • ∂ f (∥x 1 -y M ∥) ∂n . . . . . . • • • . . . 0 ∂ f (∥x N -y 1 ∥) ∂n • • • ∂ f (∥x N -y M ∥) ∂n ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ , ⃗ a = ⎛ ⎜ ⎜ ⎜ ⎝ a 0 a 1 . . . a M ⎞ ⎟ ⎟ ⎟ ⎠ , ⃗ b = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ u T (x 1 ) . . . u T (x N ) 0 . . . 0 ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ Matrix  is of dimension 2N × (M +
u(x) = a 0 + M j=1 a j f (∥x -y j ∥), x ∈ , y j ∈ ˆ (5)
The epicardial potential can then be calculated using:

u E (x) = a 0 + M j=1 a j f (∥x -y j ∥), x ∈ E ,

Experimental Methods and Protocols

MFS ECGI reconstructions were performed on data from four studies: (i) Single-site pacing in an isolated canine heart suspended in a human torso-shaped tank; data were obtained during pacing from a right ventricular (RV) anterior epicardial location; 62 (ii) RV endocardial pacing in a patient undergoing bi-ventricular pacing for cardiac resynchronization therapy (CRT); (iii) Simultaneous RV endocardial pacing and left ventricular (LV) epicardial pacing in a patient undergoing bi-ventricular pacing for CRT; (iv) Normal atrial activation in a healthy human subject. 65

Isolated Canine Hearts Suspended in a Human

Torso-Shaped Tank 62

The performance of MFS in ECGI was evaluated using data from a human torso-shaped tank. 62 The setup consisted of an isolated canine heart suspended in a homogenous electrolytic medium in the correct anatomical position inside a tank molded in the shape of a ten-year old boy. The tank had 384 surface electrodes recording torso potentials and 242 rods with electrodes at their tips that formed an epicardial recording envelope around the heart. The complete sets of torso-surface potentials and epicardial potentials were obtained by recording over several beats. The directly measured epicardial potentials by the rod-tip electrodes served as a "gold standard" for MFS ECGI validation. The torsosurface potentials provided the input data for MFS ECGI noninvasive reconstruction of epicardial potentials, electrograms and isochrones, which were then evaluated by comparison with the directly measured "gold standard". Details were provided in previous ECGI publications. 15,61,62,70 To simulate focal arrhythmogenic activity, the heart was paced from an anterior epicardial location. The same datasets were used in BEM ECGI and reported in previous publications. 62 Here, these datasets are used to evaluate MFS ECGI and compare its performance to that of BEM ECGI. The pacing protocol also provided a measure of MFS ECGI spatial accuracy (its accuracy in locating the known pacing site). After pacing, a quasi-elliptical region of intense epicardial negativity forms around the pacing site. 61,62,70,74 Bi-Ventricular Pacing In Human Subjects 65 We also applied MFS ECGI to clinical data from patients with an implanted bi-ventricular pacing device. For the biventricular pacing data, MFS ECGI accuracy in locating the pacing sites was evaluated by comparison with the pacing electrodes' positions as determined from CT images. The reconstructed activation pattern was evaluated based on the known patterns of activation generated by pacing.

Data from two heart failure patients undergoing cardiac resynchronization pacing therapy 65 are presented. Subject 1 was paced from a right ventricular (RV) endocardial site, close to the RV apex. Subject 2 was paced simultaneously from an RV endocardial site and from a left ventricular (LV) epicardial site. Body surface potentials were recorded with a 224-channel mapping system using an electrode-vest as previously described. 65 Epicardial geometry and location of the torso electrodes were obtained from CT images of the thorax. The locations of the cardiac pacing leads were also determined from these CT images. 65 Normal Atrial Activation in a Healthy Human Subject 65 MFS ECGI was applied to reconstruct atrial activation in a healthy young adult. The same data were used in BEM ECGI and reported in a previous study. 65 The atrial activation pattern was reconstructed from recorded P-wave body surface potential maps with 224 channels, together with a subject-specific torso and atrial geometry obtained using CT. The directly measured normal atrial activation pattern in isolated human hearts (Durrer et al. 25 ), was used for qualitative evaluation of the MFS ECGI reconstruction.

Informed consent was obtained according to Institutional Review Board guidelines at University Hospitals of Cleveland, which approved all human studies protocols.

Evaluation Procedures

For the tank-torso protocols, measures in terms of relative error (RE) and correlation coefficients (CC) were computed with respect to the measured data to quantitatively evaluate the accuracy of ECGI; RE and CC were defined previously. 71 RE gives an estimate of the amplitude difference and CC gives an estimate of the similarity of potential patterns or electrogram morphologies between the measured and computed data:

R E = L i=1 V C i -V M i 2 L i=1 V M i 2 CC = L i=1 V M i -V M V C i -V C L i=1 V M i -V M 2 L i=1 V C i -V C 2 For potential

Experimental Methods and Protocols

MFS ECGI reconstructions were performed on data from four studies: (i) Single-site pacing in an isolated canine heart suspended in a human torso-shaped tank; data were obtained during pacing from a right ventricular (RV) anterior epicardial location; 62 (ii) RV endocardial pacing in a patient undergoing bi-ventricular pacing for cardiac resynchronization therapy (CRT); (iii) Simultaneous RV endocardial pacing and left ventricular (LV) epicardial pacing in a patient undergoing bi-ventricular pacing for CRT; (iv) Normal atrial activation in a healthy human subject. 65

Isolated Canine Hearts Suspended in a Human

Torso-Shaped Tank 62

The performance of MFS in ECGI was evaluated using data from a human torso-shaped tank. 62 The setup consisted of an isolated canine heart suspended in a homogenous electrolytic medium in the correct anatomical position inside a tank molded in the shape of a ten-year old boy. The tank had 384 surface electrodes recording torso potentials and 242 rods with electrodes at their tips that formed an epicardial recording envelope around the heart. The complete sets of torso-surface potentials and epicardial potentials were obtained by recording over several beats. The directly measured epicardial potentials by the rod-tip electrodes served as a "gold standard" for MFS ECGI validation. The torsosurface potentials provided the input data for MFS ECGI noninvasive reconstruction of epicardial potentials, electrograms and isochrones, which were then evaluated by comparison with the directly measured "gold standard". Details were provided in previous ECGI publications. 15,61,62,70 To simulate focal arrhythmogenic activity, the heart was paced from an anterior epicardial location. The same datasets were used in BEM ECGI and reported in previous publications. 62 Here, these datasets are used to evaluate MFS ECGI and compare its performance to that of BEM ECGI. The pacing protocol also provided a measure of MFS ECGI spatial accuracy (its accuracy in locating the known pacing site). After pacing, a quasi-elliptical region of intense epicardial negativity forms around the pacing site. 61,62,70,74 Bi-Ventricular Pacing In Human Subjects 65 We also applied MFS ECGI to clinical data from patients with an implanted bi-ventricular pacing device. For the biventricular pacing data, MFS ECGI accuracy in locating the pacing sites was evaluated by comparison with the pacing electrodes' positions as determined from CT images. The reconstructed activation pattern was evaluated based on the known patterns of activation generated by pacing.

Data from two heart failure patients undergoing cardiac resynchronization pacing therapy 65 are presented. Subject 1 was paced from a right ventricular (RV) endocardial site, close to the RV apex. Subject 2 was paced simultaneously from an RV endocardial site and from a left ventricular (LV) epicardial site. Body surface potentials were recorded with a 224-channel mapping system using an electrode-vest as previously described. 65 Epicardial geometry and location of the torso electrodes were obtained from CT images of the thorax. The locations of the cardiac pacing leads were also determined from these CT images. 65 Normal Atrial Activation in a Healthy Human Subject 65 MFS ECGI was applied to reconstruct atrial activation in a healthy young adult. The same data were used in BEM ECGI and reported in a previous study. 65 The atrial activation pattern was reconstructed from recorded P-wave body surface potential maps with 224 channels, together with a subject-specific torso and atrial geometry obtained using CT. The directly measured normal atrial activation pattern in isolated human hearts (Durrer et al. 25 ), was used for qualitative evaluation of the MFS ECGI reconstruction.

Informed consent was obtained according to Institutional Review Board guidelines at University Hospitals of Cleveland, which approved all human studies protocols.

Evaluation Procedures

For the tank-torso protocols, measures in terms of relative error (RE) and correlation coefficients (CC) were computed with respect to the measured data to quantitatively evaluate the accuracy of ECGI; RE and CC were defined previously. 71 RE gives an estimate of the amplitude difference and CC gives an estimate of the similarity of potential patterns or electrogram morphologies between the measured and computed data: (8)

R E = L i=1 V C i -V M i 2 L i=1 V M i 2 CC = L i=1 V M i -V M V C i -V C L i=1 V M i -V M 2 L i=1 V C i -V C 2 For potential
Note that if we consider a static geometry, operators S N , S D and S rhs are computed once for all, since they depend only on the mesh and the conductivities in the torso represented by T . Computing S N , S D and S rhs avoids solving a 3D finite element problem at each iteration and allows a real time solving of the inverse problem.

III. NUMERICAL RESULTS

We start by generating synthetic data using an ECG simulator based on the bidomain model. We generate BSPs for two heart cases, a healthy and a fibrillation conditions. The model parameters are the same in both cases they only differ by the stimulation protocols: In the first case, the heart is stimulated in the apex and the electrical wave propagates from apex to base. We refer to this simulation as a normal or healthy case. In the second case we apply a S1-S2 protocol, in order to produce a re-entry wave, we refer to this case as a fibrillation or re-entree case. These two cases will be considered as our gold standard data. Further informations about the forward problem modelling could be found in [6], [7]. For each time step, we extract BSPs, we add 5% of noise and we use it as the given data T . we then solve the inverse problem following the KMF method explained in the previous paragraph. In Fig. 3 (respectively, Fig. 4), we show snapshots of the electrical potential distribution in the body (including heart surface and torso volume), in these snapshots we compare the inverse solution to the gold standard normal (respectively, re-entree) case. The distribution of the reconstructed electrical potential is synchronized with the gold standard electrical potential distribution in both normal and re-entree cases. We remark that the wave front is well captured but in the inverse solution it is much more smoother than in the exact solution. As shown in previous study [8], the wave front in the normal case is much accurate than in the re-entree case. Hence we will focus our study on the re-entree case. In Fig. 5 (left) (respectively right), we show a comparison between the inverse and the exact solutions show the spatial distribution of the relative error in time computing the l 2 relative error in time at each point of t heart surface geometry. We see that the error is much high in the left ventricle where its maximum reaches 75% wh in the right ventricle surface it doesn't exceed 35%. In F Our results show that the heterogeneity in the torso has an impact on the accuracy of the solution both in terms of RE and CC.

We are working on other new approches for solving ECGI problem and also quantifying the effect of the torso conductivity uncertainties on the ECGI solution Inria Bordeaux Sud-Ouest, IHU-LIRYC, CHU-Bordeaux, Université de Bordeaux.

x 1 , . . . , x N : Torso points y 1 , . . . , y M : Heart points Discretization with Finite elements method.

Fig. 1 .

 1 Fig. 1. Two-dimensional geometrical description: heart domain ⌦ H , torso domain ⌦ T (extramyocardial regions), heart-torso interface ⌃ and torso external boundary ext .

  ru T ) = 0, in ⌦ T , T ru T .n = 0, and u T = T, on ext , u T =?, on ⌃.

  find u e , for a given boundary value distribution on ⌃, we propose to define u D ( ) and u N ( ) as follows, ru D ( )) = 0, in ⌦ T , u D ( ) = T, on ext , u D ( ) = , on ⌃.

  ru N ) r (ru D ru N ), ru N ) r (ru D ), ⌧ Z ⌃ (ru D ru N ) r (ru N ),(8) Applying the Green formula on both terms we find r J( ) = (ru D ru N ).n (9)

Figure 2 .

 2 Figure 2. Finite element computational domains: Atria geometry with the different locations of stimulus used to construct the training data set (lfet). Torso geometry with different BSP measurements locations (right).

  Two-dimensional geometrical description: heart domain ⌦ H , torso main ⌦ T (extramyocardial regions), heart-torso interface ⌃ and torso ternal boundary ext .

  ru T ) = 0, in ⌦ T , T ru T .n = 0, and u T = T, on ext , u T =?, on ⌃.

  to find u e , for a given boundary value distribution on ⌃, we propose to define u D ( ) and u N ( ) as follows, 8 > < > : div( T ru D ( )) = 0, in ⌦ T , u D ( ) = T, on ext , u D ( ) = , on ⌃.

  heart domain ⌦ H , torso gions), heart-torso interface ⌃ and torso myocardial region and the torso u e = u T , on ⌃,

  D ( ) = ru N ( ) in the whole domain ⌦ T and then ru D ( ).n = ru N ( ).n on the boundary and in particular on ext . Therefore, since u D = u N = on ⌃ and by uniqueness of the Laplace solution, we obtain that u D ( ) = u N ( ) in all the domain ⌦ T . Which means that both of solutions u D ( ) and u N ( ),

  D ( ) = ru N ( ) in the whole domain ⌦ T and then ru D ( ).n = ru N ( ).n on the boundary and in particular on ext . Therefore, since u D = u N = on ⌃ and by uniqueness of the Laplace solution, we obtain that u D ( ) = u N ( ) in all the domain ⌦ T . Which means that both of solutions u D ( ) and u N ( ), satisfy both Neumann and Dirichlet boundary conditions on ext at the same time. This means that is the solution of the inverse problem.

  ru N ( )) = 0, in ⌦ T , T ru N ( ).n = 0, on ext , u N ( ) = , on ⌃.

,

  33,47,56,61,62,65-71coefficient vector ⃗ a is obtained, u(x) can be any location in the domain using:M a f (∥x -y ∥), x ∈ , y ∈ ˆ (5)Y. WANG AND Y. RUDY he auxiliary domain ˆ , which contains the as shown in Fig.1). As the fundamental fy Laplace's equation everywhere except at this representation satisfies Laplace's equamain . In addition, the specified boundary imposed at a set of boundary points (colloon the domain boundary . Since the funtions do not have singularities at points on , standard quadrature rules can be used to he surface potential and its normal gradient d on the boundary. 37 n the Appendix (Eqs. (a21) and (a22)), MFS d to discretize the Dirichlet and Neumann ditions in Eq. (1) as:

  1); ⃗ a and ⃗ b are vectors of dimensions M + 1 and 2N respectively.

  y j ∈ ˆ (6) Epicardial potentials are calculated using (6) on many epicardial nodes; numbers are provided for each dataset in the Results section. An epicardial potential map, reflecting the spatial distribution of potentials on the epicardial Solve the computed every millisecond during the cardiac cycle. The time series of reconstructed epicardial potential maps are then organized by location to provide temporal electrograms for any given point on the epicardium. A reconstructed epicardial electrogram provides the potential variation with time at a given point on the epicardium during the cardiac cycle. Epicardial isochrone maps (a map of the epicardial activation sequence) are computed by taking the time of maximum negative du E dt of the temporal electrogram ("intrinsic deflection") at a given location as the time of epicardial activation at that location.

  maps, L is the number of epicardial points at which potentials are measured and computed. V C i is Meshless Method for Inverse ECG 1275 surface, is computed every millisecond during the cardiac cycle. The time series of reconstructed epicardial potential maps are then organized by location to provide temporal electrograms for any given point on the epicardium. A reconstructed epicardial electrogram provides the potential variation with time at a given point on the epicardium during the cardiac cycle. Epicardial isochrone maps (a map of the epicardial activation sequence) are computed by taking the time of maximum negative du E dt of the temporal electrogram ("intrinsic deflection") at a given location as the time of epicardial activation at that location.

  maps, L is the number of epicardial points at which potentials are measured and computed.V C i isThe operator S rhs maps the BSP T to the residual part S rhs T , the solution of the following problem on ⌃ 8 ru) = 0, in ⌦ T , u = T, on ext , T ru.n = 0, on ⌃.

Fig. 3 .

 3 Fig. 3. Snapshots of the potential distribution in the normal case. Forward solution (left) and Inverse solution (right). The color bar scale is in mV.

Fig. 4 .

 4 Fig. 4. Snapshots of potential distribution for re-entry case. Forwa solution (left) and Inverse solution (right). The color bar scale is in mV

Fig. 6 .

 6 Fig. 6. Left (respectively, right): left (respectively, right) ventricle view the heart showing the l 2 relative error in time at each point of the geomet The color bar is dimensionless.

end for Cases metric MFS + CRESO O.C interpolated data O.C refined data Single and double stimulus (6 cases)

		RE	0.81±0.04	0.71±0.02	0.59±0.06
		CC	0.57±0.07	0.7±0.03	0.8±0.04
	Re-entry (VT)	RE	0.78±0.06	0.67±0.04	0.59±0.05
	(14 cases)	CC	0.6±0.08	0.73±0.04	0.83±0.04
		RE	0.79±0.06	0.69±0.04	0.59±0.05
	All 20 cases
		CC	0.59±0.07	0.72±0.04	0.82±0.04

⌦ T ⌦ H ext ⌃

re 1. Two-dimensional geometrical description: heart ain ⌦ H , torso domain ⌦ T (extramyocardial regions), t-torso interface ⌃ and torso external boundary ext .

This work was partially supported by an ANR grant part of "Investissements d'Avenir" program with reference ANR-10-IAHU-04. It is also supported by the LIRIMA international lab thought the EPICARD team FIMH'2017 @ Bordeaux


Florent Berthaut
email: florent@hitmuri.net

Victor Zappi
email: victor.zappi@qmul.ac.uk

Dario Mazzanti
email: dario.mazzanti@iit.it

Scenography of Immersive Virtual Musical Instruments

Keywords: Angled screens, Semi-transparent screen, Immersive virtual musical instruments, scenography, audience's experience, virtual reality

Immersive Virtual Musical Instruments (IVMIs) can be considered as the meeting between Music Technology and Virtual Reality. Being both musical instruments and elements of Virtual Environments, IVMIs require a transversal approach from their designers, in particular when the final aim is to play them in front of an audience, as part of a scenography. In this paper, we combine the main constraints of musical performances and Virtual Reality applications into a set of dimensions, meant to extensively describe IVMIs stage setups. A number of existing stage setups are then classified using these dimensions, explaining how they were used to showcase live virtual performances and discussing their scenographic level.

Index Terms:

• Human-centered computing~Virtual reality

• Human-centered computing~Scenario-based design

• Human-centered computing~Heuristic evaluations • Human-centered computing~HCI theory, concepts and models

INTRODUCTION

Context

In theatre, cinema and television, scenography relates to the study and the development of visual, experiential and spatial composition of performance. One of the main aims of this practice is to fully immerse the audience in the production, eliciting emotional and rational engagement [START_REF] Mckinney | The Cambridge Introduction to Scenography[END_REF].

This concept is surely familiar to New Media artists who have designed immersive Virtual Reality (VR) installations. Although technologically different, similar scenographic solutions are used in VR-based art to grant strong involvement of the audience, who is also the user of the interactive 3D application. However, when moving towards the context of VR performances, the concept of "user" and "audience" are no more overlapped. Performers are on stage, they interact with 3D objects inside a Virtual Environment (VE), in front of a separated audience. In this case, who is supposed to be the main target of the scenography? And is it possible to equally address both performers and spectators?

In these works, while the VE is designed to maximize performer's perception and control capabilities, spectators' virtual experience is not generally taken into account. As a result, often only performers are considered "users", as opposed to the spectators, who are barely confined outside of the VE. The concept of scenography is still present, but looses its audience-oriented connotation.

This issue is particularly important in the case of Immersive Virtual Musical Instruments (IVMI). These devices rely on the representation of sound processes and parameters as 3D entities in a VE and on fine 3D interaction techniques. In particular, their precise use always implies a certain level of immersion of the musicians, which can be granted by the use of a proper technological setup. However, the higher is the level of immersion provided by the stage setup (i.e., more refined instrument control), the more abstract and disconnected from the VE is likely to be the audience's experience.

The usage of a poor audience-oriented scenography may obviously also affect the spectators' strict musical experience, which combines sonic material and the perception of the musicianinstrument relationship. When not immersed in the VE, spectators lack important information about the instrument and performers' gestures, limiting musical engagement to only intellectual and cognitive levels [START_REF] Gurevich | Digital musical interactions: Performer-system relationships and their perception by spectators[END_REF]. This issue is common to most electronic instruments, but, in the case of IVMIs, it is not solely inherent to the nature of the instrument's metaphor, rather it can partly derive from the chosen technological setup.

So far, several examples of IVMIs have been presented in literature, some featuring metaphors relying on high levels of musician's immersion, like [START_REF] Valbom | Wave: Sound and music in an immersive environment[END_REF], [START_REF] Mäki-Patola | Experiments with virtual reality instruments[END_REF], or [START_REF] Berthaut | Drile : an immersive environment for hierarchical live-looping[END_REF], others, for their nature, more oriented towards a clear display of the musician-instrument relationship, for example [START_REF] Oliver | Fijuu2: a game-based audio-visual performance and composition engine[END_REF], [START_REF] Mulder | Design of virtual three-dimensional instruments for sound control[END_REF] or [START_REF] Zappi | Design and evaluation of a hybrid reality performance[END_REF]. Although they can all be used (some have been) for public performances, no studies have been done on how to setup a proper scenography for these instruments to engage the audience.

The aim of this work is to help instrument makers and performers, but also designers of VR applications in other fields, to create better scenographies for public exhibitions of their own systems.

Contribution

Our contribution is two-fold :

• First, we propose a set of dimensions to evaluate the stage setups of IVMIs and virtual musical performances, based on the analysis of the most common constraints in musical performances and VR.

• Then, we use these dimensions to describe, classify and compare some example stage setups, specifically evaluating features related to the concept of scenography.

CONSTRAINTS OF MUSICAL PERFORMANCES AND VIR-TUAL REALITY APPLICATIONS 2.1 Musical Performances

In this section, we define several constraints which need to be taken into account when designing a musical performance.

Understanding the Instrument

Differently from concerts performed with acoustic or electric instruments, electronic music concerts often feature new instruments, such as hardware controllers or laptops, which may distract or confuse the audience. Indeed, the sound of acoustic instruments is easy to guess from their categories, such as wind instruments, strings or percussions, and physical properties, such as size or material. Moreover, their sound and the mappings between performed gestures and sonic results are fixed and characterise the instrument. The behaviour of some non-acoustic instruments such as electric guitars or keyboards is less constrained but equally well known, mainly because they are used by several musicians.

On the other hand, new instruments are harder to understand for several reasons. First of all, they may use any sound synthesis technique and any mapping between musician's gestures and sound parameters, as described in [START_REF] Hunt | Mapping strategies for musical performance[END_REF] or [START_REF] Wanderley | Gestural control of sound synthesis[END_REF]. There is also no consistency between the sound produced by new instruments and their appearance, i.e. hardware controller shape or graphical user interface. Moreover, the behaviour of such instruments may change over time, even during a single performance. They are also less known, as they are used by fewer musicians, or even only their developer. Therefore, it is important to make sure that the audience can understand how the instrument is working. Graphical interfaces may ease this by displaying components, structures and behaviour of instruments, especially when multiple processes [START_REF] Jordà | Interactive music systems for everyone: exploring visual feedback as a way for creating more intuitive, efficient and learnable instruments[END_REF] are used. The behaviour of new instruments can be amplified for the benefit of the audience, using 3D graphical visualizations. For example, in Rouages [START_REF] Berthaut | Rouages: Revealing the Mechanisms of Digital Musical Instruments to the Audience[END_REF] 3D shapes are used to represent the components of new instruments and their sound generation, in order to enrich spectators' experience.

When designing 3D instruments, solutions such as immersion using large screens, stereoscopy and head-tracking may provide a better perception of the graphical components, and therefore a better understanding of the instrument.

Understanding Musician's Gestures

Besides understanding how an instrument works, the audience should also be able to perceive the relation between musician's gestures and their result. Such connection deeply affects the liveness of a musician's performance with an instrument, and consequently the audience's experience [START_REF] Marshall | Emotional response as a measure of liveness in new musical instrument performance[END_REF]. This can be even more crucial for instruments using several sound processes, as it may be hard to differentiate between processes controlled by musicians and autonomous processes. In performances with laptops for example, soundscapes tend to be rich and it can be almost impossible to perceive musicians gestures, as they consist of small hands and fingers movements, often hidden behind the screen. As suggested by Cascone [START_REF] Cascone | Laptop music -counterfeiting aura in the age of infinite reproduction[END_REF], the audience may feel frustrated under these circumstances, because they have no way to understand if musicians are really doing something or only reading their e-mails.

Musicians' gestures and their consequences are defined as manipulations and effects by Reeves et al. [START_REF] Reeves | Designing the spectator experience[END_REF]. They qualify public interfaces, such as new musical instruments, in relation to how much manipulations and effects are hidden from the audience. According to this taxonomy, expressive instruments reveal and even amplify both manipulations and effects, positively affecting audience's experience.

In the case of IVMIs, gestures and the instrument belong to two different worlds, the physical world and the VE. For this reason, simply showing the effects of musician's gesture on the instrument could be difficult. Some solutions should be considered to integrate the instrument in the physical space in a way that the VE blends perfectly with the real environment.

Communication Between Musicians and the Audience

Another important constraint is communication. Indeed, Bongers [START_REF] Bongers | Exploring novel ways of interaction in musical performance[END_REF] points out that musical performance involves non-verbal communication between musicians and the audience. It is fundamental that in one way, musicians can see reactions of audience, so that they can for example adapt their playing in the case of an improvisation, or simply so that they get better feeling of the ambiance. In the other way, spectators should be able to see, in addition to their gestures, expression and looks of musicians, especially in the context of a collective performance.

VR Constraints

In this section, we define what constraints influence the user experience in VR. In line with the aim of the study, first we consider a classic user-centered paradigm, then an audience-performer paradigm.

Immersion

Immersion is a key constraint in VR. The term refers to the description of the technology used to arouse in the user the psychological sense of being in the VE [START_REF] Slater | Immersion, presence, and performance in virtual environments: An experiment with tri-dimensional chess[END_REF]. This specific state of consciousness, experienced by the user and targeted in all immersive user-centered VR applications, has been defined as presence. In an optimal scenario, when a user feels "present" in a virtual world, he/she acts as if the environment were real, physically and emotionally engaged in the application.

Moving to a VR audience-performer scenario, immersion deeply affects both the audience's and the musician's experience. An immersive performance acts on the audience's feeling of presence within the VE used on stage. As a consequence, the virtual instrument and all its 3D graphical components can be perceived, to a certain extent, as "real". This aspect highly impacts the strength of virtual interaction, as seen from the eyes of the spectators; the gestures performed by the musician to play the instrument become more and more meaningful and communicative, as the instrument, from virtual, becomes real. On the other hand, when interaction takes place in a far and complete different environment (i.e. when the audience's feeling of presence is low or almost absent), performer's poses and gestures are perceived as obscure and even awkward [START_REF] Lanier | The sound of one hand[END_REF]. Moreover, even if the VE does not include interactive 3D objects, but it is capable of physically reaching and surrounding the audience, the emotional engagement of the spectators in the performance is sensibly enhanced [START_REF] Zappi | Design and evaluation of a hybrid reality performance[END_REF].

Likewise, from the musician's point of view, the perception of the VE and of the IVMI as real entities has remarkable consequences on the whole performance. In this case, immersion grants a more natural interaction with the instrument, resulting into a more expressive output, and influences control and confidence, which are strongly connected to the concept of virtuosity [START_REF] Jordà | Digital instruments and players part ii: Diversity, freedom and control[END_REF].

Reality-Virtuality Continuity

In immersive VR, presence is achieved through a combination of techniques and technologies, that are functional elements of the global system; some examples are multimodal feedbacks, detailed graphics, high framerate and large tracking area. As discussed in [START_REF] Cho | The dichotomy of presence elements: The where and what[END_REF], the effects of all these different elements are highly interconnected with one another, while the absence or the misuse of most of them produces breaks in presence [START_REF] Brogni | More breaks less presence[END_REF].

As discussed later in this work, most of the stage setups we will analyse are characterized by a combination of real and virtual objects interacting together (in first instance the virtual instrument and the musician). For this reason, it is worth focusing on the subset of immersive elements which grant a proper perceptive connection between reality and virtuality. We can call this subset Reality-Virtuality Continuity (RVC), which includes (but it is not limited to) tracking precision and real world noise filtering.

Although it can not assure a complete immersion, considering a classic user-centered paradigm the effect of RVC is twofold. First of all, from the user's perspective real physical objects integrated into the scene are visually aligned with the VE, also when the point of view changes. Moreover, all the unwanted stimuli (i.e. noise) coming from the real world surrounding the VR setup (e.g. people talking, touch or feel of VR equipment) are properly filtered, minimizing breaks of presence due to external reasons [START_REF] Slater | A virtual presence counter[END_REF].

According to an audience-performer paradigm, RVC has to be considered as an even stronger constraint, although it's fulfillment requires more complex operations. As discussed by Sanchez and Slater [START_REF] Sanchez-Vives | From presence to consciousness through virtual reality[END_REF], immersion is generally grounded on the possibility of interacting with the environment. But in most cases, spectators have not an active role in the performance; they do not interact with the scenography, they rather watch the musician playing the virtual instrument. In such an exocentric scenario, the total absence of interactive capabilities increases the effect of the perceived RVC onto the global level of immersivity of the VE on stage. In other words, action is replaced by a correct and robust perception of a real entity (i.e. the musician) interacting with virtual objects. As pointed out in [START_REF] Zappi | Design and evaluation of a hybrid reality performance[END_REF] this positively influences the audience's feeling of presence during VR performances.

From the performer's point of view, RVC is a fundamental constraint for the implementation of consistent interaction algorithms, necessary to make possible a natural usage of the virtual instrument.

Unfortunately, the immersive technologies used in virtual performances are generally incapable of preserving RVC from both sides. Often, the chosen setups penalize the musician's experience in favor of the audience's, or vice versa. These aspects and their dependencies with the scenography will be addressed in details in the next sections.

SCENOGRAPHY DIMENSIONS

In this section we define the six dimensions for the scenography of IVMIs and their impact on the constraints related to musical performances and VR that we defined in section 2.

Musician Immersion

Musician immersion relates to the immersion of the musician(s) in the VE which comprises the instrument, as defined in the previous section. It ranges from no immersion (e.g. monoscopic flat displays with fixed perspective) to complete immersion (e.g. flat stereoscopic displays with dynamic perspective or volumetric displays). It impacts mostly the interaction of the musician(s) with the instrument.

Audience Visibility

Audience visibility corresponds to the level of perception of the audience by the musician(s). In other words, it defines how well the musician(s) can perceive the audience. This dimension ranges from no perception at all to directly facing and seeing the physical audience. It impacts the communication between the musician(s) and the audience, as defined in section 2.1.3.

Audience immersion

Similarly to musician immersion, this dimension describes how well the audience perceives the VE and therefore the instrument. In addition to the values defined for the musician immersion, this dimension also includes conflicts between the musician and audience immersion, for example when they share the same screen but the perspective is computed for the musician. Therefore the fact of having a separate projection or screen improves the audience immersion. This dimension impacts mostly the understanding of the instrument by the audience.

Musician Visibility

Musician visibility corresponds to the perception of the musician(s) by the audience. It ranges from not seeing the musician at all to seeing him facing the audience. It impacts both the communication and the understanding of musicians' gestures by spectators.

Gestures Continuity

Gestures continuity relates to how the musical gestures performed by musicians in the physical space are connected to the graphic feedback of the instrument's metaphor, as perceived by the audience. It ranges from no continuity, for example when musician(s) can not be seen, to full continuity when the gestures are perfectly aligned with the components of the 3D interaction techniques. This dimension impacts the understanding of the instrument and of musicians' gestures. In turn, it depends on the audience immersion and on musician visibility.

From Virtual to Physical

The from virtual to physical dimension corresponds to how the virtual and physical spaces are merged. At one end, musician(s) and instrument are both rendered on a screen which acts as a window into the VE, completely detached from the real world. At the other end, the instrument is displayed as if it was in the physical space, ensuring perceptual consistency of the merged virtual and physical elements. Between these two ends, the instrument can be for example part of a VE contained in a virtual volume within the physical space. This dimension impacts the gestures continuity, which is easier to ensure if everything is integrated in the VE, and communication.

CLASSIFICATION OF EXISTING STAGE SETUPS

In this section we use the six IVMI scenography dimensions to analyse five different stage setups. In particular, we consider how these setups were used to stage 3 specific virtual performances; this allows for practical observations on scenography and possible improvements of each solution. A visual representation of the analysis is given on Figure 2 in the form a dimension space, that provides a quick overview of each setup's properties. In particular, the top part axes gives an indication on the level of communication. The right part axes show how well the audience perceive the instrument and musical gestures.

Head Mounted Display and Screen

We start this series of examples with one of the first and simplest stage setups ever designed to showcase a virtual performance in front of an audience. It consists of a Head Mounted Display (HMD) used in combination with a projected screen.

The most famous virtual performance capitalizing on this stage setup is probably The Sound of One Hand [START_REF] Lanier | The sound of one hand[END_REF], which took place in 1993. On stage, Jaron Lanier was in turn using different IVMIs sparse all over a VE. Sounds and notes were generated by hand movements, as they were transmitted to the instruments using a dataglove; this same device was used to move inside the VE using a point-fly technique. On stage, placed aside of the musician, a screen was used to display a 2D projection of his view point.

3D reconstruction

The primary dimension of this stage setup is musician immersion; the use of the HMD allowed for the musician to perceive a consistent world all around him and to have access to fine 3D controls. However, the use of the HMD totally prevented the audience to be seen by the musician, hence resulting into a total absence of audience visibility. Conversely, musician visibility is quite high: he played on the stage, right in front of the audience, but he was also free to move and rotate, yet partially hiding his gestures. Gestures continuity is almost absent, since the VE and the musician were perceived by the audience as two completely separated elements, the former projected onto a screen, the latter on the physical stage. For the same reason, the from virtual to physical dimension is strongly biased towards virtuality. Furthermore, the screen projection was 2D and it displayed the musician's point of view, resulting into an extremely low level of audience immersion.

The scenographic level of this setup is quite poor, since it mainly focuses on the musician and his virtual interaction. However, showing spectators a 3D projection aligned with the physical position of the musician on stage would remarkably enhance the audience's experience, providing immersion and increasing gestures continuity.

Single Screen

This setup is based on a single screen, positioned on stage behind the musician(s), as shown in happens in movie theatres.

The single screen setup was used to stage the Virtual Real performance [START_REF] Zappi | Design and evaluation of a hybrid reality performance[END_REF], in which an electronic musician played using both standard hardware controllers and non-immersive virtual instruments. A VE was displayed onto the screen, providing choreographic feedback through 3D visual. Virtual instruments were represented by 3D objects, which could be moved, touched or morphed by the performer in order to control audio effects. Such manipulation relied on optical motion capture of one of the musician's hands. This stage setup is strongly focused towards providing an intense audience experience. The stereoscopic imagery merging with the real stage strengthens both audience immersion and musician visibility. From virtual to physical dimension is moved towards the physical end, since the VE and its instruments are perceived by the audience as coherently superimposed with the physical stage. These features and the use of motion capture, allow musician's manipulation of the virtual instruments to be consistently aligned with the audience's point of view, generating a high gestures continuity. If we consider the performer's point of view, it is easy to see how musician immersion is absent, since the performer faces the audience and not the screen. On the other hand, since the musician directly sees the audience audience visibility is maximum.

Scenographies based on single screen setups can create strong involvement in the audience, capitalizing on virtual choreographies and non-verbal communication with the performers. However, such an extremely audience-centric setup makes impossible the usage of IMVIs based on complex 3D interactions. Slight setup modifications could generate a dual experience, in which the screen projection is dedicated to the performer, completely changing the scenographic outcome. The audience would no longer enjoy the perfect virtual/real environments consistency, while musicians would be immersed in the instrument, allowing for fine audio control. In this example, two screens are juxtaposed facing different directions, so as to completely enclose a part of the stage. One screen is facing the performer, sideways, the other one is angled, dedicated to the audience (Figure 5). In this way both audience and performers perceive the VE as "contained" inside of this virtual volume.

Angled Screens

Musician

The IVMI analysed using this particular setup is Drile [START_REF] Berthaut | Drile : an immersive environment for hierarchical live-looping[END_REF]. The instrument itself relies on the manipulation of 3D audiovisual objects that are associated with the nodes of live-looping trees. Handheld devices with pressure sensors are used to reach these musical objects by means of a virtual ray technique and to excite/modulate them, creating loops. Since only remote virtual interaction was implied, the setup could be successfully used to enclose the whole instrument within the stage area in between the two screens, as depicted on Figure 1. Both the projections were monoscopic.

The audience immersion is relatively low, since, although they were assigned a dedicated view on the VE, their projection was monoscopic. Similarly, the musician immersion is low due to the lack of stereoscopic projection and head-tracking. The audience visibility and musician visibility are quite high, since musician and spectators directly saw each other from the side. The gestures continuity is medium low. The virtual ray indicates what the musician was manipulating but the musician operated on the instrument standing one or more meters away from the screen, breaking the continuity between his hands and the virtual rays. For the virtual to physical dimension, this setup was at half-way, because the instrument was in a virtual volume enclosed in the physical space.

This setup and its scenographic level could be vastly improved by simply introducing stereoscopic projections for both the musician and the audience. However, as for the previous setup, only a single sweet spot (defined as the center of the seat area) would be shared by spectators; however, since the stage and the seat area are generally some meters far from each other, this would not dramatically affect the audience's experience. Furthermore, propagating the virtual ray between the musician and the virtual volume through, for example, projection mapping would specifically increase the scenographic value of this stage setup.

Semi-transparent Screen

This setup relies on the Pepper's ghost effect, obtained using an acrylic transparent screen. This screen is placed between the musician and the audience and rotated of 45 degrees on the horizontal axis, the lower part being on the audience side. Above and below it two projection screens are set. The projection screen placed above is reflected on the audience side of the semi-transparent screen while the one below is reflected on the musician side. Therefore a different rendering can be displayed on each side without interference, and the screen remains see-through. We will use another performance featuring the Drile instrument to analyse the setup. As depicted on Figure 1, during the performance, the two renderings on the opposite sides of the tilted screen showed the same 3D audiovisual objects that composed Drile, but from two opposite perspectives (one from the front, the other from the back). Doing so, the instrument appeared to be placed right in between the musician and the audience. The rest of the instrument's equipment remained the same.

The audience visibility and musician visibility are maximized since they face each other through the semi-transparent screen. The audience immersion is high. The VE was rendered through a stereoscopic passive projector. However, again only one sweet spot was shared among the audience. The musician immersion is very high since the projection on his side was also stereoscopic and since his head was tracked to adjust the rendering to his point of view. The gestures continuity is medium/high. The semi-transparency of the setup allowed the audience to perceive continuity between musicians gestures and virtual rays; nonetheless, whenever the tilted screen is not completely flat, distortion can occur and affect the perceived alignment. This setup was at the physical end of the virtual to physical dimension, the instrument being completely integrated in the physical space thanks to the Pepper's ghost effect.

Scenographies based on this setup can be very effective, also preserving the performers' experience. It is important to notice that the complexity of the equipment used may in turn introduce other issues, more related to the technical preparation of the stage (e.g. distortions due to an imprecise framing of the tilted screen).

3D Reconstruction

In this specific setup, the stage is divided into two parts, one visible to the audience and another one completely concealed (the backstage). As depicted in Figure 7, musicians perform exclusively in the backstage and are recorded by one or more cameras; the visible part of the stage hosts only a screen, where both the VE and avatars of the musicians are rendered.

Recently, again Drile has been performed using this stage setup. The instrument was prepared in the backstage and included all the equipment necessary for it to be played. Additionally, a depth camera was used for a real-time 3D reconstruction of the musician. This avatar was then displayed on the stage screen, facing the audience. Doing so, the VE was perceived by the spectators as if positioned between them and the performer.

Since the setup included stereoscopic equipment for both the IMVI and the visible stage, both musician immersion and audience immersion are particularly high. Both VE and musician were rendered on the screen, so the stage was totally virtual (from virtual to physical) and this allowed for the perception of high ges- tures continuity. On the other hand, the separation between stage and backstage determined a complete absence of audience visibility for the musicians and a limited level of musician visibility, since bounded to the resolution of the used camera.

The scenography built on this setup is strong and it does not limit the musician's freedom of action in any ways. However, it notably lacks many important characteristics typical of live performances, resembling more a recorded performance. The add of a dual system to reconstruct the spectators could be used to display them back to the musician, closing the visual communication loop.

CONCLUSION

In this paper we presented a set of dimensions that allow for the description of the scenography of performances with immersive virtual musical instruments. They rely on the analysis of the constraints of musical performances and virtual reality setups. We show that these dimensions might serve as a classification tool for existing performances but also as a design tool to help improve existing stage setups and build new ones. We hope that other researchers, instrument makers and musicians will rely on them for their own performances and help us refining them. Finally, we think that these dimensions could be adapted to non-musical immersive performances such as dance or theater.

Figure 2 :

 2 Figure 2: Dimension Spaces for the analysed stage setups.

Figure 4 .Figure 3 :Figure 4 :

 434 Figure 3: Scheme for the single screen stage setup. The musician wears an hmd and the audience sees his point of view on the screen placed on the stage.

Figure 5 :

 5 Figure 5: Scheme for the angled screen stage setup, based on the juxtaposition of two screens.

Figure 6 :

 6 Figure 6: Scheme for the stage setup based on the semi-transparent screen. People at both sides can perceive 3D projections without any interference.

Figure 7 :

 7 Figure 7: Scheme for the stage setup based on 3D reconstruction of the performer. musician and audience are in two separate part of the stage.