Caroline Charlier

Sylvain Poirée

Christophe Delavaud

Gaby Khoury

Clémence Richaud

Alexandre Leclercq

Olivier Hélénon

Study Marc Lecuit
email: marc.lecuit@pasteur.fr

Imaging of human neurolisteriosis: a prospective study of 71 cases

Keywords: Listeria monocytogenes, neurolisteriosis, magnetic resonance imaging, computed tomography scan

INTRODUCTION

Listeria monocytogenes (Lm) is a foodborne pathogen responsible for listeriosis, a severe systemic infection affecting pregnant women, immuno-suppressed individuals and the elderly. Three main clinical forms are described: maternal-neonatal infections, septicaemia and neurolisteriosis.

Neurolisteriosis typically presents as a meningoencephalitis (evidenced in 84% of cases) or less commonly as a meningitis without encephalitis (13% of cases) [START_REF] Mylonakis | Central nervous system infection with Listeria monocytogenes. 33 years' experience at a general hospital and review of 776 episodes from the literature[END_REF][START_REF] Koopmans | Listeria monocytogenes sequence type 6 and increased rate of unfavorable outcome in meningitis: epidemiologic cohort study[END_REF][START_REF] Charlier | Clinical features and prognostic factors of listeriosis: the MONALISA national prospective cohort study[END_REF]. In some Western countries, Lm is one of the leading etiological agents of meningo-encephalitis in adults, together with Herpes simplex virus, Varicella zoster virus and Mycobacterium tuberculosis [START_REF] Silk | Invasive listeriosis in the Foodborne Diseases Active Surveillance Network (FoodNet), 2004-2009: further targeted prevention needed for higher-risk groups[END_REF] [START_REF] Mailles | Infectious encephalitis in france in 2007: a national prospective study[END_REF]; and is associated with the highest morbi-mortality, despite appropriate antimicrobial therapy. Three-months mortality is around 30%, with persisting neurological impairment reported in 44% of surviving patients [START_REF] Charlier | Clinical features and prognostic factors of listeriosis: the MONALISA national prospective cohort study[END_REF]. Most neurolisteriosis cases are therefore managed in intensive care management (60%), with 33% requiring mechanical ventilation [START_REF] Charlier | Clinical features and prognostic factors of listeriosis: the MONALISA national prospective cohort study[END_REF]. The radiological presentation of neurolisteriosis remains poorly characterized and only reported as scattered observations focusing on brain abscesses or rhombencephalitis [START_REF] Cone | Multiple cerebral abscesses because of Listeria monocytogenes: three case reports and a literature review of supratentorial listerial brain abscess(es)[END_REF][START_REF] Suzuki | Magnetic resonance images in rhombencephalitis due to Listeria monocytogenes mimicking Wernicke encephalopathy[END_REF][START_REF] Ito | Listeria monocytogenes meningoencephalitis presenting with hydrocephalus and ventriculitis[END_REF] [START_REF] Armstrong | Brainstem encephalitis (rhombencephalitis) due to Listeria monocytogenes: case report and review[END_REF]. Yet, careful evaluation of central nervous system imaging in patients with neurolisteriosis is critical for a better understanding of the pathophysiology of this deadly infection, and to help refine clinical care.

MONALISA is a national prospective cohort study of invasive listeriosis implemented in France since 2009, compiling all cases of invasive listeriosis reported through national mandatory reporting, whose exhaustiveness was recently estimated around 87% [START_REF] De Valk | Changes in epidemiology and surveillance of listeriosis in France[END_REF]. For all included cases, a large dataset of clinical features and clinical isolates were collected [START_REF] Charlier | Clinical features and prognostic factors of listeriosis: the MONALISA national prospective cohort study[END_REF]. The clinical results of this study were recently published, and new prognostic factors for neurolisteriosis were evidenced: concomitant positive blood cultures, the use of adjunctive dexamethasone or mechanical ventilation requirement were independently associated with higher 3-month mortality; encephalitis signs and the number of neurological symptoms were also associated with a higher-risk of persisting neurological impairment [START_REF] Charlier | Clinical features and prognostic factors of listeriosis: the MONALISA national prospective cohort study[END_REF]. Radiological data have not been analyzed in this clinical study. No correlation between radiological and clinical data has ever been reported, and the impact of radiological abnormal images has not been studied to our knowledge. We took advantage of this collection of neuroradiological imaging data of patients with neurolisteriosis to characterize the radiological features of neurolisteriosis and determine their prognostic value. Radiological evaluation -As cases were scattered all over France, neuroimaging was performed according to local hospital practices. Brain computed tomography (CT) and magnetic resonance (MR) images performed from baseline were sent by the clinicians along with clinical data. They analyzed by two neuroradiologists highly experienced in neuroimaging of infections. Images were analyzed according to a pre-established checklist. There was no abnormal image evidenced that was not listed in the checklist. Two independent analyses were performed; in case of discrepancy, a third was performed with both radiologists to reach consensus.

METHODS

Definitions

-Meningeal enhancement was defined after injection of gadolinium chelates on T1-weighted images on MR or after injection of iodine contrast medium on helicoidal CT scan. Meningeal enhancement was classified as leptomeningeal (involvement of pia mater or subarachnoid spaces) or pachymeningeal (involvement of dura mater).

-Brain abscesses were defined as nodules or masses with peripheral enhancement after injection after injection of gadolinium chelates on T1-weighted images on MR, or of iodine contrast media on helicoidal CT scan. When available, hyper intensity on MR diffusion weighting images was also taken into account to confirm the diagnosis of brain abscess.

-Nodules evocative of abscess were defined as enhanced nodules after injection of contrast media, but lacking typical peripheral enhancement.

-Diffuse cerebral edema was defined as diffuse hyper intense signal of the cerebral white matter with effacement of the sulci on T2-weighted or Flair MRI sequences or as diffuse cerebral white matter hypodensity with effacement of the sulci on helicoidal CT scan.

-Non-specific white matter images were defined as non-specific hyper intense signal of the white matter on T2-weighted MRI sequences or as non-specific white matter hypodensities on helicoidal CT scan.

-Cerebral atrophy was defined on both MRI and CT scan as a parenchymal volume loss with enlarged cerebral sulci (cortical atrophy) and/or ventriculomegaly without bulging of the third ventricular recesses (central atrophy).

-Dilated Virchow Robin spaces were defined as linear fluid signal non-enhancing spaces located in the basal ganglia or midbrain.

-Cerebral herniation was defined as a shift of cerebral tissue from its normal location into an adjacent space.

-Contrast-enhancing ventricles were defined as ventricles with enhancement of the ependymal lining after injection of gadolinium chelates on T1-weighted images on MRI or after injection of iodine contrast media on helicoidal CT scan.

-Hydrocephalus was defined as an enlargement of the ventricles without cerebral atrophy.

-Radiological vasculitis was defined as enhanced cortical grey matter after injection of gadolinium chelates on T1-weighted images on MRI or after injection of iodine contrast media on helicoidal CT scan, possibly associated with hemorrhages or ischemic lesions.

-Hemorrhage was defined as a lesion with hyper intense signal on T1-weighted sequences or marked low signal on GRET2* weighted sequences on MRI, or as a spontaneously hyper dense lesion (not calcified) on helicoidal CT scan.

-Ischemic images were defined as focal lesions of the cortical or subcortical matter with increased diffusion weight imaging (DWI) signal and reduced apparent diffusion coefficient (ADC) values on MRI or hypodensity after injection of iodine contrast media on helicoidal CT scan.

-Clinical encephalitis was defined by the presence of at least one of the following symptoms, with no alternative cause than listeriosis identified: altered consciousness (score on Glasgow Coma Scale (GCS) <15), seizures, new onset of neurological symptoms and abnormality on electroencephalography [START_REF] Charlier | Clinical features and prognostic factors of listeriosis: the MONALISA national prospective cohort study[END_REF][START_REF] Venkatesan | Case definitions, diagnostic algorithms, and priorities in encephalitis: consensus statement of the international encephalitis consortium[END_REF].

Statistical analysis -The sample size was a convenient sample, determined by the number of available images during the study period. All tests were performed with R software (version 3.4.0). All tests were 2-tailed and p-values <0.05 (calculated by Chi 2 test, Fisher exact test or Student's t test) were considered significant. For the multivariable analysis of mortality, candidate variables were the variables showing association at a significance level of p < 0.10 in the univariable analysis; the three clinical variables previously shown associated most strongly associated with mortality in neurolisteriosis were included in the multivariable model, and a stepwise selection was done [START_REF] Charlier | Clinical features and prognostic factors of listeriosis: the MONALISA national prospective cohort study[END_REF]. Inter-rater agreement has been quantified by Cohen's kappa statistics. Disagreement was considered if one radiologist described one lesion as present when the other one described it as absent or not interpretable. The situation where one radiologist answered that a lesion was absent when the other one described it as uninterpretable was not considered as a disagreement. Confidence intervals were computed using the bootstrap method, i.e., numerical resampling. Agreement was not computed for following signs owing to their low prevalence (< 3%, i.e. less than 3 patients, for at least one of the radiologists): brain herniation, brain edema, radiological vasculitis and ventricular involvement. Observer agreement was categorized by kappa values as poor (<0.39), moderate (0.40-0.59), good (0.60-0.79) or excellent (>0.80) [START_REF] Coblentz | Observer variation in detecting the radiologic features associated with bronchiolitis[END_REF].

RESULTS

Characteristics of the study population -The patients' characteristics are described in Table 1. Fifty nine percent were men (42/71). Sixty patients (85%) reported encephalitis signs including clinical brainstem symptoms in 23 percent (16/71). Patients with neuroimaging available were similar to those of the French neurolisteriosis cohort in terms of age, gender, number and frequency of comorbidities, GCS, rate of encephalitis and focal signs, cerebrospinal fluid features, proportion of isolates with virulent clonal complexes, death and persisting impairment rates (p >0.1, data not shown) [START_REF] Charlier | Clinical features and prognostic factors of listeriosis: the MONALISA national prospective cohort study[END_REF].

Neuroradiological presentation at baseline -The first radiological procedure was performed after a median time interval of 2 days from the time of the diagnosis procedure (blood or cerebrospinal fluid sample) (interquartile range 0-11 days). The neuroradiological features are detailed in Table 2.

Altogether, abnormal neuroradiological findings were found in 62 of the 71 patients (83%). Meningeal enhancement was reported in 25/71 (35%). It consisted in pachy-or lepto-meningeal enhancement, focal (n=2) or diffuse (n=23). Twelve patients had meningeal enhancement evidenced before lumbar puncture was performed, excluding the possibility of non-specific dural and/or arachnoid enhancement, which can be observed after lumbar puncture [START_REF] Mittl | Frequency of unexplained meningeal enhancement in the brain after lumbar puncture[END_REF].

Abscesses and/or nodular images evocative of abscesses (see Methods) were reported in 10/71 cases (14%). They were unique or multiple (1 to six, median number of 2), with diameter ranging from 7 to 18 mm. They were infra-tentorial (brainstem) in 5/10 cases (50%), supra-tentorial in 4/10 cases (40%) and both supra-and infra-tentorial in one case (1/10, 10%).

Hemorrhages were reported in an unexpected high proportion of cases (11/71, 15%). They involved parenchyma in all cases and were classified as supratentorial in 7 cases, subtentorial in 1, and both supratentorial and subtentorial in 3 cases. They consisted in petechial images (7/11) or hematoma (4/11).

Parenchymal hemorrhages could also be associated with subarachnoid hemorrhage (n=2) or ventricular hemorrhage (n=1). In 7/11 cases concomitant predisposing conditions were evidenced: hypertension (n=3), pre-existing tumoral (n=2) or vascular (n=1) lesions, anticoagulation and thrombocytopenia (< 20,000/mm 3 ) (n=1, each). Importantly, major ventricular involvement (contrast-enhancing ventricles or hydrocephalus) were evidenced in 7 cases (7/71, 10%). Non-specific deep white-matter images were also frequently reported (42/71, 59%). In all but 2 cases, they could be attributed to age (> 50 years, n=39), and/or concomitant hypertension (n=20), or HIV infection (n=1).

All combinations of radiological signs were observed (data not shown). Altogether, brainstem involvement with meningeal enhancement or abscess(es) was reported in only 7/71 cases (10%). No involvement of choroid plexuses was found. MRI detected more meningeal enhancement, dilated Virchow Robin spaces and hydrocephalus than CT scan (p< 0.05, Table 2).

Correlation with clinical and microbiological features -Clinical and radiological data were compared (Figure S1). No correlation between radiological presentation and the number of immunosuppressive comorbidities was found, except for vascular images (ischemia or hemorrhage) that were more frequently reported in patients reporting above 2 immunosuppressing comorbidities (12/19) that in others (8/46, p=0.0002). All patients radiologically diagnosed with hemorrhages, ventricles involvement (contrast-enhancement or hydrocephalus), cerebral herniation, cerebral edema or vasculitis also had clinical signs of encephalitis involvement as defined above. However, there was no correlation between clinical signs and neuroradiological findings. Indeed, only 4/7 (57%) patients with radiological evidence of brainstem involvement exhibited brainstem symptoms (namely cranial nerve involvement) at the time of evaluation. Furthermore, 12/16 (75%) of patients exhibiting clinical brainstem signs did not have brainstem neuroradiological lesions on MRI (n=10) or CT scan (n=2) (Figure S1A). Similarly, 6/25 (24%) patients with meningeal enhancement did not report nuchal rigidity (Figure S1B), although all of them (25/25) had alterations of the cerebrospinal fluid (defined as cerebrospinal fluid nucleated cells count >4/mm 3 and/or CSF protein >0.5g/L and/or evidence of viable L. monocytogenes in the cerebrospinal fluid). A patient with brain abscess/nodule did not report clinical signs of encephalitis (1/10). Finally, no correlation was evidenced between isolate genotype (either hyper or hypovirulent, as reported in Maury et al [START_REF] Cone | Multiple cerebral abscesses because of Listeria monocytogenes: three case reports and a literature review of supratentorial listerial brain abscess(es)[END_REF]) and radiological presentation (data not shown).

Relationship between baseline brain images and outcome -Patients with neurolisteriosis and

ventricle involvement (hydrocephalus or contrast-enhancement) and those with parenchymal images (abscess(es), nodule(s) and/ or non-specific white matter images) evidenced significantly lower 3-month survival in the univariate analysis (respectively, 1/7 (14%) in patients with ventricle involvement versus 47/64 (73%, p=0.005) in those without and 25/46 (54%) in patients with parenchymal images versus 23/25 (92%) in those without, p=0.004) (Table S1). Parenchymal lesions (abscess(es), nodule(s) and/ or non-specific white matter lesions) remained associated with poor 3-month survival for patients in the multivariable model (Table 3). No link between radiological pattern and long-term impairment was evidenced (data not shown).

Radiological evolution -Of the 71 patients, 9 had another neuroimaging procedure performed after at least a 1-week interval, including 6 with the same type of imaging (MRI or CT scan). Images were found stable in 4 cases (including one patient with 18 mm abscesses reevaluated after 10 days, one patient with parenchymal hematoma reevaluated after 5 weeks and one patient with persisting meningeal enhancement reevaluated after 9 months), or improved in one case with brain hematoma (reevaluation after 4 and 6 weeks). One single case developed focal parietal meningitis after 7 days in the context of altered consciousness despite appropriate therapy, he died 1 week later as a consequence of severe neurolisteriosis.

DISCUSSION

The French mandatory reporting made it possible for the first time to collect a large set of radiological files from patients with neurolisteriosis, and to analyze them together with a large dataset of clinical data [START_REF] Charlier | Clinical features and prognostic factors of listeriosis: the MONALISA national prospective cohort study[END_REF]. Only culture-proven cases were included. To our knowledge, previous studies have only been focused on specific images like abscesses or rhombencephalitis, while other signs were only reported as scattered observations [START_REF] Armstrong | Brainstem encephalitis (rhombencephalitis) due to Listeria monocytogenes: case report and review[END_REF][START_REF] Cone | Multiple cerebral abscesses because of Listeria monocytogenes: three case reports and a literature review of supratentorial listerial brain abscess(es)[END_REF]. This systematic evaluation of neuroradiological features allowed an unbiased description of the neuroradiological features of neurolisteriosis. We could thereby determine that abnormal images were observed in 83% of cases, corresponding to meningeal enhancement, abscess(es) and nodular images (s) evocative of abscess, hemorrhages, ventricular involvement (contrast-enhancing ventricles or hydrocephalus), non-specific white matter images and dilated Virchow-Robin spaces, and rarely to diffuse cerebral edema and cerebral herniation.

Abscesses, meningeal enhancement, hydrocephalus, ischemia and brain edema have been reported before as scattered observations [START_REF] Alper | MR findings in listerial rhombencephalitis[END_REF][START_REF] Uldry | Early symptoms and outcome of Listeria monocytogenes rhombencephalitis: 14 adult cases[END_REF][START_REF] Coley | CT and MRI manifestations of central nervous system infection following allogeneic bone marrow transplantation[END_REF][START_REF] Marrie | Computed tomographic scanning in Listeria monocytogenes meningitis[END_REF][START_REF] Brouwer | Community-acquired Listeria monocytogenes meningitis in adults[END_REF]; however, a systematic analysis of radiological files allowed us to determine the relative frequency of each radiological sign, and to point out the rarity of brain abscesses and of brainstem involvement; although they were the cardinal features reported in the neurolisteriosis literature so far [START_REF] Kohler | Listeria brainstem encephalitis: two own cases and literature review[END_REF][START_REF] Moragas | Rhombencephalitis: a series of 97 patients[END_REF][START_REF] Tan | Brainstem encephalitis: etiologies, treatment, and predictors of outcome[END_REF]. Typical abscesses have indeed been reported in numerous case reports, as highlighted by the retrospective work by Cone and coll., who compiled 40 cases of Listeriarelated brain abscesses published from 1966 to 2003 [START_REF] Cone | Multiple cerebral abscesses because of Listeria monocytogenes: three case reports and a literature review of supratentorial listerial brain abscess(es)[END_REF]; they were however evidenced in only 6% of our case series after systematic evaluation, suggesting a bias in previous publications. Similarly, the brainstem radiological involvement called "rhombencephalitis" which is usually considered very suggestive of neurolisteriosis [23][9] was only reported in 10% of cases; this low sensitivity mirrors clinical data, as brainstem involvement is only reported in 17% of cases [START_REF] Charlier | Clinical features and prognostic factors of listeriosis: the MONALISA national prospective cohort study[END_REF]. For the first time, we evidenced the unexpected frequency of hemorrhages and ventricular involvement, that, to our knowledge, had been reported only in scattered observations so far [START_REF] Coley | CT and MRI manifestations of central nervous system infection following allogeneic bone marrow transplantation[END_REF][START_REF] Goday | Transient immunologic defect in a case of Listeria rhombencephalitis[END_REF]. Haemorrhages could occur as a consequence of associated conditions in 64% of cases; they could also reflect a Listeria endothelial tropism, although Listeria monocytogenes might invade a large array of host cells, including endothelial cells, either directly or via infected monocytes [START_REF] Wilson | Listeria monocytogenes infection and activation of human brain microvascular endothelial cells[END_REF][START_REF] Drevets | Dissemination of Listeria monocytogenes by infected phagocytes[END_REF]. The pathophysiological mechanisms underlying ventricular involvement (contrast-enhancement and/or hydrocephalus) remain to be determined, especially whether this reflects an invasion site form infected CSF, or an extension from parenchymal infection [START_REF] Kirk | Diagnostic ultrastructure of Listeria monocytogenes in human central nervous tissue[END_REF].

Altogether, these results highlight the non-specific neuroradiological presentation of neurolisteriosis, which is not associated with any pathognomonic radiological sign nor any specific combination of signs, and can affect any brain compartment (parenchyma, blood vessels, meninges and ventricles). Clinicians should be aware that the absence of radiological rhombencephalitis, brain abscess or meningeal-enhancement does not rule out neurolisteriosis and should not be taken into account for empiric anti-Listeria therapy when neurolisteriosis is considered. Diagnosis is indeed often delayed until the results of blood and cerebrospinal fluid cultures or qPCR, as clinical signs are nonspecific and direct examination of the cerebrospinal fluid evidences Gram positive rods in only 32% of cases [START_REF] Charlier | Clinical features and prognostic factors of listeriosis: the MONALISA national prospective cohort study[END_REF]. Some neuroradiological features were associated with poorer 3-month survival in neurolisteriosis in univariate analysis: ventricular involvement with hydrocephalus or contrastenhancing ventricles and parenchymal imaging combining abscess(es), and/or nodule(s) and/or white matter images; and multivariable analysis (parenchymal images defined above). Although we cannot exclude overfitting of the logistic model, because of the limited sample size, these images should be added to the other parameters shown associated with poor outcome, namely ongoing cancer, female gender, recent major weight loss, multi-organ failure, aggravation of any pre-existing organ dysfunction, mechanical ventilation, monocytopenia, positive blood culture and adjunctive dexamethasone therapy [START_REF] Charlier | Clinical features and prognostic factors of listeriosis: the MONALISA national prospective cohort study[END_REF][START_REF] Gerner-Smidt | Invasive listeriosis in Denmark 1994-2003: a review of 299 cases with special emphasis on risk factors for mortality[END_REF]. Neuroimaging is recommended in encephalitis as a part of the diagnosis procedure [START_REF] Tunkel | The management of encephalitis: clinical practice guidelines by the Infectious Diseases Society of America[END_REF]. It should be performed for all patients with confirmed neurolisteriosis, as part of the global assessment of patient's severity. Furthermore, some lesions might require specific management, like ischemic and hemorrhagic signs, and hydrocephalus or cerebral herniation requiring neurosurgical management. Because all patients with such abnormal images do not all exhibit matching clinical symptoms, neuroimaging should also be performed the 14% of patients with neurolisteriosis who do not exhibit clinical signs of encephalitis. In line with other neurological infections, MRI should be preferred to CT scan, as it displays a greater sensitivity (Table 2) [START_REF] Anslow | Cranial bacterial infection[END_REF][START_REF] Charlier | Cryptococcal neuroradiological lesions correlate with severity during cryptococcal meningoencephalitis in HIV-positive patients in the HAART era[END_REF][START_REF] Midiri | Nocardial adrenal abscess: CT and MR findings[END_REF].

The high inter-rater agreement underlines the reliability of radiological evaluation and strengthen this recommendation. However, our study has limitations. First, we did not set any harmonized neuroimaging protocol in the setting of this observational study, and the dataset reflected these heterogeneous local practices. Also, the sample size hampered further statistical analyses. We could not collect radiological files for all patients included in the MONALISA cohort, and selection bias could not be ruled out, although the study cohort appeared representative of the global picture of neurolisteriosis in France, without evidence for any distinctive clinical feature or increased severity.

CONCLUSIONS

In conclusion, this study shows that neurolisteriosis presents as a combination of neuroradiological images, none of them being specific. Signs of rhombencephalitis are uncommon, whereas, unexpectedly, hemorrhagic signs are frequent. This study identified parenchymal neuroradiological images associated with a negative prognostic value that will help improve the management of this severe infection. * Data presented are those from exam 1 and 2. When one patient had benefited from 2 exams with the same procedure (computed tomography of magnetic resonance), data from the first exam were retained for analysis. Patients could have images evidenced in both procedures therefore total numbers are not the sum of MRI plus CT scan numbers. ✝ Brain abscesses and nodules were localized in the frontal lobe, temporal lobe (n=1, each), thalami (n=2), cerebellum (n=1), brainstem (n=4), or brainstem and occipital lobe (n=1). ‡ Atrophy was classified as diffuse in 27/33 (82%) (cortical in 6, subcortical in 7 and both cortical and subcortical in 14) and as focal in 6/33 (18%). § Cerebral herniation was classified as uncal or subfalcine (n=1, each). ¶ Radiological vasculitis was defined as enhanced cortical grey matter after injection of gadolinium chelates on T1-weighted images on MR or after infection of iodine contrast media on helicoidal CT scan, possibly associated with haemorrhages or ischemic lesions. ** Ischemic images were considered as sequellar lesions in 3/8 cases, and recent in 5/8. In the 5 images reflecting recent lesions, they could be related to the radiological vasculitis in 2/5 cases, altogether they were subtentorial in 3/5 and supratentorial in 2/5 cases. ✝✝ Tumor images included meningioma (n=2) and histologically-confirmed malignant brain tumor (n=3).

  Study population -Neurolisteriosis cases were microbiologically confirmed and defined as follow: isolation of Listeria monocytogenes from the cerebrospinal fluid (CSF) or brain abscess, isolation of Listeria monocytogenes from blood cultures in patients with otherwise unexplained neurological symptoms (altered consciousness, seizures, nuchal rigidity, or focal neurological symptoms), isolation of Listeria monocytogenes from blood cultures and in CSF by qPCR. They were included from Nov 2009 to Oct 2013 from the MONALISA national prospective observational cohort study, whose design has been reported [3]. All patients or their legal representatives when unable to consent provided written informed consent. In accordance with French legislation, the study received institutional review board approval by the local ethics committee (Comité de Protection des Personnes Ile De France III, Nov-6 2009). Clinical and biological data were available at baseline and at 3-month follow-up. Radiological files were collected from clinicians. The study is registered at Clinical Trials (NCT01520597).

Figure 1 .

 1 Figure 1. Main patterns of radiological neurolisteriosis in magnetic resonance imaging (MRI) procedures. A. Axial contrast-enhancing T1-weighted MRI with a ring-like enhancing pons image corresponding to an abscess (arrow head). B. Axial contrast-enhancing T1-weighted MRI with a ringenhancing image of the right parietal lobe corresponding to an abscess (arrow head) with diffuse meningeal enhancement. C. Axial T2 gradient-echo MRI with markedly hypo signal in right cerebellar peduncle corresponding to focal bleeding. D. Axial diffusion weighted MRI with bilateral supratentorial cerebral dot-like hyper signals corresponding to ischemic images highly evocating of radiological vasculitis. E. Axial T2 spin-echo MRI with symmetric hyper signals of supra-tentorial white matter. F. Axial contrast-enhancing T1-weighted MRI with hydrocephalus and ventricular ependymal lining enhancement.

Table 2 .

 2 Neuroradiological features of 71 patients with neurolisteriosis and neuroimaging

			Lesions
		Number of patients	Number of patients	Number of patients	p value
	Neuroradiological finding	with radiological lesions *	with lesions on MR *	with lesions on CT *	CT vs MR
			N=46	N=30
		N=71
	Meninges
	Lepto and/or pachy-meningeal	25/71 (35)	20/46 (43)	6/30 (20) p=0.048	0.048
	enhancement
	Parenchyma
	Brain abscess ✝	4/71 (6)	3/46 (7)	1/30 (3)	NS
	Nodule evocative of abscess ✝	7//71 (10)	7/46 (15)	1/30 (3)	NS
	Non-specific white matter lesion	42/71 (59)	30/46 (65)	14/30 (47)	NS
	Atrophy ‡	34/71 (48)	24/46 (52)	12/30 (40)	NS
	Dilated Virchow-Robin spaces	22/71 (31)	20/46 (43)	6/30 (20)	0.048
	Cerebral herniation §	2/71 (2)	2/46 (4)	1/30 (3)	NS
	Diffuse cerebral edema	1/71 (1)	1/46 (2)	-	NS
	Ventricles
	Contrast-enhancing ventricles	2/71 (3)	2/46 (4)	1/30 (3)	NS
	Hydrocephalus	6/71 (9)	1/46 (2)	6/30 (20)	0.01
	Brain vessels
	Radiological vasculitis ¶	3/71 (5)	3/46 (7)	1/30 (3)	NS
	Hemorrhage	11/71 (15)	10/46 (22)	1/30 (3)	0.04
	Ischemia **	8/71 (11)	8/46 (17)	1/30 (3)	NS
	Concomitant tumor image ✝✝	5/71 (7)	4/46 (8)	2/30 (6)	NS
	Normal	9/71 (13)	2/46 (4)	7/30 (23)	0.02

Table 3 .

 3 Independent predictors for mortality in 71 adults with neurolisteriosis in multivariate analysis.

	Parameter

Odds ratio calculated from the multivariate model. 95% CI: 95% confidence interval.

Acknowledgements: We thank Raphaël Porcher for inter-rater agreement analyses.

Funding: Institut Pasteur, Inserm, Santé Publique France, Programme Hospitalier de Recherche Clinique, Swiss National Science Foundation.

Conflicts of interest: none.


Xavier De Tinguy
email: xavier.de-tinguy@irisa.fr

Claudio Pacchierotti
email: claudio.pacchierotti@irisa.fr

Mathieu Emily

Mathilde Chevalier

Aurélie Guignardat

Morgan Guillaudeux

Chloé Six

Anatole Lécuyer
email: anatole.lecuyer@irisa.fr

Maud Marchal
email: maud.marchal@irisa.fr

How different tangible and virtual objects can be

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

I. INTRODUCTION

Tangible objects are used in Virtual and Augmented Reality to convey the haptic sensation of touching virtual objects. However, for the illusion to work, the haptic characteristics of the tangible objects should match as much as possible those of the corresponding virtual ones in terms of, e.g., size, local shape, texture, mass. In other words, there should be a good correspondence between what users see in the virtual environment and what they touch in the real world. An easy way to achieve this important visuo-haptic matching is to create tangible replicas of all the virtual objects in the scene. For example, Billinghurst et al. [START_REF] Billinghurst | The magicbook-moving seamlessly between reality and virtuality[END_REF] presented an Augmented Reality (AR) tangible book. Users can turn its pages, look at the pictures, and read the text as in a normal book. However, if they use an AR display, they can also see 3-dimensional virtual models popping out of the pages. More recently, Harley et al. [START_REF] Harley | Tangible vr: Diegetic tangible objects for virtual reality narratives[END_REF] presented a system for diegetic tangible objects in Virtual Reality (VR). They developed four tangible objects prototypes, including a cube, a stuffed animal, a treasure chest, and a wooden boat, providing passive and active haptics. Fig. 1: Objective of our study: understanding how different a tangible object (left) can be from virtual objects (right) without the user noticing the mismatch. We focused our study on three specific criteria: width, local orientation, and curvature.

Altough compelling, creating a tangible replica of all the virtual scene may neither be feasible, e.g., if the virtual objects change dynamically during the experience, nor desirable, e.g., if the virtual environment comprises several objects. For this reason, researchers have recently started to develop solutions enabling the use of few tangible objects to render multiple virtual ones, trying to minimize the haptic mismatch. In this respect, Hettiarachchi and Wigdor [START_REF] Hettiarachchi | Annexing reality: Enabling opportunistic use of everyday objects as tangible proxies in augmented reality[END_REF] introduced a wearable system able to scan the user's surroundings and find a good match between the available virtual and tangible objects. McLelland et al. [START_REF] Mcclelland | Haptobend: Shapechanging passive haptic feedback in virtual reality[END_REF] introduced a reconfigurable device able to change its shape to match the targeted virtual object. It is composed of four rectangular rigid sections with hinged connections. The inFORM [START_REF] Follmer | inform: dynamic physical affordances and constraints through shape and object actuation[END_REF] system enables experimentation with shape-changing interfaces and dynamic physical affordances. It is made of 30×30 white polystyrene pins, actuated by push-pull rods that can extend the pins up to 10 cm from the surface. A projector placed on top of the system can create moving images on the white pins surface and a Kinect depth camera tracks the position of the user's hand. A similar approach has been also presented in [START_REF] Zhao | Robotic assembly of haptic proxy objects for tangible interaction and virtual reality[END_REF], where small robots assemble into different shapes. Sait et al. [START_REF] Sait | Physical hand interaction for controlling multiple virtual objects in virtual reality[END_REF], [START_REF] Han | Evaluating remapped physical reach for hand interactions with passive haptics in virtual reality[END_REF] mapped a single tangible object into multiple virtual ones by combining a redirection approach, that uses rotational adjustments to align the user during virtual locomotion, and a resetting approach, that introduces a discrete rotational update when the user virtually approaches a target for interaction. Redirection approaches have also been used in [START_REF] Azmandian | Haptic retargeting: Dynamic repurposing of passive haptics for enhanced virtual reality experiences[END_REF], [START_REF] Kohli | Redirected touching: Training and adaptation in warped virtual spaces[END_REF].

Although there exists substantial evidence that a mismatch between virtual and physical objects severely affects the user's illusion of presence [START_REF] Simeone | Substitutional reality: Using the physical environment to design virtual reality experiences[END_REF], [START_REF] Hershberger | Touch dominates haptic estimates of discordant visual-haptic size[END_REF], [START_REF] Lacey | Visuo-haptic multisensory object recognition, categorization, and representation[END_REF], no one has yet quantitatively studied the extent of this difference in applications of virtual and augmented reality. A first attempt has been carried out by Kwon et al. [START_REF] Kwon | Effects of sizes and shapes of props in tangible augmented reality[END_REF], who explored the effects of size and shape differences between tangible and virtual objects for interaction usability in AR. Results showed that manipulation realism increases as the tangible and virtual objects become more similar, but no quantitative measure on the minimum noticeable difference was provided. Similarly, Ban et al. [START_REF] Ban | Magicpot360: Free viewpoint shape display modifying the perception of shape[END_REF] studied the effect of shape difference between tangible and virtual objects in VR. Results show that it is partially possible to alter the shape of the virtual object w.r.t. its tangible counterpart without the user noticing. However, the authors did not provide any quantitative measure of this shape difference. Simeone et al. [START_REF] Simeone | Substitutional reality: Using the physical environment to design virtual reality experiences[END_REF], [START_REF] Simeone | Substitutional reality: Towards a research agenda[END_REF] introduced the concept of Substituional Reality, where every physical object is paired, with some degree of discrepancy, to a contextually-appropriate virtual object. By changing the degree of discrepancy between tangible and virtual objects, the authors highlight several critical mismatches able to break the immersion. For example, results show that tangible objects presenting similar affordances in parts most likely to be interacted with are the best candidates for substitution. Along this line of research, de Tinguy et al. [START_REF] De Tinguy | Toward Universal Tangible Objects: Optimizing Haptic Pinching Sensations in 3D Interaction[END_REF] also proposed an algorithm to find similar pinch grasping poses between a set of tangible and virtual objects.

The effect of visuohaptic discrepancy has been also studied from a purely perceptual point of view. For example, Hershberger and Misceo [START_REF] Hershberger | Touch dominates haptic estimates of discordant visual-haptic size[END_REF] found that, between vision and haptic information, neither modality inherently dominates the perceived size of an object. Discordant haptic information biases visual size estimates by as much as discordant visual information biases haptic size estimates. Since neither modality captures the other one completely, a discordant stimulus will be perceived as discordant. Ernst and Banks [START_REF] Ernst | Humans integrate visual and haptic information in a statistically optimal fashion[END_REF] quantitatively examined visuohaptic integration to determine whether human performance follows a maximum-likelihood estimate. This rule states that the optimal means of estimation is to sum the sensor estimates weighted by their normalized reciprocal variances. Subjects looked at and/or felt a raised ridge and were asked to judge its height. Results showed that height judgements were indeed very similar to those predicted with a maximum-likelihood estimation. Lacey and Sathian [START_REF] Lacey | Visuo-haptic multisensory object recognition, categorization, and representation[END_REF] reviewed how the similarities in humans visual and haptic unisensory object processing contribute to an integrated multisensory visuohaptic processing of object in terms of categorization, recognition, and representation. For example, a change in size produces a cost in visual recognition for both unfamiliar and familiar objects. However, neither grasping an object tighter nor enlarging the spread of the fingers leads humans to perceive a change in size [START_REF] Berryman | Representation of object size in the somatosensory system[END_REF]. This paper directly addresses the question posed in the title. Our objective is to understand how different a virtual object can be from its tangible counterpart without the user noticing (see Fig. 1). This question is important for all those working in the field of immersive environment, and its answer can open interesting avenues for the use of few tangible objects in the rendering of multiple virtual ones. Of course, the visuohaptic perception of objects encompasses several different dimensions, including the object's size, shape, mass, texture, and temperature. In this work, we started by addressing three representative haptic features -width, local orientation, and curvature, -which are particularly relevant for grasping. We carried out three human subjects experiments, one for each criterion, that we will describe in the next section. A video summarizing our work is available at https://youtu.be/xREuZbh6tLc (7.6Mo, .avi).

II. METHODS

The goal of our user studies is to measure the Just-Noticeable Difference (JND) of the discrepancy between a couple of tangible and virtual objects during 2-finger grasping, in terms of the three above-mentioned haptic dimensions: width, local orientation, and curvature.

A. Experimental setup

Fig. 2 shows the setup. Participants wear an HTC Vive headset displaying the virtual scene. A Bonita Vicon system tracks the subjects' thumb and index fingertips using markers placed on the dorsal side of their fingers (avoiding the nails). Doing so, the subjects finger pads are always left free to interact with the tangible object (TO). Two virtual fingertips mimic the motion of the subjects fingertips in the virtual environment. This simple representation of the user's hand [START_REF] Argelaguet | The role of interaction in virtual embodiment: Effects of the virtual hand representation[END_REF] has been chosen to avoid occluding the virtual object from the user's point of view (see inset of Fig. 2). The virtual scene is composed of an instruction panel and a table, supporting the considered virtual object (VO). The position of the VO in the virtual scene matches the position of the TO in the real environment. In other words, whenever users grasp the TO, they also grasp the VO. We ensure a good matching between the positions of the TO and VO by securing the TO on a 3D-printed structure, which is placed on a table in front of the user. Moreover, at the beginning of each experiment, we calibrate the system to also ensure a good matching between the subjects fingertips and their virtual avatars.

B. Experimental task and procedure

Participants are asked to grasp the TO at a designated pinch location, highlighted by two green cursors, while seeing the VO through the headset (see Fig. 2). At every new grasping trial, the system induces a discrepancy between the TO and VO by altering the considered criterion, i.e., it creates a mismatch of width, local orientation, or curvature where the subject grasped. We use only one tangible object per experiment, while several different virtual objects are shown to the participants (see Fig. 3). Whenever participants arrive at the highlighted pinch location, the object turns red and a "hold" message is displayed for 2 s on the virtual panel in front of the user. After that, the object turns green, and the panel asks the user to "release" the object. Right after releasing the object, a question appears on the panel, asking the participants to compare their perception of the considered criteria between the TO and the VO. Finally, the experimenter fakes the changing of the TO, to prevent participants from understanding that only one TO was used throughout the whole experiment. This procedure can be seen in the attached video.

Before the beginning of an experiment, we explain the procedure to the participant and we spend about three minutes adjusting the setup to be comfortable. Then, the participant spend about two minutes practicing interacting with two couples of TO and VO, different than those used during the real trials. Fig. 3: The tangible objects used for the three user studies: a cube, a trapezoid prism, and an ellipsoid. Participants are asked to grasp at the center of each shape, which is 8 cm from the table (dotted line).

III. USER STUDY #1: WIDTH A. Procedure Description

The objective of this first user study is to investigate how much the width of a virtual object can differ from the one of a tangible one without the user noticing. We use a cube as the reference shape (left object in Fig. 3), as it is a wellknow shape which is easy to recognize from any point of view. Participants are asked to grasp the virtual object and its tangible counterpart, comparing their width and answering the question: "Is the tangible object larger than the virtual one?"

As mentioned in Sec. II-A, we ensure that whenever the users touch the TO, they also touch the VO. To achieve this result, when the width of the VO and TO differs, we employ a simple virtual warping effect to (slightly) redirect the virtual fingers [START_REF] Kohli | Redirected touching: Training and adaptation in warped virtual spaces[END_REF].

B. Experimental Design and Participants

We consider the cube width as the independent variable, with a reference value for the tangible cube of w ref = 4.0 cm (see Fig. 3) and 9 comparison widths for the virtual cubes = 2.4, 3.2, 3.6, 3.8, 4.0, 4.2, 4.4, 4.8, 5.6 cm (see Fig. 4). We compute the comparison widths w ref (1 + ∆) with ∆ = {0%, ±5%, ±10%, ±20%, ±40%}. We chose this range of comparison widths so that the covered range contains JND values already registered in the literature [START_REF] Ernst | Humans integrate visual and haptic information in a statistically optimal fashion[END_REF]. Participants compared each couple of objects 7 times, yielding to 7×9=63 comparisons per participant. The experiment lasted around 10 minutes.

We enrolled 17 participants (9 males, 8 females, M = 21.59, SD = 1.58), all of whom were right-handed.

C. Results

First, we compute the percentage of answers in which the tangible object felt larger than the virtual object. As expected, as the width of the VO increases, the less often the participants feel the TO larger. Then, using Weber's law, we compute the Weber Fraction as k = ∆I/I, where ∆I refers to the JND threshold and I is the reference width. The JND threshold can be determined as the value of the stimuli in which the recognition ratio is 75%. To compute this value, we fit the psychometric curve f (x) = (1 + e (αx+β) ) to the data (with α = 1.12 and β = -0.4546), as shown in Fig. 5. The 75% JND is -0.23 cm, or 5.75% of the reference width. The corresponding Weber fraction is k = ∆I/I = 0.0575. Furthermore, the Point of Subjective Equality (PSE) is 0.16 cm. Fig. 5: User study #1: width. Psychometric curve fitting the average percentage of answers (grey points) in which participants considered the tangible object as larger than the virtual one. The 75% JND is represented in blue while the PSE is in red.

IV. USER STUDY #2: LOCAL ORIENTATION A. Procedure Description

The objective of this second user study is to investigate how much the local orientation of a virtual object's face can differ from the one of a tangible object without the user noticing. We use a trapezoid prism as the reference shape (center object in Fig. 3). Participants are again asked to grasp the virtual object and its tangible counterpart. Then, they are asked to compare the local orientation at the grasping point, answering to the question: "Are the faces of the tangible object more tilted than those of the virtual one?"

B. Experimental Design and Participants

We consider the local orientation of the prism faces as the independent variable, with a reference angle for the tangible prism of a ref = 10 • (see Fig. 3) and 7 comparison angles for the virtual prism = 2 • , 6 • , 8 • , 10 • , 12 • , 16 • , 18 • (see Fig. 6). We compute the comparison angles a ref (1 + ∆) with ∆ = {0%, ±20%, ±40%, ±80%}. At the pinch location, all virtual prisms have a width of 4 cm. Participants compared each couple of objects 7 times, yielding to 7×7=49 comparisons for each participant. The experiment lasted around 10 minutes.

We enrolled 17 participants (10 males, 7 females, M = 21.59, SD = 1.58), all of whom were right-handed.

C. Results

First, we compute the percentage of answers in which the tangible object felt more tilted than the virtual object. As before, as the tilting angle of the VO increases, the less often the participants feel the TO as more tilted. Then, we compute the Weber Fraction using the 75% JND. To compute the latter, we again fit the psychometric curve f (x) = (1 + e (αx+β) ) to the data (with α = 0.26 and β = -0.022), as shown in Fig. 7. The 75% JND is -4.38 • , or 43.8% of the reference orientation angle. The corresponding Weber fraction is k = 0.438. Furthermore, the PSE is 0.09 • .

V. USER STUDY #3: LOCAL CURVATURE

A. Procedure Description

The objective of this third and last user study is to investigate how much the local curvature of a virtual object's faces can differ from the one of a tangible object without the user noticing. We use an ellipsoid as our reference shape (right object in Fig. 3). Participants are again asked to grasp the virtual object and its tangible counterpart. Then, they are asked to compare the curvature at the grasping point, answering to the question: "Are the faces of the tangible object more curved than those of the virtual one?" Fig. 7: User study #2: local orientation. Psychometric curve fitting the average percentage of answers (grey points) in which participants considered the tangible object as more tilted than the virtual one. The 75% JND is represented in blue while the PSE is in red.

B. Experimental Design and Participants

The size of the tangible ellipsoid is 4 × 4 × 5 cm, with an estimated curvature at the grasping point of 52 m -1 in the horizontal plan and 33 m -1 in the vertical plan. We consider the curvature of the ellipsoid in the vertical plan as the independent variable, with a reference curvature for the tangible ellipsoid of c ref = 33 m -1 (see Fig. 3) and 9 comparison curvatures for the virtual ellipsoid = 12, 19, 24, 28, 33, 39, 47, 72, 269 m -1 (see Fig. 8). We compute the comparison curvatures as c ref (1 + ∆) with ∆ = {-63.6%, -42.4%, -27.3%, -15.2%, 0%, +18.2%, +42.4%, +118.2%, +715.2%} At the pinch location, all virtual ellipsoids have a width of 4 cm. Participants compared each couple of objects 7 times, yielding to 7×9=63 comparisons for each participant. The experiment lasted around 10 minutes.

We enrolled 15 participants (7 males, 8 females, M = 21.4, SD = 1.40), all of whom were right-handed.

C. Results

First, we compute the percentage of answers in which the tangible object felt more curved than the virtual object. Then, we compute the Weber Fraction using the 75% JND. To compute the latter, we fit the psychometric curve f (x) = (1 + e (αx+β) ) to the data (with α = 0.061 and β = -1.77), a shown in Fig. 9. The 75% JND is 22 m -1 , or 66.6% of the reference tilting angle. The corresponding Weber fraction is k = 0.666. Furthermore, the PSE is 29.04 m -1 . Fig. 9: User study #3: curvature. Psychometric curve fitting the average percentage of answers (grey points) in which participants considered the tangible object as more curved than the virtual one. The 75% JND is represented in blue while the PSE is in red. For clarity, the ∆ = 715.2% value is not represented.

VI. SUBJECTIVE QUESTIONNAIRE

In addition to the three user studies, the participants also fill a subjective questionnaire using a 7-point Likert scale. We asked the following questions: (Q1) "Was it easy to feel the difference of width between the tangible and virtual cubes?"; (Q2) "Was it easy to feel the difference of tilting between the tangible and virtual prisms?"; (Q3) "Was it easy to feel the difference of curvature between the tangible and virtual ellipsoids?"; (Q4) "Did it feel like you were seeing your own fingertips?"; (Q5) "Did you feel tired at the end of the experiment?".

Results show that feeling the difference in width and local orientation was quite easy (Q1: M = 4, SD = 1.57; Q2: M = 3.63, SD = 1.21). However, it was quite difficult to feel the difference in local curvature (Q3: M = 2.84, SD = 1.46). The corresponding barplots are reported in Fig. 10. The matching between the virtual and the real fingertips appears to be well perceived (Q4: M = 5.11, SD = 1.52). At the end of the experiment, participants felt a bit tired (Q5: M = 4.05, SD = 1.68).

VII. DISCUSSION

The objective of this paper is to quantify, in a VR scenario, how similar tangible and virtual objects need to be to feel the same. In other words, we wanted to measure how different virtual and tangible objects can be without the user noticing.

Toward this objective, we carried out three human subject experiments. Results showed a 75% JND of 5.75%, 43.80%, and 66.66% for the perception of width, local orientation, and local curvature during grasping, respectively around references values of 4 cm, 10 • , and 33 m -1 . Those values were selected as central values in the range of realistic values for pinch grasping.

Although no one directly addressed this question for VR, these results are consistent to similar perceptual results in the literature. Ernst and Banks [START_REF] Ernst | Humans integrate visual and haptic information in a statistically optimal fashion[END_REF] found a 84% JND of 0.04 times the average ridge height (55 mm) when providing both visual stimuli. On the other hand, the haptic-only JND was 0.085 times the average height. In our case, the 84% JND is 10.75% of the cube size (40mm). This difference might be explained by the fact that our participants could see the cube in 3D. Moreover, in this paper we are studying a discrepancy-related JND, which poses a different questions w.r.t. the one considered in [START_REF] Ernst | Humans integrate visual and haptic information in a statistically optimal fashion[END_REF]. Subjects in Ban et al. [START_REF] Ban | Magicpot360: Free viewpoint shape display modifying the perception of shape[END_REF] were asked to explore a tangible vertical surface with their index finger while seeing a slanted virtual surface on the screen (70 • ). The ratings of a questionnaire suggest that the two objects (tangible and virtual) felt almost the same. Simeone et al. [START_REF] Simeone | Substitutional reality: Using the physical environment to design virtual reality experiences[END_REF], [START_REF] Simeone | Substitutional reality: Towards a research agenda[END_REF] found that affordance and function are the most important tracts when rendering virtual objects through tangible props. For this reason, they suggest to focus on maximizing the shape similarity around areas the user is likely to interact with (e.g., a handle). Unfortunately, they did not report any quantitative data on how to do this. Finally, subjects in Kwon et al. [START_REF] Kwon | Effects of sizes and shapes of props in tangible augmented reality[END_REF] took 65% more time to grasp a tangible object when its virtual representation significantly differed. In our experiments, we used the tangible object as reference. It would also be interesting to use the virtual object as reference and change the properties of tangible ones, to observe whether there are any biases. However, as we simulate the change of the tangible object for each repetition of the experiment, we believe that the bias was minimum. Our results open new interesting avenues for the use of tangible objects different from their virtual counterpart. The psychometric curves reported in Secs. III, IV, and V enable researchers to understand when it is acceptable for a tangible object to approximate the haptic features of virtual ones. Doing so, one tangible object can be used to render multiple virtual ones without the user noticing.

From these results, we can also improve our previous work [START_REF] De Tinguy | Toward Universal Tangible Objects: Optimizing Haptic Pinching Sensations in 3D Interaction[END_REF], where we proposed an algorithm to find, rate, and extract the best pairs of pinching poses between tangible and virtual objects. Along the same line of thought, we could analyze the haptic features of a virtual environment and then, knowing that it is possible to have some discrepancy, automatically generate one or more universal tangible objects, able to provide the best possible sensations in the rendering of the virtual scene. Alternatively, one could also change the virtual scene to improve the expected visuohaptic matching and, therefore, the illusion of presence (e.g., remove certain types of surfaces which are difficult to render with the available tangible objects). It may also enables to predict the illusion of presence and take action if deemed insufficient (e.g., by increasing the number of tangible objects in use).

Although these results are interesting, they are just the beginning of a long line of research. The haptic perception of an object does not only comprise size, local orientation, and curvature. It is therefore important to extend this study to other important haptic features, such as texture, mass, and temperature. More reference values would also be needed to better determine the Weber's fraction, and it would be interesting to do so for each feature as well as for different grasping poses. We also did not take into account that humans have fingertips of different size and elasticity, which can significantly affect how they perceive a surface. Another limitation is that we only considered 2-fingers grasping. We may find different results for other types of interaction. This work also does not directly address any possible confusion due to inherent tracking issue such as occlusions or calibration residual offsets. This issue may be solved by considering different tracking techniques [START_REF] Chinello | A three revolute-revolute-spherical wearable fingertip cutaneous device for stiffness rendering[END_REF] or additional tracking error compensation methods as well as more complex haptic proxy approaches to compensate such case. We will address these points in future work. Finally, we also found that some participants had serious difficulties in recognizing even very large discrepancies. We plan to study this phenomenon in depth, to ensure a minimum guaranteed level of performance for all subjects, even the less perceptive ones.

VIII. CONCLUSIONS

Tangible objects are often used in VR and AR to provide distributed haptic sensations of touching virtual objects. However, it is often not possible to create tangible replicas of all the virtual objects in the scene and one would want to re-use some of those as much as possible, nor is it always possible to create an exact replicate. Thus, it is to be expected that there would be some discrepancy between the tangible objects and the virtual ones. This paper wants to directly address this issue, by measuring how different a virtual object can be from its tangible counterpart without the user noticing. Therefore, we tried to quantify this noticeable perceptual difference between virtual objects and their tangible counterpart during a 2 fingers pinching, by considering the perception of width, local orientation, and local curvature. Although some papers have addressed this problem from a qualitative point of view, to the best of our knowledge, this paper represents the first effort to quantify this noticeable perceptual difference between virtual and tangible objects in virtual reality.

Our experiments show that, while pinching, it was possible to induce discrepancy between what the user feel of the tangible object and what he sees from the virtual one by 5.75%, 43.8%, and 66.66% respectively for the width, local orientation and local curvature. Those results suggest that it is indeed possible, to a certain extent, to match different virtual and tangible objects without the user noticing.

Next steps would be to induce discrepancy on other parameters, such as inertia, texture, roughness, etc. as well as extending the results to other kind of grasping. These efforts would definitely open novel opportunities for using a reduced number of tangible objects to render multiple virtual ones in VR.

Fig. 2 :

 2 Fig. 2: Setup for the three experiments. Inset shows the virtual environment during the task.

Fig. 4 :

 4 Fig. 4: User study #1: width. The virtual cubes having variable widths w ref (1+∆), ∆ = {0%, ±5%, ±10%, ±20%, ±40%} are compared with the reference cube (w ref = 4.0 cm), shown in red. In the real environment, participants always grasped the tangible cube shown on the left of Fig. 3.

Fig. 6 :

 6 Fig. 6: User study #2: local orientation. The virtual prisms having variable faces orientations a ref (1 + ∆), ∆ = {0%, ±20%, ±40%, ±80%} are compared with the reference prism (a ref = 10 • ), shown in red. In the real environment, subjects always grasp the tangible prism shown at the middle of Fig. 3.

Fig. 8 :

 8 Fig. 8: User study #3: curvature. The virtual ellipsoids having variable faces curvatures c ref (1 + ∆), ∆ = {-63.6%, -42.4%, -27.3%, -15.2%, 0%, +18.2%, +42.4%, +118.2%, +715.2%} are compared with the reference ellipsoid (c ref = 33 m -1 ), shown in red. In the real environment, participants always grasp the tangible ellipsoid shown in the right of Fig.3.

Fig. 10 :

 10 Fig. 10: Barplots (7-point Likert scale) representing the participants answers to the questions on the easiness of perception of the difference between TO and VO for the three criteria: Q1 (width), Q2 (local orientation), Q3 (curvature).

*This research has received funding from the EU's H2020 research and innovation programme (grant agreement No 801413, project "H-Reality").


Charlotte Laffargue

Iwan Le Berre

Colette Ranély Vergé-Dépré

Roméo Terral

Roméo Terral The

Pointe Jarry Ter- Minal :

The Pointe Jarry terminal : A Port Trajectory

niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Background source : Plan de l'Entrée et des mouillages de la Pointe-à-Pitre, levé 1868-69, éd. 1955, corr. 1969 (Shom) (Shom) Interna�onal

A Port Trajectory

The Geohistorical analysis of Port Guadeloupe aims to reconstruct the main stages of its development and its main drivers. This poster shows the genesis and evolu�on of Jarry's container terminal. The complete containerisa�on of traffic between the French West Indies and the metropolis, led to the development of a terminal equipped with two container quay Cranes of 30 tons.

T h e J a r r y p o r t t e r m i n a l i n 2 0 0 0 ( P A G )

3

C o n t a i n e r i s a �o n ( f r o m t h e 1 9 8 0 s)

In search of land, the port migrates to Jarry Point, where an industrial-port area is developed.

A sugar wharf and a tanker pier are built as early as 1967.

Be gi nn ing of Reclama�on of the Po in te Ja rr y (1 9 6 7 , F o n d s P i e r r e t , A r c h i v e s m u n i c i p a l e s d e P A P ) 2 E x u r b a n i s a � on ( l a t e 1 9 6 0 s )

1

T h e P o r t i n t h e Ci t y

A�er 1946 and the departmentaliza�on of Guadeloupe, the port of Pointe-à-Pitre extends to the North and is equipped with new wharves. But it becomes too narrow and obsolete due to increased traffic in the 1960s.

( c a 1 9 5 0 , A r c h i v e s D é p a r t e m e n t a l e s 9 7 1 )

( 1 9 2 8 -1 9 6 5 )

  a�er the expansion of the Panama Canal, the Jarry terminal needs to be expanded. The project involves a storage pla�orm of 25 ha, 350 m of deep water docks, to accommodate vessels with a dra� of 15 m. T o a T r a n s h i p m e n t Hu b ( a ft e r 2 0 2 0 ) Ext ension proj ec t of th e c o n t a in e r t e r m i n a l f r o m 2 0 2 0 ( G u a d e l o u p e P o r t C a r a ï b e )

  à Pointe Jarry Status of Port autonome (PAG) Beginning of the containerisation The PAG changes Its status to become Grand port maritime de Guadeloupe Inauguration of the new locks of the Panama Canal Delivery of two over-panamax container cranes to Jarry Creation of the International Free Trade Zone managed by the CCI and the PAG Several events and trends had influenced the evolu�on of the Port. Is the extension of Jarry terminal a founding event ? It is not yet clear, as other drivers remain to be studied: e.g. port-city and port-local authori�es rela�onships, the evolu�on of port employment, companies network, and shipowning strategies (see TRAFIC project).


Antoine Petit

Nazim Haouchine

Frédérick Roy

Dan B Goldman

Stéphane Cotin

Frédérick Roy

Stéphane Cotin Deformed

Frederick Roy

Stephane Cotin

la diffusion de documents scientifiques de niveau

recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The considerable technical advances in augmented reality (AR) made it possible for users to have different sorts of interactions such as object grasping, rigid transform or surface deformation [START_REF] Chun | Real-time Hand Interaction for Augmented Reality on Mobile Phones[END_REF]. However, most of the time, a 3D mesh is superimposed using computer-generated texturing, making the final output unrealistic. In the context of video and image editing, recent methods using 3D stock models to replace the actual object projection in the image have been proposed [START_REF] Kholgade | 3D Object Manipulation in a Single Photograph Using Stock 3D Models[END_REF] [ZCC *12] [CZS*13] [HRC*18]. These methods give the possibility of editing objects in a scene in 3D and extending the range of manipulation. However, they are † stephane.cotin@inria.fr essentially dedicated to photo editing or assume no relative motion between the object and the camera when dealing with videos. This makes it difficult to be used for AR purpose where object pose has to be estimated over frames. the whole output realistic. Moreover, we makes it easy for users to realize complex edits during AR with little manual intervention and interactive feedback, and is based on two main contributions:

• A simultaneous simulation-and-tracking pipeline capable of transferring a finite element simulation onto a video while preserving the spatial and visual coherence of the original video. • Allowing for a wide range of edits, including rigid transforms, elastic deformations, and internal parameter updates such as mass, stiffness or gravity.

Related Works

Manipulating objects during AR involves very similar algorithms as those used in image and videos editing. This domain has gained interest from the research community with the prominence of commercial tools and the extended use of smart-phones and hand-held cameras.

One of the most active area of research is editing human faces which has received a wide attention with the Face2Face method [TZN*15; TZS*16]. A tool for photorealistic edits of facial expressions in videos in real time, based on the low-dimensional parametric face model has been proposed by Blanz and Vetter [START_REF] Blanz | A morphable model for the synthesis of 3D faces[END_REF].

Human shapes have been addressed for images [ZFL*10] and videos [START_REF] Jain | Moviereshape: Tracking and reshaping of humans in videos[END_REF], enabling quick and easy manipulation of human bodies in 2D content based on an underlying 3D morphable model. Bai et al. [START_REF] Bai | Selectively de-animating video[END_REF] focus on temporal edits by selectively de-animating videos. The idea is to compute a warp field constrained by a set of manually-provided strokes that removes the large-scale motion in these regions while leaving finer-scale, relative motions intact. objects can be moved in time while keeping their original position, giving the user the ability to manipulate the video timeline.

Most of these previous works focused on photographs and rigid object editing, while the few ones dealing with actual videos assume static scenes where no relative motion between the object and the camera is considered. Deformed Reality overcomes these assumptions and permits manipulating objects in a video stream, in the presence of camera (or object) motion. This interactive editing can be performed in parallel of camera pose estimation and 3D model transfer onto the augmented view.

Overview

Our computational pipeline is separated into three simultaneous processes, illustrated in Figure 2: (1) given a video and a 3D model, we estimate the object pose by tracking in a rigid manner the object over frames, (2) we compute the deformation based on the user input using an underlying physical model and (3) we back-project the 3D edited object onto an inpainted frame following a continuous completion and re-texturing process. The 3D CAD proxy model, used as input, is provided by the user and can be retrieved from various databases.

Model-based Rigid Pose Estimation

The issue of estimating the complete rigid 3D pose of the camera, with respect to the considered object along the image stream (cf Figure 3), is addressed based on the real-time 3D model-based tracking approach presented in [PMK13] [START_REF] Petit | Combining complementary edge, keypoint and color features in modelbased tracking for highly dynamic scenes[END_REF]. The method consists in combining, within a deterministic optimization process, geometrical information provided by tracked edge features, with a dense color information through object / background color separation, to efficiently handle 3D models of any shape and complexity.

A local non-linear minimization problem

Given a 3D model of the object, the goal is to estimate, frame-byframe, the rigid camera pose r, which transforms the object frame into the camera frame, by minimizing the function ∆(r) accounting for errors e i (r) between a set of visual features extracted from the image and the forward projection of their 3D homologous in the image plane: Figure 2: Deformed Reality pipeline: using as inputs a video and a 3D proxy CAD corresponding to an object in the scene, the user starts by aligning the 3D model onto the image. The initial pose is then estimated along with the object's edges. These edges will be used to compute a rigid object pose over frames. Simultaneously, a deformable model is built from the 3D CAD surface mesh to enable non-rigid object manipulations. The deformed mesh is textured and projected onto an inpainted image using the estimated pose to produce the final composition.

∆(r) = ∑ i ρ(e i (r)) (1)
where ρ is a robust estimator, which reduces the sensitivity to outliers. This is a non-linear minimization problem with respect to the pose parameters r, and we follow a Gauss-Newton minimization framework to handle it. At each iteration k in the process, a displacement is performed in the parameter space which is SE(3):

r k+1 = r k ⊕ δ r (2)
where ⊕ is an internal compositional law in the parameter space (SE(3)). The displacement δ P is computed through:

δ r = -λ (DJ) + De (3) with J = ∂ e(r) ∂ r (4)
and where (DJ) + is the pseudo inverse of DJ, J being, as written above, the Jacobian matrix of the error vector e(r) with respect to the pose. λ is a proportional gain and D is a weighting matrix associated to the Tukey robust estimator. Based on equation (2), the pose r, represented by its homogeneous matrix c M o , can be updated as follows, using the exponential map [MSKS04]:

c M k+1 o = exp([δ r]) c M k o . (5)
where exp([δ r]) is the exponential map enabling to express the rigid motion δ M generated by δ r at iteration k.

Visual cues

In order to benefit from the complementarity of different visual features and to overcome the limitations of classical single cue approaches, we integrate in the computation of ∆ a geometrical edgebased function ∆ g and a color-based one ∆ c with the corresponding weights w g , and w c : For ∆ c , the goal is to avoid any image extraction or segmentation that could lead to outliers and mismatch. By modeling the color (or luminance) appearance on both sides of the silhouette edges of the projected 3D model using simple statistics, a better accuracy can be achieved, with the advantage of being robust to image or motion blur, background clutter or noise.

∆ = w g ∆ g + w c ∆ c (

Efficient processing of the complete CAD model

One of the drawbacks of classical 3D model-based methods [CMPC06; DC02; PMK11] is that all the segments making of the 3D polygonal model are treated. It implies dealing with simple objects or performing manual and heavy pre-processing on the CAD model to make it compliant with real-time or computationally efficient implementations. The approach used in this work considers the direct use of a complete, but not necessarily polygonal model, which can be textured or untextured.

For efficiency, we employ a 3D rendering engine to efficiently handle 3D projections, determining visible salient edges from the rendered scene, and use these in the processing of the visual cues described in section 4.2. This enables our system to handle potentially very large models.

An advantage of this technique is to automatically handle the hidden face removal process and to implicitly handle self-occlusions. The whole set of edges and their 3D homologues, retrieved through the rendered depth buffer, will be used to compute ∆ g , while ∆ c only considers the silhouette edges.

Filtering

We incorporate a Kalman filtering process to smooth pose estimates and reduce jittering. As in [START_REF] Petit | Combining complementary edge, keypoint and color features in modelbased tracking for highly dynamic scenes[END_REF], we employ a linear Kalman filter on the camera velocity parameters v, which is then integrated to determine the pose so that: c M i+1 o = exp( v i+1 δt ) c M i o , at each time step i. We assume a constant velocity model. The camera pose and the camera displacement between successive frames can be assumed to be random variables, following Gaussian distributions. Through their covariances matrices, their uncertainty can be characterized, propagated from the low-level uncertainty, giving an indicator on the reliability of the tracking process and providing a global observation noise covariance matrix Σ P , which feeds the computation of the Kalman gain (see [START_REF] Petit | Combining complementary edge, keypoint and color features in modelbased tracking for highly dynamic scenes[END_REF] for more details).

Deformable Manipulation

We rely on physics-based modeling to enable deformable manipulation [MDM*02]. In our method, physical models are used to interactively manipulate objects in a wide range of 3D distortions including stretching, torsion, compression. In addition, the user can interactively edit physical attributes like stiffness, mass or gravity.

Using the input 3D model M, we first build using voxelization a volume V composed of co-rotated elements [NMK*06]. These elements represent the Degrees-of-Freedom (DoF) of the physical model following a finite element representation (cf Figure 4). The number of elements chosen in a tradeoff between interactive performance and sufficient accuracy. This process is done once at initialization.

A particular object deformation is specified by the displacements of nodal positions and the nodal forces. We build a stiffness matrix K depending on object's material properties, Young's modulus and Poisson's ratio. This enables us to linearize the relationship between nodal forces and nodal positions and permits real-time computation.

Finally, fixed boundary conditions q are necessary to correctly model the deformations. These constraints represent a part of the 3D model that is considered fixed.

We denote Θ(•) as the function that transforms an input 3D model M into a physical model D so that:

D = Θ(M, V, K, q) (7)
Once the volume built following Θ(•), the stiffness defined and the constrained regions chosen, the user can interact with the object at each frame of the sequence by moving the DoF producing forces that will deform the object according to its stiffness.

In addition to rigid transforms our framework enables object manipulation in a 3D manner including stretching, torsion, and compression. The produced deformations highly depend on the values of E the Young's modulus, and µ the Poisson ratio. For users not accustomed to using physical engines, pre-defined parameters are set according to object size and units. We also consider topological changes such as cutting or fracturing, and collisions that are detected by computing a proximity distance between two or more objects in the scene. Several type of forces can be applied to simulate pressure, wind or friction and act as external force on mesh surface. Physical attributes, whether intrinsic (related to the object, like stiffness, mass, and damping), or extrinsic (related to the scene, like gravity, or external forces), can be updated directly during the augmentation.

The user can thus simulate deflation and melting as well as scene dynamics like falling under high gravity or large mass.

Composition

In order to produce a final composition that is similar to the input sequence our framework measures scene lighting parameters and object material following a minimization scheme while object's background is replaced using inpainting after removing the original element from the scene. Both steps are described above.

Scene Appearance

Visually realistic augmentation requires knowledge of the object's diffuse properties and scene's light parameters. Inspired by [START_REF] Hara | Light source position and reflectance estimation from a single view without the distant illumination assumption[END_REF], we use a simplified Torrence-Sparrow reflection model that defines the specular reflection of an object's surface point as

I c (i) = k d,c , cos θ i r 2 + k s,c r 2 cos θ r exp -α 2 2σ 2 withc ∈ {r, g, b} (8)
where I(i) is the i th image pixel value, θ i the angle between the light source direction and the surface normal, θ r is the angle between the viewing direction and the surface normal, α is the angle between the surface normal and the intersection of the viewing direction and the light source direction. r represents the distance between the light source and the object surface point, k d and k s are coefficients for the diffuse and specular reflection components respectively and include light source intensity, and σ is the surface roughness.

We want to estimate an ensemble of parameters that consists of the specular reflection properties (k s ,σ ), diffuse reflection k d and light source position r from image I and the registered 3D mesh M. To do so, we start by directly calculating θ r , α and θ i using our inputs. First, the angle θ r can be obtained using the registered geometry M and camera position obtained from 3D/2D registration, then assuming a unique light source and a convex object, light source direction can be estimated by back-projecting image specular peaks on geometry normals which permits to estimate α and θ i .

Assuming a Lambertian material with constant albedo, we follow a diffuse-based constraints scheme to first estimate r knowing k d then we refine for (k s , σ ) to finally solve for (r, k d , k s , σ ) minimizing the squared error as argmin

r,k d ,k s ,σ ∑ i∈χ τ i I(i) - k d cos θ i r 2 + k s r 2 cos θ r exp -α 2 2σ 2 2 ( 9
)
where I i the image pixel value of i and τ i is a compensation factor used to avoid image compensation when computing the residual error. The domain χ represents the region of interest for the optimization scheme, where the diffuse image is used to estimate light position and diffuse reflection where the original image will be used for specular reflection estimation.

Image Completion and Re-texturing

Producing the final composition requires continuously building a new image I c as a result of the projection of the deformed and textured 3D model on the input image I.

First an inpainted image I p is generated using image completion technique following the silhouette contour of the projection of the 3D model with respect to the pose r.

We used the PatchMatch iterative algorithm [START_REF] Barnes | PatchMatch: a randomized correspondence algorithm for structural image editing[END_REF]. In order to speed-up the completion process, we only compute one iteration of the algorithm. Moreover, since most of the time the manipulations do not imply all the object, the area of the revealed part to compensate can be reduced to the difference between the original and the projected element, making the computation faster.

Once the completion is computed, the output composite image is built using the projection function Ω following the expression:

I c f = Ω P f , M (D f , T f ), I p f ( 10
)
where T is the texture of the targeted object, M (•) is a texture mapping function, f is the frame, I p is the inpainted image and P f represents the 3D/2D camera projection matrix from the object frame to the image plane.

Results

We tested our method on four examples, a Yacht sailing, an aerial view of the Statue of Liberty, a farmer with a Hat ploughing the ground and an Air balloon gliding through the air. Figure 8 shows the obtained results.

The 3D models were obtained from Internet databases (freely available). The physical models are automatically generated using a sparse hexahedral or tetrahedral grid fitting the object surface, independently of the surface discretization. The number of nodes (nodes) of the finite element model, the stiffness E, Poisson ratio ν, and number of constrained nodes (fixed nodes) used for each scenario are reported in Table 1. These parameters can be tuned as desired by the user, and pre-defined parameters are automatically set according to the 3D model size. We used the framework Sofa † for physical simulation. Zero displacement constraints are manually set on the physical model and represent the "boat pivot" for the Yacht, the "head" for the Farmer's Hat example, the "basket" for the Air balloon example and the "pedestal" for the Statue of Liberty.

In all examples, the object or the camera is moving. After a manual initial alignment, the pose is estimated over frames. Since a part of the object may be fixed (in object coordinates), the estimated pose is loaded on the virtual camera (in world coordinates). We used OpenGL for rendering.

Results for the tracking and pose estimation process are featured † www.sofa-framework.org in Figure 5, with visual overlays showing the accuracy the method.

The plots of the positions of the camera in the object frame demonstrates the benefit of filtering the complete pose to provide smooth estimates, which is essential to get visually realistic re-textured re-projections, as described in Section 6.2. Note that the camera parameters are unknown for the presented examples and are manually set, explaining the the potentially physically non-realistic estimations of the pose, and demonstrating the robustness of the pose tracking system w.r.t calibration of the camera.

We show several user interactions: stretching and pulling the Yacht and the Hat, inflating and deflating the AirBalloon, and more complex elastic deformations of the Statue of Liberty. Depending on the number of nodes of the 3D model, the process can run at interactive rates, reaching up to 17 fps. As an evaluation of the image completion and re-texturing method we show in Figure 6 the error between the original image and the in-painted and re-textured one, with the rigidly reprojected mesh, given the estimated rigid pose. The metric is the root mean square RGB error per pixel, evaluated on the whole image, and also over a bounding box around the silhouette of the projected mesh, here for the scenario of the Statue of Liberty. It shows the sensitivity of the image completion and re-texturing method especially with cluttered background, for instance when the statue passes over the island behind (frames 250-330).

Limitations

When dealing with videos with dynamic backgrounds, a temporal and coherent in-painting method is necessary to produce a final composition without visual break. The completion process considered in this work does not include temporal consistency and is computed frame-by-frame. However, we believe that using physicsbased models and scene dynamics can bring useful information to the establishment of a temporal inpainting, where object position can be predicted in time, and their silhouette can be used to compensate the new revealed parts of the video.

Another limitation of our framework resides in the robust estimation of a 3D pose, since the pose estimation may fail depending of object's background and in the presence of large occlusions (see Figure 7). This can be damaging to the whole pipeline since it also creates the region of completion and the detour the re-texturing area. We can think of recent methods based on trained histograms [START_REF] Tjaden | Real-Time Monocular Pose Estimation of 3D Objects Using Temporally Consistent Local Color Histograms[END_REF] that could be plugged to our framework. Ideally, we would like to have a 3D abstraction of the scene making the alignment fully automatic and more accurate. Approaches proposed in [START_REF] Wohlhart | Learning descriptors for object recognition and 3d pose estimation[END_REF], [START_REF] Su | Render for cnn: Viewpoint estimation in images using cnns trained with rendered 3d model views[END_REF], [KMT*17] would then be useful solutions to provide pose (re-)initialization to our pipeline. Our work could also benefit from recent works based on machine learning [CXG*16], [START_REF] Xie | Deep3D: Fully Automatic 2D-to-3D Video Conversion with Deep Convolutional Neural Networks[END_REF], where 3D reconstructions are estimated using learned models that extract shape priors from large collection of 3D data, or more recent work using convolutional neural networks [START_REF] Izadinia | IM2CAD[END_REF]. Another improvement would be to generate the 3D model directly from the input stream, using recent structure-from-motion methods [START_REF] Ondruska | MobileFusion: Real-Time Volumetric Surface Reconstruction and Dense Tracking on Mobile Phones[END_REF], or by relying on intuitive user-centered methods such as 3-sweep [CZS*13] or cuboids [ZCC*12] for simple geometry.

Conclusion

We presented "Deformed Reality" a new paradigm for augmented reality that enables the user to deform objects in scenes without producing visual breaks usually emanating from 3D mesh overlay in classical AR systems. Using an efficient method to estimate object pose and relying on a physics-based model built on-the-fly, the final composition is rendered sequentially to produce a sequence where objects are deformed and registered in real-time giving sens of manipulating the surrounding of the objects. Several limitations are to be addressed to bring our concept to a well-established technique mainly: (i) a robust real-time continuous inpainting, (ii) handling occluded regions during pose estimation and (iii) appearance and

  6) ∆ g and ∆ c are computed in a similar way to [PMK13][PMK14]. ∆ g relies on line-to-point correspondences between model and image edges and on geometrical distances accounting for these correspondences. Edges have the advantage of being robust to illumination conditions but suffer from having similar appearance, resulting in ambiguities and potential local minima.

Figure 3 :

 3 Figure 3: Pose Estimation: selected frames from the video stream showing the mesh overlaid on the input image (green contour) from camera pose estimation.

Figure 4 :

 4 Figure 4: Finite element volume and boundary constraints: the volumetric mesh V is built upon the surface model M in (a) following a grid tetrahedron volume model (b), or a grid hexahedron volume model (c). The red spheres represent the boundary constraints.

Figure 5 :

 5 Figure 5: Camera pose over frames for the statue example: (a) The measured and filtered camera translation in the object frame (from o M c ) in X, Y and Z. (b) The uncertainty indicator over the camera position estimation, computed as the trace of the covariance matrix of the estimated pose c M o (for the position parameters), for the two visual cues (edge and color) independently and jointly.

Figure 6 :

 6 Figure6: Evaluation of the image in-painting and re-texturing method by comparing the original image with the re-texturing one (statue example), with the mesh rigidly reprojected given the estimated pose. It is evaluated on the whole image (red) and on a bounding box around the projected model (green).

Figure 7 :

 7 Figure 7: Our approach may fail to correctly estimate the pose. Detection of contours may fall into ambiguities since edge/background separation is based on gradient saliency.

Figure 8 :

 8 Figure 8: 3D physics-based manipulations of various objects in using Deformed Reality pipeline. The user can interact with objects through a rich range of manipulations, including rigid and non-rigid transforms and physical parameter updates such as mass, stiffness or gravity, and thereby pull on the the Yacht's sail while moving in a race, deform the Farmer's hat with elastic response, deform the air balloon and make it deflate, inflate and fall under gravity and melt the Statue of Liberty. [video sequences are shown in the supplemental video]

Table 1 :

 1 User-selected physical parameters for each example.

		E (Kpa)	ν	Nodes Fixed Nodes
	Yacht	450	0.45	192	22
	Statue of Liberty	100	0.45	6750	225
	Airballoon	25	0.45	288	16
	Farmer's Hat	5000	0.3	1000	120

© 2019 The Author(s) Eurographics Proceedings © 2019 The Eurographics Association.