Richard P Hopton

Elizabeth Turner

Victoria J Burley
email: v.j.burley@leeds.ac.uk

Paul C Turner
email: p.c.turner@leeds.ac.uk

Julie Fisher
email: j.fisher@chem.leeds.ac.uk

Urine metabolite analysis

Keywords: Mycotoxins -fusarium, Mycotoxins -trichothecenes Food Types: Beer, Cereals and grain, Bread deoxynivalenol, urine, NMR, metabolic profiling, biomarker

teaching and research institutions in France or abroad, or from public or private research centers.

Introduction

Mycotoxins are frequent contaminants of cereal crops such as wheat, maize and barley that are intended for human consumption (CAST 2003). Deoxynivalenol (DON) was the most frequent Fusarium mycotoxin contaminant of cereals reported for European Union member countries [START_REF] Scoop | Collection of occurrence data of Fusarium toxins in food and assessment of dietary intake by the population of EU member states[END_REF]; with 57% of the 11,022 cereals tested being positive, and 13% at a level exceeding the guidelines established for DON [START_REF] Scoop | Collection of occurrence data of Fusarium toxins in food and assessment of dietary intake by the population of EU member states[END_REF]. DON is stable to much of the processing and cooking methods [START_REF] Jackson | Effect of processing on Fusarium mycotoxins[END_REF], thus DON contamination persists in the food chain (Sugita-Konishi et al. 2006), and human exposure is frequent [START_REF] Scoop | Collection of occurrence data of Fusarium toxins in food and assessment of dietary intake by the population of EU member states[END_REF][START_REF] Canady | Deoxynivalenol. In: Safety evaluation of certain mycotoxins in food[END_REF]. The toxicity of DON is clearly established in animal studies, and includes gastrointestinal toxicity, immune modulation, vomiting, feed refusal and growth retardation [START_REF] Pestka | Deoxynivalenol: Toxicology and potential effects on humans[END_REF]. There are differences between animal species in their susceptibility to DON, and this in part reflects the capacity of each species intestinal microflora to detoxify DON to de-epoxy DON [START_REF] Yoshizawa | Confirmation of DOM-1, a deepoxidation metabolite of deoxynivalenol, in Biological-Fluids of Lactating Cows[END_REF]); a detoxification pathway that [START_REF] Sundstol-Eriksen | Lack of de-epoxidation of type B trichothecenes in incubates with human faeces[END_REF] have suggested human intestinal microflora may lack. A review of human epidemiological data indicates that gastroenteritis, growth faltering, an increased susceptibility to infections and immune toxicity were possible consequences of DON exposure [START_REF] Pestka | Deoxynivalenol: Toxicology and potential effects on humans[END_REF]. Food poisoning incidents in China between the years of 1984 and 1991 implicate a role for DON in human toxicity [START_REF] Luo | Food poisoning caused by Fusarium toxins[END_REF]. A similar incident in India involving thousands of people consuming contaminated food supports this hypothesis (Bhat et al. 1989). However, the lack of a validated exposure biomarker restricts the power of such ecological observations.

One approach is to develop biological measures that are directly correlated with intake. These may take the form of measures of the parent compound or their metabolites [START_REF] Lindon | Metabonomics technologies and their applications in physiological monitoring, drug safety assessment and disease diagnosis[END_REF]) in biological fluids, including urine or serum. In swine approximately 50-60% of the ingested DON is excreted in the urine with the majority (>95%) as the parent compound (DON) or a DON glucuronide [START_REF] Goyarts | Bioavailability of the Fusarium toxin deoxynivalenol (DON) from naturally contaminated wheat for the pig[END_REF], with de-epoxy DON observed with higher dosing regimens. DON is mostly cleared within 24-48 hours of ingestion. A similar pattern of excretion in humans has been reported [START_REF] Meky | Development of a urinary biomarker of human exposure to deoxynivalenol[END_REF], Turner et al., 2008a), though no de-epoxy DON has been detected in UK urines to date (Turner unpublished observation). A urinary assay for DON [START_REF] Meky | Development of a urinary biomarker of human exposure to deoxynivalenol[END_REF] involving immunoaffinity enrichment and LC-MS quantification was recently improved by the inclusion of a labelled internal standard 13 C 15 -DON (Turner et al. 2008a,b,c). An alternative approach is to interrogate the urinary metabolome using NMR-based metabonomics [START_REF] Coen | NMR-based metabolic profiling and metabonomic approaches to problems in molecular toxicology[END_REF][START_REF] Connor | Effects of feeding and body weight loss on the 1 H-NMR based urine metabolic profiles of male Wistar Han rats: implications for biomarker discovery[END_REF][START_REF] Lindon | Metabonomics technologies and their applications in physiological monitoring, drug safety assessment and disease diagnosis[END_REF]). This latter approach may additionally reflect a more integrated measure of the temporal variation of exposure.

The current study aimed to correlate urinary DON level with one or more metabolite in the urine. Established NMR data collection and statistical analysis techniques were applied [START_REF] Bollard | NMR-based metabonomic approaches for evaluating physiological influences on biofluid composition[END_REF][START_REF] Coen | NMR-based metabolic profiling and metabonomic approaches to problems in molecular toxicology[END_REF][START_REF] Lindon | Metabonomics technologies and their applications in physiological monitoring, drug safety assessment and disease diagnosis[END_REF]) to interrogate urine samples previously analysed for urinary DON (Turner et al. 2008a). While these are preliminary data they do provide an interesting basis for the design of future studies.

Materials and Methods

Twenty-two urine samples were available from a small intervention trial conducted with UK adults. In brief, food diaries were kept over a two day period as described previously (Turner et al. 2008a within the range 4.35 -6.01 ppm were set to zero to remove effects of variable water suppressions and urea concentrations. This was conducted using Vnmrj 1.1D software (Varian Inc., Palo Alto, California, USA). The data were normalised with respect to the total area of each spectrum. This removed variation in urine concentration from sample to sample [START_REF] Craig | Scaling and normalization effects in NMR spectroscopic metabonomic data sets[END_REF]. Alternative methods for both data reduction and normalization are available [START_REF] Forshed | Peak alignment of NMR signals by means of a genetic algorithm[END_REF], Cloarec et al 2005), however the methods adopted here remain the most popular ( Gu et al, 2007[START_REF] Wang | Exploration of the direct metabolic effects of mercury II chloride on the kidney of Sprague-Dawley rats using high-resolution magic angle spinning 1 H NMR spectroscopy of intact tissue and pattern recognition[END_REF][START_REF] Smith | Large scale human metabolic phenotyping and molecular preservation[END_REF].

Partial Least Squares Discriminant Analysis (PLS-DA) was performed on the data which had been mean centred and Pareto scaled using SIMCA-P + 11 (Umetrics, Umea, Sweden).

The PLS-DA model generated was validated with > 10% extra samples. Class membership in the prediction set was achieved using a value of >0.5 to determine correct class membership.

The urine samples were dichotomised into two groups based on the urinary DON levels reported by Turner et al. (2008a)

Results

Demographic study data are provided in Tables 1 and2. There were more females (n=11) than males (n = 5) in the model, and in the overall study, females (n =15), males (n = 7).

There was no correlation between gender and the concentration of the biomarker (see below) of DON exposure (see supplementary figure). There was a significant difference in DON levels between LUD and HUD levels in each group (p<0.001). Regular medication was taken by seven individuals. Three women were taking oral contraceptives (two in LUD and one in HUD). Two women in the LUD group were taking hormone replacement therapy. One woman in the LUD did not describe her medication and one man in the HUD was taking non-steroidal anti-inflammatory medication.

The geometric mean level of urinary DON for the six test samples was 9.8ng DON/mg creatinine; 95%CI: 3.6, 27.1ng/mg). None of the demographic characteristics of this group were significantly different compared to those from the group used to build the model (data not shown). One woman with HUD intake was taking a thyroid medication.

"[Insert Table 1 about here]" Typical 1D NOESY spectra from a HUD and a LUD sample are provided in Figure 1.

Visually there were minimal differences between these spectra, but in any case spectra were normalised to accommodate any variation in total urine concentration (see below).. The separation between the LUD and HUD exposure classes was mainly in principle component one (PC1) with a small amount in PC2. Through a loadings scatter plot (Figure 3), regions of the spectrum contributing most to the separation between the classes were readily distinguished [START_REF] Holmes | The identification of novel biomarkers of renal toxicity using automatic data reduction techniques and PCA of proton NMR spectra of urine[END_REF]. "[Insert Figure 3 about here]". Two main areas are important, the first in the region ~ 3.60 -3.90 ppm, the region where signals from mannitiol arise (Holmes et al. 1997). As this is a common food additive [START_REF] Ji | Modeling of water activity, oxygen solubility and density of sugar and sugar alcohol solutions[END_REF] further correlation with this measure and urinary DON level was not investigated. In the second region four signals arising from the hippurate molecule appear 7.83 -7.55 and 3.95 ppm [START_REF] Holmes | The identification of novel biomarkers of renal toxicity using automatic data reduction techniques and PCA of proton NMR spectra of urine[END_REF][START_REF] Zuppi | 1 H NMR spectra of normal urines: Reference ranges of the major metabolites[END_REF]."[Insert Figure 4 about here]". Specifically the four integral regions for hippurate are contained in the bins centred around 7.84, 7.63, 7.55 and 3.96 ppm, in good agreement with literature values, and the integral values of these are statistically significant (p = 0.047, 0.047, 0.047 and 0.030 respectively). These higher hippurate levels in urine coincide with higher urinary DON levels. The median values for the binned integrals were 0.0109 and 0.0170 for LUD and HUD respectively indicating a 1.5 fold increase in hippurate levels in samples found to have high levels of urinary DON . Further validation of the PLS-DA model used the remaining six samples (which had been selected at random) (Table 3). The PLS-DA model (Figure 5) predicts all samples correctly in their classes. "[Insert Table 2 about here]" "[Insert Figure 5 about here]"

Discussion

This study compared urinary DON with its urinary metabolic profile in UK adult samples.

A 1 H NMR-based chemometrics approach; metabolomics, was utilised to examine samples from individuals eating their normal diet. This methodological approach has gained popularity as a rapid and non-invasive method for biomarker detection [START_REF] Lindon | Metabonomics: Metabolic processes studied by NMR spectroscopy of biofluids[END_REF][START_REF] Lindon | Metabonomics technologies and their applications in physiological monitoring, drug safety assessment and disease diagnosis[END_REF][START_REF] Bollard | NMR-based metabonomic approaches for evaluating physiological influences on biofluid composition[END_REF]. The statistical analysis undertaken here has identified two possible biomarkers; hippurate and mannitiol. Mannitol is used in processed foods, such as candies, cookies and chewing gum as an alternative to sucrose (sugar) as it has little effect on blood sugar levels [START_REF] Ji | Modeling of water activity, oxygen solubility and density of sugar and sugar alcohol solutions[END_REF]. Mannitol is also used in clinical applications [START_REF] Wakai | Mannitol for acute traumatic brain injury[END_REF]). Thus while it was identified in this study there is no strong mechanistic basis for considering this measure further. By contrast hippurate is a more interesting candidate. Hippurate is present in urine samples due to both renal and hepatic synthesis, through conjugation of glycine and benzoic acid [START_REF] Zuppi | 1 H NMR spectra of normal urines: Reference ranges of the major metabolites[END_REF]. It can also be formed through the degradation of shikimic acid, by intestinal microorganisms [START_REF] Brewster | Metabolism of shikimate in rat[END_REF]. The conjugation of glycine with benzoic acid to form hippurate has been found to be an important mechanism for elimination and detoxification of aromatic carboxylic acids [START_REF] Kasuya | Purification and characterization of a medium chain acyl-coenzyme a-synthetase[END_REF]. Both dietary factors and microbial action can influence urinary hippurate [START_REF] Lord | Clinical Applications of Urinary Organic Acids. Part 2. Dysbiosis Markers[END_REF]). The food diary information does not suggest higher benzoate intake and therefore this is not the reason for the variation in hippurate concentration. Gut microflora activity in rats (Phipps et al., 1988) and in children [START_REF] Spaapen | Urinary D-4hydroxyphenyllactate, D-phenyllactate and D-2-hydroxyisocaproate, abnormalities of bacterial origin[END_REF] were correlated with urinary hippurate levels; and DON was recently shown to modify the gut microflora in dosed swine [START_REF] Wache | Impact of deoxynivalenol on the Intestinal microflora of pigs[END_REF]. To date no studies have interrogated the metabolome of DON exposed animals and it would be interesting to further examine this putative interaction in such a model.

The statistical model built was based on a small number of participants however appropriate tests of significance have been performed; the model produced no samples outside the 95% confidence limit and all the samples in the prediction set were >0.5, which is acceptable for metabolomics data. The variation in DON levels in samples used to build this model was relatively small, (mean 8.2ng/mg; range 2.4-27.1ng/mg). However, we can make rough estimates of mean intake for each of these groups. If we assume an average volume of urine per day to be 1500ml and an approximate 50% transfer of DON to urine we can make a rough estimate of the daily intake per kg body weight. The low group would have an intake of 150ng/kg bw/day (95%CI: 110, 220), whilst high group would have an intake of 500ng/kg bw/day (95%CI: 350, 670). The lower intake is roughly in line with individuals consuming the average quantity of wheat based food items in the UK at the average level of contamination, whilst the high intake group are consuming far greater quantities of DON contaminated foods, though for both groups the intake remains within estimates predicted by [START_REF] Scoop | Collection of occurrence data of Fusarium toxins in food and assessment of dietary intake by the population of EU member states[END_REF].

Future studies should incorporate larger numbers and involve a wider range of DON

exposure. An assessment of the temporal nature of this measure during consumption of both a normal diet and during dietary intervention to restrict DON exposure would be valuable. This study highlights an NMR metabolomics approach which hopefully will

  Urine metabolite analysis as a function of deoxynivalenol exposure: an NMR-based metabolomics investigation

"

  [Insert Figure 1 about here]" The PLS-DA scores plot (Figure 2) was built with the participants split into groups of HUD and LUD. The scores plot (Figure 2) displays separation of samples on the basis of class membership; LUD (open black diamond) and HUD (open black triangles). There was a degree of overlap; due to the samples 18A and 21A. Food diary information revealed no obvious dietary reasons for this. No samples lay outside the 95% confidence limit which indicates a good model. The data were not closely clustered which may simply reflect the small sample size used. "[Insert Figure 2 about here]"

Figure 1 .

 1 Figure captionsFigure 1. Low field region of the 1 H-NMR spectrum of a urine sample taken from a

Figure 2 .

 2 Figure 2. PLS-DA scores plot generated from 1 H NMR data from urine samples taken from

Figure 3 .

 3 Figure 3. Loadings plot resulting from the PLS-DA analysis (figure 2) indicating the

Figure 4 .

 4 Figure 4. Expansion of the 1-dimensional loadings plot (P[1]), highlighting signals in the

Figure 5 .

 5 Figure 5. Scores plot indicating that the six samples used to validate the PLS-DA model are

  Figure 2

Figure Caption Figure 1 Figure 2

 Caption12 Figure Caption

  ). On day three a first morning void sample was collected. Aliquots of

	each sample were cryopreserved at -40 0 C within 6 hours of collection. Urinary DON was
	determined using an in-house immunoaffinity -LC-MS assay (Turner et al. 2008a). The
	cryopreserved samples (-40°C) were thawed, centrifuged (2000g; 15 min; 4°C), and
	duplicate aliquots were taken for NMR experiments and re-frozen at -32°C prior to
	F o r P e e r R e v i e w O n In preparation for the NMR studies the samples were thawed at room temperature. To 400 analysis. µl of urine, sodium phosphate buffer (200 µl pH 7.3 0.2M) was added. To 540 µl of this solution, deuterium oxide (10 µl) (Fluorochem, Old Glosoop, UK) was added. A solution containing 0.1% w/v solution of the sodium salt of 3-(trimethylsilyl)propanoic -2,2,3,3-d4 acid (TSP) (Sigma-Aldrich, Poole UK) and 1% w/v of sodium azide, in D 2 O was added (50 µl). The resulting solution was vortexed before transferring all 600 µl to a 5mm NMR tube (528PP-WILMAD, Sigma-Aldrich, Poole, UK). The TSP provided a chemical shift reference (δ H = 0.00 ppm). 1 H NMR spectra were acquired at 499.97 MHz on a Varian Unity Inova 500 spectrometer at 20°C. Spectra where collected using a 1-dimensional NOESY sequence [RD-90° -t 1 -90° -t m -90° -collect free induction decay]. The relaxation delay, RD, was 2.0 seconds, t m o r P e e r R e v i e w was 100 ms and t 1 was set to 3 µs and the water signal was saturated during RD and t F O n
	l y l y

m . 128 scans were collected in 32,768 data points, with a spectral width of 10,000 Hz and an acquisition time of 3.28 seconds. Spectral quality was assessed through measurement of water and TSP signals at half height and at 0.55 % peak height. An exponential line broadening of 1.0 Hz was applied to each data set, prior to zero filling to 65,536 points and Fourier transformation. Subsequently phasing and baseline correction was performed as required. Data were 'binned' into 224 equal portions between -0.05 -9.47 ppm. The bins

Table 1 .

 1 Demographics of participants for PLS-DA model building

	Page 23 of 27	Food Additives and Contaminants
	Characteristic	Total (n=16)	LUD* (n=8)	HUD* (n=8)	p-value**
		Mean (range)	Mean (95% CI)	Mean (95% CI)
	Urinary DON #	8.2 (2.4 -27.1) a	3.6 (2.6, 5.0) b	11.1 (8.1, 15.5) b	<0.001
	Sex	5m/11f	4m/4f	1m/7f	0.120
	Ethnicity -delete	16wC	8wC	8wC	n/a
	as all white
	Caucasian -refer
	to this in text
	Age	39 (21 -59)	34 (26, 42)	43 (35, 50)	0.10
	Height / m	1.69 (1.52 -1.96)	1.67 (1.59, 1.75)	1.71 (1.63, 1.79)	0.47
	Weight / kg	69 (51 -93)	69 (60, 77)	69 (61, 78)	0.98
	BMI / kg/m 2	24.2 (19.2 -32.0)	25.0 (22.6, 27.4)	23.4 (21.0, 25.8)	0.34
	* Low urinary DON (LUD), high urinary DON (HUD)
	** T-test comparison of LUD and HUD groups
	# Urinary DON ng/mg creatinine (data extracted from Turner et al., 2008a)
	a -Arithmetic mean and range
	b -Geometric mean and 95%CI
	95% CI -95% confidence interval
	m -male, f -female
	wC -white Caucasian
	n/a -not applicable
	BMI -body mass index

http://mc.manuscriptcentral.com/tfac Email: fac@tandf.co.uk

Table 2 .

 2 Demographics of participants in test group

	Characteristic	Total (n=6)
		Mean (range)
	Urinary DON #	9.8 (3.6 -27.1) a
	Sex	1m/5f
	Ethnicity	6wC
	Age	36 (24 -57)
	Height / m	1.66 (1.57 -1.78)
	Weight / kg	64 (55 -88)
	BMI / kg/m 2	23.1 (20.2 -31.2)
	# Urinary DON ng/mg creatinine (data extracted from Turner et al., 2008a)
	a -Geometric mean and 95%CI
	m -male, f -female
	wC -white Caucasian
	BMI -body mass index

Table 3 .

 3 Additional samples used to test PLS-DA model. Values > 0.50 within class

	Page 25 of 27	Food Additives and Contaminants
		Sample code	Class 1 a Class 2 b
		38A HUD *	0.09	0.91
		34A HUD	0.46	0.54
		6A HUD	0.00	1.00
		7A LUD **	0.59	0.41
		32A LUD	0.66	0.34
		43A LUD	0.74	0.26
		* High urinary DON (HUD),
		** Low Urinary DON (LUD)
		a Low urinary DON classification
		b High urinary DON classification

http://mc.manuscriptcentral.com/tfac Email: fac@tandf.co.uk

http://mc.manuscriptcentral.com/tfac Email: fac@tandf.co.ukFood Additives and Contaminants

Acknowledgements

We thank the White Rose Doctoral Training Centre, UK (RPH).


MD Andrey Zhivov
email: andyzhyvov@yahoo.com

MD Marcus Blum

MD Rudolf Guthoff

PhD Oliver Stachs

Real-time mapping of the subepithelial nerve plexus by in vivo confocal laser scanning microscopy

Keywords: confocal microscopy, mapping, cornea

Aim: To produce two-dimensional reconstruction maps of the subepithelial nerve plexus (SEP) in living cornea by in vivo laser scanning confocal microscopy in real time.

Methods:

In vivo confocal laser scanning microscopy (Heidelberg Retinal Tomograph II in conjunction with the Rostock Cornea Module) was performed on normal eyes (n=6) and eyes after LASIK (n=4). Source data (frame rate 30Hz) were used to create large-scale maps of the scanned area in Automatic Real Time composite mode. The algorithm aligns single live images onto the previously mapped composite image using landmark feature-based image processing.

Results: Real-time mapping of the SEP was performed on a large-scale area up to 3.2 mm x 3.2 mm (3072 x 3072 pixels) both in healthy subjects and in post-LASIK patients. Twodimensional structures of the SEP were imaged in all 10 eyes. Mapping quality as well as acquisition time were dependent on subject compliance and examiner experience.

Conclusion:

The described method permits real-time in vivo mapping of the SEP, thus providing the necessary basis for statistically robust conclusions concerning morphometric plexus alterations.

Introduction

In vivo confocal laser scanning microscopy (CLSM) is a powerful tool for analysing living corneal tissues in laboratory animals [START_REF] Labbe | Comparative anatomy of laboratory animal corneas with a new-generation high-resolution in vivo confocal microscope[END_REF] and humans. [START_REF] Guthoff | In vivo confocal microscopy, an inner vision of the cornea -a major review[END_REF][START_REF] Patel | Contemporary in vivo confocal microscopy of the living human cornea using white light and laser scanning techniques: a major review[END_REF][START_REF] Jalbert | In vivo confocal microscopy of the human cornea[END_REF][START_REF] Nubile | In vivo confocal microscopy of the ocular surface: where are we now?[END_REF] One of the principal limitations of this method is the small field of view for imaging, which is an obstacle to the statistically robust evaluation of findings in a large-scale area (fig 1).

Recently, several research groups have published reports of two-dimensional reconstruction mapping of stained corneal whole mounts [START_REF] Al-Aqaba | Architecture and distribution of human corneal nerves[END_REF] as well as of the living human cornea [START_REF] Stachs | Structural-functional correlations of corneal innervation after LASIK and penetrating keratoplasty[END_REF][START_REF] Patel | Mapping the corneal sub-basal nerve plexus in keratoconus by in vivo laser scanning confocal microscopy[END_REF][START_REF] Patel | Mapping of the normal human corneal sub-Basal nerve plexus by in vivo laser scanning confocal microscopy[END_REF][START_REF] Yokogawa | Mapping of normal corneal K-structures by in vivo laser confocal microscopy[END_REF]. In addition, our own group has published results of in vivo three-dimensional reconstruction of the ocular surface and cornea. [START_REF] Stachs | In vivo three-dimensional confocal laser scanning microscopy of the epithelial nerve structure in the human cornea[END_REF][START_REF] Zhivov | In vivo confocal microscopy of the ocular surface[END_REF][START_REF] Zhivov | In vivo three-dimensional confocal laser scanning microscopy of corneal surface and epithelium[END_REF] The composite procedure in all these studies was performed off-line, i.e. mapping required manual external software support for image postprocessing. Imaging artefacts (for example, applanation-induced tissue folds, epithelial compression, and movement) generate problems in image rendering. To date these artefacts have been associated with reduced image quality and have necessitated time-consuming offline processing.

We present here an experimental on-line mapping approach based on in vivo CLSM with the Heidelberg Retinal Tomograph II (HRTII) (Heidelberg Engineering GmbH, Heidelberg, Germany) in conjunction with the Rostock Cornea Module (RCM). This combination allows on-line scanning, processing and mapping of the living human cornea. The study focuses on the subepithelial nerve plexus (SEP), one of the most interesting and dynamically changing structures of the cornea.

Materials and methods

In vivo CLSM was performed with the HRTII/RCM equipped with a water contact objective (Zeiss, 63x/0.95W, 670 nm, ∞/0, Jena, Germany), as described elsewhere. [START_REF] Stave | Modified Heidelberg Retinal Tomograph HRT. Initial results of in vivo presentation of corneal structures[END_REF] Microscopy of the area is manual: the investigator has to move the objective in the x-y axis in order to depict the structures of interest and manually control the quality of the image. For better handling and image quality the patients were asked to gaze at the microscope's fixation light, which was moved manually to the opposite side away from the region of interest in order to map the maximal corneal surface.

CLSM source data (frame rate 30Hz, 384x384 pixels, 400x400 µm) were used to create large-scale maps of the scanned area by selecting the Automatic Real Time (ART) composite mode. The algorithm is based on an affine transformation of six parameters (2 x 2 transformation matrix and translation in x and y directions). In an initial step the six transformation parameters between the current live image and all images in the composite reference image pool are calculated using features inside the current live image and the reference image. If the reference image pool is empty, the actual live image is added to the pool in centre position. The affine transformation parameters are calculated using a least squares fit of the overdetermined equation system derived from live image and reference image coordinates. The actual live image can be mapped to various images inside the reference pool; therefore all determined sets of transformation parameters will be combined and averaged to a parameter set.

Live images with correlations and an overlap <75% are added to the reference image data pool. Images with ≥75% overlap as well those without correlations to the reference image data pool are rejected. Finally, the actual live image is inserted into the ART composite image on the basis of the transformation parameters by pixel-by-pixel averaging and decreasing the weighting towards the image edge. Processing of the next image is then started. The maximal ART composite image size using source images is 3072x3072 pixels (3.2x3.2 mm) and corresponds to 8x8 non-overlapping images.

In total, ten human corneas were investigated (six from healthy volunteers, aged 22-35 years, two females and four males; four from patients who had undergone LASIK, aged 30-44 years, two females and two males, 6 months postoperatively). The study was performed at the Helios Klinikum, Erfurt, Germany once approval had been obtained from the Ethics Committee of the Thuringia State Medical Board, Jena, Germany. The study was explained in detail to the patients, and informed written consent was obtained before any investigative procedures were conducted. Image acquisition was performed in ART composite mode at

Results

CLSM in ART composite

Discussion

Since Li et al. [START_REF] Li | On-line 3-dimensional confocal imaging in vivo[END_REF] presented their on-line three-dimensional confocal imaging system based on a tandem scanning confocal microscope in 2000, there have been no further attempts to design another on-line imaging system. The aim of the present article is to demonstrate a novel approach to real-time mapping of corneal structures using in vivo CLSM.

One of the main problems of mapping is image quality and the associated problems of offline processing: up to 50% of off-line data are not suitable for processing due to artefacts. [START_REF] Zhivov | In vivo three-dimensional confocal laser scanning microscopy of corneal surface and epithelium[END_REF] The present algorithm aligns only live images of sufficient quality and permits quality control in real time. In some cases different corneal layers are presented in one mapping image: this is analogous to the oblique imaging technique in section mode described elsewhere. [START_REF] Zhivov | In vivo three-dimensional confocal laser scanning microscopy of corneal surface and epithelium[END_REF] Moreover, in vivo confocal microscopes currently used in ophthalmology depict only a very small area (contact mode: HRTII/RCM 400x400 µm, i.e. 0.160 mm 2 [START_REF] Zhivov | In vivo three-dimensional confocal laser scanning microscopy of corneal surface and epithelium[END_REF] (fig 1); Confoscan 4 /Nidek 460x345 µm (www.nidek-intl.com/cs.html), i.e. 0.159 mm 2 ), and consequently the reproducible investigation and quantification of identical areas is virtually impossible. The real-time method described here enables mapping of corneal areas up to 3200x3200 µm and opens up new horizons for in vivo tissue analyses in real time. The established quantification parameters of the SEP, e.g. number and length of sub-basal nerves, nerve density and tortuosity, [START_REF] Patel | Corneal sensitivity and slit scanning in vivo confocal microscopy of the subbasal nerve plexus of the normal central and peripheral human cornea[END_REF] can now be measured in a statistically robust manner. Our system also permits follow-up studies of the SEP, which is a highly dynamic structure known to be characterised by continuous pattern changes. [START_REF] Patel | In vivo laser scanning confocal microscopy confirms that the human corneal sub-basal nerve plexus is a highly dynamic structure[END_REF] Nevertheless, the quality and size of the composite image were considerably influenced by subject compliance. To overcome these difficulties it is possible to reduce the mapping image to 1536x1536 pixels (1.6x1.6 mm) and thus to shorten microscopy time. It is also advisable to accumulate sufficient practice with the single-image mode before switching to mapping mode. However, a general examiner with an average experience will not have any problems with averagely compliant patients.

Patel et al. [START_REF] Patel | Mapping the corneal sub-basal nerve plexus in keratoconus by in vivo laser scanning confocal microscopy[END_REF] used in vivo CLSM for off-line mapping of SEP in keratoconus (mean mapped area: 6.60±0.7 mm horizontally and 5.91±0.72 mm vertically; duration of investigation: 40 minutes, including 20 minutes for microscopy). Wide-field montages were post-processed using external software with a mean of 402±57 images. Similarly, Yokogawa et al. [START_REF] Yokogawa | Mapping of normal corneal K-structures by in vivo laser confocal microscopy[END_REF] produced two-dimensional reconstruction maps of normal corneal K-structures (mean mapped area: 5.88±0.5 mm horizontally and 3.51±1.37 mm vertically; 10 minutes for microscopy, 677±211 images). Our system permits an area of 3.2x3.2 mm to be mapped with a total of 64 theoretically non-overlapping single 400x400 µm images. Given that the Supported in part by the DFG (Transregio 37, Micro-and Nanosystems in Medicine -Reconstruction of Biological Functions). Software development was performed in collaboration with Heidelberg Engineering GmbH, Heidelberg, Germany.

Copyright licence statement

The Corresponding Author has the right to grant on behalf of all authors and does grant on behalf of all authors, an exclusive licence (or non-exclusive for government employees) on a worldwide basis to the BMJ Publishing Group Ltd and its Licensees, to permit this article (if accepted) to be published in BJO and any other BMJ Group products and sublicenses such use and exploit all subsidiary rights, as set out in our licence.

Figures and Legends

Sample CLSM image (400x400 µm; depth: 57 µm) of SEP in a healthy volunteer (image scaled to corneal diameter).

  mode permitted real-time mapping of the living cornea. The acquisition time for a single composite image was up to 3 minutes. The quality and size of the composite image were considerably influenced by subject compliance and examiner experience (AZ). Real-time mapping of the SEP was performed on a large-scale area up to 3200x3200 µm (3072x3072 pixels) in all 6 healthy volunteers (fig 2). Mapping of regenerated SEP structures was achieved in all LASIK patients (n=4). Representative images obtained 6 months postoperatively show the SEP in the central flap zone (fig 3A), and the cut zone with epithelial cell ingrowth and nerves growing outside and inside the flap area (fig 3B). In some cases several corneal layers are present in one mapping image due to the focus shift during microscopy.

Figure 2 Composite

 2 Figure 2Composite CLSM image of SEP in a normal human cornea: image size (A) 1.6 x 1.6 mm (depth: 60 µm) and (B) 3.2 x 3.2 mm (depth: 62 µm) (images scaled to corneal diameter).

Figure 3 Composite

 3 Figure 3 Composite CLSM images of the human cornea 6 months after LASIK (images scaled to corneal diameter): A Central cornea with regenerated SEP nerve fibre structures; some keratocyte nuclei are visible in peripheral upper areas of the image due to the focal plane shift during image acquisition (image size: 3.2 x 3.2 mm; depth: 58 µm). B Cut zone showing epithelial cell ingrowth (*), and SEP (arrows) both outside and inside the flap area (image size: 1.6 x 1.6 mm; depth: 61 µm).

Fig. 1

 1 Fig. 1

Fig. 3

 3 Fig. 2

Acknowledgements

The authors wish to acknowledge the technical assistance of A. Kunert, MD (Helios Klinikum, Erfurt), W. Bissmann, PhD and A. Gille (Carl Zeiss Meditec AG, Jena), and the editorial support of D. Beattie.

Funding

the level of the SEP in the central cornea in healthy volunteers and in the central as well in the temporal and nasal flap areas in the LASIK group.

total CLSM time for a single composite image is less than 3 minutes with no post-processing, the advantages of the on-line system are apparent.

The method presented here permits real-time in vivo mapping of the subepithelial nerve plexus, and thus provides an essential basis for statistically robust conclusions regarding morphometric plexus changes. Our on-line system can be used for large-scale in vivo mapping of a number of two-dimensional structures following imaging with the HRTII/RCM confocal laser scanning microscope. Potential clinical applications include analysis of corneal nerve degeneration and regeneration, of surgical outcomes (e.g. corneal grafting, refractive surgery), and of contact lens wear or ocular surface disease.

Competing interests

None.


MD Markus Ritter

Julee Elledge

MD Christian Simader

MD Gabor G Deak

PhD Thomas Benesch

MD Barbara A Blodi

MD Ursula M Schmidt-Erfurth
email: ursula.schmidt-erfurth@meduniwien.ac.at

Evaluation of OCT findings in Age-related Macular Degeneration : A Reproducibility Study of two independent Reading Centers

Background/aims :

To determine the reproducibility among readers of two independent certified centers, the Vienna Reading Center (VRC) and the University of Wisconsin-Madison Reading Center (UW-FPRC) for OCT images in age-related macular degeneration (AMD).

Methods :

Fast macular thickness scans and 6 mm cross hair scans were obtained from 100 eyes with all subtypes of AMD using Stratus OCT. Consensus readings were performed by two certified OCT readers of each Reading Center using their grading protocol. Common variables of both grading protocols, such as presence of cystoid spaces, subretinal fluid, vitreomacular traction and retinal pigment epithelial detachment were compared using kappa statistics. In addition, the intraclass correlation coefficient (ICC) was calculated for center point thickness (CPT) of values remeasured manually in the presence of alignment errors.

Results :

The reproducibility was dependent on the variable measured with a kappa value of 0.81 for the presence of cystoid spaces, 0.78 for the presence of subretinal fluid and 0.795 for the presence of vitreomacular traction. The lowest reproducibility was found for the presence of retinal pigment epithelial detachment with a kappa value of 0.51. The CPT was remeasured in 29 out of 100 scans at both sites with an ICC of the remeasured thicknesses of 0.92.

Conclusion :

OCT scan data are crucial in monitoring treatment efficacy in AMD clinical trials. For comparison of results obtained by different Reading Centers, the inter-Reading Center reproducibility is essential. Although the reproducibility is generally high, the reliability depends on the selected morphological parameters.

Introduction

Optical coherence tomography (OCT) is a rapid, noninvasive imaging method capable of generating high-resolution optical cross sections of the macula. A number of studies have elucidated the usefulness of OCT in the evaluation and follow-up of retinal conditions such as age-related macular degeneration (AMD) [START_REF] Puliafito | Imaging of macular diseases with optical coherence tomography[END_REF][START_REF] Baumal | Clinical applications of optical coherence tomography[END_REF].

OCT has also become a critical tool for assessing the morphologic response of the retina to therapeutic interventions. OCT-derived measurements of the central retinal thickness are important outcome measures in clinical trials for treatment of macular edema [START_REF] Rivellese | Optical coherence tomography after laser photocoagulation for clinically significant macular edema[END_REF][START_REF] Panozzo | Role of OCT in the diagnosis and follow up of diabetic macular edema[END_REF][START_REF] Massin | Optical coherence tomography for evaluating diabetic macular edema before and after vitrectomy[END_REF] and choroidal neovascularisation [START_REF] Hee | Optical coherence tomography of age-related macular degeneration and choroidal neovascularization[END_REF][START_REF] Rogers | Optical coherence tomography findings following photodynamic therapy of choroidal neovascularization[END_REF]. In addition to quantitative metrics, OCT is widely used for qualitative assessment to establish the presence of cystoid spaces, pigment epithelium detachments (PED), subretinal fluid (SRF) or vitreomacular pathologies such as vitreomacular traction (VMT). Recently, some of these variables have been used to determine the need for initial therapy and subsequent retreatment in clinical trials evaluating novel treatment modalities in the management of neovascular AMD [START_REF] Schmidt-Erfurth | European Society for Retina Specialists' Guidelines Committee (EURETINA).Guidance for the treatment of neovascular age-related macular degeneration[END_REF]. In current clinical trials of neovascular AMD , the most commonly used OCT system is the Stratus OCT (Carl Zeiss Meditec, Dublin, California, USA) with the the fast macular thickness map (FMTM) scan mode. However, the automated algorithms frequently misidentify the inner and outer retinal boundaries. This leads to an inaccurate thickness measurement, requiring a manual remeasurement. For the evaluation of the various morphologic characteristics of exudative AMD, typically the cross hair line scans have been obtained.

In multicenter AMD clinical trials, OCT scans are being sent to a central reading center (RC) for evaluation of morphology and accuracy of automated thickness measurements. Certified readers interpret retinal morphology according to standardized protocols and the center point thickness (CPT) is remeasured manually when the automated measurement is incorrect. Controlled RC procedures provide the most reliable method to obtain realistic and solid OCT data and to conclude on the value and benefit of OCT imaging in intravitreal therapy of AMD. Certified RCs, therefore, play an important role for defining OCT strategies.

Although data have been published regarding reproducibility within OCT reading centers [START_REF] Zhang | Optical coherence tomography reader agreement in neovascular age-related macular degeneration[END_REF], little information is available regarding reproducibility between RCs. However, FDA approved trials are usually performed as two independent trials in US and non-US sites with individual RCs responsible for each trial. Also, a comparison of strategies and results from different centers allows a solid evaluation of reliability and reproducibility.

However, the comparability and reproducibility of OCT gradings of readers from independent RCs using different grading protocols have not been evaluated to our knowledge.

Methods

Data collection :

In this retrospective study, cases were collected sequentially from a list of patients with the clinical diagnosis of AMD who underwent Stratus-OCT imaging of the macula at the retina unit of the Department of Ophthalmology of the Medical University of Vienna.

Approval for the analysis of OCT images was obtained from the Institutional Review Board of the Medical University of Vienna and of the University of Wisconsin-Madison. The research adhered to the tenets set forth in the Declaration of Helsinki.

All scans were acquired by certified OCT operators following pupil dilatation using the FMTM scan mode for quantitative measures of the CPT and the 6 millimeter cross hair scan mode for qualitative assessment of retinal morphology.

The FMTM protocol acquires six 6-mm radial lines consisting of 128 A-scans per line in 1.92 seconds of scanning, whereas the cross hair protocol includes two 6 mm lines (6-12 to 9-3 o'clock) at 512 scan resolution.

Scans were analyzed with the version 4.0 software (Carl Zeiss Meditec), all patient identifying data were removed and a consecutive patient number was given. Data for each case were sent as print outs to the UW-FPRC and as digital data files to the VRC. After completion of the grading procedure, the data files with the results were independently sent by the two RCs to the Department of Medical Statistics of the Medical University of Vienna for data analysis, keeping each of the RCs blinded from the results of the other RC.

Grading Procedure :

The OCT image sets were examined as print outs at the UW-FPRC. Standard grid templates for identification of the center point and the central millimeter, printed on a transparent stock were overlaid, on the scans and used for grading of location and extent of the variables.

A validated computer assisted grading software was used at the VRC. This software imports OCT scan data exported form the Stratus-OCT and allows the grader to electronically measure various parameters in the 6 mm cross hair and FMTM scans. The distances are calculated in pixels and using the dimensions of the B scan image, the pixels are converted into micrometers. A superimposed grid indicating the center point and the central millimeter was used for grading of location and extent of parameters.

A consensus reading was performed by two certified readers at each RC. The readers were trained according to the individual protocol of the related RC and certification was awarded on the successful completion of all requirements. All readers are actively participating in the grading of Stratus OCT scans of clinical trials evaluating treatments for AMD.

6 mm cross hair scans were graded for presence and location of cysts, subretinal fluid (SRF),pigment epithelial detachment (PED),vitreomacular traction (VMT). FMTM scans were graded for presence of alignment errors at the center point. Scans with with alignment errors of more than 25% of the retinal thickness at the CP in at least one of the six FMTM scans and with a standard deviation of CPT at >10% were manually remeasured using a calibrated ruler at the UW-FPRC and digitally at the VRC.

Definition of morphologic parameters :

Before the start of the study, staff from the two RCs discussed the grading parameters but did not thereafter interact. Cysts were defined as round, minimally reflective spaces within the neurosensory retina. SRF was identified as a non-reflective space between the posterior boundary of the neurosensory retina and the retinal pigment epithelium/choriocapillaris reflection. Localisation of cysts and SRF was categorized as : 1. being outside the central millimeter, 2. involving the central millimeter. In addition, central cysts were categorized as small (<200 µm), medium (<400 µm) or large (>400 µm).

PED was described as a focal elevation of the reflective RPE band over an optically clear or moderately reflective space. A PED was graded if the elevation was greater than 400 microns at the base and/or greater than 200 microns from the surface of the RPE band to the surface of the choriocapillaris. Location of PED was described as : 1. being outside the central millimeter, 2. involving the central millimeter .

VMT was identified if a thickened posterior hyaloid was observed to insert in the foveal/perifoveal area. No grading regarding extend of VMT was performed.

Both RCs followed their own grading protocols for the selected parameters. To compare gradings between the two RCs, a comparison key was developed.

Statistical Methods :

Analyses were conducted using the software package SPSS version 17.0 (SPSS Inc, Chicago, Ill). To quantify the reproducibility of manual remeasurements of the CPT in cases of alignment errors, the intraclass correlation coefficient (ICC) was calculated. In addition, the mean, standard deviation, standard erroror, Pearson p and Bias correction factor Cb was determined. For CPT remeasurements, absence/presence, location and extend of cysts, SRF, PED and VMT, the percentage of agreement and the kappa statistic were calculated. Kappa statistics were interpreted using the ranges suggested by Landis and Koch: (10) 0 to 0.20, slight agreement; 0.21 to 0.40, fair; 0.41 to 0.60, moderate; 0.61 to 0.80, substantial; and more than 0.80, almost perfect agreement.

Results

The study included 100 eyes of 89 patients with an average age (±standard deviation) of 67± 8 years (59 female, 41 male). The OCT scans showed a wide range of morphologic changes associated with AMD including drusenoid changes, serous, haemorrhagic and fibrovascular PED, SRF, intraretinal cysts, VMT or macular edema.

Manually remeasured CPT :

The CPT was remeasured in 29 out of 100 FMTM scans at both sites (in 35 scans at the VRC and in 37 scans at the UW-FPRC). Table 1a shows the reproducilbilty as to wheather or not a remeasurement was necessary. Table 1b shows the mean, minimum and maximum CPT, standard deviation, standard error, ICC, Pearson p and Bias correction factor Cb for the remeasured CPTs at both sites. Figure 1a shows the Bland-Altman plot of the differences in the remeasured CPTs (x axis: the average of the CPT values of the two RCs; y axis: the differences of the CPT measurements between the two RCs). Figure 1b shows the scatter plot of the same measurements.

Presence of cysts, SRF, PED and VMT

Results of gradings for presence of cysts, PED, SRF and vitreomacular traction are summarized in Table 2. The shaded cells represent consistent gradings. The highest reproducibility was found for presence of cysts with a k value of 0,81 (almost perfect) the lowest for PED with a k value of 0,51 (moderate).

The number of detected VMT was low with only 2 cases identified at both sites and one additional at the UW-FPRC.

Location and extend of cysts, PED and SRF

Results of gradings for location and extent of cysts, PED and SRF are summarized in Table 3a, 4a and 4b. Table 3b shows the reproducibility of thickness gradings of cysts at the center point. The shaded cells represent consistent gradings.

Discussion

The introduction of a novel diagnostic method such as OCT of the retina into clinical practice and the acceptance/approval of used parameters by the responsible institution, requires solid proof of the value and relevance of the acquired data. In order to assess Standardized assessment of morphologic changes in OCT scans by one central RC, is becoming a widely used method of determining outcomes in large clinical trials. Benefits of using a central RC include : Standardized data acquisition procedures at the clinical sites, standardized grading procedures by the use of certified readers and masking of readers to patient data regarding treatment assignment. An additional advantage is that the reproducibility of the readings can be easily assessed [START_REF] Zhang | Optical coherence tomography reader agreement in neovascular age-related macular degeneration[END_REF]. Measurement of retinal thickness and grading for presence, location and extent of intraretinal cysts, SRF and PED are currently defined as the primary morphologic outcomes in recent clinical trials evaluating novel treatments for neovascular AMD [START_REF] Fung | An optical coherence tomographyguided, variable-dosing regimen with intravitreal ranibizumab (Lucentis) for neovascular agerelated macular degeneration[END_REF]. Although these morphologic parameters have been similarly applied by several RCs, there are no studies comparing OCT gradings between them, despite the fact that RCs often cooperate in clinical trials. RCs developed individual grading protocols, software tools and measuring devices. There is only one study comparing results from two fundus photograph reading centers in grading cytomegalovirus retinitis progression [START_REF] Hubbard | Comparability of two fundus photograph reading centers in grading cytomegalovirus retinitis progression[END_REF]. Gradings were performed on light boxes using standard grid templates and measuring circles by an ophthalmologist grader at one RC and by a group of four certified graders at the other RC. Another study investigated agreement between clinician and RC gradings of fundus photographs of diabetic retinopathy [START_REF] Scott | Agreement between clinician and reading center gradings of diabetic retinopathy severity level at baseline in a phase 2 study of intravitreal bevacizumab for diabetic macular edema[END_REF]. The agreement between diabetic retinopathy severity classification as determined by ophthalmoscopy performed by retina specialists and gradings of stereoscopic fundus photographs at a RC was determined. However, to our knowledge no study has been published on the inter-reading center reproducibility of OCT gradings.

As more competing trials are being performed world-wide with increasing numbers of study participants, more RCs are being involved to satisfy the need for certified data analysis.

The comparison of consensus readings of different OCT parameters in AMD showed a high level of agreement between the VRC and the UW-FPRC. The reproducibility was, however, dependent on the specific morphologic parameters measured with a kappa value of 0.81 for the presence of cystoid spaces, 0.78 for the presence of SRF and 0.795 for the presence of VMT. The lowest reproducibility was found for the presence of PED with a kappa value of 0.51. Reproducibility of gradings for localisation and extend of the selected parametes was lower, but still "moderate" with k values of 0.54 for SRF and 0,51 for PED and "substantial" for intraretinal cysts with a kappa value of 0,78. These findings, based on the capability of trained analysts in a standardized setting also highlight the difficulties in identifying specific morphologic pathologies in general, e.g. lesions of the RPE clearly represent a challenge for diagnosis and grading, while the identification of intraretinal cysts appears to be more reliable. These observations from a certified center condition allow an estimation of the value of such data from studies not using certified procedures. These data appear even more relevant as that a digital grading method at the VRC was compared with a paper print grading method at the UW-FPRC. In general, all reading parameters were discussed by members of the two RCs and defined prior to the study but there was no interaction between the two RCs thereafter offering completely independent analysis. Limitations of this analysis include the restricted range of morphologic paramters included and the relatively small number of study cases. In addition, this study is based solely on Stratus OCT data, which is still the most commonly used OCT in research protocols to date, however, it will be probably replaced soon by Fourier domain OCT (FD-OCT) devices.

Stratus OCT imaging is based on 6 radial, cross-sectional scans, and the information therefore is limited to a few randomly selected locations and an overall low resolution of structural details. The FD-OCT uses a fast spectral-domain technique and performs scans in a raster pattern throughout the entire macular area, at a superior resolution of 5 μm in axial and 20 μm in transverse directions. As a result, the retinal morphologic features such as RPE or vitreomacular interface abnormalities can be imaged at all locations and in more details.

In AMD, there may be up to a 92% rate of inner or outer retinal boundary errors with subsequent incorrect segmentation for retinal thickness [START_REF] Domalpally | Quality issues in interpretation of optical coherence tomograms in macular diseases[END_REF]. Recently, studies have shown the superiority of the central subfield for analysis of the central retinal thickness when compared to the center point thickness because of reduced variability. [START_REF] Krzystolik | Reproducibility of macular thickness and volume using Zeiss optical coherence tomography in patients with diabetic macular edema[END_REF]. Although the majority of the FD-OCT platforms will allow recalculation of central subfield thickness measurements, only the center point thickness can be remeasured manually on Stratus OCT when the automated algorithm fails. The CPT measurements in case of alignment errors were highly reproducible between the two RCs with a kappa value of 0,70 and an ICC of 0,92. However the mean CPT remeasured by readers of the VRC was 22 µm higher than the mean of the reameasured CPT at the UW-FPRC. The agreement on thickness gradings of central cysts was "substantial" with a kappa value of 0,75. This high degree of measurement agreement is relevant for clinical trials in which lesion thickness is a primary or secondary study end point.

A constant high level of agreement between RCs probably requires the development of a quality control program and RCs may need to consider establishing and maintaining cooperative programs to harmonize their measuring and grading procedures and settings.

Otherwise, a comparison of results from trials analyzed by different RCs cannot be certain of the impact of possible differences between RCs. Optimal standardisation and reliability is, however, essential to identify the value of new surrogate parameters in the development of novel treatment strategies which also introduce appropriate strategies for patient management and follow-up.

Figure 1a

Figure 1b

  Herein, we determine the inter-reading center reproducibility for OCT image evaluation between an established European and an established U.S. Reading center, the Vienna Reading Center (VRC) and the Reading Center of the University of Wisconsin-Madison (UW-FPRC), respectively.

  treatment efficacy and to evaluate treatment indications, it is important to gather robust data sets in a controlled manner. Few prospective clinical trials have included OCT measurements in a rigorous fashion using a standardized protocol. Due to the limited availability of solid data, the technique, although widely used in clinical routine, has not yet obtained general acceptance as a gold standard method for monitoring patients with AMD, by institutions and the community. The data available from trials such as PRONTO (Prospective Optical Coherence Tomography Imaging of Patients with Neovascular Age-Related Macular Degeneration Treated with intraocular Ranibizumab) (11), EXCITE (Efficacy and Safety of Ranibizumab in Patients With Subfoveal Choroidal Neovascularization Secondary to Age-Related Macular Degeneration) (12) and SUSTAIN (Study of Ranibizumab in Patients With Subfoveal Choroidal Neovascularization Secondary to Age-Related Macular Degeneration) (13) are uniformly based on conventional time domain (TD) OCT using the Stratus-OCT.

Table 1a . No of remeasured FMTM scans

 1a

		VRC
	UW-FPRC	no	yes	Total no.
	no 57	6	63
	yes 8	29	37
	Total no. 65	35	100
	Kappa	0,70
	Standard Error	0,07
	95% CI	0,55 to 0,84

Table 1b . Manually remeasured Center Point Tickness

 1b

	No. of remeasured scans at both sites	29
	Intraclass correlation coefficient		0,92
	95% Confidence interval		0,845 to 0,96
	Pearson p (precision)		0,94
	Bias correction factor Cb (accuracy)	0,9824
		VRC	UW-FPRC
	Mean ( µm )	298	276
	Standard Error	21,3	22,5
	Standard Deviation	114,9	121,2
	Minimum	139	115
	Maximum	501	544

Table 2 . Presence of Cystoid spaces, SRF, PED and Vitreomacular traction

 2

				VRC
		Cystoid spaces		SRF
	UW-FPRC	no	yes	Total no. no	yes	Total no.
	no 65	8	73	52	8	60
	yes 0	27	27	3	37	40
	Total no. 65	35	100	55	45	100
	Kappa	0,81			0,78
	Standard Error	0,06			0,06
	95% CI	0,69 to 0,94		0,65 to 0,90
				VRC
		PED			Vitreomacular traction
	UW-FPRC	no	yes	Total no. no	yes	Total no.
	no 32	26	58	97	0	97
	yes 0	42	42	1	2	3
	Total no. 32	68	100	98	2	100
	Kappa	0,51			0,795
	Standard Error	0,07			0,20
	95% CI	0,37 to 0,65		0,40 to 1,0

Table 3a . Cystoid spaces -Location and Extent

 3a

	Table 3b. Cystoid spaces -Thickness at the Center Point
			VRC
	UW-FPRC	0	1	2			3	Total no.
	0	83	2	1			0	86%
	1	0	7	3			0	10%
	2	0	0	2			1	3%
	3	0	0	0			1	1%
	Total no.	83%	9%	6%		2%	100
	0 -Not applicable			Kappa			0,75
	1 -Small, <200 microns		Standard Error	0,09
	2 -Medium, <400 microns		95% CI			0,56 to 0,93
	3 -Large, >400 microns
			VRC
	UW-FPRC	0	1	2	Total no.
	0	65	0	0		73%
	1	2	3	0		5%
	2	6	2	22		30%
	Total no.	73%	5%	12%		100%
	0 -Absent			Kappa		0,78
	1 -Definite, outside central 1mm	Standard Error	0,07
	2 -Definite, central 1mm involved	95% CI		0,53 to 0,82

Table 4a . Subretinal Fluid -Location and Extent

 4a

			VRC
	UW-FPRC	0	1	2	3	Total no.
	0	52	5	7	1	65%
	1	2	6	2	0	10%
	2	5	3	17	0	25%
	3	1	0	0	0	0%
	Total no.	59%	14%	26%	1%	100
	0 -Absent			Kappa	0,54
	1 -Definite, outside central 1mm	Standard Error	0,08
	2 -Definite, central 1mm involved	95% CI	0,37 to 0,69
	3 -Cannot Grade

Table 4b . Pigment epithelial detachment -Location and Extent VRC

 4b

	UW-FPRC	0	1	2	3	Total no.
	0	32	2	23	0	57%
	1	0	4	2	0	6%
	2	0	1	35	0	36%
	3	0	0	1	0	1%
	Total no.	32%	7%	61%	0%	100%
	0 -Absent		Kappa		0,51
	1 -Definite, outside central mm	Standard Error	0,08
	2 -Definite, central mm involved	95% CI		0,36 to 0,66
	3 -Cannot Grade


Christophe Lino

Marc Christie

Fabrice Lamarche

Patrick Olivier

A Real-time Cinematography System for 3D Environments

Keywords: Descriptors (according to ACM CCS): I, 3, 6 [Computer Graphics]: Methodology and Techniques, Interaction Techniques

We propose an automated system that constructs a movie in real-time from a sequence of low-level narrative elements (eg. character actions and motions, and object motions). The system computes appropriate viewpoints on these elements, and performs cuts following user-defined cinematic conventions and styles. Viewpoints are selected within specific regions of the environment (the Director Volumes) using the intersections of Visiblity Volumes and Semantic Volumes (viewpoint partitions defined around elements of interest according to characteristic cinematographic views). We then reason over Director Volumes to choose where to position the camera, and when to perform cuts given cinematographic continuity rules. Visibility issues due to dynamic occluders are handled by a local visibility sampling technique based on hardware rendering. In building annotated viewpoint partitions that handle visibility, we propose both high-level representations and a means to reason over the editing process. This represents a novel and expressive approach that couples camera planning and editing, which stands in contrast to existing techniques that are mostly procedural and do not encounter for dynamic visibility in editing.

Introduction

The provision of fully automated camera control for interactive environments, including viewpoint computation, viewpoint planning and editing, is a complex problem that raises three important issues.

1. Such a system needs to be underpinned by a narrative model that both structures the way the actions are organized, and allows us to reason as to their relative importance and presentation. Following Young's bipartite model [START_REF] Young R | Story and discourse: A bipartite model of narrative generation in virtual worlds[END_REF], interactive narratives can be considered to have two levels: (1) the story, the chronologic account of events over time; and (2) the discourse, the way in which events are organized and presented to the spectator.

How can we encode cinematic conventions and relate

these to the narrative goals of the application ? In relation to camera control, these conventions are often expressed as a set of idioms, that is, camera configurations that can be assembled to bring a story to the screen in a way that allows the viewer to understand the spatial relations/context of each scene, and the sequence of actions occurring within it. For example, such idioms would describe conventional camera placements and movements for variety of settings involving two or more characters in dialogue. An automated camera control framework must be capable of characterizing these declaratively through the use of constraints and objectives that can be solved in real-time. 3. How to develop these efficient solution mechanisms which must address the innate complexity of well understood problems such as visibility determination and path planning ? For example, the simultaneous computation of the visibility of multiple targets in a dynamic environment requires a pragmatic solution which will provide some guarantee as to the efficacy of results. Facilitating jump-cut continuity editing, where a user's view switches between two different viewpoints, requires the ability to reason more than just to local visibility of the current camera. Similarly, selecting appropriate tracking shots involves planning a camera's motion in a dynamic environment whilst simultaneously taking account of the visibility of scene elements from viewpoints along the spatio-temporal path.

We propose a fully automated system that addresses two issues, the construction of a cinematically expressive movie in real-time from a specified sequence of low-level narrative elements (e.g. character actions and motions, and object motions). Our system computes appropriate viewpoints on these elements, selected within specific regions of the environment referred to as Director Volumes) that capture both visibility and shot information pertaining to the scene. We then reason over Director Volumes to choose where to position the camera, how to plan paths between viewpoints, and when to perform cuts following user-defined cinematic conventions, given cinematographic continuity rules and styles. Our system constitutes a novel and significant step towards the expressive coupling of camera planning and editing in dynamic 3D graphical environments.

The Elements of Automated Cinematography

Camera control in computer graphics has been received significant attention in recent year. Contributions span a range of subproblems, from viewpoint computation and the specification of shot properties, to camera motion control and the planning visually felicitous paths. To contextualize our formulation automated cinematography we considered a number of significant contributions across the broad spectrum of problems to be solved, from viewpoint computation and camera motion planning, to editing.

Viewpoint Computation

The ability to compute viewpoints with specifiable properties is of particular value in a range of applications, including visual servoing, medical and scientific visualization, and image-based rendering. For such applications, viewpoints typically maximize the visibility of salient features and highlight the spatial relations between object with a view to enhancing a viewer's perception and understanding of a scene. Vasquez et al. [START_REF] Vàzquez P.-P | Automatic view selection using viewpoint entropy and its application to image-based modelling[END_REF] proposed that a good view should maximize the amount of information about a scene for a viewer. Drawing on Information Theory they use the notion of viewpoint entropy to compute a minimal set of N good views of a 3D environment (i.e. a set that maximizes viewpoint entropy).

Others have considered the problem of viewpoint selection as one of information composition. Bares et al. [START_REF] Bares | Generating virtual camera compositions[END_REF] automatically generated camera shots using the composition heuristics of expert photographers. Camera constraints are specified using storyboard frames to indicate how a desired shot should appear. The 3D environment is then analyzed to determine camera parameter setting leading to a shot closely matching the specification. Christie et al. [START_REF]A semantic space partitioning approach to virtual camera control[END_REF] proposed a partitioning of space, into semantic volumes, which capture the regions around scene elements for which a camera will have a consistent set of visual properties (i.e. qualitatively equivalent shots). However, in general approaches to viewpoint computation approach the problem without considering the temporal evolution of a scene.

Motion Planning Existing methods for camera motion planning augment classical path planning approaches with features such as path continuity and visibility along the path. Planning techniques can be divided into two main classes: local approaches, that constrain the search to a reduced set of candidate solutions, and global approaches, which require the construction of, and search over, an a priori representation of the whole environment. In terms of local approaches, techniques based on potential fields has been proposed to plan camera's position in dynamic environments [START_REF] Beckhaus | Dynamic Potential Fields for Guided Exlporation in Virtual Environments[END_REF]; the remainder of DOF being set by fixing the field of view and pointing the camera at an object of interest. Image based visual servoing has also been used to compute camera's DOF based on constraints expressed in the camera's field of view [START_REF] Courty | Computer animation: A new application for image-based visual servoing[END_REF]. Li et al. proposed to use a lazy PRM attached to the object of interest (e.g. a virtual character) to reactively plan suitable camera paths [START_REF] Cheng C.-C | Real-time camera planning for navigation in virtual environments[END_REF]. Although such local approaches are responsive to local changes and dynamic elements of the scene (i.e. occluders) they typically fail to produce camera paths with properties that account for the global properties of a scene. Integrating global knowledge of the environment significantly improves the quality of the path but generally relies on an offline analysis of static environments. For instance, [START_REF] Nieuwenhuisen | Motion Planning for Camera Movements in Virtual Environments[END_REF] used a pre-computed PRM constructed over a complete environment in the automatic generation of camera movements. Within static environments, [START_REF] Andújar | Way-finder: Guided tours through complex walkthrough models[END_REF] combined global planning, a search through a dis-cretization (cells) of an environment, with a secondary local search of each cell. Recently, Oskam et al. [START_REF] Oskam | Visibility transition planning for dynamic camera control[END_REF] presented a single target tracking real-time system that uses a global visibility-aware roadmap together with an estimate of the pair-wise visibility between portions of the scene, and which adapts at run-time to dynamic occluders. Such global planning methods enable large-scale camera transitions and can be combined with local search to follow a target and avoid obstructed viewpoints.

Editing Incorporating the editing process (where and when to cut between viewpoints) in computer graphics applications has been relatively under-addressed [START_REF] He | The virtual cinematographer: A paradigm for automatic realtime camera control and directing[END_REF] [CAwH * 96] [CLDM03] [START_REF] Friedman | Knowledge-Based Cinematography and Its Applications[END_REF]. Typically this is realized as set of idioms (series of reference viewpoints linked to a given configuration of actors) using a procedural language (usually a finite state machine): when a given action occurs, apply the associated viewpoint transition. Such approaches fail to recognize the importance of narrative discourse level [START_REF] Young R | Story and discourse: A bipartite model of narrative generation in virtual worlds[END_REF]. That is, story elements (plot and character) are defined in terms of plans that drive the dynamics of a virtual environment, and discourse elements (the narrative's communicative actions) are defined in terms of discourse plans whose communicative goals include conveying the story world plan's structure. Integration of these narrative aspects has only recently been considered [ER07] [START_REF] Jhala | Cinematic Discourse Generation[END_REF]. However, the realization has again been procedural in character, a script (i.e. a series of actions) is taken as input and an automatical visualization of the scenario w.r.t. procedural cinematic rules is produced.

The procedural application of idioms often leads to highly restrictive editing rules and cannot furnish the expressive range of real world cinematography. Instead, series of idioms are deterministically applied to a story, and the idiom application architecture (e.g. finite state machines) makes idiom composition impossible when two actions are unfolding in parallel. Crucially, such approaches do not address low-level issues, such as visibility, that must be simultaneously accounted for. Indeed, the fact that no representation of an environment takes account of the exact visibility in the whole environment is a major issue in automated camera control. Therefore, automated cinematography must incorporate a high-level editing model, in which rules (to specify spatio-temporal context) and idiom preferences (i.e. style) are non-deterministically applied, and incorporate an integrated facility to reason as to visibility in camera movement and cut planning.

Overview

We propose a real-time approach to virtual camera control and editing that directly addresses the visibility of keysubjects for both static and dynamic occluders, and provides an expressive means to reason over idioms, shots and shot transitions. An overview of this process is given (fig. 1).

Figure 1: Overview of the camera control process: building and reasoning over occlusion-free Director Volumes.

Our system takes as input a 3D interactive environment (i.e. changes are not known beforehand) and a narrative pipeline that injects events in real-time.Each class of event has an associated set of viewpoints (Director Volumes) that convey the event according to established cinematographic conventions (i.e. idioms). At runtime, a viewpoint planner first selects the most relevant event and chooses the best idiom to convey this event by filtering the Director Volumes according to the visibility of key-subjects, thereby enforcing coherency in the sequence of viewpoints, and eliciting preferred volumes. A transition planner then determines appropriate viewpoint transitions, either by using a jump cut or a path-planning process between viewpoints.

Computing Director Volumes

As already described (section 2) real-time visibility computation is a prohibitively expensive process. We address this constraint by reducing the practical complexity through two assumptions as to the geometric properties of the environment. Firstly, we use the term actor to designate a key subject of an action, free to move in the environment and we estimate its visibility using 2D convex hull of the projection of its geometry onto the floor. Secondly, we restrict the setting to static 3D indoor environments which we represent as a 2D cell-and-portal structure [START_REF] Teller | Visibility preprocessing for interactive walkthroughs[END_REF]. Cells represent the rooms and portals represent doorways and connections between cells. Hard visibility constraints exist in relation to extremities of the portals (e.g. walls or doors). Each boundary between a wall and a portal supports a stabbing line that represents a separation between visibility and occlusion. By assuming that the indoor environment resides on plane (i.e. distinct floors) we extract a 2.5D topological representation. All geometric elements in the scene above a specified height (here 1.5m) are treated as occluders.

Computing Semantic Volumes

Visual composition refers to the process of positioning the camera to furnish an image with the desired set of cinematographic properties. At the core of our approach is a representation, Semantic Volumes, that aggregates viewpoints that give rise to the qualitatively similar visual properties. The boundaries of the Semantic Volumes define when viewpoint changes occur.

We have fully specified sets of shots for key configurations of actors, which we use to characterize our Semantic Volumes (Figure 2). For example, in the case of a dialogue between two actors, a semi-space defined one side of the line of interest (LOI) captures the range of positions in which we can formulate over-the-shoulder shots of the two actors We use a BSP to define these Semantic Volumes. This structure is dynamically and procedurally processed, and allows us to efficiently characterize and partition the environment into sets of viewpoints. Some regions of space however cannot be selected to position the camera, since the actors are fully or partially occluded. Thus, an information on the visibility of the actors must be incorporated. In the sections that follow, we present a method to dynamically compute this visibility information.

A Topological Representation of the Environment

Topological representations of an environment are spatial structures that supports fast visibility computation and path planning around (typically static) obstacles. For the class of environments we consider these should be capable of addressing scene elements such a large vertical occluders, such as walls and pillars, and be capable of addressing configuration that constrain visibility (such as bottlenecks in the environment).

We use TopoPlan (Topological Planner) [START_REF] Lamarche | Topoplan: a topological path planner for real time human navigation under floor and ceiling constraints[END_REF] to preprocess the static environment geometry. TopoPlan analyses the scene geometry and generates a topological map as a triangular cell and portal decomposition based on a 2D constrained Delaunay triangulation.

The resulting representation contains pertinent information as to large vertical occluders and visibility bottlenecks, and allows us to rapidly compute the potential visibility of an object by taking into account of these large occluders.

Computing Visibility Volumes

Our process to compute the visibility of the actors has two elements: (1) a dynamic analysis of the potential visibility based on the precomputed static 2D topology and the configuration of the actors; and (2) a 3D dynamic refinement step, using a local visibility sampling that take into account dynamic occluders (e.g. the actors themselves) and elements that are not considered into topological analysis of the environment (e.g. furniture).

Using the static representation

Generating good shots of static and moving actors, requires us to maximize shot properties that relate to them, and in particular to their visibility. Whilst full visibility and complete occlusion is trivial to compute, partial visibility, which constitutes a large proportion of the shots of an actor, is more difficult to efficiently calculate. We use an original visibility analysis method, based on the precomputed topology, to create a visibility cartography for an actor.

Numerous visibility culling methods already exist to establish fully or partially occluded polygons [CF92] [BHS98]

but generally such techniques neither helps us identify fully occluded volumes nor, in the case of a partially occluded polygon, provide it with a visibility degree estimation. Consequently, we instead base our visibility analysis on the use of stabbing lines to make a dynamic categorization of the visibility volume of an actor, by which we can also estimate a partial visibility degree. Where there is no occluder between a viewpoint and the actor (for example, any viewpoint within the same topological cell), the actor will always be fully visible, we therefore only focus on viewpoints that are located in other topological cells.

From a viewpoint p, two stabbing lines are defined such that each line is tangential to, but on opposite sides, of the convex hull of the actor (fig. 3) and are referred to as extremal points. We can rapidly compute the extremal points, in linear time, by observing that a point e on the convex hull is extremal if and only if the other points immediately preceding and following e are located on the same side of a stabbing line passing through e.

To determine the visibility of an actor a with a static occluder, where point p is an extremity of this occluder, the stabbing line associated to an extremal point e of the actor separates the region where e is visible and the region where e is occluded. Since the actor is fully included between stabbing lines, visibility categorization of the whole actor is then processed by combining visibility categorization of each stabbing line. Three regions are then obtained: a Figure 3: From a viewpoint, two stabbing lines are defined with respect to an actor. By combining both stabbing lines carried by an extremity p of an occluder, a categorization into a full occlusion region, a partial visibility region and a full visibility region of the actor is obtained. 2D visibility of an actor corresponds to a segment s linking its extreme points. From a viewpoint included in the region of partial visibility, the visible portion of the actor is evaluated by using relative angle between the plane P (passing by the viewpoint and the extreme point p of the occluder) and the stabbing lines of the actor. The visibility degree is then simply evaluated.

region where the actor is fully occluded, a region where the actor is fully visible, and an intermediate region where the actor is partially visible. From a viewpoint v, the 2D visibility of an actor corresponds to a segment s linking its extreme points. We thus propose to evaluate the visibility degree of the actor by computing the visible portion of s. By drawing a line through v and p, the degree of visibility of the actor can be estimated from the angle subtended.

Integration to topological analysis In reality the representation of the environment is based on the topological analysis and is a set of convex topological cells and portals. A portal comprises two points carried by one or more walls. Thus each point of a portal defines two stabbing lines for an actor, providing a subdivision of an adjacent topological cell into visibility cells. Moreover, since topological cells are convex, visibility cells will also be convex. The environment visibility characterization (Figure 4(a)) proceeds as: (1) the decomposition of topological cells into visibility cells by using stabbing lines ; and, (2) the recursive propagation of stabbing lines to adjacent cells.

Dynamic visibility analysis structure

In this context, a BSP characterization is used for each topological cell. A node represents a stabbing line, and a leaf corresponds to a visibility cell of the actor. This structure provides: (1) an onthe-fly cell subdivision, ; and, (2) a reduction of the search complexity when characterizing a given viewpoint in the topological cell. However, occlusion due to other scene objects, not typically considered to be part of the environment (e.g. furniture), has not been considered. In the following section, we show how these elements are incorporated in the dynamic visibility computation.

Using the dynamic environment

Christie et al. [START_REF] Christie | Occlusion-free Camera Control[END_REF] presented an 3D approach to the real-time evaluation of the visibility of multiple target objects which simultaneously computes their visibility for a large sample of points. The visibility computation step involves performing a low resolution projection from points on pairs of target objects onto a plane parallel to the pair and behind the camera. By combining the depth buffers for these projections the joint visibility of the pair can be rapidly computed for a sample of locations around the current camera position. This pair-wise computation is extended for three or more target objects and visibility results aggregated in a temporal window to mitigate over-reactive camera behaviour.

In order to plan the next 3D camera position, we propose to rely on this local method. Once a 2.5D volume selected w.r.t. its potential visibility, and considering the current camera position/path, we propose to extend this volume into 3D, and then to sample the locally defined 3D potential positions around the current camera position/path. The new camera position is finally chosen among the set of the non-occluded viewpoints.

Director Volumes

In the previous sections, we have computed volumes corresponding to a semantic information, and volumes corresponding to a visibility information on an actor. Combining visibility volumes and semantic volumes, by using the compatibility between both BSP structures, allows to obtain a BSP partitioning into semantic volumes augmented with a visibility information on actors. This is the first step towards camera editing. We propose here a volume intersection method, in the aim to extract camera director volumes. BSP fusion methods have been proposed for a long time [START_REF] Naylor | Merging bsp trees yields polyhedral set operations[END_REF]. We propose to intersect visibility volume relative to each actor, obtaining a multiple visibility volume partition (fig. 4(b)). The representation of semantic information as a BSP now allows to easily intersect it with visibility information, and to obtain a camera director partition of the whole environment (fig. 4(c)). This information composition now provides a good base to reason on camera director volumes (semantic volumes augmented with a visibility information) to either make a jump, move in the environment, or maintain the camera position. In the following we will present our editing method, providing the camera paths and jumps given as input to the dynamic resolution of non-occluded camera positions.

Reasoning over Directors Volumes

In the application contexts we envisage narrative elements are not known in advance and are continuously introduced at run-time. Our automated cinematography module is therefore required to: (1) maintain rhetorical consistency with past shots; and (2) interactively interrupt the current shot to handle new elements as they become more relevant. Unlike most previous approaches [HCS96,ER07,Jha09] we enforce shot coherency rather than simply react to new narrative elements.

Narrative elements in Virtual Cinematography

The central notion around which our framework is design is the narrative element, and its coherent translation into cinematographic idioms. A narrative element is a component of the discourse and conveys relevant information or an action from the story. Each narrative element has a specific purpose and its conveyance leads to changes in a viewer's cognitive or emotional state. Narrative elements cary in accordance with the nature of the story, ranging from prototypical actions (a character stands up, walks, talks to another character) to more subtle notions such as the display of a relationship between characters (e.g. show dominance, conflict or isolation). As narrative elements are indexed in time, we refer to them as events. The Narrative Pipeline (Figure 1) therefore interactively injects events into the cinematography module.

Our editing model is built upon the notions of idioms, coherency and style. Though some conventions are implicitly encoded within idioms, it is necessary to maintain coherency when the camera changes from one event to another, when visibility fails in the middle of an idiom, or in case of interleaved idioms (parallel editing). Style Style is defined as a number of preferences as to:

Idioms

(1) shot pacing (which is the rate at which cuts are performed); (2) the camera speed (which favors either static or dynamic shots); (3) the minimum actor visibility in a shot (the level at which a character is considered occluded); and (4) order in which idioms should be considered where more than one applies.

The selection and sequencing of shots is fundamental to a viewer's understanding of the spatial, temporal and causal context. The appropriate choice of the next shot therefore needs to comply with a number of general rules, according to which we remove incoherent Director Volumes when switching viewpoints:

• Action continuity: between two shots, coherency must be maintained in the apparent speed and/or direction of motion. • Screen direction: motion within the frame (up, down, left or right) must be maintained. • Matching spatial relations: key spatial relations pertinent to the current action must be maintain, for example, in a dialogue if a character looks right towards another (off screen) character in one shot, this second actor should look towards left in the subsequent shot.

Following these rules, our editing process proceeds through a sequence of filters over the available director volumes, incrementaly removing inappropriate volumes:

• filtering according to the idioms that relate to that event according to: (1) visibility, captured in Director Volumes (i.e. suppresses idioms whose semantic volumes are occluded); (2) style, through the application of preferences between idioms; and (3) the current viewpoint, each shot has a minimum/maximum duration; • filtering according to coherency (i.e. relative position in relation to the previous semantic volume). For example, jump cuts must be between viewpoints that subtend an angle of at least 30 degree on the subject.

Results and Discussion

We present two examples that illustrate the key features or our automated cinematography system.

Our first example comprises two characters, A and B with a doorway. The narrative elements are typical character interactions for which idioms are well defined and overlap in time to illustrate the parallel editing feature of related actions:

• Event 1: A moves towards B can be captured using a sequence of three shots: an internal or parallel shot of character A; an apex shot of characters A and B; and an internal or parallel shot on character B. • Event 2: A crosses a doorway can be captured using a sequence of two shots: a side shot, 3/4 rear, or rear shot; and a front, 3/4 front or side shot.

As the first event is triggered, the specified idiom is selected and the Director Volumes are filtered by merging Visibility and Semantic Volumes (here there is no coherence to maintain with previous shots). A viewpoint is then selected in one of the Director Volumes (1.1) which in turn will constrain the choices on the next shots (1.2 and 1.3) due to the line-of-interest coherency requirement. It is possible that the second event may be ignored, depending on the temporal overlap between idioms and the minimim duration of shots. However, once the new event is selected motion coherency is enforced due to the left-to-right motion of the subject on the screen, which filters the availables volumes for shot 2.1, The second scene takes place in an environment will multiple pillars. Two characters A and B are engaged in a discussion while walking in the environment. The only narrative actions are the dialogue utterances. This example displays demonstrates how our system overcomes some challenging problems. Firstly, around t = 1 s. utterances of characters A and B are very short. The system therefore ignores some of the events or postpones their shooting to avoid overly reactive cutting between shots. Secondly, around time t = 5 s., no apex shots are available to frame both characters. The filtering system therefore selects the next appropriate idiom in the list (an over-the-shoulder shot). Events and shots are displayed in figure 7.

Conclusion

In summary, we have presented a real-time cinematography system for virtual environments relies on an encoding of cin-ematic idioms and coherency rules to produce appropriate edits and camera paths from a set of narrative actions. Our method relies on a spatial partitioning providing a characterization into visibility cells (fully visible, partially visible, or fully occluded) and characteristic viewpoints (the semantic volumes). To improve the visibility estimation, we utilized a local visibility sampling technique to handle minor and dynamic occluders. We then used these cells to reason at a symbolic level on how shot transitions should be performed, utilizing a user defined encoding for style and cinematic convention. The expressiveness of our system stands in stark contrast to existing approaches that are either procedural in character or do not account for dynamic visibility.

Figure 2 :

 2 Figure 2: Example of semantic volumes for a key configuration of actors (adapted from [Ari76]).

Figure 4 :

 4 Figure 4: (a) Partitioning into visibility cells. Regions are divided into three groups: full visibility cells (white), full occlusion cells (black), and partial visibility cells (grey level) (b) Partitioning into multiple visibility cells (Two actors)(c) Semantic volumes linked to a configuration with two actors, and merged with the visibility information. Each semantic volume is colored, and the visibility degree is represented by the coloration degree of the cell.

Figure 5 :

 5 Figure 5: From story, extraction of a tree of possible idioms is processed, to plan the sequence of camera viewpoints

Figure 6 :

 6 Figure 6: Conveying two related actions in parallel while maintaining coherency over the line of action and the line of interest.

Figure 7 :

 7 Figure 7: Following two actors in a dialogue scene. Camera avoids reactive switches with short utterances and uses onecharacter shots when apex shots are not available.

  An idiom is a sequence of reference shots that convey a narrative element. In contrast to approaches that consider only a single solution viewpoint [HCS96, CAwH * 96], each shot is specified in terms of Semantic Volumes that relate to key subjects.

	Coherency Coherency affords compliance with cinematog-
	raphy conventions that require maintenance of the spatio-
	temporal context when the camera changes viewpoint.
	Conventions are well established in relation to the actors'
	motions, spatial locations, the line-of-interest and actions.

Nous proposons une méthode temps-réel d'automatisation de la construction d'un film à partir d'une liste d'éléments bas-niveau (eg. actions/mouvements des personnages, mouvement des objets). Notre système calcule des points de vue appropriés sur ces éléments, et effectue des coupures de montage selon des conventions cinématographiques définies par l'utilisateur et le style. Les points de vue sont sélectionnés dans des régions spécifiques de l'environnement (les volumes directeurs) en utilisant l'intersection de volumes de visibilité et de volumes sémantiques (partitions des points de vue définies autour des personnages principaux selon des placements caractéristiques de la cinématographie). Nous raisonnons ensuite sur ces volumes directeurs pour choisir où positionner la caméra, et quand effectuer des coupures étant donnés des règles de continuité cinématographiques. Les problèmes de visibilité dû à des occultants dynamiques sont gérés par un échantillonnage local basé sur des rendus. En construisant une partition en points de vue annotés qui gèrent la visibilité, nous proposons une représentation haut-niveau et un moyen de raisonner sur le processus de montage. Cela représente une approche originale et expressive qui associe la planification du positionnement et des sauts de la caméra, qui contraste avec les techniques existantes qui sont pour la plupart procédurales et ne prennent pas en compte la visibilité dynamique dans le montage.