A Giachetti

F Fornasa

F Parezzan

A Saletti

L Zambaldo

L Zanini

F Achilles

A E Ichim

F Tombari

N Navab

S Velasco-Forero

Shrec'16 Track: Retrieval of Human Subjects from Depth Sensor Data

Keywords: Descriptors (according to ACM CCS): I, 4, 8 [IMAGE PROCESSING AND COMPUTER VISION]: Scene Analysis-Shape

In this paper we report the results of the SHREC 2016 contest on "Retrieval of human subjects from depth sensor data". The proposed task was created in order to verify the possibility of retrieving models of query human subjects from single shots of depth sensors, using shape information only. Depth acquisition of different subjects were realized under different illumination conditions, using different clothes and in three different poses. The resulting point clouds of the partial body shape acquisitions were segmented and coupled with the skeleton provided by the OpenNI software and provided to the participants together with derived triangulated meshes. No color information was provided. Retrieval scores of the different methods proposed were estimated on the submitted dissimilarity matrices and the influence of the different acquisition conditions on the algorithms were also analyzed. Results obtained by the participants and by the baseline methods demonstrated that the proposed task is, as expected, quite difficult, especially due the partiality of the shape information and the poor accuracy of the estimated skeleton, but give useful insights on potential strategies that can be applied in similar retrieval procedures and derived practical applications.

Introduction

One of the most interesting application of non-rigid shape retrieval is certainly the one related to human subjects re-identification, that can have relevant applications, for example in security and surveillance. Recent papers [PSR * 14,WMKS * 15] demonstrated that quite good retrieval performances can be obtained for human bodies acquired with a whole body scanner. However, many real world applications could not be based on the acquisition of complete high resolution models of the subjects, but rather partial scans of low quality, like, for example, those that can be acquired with a depth sensor. Low cost depth sensors like Microsoft Kinect, Intel Realsense, etc., are now widely available and, despite some technical limitations due to the technologies used (IR structured light or Time of Flight sensing), they can now be used similarly to conventional cameras in surveillance and monitoring applications. This fact suggested us to test geometry-based shape retrieval methods on the practical and extremely challenging task of retrieving from a database partial models of a human subject acquired with a low end depth sensor given an example. Human reidentification depth sensor datasets have already been proposed in the Computer Vision community [BCDB * 12,MFB * 14] and it is surely interesting to evaluate the contribution of purely geometric methods on this kind of applicative tasks.

Data acquisition and proposed task

Models have been created by placing a depth sensor (Asus Xtion Live Pro) in a position simulating typical a surveillance acquisition setup, with the depth camera placed in an elevated position (about 2.2m. from the floor), looking down with an angle of 22 degrees with respect to the horizontal plane (see Figure 1).

With this setup, we acquired depth maps of a group of subjects in three different poses with three different clothing (two scans with different coats and one without) and two different illumination conditions (natural light and artificial light), for a total of 18 scans for each subject. For each subject we recorded the acquired point clouds and the corresponding skeletons provided by the OpenNI functions (Figure 2). Rough point clouds have been processed in order to transform the coordinate system in order to have the normal to the floor plane along the y direction and subject x-z position approximately in the origin. Finally clouds' points belonging to the floor and to the environment were removed and a smoothing procedure based on the original structured point cloud connectivity was applied. Participants were finally provided with the rough and smoothed point clouds as ASCII .ply files, the skeleton file with coordinates of 15 nodes (HEAD, NECK, LEFT SHOULDER, LEFT ELBOW, LEFT HAND, RIGHT SHOULDER, RIGHT EL-BOW, RIGHT HAND, TORSO, LEFT HIP, LEFT KNEE, LEFT FOOT, RIGHT HIP, RIGHT KNEE, RIGHT FOOT) and connectivity in .off format, and a triangulated mesh obtained with Meshlab [CCC * 08] implementation of the ball pivoting algorithm [BMR * 99] (Figure 4).

We acquired a total of 50 subjects (20-25 years old males and females), subdividing then the dataset in a training set of 180 models of 10 subjects and a test dataset with 720 scans of 40 different subjects. The training data set was also provided with label information and could in principle used to set algorithm parameters or train supervised methods.

Figure 3 shows examples of (cleaned) point clouds showing differences in the models of the same subject in the different acquisition conditions (lights, pose, clothing).

Participants were finally asked to send up to three dissimilarity matrices evaluated distances between all the shapes in the test set. Furthermore, Precision-Recall plots have been analyzed and from the PR curves the Mean Average Precision (MAP), e.g. the average of all precision values computed for each subject in the retrieved list was estimated. An analysis of the effects of pose and clothing on the retrieval scores of the different methods has then been performed, and will be discussed in Section 6. As basic method to characterize shape we propose the following descriptors:

Lengths of Skeletal Segments (LSS): we only used the lengths of the 14 skeletal segments as shape descriptors and used Euclidean distances to evaluate the distance matrix.

Statistics on Shape Points Clusters (SPC): we clustered the smoothed cloud points according to the closest skeletal segment and computed statistics on the point distribution. We used as descriptor components the average distance of the points from the segment (SPC Mean), the standard deviation of the distance (SPC STD) and the normalized concatenation of Mean and Standard deviation (SPC M+S). Euclidean distance was used to compute dissimilarity. As in some poses point clusters related to some skeletal segments are empty, we replaced the differences of the related components with the average of the well defined differences.

Participants and methods proposed

Only two groups participated to the contest. Unfortunately the difficulty of the task makes probably too difficult the use of standard geometry processing descriptors used in watertight mesh retrieval and the use of shape only information is not usual in the Computer Vision community. We received results and methods from the "TEAM TUM-EPFL", composed by F. Achilles, A-E. Ichim, F. Tombari, and N. Navab. and from Santiago Velasco-Forero (CMM MINES Paristech). In the following we describe the methods proposed by the participants.

submitted to Eurographics Workshop on 3D Object Retrieval (2016)

Step in Depth+Mesh+Skeleton classification by S. Velasco-Forero

For a given couple of (mesh,skeleton) denoted by (Mesh, Ske), tlhe goal was to produce a descriptor capturing the interaction between skeleton points in Ske and the mesh data Mesh calculated from depth information. Due to low quality of the mesh, in many cases Mesh contains holes and isolated part. However, Ske is complete by included hidden parts by symmetry. Thus, in this method, the author has projected Ske on Mesh and used the upper part of allpairs distance as descriptor.

Here is a detailed description of the proposed algorithm. Three measures of similarities have been then considered via Lpnorm distances, as follows,

Dist(a, b) = |Ske| ∑ i=1, j>i (|Upper(Da)(i, j) -Upper(D b )(i, j)| p ) 1/p
(2) with p = 1, 2, 1.5. The first two cases are well-known as cityblock(Manhattan) and Euclidean distances and are referred in results as DMS -C and DMS -E. The third is referred to as DMS -M (method using Minkowski distance).

submitted to Eurographics Workshop on 3D Object Retrieval (2016) 5.2. Shape parameter estimation using a ConvNet and confidence weighting by F. Achilles, A. E. Ichim, F. Tombari, N. Navab

In order to retrieve human shapes from depth data, the group parameterized human mesh models using blend shapes [PSS99, BRM08]. To build up the database, the mesh model is rendered in several depth images with varying viewpoint-and shape parameters. Additionally, motion capture sequences were used to animate the model and hence induce robustness with respect to pose changes.

The connection between depth input and shape parameters were learned using a convolutional neural network, which jointly estimates pose and shape from depth. During testing, the estimated shape parameters were used to build up the dissimilarity matrix. BodyNet (BN) was applied in three different variants which are specified as follows:

1. BN1 (naive) Each shape vector is subtracted from those of the other samples and the sum of squared distances (SSD) is taken as a dissimilarity metric. 2. BN2 (augmented) The input at test-time were augmented by applying translations and horizontal flipping. This way, 50 shape vectors were estimated for each sample, which allowed to compute the centroid of the resulting parameter distribution. Intuitively, the centroid resembles a more robust shape descriptor than a single estimation. SSD was again used for computing the dissimilarities between the respective centroids. 3. BN3 (confidence weighted) After applying the test-time augmentation of BN2, the variance in the distribution of estimated parameters can be used to impose a confidence weighting. The variance vector v of each sample was normalized as v =

v-min(v) max(v)-min(v) and w = 1-v was used as weight vector. With element-wise multiplication ( ), the dissimilarity computation for two samples s 1 , s 2 changed to

f d 1,2 = ∑ (w 1 w 2 ) (shapeVec 1 -shapeVec 2 ) 2 ∑ w 1 w 2 , (3)
such that shape vectors were primarily compared at the parameters that were estimated with a higher confidence.

Experimental results

As a first result, given the dissimilarity matrices submitted, we compute the global retrieval scores, that are reported in Table 1 The first interesting result that can be seen is that lengths of skeletal branches are not informative at all. This can clearly be due to the large error in the segment fitting.

Retrieval scores are low, except for the NN, that, however, is certainly biased by the similarity of the meshes of the same subject with same clothing. To see this we also tested the average retrieval scores in the two subsets obtained with single illumination conditions. Results are shown in Table 1.

Figures 6 and7 show the corresponding Precision-Recall plots, where it is clear that the recall values drop quickly when precision grows. DMS runs are consistently the best ones, showing that the NN FT ST E-m DCG mAp LSS 0,01 0,022 0,046 0,031 0,314 0,056 SPC_Mean 0,693 0,165 0,224 0,152 0,505 0,164 SPC_STD 0,771 0,145 0,194 0,131 0,488 0,151 SPC_M+S 0,842 0,180 0,233 0,159 0,528 0,179 DMS-E 0,893 0,235 0,305 0,207 0,605 0,246 DMS-C 0,925 0,247 0,319 0,216 0,620 0,262 DMS-M 0,931 0,249 0,325 0,220 0,624 0,267 BN1 0,381 0,110 0,156 0,106 0,427 0,110 BN2 0,870 0,163 0,205 0,140 0,512 0,170 BN3 0,907 0,170 0,211 0,143 0,524 0,174

Table 1: Retrieval scores obtained with submitted methods and baseline methods on the full models dataset.

combination of skeleton and shape holds a relevant amount of information even when the accuracy is low. The idea to estimate distances on skeletal points projected on the mesh might have reduced the error created by the inaccuracy of the skeleton estimation. DMS is also the only method tested that uses mesh information instead of point cloud data. This may also have had a role due to the removal of other noisy information from the original data.

It may, in any case, appear strange that the more sophisticated approach (BN) does not provide the best results, but, actually the procedure of estimating shape parameters from simulated renderings may suffer of limits in pose sampling. The methods is in fact pose-biased as revealed by our further analyses. Biases and peculiarities of the different methods are indeed quite different and reveal interesting information. In order to understand this, we analyzed some figures derived from the best run of each group. If we consider, for example, the percentage of wrong first neighbor retrievals of each one (see Table 3), we see that the behavior of the algorithms is quite different: BN tends to retrieve more likely subjects in the same pose as the input while DMS tends to retrieve most likely subjects with same clothing.

submitted to Eurographics Workshop on 3D Object Retrieval (2016) NN FT ST E-m DCG mAp LSS 0,026 0,025 0,047 0,036 0,268 0,059 SPC_Mean 0,172 0,094 0,150 0,092 0,408 0,088 SPC_STD 0,119 0,072 0,117 0,077 0,366 0,079 SPC_M+S 0,208 0,110 0,165 0,094 0,415 0,127 DMS-E 0,494 0,188 0,260 0,143 0,474 0,210 DMS-C 0,513 0,199 0,274 0,151 0,489 0,225 DMS-M 0,521 0,204 0,279 0,155 0,494 0,231 BN1 0,044 0,035 0,069 0,050 0,323 0,063 BN2 0,256 0,111 0,153 0,082 0,366 0,119 BN3 0,303 0,121 0,160 0,085 0,379 0,129 Table 2: Averaged retrieval scores obtained with submitted methods and baseline methods on data subsets acquired with same illumination conditions

Figure 7: Precision-recall plots for the retrieval with all the methods proposed on a subset of the original data with same illumination conditions

If we consider the first 17 shapes retrieved, corresponding to the first tier statistics, and we look at the number of correctly labelled ones, we find other interesting insights. The number of retrieved shapes of the same subject is always far from the ideal one (17), but however, quite higher than the results that would have been obtained by random sampling. If we consider the number of retrieved models with the same pose and clothing and compare it with the expected number resulting in random sampling, of the input we see an evident bias, especially for pose, but different behaviors of different methods. If we look at the correctly labelled retrieved models of this test, we see that all the methods are able to retrieve also examples of models of the same subject in different pose or with different clothes, with the exception of BN that seem not able to retrieve models in different poses. Table 5 shows the average number of correctly labelled retrieved shapes among the first 17 having different illumination, clothing and pose with respect to the query.

This fact tells us that, even if the task is quite hard, relevant information is encoded in the low quality scans and skeletons and probably the scores could be easily improved. One method of doing this could be simply merging the characterizations given by the different methods proposed here. This fact can be also deduced by looking at Figure 8. Here we plot confusion matrices obtained with classification of first (left), second (center) and third (right) retrieved models in the best runs of the different groups. Dark colors correspond to higher number of models retrieved, and dark spots not in the principal diagonals correspond to retrieved models not of the query subject. It is possible to see that errors of different methods are not often done on different subjects, meaning that the algorithms characterize different features.

Discussion

The retrieval performances obtained in our task show that, as expected, it is hard to retrieve instance of the same human bodies from low resolution depth maps allowing change of pose and clothing and using only shape information without color.

All the methods proposed could be, however relevantly improved, and this is quite obvious, as the amount of time available for the test was quite small. Statistics on point clusters (SPC) can be enhanced adding more estimations. As the accuracy of the given skeletal segments is low, this may be not used to estimate descriptors, but only for clustering, estimating as descriptors coefficients of cylinder or ellipsoid fitting for example. Furthermore, learning can be applied by training optimal feature space projection for classifying example data with known subject labelling. Similar considerations hold also for DMC descriptors.

BodyNet based methods could be improved in several ways, e.g. with a denser sampling of training poses. The huge effect of the augmentation step demonstrate the sensitivity of the estimated parameters on the input pose and the possibility of enhancing the results with simple heuristics.

Our analysis also showed that the different approaches proposed are in some sense complementary, each one performing better on different subjects and different conditions. This means that a smart feature fusion technique could also be successful in joining the different descriptors into a single one providing better scores.

Finally, supervised learning, that demonstrated a great effect in improving the retrieval performances of shape descriptors applied to whole human body scans [LBBC14] could surely be used also on to enhance the retrieval scores. As the dataset presents a sufficient number of models and a training set with different subjects with respect to the tested one is available, we think that interested researchers could surely test different supervised approach for this task with a relevant possibility of enhancing the retrieval scores.

In any case, we think that the insights coming from the analysis performed on the results can be extremely useful for the design of effective real-world applications.

It should be considered that the quality of the depth images is increasing and future generations of sensors and API may provide more precise reconstruction and improved algorithms could provide better skeleton estimates. Furthermore, it should be considered that, in real world applications, color information could be used as well as dynamic information, here not exploited. It is clear that, using the time evolution of the point clouds, it is possible to enhance the quality of the human body characterization obtained with a single depth sensor. A recent work by Ichim et al. [IT16] showed, for example, the possibility of reconstructing accurate parametric reconstructions of human bodies from time evolving depth images, while in this context parametric reconstruction quality was certainly limited by the use of a single shot approach.

Figure 1 :

 1 Figure 1: Acquisition setup

Figure 2 :

 2 Figure 2: Example of RGB image (left) depth map (center) and skeleton (right) obtained by the sensor.

4.

  Baseline methods by A.Giachetti, F.Fornasa, F.Parezzan, L.Zanini

Figure 3 :

 3 Figure 3: Top row: same subject acquired with different illumination conditions. Middle row: same subject in the three poses. Bottom row: same subject with three different coats.

1 .

 1 For each v ∈ Ske compute its closest point in the mesh, i.e. Proj Mesh (v) = arg min w∈Mesh ||vevery pair of points in the skeleton compute the pairwise

Figure 4 :

 4 Figure 4: Examples of triangulated meshes provided to participants together with the registered skeletal segments.

Figure 5 :

 5 Figure 5: Mesh data an its correspondent Ske (In blue). Results of the operator Proj Mesh (Ske) are shown in red.

Figure 6 :

 6 Figure 6: Precision-recall plots for the retrieval with all the methods proposed on the full dataset.

SPC_M+STable 5 :

 5 Average number of correct (same subject of the query) First-Tier retrieved models acquired in different poses, different clothing and different illumination conditions with respect to the query model.

Figure 8 :

 8 Figure 8: Top row: confusion matrices obtained with classification of first (left), second (center) and third (right) retrieved model in the best SPC run. Middle row: confusion matrices obtained with classification of first (left), second (center) and third (right) retrieved model in the best BN run. Bottom row: confusion matrices obtained with classification of first (left), second (center) and third (right) retrieved model in the best DMS run.

Table 3 :

 3 Percentage of wrong NN retrievals sharing the same pose or the same clothing of the query model with the three best runs for each group.

		SPC_M+S BN3 DMS-M
	same pose	76 %	95% 64 %
	same clothing 55 %	39% 74 %
	same lights	54 %	40% 64 %

Table 4 :

 4 Average number of retrieved models of same subject, same pose or same clothing among the first 17 shapes retrieved (1st tier), compared with the expected values in random retrieval.

submitted to Eurographics Workshop on 3D Object Retrieval(2016)


U Bortolozzo

A Montina

F T Arecchi

J P Huignard

S Residori

Spatiotemporal pulses in a liquid crystal optical oscillator

Keywords: numbers: Pacs: 05.45.-a, 42.70.Df, 42.65.Sf

A nonlinear optical medium results by the collective orientation of liquid crystal molecules tightly coupled to a transparent photoconductive layer. We show that such a medium can give a large gain, thus, if inserted in a ring cavity, it results in an unidirectional optical oscillator. Dynamical regimes with many interacting modes are made possible by the wide transverse size and the high nonlinearity of the liquid crystals. We show the generation of spatiotemporal pulses, coming from the random superposition of many coexisting modes with different frequencies.

Optical oscillators have been extensively studied in the past, in particular ring cavities with photorefractive gain [1,2] have attracted much attention, both experimentally and theoretically [3,4]. Here we present a different optical medium that results from the collective orientation of liquid crystal molecules tightly coupled to a photoconductive layer and pumped in a two-wave mixing configuration [5,6]. We show for the first time that, when inserted in a ring cavity, the liquid crystal light valve has a gain large enough to overcome the losses, thus resulting in an unidirectional optical oscillator. The wide transverse size and the high nonlinearity of the liquid crystal light valve allows us to explore dynamical regimes remained inaccessible up to date, where a huge number of modes are interacting. By changing the control parameters, the system displays either regular regimes, such as the alternation of Gauss-Laguerre cavity modes and the emission of out of axis large order modes, or space-time chaotic behaviors like the generation of high amplitude spatiotemporal pulses appearing in random space points.

In this letter, we focus our attention mostly on the spatiotemporal pulses. We show that they are confined along the three space directions and that they behave as coherent structures, with a characteristic size and life-time. We present a theoretical model for the liquid crystal oscillator and, for a set of parameters consistent with the experiment, we confirm the appearance of spatiotemporal pulses by numerical simulations. It is worth, here, to distinguish these spatiotemporal pulses from the solitary or localized structures that arise when a system presents bistability between a spatially homogenous and a pattern state [7]. While spatiotemporal pulses originate from a coherent superposition randomly taking place within a large number of coexisting modes, localized structures are isolated cells of a corresponding spatial pattern. Moreover, spatiotemporal pulses are confined both in space and time, while localized structures are confined only in space and can persist for indefinite time if not erased by a local, and large enough, perturbation. Spatiotemporal pulses are somehow reminiscent of the pulsating solitons predicted in the complex Ginzburg-Landau equation (CGLE) for mode-locked lasers [8] or of the random-phase solitons found in the nonlinear Schroedinger equation (NLSE) for nonlinear lattices [9]. In our case, due to the large time scale separation between the fast decay of the field in the cavity (τ c ≃ 10 -7 s) and the slow response time of the liquid crystals (τ ≃ 10 -3 s), the model cannot be reduced to a single equation for the electric field, the dynamics being slaved by the slow evolution of the refractive index. Nevertheless, the model contains the main ingredients for the creation of spatiotemporal pulses, that is, a large number of modes with partially uncorrelated phases and interacting through the nonlinear medium. Note that similar mechanisms are also at the basis of the collapsing filaments predicted in optical turbulence [10]. The experimental setup is schematically represented in Fig. 1. The cavity, that has a total cavity length L = 240 cm, consists of three high-reflectivity dielectric mirrors and a lens of f = 70 cm focal length, which enhances the mode stability providing a nearly spherical configuration. This configuration also ensures the presence of different longitudinal modes that oscillate at different frequencies and thus can lead to the formation of spatiotemporal pulses [11]. The gain medium is a liquid crystal light valve with one of the walls made of a thin slice, 1 mm thickness, of the photoconductive B 12 SiO 20 (BSO) crys-tal [6]. An external voltage V 0 is applied to the light valve by means of transparent electrodes deposited over the glass window and the external side of the BSO crystal. The BSO acts as a transparent photoconductor, thus modulating the voltage across the liquid crystals as a function of the intensity of the light passing through the cell [5]. The working point of the liquid crystals is fixed at V 0 ≃ 20 V , frequency 25 Hz. The thickness of the liquid crystal layer is d = 14 µm and the lateral size of the cell is 20x30 mm 2 . The cell is pumped by an enlarged and collimated (10 mm diameter) beam from an Ar + laser (λ = 514 nm), intensity I p ≃ 2 mW/cm 2 . The light amplification in the cavity is based on twowave mixing (2WM) interactions in the liquid crystals [6]. The pump beam and the cavity axis are an angle θ = 3 mrad. The cavity field E c is spontaneously generated and has a frequency almost equal to that of the pump field E p , so that the 2WM is nearly degenerate. In fact, we observe a frequency detuning of a few Hz, which is automatically selected by the cavity in order to maximize the gain and that corresponds to a refractive index grating slowly moving inside the liquid crystals. It is well known that a moving grating enhances the two-wave mixing gain for photorefractive crystals [START_REF] Gunter | Photorefractive crystals and their applications[END_REF], however this frequency shift was not observed before for a liquid crystal light valve. Also note that the liquid crystal light valve is a thin medium, so that the 2WM here takes place in the Raman-Nath regime [START_REF] Yariv | Optical Waves in Crystals[END_REF]. The Fresnel number of the cavity, which is the ratio of the area of the diffraction limiting aperture to the area of the fundamental Gaussian mode, is controlled by a diaphragm placed in front of the light valve and can be changed from F = 1 to approximately F = 500, which implies changing from a single transverse mode oscillation to a regime where a huge number of modes are interacting. An important feature, that differentiates our system from previous photorefractive cavities, is that changing the voltage V 0 changes the uniform refractive index n c of the liquid crystals, and thus the frequency detuning between the lowest order cavity mode and the pump beam. In order to compensate the detuning, the cavity adjusts its emission by changing the length of the out of axis transverse wave vectors [START_REF] Bortolozzo | [END_REF], so that the number of active modes can be changed by keeping the Fresnel number fixed and by varying the voltage V 0 .

For the purpose of visualization, a small fraction (4%) of the cavity field is extracted by a beam sampler and directed to a CCD camera. A lens is inserted on the path of the extracted beam in such a way that the transverse intensity distributions recorded by the CCD are equivalent to the intensity distributions at the entrance plane of the liquid crystal light valve, z = 0, by taking the z coordinate along the cavity axis. For a fixed Fresnel number F = 500, and by changing V 0 and the pump intensity I p , we have determined the experimental phase diagram, as reported in Fig. 2a. Cavity mode oscillations are in the larger grey area whereas spatiotemporal pulses are in the darker area. The low V 0 regimes are similar to those previously reported for low F photorefractive cavities, with the alternation of low order Gauss-Laguerre modes [1,2]. The transition to the high V 0 regimes is accompanied by the emission of high order and out of axis symmetrical modes. For intermediate values of V 0 and I p , a large number of modes interact through the nonlinear medium and give rise to the formation of spatiotemporal pulses, appearing as large intensity peaks over a lower amplitude and "speckle-like" background. Instantaneous snapshots of the transverse intensity distributions I c (x, y), where x, y are the coordinates in the transverse plane, are displayed in Fig. 2b-g. The transverse size of the oscillating field increases with V 0 , up to a large ring for high V 0 . The spatiotemporal pulses appear when a large number of modes is populating the whole size of the area illuminated by the pump beam. A typical snapshot corresponding to this case is displayed in Fig. 2g and the corresponding spatio-temporal plot is shown in Fig. 2h.

To investigate the dynamics of spatiotemporal pulses we have fixed V 0 = 20.3 V r.m.s. and I p = 2.0 mW/cm 2 and we have recorded several movies. The spatiotemporal pulses are identified by applying on each frame a threshold of for instance 3 times the average intensity, calculated as Īc = N k=1 I c (x k , y k )/N , with I c (x k , y k ) the intensity of the cavity field in the k-th pixel point and N the total number of pixels composing the image. The extension of the pulses in the z direction is investigated in the following way: we have divided in three parts the beam extracted from the cavity, directed each part through an optical path of different length and recorded the three transverse intensity distributions by using separate CCDs. The electronic delay time of the trigger being negligible with respect to the liquid crystal response time, we can consider that the CCDs simultaneously record the slaved field evolution in three different z positions. By inspecting the movies, we have found that the typical longitudinal extension of the pulses is approximately 30 cm, so that we have chosen to record the intensity distributions I c (x, y, z i ) i = 1, 2, 3 at distances corresponding to z 1 = 0, z 2 = 5 and z 3 = 32 cm. We cut the spatiotemporal plots along the x, y or t direction and we obtain the 1D spatial or temporal profile of the pulses. The spatial, temporal, profiles of a pulse are displayed in Fig. 3,4, respectively. It can be seen from Fig. 3c and Fig. 4b that at the distance z 3 the pulse has disappeared. By taking the half height width of the pulses with I c (x, y, z) > 3 Īc and by averaging over more than one hundred profiles, we find that the typical transverse size of a pulse is 250 ± 50 µm whereas its average lifetime is around 0.5 ± 0.1 s. The model, derived by coupling the Maxwell equations for the cavity field with a Debye relaxation equation for the refractive index [START_REF] Newell | Nonlinear Optics[END_REF], takes into account the Kerr nonlinearity of the medium as well as the two-wave-mixing mechanism of photon injection inside the cavity. The liquid crystal light valve is positioned in z = 0, perpendicularly to the cavity axis and r ⊥ denotes the coordinates in the transverse (x, y) plane. The refractive index n(r ⊥ , t) satisfies the equation τ ∂

t n = n c -n + l 2 0 ∇ 2 ⊥ n -α|E( r ⊥ , t)| 2
, where α > 0 is the nonlinear coefficient of the light valve, n c is a constant value determined by the voltage V 0 , E( r ⊥ , t) = E p e i( kP • r-ωpt) + E c e i(kcz-ωpt) + c.c. is the total electric field, with E p the pump amplitude and E c = E c ( r ⊥ , t) the complex amplitude of the cavity field. The intensity,

|E| 2 = |E p | 2 + |E c | 2 + (E * p E c /2)e -i k ⊥ • r ⊥ + c.c.
, gives rise to a refractive index change with two components, one varying slowly in space, the other corresponding to a spatial grating with wave number k ⊥ = k p -k c . We thus write the refractive index as n = n c -α|E p | 2 + n 0 + (n 1 /2)e -i k ⊥ r ⊥ + c.c.. By substituting this expression in the equation for n we obtain

τ ∂n 0 ∂t = (-1 + l 2 0 ∇ 2 ⊥ )n 0 -α|E c | 2 (1)
τ ∂n 1 ∂t = -1 + l 2 0 ∇ 2 ⊥ + l 2 0 | k ⊥ | 2 n 1 -2il 2 0 k ⊥ • ∇n 1 + -αE * p E c
for the slowly varying fields n 0 e n 1 . The wave equation for the cavity field is written by adopting, as usual, the slowly varying amplitude approximation and by considering a planar cavity. Due to the large scale separation between the medium response time and the cavity roundtrip time, we neglect the time derivative and obtain

∂E c ∂z = i 2k c ∇ 2 ⊥ + ik c n 0 W (z) + iδ -γ c c E c + (2) + ik c n 1 W (z)E p ,
where W (z) = 1 in the liquid crystal light valve, z = 0 to d, and 0 elsewhere, δ is the frequency detuning between the lowest order cavity mode and the pump field and γ c is the cavity loss rate. When deriving this equation we have considered n 0 and n 1 small. Inside the liquid crystal light valve we neglect the transverse Laplacian and we integrate in z the field equation, Eq.( 2), thus obtaining

E c (d) = e ikcdn0 [J 0 (2k c d|n 1 |)E c (0) + iJ 1 (2k c d|n 1 |)E p (0)] ,
(3) which accounts for Raman-Nath diffraction [START_REF] Yariv | Optical Waves in Crystals[END_REF] of the pump in the 2WM process. Outside the liquid crystals the field evolution is governed by diffraction, thus the transverse Laplacian has to be retained. By considering that the cavity field has to satisfy the periodic boundary conditions imposed by the cavity, and by taking into ac-count the presence of the lens, it can be shown [16] that the field at the entrance of the cell is given by

E c = i ∞ m=1 Bm ĈE p (0), (4)
where

B = e i L 2 2kc ∇ 2 ⊥ e -i kc 2f r 2 ⊥ e i L 1 2kc ∇ 2 ⊥ + iδ-γc c L e idkn0 J 0 (2k c d|n 1 |) (5) and Ĉ
= e i L 2 2kc ∇ 2 ⊥ e -i kc 2f r 2 ⊥ e i L 1 2kc ∇ 2 ⊥ + iδ-γc c L e idkn0 J 1 (2k c d|n 1 |), (6)
with L 1 the distance between the liquid crystal light valve and the lens, L 2 = L -L 1 . The interpretation of the above equations is quite simple. The first term of the sum, E (0) c = i ĈE p (0), is Eq.( 3) with E c (0) = 0 plus the field evolution in the cavity, thus accounts for the field generated by the pump in one cavity round-trip. The following terms sum up the field evolutions on the successive cavity loops. Inside the liquid crystal light valve, the operator B gives the same field equation as Eq.( 3) but with E p (0) = 0. At variance with usual treatments, where the mean-field approximation is used to eliminate the z dependence of the field, our model keeps this dependence, which is necessary to accounts for the formation of 3D patterns. Previously, the z behavior was considered for optical parametric oscillators [START_REF] Berre | [END_REF] and in dispersive self-focusing media [18], where optical bullets have been predicted as the 3D analogous of optical solitons. We have performed numerical simulations of the model equations, Eq.(1) and Eq.( 4) in the 2D+1 approximation, that is, by keeping the time and the x,z spatial dependences. The parameters are chosen from the typical values of the experiment, τ = 40 ms, l 0 = 30 µm,I p = 2 mW/cm 2 and α = 4 cm 2 /W . The number of terms in the sum is truncated to the number of round-trips given by the average lifetime of photons in the cavity. In Fig. 5 is shown the profile of a spatiotemporal pulse along z. In Fig. 6 we show the intensity distribution calculated in the x, z plane at a fixed time. In agreement with the experiment, spatiotemporal pulses appears as filaments confined both in the transverse direction and along the light propagation direction z.

In conclusion, we have shown a new type of nonlinear optical oscillator, which includes a thin liquid crystal light valve as the gain medium, and we have given evidence of the formation of spatiotemporal pulses.

U. Bortolozzo acknowledges a fellowship of the Ville de Paris. A. Montina acknowledges a fellowship of the Ente Cassa di Risparmio di Firenze, under the project "dinamiche cerebrali caotiche".

FIG. 1 :

 1 FIG. 1: Experimental setup.

FIG. 2

 2 FIG. 2: a) Experimental phase diagram and instantaneous snapshots taken at b) c) V0 = 10 V , Ip = 2.4 mW/cm 2 , d) e) V0 = 14 V , Ip = 2.0 mW/cm 2 , f) V0 = 25 V , Ip = 1.3 mW/cm 2 , g)V0 = 20 V , Ip = 2.0 mW/cm 2 ; h) space-time plot obtained for the same parameter values as in g).

FIG. 3 :

 3 FIG. 3: Spatial profiles recorded in a) z1, b) z2 and c) z3.

FIG. 4 :

 4 FIG. 4: Temporal profiles recorded in a) z2 and b) z3.

FIG. 5 :

 5 FIG. 5: Numerical pulse profile along z.

FIG. 6 :

 6 FIG. 6: Numerically calculated field distribution in the xz plane at a fixed time; the dashed line marks the lens position (L1 = 1.3 m, L2 = 1.1 m and f = 0.7 m).