Fabien Danieau
email: fabien.danieau@technicolor.com

Julien Fleureau

Technicolor Audrey Cabec

Paul Kerbiriou

Technicolor Philippe

Guillotel Technicolor

Nicolas Mollet

Marc Christie

Anatole Lécuyer

A Framework for Enhancing Video Viewing Experience with Haptic Effects of Motion

Keywords: H, 5, 1 [Information Interfaces and Presentation]: Multimedia Information Systems-Video; H, 5, 2 [Information Interfaces and Presentation]: Multimedia Information Systems-Haptic I/O;

This work aims at enhancing a classical video viewing experience by introducing realistic haptic feelings in a consumer environment. More precisely, a complete framework to both produce and render the motion embedded in an audiovisual content is proposed to enhance a natural movie viewing session. We especially consider the case of a first-person point of view audiovisual content and we propose a general workflow to address this problem. This latter includes a novel approach to both capture the motion and video of the scene of interest, together with a haptic rendering system for generating a sensation of motion. A complete methodology to evaluate the relevance of our framework is finally proposed and demonstrates the interest of our approach.

Introduction

New technology developments allow the creation of more and more immersive multimedia systems. 3D images and sound spatialization are now present in the end-user living space. But these systems are still limited to the stimulation of two senses, sight and hearing, while researches in virtual reality have shown that haptic perception seems to be strongly connected to the feeling of immersion [START_REF] Reiner | The Role of Haptics in Immersive Telecommunication Environments[END_REF].

In line with these works, we focus here on the way to enhance a natural video viewing experience with "realistic" haptic effects in a consumer environment. More precisely, our motivation is to develop a framework to make the user feel the motion embedded in the multimedia (Audio/Video) content he is watching. In this work, we target an automatic way to both produce and render the motion effects embedded in an audiovisual content. We especially consider the case of a first-person point of view A/V content, for which we expect the viewer to feel what the main actor is currently feeling in terms of motion. In this context, a comprehensive framework is presented and includes:

1. An approach to both capture motion and video of the scene of interest, 2. The method necessary to send and transform the original motion information to the back-end haptic device, 3. A haptic rendering scheme for generating a motion effect on a force-feedback device,

The remainder of this paper is as follows. Section 2 describes existing works proposing to produce or render motion effects. Section 3 presents the framework including a description of the main components of the proposed workflow. Section 4 details the methodology adopted to assess the proposed system as well as the obtained results. Finally section 5 provides conclusions and perspectives.

Related Work 2.1 Haptic effects for audiovisual contents

The production and the rendering of haptic effects for audiovisual contents are two typical issues identified in the pioneer theoretical work of O'Modhrain and Oakley [START_REF] O'modhrain | Touch TV: Adding feeling to broadcast media[END_REF].

The first issue deals with the content creation, i.e. how the haptic effects can be produced in order to be added to the audiovisual stream. In our context haptic effects are "motion effects". According to O'Modhrain and Oakley there are two ways to create such effects: the off-line creation and the real-time generation. In the first case haptic effects are synthesized and the editor of the content manually adds effects to the media. In the second case the effects are directly captured from physical sensors using specific device.

The second issue refers to the visualization of the content, more precisely to the rendering of the haptic cues. The haptic feedback should be rendered by a technology able to produce a wide range of haptic sensations. Moreover several constraints might appear if the content has to be displayed in a user's living space, potentially shared.

Only a few contributions exploring the interest of haptic feedback for audiovisual contents have been reported in the literature. Most of them rely on the use of vibrotactile devices (Rahman et al. [START_REF] Rahman | Adding haptic feature to YouTube[END_REF], Kim et al. [START_REF] Kim | A Tactile Glove Design and Authoring System for Immersive Multimedia[END_REF]) or force-feedback devices (Gaw et al [START_REF] Gaw | Haptically Annotated Movies: Reaching Out and Touching the Silver Screen[END_REF], Cha et al. [START_REF] Cha | Touchable 3D video system[END_REF]). But the haptic effects proposed by these contributions are relatively simple : artificial vibrations patterns or abstract force-feedback. To our best knowledge, no haptic effects of motion have been introduced.

Simulation of motion

In the context of motion rendering, motion simulators are well-known devices designed to make the user feel motion. They are intensively used as driving or flight simulators for learning purposes or in amusement parks. Most of them are based on the Stewart's platform [START_REF] Dasgupta | The Stewart platform manipulator: a review[END_REF]. It is a 6-DOF platform moving thanks to 6 hydraulic cylinders. Motion platforms are very immersive but they remain expensive for end-user customers and are not designed to be integrated in a user's living space. In a less invasive way, the sensation of motion can be also induced by a force-feedback device. Ouarti et al. [START_REF] Ouarti | Method for simulating specific movements by haptic feedback, and device implementing the method[END_REF] apply a force on the user hand and the system is expected to generate an illusion of motion with force-feedback.

While the interface is pulling the hand, the user feels moving forward. In our work we consider using a force-feedback device to render the effect of motion instead of a motion simulator. Such a technique takes advantage of its low cost and of its convenient size to be compatible with consumer applications.

In the context of motion capture, motion effects can be produced thanks to external sensors such as i) Inertial Measurement Units (see [START_REF] Brady | Relay: a futuristic interface for remote driving[END_REF] or [START_REF] Slyper | Action capture with accelerometers[END_REF] for examples in the context of radio-controlled cars and actor modeling respectively) or ii) the camera used to record the scene combined with motion extraction techniques (see Hu et al. [START_REF] Hu | A Survey on Visual Surveillance of Object Motion and Behaviors[END_REF] for a survey). Extraction algorithms can be helped by adding markers into the filmed scene (Sigal et al. [START_REF] Sigal | Humaneva: Synchronized video and motion capture dataset for evaluation of articulated human motion[END_REF] relied on this technique to perform motion capture) or by using a camera enhanced with infra-red capabilities (Microsoft Kinect device for instance). Thus several motion capture techniques exist but they are not designed to enhance a video viewing session. They target other applications such as human behavior modeling or human computer interaction.

Outline of the system

The system we propose is a comprehensive framework designed from the production of motion effects to their rendering on dedicated devices. Three main steps compose this framework as depicted in Figure 1.

Motion capture with physical sensors

The first step consists in i) capturing the motion effects for the renderer and ii) recording the audiovisual content that should be displayed simultaneously with the renderer. In the context of our work, a combined system making use of, on one hand, an Inertial Measurement Units (IMU) and, on the other hand, of a High Definition (HD) camera dedicated to sportive activities has to be designed.

The IMU we chose is the Ultimate IMU board which combines an ADXL345 accelerometer, an ITG-3200 gyroscope and a HMC-5843 electronic compass (cf. Figure 2-Left,B). The first component records the 3-axis accelerations of the board, the second one quantifies the rotation of the board around its 3 axes and the last component allows a geocentric orientation by giving an estimation of the local magnetic field. An additional micro-SD memory card may be embedded on the board and allows the recording of the three raw signals. A dedicated middleware has been developed and uploaded to the Ultimate IMU to set the recording process to 30Hz.

Complementary, a Camsports HDS-720p (see Figure 2-Left,A) was selected to record the scene corresponding to the current point of view of the actor. The camera is a HD bullet camera. It uses a 120 degrees wide-angled lens and integrates a built-in 4GB memory chipset. The spatial resolution of this device is 1280 x 720p at a frequency of 30fps. It is water-proof and is able to handle harsh environments. It finally integrates a mono-channel microphone. A complete integrated prototype combining the IMU, its battery and the camera has been developed. As the system is designed to be fixed on an actor (first-person point of view recording), it is enough robust to resist to different conditions of recording (cf. Figure 2-Right).

To synchronise the IMU and the camera, which are obviously independent, and do not offer possibilities of external synchronisation, a mechanical trick is used (very similar to the A/V synchronisation techniques traditionally used in movie making). Before each record, three little pats are given on the prototype which cause a fast and big peak in both the acceleration signals of the IMU and the audio stream of the camera. Basic signal processing techniques are then used to make those peaks match in both signals (variance-based threshold).

Processing of the captured motion signals

Once the different motion information have been recorded, they have to be processed to be compatible with the input of the haptic device that is used. As stated previously, Ouarti et al. [START_REF] Ouarti | Method for simulating specific movements by haptic feedback, and device implementing the method[END_REF] relied on a force-feedback device to make the user feel a sensation of self-motion. A force in the user hand was applied while the subject was watching a visual stimulus. The force (orientation and direction) is highly correlated to the motion embedded in the visual content and creates an illusion of self-motion even though the application point (hand) is different from the stimulating point where the accelerations have been recorded (chest).

In our context, we propose to replace the single acceleration of the approach of Ouarti et al. [START_REF] Ouarti | Method for simulating specific movements by haptic feedback, and device implementing the method[END_REF]

by using the three raw accelerations a[k] = {ax[k], ay[k], az[k]}

T recorded during the capture step at each sample k to modulate the 3-axis effort applied to the subject's hand. The methodology described here is generic and may be applied to other kinds of rendering devices. Though this approach is quite direct and requires very few processing, several points require specific attention (see Figure 3).

Gravity removal

The main processing to apply is linked to the gravitational component g included in the raw acceleration. This latter is quite important regarding the other external sources of acceleration and can mask some useful information needed to render a motion feeling. Our empirical observations showed that removing this specific contribution enhance the user experience. A two-step methodology is therefore applied to remove this component from the original signal. In a first step, the board orientation is estimated using the approach described by Sabatini [START_REF] Sabatini | Quaternion-Based Extended Kalman Filter for Determining Orientation by Inertial and Magnetic Sensing[END_REF]. This latter especially combines the use of i) a quaternion-based representation of the board attitude and ii) a dedicated extended Kalman filter to estimate the board orientation by fusioning the information coming from the three sensors (gyroscope, accelerometer, magnetometer). This operation allows to estimate the direction of the gravity (vertical) ng[k] in the accelerometer frame (frame A) at each time sample k. The raw acceleration vector is therefore updated by removing the quantity,

g ng[k], from each sample a[k].
The processing output signal

p[k] = {px[k], py[k], pz[k]
} T may be formalized, at each time sample k, by:

p[k] = a[k] -g ng[k]
(1)

Filtering and Additional effects

A filtering step is necessary to reduce the noise of the original signal. For practical reasons, the filtering is actually performed on the IMU. More precisely, the 3 sensors data were natively sampled at 200Hz but due to limitations with the writing speed on the embedded micro-SD, samples were averaged and down-sampled at 30Hz. This averaging step contributes to a low-pass filtering of the raw signal. Extra operations enhancing the signal for a better rendering for the end-user may be optionally addressed. They may be simple operations to remove artifacts or artificial modulation (reduction or amplification) of some parts of interest in the signal a to underline specific haptic events.

Haptic rendering of the motion effect

An open-loop rendering system was introduced to display the force-feedback. This way the signal p was simply rendered as a force vector F, defined as

F[k] = {Fx[k], Fy[k], Fz[k]} T .
However a few transformations of the signal p should be performed in order to render a force suitable for the haptic device (Figure 4). The processed signal p is sent to a haptic rendering algorithm controlling a haptic device which is, in our framework, a Novint Falcon device. This latter is a 3-DOF forcefeedback device, able to apply a force along three axes.

Axis alignment

To be rendered on the haptic device, an axis permutation of the signal p has to be performed to align the axes of the accelerometer (frame A) with the axis of the device (frame D). The associated permutation matrix is termed M A D .

In our context, the matrix simply switches the y and zaxis of A in D. A complementary step reverses the z-axis as the force-feedback device is placed in front of the user and it is supposed to pull the user's hand when the recorded acceleration is positive on the z-axis.

Gain

Then a scaling of the raw data is necessary to adapt the amplitude of the signal p to the renderer input range. The scaling factors sx, sy and sz for each axis are assumed to be constant (independent of the time sample) and empirically set according to experimental feedbacks. The associated diagonal scaling matrix is termed diag(sx, sy, sz) and in the context of our work sx = sy = sz.

The force rendered by the haptic device may be finally formalized by:

F[k] = diag(sx, sy, sz)M A D (p[k]) (2)

Multimedia player with haptic effects

Once the force F is computed, it has to be rendered by a haptic device. A force F is computed for each IMU sample and oversampled (point sampling, i.e. piecewise constant interpolation) to meet the requirements of the 1kHz haptic rendering loop frequency.

The rendering algorithm was integrated in a home-made multimedia player allowing the haptic rendering on a forcefeedback device and the A/V rendering in a synchronized way. The HAPI1 was used to control the Novint Falcon. This high-level C++ library allows to simply create force vectors which are then rendered on a force-feedback device.

The framework presented here allows to produce and render realistic haptic effects of motion. The rendering stage relies on a force-feedback device to generate a sensation of motion. Accelerations of the camera are rendered on this device. But the framework is not limited to one kind of haptic device. The positions of the IMU, and so of the prototype, are computed in the processing step in order to remove the gravity. These data could be used to render motion effect on a classical motion simulator which will reproduce the position of the camera.

User Evaluation

A dedicated experimental protocol has been developed to assess the impact of the haptic feedback on the user's Quality of Experience (QoE). QoE relates to the subjective user experience with a service or an application [START_REF] Kilkki | Quality of experience in communications ecosystem[END_REF]. The protocol designed to evaluate the QoE is presented hereafter as well as the associated results.

Experimental Setup

Capturing test sequences

Our motion capture prototype was used to create several samples of audiovisual contents enriched with haptic data. We identified 4 scenarios to represent different kinds of motion feelings (Figure 5). The prototype was placed on an actor's chest and we obtained the following videos sequences as well as the corresponding haptic data:

-Bike. The objective of this scenario is to capture lowamplitude movements. The actor is performing outdoor cycling and a succession of vertical movements with small amplitude are captured. (duration 61s).

-Horse. In this case the actor is riding a galloping horse and feels recurrent top-down movements. Highamplitude vertical movements are captured. (duration 60s). -Car turning. In this scenario, the actor is inside a car engaged in a roundabout. The centrifugal force makes him feel pushed on a side. The captured motion is felt as strong and long. (duration 45s). -Car Braking. This last scenario aims to capture a strong punctual movement. The actor is in a car strongly braking and he feels a strong force pushing him forward during few seconds. (duration 75s).

Duration of each sequence is around one minute.

Haptic setup

The playback system was composed of a 15" laptop and a Novint Falcon device (Figure 6). The user is comfortably seated in front of the computer where one of the captured video sequence is displayed and experiences haptic feedback by holding the Novint Falcon device in his dominant hand. Our home-made Haptic/Audio/Video Player was used to play the video in fullscreen mode.

Experimental Protocol

Variables

In order to evaluate the user's QoE for each sequence we defined three types of haptic feedback to be rendered with the video:

-Realistic Feedback. The captured haptic feedback, consistent with the sequences. -No Feedback. Only the audiovisual content is displayed. The goal of this condition is to measure the QoE of a classical A/V content. This will be used as a reference to evaluate the interest of a haptic feedback for a video. -Random Feedback. A random haptic feedback made of a low-pass filtered white noise (cutoff frequency Fc = 0.5Hz) of the same length and amplitude of the consistent haptic feedback. This feedback is not consistent with the video and will be used to evaluate the interest of providing a realistic haptic feedback.

Combining the whole set of possibilities we get 12 conditions (4 videos sequences x 3 haptic feedbacks) which were tested in each experiment in order to evaluate the QoE, our independent variable. These conditions will be presented in a random order to the participants.

Measures

A questionnaire was designed to evaluate the QoE of a video enriched with haptic feedback. It was built around the Presence [START_REF] Witmer | Measuring presence in virtual environments : A presence questionnaire[END_REF] and Usability [START_REF] Tullis | Measuring the user experience: collecting, analyzing, and presenting usability metrics[END_REF] concepts.

Presence aims at measuring how much the user feels being physically situated in a virtual environment. Witmer and Singer [START_REF] Witmer | Measuring presence in virtual environments : A presence questionnaire[END_REF] identified four factors to determine the presence: Control, Sensory, Realism and Distraction. "Control" determines how much the user can control and modify objects within the virtual environment. "Sensory" characterizes how each sensory modality is solicited during the interaction. "Realism" describes how much the environment is realistic and consistent with user's representation of the real world. "Distraction" identifies how much the user is disturbed by the apparatus used to create the virtual world. From this definition we focused on two factors: Realism and Sensory. As the user is passive with our system, Control factor was not relevant here. Moreover we did not measure Distraction in our QoE questionnaire but this aspect was interesting and was evaluated in a second questionnaire (post-test questionnaire).

Usability is defined by the norm ISO 9241-11 and aims at measuring how easy a system is to use. Three factors composed this concept: Efficiency, Effectiveness and Satisfaction. This latter measures how well the user enjoyed the system. "Effectiveness" means how well a user can perform a task while "Efficiency" indicates how much efforts are required. These two factors were not totally suitable for our system in the sense that it was not designed to perform a task. We preferred to use the term of Comfort to measure how well was the system to provide feedback. Satisfaction was however fully relevant in our situation.

Hence, the QoE of our system was evaluated by 4 items : Realism, Sensory, Comfort and Satisfaction (cf. Table 1). We defined only one question by item supposed to be rated on a five-point Likert-scale. The QoE is computed by the sum of these 4 items. This way the QoE questionnaire is easy to fill in and can be submitted for each condition.

A second questionnaire was designed in order to collect more information about user's feelings. It was composed by

Factor

Question

Realism

How much did your experiences in the virtual environment seem consistent with your real-world experiences? Sensory How much did the haptic feedback improve the interaction? Comfort How much was the system comfortable? Satisfaction How much was the system pleasant to use? Table 1: QoE Qestionnaire. Each question is rated on a 5-point Likert-scale from 1 (Not at all) to 5 (Totally) several open-ended questions supposed to be submitted at the end of the experiment. The participant was asked to label each haptic feedback, to indicate which one seemed the more realistic and why. To answer these questions he had the possibility to replay every sequences classified into 3 groups : Feedback 1, Feedback 2 and Feedback 3. He was also asked to tell how the system was tiring and comfortable to use, and finally to imagine several applications.

Procedure

15 participants have taken part to the experiment. They were aged from 21 to 59 (M=27.8 SD=9.7), 9 were Male, 1 participant was left-handed, 8 never used a Novint Falcon device. The whole experiment lasted from 30 to 40 minutes. The procedure for each user was organized as follows:

1. Fill in a consent form. 2. Fill in an information sheet. 3. Demonstration of the falcon and its capacities (forces it can provide). This step aims of reducing the "surprise effect" for novice users. 4. Presentation of the 12 conditions in a random order.

For each one:

4a. Experience the multimedia content (video sequence + haptic feedback), 4b. Answer to the QoE questionnaire.

5. Answer the post-test questionnaire. Possibility to retry every sequences.

Results

The data collected were 4 notes (Realism, Sensory, Comfort and Satisfaction; from 1 to 5) for each condition per participant. The sum of these notes gives the QoE per conditions per participants (Figure 7). The QoE per feedback per participant was also computed. This metrics is termed QoE All . The normality of the distributions was tested with the Shapiro-Wilk test and was rejected most of the time.

Hence non-parametric tests were used to analyze the results presented in this section. We first found that the QoE for each sequence follows the same pattern. The QoE All of the "Random Feedback" condition (M = 10.2 SD = 1.6) is higher than the QoE for the "No Feedback" condition (M = 7.5 SD = 2.1), and the QoE of the "Realistic Feedback" (M = 15.3 SD = 2.6) is higher than the random one. Figure 7 depicts this pattern. This result is significant according to the Friedman Anova (χ 2 = 24.71, df = 2, p < 0.05).

Figure 8 shows the mean score for each item of the QoE All for the three feedback conditions. The more realistic the feedback is the higher are the Realism, Sensory and Satisfaction items: Realism MNone = 1.3 M Random = 1.9 M Realistic = 3.8, Sensory MNone = 1.1 M Random = 2.4 M Realistic = 3.9 and Satisfaction MNone = 2.1 M Random = We also observed that the QoE All for the realistic haptic feedback remains the same for those who never used a Novint Falcon (M = 15.5 SD = 2.9) and for those who did (M = 15.25 SD = 2.5). The expertise of the participant do not affect the result significantly (Wilcoxon Mann-Whitney test, p < 0.05).

Discussion

The main result of this study is that QoE increases with haptic feedback and more particularly with haptic feedback consistent with audiovisual content. Moreover participants' expertise with the Novint Falcon do not affect the QoE. This observation let us think that our main result is not due to a "surprise effect". However the low score obtained by sequences with no feedback can be in part due to our experimental protocol. Whatever the condition, participants were asked to hold the Novint Falcon device in their dominant hand. Thus they might have been frustrated when they felt no haptic feedback. Obviously if there is a haptic device people are expecting haptic feedback.

We also observed that haptic feedback may change user's perception of the audiovisual content, especially if the meaning of the video is ambiguous. For instance one cannot see a bike in the bike sequence although a head of a horse is visible in the horse sequence as well as a part of a car in the two car sequences. During the experiment a participant thought that the bike sequence represented a buggy riding video because he felt that the haptic feedback (realistic feedback condition) was closed to his own buggy driving experience. Thus it appears that users build a mental representation of the multimedia content consistent with their own experience, and it is interesting to see how haptic feedback can influence this representation when audiovisual content is ambiguous. Another interesting behavior was observed while participant experienced video enriched with random feedback. Most of them tried to find a meaning for this haptic feedback, consistent with their own personal experience. This observation may explain higher QoE for random feedback than for no feedback. The phenomenon was particularly highlighted in the Car Turning and Car Braking conditions. Several participants supposed that the haptic feedback was mapped to the gear shift of the car. This can also explain why QoE for Random Feedback in these two conditions is better than in Bike and Horse conditions.

Finally participants reported in the post-test questionnaire to feel comfortable all along the experiment although the position of the arm and the hand-grip were reported as quite uncomfortable. This setup is obviously not suitable for watching a 2-hours movie. Of course, the limitations mainly due to the haptic rendering system and to the screen size, should be improved to reduce the user's fatigue in a out-of-the-lab context.

Conclusion and Perspectives

In this paper we presented a comprehensive framework to add realistic haptic effects of motion to a video. A novel approach to capture both audiovisual content and motion of the camera was first detailed. Then an original way to render this multimedia content was described. Our playback system relies on a force-feedback device to make the user feel the captured motion. Finally we presented a method to evaluate users' Quality of Experience with our system. Results show that the user experience increases with a realistic haptic feedback.

The work presented here focuses on the production and on the rendering of haptic effects. A third issue important to consider would be the distribution of the audiovisual content enriched with haptic effects. Cha et al. [START_REF] Cha | A Framework for Haptic Broadcasting[END_REF] introduced the notion of haptic broadcasting which correspond to the techniques to transmit haptic effects synchronized to the other components of the media. In a similar way, Walt [START_REF] Waltl | Enriching Multimedia with Sensory Effects: Annotation and Simulation Tools for the Representation of Sensory Effects[END_REF] described a data format to formalize haptic effects and to synchronize them to an audiovisual content. These techniques would be an interesting extension for this framework, especially in a consumer context where users used to watch videos through streaming platforms.

The first user study yielded promising results. Therefore, it would be first interesting to conduct a large-scale study in order to characterize more precisely the user's feelings and attitude with the system as well as the design of a more ecological setup. Moreover research efforts are necessary to determine when the user perceives a haptic feedback as consistent or not with an audiovisual content. This will help to finely design haptic effects necessary to trigger an immersion feeling. Second, sequences representing a diversity movements have also to be captured. More fundamentally, it is important to understand how users feel a sensation of motion through a force-feedback device. Finally it would also be interesting to evaluate the perception of effects with third-person point of view records.

To conclude, the framework we proposed in this paper allows to create more immersive audiovisual content by involving our sense of touch. This brings a new way to experience multimedia content and can enhance many viewing contexts such as movies, extreme sports videos or video games.

Figure 1 :

 1 Figure 1: Workflow Overview. Data are simultaneously captured by a camera and a motion sensor. Then motion data are converted into a signal suitable for the dedicated haptic renderer. Finally both video and motion are rendered simultaneously.

Figure 2 :

 2 Figure 2: Overview of the device capturing both video and motion. Left: The integrated prototype is composed by a (A) Camsports HDS-720p and a (B) Ultimate IMU board with its battery. Right: The prototype is fixed on an actor's chest and records his motion on 3 axes.

Figure 3 :

 3 Figure 3: Processing Overview. The gravity is removed from the accelerations motion signals captured. Optionally extra processing can be performed such as low-pass filtering.

Figure 4 :

 4 Figure 4: Pre-Rendering Overview. The signal from the processing step is aligned to the axes of the haptic device and then amplified.

Figure 5 :

 5 Figure 5: Tests Scenarios. Top Left -Outdoor cycling. Top Right -Horse riding. Bottom Left -Car engaged in a roundabout. Bottom Right -Car strongly braking.

Figure 6 :

 6 Figure 6: A participant experiences an audiovisual content enriched with haptic feedback.

Figure 7 :

 7 Figure 7: QoE of each sequence and haptic feedback. For each sequences, participants found that a realistic haptic feedback improves the experience. Interestingly a random feedback was more appreciated that no feedback.

2. 8 M

 8 Realistic = 4.1. These results are also significant (Friedman Anova: χ 2 = 24.0.3, df = 2, p Realism < 0.05; χ 2 = 26.68, df = 2, pSensory < 0.05; χ 2 = 23.05, df = 2, p Satisf action < 0.05). However Comfort appears to be relatively stable all along the experiment (MNone = 2.9 M Random = 3.2 M Realistic = 3.6). But this result is less significant (Friedman Anova, χ 2 = 8.79, df = 2, p = 0.012).

Figure 8 :

 8 Figure 8: QoE of each haptic feedback and details of the components. The Comfort component of the QoE remains the same whatever the feedback perceived. However the three others increase with no feedback, random feedback and realistic feedback respectively.

http://www.h3dapi.org/


Fabien Danieau
email: fabien.danieau@technicolor.com

Anatole L Écuyer

Philippe Guillotel

Julien Fleureau

Nicolas Mollet

A Lécuyer

Enhancing audiovisual experience with haptic feedback: a survey on HAV

Keywords: haptic interfaces, multimedia, audiovisual, user experience !

Haptic technology has been widely employed in applications ranging from teleoperation and medical simulation to art and design, including entertainment, flight simulation and virtual reality. Today there is a growing interest among researchers in integrating haptic feedback into audiovisual systems. A new medium emerges from this effort: haptic-audiovisual (HAV) content. This paper presents the techniques, formalisms and key results pertinent to this medium. We first review the three main stages of the HAV workflow: the production, distribution and rendering of haptic effects. We then highlight the pressing necessity for evaluation techniques in this context and discuss the key challenges in the field. By building on existing technologies and tackling the specific challenges of the enhancement of audiovisual experience with haptics, we believe the field presents exciting research perspectives whose financial and societal stakes are significant.

INTRODUCTION

I N 1962, Heilig introduced Sensorama, a system where one could watch a 3D movie, sense vibrations, feel the wind and smell odors [START_REF] Heilig | Sensorama simulator[END_REF]. This pioneering work opened the path for research in virtual reality, providing high-end interfaces that involve real-time simulations and interactions through multiple sensorial channels [START_REF] Burdea | Virtual reality technology[END_REF]. The importance of the sense of touch (haptics) to the immersion of the user in the virtual reality environment has been particularly studied. But the use of haptic technologies is much wider than the field of virtual reality: numerous applications have been found for them in medical, robotic and artistic settings. However, it is virtual reality that has triggered the development and evaluation of numerous haptic interfaces that enable the study of complex physical interaction with virtual objects [START_REF] Biggs | Haptic interfaces[END_REF], [START_REF] Hayward | Haptic interfaces and devices[END_REF].

In contrast, research and technology for audiovisual entertainment remains essentially focused on improving image and sound. Although the potential industrial impact appears to be significant, haptic feedback in a multimedia context, in which haptic feedback is combined with one or more media such as audio, video and text, remains underused. Only a few systems, known as "4Dcinemas", currently exploit this technology. However, the number of articles reporting the potential of haptic feedback for multimedia is increasing. In parallel, contributors working with virtual reality, such as Reiner [START_REF] Reiner | The role of haptics in immersive telecommunication environments[END_REF], have showed haptic feedback to be a key factor in user immersion, and thus of great interest to entertainment applications.

Recent works defend this view. O'Modhrain et al. have demonstrated that the benefits of haptic feedback observed in virtual reality are applicable to multimedia applications [START_REF] O'modhrain | Touch TV: adding feeling to broadcast media[END_REF]. Haptic feedback may open new ways to experience audiovisual content: the relation between users and audiovisual content is not limited to a passive context where the user just listens and watches but could enable physical involvement in a more immersive experience [START_REF] Magnenat-Thalmann | Haptics in virtual reality and multimedia[END_REF]. As well as physical sensations in parallel with the audiovisual content, the user could expect to receive a complementary piece of information or to intensify an emotion through haptic interaction, moving beyond simple immersion. The combination of haptics and audiovisual content becomes the complete medium of haptic-audiovisual (HAV [START_REF] Saddik | Haptic technologiesbringing Touch to multimedia[END_REF]) content, worthy of study distinct from virtual reality, with its own specific requirements and scientific challenges.

This fresh field of study introduces many new questions. How to deliver haptic technology to the user? To what extent can haptics affect the user's perception and understanding of the audiovisual content, and how can haptics be employed efficiently in conjunction with image and sound? What about the acceptability of complex haptic interfaces for users? How will the quality of the user experience be evaluated? Moreover, to what extent can the same haptic effect be experienced in different viewing scenarios (mobile tv, cinema or user living space, potentially shared) with possibly different devices?

The aim of this review is to gather and classify the results obtained in this young field of research by identifying its key challenges. We then propose future paths for research.

The paper is organized as follows. We first describe a general workflow for adding haptic effects to audio-visual content and build on this workflow to detail its three main stages: (i) production of haptic effects, (ii) distribution of haptic effects and (iii) rendering of haptic effects. We then discuss techniques for evaluating the quality of experience of users (QoE) in such systems. We conclude by discussing the developing prospects in the field.

A WORKFLOW FOR ADDING HAPTIC FEEDBACK TO AUDIOVISUAL CONTENT

This review is organized in a manner analogous to the typical workflow for video-streaming. This comprises three stages: (i) production, (ii) distribution and (iii) rendering (see Figure 1). We use the term "haptic effect" to designate the use of a haptic feedback in audiovisual content (a generalization of the term employed in the specific context of video viewing [START_REF] O'modhrain | Touch TV: adding feeling to broadcast media[END_REF], [START_REF] Yamaguchi | Automatic generation of haptic effect into published 2D graphics[END_REF], [START_REF] Cha | Touchable 3D video system[END_REF]).

The first stage in the workflow deals with the production of the content, i.e. how haptic effects can be created or generated in synchronization with the audiovisual content. Three techniques emerge from the literature: the capture and processing of data acquired from sensors, automatic extraction from a component of the audiovisual content (image, audio or annotations) and manual authoring of haptic effects. These production techniques and tools will be reviewed in Section 3.

The second stage in the workflow deals with the distribution of haptic effects. Current technologies allow mass distribution of media over networks, so there is a strong requirement for haptic effects also to be distributable in this way. This raises questions on formalizing haptic effects. The synchronized transmission of haptic effects over networks is termed haptic broadcasting [START_REF] Cha | A framework for haptic broadcasting[END_REF]. Various models, formalizations and techniques for this are reviewed in Section 4.

Finally an encoded haptic effect is rendered on a specific haptic device and experienced by the user. Section 5 offers an overview of the wide range of published techniques classified by the type of device (wearable, handheld, desktop or seat).

The evaluation of the user experience is a key aspect that cuts across production, distribution and rendering. Most interest to date has focused on the technical aspects of these three stages, but there is also a clear necessity to measure the quality of haptic-enhanced audiovisual experiences and provide common tools and metrics for such evaluations. This quality of experience (QoE see [START_REF] Kilkki | Quality of experience in communications ecosystem[END_REF]) is reviewed Section 6.

PRODUCTION

Production is the task of creating haptic effects in order to enhance audiovisual content. Three methods to create them have been reported in the literature: (i) capturing haptic effects from the real world using physical sensors, (ii) generating haptic effects by an automated analysis of audio and/or visual contents, and (iii) manually synthesizing haptic effects from scratch or by editing effects obtained with the previous methods. Haptic effects will be classified according to their perceptual characteristics (tactile, kinesthetic, and proprioception).

Haptic Perception

Haptic Effect Ref. Individual effects can be combined to create complex effects.

Haptic effects for audiovisual content

Waltl's classification of haptic effects is the most exhaustive yet published [START_REF] Waltl | Enriching multimedia with sensory effects: annotation and simulation tools for the representation of sensory effects[END_REF]. He details several sensory effects such as taste, smell and haptic. Haptic effects reported were temperature, wind, whole body vibration, water sprayer, passive kinesthetic motion and force (the user simply holds a force-feedback device), active kinesthetic (the user can explore actively the content thanks to a force-feedback device), tactile and rigid body motion (the whole body of the user is moved as in motion simulators). This classification linked each effect to a specific device.

In contrast, the classification we propose is based on haptic perceptual capabilities. Haptic feedback is often separated into two categories: tactile and kinesthetic feedback. There are three types of tactile stimuli: perception of vibration, of pressure [START_REF] Shimoga | A survey of perceptual feedback issues in dexterous telemanipulation. II. Finger touch feedback[END_REF] and of temperature [START_REF] Jones | The psychophysics of temperature perception and thermal-interface design[END_REF]. Two types of kinesthetic stimuli may be defined [START_REF] Jones | Kinesthetic sensing[END_REF]: perception of movement and limb position and the perception of forces. Finally, haptic perception may result from the motion of the user's own body [START_REF] Berthoz | The brain's sense of movement[END_REF]. Both the vestibular system and proprioception contribute to the perception.

We then propose a table summarizing haptic effects in HAV systems in which each category is mapped to contributions from the literature (see table 1). The reader may also refer to the guidelines for the design of vibrotactile effects [START_REF] Van Erp | Guidelines for the use of vibro-tactile displays in human computer interaction[END_REF] or haptic feedback in multimodal environments [START_REF] Hale | Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations[END_REF]. These individual effects can be combined to create more complex effects. For example, the haptic effect associated with an explosion might be defined with a combination of temperature and vibration.

Haptic effects are mostly used to represent physical events which occur in the scene (see references in table 1). The user perceives stimuli which are directly related to the audiovisual content (e.g. bumps when driving off-road), augmenting the physical event and the sense of "being physically present". However other

Haptic renderer

Automatic extraction

E N C O D E R E N C O D E R D E C O D E R D E C O D E R Figure 1.
Workflow for adding haptic effects to audiovisual content. In this review, we consider haptic effects as a component of a multimedia content. Effects are typically produced, distributed and rendered in the user living space in parallel to the audiovisual content.

aspects of an audiovisual content, such as ambiance, can be enhanced [START_REF] Kim | A tactile glove design and authoring system for immersive multimedia[END_REF]. The role of haptic effects in audiovisual content is analogous to that of audio in movies: audio is used for increasing the realism (sound effects) and to create ambiance (music). In movies, a clear separation is drawn between diegetic sounds (a sound for which the source belongs to the diegesis, the recounted story) and non-diegetic sounds (a sound for which the source is neither visible nor implied in the action, typically such as a narrator's comment or mood music). Nondiegetic haptic effects have similar potential. Non-visual content could be augmented by providing additional information that is perceived by the user.

The use of haptic effects to enhance ambiance or emotion is not straightforward. The haptic effect designer may explore results from research on affective haptics: recent works attempt to communicate affect with haptic feedback [START_REF] Smith | Communicating emotion through a haptic link: design space and methodology[END_REF] or trigger users' emotions with the help of haptic patterns [START_REF] Lemmens | A body -conforming tactile jacket to enrich movie viewing[END_REF], [START_REF] Tsetserukou | Affective haptics in emotional communication[END_REF].

Capturing haptic effects from the real world

One approach to creating haptic effects is to capture haptic effects related to an object or actor in a scene. Piezo-electric sensors can also be used to capture forces [START_REF] O'modhrain | Touch TV: adding feeling to broadcast media[END_REF] or vibrations but, most of the time, accelerometers are used to record accelerations and deduce forces applied to the targeted object. Brady et al. equipped a radiocontrolled car to capture accelerations on X, Y and Z axes [START_REF] Brady | Relay: a futuristic interface for remote driving[END_REF]. These recorded data were then directly transmitted and rendered to the user's control device. Recorded accelerations on X and Y axes control an embedded 2DoF force-feedback device and acceleration on the Zaxis drives a vibration device. Similarly, Danieau et al. placed a camera together with an accelerometer on an actor's chest to capture a first-person point-of-view video and the associated motion [START_REF] Danieau | A framework for enhancing video viewing experience with haptic effects of motion[END_REF]. Different scenarios were used to capture different kinds of movements: riding a bike, riding a horse and being in a car as it braked or turned. The videos were then replayed with haptic effects of force generated from the recorded accelerations. Kuchenbecker et al. recorded haptic events in a database to enable replay later [START_REF] Kuchenbecker | Event-based haptics and acceleration matching: portraying and assessing the realism of contact[END_REF]. The authors recorded accelerations resulting from the impact of a stylus on different materials (wood, foam). These accelerations were transduced into forces and replayed by a force-feedback device when the user touched virtual materials.

A second approach consists of capturing haptic effects related to a whole scene. Depth (or 2.5D) cameras have been used to build touchable images [START_REF] Cha | Touchable 3D video system[END_REF]. A more precise result could be obtained with 3D trackers [START_REF] Mcdaniel | A methodology to establish ground truth for computer vision algorithms to estimate haptic features from visual images[END_REF] but these devices are more expensive and the analysis of the scene would take longer. The problem of capturing haptic effects remains strongly constrained by the available hardware. In contrast to video and sound recording, only a limited number of devices exist, mainly accelerometers and 3D cameras with considerable variations in precision and cost.

Automatic extraction of haptic effects from audiovisual content

Haptic effects can also be created automatically by extraction. The key idea is to generate haptic effects which are consistent with media content in order to highlight specific aspects. For example a scene showing an explosion could be enhanced by haptic feedback such as vibrations and heat. Video and sound analysis might be used to detect explosions and then automatically add haptic effects.

Automatic extraction can occur in the production stage or in the rendering stage (see figure 1). In the production stage, haptic effects are automatically generated and can be modified by the creator. In the rendering stage, haptic effects are automatically generated on the client side.

Generation from visual content

A classical way to extract content from an audiovisual media consists in using video analysis techniques. Typical algorithms rely on feature detectors to extract points of interest inside an image to build derived information (e.g. object identification) [START_REF] Tuytelaars | Local invariant feature detectors: a survey[END_REF]. There are significant variations in the features they offer such as robustness to light variations, motion and computational cost. Some specific algorithms are dedicated to the detection of specific features such as faces [START_REF] Zhao | Face recognition: a literature survey[END_REF] or motion [START_REF] Hu | A survey on visual surveillance of object motion and behaviors[END_REF]. Detecting events is also possible. Video abstraction [START_REF] Truong | Video abstraction: a systematic review and classification[END_REF] and video data mining [START_REF] Zhu | Video data mining: semantic indexing and event detection from the association perspective[END_REF] have both been used for event detection but are restricted to specific subjects such as sports, where the potential range of events is limited. Once a targeted event is detected in the audiovisual content, a haptic effect could be generated. For instance, Réhman et al. have shown how to automatically extract events from a soccer game video and to display them with a vibrotactile device [START_REF] Ur Rehman | Turn your mobile into the ball: rendering live football game using vibration[END_REF]. Five vibration patterns were designed to represent the position of the ball on the field, to the team leading the game or to the goals. However the main focus was on how to render the effects rather than the video analysis. An earlier study was conducted in the context of sensory substitution, but the aim was to use haptic feedback to replace visual information rather than using haptics to enhance these data [START_REF] Bach-Y Rita | Sensory substitution and the human-machine interface[END_REF].

The difficulty of direct extraction of haptic information from video was pointed out by Mc Daniel et al. [START_REF] Mcdaniel | A methodology to establish ground truth for computer vision algorithms to estimate haptic features from visual images[END_REF]. To simplify the problem, the authors built a database which maps visual information (a picture of an object) to haptic information (the 3D shape of the object). The database is used to generate appropriate haptic feedback for each object identified from visual information.

Even if computer vision provides a broad range of tools, most techniques to analyze and generate haptic feedback have not been yet explored in detail. The robustness and adaptability of the detection algorithms remain typical issues in the field [START_REF] Tuytelaars | Local invariant feature detectors: a survey[END_REF].

Generation from audio content

Haptic effects can also be created from the audio content within audiovisual media. The main approach is to transduce an audible signal into a signal suitable for vibration motors. Chang and O'Sullivan used a bandpass filter to isolate frequencies compatible with a targeted vibration motor and then amplify and render the output signal on this device [START_REF] Chang | Audio-haptic feedback in mobile phones[END_REF]. This system was developed for mobile phones which then vibrate according to ringtones. The MOTIV 1 development platform from Immersion is a similar commercially available system. The "Reverb" module allows the automatical addition of haptic effects to any application using the output audio stream.

The approach selected by Nanayakkara et al. is even more direct and does not require any processing of the audio stream [START_REF] Nanayakkara | An enhanced musical experience for the deaf: design and evaluation of a music display and a haptic chair[END_REF]. The authors developed a chair for deaf 1. http://www.immersion.com/products/motiv/ people which renders music and vibration. The sound is played by speakers attached to the seat, which are specially designed to propagate vibrations to the surface they are attached to.

Most research follows this straightforward technique of the transduction of audio into vibrations. The approach could be extended by attempting to represent the information conveyed by the audio stream. Audio analysis techniques to extract specific features would then be useful. For example the system described by Zhang and Kuo permits the identification of music, speech and environmental sound in an audio signal [START_REF] Zhang | Audio content analysis for online audiovisual data segmentation and classification[END_REF].

Generation from metadata

Metadata can contain information about movements or physical properties of objects within the media. Yamaguchi et al. extracted data from a Flash2 animation to compute force feedback as the user explores the content [START_REF] Yamaguchi | Automatic generation of haptic effect into published 2D graphics[END_REF]. Since this format allows access to the geometry and position of elements within a 2D animation, it is possible to compute a force-feedback related to one of the objects in the scene. The authors defined a virtual mass for the targeted object and then computed a force-feedback relative to the acceleration and mass of this object. This technique can be applied to computer animations where a 3D model of the scene is available. But the system remains specific to animations and is not suitable for standard video. However some data formats allow for the description of audiovisual content. The MPEG-7 standard focuses on the description of multimedia content and can contain a description of movement within a scene [START_REF] Chang | Overview of the MPEG-7 standard[END_REF], opening many possibilities for the generation of haptic effects.

Graphical creation tools for synthesizing haptic effects

Although haptic effects can be created automatically, the need to create them before their integration with audiovisual content remains. Original effects may need to be edited. Neither of these functions can be automated.

Two main categories of graphical creation tools have been designed. The first allows users to specify the behavior of one or several actuators. In this case the designer has to use the same device as the end-user. In the second category the designer edits haptic cues that the user will perceive without referring to specific hardware. Various data formats and graphical tools are summarized in table 2.

Device-oriented effects

The behavior of an actuator is typically controlled by specifying a curve representing the amplitude of the stimulation (vibration or the force in time). The Hapticons editor [START_REF] Enriquez | The hapticon editor: a tool in support of haptic communication research[END_REF] was created to edit trajectory patterns called "haptic icons" on a 1DOF force feedback device (a knob). This kind of tool is already used in the industry. The aforementioned MOTIV 1 development platform provides a curve editor for designing vibrotactile patterns for various devices (mobile phones, gamepads, etc.).

Quite different graphical interfaces are used to edit the behavior of an array of motors. The user must specify the behavior of each motor in time. Representative examples have been developed by Rahman et al. [START_REF] Rahman | Adding haptic feature to YouTube[END_REF] and Kim et al. [START_REF] Kim | A tactile glove design and authoring system for immersive multimedia[END_REF].

User-oriented effects

The second type of graphical tool focuses on describing what the user should feel instead of defining how actuators should behave. This implies that the haptic rendering is handled by dedicated software.

Ryu et al. have created the posVib Editor to edit vibrotactile patterns [START_REF] Ryu | posVibEditor: graphical authoring tool of vibrotactile patterns[END_REF]. The intensity of the vibration felt by the user is represented by a curve.

The MPEG Rose Annotation tool was designed to associate sensory effects to multimedia content [START_REF] Waltl | Enriching multimedia with sensory effects: annotation and simulation tools for the representation of sensory effects[END_REF] (see Section 3.1). It allows the designer to tune sensory effects all along a movie. One or several effects can be added on a timeline which determines when they start and when they finish.

A different approach consists in describing material properties of objects within a scene. It implicitly determines what users feel when they touch objects. This type of tool resembles a 3D editor in which the author directly visualizes the 3D object being manipulated, but haptic (friction, stiffness) rather than visual properties are edited. We refer the readers to the presentation of the K-Haptic Modeler [START_REF] Seo | K-HapticModeler: a haptic modeling scope and basic framework[END_REF] as well as the HAMLAT tool [START_REF] Eid | HAMLAT: a HAML-based authoring tool for haptic application development[END_REF] which is a graphical editor for HAML (see Section 4.1.1).

DISTRIBUTION

The second stage consists in formalizing haptic effects into data to be synchronized, stored and transmitted with the audiovisual media. Even though the range and nature of haptic effects is not yet well defined, there have been several attempts at providing formalizations. These formats are summarized in table 2 which displays, when available, the associated authoring tools (see Section 3.4), and solutions to transmit haptic effects over the network (see Video Container column of table 2).

Data formats for haptic effects

Though there are several contributions which use dedicated formats to encode haptic feedback for audiovisual content, most approaches rely on generic formats. We consider two ways to formalize haptic effects: "deviceoriented" that defines the actuators' precise behavior, and "user-oriented" that describes effects from the user's point of view. The formats presented in this section are however suitable for both usages. Choosing between them only influences the way in which the rendering stage has to be handled: device-oriented data are used to control haptic devices directly, but user-oriented data must be interpreted. Since there is no obvious way to classify the encoding of haptic effects, we will use a per-format classification. We will detail contributions based on XML, a versatile description language, CSV a simple format to store data and VRML, a language dedicated to descriptions of 3D worlds. These formats are summarized in Table 2.

The issue of formalizing haptic effects has been solved by companies such as D-Box 9 or Immersion 1 who have developed commercial solutions for rendering haptic effects along with audiovisual content. D-Box have created a proprietary language to add haptic effects to a movie, called D-Box Motion Code TM . However, details of these formats are not currently available and the effects cannot be edited by the end-user.

XML-based

The first method of formalizing haptic feedback relies on XML language. The Haptic Application Meta-Language (HAML [START_REF] El-Far | Haptic Applications Meta-Language[END_REF]) is a generic format for describing haptic feedback which contains information about the haptic device, haptic rendering and visual rendering (see Listing 1). The purpose of this format is to be able to use any haptic interface with any virtual world, the system adapting the haptic rendering to the capabilities of the haptic interface used. This language is dedicated to virtual reality applications but it could be used to describe scenes in audiovisual content: objects and their location, geometry, haptic properties (stiffness, damping, friction), etc. This format respects the MPEG-7 standard which yields standardized tools to structure and organize descriptions of multimedia content [START_REF] Chang | Overview of the MPEG-7 standard[END_REF]. [START_REF] Saddik | Haptic technologiesbringing Touch to multimedia[END_REF]). Here, the haptic properties (stiffness, friction and damping) of a 3D cube are defined.

Closely related to video viewing, the Sensory Effect Description Language described by Waltl also relies on XML [START_REF] Waltl | Enriching multimedia with sensory effects: annotation and simulation tools for the representation of sensory effects[END_REF]. This language is designed to add sensory effects to any multimedia content: movies, video games, web content, etc. Users can create groups of effects and synchronize them with other media (see Section 3.1 for the list of effects). For each effect the designer can specify at least its intensity and duration. However devices Table 2 Overview of existing formats to edit and store haptic effects. Two types of haptic effect can be described: effects focused on what the user will perceive (user-oriented), and effects focused on how the actuators will behave (device-oriented). Most of the time a graphical user interface is designed to easily edit data. Some formats are to be embedded with a container enabling both audiovisual and haptic contents to be distributed via streaming platforms. and techniques to render effects are not specified. If converting an intensity into vibrations is simple, the rendering of a forward movement over 2 meters with an acceleration of 30cm.s -2 is less straightforward (see Listing 2). At the time of writing this paper, this language is close to being standardized by the MPEG working group as the MPEG-V format [START_REF]ISO/IEC FDIS 23005-3 Information technologymedia context and control -Part 3: sensory information[END_REF]. <s e d l : E f f e c t x s i : type=" sev : VibrationType " a c t i v a t e =" t r u e " i n t e n s i t y -range=" 0 100 " i n t e n s i t y -value=" 10 "/> 7 <s e d l : E f f e c t x s i : type=" sev : WindType" a c t i v a t e =" t r u e " i n t e n s i t y -range=" 0 100 " i n t e n s i t y -value=" 5 "/> ).

Here a "Move Toward" effect is defined followed by a group of effects combining "Wind" effect and a "Vibration" effect.

In an approach dedicated to instant messaging applications, Kim et al. [START_REF] Kim | A bidirectional haptic communication framework and an authoring tool for an instant messenger[END_REF] developed an XML-based format to exchange haptic feedback called "TouchCons". This allows users to send haptic messages such as vibration patterns or thermal effects. Two main files are used in this system. First, the Library XML describes a list of haptic messages and how they should be rendered (device used, intensity, duration). Second, the Device XML describes the available devices and associated capabilities. To send a message, the user chooses one from the Library XML file. When he receives a message, it is rendered according to the capabilities of the devices listed in the user's Device XML file. This framework could be used, instead of TouchCons, to describe haptic effects and then to send them to the end-user. The effects would be then rendered according to the user's devices configuration.

Finally XML representation can be used to determine the behavior of actuators directly. For example, Rahman et al. [START_REF] Rahman | Adding haptic feature to YouTube[END_REF] described vibration patterns of a vibrotactile array: the vibration intensity of each motor is specified in an XML file. This approach is simple but the effects described can be rendered only by a specific device.

CSV-based

Comma Separated Values (CSV) is a file format where data are stored in a simple text file separated by commas. Enriquez et al. relied on this format to store knob positions [START_REF] Enriquez | The hapticon editor: a tool in support of haptic communication research[END_REF]. This direct approach is simple but device specific. Danieau et al. [START_REF] Danieau | A framework for enhancing video viewing experience with haptic effects of motion[END_REF] also used this type of format but the authors stored information about the motion embedded in a video (acceleration in m.s -2 on 3 axes for each instant t). The motion effect is then rendered by the user's haptic device.

VRML-based

A third method used to describe haptic content uses VRML/X3D. This language serves to represent 3D worlds and contains information needed by visual rendering systems. Sourin and Wei [START_REF] Sourin | Visual immersive haptic rendering on the web[END_REF] proposed an extension of this language by adding haptic rendering techniques. One purpose of this language is to transmit virtual objects and their associated haptic rendering algorithms over the internet. In a similar way to HAML, this solution allows an audiovisual scene and the associated rendering techniques to be described.

The two techniques presented hereafter are based on the MPEG-4 BIFS format, also known as MPEG-4 Part 11 [START_REF]Information technology -coding of audiovisual objects -Part 11: scene description and application engine[END_REF]. BIFS, which stands for Binary Format for Scenes, is a scene description protocol based on VRML. Cha et al. extended this format to add haptic properties to a video [START_REF] Cha | Touchable 3D video system[END_REF]. The authors built a "touchable" movie, i.e. a movie in which spectators can feel the depth of the images using a force-feedback device. For each frame of the video the authors associated texture properties (stiffness, static friction and dynamic friction; see Listing 3).

Haptic-video containers

A container is a meta-file format that can hold several files in a single file which makes distribution easier. In the HAV context, a container is a single file that regroups haptic, visual and audio content. This stage is depicted in Figure 1. All components are compressed and synchronized into a single container for network transmission [START_REF] Wu | Streaming video over the Internet: approaches and directions[END_REF]. These containers are mainly used in multimedia applications to store both audio and visual content into a single file.

Several containers embedding audio and video exist (ogv, avi, mp4, etc.), but those combining haptic content are less common. A simple solution would consist of directly embedding the file containing the haptic data into a container that allows the attachment, such as the mkv container. O'Modhrain and Oakley used on the Flash standard to distribute videos enhanced with haptic effects [START_REF] O'modhrain | Adding interactivity: active touch in broadcast media[END_REF]. They integrated haptic feedback in their home-made animation and the media was played by a web browser embedding the Immersion Web plugin. This alternative is suitable for distribution purposes, although limited to the rendering capability of the plugin and to a specific type of audiovisual content (animation).

To take advantage of streaming platforms, one solution is to develop formats for haptic effects compatible with video containers that permit playback as they are downloaded. Some formats (see Section 4.1) were designed to support this streaming feature. Modified MPEG-4 BIFS [START_REF] Cha | Touchable 3D video system[END_REF] can be embedded into a classical MPEG-4 container. In a similar way MPEG-V is compatible with the MPEG-2 TS container [START_REF] Yoon | 4-D broadcasting with MPEG-V[END_REF]. This streaming challenge has been identified as haptic broadcasting by Cha et al. [START_REF] Cha | A framework for haptic broadcasting[END_REF]. This is a specific challenge different from the classical transmission of data for teleoperation [START_REF] Hokayem | Bilateral teleoperation: an historical survey[END_REF]. The purpose is not to control a device remotely but to send multimedia containing audio, video and haptic content. The two formats presented are at an early stage of development but demonstrate the possibility of haptic broadcasting.

RENDERING

Once the haptic content has been transmitted to the user, the haptic device needs to decode and render the content to provide the appropriate effect (in the same way that video is displayed on the screen or audio is rendered on the speakers, see Figure 1). Here we review a list of haptic interfaces proposed for "enhanced" video viewing.

Table 3 Overview of existing haptic devices used for enhancing audiovisual content.

We classified these devices into four categories: wearable devices, handheld devices, desktop devices and haptic seats. The results are presented in Table 3.

Wearable devices

Wearable devices are designed to be worn by as the user experiences audiovisual content. Typically they are composed of several vibrotactile actuators embedded into clothes, as detailed in Rahman et al. [START_REF] Rahman | Adding haptic feature to YouTube[END_REF]. This topic has been intensively studied for virtual reality purposes [START_REF] Lindeman | Wearable vibrotactile systems for virtual contact and information display[END_REF] and many devices have been designed.

In the HAV context, exploring the idea of enhancing live sports experience, Lee et al. [START_REF] Lee | Immersive live sports experience with vibrotactile sensation[END_REF] proposed a device with vibrotactile sensation through an assembly of 7x10 vibrotactors attached to the user's forearm. This prototype was used to render movements of the ball on the field during a soccer game. The tactile array was mapped to the field and vibrations were triggered at ball locations. According to the authors this device allows the user to better understand ambiguous game situations.

Kim et al. designed a tactile glove for immersive multimedia [START_REF] Kim | A tactile glove design and authoring system for immersive multimedia[END_REF], [START_REF] Kim | Effects of different haptic modalities on students' understanding of physical phenomena[END_REF]. It contains 20 tactile actuators per glove (4 per finger). The gloves are wireless-controlled and produce vibrotactile patterns as the user watches a movie. These patterns were first created, then synchronized with the video.

A tactile jacket has also been developed by Lemmens et al. [START_REF] Lemmens | A body -conforming tactile jacket to enrich movie viewing[END_REF]. They explored the influence of tactile devices on spectators' emotional responses, and designed a tactile jacket with 16 segments of 4 vibration motors covering the torso and the arms. Motors are activated following patterns related to specific emotions. For example, the feeling of love is enhanced by activating motors overlying the abdomen in a circular manner.

Palan et al. [START_REF] Palan | Tactile Gaming Vest (TGV)[END_REF] presented a vest with embedded vibration motors, solenoids and Peltier elements. The vest was designed to display three haptic effects as realistically as possible: gunshots, slashing and blood flow, with the motivation of improving video games experience. Similarly, a commercially available jacket manufactured by TNGames 3 produces effects such as explosions, gunshots or accelerations using 8 air cells.

While the embedded devices do not yield a significant change in weight or wearability of clothes, being composed of simple vibrotactile actuators, the range of possible haptic effects is rather limited.

Handheld devices

Users can experience haptic feedback through portable devices held in the hand. Vibrotactile technology appears well-suited for portable devices. For years, the gaming industry has used vibrating joypads to enhance immersion video games. Mobile devices (phones and tablets) are now equipped with vibration motors which may be used to enhance multimedia contents 4 . Using this technology, Réhman et al. relied on a mobile phone equipped with a vibration motor to display haptic cues related to a soccer game [START_REF] Ur Rehman | Turn your mobile into the ball: rendering live football game using vibration[END_REF]. Alexander et al. developed a prototype of a mobile TV providing tactile feedback using ultrasound [START_REF] Alexander | Adding haptic feedback to mobile tv[END_REF]. The device is a screen with a 10x10 array of ultrasonic transmitters set on the reverse side. The user holds the device to observe the audiovisual content and experiences haptic feedback through the fingers.

The remote control developed by O'Modhrain and Oakley is a different sort of handheld device that provides force-feedback [START_REF] O'modhrain | Adding interactivity: active touch in broadcast media[END_REF]. A gaming joystick was rehoused in a device resembling a remote control. Similarly Yamaguchi et al. used a computer mouse with a 2DOF force-feedback joystick [START_REF] Yamaguchi | Automatic generation of haptic effect into published 2D graphics[END_REF].

As with clothes-based devices, handheld devices cannot embed heavy actuators and so only a restricted range of haptic effects can be rendered. However, the use of a common device in the user living space (remote control, mobile phone) seems well on the way to popular acceptance.

Desktop devices

In virtual reality settings, force-feedback devices are mainly used to interact with virtual objects. The user can feel and often modify the displayed content. With video viewing the user cannot modify the content. The user receives haptic cues, sometimes while actively exploring the content, but the audiovisual content does not change. For example in the solution devised by Gaw et al. [START_REF] Gaw | Haptically annotated movies: reaching out and touching the silver screen[END_REF], the user holds a force-feedback device and is guided along a prerecorded path while viewing a 3. http://tngames.com/ 4. http://www.immersion.com/markets/mobile/products/ movie. The same technique was used by Kim et al. to enhance educational videos with a Phantom 5 device [START_REF] Kim | Effects of different haptic modalities on students' understanding of physical phenomena[END_REF]. In a similar way, Danieau et al. used a force-feedback device to enable the user to feel the captured acceleration associated with a video [START_REF] Danieau | A framework for enhancing video viewing experience with haptic effects of motion[END_REF] .

These devices have also been adapted to the task of "touching" images in a video [START_REF] Cha | Touchable 3D video system[END_REF]. In this study the user could actively explore the video content and received haptic feedback through a Novint Falcon device 6 .

Other desktop devices have been designed to convey haptic feedback to the user without direct contact. An example is a fan which generates air streams, simulating the haptic effect of wind. Associated with a thermal device, a fan may be used to create temperature variations [START_REF] Dionisio | Projects in VR virtual hell: a trip through the flames[END_REF]. Fans providing wind effects are commercially available. The Philips amBX system 7 generates not only wind effects but also lighting effects and enables keyboard vibration. This kind of device is simple to use, which results in more ecological interaction.

Contact with virtual objects is possible without directly handling a device. Hoshi et al. [START_REF] Hoshi | Noncontact tactile display based on radiation pressure of airborne ultrasound[END_REF] used ultrasound to exert pressure remotely on a user's skin. Their prototype was composed of an array of 324 airborne ultrasound transducers, able to exert a force of 16mN at a 20mm focal point diameter over a 180x180mm surface. This invisible surface is created at 200mm above the device. Combined with a 3D display system, the author succeeded in creating touchable floating images. A similar system has been previously developed by Suzuki and Kobayashi [START_REF] Suzuki | Air jet driven force feedback in virtual reality[END_REF], based on air jets.

Haptic seats

Our fourth device category is the haptic seat. The user sits on a modified chair and passively senses haptic effects.

Vibrotactile actuators have once again been used in a number of ways. The tactile blanket [START_REF] Dijk | A tactile actuation blanket to intensify movie experiences with personalised tactile effects[END_REF], a variant for the theme Lemmens' Jacket [START_REF] Lemmens | A body -conforming tactile jacket to enrich movie viewing[END_REF], is equipped with 176 actuators and displays vibration patterns designed to enhance the user's emotion.

More recently Israr and Poupyrev [START_REF] Israr | Tactile brush: drawing on skin with a tactile grid display[END_REF] embedded an array of 12 vibrotactile actuators in the back of a chair, with an original controller. The user experienced the tactile illusion of a continuous stimulus though the actuators were at discrete locations.

Several commercial products in this category are already available. One example is the "couch shaker" from The Guitammer Company 8 . This device uses actuators to shake the couch or sofa, operating like a subwoofer which propagates low-frequency vibrations to the couch instead of playing sounds. Some seating devices attempt to provide more complex effects such as motion. Typically such seats are fixed on actuators or motion 5. http://www.sensable.com 6. http://www.novint.com 7. http://www.ambx.philips.com 8. http://www.thebuttkicker.com platforms. For example, the D-Box 9 seat features 3 DOF: pitch, roll and heave.

Haptic seats are commonly encountered in theme parks or amusement arcades where they are typically used as motion simulators. Some of them even embed several devices to provide a wide range of effects (water spray, air blast, leg ticklers, etc. See the MediaMation 10 company.) These devices are not, however, adapted to the end-user living space and their cost is prohibitive for the mass market. In contrast, the D-Box 9 seat is a consumer product designed for living room use though it remains expensive. Devices based on vibrotactile arrays are also available but the range of tactile effects which can be rendered is quite limited.

QUALITY OF EXPERIENCE

Haptic effects aim at enhancing the audiovisual experience. This means that the quality of experience (QoE) of a video viewing session with haptic feedback would be higher than when haptic feedback is not present. But how should this hypothesis be assessed? Jain discusses the necessity of capturing the QoE for system evaluation [START_REF] Jain | Quality of experience[END_REF]. He underlines the difficulty of identifying and measuring the factors that characterize this metric due to its subjective nature.

Nevertheless Hamam et al. [START_REF] Saddik | Haptic technologiesbringing Touch to multimedia[END_REF], [START_REF] Hamam | A quality of experience model for haptic user interfaces[END_REF] have proposed an initial model for the evaluation of QoE in multimedia haptics which identifies four factors: rendering quality, and the user-centered measures of physiology, psychology and perception. The rendering quality is dependent on the quality of the visual, audio and haptic feedback. Perception measures describe the way the user perceives the system depending on the user's experience, fatigue and other factors which may alter the user's perception. Physiological measures identify how the system modifies the user's biological state, and psychological measures highlight changes in mental state. The authors detail an exhaustive list of parameters related to each factor (e.g. respiration rate, body temperature or blood pressure for physiological measures). While this provides a taxonomy of the different factors influencing the quality of experience, techniques to evaluate them were not presented.

In this section we detail classical techniques to measure the QoE of HAV systems. The typical approach found in the literature is a subjective measure based on questionnaires. Other approaches capture biosignals which provide an objective measurement of the user's physiological state from which emotional state is inferred.

Subjective measures: questionnaires

Most contributions in HAV rely on simple questionnaires to evaluate the impact of haptic feedback on the quality 9. http://www.d-box.com 10. http://www.mediamation.com of experience. Participants are usually asked to respond to questions on a Likert-scale. For example, Kim et al. [START_REF] Kim | A tactile glove design and authoring system for immersive multimedia[END_REF] studied the benefits of vibrotactile feedback for enhancing movies by using 4 general questions (Is this more interesting than movies? Is the tactile content easy to understand? Is the tactile content related to the scene? and Does the tactile content support immersion?). Ur Rhéman et al. covered the same aspects using a more detailed questionnaire [START_REF] Ur Rehman | Turn your mobile into the ball: rendering live football game using vibration[END_REF], while other authors have limited their analysis only to user satisfaction (see [START_REF] Waltl | Increasing the user experience of multimedia presentations with sensory effects[END_REF]).

A more elaborate approach characterizes the quality of experience using multiple factors. Hamam et al. [START_REF] Hamam | Effect of haptics on the quality of experience[END_REF] evaluated the five factors (extracted from their model described above) of realism, usefulness, intuitivism, fatigue and QoE. Danieau et al. [START_REF] Danieau | A framework for enhancing video viewing experience with haptic effects of motion[END_REF] identified 4 factors: sensory, comfort, realism and satisfaction. "Sensory" characterized how the haptic feedback contributed to the immersion. "Realism" described the realism of the simulation and how it was consistent with the user's representation of the real world. "Comfort" measured the overall comfort of the system (a proxy for acceptance). "Satisfaction" measured how much the user enjoyed using the system. These 4 factors were combined into one QoE measure. This variation highlights the need for a standardized questionnaire to better evaluate and compare different systems. Identifying the factors to be measured is not an easy task, but several have already been evaluated in a systematic way: comfort, interest, acceptance and satisfaction. They can serve as a basis on which to build a subjective measure of the QoE.

Objective measures: physiological data

Another approach to the evaluation of the quality of experience consists of measuring changes in the user's physiological state. The QoE cannot be directly determined from this measure, but it can be used to infer the user's emotional state, which contributes to the QoE. To the best of our knowledge, no work has been done using these techniques in the context of HAV systems. Nonetheless, inspiring results can be found in the context of virtual reality applications and video viewing.

In the context of virtual reality, Meehan et al. gathered heart rate, skin conductance and skin temperature data from subjects in a stressful virtual environment [START_REF] Meehan | Physiological measures of presence in stressful virtual environments[END_REF]. These measures helped to determine the user's feeling of "presence" and were compared to subjective users' self-reports (see [START_REF] Schuemie | Research on presence in virtual reality: a survey[END_REF] for a survey on "presence"). These authors suggest that heart rate has the strongest correlation with a sensation of presence. Skin conductance correlated less strongly and skin temperature not at all. Haptic feedback significantly improved presence.

Mandryk et al. observed biosignals in video game players to determine their user experience [START_REF] Mandryk | Using psychophysiological techniques to measure user experience with entertainment technologies[END_REF]. Skin conductance, heart rate, facial muscle activity and respiration rate were captured. The authors concluded that, for most participants, playing against a friend is more enjoyable than playing against the computer. The physi-ological measures were significantly consistent with the self-reported measures.

In a video viewing context, Fleureau et al. studied the potential of physiological signals for detecting emotional events [START_REF] Fleureau | Physiological-based affect event detector for entertainment video applications[END_REF]. Participants simply watched several videos while their heart rate, skin conductance and facial muscle activity were recorded. A detector based on machine learning techniques was designed. Given the user's biosignals, the system was robustly able to determine whether user were experiencing an emotional event and if this event was positive or negative.

The physiological chosen signals in these studies were mostly similar: heart rate, galvanic skin response, and facial muscle activity. All yielded significant results despite the various settings of virtual reality, video games and video viewing. The implications for the evaluation of HAV experiences is clear. Furthermore, closed-loop systems, in which physiological signals are used to control the nature and intensity of haptic events offer interesting possibilities for adapting the haptic effects to the individual user.

DISCUSSION

We have presented an overview of how haptic effects can enhance audiovisual content. Studies relevant to each stage of haptic production, distribution and rendering have been presented. Some of these studies present solutions that address all stages and may be seen as implementations of the generic workflow displayed in Figure 1. These general approaches are summarized in Table 4.

While the existing solutions clearly demonstrate how haptic effects can be used with audiovisual content using tactile or kinesthetic feedback, the studies reported do not explore combinations of effects (e.g. kinesthetic and tactile). This is mostly because the devices studied have generally had only one type of actuator. As a consequence, the range of effects that can be generated is narrow and the conjunction of effects is rarely explored, despite the significant potential benefits. Furthermore, there appears to be a gap between the use of portable haptic interfaces (wearable or handheld), conveying weak effects, and complex devices (motion simulators) which are not adapted to the user living space. There is a clear opportunity to design new haptic devices dedicated to audiovisual enhancement. This implies in turn a better understanding of the requirements for HAV systems, which seem to differ significantly from those in virtual reality systems.

Further research on user perception should be conducted to determine relevant haptic stimuli for effective audiovisual entertainment. The link between haptic stimuli and user experience is not thus far well established. Haptic effects are mainly used in a similar way to the use of haptic feedback in virtual reality: to immerse the user physically in the audiovisual scene. The use of haptic effects to enhance non-diegetic aspects of a video such as the ambiance or emotions has been little studied. This appears as a key challenge and opportunity in this nascent field.

The distribution stage also requires research effort. Each solution currently uses a different technique to formalize haptic effects in the absence of a common definition for haptic effects. Only half of the studies have proposed methods for the transmission of the media to a remote display device. But several techniques allowing haptic broadcasting are emerging. Multimedia containers embedding audiovisual and haptic effects are currently being developed and standardized (MPEG-V, MPEG-4 BIFS). The MPEG-V format is a promising standard for distribution currently under development by the MPEG group. The draft standard presents a list of haptic effects along with an XML-based method to describe them. This format is also designed to be compatible with streaming technologies. However the new standard will have to follow the evolution of this emerging field of study. New haptic effects and devices will almost certainly be developed.

In most solutions haptic effects are synthesized: authors manually create and synchronize haptic effects to the audiovisual content. Each solution currently offers a different technique for editing haptic effects, though general editing tools may arrive with the advent of new standards. The automatic extraction of haptic cues from visual content has also been reported. Such cues are currently limited to specific audiovisual content: soccer games following pre-defined rules, and animations where the position and geometry of objects is already known. The automatic extraction of haptic effects for any audiovisual content remains a complex task, and more work will be necessary to adapt current algorithms to this new purpose. Extraction can be facilitated by metadata that describe the content of the media, but extracting haptic effects from videos is a new challenge for which new specific techniques will have to be designed.

One final aspect to be discussed in this review is the quantification of the benefits lent to audiovisual content by haptic effects. Some of the studies presented here have conducted user evaluations, mostly based on questionnaires. Most show that haptic effects enhance the user experience but the various studies are heterogeneous and hardly comparable. There is pressing need for shared validated tools to evaluate this quality of experience.

CONCLUSION

In this review we have explored the possibilities provided by haptic feedback for enhancing audiovisual content. Several trends can be identified within this emerging field. The studies presented have been arranged against a generic workflow and the key challenges that pertain to this new way of experiencing videos identified.

The first stage, related to production of haptic effects, is the identification and generation of haptic effects which must be delivered to the user during the display of the media. We detailed different formats to store and synchronize haptic effects to audiovisual media, from a simple text-based representation to standardized XML formats. The key issue is the creation of haptic feedback. While a number of authoring tools are available, these effects may also be captured from physical sensors or generated from an other part of the media (video, audio or metadata).

Once the media has been enriched with haptic effects, it must be sent to the user. Media streaming platforms to distant users is now a common method of distribution. This stage is dependent on the way haptic data are stored. Though these issues are largely solved for audiovisual media, there are few standards for media with haptic effects. However some pioneering contributions have demonstrated the feasibility of this approach.

In the last stage the user perceives the media through a haptic device. These haptic interfaces are generally designed and dedicated to the purpose of displaying haptic cues during video viewing.

The results of our survey suggest that research effort is needed in the design of data formats and technology for distributing HAV content. The development of haptic media creation tools is also necessary. This may lead to a new type of professional activity in the cinema industry. Just as 3D movies now need "stereographers", so will new HAV content require "haptographers". Moreover the development of tools to evaluate the quality of experience and the acceptance of such systems is mandatory. Tackling the challenges of this young but promising field of study will yield new tools and methods for adding haptic content to multimedia, leading to a more compelling user experience in combination with audiovisual content.

8 <T a c t i l e> 9 <S 10 <Damping>0 . 9</Damping> 11 <S 13 </ T a c t i l e> 14 <Listing 1 .

 89101113141 t i f f n e s s>0 . 8</ S t i f f n e s s> F r i c t i o n>0 . 5</ S F r i c t i o n> 12 <D F r i c t i o n>0 . 3</ D F r i c t i o n> Example of an xml-based file (HAML

1 <s e d l : SEM> 2 <s 4 </ s e d l : E f f e c t> 5 <s

 1245 e d l : E f f e c t x s i : type=" sev : RigidBodyMotionType " a c t i v a t e =" t r u e " s i : p t s =" 1593000 "> 3 <sev : MoveToward d i s t a n c e =" 200 " a c c e l e r a t i o n =" 30 "/> e d l : GroupOfEffects s i : p t s =" 1647000 "> 6

8 </ s e d l : GroupOfEffects> 9 </ s e d l : SEM> Listing 2 .

 892 Example of an xml-based file (MPEG-V[START_REF] Waltl | Enriching multimedia with sensory effects: annotation and simulation tools for the representation of sensory effects[END_REF]

25 }

 25 t i c S u r f a c e H a p t i c T e x t u r e S u r f a c e { 7 s t i f f n e s s R a n g e 0 . 1 10 8 s t a t i c F r i c t i o n R a n g e 0 . 2 associated with the visual scene). Thus the MPEG-4 BIFS format extended by Cha et al. can both describe a 3D scene and/or contain data to drive vibrotactile arrays. These two possibilities have been implemented by Kim et al. for adding haptic textures effects or vibration effects to educational videos[START_REF] Kim | Effects of different haptic modalities on students' understanding of physical phenomena[END_REF].

Table 1

 1 List of potential haptic effects for audiovisual content.

http://www.adobe.com/products/flash.html

Type of interface

Wearable

Vibrotactile armband 7x10 vibration motors Vibrations (related to position of a ball during a soccer game) [START_REF] Lee | Immersive live sports experience with vibrotactile sensation[END_REF] Vibrotactile glove 20 vibration motors (4 per finger) Vibrations [START_REF] Kim | A tactile glove design and authoring system for immersive multimedia[END_REF] Vibrotactile armband or jacket Array of vibration motors (variable size) Vibrations [START_REF] Rahman | Adding haptic feature to YouTube[END_REF] Vibrotactile jacket 16x4 vibration motors Vibrations (related to user's emotions) [START_REF] Lemmens | A body -conforming tactile jacket to enrich movie viewing[END_REF] Vibrotactile vest Vibration motors + solenoids + peltier elements Pressure (gunshot), temperature (blood flow), vibrations (slashing) [START_REF] Palan | Tactile Gaming Vest (TGV)[END_REF] Vibrotactile Table 4 Summary of existing schemes for adding haptic effects to audiovisual content. Each system offers a solution for synchronizing and rendering haptic feedback within an audiovisual content. Some schemes also specify ways to distribute the media over the network.