I also thank the three labs: U2IS at ENSTA-ParisTech, LISA at CEA-list, and Limsi at University of Paris-Sud, for hosting me during my thesis. These labs are a prolific environment where to work, organizing or participating to numerous events and conferences.

I thank my colleagues from the three labs for participating to my experiments, for being patient when suffering the vagaries of electronics, and for the technical support they provided. I thank them for their good mood, the many activities we did together and the memories I will keep. These activities were sporty, playful and tasty.

I am not citing all of them here, so I focus on my roommates in the labs: Paul-Alexis NOVICOFF, Roxana AGRIGOROAIE, and David-Octavian IACOB. I thank them for their support, for responding to my questions, and for the discussions about their own thesis subject. I deeply thank my family for encouraging me from the start to pursue my ambitions in robotics, and for ensuring my life would be easier during these three years.

I also thank my grand-parents for their interest on my thesis subject.

Last but not least, I thank my numerous bikes who gave their life to this cause and to the success of my thesis.

Introduction

Many years ago, the industrial robots were designed in constraint environments, behind fences and humans were forbidden in near vicinity. These robots were rigid and designed for task-oriented purposes, hence they did not allow human-robot interactions. Recently, new applications required the robots to leave these fences and explore open spaces. Some of these robots were set in human environments.

Such environments have the property of being human-centred and social. It means that interactions and communications between individuals occur all the time. In this context, Social Robotics field emerged.

Social robots have to sustain daily and long term interactions with humans. They have to generate appropriate behaviors, relevant to the social context. They not only perform technical tasks but also they are viewed as social companions. They have to constantly adapt their behavior, make the interaction more pleasant, and more natural. All these elements require high level decision making with communication skills for communication purposes, and adaptive behaviors for naturalness of the interactions.

The development of such abilities is part of the Social Robotics field. The recent progress in the field enhanced social functions of the robots. In education, social robots act as teachers, individual tutors, or learning pairs [START_REF] Leite | As time goes by: Long-term evaluation of social presence in robotic companions[END_REF][START_REF] Deshmukh | Towards empathic artificial tutors[END_REF], Mubin et al., 2013 (review)). Social robots assist people such as the elderly [START_REF] Heerink | Assessing acceptance of assistive social agent technology by older adults: the almere model[END_REF][START_REF] Gross | Progress in developing a socially assistive mobile home robot companion for the elderly with mild cognitive impairment[END_REF][START_REF] Wada | Living with seal robots-its sociopsychological and physiological influences on the elderly at a care house[END_REF]Shibata, 2007, Robinson, MacDonald, andBroadbent, 2014 (review)), or children in hospitals [START_REF] Jeong | A social robot to mitigate stress, anxiety, and pain in hospital pediatric care[END_REF]. They also assist in medical purposes, for rehabilitation [START_REF] Tapus | User personality matching with a handsoff robot for post-stroke rehabilitation therapy[END_REF] or autism [START_REF] François | A long-term study of children with autism playing with a robotic pet: Taking inspirations from non-directive play therapy to encourage children's proactivity and initiativetaking[END_REF][START_REF] Chevalier | Impact of sensory preferences of individuals with autism on the recognition of emotions expressed by two robots, an avatar, and a human[END_REF].

Other fields are entertainment [START_REF] Tanaka | Socialization between toddlers and robots at an early childhood education center[END_REF][START_REF] Fernaeus | How do you play with a robotic toy animal?: a longterm study of pleo[END_REF] and domestic/household [START_REF] Graaf | Longterm evaluation of a social robot in real homes[END_REF][START_REF] Klamer | Adventures of Harvey"-Use, acceptance of and relationship building with a social robot in a domestic environment[END_REF]. Social robots can also be used in public spaces, as a receptionist [START_REF] Gockley | Designing robots for long-term social interaction[END_REF], or as museum guide [START_REF] Stubbs | Long-Term Human-Robot Interaction: The Personal Exploration Rover and Museum Docents[END_REF].

Importance of emotions and touch in Social Robotics

Social robots have two main purposes: communicate, and act in a natural and pleasant manner. Emotions and Touch are two components of high importance.

One of the reasons is the anthropomorphic/anthropological point of view. If the robots have the same abilities as humans, they should be more adapted to live with humans, and the interactions would appear more natural. As emotions are fundamental for human beings, and it is often assimilated to being alive, robots should express emotions. We can notice that the work on emotions in robotics does not only enables them to express emotions, but it also helps them to detect cues about the affective state of the interaction partner. This perception-expression loop is necessary to fully simulate the emotional process. On another hand, touch is a very rich sense, that provides information about shape, texture, and/or temperature of an object, allows dexterity while manipulating an object. The sense of touch is used in social interactions, such as hugs or handshakes.

Anthropomorphism is not the only reason why Emotions and Touch are important to implement in a social robot. Naturalness of the interaction is also improved by the ability to conform with the changing environment, and the environment that humans evolve in is tremendously complex.

Emotions

Among the environment variables we can find the social context. A social robot needs to be socially aware, and execute appropriate actions to be socially accepted.

Hence, the robot should know for instance: its own social function, the type of environment (e.g., private/public, with one or several persons), the external characteristics of the individual it is interacting with (e.g., child/adult/elderly, female/male), the internal characteristics of the individual (e.g., previous interactions with the robot, personality, tastes), the social signals communicated during the interaction (e.g., mood, social engagement, emotion, and so on). The internal state of the user is part of the social context and is a prominent component a social robot should measure. Nevertheless, it is very difficult to determine. It is dynamic and has several levels (i.e., from the long term personality, to the mood and the short term emotional state). The emotional state is the most ephemeral of the internal states, however, it has a strong impact on user experience. The social robot should be able to measure 1.2. Importance of emotions and touch in Social Robotics it in real time, decide for an appropriate response, and express back a corresponding emotion.

Besides the information given by the social context, Emotions are also a communication tool. When communicating, several types of information are exchanged:

the verbal and the non-verbal cues. Some of the non-verbal cues are emotion expressions. Measuring and interpreting these cues is important to appreciate all the information that is communicated, and enhances the perceived robot intelligence by the user. One should notice that emotion expression is not necessarily synonymous of experienced emotion. Nevertheless, in the emotional process, there is unconsciously/ or consciously the preparation for an action. So, observing the emotional cues enables to predict the action of one's partner. And this is beneficial for communication and collaboration. The action prediction abilities are also very important for the naturalness of the interaction.

Finally, emotions have the advantage to create a private link between the individuals, it involves intimacy. Empathy also reinforces this connection. As this kind of bond enhances engagement in the social interaction, and as engagement is a source of success of the social interaction, empathic abilities are important considerations for social robots.

Touch

The touch modality is fundamental as it enables us to act on the environment at the same time we perceive it. The physical contacts we have with objects are numerous and have many purposes. However, there is a part of the time during which we do not touch objects, but other humans or animals. This is called social touch [START_REF] Gallace | The science of interpersonal touch: an overview[END_REF][START_REF] Field | Touch for socioemotional and physical well-being: A review[END_REF]Silvera-Tawil, Rye, and Velonaki, 2015;[START_REF] Huisman | Social touch technology: A survey of haptic technology for social touch[END_REF]. When a social touch occurs, not only the physical contact is exerted, but also a social meaning is conveyed. For instance, holding the hand of a child while crossing the street does not only keep him close, it is also reassuring him. Even though no specific meaning was communicated by the toucher, it can have psychological effects on the partner. [START_REF] Field | Touch for socioemotional and physical well-being: A review[END_REF] showed that touch deprivation in childhood increases violent behaviors. [START_REF] Stack | Tactile stimulation as a component of social interchange: New interpretations for the still-face effect[END_REF] found that children that are regularly touched during a period of time have more tendency to smile. [START_REF] Goldstein | Brain-to-brain coupling during handholding is associated with pain reduction[END_REF] found using neural signals that being touched by a partner reduces pain. [START_REF] Morrison | Keep calm and cuddle on: social touch as a stress buffer[END_REF] showed that touch enables stress buffering. These results indicate that touch is important for well-being. This is the reason why social robots should have touch abilities. Some of the following results were observed in psychology as in Social Robotics as well.

Another reason to use touch in Social Robotics, is that touch has communication properties. Messages, with high level signification, can be conveyed through touch, and it involves several aspects: the context, the location of touch, and the tactile pattern. The context has a very strong effect on the meaning given to the interaction [START_REF] Gallace | The science of interpersonal touch: an overview[END_REF]. For instance, the gender, the intention of the toucher, the time, the place of the interaction, if people are around, if the toucher is a stranger, and so on, can determine if the touch is pleasant or unpleasant. The location of the touch can also induce comfort or discomfort [START_REF] Heslin | Meaning of touch: The case of touch from a stranger or same sex person[END_REF]. [START_REF] Nguyen | The Meanings of Touch: Sex Differences[END_REF] found that touch on the hands is considered as friendly, loving and pleasant while touch on thighs indicates sexual desire. Thirdly, the touch pattern is highly informative. Pinching, squeezing, stroking, tapping are basic patterns that can be recognized by a measurement system [START_REF] Jung | Touching the Void-Introducing CoST: Corpus of Social Touch[END_REF].

Stroking can induce different levels of pleasantness depending on the speed [START_REF] Huisman | Social touch technology: A survey of haptic technology for social touch[END_REF]. Some pre-defined patterns exist like the hug, the handshake, holding the hand, or taping the shoulder, that have social meaning. For instance they can all be greeting manners, but in different contexts.

As previously mentioned when we presented illustrative examples of social touch messages, another property of touch is to convey emotions. It is possible to feel an emotion when being touched (see the pleasure levels reported previously). It is also possible to express an emotion [START_REF] Alenljung | User experience of conveying emotions by touch[END_REF] and to recognize an emotion that has been communicated. An individual can recognize such emotions [START_REF] Hertenstein | The communication of emotion via touch[END_REF], or alternatively a measurement system can be used [START_REF] Silvera-Tawil | Interpretation of social touch on an artificial arm covered with an EIT-based sensitive skin[END_REF].

When touching someone, we enter in his/her personal space, and if this is socially accepted, touch may be a way to create a social connection with the individual. The social interaction may be enhanced by this aspect. For instance, the "Midas Touch" effect, was observed between humans [START_REF] Crusco | The Midas Touch: The effects of interpersonal touch on restaurant tipping[END_REF]: people offered more tips in a restaurant when they were touched by the waiter. This effect has also been highlighted face to a robot by Fukuda et al., 2012. Pro-social behaviors were observed in children after hugging a robot Shiomi et al., 2017a. Finally, if touch is beneficial to emotions and emotions are beneficial to social interactions, the sense of touch should be considered in social human-robot interaction.

All these arguments are in favor of the importance of emotions and touch to communicate efficiently, and in a natural manner. Hence, implementing in social robots the abilities of: being touched, perceive emotions, interpret the touch and emotion, express emotion, and touch the individual with whom it is interacting, is a promising consideration for enhanced social interactions.

Some properties of emotions and touch are used to go beyond the goal of improving social interactions. Specific applications where emotional abilities or touch performance are required in robots, exist. For instance, for teaching emotions to autistic children [START_REF] Chevalier | Impact of sensory preferences of individuals with autism on the recognition of emotions expressed by two robots, an avatar, and a human[END_REF], to induce positive emotions in a hospital context, or to buffer stress or increase efficiency at work (Shiomi et al., 2017b).

1.3. Motivations of the thesis

Motivations of the thesis

We highlighted the importance of emotion and touch in social interaction. The main research question that this thesis is focusing on is: Can a social robot infer the emotional state of an individual using the tactile modality during a touched social interaction?

To answer this question, one has to carry out a multi-disciplinary research. The domains of interest are: (2) Psychology Psychological consequences of touch.

We need to measure the human behavior and his/her internal state.

-The robot should make the interaction more intuitive and should not be psychologically invasive.

-The experiments have to be based to authentic and spontaneous emotional states of the participants.

(3)

Robotics

Hardware capabilities for robots, to measure touch.

The robot applications are designed for the humans (i.e., the findings of this research should be used in an appropriate manner).

The physical contact with the robot should be realistic, comfortable (e.g., movements, texture), and non invasive.

To the best of our knowledge, this question with all these different ingredients and research aspects combined has not been answered yet. However, several researches partially contributed to these domains. Some robots look impressively human-like (Hanson Robotics, 2018) even though it is not always the strategy chosen for appearance design. These robots are able to express emotions such as: facial expressions in a human-like way [START_REF] Aly | Multimodal Adapted Robot Behavior Synthesis within a Narrative Human-Robot Interaction[END_REF] or with a simplified head [START_REF] Kędzierski | EMYS-emotive head of a social robot[END_REF], gesture and voice [START_REF] Beck | Towards an affect space for robots to display emotional body language[END_REF], colors (Le [START_REF] Maitre | Self-talk discrimination in humanrobot interaction situations for supporting social awareness[END_REF]. However, robots able to infer the emotion of the user are less common. Some non-intrusive techniques exist using audio-visual cues [START_REF] Wu | Survey on audiovisual emotion recognition: databases, features, and data fusion strategies[END_REF] using: facial expressions [START_REF] Valstar | The first facial expression recognition and analysis challenge[END_REF][START_REF] Wilhelm | Test battery for measuring the perception and recognition of facial expressions of emotion[END_REF], posture [START_REF] Bianchi-Berthouze | On posture as a modality for expressing and recognizing emotions[END_REF], and prosody [START_REF] Grimm | Emotion estimation in speech using a 3d emotion space concept[END_REF]. Methods using touch [START_REF] Silvera-Tawil | Interpretation of social touch on an artificial arm covered with an EIT-based sensitive skin[END_REF] are scarcer, and still not applied to robotics and spontaneous emotions.

Nowadays, robots are able to be controlled by force and be controlled despite a soft structure (e.g., the MEKA robot [START_REF] Orefice | Let's handshake and I'll know who you are: Gender and personality discrimination in human-human and human-robot handshaking interaction[END_REF]). This gives them the possibility to interact physically with humans. Hence we observe the development of studies in physical social interaction, for instance to tackle the handshake manner (Melnyk, 2014). Several systems exist to measure and generate touch features [START_REF] Kontarinis | A tactile shape sensing and display system for teleoperated manipulation[END_REF][START_REF] Bolopion | Remote microscale teleoperation through virtual reality and haptic feedback[END_REF][START_REF] Sarakoglou | A High Performance Tactile Feedback Display and Its Integration in Teleoperation[END_REF][START_REF] Manus | Manus VR[END_REF], however, they are mostly for tele-operation purposes which is only the informative channel of touch. There is a lack of social context inference using tactile data in the literature (Silvera-Tawil, Rye, and Velonaki, 2015). Some studies tackled the perception of different tactile behaviors of a robot by users [START_REF] Gaffary | The impact of combining kinesthetic and facial expression displays on emotion recognition by users[END_REF]. Others aimed at recognizing if a touch has social meaning, or if it is only informative or object contact. [START_REF] Knight | Real-time social touch gesture recognition for sensate robots[END_REF][START_REF] Avelino | Human-aware natural handshaking using tactile sensors for Vizzy, a social robot[END_REF]. It was also possible to detect the general context of handshake interactions (Melnyk, 2014), or levels of affection in whole body touch [START_REF] Cooney | Recognizing affection for a touch-based interaction with a humanoid robot[END_REF].

Very few studies aimed at inferring the emotional state through touch. Some

were able to discriminate tactile patterns [START_REF] Jung | Touching the Void-Introducing CoST: Corpus of Social Touch[END_REF][START_REF] Yohanan | Design and assessment of the haptic creature's affect display[END_REF], and others could recognize emotions in tactile data [START_REF] Silvera-Tawil | Interpretation of social touch on an artificial arm covered with an EIT-based sensitive skin[END_REF], or kinematic data [START_REF] Gaffary | The impact of combining kinesthetic and facial expression displays on emotion recognition by users[END_REF]. Nevertheless, when studying affective touch expressions, most of the authors use acted emotions. This goes against the spontaneous nature of emotion expression, and we think a strong bias may be created. Besides, in some of these studies, there is no social interaction, the user is only manipulating a passive device.

The lack of research in the touch area may be due to several reasons: (1) The data collection is time consuming as the participants have to be in the lab to perform the touch interactions (i.e., no large scale online questionnaires or picture databases can be used).

(2) Touch is a complex modality to measure, as it is multi-modal (e.g., temperature, vibration, force), and it is distributed on a large surface (i.e., the whole robot body). The comparison between measurement device is often challenging.

(3)

Many social contexts may change the meaning of touch. And the comparison of the results also becomes complicated. (4) It is possible to rely on questionnaire (e.g., how often are you touched by someone? What would you feel if you were touched in this location?). However, such questionnaires do not allow to collect physical data.

Thesis outline

As a result, due to the lack of touch knowledge in social human-robot interaction,

we are motivated at collecting tactile data during such interactions. We intent to contribute in the field of artificial skins in order to collect this data in an easy way.

Between Psychology that studies subjective responses to human-human interactions, and Social Robotics that measures physical data during human-robot interactions, little studies (like Melnyk et al., 2014) start from measuring physical data during human-human interactions. We aim to contribute in this area as we think that the robot shape, appearance and abilities are going to change in the future.

We also aim to contribute in social context inference through touch as this modality has not been deeply studied from this point of view, but may provide useful information.

Finally, we are motivated at contributing in the field of emotion recognition through touch, given the importance of emotions in the success of social interactions.

However, we want to study spontaneous emotions, in an ecological1 environment.

Thesis outline

The overall methodology of our work consists of several steps: (1) Measure, or generate if possible, the internal state of an individual.

(2) Choose a social agent that interacts with the individual (e.g., a social robot, or another individual).

(3) Perform touched social interactions between both partners. (4) Measure the tactile data during the interaction using a specific device. (5) Observe the differences in the tactile data depending on the internal state of the individual. (6) Model these differences to be able to recognize autonomously the internal state of the individual.

Chapter 2: Details the states of the art of four aspects that are used in our research: the handshake interactions, the personality, the emotions, and the artificial skins. The handshake interactions were found appropriate candidates to study social touch. We detail why we limited the rest of the thesis on the study of handshake interactions. We explain why personality traits are relevant in Social Robotics, and their importance in handshake interactions. The long term aspect of personality, for instance, makes it simpler to study in a lab context than emotions. This led us to start by contributing in long term internal states field (such as personality and gender), before tackling the heart of the thesis which are emotions.

Chapter 3: Presents a study to determine the effect of long term internal states, such as personality and gender, on the handshake manner. It also presents an instrumented glove that measures pressure and movement, that we developed for this purpose. Handshake interactions are collected for both human-human and humanrobot interactions.

Chapter 4: Describes a study to determine the effect of short term affective states, such as spontaneous emotions, on the handshake manner. Elicitation methods are reviewed and an emotion elicitation tool using virtual reality is designed and evaluated. Two gloves were designed and improved. The pressure distribution data was analyzed during handshake. Human-human and human-avatar (in the virtual reality environment) interactions were both studied. We investigated the effect of emotions on this data.

Chapter 5: Presents a study to determine the effect of medium term affective states, such as spontaneous moods, on the handshake manner. A consistent comparison between human-human and human-robot is proposed.

All along the chapters and in appendixes described bellow, we give physical cues about the handshake manner in general, such as: interaction effects of the pressure by the partners, the inter-individuals differences, or how to describe the spacial pressure in a handshake.

Appendix A: Presents the detailed study to determine the role of the sensors used in Chapter 4 and Chapter 5. The role of a sensor is to represent the action of the handshake partner that presses it. The study answers the question: which partner activates which sensor?

Finally, Appendix B: Presents a description of the pressure data collected in Chapter 5, with a reduced number of features. The description contains both spacial and pressure magnitude information, and the correlations are analyzed.

Introduction

In the introduction of this thesis, we showed that our work addresses three research fields (i.e., Sociology, Psychology, and Robotics). These disciplines largely differ from their object of study and methodology. Sociology tackles the social interactions in general, while psychology addresses the behavior and mind of individuals, including conscious and unconscious phenomena. Both are part of Social Sciences, which study the relationship between individuals. Robotics consists in endowing machines with observation, action, adaptation, and autonomy abilities, in order to assist humans in their tasks. We are interested in several components of these disciplines for our work, which we present in this chapter.

We detail in Section 2.2 the handshake manner that is a social interaction, that embeds rich meanings.

Then, we present two psychological aspects that may impact our way to perform handshakes. The first one is the personality (see Section 2.3), which corresponds to long term psychological characteristics of an individual. The second one is the emotion (see Section 2.4), which corresponds to a short term phenomenon. As it is a very complex matter, many theories and approaches were proposed. We present some of them in detail.

Finally, robotics relies greatly on hardware, and the ability to perceive. We present in Section 2.5 a review of systems that enable to measure tactile data, which is an important challenge in our research.

Handshake: a social interaction

In this thesis, we investigate the effect of the psychological state of an individual on the tactile data exchanged during a social-touch interaction. Hence, we need to choose an interaction that involves physical contact, and is subject to a psychological communication between the individuals. A social interaction that fulfills these requirements is the greeting handshake. Now, we describe what is a handshake and what it represents, and why we choose to work on it.

First of all, [START_REF] Hall | The handshake as interaction[END_REF] give a definition of handshake. "The Handshake is a social act wherein we have a combination of contact experience between two hands of different persons that communicates the degree of mutual physical resistance as well as the exchange of social identities and the beginning and ending of social activity." This is also a non verbal form of communication. This definition proposes: (1) the concepts of social act and communication, (2) the involvement of two individuals, (3) the existence of a physical contact, and, (4) that handshake is impacted by social identities (i.e., person's sense of who they are based on their social group membership [START_REF] Tajfel | Social psychology of intergroup relations[END_REF], which suggests that the internal state of the individual may have an effect). This definition makes handshake directly relevant to our work.

In their paper, Hall et al, give a precise description of the handshake manner, from its origin to the different points of view the former sociologists had about it, and among them, [START_REF] Schiffrin | Handwork as ceremony: the case of the handshake[END_REF]. Even though it was several decades ago, to our knowledge, most of psychological studies about handshake, are based on this work.

According to [START_REF] Schiffrin | Handwork as ceremony: the case of the handshake[END_REF], the handshake manner is an "access ritual". It means it enables to get an entrance point in the "Self" of someone following a certain convention. It is structured as follows: one person offers an access to his/her "Self" to his/her partner (e.g., like a present), and at the same time, he/she requests an access to the other. This first person is called the initiator of the handshake. On a second phase, the partner accepts the offer, and grants the access to himself/herself. This explains why, as specified by [START_REF] Hall | The handshake as interaction[END_REF], a non response of the partner, or if he/she has a very passive hand during handshake, creates immediately embarrassment and social tension, as the convention is broken. This also shows that the internal state of the individuals is involved in handshake manner, and is interesting for our study.

The fact that this social act is based on conventions is due to an historical background of handshake. Indeed its origin is very old and several sources support handshakes were used in ancient Greece, and was also used in medieval Europe by high social classes. It is now used worldwide by all types of classes. However, depending on the epoch, location, and social context, the handshake manner has different forms and meanings. Several examples are given by [START_REF] Schiffrin | Handwork as ceremony: the case of the handshake[END_REF], or can be found in the public domain. In ancient Greece, handshake was used to show friendliness or hospitality. In the medieval Europe, kings performed handshake to knights to show their trust. In some contexts, handshake can be used to congratulate, or for reconciliation [START_REF] Schiffrin | Handwork as ceremony: the case of the handshake[END_REF].

The handshake manner also has different forms and meanings (wisc-online, 2018).

In Korea, the younger holds the arm of the older with his/her left hand. In China, the handshake can last longer. Depending on the country, different firmness are accepted. Too soft or too strong handshakes can be perceived un-polite. Beside these meanings, duration and firmness variations, the grip can be very distinct. The In some countries, the traditional greeting does not involve handshake. In India, the Namaste gesture is used, or in New-Zeland people touch their nose and forehead.

Despite these numerous differences, either in terms of greeting and handshake forms and meaning, it seems that greeting is often related to a physical contact, and grants access to internal aspects of the individuals. Nowadays, in Europe, handshake is mainly used for greeting or leaving someone, using the standard type (a). In this thesis, we are interested in the greeting handshake between acquaintances. We restricted our work on acquaintances as we may need repeated measures with the same individuals, which prevents from catching the "first" handshake between unknown persons. Besides, we supposed that in everyday life, apart from professional situations, people rarely start physical interaction with unknown persons.

According to [START_REF] Schiffrin | Handwork as ceremony: the case of the handshake[END_REF], three messages are exchanged at the same time during handshake: (1) the former separation did not damage the relationship, (2) a new request of social access is performed, and, (3) the following relationship is prepared.

There is the idea that if the handshake succeeds, the partnership can be effective for the rest of the interaction. However, the handshake manner involves so much parameters, including the combination of both partners (e.g., distance between the partners, gaze, duration, number of oscillations, firmness), that there is a need to standardize it. Many lessons in management or trading for instance give advices on how to perform the "perfect" handshake. They also give many examples of "bad" handshakes, with nick names like "dead fish", or "knuckle-buster" [START_REF] Businessgovau | The Top 10 Bad Business Handshakes[END_REF]. This reinforces the idea that the handshake is not more than just a ritual, it is an automatic procedure. When one knows how to do a "perfect" handshake, his/her manner will never change, whatever his/her personality, or emotional state. [START_REF] Stryker | Fundamental principles of social interaction[END_REF] showed that managers mainly take into account what their employees say, and forget about assessing their handshake.

Personality: psychological constants

However, given another opinion, the handshake can not be fully controlled and it can be socially interpreted. It has a meaning, it is an extension of ourselves ( [START_REF] Vanderbilt | Amy Vanderbilt's etiquette[END_REF]. Therefore, the personality of an individual may interfere with the handshake manner. Besides, as during handshake one has to anticipate the firmness of his/ her partner, evaluate it, and adjust his/ her own strength in real time, the tactile sense is crucial. This makes the handshake interaction a good candidate to investigate the effect of individual's internal state on the tactile data exchanged during a touched social interaction.

Personality: psychological constants

The personality (from the psycho-social point of view) is all the behaviors and attitudes that characterize an individual. This means that facing a specific situation, an individual will "always" (given the stability of his/her personality) have the same response that is his/her own. Therefore, observing and classifying these reactions leads to the inference of psychological dimensions, which are personality traits. It was found that personality is stable through time: Small et al., 2003 found stability over 6 years, and[START_REF] Mcgue | Personality stability and change in early adulthood: A behavioral genetic analysis[END_REF][START_REF] Mcgue | Personality stability and change in early adulthood: A behavioral genetic analysis[END_REF] fond little changes overs 10 years.

The research on personality is addressed in many fields. For instance, in Human-Computer Interaction [START_REF] Liu | Two techniques for assessing virtual agent personality[END_REF]Junior et al., 2018=, or Social Robotics (Mileounis, Cuijpers, and[START_REF] Mileounis | Creating robots with personality: The effect of personality on social intelligence[END_REF], researchers aim at giving consistent behavior patterns to virtual agents or social robots, implementing artificial personality. Other researches also try to infer the personality of the user [START_REF] Gunes | Real-time prediction of user personality for social humanrobot interactions: Lessons learned from public demonstrations[END_REF].

Research on psychology about personality has been active for decades. The approach directing most of the work (we present some examples in the section), and that led to the definitions of personality traits, is the following:

1) If personality concerns our behaviors, we have words to describe them, so we can make a corpus of such words from dictionaries.

2) These words can be redundant, or have fuzzy meaning, so we can do a lexical reduction.

3) These words can also be classified given their meaning, so we can do a lexical clustering.

4)

Using these clusters defined by the aggregated words, it is possible to evaluate people. Then these people are associated to a primitive personality description.

5)

Analyzing the correlations between the evaluations, it is possible to reduce the dimensions of the descriptions. This is called factor analysis, and it generates higher level personality traits.

6)

Finally, questionnaires can be written to directly evaluate the factors in subjects. The questionnaires have to be checked for consistency, and repeatability. And they can be used in other fields of research to have a relevant estimation of individuals personality.

The first steps [START_REF] Geslin | Gamers' versus nongamers' emotional response in virtual reality[END_REF], 2)) were performed in 1936 by [START_REF] Allport | Trait-names: A psycho-lexical study[END_REF]. They extracted 18000 words from the English dictionary, and classified them in 4 categories: "real" personality traits (e.g., aggressive, introverted, sociable), temporary states (e.g., stunned, chatting), social evaluation (e.g., insignificant, acceptable, worthy), and metaphorical terms (e.g., pampered, crazy, malformed). When selecting only the "real" personality traits, 4500 terms remain. Starting from these words, Cattell, 1943 grouped them as synonyms (step 3)). This semantic analysis led him to create a list of 171 pairs of adjectives. After evaluating subjects with these adjectives, and analyzing the correlations, he found 60 clusters that led to 35 bipolar personality clusters (steps 4), 5)). In 1957, Cattell developed the theory of 16 principal factors to describe the personality, and proposed a questionnaire (step 6)). Some details are presented by [START_REF] Cattell | The sixteen personality factor questionnaire (16PF)[END_REF]. The goal of many researchers in the following years was to reproduce these results, or to continue to reduce the dimensionality of the factors. They began with the 171 pairs of adjectives, the 35 clusters, the 16 factors, or started back from step 1).

Eysenck, 1991 reviewed most of these studies, and stated that the level of repeatability of the results is very low. He found it very difficult to find the 16 factors from new datasets, and it appeared that the number of factors was lower. For instance, [START_REF] Tupes | Stability of personality trait rating factors obtained under diverse conditions[END_REF] used the 35 bipolar clusters of Cattell, and found 5 principal factors. They observed a great consistency of the results. [START_REF] Norman | Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings[END_REF] found the same factors and named them as Extroversion, Agreeableness, Conscientiousness, Emotional Stability, and Culture. He checked these factors orthogonality.

Norman also created a new corpus of 75 clusters of words starting from a 2800 word database. This corpus was used by [START_REF] Goldberg | An alternative 'description of personality': the big-five factor structure[END_REF] to show, using several statistical methods, high robustness of the 5 factors. They were then called as the Big5 (i.e., Extroversion, Neuroticism, Agreeableness, Conscientiousness, Openness). [START_REF] Eysenck | Dimensions of personality: 16, 5 or 3?-Criteria for a taxonomic paradigm[END_REF] had another approach than the authors previously cited. He had a top-down procedure. Eysenck proposed his own theory about personality. It can be split in 4 levels: (1) observed specific behaviors, (2) repeated occurrences of this behavior, (3) this behavior is correlated with other behaviors, which creates a trait, and (4) several traits are correlated which creates a personality dimension.

He firstly proposed general personality dimensions, and then searched for the corresponding behaviors. He started in 1947 from two general traits, based on biology studies: Extroversion/ Introversion, and Neuroticism/ Stability. Later, he added Psychotisime/ Socialisation, and Lie/ Social desirability to propose a revised version of a personality questionnaire: the Eysenck Personality Questionnaire (EPQ-R) [START_REF] Eysenck | A revised version of the psychoticism scale[END_REF]. Eysenck also assessed that Cattell used in practice 4 secondary factors: Extroversion, Anxiety, Emotivity, and Submission.

Given Eysenck, the labels of these secondary factors mostly correspond to his own 4 dimensions. The Big5 also has the same 2 first dimensions, the three others partially represent Psychotisime.

To summarize the above discussion, all the psychologists do not use the same approach, do not agree on the number of personality traits, and their name. However it is clear that Extroversion and Neuroticism are two main dimensions that regularly appear in the papers. The community is also in favor of the Big5 as its robustness has been demonstrated several times. John and Srivastava, 1999 developed a questionnaire (Big Five Inventory) to measure these 5 factors, which is described and compared to other questionnaires. It has the advantage to be short (44 questions), to be composed of sentences describing situations instead of a list of adjectives, has been largely used, and its consistency has been proved. A French version exists and has been evaluated [START_REF] Plaisant | Validation par analyse factorielle du Big Five Inventory français (BFI-Fr). Analyse convergente avec le NEO-PI-R[END_REF]. In our work, we used both versions of the questionnaire, depending on the spoken language of our participants.

In our work, we take advantage of the following properties of personality: (1)

It is an internal characteristic of an individual that is important to assess during an interaction. (2) This characteristic is related with behaviors and attitudes of the individual, so it may be expressed during the social interaction. (3) It is a long term characteristic, so it can be measured prior or after an experiment, without interfering with the data. (4) It is possible to measure it using short questionnaires, such as the Big Five Inventory that has 44 questions. Our contributions in the field of personality and tactile social interaction is in Chapter 3.

Emotions: theoretical overview

The emotion an individual can experience is a dynamic process that depends on a specific event and the environment [START_REF] Scherer | What are emotions? And how can they be measured?[END_REF]. Contrary to the personality traits or gender, the effects of emotions are more complex to study, due to the time and context dependency. This dynamic and event-related aspects of emotions are two properties that were systematically stated in our literature review. However, emotions are far from being subject to consensus. The first question of What are emotions? is not answered yet [START_REF] Kappas | The science of emotion as a multidisciplinary research paradigm[END_REF]. A large number of theories were developed and evolved since Darwin's discoveries [START_REF] Darwin | The expression of the emotions in man and animals[END_REF], and many are still used nowadays. Among them, we can cite [START_REF] James | What is an emotion?[END_REF][START_REF] Cannon | The James-Lange theory of emotions: A critical examination and an alternative theory[END_REF][START_REF] Lazarus | Emotions: A cognitivephenomenological analysis[END_REF][START_REF] Averill | A constructivist view of emotion[END_REF][START_REF] Frijda | Emotion, cognitive structure, and action tendency[END_REF][START_REF] Levenson | Voluntary facial action generates emotion-specific autonomic nervous system activity[END_REF][START_REF] Scherer | Appraisal processes in emotion: Theory, methods, research[END_REF][START_REF] Ledoux | The emotional brain, fear, and the amygdala[END_REF]. These authors come from different backgrounds and do not study the same aspects of emotions, which explains the variability of emotional theories. We present some of them in order to see what components are involved in the emotional process, and what are the challenges we have to face in order to study emotions in a social interaction context. We propose in this section an overview of the various theories about emotions.

The causes, process, and goals are detailed. We also insist on the importance of emotions in social interactions. In short, to the best of our understanding on the subject, we address these three questions: (1) How does the emotional process work?

(considering what causes emotions?, which components are involved?, and what are the consequences of emotions?). (2) What is the purpose of emotions? And finally, (3)

an intent to answer What are emotions?, but we better answered What makes emotions different from the others affective states?

Emotional theories

Emotions are a wide subject, that has been intensively investigated in the last century, and they involve many disciplines. However, still no consensus exists [START_REF] Kappas | The science of emotion as a multidisciplinary research paradigm[END_REF]. This led to the creation of many theories from several points of view: Ethology and Naturalism, Neuro-physiology, Cognitive Sciences, or Social Sciences. We present in this sub-section an overview of some of these theories.

Ethology and Naturalism

Ethology and Naturalism focus on the behavior of animals and the evolution of species. Ethological studies cannot explain how the emotional process works inside the body, however they give some properties of emotions. Darwin, 1872 noticed that animals have rich emotion expressions, which are included in their communication system, and, that are necessary for individual regulation. Ekman, 1992 goes along Darwin when he says emotions enable to face a specific number of fundamental life problems, leading to separated emotion patterns, called basic emotions (i.e., fear, anger, surprise, joy, sadness, and disgust [START_REF] Ekman | Basic Emotions In[END_REF]). He precises that these patterns evolved with the species depending on the problems they had to face (i.e., adaptive property of emotions), which is an evolutionist theory. Plutchik, 2001 joins these ideas and proposes 8 basic emotions with antagonist properties, and provides examples of compositions of the basic emotions. These resulting 16 emotions can also vary in term of activation. Ekman, 1992 worked on human facial expression, which is one of the behavioral response of emotions. He found several properties of emotions: their universality (i.e., cultural and even species similarity, triggered by the same fundamental needs), their rapidity (i.e., duration of the trigger, reaction time, and duration of the emotional episode), and "pre-cabled" (i.e., unintentional and difficult to avoid response, repeatability depending on the trigger, trigger and response bi-directionality1 ).

To summarize these findings, we can say that emotions are in the basis of human race and are necessary, as it enables to fulfill the fundamental needs and to solve problems. 6 basic emotions exist, meaning that they appear to be universal, physiologically pre-cabled, and uncontrollable. This may explain why the response is fast.

Finally, emotions have communication purposes.

Neuro-physiology

Neuro-physiologists [START_REF] Ledoux | The emotional brain, fear, and the amygdala[END_REF] found neural connections that explain the speed of emotional reactions facing "great needs". These connections are low level and link the sensory thalamus, that detects raw characteristics of the situation, with the amygdala that triggers to the hypothalamus, the motor center and sensory areas.

The hypothalamus generates the physiological response through the autonomous neural system (e.g., heart rate, breath frequency, skin sweat). The motor center activates behavioral responses (e.g., facial expression, muscle contraction). And the sensory areas focus the attention on the event that triggered the emotional response.

These responses are objective and can be measured to assess the individual emotional state. This circuit is well explained for fear or anger. LeDoux, 2003 details the fear circuit, that is summarized in Figure 2.2. The previous description corresponds to the "short circuit". Indeed, a "long circuit" is related to the cognitive evaluation of the situation, leading to a subjective conscious representation of the emotions, and emotion regulation. However, these neuro-physiological fundings cannot be generalized to all emotions. Another circuit was found for joy, involving other parts of the brain, like the reward system to motivate to renew the action that led to this emotion [START_REF] Costa | Emotional imagery: assessing pleasure and arousal in the brain's reward circuitry[END_REF]. FIGURE 2.2: Schematic of the neural process of fear. The large red arrows show the short circuit, and the small black arrows are for the long cognitive circuit. This schematic was adapted from LeDoux, 2003. We added the consciousness limit, but it should not be seen as firm frontier.

Despite neural circuits were not found for all the basic emotions, a hypothesis remains in the research community, started by James, 1884, and partially experimentally observed by [START_REF] Levenson | Voluntary facial action generates emotion-specific autonomic nervous system activity[END_REF]. It suggests that a low level emotional processing leads to physical responses, which are introspectively observed by the individual to be cognitively evaluated, leading to the subjective emotional experience. This approach is called "peripheral" and is contradicted by several researchers placed in the "cognitivist" point of view.

To summarize the neuro-physiological approach, the pre-cabled assumptions are physically observed, as some specific circuits were found for a few basic emotions.

However, this neither means that every emotions have a circuit, nor that an emotion necessarily activates its circuit. Besides, emotions involve many systems in the brain that are not cognition. This confirms that emotions can be uncontrollable and have a strong response in the body and behavior. Nevertheless, the fact that the emotional response is automatic, does not reject the assumption that an appraisal system can detect the changes in the body, making the emotion conscious.

Cognitive Sciences

What motivates the cognitivist approach is that despite the consistency of the emotional response given a trigger event claimed by Ekman, 1992, several factors can lead to different responses while caused by the same stimuli. Cognitive Sciences tackle information process mechanisms, memory, or knowledge. The authors in Scherer, Wallbott, and Summerfield, 1986 2 found that facing the same situation, everyone do not feel the same emotion, that the way an emotion is expressed depends on the individual, and that one person does not experience the same emotional states when confronted several times to the same situation. This leads to consider the existence of a psychological profile concerning emotion perception and expression. This also highlights the importance of memory and situation context. Besides, as developed in Section 2.3, one personality trait of the Big5 is neuroticism. This trait is a predisposition to feel negative emotions. Also, in the personality model of [START_REF] Norman | Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings[END_REF], one of the personality traits is emotional stability. This confirms the fact that personality has a weight in the emotional process. [START_REF] Scherer | Experiencing emotion: A cross-cultural study[END_REF] classified the participants in three groups: "sensitizers", "internalizers", and "externalizers". These categories may also be part of the personality. Besides, people have the tendency to express more their emotions either verbally, physiologically, or through movement.

Cannon, 1927 marked the origin of the cognitivits theories. The baseline of these theories is that all the observable emotional responses (i.e., physiological or behavioral) are directly the consequence of a cognitive process that evaluates the situation depending on the context, the psychological current state and profile of the individual, his/her memory, and his/her motivation. In other words, the peripheral approach states the physiology leads to cognition while the cognitivits say the cognition leads to physiology.

Cognitivist researchers are numerous and we can cite [START_REF] Arnold | Feelings and emotions: The mooseheart sympo-sium, chapter An excitatory theory of emotion[END_REF][START_REF] Lazarus | Emotions: A cognitivephenomenological analysis[END_REF][START_REF] Frijda | Emotion, cognitive structure, and action tendency[END_REF][START_REF] Collins | The cognitive structure of emotions[END_REF][START_REF] Roseman | Appraisals of emotioneliciting events: Testing a theory of discrete emotions[END_REF][START_REF] Scherer | Appraisal processes in emotion: Theory, methods, research[END_REF]. Their theories are very similar and can be placed in a cause-appraisal-to-action-generation process, divided in three steps as defined by [START_REF] Lazarus | Emotions: A cognitivephenomenological analysis[END_REF]. The first one (1) is a primary appraisal of the event that triggered the emotional process.

This takes into account the record of previous emotional experience of the individual [START_REF] Arnold | Feelings and emotions: The mooseheart sympo-sium, chapter An excitatory theory of emotion[END_REF], how the situation was anticipated, how it impacts the motivations/interests of the individual and the pleasure it can generate, what the situation would be if no action is taken and how difficult the action would be, and other context data. After assessing the situation and its impact, (2) the best action has to be chosen [START_REF] Frijda | Emotion, cognitive structure, and action tendency[END_REF], and the individual has to prepare for it (with a physiological response). An action has to be associated with a purpose. Then, (3) the action is executed, and (4) some of the theories propose a reevaluation step after the action.

Arnold, 1950 claims that the physiological response is observed cognitively. [START_REF] Lazarus | Emotions: A cognitivephenomenological analysis[END_REF] states that if the action was unsuccessful, the physiology has to be regulated to cope with the situation, and [START_REF] Frijda | Emotion, cognitive structure, and action tendency[END_REF] supports that the result of the action is a new situation to appraise. The literature is summarized in Table 2.1 with the questions answered during the appraisal process. However, these theories do not define well to which moment the subjective conscious representation of emotions occurs.

To summarize the cognitive approach, emotional process involves more components than the stimulus and the neural system of the individual. It highly depends on the context. Besides, it also depends on the individual and his/her personality.

Cognitivists state that the emotional response comes from a cognitive process. The overall process is divided in several steps with different implications. There is a first appraisal of the event itself considering if it was anticipated and what can be its consequences on the self-interest. This task is helped by the memory of former experiences. The second step is the action decision, motivated by the individual's purpose. This is just followed by the body response and change in behavior. Hence, two elements are important for the emotional process: (1) the motivation/interests of the individual, and (2) the goals of the action. This process can be long and complex, however it is not necessarily conscious.

Social Sciences

According to evolutionists like [START_REF] Darwin | The expression of the emotions in man and animals[END_REF]or Ekman, 1992, emotions Steps of appraisal [START_REF] Arnold | Feelings and emotions: The mooseheart sympo-sium, chapter An excitatory theory of emotion[END_REF][START_REF] Lazarus | Emotions: A cognitivephenomenological analysis[END_REF][START_REF] Frijda | Emotion, cognitive structure, and action tendency[END_REF][START_REF] Collins | The cognitive structure of emotions[END_REF][START_REF] Roseman | Appraisals of emotioneliciting events: Testing a theory of discrete emotions[END_REF][START_REF] Scherer | Appraisal processes in emotion: Theory, methods, research[END_REF] This shows how important emotions are in social interactions, that is why Social Sciences deeply studied this phenomenon.

Memory

A social interaction is composed of a transaction performed in two steps [START_REF] Cosnier | Psychologie des émotions et des sentiments[END_REF]. First the information has to be understood, then it has to be interpreted.

Several modalities are usually used during interpersonal social interaction: the verbal, vocal, and kinesthetic modalities. The verbal modality is more efficient to convey raw information, as it enables abstraction and conceptual operations. This is used in the understanding step. Through vocal and behavior, affects can be conveyed, allowing interpretation. Some fast affect expressions can connotate specific words of the discourse of the speaker, or show approval or perplexity of the listener.

More long term attitudes are linked to the context of the discourse. However, in this sequence of affect expressions, some are affects linked to real emotional experiences of the individual, others are acted for the discourse purpose [START_REF] Cosnier | Psychologie des émotions et des sentiments[END_REF]. In these conditions, it can be complex to infer the emotional state of one's partner.

Emotion inference can be helped by a specific ability of humans: empathy. Empathy is a component of emotion contagion [START_REF] Hatfield | Emotional contagion[END_REF].

It is the ability to put oneself in the place of the other and act, think, or feel like him/her. Three empathies exit: (1) empathy of action, (2) empathy of thought, and

(3) empathy of affect. We are interested in the last one. It is the result of two steps. First, the individual mimics and synchronizes with the observable behavior of his/ her partner. For instance, when observing expressive faces, our own face muscles contract the same way [START_REF] Hatfield | Emotional contagion[END_REF]. Then, there is the feedback step. When introspectively observing our own state, we would start to feel the corresponding emotion, and even have physiological response [START_REF] Levenson | Voluntary facial action generates emotion-specific autonomic nervous system activity[END_REF]. This process remains unconscious.

Besides the emotion contagion property of empathy, it is socially beneficial. Empathy can strengthen affinities between individuals. [START_REF] Maisonneuve | Psycho-sociologie de l'amitié[END_REF] showed that the convergence of attitudes and moral judgments favors affinities. Besides, as we stated previously, in the cognitive level, emotions depend on the motivations and purpose of action. As a consequence, similarities of emotions facing the same event enhance relationship between individuals. It can also be seen the other way: affinities ease the operation to put oneself in the place of the other, and thus empathy. So, if empathy has such importance in social life, this can lead people to "look" empathic and act the emotions of the others.

To summarize the social aspect of emotions, the emotional process includes a preparation to action, and the emotional output is an emotion expression. This is a way to communicate what is the action the individual has prepared. This is also a way to regulate the individuals between themselves. During a social interaction, emotion expressions convey interpretation of the discourse. However, these expressions do not necessarily correspond to what is felt by the individual, some can be acted for discourse purposes. An ability that enables a deep connection and synchrony between individuals is the empathy of affect. Such empathy is important to be fully engaged in the interaction, and to strengthen affinities. This also means that emotions can propagate between individuals. This shows that not only emotions ease communication, emotions also can be transfered, and the whole process is socially beneficial for individuals. This also addresses the fact that people can act an emotion, they can express it without feeling it (e.g., for discourse connotation, or to create affinity). Some researchers [START_REF] Averill | A constructivist view of emotion[END_REF] in social science, called "constructionists", even consider emotion as a social construction, a language developed and learned culturally.

Emotions are an independent structure

After having described the various levels of emotional reaction and process, it is still difficult to see where the emotion lies. Some authors consider emotions as an independent mechanism, but with interactions with physiology or cognition. [START_REF] Scherer | Appraisal processes in emotion: Theory, methods, research[END_REF] sees emotions as the simultaneous modification of 5 subsystems: perceptivo-cognitive, neuro-physiological, motivational, motor, and regulation. [START_REF] Zajonc | Feeling and thinking: Preferences need no inferences[END_REF] insists on the independence between emotions and cognition.

Both can interact (i.e., emotions can alter our cognitive attention, the observation of the physiological response can lead to a cognitive representation of the situation, it is possible to consciously regulate our emotions), but emotion can exist without any cognition process. The author states that in any case, the emotion prevails to the cognition, as the emotional process requires much less resources and is faster.

Sources and goals of emotions

In the presented emotional theories, two important aspects of emotions were noticed: the sources and the goals of emotions.

As it has been presented earlier, emotions can be induced by circumstances (i.e., uncontrolled events), by self or other's action, by mental representations (e.g., memory, thought), or by empathy.

The purpose of emotions, defined in the literature are as numbered as the theories about it. Evolutionists see emotions as adaptive, necessary to survive, and is part of the communication instinct (Ekman, 1992). Cognitivists argue emotions enable to prepare for an action, with a specific goal. [START_REF] Frijda | Emotion, cognitive structure, and action tendency[END_REF] proposes an action and a purpose for each basic emotion, presented Table 2.2 (except for sadness but we added a suggestion in the table).

Finally, a goal of emotions is to ease communication. This is helped by a large and differentiable expressive panel [START_REF] Darwin | The expression of the emotions in man and animals[END_REF]. This richness was studied by Ekman, 1992 for facial expressions, which give to emotions the properties of "decontextualisation", and "conventionalization". As it is also possible to consciously act an emotional expression, this may be the basis of a language. As it was explained [START_REF] Averill | A constructivist view of emotion[END_REF].

Definitions

We presented in the previous sub-sections several theories about emotions. However, a question remains: What are emotions?. In our literature review, rare are the authors that give a clear definition of emotions in a few sentences. This is not what we aim to do here. Nevertheless, we summarize the emotion properties in a consistent manner, and highlight the differences with others affective states.

All the theories we talked about are not from the same research community. The emotion field is highly multi-disciplinary, and it is possible that the authors do not refer to the same things when talking about emotions [START_REF] Kappas | The science of emotion as a multidisciplinary research paradigm[END_REF]. It is possible that these theories are not contradictory, but can be understood as a whole [START_REF] Cosnier | Psychologie des émotions et des sentiments[END_REF]. For instance, we can concede that emotions are the result of a cognitive appraisal of the situation, considering the expectations of the individual. But at the same time, we can state that when the person is not attentive to the situation, and an unexpected event occurs, a shorter emotional circuit is preferred. And then, an automatic and uncontrolled emotion reaction rises. The delayed appraisal may then regulate the emotional response.

We divided the properties of emotions in three aspects that are detailed below:

(1) automatic process facing fundamental needs, (2) cognitive appraisal considering interests, and (3) communication tool.

(1) Some basic emotions are pre-cabled with neuro-physiological circuits. They are activated when an event, that is related to fundamental needs, occurs. They have strong and automatic body responses, which is not easily controllable. These circuits are a-priori unconscious.

(2) In a higher level, when the event does not require an urgent reaction (e.g., when it is not something totally unexpected), the cognitive system is activated to appraise the situation. It assesses if the event fits the individual's motivation and interests, and what action should be done to change the situation. This cognitive process is highly dependent on the context and the individual's psycho-physical state. All this can be performed unconsciously or consciously. When conscious, a subjective representation of the internal state can be made.

There is a link between (1) and (2). The cognition can regulate the automatic body response, and the cognition can take into account the body state in the evaluation process.

(3) Emotions have a communication purpose. It enables to communicate action intention. It has a social regulation purpose and can strengthen affinities. It enables to engage in a conversation and is a tool to ease communication. Emotions can also propagate between individuals. And emotions can be acted. Hence, one has to notice the difference between emotional response and emotion expression.

Other properties that are suggested in most of the theories are the following: An emotion is induced by a specific and ephemeral event, called stimuli. The response is fast (less than a second) and the duration is short (several seconds or minutes).

All these properties of emotions are summarized again in the next sub-section, considering why emotions are important in the field of Social Robotics. Before that it is necessary to present other affective states that are not emotions, and show that they should not be mingled. Indeed, emotions, feelings, passions, and moods are all affective states [START_REF] Cosnier | Psychologie des émotions et des sentiments[END_REF].

A feeling lasts longer than an emotion and is related to a precise object. It can continue whenever the object is in a distance [START_REF] Cosnier | Psychologie des émotions et des sentiments[END_REF].

A passion also does not have time limit, and is directed to an object not necessary in presence. But besides, there is a dependence of the individual to this object [START_REF] Cosnier | Psychologie des émotions et des sentiments[END_REF].

A mood is medium term (several hours or sometimes days), but it is not linked to a precise object or event. It is more difficult to determine the cause of it as several events are involved. It is a background state, with a a lower intensity that an emotion [START_REF] Desmet | Mood measurement with Pick-A-Mood: review of current methods and design of a pictorial self-report scale[END_REF]) but when at its maximum level, it seems like a continuous emotion (e.g., Irritable can be converted into continuous anger). [START_REF] Beedie | Distinctions between emotion and mood[END_REF][START_REF] Scherer | What are emotions? And how can they be measured?[END_REF][START_REF] Cosnier | Psychologie des émotions et des sentiments[END_REF] Each of these affective states can be associated to a subjective representation, which is called an affect. Then the individual is consciously aware of its affective state [START_REF] Cosnier | Psychologie des émotions et des sentiments[END_REF].

Conclusion on emotional theories

To conclude this introduction on how the emotional process works, and what it involves, we summarize some emotion properties and present how they are beneficial for Social Robotics. We also present some challenges, and the emotional components we need in our work.

Emotion applications in Social Robotics

In the literature review we found that emotions are at the basis of humans and some animals evolution. Hence, emotion capabilities appear as an evidence of being alive.

A field part of Social Robotics work at developing Geminoids. These are robots aim to look the closest to humans as possible. We can cite the examples of Sophia (Hanson Robotics, 2018) or Erica [START_REF] Glas | Erica: The erato intelligent conversational android[END_REF]. Thus, emotions are a component taken into account in their design.

We also found that emotions are highly beneficial for social interactions. They ease the communication, create engagement and generate a strong link between the partners. The empathy plays an important role in this aspect. This could make human-robot interactions more natural and spontaneous. Several experiments study the benefits of using empathic robots [START_REF] Riek | Real-time empathy: Facial mimicry on a robot[END_REF][START_REF] Hegel | Playing a different imitation game: Interaction with an Empathic Android Robot[END_REF][START_REF] Castellano | Multimodal affect modeling and recognition for empathic robot companions[END_REF][START_REF] Leite | The influence of empathy in human-robot relations[END_REF]. Some of them target educational applications.

Finally, we noticed that emotions are not only basic abilities, or conversational tools, it also has a high level aspect. They require to define motivation and interests, and use high level cognition for appraisal and decision making for action. These cognitive process are useful to design artificial intelligences. Several robots use emotion based decision making models [START_REF] Velásquez | Modeling emotion-based decision-making[END_REF][START_REF] Yu | An emotion-based approach to decision making and self learning in autonomous robot control[END_REF][START_REF] Hollinger | Design of a social mobile robot using emotionbased decision mechanisms[END_REF].

Emotional studies challenges

Despite these appealing possibilities, emotions are challenging to study. (1) Emotions have many sources, many purposes, several levels of complexity (from reflex response, to social outcomes and relationship construction), and can be from specific (basic emotions) to fuzzy (mixed emotions). (2) It uses several circuits, with several timing, and has several connections between the cognitive and physical systems, between controlled and automatic responses, and between unconscious and subjective representation. (3) The physiology does not imply emotion experience (i.e., it can be aroused by other environmental causes), the emotion expression does not imply emotion experience (e.g., it can be acted), and subjective representations do not imply emotion experience (e.g., the individual thought can be biased).

The link with our research

In this thesis, we investigate if a given emotion, experienced by an individual, alters the tactile pattern of the contact between two partners, during a tactile social interaction. In other words, we want to see if a connotative component (that depends on the experienced emotion) is added to the informative message communicated in the tactile interaction itself. So, we are interested in the behavioral component of emotions, we want to see if the tactile modality is a channel for this behavior expression during social interaction, and if this behavior can be detected by an artificial device.

The emotion studied here has to be spontaneous in order to let us observe its natural expression. Besides, the emotion has to be induced by an external source from the interaction, so that it does not interfere with the expression.

In practice, to answer these questions using experiments, several more components are required. In order to study spontaneous emotions, we need to elicit them in the participant. Indeed, emotions are generated by an event, so this event has to occur in the lab context. So, we have to find what kind of stimuli (i.e., emotionally charged content) can be used for emotion elicitation. We chose to excite several modalities to maximize the chances to have a strong emotional response.

Another component we need is a way to measure the experienced emotion. This is in order to verify that our participants experience an emotion and that it corresponds to what we expected to elicit. Measuring the physiological response is a way to infer some dimensions of the emotions (e.g., pleasure, arousal). We tried to use it but we were not able to process efficiently the data. Another way is to use questionnaires to assess the subjective representation of the emotion. This implies that the emotion is sufficiently strong and the emotional process is completed.

To sum-up, we need to elicit emotions, to measure them, to see if it is communicated through touch during a social interaction, and to see if the touch changes are observables by an artificial device. Our contributions in this field are in Chapter 4.

Spatial pressure measurement: a review

This thesis aims to detect if the tactile data, exchanged during a touched social interaction, is altered by the internal state of the user. We chose to study one interaction in particular, which is handshake. Hence, we needed a system able to measure this tactile data, on the hand. This section gives some cues in order to design our own device.

We first provide some properties of the tactile modality, and some requirements for artificial skin design. Then, we present a classification of the technologies that exist, and which were developed in order to sensitize the body of robots or machines.

Finally, we justify the technology we chose.

Touch properties

Vision, audition, and touch are often viewed as the three most important senses of humans and animals. They enable them to perceive their environment. Beyond other senses, touch allows us to act on the environment at the same time we perceive it, which confers the richness of this modality. Touch enables us to prevent from injury, to keep balance, to control movements, and even to be self-aware. Touch offers the ability to manipulate objects/tools, which may have been a key for human dexterity and evolution.

Touch can be defined by several characteristics as follows: It can be active (e.g., put a ball in rotation to assess its inertia, move the finger on a surface to perceive its roughness), or passive. It can be cutaneous (i.e., all that is measured by the skin), or kinesthetic (i.e., the information form the joints, tendons, muscles). The combination of the cutaneous with the kinesthetic channels is called haptic. The tactile modality is cutaneous only. As it is linked with the skin anatomy, it is defined by what the mechanoreceptors in the skin can measure. Four types of mechanoreceptors measuring displacement of the skin exist, they can detect high or low frequency stimuli, and have a fast or slow adaptation rate. They also have different spatial resolutions and displacement ranges [START_REF] Olausson | The neurophysiology of unmyelinated tactile afferents[END_REF].

Artificial skins: requirements

Not only the living beings embody the sense of touch, machines also highly need physical perception. These last years this field developed a lot. Co-bots use kinesthesis to be force-controlled, and tactile screens use the cutaneous aspect. Social robots need both aspects to fully understand the social cues during an interaction.

Considering the tactile modality only, researchers in Social Robotics state that it is necessary to measure either the location, and the time aspect of the contact. This is in order to infer the "social load" dimension during social human-robot interactions [START_REF] Silvera-Tawil | Interpretation of social touch on an artificial arm covered with an EIT-based sensitive skin[END_REF]. This justifies the need of artificial skins in this field.

Several literature reviews tackle the topic of artificial skins, either from the technical point of view [START_REF] Dahiya | Robotic tactile sensing: technologies and system[END_REF][START_REF] Dahiya | Directions toward effective utilization of tactile skin: A review[END_REF], or from the applications on social human-robot interactions point of view (Silvera-Tawil, Rye, and Velonaki, 2015). They give requirements for artificial skins. The skin should measure constant and dynamic forces, should be accurate at measuring large ranges of forces, and with a low response time. Beyond the force aspect, the skin should detect the force direction (i.e., the sheer or normal directions), the temperature, and local acceleration data: it has to be multi-modal. The sensitive area should be continuous in space, or at least, with high spatial resolution. It should also be multi-touch sensitive, and robust (considering the mechanics, the electronics, and the transduction stability through time and temperature). Finally, the mechanical aspects of the skin should fit the robot's geometry and purpose. So, it has to be soft (for comfort but also to absorb impacts), flexible, and even stretchable.

All these requirements make the artificial skin design task very challenging, but it is also emphasized by all the components that have to be addressed. These components are: the surface material (i.e., thickness, hardness, texture), the transducer technology, the wiring, the signal conditioning, the local processing, the electronics, the communication, the low level interpretation of the data, and data fusion and context-aware process.

A technological review

Despite the complexity of artificial skin design, several technologies and implemented systems have been developed. These technologies can be classified given the transducers used, and the skin architecture.

The first architecture that is usually used, is to connect a collection of individual sensitive cells to the acquisition device. Each cell uses a physical transduction principle, that converts the applied pressure along the sensitive area, into an electric signal. These transducers use many working principle: optical, piezo-resistive, piezo-capacitive, magnetic, transistors, and so on. However, in most of the cases, the information path is as follows: the applied force on an area is transfered to a rigid structure which spreads the force in an defined surface. Hence, this surface is exposed to a pressure which deforms a test body (material with a known stiffness).

This deformation creates a displacement, which alters the electric characteristics of the transducer. We present below some examples of piezo-sensitive cells (also called taxels in the literature).

Transducers examples

Optical: This technology is called non-coupled, as between the mechanical displacement and the electric signal, the information passes through an additional modality.

Here, the intermediary modality is light. As an example, we can refer to [START_REF] Ohmura | Conformable and scalable tactile sensor skin for curved surfaces[END_REF]. They used a light emitter next to a receiver, below a "transparent" foam layer. The light is emitted in direction of the foam where it is diffused and reflected to the receiver. The higher the pressure the more light is reflected. Everything holds in a flexible substrate. This was designed to cover a humanoid robot arm. Another example is a multi-direction force-sensitive cube [START_REF] Kadowaki | Development of soft sensor exterior embedded with multi-axis deformable tactile sensor system[END_REF]. A schematic is presented in Figure 2.3. Three light emitters are placed in the bottom of the cube, and three receivers are on the top. It is then possible to detect the position and orientation of the top surface of the cube compared to the bottom surface, and hence, estimate the magnitude and direction of the applied force and torque. An advantage of optical technologies is the immunity to electromagnetic disturbance. However, it is highly energy consuming.

Magnetic: The same way as optical techniques, the magnetic piezo-sensitive cells are non-coupled. Instead of light beams, it uses a magnetic field. Usually, a small magnet is placed in a deformable substrate, and Hall-effect sensors are placed below [START_REF] Chathuranga | Disposable soft 3 axis force sensor for biomedical applications[END_REF][START_REF] Paulino | Low-cost 3-axis soft tactile sensors for the humanfriendly robot Vizzy[END_REF]. It is also possible to have a 3-dimensional force representation (see Figure 2.4). Compared to optical sensors, it is lower power consuming, however it is sensitive to magnetic interferences.

Transistors: It is possible to use electrical properties of specific materials in order to manufacture field effect transistors which are sensitive to pressure. Indeed, This technology is very compact, highly sensitive, enables to measure dynamic responses, and embodies a local basic data processing. However, this is still difficult to have a high scalability due to the manufacturing complexity. Resistive: The piezo-resistive sensors consist in a conductive rubber/polymer film, or fabric, sandwiched between two electrodes. When pressed, the resistance between the electrodes is reduced, or increases, depending on the architecture of the sensor. Several phenomenon can be used.

(1) The resistance can be changed due to a geometric variation. Most of all conductive/resistive materials that are used as a dipole, show a lower resistance value when the distance between the two poles is lower, or when the section to be crossed between the poles is larger. When pressed, or stretched one or two of these variables are changed.

(2) The resistance can change due to the modification of the surface of contact between the electrodes and the resistive layer changed. When pressed the surface increases, so the resistance drops down, until a certain point when the electrode is fully in contact. The sensitivity of the sensor, and the range of pressure to apply before saturation, are highly depend on the roughness of the electrode.

(3) The resistance changes because the resistivity of the material depends on the pressure applied. This happens with heterogeneous materials, such as highly conductive particles included in a highly resistive foam or silicon matrix. When pressed, the phenomenon (1) and (2) happens at the microscopic level: the surface of contact between the particles and the silicone increases while the distance that separates them diminishes, so the resistance goes down. Two examples are shown by Kwon et al., 2016, with porous foam, and[START_REF] Pan | An ultra-sensitive resistive pressure sensor based on hollowsphere microstructure induced elasticity in conducting polymer film[END_REF] with conductive spheres in silicon. Choong et al., 2014 also used micro structures (pyramids), but only at the surface of a resistive film, to produce (1). These sensors are so sensitives that it is possible to measure heart beat only with the blood pressure variation. At a larger scale, sensors using mostly (2) are commercialized (Interlink Electronics, 2018). In this latter design, the electrodes are side by side and use the polymer as a bridge between them. So when pressed, the surface of contact increases leading to a decrease of the resistance, but also the thickness of the bridge simple electronics, and to be simple to construct [START_REF] Dahiya | Robotic tactile sensing: technologies and system[END_REF]. However, it induces non linearity and hysteresis. This list of transducer technologies is not exhaustive. We can for instance cite ultrasonic methods, that measure perturbations of an acoustic wave propagate in a material when a pressure is exerted [START_REF] Liu | An acoustic multi-touch sensing method using amplitude disturbed ultrasonic wave diffraction patterns[END_REF]. This allows a space-continuous measurement, however the touched surface needs to be rigid.

What we can notice about these technologies is that the information is conveyed through several modalities (e.g., mechanical, electrical), which requires a conversion at each step. Thus, the overall characteristics of the sensor depend on the efficiency of conversion of each step. An element that often introduce hysteresis and long response time is the test body that converts the pressure into displacement. Besides, this component often defines the range of force that the sensor is able to measure.

So, a lot of attention should be paid to the test body.

Types of architecture

Another aspect that characterizes a tactile skin, is the architecture chosen to sensitize the whole skin surface. Indeed, we presented some transducer methods to convert a force into an electrical signal, but this is done locally. Strategies exit to use these transducers in different points of the skin, without duplicating each cell with associated electronics thousands of times.

Matrix: This consists in creating rows and columns whose each intersection is a piezo-sensitive cell. Instead of wiring each cell individually, one column is excited while one line is read. This gives the data of the corresponding cell. Using this process, all the lines and columns are scanned. This is used for instance with capacitive transducers [START_REF] Lee | A flexible polymer tactile sensor: Fabrication and modular expandability for large area deployment[END_REF], or with piezo-resistive cells [START_REF] Yang | An integrated flexible temperature and tactile sensing array using PI-copper films[END_REF]. The advantage of this technique is that it drastically reduces the complexity of wiring. However, some unexpected phenomenons may occur, mostly with the piezo-resistive technology. These are the cross talk effect [START_REF] Yang | An integrated flexible temperature and tactile sensing array using PI-copper films[END_REF], or the ghost effect [START_REF] Kheddar | A tactile matrix for whole-body humanoid haptic sensing and safe interaction[END_REF].

Fibers: A similar method as matrix is the use of fibers. The fibers are meshed, like for fabrics, but each crossing of perpendicular fibers is a sensitive cell. For instance, we have the system of [START_REF] Lee | Conductive fiber-based ultrasensitive textile pressure sensor for wearable electronics[END_REF]. Each fiber is composed of a conductive core coated with a dielectric material. This uses the piezo-capacitive effect. Gong et al., 2014 used gold nano-wires, and[START_REF] Lou | An ultra-sensitive and rapid response speed graphene pressure sensors for electronic skin and health monitoring[END_REF] used elastic nano-fibers coated with conductive material, to benefit from the piezo-resistive effect.

Electrical tomography: This technique is particular in the sense that no wire is required in the sensitive area. The measurement is performed only in the periphery of the skin, and a finite element reconstruction enables to estimate the map of the electrical characteristics of the skin. An examples of electrical capacitance tomography is presented in Yang, 2010, however, this is not suitable for a robotic skin.

Piezo-resistive films present numerous advantages for this task. It is flexible and stretchable, and it is easier to reproduce the resistance map than the capacitance map (see Figure 2.9). The principle was demonstrated by Alirezaei, Nagakubo, and Kuniyoshi, 2007, then some algorithms were improved by Tawil, Rye, and Velonaki, 2011, and the technology was applied to design a robotic skin and control a robot [START_REF] Pugach | Touch-based admittance control of a robotic arm using neural learning of an artificial skin[END_REF].

Conclusion on the spatial pressure sensors

To conclude this technical review, a large panel of technologies exist to measure the pressure data, in a human-robot interaction scenario. They involve numerous transduction principles and architectures. However, these devices were mostly developped for research purposes, or for specific projects. Few systems are commercialized, or are easily available.

We can cite the following available systems: There are multi-modal modules described in Mittendorfer and Cheng, 2012. There is a collection of 20 piezo-resistive blocs built in a single patch [START_REF] Tekscan | Grip system[END_REF], that was designed for research in ergonomics. This was used in a research about inter-individual and human-robot handshakes [START_REF] Wang | Design and performance of a haptic data acquisition glove[END_REF]. However, this system only measues the contact between the fingers/palm and the object gripped. It does not allow to measure the grasp of the handshake partner on one's hand. Another remark we can do about these technologies, is that in order to increase the robustness, and prevent from wiring issues, the sensors are directly integrated in the final substrate. This substrate forms a patch, with a given shape. This shape is highly dependent on the application the designers had in mind when making the sensor. This makes difficult to find a sensor that corresponds to our application. In our case, we are interested in sensitive gloves to measure handshake data. However, most of the sensitive gloves are made to measure our own action on the environment (e.g., how we touch objects Tekscan, 2018, to control a device with the finger movement [START_REF] Manus | Manus VR[END_REF]. So, tactile skins have to be custom made every time we need it. To our knowledge, the only type of devices that aim to be integrated in any kind of application are the modular sensors [START_REF] Mittendorfer | Integrating discrete force cells into multi-modal artificial skin[END_REF]. However, they are still too large for handshake measurements.

Finally, we chose to use FSR sensors Interlink Electronics, 2018, placed on a glove, given the locations that are the most relevant for handshake interaction studies.

These sensors have the advantage to be available, flexible, and easy to integrate.

Such a glove was used in Chapter 3, however, in Chapter 4 and Chapter 5, we designed our own piezo-resistive cells in order to increase compactness and robustness, and multiply the regions of interest.

How to go beyond the state of the art?

In this chapter, we presented four reviews about: the handshake manner, the personality, the emotions, and the artificial skins. These four components are used in this thesis as we tackle the following question: Is it possible to infer the internal state of an individual (such as personality or emotion), during a tactile social interaction (such has greeting handshake), using the tactile data collected by an artificial system? This question tackles several aspects of the literature that were not answered to the best of our knowledge.

Handshake

Concerning the handshake interaction, several considerations have been discussed.

(1) It is a type of greeting interaction. Greeting can have many forms and meanings. However, it seems it is conventionalized depending on culture and social context. It also seems that greeting often involves touch and gives access to the internal state.

At least the handshake interaction was stated to have these properties. These are sociological results but when reviewing in this field, it is difficult to find large scale studies, like cross-cultural questionnaires, for numerical validation. This would be beneficial, for instance, to help a social robot to know what would be the most socially accepted interaction. And maybe, performing physical measurements would give cues about the strength a robot should exert.

(2) It is stated that during handshaking, three messages are exchanged. However, these messages are theoretical and we do not know what the physical content can be. For instance we can wonder if during the "preparation for the relationship to follow", the personality or the emotional state are exchanged through tactile data.

(3) If we posit the usual handshake interaction used in European countries is the standard one (Figure 2.1a)), to which extent is it standardized? Two views are opposed about this question: fully standardized, or an "extension of ourselves". This question, however, needs to be answered, to know if we can expect to detect social cues or internal state cues in the handshake manner.

(2) and (3) are tackled in this thesis as we try to observe variations in the handshake depending on the internal state, but also we observe how repeatable is a handshake for an individual.

Personality

The goal of our thesis is to infer emotions in the tactile data of a social interaction.

However, as a first step, we can enlarge the question to "internal state characteristics". Personality is part of the internal state, and we noticed in the review that several properties make it relevant in our research. These properties are: impact on social interaction, behavioral expressions, long term characteristics, and possibility to measure it.

We provided the history of personality modeling that led to develop personality questionnaires. We found out that most of these researches, lead to a limited number of personality traits. Among them, two are always cited: extroversion and neuroticism degrees. In the Big5 Inventory [START_REF] John | The Big Five trait taxonomy: History, measurement, and theoretical perspectives[END_REF], the questions about extroversion degree concern sociability, and new situation seeking. The neuroticism trait is link with emotional stability. This is another reason why these traits, are important in our research, and in Social Robotics in general.

Many studies in Human-Computer Interaction or Social Robotics try to infer personality observing the behavior or preferences of the user. We contribute in this field as we try in Chapter 3 to observe differences in the handshake manner depending on extroversion degree.

Emotions

The literature review about emotions was dense, given the large variety of theories about it, the large number of components it involves, and the numerous implications it has. A detailed summary of these implications, about emotion properties, Social Robotics applications, the place emotions have in our work, was proposed in the emotion definition Section 2.4.3 and conclusion Section 2.4.4. Some keywords are reminded here: the strong and automatic response; the physiological, behavioral and cognitive aspects of the response; the high dependence to the context; the dependence to the individual's psycho-physical state (i.e., individual dependent property); the unconscious or conscious process/ representation; the communication properties; the communication of action intention; the enhanced engagement in conversation; the empathy; the emotion contagion; the difference between emotional response and emotion expression; the possibility to act emotions. Among these properties, there is the behavioral response that indicates that emotions can be measured. We want to contribute in this field, as we try to measure the emotion using touch during social interactions. This is presented in Chapter 4.

However, several challenges remain to study emotions, and some are tackled in Chapter 4.

(1) Few efficient tools were found to elicit emotions in participants of a lab experiment (Section 4.3). However, it is important to control the emotion of the participant, which is in general the independent variable of the study, in a reliable way.

We contribute in this field as we developed and evaluated an elicitation tool.

(2) Studies about emotions are highly time dependent. It means that the dynamic aspect of emotions makes it difficult to synchronize the measurements with the emotion elicited and experienced. However, in Section 4.3.1, little contributions were found that characterize numerically this time aspect.

(3) In Section 4.2, we discussed methods to measure emotions. One of them, that we use in our work, evaluates the subjective representation of emotions. However, the review on emotion theories did not answer where exactly is the limit, or interaction, between subjective representation and the real emotion experienced.

(4) Finally, it seems interesting, from the Social Robotics point of view, to understand or find a way to detect the differences between an acted emotion for discourse purposes, and an emotion felt by the individual. This could prevent from a missinterpretation by the robot.

Artificial skins

In this chapter, we also reviewed the devices that enable to measure spacial pressure. We showed that the touch sense is rich and provides substantial information about the environment. This is what motivates robots to have tactile abilities. Besides, it was found that touch has a social load during touched social interaction, which is important for Social Robotics. As the location and time aspects of touch are meaningful, this leads to the creation of artificial skins.

A high number of requirements are associated with these skins: flexibility, multimodality, multi-touch, large surface, and so on. Numerous technologies tackle these challenges. However, we found that they often suffer from a lack of scalability, miniaturization, and multi-range measurement properties.

Besides, little devices are available in the market, and the ones that exist make it difficult to match with the targeted application. FSR technology appears to be widely used in this field, and we found several examples of instrumented gloves. These gloves are used for handshake characterization, which corresponds to the direction of our research. We contribute in this field as we designed several gloves that use FSR technology (Section 3.4.1). And we also designed our own sensors using piezoresistive property (Section 4.5.1).

Other challenges were presented in the introduction chapter (Section 1.3). They are about social touch studies, with the time consuming, multi-modal signals, or social context dependency concerns. We also contribute in this field, as the overall thesis tackles physical measures during touched social interactions.

Introduction

In Chapter 2 Section 2.3, we presented some theoretical background on personality.

Two points are worth noting: (1) the personality is defined by the behavior of an individual facing a certain situation, and (2) this is a long lasting internal trait.

In Chapter 2 Section 2.2, we highlighted that even though handshake is commonly and daily used, it has a strong meaning. The type of handshake can witness for cultural or context differences. Furthermore, it is a way to give access to oneself. Thus, it may be a way to express one's personality, and the behavioral effect of this personality may be observable directly in the physical contact between the individuals.

As a consequence, it appears relevant to study the link between the handshake manner and personality. The long term aspect of this psychological dimension is even more an advantage for experiments since it enables to measure the participant's personality separately from the handshake interaction. Besides, gender is another constant characteristic that represents either physical or psychological differences [START_REF] Chaplin | Handshaking, gender, personality, and first impressions[END_REF].

In this chapter, we are investigating the handshake manner depending on personality and gender. We first present the related work concerning this question in Section 3.2. We also introduce some work that performed quantitative measurements of handshake in order to model them. Then, a preliminary experiment is depicted in Section 3.3 to detect contact points in interpersonal handshake. In Section 3.4, we describe a device we designed and developed to measure the pressure distribution and movements during handshake. Finally, an experiment to observe the effect of personality and gender on numerous handshake samples from 36 participants is presented in Section 3.5.

Does personality impact handshake? A review

Personality, gender, and handshake

The link between personality and handshake has been investigated by several researchers, either from psycho-sociology, management, or Social Robotics. Åström, 1994 used a specific personality model [START_REF] Cecarec | Matning av psykogena behov med frageformularsteknik [Measuring psychogenic needs by questionnaire[END_REF], based on 5 factors, to describe the participants of his experiment. These participants had to evaluate subjectively handshakes they were performing. The results showed that strength was positively correlated with rational dominance of women and aggressive nonconformism of men. He also observed that these two personality traits from Cecarec's model correspond to extroversion. [START_REF] Chaplin | Handshaking, gender, personality, and first impressions[END_REF] used the Big5 questionnaire completed by shyness, emotional expressiveness, and positive-negative affect. Handshakes were blindly and subjectively evaluated by pairs of students. It was found that strong handshakes are positively related to extroversion and emotional expressiveness, and, negatively linked to shyness and neuroticism. A firmer handshake gives a better impression. Male handshakes are stronger. Chaplin also found correlations between observed features: a firm handshake is linked with its duration, if there is eye contact, completeness of grip, and vigor.

From the research in management, [START_REF] Stewart | Exploring the handshake in employment interviews[END_REF] tackled job interviews.The study used the Big5 model to rate 98 students that participated to fake job interviews.

Their handshake manner was evaluated and the authors found a positive correlation of the strength with both gender (i.e., being male) and extroversion. There is also a relation between vigor and extroversion, and between completeness of grip and gender. This shows that personality or gender can alter how handshake is perceived, while other studies highlighted the social importance of handshake. Dolcos et al., 2012 observed neurological and qualitative variations depending on viewing videos of people handshaking or not before interacting. The presence of handshake caused better evaluations of the interaction, and the partner had better interest for further interactions. These results also depend on culture and gender as investigated by [START_REF] Katsumi | When nonverbal greetings "make it or break it": the role of ethnicity and gender in the effect of handshake on social appraisals[END_REF] To continue with handshake as a beneficial tool, Bevan and Stanton Fraser, 2015 studied handshake with a tele-operated robot. The authors showed it promoted better negotiation outcomes. This comforts us in believing that important information is conveyed by handshake and that personality may take a large part in it.

Other studies, cited in this paragraph, aimed to detect psychological data of individuals during handshake, using quantitative data (i.e., instrumented gloves measuring pressure and movement). In Garg, Mukherjee, and Rajaram, 2017, the participants were asked to act a personality, either submissive or dominant. They were able to discriminate these conditions using the handshake data, mostly the movement features. However, the fact that the personality is acted in their experiment goes against two principles: (1) the personality characterizes an individual and (2) the personality cannot be easily changed. Thus, it does not correspond to ecological measurements. [START_REF] Tagne | Measurement and analysis of physical parameters of the handshake between two persons according to simple social contexts[END_REF][START_REF] Tagne | Measurement and analysis of physical parameters of the handshake between two persons according to simple social contexts[END_REF]Melnyk et al., 2014 worked on the characterization of the handshake movement and pressure. They were able to recognize the social context of the handshake. However the personality impact is notified as future work.

To conclude this section, the personality and gender effects on handshake have been observed subjectively in psychological studies. Some works have been started to quantize the results using instrumented gloves. However, a lot remains to be done in this field, and we contribute to it in this chapter, creating a corpus of quantitative handshake measurements and assessing their correlation them with gender and personality.

Quantitative measurements of handshake

Other researchers aim at collecting quantitative data and compute models of either interpersonal handshakes, or human-robot handshakes. These studies are mostly part of Social Robotics to make human-robot handshakes more natural, or part of Human-Computer Interaction to develop tele-operated handshake devices. glove with 9 piezo-resistive cells to control a tele-operated handshake system. Pedemonte, Laliberté, and Gosselin, 2016 measured the grip force while handshaking a 3D-printed hand, in order to design a tele-operated handshake system. One of the most recent work on quantifying tactile contact during handshake was done by [START_REF] Knoop | Handshakiness: Benchmarking for human-robot hand interactions[END_REF]. The authors measured the contact surface during interpersonal and human-robot handshakes using colored paint. They also measured the maximum pressure exerted using a piezo-sensitive film. Thus, the collected data is continuous by space, but do not contain the time aspect of the handshakes.

As a result, this review shows that the movement during handshake has been quantitatively studied, modeled, and implemented in social robots. However, to the best of our knowledge, no intent to infer the internal state of individuals was done. Concerning the tactile data, the state of the research field is even less advanced. To our knowledge, very few studies recorded the pressure exchanged during handshake using sensitized gloves. Besides, these gloves are very heterogeneous, in terms of the sensor technology used, and the number and location of the sensors. This makes difficult to compare the results, but even though, these results are not publicly available. And, as for the movement data, no inference of internal state was performed using tactile data during handshake.

Contribution of this chapter

In the two previous subsections, we presented studies in psychology that found effects of gender and personality on handshake subjective evaluations. Other researches enabled to model the handshake movement, and a few measured the tactile data. However, if the data is recorded, observed, and sometimes copied by a social robot, or only a robotic arm, to our knowledge, no study has investigated how the internal state of the individual (i.e., gender and personality, or even mood and emotional state) alters these data. Our contribution goes beyond the state of the art, as it tackle this specific question: How can we infer gender and personality of an individual given his hand movement and pressure during a handshake interaction?

To be more precise, we do not observe the effect of all of the 5 personality traits (see the personality theory in Chapter 2 Section 2.3) on handshake. In several of the previous cited psychological studies, the degree of extroversion was found to have an effect on handshake. Besides, this personality trait is seen as a facilitator for more social interaction. [START_REF] Lopes | Emotional intelligence and social interaction[END_REF] found that extroversion is positively correlated with higher self-reports and positive interaction with friends. Thus we posit this personality trait is more appropriate to be communicated during an engagement interaction which is the greeting handshake. This is why we study this trait in this chapter.

We also can remind a remark we did concerning the review of pressure measurement during handshake: the data is not publicly available, and the comparisons between setups is difficult. In this chapter we also design our own instrumented glove (see Section 3.4.1). But it concerned a first experiment in our thesis. In the two next chapters we improved the glove with more sensors, and try to compute geometric components that could be computed with any setup, no matter the resolution and location of the sensors.

Geometric aspect of handshake (Experiment 1)

We described in the previous sections what is a handshake from the social point of view, and we presented some psychological aspects of this interaction. However, these studies remain subjective as there is no physical measurement on the handshake manner itself. In order to design an objective measurement system, it is important to know the positions of contact between the two hands during an interpersonal handshake. Thus, we carried out an experiment in which a participant wears a glove soaked with ink and handshake another person with a clean glove. We could determine the most often touched zones, which are the most relevant locations to put the sensors on. This work was briefly presented in [START_REF] Orefice | Let's handshake and I'll know who you are: Gender and personality discrimination in human-human and human-robot handshaking interaction[END_REF], and at the same time, [START_REF] Knoop | Handshakiness: Benchmarking for human-robot hand interactions[END_REF] published another study with the same purpose. As a consequence, the results in this section are compared with this latter publication.

Protocol and data extraction

The goal of this experiment is to determine the most often touched areas during interpersonal handshake in order to position pressure sensors on a future instrumented glove. As the sensors remain in fixed locations for all the conditions, the identified areas have to be the most generalizable as possible. Thus, we choose participants that belong to a large spectrum of characteristics. As [START_REF] Chaplin | Handshaking, gender, personality, and first impressions[END_REF] noticed that males and extroverted persons have a firmer handshake, we can expect that gender and degree of extroversion would change the deformation of the partner's hand, thus altering the contact area. Beside, the size of the hands was found to be an important parameter as it directly impacts the contact surface.

For these reasons, the data was monitored from 8 participants (also called senders)

as a combination of gender (male, female), extroversion personality trait (introvert, extrovert), and hand size (small, large), handshaking both a small handed woman and a large handed man. The two latter who were called receivers can also be seen as experimenters. The personality traits were collected using the Big5 questionnaire [START_REF] John | The Big Five trait taxonomy: History, measurement, and theoretical perspectives[END_REF] presented in Chapter 2 Section 2.3. We attributed the extrovert personality trait to a participant when he/she scored more than 3.0 in the corresponding questions. We computed the median value of the hand length of the 10 participants (i.e., 173mm), and attributed the large hand characteristic to a participant when his/her hand is longer than 173mm. The participants aged between 25 and 59 and were 33 years-old in average. The distribution of the participant characteristics is given in Table 3.1. The experimental protocol was as follows: The two experimenters stay in a room wearing a latex glove on the right hand to protect from the ink. They also wear a white cotton glove that they change at each participant. When the participant arrives, he/she also puts on the latex and cotton gloves. He/she immerses the right hand in a blue ink container and wrings it. He/she performs handshakes randomly with both experimenters, removes the glove, measures his/her hand, and fills the personality questionnaire. During this time, the experimenters take pictures of their glove, on 4 sides, before removing it.

In order to extract the contact areas from the pictures, several steps were performed for each side of the hands. One example is presented

Results

After extracting the data of the 16 printed gloves, we separated the ones of the largehanded receiver from the ones of the small-handed, and averaged the meshes. This leads to the Figure 3.2. It can be seen that the small hand is more fully covered by ink than the large one. The large hand has untouched zones in the palm. This means that when handshaking a small hand, one fully grasps the hand and both palms touch each other. However, when the hand is larger, one would close his fingers only on the top and bottom of the partner's hand, without being able to bring it closer.

Hence, the palms do not contact. This may suggest that the relevant information is held by the actions of the figers (top or bottom), not in the palm contact.

We merged all the gloves to plot Figure 3.4. The contact area is very large. Thus, it would involve a lot of sensors to represent all the contact pressure distribution.

We decided to use the location that are close to always touched. They are represented by circles in the figure, and 8 sensors will be placed there. However, it may have been interesting to observe the zones that are not always touched as they can give behavioral information. For the rest of our work, we divided the positions and the sensors in two groups: the ones that are touched by the participant (group r, as the information is received by the experimenter) and the ones that are touched by the experimenter (group s, as sent by the experimenter). We decided to put sensors in zones belonging to both groups because it enables to differentiate the pressure applied by each partner (participant and experimenter). Indeed, as we want to detect intrinsic information from the participant, it is interesting to study how his/her response depends on the experimenter's action.

Before concluding this section, we can compare our results with other studies.

Our work was briefly presented in a paper we published as a conference proceeding [START_REF] Orefice | Let's handshake and I'll know who you are: Gender and personality discrimination in human-human and human-robot handshaking interaction[END_REF]. A few month later, [START_REF] Knoop | Handshakiness: Benchmarking for human-robot hand interactions[END_REF] carried out a similar experiment. 12 handshakes were performed within heterogeneously hand-sized subjects, with paint transfer (directly in the skin that time). The found contact zones are presented in Figure 3.3. Their results are consistent with ours as the most often touched zones have the same locations. We also can see a higher variability of the thumb's position on the experimenter's hand back the same way as in our result figure. This comforts us on the choice of the sensor positions.

Conclusion of the experiment

In the previous described sections, an experiment that aimed to detect the contact areas during interpersonal handshakes, considering different hand sizes and internal states of the participants like gender and extroversion personality trait was presented. This experiment enabled to find the contact areas that are close to always touched. These locations are the most appropriate to sensitize as they will always provide a signal during handshake. However, we noticed that some zones are not always touched, which may make them relevant to discriminate behavioral aspects (e.g., maybe an extrovert person touches more sensors than an introvert one). When more sensors are available, these areas should also be sensitized. After the analysis, we selected 8 locations. We also made an assumption about the role of these locations, as they can either represent one partner's action or the action of the other. This assumption was validated by another quantitative study in Appendix A.

At the end, based on these 8 locations, we design an instrumented glove (see Section 3.4.1), with 8 discrete pressure sensors. This glove is used to collect numerous handshake interactions, in a natural manner, with a large panel of participants having different internal characteristics (i.e., gender and degree of extroversion). The overall procedure is repeated for the human-robot experiment in Section 3.6, even though less participants are involved.

3.4 Pressure and movement in handshake: a dedicated measurement system

Design of an instrumented glove

To measure the data during interpersonal handshakes, we designed a glove on which we sewed piezo-resistive sensors and an Inertial Measurement Unit (IMU). Our choices were directed by two objectives: (1) the measurement device has to be the as "transparent" as possible, and (2) it has to be wearable. The transparency (1) had to enable to measure without disturbing the interaction, so the sensors had to be thin and flexible, the glove material had to be soft, and the wiring had to be hidden.

All the system had to be embedded (2) in order to measure handshakes in real life, directly in the individuals usual environment.

We built a fully embedded system. The electronics diagram is presented in Fig- The glove and sensors: The glove is made of cotton fabric on which 8 pressure sensors are sewed. In order to maintain the sensors in fixed positions, the glove is reinforced by pieces of rubber. A clean elastic fabric covers the whole glove and the electronics.

For pressure measurement, we used piezo-resistive sensors (Interlink FSR-400 and FSR-402). The position of the sensors are indicated in Figure 3.5. Seven sensors are 8mm (active size 5.6mm), and one is 18mm (active size 12mm), corresponding to P 7 as the touched zone is larger (see Figure 3.4). Four of the sensors are located on the thumb (P 0 ), index finger (P 1 ), middle finger (P 2 ), and ring finger (P 3 ) of the experimenter. They belong to the group s. Three are located on the little finger (P 4 ), the top (P 5 ), and the bottom (P 7 ) of the experimenter's hand. They belong to the group r, and one is present in both groups: the palm (P 6 ). The IMU is a MPU-6050 that measures 16-bit 3-axis signed acceleration and angular speed. The sensitivity was set to 4g and 500deg/s for the acceleration and angular speed respectively. It is located on the back of the hand, with the x-axis pointing at the fingers and the z-axis pointing at the thumb (see Figure 3.5). In order to remove the gyrometer bias and compute the initial orientation of the glove, a calibration step was performed before every measurement. During this step, the hand had to be motionless for 2 seconds. , where d is the output of the ADC. Then, given the characteristics of the sensors, the corresponding pressure can be found by the following law: P = exp (a. log (R FSR )+b)

S

, where a and b are coefficients estimated after a calibration phase, and S is the surface of the active part of the sensor. The order of magnitude of the resistance drop due to an applied pressure is as follows: 60kΩ corresponds to a pressure of 23kPa, and 4kΩ to 226kPa.

Control unit: The control unit is a 16MHz Arduino Mini-Pro 5V. The acquisition is done at a 300Hz rate. It manages the communication with the ADC and IMU using the SPI and I2C port, and, stores the data, receives the commands, and so on.

Communication and storage: In order to control the Arduino (i.e., start/stop the acquisition and store the data), and to save the experimental conditions (i.e., date, name of the participant, remarks about the interaction), we developed a User Interface (UI) application on a smartphone that is able to communicate with the Arduino through Bluetooth. We use the Bluetooth module HC-05 that is wired to the serial port of the Arduino. The data recorded by the Arduino during the handshake is stored in real time on a SD card, through the SPI port.

Power supply: All the electronics and sensors are powered by a single-cell Li-ion battery. The 3.7V of the battery are boosted to 5V by a specific circuit, and regulated by the internal circuit of the Arduino. In order to charge the battery we use the TP4056 circuit.

Feature extraction of a handshake

At this stage of our work, we have a glove able to measure time dependent pressure at several points, the acceleration of the hand in 3 directions, and the rotation speed around 3 axis. It is now important to extract relevant features from this data.

Four examples of recorded data with the glove can be seen in Figure 3.8. The coordinates of the (X, Y, Z) axis used to name the following features are shown in Figure 3.5. Z is vertical, and X goes from the experimenter towards the participant.

Firstly, the orientation of the hand seems to be an important characteristic, given advices in commerce or management courses. For instance, when the palm points down, it shows dominance, while it points up to show submissive behavior. Thus, we computed two orientations of the hand: the supination angle (i.e., positive when the palm is up, negative when down (pronation)) θX, and the adduction angle (i.e., positive when the fingers point to the ground, negative otherwise (abduction)) θY.

In order to compute the orientation of the hand, we removed the offset of angular speed while the hand is motionless. Then we integrated it through time while fusing the data with the accelerometer data, using a Kalman filter. Then we extracted the maximum value of these angles reached during the handshake, in the global coordinates: δθX and δθY.

Another important data is the acceleration that characterizes how dynamic is a handshake. As the acceleration is recorded in the accelerometer coordinates, we used the information about orientation of the hand to rotate the data. Moreover, we removed the gravity from the vertical Z axis. The vertical acceleration is displayed in an illustrative example (Figure 3.8). However, the acceleration is not necessarily only vertical, thus we projected it on the principal axis of the movement. This signal is called A, but we also integrated it through time to get the speed S. We computed the amplitudes of these signals (i.e., A amp and S amp ). As the acceleration also has an offset, it creates a continuous increase of speed, so we computed this offset A o f f and the maximum speed reached S max .

The main direction of the movement is characterized by two global angles (αY and αZ). αY implies that the movement is out of the frontal plan and αZ indicates it is out of the sagital plan.

As observed in Figure 3.8, the number of oscillations changes depending on the handshake, so it was expected to be a relevant information for our study. The signal can be split in three phases: the contact (when the touch starts, which can create a shock), the synchrony (when the hands maintain the contact and move in synchrony), and, the release (when the move stops and the pressure drops down). This corresponds to the description previously mentioned by Melnyk et al., 2014. One can also notice that the pressure signal follows these oscillations, most often with the same phase. However in some examples (e.g., Figure 3.8d) sensor P 5 ), it is in phase opposition. This shows the complexity of the interaction, with the forces being the cause of the movement, or its consequence. To count the number of oscillations, we counted the number of intersections of the acceleration with its average line, during the synchrony phase. This feature is called OscNb.

In order to perform all the procedure described above, the acceleration signal was cut from the first instant of movement until the movement ends (the hand has to be static before and after the handshake). This was computed using threshold on the norm of the acceleration and angular speed. The pressure data was also cut like previously, and we extracted the first and last instant the norm of the derivate of all the pressure sensors was higher than 36kPa/s. This enables to compute another important feature: the duration of the pressure TP. All the samples were manually checked and corrected in case of miss-detection.

As the sensors can be pre-constrained by the tight glove, we removed the induced pressure offset by subtracting the minimum value of the pressure for each sensor. Then we get the maximum (P iMax ), mean (P i ) and maximum derivative (dP iMax ) of pressure signals for each of the 8 sensors, and global maximum (P MaxMax ) and mean P. Finally, PG j is the mean signal of the sensors in the group j. These groups are defined during the analysis. All the parameters are summarized in Table 3.2.

Pressure and movement in handshake: impact of personality and gender (Experiment 2)

After having assembled an instrumented glove and defined a methodology to extract measurements from it, we carried out an experiment to see how efficient it is to detect characteristics of a handshake. We first want to see if it is possible to discriminate a firm from a soft handshake, and to propose a method to extract the relevant features and evaluate the detection efficiency. A second objective is to observe if a handshake is able to convey intrinsic information about an individual

Protocol

The experiment has been carried out with 36 participants (11 females (F) and 25 males (M)) and an experimenter. All the participants answered a pre-experiment questionnaire in which they were asked to specify their age, the width and length of their hands, and they filled-in a Big Five Personality test [START_REF] Goldberg | An alternative 'description of personality': the big-five factor structure[END_REF]) that determines the five personality traits (i.e., extroversion, agreeableness, conscientiousness, neuroticism, and openness). We classified the participants in two categories concerning the extroversion personality trait: introverts (I), whose score is below or equal to 3.0 and extroverts (E), whose score is higher than 3.0. We got 20 E and 16 I. The participants aged between 21 and 52, and were 28 years-old in average.

We combined the width (W) and length (Le) of the hands to associate a hand size category to the participants: small hand (S) or large hand (L). We found an important correlation link between W and Le as the Pearson coefficient is r = 0.78, p value < 1e -3 . We computed a linear regression of these two parameters in our dataset as follows: W = b + a * Le, where a = 0.42mm, and b = 7.2mm. We also computed the median values of these parameters: Me W = 85 and Me Le = 182. We projected this median point in the regression line previously computed, given this equation:

sep = Me Le √ 1 + a 2 + Me W * a √ 1 + a 2 = 201.3mm (3.1)
The result (sep) is the separation value between small and large hands, after projecting all the hand sizes in the regression line. This threshold is calculated independently from the gender despite the correlation between the hand size and the gender.

We chose this because it is the geometrical aspect that interests us for the pressure distribution. That makes 18 S and 18 L. The characteristics of the experimenter are as follows: male, introvert, large handed.

The distribution of the participants' characteristics is shown in Table 3.3. The experimenter wears the glove and controls the data acquisition. The experiment occurs in different days, and before seeing each participants, all the sensors are checked to detect wiring failure. We tried to make this experiment as ecological as possible. Thus, it takes place in the participants' environment (their office), the experimental setup is hidden to them and the handshake context is an usual greeting between acquaintances, as handshake is their habitual form of greeting. Despite this wish of natural interaction, some rules had to be expressed: the participant has to initiate the handshake, he/she can begin when the experimenter looks at him/her (because of the calibration phase), the context is to handshake the experimenter like he/she does every other day and several handshakes are performed. The experimenter handshakes normally four times (it is the normal firmness: N), then twice softly (so), twice firmly (fi), once softly, and a last time firmly. It makes a total of 10 handshakes. During (so) the experimenter has an almost passive hand and during (fi) he closes harder and tries to lead the movement. During N the pressure exerted by the experimenter has a mean of 67.7kPa and a standard deviation of 16.2kPa. This has been measured squeezing a passive object with the gloved hand.

The experimenter changed his handshake firmness in order to have a simple experimental condition to detect in the recorded data. It was also to analyze if the behavior of the experimenter has consequences on the participant's action. Finally, it was to see if the gender and extroversion of the participant expresses better while the experimenter is passive, or highly active.

To summarize, the experiment aimed to answer 5 questions: (1) Is it possible to detect the firmness of the experimenter? (2) What is the effect of the firmness on the group r sensors? (3) Can we detect the gender of the participant? (4) Can we detect the degree of extroversion of the participant? (5) Is it easier to detect it during a soft or a firm handshake?

Our first hypothesis (H1) is that the firmness is detectable and the maximum pressure for the group s increases as the handshake is firmer. Our second hypothesis (H2), based on the results of [START_REF] Chaplin | Handshaking, gender, personality, and first impressions[END_REF]Stewart et al., 2008 studies, is that gender is easy to discriminate in normal firmness handshakes : women handshake softer than men. Our third hypothesis (H3), also based on these authors work, is that introvert persons handshake softer than extrovert ones.

Classification and results

Order of magnitude of the data

The first step of our analysis is to observe the distribution of the data. Some values are given Table 3.4. The duration of a handshake is usually less than one second, and there is between 1 and 4 oscillations. There is usually an average upward acceleration, and there is a high variability of the speed reached. The experimenter's hand is often open and inclined to the ground, the movement is out of the frontal plan. There is also a high variability of the pressure exchanged, which can be caused by the experimental conditions. The maximum pressure exerted can be very high (often P MaxMax = 200kPa), but as this variable does not give information about the position of these overloads, one would prefer to use the global mean pressure (P).

Finally, one can notice the two groups of sensors have mostly the same distribution (GP e is for the action of the experimenter and GP p is for the participant, as it will be detailed later in the analysis). When analyzing the sensors individually, we notice that the pressure distribution does not follow a normal law. Many samples have values close to zero. This is mostly seen with the sensors P 5 , which is on the top of the experimenter's hand. The null values for this sensor correspond to a position of the participant's thumb away from the sensor.

Feature reduction

The next step is to reduce the number of features and find the most relevant to describe the handshakes. We have measured and computed features related to 5 physical aspects (i.e., duration, speed, acceleration, orientation, and pressure), and if they are relevant, we try to keep at least one feature for each. We group the features that contribute the same manner to the description of the data, and we remove the ones that are redundant as computed from the same source signal. For instance, we compare the mean, maximum, and maximum derivate of the pressure signals.

We compute the Spearman correlation coefficients (ρ) with a significance level lower than p value < 1e -3. They all show a high correlation: higher than 0.90 between mean and maximum, and higher than 0.75 between mean and maximum deviate. Apart from P 5 (ρ P5-TP = 0.15), none correlation is found with the duration of handshakes.

We choose to use only the maximum pressure of a sensors to represent the measure, as it indicates to which extreme the participant was able to exert a pressure during handshake.

Figure 3.9 shows the correlation circle of the features. This circle converts the correlation matrix into distances between features and projects them in a 2D space that preserves the largest variance of the data. The two components of this space are called principal components and are part of the Principal Component Analysis (PCA). They are linear combinations of all the features. The position of each feature is defined by the weight it has on the linear combination of the components. Thus, a feature close to the periphery is fully described by the 2 components. The interpretation of the distance between two variables close to the periphery is more reliable than the ones in the center. When separated by 90 • , the two variables are fully uncorrelated. We use this figure to visually detect which features form a group that contains the same information, and the ones that are uncorrelated. We also check the values of the correlations using Spearman coefficients.

Observing the figure, we see the movement features are independent to several pressure data (e.g., P 0 , P 1 , P 2 , P 3 .). The duration (TP) is slightly correlated with orientation and speed data (ρ ≈ 0.3 for all comparisons). The same result is found when comparing acceleration (A o f f ,A amp ) with speed and orientation, however there is no correlation with the duration. Thus it is important to keep both acceleration and duration for the rest of the analysis. As ρ Ao f f -Aamp = 0.8, we only keep A amp . The correlation of speed and orientation data is ρ ≈ 0.5, so we keep only one of each (i.e., S amp and δθY). The global mean pressure (P) is slightly correlated with acceleration and speed (ρ ≈ [0.2, 0.3]). This means a firmer handshake is more dynamic. This is consistent with the psychological observations of Chaplin et al., 2000. However, this correlation is low, which is visible in the figure.

So, at this stage and considering the movement features, we have selected: TP, A amp , S amp , and δθY. We still have to study αY, αZ, and OscNb, which are in the center of the circle. The correlation circle indicates that the two first principal components (i.e., F1 and F2) describe 45% of the global variability. This means that using We now tackle the pressure features. Analyzing the sensors individually, we found that P 5 , in the top of the experimenter's hand, is rarely touched (i.e., 76% of the values are bellow of 0.5kPa). This is due to the high variability of the position of the participant's thumb. To overcome this limitation, in the next experiment several sensors are placed in the area. This explains why the correlation coefficients between P 5 and the other sensors are not significant. This sensor is not used in the analysis.

Low correlations are also found for P 4 (ρ ≈ 0.3) while there is not this contact issue (6% of samples bellow 0.5kPa). The correlations between P 1 , P 2 , and P 3 are much higher (ρ ≈ [0.7, 0.8]), thus it is relevant to merge then in a single variable: PG e , the mean of their values, which represents the experimenter's action. P 0 is also correlated with these sensors (ρ ≈ 0.6) but is kept separated. P 7 is correlated to a lower level to all the sensors (ρ ≈ [0.3, 0.5]) and a second group is created, representing the participant's action: GP p , gathering P 4 , P 6 , and P 7 .

The 9 remaining features are: TP, OscNb, A amp , S amp , δθY, αY, P 0Max , GP eM Ax and, GP pMax .

Impact of the conditions on handshake

In order to observe the impact of the experimental conditions on the extracted features, we compute if the mean differences are significant. This can be done by the Analysis of Variance (ANOVA) algorithm. For each feature, we compute a One-Way ANOVA, with a significance level p value < 0.05 (i.e., given the data, the result has a risk of 5% to be obtained by chance). This is the usual value used in cognitive science. When the mean difference is significant, we compute a post-hoc Tukey test, that does a pairwise comparison and gives a tolerance interval of the difference. We give the relative increase or decrease ratio of one group compared to the reference group when it is significant.

Our corpus is composed of two datasets: one homogeneous in terms of experimenter firmness (2 handshakes x 3 types of firmness: soft, normal, firm) with 216 samples of 36 participants called "dataFirmness"; and another one with only normal handshakes (4 per participant), it has 132 samples of 33 participants called "dataNormal". Table 3.5 presents the differences of the features depending on the firmness of the experimenter during handshake, using the "dataFirmness" corpus. Upon the 8 selected variables (see the previous subsection and we removed OscNb), 6 have a significant response. The first result is that soft handshakes last significantly longer than normal or firm ones. The maximum acceleration is also lower. The principal direction of this acceleration (αY) is further from the frontal plan, but this can be due to the difficulty to compute this feature with low acceleration signal. This feature is removed from the rest of the analysis. Concerning the applied pressure the significance of the differences is much higher (i.e., the F value are higher than 10), and the firm handshakes induce higher pressure than normal or soft ones. The effect is stronger for the sensors activated by the experimenter (i.e., the values are multiplied by 2 or 4) than for GP p . This makes sense as the firmness condition directly depends on the experimenter's action. This also shows that the action of the experimenter has consequences on the participant's behavior.

Table 3.6 displays the results for the gender condition, using the "dataNormal" corpus. First, the female handshakes last longer, they reach higher speed while using less sharp acceleration. This means the oscillations are smoother. Moreover, a high significance is found for the hand orientation. The experimenter's hand points down, but one should have in mind it may not be due to a psychological aspect.

It may be due to the cases when the participants are shorter than the experimenter.

Concerning the pressure, gender has an effect on the action of both individuals. The experimenter unconsciously handshakes softer the females, who also handshake softer themselves. However it is difficult to know the origin of this difference. Is it Table 3.7 shows the effect of the participant's degree of extroversion, using the "dataNormal" corpus. This psychological aspect has no effect on the duration of the handshake, but the maximum speed is slightly altered (i.e., introvert persons reach higher speed). The orientation of the hand also changes. The experimenter's hand points down which means introverts have the hand pointing up. This may confirm an idea given by several courses in management, saying that a handshake is submissive when the hand points up. The movement is also more out of the frontal plan. Surprisingly, the effect on the pressure exerted by the participant is very little (the F value is 4.0 while the critical value for p value < 0.05 is 3.9). This results contradicts our third hypothesis (H3). However PG eMax has a strong difference meaning that the experimenter handshakes stronger introverts. The experimenter does not know a-priori and consciously the personality of the participants. However he knows them as acquaintances and this may have unconsciously altered his behavior.

Recognition of the experimental conditions

After having observed how the measured features change depending on the conditions we take them all into account in order to determine if it is possible to find back the conditions from the data. We choose as learning algorithm the LDA (Linear Discriminant Analysis) that finds a new space of representation of the data that maximizes the inter-class variance and minimizes intra-class variance for a given parameter. The new created components allow to discriminate more easily the categories. This method is supervised, thus attention must be paid to the overfit problems (i.e., the computed model fits well the data provided, but cannot be generalized to new data). That is why a maximum of 6 features are used for datasets of 132 or 216 samples.

As a first example, we try to detect the firmness condition of the experimenter using the 4 selected features: TP, A amp , GP eMax and, P 0Max . (PG pMax was removed as it represents only the participant's behavior). Two components are created by LDA: LD1 and LD2. When all the samples are projected in the LDA space, the Bayes method enables to create frontiers that separate the learned categories (see The success rate is high but it is important to make sure that the data is not overfitted. We check whether the score is not too different between learning samples and evaluation samples using the "testset" method. We split the dataset in two same-sized subsets, and we train on the first and evaluate on the second, the split is done randomly. We compute LDA 2000 times, recording either the learning The second analysis tackles the gender recognition. We use the "dataNormal" corpus and compute an LDA using 6 features: TP, A amp , S amp , δθY, GP eMax and, GP pMax . The score of this model is 76.5%. When looking at how LD1 is calculated, it follows the signs of the mean differences in Table 3.6. However, the weight of PG eMax is much lower than the others (the weight of PG eMax is 7 times lower than PG pMex ). It shows that in the end, the behavior of the participant is more efficient to discriminate his/her gender than the experimenter's action. For a third analysis, we used the same dataset to compute an LDA in order to discriminate the degree of extroversion of the participants. We used the 5 following features: S amp , δθY, αY, GP eMAx and, GP pMax . The score of this model is 68.2%.

All the features have exactly the same weight in LD1, apart from GP eMax that is 1.5 higher than the others. This shows that the experimenter's behavior has a certain impact on the discriminability of the degree of extroversion.

Impact of firmness on the detection rates

Finally, we tackled the following question: does the experimenter's firmness help or complicate the detectability of internal characteristics of an individual? Thus, we used the "dataFirmness" dataset composed of the three kinds of firmness, and computed

LDAs depending on gender and degree of extroversion for the soft and the firm conditions. In the soft condition, the score for gender is 79.2%, while it is 73.6% in the firm condition. During the normal condition, it is 76.5%, this means that when the experimenter is passive, the gender is more easily detected. Concerning the degree of extroversion, however, no significant differences in the classification scores depending on the firmness of the experimenter is observed.

To summarize this subsection, Table 3.8 shows the discrimination scores.

Discussion

We discuss in this sub-section how the results fit our three hypotheses.

It has been checked that the proposed analysis method is able to recognize the handshake firmness of the individual that wears the glove with a success rate of 75%. The main features for this are the handshake duration, direction of movement, maximum acceleration, and the pressure of the individual that wears the glove. The results are consistent with the first hypothesis (H1) as the pressure is higher with firmness and the glove pressure is discriminative.

The experiment results are also in favor of the second hypothesis (H2): it is possible to recognize gender through handshake as the success rate is 77%. An important feature apart from the participant's pressure is the hand inclination. This means that there is a behavioral effect of the participants depending on their gender. It has also been found that when the experimenter has a softer handshake it is easier to recognize gender.

However, when looking at these results, it can also be assumed that the handshake manner differences are due to hand size rather than gender. Indeed, female participants are more likely to have smaller hands and this geometrical aspect could change the probability to touch a sensor and give a smaller inertia to the hands. In our "dataNormal" dataset, we do not have any female with large hands but we have 7 males with small hands, making 28 samples of "exception". It can be noted that only 6 points out of these 28 samples are misclassified and 3 are from the same person who had a projected hand size of 193mm. This tends to confirm that the gender discrimination is made from behavioral components rather than geometrical.

The extroversion discrimination is less obvious as the success rate is around 62%.

If some important features are movement characteristics, the most relevant pressure information is linked to the behavior of the individual that wears the glove (i.e. the experimenter). We were not able to find this level of success rate without the GP eMax feature. This means that detecting extroversion through contact pressure might not be relevant in a human-robot interaction scenario. This goes against the third hypothesis (H3) but further studies would need to be carried out (taking into account for instance physiological data: hand temperature and dermal conductivity). Nevertheless, the participant action (GP pMax ) still showed lower pressure in the introvert case. Deeper investigation with more precise measurements and analysis may be able to use this information to discriminate the degree of extroversion.

Conclusion of the experiment

This experiment had two objectives: (1) evaluate our measurement system to see if it enables to distinguish behaviors acted by one of the handshake partners, which is here the firmness of the experimenter, and (2) see if differences are observable between gender and extroversion degree of the participants. We collected handshake measurements, in real life conditions, with 36 participants.

Before answering the research questions, we reduced the number of variables to study, as they consist in several aspects of 5 physical measures (i.e., duration, speed, acceleration, orientation, and pressure). We grouped these variables when they were correlated, and we kept a set that contains at least one of the 5 physical data. Afterwards, we kept only the features that were relevant to describe the different conditions of the experiment. This selections was done by the significant ANOVAs depending on a given condition. Then, these variables were used in a model/recognition of the conditions method (i.e., LDA).

We found recognition rates higher than chance for the three experimental condition categories (i.e., Firmness, gender, and extroversion degree). These rates are associated with the variables that enabled to learn and test the models. Finally, we found that the gender is easier to recognize when the experimenter is passive, and it is not necessarily detected due to the hand size differences between men and women. All these results contribute to the state of the art as it gives some physical and measurable cues on how gender and extroversion degree alter handshake manner. Besides, this is encouraging about the usability of our device, which may be able to provide deeper information about the individual internal state during handshake (e.g., mood or emotion).

3.6 Can personality be detected during human-robot handshake? (Experiment 2 bis)

In the previous section, we presented an experiment characterizing the handshake differences depending on firmness, gender and extroversion degree, but only between two humans. It is important to notice that one of the partners was always the same (i.e., experimenter), but could interfere in the data, or at least have a non constant behavior between interactions. Nevertheless, it was important to start with human-human interactions for several reasons. The third reason, which is closer to the current research questions, and is more a methodology point of view, is the following. The only hypotheses we could make about how gender and extroversion degree can be conveyed through handshake, come from psychological studies, that investigated human-human interactions. So, we needed to verify these results, and complete them with objective measures to contribute consistently to the field. We found that indeed, gender and extroversion degree can be communicated. However, there is no reason it also happens face to a social robot, whom the user may have a different social representation. Maybe what 3.6. Can personality be detected during human-robot handshake? (Experiment 2 bis) 67

we convey depends on the social intelligence we attribute to our partner. This paragraph comes from some of our reflections we had, that motivate, guide, and justify the choices we made in this thesis. Several hypothetical terms are used as, to the best of our knowledge, these questions were not answered by the literature.

It is however important to know, given the social robots that are available now, if some psychological cues can be conveyed to a robot during handshake, the same way it is to a human partner. We present in this section a small experiment, that follows almost the same protocol as the previous section, but using the Meka robot.

This robot is described in Section 3.6.1, and the results are presented and compared with the human-human interactions in Section 3.6.2.

Experimental setup and protocol

The experiment presented in this section was conducted with the Meka humanoid robot pictured Figure 3.14. This compliant robot has been designed for human-robot social interaction studies. Its joins are intrinsically safe and the actuators are able to simulate customized stiffness. It has a moving head, is able to simulate facial expression, has an omnidirectional base, a customized body height and two arms with 7 degrees of freedom (DOF). Its hands have 5 cable driven DOF: a 2 DOF thumb and 3 fingers. We can adjust the close ratio (cr), stiffness (kh for the hand and ka for the arm), and speed. Its hands are larger than a human hand and is designed for dexterous manipulation.

The glove we designed for the robot has one layer of fabric on which we embedded 8 pressure sensors in the most touched areas found by a ink experiment similar to the one in Section 3.3. However, because the hand is large, participants did not touch the same areas and it was difficult to instrument the whole surface. An accelerometer is also sewed on the back of the robotic hand. Then the glove is covered by another fabric layer. The acquisition was done using an Arduino transferring data through USB and commands were sent using ROS.

After having filled the same questionnaire as in the previous experiment (Section 3.5), participants were introduced to the robot. They had five minutes to perform handshake training with the the robot exerting various closing ratios and strength.

During a handshake the participant starts the move and when he/she is close to the robot hand, the experimenter sends the closing command. The closing duration is about one second, until the participant starts to open his/her hand. If after a few seconds of interaction, the participant did not start to remove his/her hand, the experimenter sends the open request. 9 measures were made: 3 soft (cr=50%, kh=50%, ka=30%), 3 normal (cr=70%, kh=70%, ka=60%), and 3 firm (cr=80%, kh=85%, ka=90%).

Then, we extracted the same features as in the human-human experiment.

The experiment was carried out with 8 participants (7 M and 1 F, 4 E and 4 I) making a 72 samples dataset.

Experimental results and comparison

The first thing we can notice is that some sensors are never or hardly touched but the touched sensors are consistent depending on the participant. For instance P 0 is never touched, P 7 is touched by one person, P 6 by two, P 1 by three, P 2 , P 3 , P 4 , P 5 by five. No correlation can be made between the fact a sensor is touched and the extroversion value. We decided to take into account only the touched sensors and calculate the mean and maximum values of this selection.

We first checked if a firmer robot handshake produces higher sensor responses.

We calculated an one-way ANOVA on firmness category and we found that only discrimination between firm and soft handshakes is significant and it is using the P MaxMax feature. P MaxMax is higher for firm handshakes than for soft ones (δµ=87.1%, F value (2, 69) = 4.1, p value = 0.021), P behaves the same way but less significantly. This result is consistent with the previous experiment and the order of magnitude of this pressure gap is similar (δµ = 99.3%, F value (2, 213) = 38.7, p value < 1e -3). Unfortunately, we cannot determine which sensor belonging to group s or group r is responsible for this difference.

Given the fact our number of participants is very small we only can do a qualitative comparison between the two experiments. We can say that the only female of the experiment handshakes slightly softer than males (µ=25.8 kPa, σ=22.3 versus µ=29.4 kPa, σ=16), the maximum acceleration amplitude is also lower, the maximum speed higher and the frequency lower. So apart from the inclination of hand feature, this is consistent with the human to human results. Similarly, for the extroversion study we found that the two out of three main features selected in the previous study (δθY, SAmp, but not P and αY) behave the same way: δθY is higher for introverts (µ=11.8 • , σ=6.8 (I) versus µ=9.8 • , σ=6.6 (E)), SAmp is higher (µ=0.21m.s -1 , σ=0.12 (I) versus µ=0.18m.s -1 , σ=0.12 (E)), and P is equal. This last information is still against of our third hypothesis (H3). Indeed in the previous experiment, the sensors with a stronger response belonged to the group s (the group definition is in Figure 3.4). As the robot has the same behavior whatever the participant personality, it shows that there is no strong link between the extroversion of the participant and the pressure he/ she applies.

The comparison with the human-robot handshakes showed some consistency with respect to the human-human interaction. But this has to be taken with caution, as the number of participants involved in the human-robot interaction experiment was too small. Some more work should be done in this direction to see if results concerning human-human interaction and human-robot interactions are similar, if in both cases psychological content can be spontaneously conveyed from an individual to his/her partner through the tactile modality.

Conclusion of the chapter

To conclude this chapter, we remind its structure. The main goal of the chapter was to study how the handshake manner is altered by psychological constants characterizing an individual. We started by presenting how gender and personality were found to have effects on handshake, given psychological studies. The handshake manner was also quantitatively described by researchers with more technical background, but no study correlated these results with internal state of the participants.

The contribution of this chapter is thus to find a link between objective handshake measurements and internal state. We developed a glove to acquire these measurements, using the results of a preliminary experiment that gave us the locations of the touched areas during handshake. We used this device to collect interpersonal and ecological handshakes to be compared with the gender and extroversion degree of the participants. Several results were found, and we could detect the firmness of the experimenter, and the gender and extroversion degree of the participants with above than chance rates. Finally, these results were compared with human-robot handshake interaction measurements.

The contributions are the following: (1) The contact areas during handshake, that are consistent with other researchers results. (2) Not only pressure data, but also acceleration and duration enabled to detect the experimenter's handshake firmness, with a success rate of 75% for three firmness conditions. (3) It is possible to recognize the gender of an individual using the physical data during handshaking, with a success rate of 77%. The most important features are speed, hand inclination, and participant's pressure. (4) The extroversion degree also has an impact on handshake, although with a lower size effect (62% of success rate). This was obtained using hand inclination, speed, but the pressure effect was mostly due to the experimenter's unconscious action. (5) The last results have to be taken carefully, but we found some similarity when interacting face to a human or face to a social robot. These results were part of a contribution in a conference in 2016: [START_REF] Orefice | Let's handshake and I'll know who you are: Gender and personality discrimination in human-human and human-robot handshaking interaction[END_REF] These results and contributions are encouraging as they show that a few pressure sensors in a glove and some movement measurements enable to detect internal states of an individual. These states are long term, so it is possible to measure them in a totally deported context from the interaction measures. The center of our thesis, however concerns affective states, which are much short term. Given the success of these first experiments, we aim to investigate how affective states alter handshake, but this short term property raises new challenges. The affective measurements have to be collected soon after or before handshaking, and should correspond to what is experienced during the interaction. Besides, affective states can be modified during the experiment and become controlled experimental conditions. However we have to make sure that these experienced affective states are still running during the interaction. In the rest of the thesis, we investigate two types of affective states: the emotions, and the moods. The next chapter (Chapter 4) tackles emotions. We designed a tool to generate controlled emotions in the participants, and then used it to compare the handshake manner before and after emotion elicitation.

Chapter 4

Impact of emotions on handshake

Introduction

In Chapter 2, Section 2.4, we defined what are the emotions and presented various theories about them. In Chapter 2, Section 2.2, we detailed how the handshake manner is important to engage an interaction and to reveal some aspects of our internal state. In this chapter, we investigate, through an experiment, if the handshake manner can be altered by the experienced emotion of the individual.

We firstly introduce in Section 4.2 some methods to measure emotions. They can be used to verify which emotion is experienced by the participant. We then present a review on how emotions can be generated in a laboratory setup, in Section 4.3. This information is exploited in our work that focuses (1) on the elicitation process of a spontaneous emotional state into an individual, (2) on the measurement of the response in order to validate the experimental condition, and (3) on the analysis of the handshake data to detect differences depending on emotions.

After these reviews, we describe in Section 4.4 an emotion elicitation tool that we designed and developed, using Virtual Reality (VR), and audio-visual stimuli from public databases. We explain which emotions we chose to elicit, and depict the virtual environment with the selected stimuli.

Then, this tool is evaluated, based on questionnaires. Physiological data was also measured but could not be analyzed. Given the subjective responses of the participants, the tool generated the expected emotions. This led us to the second phase of our work: the use of the VR tool for emotion recognition through handshake.

We present in Sectione 4.5 an experiment involving VR, handshakes with virtual avatars, and with a real human, and pressure measurement. The results obtained are presented and a discussion, that aims to improve future research in this direction, is also provided.

Emotion measurements

The emotion theories shown in Chapter 2 revealed that emotions consist in a response to an event. Thus, the response can be measured. This is important in our research as we not only need to elicit an emotion in our participants,but we also have to check that the right emotion occurred.

Many researches tackled this question, in Psychology, Social Robotics and Computer Science. We already specified that the emotion can have physiological, behavioral, or cognitive consequences. These are the three modalities that can be used to detect emotions. Table 4.1 summarizes the related literature.

When focusing on physiology, one can measure heart rate, respiration rate, or skin conductance [START_REF] Rainville | Basic emotions are associated with distinct patterns of cardiorespiratory activity[END_REF][START_REF] Kim | Bimodal emotion recognition using speech and physiological changes[END_REF][START_REF] Valenza | Revealing real-time emotional responses: a personalized assessment based on heartbeat dynamics[END_REF][START_REF] Ménard | Emotion Recognition based on Heart Rate and Skin Conductance[END_REF][START_REF] Nakasone | Emotion recognition from electromyography and skin conductance[END_REF]. Concerning the behavioral aspects, many studies looked at audio-visual features due to their non-invasive nature. One can detect key-points on the face representing "Action Units" [START_REF] Pantic | Machine analysis of facial expressions[END_REF][START_REF] Valstar | The first facial expression recognition and analysis challenge[END_REF], detect the skeleton pose or movement [START_REF] Mccoll | Affect detection from body language during social HRI[END_REF][START_REF] Dael | Emotion expression in body action and posture[END_REF][START_REF] Karg | Body movements for affective expression: A survey of automatic recognition and generation[END_REF], or measure the prosody of the voice [START_REF] Grimm | Emotion estimation in speech using a 3d emotion space concept[END_REF][START_REF] Kim | Bimodal emotion recognition using speech and physiological changes[END_REF][START_REF] Koolagudi | Speech emotion recognition using segmental level prosodic analysis[END_REF][START_REF] Ramakrishnan | Recognition of emotion from speech: A review[END_REF]. To the best of our knowledge, no method, either physical or behavioral, proposes a standard and automatic technique to detect emotions. Physiology only provides tendencies like arousal, while facial expression gives a probability to fit one of the basic emotions. To assess the cognitive response of emotions, one can analyze the vocabulary used by a participant while speaking or writing [START_REF] Kao | Towards text-based emotion detection a survey and possible improvements[END_REF]. Thus, it is possible to access the unconscious cognitive emotional response. Another method consists in asking the participant to answer a questionnaire. The answers correspond to the conscious subjective representation of the participant on its own emotion. Several questionnaires were developed corresponding to two representations of the emotion. One is discrete, and the participant is asked to evaluate to which level a group of adjective corresponds to his/her emotion [START_REF] Schaefer | Assessing the effectiveness of a large database of emotioneliciting films: A new tool for emotion researchers[END_REF]. The adjectives are synonyms of the basic emotions, or can depict a mixture of emotions. The second representation is in a three dimensional space (i.e., PAD: Pleasure (or valence), Arousal (or activation), Dominance) [START_REF] Russell | Evidence for a three-factor theory of emotions[END_REF]. The participant is asked to evaluate how his/her emotion fits each dimension. A questionnaire using figures representing each dimension without having to explain their meaning is generally used [START_REF] Bradley | Measuring emotion: the self-assessment manikin and the semantic differential[END_REF]. This questionnaire is called Self Assessment Manikin scale (SAM).

As explained in Chapter 2, the emotional response does not necessarily appear on the three modalities at the same time. The cognitive emotion may also not fit the emotion expressed by physiology or behavior. The questionnaires can also be biased as the participant may not answer what he/she really feels but what he/she thinks he/she was supposed to feel in the experiment. In our work, we used cognitive evaluations using both discrete and continuous questionnaires. We also tried to check the consistency with physiological measures, but could not conclude on it. We think that if the subjective response exists, and if the participants answer honestly, there is more chance for the emotional process to be complete. As a consequence, the behavioral reaction should be visible.

Elicitation methods: a review

After presenting what an emotion is, how the emotional process works, what its impact on an individual is and how it can be measured, this section presents how to elicit an emotion, in an experimental context. Indeed, most of the researches about how the emotion alters a phenomenon, require to generate a specific emotion in subjects and observe the consequences. In our case, we would like to elicit emotions and measure how it alters the handshake manner.

This section details three main methods used to elicit emotions and presents databases of stimuli that can be used. The emotional response duration aspect, which is an important element in the design of elicitation systems, is also discussed.

Finally, we provide some examples of studies that used VR for emotional purposes and discuss some important aspects in the designing step and some concepts to be used so as to improve VR elicitation.

Usual elicitation methods

The emotion elicitation methods can be grouped in three categories. The first one (1)

refers to an internal process that generates the emotion (e.g., the participant reads a certain situation and imagines it is happening to him/ her, the participant thinks about particular memories, self elicitation, etc.). In the second category (2), the participant is passive as the external stimuli (e.g., images, videos, music, narration, facial expressions, etc.) are presented to him/her. In the third type (3), stimuli come out during an interactive task (e.g., the participant takes part in a scenario with actors, plays a game, explores a virtual environment, etc.). In this sub-section, we introduce briefly various elicitation methods used in these categories, with examples from the literature. Then, we discuss the impact of time, which we have to take into account during elicitation.

Internal process

(1) Many studies use the first elicitation category (i.e., internal process) as it does not require any material, and it can easily be set-up. Making participants read about a situation or thinks about particular memories can be applied to any individual.

However, it is difficult to control what emotion is really elicited depending on the participant. Self elicitation is a specific method, which is much more reliable. For self elicitation, the participant has to be an actor, that has deep knowledge on emotions, and knows how to reproduce them. However, it is more difficult to find numerous participants for these experiments. Picard, Vyzas, and Healey, 2001 used only one actress and measured her physiological reaction while she self-elicited 8 emotions every morning for a duration of 6 weeks. This method allowed to have consistent physiological alteration during several minutes.

(2) Most of the studies on emotion use discrete stimuli displayed on a screen or played in headphones. For this kind of task, datasets that have already been emotionally evaluated are required. We present in the three next paragraphs such external stimuli (i.e., images, music, and videos).

External stimuli: Images

The IAPS database [START_REF] Lang | International affective picture system (IAPS): Affective ratings of pictures and instruction manual[END_REF] is well-known as it has been evaluated and tested by many studies, on several emotional dimensions, which enables to compare the experimental results. It is composed of 944 images presented during 6s in average. It has been evaluated in terms of PAD by the authors of the database, and discrete emotions [START_REF] Mikels | Emotional category data on images from the International Affective Picture System[END_REF]. The questions asked to the evaluators in these studies was clearly to judge their own emotional state. Dan-Glauser and Scherer, 2011 created a corpus of 730 topic-specific images displayed during 4 seconds. It contains four categories of negative emotions, and only one positive, given that negative ones are more studied in the literature. They are evaluated in terms of pleasure and arousal, however, for the pleasure question it was not clearly specified to mark the emotion internally experienced when looking at the picture.

These two databases contain photographies of objects, nature, animals, or humans in certain situations. One could think about drawing or artistic painting, but they are not used as they may be more subject to interpretation.

The use of images is interesting as the participant discovers instantly all the emotional content. However, the emotion does not last in time. The same way, some real life event related sounds can be used to induce emotions. Bradley and Lang, 2000 used 60 samples of 6 seconds audio stimuli, like screams, bombing, laugh, or roller coaster. They are evaluated in terms of PAD, and the physiological response is also recorded. The results are highly similar to visual stimuli, which leads to the hypothesis that whatever the perceptive entry used, all the emotional signals are conveyed to the same emotion center in the brain. Multi modal elicitation may be an interesting direction.

As briefly introduced, visual and audio stimuli can elicit emotions. However, the duration of these stimuli is very short and does not last long enough after the exposure. Music has the benefit of having a time dimension. It has been widely studied for its link with emotions.

External stimuli: Music

Konečni, 2008 admits that music can carry highly emotional content. This is due to the temporal aspect, and the fact that music makers have high knowledge about emotions. It is possible to allude emotion expression (e.g., heart beat for fear musics), or express events and stories. However, the author raises doubts regarding the direct link between music and emotion generation. The first concern is about the definition of emotions. He agrees with the literature on the fact that there is physiological response while listening to music. But there is not necessarily a subjective emotion representation, so one cannot speak about emotion. We argue, given the theories presented in Chapter 2, that an emotion does not necessarily require a subjective representation to exist. The second issue is that authors occasionally confuse the fact music can express or induce emotions. Indeed, we can do the same remark about images. A picture representing a sad person expresses sadness but it does not imply that the watcher will feel sad. Konečni, 2008 also highlights a scale issue.

The emotion intensity "felt" while listening a music is lower than what the music "expresses". The author showed in another study that the emotion intensity is lower when remembering an emotional situation than what is remembered to have been felt during the situation. When listening music, the intensity is even lower than during this remembering operation.

In summary, physiological responses may not be related to an emotion. There is a loss of emotional signals between what is expressed by the music and what

is felt, and, the emotional response is low. [START_REF] Konečni | Does music induce emotion? A theoretical and methodological analysis[END_REF] proposes the influence of mediators. One of these mediators can be the association memory which relates the music (or type of music) with an emotionally charged event. It can also be the automatic physio-motor response (e.g., dance) that rises an emotion. The author also refers to non emotional subjective states (i.e., being moved, aesthetic awe). However these states occur rarely, and require personal cognitive associations, or to sublimate the overall situation (e.g., listening a music played by an orchestra).

To complete Konevcni's statement [START_REF] Konečni | Does music induce emotion? A theoretical and methodological analysis[END_REF], we can refer to the work of [START_REF] Lundqvist | Emotional responses to music: Experience, expression, and physiology[END_REF]. These authors, to study the elicitation property of the music only, chose not to use music that can be related to past experiences of the participants. Thus, they created a corpus from scratch. Their participants still felt emotions, which were correlated with facial muscles activity.

Based on these discussions, we confess not to be confident about how the music induction process works, and how to choose the best stimuli for our purposes. Still, reactions were observed while exposed to music, were they induced by direct elicitation or through mediators. Besides, some studies, presented bellow, created music-based databases, which are evaluated as "felt emotion". These databases were designed for elicitation purpose. [START_REF] Vieillard | Happy, sad, scary and peaceful musical excerpts for research on emotions[END_REF] created 56 sequences of piano notes lasting 15 seconds, aimed to elicit 4 emotions (i.e., peaceful, sad, happy and fear). They evaluated how these samples express, or induce emotions, in terms of discrete emotion, valence and arousal. They found a consistency between the emotional evaluations and the emotions targeted by the music maker. Moreover, they observed a consistency of the emotion categories depending "expressed by the music" or "experienced by the participant" evaluations. On the contrary to what observed Konečni, 2008, the "experienced by the participant" condition showed higher intensity than "expressed by the music".

Finally, Eerola and Vuoskoski, 2010 created a corpus of 460 extracts of movie soundtracks lasting 15 seconds, in order to perform a preliminary experiment. The samples are evaluated in terms of discrete emotions (i.e., happy, sad, tender, fearful and angry). The authors then could produce a selection of 16 samples of between 45 and 77 seconds eliciting happiness, sadness, fear, or a tender emotion. We use this database (Eerola and Vuoskoski, 2010) in our work.

External stimuli: Videos

Video content has the advantage to depict more complex situations and has also a stronger effect on emotions. It has been demonstrated that it is one of the easiest techniques to implement in a laboratory, reproducing an optimal artificial model of reality [START_REF] Schaefer | Assessing the effectiveness of a large database of emotioneliciting films: A new tool for emotion researchers[END_REF]. Bednarski, 2012 divided videos in two categories:

with direct engagement, and subjective engagement (i.e., the emotion appears from analyzing the situation of the characters). She showed that the direct engagement is more efficient to elicit positive emotions. To succeed subjective engagement, the video maker has to improve the empathic situation. The videos can also come from various sources (e.g., movies, landscapes, animals).

Several databases containing movie extracts are available. Bednarski, 2012 focused on positive emotions. She evaluated 14 movie extracts chosen to induce 7 positive emotions. Each movie lasts between 45 seconds and 5 minutes. The evaluation is performed using 27 adjectives on 7-point Likert scales. The most efficient stimuli elicit joy, awe, serenity, and interest.

Bartolini, 2011 studied 45 movie extracts aimed to elicit 9 emotions (i.e., the 6 basic ones more 3 positive emotions). The extracts last between 30 seconds and 7 minutes. They were evaluated using 19 discrete adjectives, and using the PAD scale.

Interactive task

The "interactive task" emotion elicitation method (3) consists in designing a controlled scenario, and make the participant take part of it. However, the participant should not be aware of what the scenario is really for. These scenarios can be psycho-social (several accomplices are involved) or individual (the participant is alone). Amodio, Zinner, and Harmon-Jones, 2007, give some cues on how to design a psycho-social scenarios. They consider that the experimenter firstly has to construct a cover story, to distract the participant from the real goal of the experiment, hide the real independent variables that are adjusted by the scenario, and hide what is really measured. This story should make sense, and if possible, involve the participant so that he/she is more attentive to the conditions. The experimenter, who interact with the participant, should train rigorously to have a repeatable behavior, and be neutral. It is also better to perform within-subject analysis as the behaviors are highly dependent on the participant. Given the authors, this may be problematic, as one condition experienced by the subject may affect other conditions, the participant may be able to detect the difference between the conditions, or it may not make sense to performed repeatedly the same task. The authors state this method is costly in terms of design and execution, however it has several advantages. The emotional response is spontaneous and close to what the participants would experience in real life, it is not affected by bias such as social desirability, it enables to elicit complex emotions, and, as social interactions are widespread emotion sources, it is a ecological way to elicit emotions in the lab context.

In our research, we are very motivated to study ecological and spontaneous emotional states, so this method is interesting in our point of view. However, in order to work efficiently, one has to take high caution in the design and be aware Another way to use interactive task methods is to use games in which difficulty can be adjusted to induce stress, joy, or disappointment. In this scenario, the participant is involved in the elicitation, and if the game is immersive enough, he/she may forget about the lab context. However, apart from joy and disappointment, the richness of the emotions that can be elicited is low.

A third way to use this category of elicitation is to make the participant walk around a virtual and controlled environment. As the individual controls the movement, he/she is involved is the process. Therefore he/she perceives all the emotional stimuli just as he/she was discovering them by himself/herself. We detail such techniques using virtual reality in Section 4.3.2.

Time aspect

The time aspect is very important for emotion elicitation or detection. When we exposed the emotional theories, we noticed the "short-time" cause, the "short-time" and "fast" reaction, and the "short" duration. However, we need to know to what extent these "short-times" are. This would give us the information of how long should the elicitation phase be, and how long after we can perform the handshake measurement. Despite the importance of these questions, to our knowledge, surprisingly few studies tackled time aspect of emotions.

Picard, 1997 proposed a theory about the time-dependent emotional intensity following several trigger events. He does an analogy with a bell ringing. To be activated, an emotion requires a certain stimulus threshold. It also saturates after a certain stimulus value. When elicited, the emotional arousal increases quickly, then it decreases gradually before vanishing. Another property is that, with repeated stimuli, the maximum emotional arousal reached is higher, and the emotion remains longer. These properties, although not proved experimentally to our knowledge, are very interesting for elicitation tool design.

Few studies give time ranges of the emotion duration. This duration depends on the modality used to observe the emotion. [START_REF] Ekman | Expression and the nature of emotion[END_REF] assesses that the facial expression response lasts between 0.5 to 4 seconds. [START_REF] Valenza | Revealing real-time emotional responses: a personalized assessment based on heartbeat dynamics[END_REF] was able to evaluate the emotional response measuring heart rate using a 10 seconds window. The authors found that the physiological regulation occurs 6 seconds after the stimulus, marking the boundary between the low level emotional process to high level cognitive evaluation. In their experiment, the images were displayed during 10 seconds. Kim, 2007 measured several expression modalities and used different time windows (between 2 and 6 seconds for prosody, and between 3 and 15 seconds for physiology). The authors reached a success rate of 50% upon 4 emotional conditions, meaning that the emotional response was observable during this range of time. [START_REF] Garrett | Time course of the subjective emotional response to aversive pictures: relevance to fMRI studies[END_REF] continuously presented 4 seconds aversive IAPS images to participants during 32 seconds. The participants had to answer in real time to a questionnaire to assess their arousal. 8 seconds after the start, the emotion felt drops by 57%, and after 16 seconds it drops down by 74%. These data indicate that the emotion duration ranges below half a minute.

To conclude on the time aspect, most of the studies target short-lived emotions.

The response is measured in real time, or right after the stimulus. To our knowledge, no one has intended to make the emotion last long after the elicitation phase.

In order to investigate the effect of emotions on social interaction, it is important to have a continuous emotional state. Picard, 1997 pointed out the cumulative property of emotions. We took into account these results while designing our experiment.

We present stimuli repeatedly and frequently to increase the level of emotion and make it more stable. We also consider that using several modalities at the same time should improve consistency and make the participant, surrounded by emotional content, "live" his/her emotion. Moreover, we assume that elicitation would be more efficient if the participant is active during the experiment and contributes to his/her emotion. That is why the third category of elicitation (3), which uses interactive tasks, is an interesting approach for our work.

Elicitation using virtual reality

A very promising media is virtual reality (VR). Indeed, a succession of stimuli can be implemented, a combination of several modalities is available, and, the participant is an actor as he/she controls the camera direction and the First Person controller.

Moreover, [START_REF] Bouvier | La présence en réalité virtuelle, une approche centrée utilisateur[END_REF] showed that there is a link between sense of presence and emotion arousal.

Previous research works described how to design a virtual environment to elicit emotions and how to increase presence. Geslin, Jégou, and Beaudoin, 2016 found several parameters that can be adjusted depending on the coordinates of the emotion in the circumplex space (i.e., the Pleasure-Arousal space [START_REF] Russell | Evidence for a three-factor theory of emotions[END_REF]). For instance, the light and coloration influence pleasure, fast moves and open spaces induce high arousal and pleasure. Several environments were designed to elicit fear and anxiety. [START_REF] Felnhofer | Is virtual reality emotionally arousing? Investigating five emotion inducing virtual park scenarios[END_REF] designed five environments in a park with different lighting, sounds, added elements, and characters. The authors elicited joy, anger, anxiety, boredom, and sadness. Between joy and anxiety the design changed from chirping of birds and sunny day to owl cry and dark night. The authors assessed that virtual reality enables to use rich and interactive stimuli, and that it is easy to adjust emotion through some specific parameters.

Many studies tackled the sense of presence in virtual environments. The sense of presence defines how participants forget about surrounding environment and believe they take part of the VR media. [START_REF] Baños | Immersion and emotion: their impact on the sense of presence[END_REF] found that presence depends on the display used (screen, video-projector, VR). For Bouvier, 2009, presence depends also on how the participant takes part in the game, and hence an important effort has to be made in the framework of the experiment. The concept of presence is important in our work as if presence is strong enough, emotional arousal will be amplified.

Geslin, Bouchard, and Richir, 2011 studied the impact of being a regular gamer or not on the physiological response while playing. The authors also explained the concept of "Flow", used by game designers, which is the balance between boredom (extra skills) and anxiety (extra challenges). In the state of "Flow", the participant is the most receptive to emotions as he/she totally forgets the surrounding environment and the technical aspect of the game.

To summarize this review on VR elicitation, we can say that VR is an interactive way to elicit emotions. It has the advantage to be modular, enabling to adjust the emotions through defined parameters. Improving the sense of presence and the state of "Flow" are important considerations to increase the elicitation efficiency.

Conclusion on elicitation methods

To conclude this review, if self elicitation using actors has proved its strength, it is still complicated to implement for a large cohort of participants.

Displaying audio-visual stimuli is the most widely used technique, mostly for the real-time measurements, like physiology. Several databases have been created

containing images, sounds/music, and videos. However these stimuli largely differ in terms of type of content, and duration. Indeed, some can express a social situation calling for empathy, display a situation that evokes a memory by association to the participant, or simply rise an emotion through the content itself. These correspond to the possible emotional sources presented in the theory section. For all of these stimuli a response was observed. However, one can suspect that these responses are very short term and may not be measurable during experimental social interaction.

Virtual reality is a promising media to elicit emotions. It involves the participants in the elicitation, and makes them forget about the environment. It allows to control the experimental conditions, as each participant is exposed to the same environment and content, and it is easy to adjust the content for each emotion. It enables interactive tasks within the environment, and still allows the usage of previous elicitation methods with discrete stimuli.

Design of an emotion elicitation device using VR

This section justifies our choices concerning the design of an emotion elicitation tool.

Firstly the choice of the emotions to generate are presented. The emotional content is then detailed. Finally, the tool was evaluated and the results are discussed.

Choice of the emotions to elicit

The first step to design our emotion elicitation tool is to choose relevant emotions whose impact is analyzed in a handshake social interaction.

Several ideas can guide our choice: to follow a possible application of an emotion recognizer, to use emotions that are opposed in the (Pleasure-Arousal-Dominance scale), or to use basic emotions.

The general question of our research is to know if emotions can alter the handshake. If the answer is true, one can expect that very different emotions may lead to different handshake manners. We chose basic emotions, which have strong physiological and behavioral responses (see Chapter 2 Section 2.4), and we take the ones that differ the most in the PAD scale. In order to have a baseline, we add a neutral condition (N), aimed to elicit medium pleasure and dominance, with low arousal.

Given the Russell's emotion database [START_REF] Russell | Evidence for a three-factor theory of emotions[END_REF] 1 , this corresponds to the "relaxed" emotion with zero pleasure. One can call this mixture as "calm". We chose three other emotions: happiness (H), fear (F), and sadness (S).

We projected these four emotions in the PAD space (see Figure 4.1), with the standard deviations corresponding to the experiment of [START_REF] Russell | Evidence for a three-factor theory of emotions[END_REF] This shows that the selected emotions are highly different, and mostly concerning the Pleasure-Arousal dimensions (see Figure 4.5). Finally, in case the short-term emotion elicitation would not work, we added two mood conditions. One positive mood, and one negative mood. They were induced by making the participant virtually walk in a landscape with a positive or negative ambiance.

Emotional content

Concerning the "mood" elicitation, we used an outdoor space to leave the participant free to move. The positive ambiance was designed to be relaxing and the negative one to be oppressive. The first one was represented by a sunny day, green grass, beach with calm sea, and sea sound, river with river sound, some trees with bird songs, a cliff to see the landscape and a lake (see Figure 4.2a)). Some flying butterflies and a running horse were added. The negative ambiance was represented by the same environment but the sky was cloudy and darker, grass was muddy, sea was choppy with storm sound, it was raining with thunder lighting and sounds, the bird songs were replaced by raven cry and flights (see Figure 4.2b)). These choices were made based on some researches from the literature presented in Section 4.3.2. The storm and raven activity is supposed to increase arousal of the negative mood and the light and colors of the sunny day is supposed to improve valence of positive conditions.

The participants were asked to explore the virtual environment during 3 minutes. They were free to move but a path helped them to orient themselves so that everyone see all the emotional content. For emotion elicitation, the participant reaches in a corridor with pictures displayed on both walls. He/she has 1.5 minute to cross the corridor and gaze at the 10 pictures (see Figure 4.3). During this step, a music is played. Then, a door opens and the participant enters a theater room with some sit characters. A movie sample of 2 minutes is displayed. When the video ends, the participant removes his/ her VR helmet. The choice of mixing several modalities (image, sound, video) was made on the hypothesis that multi-modality stimulation should strengthen the experienced emotion. We chose 1.5 minute for picture viewing so that the participants spend around 6 seconds to watch each picture, with 3 seconds between each. We did not change lighting and texture conditions as the first evaluation focuses on the role of the proposed three modalities only. The presence of virtual agents was to prevent from an oppressive feeling due to loneliness. These agents were in the same situation as the participant, watching the video.

The stimuli were selected as follows. For images2 , we used the IAPS database [START_REF] Lang | International affective picture system (IAPS): Affective ratings of pictures and instruction manual[END_REF], which was originally evaluated in the PAD space. However, we also used the work of [START_REF] Mikels | Emotional category data on images from the International Affective Picture System[END_REF] that attributed categories to the images. They are annotated as anger, disgust, fear, sadness, or undifferentiated for negative pictures, and amusement, awe, contentment, excitement, or undifferentiated for positive pictures. Some of them were the combination of two or three categories. For the happiness emotion, we extracted all pictures noted as amusement, contentment, or a combination of these two categories with another one. We ranked them with highest scores for these dimensions, suppressed the erotic pictures and selected the ones with high pleasure and arousal. We also avoided having several pictures representing the same thing/scene. For the fear emotion, we used the fear category and its combinations and used the same method of selection, with low pleasure and high arousal. We also did this for sadness (i.e., low pleasure and low arousal), but we removed the shocking pictures. For the neutral condition, we used the PAD space and selected pleasure just above the medium value and choose the ones with low arousal. For music we used the Eerola et al. database [START_REF] Eerola | A comparison of the discrete and dimensional models of emotion in music[END_REF]. We needed around 2 minutes music but Eerola et al. only evaluated 4 musics by emotion lasting around 1 minute. We decided to use, for each elicitation, two musics of 1 minute, from the 16 samples database. We selected the ones with high score for the given emotion, with low level of confusion. For the neutral emotion, we used the "tenderness" component of the database.

Finally, for the movie extracts, we used a French database [START_REF] Schaefer | Assessing the effectiveness of a large database of emotioneliciting films: A new tool for emotion researchers[END_REF] that classifies stimuli by category. We selected the ones with high score for the given emotion, with also low level of confusion and durations fitting the scenario (i.e., between 1 and 2 minutes). However, all the durations could not be similar.

The selected audio and video stimuli are presented in Table 4.2.

Evaluation of the system (Experiment 3) Presentation of the experiment

In order to confirm that our setup was able to elicit the expected emotions, we carried out an experiment which implied several aspects. To assess the emotion felt during elicitation, two methods were employed: the cognitive response of the participants using questionnaires, and the physiological response.

To tackle physiological measurements, the participants were wearing an ECG sensor for heart rate analysis, a GSR sensor for skin conductance and a respiration sensor to measure the respiration frequency. These sensors were from the TEA Captiv' package.

To measure emotions from the cognitive aspect, we used two tools. The first one is the Self Assessment Manikin scale (SAM) [START_REF] Bradley | Measuring emotion: the self-assessment manikin and the semantic differential[END_REF], which returns the emotional state on the three dimensional PAD 9-point Likert scale. The second one is composed of discrete adjectives to select. Several responses were accepted and additional adjectives could be proposed by the participant. The provided adjectives, with their French translation were extracted from the ones used by [START_REF] Schaefer | Assessing the effectiveness of a large database of emotioneliciting films: A new tool for emotion researchers[END_REF]. We used the French version for the French-speaking participants. We selected two groups to represent each of our emotional conditions, and we added two which should not be elicited (i.e., anger and surprise). The adjectives are presented in Table 4.3. These questions were either asked after each elicitation phase in VR, and on a post-experiment questionnaire that evaluates every stimuli used in the experiment (images, music and videos).

Other

Angry, Irritated, Mad En colère, Irrité(e) Surprised, Amazed, Astonished Surpris(se), Étonné(e), Stupéfié(e) Three questionnaires were used for this experiment. (1) A pre-experiment questionnaire provided the personality data of the participant (we used the Big5 personality test [START_REF] Goldberg | An alternative 'description of personality': the big-five factor structure[END_REF])), and precedence about VR use, or gaming habits. We also asked for specific phobias of the participant to detect overreactions. (2) The inexperiment questionnaires reported information about the global VR elicitation. We measured the emotional state, the variability of the emotion felt (5-point Likert scale for emotional variability, and a question about intensity evolution: instantaneous, oscillating, or progressive). We also asked to which modality the participant was the most attentive and receptive, and the problems that appeared during VR (mobility, artifacts, nausea). (3) After the experiment, the participants had to watch again every stimulus separately and evaluate the emotion felt, and if the emotion was more intense than in the VR elicitation. For the videos, we also asked if the sample duration was adequate (the emotion takes a longer time before being elicited, or it has time to vanish before the stimulus ends).

Experimental protocol

The experiment was carried out by 16 participants. Unfortunately, no gender balance could be insured but we made sure to select stimuli that were gender independent in the databases. The first step of the experiment was the pre-questionnaire completed online. The goal of the experiment, the procedure, and the risks were then explained. A consent form was signed and the participant could ask questions.

The game play was explained to the participant, the VR helmet was adjusted, and participants had to complete a short game to accustom to the joystick and VR view. no particular issue was detected with the participant, the physiological sensors were put in place and the signal was checked. Then the acquisition and the video camera were started. Each elicitation occurred in four steps. First, a relaxation phase with the VR helmet on and a relaxing music of 2 minutes. Then, the VR environment was displayed, it lasted between 3 and 4 minutes. After that a quick questionnaire was answered, and a cognitive game of 2 minutes (a Sudoku) enabled the emotion to vanish. After the 6 elicitation steps, the physiological sensors were removed and the participant was thanked and could provide some comments about the experiment. Finally, a post-questionnaire had to be filled online about stimuli. The first two VR environments were randomly positive or negative ambiance, then the four emotional conditions were displayed also randomly.

Results

In this analysis we use "null hypothesis" statistical tests. The output of these tests is a p value which corresponds to the probability to have rejected the "null hypothesis" whereas it was true. We choose the critical p value of 0.05, as it is the usual significance threshold used in cognitive science research.

In this sub-section, we first detail how the mood elicitation was evaluated, then the same is done about the 4 emotion conditions, then we analyze the results of each stimulus evaluations. Finally, we compare the modalities.

Mood elicitations

The "mood" elicitation has two conditions: a positive mood (P), designed to be relaxing (i.e., high valence and low arousal), and a negative mood (N), designed to be oppressive (i.e., low valence and high arousal).

The PAD evaluation made after each elicitation is performed in a 9-point Likert scale, thus non parametric statistical test can be used. Besides, the evaluations of P and N are dependent on the participant as each of them see both conditions. We compute a Wilcoxon signed-rank test 3 that detects whether the difference of marks of each participant is significantly different from zero.

One significant result concerns the arousal dimension (N Arousal = 6.19 ± 1.5, P Arousal = 4.56 ± 2.1, p value = 4e -4 , rc(16) = 1.0 4 ). The arousal of the negative condition is higher than the positive one, which corresponds to what we targeted.

However, the pleasure is equally evaluated.

The positive condition is largely considered as "calm" and "joyful", but, the evaluation of the negative mood depends on the order it was presented. When presented at first, the negative condition is viewed as "calm" ("calm": 6 times, "nervous": 2 times, upon 10 adjectives), but when it is presented after the positive condition, it is seen more "nervous" ("calm": 2 times, "nervous": 5 times, upon 13 adjectives).

So, the adjective reports confirm our expectations about the positive elicitation (P). However, the (N) condition suggests that our elicitation method is subject to order effects.

It is also interesting to notice that the negative condition is perceived as variable in terms of emotional category, but also in term of emotional intensity. The emotional intensity neither reaches its maximum instantly, nor progressively, but it is oscillating during the elicitation.

Emotion elicitations

The targeted (see What is important to assess in order to determine the efficiency of the elicitation, is if we have significant differences in the evaluation measurements depending on 3 We use the Pratt treatment that includes the zero-differences in the ranking process. 4 rc is the rank correlation, computed as follows: rc(s) = S-2T S where s is the sample size, S is the total rank sum and T the T-statistic output from the Wilcoxon test. rc = 1.0 means all the differences have the same sign. rc = 0 means the signs are equally distributed. This is a representation of the size effect.

the condition. For each PAD component, we compute a Kruskal-Wallis test. The null hypothesis of this test is that all the samples (i.e., whatever the emotional condition) follow the same non parametric distribution. They are all rejected, so there is at least one emotional condition, for each dimension, that is evaluated significantly different from the others. The results are as follows: Pleasure (H(3) = 25.2, p value < 1e -4 ), Arousal (H(3) = 19.4, p value = 2e -4 ), and Dominance (H(3) = 8.7, p value = 0.03)5 . 2.5 ± 0.9 3.0 ± 1.8 3.9 ± 1.4 5.3 ± 1.5 3.7 ± 0.9 5.2 ± 2.2 FIGURE 4.5: Expected PAD values for the four emotional conditions, in the circumplex space [START_REF] Russell | Evidence for a three-factor theory of emotions[END_REF]. The red numbers represent the Dominance component. The scale used is the 9point Likert scale of the questionnaire.

In order to determine if these significant differences are caused by the conditions that were targeted to be different, we perform pairwise comparisons. We use the Mann-Whitney U tests, whose null hypothesis is that both subsets belong to the same distribution. The sign difference of "N/H" is "<" meaning that H values are significantly higher than N. Note2: A rbc close to 1.0 means that the effect of the condition is important. Close to 0, there is no effect.

We count the number of adjectives to describe the elicitation phases, that correspond to what was targeted. It shows that happiness and sadness have good scores (79% and 77%), while the neutral case is noisy (56%), and the fear case is often confused with "surprise" (50%).

Considering the answers about emotion stability, the fear case appears to elicit more diversified emotions than the others, however, the intensity sounds to be more progressive (10 participants perceived it progressive, 3 immediate, and 3 oscillating).

The neutral case is not seen as emotionally progressive, and we cannot conclude for the other emotions considering this variable.

Detail of the stimuli emotional impact

The The sign difference of "N/H" is "<" meaning that H values are significantly higher than N. Note2: A rbc close to 1.0 means that the effect of the condition is important. Close to 0, there is no effect.

Besides the PAD evaluation, we analyzed the adjectives questionnaire. Fear is the emotion that have the most errors in adjectives qualification. It has 57% of mistakes compared to 65% for sadness and happiness, and 71% for neutral.

This analysis shows that the images do not induce contrary emotions than expected, and may have an important effect on the consistency of the global VR elicitation. It is nevertheless important to verify that all the images are consistent, or if some are misleading.

The images we used come from the IAPS database [START_REF] Lang | International affective picture system (IAPS): Affective ratings of pictures and instruction manual[END_REF] and we determine if the evaluations of our participants are consistent with the database. The authors provide the mean and standard deviation of the three PAD components, for each image. These values were computed using 100 participants who evaluated 40 images upon 960. Thus, we can estimate that each image has been evaluated by 6 participants. This is little to assume for normality, especially since their and our data come from Likert scales. However, we computed One Sample t-tests. This test has the null hypothesis that a distribution's mean is equal to a given value (i.e., in our case, the mean value given by IAPS). This test was performed for each image and each component. The results are presented in Figure 4.6. This shows that few images are evaluated differently from the database. What can be noticed, is that most of these images display lower pleasure in our case. This is beneficial for the fear condition, but not for the happiness one. In order to detect which images are the most efficient to induce the correct emotion, we calculated two types of scores for every image. The first type of score takes into account the qualification mistake rate. The second type of score averages the distance of each PAD component form the emotional reference [START_REF] Russell | Evidence for a three-factor theory of emotions[END_REF]. The distance is computed as described in Videos: Concerning the videos, as we only have one video per condition, we qualitatively comment the data. First, the pleasure dimension is well distributed: low pleasure for sadness, neutral is central, and high pleasure for happiness. However, the fear condition has two outliers with high pleasure, so its distribution is not so clear. The neutral condition has low arousal as expected. The dominance is split into two groups: fear and sadness with low dominance compared to neutral and happiness with medium dominance. On average the emotional responses match the emotions we aimed to elicit. About the adjectives qualification, the fear case has also a bad qualification score, like for the images: 46% compared to 68% for sadness, 82% for neutral, and even 91% for happiness. These last scores show that neutral and happiness were very efficient to elicit the right emotion. The sad video was often qualified as "calm". When asked about the length of the videos and emotion dissipation, half of the participants thought that the videos were too long. However, for the sadness condition, it was the case for most of the responses. This means that the videos may create boredom, and may let the emotion created earlier vanish. So we have two choices: shorten the video, or find a sample that involves more the participant in the narration. We chose the second choice for the sadness video.

Music: We evaluated two music samples per emotion and could select the most efficient ones (i.e., with the PAD values closer to what is expected). With the remaining music samples7 , we observed the following: the pleasure dimension is shifted with high values (An explanation could be that listening to music, in general, generates positive emotions). Happiness induces high pleasure, neutral is slightly above the center and sadness and fear are slightly below. For the arousal, all the samples are centered apart from happiness which is higher. For dominance, fear has low values, sadness is slightly below the center, while happiness has high values. Thus if only happiness elicited high arousal through music, in average, the chosen music samples are able to generate the right PAD responses.

Concerning the adjectives, the sadness condition has low qualification scores, it is often called as "calm" while the happy music elicits well happiness.

Comparison of the modalities

Finally, we can compare which modality is more efficient depending on the emotion elicited. After the VR experiment, we asked the participants to which modality they paid the most attention and to which one they were the most receptive. We found that for sadness, they were more attentive and receptive to images. The same effect is seen for fear with the video. In general, little attention was given to music. However, participants were the most receptive to this modality. Music had very little effect on fear. Several persons were more receptive than attentive to music for happiness. This may be due to a certain surprise of the energy of the happy music, and explains why it was the only condition where the music was evaluated with high arousal. Hence, we keep this music for the next experiment. Many people did not pay attention to a particular modality but they could choose an answer for receptivity. Finally, the ones who were the most attentive to videos or music were even more receptive to these modalities. This means that even when one pays no attention to a stimulus, it can still elicit emotion, and when attention is given to it, the response increases.

We also compared the emotional intensity and category differences between viewing the stimuli in the post-questionnaire (on a screen) and in VR. We used one Sample t-tests to estimate if one of the modality (i.e., image, music, or video) has a significant difference of intensity induction between VR and post-questionnaire (i.e., if the 5-point Likert scale measurements are different from 3.0). Only the music is perceived more intense in post-questionnaire, as possibly the subjects gave more attention to what they heard. Nevertheless, the video has measurements slightly below the center (i.e., the mean is 2.8). In the comments,the participants judged the videos more immersive in VR, which explains why the induced emotion was perceived more intense in this case. Concerning the type of emotion induced, for all the modalities, less than 10% of the subjects answered that they did not feel the same emotion in VR than at home.

What can be noticed from this analysis, is that VR does not alter the way each stimuli is perceived. Besides it enables a multi-modal elicitation, and each individual has a different sensitivity depending on the modality.

Discussion about the experiment

The experiment taught us a lot about how the stimuli we proposed are perceived and how virtual reality alters this perception.

The perception of the positive mood condition matched what we aimed to elicit, as it was viewed as "calm", and "joyful". The arousal was also lower than for the negative condition but still close to the medium value. This effect may be because we kept some action in the scene and the field of view was large. That confirms the model of Geslin, Jégou, and Beaudoin, 2016. On another hand, the negative condition was misjudged when elicited before the positive condition. This order effect could confirm the idea that one is more sensitive to a negative emotion after a positive one than the other way around. The emotion was also perceived as varying through time. This variation combined with the order effect makes difficult to know what is really elicited by the negative condition. Besides, in both conditions, the pleasure component did not show significant differences. As a consequence, the use of our mood elicitation system is not appropriate for our study.

Concerning emotion elicitation, we notice that happiness and neutral cases are very efficient. For the VR evaluation, they have the right PAD values. Happiness has good qualification scores, neutral is more noisy but it is because it does not represent a specific emotion. The joyful and neutral images, videos, and music, are all consistent in terms of: (1) the expected differences from the sadness and fear conditions, and (2) the success rate of adjectives qualification. This consistency in stimuli induced the right emotions in the VR environments. Besides, the fact that music in general induced more pleasure than expected, and the fact that in the happiness condition the participants were more receptive to music, may increase this consistency.

The sadness condition has the right pleasure and good qualification in the VR questionnaire. The arousal level is higher in absolute value, but when comparing to happiness and fear, it stays below. For all the modalities, the sadness condition is well located in the pleasure scale compared to other conditions. Images also correspond to what was expected in terms of PAD and adjective qualification. Participants were more receptive to the image modality for this emotion. This may have counterbalanced the too "calm" and long video, and the mid-aroused, mid-dominant and "calm" music. The sad temporal variation was not noted as progressive, thus one can either shorten the video/music, or choose more appropriate extracts. We chose the second option for the Sadness condition.

The fear case is more problematic. It can be seen when comparing VR evaluation with the expected PAD values: fear does not fit well, with more pleasure and more arousal. When comparing between the conditions, the dominance remains below, however, the pleasure is not easily differentiable from the neutral case. The good qualification scores are also low, but in fact the high level of confusion with "surprise" may explain why the pleasure evaluation is higher (the surprise emotion has the same arousal as fear but opposite pleasure). These observations about pleasure and adjectives are also valid for images and videos. The fact that for fear, the participants were more receptive to the video can explain the similarity of evaluation between video and the global environment (even though evaluations of music are better). In general, several emotions appeared and the intensity was only progressive. This progressiveness could be due to the suspense of the movie. This confirms that the attention given to a modality can change the emotion felt in the whole elicitation. Given these results, some work is required to improve the efficiency of the fear condition.

To conclude this discussion, we can say that the use of VR did not alter the stimuli efficiency. Besides, the multi-modality was proved to be important. Indeed, not all the participants focused their attention, or were receptive to the same modality. A link was found between the overall evaluations and the modality that caught the most attention. The modality that has the strongest impact on a elicitation condition are different depending on the conditions (i.e., music for happiness, images for sadness, or video for fear). However we cannot determine whether it is due to the emotion category itself that is more easily elicitable by a modality, or if it is the chosen extracts that had different elicitation efficiency. In any case, in the coming experiment, we keep these three modalities and try to give them a balanced weight in the scenario.

The results in this section were part of a contribution in a conference in 2017:

Orefice et al., 2017.

Device improvement

Given the efficiency of the stimuli at eliciting emotions, the importance of multimodality, and the advantages of VR, we decided to continue our work using these four emotion conditions. Nevertheless, some improvements had to be done. First, we decided to increase the realism of the scene, enriching the 3D models and lighting. We also wanted to balance the duration of each modality (i.e., around 1 minute per modality). We chose to allow differences in the four scenarios, as the purpose is not anymore to evaluate the stimuli, but to use the appropriate scenarios to induce the right emotion. Finally, we chose to involve more the participant in the process, using "active" elicitation. Thus, we exploited the Geslin, 2013 guidelines. The author proposed to generate sadness by mimicry with a sad person or the loss of someone the participant has been attached with. Fear can be induced by mimicry with a group panicking. The unknown creates fear, with deformity and darkness. Finally, happiness comes from accomplishment, or thanks to a gain. In the VR world, the participant firstly walks along a corridor whose wall texture depends on the emotion (golden wallpaper: N and H; muddy plaster: S and F). This is in order to set the pleasure level. This step lasts 10s and a music is broadcast in the headphones. This step is music-only to let the participant pay attention to it. The participant enters a circular room in which five pictures are displayed on the wall (i.e., the pictures selected after the previous experiment), the music is still playing.

This room is large and circular to leave space, to facilitate the mobility, and to give a general view of the pictures. After observing the pictures during 45s, the participant has to reach the center of the room. He/she enters a lift, which makes him/her the responsible for the next scene to load.

The H condition has a specific scene. It is a 2D mini-game where the participant controls a little bird and has to avoid obstacles during 50s. Then he/she wins and earns cookies distributed after the experiment. Thus, greed induced by the granted cookies, and success, can emphasize the happy feeling. Afterwards, for all the emotional conditions, the participant sits with other social agents in a theater room.

He/she watches a movie sample during 1min (except for the S and F conditions).

For the S condition, we chose a longer video (3min) to have time to become attached to the characters that is going to die. We consider that the video extract we chose in Table 4.7 corresponds well to the "get attached" plus "loss" criterion. Moreover, it was specifically designed to induce sadness. And, for the F condition, after 1min video, zombie growling can be heard in the headphones and zombies appear in the field of view. Besides, we made the movie spectators run away at seeing zombies, to create group mimicry. This animation lasts 1min, while the video continues, before the participant retrieves the joystick control. After the video, the participant reaches the theater's exit.

In Table 4.7 we define the stimuli used from the databases.

Effect of emotion on handshake (Experiment 4)

As soon as we had a system able to elicit emotions in an immersive way, we used it in an experiment involving emotion elicitation, and handshake measurements. These handshake measurements were performed with improved instrumented gloves compared to the experiment in Chapter 3. This is described in Section 4.5.1. The general procedure is to handshake the participant before elicitation, then after, and compare the pressure exchanged. In this chapter, and also in the next one, we only analyze the pressure data even though we also measure the movement. This is due to the little number of contributions in the tactile plus handshake area (see Chapter 3 Section 3.2). The movement can be analyzed in future work.

In order to keep consistency in the contexts of handshake and elicitation, the interaction is part of the scenario. Thus the participants handshake a virtual agent (physically played by the experimenter), placed in the virtual environment scenes (see Figure 4.8). This requires a specific experimental setup described in Section 4.5.2.

The protocol, results and discussion are detailed in the following sections.

Glove improvement: design of new sensors

In order to have more precise measurements in terms of spatial distribution, we decided to increase the number of sensors compared to Chapter 3. In order to increase the sensors robustness we made them softer. This implied to create home-made sensors. We also wanted to measure both the participant's action, and the experimenter's perception. Thus we designed two gloves. These were identical in order to also measure the experimenter's action. The pressure sensors we made are piezo-resistive and they are described in Figure 4.9. They use a resistive rubber layer, 0.1mm thick, sandwiched between two copper electrodes. When pressed, the contact surface between rubber and copper increases and the distance between the electrodes decreases, resulting in a resistance drop down. After a certain threshold, the logarithm of resistance lowers as an affine function of the pressure. The sensors we designed have an electrode surface of 2x6mm 2 . The resistance is measured using 10kΩ voltage divider and three 12-bits 8-channels analog to digital converters (i.e., MCP3208). The sensors are sewed on an elastic-fabric glove, and covered back by a second glove, to protect the sensors and hide the wiring. We calibrated individually the 46 sensors, 8 times, once they were in place, from 0 to 80kPa (72kΩ), 150kPa (27kΩ), or 250kPa (6kΩ). The pressure was applied using a 18x18x10mm 3 piece of foam pushed by a force cell. We modeled the logarithm of the resistance (R) as a square polynomial of the pressure (P) until P lim = 25kPa, and then as a linear law (see the equation below). R(P)= exp(a 1 .P 2 + b 1 .P + c 1 ) if P < P lim , exp(a 2 .P + b 2 ) if P >= P lim P lim was chosen after viewing all the calibration points of log(R) as a function of P. a 2 and b 2 were computed to fit a linear regression of the data above P lim , and a 1 , b 1 , c 1 were computed given the continuity and tangent constraints, and so that R(0) = 1e 8 Ω. An example of the computed model is shown in Figure 4.9c).

Each glove is composed of 23 pressure sensors, distributed as depicted in Fig- Chapter 3), we only increased their number. Here also, we can define two groups of sensors per glove: the ones activated by the glove owner (i.e., group s), and the ones activated by its partner (i.e., group r). 9 sensors belong to group s, and 10 sensors belong to group r. In Chapter 3, these roles were attributed from assumptions, but here we use the results of an intermediary experiment, which is described in Appendix A.

We can also remind some notations that are presented in the appendix and is used in this chapter: Gp is the group of all the sensors activated by the participants (i.e., the 9 from his/her glove and the 10 from the experimenter's glove). The same way, the action of the experimenter is measured by the Ge sensors. Compared to the instrumented glove used in Chapter 3, the acceleration and angular speed are still recorded, but the accelerometer chip is able to compute the orientation of the glove using data fusion, while it was post-processed in the first version. The data is acquired in real time, at 100Hz and transmitted through Bluetooth for the participant, and USB cable for the experimenter directly to a computer.

Overall experimental setup

As this experiment required both to elicit emotions and perform handshake measurements, many components needed to be set up. We can group them in three categories (i.e., VR control, physical link with VR, handshake measurement) that are detailed in this sub-section and in Figure 4.11.

The first components concern the VR. Like in the previous experiment (see Sec- The second set of components aims at reproducing the physical handshake interaction into the virtual environment. Indeed, the participant sees an avatar in front of him/her (see Figure 4.8a)e)) and has to handshake with it. The avatar is actually controlled by the experimenter's movement, and the participant can see the hand of his/her avatar controlled by his/her own movement. This is achieved using a fusion technique between the absolute position of the hands, and the acceleration/ angular speed acquired in real time by the gloves. The position is obtained thanks to a Kinect that detects green paper markers on the shoulder and wrist of both individuals. The shoulders adjust the position of the avatars, and the wrists adjust the relative avatars' hand position. The orientation of the avatars' hand is controlled by the accelerometer data. In order to have the whole system work, both individuals have to face each other in a specific position as shown in Figure 4.11. The participant is formally asked not to physically walk during the experiment, but he/she can turn around. Besides, the VR helmet can have a rotation drift. Thus, before handshaking, the participant's view is reset corresponding to his/her real body angle. This makes him/her turn around back in the right orientation to face the avatar, and the experimenter. The body angle it obtained by a fifth green marker. Finally, when the handshake starts, the experimenter can see the view of the participant to adjust his position.

The third type of components are for handshake recording. Both the participant and the experimenter hold an instrumented glove on their right hand. All the participants are chosen to be right-handed. The gloves stream the 23 pressure values and acceleration/ angular speed to the main computer where they are saved. Besides, a video camera records the whole interaction. An example of handshake interaction can be seen in Figure 4.12.

FIGURE 4.12: Exemple of handshake interaction with the virtual agent, and view of the participant

Experimental protocol

The experiment was carried out with the help of 12 participants, but due to measurement problems, only 10 (4 women and 6 men, with an average age of 25 years old) are shown in the data analysis. They firstly have to answer a questionnaire (Qpre-exp) including personal information and background in video games.

After explaining all the phases of the experiment and allowing to ask questions, the participant is entering a training phase to get used to the VR setup and the joystick. Then the gloves and optic markers are installed. Three handshakes are performed and recorded without helmet. Back in the virtual world, the participant is able to see his/her virtual hand and virtual body moving. Some handshakes are achieved with the virtual agent until the participant is ready, then three handshakes are recorded. These first steps of the experiment are covered in 20 minutes. Three elicitations are completed. For every elicitation there is a relaxation phase with music (2min), two handshakes without helmet and one with the agent. The participant plays the elicitation scenario. The emotion displayed is chosen randomly. After reaching the theater's exit, two handshakes are performed with the agent, and two with the experimenter without the helmet. This process lasts 7min. Then the participant is asked to fill-in a questionnaire(Qpost-el). This questionnaire is used to measure the emotion felt during the experiment (using Pleasure-Arousal-Dominance Manikin scale, ordering groups of adjectives (the same as in Section 4.4.3), and evaluating the emotion fluctuation during VR). It also checks if the emotion changed during the handshaking act. Finally, we included questions about modalities efficiency, for instance, if images were more efficient than the video to elicit a specific emotion. This step lasts 4min. The whole experiment lasts one hour. At the end, the participant receives as a gratification some cookies.

Results

This experiment enabled to collect 10 participants x (3 trainings x 2 agents + 7 x 3 emotions) = 270 handshakes. Each handshake sample is associated with 46 timedependent pressure data, combined with acceleration, position and orientation data.

In this study, we only focus on pressure, and more precisely, we extracted the maximum value reached by each sensor during the handshake. It is noted Pm. We chose this feature because the experiment in Chapter 3 showed a high correlation between the maximum and average pressure. Hence, a large part of the information contained by these features is redundant. We also think that Pm contains more variability as it shows to which extend the participant was able to press during the handshake.

The first analysis explores the global variability of the data, to see how it is impacted with the numerous experimental conditions (e.g., participant, before or after VR, with the virtual agent or not). Then, we present how the elicitation scenarios were perceived by the participants, to see if the handshakes were performed with different emotional states. Finally, the impact of emotional condition on handshakes is explored.

Description of the data variability

First, we introduce the notations used to describe the experimental conditions. A handshake is performed during the training phase (T), or during an emotional condition phase (neutral: N, happiness: H, sadness: S, fear: F). The handshake can be with a human (tH) or face to avatar (tA). The order of handshake can occur before (o1) or after (o2) VR.

Previous studies, like the one presented in Chapter 3, showed that the handshake manner changes depending on the individual, and especially depending on his/her personality or gender. It is important to investigate how our data is impacted with this inter-participant variability. We take all the handshakes before elicitation (o1)

including T, face to the experimenter (tH), which represents a baseline of 9 handshakes per participant. An interesting observation would be to see, for each sensor i, if Pm i is significantly different depending on the participant. The distribution of Pm i is not Gaussian. This is due to the fact that when a sensor is not touched, the measured value is zero. This creates a high asymmetry in the distribution. As a consequence, one cannot use parametric statistical tests. We use the Kruskal-Wallis test for each sensor, compared to the participants. The result is that every sensor has a response significantly different (p value < 0.05) depending on the participant (the median values of the output of the tests are: H(9) = 51.8, p value < 1e -4 ) 8 . When we take all the data (i.e., tH, tA, o1, and o2), the difference is even more important (H(9) = 134.0, p value < 1e -4 in average). This may suggest that the participants have different responses to the conditions. However, when the Kruskal-Wallis null hypothesis is rejected, it means that at least one participant has a different behavior than the others. We compute Mann-Whitney U tests to account for pairwise differences between the participants. 45.9% of the 4140 tests are significant for the baseline subset, and 72.5% when applied to all the samples. To conclude on these results, most of the participants have a different effect on the 46 sensors, and this variability increases when we include the different elicitation conditions and the use of a virtual agent. One can moreover notice that it concerns either the sensors activated by the participant or the experimenter. So, the experimenter unconsciously changes his handshake depending on the participant. Given the later result, it is important to characterize the amplitude of variability induced by the experimenter. The idea is to assess if the fact that a sensor belongs to the group of the 19 sensors activated by the experimenter (Ge) makes the variability of its data lower than if it belonged to Gp. We first take the baseline subset (with 9 handshakes x 10 participants = 90 samples). We compute, for each participant, the standard deviation σ i of Pm i , for each of the 38 sensors that belong to Ge and Gp. This makes a dataset of 38 sensors x 10 participants = 380 values of σ. This was in order to remove the effect of pressure amplitude variation depending on the participant observed in the previous paragraph. We compute a Kruskal-Wallis test of σ depending on the group Ge or Gp. The result is not significant and shows that the variability of the experimenter's action is similar to the participant's action in the baseline samples (H(1) = 3.2, p value = 0.073). However, the difference is significant when we use all the samples (i.e., tH, tA, o1, and o2): H(1) = 4.6, p value = 0.031. This may confirm the fact that the emotional conditions, or the agent condition have an effect on the participants. Nevertheless, one should be aware that the samples used for this analysis are not totally independent from the participant (i.e., each individual may have a different order of magnitude of the standard deviation value). Therefore, we computed, for each individual independently, the Levene test whose null hypothesis is the equality of variance between two groups. The difference of variance between the action of the experimenter and the participant is significant in 7 cases out of 10 (the median values of the outputs of the 10 Levene tests are: w(1, 25) = 42.5, p value < 1e -4 )9 . Upon these 7 significant differences, 4 show the experimenter variance is lower than the participants. As a conclusion to these results, the behavior of the experimenter is generally more stable than the participant's and may have less effect on the measurements than the consequence of the emotion elicitation. In the remaining part of the study, we only use the sensors activated by the participant (i.e., Gp).

Elicitation efficiency

Concerning emotion perception from the participants, the performance of the elicitation device is mitigated. All the participants had positive pleasure in the H condition, however, this elicitation was often marked as "Tender", "Calm", and sometimes as "Surprising". The S condition was seen with low pleasure but in two exceptions it was perceived with high pleasure. The main adjectives were "Anxious" and "Sad", but several participants noticed "Joyfull" and "Exciting". This emotion was largely stable apart from two samples, given them, the intensity was growing. The F case has alternatively high and low pleasure, but always high arousal. Many adjectives were used to describe this elicitation but "Surprise" is always selected. The N condition was a mix of all kind of emotions.

Overall, in terms of PAD space and adjective description the elicitation tool targeted the right emotions, but some outliers may bias our experiment. It is nevertheless important to evaluate if the emotion was still present while handshaking.

In only half of the conditions, the participants felt the same emotion handshaking the avatar than during the elicitation phase. However, it is interesting to notice that it is worse when handshaking with the experimenter. This means that a continuity of elicitation context is important to preserve the emotion during the interaction phase, but it can also exhibit emotion dissipation.

Impact of emotions

Using the 19 sensors of Gp, we aim to see if the magnitude of the pressure, or its location in the hands, varies depending on the elicited emotion. The first step is to describe the handshakes, with a reduced number of features, that are representative of the information. In the Appendix B we propose such a description, and characterize the distribution and relationship between the components. However, this was only used in the last experiment presented in Chapter 5. When applied to the measurements of this section, no significant results were found about the localization of the pressure center. As a consequence, we only present here the results concerning Before investigating the emotion effect, it is important to know whether we can merge the data in the tH condition with the tA condition. Indeed, the first condition has the advantage to prevent from the disturbance induced by doing a virtual handshake. The second one was performed in the same context as the elicitation which should make the data more dependent on the emotion category, considering emotion continuity. We computed Mann-Whitney U tests to account for pairwise differences between tH and tA, using all the data (i.e., all the elicitation conditions including the training phase (T), before, and after VR). However, this was computed for each participant individually. We have 27 samples per participant. Only 4 participants out of 10 show significant differences while interaction with the experimenter or viewing the virtual agent, for at least one of the three variables. What is interesting it that every time the difference is significant, the pressure exerted is lower when interacting with the virtual agent. However, we have to notice that most of all the significant comparisons have p values between 0.01 and 0.05, so the effect of the condition is limited. Besides, none of these participant reported being disturbed by the avatar interaction. Only one remarked the virtual hand movement was not fluent enough. Given these results, we made the decision to merge both tH and tA datasets.

We investigate now if the fact of performing a handshake after the VR scenario, compared to doing it before, whatever the emotional content, induces a significant difference. We selected, for each participant and elicitation condition, the first handshake with the experimenter before VR to be compared with the first handshake with the experimenter after VR. The same was done for tA. We then merged all these pairs in one dataset. We computed a Wilcoxon signed-rank tests to compare between o1 and o2 for each of the three variables (i.e., maxTop, maxMid, and maxBot). Only one result is significant, which is for maxMid: p value = 0.006, rc(60) = 0.43. The pressure exerted in the palm after VR appears to be higher, whatever the emotional condition. However, we need to analyze the effect of each emotional condition individually. Thus, we computed 4 Wilcoxon signed-rank tests on the emotional subsets containing between 12 and 16 samples each. Very few comparisons are significantly different. The pressure maxMid it higher after VR than before in the Neutral condition (p value = 0.047, rc(16) = 0.58) and Sadness condition (p value = 0.030, rc(16) = 0.62). We also found that maxTop is lower after VR in the Sadness case (p value = 0.030, rc(16) = 0.65). These three results are very close to the significance threshold (i.e., p value < 0.05), besides maxTop appears to behave the opposite way than maxMid for the same Sadness condition, and we found an effect of the Neutral condition. All these observations do not enable us to conclude concerning the effect of the emotional conditions on the pressure exerted during handshake.

Conclusion of the experiment

In this experiment, we used the elicitation device developed in Section 4.4.3 and added a handshake interaction, between the participant and the experimenter, or a virtual agent. We firstly studied the variability of these handshakes in terms of pressure. The inter-participant variability was visible on the response of each sensor.

This indicates that the handshakes differ in terms of magnitude of the pressure and localization of the touched sensors. Besides, the experimenter's action also depends on the participant. It has nevertheless been showed that the variability of the sensors activated by the experimenter, is lower than the variability of the ones activated by the participants. This is in favor to the following assumption: the participants have a less constant handshake manner due to the impact of the experimental conditions than usual.

The participants were subject to different emotion elicitation scenarios, and we verified the consistency of the induced emotions. The questionnaires demonstrated that the elicited emotions are not systematically well received and maintained until the measurements. Besides, a significant difference was observed between handshaking the virtual agent and the experimenter (i.e., the firmness with the virtual agent is lower) for some participants. This suggests that even though they did not report it, the interaction with the virtual agent, may disturb the handshake manner. Considering the effect of the emotional conditions, no difference was visible in the data.

Several remarks can be identified concerning these results. The fact that no impact of emotions was observed in our data can be due to three reasons. (1) The expected emotions may not be efficiently felt by the participants during the handshake interaction. The verification measurements performed through questionnaires were mitigated, whereas the elicitation stimuli were approved by a previous experiment in Section 4.4.3. The duration of the elicitation may have finally made the emotion vanish when handshaking, as it was said by some participants. The complexity of the setup may have disturbed the interaction as being not natural anymore.

(2) The emotional state may not be transmitted through handshake. To our knowledge, this experiment is the first of its kind, nobody has shown the ability of handshakes to convey emotions. Some ideas can explain a failure to convey emotions. The emotions appear after a specific and punctual event. The handshake is a way to initiate an interaction, so, at this point, the object of the emotion disappears, and the attention is focused on the new interaction. A more appropriate way to make someone exchange his/her emotion through handshake may be to make the handshake partner be the object of emotion himself. For instance, make the participant be afraid of the person he/she has to handshake. However, this method creates a socio-emotional context that changes the representation the participant has of his/her partner. It would be difficult to differentiate the social patterns from the emotional ones. An other way to study handshake compared to the internal state of an individual in such a way the internal state is independent from the handshake partner, may be to focus on mood instead of emotion. Indeed, the mood is not linked to a specific object, so it may be more appropriate to be preserved during the social interaction, and be conveyed through handshake.

(3) The expression of emotion through handshake may be too subtle to be observed by our measurement system. More sensitive sensors and higher spacial resolution may allow a more precise representation of handshake, or to compute more complex features, like a description of the timedependent signals. Besides, increasing the number of samples should improve the robustness of the models. Further studies should focus on little number of participants but with multiple elicitation and handshake iterations for significance. This leads to a new question as stimuli may not be efficient repetitively. This could be addressed using larger databases of efficient stimuli.

Conclusion of the chapter

In this chapter, we showed how the available knowledge about emotions gives cues on how to measure them, either physically or cognitively. We presented some of these methods that can be used in our work. Indeed, when studying a specific response modality of emotions, it is important to assure by robust ways that the right emotion is experienced.

We then provided a classification and examples of techniques to elicit emotions in a controlled manner. Making professional actor self-induce an emotion is not suitable for a large cohort of participants, and asking non professionals to act an emotion would not lead to an ecological procedure. Indeed, when studying touch interaction and overall social touch interaction, acting emotion would drive to a substantial expression bias.

The use of emotionally charged audio-visual stimuli is a good alternative especially since annotated databases are available. Having a succession of such stimuli eliciting through several modalities may be an efficient way to increase the duration of the emotional state and its intensity.

To prevent from the drawback of passivity of the participant while eliciting, the use of virtual reality appears to be beneficial. The engagement of the participants would increase and his/her emotional response and its consequences in the social interaction may be more natural.

We designed VR scenarios to elicit four emotions. We evaluated them using questionnaires, and the results encouraged us to use this system in an experiment involving handshakes.

We carried out an experiment in which participants were subject to emotional content made to induce spontaneous emotion experiences. They performed handshakes before and after elicitation. A high inter-participant variability was observed in the pressure exchanged during handshake. However, no effect of the emotion condition was seen. This can be due to many reasons. Among them, the fact that handshakes may not be the most appropriate way to measure emotion impact on tactile interaction. Nevertheless, we assume that the mood has several advantages for our research.

In the next chapter, we present an experiment involving little number of participants performing repeated handshakes at different days. These handshakes are associated with the spontaneous mood of the participants. We also compare the interaction facing the experimenter, with the one facing a robot.

Chapter 5

Mood and handshake

Introduction (Experiment 5)

In Chapter 4 we discussed that emotion may not directly impact a handshake manner. In Chapter 2 Section 2.4.3 the mood was defined as a long term internal state, not linked to a precise object or event. Therefore, we consider that there is more chance for the mood to be communicated through handshake. This internal state could still be present during handshake interaction, even though this interaction is unbound to what caused the mood. As a consequence, after having studied long term internal characteristics of individuals (i.e., personality and gender) in Chapter 3 and short term affective states (i.e., emotions) in Chapter 4, we study in this chapter medium term affective states (i.e., moods).

In Chapter 4, we found that a lack of samples per emotional condition and participant, may be a reason why we did not observe a correlation between emotion and handshake. In this chapter, we used repeated measurements per participant.

Besides, in order to have ecological measurements, we studied spontaneous moods.

This means that we did not try to control the affective state of the participants. As the mood variation time is several hours to days, the experiment was carried out at distinct days. We still used human-human interactions in order to get a baseline unrelated to the physical limitations or appearance of a social robot. Finally, we also performed the measurements with a social robot, and compared the results.

In this chapter, we firstly present, in Section 5.2, some setup components such as:

the robot we used, the instrumented glove we designed for it, some improvements in the data acquisition, and how we measured the mood. Then, we detail the experimental protocol with the participants' characteristics in Section 5.3. The results, in Section 5.4, consist in several steps. We describe the global data distribution we collected. We discuss the mood distribution as it is an uncontrolled experimental condition. Finally, we present the impact of the mood on the pressure data during handshake. We also add some results concerning the inter-individual variability and the behavioral consistency between the human and robot agents.

Experimental setup

Handshaking with a social robot

In the experiment described in this chapter, we have two main conditions: the participant handshakes with a human agent and then handshakes with a social robot.

The human agent, like in the previous chapters, is a single experimenter, in order to have comparable results between the participants. The social robot we used is the Pepper robot. Its hand is comfortable, child-sized, with 5 fingers and has one actuator to be closed. As the shape of its hand does not exactly fit a human hand, we designed an instrumented glove specifically for the robot. The position of the sensors is presented in Figure 5.1. Some precisions need to be pointed out on how a handshake is performed with the robot. A picture of the hand grasping is shown in Figure 5.2. A handshake occurs as follows: when the participant is in front of the robot, the latter rises its hand saying "Good morning". Then the participant approaches his/her hand, the robot closes its own, and the participant shakes until the hand opens. The robot is programmed to wait 1.5s before opening the hand. During a handshake, the arm of the robot stays rigid. The closure of the robot's hand is triggered thanks to a

Hall-effect sensor placed at the center of the robot's hand, and a small magnet in the participant's glove. The closure command is sent 3cm before touch.

Measuring the mood

In this chapter, we investigate the consequences of mood on handshake manner.

Therefore, it is important to define precisely what is the mood and how it differs from emotions. After giving some theoretical background about mood, we present a questionnaire we used to measure it.

The main differences between mood and emotion are their duration, and their relation with the cause or object that induced them. Emotions are linked to an object, like explained in Chapter 2 Section 2.4.3. As presented in Frijda, 1994, we are angry against something. This object is usually identified by the individual [START_REF] Ekman | The nature of emotion: Fundamental questions[END_REF]. This object can be a punctual external event, or an internal event (e.g., a memory). The emotion experience and response is short term (seconds to minutes). On an other hand, moods are longer term [START_REF] Beedie | Distinctions between emotion and mood[END_REF],

and not linked to an object as the cause is not identifiable [START_REF] Ekman | The nature of emotion: Fundamental questions[END_REF]. However, similar to emotions, moods impact behavior and attitudes. [START_REF] Desmet | Mood measurement with Pick-A-Mood: review of current methods and design of a pictorial self-report scale[END_REF] gave the following characteristics to the mood: "moods are low-intensity, diffuse feeling states that usually do not have clear antecedents, are not directed at a particular object, and can last for hours or days but are limited in time". to assess the mood. Each quadrant was divided in two parts, to increase precision in the scale, by defining 8 distinct moods. A neutral mood was also introduced (see Figure 5.3).

Numerous mood questionnaires have been developed, and differ in many ways.

Indeed, they can be used for many applications, such as the evaluation of the long term effect of a new product on individuals' life quality, or detecting depression in a medical context. [START_REF] Polak | Measuring mood: considerations and innovations for nutrition science[END_REF] propose a review of the mood measurement tools, in the context of nutrition.

Unlike verbal questionnaires in which participants have to answer several questions (e.g., PANAS-X [START_REF] Watson | Toward a consensual structure of mood[END_REF] with 60 questions, or the Brief Mood Introspection Scale (BMIS) [START_REF] Mayer | The experience and meta-experience of mood[END_REF] with 16 questions), the PAM is graphical. The participant has to pick an image that represents the mood.

We chose this questionnaire in our experiment, as it sounded to have a sufficient granularity and does not require high cognitive load or time to answer.

In the PAM, each mood is represented by an expressive drawn character, placed in a circumplex space (see Figure 5.3). Three versions were created, using three types of character: a female, a male, and a neutral gender robot. We did not choose the robot version in case the participant try to infer a mood for the robot instead of themselves. Even though no gender differences was observed by the questionnaire designers, we chose a unique version for all the participants. We chose the female one. We did not put the labels of the moods, as we wanted the participants to compare themselves directly to the characters, not the semantic.

Every day, the mood of the participant was measured using the PAM questionnaire. This questionnaire is a set of 9 moods (i.e., Neutral state, Calm, Relaxed, Cheerful, Excited, Tense, Irritated, Sad, and Bored), displayed as unlabeled expressive drawn female characters on a circumplex space (i.e., Pleasure-Arousal).

Experimental protocol

This experiment aimed, for a small set of participants, to measure if their handshake pressure changes depending on their mood, the day of the measurement. Thus, many days were required to have repeated measurements and enough variability in the mood condition. 11 participants performed 8 handshakes (4 with the experimenter and 4 with the robot) every morning of 16 non-consecutive days. As we wanted to measure the behavioral response of the participant only, they were clearly asked to initiate the handshake. A total of 1408 handshakes were recorded (with 128 Our 11 participants were 3 females and 8 males, between 24 and 30 years-old, and all right handed. They all had technical background about Robotics.

Before the 16 days of experiments, the participants came for a training day. They signed a consent form and the mood questionnaire, and the protocol was explained to them. They also executed a few handshakes. The protocol used in the 16 remaining days occured as follows. When the participant arrives in the lab in the morning, he/she comes spontaneously (or is invited to come) in the experimentation room. He/she puts on the tactile glove, and performs one handshake with the robot (see Figure 5.2). Afterwards, a handshake is performed with the experimenter (see Figure 5.4). These correspond to the "first" handshakes of the day. Then, to have repeated measurements, the participant shakes 3 times with the robot, and 3 times with the experimenter. Finally, he/she answers the mood questionnaire and removes the glove. The questionnaire is answered after the handshakes so that the participant cannot alter his/her handshake manner depending on the answer. The way the robot handshakes could mislead it as the initiator, however this was motivated by technical reasons, and the participants were still ask to initiate.

Results

As explained in the introduction of this chapter, the main goal of this experiment was to compare the handshake manner with the mood of the participants. Before answering this question, we have to find a way to represent the pressure data so as to make it comparable with future experimental setups. This description uses 6 components which are detailed in Appendix B. In Section 5.4.4, we present how these components are participant-dependent. This indicates that handshakes cannot be compared directly, without a between-participant standardization. We also investigate, along our analysis, if the observations made from the human-human interactions, are similar with the human-robot interactions. We also examine the overall distribution of the computed components, before and after the standardization process, in Section 5.4.1. Given all this information, we were able to process the mood analysis. The results are detailed in Section 5.4.3.

This experiment enabled to record 704 handshakes per type of interaction (i.e., Human-Human or Human-Robot). The pressure data was extracted as follows: the start and end of handshakes were manually selected considering the first rising and last decrease of pressure. Then, the voltage signals were converted to pressure, based on the 70 calibration functions of the sensors (see Chapter 4 Section 4.5.1).

Putting on the gloves creates a pre-strain on the sensors, which is summed to the pressure exerted due to the interaction contact. We decided to remove this offset by subtracting the minimum output of the sensors measured during handshake.

Pressure data distribution

In this sub-section, we firstly present the 6 components used to describe the participant's handshakes. We then give some order of magnitude of these measurements, and present how they are distributed in order to choose statistical tests to compare them with the participants' mood.

Like in Chapter 4 Section 4.5.4, we use in this result section the 19 sensors activated by the participants (Gp), and compute 6 components as described in Appendix B. The sensor locations are presented in Figure 5.5, and we remind here how the components are calculated. We separated the sensors in two groups one for the top of the grasp, and one for the bottom. We then computed two geometrical and one magnitude features for each group. The notation is summarized in Table 5.1. 5.2 shows the order of magnitude of the maximum pressure applied by the participants, in both the "Top" and "Bottom" groups. As the data in the corpus has a collection of experimental conditions (i.e., participant, mood, agent, day), one should analyze the distributions separating the conditions. Thus, we start by observing the normality of each component, for each participant individually. We find that in several cases, the data is not normally distributed (see Table 5.3). In those cases, the distribution can either be uniform, asymmetric, or a combination of two Gaussians (see some examples in Figure 5.6).

We find that either with the robot or the human agent, YTop and XBot are normal for most of the half of the participants. For XTop and maxBot, this ratio is opposite depending on the agent. YBot and maxTop have medium ratios. In some cases, for the maximum pressure values, the non normality is due to a high number of low pressure values. This artifact is however much less visible than in the individual sensor distributions. Concerning the cases where the feature distribution is a combination of two Gaussians, one could imagine that it is due to separate reactions for two types of mood for instance. However, this hypothesis is difficult to verify for each participant individually as each mood is represented by too few samples per participant. This is the reason why, in the remaining part of the study, we analyze two separate datasets gathering all the participants, one concerning the human agent, the other concerning the robot agent.

We manipulate the data in order to make it independent on the participant. Indeed, given the results in Chapter 3, and as it is detailed in Section 5.4.4, the handshake manner shows high differences depending on the participant. Thus, we standardize the data (i.e., we remove the mean value, and divide by the standard deviation), for each component, and each participant. One has to keep in mind two things about this manipulation: (1) this is done even for the sub-datasets that are not normally distributed.

(2) the data is not totally participant independent, as each individual may have different reactions when subject to the same affective state.

In the rest of this section, we detail how these two new datasets (one for each agent) are distributed. We computed Wilks-Shapiro1 tests for each of the standardized components and the results are in Figure 5.7. Most of the distributions are not normal even though they visually seem close to Gaussians. As our goal is to observe the effect of the mood in the data, we have to study the mood subsets individually to check their normality. As it would be too tedious, we only evaluate the normality of the residual2 of One-Way ANOVAs of each component depending on the mood (see Table 5.4). This evaluation is a usual shortcut used to verify The second requirement for ANOVA concerns equality of variance between subsets. This is evaluated using the Levene test. The null hypothesis of this test is that all the variances are equal, so we need this hypothesis no to be rejected by a p value lower than 0.05, in order to use ANOVAs. The results of the Levene3 tests depending on the mood (see As a conclusion to this subsection, we defined 6 components that describe the handshakes we collected. We gave the ranges of maximum applied pressure in our dataset. We observed how the data is distributed for each participant and found that it does not necessarily follows normal laws. We removed the inter-participant variability standardizing the data, but we still noticed that the impact of affect on handshake can be participant dependent. We presented how the remaining data is distributed, and showed that it is closer to Gaussians. Several components have normal residual when computing One-Way ANOVAs depending on the mood.

Some of these also show equality of variances, which enables to use ANOVAs.

Mood distribution

Each handshake measurement is associated with the mood of the participant. This mood belongs to a set of 9 labels, that are placed on a circumplex circle, centered on the neutral mood, and with two independent dimensions: Pleasure and Arousal (PA) (see Figure 5.3). Nevertheless, moods were non-controlled conditions, the participant performed the experiment with the mood they had that day. Thus the chosen mood labels may not be homogeneously distributed among the 11 participants x 16 days = 176 experiments. Data extraction revealed the two main moods are Cheerful and Calm, which were chosen 39 and 35 times, respectively. The Relaxed mood and Neutral state were chosen 23 times. The negative moods were less present:

Tense (15), Bored [START_REF] Nakasone | Emotion recognition from electromyography and skin conductance[END_REF], Irritated (8), and finally Sad with only 6 occurrences. Excited was selected 13 times. These numbers have to be multiplied by 4 handshakes per day and per dataset. Some samples had also to be removed due to measurement errors (3% of the samples). Table 5.5 gives the total number of handshakes with complete data associated with the mood categories, for both agents.

Impact of the mood on pressure

Given the results in Section 5.4. Several results are significant, such as YBot and maxBot for both agents. In Appendix B it has been shown that these two features are highly positively correlated (i.e., Spearman coefficients of 0.6), so we only study in detail maxBot. In the robot case, two other features show significant differences depending on the mood: XTop and XBot. These two features are slightly negatively correlated (i.e., Note1: The sign of "Bored/Excited" being "<" means "Bored" has lower maxBot than "Excited". Note2: rbc is the rank-biserial correlation, close to 1.0 it means that the effect of the condition is important. Note3: To get the size of the mood subsets, please refer to Table 5.5.

observed between opposite moods in the PA space. Finally, the moods in the same quarter of the PA space do not induce significant differences.

Considering the sign of the differences, one can notice that low-aroused moods induce lower pressure than high-aroused ones, every time the difference is significant. In order to verify this observation, we compute the arousal component of the moods, placing each mood evenly spaced in a unit circle, and calculating the sinus of this position. We then compute the Spearman coefficients between the mood arousal and maxBot in both human and robot datasets. We find a significant positive correlation (human case: (ρ = 0.11, p value = 0.004), robot case: (ρ = 0.14, p value = 2e -4 )). However, the value of these coefficients is low.

To finish with the bottom-grasp analysis, we verified that YBot, which is correlated with maxBot, behaves the same way depending on the mood, and if it enables to give more relevant information about the handshakes. The significant sign differences are presented in Table 5.8. Considering the human agent case, the results are consistent, however, in the robot case, the significant differences are not for the same mood pairs. Besides, two comparisons show higher YBot values for less aroused moods, but the p values in these cases are the highest (i.e., higher than 0.02).

The last analysis that we performed concerns the XTop component, in the robot interaction case. The pairwise comparisons are presented in Table 5.9. This time, only the "Tense" mood can be distinguished from the three other moods. The difference is even visible when compared to "Irritated", which is close in the PA space. For each of these three comparisons, XTop is lower in the "Tense" case, meaning that

= Pleasure = Arousal Bored/Excited < < = < Sad/Excited < < = = Tense/Relaxed = = < = (2) -Pleasure = Arousal Sad/Irritated = < = = Bored/Irritated = = < < Bored/Tense < < = < (3) + Pleasure = Arousal Calm/Cheerful = = < < Relaxed/Cheerful = = > = Relaxed/Excited = = = = (4) = Pleasure = Arousal Calm/Relaxed = = < = Tense/Irritated = = = = Sad/Bored = = = = Note1:
The sign of "Bored/Excited" being "<" means "Bored" has lower YBot or maxBot than "Excited". Note1: The sign of "Bored/Excited" being "<" means "Bored" has lower maxBot than "Excited". Note2: rbc is the rank-biserial correlation, close to 1.0 it means that the effect of the condition is important. Note3: To get the size of the mood subsets, please refer to Table 5.5.

the grasp is less deep into the hand of the robot (see Figure 5.5 for the geometric description).

To conclude this sub-section, we summarize some finds. We analyzed the global effects of the mood in our datasets using Kruskal-Wallis tests. We found several features that are altered by the mood. Thus, we conducted pairwise analysis, for the maximum pressure exerted in the bottom part of the hand. Several encouraging results were found: (1) Upon 18 comparisons of distant moods, 8 showed significant differences. (2) Differences were observed either in interpersonal handshakes or human-robot handshakes.

(3) An effect of the mood arousal was found such as higher aroused moods induce higher pressure in handshakes.

We investigated if the fact to exert pressure using the fingertips, or the first phalanx gives a relevant information about the mood. The results were mitigated.

Finally, we found that the deepness of the grasp (i.e., if the thumb of the participant goes far in direction of the robot's arm) can be different when the participant is anxious.

Inter-individual variability and angent consistency

In this sub-section, we are interested in how the handshake data is "subject-specific".

In Chapter 3 we found that the handshake manner depends on gender and extroversion degree. In Chapter 4, we observed a between-participant variability, and the sensors activated by the participants measured more variations than the experimenter only. In Section 5.4.1, we found that some participants induce Gaussian distributions on some of the 6 components, while others have different types of distribution. As a consequence, it is important to investigate to which extent this differentiation occurs, and if the behavior of a participant is consistent facing the human agent, or the robot.

The results in Section 5.4. Note4: nb sign is the number of mean differences of the pairs with the same sign for both agents, given that the difference is significant for both agents. The percentage is the ratio between nb sign and nb a .

an average of 0.5. These observations appear either in the handshakes face to the human or the robot. The results reveal that XTop contains high numbers of behavior differentiation due to the participant. This means that the position of the thumb above the agent's hand is efficient in discriminating participants, and it confirms the observations on the variability of this area by [START_REF] Knoop | Handshakiness: Benchmarking for human-robot hand interactions[END_REF]. The feature maxBot is also important.

Another question we are interested in answering is if these specificities on the behavior is found in both agents for a given participant. We count the number of pairs that show significant difference in both cases, and note it as nb a in Table 5.10.

Here again, high values are found. Between 70% and 80% of the significant differences with both agents are from the same pair of individuals. This is true for the 6 Despite this remark, even if a bias exists, the level of significance is very high.

To conclude on these results, there is a high inter-participant variability while handshaking. These differences appear despite distinct days of experiment, and distinct moods of the individuals. Besides, an important consistency was found, for the persons with a specific behavior, between the interaction with the human or the robot. These are very promising results that indicate a certain stability of an individual's handshake, and may suggest that interacting with a robot do not change our behavior.

Conclusion of the chapter

In this chapter, we studied how the mood alters the handshake manner, either while interacting face to a human or a robot. We firstly presented the social robot we used and some technical description of human-robot handshake protocol. We gave some theoretical background of what is the mood compared to emotions, and gave a way to measure it. In this chapter, we did not intend to control the mood of our participants, we only measured it. With a few number of participants, but for repeated days, we were able to collect 1408 handshake interactions.

We characterized the pressure data exchanged during handshake by 6 features, and described their distribution. We found that they do not necessarily follow normal laws, even when we take the data of each participant individually. We created a corpus that intend to remove the inter-individual variability. However, we still noticed that if the overall handshake manners became comparable, the behavior differences in the handshakes, that depend on the mood, may still be participantdependent. These new variables were still not all Gaussian, and the variance depending on the mood were not equal, for all the components. This prevented us from using parametric statistical methods.

After having described how the uncontrolled mood conditions are distributed, we observed a global effect of the mood on the pressure data for both agent conditions. The bottom of the grasp was mostly altered, but the position of the participant's thumb in the robot case showed also a significant difference. We found that several pairwise comparisons are significant, and mostly the ones between negative moods. It was showed that the arousal is linked to the pressure (i.e., higher arousal induces higher pressure), and that being anxious may change the position of the thumb (i.e., being "Tense" induces a less deep grasp towards the robot's hand).

Other results are interesting. We characterized the inter-individual variability of handshakes. We found that many participants have significant differences between each other. This is the case for all the variables but mostly for the position of the thumb (i.e., XTop) and the maximum pressure in the bottom of the grasp.

These behavior specificities are highly repeatable, day after day, mood after mood, and it is consistent between both human and robot agents. These results were part of a contribution in a conference in 2018: Orefice et al., 2018. Several remarks can be done. As it was repeatedly reminded, our data is not fully independent. Several handshakes were performed the same day, and all the participants are grouped in the same dataset. The participant-wise standardization may not have removed their affective behavior, which may be opposite from one to another. It would have been highly interesting to have performed the experiment in such a long period of time that we would have tens of non-consecutive samples per mood and per participant, letting us analyze the data individually. Another remark is that we could not compare the positive mood with the negative ones, due to heterogeneity of the distribution. Thus we cannot estimate the impact of pleasure on handshakes. Finally, given the spatial distribution of the sensors and their precision, we can only talk about tendency in our data. We cannot determine for instance the detectability of mood in handshake. The results tell us that the mood impacts the behavior during handshake, and that the arousal is an important dimension. Future work should investigate to which extent the moods are discriminable. The main goal of this thesis was to find out if cues about emotion can be detected during a touch-based human-robot social interaction. This was motivated by the fact that emotions and touch play an important role in social interactions. The number of applications for social robots is increasing, hence it is important to enhance their social abilities. This is done by designing more natural interactions, and by endowing robots with emotional touch abilities, such as emotion inference through touch.

After reviewing the literature, we found out that some challenges were still to tackle. (1) There is a lack of understanding of the handshake manner with quantified data (e.g., physical measurements, or surveys). Such measurements could improve the knowledge about: the cultural differences, to which extent the handshake manner is standardized and if a real social message is transmitted, and what type of message is conveyed.

(2) On the technological level, there is a lack of commercially available spacial pressure sensors to measure handshake manner.

(3) There is a lack of social touch datasets, or number of samples per experiment. (4) There is a lack of efficient emotion elicitation tools that have a powerful and durable response. (5)

Few works aim to infer social cues using a robot, while other studies make a robot express social cues and observe the reaction of the user. Among the ones that infer social cues, few infer the emotional state of the user. Moreover, few use touch to infer social cues. Even less in a social interaction context. To the best of our knowledge, studying emotional state inference through tactile social interaction, has not been covered.

The expected contributions to address these challenges were as follows: (1) Collect tactile data during human-human or human-robot social interaction (here restricted to handshake). (2) Design a "transparent" and embedded system to measure tactile data during handshake interactions. 6.1 Summary of the thesis

Chapter 2

In Chapter 2, we presented an extensive and interdisciplinary literature review about:

the handshake manner, the personality, the emotions, and the artificial skins. It provided useful information for the whole thesis as these notions are the core of our research. Based on this state of the art, we decided to focus only on one touched social interaction: the handshake manner. We also decided to widen our research question (that concerned initially only emotions) to study the expression of other internal states, that are more stable in time: the personality and the gender.

Chapter 3

Goal of the chapter:

The main goal of Chapter 3 was to investigate how the handshake manner is altered by psychological constants characterizing an individual. We chose to study two characteristics: the gender and and the extroversion degree.

Challenges:

Concerning the physical measurements of handshake interaction, few studies tackled the pressure data. The authors did not try to infer internal states of the individuals. The pressure spacial distribution also was not intensively investigated.

Solutions:

We decided to build our own glove, using FSR sensors (presented in Section 2.5) that are commercially available. In order to locate where to position the sensors on the glove, we performed an experiment, using as diversified participants as possible.

We ended-up with a glove able to measure pressure in 8 locations, and acceleration/ orientation.

Experiment:

We used this device, worn by the experimenter, to collect interpersonal and ecological 1 handshakes to be compared to the gender and extroversion degree of the participants. We demonstrated that we could detect the firmness of the experimenter, and the gender and extroversion degree of the participants with an above than chance rate. Finally, these results were compared with human-robot handshake interaction measurements.

Contributions:

Our main contributions in this chapter are the following: (1) The contact areas during handshake, that are consistent with other researchers results.

(2) Not only pressure data, but also acceleration and duration enabled to detect the experimenter's handshake firmness, with a success rate of 75% for three firmness conditions. (3) It is possible to recognize the gender of an individual using the physical data during handshaking, with a success rate of 77%. The most important features are speed, hand inclination, and participant's pressure. (4) The extroversion degree also has an impact on handshake, although with a lower magnitude (62% of success rate). This was obtained using hand inclination, speed, and pressure. We noticed that a part of the pressure differences was caused by the experimenter's action.

(5) We found some similarity when interacting with a human or with a social robot, nevertheless this last result has to be taken with caution. [START_REF] Orefice | Let's handshake and I'll know who you are: Gender and personality discrimination in human-human and human-robot handshaking interaction[END_REF] Ideas for improvement:

The results found in this chapter are promising, however some improvement could be proposed. (1) The instrumented glove we built for the experiment of this chapter had little number of sensors. Thus, we were not able to observe a relation between the contact positions in specified areas, and the individual. Only 3 positions were analyzed (i.e., participant's action, experimenter's thumb action, and experimenter's action). As observed in Chapter 5, a high resolution sensitive top of the hand may be provide more information.

(2) The gender was detected, but only tendencies were found to reject the possibility that it is due to anthropomorphism effects of gender differences (i.e., tallness, size of the hand). This should be investigated more deeply.

(3) An effect of extroversion degree was found however, the experimenter's action had an impact on these results. It is an interesting result that depending on the personality of the partner, one has a different behavior. Nevertheless, in order to isolate the behavior of the participant only, a robot could be helpful. (4) We were not able to compare rigorously the human-human handshakes with the human-robot ones. This should be investigated deeper in future studies.

Chapter 4

Goal of the chapter:

The main goal of Chapter 4 was to analyze the differences in tactile data depending on the emotion experienced by the user, in a spontaneous and ecological2 manner. This was motivated by the encouraging results on the detectability of long term characteristics of an individual (Chapter 3). Besides, it was shown in Chapter 2 that emotions have strong behavioral responses, and it can be communicated. Hence, we expected to find variations in the handshake manner. However, the study of emotion rose challenges, as it is a short term affective state. When designing this study, we decided to set two constraints: to use spontaneous emotions, and to have the source of emotion different from the interaction (so that the social content do not interfere with the emotion expression). We decided to use the following scenario for the experiment:

• Perform handshakes

• Generate emotions in the participant using a pre-evaluated tool

• Perform again handshakes

• Verify the emotion felt by the participant was correct

Challenges:

This procedure brought out two challenges: How to elicit spontaneous emotions? and How to measure emotions?. These two questions were dealt with in a literature review.

Concerning emotion elicitation, many studies used passive stimuli observed by the subject, and the response was measured in real time. In our case we needed the elicitation phase to be generated before the measurement, hence the emotional state had to last long enough.

Solutions:

We found out that using Virtual Reality is a way to involve the participant in his/her own elicitation, to make him/her forget about the lab context, and then elicit a stronger and more ecological emotion. In VR, several stimuli of different modalities were included to enhance the effect. To evaluate the experienced emotion, we used questionnaires. We created virtual environments that includes emotional stimuli selected from pre-evaluated databases. It was aimed to elicit 3 basic emotions and a neutral condition. The subjective evaluations of the tool encouraged us to use it in the main experiment.

Experiments:

The experiment needed the use of a virtual agent in order to perform handshakes without breaking the VR experience. We also performed handshakes without the VR helmet. The virtual agent was controlled by the experimenter. For this experiment, we also created new gloves with more home-made sensors (i.e., 23). Two gloves were used to observe the action-reaction effect. The handshakes were performed before and after elicitation and were compared. A high inter-participant variability was observed in the pressure exchanged during handshake. The variability of the participants was higher than the experimenter's, which indicated a possible effect of the conditions on the handshake manner. However, no effect of the emotion condition was observed. This can be due to many reasons that were discussed.

Contributions:

The contributions of this chapter are the following: (1) Cues on how to design an emotion elicitation tool using VR and stimuli from databases were presented.

(2)

We were able to create subsets of "most efficient stimuli" from these databases.

(3)

The elicitation tool was evaluated with the following results: the neutral, sadness, and happiness conditions were efficiently elicited, while the fear condition was more subject to variations. (4) VR did not alter the stimuli efficiency and multi-modality was important: the participants were not all receptive to the same modalities, there was a link between this receptivity and the overall VR evaluations. (5) The sensors designed for the instrumented gloves are low cost easy to make cells that can be duplicated to fully cover the contact areas. (6) There is inter-participant variability in the handshake manner. The magnitude and location of the pressure are different depending on the participant. (7) The pressure is lower while interacting with the virtual agent than face to the human. (8) No differences induced by the emotion condition was found, but this may be due to several reasons described in Section 4.5.5.

Issues and ideas for improvement:

Some of the results found in this chapter can contribute to the fields of emotion elicitation and physical interaction in VR, however the targeted emotion detection in handshake did not show the expected results. Some improvements could be applied to pursue the research. (1) We conducted the experiment with a dozen of participants, however each participant experienced only 3 emotional conditions, one time. Even though several handshakes were performed for each condition, the interindividuals variability suggests to use more repeated measurements for one participant. This rises the issue of multiple elicitation.

(2) The elicited emotions were not systematically well perceived. Besides, in some cases, the emotion had time to vanish before the interaction. Hence, some more work should be done in this direction.

(3) An important issue is that we only relied on subjective questionnaires to evaluate the elicitation tool. The participants were asked to be honest about their reported experienced emotional state, however a bias can appear as the emotional conditions were obvious to recognize. (4) One should verify the experienced emotion trough physiological measurements for instance. However, it is a challenging solution as the sensors are highly invasive, and the emotional state is not trivial to recognize through physiology.

(5) If one wants to continue in the field of spontaneous emotions and handshakes, an important work remains in the elicitation step. One may change the methodology and study an emotion elicited by the handshake partner, despite the interferences between social and emotional cues.

Chapter 5

Goal of the chapter:

The main goal of Chapter 5 was to detect the effect of spontaneous mood on the handshake manner. Similar to Chapter 4, this was motivated by the encouraging results on the detectability of long term characteristics of an individual (Chapter 3).

Besides, the mood appeared to be a promising candidate to be observed in handshakes. The mood is a longer term affective state than emotion, hence it can be measured at any moment during the experiment and still correspond to the one felt during the interaction. The mood is not linked to a specific object, so it should not interfere with the fact that the handshake partner is not the source of the mood.

Challenges and solutions:

To the best of our knowledge, no study tried to infer the mood of an individual through tactile interaction. However, as the handshake interaction grants access to oneself, we suggested the mood may be a socially relevant information to share when greeting. Like in Chapter 3, we used the strategy of studying both humanhuman and human-robot interactions. On another hand, an issue often noticed in Social Robotics, is that the results are collected during short term interactions (e.g., the participants interact only once the first time they meet the robot). In this experiment, we used longer term interactions, as the participants performed handshakes every mornings of 16 days. Besides, the mood condition was not controlled, it was the spontaneous mood of the participants.

Experiment:

This experiment enabled to collect 1408 handshake interactions, divided between a human agent and a robot agent, and involving 11 participants. They were described by 6 components representing spacial and magnitude aspects of the contact pressure.

We found a significant effect of the mood on these components, that was visible for both agents. We also found a high inter-participant variability.

Contributions:

The contributions of the chapter are the following: This suggests the idea that in the future, a robot could recognize an individual only using his/her handshake manner.

Ideas for improvement:

In order to extend these results, one should perform these measurements on a longer period of time. Hence, each participant would be studied individually, without the standardization phase. At the same time, one would be able to compensate the heterogeneity of the mood distribution.

these observations. Besides, several questions that were not tackled in this work deserve some attention.

Here are examples of the contributions that could, or should, be addressed in the future.

The greeting handshake manner should be deeply investigated. Indeed, this interaction is not only relevant in Social Robotics due to sociological and psychological results, it is the first interaction a social robot would do in a real life context. And

this is what politicians do when meeting a robot in public events. As it is often said, the 20 first seconds can tell who you are. So, a robot should be able to perform consistent handshakes and detect the social context like a human. Closer from our time line, quantitative experiments should give cues about: cultural aspect, standardization effects, and what type of message is exchanged (i.e., no message, social message, internal state message). Then one can try to infer these message meanings through handshake.

The mood was found to be expressed during handshake. One should investigate deeper this aspect, with more participants, in a longer period of time. But one should also find another type of interaction, which seems relevant for this like hugging.

In order to continue with the internal state, we need to considerably improve the tools required to study spontaneous emotions before investigating its impact on touch. First, the time aspect of emotions should be deeply and numerically investigated: When does it start? How long to show the stimuli? When does it vanish? Some work has to be done considering the object to which the emotion is related. Finally, the experimental scenario and elicitation method should be tested and rigorously approved before performing the experiment with tactile measurements.

My publications

During this thesis, we published our contributions in three conference proceedings. has a firm handshake (P), and only the experimenter has a firm handshake (E). Then he/she performs 3 normal handshakes, 2 E, 2 P, 2 N, 2 P, and 2 E. This distribution of conditions was chosen to remove order effect, and to prevent from having all the samples of a condition being consecutive.

In the data analysis, we were only interested in the maximum pressure reached by a sensor during the handshake interaction.

A.3 Data analysis

Before detailing the results in the rest of this section, we present some notations.

Each glove has 23 sensors, which are identified by a number starting from 0. The Figure A.1 gives the positions of the sensors and their identifier projected on a hand picture. Some sensors are more activated when the glove owner handshake firmer, we consider they represent the glove owner's action. We call them S, as they enable the glove owner to "Send" the information to his/ her partner. Other sensors are more activated when the other is firm, we call them R, as they enable the glove owner to "Receive" the information from the other. Finally, some are activated by both partners, we call them "?", as their role is undetermined.

A.3.1 Discrimination of the sensors role

In this subsection, we present how we determined the role of the sensors, which can be S, R, or "?". We first noticed that the maximum pressure distribution of each sensor is not Gaussian. Indeed, the values are always positive but a sensor can be not or bearly touched, which gives a high number of "close to zero" values. The sensors actually follow exponential distributions. This observation led us to use non-parametric statistical tests. We used Mann-Whitney U tests to compare each sensor depending on the E and P conditions. The null hypothesis of this test is that both subsets belong to the same distribution. The Table A.1 summarizes the results of the pairwise comparisons, with the p value of the tests, the rank-biserial correlation coefficients rbc1 , and the sign of the difference when the null hypothesis is rejected (i.e., when p value < 0.05). The two central columns correspond to the results for the two gloves, and the solution columns corresponds to the interpretation of the results.

The first analysis if the following: the consequences of the firmness conditions are observable directly on the sensors, as several comparisons show significant differences. The first solution column corresponds to the role attribution that is directly consistent with the two gloves. For instance, for the sensor "0" located on the thumb:

When the experimenter has a firm handshake (E), the pressure on the participant's Note1: The sign of "E/P" being "<" means that when the experimenter only has a firm handshake, the maximum pressure reached by the sensor is lower than when the participant is firm. Note2: A rbc close to 1.0 means that the effect of the condition is important. Close to 0, there is no effect.

142Appendix A. Discrimination of the action of the participant or the experimenter thumb is lower than when the participant has a firm handshake (P). The inverse result is found considering the experimenter's thumb. This shows that "0" is a S sensor.

The only contradiction is for the sensor "20" for which the relation is not inverted.

In some cases, there is no significant differences for a sensor in a glove depending on the condition. However, when observing the glove of the partner, a difference is observed. That is how we filled the second solution column. Finally, in a few cases, the sensor response is undifferentiated in both gloves. We marked them as undetermined in the third solution column. But for "18" and "20", we used geometric considerations to attribute them the role of their neighbors.

In the last solution column, we summarize the role attribution. 9 sensors are S, 10 are R, and 4 are undetermined, meaning they can be activated by both partners.

These sensors are drawn in In the end, among the 46 sensors, we created two groups: the ones activated by the participant (Gp), and the ones activated by the experimenter (Ge). For instance, Gp is composed of 9 sensors in the participant's glove (i.e., {0,1,2,3,4,5,6,7,8}), and 10 sensors in the experimenter's glove (i.e., {10,12,13,14,16,17,18,19,20,21}).

A.3.2 Loop effect in the action

In this subsection, we characterize to which magnitude the pressure changes between the normal and firm conditions. Then we highlight a loop effect, meaning that the action of one partner can have repercussions on the other. We quantify the magnitude of this repercussion.

First, we characterize the normal handshakes (N) for the two groups of sensors created in the previous subsection. We give respectively the mean, median, and maximum values for all the Gp sensors and all the normal handshake interactions: µ = 9.7kPa, Me = 5.4kPa, max = 59.4kPa. There is a significant difference of the maximum value of the pressure depending on the group (i.e., Gp or Ge): the Mann-Whitney U test result is p value = 0.002 and rbc = 0.1. The Experimenter's firmness is lower comparing to the Participant's: for Ge in normal handshakes, µ = 8.8kPa, Me = 4.6kPa, max = 52.0kPa.

Comparing N with E or P, most of the sensors show a significant difference.

26/46 correspond to higher pressure in the case of E, and only 2 show less pressure.

These two sensors are the previously cited "20" in the experimenter's glove, and the thumb of the participant "0". In the case of P, 37/46 sensors show higher pressure.

Thus, the firmness condition is well visible in the sensors data, compared to normal handshakes, even in the sensors of the partner that was not supposed to be firm.

We give orders of magnitude of the pressure increase for the sensors activated by the one supposed to handshake firmer. The maximum pressure reached do not substantially increase: +58% for Gp in the P case, and +42% for Ge in the E case. However, the median value is multiplied: +335% for Gp in the P case, and +116%

for Ge in the E case. This means that the maximum strength increase slightly, but more sensors reach this level in each handshake. One could notice that the amount of increase of the experimenter is smaller than the participants.

What is now important to evaluate, is if in the E case, Gp shows higher pressure that in N handshakes (and the same for Ge in the P case), and compare this increase with the previous cited values. The maximum pressure reached by Gp sensors when E compared to when N is significantly higher (the Mann-Whitney U test result is: p value = 036, rbc = 0.1, increase:+14%). The median value increases with the same range: +15%. The same way, there is an increase of Ge sensors when P compared to when N (p value < 1e -3 , rbc = 0.2, increase:+49%). The median value increases even more: +74%. These results reveal three thing: (1) The existence of a loop effect (i.e., when one partner has a firm handshake, the sensors activated by the other show significantly higher pressure compared to his/her normal behavior).

(2) The range of this increase has a lower magnitude compared to when the individual is directly asked to have a firm handshake (e.g., +14% compared to +58% for the action of the participant), which means that what is measured by the Gp group is mostly the behavior of the participant.

(3) This difference of magnitude is smaller concerning the experimenter's sensors, compared to the participant's sensors, which means that the experimenter's behavior does not bias much the handshake measurements.

A.4 Conclusion

In this appendix, we conducted an experiment to investigate how the sensors of our handshake measurement device, enable to quantify the action the partner wearing the glove. The first analysis showed that several sensors were directly activated by one of the partner, and a few are activated by both. We created two groups of sensors, distributed in both gloves, that represent the action of the participant, and 144Appendix A. Discrimination of the action of the participant or the experimenter the action of the experimenter. These groups are used in the different chapters of this thesis.

We also characterized the magnitude of the pressure applied by a few participants, during their "normal" handshakes, and highlighted how these values increase when they are asked to have a firm handshake.

Finally, we investigated if the action measured of one partner comes from his/her own intention, or if it is due to a loop effect that reproduces the action of his/her partner. We found that this loop effect exists, but the repercussion magnitude is smaller than the intention itself. Besides, it was found that the experimenter alters the data in a reasonable way, as the repercussion of his behavior on the participant's action is even smaller that what the participant can induce on the experimenter.

These results encourage us to use only the 19 Gp sensors to analyze the behavior of the participants during handshake. Note1: The numbers correspond to the sensot id, and the letter is for the glove owner. Note2: Considering Appendix A, the sensors p9 and p11 are activated by both partners, p10 is activated by the experimenter, and e10, e12, e13, and e14 belong to Gp.

each phalanx number. Then we compute the barycenter between these 3 phalanx with the following weights: -1, 0, and 1 for the finger tips.

Here we do not have enough sensors in the bottom of the experimenter's hand to measure the position of center of pressure in his hand. Nevertheless, it could be interesting to add these components to assess the position of the participants' hand on the experimenter.

Finally, we create maxBot as the maximum pressure reached by all the Bottom sensors, apart from e16, e17 and e18.

In the end, we dispose of 6 components, 4

B.3 Efficiency of the description

After having created the 6 geometrical an magnitude components based on the role of the sensors defined in Appendix A, and on arbitrary decisions, we have to evaluate the efficiency of this description.

One of the issues that motivated the creation of these components was the fact the responses of the sensors contain a lot of null values. In other words, when a sensor is not touched, it returns zero (see Figure B.4 (top)). This makes it impossible to analyze the measurements of each sensor individually, we need the information of the sensors located around. This is the reason why we created geometrical components.

Besides, in order to get the magnitude of the pressure applied in the corresponding area, we could not compute the mean of all the sensors. The null values would have biased the result. This is why we computed the maximum pressure. Given these concerns, it sounds important that our components do not contain these null values artifacts. aspects in the handshake manner. Despite these correlations were found for both agents, the level of correlation is low (i.e., -0.22 and -0.17) Another aspect this matrix provides is the link between Y components, and the magnitude of the pressure. We find medium correlation levels between YTop and maxTop, and between YBot and maxBot, for both agents. This indicates that when an individual handshakes firmer, it is mostly due to his/her finger tips. Hence, when one handshakes firmer, he/she fully enrolls the partner's hand and presses with the last phalanx. This is also a new result that is important to notice.

The correlation of the magnitude of pressure between the Top and Bottom groups was expected: if the pressure is applied on the top of something, it has to be also applied on the bottom, in a way or another. However, the level of correlation is not so important, which highlights the importance of keeping both variables in the analysis.

The last result is about the low number of significant correlations. If we put the level from which a correlation start to be visible at ρ > 0.2, upon 22 out of 30 coefficients can be rejected. This reveals that little redundancy is included in our components.

B.4 Conclusion

This appendix aimed to create components that describe the pressure exchanged during handshake interactions, with several objectives. We wanted to remove the features that contain a large part of empty information (i.e., when a sensor is not

( 1 )

 1 Social Sciences to study social interactions; (2) Psychology to study emotions or others internal states; and (3) Robotics from the hardware point of view (e.g., creation of tactile sensors, robot control) and the artificial intelligence point of view (e.g., learning, emotion model on the tactile data, behavior generation). Additionally, three components have to be taken into account transversally across these domains: (4) Touch, (5) Humans, and (6) Naturalness. We present bellow what aspects are important for our work.

1

 1 Emotional theories . . . . . . . . . . . . . . . . . . . . . . . . Ethology and Naturalism . . . . . . . . . . . . . . . . . . . . Neuro-physiology . . . . . . . . . . . . . . . . . . . . . . . . Cognitive Sciences . . . . . . . . . . . . . . . . . . . . . . . . Social Sciences . . . . . . . . . . . . . . . . . . . . . . . . . . Emotions are an independent structure . . . . . . .

  Figure 2.1 presents some of them. There is the classic style (a)), the political one with the left hand upon the other (b)), the 60's afro-american style (c)), and the Nigerian handshake with the finger snap (d)).

FIGURE 2 . 1 :

 21 FIGURE 2.1: Exemples of various handshake styles

FIGURE 2

 2 FIGURE 2.3: 3D optical piezo-sensitive cell (extracted from[START_REF] Kadowaki | Development of soft sensor exterior embedded with multi-axis deformable tactile sensor system[END_REF]

FIGURE 2

 2 FIGURE 2.5: Schematic of a piezo-sensitive cell based on FET transistor (extracted from Adami et al., 2012)

FIGURE 2 . 6 :

 26 FIGURE 2.6: a) Schematic of a piezo-capacitive cell (extracted from Ulmen and Cutkosky, 2010); b) Picture of a module with 12 piezocapacitive cells (extracted from Cannata et al., 2008); c) Schematic of a 3D piezo-capacitive cell, composed with 3 sub-cells (extracted from Nazeer, 2012)

  is reduced (phenomenon (1), leading to the inverse effect). However we performed our own tests, and found that (1) is negligible compared to (2). These sensors are used in many human-machine applications. Vidal-Verdú et al., 2011 created a robot skin made of these sensors and added a polyurethane cone to convert the force into a pressure distributed on the sensitive area (see Figure 2.7).

FIGURE 2

 2 FIGURE 2.7: Piezo-resistive cells assembled in a skin (extracted from Vidal-Verdú et al., 2011)

FIGURE 2

 2 FIGURE 2.8: Piezo-resistive cells made of fabric (extracted from Bhattacharjee et al., 2013). a) Schematic of a cell; b)-c) Picture of the electrodes layer; d) The electrodes are covered by the resistive and conductive layers.

FIGURE 2

 2 FIGURE 2.9: Examples of Electrical Impedance Tomography (EIT) sensors (extracted from[START_REF] Alirezaei | A highly stretchable tactile distribution sensor for smooth surfaced humanoids[END_REF].

FIGURE 2 .

 2 FIGURE 2.10: Modular multi-modal sensor (extracted from Mittendorfer and Cheng, 2012). a) Picture of a the full module; b) Schematic of a Piezo-capacitive cell.

  Finally, individual piezo-resistive sensors also called Force Sensitive Resistances (FSR) (Interlink Electronics, 2018) are widely available. They were used by Melnyk et al., 2014 and Tagne, Hénaff, and Gregori, 2016 who designed two gloves, with 6 sensors, that measure inter-individual handshake data. Garg, Mukherjee, and Rajaram, 2017 also designed a glove to measure handshakes. They used flex sensors, that detect to which level the hand is closed. Some pictures of these gloves are presented in Figure 2.11.

FIGURE 2 .

 2 FIGURE 2.11: Exemples of gloves that use pressure sensors. a) Extracted form Wang, Hoelldampf, and Buss, 2007, and b) the modified version for handshake measures; c) Extracted from Garg, Mukherjee, and Rajaram, 2017; d) Extracted from Melnyk et al., 2014

1

 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Does personality impact handshake? A review . . . . . . . . . . . 3.2.1 Personality, gender, and handshake . . . . . . . . . . . . . . 3.2.2 Quantitative measurements of handshake . . . . . . . . . . . 3.2.3 Contribution of this chapter . . . . . . . . . . . . . . . . . . . 3.3 Geometric aspect of handshake (Experiment 1) . . . . . . . . . . . 3.3.1 Protocol and data extraction . . . . . . . . . . . . . . . . . . . 3.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.3 Conclusion of the experiment . . . . . . . . . . . . . . . . . . Design of an instrumented glove . . . . . . . . . . . . . . . . 3.4.2 Feature extraction of a handshake . . . . . . . . . . . . . . . 3.5 Pressure and movement in handshake: impact of personality and gender (Experiment 2) . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Classification and results . . . . . . . . . . . . . . . . . . . . Order of magnitude of the data . . . . . . . . . . . . . . . . . Feature reduction . . . . . . . . . . . . . . . . . . . . . . . . . Impact of the conditions on handshake . . . . . . . . . . . . Recognition of the experimental conditions . . . . . . . . . . Impact of firmness on the detection rates . . . . . . . . . . . 3.5.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.4 Conclusion of the experiment . . . . . . . . . . . . . . . . . . 3.6 Can personality be detected during human-robot handshake? (Experiment 2 bis) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.1 Experimental setup and protocol . . . . . . . . . . . . . . . . 3.6.2 Experimental results and comparison . . . . . . . . . . . . . 3.7 Conclusion of the chapter . . . . . . . . . . . . . . . . . . . . . . . . 69

  Yamato, Jindai, and Watanabe, 2008 used a motion-capture room to characterize the movement of interpersonal handshakes.[START_REF] Sato | Synchronization based control using online design of dynamics and its application to human-robot interaction[END_REF] generated command models for a robotic arm and focused on synchrony with the user, to dynamically control handshake interactions.[START_REF] Zeng | Human-robot handshaking: A hybrid deliberate/reactive model[END_REF] had a more human-based approach, as they mimicked the human kinesthesis to command the robot, and associated an adaptive model to fit in real time with the move of the individual.[START_REF] Avraham | Toward perceiving robots as humans: Three handshake models face the turing-like handshake test[END_REF] succeeded a Turing test while controlling the movement of a robotic handshake. Some work is also made on modeling the information exchanged during handshake, using tactile sensors.Wang, Hoelldampf, and Buss, 2007 used piezo-resistive sensors to analyze 600 interpersonal handshakes. The authors showed that the most used regions in the hand are the middle and ring fingers. However, their glove also measured acceleration and movement data and the authors analyzed more these features than the pressure data.Kim et al., 2013 used a

  Figure 3.1. a) The background is subtracted and the contour of the hand is computed. b) The hand is meshed following a specific procedure. A manual selection of 33 joint points is performed (see orange points in Figure 3.1b)), and subdivisions are automatically generated. Then, the mesh links all the points as displayed in the figure. Finally, the color is thresholded to detect all the blue pixels. c) Each cell of the mesh is then weighted by the ratio of touched pixels by the size of the cell. d) Every cell value is then transfered to a standard meshed hand.

FIGURE 3 . 1 :

 31 FIGURE 3.1: Exemple of contact distribution on the receiver's glove, and extraction of the data

FIGURE 3

 3 FIGURE 3.2: Average of the 8 large-handed gloves (top) and 8 smallhanded gloves (bottom)

  FIGURE 3.3: Distribution of contact over several trials of the handshaking interactions from an experiment carried out by Knoop et al., 2017. Blue/dark indicates contact in this area in none of the trials, and yellow/light indicates contact in this area in all of the trials.

FIGURE 3

 3 FIGURE 3.4: Average of areas touched by the inked hand during handshake (blue) drawn on a standard right hand. The selected points are rounded and associated to sensors for the glove design. They are divided between group s (sent) in red and group r (received) in yellow

  ure 3.7, and the final system is shown in Figure3.6. The different parts of the glove are presented below.

  FIGURE 3.5: The glove components and definition of the coordinates

  FIGURE 3.7: Electronics diagram

FIGURE 3

 3 FIGURE 3.8: Examples of handshake data, for 4 handshakes with different participants

FIGURE 3

 3 FIGURE 3.9: Correlation circle of the features

Figure 3 .

 3 Figure 3.10). The component LD1 represents 97% of the projected data variability,and is mostly directed by the pressure features. However, as seen in Table3.5, the duration and acceleration also have a weight in the discrimination. They are involved in LD1 but describe the majority of LD2 (representing the remaining 3% of variability). The data distribution projected on the first component is displayed in Figure3.11a). We compute the rate score of true detections, it reaches 75.0%.

FIGURE 3 .

 3 FIGURE 3.10: LDA space for firmness discrimination. The small dots represent the true detections, the large ones represent the wrong detections. The dot's color indicates the real category of the sample. The colored areas are the calculated class attribution from the LDA model.

  FIGURE 3.11: a) Distribution of the projected data on LD1 depending on the firmness. b) Distribution of the detection scores for the learning sets and evaluation sets.

Figure 3 .

 3 12a) gives the two distributions of male and female handshakes projected on LD1. The overfit evaluation gave the following results: the learning tests scores are (µ=78.2%, σ=4.3%) and for the evaluation the scores are (µ=72.4%, σ=4.4%) (see Figure3.12b)). In that case, the random choice is 50%, so the fact we get at least 72.4% shows the discriminability of the gender, which is in favor of our second hypothesis (H2).

FIGURE 3 .

 3 FIGURE 3.12: a) Distribution of the projected data on LD1 depending on the gender. b) Distribution of the detection scores for the learning sets and evaluation sets.

Figure 3 .

 3 13a) gives the two distributions of extrovert and introvert handshakes projected on LD1. The overfit evaluation gives the following results: the learning tests scores are (µ=73.2%, σ=7.2%) and for the evaluation the scores are (µ=62.2%, σ=4.7%) (see Figure3.13b)). The random choice is also at 50%, and the results are slightly above. It nevertheless indicates the discriminability of extroversion through handshake manner. Moreover, there is a tendency in favor of the third hypothesis (H3), as GP pMax is lower for introvert persons, even though the corresponding F value is lower than for GP eMax .

FIGURE 3 .

 3 FIGURE 3.13: a) Distribution of the projected data on LD1 depending on the degree of extroversion. b) Distribution of the detection scores for the learning sets and evaluation sets.

( 1 )

 1 Social robots are very recent technologies, and few models are industrially produced. Thus, they may substantially evolve in the next years. Depending on their cognitive abilities, and how the room the users will grant them in the society, the social representation of the robots by the users may change. Also, the physical shape of the robot may evolve, which may change the physical data that can be measured in the hand, for instance. Also, the mechanical abilities may improve, leading to more controllable handshake dynamics. So, creating a baseline only on the actual robots, and moreover on one specific model of the actual robots, might not be a long term contribution. (2) The current tendency of social robots is to look like humans (see the Sophia robot (Hanson Robotics, 2018)). So, creating models of how humans behave between themselves may hypothetically be directly usable in future human-robot interactions. (3)

FIGURE 3 .

 3 FIGURE 3.14: Human-Robot handshake using Meka humanoid robot

1

 1 measurements . . . . . . . . . . . . . . . . . . . . . . . . . 72 4.3 Elicitation methods: a review . . . . . . . . . . . . . . . . . . . . . . 74 4.3.1 Usual elicitation methods . . . . . . . . . . . . . . . . . . . . 74 Internal process . . . . . . . . . . . . . . . . . . . . . . . . . . 75 External stimuli: Images . . . . . . . . . . . . . . . . . . . . . 75 External stimuli: Music . . . . . . . . . . . . . . . . . . . . . 76 External stimuli: Videos . . . . . . . . . . . . . . . . . . . . . 77 Interactive task . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Time aspect . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 4.3.2 Elicitation using virtual reality . . . . . . . . . . . . . . . . . 80 4.3.3 Conclusion on elicitation methods . . . . . . . . . . . . . . . 81 4.4 Design of an emotion elicitation device using VR . . . . . . . . . 81 4.4.1 Choice of the emotions to elicit . . . . . . . . . . . . . . . . . 81 4.4.2 Emotional content . . . . . . . . . . . . . . . . . . . . . . . . 82 4.4.3 Evaluation of the system (Experiment 3) . . . . . . . . . . . 85 Presentation of the experiment . . . . . . . . . . . . . . . . . 85 Experimental protocol . . . . . . . . . . . . . . . . . . . . . . 87 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 4.4.4 Discussion about the experiment . . . . . . . . . . . . . . . . 95 4.4.5 Device improvement . . . . . . . . . . . . . . . . . . . . . . Glove improvement: design of new sensors . . . . . . . . . . 99 4.5.2 Overall experimental setup . . . . . . . . . . . . . . . . . . . 101 4.5.3 Experimental protocol . . . . . . . . . . . . . . . . . . . . . . 103 4.5.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Description of the data variability . . . . . . . . . . . . . . . 104 Elicitation efficiency . . . . . . . . . . . . . . . . . . . . . . . 105 Impact of emotions . . . . . . . . . . . . . . . . . . . . . . . . 106

  how costly it is. Moreover, the subject should be totally naive to what the experimenter is investigating. Our research is still at the stage of exploration, we do not have a-priori the knowledge of what should be measured, and if there is truly something to measure. That is why we chose to use less advanced techniques.

FIGURE 4 . 1 :

 41 FIGURE 4.1: PAD values of the selected emotions from[START_REF] Russell | Evidence for a three-factor theory of emotions[END_REF] We added a neutral/calm emotion.

FIGURE 4

 4 FIGURE 4.2: Screen-shots of virtual environments for mood elicitation. a) for positive mood, b) for negative mood.

FIGURE 4 . 3 :

 43 FIGURE 4.3: Map and screen-shots of virtual environments for emotion elicitation.

( 1 )

 1 Evaluate the overall outcome for each elicitation scenario. (2) Evaluate and select the most efficient stimuli from the presented ones. (3) Compare and validate the importance of each modality.

FIGURE

  FIGURE 4.4: Experimental setup

Figure 4 .

 4 Figure 4.4 presents a participant immersed in a VR scenario. The VR helmet was a VR Box 2 with a Samsung Galaxy S7 smartphone. The elicitation was made using Unity and the rendered image was streamed to the smartphone using Riftcat. All the VR calculations were made by the OpenVr library and the SteamVr plug-in. If

  Figure 4.1 and Figure 4.5) and measured PAD scores for the emotion elicitation of the global VR environments are presented in Table 4.4. When qualitatively observing the data, one can notice that the evaluations correspond to what was expected, apart from these exceptions: the fear condition induces more pleasure, the sadness condition shows more arousal, and both fear and sadness are more dominant. Happiness and neutral look to fit in all dimensions.

  Fear and sadness conditions show lower arousal, which is not beneficial in the fear case. Happiness however has higher arousal. Finally, only three images differ from the database in term of dominance. This means our image selection method was consistent with what was perceived by the participants.

FIGURE 4 . 6 :

 46 FIGURE 4.6: One Sample t-tests results for comparison between individual image evaluation and the IAPS database Note1: Only the significantly different images are presented. Note2: "E" means the elicitation condition (N: Neutral, H: Happiness, F: Fear, S: Sadness). Note3: "IAPS" indicates the identification number of the images in the database. Note4: "Mean diff" is the difference between what we measured and the database. When negative, our value is lower. Both scales are 9-point Likert scales. Note5: The colors represent how the sign of the mean difference is in favor to our elicitation (i.e., in green it is beneficial).

  Figure 4.7. When the mark is distanced from the mean plus or minus the standard deviation of the global emotional category distribution, if it is on the side away from the reference, then the score attributed is 1. Each picture is associated with the mean of these four scores (i.e., adjective, pleasure, arousal, and dominance distances) on the 11 evaluations (as we have 11 participants). Half of the pictures have a total score above 22% (apart from two fear images), and this half is equally distributed depending on the elicitation condition. As a consequence, we have 5 images per emotion condition that are qualified as "good" pictures. They are used in the next study.

FIGURE 4

 4 FIGURE 4.7: PAD distance score attribution method for images

  The final scenarios, to elicit neutral emotion (N), happiness (H), sadness (S), and fear (F), are presented below, with screen-shots of the VR content in Figure4.8.

FIGURE 4

 4 FIGURE 4.8: Screen-shots of VR content: (a) Nice corridor; (b) Dirty corridor; (c) Picture room (here in F condition); (d) Mini-game (H condition); (e) Theater room (video of S condition); (f) Zombies animation (F condition)

FIGURE 4 . 9 :

 49 FIGURE 4.9: Design of the piezo-resistive sensors. a) Operating principle, b) picture , and c) calibration laws with corresponding model for one sensor.

  ure 4.10. The positions were determined from the previous version of the glove (see

FIGURE 4 .

 4 FIGURE 4.10: Glove schematic and sensor positions (here the participant's glove). The sensor groups are described in Chapter 3 Section 3.3: group s on participant's glove in orange and group r in green dot-line.

  tion 4.4.3), we use a Windows 10 computer that runs Unity and streams the VR content to a smartphone placed in a VR helmet, through Wifi. What is seen by the participant is simultaneously displayed on a screen and recorded. The audio content is played in headphones. The participant controls his/her character using a Bluetooth joystick. The experimenter controls the VR scenes, emotional conditions, and all the acquisitions using a mouse close to him.

FIGURE 4 .

 4 FIGURE 4.11: Global experimental setup

  three variables: maxTop, maxMid, and maxBot. They correspond to the maximum pressure respectively in the top of the grasp (i.e., thumb of the participant and the top of the experimenter's hand), the middle of the grasp (i.e., the palm of the experimenter's hand), and the bottom of the grasp (i.e., the 4 fingers of the participant and the bottom of the experimenter's hand). A schematic is presented in Figure4.13.

FIGURE 4 . 13 :

 413 FIGURE 4.13: Location of the sensors used to compute the three variables maxTop, maxMid and maxBot

1

 1 setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 5.2.1 Handshaking with a social robot . . . . . . . . . . . . . . . . 112 5.2.2 Measuring the mood . . . . . . . . . . . . . . . . . . . . . . . 113 5.3 Experimental protocol . . . . . . . . . . . . . . . . . . . . . . . . Pressure data distribution . . . . . . . . . . . . . . . . . . . . 117 5.4.2 Mood distribution . . . . . . . . . . . . . . . . . . . . . . . . 122 5.4.3 Impact of the mood on pressure . . . . . . . . . . . . . . . . 122 5.4.4 Inter-individual variability and angent consistency . . . . . 126 5.5 Conclusion of the chapter . . . . . . . . . . . . . . . . . . . . . . . . 128

FIGURE 5 . 1 :

 51 FIGURE 5.1: Sensor positions on the Pepper's glove. Note: The S and R annotations correspond to the role attributions given in Appendix A for the humans gloves.

FIGURE 5

 5 FIGURE 5.2: Picture of a handshake with Pepper

FIGURE 5

 5 FIGURE 5.3: Characters of the PAM to represent the moods in the circumplexe space (adapted from[START_REF] Desmet | Mood measurement with Pick-A-Mood: review of current methods and design of a pictorial self-report scale[END_REF]

FIGURE 5

 5 FIGURE 5.4: Picture of a handshake between a participant and the experimenter

FIGURE 5 . 5 :

 55 FIGURE 5.5: Description of the 4 position components.In red the sensors used to compute the "Top", and in blue the ones for the "Bottom". The maximum values of the pressure are also computed for the "Top", and "Bottom" groups. The robot's glove has 4 sensors in the top instead of three.

FIGURE 5 . 6 :

 56 FIGURE 5.6: Examples of non normal distributions for a given component, participant, and agent. The solid-line plots correspond to the computation of a normal law considering the mean and standard deviation of the data.

  FIGURE 5.7: Distribution on the 6 standardized components, for both agents datasets. In green the histograms when the distribution is Gaussian, in red-dashed when it is not normal. The associated Shapiro p values are specified.

( 3 ) 6 )

 36 Compare tactile measurements between human-human and human-robot interactions to link Social Robotics to psychological results. (4) Perform social context inference (here we focused only on personality and gender) using the tactile data. (5) Design an emotion elicitation tool adapted for handshake interaction studies. (Analyze the differences in tactile data depending on the emotion experienced by the user, in a spontaneous and ecological manner.

( 1 ) 6 )

 16 The mood affects the handshake manner. (2) The negative moods are easier to discriminate. (3) High arousal induces higher pressure. (4) There is a consistency of these results between interacting with the human or the robot agent. (5) Being "Tense" induces a less deep grasp towards the robot's hand. (Individuals have handshake manner very distinct from each others. (7) The specificities of the individuals are repeatable between the agents.

  [START_REF] Orefice | Let's handshake and I'll know who you are: Gender and personality discrimination in human-human and human-robot handshaking interaction[END_REF] which corresponds to Chapter 3. Orefice et al., 2017 which corresponds to Section 4.4. Orefice et al., 2018 which corresponds to Chapter 5.

  Figure A.1.

FIGURE A. 1 :

 1 FIGURE A.1: Sensors location, and final role attribution

FIGURE B. 2 :

 2 FIGURE B.2: Separation of the Gp sensors into two groups: a) the Top, and b) the Bottom.

  Touch properties . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.2 Artificial skins: requirements . . . . . . . . . . . . . . . . . . 2.5.3 A technological review . . . . . . . . . . . . . . . . . . . . . . Transducers examples . . . . . . . . . . . . . . . . . . . . . . .6.2 Personality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 2.6.3 Emotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 2.6.4 Artificial skins . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

	2.5.1

. . . . . 2.4.2 Sources and goals of emotions . . . . . . . . . . . . . . . . . 2.4.3 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.4 Conclusion on emotional theories . . . . . . . . . . . . . . . Emotion applications in Social Robotics . . . . . . . . . . . . Emotional studies challenges . . . . . . . . . . . . . . . . . . The link with our research . . . . . . . . . . . . . . . . . . . . 2.5 Spatial pressure measurement: a review . . . . . . . . . . . . . . . Types of architecture . . . . . . . . . . . . . . . . . . . . . . . 2.5.4 Conclusion on the spatial pressure sensors . . . . . . . . . . 2.6 How to go beyond the state of the art? . . . . . . . . . . . . . . . . 2.6.1 Handshake . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2

TABLE 2 .

 2

1: Classification of the appraisal process in literature

TABLE 2

 2

		.2: Examples of action and goal of action for each basic emo-
		tion (adapted from Frijda, 1987)
	Emotion Action	Goal
	Fear	Avoidance	Protection
	Anger	Attack / Threat	Recover control
	Disgust	Rejection	Protection
	Joy	Be with	Access to consumption
	Surprise Interrupt	Reorientation
	Sadness Introspection / Acceptance Reconstruction
	previously, this kind of language could have been culturally constructed as a new
	communication channel

TABLE 3

 3

		.1: Characteristics of the participants
		Gender Extroversion Hand size	Hand length (mm)
		Female	Introvert	Small	161
		Female	Introvert	Small	170
		Female	Extrovert	Large	173
	Senders	Female Male	Extrovert Extrovert	Large Small	180 171
		Male	Introvert	Small	172
		Male	Introvert	Large	187
		Male	Extrovert	Large	197
	Receivers	Female Male	Introvert Extrovert	Small Large	163 189

TABLE 3

 3

	.2: List of the different parameters studied during handshake
	Duration	TP
	Number of oscillations	OscNb
	Acceleration offset	A o f f
	Maximum acceleration amplitude	A amp
	Maximum speed amplitude	S amp
	Maximum speed	S max
	Supination angle	δθX
	Adduction angle	δθY
	Movement direction (non frontal)	αY
	Movement direction (non sagital)	αZ
	Maximum pressure on sensor i	P iMax
	Mean pressure on sensor i	P i
	Maximum pressure derivate for sensor i	dP iMax
	Global maximum pressure	P MaxMax
	Global mean pressure	P
	Max or mean pressure for group j	PG jMax ; PG

j such as his/her gender, degree of extroversion (i.e, internal characteristics), or hand size (i.e, external characteristics).

TABLE 3

 3

		.3: Characteristics of the participants
	Extroversion	Extrovert (20) Introvert (16)
	Size of hand	Small Large Small Large
	Gender	F (11) M (25)	6 3	0 11	5 4	0 7

TABLE 3

 3

	.4: Order of magnitude of the variables, during "normal"
		handshakes
	Variable Unit	Mean Std Min Max
	TP	s	0.80 0.17 0.43	1.37
	OscNb		2.5	0.9	1	6
	A o f f A amp	m/s -2 m/s -2	23.5 13.0 34.0 15.4	1.9 8.9	54.6 73.0
	S amp	cm/s -1	19.2	9.3	5.9	56.0
	S max	cm/s -1	10.5	9.8	0.0	56.0
	δθX	deg	24.8 11.1	8.6	61.8
	δθY	deg	13.5	4.6	3.8	28.4
	αY	deg	63.1	8.3 39.2	83.5
	P	kPa	10.7	6.7	0.6	33.7
	P MaxMax kPa	89.0 45.2 17.2 278.2
	PG pMax kPa	32.4 19.5	0.0 102.8
	PG eMax	kPa	21.4 18.7	0.2	76.4

TABLE 3 .

 3 5: Effect of the experimenter's firmness on the extracted features: ANOVAs, mean increase compared to the reference group, and confidence interval from the Tukey tests

	Variable	F value (2, 213)	Reference group	Compared group	Mean increase	Confidence interval
	TP	7.1	Normal Firm	Soft Soft	+10.8% +14.2%	[+1.7%, +19.8%] [+4.9%, +23.5%]
	A amp	4.4	Firm	Soft	-22.6%	[-40.6%, -4.7%]
	αY	6.8	Firm	Soft	+10.7%	[+3.8%, +17.5%]
	GP pMax	34	Normal Firm	Firm Soft	+60.7% -48.6%	[+37.3%, +84.2%] [-63.2%, -34.0%]
			Normal	Firm	+100.7% [+74.7%, +126.8%]
	GP eMax	107	Normal	Soft	-58.8%	[-84.9%, -32.8%]
			Firm	Soft	-79.5%	[-92.5%, -66.5%]
	P 0Max	66	Normal Firm	Firm Soft	+280.9% [+204.7%, +357.1%] -91.8% [-111.8%, -71.8%]

due to a psychological effect of the gender leading females to have a softer behavior? Is it because of their physical strength? Or is it because of the experimenter's action that have consequences on his partner? This experiment cannot answer these questions, as our goal is to see if there are differences in the handshake manner as a global interaction act.

TABLE 3 . 6 :

 36 Effect of the participant's gender on the extracted features

	Variable	F value (1, 130)	Reference group	Compared group	Mean increase	Confidence interval
	TP	8.1	Female	Male	-13.2% [-22.4%, -4.0%]
	S amp	6.8	Female	Male	-19.5% [-34.3%, -4.8%]
	A amp	4.3	Female	Male	+24.3% [+1.3%, +47.3%]
	δθY	16.2	Female	Male	-21.1% [-31.5%, -10.7%]
	GP pMax	5.3	Female	Male	+31.0% [+4.4%, +57.6%]
	GP eMax	4.5	Female	Male	+27.8% [+1.9%, +53.8%]

TABLE 3

 3

		.7: Effect of the participant's degree of extroversion on the
			extracted features
	Variable	F value (1, 130)	Reference group	Compared group	Mean increase	Confidence interval
	S amp	5.3	Extrovert Introvert	+20.2% [+2.9%, +37.5%]
	δθY	6.6	Extrovert Introvert	+16.2% [+3.7%, +28.7%]
	αY	6.0	Extrovert Introvert	+6.7%	[+1.3%, +12.2%]
	GP pMax	4.0	Extrovert Introvert	-18.6% [-37.0%, -0.2%]
	GP eMax	10.8	Extrovert Introvert	-28.0% [-44.9%, -11.2%]

TABLE 3

 3

		.8: Summary of the discrimination scores depending on the
			tested category and conditions
		Soft	Normal Firm	Learning set	Evaluation set
	Gender	79.2%	76.5%	73.6% µ = 78.2% ; σ = 4.3 µ = 72.4% ; σ = 4.4
	Extroversion 65.3%	68.2%	68.1% µ = 73.9% ; σ = 7.2 µ = 62.2% ; σ = 4.7
	Firmness		75.0%		µ = 73.9% ; σ = 5.6 µ = 67.6% ; σ = 3.6

TABLE 4 .

 4

		1: Exemples of studies that measured emotions using vari-
			ous modalities
	Modality	Measure	Feature	Reference
		Heart rate	Frequency, frequency variation	Rainville et al., 2006; Kim, 2007; Valenza et al., 2014; Ménard et al., 2015
	Physiology Respiration rate	Frequency, frequency variation	Rainville et al., 2006; Kim, 2007
		Skin conductance	Picks, duration of the picks	Nakasone, Prendinger, and Ishizuka, 2005; Kim, 2007; Ménard et al., 2015
		Facial expression Action units	Pantic and Bartlett, 2007 (review); Valstar et al., 2011
	Behavior	Posture	Body movements, body posture	McColl and Nejat, 2012; Dael, Mortillaro, and Scherer, 2012; Karg et al., 2013
				Grimm and Kroschel, 2007;
		Voice	Prosody	Kim, 2007; Koolagudi, Kumar, and Rao, 2011; Ramakrishnan,
				2012 (review)
		Vocabulary	Key-word	Kao et al., 2009 (review)
	Cognitive	Subjective representation	Discrete questionnaire Continuous	Schaefer et al., 2010 Russell and Mehrabian, 1977;
			questionnaire	Bradley and Lang, 1994

used this database (Schaefer et al., 2010) for our research.

  [START_REF] Carvalho | The emotional movie database (EMDB): A self-report and psychophysiological study[END_REF] evaluated 52 movie clips of 40 seconds, without sound, in terms of PAD. An attention was granted to homogeneity of durations.[START_REF] Schaefer | Assessing the effectiveness of a large database of emotioneliciting films: A new tool for emotion researchers[END_REF] created a corpus of 70 film extracts lasting between 1 and 7 minutes. They were selected to elicit anger, sadness, fear, disgust, amusement, tenderness, and neutral state. The evaluation was performed using discrete adjectives, and the Positive and Negative Affect scale. We

TABLE 4

 4

			.2: Source of the selected music and video stimuli
	Emotion	Modality Title / Track	Start	Duration
	Neutral	Music	The Godfather III, track 5 The Portrait of a Lady, track 3 00:20 01:13	01:06 01:00
		Video	The Lover	1:10:30 00:48
	Happiness	Music	The Untouchables, track 6 Pride & Prejudice, track 4	01:26 00:10	01:07 01:00
		Video	The dinner game	0:50:10 01:08
	Sadness	Music	Pride & Prejudice, track 13 The English Patient, track 18 00:00 00:40	00:50 01:00
		Video	Dangerous minds	1:22:10 02:08
	Fear	Music	The Fifth Element, track 17 Hannibal, track 1	00:00 03:57	01:01 01:00
		Video	Scream 1	0:07:40 02:30

TABLE 4

 4

		.3: Groups of adjectives used in the questionnaire (extracted
		from Schaefer et al., 2010)
	Elicitation	English version	Group of adjectives French version
	Neutral/	Calm, Serene, Relaxed		Calme, Serein(e), Détendu(e)
	Calm	Moved, Tender		Attendri(e)
	Happiness	Warm hearted, Gleeful, Elated Joyful, Happy, Amused	Heureux(se), Exalté(e), Épanoui(e) Joyeux(se), Amusé(e), Gai(e)
	Fear	Fearful, Scared, Afraid Anxious, Tense, Nervous		Apeuré(e), Effrayé(e), Terrifié(e) Anxieux(se), Tendu(e), Nerveux(se)
	Sadness	Sad, Down-hearted, Blue Disgusted, Turned off, Repulsed Dégoûté(e), Écoeuré(e), Répugné(e) Triste, Cafardeux(se)

TABLE 4

 4

		.4: Expected and measured PAD scores for emotion elicita-
		tion of the global VR environments
			PAD
	Elicitation	Pleasure	Arousal	Dominance
		Expected Measured Expected Measured Expected Measured
	Neutral	5.

1 ± 1.2 4.9 ± 2.5 3.2 ± 1.5 3.6 ± 2.0 5.2 ± 2.0 6.3 ± 2.2 Happiness 8.2 ± 0.8 7.6 ± 1.6 7.0 ± 1.0 6.5 ± 1.7 6.8 ± 1.5 7.1 ± 1.5 Fear 2.4 ± 0.8 4.1 ± 2.2 7.4 ± 1.3 6.5 ± 1.6 3.3 ± 1.2 5.1 ± 2.1 Sadness

Most of the comparisons match what was expected, apart from the pleasure between N/F, which is nevertheless validated by H/F.

  Table4.5 summarizes the results of the pairwise comparisons, with the following information: the p values of the tests, the rank-biserial correlation coefficients rbc 6 , if the null hypothesis is rejected, the sign of the difference, and if the results fit our expectations.

TABLE 4

 4

		.5: Results of the pairwise Mann-Whitney U comparisons,
		for emotion elicitation of the global VR environments
	Dimension Pair	p value	rbc (16,16)	Rejected	Sign of the difference	Fits expectation
			Expected to be different
		N/H 0.001	0.6	True	<	yes
	Pleasure	N/F 0.184 N/S 0.015	0.2 0.5	False True	= >	no yes
		H/F 1e -4	0.8	True	>	yes
	Arousal	H/N 4e -4 H/S 0.019	0.7 0.4	True True	> >	yes yes
	Dominance	H/F 0.007 H/S 0.007	0.5 0.5	True True	> >	yes yes
			Expected to be equal
	Pleasure	S/F	0.087	0.3	False	=	yes
	Arousal	H/F 0.427	0.0	False	=	yes
	Dominance S/F	0.500	0.0	False	=	yes
	Note1:

each PAD dimension is significantly altered by the image categories.

  post-experiment questionnaire (filled by 11 participants) allowed to evaluate every individual stimuli (40 images, 4 videos, 8 musics), choose the most efficient ones, and detect the least appropriate ones. Images: Using the same method as in the previous section, we compare the PAD values of the images with the expected values (see Figure 4.1 and Figure 4.5). The Kruskal-Wallis test gives the following: Pleasure (H(3) = 252.8, p value < 1e -4 ), Arousal (H(3) = 29.3, p value < 1e -4 ), and Dominance (H(3) = 112.2, p value =< 1e -4 ). Thus The H-values are very high for Pleasure and Dominance, showing an important differentiation depending on the image categories. The results of the pairwise Mann-Whitney U comparisons are presented in Table 4.6.

The results are consistent with what was targeted. The only difference concerns the fear condition. It induces more pleasure than sadness, and less arousal than happiness. The arousal values even dropped down and are blend with sadness.

TABLE 4

 4

		.6: Results of the pairwise Mann-Whitney U comparisons,
			for emotion elicitation of the images
	Dimension Pair	p value	rbc (110,110)	Rejected	Sign of the difference	Fits expectation
			Expected to be different
		N/H < 1e -4	0.5	True	<	yes
	Pleasure	N/F < 1e -4	0.7	True	>	yes
		N/S < 1e -4	0.8	True	>	yes
		H/N < 1e -4	0.4	True	>	yes
	Arousal	H/S 1e -4	0.3	True	>	yes
		F/S	0.182	0.1	False	=	no
	Dominance	H/F < 1e -4 H/S < 1e -4	0.6 0.6	True True	> >	yes yes
				Expected to be equal
	Pleasure	S/F	4e -4	0.2	True	<	no
	Arousal	H/F 0.009	0.2	True	>	no
	Dominance S/F	0.311	0.0	False	=	yes
	Note1:

TABLE 4 .

 4 7: Emotional stimuli reference (Images are referred by identifiers in[START_REF] Lang | International affective picture system (IAPS): Affective ratings of pictures and instruction manual[END_REF], The sad video was slightly shorten).

	Emotion	Modality Image id / Title / Track	Start	Duration
		Images	2038; 2102; 2191; 2214; 2850
	Neutral	Music	The Portrait of a Lady, track 3	00:20	01:00
		Video	The Lover	1:10:30 0:48
		Images	1340; 1811; 2352; 5910; 8380
	Happiness	Music	Pride & Prejudice, track 4	00:10	01:00
		Video	The dinner game	50:10	1:08
		Images	2205; 2700; 2900; 3230; 9561
	Sadness	Music	The English Patient, track 18	00:00	01:00
		Video	Changing Batteries (MMU, Malaysia) 00:38	3:00
		Images	1050; 1300; 1302; 1930; 3500
	Fear	Music	Hannibal, track 1	03:57	01:00
		Video	Scream 1	07:40	2:30

  [START_REF] Watson | Toward a consensual structure of mood[END_REF] based their work on several previous factor analysis of adjectives studies. It was found that usually two dimensions appear: the pleasure and arousal, just like for emotions. From these two dimensions, four quadrants (i.e., combinations of energized/calm and unpleasant/pleasant) were defined, as four ba-

	sic moods, correlated with several adjectives. Based on this representation, Desmet,
	Vastenburg, and Romero, 2016 created a questionnaire (called Pick A Mood: PAM)

TABLE 5

 5 = 90, σ = 38 µ = 116, σ = 40 maxBot (kPa) µ = 81, σ = 42 µ = 78, σ = 42It is now important to discuss how the data is distributed in our dataset, and more specifically, to check for the normality of these distributions. In order to detect if a distribution follows a normal law, we use the Wilks-Shapiro test. The null hypothesis for this test is that the distribution is normal, so when the p value is lower than 0.05, it suggests the data is not normally distributed.

	.2: Distribution of maximum pressure
	Human agent Robot agent
	maxTop (kPa) µ

TABLE 5

 5

	.3: Number of participants that show Gaussian distributions
	depending on the component
	Component Human agent Robot agent
	XTop	3	7
	YTop	8	7
	maxTop	6	3
	XBot	9	9
	YBot	4	6
	maxBot	7	2
	Note: In total we have 11 participants.

XTop, YTop, and XBot for the human case, and XTop for the robot case are normally distributed and have equality of variance.

  Table 5.4) are as follows: for the human agent, only XTop, YTop, XBot, and maxBot have equality of variance, and for the robot agent, only XTop, XBot, and maxTop have equality of variance. So, if we want to strictly fit the requirements in order to compute ANOVAs, only

But most of the components do not verify the requirements for ANOVA. Given all these observations, we choose to use, in the rest of this chapter, less conservative 4 tests than ANOVAs and t-tests. We use the Kruskal-Wallis test instead of the One-Way ANOVA, and the Mann-Whitney U test instead of the t-test.

  In the cases these tests show a significant effect of the mood in the data, for the few variables that verify the ANOVA requirements, we can compute One-Way ANOVAs to obtain a more precise characterization of this effect.

TABLE 5 .

 5 4: Wilks-Shapiro and Levene tests to evaluate if the ANOVA requirements are valid for the mood study

				Human agent
	Component	Wilks-Shapiro test		Levene test
		W	p value	Normal	w (8,676)	p value	Equal variance
	XTop	0.996 0.052	Yes	0.83	0.574	Yes
	YTop	0.998 0.629	Yes	0.53	0.833	Yes
	maxTop	0.990 0.000	No	2.51	0.011	No
	XBot	0.997 0.343	Yes	0.57	0.804	Yes
	YBot	0.975 < 1e -4	No	2.67	0.007	No
	maxBot	0.989 < 1e -4	No	1.42	0.186	Yes
				Robot agent
	Component	Wilks-Shapiro test		Levene test
		W	p value	Normal	w (8,679)	p value	Equal variance
	XTop	0.997 0.240	Yes	0.98	0.446	Yes
	YTop	0.984 < 1e -4	No	2.77	0.005	No
	maxTop	0.954 < 1e -4	No	1.53	0.142	Yes
	XBot	0.995 0.042	No	1.10	0.360	Yes
	YBot	0.977 < 1e -4	No	2.25	0.022	No
	maxBot	0.988 < 1e -4	No	2.79	0.005	No

TABLE 5

 5

	.5: Number of handshake samples associated with the moods
	Mood	Human agent Robot agent
	Neutral	90	90
	Calm	139	143
	Relaxed	88	88
	Cheerful	153	154
	Excited	51	51
	Tense	59	59
	Irritated	31	31
	Sad	24	23
	Bored	50	49

  1, we use Kruskal-Wallis 5 tests to see if the mood alters the 6 following standardized components: XTop, YTop, maxTop, XBot, YBot, and maxBot. The results are in Table 5.6.

TABLE 5 .

 5 7: Mann-Whitney U pairwise comparisons for the interaction with the human and robot agents, for the maxBot feature.

		Mood location in PA space	Pair	Human agent p value rbc sign p value rbc sign Robot agent
	(1)	= Pleasure = Arousal	Bored/Excited Sad/Excited Tense/Relaxed	0.003 0.3 0.005 0.4 0.345 0.0	< < =	0.001 0.4 0.126 0.2 0.474 0.0	< = =
	(2)	-Pleasure = Arousal	Sad/Irritated Bored/Irritated Bored/Tense	0.048 0.3 0.054 0.2 0.008 0.3	< = <	0.135 0.2 0.006 0.3 0.009 0.3	= < <
	(3)	+ Pleasure = Arousal	Calm/Cheerful Relaxed/Cheerful 0.409 0.0 0.093 0.1 Relaxed/Excited 0.079 0.1	= = =	0.001 0.2 0.172 0.1 0.134 0.1	< = =
	(4)	= Pleasure = Arousal	Calm/Relaxed Tense/Irritated Sad/Bored	0.162 0.1 0.355 0.0 0.239 0.1	= = =	0.082 0.1 0.234 0.1 0.239 0.1	= = =

TABLE 5 .

 5 8: Mann-Whitney U pairwise comparisons sign differences, for the interaction with the human and robot agents, for YBot and maxBot features.

	Mood location in PA space	Pair	Human agent YBot maxBot YBot maxBot Robot agent
	(1)

TABLE 5

 5

		.9: Mann-Whitney U pairwise comparisons for the interac-
		tion with the robot agent, for the XTop feature.
		Mood location in PA space	Pair	Robot agent p value rbc sign
	(1)	= Pleasure = Arousal	Bored/Excited Sad/Excited Tense/Relaxed	0.390 0.279 < 1e -3 0.3 0.0 0.1	= = <
	(2)	-Pleasure = Arousal	Sad/Irritated Bored/Irritated Bored/Tense	0.392 0.498 < 1e -3 0.4 0.0 0.0	= = >
	(3)	+ Pleasure = Arousal	Calm/Cheerful Relaxed/Cheerful 0.173 0.484 Relaxed/Excited 0.347	0.0 0.1 0.0	= = =
	(4)	= Pleasure = Arousal	Calm/Relaxed Tense/Irritated Sad/Bored	0.151 0.001 0.385	0.1 0.4 0.0	= < =

This means that at least one participant has a behavior different from the others, for all the components.

  1 showed that only half of the distributions of the 6 components and 11 participants are Gaussian. So we do not use parametric analysis in this study. We start with the computation, for each component and each agent, of a Kruskal-Wallis test depending on the participants. The tests are significant for all the 6 components, with p values < 1e -4 . The H values corresponding to these tests are shown in Table5.10.

	In order to see if this is due to only one person or if all the participants behave
	differently from each others, we compute pairwise comparisons Mann-Whitney U
	tests. In other words, for each pair of participants, we test if the distribution of one
	component is significantly different. As we have 11 participants, we compute 55
	comparisons. We count the number of significant differences (p value < 0.05), noted
	nb

H or nb R depending on the agent. This is presented in Table

5

.10. We notice that a high number of comparisons are significant, which indicates that more than one individual is involved in this differentiation. Besides, the size effect when the comparison is significant, rbc (rank-biserial correlation) is between 0.2 and 1.0, with

TABLE 5

 5 : All the p values of the Kruskal-Wallis tests are below 1e -4 . Note2: nb i is the number of significant pairwise comparisons, using Mann-Whitney U tests (significance level: p value < 0.05). nb H : only using the human agent interaction, nb R : only for the robot, nb a : significant for both agents. Note3: As we have 11 participants, a number of 55 pairwise comparisons were computed.

	.10: Between-participant differences and number of consis-
			tent behaviors
	Feature	Human agent Robot agent H(10) nb H H(10) nb R nb a Both agents nb sign
	XTop	247	43	311	48	37 27 (73%)
	YTop	121	41	230	40	31 17 (55%)
	maxTop	140	43	153	40	29 25 (86%)
	XBot	144	40	208	42	29 23 (79%)
	YBot	121	39	103	40	32 28 (88%)
	maxBot	235	42	135	42	34 30 (88%)

Note1

As a result, upon 55 comparisons, between 25 and 30 show a significant difference, in both agent conditions, and with the same sign difference. This indicates a very good consistency of the behavior of the partici- pants, either facing the human, or the robot.

  components. Moreover, if we test if the sign of the mean difference between these pairs is equal for both agents, we get the nb sign column in the table. Most of the signs are equal (apart from YTop). One remark we can point out, is that the statistical method employed has a bias in the computation of the p values used to determinate significance of a test. Indeed, we performed consecutive null-hypothesis statistical tests, in the same database, only changing the subsets that are compared. Usually, the test that is computed for this type of analysis is the post-hoc Tukey test. It takes into account the whole distribution, as it uses the F value of an ANOVA. However, as our data is not Gaussian, we could not use it. No such test exists, to our knowledge, for non-parametric analysis.

  Chapter 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 6.1.2 Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 6.1.3 Chapter 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 6.1.4 Chapter 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

	Chapter 6
	Conclusion
	Contents
	6.1 Summary of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . 132
	6.1.1

6.1.5 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 6.2 Overall contributions and perspectives . . . . . . . . . . . . . . . . 137 6.3 My publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

TABLE A .

 A 1: Mann-Whitney U pairwise comparisons for each of the 46 sensors outputs depending on the firmness conditions

		Participant's glove Experimenter's glove	Solution
	Sensor Pair p value	rbc sign p value	rbc	sign	1 2 3 4
	0	E/P < 1e -3 1.0	<	< 1e -3 0.9	>	S	S
	1	E/P < 1e -3 0.8	<	0.001	0.6	>	S	S
	2	E/P 0.011	0.5	<	0.066	0.3	=	S	S
	3	E/P 0.001	0.7	<	0.220	0.2	=	S	S
	4	E/P < 1e -3 0.7	<	< 1e -3 0.8	>	S	S
	5	E/P < 1e -3 0.8	<	0.001	0.7	>	S	S
	6	E/P < 1e -3 0.8	<	0.018	0.4	>	S	S
	7	E/P < 1e -3 0.7	<	0.231	0.2	=	S	S
	8	E/P 0.022	0.4	<	0.031	0.4	>	S	S
	9	E/P 0.214	0.2	=	0.267	0.1	=		? ?
	10	E/P 0.305	0.1	=	0.001	0.7	<	R	R
	11	E/P 0.411	0.1	=	0.079	0.3	=		? ?
	12	E/P 0.003	0.6	>	0.097	0.3	=	R	R
	13	E/P 0.410	0.1	=	0.001	0.7	<	R	R
	14	E/P 0.252	0.1	=	0.001	0.7	<	R	R
	15	E/P 0.382	0.1	=	0.448	0.0	=		? ?
	16	E/P 0.173	0.2	=	< 1e -3 0.7	<	R	R
	17	E/P 0.261	0.1	=	< 1e -3 0.8	<	R	R
	18	E/P 0.068	0.3	=	0.073	0.3	=		R R
	19	E/P 0.243	0.1	=	0.024	0.4	<	R	R
	20	E/P 0.015	0.5	>	< 1e -3 1.0	>	?	R R
	21	E/P 0.107	0.3	=	< 1e -3 0.7	<	R	R
	22	E/P 0.097	0.3	=	0.106	0.3	=		? ?

  are spatial and 2 represent the pressure magnitude. They are represented in Figure B.3, and the definitions are summarized in Table B.1.

TABLE B

 B

			.1: Computation of the 6 components
		Component Computation
	T	XTop	Barycenter of the sensors on the top of the agent's glove
	O P	YTop	Difference between the participant's thumb and the maximum pressure on the top of the agent's glove
		maxTop	Maximum pressure of the "Top" sensors
	B O	XBot	Barycenter of the maximum of the two last phalanx of each finger
	T T	YBot	Barycenter of the maximum of the first, second, and last phalanx
	O M	maxBot	Maximum pressure of the sensors in the four fingers of the participant

TABLE B .

 B 2: Correlation matrix using Spearman coefficients, between the 6 components, for both agents

					Human agent
			XTop		YTop		XBot		YBot		maxTop
		ρ	p value	ρ	p value	ρ	p value	ρ	p value	ρ	p value
	YTop	0.08	0.041
	XBot	-0.22 < 1e -3 0.09	0.020
	YBot	-0.06	0.096	0.04	0.278	0.13	0.001
	maxTop 0.32	< 1e -3 0.45 < 1e -3 -0.08	0.049	-0.04	0.329
	maxBot	0.19	< 1e -3 0.04	0.316	-0.03	0.473	0.40	< 1e -3 0.33 < 1e -3
					Robot agent
			XTop		YTop		XBot		YBot		maxTop
		ρ	p value	ρ	p value	ρ	p value	ρ	p value	ρ	p value
	YTop	-0.14 < 1e -3
	XBot	-0.17 < 1e -3 0.11	0.003
	YBot	-0.06	0.122	0.14 < 1e -3	0.19	< 1e -3
	maxTop 0.06	0.106	0.43 < 1e -3 -0.05	0.194	-0.12	0.001
	maxBot	-0.04	0.337	0.18 < 1e -3	0.15	< 1e -3	0.43	< 1e -3 0.35 < 1e -3

Ecological here means that the study must approximate the real-world that is being examined.

[START_REF] Levenson | Voluntary facial action generates emotion-specific autonomic nervous system activity[END_REF] found that activating specific muscles of the face (i.e., the usual response to an emotion), can be a trigger to an emotion and can induce the corresponding physiological response.

[START_REF] Scherer | Experiencing emotion: A cross-cultural study[END_REF] proposed a large scale questionnaire to know how people experience emotions in their ordinary life.

[START_REF] Russell | Evidence for a three-factor theory of emotions[END_REF] projected 151 emotion labels in the PAD space. However, the "calm" emotion was not referenced.

The IAPS identifiers of selected images are as follows:Happiness={1340, 1750, 1811, 2092, 2311, 2352, 2550, 4626, 5910, 8380}, Fear={1050, 1300, 1302, 1113, 1930,

3280, 3500, 5971, 6610, 9600}, Sad- ness={2205, 2700, 2800, 2900, 3230, 3350, 6838, 9050, 9561, 9910}, and Neutral={1616, 2038, 2102, 2191, 2214, 2487, 2850, 7057, 7506, 8160}

H(g -1) is the H-statistic of Kruskal which follows approximately a chi-squared distribution. g -1 is the degree of freedom, g being the number of groups.

rbc(n 1 , n 2 ) is the rank-biserial correlation, computed as follows: rbc = 1 -2U n 1 n 2 , where U is the Mann-Whitney U test statistic, and n 1 and n 2 are the subset size in both compared groups. This is a representation of the size effect.

The selected samples are: N (The Portrait of a Lady), H (Pride & Prejudice), S (The English Patient), F (Hannibal). See details in Table 4.2.

H(g -1) is the H-statistic of kruskal which follows approximately a chi-squared distribution. g -1 is the degree of freedom, g being the number of groups.

w(k -1, nk) is the Levene test statistic which follows approximately a F-distribution, k is the number of groups, and n is the number of samples.

Note: In the first line we distinguish a combination of sub-distributions, in the second line are asymmetries, and in the third line are "uniform" distributions, and the ones with numerous low pressure values.

The Wilks-Shapiro test statistic is noted W and enables to compute p values .

The residual is the collection of the differences between each sample and the mean of the group it belongs to.

The Levene test statistic is w(k -1, nk) which follows approximately a F-distribution, k is the number of groups, and n is the number of samples.

A more conservative test has less chances to reject the null hypothesis while it was really true

H(g -1) is the H-statistic of Kruskal which follows approximately a chi-squared distribution, g is the number of groups. When the null hypothesis is rejected, at least one condition has a significant impact on the distribution.

rbc(n 1 , n 2 ) is the rank-biserial correlation, computed as follows: rbc = 1 -2U n 1 n 2 , where U is the Mann-Whitney U test statistic, and n 1 and n 2 are the sample size in both compared groups. This is a representation of the size effect.

Ecological here means that the study must approximate the real-world that is being examined.

rbc(n 1 , n

) is the rank-biserial correlation, computed as follows: rbc = 1 -2U n 1 n 2 , where U is the Mann-Whitney U test statistic, and n 1 and n 2 are the sample size in both compared groups. This is a representation of the size effect.

Acknowledgements

First of all, I want to thank Digiteo and Labex Digicosme for founding this thesis. I thank my supervisors Adriana TAPUS, Moustapha HAFEZ, and Mehdi AMMI, who created this collaborative project, for hiring me. I am grateful to them for having accompanied me in the meanders of the many disciplines and perspectives in which my thesis is involved. They supported me and helped me when the dead-lines were dangerously close. Most of all, they trusted me and gave me a lot of freedom in my work, which made my thesis tackle subjects in which I blossomed.

I thank the jury members E. ANDRE, F. VIDAL-VERDU, M.A. AMORIM, G. CHENG, P. HENAFF, and D. HEYLEN for their reports, their constructive questions, and for having traveled from far to attend my defense.

Spearman coefficients of 0.2 in Appendix B). The fact the XTop significance is only visible with the robot agent may be due to a higher precision of the computation of this feature as the robot has 4 sensors in the top of its glove, compared to 3 for the experimenter. So, we also study in detail the XTop values in the case of the human-robot interaction. We can also remind that this feature fits the requirements to compute ANOVAs depending on the mood, F value (8, 677) = 2.28, p value = 0.021, which is also significant. We now detail how the maximum applied pressure in the bottom of the grasp (maxBot), changes depending on the mood. We observe the differences between two moods in pairwise comparisons, using Mann-Whitney U tests. It has been shown in the previous Section 5.4.2 that the mood distribution is heterogenous, leading to highly different number of samples depending on the conditions. The Mann-Whitney U test does not require to have the same number of samples in the pairs, however, we decide to compare only the moods that have balanced sample size. The results for both agents are presented in Table 5.7, with the p value of the tests, the rank-biserial correlation coefficients rbc 6 , if the null hypothesis is rejected, and the sign of the difference. The comparisons are grouped in four types: (1) the opposed moods in the Pleasure-Arousal (PA) space, (2) negative moods with opposite arousal, (3) positive moods with opposite arousal, and (4) moods in the same circumplex quarter.

Several comparisons show significant differences. It is mostly the "Bored" mood that induces lower pressure than the other conditions, for both agents. The size effect is however weak (i.e., rbc is between 0.2 and 0.4). The type of comparison that reveals the highest number of significant differences is (2) when comparing negative moods between themselves. The positive moods are more difficult to discriminate. The distribution of the mood states did not allow to compare negative with positive moods at the same level of arousal. However, some differences were

Appendix

Other contributions were provided along the thesis. They are presented in Appendix A and Appendix B.

Appendix A aimed to detect if a sensor is activated by the participant, by the experimenter, or by both. We were able to create three groups of sensors that were used in Chapter 4 and Chapter 5. We also characterized the magnitude of the pressure applied by the participants. As a last contribution, we highlighted an action-reaction process between both partners. However, the intensity of the reaction of an individual is lower than if it was caused by his/her own action.

Appendix B aimed to create components that describe the pressure exchanged during handshake interactions. We determined the area that are rarely touched. We accurately explained how we created 6 components. They describe either pressure location and pressure amplitude features. Some cues are given about handshake manner in general as we observed the correlations between these components.

Overall contributions and perspectives

Along this thesis, we brought several contributions that could be very useful in the targeted fields. We were able to collect tactile data from a large number of handshake interactions, in different conditions. We were able to design sensitive gloves with numerous sensors, that were "transparent" and wireless. We were able to study the pressure exchanged during handshake interactions. We could describe and analyze the data in terms of spacial distribution and pressure magnitude. The descriptor was designed to be compatible with other measurement systems. We were able to compare handshake interactions face to a human or face to a robot. We found, at every stage of our analysis, significant similarities between both agents. We were able to compare the handshakes between individuals. In all our datasets, we observed significant differences in the handshake manner depending on the participant. These differences (it was only studied in one experiment, in which we also took into account the movement features) were correlated with gender and extroversion degree.

Hence, it was possible to infer these dimensions of the social context from handshakes with success rates above than chance. We designed an emotion elicitation tool and evaluated it. Nevertheless its efficiency in a study that tackles emotion versus handshake manner is questionable. No results were found about the impact of emotions on handshake but we discussed the possible reasons. We were able to collect repeated handshakes with the same participants, for a "long" period of time, and label them with the spontaneous mood of the participants. We found a significant, and promising for the future, effect of the mood on handshake manner. The arousal dimension of the mood may play an important role in the pressure exchanged.

Despite these promising contributions, one has to notice that this work is a first step in an unexplored domain. Therefore, a lot of work remain to characterize deeply Appendix A

Discrimination of the action of the participant or the experimenter A.1 Introduction

As explained in Chapter 4 Section 4.5.1 we improved the handshake measurement system compared to Chapter 3. We desinged two gloves of 23 sensors. The gloves are duplicated so that we can observe the behavior of the participant the same manner we can observe the experimenter's action. However, the action of one partner does not concerns only the sensors in his/her own glove. The sensors on the fingertip, for instance, represent the action of the glove owner, while the sensors in the top and bottom of the hand correspond to the action of his/her partner.

In order to determine which sensor represents the action of which individual, we conducted a dedicated experiment. In Section A.2 we describe the protocol, and in Section A.3 we present an analysis to discriminate the role of the sensors. We also observed how the action of one partner has consequences on the action of the other, highlighting a loop effect.

A.2 Experimental protocol

This experiment aimed to study two things: (1) the differences on the sensors response depending on which partner has a firm handshake, which should enable to determine which sensor is activated by who, and (2) to which extent the action of one has repercussions on the action of the other, comparing to his/her normal handshake firmness.

As a preliminary experiment, few samples were collected, with the participation of 4 persons (1 female and 3 males, between 24 and 33 years-old, three are extrovert and one is introvert). They performed 13 handshake interactions with the same experimenter, who is also the same as in the experiments of Chapter 4 and Chapter 5.

The participant and the experimenter wear an instrumented glove on the right hand.

The gloves are attributed the same way as in the other experiments. The 13 handshakes occure as follows. The participant is informed he/she has to perform three types of handshake: both partner have a normal firmness (N), only the participant

Appendix B

Computation of 6 handshake pressure components

B.1 Introduction

In this appendix we present a manner to represent the handshake pressure data.

This uses a combination of sensors in the second version of our instrumented gloves (see Chapter 4 Section 4.5.1).

In order to analyze the handshake data, we cannot represent each sensor pressure as it would not be possible to interpret the results. Besides, all the sensors are not touched at each handshake. This would introduce null values for one sensor while the one near it would have a positive value. This is a drawback when we want to use statistical tests. We need fewer components that are meaningful.

Our gloves are able to measure the spatial distribution of the pressure. Hence, in order to simplify the description, we only need to know: What is the maximum magnitude of the pressure? Where was it reached? Of course it does not represent all the information. Other aspects could be affected by the experimental conditions.

Nevertheless it is a good start.

In Section B.2 we present the choices we maid to create 6 components. Then, in Section B.3, we evaluate the efficiency of this description.

B.2 Definition of the components

Each instrumented glove is made of 23 pressure sensors. As explained in Appendix A, three groups of sensors among the 46 available can be created: 19 are activated by the participant (Gp), 19 by the experimenter (Ge), and 8 by both. In the chapters that use this appendix, we are only interested on describing the behavior of the participant.

Thus, we only use the Gp sensors.

In Chapter 3 we found a high correlation between the maximum and average pressure for each sensor. That is why we think, in a first approximation, that the time profile of the pressure, on average, may not contain more information than the maximum value of the signal. We also think that this feature contains more variability as it shows to which extend the participant was able to press during the handshake. These are the reasons why we only analyzed the maximum values reached by a given sensor during the handshake interaction. The objectives of this appendix is to compute geometrical components that represent the handshake data. In order to have a meaningful computation, we need more sensors on the participant's fingers than we have in Gp (in green in Figure A.1).

We add the sensors p9, p10, and p11 in the participant's glove. Even though they are partially activated by the experimenter, the pressure magnitude is low, so they would not interfere too much in the results. However the sensor p12, activated mostly by the experimenter, reaches too high values to be selected. We also remove the sensors in the palm of the experimenter (the sensors e13, and e14, but also e10), We think it is important to describe the geometrical aspect of the pressure distribution. Indeed, the only work we found in the literature, that tackles where the pressure is exerted in the handshake, was done by [START_REF] Knoop | Handshakiness: Benchmarking for human-robot hand interactions[END_REF]. However, the authors did not describe it in a numerical manner. We think that this geometrical aspect may convey information about the handshake behavior. Some attention has to be paid of while designing the geometrical descriptor. The data has to be comparable between different measurement devices. For instance, the sensors can be changed, as well as their number and location, but the computed geometrical components should represent the same information.

Description of the Top group:

We start to describe the data of the "Top" group. It is composed of three sensors in the top of the experimenter's hand, and one on the participant's thumb. A first component can be computed: it represents the position of the thumb along the three sensors in the top. We call it XTop (see Figure B.3). The numerical value corresponds the the barycenter of the pressure on these three sensors, associated with weights (i.e., -1, 0, and 1). XTop is not sufficient to describe the position of the thumb as the contact can be from the thumb tip, or the middle phalanx. We compute a YTop component that represents the perpendicular direction (see Figure B.3). However, our gloves do not have several sensors in that direction. The idea is that if one of the sensors shows a high pressure in the top of experimenter's hand while the participant's thumb shows no pressure, the contact is due to the middle phalanx, and the thumb tip stays in the air. If no pressure is felt in the top but the thumb shows high pressure, it means the thumb enrolled the hand and presses behind the sensors. If the pressures are similar, it means the thumb presses directly on the sensors. Hence, YTop is the difference between the thumb's pressure and the maximum pressure in the top. This is not the ideal representation, one should place more sensors in the back of the hands to have more precise measurements. However, in our case, this gives an idea of the variability along this axis.

Finally, concerning the "Top" group, we compute the maximum pressure reached in the area (taking into account the 4 sensors). This is named as maxTop. The maximum values of the pressure are also computed for the "Top", and "Bottom" groups.

Description of the Bottom group:

We now describe the components to describe the "Bottom" sensors. We represent the pressure position information on the fingers of the participant. Hence, we do not take into account the sensors in the bottom of the experimenter's hand yet.

We create a component called XBot, that corresponds to which finger presses the most. Indeed, we made the hypothesis that some handshake could be performed using more the index and middle fingers, and others using more the ring and little fingers. We use the 8 sensors in the last two phalanx in order to have the same number of sensors per finger. We take the maximum pressures of each pair, and compute the barycenter between these 4 values and the finger weights. The weights are attributed from the little finger to the index finger with this sequence: -1.5, -0.5, 0.5, 1.5. This is represented in Another goal of the creation of the components, is to describe, with few variables, the information contained in our original 19 sensors. We expect this information to be as complete as possible. We also would like to prevent from having redundant information. To evaluate these aspects, we compute the correlation matrix of the 6 components. We do not use the Pearson coefficients as we are not sure of the normality of the data. We use the Spearman coefficients, with the associated p values , for both interactions (i.e., with the experimenter and with the robot). The results are presented in Table B.2.

We now analyze this matrix. Several significant correlations are found, but first we have to notice that for both agents, and for both Top and Bottom groups, X is not correlated with Y. This means that these axis, which are geometrically perpendicular, are also orthogonal from the behavior point of view. In other words, it is important to keep 2 dimensions to describe the geometrical aspect of a handshake in the Top and in the Bottom areas. We find a negative correlation between XTop and XBot, for both agents. This indicates that when the participant has the thumb further in the agent's hand (i.e., has a deeper grasp towards the partner), he/she presses higher with the two last fingers. This can look contradictory, however we think it can be assimilated to a torque applied by the participant. He/she may be trying to rotate the hand to point it down. This is important to notice as the measurement of spacial distribution can be a way to estimate the torque exerted, which may be used to infer behavioral 1 The data we use in this appendix was collected in the experiment presented in Chapter 5.

touched during the whole handshake). We also wanted to reduce the number of features to a set of few components. They had to represent well the overall information and not to be redundant.

We first observed the distribution of all the sensors, were they activated by the participant or the experimenter. We could determine the ones that are rarely touched.

Then, we took the 19 sensors activated by the participant, but rearranged the selection, based on the previous observations, in order to increase the size of the studied areas. We precisely explained the choices that led us to the definition of 6 components. They either represent geometrical aspects of the pressure, and its magnitude.

This representation aimed to be independent from the used setup. We then verified that these components do not contain an important part of null information. Finally, we analyzed the efficiency of the representation observing the correlation matrix.

We found several results that not only validate the efficiency of the description, but also give new cues, that were never explored to the best of our knowledge, about the handshake manner itself. For instance, either in the Top and Bottom areas of the hands, we need 2 dimensions to describe the position of the pressure center. A kind of rotation intention was found in some handshakes, that may correspond to some specific behavior while handshaking. The firmness of a handshake is mostly caused by the finger tips of the individual. In this thesis, we investigate the possibility for a robot of detecting this emotion using the tactile data during a touched interaction.

We study tactile data exchanged during handshake interactions. These are executed either between two individuals, or between a human and a robot. Measurement devices are developed, for both interaction partners, to collect tactile data. Several conditions are tested to see their effect on handshake manner: the gender of individuals, their personality, their mood, and their emotions. Concerning the emotions, we design a tool that uses virtual reality, that enables to elicit emotions, in a controlled manner. The results suggest that it is possible to detect gender and extroversion degree, using pressure and movement data. Besides, differences exist depending on the mood, either in terms of contact position and pressure magnitude. These results are consistent depending on the interaction with the human or the robot.