Annie Pauzie
email: annie.pauzie@ifsttar.fr

O Orfila

Methodologies to assess usability and safety of ADAS and automated vehicle

Keywords: human factors, automation, methodology, road safety, human centred design, acceptability, trust, situation awareness, workload

In the framework of future innovation and for the sake of road safety, there is a great hope in fully supporting, or even replacing, the human driver by reliable technology. But, due to the novelty of this context, an important care will have to be devoted to investigate drivers' expectation, needs, behavior and functional abilities to reach this goal. In this context, this paper reviews several human factors issues related to partial and fully automated vehicles, with discussion of strengths and weaknesses of methods investigating driver automation acceptability, trust, situation awareness and workload. Main results of these parameters in relation to automated driving are presented and relevant methodologies to investigate these human variables are discussed in the perspective of real road experiments context.

INTRODUCTION

Several studies showed the importance of the human centred approach for the design and the implementation of technology in transport, and more especially in automotive context (Barnard et al., 2010). Indeed, taking into consideration the knowledge on human behavior, drivers' functional capacities and needs at the early stages of a prototype development helps designers to set up safer systems [START_REF] Pauzié | Human centred design for ITS: roadmap for future research, ERTICO invited session[END_REF]. In the perspective of the partial and fully automated vehicles, several issues linked to human factors will have to be carefully studied in order to reach one of the main ambitious objectives of self-driving: increase road safety in a zero accident perspective.

ISSUES IN DRIVER CENTRED DESIGN AND AUTOMATED VEHICLE

The fact that human will delegate some or the total control of her/his car to systems is raising new human factors issues in terms of acceptability, trust, situation awareness and mental workload.

Technology acceptability deals specifically with perceived usefulness and perceived ease of use [START_REF] Davis | Perceived usefulness, perceived ease of use, and user acceptance of information technology[END_REF] and is influenced by belief, concern and expectation of the population of users. A professional survey compared the attitudes surrounding autonomous vehicles. Polling 17,400 vehicle owners, it showed that opinions are split in the total population (Power and Associates, 2012). Social acceptability is linked to the idea that automation will deprive people of personal control over their vehicles and are possible concerns of automated driving [START_REF] Howard | Public perceptions of selfdriving cars: The case of Berkeley, California[END_REF][START_REF] Shladover | Why we should develop a truly automated highway system[END_REF], this specificity being diversely accepted depending on the drivers' personality, cultural background and generational belonging. An early survey conducted by [START_REF] Bekiaris | Driver needs and public acceptance regarding telematic in-vehicle emergency control aids[END_REF] found a definite rejection of automated driving. More recently, after experiencing highly automated driving (HAD) in a driving simulator, only 13 of 38 participants stated that they wished to have the system in their car, possibly because the system did not prevent incidents [START_REF] Schieben | Test results of HMI in use on cars and with simulators[END_REF]. Some drivers liked the increase of comfort and safety while some respondents did not like the idea of handing over control to automation [START_REF] Flemisch | HAVE-IT. Highly automated vehicles for intelligent transport[END_REF], People can find the lack of control unsettling, believing the technology to be unreliable and the programming to be incapable of proper control, worrying about the risk of computer malfunction [START_REF] Klayman | Self-Driving Cars Coming Our Way, But Don't Throw Out Your License Just Yet[END_REF]. [START_REF] Payre | Fully automated driving: impact of training on emergency manual control recovery, 22nd ITS World Congress[END_REF] showed that the driver's acceptability is related to the type of road context, with preference of delegating the control of the vehicle on highways, in traffic congestion and for automatic parking.

Acceptance is linked to usability characteristics of the system leading to trust. It is vital for successful implementation and is a precondition for these systems to achieve the benefits they claim [START_REF] Najm | Evaluation of an automotive rear-end collision avoidance system[END_REF]. In-vehicle system acceptance depends upon drivers' motivation and driving style, with a high variability among the population. At this stage, despite numerous studies on acceptance of driver assistance systems, neither a common definition nor a standardized measurement procedure is available [START_REF] Ghazizadeh | Extending the Technology Acceptance Model to assess automation[END_REF].

Acceptability and acceptance of automation are major challenges allowing success in implementation and a prior condition to get the expected benefits [START_REF] Najm | Evaluation of an automotive rear-end collision avoidance system[END_REF].

Trust is considered as a key variable for reliance on, and misuse/disuse of, automated systems [START_REF] Kazi | Assessing drivers' level of trust in Adaptive-Cruise-Control and their conceptual models of the system: Implications for system design[END_REF] and has a direct impact on the level of acceptance. Trust is a complex interaction involving dispositional trust (culture, age, gender, personal traits), situational trust (setting, difficulty, task, risk), initial, learned trust (pre-existing knowledge), and dynamic, learned trust (system performance, reliability, validity, errors) [START_REF] Hoff | Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust[END_REF]. The issue of trust in the system is needed to be considered both in partial and full automation, being crucial in the second case where the driver will be then able to be fully involved in an other activity requiring a lot of attention, or to really relax or even have a nap. There might be an issue not only related to lack of trust in the reliability and performance of the automatic system, but also an issue of over-trust where the human simply trusts the technology too much [START_REF] Strand | Semi-automated versus highly automated driving in critical situations caused by automation failures[END_REF], leading to misuse and disuse [START_REF] Parasuraman | Humans and automation: Use, misuse, disuse, abuse[END_REF][START_REF] Körber | Potential Individual Differences Regarding Automation Effects in Automated Driving[END_REF]. This will have to be highly considered in the scenario of driver getting back control to the driving task.

Situation awareness can be defined as ''knowing what's going on so you can figure out what to do" [START_REF] Adam | Fighter cockpits of the future[END_REF]. Endsley (1988a) which states that situation awareness is ''the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future".

It is important to consider this variable in the framework of automated vehicle, studies already showed that partly automation can lead to impoverish situation awareness [START_REF] Merat | Automated driving, secondary task performance and situation awareness[END_REF][START_REF] Carsten | Driver assistance systems: Safe or unsafe[END_REF]) with probably higher effect for highly automated situation. Indeed, a wealth of evidence from simulator studies shows that Highly Automated Driving evoke long response times and an elevated rate of (near-) collisions in critical events as compared to manual driving [START_REF] Strand | Semi-automated versus highly automated driving in critical situations caused by automation failures[END_REF]. [START_REF] Merat | How do drivers behave in a highly automated car[END_REF] found that drivers of a highly automated car took 2.5 s longer to press the brakes in response to a red traffic light than people driving manually in addition to slow brake response times with respect to emerging and oncoming vehicles. [START_REF] Damböck | Requirements for cooperative vehicle guidance[END_REF] found longer reaction times in a scenario where the lead vehicle braked hard with the longitudinal controller failing at the same moment and in a scenario where a wild animal ran onto the road, undetected by the sensors. However, there are counterexamples, where drivers successfully avoid collision in critical event scenarios. Essentially, if the automation fails unexpectedly with very little time for the human to respond, then almost all drivers crash [START_REF] Flemisch | Cooperative control and active interfaces for vehicle assistance and automation[END_REF], but if drivers receive a timely warning then almost all drivers will safely avoid collision [START_REF] Gold | Take over!" How long does it take to get the driver back into the loop[END_REF].

The level of situational awareness is a direct consequence of drowsiness, distraction, health status, fatigue, vigilance and involvement in activities not linked to driving task. Its evaluation will be especially important in a context of highly automated systems that requires the human driver to cycle in and out of the automated driving mode during a trip.

Mental workload is a psychological construct, difficult to define and difficult to assess, depending upon the task demands in relation to the amount of resources the operator is willing or able to allocate, and is therefore a relative concept [START_REF] Waard | The measurement of drivers' mental workload[END_REF]. In the framework of full automation, one of the most critical scenario in relation to mental workload will be for the driver to perform transitions from manual to automated control, or vice versa. Some authors argued that adjusting the automation status is itself a secondary task that should not induce too much extra workload [START_REF] Thompson | Cellulartelephone conversation while driving: effects on driver reaction time and subjective mental workload[END_REF]; nevertheless, in this particular scenario, the issue of the drivers ability to take the control back according to their current state, and the consequently induced cognitive cost for them to manage correctly the action, still needs to be clearly understood. Furthermore, workload can be reduced as the driver is relieved from the cognitive activity associated with manual driving and from the physical activity of moving the pedals and steering wheel [START_REF] Dragutinovic | Behavioural effects of Advanced Cruise Control use -A meta-analytic approach[END_REF].

Workload and situation awareness are two of the most important Human Factors constructs predictive of performance and safety [START_REF] Mccauley | Issues pertaining to the driver's role in automated highway systems (AHS): vigilance, supervisory control, and workload transition[END_REF][START_REF] Parasuraman | Situation awareness, mental workload, and trust in automation: Viable, empirically supported cognitive engineering constructs[END_REF][START_REF] Sarter | Situation awareness: A critical but ill-defined phenomenon[END_REF][START_REF] Stanton | A proposed psychological model of driving automation[END_REF]. Partial automation could raise driver's workload and situation awareness issue if the driver has to remain vigilant and monitor the automation status (Winter et al. 2014[START_REF] Merat | Automated driving, secondary task performance and situation awareness[END_REF]. They are human cognitive state rather than causal agents (Flach, 1995) and therefore require well defined measurement procedures [START_REF] Hand | Statistics and the theory of measurement[END_REF].

The driver's workload will have also to be assessed in conjunction with stress while performing transition from automated to manual control, in order to ensure safe handoff from the automated vehicle function to the driver in different driving conditions in the time allocated by the system All these variables will have to be deeply investigated due to the novelty of the situation and the consequently poor level of knowledge and understanding gathered in real road context experiments at this stage. Fortunately, several interesting driving simulator studies allowed setting up some basic understanding of human behavior and mental state in partial and full automation situation (Winter et al. 2014[START_REF] Merat | Transition to manual: Driver behaviour when resuming control from a highly automated vehicle[END_REF]. Due to the maturity of the technology nowadays, we reach the point where methods to investigate these human factor parameters have to be relevant for real road investigation.

METHODOLOGIES TO INVESTIGATE HUMAN FACTORS ISSUES IN REAL ROAD CONTEXT

Methodologies to investigate human factors will have to be adapted to the level of automation, as each of this level will raise specific issues regarding driver abilities requirements closely linked to the modalities of automated functionalities, and will have to be conducted in realistic context. So far, there are almost no studies testing highly automated driving in natural environment. Fully autonomous vehicles are not yet ready, and vehicles with highly automated driving technologies are still rather rare. Therefore, existing studies on automation have used interviews, online questionnaires [START_REF] Kpmg | Self-driving cars: are we ready[END_REF][START_REF] Payre | Fully automated driving: impact of training on emergency manual control recovery, 22nd ITS World Congress[END_REF][START_REF] Rödel | Towards autonomous cars: the effect of autonomy levels on acceptance and user experience[END_REF] or different contexts of driving simulation [START_REF] Strand | Semi-automated versus highly automated driving in critical situations caused by automation failures[END_REF][START_REF] Lee | Can autonomous vehicles be safe and trustworthy? Effects of appearance and autonomy of unmanned driving systems[END_REF]. However, human factors studies required, at this stage of technical maturity, more real-life studies to get ecological valid data. For example, in this purpose, the University of Michigan and MIT have created a mock-up set of busy streets in Ann Arbor to provide tests for self-driving vehicles in an urban environment [START_REF] Knight | Town built for driverless cars[END_REF], aiming to conduct a large-scale test with 2,000 driverless cars on the road within the next eight years. In the same vein, other projects using test tracks and dedicated experimental roads will be conducted in the coming decade to be able to gather reliable data on human behavior and automated vehicle. In this framework, adapted methodologies will have to be identified to investigate relevant human factor variables that could potentially impact system safety and usability.

Acceptability, acceptance and trust

To investigate acceptability of partial and fully automated vehicles, several methodologies have been used, more or less combined, such as online questionnaires, professional surveys, focus group interviews, questionnaires, in-depth interviews [START_REF] Casley | Driver workload comparisons among road sections of automated highway systems[END_REF].

The online questionnaire has a great advantage in terms of high number of participants that can be involved in the survey with reasonable time investment for researchers; the limit of this method is that participants cannot be observed while answering [START_REF] Leggett | Social desirability bias in contingent valuation surveys administered through in-person interviews[END_REF]. So, this type of investigation can be biased being based on imagination and not actual experience [START_REF] Rödel | Towards autonomous cars: the effect of autonomy levels on acceptance and user experience[END_REF]). Indeed, it is difficult to test something that is not fully implemented in a real world context, as end-users tend to think in an abstract manner rather than thinking about the situation in the real world. The methodology of focus groups and in-depth surveys allowed overcoming this limit, with more contextualized data in relation to the personality and the motivation of the respondent, getting closer to a real world situation and to avoid as much as possible imaginative thoughts. But, due to the lengthy of the process, it induces smaller size of interviewers sample that online questionnaire. Methodology of focus groups and in-depth surveys can be combined with some real road tests to get subjective point of view and opinions based upon real experience of the situation.

To study acceptance, most of the investigations have been conducted on driving simulator [START_REF] Tanaka | Research on the threshold of scare or secure by braking in Advanced Driver Assistance System[END_REF]. Nevertheless, [START_REF] Neale | Commentaries in: Human Factors issues for automated highway systems (AHS)[END_REF] and Farber (1999) stated that simulators do not provide an accurate representation of the phenomenology of real automated driving.

Trust and autonomous driving have been investigated in simulation and experimental studies [START_REF] Helldin | Presenting system uncertainty in automotive UIs for supporting trust calibration in autonomous driving[END_REF]Verbene et al., 2012). This variable is usually evaluated through questionnaires before and after experience of driving. In addition to these participants 'ratings, [START_REF] Gold | Trust in automation -Before and after the experience of take-over scenarios in a highly automated vehicle[END_REF] recorded gaze behavior during a driving simulator session in order to measure a potential change of trust by a change in scanning behavior. In this study, results indicate that horizontal gaze behavior could not be confirmed as a metric for measuring trust in automation.

Situational awareness, workload and stress

Several measures of situational awareness have been developed, the most widely used among them being the Situation Awareness Global Assessment Technique (SAGAT, [START_REF] Endsley | Situation awareness global assessment technique (SAGAT)[END_REF] and the Situational Awareness Rating Technique (SART). SAGAT provides an objective measure of situational awareness based on queries provided to the operator to assess his or her knowledge of what was happening at the time during freezes in a simulation while SART provides a subjective rating of situational awareness by operators [START_REF] Endsley | A Comparative Analysis of Sagat and Sart for Evaluations of Situation Awareness[END_REF].

The main advantage of SAGAT is that it allows an objective index of situation awareness. Its main disadvantage is that it requires to temporarily freeze the road scenery in a simulation context to test whether the driver has observed and understood the host vehicle's state, the road infrastructure, objects in the environment, and the behaviours of other road users. It has also been frequently asserted that another disadvantage of this method is that the technique relies on memory [START_REF] Dreyfus | Formal models vs. human situational understanding: Inherent limitations on the modeling of business expertise[END_REF]Nisbett and Wilson, 1977).

The method SART provides an assessment of situation awareness based on user's subjective opinion with a set of components determined through analysis to be relevant according to the context. Participants rate on a series of bipolar scales the degree to which they perceive (1) a demand on their resources, (2) supply on their resources and (3) understanding of the situation. These scales are then combined to provide an overall SART score for a given context. The main advantages of SART are that it is easy to use and can be administered in a wide range of task types and can be used in real world tasks as well as simulations. Potential limitations of SART have been asserted to include [START_REF] Endsley | Measurement of situation awareness in dynamic systems[END_REF]: (1) the inability of individual to rate his own situation awareness (2) the possible influence of the subjective evaluation of the performance level on the subjective rating of the situation awareness (3) possible confounding with workload issues while situation awareness may operate as an independent factor from workload in many situations [START_REF] Endsley | Situation awareness and workload: Flip sides of the same coin[END_REF]. Furthermore, it has been shown that the SART scores were highly correlated with confidence level and subjective performance, leading to consider that subjective situation awareness ratings be viewed as good indices of these aspects, but perhaps not veridical representations of situation awareness itself [START_REF] Endsley | A Comparative Analysis of Sagat and Sart for Evaluations of Situation Awareness[END_REF].

Situation awareness can be also evaluated while recording driver's behavior and attitude: in an interesting study, [START_REF] Omae | Measurement of driver's reaction time to failure of steering controller during automatic driving[END_REF] let 30 drivers experience a ride in a real highly automated vehicle on a test track. Even though the participants were told that the automation could display steering failures that required manual intervention, 8 participants fell asleep during the drives. Some participants started reading, operating their mobile phone, crossing their legs, or leaning out of the window. The drivers stated that they engaged in these behaviours because the task was boring and they had nothing to do. Situation awareness can be inferred from drivers' eyemovements recording. For example, drivers in highly automated driving contexts are less likely to gaze at the road centre than manual drivers (Barnard andLai 2010, Carsten et al. 2012), which indicates that they have altered situation awareness compared to manual driving. This variable can also be inferred by measuring performance of driver's response at critical events.

Several methods have been developed to measure mental workload (Colle H.A, 1998): measurements of physiological parameters, dual-task method [START_REF] Reid | The subjective workload assessmenttechnique: a scaling procedure for measuring mentalworkload[END_REF] and methods that elicit drivers' subjective judgments about the workload they have experienced.

In terms of physiological measurements, it has been shown that cognitive load can reduce skin conductance (Cha, 2003), increases eye-blink rate (Cha, 2003;[START_REF] Damböck | Requirements for cooperative vehicle guidance[END_REF][START_REF] Merat | Preface to the special section on human factors and automation in vehicles designing highly automated vehicles with the driver in mind[END_REF] and can be measured through heart rate variability (e.g., [START_REF] Brookhuis | Driving with a congestion assistant; mental workload and acceptance[END_REF][START_REF] De Waard | Driver behavior in an emergency situation in the Automated Highway System[END_REF][START_REF] Mayser | Driver workload monitoring[END_REF][START_REF] Takada | Evaluation of drivingassistance systems based on drivers' workload[END_REF][START_REF] Takano | A study of evaluation for driver's mental workload while driving heavy duty vehicle with ACC including EBS brake control[END_REF][START_REF] Törnros | Effects of ACC on driver behaviour, workload and acceptance in relation to minimum time headway[END_REF][START_REF] Wille | Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence[END_REF]. Using this variable, it has been shown that partial automation tend to reduce heart rate as compared to manual driving, indicating a reduction of workload [START_REF] Carsten | Control task substitution in semiautomated driving. Does it matter what aspects are automated? Human Factors[END_REF][START_REF] De Waard | Driver behavior in an emergency situation in the Automated Highway System[END_REF]). However, not all studies are consistent in this respect. Furthermore, measurement of physiological parameters raises important difficulties in real road context, that is why most of the workload driver's measurement in this case has been conducted using questionnaires or tests to assess cognitive cost. Subjective or self-assessed measures allow estimates from individual's reports concerning the workload or effort expenditure that was experienced during the task (Tokunaga R.A, 2000). These measures are often used in practice because they have many advantages in terms of easy running process over objective measures [START_REF] Patten | Using mobile telephones: cognitive workload and attention resource allocation[END_REF]. Among these methodologies, one of the more adapted to the context of driving is the DALI (Driving Activity Load Index, Pauzie, 2008), a revised version of the NASA-TLX, the latter one being created by the army and originally designed to assess pilot workload in the aviation domain. The DALI has allowed testing drivers' mental workload in various contexts linked to the use of in-vehicle technologies [START_REF] Harvey | A usability evaluation toolkit for In-Vehicle Information Systems (IVISs)[END_REF][START_REF] Tretten | Information Design Solutions for Automotive Displays, Focus on HUD[END_REF][START_REF] Tretten | Where Should Driver Information be Placed? A Study on Display Layout[END_REF][START_REF] Kim | Effects on Productivity and Safety of Map and Augmented Reality Navigation Paradigms[END_REF][START_REF] Kern | Making Use of Drivers' Glances onto the Screen for Explicit Gaze-Based Interaction[END_REF].

CONCLUSIONS

Automated driving is an important hope for road safety. Some results from a real road study showed that a partially automated car outperformed human drivers with better responses to unanticipated events, faster than manual drivers did [START_REF] Davis | Evaluation of semi-autonomous convoy driving[END_REF]. In the same vein, [START_REF] Marcus | Moral machines[END_REF] stated that, in a few decades, automated driving will be so reliable that humans will not be legally allowed to drive Nevertheless, at this stage, several questions need to be resolved in terms of human factors and will require deep investigations in realistic conditions to evaluate how the driver will accept, face, use and manage this situation according to the diversified scenarios linked to the several levels of automation.

The methodologies to assess drivers' requirements and capacities will need to be tested and maybe to be adapted taking into account the novelty of this situation in order to be fully efficient to prepare the reality of this huge step in the driving task concept and to ensure expected positive consequences for the road safety.