Virpi Roto
email: virpi.roto@aalto.fi

Philippe Palanque
email: palanque@irit.fr

Hannu Karvonen
email: hannu.karvonen@vtt.fi

Engaging Automation at Work -A Literature Review

Keywords: Work Automation, Interaction Design, User Experience, Engagement, Human-Computer Interaction, Human Factors, Literature Review

Automation pervades workplaces in an increasing pace and its effects on work practices and roles are far-reaching. Work tasks are typically automated with efficiency, effectiveness and safety in mind, but less attention is paid on the user experience aspects. As the amount of direct human control over technology is often decreased with automation, the human aspect of those systems might seem less essential and thus human-system interaction designers may not be consulted when automation is designed. Yet, fully autonomous and unmanned systems are rare, as humans often still have to monitor, intervene, maintain and control the automated environmentsbe it on-site or remotely. This paper discusses the need for better interaction design of automated systems with a focus on engaging user experiences in work environments. Results of a systematic literature on engaging user experience design in automation solutions used at work revealed that experiential human-automation interaction design is a neglected research topic. Therefore, we call for more research on automation design that improves not only efficiency, i.e., the pragmatic aspects of user experience, but also employee engagement and other emotional aspects of user experience. It is time for a turn to the experiential to take place also in the work automation context.

Introduction

The amount of research on the human aspects of automation is increasing rapidly. These human aspects include, for example, topics such as human-automation interaction (e.g., [START_REF] Hancock | Human-automation interaction research: Past, present, and future[END_REF]), the ethical aspects of automation (e.g., [START_REF] Kumfer | A Human Factors Perspective on Ethical Concerns of Vehicle Automation[END_REF] and acceptance issues [START_REF] Ghazizadeh | Extending the Technology Acceptance Model to Assess Automation[END_REF]. However, the user experience (UX) factors with automation especially in complex work environments has gained little scholarly attention [START_REF] Savioja | User experience: does it matter in complex systems?[END_REF]. Bainbridge [START_REF] Bainbridge | Ironies of automation[END_REF] was among the first to briefly discuss some engagement issues of automation, such as boredom, reduced feeling of achievement, and lower job satisfac-tion. User experience research is mostly focused on consumer products, e.g., Hassenzahl et al. [START_REF] Hassenzahl | Convenient, clean, and efficient?: the experiential costs of everyday automation[END_REF] raised the question of the experiential cost of automation using a coffee machine as an example. Both lines of research agree that the automated process may become meaningless to the users, and what might have previously been engaging tasks for the users may be degraded to, for example, waiting time [START_REF] Bainbridge | Ironies of automation[END_REF][START_REF] Hassenzahl | Convenient, clean, and efficient?: the experiential costs of everyday automation[END_REF]. Following these lines of thought, this paper discusses the relevance of designing for engaging automation, especially at work environments where the users of technical systems can seldom decide on the level of automation they want. The specific focus on work here means that we study employees as expert users of automation. In addition, the work context serves as a specific environment for studying engagement, since long-term engagement with the tools at work may often rise from professional, rather than entertainment aspects.

We see engagement as a consequence of successful user experience design, which has gone beyond removing problems and frustration to providing motivating, exciting or pleasurable experiences. The design of engaging user interactions with automation is an important, but largely neglected area of research. Interaction designers may see automation as a threat to their jobs, because the common thinking is that with higher levels of automation, no user interface or human interaction is required. However, workplace automation progresses in stages, and on different levels of automation (LoA, see e.g. [START_REF] Parasuraman | A model for types and levels of human interaction with automation[END_REF]), the human operator has a different role. While there are several models of LoAs, it is typical in all of them that the amount of human control decreases with the increasing automation level.

From the human perspective, automation refers to a device or a system that accomplishes (partially or fully) a function that was previously, or conceivably could be, carried out (partially or fully) by a human operator [START_REF] Parasuraman | Humans and automation: Use, misuse, disuse, abuse[END_REF]. By engaging automation design for work, we refer to design that changes the related work processes less human-dependent, but with the goal of making the work more interesting for the employees instead of making the work more monotonous and boring, which can often happen [START_REF] Cummings | Boredom in the workplace: a new look at an old problem[END_REF].

Engagement has been interpreted in many ways, as it can mean different things depending on the context. This paper tackles the question of what is known about engagement in relation to using automated systems at work. Our approach to answer the question is a systematic literature review. Specific research questions regarding the literature review are as follows:

RQ1: How much is published about designing experiential automation systems for work contexts? RQ2: Where this topic is being researched? RQ3: What are the limitations of the current research?

Related research

Before going into the literature review, it is important to clarify the key concepts of this paper, namely, engagement and automation.

Engagement with Technology

To better understand the concept of engagement in the specific case of using interactive technical systems at work, we approach the concept from two neighboring research areas: work engagement and engagement with interactive tools. Work engagement can be defined as "a positive, fulfilling work-related state of mind that is characterized by vigor, dedication, and absorption" [10], see Table 1 for details. The definitions of technology engagement are context-specific, and only a few publications in the field of interactive systems provide a definition for engagement. Publications on student engagement in e-learning and beyond provide perhaps the most developed view to the concept of engagement. In this field, [START_REF] Kahu | Framing student engagement in higher education[END_REF] identified that the key problem in engagement literature is mixing the state of engagement, factors that influence student engagement, and the immediate and longer-term consequences of engagement, and provides a framework making this distinction in higher education setting. While the context in [START_REF] Kahu | Framing student engagement in higher education[END_REF] is not work-related and many of the factors and consequences are not applicable in the domain of interactive tools used at work, we utilize the state of engagement, the factors affecting engagement, and the consequences of engagement as one conceptual framework that we will use in the literature review analysis.

Turner [START_REF] Turner | The anatomy of engagement[END_REF] sees engagement with technology as being positive and exploratory. The exploration aspect creates a "space" in which engagement occurs, and this space includes affordances of the technical artefact, which people subsequently exploit. According to [START_REF] Turner | The anatomy of engagement[END_REF], people engage with something and continue to do so basically because they enjoy doing that activity. Therefore, affect has an important role in engagement. In addition, people engage with technology, because it allows them to achieve their purposes and these purposes can be considered as a reciprocal expression of themselves (ibid.).

Sidner et al [START_REF] Przybylski | A motivational model of video game engagement[END_REF] defines engagement in human-robot interaction as "the process by which interactors start, maintain and end their perceived connection to each other during an interaction". This definition is also applicable with automated systems in which interaction between the human and the automated system is an active one, such as with service robots.

The publications discussing designing for engagement with automation are becoming more popular as the automation level of different environments increases. Automotive user interfaces seem to be the latest arena for engagement research, and in this context, engagement means the focus of attention. For instance, in cars, if drivers are disengaged from driving, it takes around 40 seconds to resume adequate and stable control of driving [START_REF] Merat | Transition to manual: Driver behaviour when resuming control from a highly automated vehicle[END_REF]. It is clear that in emergency situations, disengaged drivers are not able to react quickly enough. Instead of requiring drivers or users simply to keep their attention in the system, we should ask how to motivate the users to keep their attention on the system. Engagement to us means not only the focus of attention, but also motivation and enjoyment.

As we can see already from the literature above, there are many varying definitions for the term engagement. Since it is hard to follow a single definition, in this paper we build on [10] and discuss engagement in a continuum of short-vs. long-term engagement, from absorption (full concentration in work activities) to vigor (energy and mental resilience) and dedication (e.g., significance, enthusiasm, pride). This helps us to analyse the existing definitions and position them. For example, the definition of Sidner's human-robot interaction [START_REF] Sidner | Where to look: a study of human-robot engagement[END_REF] engagement is on the short-term side, i.e., during an interaction episode, while work engagement is typically long-term.

Automation of Work Tools and Relationship with Usability

The strive for more automation in the workplace has been driven by a set of considerations that proved useful before the computer age, but must be reconsidered nowadays. One of the first considerations was the one proposed in the MABA-MABA (Machine Are Better At -Man Are Better At) by Paul Fitts [START_REF] Fitts | Human engineering for an effective air navigation and traffic control system[END_REF] which was considering the best player, according to abstract tasks, between a human and a (steam) machinery (see Fig. From this figure it is interesting to note that only in number of functions, machines were better than human (3 positive scores for human and 6 for machines). Carver and Turoff [START_REF] Carver | Human-computer interaction: the human and computer as a team in emergency management information systems[END_REF] extended this notion of best player, replacing the steam machinery with computer systems (see Fig. 2). [START_REF] Carver | Human-computer interaction: the human and computer as a team in emergency management information systems[END_REF] These two classifications (and especially the first one) have been extensively used both as a design driver and as a mean of assessing the adequateness of function allocation between humans and machines/computers. However, in research, it has been clearly identified that actual work requires the involvement of both machines and humans ("In modern defence systems, the majority of functions need both humans and machines to undertake complementary tasks'' ( [START_REF] Goom | Function allocation and MANPRINT[END_REF], p. 53), cited in [START_REF] Dearden | Allocation of function: scenarios, context and the economics of effort[END_REF] ).

Fig. 2. Best player between human and computer from Carver and Turoff

Another major consideration pushing towards more automation in the workplace, came from studies on human error. Indeed, while some HCI contributions such as the Human Processor Model [START_REF] Card | The Model Human Processor: An Engineering Model of Human Performance[END_REF] were describing human behaviour as "flawless", other work (e.g. [START_REF] Reason | Generic error modelling system (GEMS): a cognitive framework for locating common human error forms[END_REF]) was highlighting that (within this human processor model) there was room for error within each processor (perceptive, cognitive and motor). While computer technologies were not getting more reliable (due to the fact that the computing field is evolving at high speed), the human error was always seen more prominent due to the impression that systems failures can always be fixed and fixing human errors is (due to the human nature) unfeasible. It was only much later that work from the dependable computing field was providing a more comprehensive view on failures, errors and faults integrating both human and systems errors in a single conceptual framework [START_REF] Avizienis | Basic concepts and taxonomy of dependable and secure computing[END_REF]. This kind of work highlights the fact that having more automation does not mean less errors as automation will fail, at least as much as other parts of the system will fail.

The fact that human and partly-autonomous systems are jointly involved when work has to be performed requires preventing as much as possible errors from both sides keeping in mind that allocation of function between the two is not a "zero sum" game: "automating a function does not necessarily result in a reduction in workload equivalent to the human work necessary for that function" [START_REF] Dekker | Function Allocation: A Question of Task Transformation not Allocation[END_REF]. This means that human work and the technical system with which the working human is cooperating have to be carefully designed. It also means that considering that the human will be able to take over when automation fails is not acceptable [START_REF] Matthews | Predicting battlefield vigilance: a multivariate approach to assessment of attentional resources[END_REF], especially when automation has been designed to replace the human operator.

An example of careful design can be taken from aeronautics where the accident rate is so low (less than one in ten million flights for the new generation of large civil aircrafts) that the underlying processes can be seen as exemplary.

Landing an aircraft may be perceived as an activity requiring very highly skilled operators and being safety critical, as an error might have catastrophic consequences. However, since the early 40s work has been carried out by the aircraft industry to provide pilots with an Autoland function. It was only on December 28th 1968 that the first civil aircraft (Airbus Caravelle) was qualified to use the Autoland function with a focus on reliability of the system offering segregation, diversity and redundancy mechanisms. Landing activities require performing flare (the nose of the plane is raised), managing thrust (reducing the value gradually), managing the roll (rotation of the aircraft along the axis running from nose to tail) and managing the alignment with the runway. Developing the Autoland function was done gradually performing first AutoFlare only, then adding AutoThrottle servos to handle thrust, and finally utilizing the autonomous management of roll leading to a fully autonomous landing system. While this system replaces humans very efficiently when visibility is low (including levels of zero visibility) its use is also extremely limited depending on other weather conditions, such as wind. For example, for a Boeing 747-400 the limitations are a maximum headwind of 25 kts, a maximum tailwind of 10 kts, a maximum crosswind component of 25 kts, and a maximum crosswind with one engine inoperative of five knots. This demonstrates the fact that careful design and deployment of automation in the workplace might increase feasibility of operations and the overall performance of a system. However, while work (and working conditions) and automation are considered, nothing is said about user experience and engagement of operators in such contexts even though the Hedonist magazine has a dedicated column called "on wings" dedicated to flying aircrafts [1] .

In other domains such as walk up and use systems, automation is added mainly to reduce required labor and thus to reduce the cost of workforce. This usually comes along with a reduction of the number of tasks covered, more sequential tasks, more cumbersome and less enjoyable. Cash machines are a widely used example demonstrating this when comparing with getting cash at the cashier. At the cashier, one can request a 100 Euros note because it is green and the person wants to make a green gift to a kid. In the user's own branch, it is often not necessary to prove identity as the cashier would know personally most of the clients. At the cashier, a PIN number and a card are not required, etc. Beyond cost of workforce reduction automations increases ubiquity and availability of service throughout the day. 2.3

Designing Interactive Systems for Engagement Designing for engagement shares much of its history with user experience (UX) design. UX has been a research topic, since the 3rd wave of human-computer interaction started to pay more attention to the emotional, and not only the pragmatic aspects of technology use [START_REF] Bødker | When second wave HCI meets third wave challenges[END_REF]. Since user experience design in practice often equals to user interface design and simply improving usability, we chose to focus on engagement, which requires more than problem-free use of interactive systems.

Ramsay, Barbesi and Preece [14[] were among the first ones to raise engagement as one of the web site design goals, and since then, the means for designing for engagement has been studied, for example, in the contexts of e-learning [START_REF] Cousin | Designs for network learning: A communities of practice perspective[END_REF][START_REF] Dickey | Engaging by design: How engagement strategies in popular computer and video games can inform instructional design[END_REF][START_REF] Hawthorn | Interface design and engagement with older people[END_REF][START_REF] Kay | Evaluating learning, design, and engagement in web-based learning tools (WBLTs): The WBLT Evaluation Scale[END_REF], games [START_REF] Przybylski | A motivational model of video game engagement[END_REF] , and interactive public displays [START_REF] Jacucci | Worlds of information: designing for engagement at a public multi-touch display[END_REF]. In automation contexts, designing for engagement has been mostly studied in human-robot interaction (e.g., Sidner et al. [START_REF] Sidner | Where to look: a study of human-robot engagement[END_REF]; Rich et al. [START_REF] Rich | Recognizing engagement in human-robot interaction[END_REF]), especially service robots. Robots are one way to replace human work with automation, but at workplaces, automation often comes in other forms.

Method

The main method used in this work, a systematic literature review, aims to find all scientific literature in a specific topic. According to APA [START_REF] Apa | [END_REF] p.10, literature reviews can help to • define and clarify the problem;

• summarize previous investigations to inform the reader of the state of research;

• identify relations, contradictions, gaps, and inconsistencies in the literature; and

• suggest the next step or steps in solving the problem.

In this chapter, we report the literature review method used in this research, including the suitable parts of the structure recommended by Kitcheman et al. [START_REF] Brereton | Lessons from applying the systematic literature review process within the software engineering domain[END_REF]. The publications filtering process was similar to the one used in [START_REF] Bargas-Avila | Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience[END_REF].

Search Process

First, we tested three databases of scientific publications to investigate their coverage of relevant publications in the area of engagement with automation. Scopus database was selected, as the search with keywords 'engagement' and 'automation' yielded the highest number of publications. The final search was executed in October 2018. The search process was done in 4 phases. In phase 1, one of the researchers executed the Scopus search and extracted the results to a table with publication information, including the title, keywords and abstract. In phase 2, the researcher checked the keywords and title to estimate the potential relevance of the publication. The same continued in the phase 3, where the researcher evaluated the abstract. In the last, 4th phase, the full text was studied to determine relevance of the remaining publications. In this last phase, the selection of potentially relevant publications was distributed to three researchers, so that each publication was examined by one researcher.

Inclusion and Exclusion Criteria

During the 4 phases of filtering, we included publications that focused on the engagement while interacting with automated systems at work. We did not set any criteria for the quality of the publication venue or publication. In phase one, the search keywords were selected based on our focus on engagement in relation to using automated systems at work: 'automation', 'work', 'interaction', and 'engagement'. Based on the initial search results, which were very limited, we expanded the search criteria to cover not only 'engagement' but also any emotional user experience, and not only 'automation' but also 'cyber-physical' as we expected cyber-physical systems to be used in industry.

Conference reviews (cr) and other reviews (re) were excluded from the search criteria. The final search phrase for Scopus was: TITLE-ABS-KEY ( ( automation OR cyber-physical ) AND work AND interaction AND ( "user experience" OR engagement OR emotional ) ) AND ( EXCLUDE ( DOCTYPE , "cr" ) OR EXCLUDE ( DOCTYPE , "re" ) )

In phases 2 and 3, evidence on emotional aspects in work automation were searched from the keywords, title, and abstract. Publications that were clearly out of scope were excluded, others were kept in for a more careful study. The most typical reason for exclusion was that 'work' did not refer to a work context, but 'work' appeared in phrases referring to the research reported in the publication, such as 'this work' or 'future work'.

In phase 4, a more thorough evaluation of the full publication was done to see if publication was reporting some emotional aspects of interacting with automation systems at work. The most typical reason for rejection in this phase was, still, that 'work' did not refer to work context. In addition, papers that did not investigate human-automation interaction were excluded at this point of the filtering process.

Results

Search Results

The total number of publications found in phase 1 with the Scopus search phrase was 56. In phase 2, 19 publications were excluded based on keywords and abstract. Out of 37 publications entering phase 3, 26 were excluded based on the abstract. Out of these, 22 were excluded because they were not addressing work context, but smart home or city, or e-services for elderly or young people. One publication was excluded because term 'automation' was appearing only in the copyright statement that Scopus appends to the abstract (International Journal of Automation and Smart Technology). This paper was not about automation but smart technology and thus, did not fulfil the search criteria. Another publication was an extended abstract of a keynote talk, which did not go into the required level of detail to be analysed. Two other publications were excluded in this phase due to lack of automation or human-automation interaction focus. Only 11 publications made it to phase 4, the full text study. Out of these, one was excluded due to lack of work context and one because it discussed human-automation interaction only as a motivation behind to the reported risk management study. The remaining nine publications are discussed in the next section.

Relevant Publications

The nine relevant publications were analyzed by the three main characteristics of relevant applications: Automation, Engagement/User experience, and Work context. A short description of each publication follows with analysis on the engagement. [32] presents a prototype that exploits gesture-based interaction technique in the domain of flexible manufacturing systems (partly-autonomous loading stations controlled by operators). The objective of the paper is to assess the impact of novel interaction techniques (more precisely gesture-based ones) on user experience. While the loading stations are party autonomous, the paper focusses on UX (especially emotional UX, frustration and competence) and the work of the operators in terms of tasks. In that paper, automation is not part of the design space and is only considered as a behavioral aspect of the loading stations. For this reason, the paper appears in Fig. 3 at the intersection between Work and User Experience.

[33] looks at the UX elements in machinery automation from interaction design perspective. In addition, the authors describe the design implications of these elements and the benefits and challenges of applying UX to this domain. However, in the paper, machinery automation is used as the name for big machines in mining and forestry. Therefore, it is not related to automation beyond the actual device. Based on this, taking into account the sections in Fig. 3, the paper recognizes very well the aspects of work and UX, but not automation and thus appear in at the intersection of Work and UX.

[34] present a prototype called intelligent interaction and emotion that offers to a user a standard WIMP (Windows Icons Menus and Pointing devices [START_REF] Van Dam | Post-WIMP user interfaces[END_REF]) user interface to control an air-cleaning device. The command and control system embeds contextual information affecting its behavior and good user experience is triggered by the possibility of breathing air of good quality produced by the device. For this reason, the system (device + command and control system) is autonomous and automation is distributed between these two components of the system. As this device belongs to the category of home appliances, it does not contribute to the work to be carried out by the user. This paper thus appears at the intersection between UX and Automation in Fig. 3.

[35] presents a theoretical study about how current emotional status of persons influence their reliance on automation. The case study is taken from security scanning

Automation

User Experience Work [START_REF] Heimonen | Designing Gesture-Based Control for Factory Automation[END_REF] [33] [START_REF] Merritt | Affective Processes in Human-Automation Interactions[END_REF] [40]

[37] [START_REF] Bodker | Complex Mediation[END_REF] [39]

[38] [START_REF] Cao | Case Study of Air Cleaner by the Intelligent Interaction and Emotion[END_REF] (for instance at airports) and the use of an Automated Weapon Detection system. The experiment has only been performed with students and does not relate to any work practice, hence its positioning at the intersection of automation and UX in Fig. 3. However, the paper concludes that positive emotional status increases reliance (i.e. likelihood to follow recommendations from an autonomous system) on automation.

[36] is a theoretically oriented paper examining how to combine the two traditions of activity theory and semiotics. In specific, the interest in [START_REF] Bodker | Complex Mediation[END_REF] lays in analyzing technology-mediated work both from activity theory and semiotics perspective. The paper discusses various automation examples from ship control and wastewater treatment domains. The emphasis is on understanding the work activity and automated tools used in work. However, the UX or engagement aspects of these automated tools are not considered in the article, so this publication is placed in the intersection between work and automation in Fig. 3.

[37] is a practical paper describing the technical solution for the use of smart glasses as part of a cloud computing system in factory context. [START_REF] Hao | The role of wearable devices in meeting the needs of cloud manufacturing: A case study[END_REF] aims to improve employee experience by freeing their hands to the actual work with wearable glasses, and by making communication more fluent with real-time video coming from the glasses to, e.g., the helpdesk operator. This makes communication between the back office and the shop floor more effortless, therefore we categorize this work as a work automation case. Beyond the general recommendation of using wearable technology, the paper does not discuss emotional aspects of automation. Therefore, it is placed at the intersection between work and automation. [START_REF] Filho | Using wearable and contextual computing to optimize field engineering work practices[END_REF] is very similar to [START_REF] Hao | The role of wearable devices in meeting the needs of cloud manufacturing: A case study[END_REF] in its approach to provide good experiences in context of industrial fieldwork via wearable technology. The user experience goal is to provide employees "the right information to the right user at the right time and place". In addition to this pragmatic user experience, the emotional experiences of workflow automation are not discussed. The paper lists workflow automation as one of the keywords, and we categorized it as such a case: it is about automation and work but not about engagement.

[39] focuses on training for operators in the domain of manufacturing systems. The paper presents a tutoring system exploiting augmented reality interaction techniques integrating motion detection and force feedback. When manufacturing systems are getting more complex and embed more automation, classical on-the-job training is more difficult to perform. The paper is thus centered on work and how automation influences the acquisition of needed skills to perform that work. The user experience aspect is only touched in the paper through the notion of emotional knowledge (i.e. trainees and trainers being able to share information together with the emotions they were feeling), which was facilitated by the new technology made available at training time. The publication was placed between Automation and Work. [START_REF] Kymäläinen | Evaluating future automation work in process plants with an experiencedriven science fiction prototype[END_REF] investigates conducting experience design research in industrial work context based on UX goals. The particular focus is on a science-fiction prototyping method: how to utilize futuristic interaction videos as means of UX-goal driven design. The system featured is an automation system utilized in a production environment. Therefore, all the three elements of UX, automation and work are featured in this paper and the paper is placed in the middle of Fig. 3.

Discussion

The overarching question that motivated this study was what is known about engagement in relation to using automated systems at work? The answer is simple: very little. This paper reported a systematic literature review in the intersection of automation, work, and user experience/engagement, and identified only one publication covering all three. Due to this limitation, eight closest works were included in the analysis as well. With this set of publications, we were able to answer our four specific research questions:

RQ1: How much is published about designing experiential automation systems for work contexts?

The literature review revealed a clear lack of research in the area of engaging work automation. Even after expanding the scope from engagement to user experience and emotional aspects in using automated systems, only one publication was identified to cover automation, work, and engagement or user experience. RQ2: Where this topic is being researched? It was interesting that 4 out of the 9 publications closest to the scope of this research originated from the Nordic countries. These countries are the ones that developed the participatory design approach to take the employees' perspective into account when designing work tools. This is known as the Scandinavian Tradition of democratic design. It is thus not surprising that the first publications introducing human-centred, experiential design of automation at work come from Scandinavia, especially Finland [32, 33, 35,].

RQ3: What are the limitations of the current research? Automation is a vast field of research. Thousands of scholars are studying automation from technical, business, societal, human factors, etc. perspectives. Similarly, the research on work expands to various topics with a huge volume of scientific publications. Also user experience research has exploded during the previous years, as businesses have realized its importance to long-term business success. Given the volume of research done in these three fields, it is surprising that only a handful of scholars have investigated user experience of automation in work context. The answer to question of the limitations in the current research is simple: there is still a very limited number of publications tackling this recently born research topic.

Limitations of this study

A literature review is always dependent on its defined scope. We used only one literature database, Scopus, for this literature review, so the set of literature might be wider if other databases would have been covered. This literature review required the search to appear in the title, keywords, or abstract of the publication. While we consider this search criteria adequate to find the key publications focused in the specific scope of ours, more examples of engaging work automation could be found in publications whose main focus is not on this topic. Finally, the selection of search keywords limits the scope of literature reviews, and the review could be expanded by adding new keywords.

Conclusions

We have discussed the concepts of engagement and automation, reviewing their definitions and providing examples. We found that these concepts are highly context-sensitive, and there is little research on them in the work context. Automation design requires a multitude of skills and cross-disciplinary research, as automated systems should be reliable, dependable, resilient, secure, usable, accessible, and provide good user experience. Much attention has been placed on the first aspects, but on designing for user experience in automation where users' role is to monitor, intervene and control automated processes is still in its infancy. As working with automation is an increasing part of our life, the experiential aspects, such as engagement, become more important.

For safety-critical environments, increasing engagement via playfulness or gamifying the work might not be a good approach as these types of activities can draw too much attention and compromise safety in some situations. Especially, engagement to secondary tasks on the cost of the primary task would not be beneficial in work environments.

The concept of user experience is not well defined, therefore, it is hard for automation researchers to start working on this area and publish their research. Also the definition of engagement is highly context specific, and there are different interpretations of it even within the field of automation. In the case of car driving, engagement refers to the driver's focus of attention (e.g. in [START_REF] Merat | Transition to manual: Driver behaviour when resuming control from a highly automated vehicle[END_REF]), while in human-robot interaction, engagement refers to the perceived connection between the human and the robot during their interaction ( [START_REF] Sidner | Where to look: a study of human-robot engagement[END_REF]). The analysis of engagement literature, reported in chapter 2, we found it useful to analyze the definitions of engagement on the continuum from shortterm to long-term. As the focus of this research is on work context, we are promoting engagement with automation as an emotional, motivational concept, in line with Schaufeli et al. 's definition [10] of the concept of work engagement.

This paper reported a systematic literature review to tackle question what is known about engagement in relation to using automated systems at work. The nearly empty intersection of automation, work, and user experience research in Fig. 3 may be due to the following reasons. Bringing automation to work context is motivated by increased efficiency and workforce savings, and the role of employees is seen disappearing as automation proceeds to levels where human operation is not required any more. Therefore, the human aspects in automation are considered of minor importance, while full attention is placed on the internal functionality of the automation system.

Fig. 1 .

 1 Fig. 1. a) Illustrations of the Fitts list ([26], pp. 7-8)b) MABA-MABA list from Fitts[START_REF] Fitts | Human engineering for an effective air navigation and traffic control system[END_REF]

Fig. 3 .

 3 Fig. 3. Positioning of literature with respect to the three relevant aspects; User Experience/Engagement, Automation, and Work.

Table 1 .

 1 Work engagement factors by [10]. of work engagement, and in how to maintain work engagement as the level of automation increases. Therefore, our focus in this paper is on the research of engagement with technology and with interactive work systems.

	Factor	Definition
		High levels of energy and mental resilience while working, the
	Vigor	willingness to invest effort in one's work, and persistence even in
		the face of difficulties

Dedication

Being strongly involved in one's work and experiencing a sense of significance, enthusiasm, inspiration, pride, and challenge

Absorption

Being fully concentrated and happily engrossed in one's work, whereby time passes quickly and one has difficulties with detaching oneself from work Specifically, we are interested in the role of interactive work tools in the formation

We hope this research the discussion on the need for further investigation of the experiential aspects of automation systems. A natural next step in future research is to broaden the literature review to automation engagement outside work context, and then test if the means of engagement in other contexts are applicable in the work domain. Since this literature review pointed us to an unexplored territory of research at the intersection of engaging user experiences, automation, and work, the list of relevant research questions for this area is long. Some of the interesting questions include:

 What does good user experience mean in case of work automation?  How does engagement with automation differ at work and in leisure contexts?  What does appropriate level of engagement mean in case of work automation?  How can automation systems contribute to improving work engagement?  Is design for engagement possible before acceptance and trust are there?  What are the engaging interaction techniques and data visualizations for work automation?

The fundamental idea behind this work is that automated systems do not entirely eliminate the interaction between humans and the system, but there are different levels of human control. Humans must be able to intervene automated processes when needed. Instead of dismissing user experience design for automated systems, we need to rethink the interaction style ( [START_REF] Kymäläinen | Evaluating future automation work in process plants with an experiencedriven science fiction prototype[END_REF]). The need for good user experiences does not disappear with automation.


1

The Right of access to the courts under the State of emergency in France 1 Olivier Le Bot, Professor of public law at Aix-Marseille university In France, the State of emergency legal regime is regulated by an act of 1955 2 . This statute was passed during the Algerian war, in order to strengthen administrative police powers in crisis situations.

According to article 1, this exceptional regime can be decreed in two situations: in "cases of imminent danger resulting from serious breaches of public order, or in case of events threatening, by their nature and gravity, public disaster".

Its implementation opens a number of exceptional powers to public authorities, especially for the minister of the interior and the prefects (ie governors that represent the state government at local level).

For example, those authorities can:

-put individuals under house arrest (people are confined to their homes during the night and, during daytime, have to report one, two or three times to the authorities) 3 ; -disband associations or groups 4 ; -prohibit certain public meetings and provisionally close certain meeting places 5 ; -pronounce curfews 6 ; -authorize the police to inspect vehicles, to carry out identity verifications and to control luggage 7 ; -authorize administrative searches in houses without judicial oversight 8 . In ordinary times (when the State of emergency is not activated), the last two measures can be ordered only by the judicial authority (either a judge, or a public prosecutor). But those repeals had absolutely no effects for the applicants. Why? Because the Constitutional court has systematically decided (as authorized by the Constitution) not to order the immediate cancellation of the unconstitutional provisions but to postpone this annulment for several months, taking into consideration the principle of legal security and in order to give time for the Parliament to rewrite the law.

The

II. The right to an ordinary judge

The second question, related to the right of access to court, deals with the control exercised not on legal provisions themselves but on individual measures taken on the ground of these provisions. This question is related to the right to get access to an ordinary court in order to review the applications of the legal provisions.

Is this control effective?

One must distinguish two situations.

Most of the decisions are subject to an effective judicial review. Referred back by the individual concerned by a given public decision, the administrative court can exercise its control in a very short period of time (in 2, 3 or 4 days) to evaluate whether the decision is justified and proportionate. If not, the judge can suspend the disputed decision [START_REF]Council of State (which is the supreme administrative court), ord[END_REF] . For example, 20 % of the measures of house arrest were considered illegal by the administrative court, and therefore suspended [START_REF] Commission | Opinion on the Draft Constitutional Law on "Protection of the Nation[END_REF] . As a result, we can agree with the Constitutional court to consider that the control exercised by the ordinary judge on those measures is effective [START_REF]Constitutional Council, decision n° 2015-527 QPC of 22[END_REF] . However, the control cannot be considered as effective for decisions that produce all of their effects in a very short period of time. It is the case of two types of decisions:

-decisions that authorize the police to inspect vehicles, to carry out identity verifications and to control luggage; -and the decisions that authorize house searches.

The house search is resorted in 2, 3, 5 hours, maybe 6. In any case, the targeted person has not the time to refer the decision back to court and to get a judgement on its legality before the house search is over. When the judge rules on the case, it's too late: if the administrative decision was illegal, the harm is done, and it cannot be rectified. The problem is all the more important because 40 % of the judgements have concluded that the disputed house searches were illegal14 .

The situation is the same with decisions authorizing the police to inspect vehicles, to carry out identity verifications and to check luggage. Those decisions were adopted every day, for a period of 24 hours. This time is too short to challenge its legality before a judge.

What is the position of the Constitutional court on this situation?

The Court did not rule on the provision of the 1955 act related to the power to authorize the police to inspect vehicles, to carry out identity verifications and to check luggage15 .

But it ruled on the article related to the conduct of house searches 16 .

And it found that the constitutional right to access to court was not violated. Why? For two reasons. First, because "of the particular circumstances which led to the declaration of the State of emergency"17 . Secondly because a judge can repeal the decision allowing the illegal house search and it can order the administration to compensate the damage suffered as a result of that decision.

We can estimate, as those two measures (annulment and compensation) only occur after the complete execution of a possible illegal house search, that they cannot be considered as providing an effective judicial protection.

With this decision of the Constitutional Council, we are far, far away, from the contemporary conception of what an effective judicial protection should be.

  State of emergency has been decreed eight times 9 : -the first three times in relation to the Algerian war; -the next three times in French overseas territories; -once during the 2005 riots in the suburbs of Paris; -and the last one in 2015 after a series of terrorist attacks in Paris. The measures taken under the State of emergency regime generate important effects for individuals and can create significant limitations of fundamental rights. Therefore, the rule of law implies the intervention of a judge to exercise a control. This issue has two dimensions: one the one hand, the right to a constitutional judge and, on the other hand, the right to an ordinary judge. I. The right to a constitutional judge The first dimension is related to the right to constitutional review. This implies the intervention of a judge to review the constitutionality of the law. Is this control actually exercised? Yes and no. Yes, formally: any person can challenge the constitutionality of a legislative provision through a priority preliminary ruling on constitutionality. It leads the constitutional court, in an ex-post review, to evaluate if the 1955 Act dealing with the State of emergency respects the constitutionally protected rights and freedoms. Yes again, because most of the challenged provisions of the 1955 Act have been declared unconstitutional. Without self-restraint, the Constitutional Court has reviewed the challenged provisions with the same criteria and the same intensity than in ordinary time. As a result, 8 of the 9 decisions handed down by the constitutional council regarding the law of 1955 have led to the unconstitutionality of some provisions of the actin most cases due to their vagueness 10 .

http://www.senat.fr/rap/l15-177/l15-1776.html#toc50.

Constitutional Council, decisions n° 536 QPC, n° 567/568 QPC, n° 600 QPC, n° 624 QPC, n° 635 QPC, n° 677 QPC and n° 684 QPC.

Report from the National Assembly n° 591 of 28 June 2017, p. 14 : 31 annulations, for 78 judgements handed down. Direct link here: http://www.senat.fr/rap/l16-591/l16-5912.html#toc17.

More precisely, it did not rule on the issue of the right to get access to court, because it declared this power unconstitutional for violating other rights (the freedom of movement and the right to privacy) : Constitutional Council, decision n° 2017-677 QPC of 1 st December 2017 (http://www.conseil-constitutionnel.fr/conseilconstitutionnel/francais/les-decisions/acces-par-date/decisions-depuis-1959/2017/2017-677-qpc/decision-n-2017-677-qpc-du-1er-decembre-2017.150249.html)

.16 Constitutional Council, decision n° 2016-536 QPC of 19 February 2016, §

[START_REF]Council of State (which is the supreme administrative court), ord[END_REF]. 17 § 11.


Alexandre Bruckert

Hamed R Tavakoli

Zhi Liu

Marc Christie

Olivier Le Meur

Deep Saliency Models : the Quest for the Loss Function

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

INTRODUCTION

Despite decades of research, visual attention mechanisms of humans remain complex to understand and even more complex to model. With the availability of large databases of eyetracking and mouse movements recorded on images [START_REF] Bylinskii | Mit saliency benchmark[END_REF][START_REF] Jiang | Salicon: Saliency in context[END_REF], there is now a far better understanding of the perceptual mechanisms. Significant progress has been made in trying to predict visual saliency, i.e. computing the topographic representation of visual stimulus strengths across an image. Deep saliency models have strongly contributed to this progress. However, as recently pointed out by Borji [START_REF] Borji | Saliency prediction in the deep learning era: An empirical investigation[END_REF], a neglected challenge in the design of a deep saliency model is the choice of an appropriate loss function. In [START_REF] Jetley | End-to-end saliency mapping via probability distribution prediction[END_REF] a probabilistic end-toend framework was proposed and five relevant loss functions were studied. Yet, to the best of our knowledge, none of the papers concerning the challenges in designing deep saliency models have investigated this aspect properly, despite its influence on the quality of the results. Important questions therefore arise: how do different loss functions affect performance of deep saliency networks? Which loss functions perform better than others and on which metrics? Is there actually substantial benefits in combining loss functions? And how does the combination of loss functions perform with respect to individual loss functions? In this work, we seek answers to such questions by conducting a series of extensive experiments with both well-known and newly designed loss functions. For this purpose, we first categorize loss functions per type of metric : (i) pixel-based comparisons e.g. Mean Square Error, Absolute Errors Exponential Absolute Difference, (ii) distribution-based metrics e.g. Kullback-Leibler divergence, Bhattacharya loss, binary cross-entropy, (iii) saliencyinspired metrics such as Normalized Scanpath Saliency or Pearson's Correlation Coefficient, or (iv) perceptual-based metrics, gathering two novel metrics we propose in this paper which are inspired from image style transfer, and measure the aggregation of distances computed at each convolutional layer, between the convoluted reference image and the generated saliency map. We then design a novel deep saliency model to provide a fixed network architecture as a reference on which all the loss functions will be evaluated. Our evaluation strategy then consists in evaluating the impact of all the loss functions taken individually, on our fixed network with a fixed image dataset (MIT). Then by building on the common agreement that different metrics favor different perceptual characteristics of the image [START_REF] Borji | Saliency prediction in the deep learning era: An empirical investigation[END_REF], we propose to further explore how the combination of loss functions, typically aggregating pixel-based, distribution-based, saliency-based and perceptual-based functions, can significantly influence the quality of the training. To demonstrate the generalization capacity of our combined metric, we measure its impact on different datasets (CAT2000 and FiWi) and also with a different network architecture (SAM-VGG). The contributions of this paper are therefore (i) to demonstrate how the choice of the loss function can strongly improve (or depreciate) the quality of a deep saliency model, and (ii) how an aggregation of carefully selected loss functions can lead to significant improvements, both on the fixed network architecture we proposed, but also on some other architectures and datasets. The paper is organized as follows. Section 2 presents the related works. The loss functions for training a deep architecture aiming to predict saliency map are described in Section 3. Section 4 presents the comprehensive analysis of loss functions and their combinations. Conclusions are drawn in the last section.

RELATED WORKS

Computational models of saliency prediction, a long standing problem in computer vision, have been studied from so many perspectives that going through all is beyond the scope of this manuscript. We, thus, provide a brief account of relevant works and summarize them in this section. We refer the readers to [START_REF] Borji | State-of-the-art in visual attention modeling[END_REF][START_REF] Borji | Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study[END_REF] for an overview. To date, from a computer vision perspective, we can divide the research on computational models of saliency prediction into two era (1) pre-deep learning, and (2) deep learning. During the pre-deep learning period, significant number of saliency models were introduced, e.g. [START_REF] Itti | A model of saliencybased visual attention for rapid scene analysis[END_REF][START_REF] Neil | Saliency based on information maximization[END_REF][START_REF] Harel | Graph-based visual saliency[END_REF][START_REF] Hou | Image signature: Highlighting sparse salient regions[END_REF][START_REF] Zhang | Saliency detection: A boolean map approach[END_REF], and numerous survey papers looked into these models and their properties, e.g. [START_REF] Borji | State-of-the-art in visual attention modeling[END_REF][START_REF] Zhao | Learning saliency-based visual attention: A review[END_REF]. During this period the community converged into adopting eye tracking as a medium for obtaining ground truth and dealt with challenges regarding the evaluation and the models, e.g. [START_REF] Riche | Saliency and human fixations: State-of-the-art and study of comparison metrics[END_REF][START_REF] Borji | Analysis of scores, datasets, and models in visual saliency prediction[END_REF]. This era was then replaced by saliency models based on deep learning techniques [START_REF] Borji | Saliency prediction in the deep learning era: An empirical investigation[END_REF], which will be the main focus of this paper. We therefore outline the recent research developments of deep saliency model era from two perspectives, (1) challenges of deep models and works that addressed them, and (2) the deep saliency models. We, then, stress the importance of task specific loss functions in computer vision.

Challenges of deep saliency models.

The use of deep learning introduced new challenges to the community. The characteristics of most of the models shifted towards data intensive models based on deep convolutional neural networks (CNNs). To train a model, a huge amount of data is required; motivating the search for alternatives to eye tracking databases like mouse tracking [START_REF] Jiang | Salicon: Saliency in context[END_REF], or pooling all the existing eye tracking databases into one [START_REF] Bruce | A deeper look at saliency: Feature contrast, semantics, and beyond[END_REF]. To improve the training, Bruce et al. [START_REF] Bruce | A deeper look at saliency: Feature contrast, semantics, and beyond[END_REF] investigated the factors required to take into account when relying on deep models, e.g., pre-processing steps, tricks for pooling all the eye tracking databases together and other nuances of training a deep model. Authors, however, considered only one loss function in their study. Tavakoli et al. [START_REF] Tavakoli | Saliency revisited: Analysis of mouse movements versus fixations[END_REF] looked into the correlation between mouse tracking and eye tracking at finer details, showing the data from the two modalities are not exactly the same. They demonstrated that, while mouse tracking is useful for training a deep model, it is less reliable for model selection and evaluation in particular when the evaluation standards are based on eye tracking. Given the sudden boost in overall performance by saliency models using deep learning techniques, Bylinskii et al. [START_REF] Bylinskii | Where should saliency models look next?[END_REF] reevaluated the existing benchmarks and looked into the factors influencing the performance of models in a finer detail. They quantified the remaining gap between models and human. They argued that pushing performance further will require high-level image understanding. Recently Sen et al. [START_REF] He | Understanding and visualizing deep visual saliency models[END_REF] investigated the effect of model training on neuron representations inside a deep saliency model. They demonstrated that (1) some visual regions are more salient than others, and (2) the change in inner-representations is due to the task that original model is trained on prior to being fine-tuned for saliency.

Deep saliency models.

The deep saliency models fall into two categories, (1) those using CNNs as a fixed feature extractors and learn a regression from feature space into saliency space using a noneneural technique, and (2) those that train a deep saliency model end-to-end. The number of models belonging to the first category is limited. They are not comparable within the context of this research because the regression is often carried out such that the error can not be back-propagated, e.g., [START_REF] Vig | Large-scale optimization of hierarchical features for saliency prediction in natural images[END_REF] employs support vector machines and [START_REF] Hamed | Exploiting inter-image similarity and ensemble of extreme learners for fixation prediction using deep features[END_REF] uses extreme learning machines. Our focus is, however, the second group. Within end-to-end deep learning techniques, the main research has been on architecture design. Many of the models borrow the pre-trained weights of an image recognition network and experiment combining different layers in various ways. In other words, they engineer an encoder-decoder network that combines a selected set of features from different layers of a recognition network. In the following we discuss some of the most well-known models. Huang et al. [START_REF] Huang | Salicon: Reducing the semantic gap in saliency prediction by adapting deep neural networks[END_REF] proposed a multi-scale encoder based on VGG networks and learns a linear combination from responses of two scales (fine and coarse). Kümmerer et al. [START_REF] Kummerer | Deep gaze i: Boosting saliency prediction with feature maps trained on imagenet[END_REF] use a single scale model using features from multiple layers of AlexNet. Similarly, Kümmerer et al. [START_REF] Kummerer | Understanding low-and high-level contributions to fixation prediction[END_REF] and Cornia et al. [START_REF] Cornia | A Deep Multi-Level Network for Saliency Prediction[END_REF] employed single scale models with features from multiple layers of a VGG architecture. There has been also a wave of models incorporating recurrent neural architectures. Han and Liu [START_REF] Liu | A deep spatial contextual long-term recurrent convolutional network for saliency detection[END_REF] proposed a multiscale architecture using convolutional long-short-term memory (ConvLSTM). It is followed by [START_REF] Cornia | Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model[END_REF] using a slight modified architecture using multiple layers in the encoder and a different loss function. Recurrent models of saliency prediction are more complex than feed-forward models and more difficult to train. Moreover, their performance is not yet significantly better than some recent feed-forward networks such as EML-NET [START_REF] Jia | EML-NET: an expandable multi-layer network for saliency prediction[END_REF]. In the literature of deep saliency models, a loss function or a combination of several ones is chosen based on intuition, expertise of the authors or sometimes mathematical formu-lation of a model. Kümmerer et al. [START_REF] Kuemmerer | Informationtheoretic model comparison unifies saliency metrics[END_REF] introduces the idea that information-theory can be a good inspiration for saliency metrics. They use the information gain to explain how well a model performs compared to a gold-standard baseline. Consequently, they use the log-likelihood for a loss function in [START_REF] Kümmerer | Deepgaze ii: Reading fixations from deep features trained on object recognition[END_REF], achieving state-of-the-art results in saliency prediction. Jetley et al. [START_REF] Jetley | End-to-end saliency mapping via probability distribution prediction[END_REF] are part of the very few who specifically focused on the design of a loss functions for saliency models. They proposed the use of Bhattacharyya distance and compared it to 4 other probability distances. In this paper, in contrast to [START_REF] Jetley | End-to-end saliency mapping via probability distribution prediction[END_REF], we (1) adopt a principled approach to compare existing loss functions and their combinations and (2) investigate their convergence properties over different datasets and network architectures.

Deep Learning, loss functions and computer vision.

With the application of deep learning techniques to computer vision domain, the choice of appropriate loss function for a task has become a critical aspect of the model training. The computer vision community have been successful in developing task tailored loss functions to improve a model, e.g., encoding various geometric properties for pose estimation [START_REF] Kendall | Geometric loss functions for camera pose regression with deep learning[END_REF], curating loss functions enforcing perceptual properties of vision for various generative deep models [START_REF] Johnson | Perceptual losses for real-time style transfer and superresolution[END_REF], exploiting the sparsity within the structure of problem, e.g., class imbalanced between background and foreground in detection problem, for reshaping standard loss functions and form a new effective loss functions [START_REF] Lin | Focal loss for dense object detection[END_REF]. Our efforts follows the same path to identify the effectiveness of a range of loss functions in saliency prediction.

LOSS FUNCTIONS FOR DEEP SALIENCY NETWORK

Before delving into the description of loss functions, we present the architecture of the convolutional neural network that will be used throughout this paper. After this presentation, we elaborate on the tested loss functions.

Proposed baseline architecture

Figure 1 presents the overall architecture of the proposed model. The purpose of designing a new architecture is only to perform a comparison with existing architectures. Our architecture is based on the deep gaze network of [START_REF] Kümmerer | Deepgaze ii: Reading fixations from deep features trained on object recognition[END_REF] and on the multi-level deep network of [START_REF] Cornia | A Deep Multi-Level Network for Saliency Prediction[END_REF]. The pre-trained VGG-16 network [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF] is used for extracting deep features of an input image (400 × 300) from layers conv3 pool, conv4 pool, conv5 conv3. Feature maps of layers conv4 pool and conv5 conv3 are rescaled to get feature maps with a similar spatial resolution. That feature map with 1280 channels is then fed into a shallow network composed of the following layers: a first convolutional layer allows us to reduce by a factor ten the number of channels, which are then processed by an ASPP (an atrous spatial pyramid pooling [START_REF] Chen | Rethinking atrous convolution for semantic image segmentation[END_REF]) of 4 levels. Each level has a convolution kernel of 3 × 3, a stride equal to 1 and a depth of 32. The dilatation rates are 1, 3, 6, and 12. The ASPP benefit is to catch information in a coarse-to-fine approach while keeping the resolution of the input feature maps. The output of the four pyramid levels are then merged together, i.e. leading to 4 × 32 maps. The last 1 × 1 convolutional layer reduces the data dimensionality to 1 feature map. This map is then smoothed by a Gaussian filter 5 × 5 with a standard deviation of 1. The activation function of these layers is a ReLU activation.

The network was trained over the MIT dataset composed of more than 1000 images [START_REF] Judd | Learning to predict where humans look[END_REF]. We split this dataset into 500 images for the training, 200 images for the validation and the rest for the test. We use a batch size of 60, and the stochastic gradient descent. To prevent over-fitting, a dropout layer, with a rate of 0.25, is added on top the network. The learning rate is set to 0.001. During the training, the network was validated against the validation test to monitor the convergence and to prevent over-fitting. The number of trainable parameters is approximately 1,62 millions. In the following section, we present the different tested loss functions used during the training phase.

Loss functions

Let I : Ω ⊂ R 2 → R 3 an input image of resolution N × M .
We suppose S and Ŝ the vectorized human and the predicted saliency maps, i.e. S and Ŝ are in R N ×M . Let also S f ix be the human eye fixations map, i.e. a N × M image with 1 or 0 pixels. In the following, we present thirteen loss functions L tested in this study. There are classified into four categories according to their characteristics: pixel-based, distributionbased, saliency-inspired and perceptual-based.

Pixel-based loss functions

For pixel-based loss functions L, we assume that S and Ŝ are in [0, 1]. We evaluate the following loss functions:

• Mean Squared Error (MSE) measures the averaged squared error between prediction and ground truth:

L(S, Ŝ) = 1 N × M N ×M i=1 (S i -Ŝi ) 2 (1)
• Exponential Absolute difference (EAD):

L(S, Ŝ) = 1 N × M N ×M i=1 exp(|S i -Ŝi |) -1 (2)
• Absolute Error (AE):

L(S, Ŝ) = 1 N × M N ×M i=1 |S i -Ŝi | (3)
• Weighted MSE Loss (W-MSE):

L(S, Ŝ) = 1 N × M N ×M i=1 w i S i -Ŝi 2 (4)
The weight w i allows to put more emphasis on errors occurring on salient pixels of the ground truth S. Two functions are tested in this paper. In [START_REF] Cornia | A Deep Multi-Level Network for Saliency Prediction[END_REF], authors defined the loss function (MLNET):

w i = 1
α-Si and α = 1.1. Therefore, when S i = 1, the error is multiplied by a factor 10, whereas when S i = 0, the multiplying factor is equal to 0.90. We also consider a weighting function based on a parametric sigmoid function (SIG-MSE):

w i = k 1+exp(-k×(Si-λ))
, where k = 10 and λ varies between 0 and 1.

Distribution-based loss functions

For the distribution-based loss functions, we consider that the vectorized human and the predicted saliency pixels represent a probability to be salient. For that the network presented in section 3.1 is modified in order to output pixel-wise predictions that can be considered as probabilities for independent binary random variables [START_REF] Pan | Salgan: Visual saliency prediction with generative adversarial networks[END_REF]. An element-wise sigmoid activation function is then added as being the last layer. The following loss functions are investigated:

• Kullback-Leibler divergence (KLD) measures the divergence between the distribution S and Ŝ:

L(S, Ŝ) = N ×M i=1 Ŝi log Ŝi S i (5)
• Bhattacharya loss (BHAT) measures the similarity between the distribution S and Ŝ:

L(S, Ŝ) = N ×M i=1 S i Ŝi (6)
• Binary Cross Entropy (BCE) assumes that the saliency prediction Ŝ as well as the ground truth saliency map S are composed of independent binary random variables:

L(S, Ŝ) = - N ×M i=1 S i log Ŝi + (1 -S i ) log(1 -Ŝi ) (7)
• Weighted Binary Cross Entropy (W-BCE): compared to the BCE loss, a global weight w is introduced to consider that there are much more non salient areas than salient areas [START_REF] Wang | Video salient object detection via fully convolutional networks[END_REF]. It allows to put more emphasis on errors occurring when S → 1 and Ŝ → 0 (w >> 0.5) or when S → 0 and Ŝ → 1 (w << 0.5):

L(S, Ŝ) = - N ×M i=1 w × S i log Ŝi +(1 -w)(1 -S i ) log(1 -Ŝi ) (8)
• Focal Loss (FL) : In order to deal with the large foregroundbackground class imbalance encountered during the training of dense detectors, Linet al. [START_REF] Lin | Focal loss for dense object detection[END_REF] modified the binary cross entropy loss function. Such class imbalance is also relevant in the context of saliency prediction, for which the ground truth saliency map mainly consists of null or close to zero, creating a similar phenomenon. The approach is quite similar to W-BCE, except that the weight is locally adjusted and based on a tunable γ power of the predicted saliency. As in [START_REF] Lin | Focal loss for dense object detection[END_REF], we set by default the γ value equal to 2:

L(S, Ŝ) = - N ×M i=1 (1 -Ŝγ i ) × S i log Ŝi + Ŝi γ (1 -S i ) log(1 -Ŝi ) (9)
• Negative Logarithmic Likelihood (NLL): As shown by Kümmerer et al. [START_REF] Kuemmerer | Informationtheoretic model comparison unifies saliency metrics[END_REF], information theory provides strong insights when it comes to saliency models. For instance, in [START_REF] Kümmerer | Deepgaze ii: Reading fixations from deep features trained on object recognition[END_REF], they use maximum likelihood learning. Let I be the set of fixations in an image, and N i the number of those fixations. The logarithm of the prediction at the coordinates of each fixation is then computed :

L( Ŝ) = - 1 N i i∈I log Ŝi (10)

Saliency-inspired loss functions

Saliency predictions are usually evaluated using several metrics [START_REF] Le | Methods for comparing scanpaths and saliency maps: strengths and weaknesses[END_REF]. Those metrics are good candidates to use as loss functions, since they capture several properties that are specific to saliency maps.

• Normalized Scanpath Saliency (NSS): This metric was introduced in [START_REF] Robert | Components of bottom-up gaze allocation in natural images[END_REF], to evaluate the degree of congruency between human eye fixations and a predicted saliency map. Instead of relying on a saliency map as ground truth, the predictions are evaluated against the true fixations map. The value of the saliency map at each fixation point is normalized with the whole saliency map variance:

L(S f ix , Ŝ) = 1 N × M N ×M i=1 Ŝi -µ( Ŝi ) σ( Ŝi ) S f ix i ( 11
)
• Pearson's Correlation Coefficient (CC) measures the linear correlation between the ground truth saliency map and the predicted saliency map :

L(S, Ŝ) = σ(S, Ŝ) σ(S)σ( Ŝ) (12)
where σ(S, Ŝ) is the covariance of S and Ŝ.

Perceptual-based loss functions

We propose two new loss functions for deep saliency, that have been applied with success in image style-transfer problems [START_REF] Johnson | Perceptual losses for real-time style transfer and superresolution[END_REF][START_REF] Ecker | A neural algorithm of artistic style[END_REF]. The idea is to compare the representations of the ground truth and predicted saliency maps that are extracted from different layers of a fixed pre-trained convolutional neural network. The idea behind those losses is to take into account not only the saliency map, but also the deep hidden patterns that could exist, as well as the potential relationship between such patterns. Let φ j (S) be the activation at the j th layer of the VGG network when fed a saliency map S. φ j (S) is then of size C j × H j × W j , where C j represents the number of filters, H j and W j represent the height and width of the feature maps at the layer j, respectively. We also denote J the set of layers from which we extract the representations.

In this work, we extracted the outputs of the 5 pooling layers of a fixed VGG-16 network [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF] pre-trained on the ImageNet dataset, representing a total of 1920 filters:

• Deep Features loss (DF) measures the Euclidean distance between the feature representations:

L(S, Ŝ) = j∈J 1 C j × H j × W j φ j (S) -φ j ( Ŝ) 2 (13)
• Gram Matrices of Deep Features loss (GM) : In order to leverage the potential statistical dependency between features maps, we propose a new loss relying on Gram matrices. For this purpose, we reshape the output φ j (S) into a matrix ψ of size C j × (H j W j ). Then, for each layer j, the Gram matrix G φ j (S) of size C j × C j is defined as follows:

G φ j (S) = 1 C j × H j × W j ψψ (14)
The loss function is then the sum of the squared Frobenius norm of the difference between the Gram matrices G φ j , j ∈ J:

L(S, Ŝ) = j∈J G φ j (S) -G φ j ( Ŝ) 2 F (15)

Center-bias regularization

Since our model does not take into account the center-bias with a learned prior, like in [START_REF] Cornia | Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model[END_REF][START_REF] Cornia | A Deep Multi-Level Network for Saliency Prediction[END_REF], we add a regularization term. We compute a center-bias map B as the mean of the ground truth maps from the training part of the MIT dataset, and add to the loss function the regularization term R i for each pixel i:

R i = α( Ŝi -B i ) 2 (16)
We empirically set the parameter α to 0.1, even though it could be optimized to improve final results. This regularization will later be referred as R in Table 2. The center-bias map B is illustrated in Figure 2.

Linear combinations

All presented loss functions evaluate different characteristics of the predicted saliency maps. We can then hypothesize that a linear combination of some of those loss functions could lead to better results, as it would aggregate the particularities of all measures (a strategy already adopted by [START_REF] Cornia | Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model[END_REF]). We decided to evaluate two linear combinations: the first one (LC 1) combining KLD, CC and NSS loss functions, and the second one (LC 2) adding Deep Features loss, Gram Matrices loss and sigmoid-weighted MSE. This specific combination was chosen because it relies on an existing successful combination and also aggregates the four types of metrics together. We followed the work of [START_REF] Cornia | Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model[END_REF] to set the coefficients for the first linear combination: -1 for the NSS, -2 for the CC, and 10 for the KLD. In the LC 2 combination, we kept those coefficients and set arbitrarily all the other ones as 1. We used the best λ parameter that we tested for the sigmoid-weighted MSE in the linear combination (λ = 0.55).

EXPERIMENTS

Testing protocols

To carry out the evaluation, we use seven quality metrics applied on the MIT benchmark [START_REF] Bylinskii | Mit saliency benchmark[END_REF][START_REF] Le | Methods for comparing scanpaths and saliency maps: strengths and weaknesses[END_REF]:

CC (correlation co- efficient, CC ∈ [-1, 1]), SIM (similarity, intersection be- tween histograms of saliency, SIM ∈ [0, 1]), AUC (Area Under Curve, AU C ∈ [0, 1]), NSS (Normalized Scanpath Saliency, N SS ∈ ]-∞, +∞[), EMD (Earth Mover Distance, EM D ∈ [0, +∞[) and KL (Kullback Leibler divergence, KL ∈ [0, +∞[).
The similarity degree between prediction and ground truth is computed over 299 images.

Performance of the proposed model

Table 1 presents the performance of the proposed model (when trained with the MSE loss function) compared to existing models, i.e. Itti [START_REF] Itti | A model of saliencybased visual attention for rapid scene analysis[END_REF], Rare2012 [START_REF] Riche | Saliency and human fixations: State-of-the-art and study of comparison metrics[END_REF], GBVS [START_REF] Harel | Graph-based visual saliency[END_REF], AWS [START_REF] Garcia-Diaz | Saliency from hierarchical adaptation through decorrelation and variance normalization[END_REF], Sam-ResNet & Sam-VGG [START_REF] Cornia | Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model[END_REF], SalGan [START_REF] Pan | Salgan: Visual saliency prediction with generative adversarial networks[END_REF], ShallowNet & DeepConvNet [START_REF] Pan | Shallow and deep convolutional networks for saliency prediction[END_REF]. All models are evaluated on the test dataset as defined in Section 3.1. According to the evaluation, the proposed model performs rather well and is in the top 4 models. The best performing models are Sam-ResNet and SalGan. Do note however that other models are not specifically trained on the MIT dataset, unlike ours.

Loss function performance

Table 2 presents the performance obtained with the different loss functions.

Which category of loss functions provide the best performances? Except the perceptual-based loss functions, results suggest that the pixel-based, the distribution-based and the saliencyinspired loss functions perform similarly. However, the perceptual-based loss functions we introduced, namely DF and GM, do not perform well individually compared to the aforementioned losses. It might be due to the feature maps used for these losses which are extracted using a deep convolutional network that was trained using natural images, and not saliency maps. The representation of the saliency maps in the feature space might then not be appropriate for this task.

Designing a stronger loss from weaker losses.

Results also suggest that a simple linear combination of well known loss functions increases the ability of the network to predict saliency maps. While keeping the number of trainable parameters unchanged, we succeed in improving up to 14% the correlation coefficient when we compared the best linear combination (CC = 0.7291) to the classical MSE (CC = 0.6388). In a more general way, linear combinations of the loss functions systematically improve the results on most of the metrics. Such fluctuations between the performances of the different loss functions confirm our hypothesis that the choice of the loss function is a critical part of designing a deep saliency model. Moreover, the aggregated loss of KL + CC + NSS + DF + GM improves the SIM, AUC-B and KL scores compared to the aggregation KL + CC + NSS. This reveals the influence of perceptual-based losses (DF+GM) in deep saliency models, and probably calls for future work in this direction. Beyond this quantitative analysis, Figure 3 illustrates predicted saliency maps obtained for some of the tested loss functions. Qualitatively speaking, the saliency maps obtained when using the combined loss KLD + CC + NSS + DF + GM + SIG-MSE + R look very similar to the ground truth maps. For instance, they are very condensed around the salient regions, with little noise.

4.4. Does the best loss generalize well over different datasets and a different architecture?

In this section, we test how well the best loss generalizes over two datasets, i.e. CAT2000 and FiWi, and one architecture, i.e. SAM-VGG.

CAT2000 and FiWi datasets. CAT2000 eye tracking dataset [START_REF] Borji | Cat2000: A large scale fixation dataset for boosting saliency research[END_REF] is composed of 2000 images belonging to 20 different categories whereas FiWi dataset [START_REF] Shen | Webpage saliency[END_REF] is composed of more than 140 screen shots of webpages. Performances are given in Table 3. Results indicate that the loss function based on the linear combination of KLD, CC, NSS, DF, GM, SGI-MSE and R allows to significantly increase the ability to predict visual saliency. Compared to the MLNET-MSE loss function, the gain in terms of CC is 16.3% and 7.1% for CAT2000 and FiWi datasets, respectively.

SAM-VGG with linear combinations of loss functions.

We also retrain SAM-VGG network over the training dataset, as described in Section 3.1, by considering two linear combinaisons (LC 1 and LC 2). Table 4 presents the results.

The results confirm that the linear combination approach improves the prediction and hence generalizes well independently of the dataset and the network architecture. Even if the performances on the test datasets do not reach state-of-theart techniques (see for instance [START_REF] Chang | An element sensitive saliency model with position prior learning for web pages[END_REF] for FiWi), due to the fact that our model was only trained on natural images in MIT, the hierarchy of the loss functions we tested remains consistent, emphasizing the benefit of the linear combination. Figure 4 illustrates a predicted saliency map generated when the SAM-VGG network is trained with a stand-alone loss function, i.e. MLNET-MSE, and with a combination of loss functions.

CONCLUSION

In this paper, we introduced a deep neural network which purpose was to evaluate the impact of loss functions on the pre- diction capacity when it comes to deep saliency models. We evaluated several well-known and commonly used losses, and introduced a new kind of loss function (a perceptual-based loss) that, to the best of our knowledge, has not been applied to saliency prediction. These new loss functions seem to improve, at least partially, the performances of a deep saliency model. Further work on the exact contribution of this kind of losses however still remains necessary. We showed that a simple linear combination of different losses can significantly improve over individual losses, especially when when different types of loss functions are combined (pixel-based, distribution-based, perception-based, saliency-based).

More importantly we showed that this combination strategy generalizes well on different datasets and also with a different deep network architecture. Optimization on the coeffi- cients of those linear combinations is also possible to obtain the best performances possible out of the combination. This approach could moreover easily be extended to other kinds of architectures, not necessarily based on convolutional neural networks.

Finally, one of the main idea that motivated our work was to highlight the importance of the choice of the loss function.

We showed that a careful design of the loss function can significantly improve the performances of a model without increasing the number of trainable parameters.

Fig. 1 :

 1 Fig. 1: Architecture of the proposed deep network.

Fig. 2 :

 2 Fig. 2: Averaged colored saliency map of the training part of MIT dataset. Horizontal and vertical marginal distributions are also plotted, illustrating the center bias.

Fig. 3 :

 3 Fig. 3: (a) Visual stimulus; (b) Ground truth saliency map; (c) KLD + CC + NSS + DF + GM + SIG-MSE + R combination; (d) NSS; (e) KL + CC + NSS + R; (f) MLNET-MSE; (g) KLD; (h) SIG-MSE (λ = 0.55); (i) CC.

Fig. 4 :

 4 Fig. 4: Example of good predictions by the combination loss while a single loss makes bad predictions (for SAM-VGG model). (a) original image; (b) Ground truth saliency map; (c) KLD + CC + NSS + DF + GM + SIG-MSE + R combination (CC = 0.8681 and 0.7967); (d) MLNET-MSE (CC = 0.4320 and 0.4491) .

Table 1 :

 1 Performance of our model. Best performances are in bold. (AUC-B=AUC-Borji; AUC-J=AUC-Judd)

	Itti	0.28	0.35	0.71	0.71	0.79
	Rare2012	0.46	0.43	0.78	0.79	1.40
	GBVS	0.49	0.44	0.81	0.81	1.38
	AWS	0.39	0.40	0.75	0.76	1.22
	Sam-ResNet	0.68	0.60	0.85	0.79	2.43
	SalGan	0.70	0.58	0.86	0.86	2.17
	Sam-VGG	0.64	0.57	0.85	0.78	2.19
	ShallowNet	0.59	0.47	0.83	0.84	1.62
	DeepConvNet	0.60	0.49	0.84	0.85	1.73
	Our model	0.64	0.45	0.81	0.84	2.04
	(ranking)	3/10	5/10	6/10	3/10	4/10

CC ↑ SIM ↑ AUC-J ↑ AUC-B ↑ NSS ↑

Table 2 :

 2 Performance of the loss functions. Best performances are in bold and the second and third best performances are in italic. R represents the center-biais regularization. (AUC-B=AUC-Borji; AUC-J=AUC-Judd)

	CC ↑	SIM ↑ AUC-J ↑ AUC-B ↑ NSS ↑ EMD ↓	KL ↓

Table 3 :

 3 Performance of proposed model over CAT2000 and FiWi datasets with MLNET-MSE (W-MSE), KLD + CC + NSS + R (LC 1) and KLD + CC + NSS +DF +GM + SIG-MSE (λ = 0.55) + R (LC 2). Best performances are in bold. (AUC-B=AUC-Borji; AUC-J=AUC-Judd)

		CC ↑	SIM ↑ AUC-J ↑ AUC-B ↑ NSS ↑
	CAT2000
	W-MSE	0.508	0.4017	0.8221	0.8016	1.9486
	LC 1	0.5535 0.4261	0.8273	0.8187	1.9332
	LC 2	0.5937 0.4203	0.8309	0.8372	1.9375
	FiWi
	W-MSE	0.3954 0.3872	0.7312	0.7114	0.8050
	LC 1	0.4118 0.4157	0.7621	0.7390	0.8214
	LC 2	0.4236 0.4183	0.7636	0.7681	0.8205

Table 4 :

 4 Performance of SAM-VGG over MIT dataset with MLNET-MSE (W-MSE), KLD + CC + NSS + R (LC 1) and KLD + CC + NSS +DF +GM + SIG-MSE + R (LC 2).Best performances are in bold. (AUC-B=AUC-Borji; AUC-J=AUC-Judd)

		CC ↑	SIM ↑ AUC-J ↑ AUC-B ↑ NSS ↑
	MIT
	W-MSE 0.7351 0.6769	0.8521	0.7884	2.0037
	LC 1	0.7499 0.6502	0.8635	0.7912	2.1694
	LC 2	0.7511 0.6472	0.8712	0.8017	2.0741