Tanja Heuer
email: ta.heuer@ostfalia.de

Ina Schiering
email: i.schiering@ostfalia.de

Reinhard Gerndt
email: r.gerndt@ostfalia.de

Privacy and Socially Assistive Robots -A Meta Study

Keywords: socially assistive robot, Robot Ethics, HRI, Human-Robot Interaction, Human-Robot Friendship, Machine Ethics, Privacy

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Some years ago robots were exclusively present in research laboratories and industrial facilities. Because of a miniaturization of components and the decline in price, robots are increasingly used for personal applications as e.g. smart home applications and social interaction now. According to the International Federation of Robotics (IFR) "about 5.4 million service robots for personal and domestic use were sold [in 2015], 16% more than in 2014." [24]. Two thirds of those "are robots for domestic tasks, including vacuum cleaning, lawn-mowing, window cleaning and other types" [24] and it is estimated that these numbers will rise up to 30 million until 2019. "Sales of robots for elderly and handicap assistance will be about 37,500 units in the period of 2016-2019. This market is expected to increase substantially within the next 20 years" [24].

Tasks of domestic robots (for elder care) are mainly categorized into three areas [START_REF] Sharkey | Granny and the robots: Ethical issues in robot care for the elderly[END_REF]: "(1) to assist the elder[s], and/or their carers in daily tasks; [START_REF] Banks | Animal-assisted therapy and loneliness in nursing homes: use of robotic versus living dogs[END_REF] to help monitor their behavior and health; and (3) to provide companionship". The aim of social robots is a (semi-) autonomously interaction with humans in a respectful way [START_REF] Lee | Can robots manifest personality? an empirical test of personality recognition, social responses, and social presence in human?robot interaction[END_REF]. Several of the intended domestic tasks require sensors as e.g. cameras or microphones to allow individualized user interactions. Users of domestic robots expect a conversation in a "natural human-like manner" [START_REF] De Graaf | What makes robots social? a user's perspective on characteristics for social human-robot interaction[END_REF]. Because of these user expectations, robots are often considered as living beings like humans or animals and are treated as friends that accompany humans in their private environment [START_REF] De Graaf | An ethical evaluation of human-robot relationships[END_REF]. Robots will be dangerously ubiquitous, but, as for other smart devices as e.g. smart phones, wearables, etc. that accompany users in their daily life, the privacy risks are not obvious. Hence, privacy should be an important aspect in robotics, but is not taken into consideration in a satisfactory way yet. At the moment robots are designed from an engineering perspective with a main focus on functionality. Privacy and other ethical aspects of robotics are investigated in general, but not adequately integrated in the engineering process of robotics.

In this paper the role of privacy in "Robotic Ethics" is investigated in the context of user studies with assistive robots. There the perception concerning privacy of different user groups is investigated and the effect of the type of robot on the perception of privacy is examined. Approaches to make privacy risks more transparent are discussed as a first step to foster the consideration of privacy in robot design processes.

The considered so called Socially Assistive Robots (SAR) together with an approach for categorization are described in section 3. An overview of ethical concerns towards robotic is presented in section 4. The role of privacy in this ethical context is emphasized. In section 5 an overview about Socially Assistive Robots is given and privacy concerns of users are stated and evaluated in section 6 followed by discussion and conclusion.

Methodology

The basis of the meta study is a literature survey. The retrieval was based on queries in databases and search engines for scientific publications, i.e Google Scholar and ScienceDirect. In addition, the following relevant conferences were considered ISCR (International Conference on Social Robotics), ISRR (International Symposium on Robotics Research) and RO-MAN (Robot and Human Interactive Communication). Main key words as "social robot", "robot/robotic and privacy", "robot/robotic study" and "robot/robotic and ethical aspects" were used. In addition, other surveys as Leite et al. [START_REF] Leite | Social robots for long-term interaction: a survey[END_REF] who already gave an overview of social robots for long-term interactions, are considered. Because of an increasing use of robots for home and health care the collection of papers has been limited to the last ∼10 years.

Investigated robots are sorted into categories explained in section 3. The categorization which is adopted by Fong et al. [START_REF] Fong | A survey of socially interactive robots[END_REF] points out the different types of robots and their appearances. Based on this categorization, it should be examined whether the perception of privacy depends on the appearance or on specific functionalities. Further we want to investigate, how participants and users perceive privacy issues and how users express their concerns in the context of the studies.

For each paper the duration, target group and the number of participants is described and summarized in Figure 5. Duration gives an overview of the length of the studies. If a study for example just lasts one hour, the user may not think about privacy risks of the investigated robot. Also the target group is important for the assessment of human-robot interaction (HRI), since children might interact in a different way with robots than older people. The number of participants is also important. Studies with a larger number of participants could achieve more significant results about the perception of a robot.

The main focus in this meta study is on the ethical, especially privacy concerns, which are mentioned by the users during the HRI studies. With the introduction of individualized conversation between robots and users, the utilization of sensors, e.g. cameras and microphones in addition to standard sensors for autonomous behavior as ultrasonic sensors or laser range scanners, increases. But even though sensors use is increasing, it is not transparent to the user which data is recorded, stored and transmitted. In the studies users express skepticism, interest and curiosity. It is interesting to investigate, which of these attitudes prevails.

Socially Assistive Robots

In the context of this study social robots are in the focus. A robot is called social, when it is able to react to human actions in a (semi-) autonomously way either by speech or movements. Movement in this sense includes moving only the head or another part of the robot as an arm or a leg. The area of socially assistive robots (SAR) is defined as "a class of robots that is the intersection of assistive robotics [...] and socially interactive robotics [...]" [START_REF] Feil-Seifer | Socially assistive robotics[END_REF]. Hence, socially assistive robots are able to substantially support human beings in many areas of their daily life: companionship, entertainment, security, transportation, education, customer service, personal assistance, sales and tourist guidance [START_REF] Sharkey | Granny and the robots: Ethical issues in robot care for the elderly[END_REF]. A huge variety of robots are developed to realize one or more of these tasks. Robots in this sense are divided into four categories defined by Fong et al. [START_REF] Fong | A survey of socially interactive robots[END_REF] as described in the following.

Functional The focus of this category is the practical benefit of the robots. These robots are designed with the focus on the intended use. For instance, requirements in health care like a storage space or a removable tray need to be taken into account for the construction [START_REF] Pineau | Towards robotic assistants in nursing homes: Challenges and results[END_REF]. As an example the robot Pillo serves as a reminder of medication (see Fig. 1). It is able to store medicine and to dispense the medicine depending on the needs of the user [START_REF] Amirfar | A little robot with big promise may be future of personalized health care[END_REF]. Therefore it has a closed compartment for the medicine with restricted access and a special place is designated for a glass of water. Although it is designed in an abstract functional way, it shows eyes on its screen to look sympathetic.

Caricatured Cartoons and caricatures are the background of this category. In cartoons, for example, characteristic features of a creature are particularly emphasized. By highlighting these features, other elements are neglected. The attention of the user should be focused on the emphasized part. Scheeff et al. used cartooning techniques to design the robot Sparky [START_REF] Scheeff | Experiences with sparky, a social robot[END_REF]. This robot looks like a small turtle with focus on the face and the facial expressions. Another example is the Care-O-bot 4 (see Fig. 2). It is equipped with arms for fetch-and-carry tasks and wheels instead of legs for moving. The head is a touch screen and it represents a face by showing abstract eyes as the Pillo.

Zoomorphic A third design of robots is the imitation of zoomorphic creatures.

There the robot design is either based on realistic or fantasy animals. In the context of children and elder care animal-like creatures are often used, eg. Paro and Aibo for entertainment. Fong et al. [START_REF] Fong | A survey of socially interactive robots[END_REF] suggest that for humans it is easier to build relationships to animals and interactions are possible on a lower level. These robots often do not have a wide range of functionalities but mainly act as a companion. Paro, a seal robot is able to move its head, tail and the eyes when it is touched (see Fig. 3). Additionally it is able to express a sense of well-being by sounds.

Anthropomorphic Anthropomorphism signifies the attribution of human appearances, behaviors and other characteristics to non-human-objects. The aim of anthropomorphic robots is to make human like actions of a robot easier to understand for human beings [START_REF] Duffy | Anthropomorphism and the social robot[END_REF]. A meaningful human-robot interaction in a social way is supposedly easier when the robot has a physical and a mental personification [START_REF] Breazeal | Designing sociable robots (intelligent robotics and autonomous agents)[END_REF][START_REF] Scassellati | Foundations for a Theory of Mind for a Humanoid Robot[END_REF]. DiSalvo et al. [START_REF] Disalvo | All robots are not created equal: the design and perception of humanoid robot heads[END_REF] suggest "that interaction through speech and movement will greatly effect the perception of humanness in robots". With the humanness of a robot, other deficits can be covered up.

Ethical Concerns in SAR and the Role of Privacy

In this section different concepts of ethical concerns in SAR are introduced. In 1942 Isaac Asimov created the first general three laws of robotics stating that a robot must obey a human being and is not allowed to hurt human beings.

With increasing interest in the use of robots in various application areas, these rules were further specified in 2009 [START_REF] Murphy | Beyond asimov: the three laws of responsible robotics[END_REF]. These two approaches comprise mainly a professional standard for a safe human-robot interaction but specific ethical aspects of this safe interaction are not considered. A detailed overview of ethical concerns is proposed by Sharkey & Sharkey [START_REF] Sharkey | Granny and the robots: Ethical issues in robot care for the elderly[END_REF] and Coeckelbergh et al. [START_REF] Coeckelbergh | A survey of expectations about the role of robots in robot-assisted therapy for children with ASD: Ethical acceptability, trust, sociability, appearance, and attachment[END_REF] focused on specific topics in health care. Whereas Sharkey & Sharkey [START_REF] Sharkey | Granny and the robots: Ethical issues in robot care for the elderly[END_REF] discuss concerns of using robots in elder care, Coeckelbergh et al. [START_REF] Coeckelbergh | A survey of expectations about the role of robots in robot-assisted therapy for children with ASD: Ethical acceptability, trust, sociability, appearance, and attachment[END_REF] investigate ethical concerns of using robots in therapeutic childcare (see Fig. 4).

These detailed approaches show in general a broad consensus. Aspects like loss of privacy and data protection are mentioned. Also the ethical acceptability towards robots is introduced [START_REF] Coeckelbergh | A survey of expectations about the role of robots in robot-assisted therapy for children with ASD: Ethical acceptability, trust, sociability, appearance, and attachment[END_REF]. Several of these ethical concerns comprise also privacy aspects. For example, the personal liberty and the self-control of personal thinking and acting needs to be guaranteed. If the robot is regarded as a friend, emotions and attachment play a role. A friend is more entrusted than a machine. Therefore this needs to be investigated in a more detailed way.

Denning et al. [START_REF] Denning | A spotlight on security and privacy risks with future household robots: attacks and lessons[END_REF] developed a questionnaire of considerable fundamental secure and privacy-respecting issues for the development of household robots. They focused on network security and possible attacks, e.g. spying on homes, acquisition of login credentials. The questions are divided into Social, Environmental, Technical Questions and Security and Privacy Questions (see Table 1).

Studies about (Socially) Assistive Robots

This section describes studies about assistive robots of the last decade with a focus on ethical concerns and privacy aspects mentioned by the participants. To answer the questions about user groups, the overview of the studies is divided into categories shown in Figure 5 on the x-axis. Participants are divided into children, families, older people and mixed participant groups. The y-axis lists the different types of robots investigated. The duration and the number of participants of the studies is represented by icons. If a robot is used for a longer time by the participants, more interaction is possible and the user is able to build up a personal attitude towards the robot. The number of participants is an important indication of the significance of the study.

Functional

Families: Studies with the vacuum cleaner Roomba are all carried out in family environments. Families comprise in this context single households, households with children or couples. These studies were conducted in 2007 [START_REF] Breazeal | How Robotic Products Become Social Products: An Ethnographic Study of Cleaning in the House[END_REF], 2009 [START_REF] Sung | Robots in the wild: understanding longterm use[END_REF], 2011 [START_REF] Fink | People's perception of domestic service robots: Same household, same opinion?[END_REF] and 2013 [START_REF] Fink | Living with a vacuum cleaning robot[END_REF] and evaluated the change of cleaning routines and activities over time. Ethical concerns are mentioned by Fink et al. [START_REF] Fink | Living with a vacuum cleaning robot[END_REF] stating "When people did not trust/rely on the robot, they did not want to leave the room/home when the robot was switched on". Even if the robot has no camera or microphone "[...] the Roomba changed people's cleaning activities [...]" [START_REF] Breazeal | How Robotic Products Become Social Products: An Ethnographic Study of Cleaning in the House[END_REF].

Older People: Beer et al. [START_REF] Beer | Mobile remote presence systems for older adults: acceptance, benefits, and concerns[END_REF] conducted a study on older people using the MRP (mobile remote telepresence system). Twelve older adults took part in this study which consisted of two sessions. The goal was the improvement of social communication with other people. The robot Cafero is used in a residential home by residents and staff for about 30 minutes [START_REF] Broadbent | Human-robot interaction research to improve quality of life in elder care -an approach and issues[END_REF]. It provides medication management based on web services and uses a camera for face recognition. There are no concerns mentioned and the researchers ensured the confidentiality of the data and employed anonymization techniques. In 2016 the PeopleBot was used and contrasted to Care-o-Bot in a study at the University of Hertfordshire on older people [START_REF] Syrdal | Views from within a narrative: Evaluating long-term human-robot interaction in a naturalistic environment using open-ended scenarios[END_REF]. The robot e.g. offers functionalities as video telephony and the participants were allowed to use the robots for whatever they want to and whenever they like.

Mixed: Other studies also at the University of Hertfordshire [START_REF] Koay | Living with robots: Investigating the habituation effect in participants' preferences during a longitudinal human-robot interaction study[END_REF][START_REF] Koay | Five weeks in the robot house-exploratory human-robot interaction trials in a domestic setting[END_REF][START_REF] Syrdal | He knows when you are sleeping-privacy and the personal robot companion[END_REF] explored the habituation effects of different participant groups working with the PeopleBot. Within the five weeks the preferences of the participants changed because they got used to the robot. Another important aspect is the fear of the loss of control mentioned by participants because of the autonomous behavior of PeopleBot [START_REF] Koay | Living with robots: Investigating the habituation effect in participants' preferences during a longitudinal human-robot interaction study[END_REF]. In a further investigation the researchers focused on attitudes towards privacy. The participants attended a session where the concerns of using personal robots were in the focus. As main issues the storage of personal information and which data is stored are mentioned. Other concerns regarding usability and applicability are stated [START_REF] Syrdal | He knows when you are sleeping-privacy and the personal robot companion[END_REF]. Kidd et al. [START_REF] Kidd | Robots at home: Understanding long-term human-robot interaction[END_REF] introduced a robot called Autom for helping people to reduce their weight. A camera is included for face recognition and the conversation between robot and the human is personalized based on prior usage. The ACE (Autonomous City Explorer) [START_REF] Weiss | Robots asking for directions: the willingness of passers-by to support robots[END_REF] was driving through Munich, Germany for one day asking persons for the right way. Equipped with a stereo vision camera, a touch monitor and a loudspeaker it interacted with passers-by. No ethical aspects were mentioned. A study with Spunik [32] evaluated the differences of HRI with an avatar (i.e. virtual representation of a robot on a screen) or a physical robot. Participants expressed privacy as one of their main ethical concerns during that study. To work effectively as assistant, the robot needs a lot of knowledge about the user. Another important aspect is the "damage of the privacy space", when the robot is able to move on its own.

Caricatured

Children: RoboVie is used in different studies of Kanda et al. [START_REF] Kanda | Interactive robots as social partners and peer tutors for children: A field trial[END_REF][START_REF] Kahn | robovie, you'll have to go into the closet now: Children's social and moral relationships with a humanoid robot[END_REF]. RoboVie acts for example as teacher for children in Japan [START_REF] Kanda | Interactive robots as social partners and peer tutors for children: A field trial[END_REF] or as friend in Washington [START_REF] Kahn | robovie, you'll have to go into the closet now: Children's social and moral relationships with a humanoid robot[END_REF], but no concerns are brought up by the users. Kanda et al. [START_REF] Kahn | robovie, you'll have to go into the closet now: Children's social and moral relationships with a humanoid robot[END_REF] attribute the ability of friendship to the robot. But already in an earlier study [START_REF] Kanda | Interactive robots as social partners and peer tutors for children: A field trial[END_REF] it is shown, that children lost their interest in the robot after a certain amount of time.

Older People: Scitos [START_REF] Döring | Robotic companions for older people: A case study in the wild[END_REF] interacted with people older than 60. The study investigates the idea of a robot as companion. The only hint towards ethical concerns is mentioned as an idea for future work: "Will there be effects of habituation Mixed: RoboVie was investigated in the context of shopping mall by Kanda [START_REF] Sabelli | Robovie as a mascot: A qualitative study for long-term presence of robots in a shopping mall[END_REF]. There it could be used as a shopping assistant for everyone in Japan for three years. As an interesting result most of the adults only tried to interact with the robot when their children wanted to make use of RoboVie. Also Careo-Bot assisted in an electronic market where it should show customers the way to products they asked for [START_REF] Schmitt | Wie wirkt der care-o-bot 4 im verkaufsraum? Mensch und Computer 2017-Usability Professionals[END_REF].

Zoomorphic

Children: Aibo is used in different studies on children. Kahn et al. [START_REF] Kahn | Robotic pets in the lives of preschool children[END_REF] tested the impact on children in preschool, Weiss et al. [START_REF] Weiss | i love this dogchildrens emotional attachment to the robotic dog aibo[END_REF] evaluated "first time reactions in HRI" in a shopping mall and Stanton et al. [START_REF] Stanton | Robotic animals might aid in the social development of children with autism[END_REF] tried to support autistic children. All the interventions last less than one hour per interaction and no privacy aspects are stated. Weiss et al. [START_REF] Weiss | i love this dogchildrens emotional attachment to the robotic dog aibo[END_REF] point out the goal of "create[ing] an awareness for robots in general". iCat [START_REF] Leite | As time goes by: Long-term evaluation of social presence in robotic companions[END_REF] interacts with children during a chess game. The robot is able the show facial expressions and has a camera in its nose.

Although it is a study investigating children no ethical aspects are mentioned. Kozima et al. introduced a robot called Keepon [START_REF] Kozima | Keepon[END_REF]. Equipped with a camera and a microphone it is able to react on childrens behavior in a non-verbal way.

Families: Another toy robot is Pleo [START_REF] Fernaeus | How do you Play with a Robotic Toy Animal? A long-term study of Pleo[END_REF], a dinosaur. It has a camera and two microphones. Six families lived with Pleo from two up to ten month. Important statements of users were "no long-term interest" to use the robot and "with another kind of toy, it may not have been a problem to claim that it sees, but several parents in this study seemed to take such information more literally as technical features.". Parents expressed concerns about the cameras, instead of just accepting them as eyes of the robot. Another study employed the rabbit robot Karotz [START_REF] De Graaf | Why do they refuse to use my robot?[END_REF]. Karotz has a camera, a microphone and is able to connect to the internet although the internet connection is not really used.

Older People: Paro is a small seal robot with tactile and noise sensors. Wada et al. [START_REF] Wada | Living with seal robots-its sociopsychological and physiological influences on the elderly at a care house[END_REF][START_REF] Wada | Long-term robot therapy in a health service facility for the aged-a case study for 5 years[END_REF] investigate how older people with dementia react on that robot. Even though Paro has limited functionalities, one old lady prefers playing with Paro rather than with other habitants of the retirement house. Additionally, the psychological effects [START_REF] Robinson | The psychosocial effects of a companion robot: a randomized controlled trial[END_REF] and the general effects in care with older people of using Paro were investigated without significant results [START_REF] Wagemaker | Advances in mental health care: Five n= 1 studies on the effects of the robot seal paro in adults with severe intellectual disabilities[END_REF]. In 2010 a series of studies with the Karotz started in the Netherlands. Two studies are conducted with people older than fifty [START_REF] Klamer | adventures of harvey-use, acceptance of and relationship building with a social robot in a domestic environment[END_REF][START_REF] De Graaf | Sharing a life with harvey: Exploring the acceptance of and relationship-building with a social robot[END_REF] to get an overview of how older people use robots. Ethical aspects are not mentioned, but users did not like to interact with the robot, because of the lack of functionality. A study with Aibo investigated the differences in the interaction with a robot and a real dog [START_REF] Banks | Animal-assisted therapy and loneliness in nursing homes: use of robotic versus living dogs[END_REF]. The robot iCat was used to investigate the acceptance of robots as companions with 30 participants (22 female, 8 male) [START_REF] Heerink | The influence of social presence on acceptance of a companion robot by older people[END_REF].

Anthropomorphic

Children: Kaspar, a robot with cameras in the form of eyes, played a game with six children under ten years with "low-level socially communicative behaviors" [START_REF] Wainer | A pilot study with a novel setup for collaborative play of the humanoid robot kaspar with children with autism[END_REF]. The session lasts less than one hour. Again no ethical aspects are discussed. Another study with one autistic child was made with a NAO robot [START_REF] Shamsuddin | Initial response in hri-a case study on evaluation of child with autism spectrum disorders interacting with a humanoid robot nao[END_REF] for a session of 14 minutes. The robot has four directional microphones, loudspeakers, two cameras and can connect to the Internet autonomously. Concerning ethical aspects it is said that "[...] proven reliability and safety need to be gained in advance" and "safeguarding the wellbeing [...] is crucial to ensure that their rights are always protected", but without any further explanation.

Privacy and Robotics

In this section the perception of privacy in studies about social robots is discussed and contextualized in the categories of types of robots and user groups.

Robot Groups Functional: A lot of studies are conducted in a public space. Users have typically not much time to interact with a robot. Mainly during studies in a home context, ethical concerns are identified. People are changing their habits, even though the robot being in their home has no sensors as cameras or microphones. The reason may be the robots functional appearance. Roomba is considered as a device that can move independently and users do not have control over it [START_REF] Fink | Living with a vacuum cleaning robot[END_REF].

Caricatured: RoboVie and Care-o-Bot were used during several studies in different areas. But for the variety of functionalities there are only very limited examples of real use. Many of the available functionalities are not used or problematic [START_REF] Schmitt | Wie wirkt der care-o-bot 4 im verkaufsraum? Mensch und Computer 2017-Usability Professionals[END_REF]. The Care-o-Bot was e.g. of limited use in an electronic market and because of interfering noise. Additionally, a laboratory study was Users described the robot as interesting, but not very useful. Cafero was designed explicitly for the requirements of users but the study does not give any hints about the user perception. Instead the researchers list general issues of robotic studies as risks, quality of life (QoL), acceptance or confidentiality [START_REF] Broadbent | Human-robot interaction research to improve quality of life in elder care -an approach and issues[END_REF].

Zoomorphic: A lot of robots are already developed in this area. In the study with Pleo [START_REF] Fernaeus | How do you Play with a Robotic Toy Animal? A long-term study of Pleo[END_REF] the participants treated it as a real animal and were disappointed of restricted functionalities. Despite the fact, that Karotz has a microphone and a camera, its functionalities are very limited. However, Paro is an already accepted, commercially available robot in elder care. Regardless of the minimal set of functionalities, it is able to react and react to peoples actions in a satisfying way. Apart from Paro, there are not many zoomorphic robots that are accepted or can be used satisfactorily, independent of the user group.

Anthropomorphic: This category of robots is recently developed. Because of limited availability and the costs user interaction in studies is very restricted and it is not possible to investigate this area.

User Groups In several studies, users point out their concerns of using robots. Either they do not want to use a robot at all because of privacy concerns regarding data collection and the loss of private space. Or the users are afraid of using a robot because of a lack of technical expertise. Some users accept the robot in their household but change their behavior because of the fear of surveillance. Robot engineers often state that privacy is an important aspect in robotics but there is no typically further information how concerns should be addressed and how privacy might be achieved. Koay et al. [START_REF] Koay | Five weeks in the robot house-exploratory human-robot interaction trials in a domestic setting[END_REF] state that participants were glad, that the robot learns helpful information for improving functionality, but in a related study participants expressed their concerns about sensitive information [START_REF] Syrdal | He knows when you are sleeping-privacy and the personal robot companion[END_REF].

Children: Interestingly, in studies on children ethical concerns are rarely considered. It is pointed out that privacy is important and respected, but concerns expressed by parents are not described. Children do not have the same perspective as adults and they have a more intuitive way of interacting with technology. Additionally, it is stated that children loose interest in robots after a certain time.

Families: Until now, not many studies were conducted in family homes, because robots are not sufficiently robust, as stated by Graaf et al. [START_REF] De Graaf | Why do they refuse to use my robot?[END_REF].The Roomba studies also incorporate families. In the case of Pleo, parents had concerns about accepting cameras as eyes [START_REF] Fernaeus | How do you Play with a Robotic Toy Animal? A long-term study of Pleo[END_REF].

Older People: Since elder care is one of the two main areas in social robotics, many studies are available. Although the studies do not directly focus on privacy, many users express concerns about various aspects of the area. Especially older people seem to be more sensitive with respect to robots. In the study of Graaf et al. [START_REF] De Graaf | Sharing a life with harvey: Exploring the acceptance of and relationship-building with a social robot[END_REF] users were able to decide by clicking a button whether they agree to be filmed but it was not transparent if the robot is recording or not.

Discussion

Pillohealth4 realizes personalized medication with the help of Pillo via face recognition. It promotes a secure storage of the medication and an additional mobile application is provided. But what happens in case of errors of the face recognition procedure? What happens, if attackers can manipulate Pillo which is connected to the internet? These questions should not only be raised for Pillo, but for all robots in a social context.

Currently, there are not a lot of robots commercially available [START_REF] Broadbent | Acceptance of healthcare robots for the older population: review and future directions[END_REF]. Paro, Roomba and lawnmowers are mainly the only robots on the market. An important result of the studies is that a lot of users are not convinced and satisfied of using robots so far. Researchers still have to face a lot of problems as speech and image recognition and restricted functionalities. Additionally, participants stated a lot of concerns regarding sensors like cameras or microphones, storage of personal data or autonomous behavior. In most of the cases, clearly defined ethical principles are missing. Even though some robot features are not as sensitive as medication, the ethical consequences of every single feature need to be investigated. It is important for users, to be aware of the privacy risks of robots.

Therefore more transparency and a thorough consideration of privacy issues is essential to raise user acceptance. To figure out which aspects need to be dealt with, the features of robots need to be investigated. As already proposed in the questionnaire 1, sensors and actors should be considered because of the inherent risk. By using such a questionnaire, users might be able to understand the risks and possible countermeasures. Because the questionnaire has a focus on network security, it is important to address privacy aspects in addition. The model of seven types of privacy is a general approach, to investigate ethical aspects with a focus on privacy [START_REF] Finn | Seven types of privacy[END_REF]. The following Table 2 gives an overview based on typical sensors employed in robotics. Depending on sensors and accompanying functionalities, different types of privacy are violated, e.g. with a camera, a robot is able to track people and their environment. This may influence users in their behavior [START_REF] Fink | Living with a vacuum cleaning robot[END_REF]. Especially in the care area, this must be clearly transparent [START_REF] De Graaf | Sharing a life with harvey: Exploring the acceptance of and relationship-building with a social robot[END_REF]. Concerning these aspects further correlation to other smart devices need to be investigated.

Another important privacy issue is the fact of Thoughts and Feelings. Robots are described as a friend or companion in many of the studies. That implies blind trust and an absolutely openness towards them [START_REF] De Graaf | An ethical evaluation of human-robot relationships[END_REF]. Through the humanization of human-robot interaction, the topic of privacy gains even more importance. Cynthia Breazeal introduced the relationship between emotional engagement and functional utility [START_REF] Breazeal | A social robot in every home[END_REF][START_REF] Heerink | The influence of social presence on acceptance of a companion robot by older people[END_REF]. The more functionality a robot has, the more intimate is the emotional binding. The more functional it looks like, the more it is seen as a machine and not as a companion . Nevertheless, appearance of a robot is not the main characteristic for being skeptical.

In general, robots should not be too complex but more useful and helpful for daily routines. To address privacy concerns, transparency is needed concerning collection of personal data and it is important that the user is able to intervene, e.g. by switching off certain features. A lot of participants criticized the available features and missed some helpful ones. A robot e.g. does not need a camera when the functionalities using the camera are not perceived beneficial. Therefore it is of utter importance to involve potential users in robot design [START_REF] Broadbent | Human-robot interaction research to improve quality of life in elder care -an approach and issues[END_REF][START_REF] Lee | Steps toward participatory design of social robots: Mutual learning with older adults with depression[END_REF]. Users should be asked about preferences and aversions. This would ensure whether a specific sensor is really needed or not and thereupon a more specific analysis is possible.

Conclusion

Summarized, only a few ethical aspects have been investigated in robot studies so far. There are already some doubts mentioned regarding reduction of human contact, loss of control and loss of privacy related to recordings. The concern of emotions & attachments is not seen as problem so far.

Privacy should be a broad and important topic in robotic research. Zoomorphic robots, preferably used in health care are mostly simplistic but on the other hand very successful. With increasing demands, the complexity of robots will in-crease. Cameras, microphones and internet access will be a standard for robots. This allows an enlargement of robot functionalities, especially concerning social conversation and interaction between user and robot. Therefore the parallels to already available intrusive technical devices like mobile phones and their mobile applications are important to investigate. But for robots to be successful on the market, it must be helpful. If the robots have sensors it must be clear, why and for what reasons and scenarios they are needed. The two main target groups for social robots children and older people (with physical/psychological deficits) might have difficulties to be aware of the risks using them and the high complexity. Therefore risks and requirements need to be investigated during the whole development process to get a more transparent view.

Fig. 1 .

 1 Fig. 1. Pillo 1

Fig. 2 . 2 Fig. 3 .

 223 Fig. 2. Care-O-bot 4 2 Fig. 3. Paro 3

Fig. 4 .

 4 Fig. 4. Overview of General Robotic Laws and Ethical Concerns

Fig. 5 .

 5 Fig. 5. Overview of Robot Studies

Table 1 .

 1 Questionnaire by Denning et al.[START_REF] Denning | A spotlight on security and privacy risks with future household robots: attacks and lessons[END_REF]

	Social, Environmental, and	Security and Privacy Questions
	Technical Questions
	What is the intended function of the	Does the robot create new or amplify
	robot?	existing privacy vulnerabilities?
	How mobile is the robot?	Does the robot create new or amplify
	What actuators does the robot possess? existing physical integrity vulnerabilities?
	What sensors does the robot possess?	Does the robot create new or amplify
	What communication protocols does the	existing physical safety vulnerabilities?
	robot support?	Does the robot create new or amplify
	Who are the intended users of the robot?	existing psychological vulnerabilities?
	What is the robots intended operational	Can the robot be combined with other
	environment?	robots or technologies to facilitate an
	Besides the intended users of the robot,	attack?
	what other people (and animals) will be
	in the the robots environment?
	What kind of development processes are
	in place?

Table 2 .

 2 Types of Privacy and the impact of Technical Conditions

	Privacy of	Camera	GPS	Loud-	Micro-	Tactile	WiFi
	...			speaker	phones	Sensors
	the Person
	Behavior	X	X	X	X	X
	and Action
	Communi-			X	X	X	X
	cation
	Data and	X			X		X
	Image
	Thought			X		X
	and
	Feeling
	Location	X	X
	and Space
	Association	X			X		X

https://assets.entrepreneur.com/content/3x2/1300/20160630152708-pillo.jpeg

http://www.care-o-bot-4.de/

https://www.thestar.com/life/breakingthrough/2014/06/09/robot gets seal of approval.html

http://pillohealth.com/

Acknowledgment This work was supported by the Ministry for Science and Culture of Lower Saxony as part of the program "Gendered Configurations of Humans and Machines (KoMMa.G)".


Felix Bieker

Nicholas Martin

Michael Friedewald

Marit Hansen
email: fbieker|marit.hansen@datenschutzzentrum.de

Data Protection Impact Assessment: A Hands-On Tour of the GDPR's Most Practical Tool

Keywords: Data Protection Impact Assessment, Risk to Rights and Freedoms, General Data Protection Regulation, Data Protection, EU law

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The General Data Protection Regulation (GDPR) will replace the Data Protection Directive on 25 May 2018. Among the regulatory and governance instruments it introduces is the Data Protection Impact Assessment (DPIA), which serves to mitigate risks to the rights and freedoms of natural persons and is a tool for controllers to conform to the GDPR's legal requirements. DPIA builds on Privacy Impact Assessments (PIAs), as they have been encouraged by academia [START_REF]Privacy Impact Assessment[END_REF], Data Protection Authorities (DPAs) [START_REF]Privacy Risk Assessment: Methodology (how to carry out a PIA)[END_REF] and the European Commission (e. g. for RFID applications [START_REF]Privacy and Data Protection Impact Assessment Framework for RFID Applications[END_REF]). However, DPIA focuses on conformity to EU data protection law and thus has a more specific scope. It is a very useful tool for controllers to control their processing of personal data and ensure compliance.

When a high risk to the rights of individuals is likely, carrying out a DPIA is mandatory according to Article 35(1) GDPR. While non-compliance with this obligation may incur a penalty of up to 2% of the world-wide annual turnover of a business according to Article 83(4)(a) GDPR, the notion of high risk is not defined in the Regulation. Rather, Article 35(3) GDPR lists a few examples of data processing operations, which could potentially pose a high risk. Similarly, the GDPR does not offer much advice about how to carry out a DPIA; much less a methodology. Article 35(4) GDPR contains only minimal requirements, and provides no further guidance about how to implement these in practice. Furthermore, existing processes for Privacy Impact Assessments (PIA) may not take due account of the GDPR's legal requirements, such as data protection by design and by default, which is now enshrined in Article 25 GDPR, or the risk-based approach adopted in this new legislation. Thus, the goal of the workshop was to acquaint participants with the DPIA framework, how it can best be carried out and which specific issues may arise. Participants were first introduced to the DPIA framework developed by the German research consortium Privacy Forum (Forum Privatheit) [START_REF] Friedewald | White Paper Datenschutz-Folgenabschätzung[END_REF][START_REF] Friedewald | A Process for Data Protection Impact Assessment under the European General Data Protection Regulation[END_REF] and focuses on the rights of individuals. This framework is based on the legal requirements of the upcoming GDPR, in particular Article 35, as well as the Standard Data Protection Model (SDM) methodology adopted by the German data protection authorities [START_REF]The Standard Data Protection Model (SDM)[END_REF], which operationalises these legal requirements, and best practices. The framework takes account of the Guidelines on Data Protection Impact Assessment of the Article 29 Data Protection Working Party [START_REF]Guidelines on Data Protection Impact Assessment (DPIA) and determining whether processing is "likely to result in a high[END_REF]. In order to raise the participants' awareness of the risks to the rights and freedoms of individuals two case studies were discussed with a view to identifying the relevant risks by applying the data protection goals systematised in the SDM.

Introduction to Data Protection Impact Assessments

A DPIA begins before any data are processed and continues throughout the life cycle of a project and its data processing operations. It is a useful tool for any controller to implement their obligations under the GDPR and allows them to document this, as they are obliged to under Article 5(2) GDPR. At the heart of this process is the analysis of risks to the rights and freedoms of individuals that may emanate from the processing of personal data and is the basis for mitigating these risks through technical and organisational measures. This can best be achieved in four phases, as detailed in Figure 1 below.

Fig. 1. Framework for Data Protection Impact Assessment

In the preparation phase (1.), a team is assembled to carry out the assessment and relevant information about the envisaged processing collected. In the execution phase (2.) the sources of risk (i.e. attackers) are identified, the gravity of the interference is determined and the risks for the rights and freedoms of natural persons are evaluated. Furthermore, the controller identifies appropriate measures and documents the results of the evaluation in a DPIA report. On the basis of this evaluation, the controller then decides whether to carry out the envisaged processing operation or not. If the DPIA finds that the risks to the rights of individuals remain high even with the identified measures, the controller has to consult the supervisory authority according to Article 36 GDPR before the processing can start. The controller may also decide to abandon the processing operation.

If adequate measures could be identified to address the risks and ensure the protection of the rights of individuals (or this is achieved during the consultation with the supervisory authority), the controller implements these measures, tests and documents their effectiveness and demonstrates compliance with the GDPR (3.), before approving the processing operation. In the review phase (4.) the controller monitors the risks for the rights and freedoms of natural persons and repeats (parts of) the assessment when necessary.

The Standard Data Protection Model and Risk Analysis

The Standard Data Protection Model is a methodology to ensure effective compliance with data protection obligations and allows for auditing and control through transparent processes. This is achieved by formulating explicit data protection goals, which are derived from the legal requirements of data protection law. The data protection goals are the following:

Data Minimisation. Data minimisation substantiates and operationalises the principle of necessity, which requires of any process as a whole as well as any of its steps not to collect, process and use more personal data than necessary for the achievement of the purpose of the processing. Data minimisation is to be taken into account proactively as an element of data protection-friendly design. Starting with the design of information technology by the manufacturer and its configuration and adaptation to the operating conditions, to its use in the core and auxiliary processes of the operation, for instance in the maintenance of the systems used; from the collection of personal data, through its processing and use, to its erasure or complete anonymization; throughout the entire life cycle of the data.

(1) Availability. Personal data must be available and can be used properly in the intended process. Thus, the data must be accessible to authorised parties and the methods intended for their processing must be applied. This presupposes that the methods can deal with the available data formats. Availability comprises the ability to find specific data (e.g. by means of address directories, reference or file numbers), the ability of the employed technical systems to make data accessible to individuals in an adequate manner, and the possibility to interpret the content of the data (semantic ascertainability).

(2) Integrity. On the one hand, information technology processes and systems must continuously comply with the specifications that have been determined for the execution of their intended functions. On the other hand, integrity means that the data to be processed remain intact, complete, and up-to-date. Deviations from these properties must be excluded or at least be ascertainable so that this can either be taken into consideration or the data can be corrected. If the protection goal integrity is understood as a form of accuracy within the meaning of Article 5(1)(d) GDPR, this leads to the claim that there is sufficient congruency between the legal-normative requirement and common practice, both in terms of technical detail as well as in the broad context of the procedure and its overall purpose.

(3) Confidentiality. No person is allowed to access personal data without authorisation. A person is not only unauthorised when it is a third party external to the controller, regardless of whether they act with or without a criminal intent, but also employees of technical service providers who do not need access to personal data for the provision of the service, or persons in organisational units who are unrelated to the respective procedure or data subject.

(4) Unlinkability. Data shall be processed and analysed only for the purpose for which they were collected. Data sets can in principle be processed for further purposes and can be combined with other, potentially publicly available data. Larger and more meaningful data sets also increase the potential for abuse, i.e. to use the data unlawfully, for purposes beyond the legal basis. Such further processing is lawful only in strictly defined circumstances. The GDPR only allows them to be used for archival purposes which are in the public interest, for scientific or historical research purposes or for statistical purposes, and explicitly calls for safeguards for the rights and freedoms of the data subjects. These safeguards are to be achieved through technical and organisational measures. In addition to measures of data minimisation and pseudonymisation, other measures that allow the further processing to be separated from the source processing are also suitable, ensuring separation both on the organisational and on the system side. The data base can, for example, be adapted to the new purpose by pseudonymisation or reduction of data volume.

(5) Transparency. The data subject as well as the system operators and the competent supervisory authorities must be able to understand, to a varying extent, which data are collected and processed for a particular purpose, which systems and processes are used for this purpose, where the data flow for which purpose, and who is legally responsible for the data and systems in the various phases of data processing.

Transparency is necessary for the monitoring and control of data, processes, and systems from their origin to their erasure and is a prerequisite for lawful data processing.

Informed consent, where it is necessary, can be given by data subjects only if these criteria are met. Transparency of the entire data processing operation and of the parties involved can help ensure that data subjects and supervisory authorities can identify deficiencies and, if necessary, demand appropriate procedural changes.

(6) Intervenability. Data subjects are effectively granted their rights to notification, information, rectification, blocking and erasure at any time, and that the controller is obliged to implement the appropriate measures. For this purpose, controllers must be able to intervene in the processing of data throughout the process; from the collection to the erasure of the data.

All of these protection goals can be linked to specific provisions in the GDPR, and all except availability, which is an implicit requirement throughout the GDPR, are mentioned in the principles relating to personal data processing in Article 5(1) [START_REF]The Standard Data Protection Model (SDM)[END_REF].

Consequently, they can be used in the assessment of the risks to the rights and freedoms of natural persons in order to identify potential sources of risks and potential damages to these rights and freedoms. According to recital 76 the likelihood and severity of potential damages should be determined objectively and with reference to the nature, scope, context and purposes of the processing. However, as the phrasing of the risk already makes clear, recital 75 emphasises that this potential damage includes also non-material damage, such as discrimination, reputational damage, social disadvantages, the deprivation of data subjects' rights or preventing them from exercising control over their personal data. When read jointly with the second sentence of recital 94 it further becomes clear that besides such potential damages, interferences with fundamental rights, for instance the right to the protection of personal data under Article 8 CFR, the right to private life under Article 7 CFR, freedom of speech under Article 11 CFR or the right to be protected against discrimination under Article 21 CFR, are also risks to be considered in this assessment [START_REF] Bieker | Die Risikoanalyse nach dem neuen EU-Datenschutzrecht und dem Standard-Datenschutzmodell[END_REF].

When considering this fundamental rights dimension of risk in the GDPR it also becomes clear that mathematical formulas, such as the common 𝑅𝑅 = ∑ 𝐼𝐼 𝑘𝑘 × 𝑝𝑝(𝐼𝐼 𝑘𝑘 ) 𝑛𝑛 𝑘𝑘=1

, where R is the risk, which is the product of the impact multiplied by the probability of potential damage, are not applicable here. Instead, the evaluation should classify the effect of potential damages or interferences with fundamental rights as well as the likelihood of their realisation into certain categories, such as marginal, limited, serious and severe. By applying each data protection goal to a processing operation to identify potential risks and then evaluating these, controllers can ensure that they fulfil their obligations with regard to the rights and freedoms of natural persons.

Data Subject Participation

As data protection law is ultimately concerned with safeguarding the rights of individuals, scientific studies have been demanding for years that DPIAs (or PIAs) should not only include the views of (technical and legal) experts. Making use of the expertise of technical experts alone may lead to a very narrow perspective (e.g. limiting an assessment to legal aspects alone) and also to a technocratic-paternalistic approach that takes decisions without taking citizens' concerns duly into account. Rather, a comprehensive and broad consultation of stakeholders is necessary to increase the quality of the assessment results and their legitimacy. However, the questions of which actors are considered to be 'relevant' at all and who determines this [START_REF] Wright | Integrating privacy and ethical impact assessment[END_REF] should be kept in mind. This idea is reflected in Article 35 [START_REF] Wright | Integrating privacy and ethical impact assessment[END_REF] GDPR that stipulates that the views of data subjects or their representatives on the intended processing should be taken into account -if appropriate. The provision limits the controller's obligation to allow participation with reference to the effort needed or other conflicting interests (security, intellectual property rights, etc.).

Without prejudice to such limitations, the question arises as to how the different stakeholder groups and interests can be involved in the evaluation process of a DPIA. In terms of methodology, relatively simple and proven methods are available, with which companies already have experience in the areas of product design and marketing (e.g. focus groups), but of course more elaborate methods from participatory Technology Assessment (pTA) can also be used [START_REF] Hennen | Why do we still need participatory technology assessment?[END_REF]. However, an evaluation with data subject participation poses particular challenges in terms of timing and circumstances:

• The consultation should best start early in the process, so as to allow for an impact on the design of the processing operation.1 • The involvement of data subjects can be problematic, as careful and systematic assessment often requires expertise that lay people usually do not have. The key question here is therefore how this expertise can be conveyed to enable discussions on equal footing between lay people and experts. • The vocabulary used in the evaluation process has implications for the intensity and quality of the involvement of different groups of actors. For example, certain forms of wording are likely to favour particularly technophile actors or those with legal knowledge. It will therefore be crucial to organise a neutral translation process between the different groups.

Extensive participatory DPIAs involving external stakeholders are, however, likely to remain the exception, since this process is time-consuming and could lead to consultation fatigue among certain stakeholder groups. Under normal circumstances the data subjects' views should be taken into account by involving units from the organisation that are in close and regular contact with the data subjects, i.e. sales, service or the works or staff council [START_REF] Kiesche | So funktioniert die Folgenabschätzung[END_REF].

Hands-On: Practical Assessment of the Risks for the Rights and Freedoms of Natural Persons

After the input statements, participants were divided into two groups to discuss the two following case studies and identify risks for the rights and freedoms of natural persons. These were then summarized by participants of each group and discussed with all participants.

Case Studies

Case Study 1: Smart Surveillance in Train Stations.

After successful pilots, the national police force of an EU Member State has proposed setting up cameras with automated biometric recognition and behavioural analysis capabilities in all of the country's train stations. The system will have access to the images and biometric data from the national identity-card database, as well as police databases of terrorist and criminal suspects, political extremists and religious fanatics, and persons of interest or concern. It is supposed to be able to identify individuals with a very high degree of accuracy. The data will be stored for up to 1 year. Besides identifying individuals, the system performs behavioural analysis to identify a range of suspicious behaviours (e.g., looking about a lot, avoiding security personnel, leaving luggage behind). It also identifies dangerous behaviour or behaviour indicating suicidal tendencies, especially of vulnerable individuals (e.g. drunken gait, straying close to platforms).

When the system picks up suspicious (or dangerous) behaviour or individuals, it sends automated messages to the station security personnel, city anti-terror units, and/or the station health and safety personnel (for drunks, etc.), whereupon these initiate enhanced monitoring or other interventions (e.g. arrest).

Case Study 2: Emotional Decoding for In-Store Advertising.

A supermarket chain operating in an EU Member State has revamped its in-store advertising system with a smart camera system operated by Echeloon, a company specialising in targeted advertising. Through a camera integrated into a screen displaying advertisements, the system recognises when and for how long a person looks at the screen, their sex (and what it presumes to be their gender), approximate age and worn attire. Furthermore, the system deduces the customer's presumed emotional state (anger, happiness, anxiety, etc.) from their facial expression. The data is then used to personalise advertisements to pre-defined groups of customers and their presumed interests and preferences. Additionally, the system can promote special offers to certain groups of customers or offer specific rebates to an individual customer.

Customers are informed about video surveillance at the entrance of the market where the terms and conditions are posted on signs. However, they contain no reference to the smart camera system. In a press release the chain stated that the system is operated exclusively by Echeloon and the supermarkets have no access to the data. It goes on to state that the system processes only encrypted data and any photos of customers are processed automatically and deleted once the data has been extracted, after approximately 150 milliseconds. Thus, Echeloon assures, no personal data were collected and there was no obligation to inform customers specifically. 2

Group Discussions: Applying the Data Protection Goals to the Case Studies

The group discussions as well as the legal background of the case studies will be discussed in the following. In order to determine whether there is a high risk, the controller, in a first step should refer to examples of high risk processing operations provided by the Article 35(3) GDPR, recitals 71, 75 and 91 as well as the Article 29 Working Party [START_REF]Guidelines on Data Protection Impact Assessment (DPIA) and determining whether processing is "likely to result in a high[END_REF]. These include the innovative use of technology, data processing on a large scale as well as publicly accessible areas such as privately-owned shopping centres.

While the case study both fall within these examples and it is thus indicated that a high risk is likely and a DPIA should be carried out, the workshop aimed to enable participants to engage in the evaluation of the risks to the rights of individuals, as it is required in step 2.4 of the DPIA framework.

During the group discussions, the workshop participants sought to apply the Data Protection Goals to the case studies, to analyse the risks to the rights of individuals posed by the processing operations described in the cases. This section summarizes the results of these discussions. Due to the natural flow of the debate, not all protection goals were given equal attention.

The question of whether a processing operation is lawful is paramount in data processing. It is thus assessed as one of the first steps in a DPIA (see step 1.5 of the DPIA framework). While the case studies pose serious questions as to their lawfulness, 3 this was not a focus of the workshop and was therefore not addressed in detail. As the workshop focused on the identification of risks to the rights of individuals, the case studies did not include specific information on legal bases. However, as will be seen below the protection goals are able to identify the risks to the rights of individuals caused by the processing operations lined out in the case studies, which also uncover risks concerning the lawfulness of the processing as the SDM operationalises the legal requirements of the GDPR.

Case Study 1: Smart Surveillance in Train Stations. 3 Participants discussed that while the first case study would have to be based on an express legal basis of national law, the scope of the processing raised serious question of its proportionality. Regarding the second case study participants pointed out that Echeloon's claim that the data processed was not personal was not true, as the duration for which data are processed, and whether or not they are encrypted, is irrelevant to whether the data classify as personal data in the sense of Article 4(1) GDPR. Participants pointed out that the general information at the entrance of the store was not sufficient to obtain informed consent within the meaning of Article 7 GDPR, as it included only general information on video surveillance and not the specific processing operation of emotional decoding. It should be added that just as in the first case study, as the system identifies individuals by use of biometric data, it processes special categories of data according to Article 9 GDPR and thus explicit consent would be required. Beyond these issues, the case further raises issues of price discrimination based on age, gender, race or income (through the analysis of worn attire).

Data Minimisation. The first group found that there are several issues concerning data minimisation, which are linked to the extremely broad purpose of the smart surveillance system. It is supposed to identify not only various kinds of offenders, suspects or persons of interests included in a police database, but also any individual in the train station that acts suspiciously and thus allows for the tracking of all passengers frequenting the train station. Furthermore, the system is supposed to alert authorities of dangerous behaviour to prevent harm to individuals. As participants pointed out, these sweeping purposes can already be seen as colliding with the principle of purpose limitation of Article 5(1)(b) GDPR.

Participants also found that the storage of the cameras' raw data in a centralised system for one year violated the principle of data minimisation, as it was not specified why the data would be needed retrospectively, if the person identified did not lead to a match with the police database or act suspiciously or dangerously. Essentially, storage of the raw data beyond the assessment of their identity/behaviour would entail mass-scale data retention on train passengers, the vast majority of whom are neither suspects nor persons of interest. Furthermore, it was questioned, whether the purposes of identifying suspects or persons or interest and dangerous behaviour could not be met through other, less data-invasive means than the proposed smart-camera system.

Availability. As to the availability of the data it was discussed that it had to be ensured that the automated algorithm that automatically notifies the pre-defined authorities is revisable and allows for review of its functionality by the controller, e.g. through a logging mechanism. However, the contents of these logs should, with regard to the protection goal of data minimisation, only log data that are necessary to monitor the correct functioning of the system.

Integrity. Concerning the integrity of the data processing operation, the first group had general concerns about the properties of the system and the cameras and their safety and security. As the case study did not specify any of these issues an actual system would have to ensure that the entire surveillance system continuously complies with the specifications (including a definition of data flows, concerning access and sharing of the data) and the data processed in it would remain complete or any changes made by employees or external parties could be traced. In this regard the participants emphasised further that, given the amount of passengers frequenting a major train station, even a highly accurate algorithm would produce a significant amount of false positives and false negatives. Hence, it would have to be ensured that these are minimised and the persons operating the surveillance system would be able to adequately interpret these results in order to avoid the risk of false accusations against train passengers. However, the complicated nature of human-machine interactions -especially in the context of hierarchical organisations -exacerbates the risks that false positives or other analytical errors pose to data subjects. Given the complicated and 'inhuman' nature of machine 'thought' [START_REF] Burrell | How the machine thinks: Understanding opacity in machine learning algorithms[END_REF][START_REF] Metz | How Google's AI viewed the Move no Human could Understand[END_REF], staff operating and responding to the system can be presumed not to have more than a rudimentary understanding of how the system reaches its conclusions. Given that they are by definition also likely to hold only low-ranking and possibly insecure positions in their organisa-tion, they will likely be highly reluctant to question or go against the conclusions drawn by the system: even in the event of the system reaching very questionable conclusions, they will likely have organisational incentives to go along with the machine's conclusions, rather than go against the machine.

Confidentiality. The surveillance system entailed multiple risks with regard to the protection goal of confidentiality, the participants of the first group found. Given the broad database, access to the data would have to be defined restrictively and authorized access would have to be logged. This was needed in order to ensure that misuse of the collected data could be prevented or at least be detected and prosecuted. Persons with access to the system would be able to track the daily movements of a vast amount of people. This of course, was not only limited to the controller, who could also be tempted to expand the purposes of the processing even further, but also made the system a high-level target for third party attackers and hackers. A further point of concern was the interface of the system, such as when dangerous or suspicious behaviour is identified and interventions by the station police or security personnel are triggered. Participants stated that it would be a crucial question how much and which data about the individual concerned were made available to the security staff.

Further risks to the rights of the individuals could emanate from the storage location of the data. Participants argued that if, for example, the data were to be stored in a cloud rather than locally, the risk to the confidentiality of the data would be increased even further. Participants again pointed to the risks presented by false positives, false associations, and the potential for bias and subjectivity to infect the analysis. Given the very large number of individuals passing through major train stations, even error rates of less than 1 percent can quickly result in thousands of misidentifications with potentially very serious consequences for the individuals concerned and could subject them to discrimination.

Unlinkability. Due to the already overly broad purpose of the surveillance system, the participants focussed especially on unlinkability. The automated matching of individuals with the entire police database was seen as a heavy interference with the rights of individuals. Further, the possibility to identify any individual by matching their photo to the national ID card database was seen as yet another heavy interference with the rights of individuals on a mass-scale. The participants argued that the processed data could easily be used beyond their original purpose in order to discriminate certain groups of people. Due to the raw data of the camera footage being stored, this could also be done retroactively and the data could be combined with data from other sources to track the movements of individuals. Additionally, the data flows and the authorities that can access the data were not sufficiently clear. Lastly, the purpose of the collection could be expanded even further and the system could be linked to other state systems, for instance those of the welfare or health authorities, for instance to monitoring welfare recipients for signs of undeclared employment or other benefit fraud.

Transparency. Concerning the protection goal of transparency, it was argued that the train passengers were confronted with the risk of not knowing when, how or why their data was being processed. The individuals would have to be informed of the fact and the amount of surveillance as well as how the data is processed, including whether it is shared with other authorities or private parties, the participants found in their discussion. This had to include the monitoring and/or certification of the algorithm that carries out the biometric recognition and behavioural analysis.

Due to high numbers of individuals concerned they were already subject to a risk of being falsely identified as a suspicious person or as behaving dangerously, especially as these terms were not defined sufficiently. Furthermore, individuals could be identified merely because a person of interest for the police would ask them for the time, as one participant remarked, or the algorithm would identify their behaviour as dangerous. Thus, there was the additional risk of not being able to determine when an individual's behaviour would be registered by the system.

Intervenability. Similarly, the individuals faced risks concerning their possibilities of intervention with regard to the surveillance system. The participants argued that the lack of transparency led directly to a risk of the data subjects' not being able to exercise their rights. Furthermore, there was no second instance before the data was shared by the automated system. It was unclear how (and if at all) data subjects who have been identified as suspicious or engaged in dangerous behaviour may challenge a decision, and indeed how they would even find out about such decisions.

Case Study 2: Emotional Decoding for In-Store Advertising.

Data Minimisation. With regard to the purpose of targeted advertising to customers of a supermarket, the participants of the second group found it questionable whether all of the envisaged categories of data (sex and presumed gender, approximate age, worn attire and emotional state) where strictly necessary, as demanded by the principle of data minimisation. The data collected concern special categories of data according to Article 9 GDPR, as the system uses the biometric data to identify individuals 4 and allows conclusions on categories such as race, ethnicity, religious beliefs (e.g. when wearing a hijab or kippah). Furthermore, the data on the emotional state of customers were derived from the biometric data, could arguably be seen as health data, as Article 9 GDPR includes data relating to mental health (cf. Article 4 (15) GDPR). These broad categories of data, the participants argued, were not necessary to personalise product offers in a supermarket. While the automated deletion of the pictures taken by the system is a step to reduce the amount of data used, the sensitive biometric data is retained indefinitely and therefore the dataset is not reduced to the minimum required to achieve the intended purpose.

Availability. The availability of the data here is not an issue, as they are highly available.

Integrity. Much as in the first case study, the participants of the second group found that concerning the integrity of the data processing operation the properties of the system had to be further defined.

Confidentiality. As the data is processed by a processor, the risk of disclosures is higher. Thus, employees of both the controller and the processor could potentially use the biometric data stored in the system for an unspecified period and use them in other processes, such as biometric identification, for identity theft or fraud. Furthermore, other customers or employees could observe the targeted advertisements on the display, which could cause the individual distress, which could, depending on the promoted product, range from mild embarrassment to more serious consequences.

Unlinkability. With regard to the storage of the data that is derived from the pictures taken of customers, it was pointed out that the continued storage and further use for other purposes would pose risks to the data subjects, given the nature of the data, which relates to the private life of the individual. For example, if the further processing was aimed at assembling profiles of shopping behaviour -perhaps even drawing on data generated at other stores that use the same camera system -this would amount to tracking of individual preferences.

Transparency. In the group discussion transparency was the main issue. The participants argued that the system provided no transparency to data subjects as they were not at all informed about the system. This also extends to the analytical principles governing the system's algorithms: How and on what basis does the system identify certain kinds of behaviour as suspicious or dangerous (including to the individual him or herself)? How reliable is this identification?

The system could also be used to manipulate the emotions of data subjects (e.g., making them unhappy by denying them expected promotions or giving them the 'wrong' ones; making them happy by giving them particular discounts, etc.).

Intervenability. As data subjects are not informed of the processing, they would also have no means of intervention in the processing and thus be faced with a negation of their data subject rights.

Conclusion

The main objective of the workshop was to introduce participants to the DPIA methodology developed by Privacy Forum with a particular focus on the evaluation of risks based on the systematic approach of the SDM, which operationalises the legal requirements of EU data protection law. This was achieved by means of an introductory presentation, and a hands-on exercise in which the workshop participants analysed two data processing operations with regard to the risks they pose to the rights of individuals. As was to be expected both groups found that due to the numerous risks to the rights of individuals the envisaged processing operations of both case studies could not be carried out.

Beyond the details of the case studies and the particular methodology presented, the workshop discussions yielded insights that are of more general significance for DPIA processes. The discussions among participants confirmed that a multidisciplinary perspective is needed in order to identify and mitigate risks to the rights of individuals in a coherent and holistic manner. The workshop demonstrated that the SDM's data protection goals allow for a structured analysis of risks to the rights of individuals in accordance with the requirements of data protection law. Due to the manifold risks data processing entails such a structured analysis is crucial and at the heart of every DPIA. Nevertheless, the risk analysis in accordance with the GDPR needs further refinement and research. The discussions showed that it can be difficult to discuss risks for rights of individuals, if the legal basis for the processing and the potential risk sources, i.e. attackers, have not been identified beforehand, as stipulated in the DPIA framework. Furthermore, the fine-grained evaluation of the risks to the rights of individuals requires clarification. While recital 75 GDPR refers to the varying likelihood and severity of potential damages, which originated in information security, will have to be adapted in order to allow for the correct application within the fundamental rights framework of the GDPR and in conformity with the requirements of the EU Charter of Fundamental Rights. This future work can then also be integrated in the SDM in order to provide controllers, processors, manufacturers and supervisory authorities with guidelines on how to assess risks to the rights of individuals in practice.

However, when a DPIA is conducted prior to market launch or in parallel with the development process the inclusion of external persons may be undesirable. Reasons might be the immaturity of the technology, the organisation's desire for confidentiality or fear of a bad public image.

For further discussion on legal, regulatory and ethical issues surrounding 'smart' advertising of various kinds see[START_REF] Kiesche | So funktioniert die Folgenabschätzung[END_REF].

Unlike the pictures itself, these data are also stored and further processed.

Acknowledgement

This work is partially funded by the German Ministry of Education and Research within the project 'Forum Privacy and Self-determined Life in the Digital World', https://www.forum-privatheit.de/forum-privatheit-de/index.php.