Roberto Carbone
email: carbone@fbk.eu

Silvio Ranise
email: ranise@fbk.eu

Giada Sciarretta
email: giada.sciarretta@fbk.eu

Design and Security Assessment of Usable Multi-factor Authentication and Single Sign-On Solutions for Mobile Applications

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

This paper is a report of the workshop "Secure and Usable Mobile Identity Management Solutions: a Methodology for their Design and Assessment" presented at the 13th International IFIP Summer School 2018 on Privacy and Identity Management -Fairness, accountability and transparency in the age of big data held in Vienna, Austria.

Context. We focused on the design and security assessment of solutions for mobile native applications (hereafter native apps) with two features: Single Sign-On (SSO) and Multi-Factor Authentication (MFA); these two features are extremely important indeed: SSO allows users to access multiple apps through a single authentication act performed with an identity provider (IdP), for example Google or Facebook; while a MFA is a procedure that enhances the security of an authentication process by using two or more authentication factors (e.g., a password combined with the use of a fingerprint). A good design choice is to combine these features to have a good balance between usability and security.

While there exist many secure MFA and SSO solutions for web apps, their adaptation in the mobile context is still an open challenge. The majority of mobile MFA and SSO solutions currently used are based on proprietary protocols and their security analysis lacks standardization in the structure, definitions of notions and entities, and specific considerations to identify the attack surface that turns out to be quite different from well understood web scenarios. This makes a comparison among the different solutions -in order to choose the proper solution for a specific scenario -very complex or, in the worst case, misleading. Due to this lack of specifications and security guidelines, designing a mobile MFA and SSO solution from scratch is not a simple task; and as its security depends on several trust and communication assumptions, in most cases, could result in a solution with hidden vulnerabilities. In addition, it is necessary to take into account the legal aspects of the country where the MFA and SSO solution will be deployed. However, when innovative solutions are analyzed it is not an easy task to understand which legal obligations follow.

The main goal of the workshop was to make participants aware of the current security and usability issues of MFA and SSO solution in the mobile context. Participants were first introduced to the context and then, through the use of exercises we openly discussed several illustrative scenarios. Finally, we described our methodology for the design and security assessment of mobile MFA and SSO solutions. The design space is characterized by the identification of: (i) national (e.g., Sistema Pubblico di Identitá Digitale -SPID for Italy [START_REF]DPCM of 24[END_REF]) and European (e.g., electronic IDentification Authentication and Signature -eIDAS [16]) laws, regulations and guideline principles that are particularly relevant to digital identity; (ii) a list of security and usability requirements that are related to authentication solutions; (iii) a set of implementation mechanisms that are relevant to authentication and authorization on mobile devices and provide an easy way to satisfy the requirements in (ii). To validate our approach, we applied it to a number of real-world scenarios that represent different functional and usability requirements. In this workshop, we applied our methodology to a real use-case scenario (called TreC) that supports the usage of mobile health apps. TreC (acronym for "Cartella Clinica del Cittadino", in English Citizen's Clinical Record) is an ecosystem of services that supports doctors and patients in the health-care management, by enabling all citizens living in the Italian Trentino Region to access, manage and share their own health and well-being information through a secure access (currently used by more than 80.000 patients).

Workshop Objectives. The main objectives of the proposed workshop were the following:

to enable the audience to acquire the basic notions and the state of the art of MFA and SSO solutions for native apps; to create awareness of usability and security problems together with legal provisions related to authentication in mobile computing; to provide an overview of the techniques commonly used to analyze the security of an authentication solution; to perform an experimental evaluation of security and usability of MFA solutions for native apps.

Expected contributions from the audience members. To raise the participants' awareness of the current possible limitations on usability and security of mobile MFA and SSO solutions, several questions and exercises were discussed together. The information extracted from the discussion has been useful in two ways. On the one hand, we were able to validate our hypothesis on usable solutions and to understand which security level is perceived. On the other hand, we were able to evaluate our methodology asking feedback to possible user.

Intended audience, including possible assumed background of attendees. The workshop was oriented to academic researchers, (PhD) students, security experts from industries that work on or want to approach the field of identity management. The attendees did not require a specific background on authentication to follow the main part of our workshop, as our step-by-step teaching approach enabled them to grasp the information presented even if some of the concepts were new or not consolidated. A dozen technical (IT security) and legal researchers took part to our workshop. The slide of the workshop are available at https://st.fbk.eu/workshop-ifipsc-18.

Paper Structure. In Sect. 2, we describe the content of the workshop on MFA and SSO solutions for native apps. Sect. 3 details the workshop structure and the assigned exercises. In Sect. 4, we present the exercises and discuss the outcomes. Finally, in Sect. 5 we discuss some lessons learned and describe our on-going work on this topic.

Content of the Workshop

We make a large use of our digital identities in our everyday life, from accessing social apps to security critical apps like e-health or e-banking apps. Underlying these transactions there is the exchange of personal and sensitive data, which could be exploited by a malicious intruder to impersonate or even blackmail a user. For this reason, many Identity Management (IdM) solutions have been designed to protect user data. In general, IdM refers to different aspects of the digital identity life-cycle (e.g., the creation and the provision of identities, password management and so on). In this workshop, we focused on the aspects related to authentication.

Password-based Authentication. A common authentication mechanism is the password-based authentication, however its use is resulting in many attacks (e.g., identity theft). There are two main reasons. First, users are very bad in inventing and remembering passwords. [12] shows the list of the top 100 worst passwords of 2017, where at the first place there is "123456". This password is very easy to remember but it is also easy to crack; attackers have rainbow tables and dictionaries that contain this kind of credentials. Second, users re-use their passwords on several services: as reported in [10], more than the 54% of people use only 5 or fewer different passwords across their entire online life. This means that, if their credentials are compromised, for example after a guessing attack (easily performed by an hacker if the password selected is one of the 100 worst passwords [12]) then the attacker can access all the user data in different online services. There exist "complexity tips" which allow users to choose proper passwords. For example, though being nine digits long, a password such as '123456789' could be instantly and easily cracked. According to former NIST recommendations, properly complex passwords should contain lower-and upper-case letters, as well as special symbols and numbers, for them to be secure enough that it could take years to crack them (via brute-force or dictionary attacks). However, the current NIST guidelines [START_REF][END_REF] recommend to follow an entirely different scheme, i.e. a good password should be made of a random, long phrase.

Password-based Authentication and SSO. To permit the user to choose a complex password and access different services, an advisable design choice is to combine the password-based authentication with a Single Sign-On (SSO) solution. SSO allows users to access multiple apps through a single authentication act performed with an IdP, for example Google or Facebook. A common practice is to adopt the state-of-the-art standards, like SAML 2.0 [START_REF]Profiles for the OASIS. Security Assertion Markup language (SAML) V2[END_REF] and OpenID connect [START_REF]OpenID Connect Core 1[END_REF] (OIDC). SAML 2.0 is pervasively used in the corporate environment, while OIDC is mainly used in social apps. There are two main advantages of using SSO. First, users do not need to register with an app to access it. Thus the user can choose a single complex password (providing usability and security). Second, if a user has already an active login session with an IdP, then she can access new apps without entering her IdP credentials anymore (providing usability). A SSO security drawback is that users are using only one password to access many services. Again, the user could select a more complex password, but still there is a single point to failure.

Multi-Factor Authentication and SSO. A better design choice is to combine SSO with MFA solutions. Where a MFA is a procedure that enhances the security of an authentication process by using two or more authentication factors, such as combining a password that is something you know, with a hardware token that is something you have or the use of a fingerprint that is something you are.

Usually a MFA procedure requires the generation of a OTP (One Time Password). That is an una-tantum code that proves the possession of the OTP generator and optionally, if protected by a PIN, proves the knowledge of the PIN as well. There are different OTP generation approaches, in this workshop we focus on the Time-based OTP approach, where the OTP is generated starting from the current time of the operation and a secret key shared between the OTP generator app and the IdP. IdP must validate this value: only OTPs that fall into a short temporal range are accepted.

There are many MFA solutions on the market, and some of them are based on FIDO [START_REF] Fido | [END_REF]. FIDO is a standard for password-less and MFA authentication that allows online services to augment the security of their existing password-based solution by adding a MFA procedure. To provide a FIDO two-factor solution, the organization must provide to its users a physical FIDO U2F device, like a USB key.

MFA and SSO solutions for native apps. In this workshop, we focused on the design and security assessment of solutions for native apps with two features: SSO (for usability) and MFA (for increasing security). We focused on native apps -which differ from browser-based mobile apps as they are not accessed through a browser but they need to be downloaded from a marketplace -as the market is pointing on their use. Think about the many times you are suggested to download the app while you are navigating it in the browser; or the limitations that you can have from a browser-based version of an app. For example, this is the case of the browser-based version of TripAdvisor that provides less functionalities compared to the native app, such as the management of the reviews: users can read reviews on the browser version, but if they want to contribute on one review they have to download the native app.

As we will detail in Sect. 4, the known standards and solutions currently available for browser-based authentication (e.g., SAML or OIDC) cannot be easily reused in the mobile context, as browser-based and native apps are based on different security assumptions. Even if these are very good solutions there are still some limitations. Being proprietary protocols they cannot be customized and they do not necessarily satisfy the security requirements of a company. For example, an identity used in the social network solution is self-declared by the user. And so, it cannot be used by a company which needs to be sure of the real identity of the user. A first attempt of designing a solution for mobile authentication was carried on by big companies (e.g., Google and Facebook) that have designed their own solutions based on their security assessment. At the same time, the OAuth working group has released some guidelines. The current best practice was released in 2017. The solution proposed is called "OAuth 2.0 for Native Apps" [START_REF]OAuth 2.0 for Native Apps[END_REF]. Even if it is a good starting point, it does not cover some aspects. For example, it does not mention how to extend the protocol to support MFA or more complex environments, where for example different standards are used. Thus, in some specific cases and based on the requirements, a company is required to design a new ad-hoc solution.

The importance of a careful design phase. Designing a security protocol from scratch is not a simple task, as many aspects must be taken into account, such as how to establish trust, how to choose the right communication channel or how to evaluate the compliance of the designed solution with the current legal obligations, thus it is not recommended. Moreover, after the design, it is not simple to choose the right method to evaluate the corresponding security. Given all these aspects, it is clear that the design phase is not trivial and wrong design choices could lead to serious security and usability problems.

An example of a wrong design choice is the use of SMS as a second-factor authentication. This authentication method consists of the following steps: first the user has to enter her credentials into the app, then she will receive an SMS containing a OTP, and finally this OTP is entered by the user in the app. NIST [7] points out that this solution could be vulnerable to two kinds of attacks: Social Engineering. "An out of band secret sent via SMS is received by an attacker who has convinced the mobile operator to redirect the victim's mobile phone to the attacker" (e.g., using SIM swap [START_REF] Cranor | Your mobile phone account could be hijacked by an identity thief[END_REF]). Endpoint Compromise. "A malicious app on the endpoint reads an out-of-band secret sent via SMS and the attacker uses the secret to authenticate.".

Even if these attacks are well known in the security community, there are still many companies that are using this authentication method. This is causing the spread of many security breaches. For example, this is the case of the social network platform Reddit, attacked in August, 2018. In [START_REF]We had a security incident. Here's what you need to know[END_REF], the Reddit security experts specified that the attack was related to the use of SMS and that they are now moving to a token-based second factor authentication method. Another example is described in [2], where a user was victim of two SIM hijacking attacks and now he is suiting the telecommunication company for a total of 224 million dollars. This example clearly demonstrates how a wrong design choice could damage not only the end-user but also the company.

Our methodology. Given that designing a new security protocol from scratch is not an easy task and could result in a solution with hidden vulnerabilities, we have contributed with the definition of: a reference model for MFA and SSO for native apps, and a methodology to assist a designer in the customization of our model and in the analysis of its security and usability. Our reference model is inspired by the Facebook solution [START_REF] Facebook | Getting started with the Facebook SDK for Android[END_REF] and OAuth 2.0 for native app [START_REF]OAuth 2.0 for Native Apps[END_REF]. We have extended these solutions in a way that they can be used by any IdP willing to provide its own SSO solution, meaning that the resulting SSO does not necessary leverage on identity provided by social IdP, and (optionally) a MFA. Currently, our models support two different OTP generation approaches: TOTP and Challenge-Response. Full details about the reference model based on TOTP can be found in [START_REF] Sciarretta | Design, Formal Specification and Analysis of Multi-Factor Authentication Solutions with a Single Sign-On Experience[END_REF].

Together with the reference model we have defined a methodology to assist a designer in the customization of our reference model and in the analysis of the resulting security and usability. In the first phase, we ask the designer to clarify the application scenario by filling a table that we provide (specifying the entities involved, the type of data that will be processed and which are the authentication requirements). Given this table, in the customization phase we are able to instantiate our model for this specific scenario and we provide as output a message sequence chart of the flow and a set of assumptions and security goals. These values are then given as input to the security analysis phase. We provide a semi-formal and a formal analysis. The output of this phase is a security analysis report. If some serious attacks are found then the designer has to go back in the customization phase and change the design otherwise the designer can proceed with the last phase. In the usability analysis phase we are asking to validate the usability satisfaction. If no problems are reported then the final solution is generated; otherwise the designer has to go back to the definition of the requirements and refine the design accordingly.

To validate our methodology, we have applied it to different real-world scenarios which consider different authentication and usability aspects. During this workshop we had detailed the e-health scenario TreC.

Structure of the Workshop

To raise the participants' awareness of the current limitations on usability and security of mobile MFA and SSO solutions, together with the background context described in Sect. 2, several questions (labeled with ) and exercises (labeled with ) were discussed together. Fig. 1(a) shows the background and current position of the participants (divided in two groups) and the outline of the exercises. The workshop followed the following structure:

Introduction and Problem Statement. In this introductory part we provided participants with the background described in Sect. 2 and we pointed out which are the current limitations related to the development of usable authentication solutions that are also secure in the mobile context. Browser-based vs Mobile Authentication. After the description of the browser-based OIDC standard, we asked the participants if -in their opinion -this solution (and more in general, any browser-based solutions) can be reused also in the mobile context. We decided to ask this question to evaluate the participant's awareness and understanding of the differences between a mobile and a browser-based solution and the fact that we cannot easily reuse solutions that are developed in one context in another. Design Choices: Security and Usability Problems. Designing a security protocol from scratch is not easy and the design phase is very important in terms of striking the right balance between security and usability. To raise this warning, during the workshop we asked participants to identify which are the security and usability problems related to two wrong design choices: User Agent Choice: embedded browser. This exercise is related to the choice of using an embedded browser as user agent. So we asked the participants to evaluate the usability and security of a solution where users enter their credentials in an embedded browser, namely a browser that is managed by an app.

OTP Choice: app that shows the OTP value to the user. During this exercise, we asked the participants to evaluate the usability and security of an OTP generator app that shows the OTP value on the smartphone screen. Methodology Overview: TreC Scenario. In this part, we described our methodology for the design and security assessment of mobile authentication solutions, applied directly to a real-world use case scenario, called TreC. e-Health Legal Compliance. Being TreC a personal health record platform, we briefly mentioned the Italian legal aspects concerning health data, and more in general to sensitive data. Then, with the aim of having a broader view on the legal aspects, we asked the participants to present the legal obligations of their country. Usability Discussion on TreC. In relation to the TreC solution, we discussed with the participants some usability problems that resulted from proposing our solution to patients. TreC activation phase. We asked the participants to suggest some changes in order to simplify the TreC activation phase taking into account security. Conclusions and On-Going/Future Work. Finally we summed up the main points of the workshop and presented our on-going and future work.

The answers and the discussions are reported in the following section.

Outcomes of the Exercises

In this section, we report the solutions to the exercises and discussions introduced in Sect. 3. To promote an interdisciplinary approach, we divided the participants in two groups based on their backgrounds and current position (see Fig. 1(b)). Each group included four men and a woman.

Browser-Based vs Mobile SSO Solutions

Q1. Can we use browser-based authentication and SSO solutions for native apps? We gave 5 minutes to discuss the problem and then we asked for the individual answers.

Participant's Answers: 5 "yes" and 5 "no". With 2 participants that voted "yes" saying that actually they would prefer to vote for "it depends", clarifying that it depends on the scenario and the security level required.

Our Answers: The use of a browser-based authentication protocol in the mobile context requires a detailed understanding about the differences between the two scenarios [START_REF] Chen | OAuth Demystified for Mobile Application Developers[END_REF]23], and at the end it is pretty clear that we need to design a new flow ad-hoc for the native apps. Let us have a look at the differences. The first difference is the Service Provider (SP) type: SP is not an app running in the browser but it is a native app. So we have to consider all the vulnerabilities related to a mobile platform.

Secondly, the User Agent (UA) that is used by the user to interact with the SP could be of different types: it could be a browser embedded inside the SP app, or an external browser that is installed in the user smartphone, or even an app released by the OIDC provider. So, we must take into consideration that now the SP app and the UA could not be played by the same entity as was in the browser case.

In addition, the redirection mechanisms between the OIDC provider and the app are different. Indeed, in a browser redirection you can uniquely identify the SP using its hostname. This ability is not always available in the mobile case as you are required to redirect to a specific app in the user's smartphone.

Finally, in the mobile case, the use of a SP backend is optional (you can have a native app that does not require a backend), so we have to adapt the flow by directly managing the authentication from the mobile. So in this case, we cannot use the client secret for authenticating the SP since in a mobile device we cannot store a secret as all the stored values are readable, at least by the owner of the smartphone.

For all these reasons, the reuse of available browser-based solutions in a mobile context is not obvious and it is necessary to redesign them taking into account the differences highlighted above.

Evaluation: The question was not fully clear and some participants were not able to answer.

Wrong Design Choices

In some specific cases and based on the requirements, a company or an organization could be forced to design a new authentication solution to provide secure mobile authentication solutions to their employees. However, designing a security protocol from scratch is not a simple task, as many aspects must be taken into account. In this section, we report the two scenarios that have been proposed to the workshop participants to highlight examples of wrong design choices. We proposed two exercises and asked Group 1 to tackle Exercise 1 (E1 ) and Group 2 to solve Exercise 2 (E2 ) in parallel. We allowed 10 minutes to elaborate a solution to each group.

User Agent (UA) Choice: Embedded Browser E1. We asked the participants to evaluate the usability and security of a solution where users enter their credentials in a browser that is managed by a native app (so called embedded browser). In detail, we asked the members of Group 1 to focus on the following questions:

1. How does an embedded browser work? 2. Are there any security issues? 3. How would you rate the user experience when accessing multiple apps?

Participant's Answers:

1. An embedded browser is a component inside your app that opens a website URL. 2. A native app has full control of the embedded browser, so if you are typing a password, the attacker can read it. 3. Not so good. Since an embedded browser is a separate browser-instance for all the native apps, users have to re-enter their password for all the native apps.

Our Answers:

1. An embedded browser is defined in [START_REF]OAuth 2.0 for Native Apps[END_REF] as "a user-agent hosted inside the native app itself (such as via a web-view), with which the app has control over to the extent it is capable of accessing the cookie storage and/or modifying the page content". The relevant bit from the point of view of security is that a native app is in control of the embedded browser. 2. The use of this type of browser is widely discouraged, as there is a loss of isolation between the app and the browser [START_REF] Luo | Attacks on WebView in the Android system[END_REF]. If the app is malicious, then it can steal the user credentials or change the authorization permissions. This is an example of a JavaScript added by a malicious app to steal user credentials:

webView . e v a l u a t e J a v a s c r i p t ( ' ' ( f u n c t i o n ( ) { r e t u r n document . getElementById ( ' pwd ' ) . v a l u e ; } ) ( ) ; ' ' , new V a l u e C a l l B a c k<S t r i n g >() { @Override p u b l i c v o i d o n R e c e i v e V a l u e ( S t r i n g s ) { Log . d ( ' ' WebViewField ' ' , s ) ; } } ) ;

3. An additional limitation when choosing an embedded browser is that it does not provide a SSO experience. Indeed, if the browser is integrated within the app, then the login session information is stored in (and only accessible to) the app and it is therefore not available to other apps. This forces the user to re-enter credentials even if she has an active login session with an IdP. This is a frustrating experience, especially due to the small-virtual keyboard of a smartphone.

Evaluation:

The participants were able to answer in the correct way.

OTP Choice: App that Shows the OTP Value to the User E2. This exercise is related to the choice of using a native app for showing an OTP value to the user.

1. How would you rate the user experience when accessing a native app? 2. Is there any security issue? 3. List one or more OTP choice alternatives.

Participant's Answers:

1. We thought that usability really takes a hit if you have to switch between your native app and the OTP app all the time. This is the main thing. The usability can also be affected if you have multiple devices lost or broken. 2. Malicious apps trying to exfiltrate the OTP code or shoulder-surfing attacks (you are just behind someone and take a look at the OTP code that can be memorized or can be picked using a mobile camera). 3. Hardware tokens.

Our Answers:

1. Moving from an app to another is burdensome for the user in terms of time and difficulty. 2. To avoid the burden of remembering the OTP value, some apps (e.g., MySiel-teID1 app of the Sielte SPID IdP) have a button for coping the OTP value. This is a serious security issue as the clipboard can be accessed by any app installed in the smartphone so that malicious apps can easily steal the OTP that has been copied. 3. An alternative choice is the use of a solution that does not ask the user to enter the OTP value, but after the PIN input, the OTP value is sent to the IdP server in a transparent way (namely, without any involvement of the user). The other alternative is the use of external OTP generators, such as FIDO keys or eID smartcard with capability of implementing a challengeresponse OTP approach (using NFC and a smartphone as a card reader). The advantage of using an eID card over other external hardware tokens is that usually users bring it with them to permit an in-person identification; thus we can assume that an eID card is always available to the user also for online access.

Evaluation:

The participants were able to answer in the correct way.

e-Health Legal Compliance

Being TreC a personal health record platform, we briefly mentioned the Italian legal aspects concerning health data. Then, with the aim of having a broader view on the legal aspects, we asked the participants to present the legal obligations of their country. At the time of running the workshop, when dealing with sensitive health data -that are a particular type of personal data -in Italy, we had2 to follow the Data Protection Code [START_REF][END_REF]. [START_REF][END_REF] says that the data controller shall adopt the minimum security measures in order to protect these data. More technological details can be found in CAD [13] that is the Italian code for the public administration. In particular, [13] specifies which digital identities can be used in this context: CNS (a smartcard used to access online services of the public administration), CIE 3.0 (the Italian electronic identity card) or the Italian national ID scheme called SPID (from the second level up).

Q2. Which legal obligations do you have to follow when dealing with e-health data in your country?

Participants' Answer (from Germany): We apply the GDPR, especially for sensitive health data. The basic approach is: you are not allowed to process any data if you cannot provide the level of security that is necessary to protect the data; it is not a minimum standard but it must be the state-of-the-art. Additionally to the GDPR there are also rules/guidelines specific for health data.

Extra Discussion: we report here different interesting discussions that came up in relation to this question.

The first is a discussion on the minimum security measures required in [START_REF][END_REF]. We explained that the Annex B of [START_REF][END_REF] specifies a set of specific security measures, such as: sensitive data must be protected with an authentication method with passwords of at least 8 characters. [START_REF][END_REF] was in force pre-GDPR and we agree with the participants on the fact that the GDPR approach is the opposite: they do not suggest any kind of measures; they should be state-of-the-art, and it should be proved that companies have done their best to comply and provide enough security.

Then, we have discussed who is in charge of managing health data. In Italy, each region is responsible for applying and developing solutions for healthcare. In Germany, it depends of what you do. For example, hospitals have to follow the federal state law, while the health insurance is the same for all the country.

Finally, we briefly discussed about the privacy concerns arising by the adoption of a SSO solution. A participant draw our attention to the fact that the usage of a SSO protocol every time a user wants to log-in creates a single point of knowledge that can become a major threat to privacy. This problem is particularly acute when considering the so-called "consumer SSO": what happens to consumers data when they enable access to other applications or accounts from Facebook, Gmail, LinkedIn, Twitter, or a host of other providers? Indeed, this enables service providers with the capability of monitoring and collecting information on consumers habits and preferences as they browse the Web. While acknowledging the importance of these and related privacy concerns, we consider them outside the scope of the workshop which focuses on the trade-off between security and usability of SSO solutions. We just observe how security and privacy may be contentious even in SSO solutions: the capability of tracking and profiling users can be used by an identity provider to spot when an attacker is trying to impersonate a legitimate user. This is known as behavioral authentication and is provided by, e.g., Google which alerts if a user account is accessed from a location which is not among the usual ones.

Evaluation: Different interesting discussions arose from this question.

TreC Activation Phase

TreC activation phase is performed by the patient, only once and after she download the OTP-PAT app. It is performed partially on her laptop and partially on her smartphone:

-On her laptop, patient logs in using her CNS (the card is read by desktop smartcard reader), and generates a temporary code (it lasts 5 minutes). -On her smartphone, patient downloads the OTP-PAT app using an official marketplace. Then, she enters the temporary code together with her credentials into OTP-PAT. If the login is successful, the activation phase is completed by patient with the creation of a PIN code.

As a consequence of this phase, OTP-PAT obtains two values: a token that is used as a session token in place of the user credentials to provide a SSO experience; and a seed value -stored encrypted with the PIN code selected by patient during this phase -that is used to generate OTPs. For the TreC scenario, we have performed a pilot involving a controlled set of patients. Regarding usability, we found out that the activation phase is considered too complex. The main reason was the use of a smartcard reader that for being used, needs the installation of a specific software, which sometimes does not work properly. In addition, users were annoyed by the requirement of choosing complex passwords inside the mobile and they tend to easily forget them.

E3. What would you suggest to change in order to simplify the activation phase taking into account security? Note that the activation phase must provide a good level of assurance on the real identity of the user. In the previous solution this was implied by the use of a smartcard and the generation of an activation code, specific for the particular installation of the app. We gave 10 minutes to debate this issue.

Participant's Answers: they came up with two ideas:

1. Use one of the existing identity infrastructures. For example, in Austria they can use an identification method provided from the post system or in Sweden, if you have a bank account, then you can request a BankID that you can use for accessing online services; 2. Perform a face-to-face authentication and then send an activation code via email.

In addition, they observed that the objection on the password complexity is just an educational thing: people do not understand the security behind this design choice.

Our Answer: We change the activation phase follows:

-On her laptop, patient logs in with one of the authentication solutions that are available (for example SPID) by using a high level of assurance (e.g., requiring a second factor authentication) and obtains a QR code. -On her smartphone, patient downloads the OTP-PAT app using an official marketplace. Then, she scans the QR code using OTP-PAT app and enters a temporary code obtained on her email. If the login is successful, then the activation phase is completed by patient with the creation of a PIN code.

As an alternative, if some users do not want to activate the app online, they can go to one of the office of the healthcare organization, prove their identity by using an identity card (in-person identification) and finally they receive a printed version of the QR code. Note that this solution avoids the need to manually enter username and password since the identity information of the patient is inside the QR code obtained after a strong authentication (in-person or online).

Evaluation:

The participants were able to propose valid alternatives.

Lesson Learned and On-Going Work

During this workshop, we had the opportunity to discuss our work with researches from technical and legal backgrounds. On the one hand, our main goal was to create awareness of usability and security issues related to authentication in the mobile context. On the other hand, the information extracted by their answers and discussions gave us the opportunity to validate our hypothesis on the usability and security of the solution that we have proposed for the TreC scenario.

Regarding the exercise session, we observed a high interest and participation. The participants were able to answer in the correct way to almost all questions. Only the first question related to the possibility to re-use a browser-based solution in the mobile context was not fully understood. The problem was thatbeing us security experts -we intended that question from a security perspective, while in a broader context this was unclear.

In the workshop, we made an effort to clearly define the security and usability problems in each one of the proposed exercises and questions. Indeed, this is rarely the case in a real-world scenario whereby striking the best possible balance between security and usability turns out to be a daunting task. This is so because of tight development schedules, focus on functionalities rather than security-bydesign, and the unawareness of developers about the security implications of certain implementation decisions. The combined effect of these factors results in the presence of severe (and exploitable) vulnerabilities in a large amount of applications. To alleviate this state of affair, we developed a plug-in for the automated synthesis of secure authentication solutions in mobile applications in the context of the EIT Digital activity "Security Tools for App Development" [START_REF]API Assistant: automated security assessment of 3rd party apps for the API economy[END_REF] (STAnD). The basic idea underlying STAnD is to provide native app developers with tools that help them to take into consideration more and more security aspects. STAnD will be composed by a plugin for code hardening and a wizard that allows developers to configure and customize their solutions: developers are presented with a series of choices and then the code is automatically produced according to their answers. A second feature of STAnD will be the possibility to validate the app code by submitting the APK to a managed service that will check if there are security problems (e.g., the need for obfuscating the code related to the handling of a key).

  Question and exercise structure.

1 Position

 1

Fig. 1 .

 1 Fig. 1. Interactive Workshop Structure.

https://play.google.com/store/apps/details?id=it.company.sielte

During the preparation of this workshop, the Italian government was in the process of adopting the EU General Data Protection law (GDPR[18]), with some delay compared to other member states due to the national elections. Now,[START_REF][END_REF] was amended by the decree adapting the national legal system to the GDPR 2016/679 (Legislative Decree No. 101 of 10 August 2018).

Acknowledgments. This work has partially been supported by the Activity no. 18163, "API Assistant -Automated security assessment of 3rd party apps for the API economy", funded by the EIT Digital.


Sascha Van Schendel
email: s.vanschendel@tilburguniversity.edu

Risk Profiling by Law Enforcement Agencies in the Big Data Era: Is There a Need for Transparency?

Keywords: Risk Profiling, Transparency, Law Enforcement, Procedural Safeguards, False Positives, Discrimination, Data Protection, Criminal Law, Explanation

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Risk assessment has become very popular in all sectors of society, including in the prevention against crime. Over the last years, the term 'Big Data' has taken flight and has increasingly received much attention in government policies and practices. [START_REF] Van Der Sloot | International and Comparative Study on Big Data[END_REF] The use of Big Data analysis is in part the reason for a strong emphasis on preventing and minimizing risk in society. Having the tools to analyze huge volumes of data and extract information from them, possibly completely by automated means, facilitates processes such as the creation and analysis of risk profiles. [START_REF] Marks | Automatic justice? Technology, Crime and Social Control[END_REF] The use of profiles grows as they can be constructed and applied more easily, while at the same time the construction, analysis and application of the profiles become more complicated and opaque. The use of risk profiles to find suspects or determine if someone poses a risk to society has traditionally been an important tool to national law enforcement agencies to efficiently make use of their powers. Some scholars have described the emphasis on risk in criminal justice as entering into an era of actuarial justice [START_REF] Kemshall | Understanding risk in criminal justice, Crime and Justice Series[END_REF] in which we focus on analyzing risk in a mathematical way, the rise of 'the logic of risk' [START_REF] Ericson | Policing the Risk Society[END_REF], or 'the new paradigm of criminal law' [START_REF] Koops | Technology and the Crime Society: Rethinking Legal Protection[END_REF]. While there are arguments to make in favor of law enforcement agencies making their practices more efficient by using risk profiles [START_REF] Zouave | An Inconvenient Truth: Algorithmic Transparency & Accountability in Criminal Intelligence Profiling[END_REF], this development is not without its issues and raises issues towards those affected by this practice. This leads to the first hypothesis of the paper: risk profiling in the Big Data era creates challenges. Based on this hypothesis of risk profiling creating challenges, this paper addresses the question whether we need transparency of risk profiling by law enforcement actors, from the perspective of protecting fundamental rights of those affected by the use of risk profiles. This research question also contains the second hypothesis of this paper, namely that transparency could be an interesting angle to tackle the challenges. The aim of this paper is to shed light on the challenges of risk profiling. Exploring whether transparency is a way to approach these challenges is intended as a starting point of a discussion. This paper does not provide an analysis of how transparency will solve the challenges of risk profiling, nor does the author outline what transparency should look like in this context. This is the topic of future research of the author.

Section 2 of this paper briefly maps risk profiling by law enforcement actors in practice. One specific example is taken as a case study to be explored in more detail. This example is SyRI (System Risk Indication), a Dutch risk profiling system. This example was chosen as it is currently under review in a national court case. SyRI is a good example of risk profiling that presents the starting point of a criminal investigation. Some parallels are drawn to the USA in Section 2, as some of the types of use of risk profiles are still very minimal in the European Union but might become more prominent following the USA's example. Section 3 describes the main challenges of the use of risk profiling, grouping them under three main, non-exhaustive headers: errors, discrimination and stigmatization, and lack of procedural safeguards or outdated safeguards. Section 4 describes why transparency might be an interesting angle to approach the challenges. For this purpose, Section 4 introduces and briefly describes Heald's 'varieties of transparency' [START_REF] Heald | Varieties of Transparency[END_REF] as a theoretical model. Subsequently, Section 4 narrows transparency down to foster further discussions, as transparency in itself is a very broad concept. For this purpose a bottom-up approach to the issues is chosen, focusing on explanations as a means of transparency. The focus on explanations is all the more relevant after the introduction of the General Data Protection Regulation [START_REF]679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data[END_REF] ('GDPR'), as it contains references to explanations in the context of automated decision making. Section 4 will therefore also briefly mention transparency and explanations under the GDPR and the Law Enforcement Directive [START_REF]680 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data by competent authorities for the purposes of the prevention, investigation, detection or prosecution of criminal offences or the execution of criminal penalties, and on the free movement of such data, and[END_REF] ('LED').

2

Risk Profiling in Practice

What is Risk Profiling?

Risk profiling, for the purpose of this paper, is categorizing or ranking individuals or groups, sometimes including automated decision making, using correlations and probabilities drawn from combined and/or aggregated data, to determine the level of risk that is posed to the security of others or national security by those individuals or groups. 1 The most prominent type of risk here is the likelihood of an individual or group (re)committing crime.

Risk profiling can take many forms in the law enforcement context. Risk profiling can be used in concrete criminal investigations where there is already an identified suspect or perpetrator and a profile is applied to this person. A first instance is to make decisions about which police powers to employ. Brkan gives the example of automated decision making to determine whether to seize a mobile device. [START_REF] Brkan | Do algorithms rule the world? Algorithmic decision-making in the framework of the GDPR and beyond[END_REF] Risk profiling of an identified individual can also be targeted towards future behavior. This can be risk profiling to determine whether someone is allowed bail or probation specifically whether that person is at risk of reoffending, or risk profiling in sentencing determining the duration of incarceration. The most famous example is from the USA, namely COMPAS. COMPAS is an algorithm used by judges and probationand parole officers to assess a criminal defendant's likelihood of reoffending. [START_REF] Van Brakel | Pre-Emptive Big Data Surveillance and its (Dis)Empowering Consequences: The Case of Predictive Policing[END_REF] There are types of risk profiling where the target is a location. These are often types of predictive policing drawing from various sources of data, ranging from non-personal data such as the distance to the highway to different forms of personal data pertaining to inhabitants of that area such as the history of criminal records. Algorithms can in this way pinpoint the level of risk for areas, so that police officers can be deployed accordingly. This type of risk profiling is very popular in the USA, but also exists in Europe.

[12] Such as in the Netherlands, where the Crime Anticipation System is used, creating a grid that is updated every 14 days which shows for each square what crime is likely to take place and on which time of day. This system was at first only applied in the capital, Amsterdam, but is now being used in various other cities. While such a system is targeted at the risk level of a location, it indirectly profiles the residents of that area. This is where discussions on stigmatization and self-fulfilling prophecies come in: by attaching a risk label to a certain area and sending police patrols there accordingly, this can impact the view residents and outsiders have of this area plus lead to an increase in crime detection further increasing patrols and measures taken against residents of this area. Indirectly the residents are also profiled as high risk. Of course this means that there is an assumption that the suspects or perpetrators would reside in this area, while this does not have to be reality.

Besides the above described type where law enforcement applies profiles to an already identified individual or area, risk profiles are also used to detect individuals -or groups-that fit the profile. In these cases an algorithm finds individuals that fit the risk profile in a haystack of data. These individuals are likely to commit a crime or are likely to have committed an undetected crime. This type of profiling does not take place within the boundaries of a specific criminal investigation but rather leads to the starting point of one. Risk profiling to detect individuals can take the form of 'heatlists', similar to the heatmapping or area profiling described above. An example from the USA is the system Intrado Beware, which is a mobile, cloud-based application, sold to the police, that gathers contextual information from social media, commercial data and criminal data, creating a risk score -green, yellow, red-for individuals.[12] Intrado Beware is slightly different from the standard model of detecting people who have committed a crime, as it is more targeted towards providing police information about the person they are about to encounter and identifying whether they are a risk in the sense of posing a risk to the security of the police officer. Another example of finding individuals that match the risk profile comes from the Netherlands, which is described in the section below.

The SyRI Case

An example of risk profiling can be found in the Netherlands in the SyRI ('System Risk Indication') program. SyRI was officially launched in 2014 and is employed by the Dutch Ministry of Social Welfare & Employment. It is a system in which many databases are combined -ranging from tax data and data about social benefits to data about integrating in Dutch society and education-, creating a large data pool to detect fraud. [START_REF] Besluit | Staatsblad[END_REF] SyRI targets three types of fraud: unlawful use of social benefits, taxation fraud, and fraud with labor laws. [START_REF] Besluit | Staatsblad[END_REF] Due to the broad scope and large governmental database, almost every citizen of the Netherlands is present in the database. Using a predetermined risk model, the system searches for correlations in the database flagging a potential case of fraud based on the model used for that specific search.

[14] The individual is given a risk indication, which is forwarded to the Dutch National Police and/or prosecuting office, who then decide whether to investigate further. The risk indication is stored in a register which relevant public bodies can access. [START_REF] Besluit | Staatsblad[END_REF] So even though SyRI is not a specific risk profiling program of law enforcement solely, law enforcement is one of the parties that can be included in a cooperation to use SyRI and the risk score of SyRI can be the data point that starts a criminal investigation.

Even though SyRI has been used for a couple of years now, its use has not been without resistance. There have been parliamentary debates centered on the question whether SyRI met proportionality demands and whether its legal basis was not too broad. The program raises issues of transparency, mainly awareness and contestability. Most citizens are not aware that their data is in this system nor that they might be flagged. Most people are confronted with the existence of the system when they receive an administrative fine or encounter another negative consequence. Besides possible privacy and data protection issues that follow from a system that uses so much data, there are serious issues with possibilities to contest the system and correct errors. In March 2017, several NGOs and two citizens took up the initiative to launch a court case, which is still ongoing, to test whether SyRI is compliant with EU data protection legislation, the fundamental right to privacy and the right to fair trial under article 6 of the European Convention on Human Rights.[15] One of the points that is debated is the secrecy of the risk models, but also the lawfulness of the automated decision making and the broadness of the legal basis. [15] In this sense the problematic aspects of SyRI illustrate the challenges following from data driven policing or policing in the Big Data era, such as risk profiling.

Risk Profiling: Challenges

This section groups the challenges of risk profiling under three main headings: errors, discrimination and stigmatization, and lack of procedural safeguards or outdated safeguards. This is a non-exhaustive list but aims to give an oversight of the main challenges based on literature about profiling, algorithms, predictive analysis and data analysis in the law enforcement domain.

Errors: Relying on Statistics and Probabilities

Most profiles are probabilistic, describing the chance that a certain correlation will occur. [START_REF] Hildebrandt | Defining Profiling: A New Type of Knowledge? In: Profiling the European Citizen[END_REF] In most cases the individuals included under the profile do not share all the attributes or characteristics of the group profile. [START_REF] Hildebrandt | Defining Profiling: A New Type of Knowledge? In: Profiling the European Citizen[END_REF] This is especially true for nondistributive profiles, which are framed in terms of probabilities and averages, comparing members within a group or category, or comparing those groups or categories to each other. [START_REF] Vedder | KDD: The challenge to individualism[END_REF] This means that there is always an inherent risk of errors in the use of profiles, as it might include people erroneously within a profile or might miss certain individuals, leaving them out of scope. The first category is false positives, the second situation is false negatives. [START_REF] Hildebrandt | The Challenges of Ambient Law and Legal Protection in the Profiling Era[END_REF] In case of false positives, people would be incorrectly classified in a group or profile. This in turn could have consequences for decisions taken to the disadvantage of these persons, or they could be erroneously subjected to police powers. In the case of a false negative, we encounter the more traditional problem of law enforcement, namely overlooking someone who should be a suspect or miscalculating the risk of recidivism. Especially in the context of terrorism threats, risk profiles aim at minimizing false negatives, as the societal consequences are a lot graver when allowing for a false negative than a false positive. [START_REF] Leese | The new profiling: Algorithms, black boxes, and the failure of anti-discriminatory safeguards in the European Union[END_REF] Mittelstadt et al. talk about these issues in terms of 'inconclusive evidence', meaning that algorithms often draw from statistics and in doing so create only probable outcomes that are focused more on actionable insights than causal relations. [START_REF] Mittelstadt | The ethics of algorithms: Mapping the debate[END_REF] Algorithms become increasingly complex and autonomous, which makes it harder for law enforcement to be transparent about why they receive a certain outcome. Mittelstadt et al. refer to this complexity and opaqueness as 'inscrutable evidence', where humans have trouble interpreting which data points lead to the conclusion. [START_REF] Mittelstadt | The ethics of algorithms: Mapping the debate[END_REF] Risk profiling in the Big Data era relies heavily on algorithms and statistics. Statistics offer insight into numbers, for example how many people re-offend within an amount of years. Algorithms can be used to combine statistics, mine them for patterns, and make a prediction about an individual's behavior by applying this information to their situation. This does not mean however that this person acts according to the statistics nor that the conclusion based on combining statistics is right. If the process becomes more complex and opaque it can become harder for law enforcement agencies to demonstrate why they received this outcome.

Discrimination & Stigmatization

The trend of risk management combined with the strong focus in politics on terrorism prevention can push law enforcement to target specific groups, especially with the pressure to fully use technologies such as algorithms and Big Data analysis. The technology, to a large extent, takes over tasks that were not fully automated before. Now algorithms take over the task of detecting the patterns, creating the profiles and finding correlations. [START_REF] Koops | Technology and the Crime Society: Rethinking Legal Protection[END_REF] As these technologies are not foolproof -just as police officers' instincts and human observation and logic are not foolproof-this does pose a threat of discrimination and stigmatization of certain groups. The technology might 'over target' specific groups. It has been shown already that risk-based policing targets certain societal groups within different EU countries, such as North African youths, soccer supporters, Roma, and Muslims. [START_REF] Leese | The new profiling: Algorithms, black boxes, and the failure of anti-discriminatory safeguards in the European Union[END_REF] The technology might increase racial or ethnical profiling especially. For example, in the Netherlands, the existence and possible condoning of ethnic profiling by police officers has been a topic of societal debate for years. [START_REF] Van Der Leun | Ethnic Profiling in The Netherlands? A Reflection on Expanding Preventive Powers, Ethnic Profiling and a Changing Social and Political Context[END_REF] While these types of debates were mainly targeted at racial profiling based on 'police instinct', automated profiling possibly increases racial profiling. [START_REF] Leese | The new profiling: Algorithms, black boxes, and the failure of anti-discriminatory safeguards in the European Union[END_REF][START_REF] Mittelstadt | The ethics of algorithms: Mapping the debate[END_REF] As Van Brakel explains: "Predictive mapping can potentially lead to ethnic profiling. If arrest rates are a measure for predicting in which areas most crime occurs, for instance, and if it is clear that arrest rates are disproportionately higher in particular population groups as a result of ethnic profiling there is a clear bias in the prediction, and the mapping can lead to even more ethnic profiling".

[12] Referring back to the example of the Dutch predictive policing application CAS, ethnic profiling has already been demonstrated to be an issue. [START_REF] Van Der Leun | Ethnic Profiling in The Netherlands? A Reflection on Expanding Preventive Powers, Ethnic Profiling and a Changing Social and Political Context[END_REF]. When using automated means, all the data analysis is scaled up, increasing the scale of the problematic aspects. Profiling in itself is a discriminatory process, which is not illegal in itself, but can become illegal discrimination if based on factors such as race or religion. [START_REF] Schermer | The limits of privacy in automated profiling and data mining[END_REF] Article 11 of the Law Enforcement Directive prohibits the use of sensitive data -officially called 'special categories of data' 2 -unless suitable safeguards are in place to protect the interests of the data subject. So when using sensitive data such as ethnicity or religion extra safeguards might need to be put into place. However, provisions that forbid the use of these types of factors or require extra safeguards, do not prevent the use of proxies. The use of proxies could nonetheless be discriminatory, such as using zipcodes or income as a proxy for ethnicity. Discrimination following directly from automated decision making is forbidden as far as the special categories of data go. Profiles that focus on other characteristics -or proxies for those characteristics-, such as age can also be deemed illegal. Recently, a court in the Netherlands ruled that the use of a risk profile -of single men of 55 years or olderwas in violation of the right not to be discriminated against. [23] It is extremely hard, however, to tackle illegal discriminatory profiling if the impacted individuals are not aware that they are placed in a certain profile. As Leese states: "as datadriven profiles produce artificial and non-representational categories rather than actual real-life social groups, the individual is likely to not even notice when he or she becomes part of a 'risky' category". [START_REF] Leese | The new profiling: Algorithms, black boxes, and the failure of anti-discriminatory safeguards in the European Union[END_REF] Besides individuals not being aware, the actors operating the algorithm might also be unaware of illegal discrimination happening in their dataset or algorithm, or they might be unaware that their use of proxies has the same result as the illegal discrimination based on certain characteristics. These problems are only more difficult to detect and address as systems get more complex.

2 Special categories of data under the GDPR and LED are data that are deemed especially sensitive and therefore receive more protection. The set categories are: data revealing racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership, and genetic data, biometric data for the purpose of uniquely identifying a natural person, data concerning health or data concerning a natural person's sex life or sexual orientation.

Procedural Safeguards

Automated risk profiling works in a different way than the more traditional policing, creating challenges in the way in which safeguards are set up. First, risk profiling is a form of proactive or even preventive policing. This forms a contrast to the more traditional reactive policing. Koops has referred to a shift in paradigm in criminal law which for example contains a focus on prevention, risk, groups, profiling, and statistics. [START_REF] Koops | Technology and the Crime Society: Rethinking Legal Protection[END_REF] An issue is that safeguards might be linked to the prosecution phase, leaving out the opaque pre-investigation practices where a lot of data is already analyzed. [START_REF] Koops | Technology and the Crime Society: Rethinking Legal Protection[END_REF] In reactive policing the focus for checks and balances is traditionally on the judge, who comes in at the later investigation stages or only at the trial. However with risk profiling someone might be arrested erroneously and released shortly after. Similarly, with risk profiling used in general policing, a lot of data is analyzed and privacy infringements could take place there as a consequence, but go undetected because there is no criminal investigation of a specific suspect yet. Second, in the information society decisions are increasingly made based on group profiles. [START_REF] Hildebrandt | The Challenges of Ambient Law and Legal Protection in the Profiling Era[END_REF] In literature on data protection and privacy there are increasingly more debates on the possibilities for collective procedures to address types of data processing such as Big Data analytics and group profiling. [START_REF]Personal data for decisional purposes in the age of analytics: From an individual to a collective dimension of data protection[END_REF] Vedder, in his work on KDD (Knowledge Discovery in databases), already signaled a tendency of treating people based on group characteristics. [START_REF] Vedder | KDD: The challenge to individualism[END_REF] This tendency has only grown with modern risk profiling, as risk profiling requires statistics and categorizing or ranking of people. Vedder discusses data that for example used to be personal data but during time has become part of a broader set of anonymous data, at some stage the data became part of aggregate data and individual identifiers were replaced with group identifiers. [START_REF] Vedder | KDD: The challenge to individualism[END_REF] As Vedder precisely states, using generalizations and categorizations based on profiles can be highly problematic when they are used as a basis for policy and people are treated as a member of a group instead of on their own merits. [START_REF] Vedder | KDD: The challenge to individualism[END_REF] However, safeguards and rights are often linked to individual decision making. Automated decision making under article 11 of the Law Enforcement Directive, which produces an adverse legal effect concerning the data subject or significantly affects him or her, is prohibited unless authorized by national law and provided with appropriate safeguards. Profiling is concerned with creating a set of correlations on the aggregate level and subsequently applying it to individuals or groups. One could argue that only the application of a profile to an individual situation is regulated here. Brkan gives the example of a group being the target of profiling by making an automated decision to patrol certain areas, affecting the lives of the people who live in such an area. [START_REF] Brkan | Do algorithms rule the world? Algorithmic decision-making in the framework of the GDPR and beyond[END_REF] Again, reference could be made to the Dutch predictive policing system, CAS, indicating where and when which crimes are likely to take place. Based on those risk indications police officers are deployed, but it is not clear whether the decision to target areas as high risk areas meets the criteria of article 11 of the Law Enforcement Directive to require further safeguards.

Transparency

Using Transparency to Address Challenges

Having presented the most prominent challenges of risk profiling by law enforcement agencies, the issue is how to address these challenges. I propose to look at the concept of transparency for solutions to these challenges.

Transparency has possibilities to expose flaws or give insight into decision making. A lot of the challenges relate to processes being opaque. For example, maybe it is not visible that someone is placed in the wrong category or that there is illegal discrimination taking place. Or, because of a lack of procedural safeguards in the early investigation, mistakes do not come to light. In that sense transparency also increases possibilities of awareness. A lot of people are simply not aware that they are being profiled or that a decision about them, for example concerning arrest or deploying investigative measures, is based on a risk profile. A lack of awareness makes it difficult for those affected by risk profiling to check for compliance with their rights when necessary, such as the right to fair trial, equality of arms, privacy, or the principle of nondiscrimination. Therefore transparency might be interesting to look further into. However, transparency is a very broad concept and has different meanings even within one discipline. Several authors have already described the relation between transparency and a concept that is often connected to it, namely 'openness'. For example Birkinshaw proposes that transparency and openness are close in meaning but are both broader than merely access to (government) information. [START_REF] Birkinshaw | Freedom of Information and Openness: Fundamental Human Rights[END_REF] Larsson also does not consider openness and transparency to be the same concept, as according to Larsson transparency goes beyond openness and also includes simplicity and comprehensibility. [START_REF] Larsson | How Open Can a Government Be? The Swedish Experience[END_REF] Heald remarks that transparency has become 'the contemporary term of choice' for describing an openness of public actors about actions and decisions they make. [START_REF] Heald | Varieties of Transparency[END_REF] However, Heald makes various distinctions within the concept of transparency. [START_REF] Heald | Varieties of Transparency[END_REF] These distinctions are helpful to dismantle the broad concept and distinguish which functions or solutions transparency actually offers. Therefore Heald's work on transparency is briefly discussed here as a theoretical framework.

First, Heald makes explicit different directions of transparency. There are two directions of vertical transparency: upwards and downwards. Upwards transparency can be seen in hierarchical terms of allowing the superior to observe behavior or results. Downward transparency can be seen in terms of democracy, allowing the ruled to observe behavior or results of their rulers. [7, p. 27] Second, Heald discusses the two directions of horizontal transparency: outwards and inwards. Transparency outwards occurs when the hierarchical agent can observe behavior outside of its organization or institution, so as to understand the domain it is operating in and observe the behaviour of peers. Transparency inwards occurs when those outside of the organization can observe what is happening within the organization. [7, p. 28] Next Heald distinguishes different varieties of transparency in general using three dichotomies: event transparency versus process transparency; transparency in retrospect versus transparency in real-time; nominal transparency versus effective transparency. When distinguishing between events and processes, an event can for example be the input or output data. When providing process transparency one can be transparent about the procedural factors -which rules are followed-or operational aspects -how are the rules applied in this situation-. [7, p 29-32] Another dichotomy is the temporal one, so one can allow for transparency after the fact -in retrospect-or one can continuously allow for transparency so that transparency takes place in real-time. [7, p. 32-33] For the last dichotomy Heald states that there can be a gap between nominal and effective transparency, which he labels the 'transparency illusion'. [7, p. 34] Allowing for transparency does not always mean that it is effective: "For transparency to be effective, there must be receptors capable of processing, digesting, and using the information". [7, p. 35] Also, transparency is not effective when it creates an information overload. [7, p. 35] After having some more insight into the concept of transparency, it is interesting to see how this theory relates to the problem at hand. First, concerning the vertical transparency: in the context of data processing by law enforcement actors, upwards transparency is concerned with transparency towards oversight authorities such as Data Protection Authorities or (investigatory) judges. Downwards transparency is directed towards the people that are the subject of the process, in the case of automated decision making this concerns for example the data subjects. When looking at horizontal transparency, inwards transparency can be offered to oversight authorities, the people affected by the data processing, the democracy or people at large, and so forth. In the context of law enforcement outwards transparency is not so relevant. When distinguishing between events and processes it becomes clear that in the case of risk profiling there is a large variety in what transparency could be given about. Transparency can for example concern events such as the input of new data, or the outcome that the algorithm gives. On the other hand transparency could be given about the process, such as procedural aspects like the decision rules, which in this case could be the algorithm itself. Concerning the process, transparency could also be provided about the operational aspects, focusing on a specific situation, explaining why the decision rules have in this case led to this outcome. With regard to the temporal dimension transparency could be given in retrospect, for example notifying oversight authorities or individuals that a decision has been made based on a risk profile. Or transparency could be offered in real-time, which in the case of law enforcement seems complicated, as this might pose difficulties for ongoing investigations. With regards to the dichotomy between nominal and effective transparency, a lot of issues are left open. To determine the effectiveness of transparency of risk profiling would be quite difficult.

Based on the description above, there is still a lot of variation possible in to whom transparency is offered, about what elements of risk profiling transparency is given, and what constitutes effective transparency. Transparency in risk profiling could have varying functions. However, going back to the research question of this paper, -to determine whether transparency could help with the challenges from the perspective of protecting the rights of those affected by the risk profiling-transparency needs to be narrowed down further along Heald's varieties of transparency. In focusing on those affected by risk profiling, downwards-inwards transparency is the relevant variety. When targeting transparency towards data subjects, and others that might be affected, three steps could be distinguished. The first step is to make data subjects aware that data processing and risk profiling is taking place. The second step is to explain to data subjects what is going on and how certain decisions are made. These two steps enable the third step, being able to contest profiles and automated decisions and receive due process. Perceiving transparency in this bottom-up way makes it easier to grasp the overall concept of transparency and connects to the challenges. It is after all important in safeguarding the rights of individuals affected that they are not erroneously profiled, illegally discriminated against, or undergoing a process without enough procedural safeguards to protect fundamental rights such as the right to a fair trial.

Alternatively, it is interesting to assess in the context of upwards transparency how law enforcement actors will explain their profiling practices and decisions to judges, or other competent authorities, when the analysis becomes more intricate and decisions more data driven. This is, however, a dimension of transparency that will largely take place behind closed doors and very difficult to analyze as researchers.

4.2

Food for Thought: Explanations as a Means of Transparency?

One aspect or means of offering transparency to data subjects, and others affected by risk profiling, is that of providing explanations of the profiling. Explanations of profiling and automated decision making have become very relevant with the reform of EU data protection legislation. To go further into this, a brief description of EU data protection legislation in the context of transparency is needed. EU data protection legislation consist of several pieces of law. In 2016 the reform package for Data Protection legislation on the European Union level was adopted, introducing the General Data Protection Regulation [START_REF]679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data[END_REF] ('GDPR') and the Law Enforcement Directive [START_REF]680 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data by competent authorities for the purposes of the prevention, investigation, detection or prosecution of criminal offences or the execution of criminal penalties, and on the free movement of such data, and[END_REF] ('LED'). Before the introduction of the LED, data protection in this area was left in part to national legislation, partly standardized by Convention 108 of the Council of Europe [START_REF]Convention for the Protection of Individuals with regard to Automatic Processing of Personal Data[END_REF], and in part regulated by a variety of specialist and sector specific instruments, creating a very fragmented landscape. [START_REF] Hert | The Police and Criminal Justice Data Protection Directive: Comment and analysis[END_REF] The LED repeals the Council Framework Decision 2008/977/JHA [START_REF]Council Framework Decision[END_REF], which was very narrow in scope, only applying to cross-border transfers and exchanges of personal data, excluding domestic processing of personal data. [START_REF] Marquenie | The Police and Criminal Justice Authorities Directive: Data protection standards and impact on the legal framework[END_REF] As the regulation of the processing of personal data by national law enforcement agencies has been left out of harmonization so far, a wide margin is left to the criminal procedural law of Member States to lay down requirements and safeguards. For data processing in the private sector it is logical to look for requirements and safeguards in the GDPR, but for data processing by national law enforcement agencies the LED needs to be seen together with the safeguards and requirements following from Member States' legislation that arranges the competencies of these actors. The current Law Enforcement Directive does not contain a general principle of transparent processing. Under relevant Council of Europe law this is different. The newest version of Convention 108, which also applies to the law enforcement domain, does contain a principle of transparent data processing under article 5. 3 When comparing the GDPR and the LED, a fundamentally different approach with regard to transparency becomes visible. Transparency takes a predominant place within the GDPR in the form of transparent processing 4 , combined with various rights towards the data subject. Transparent processing in this sense can mean that data subjects are informed about processing before it takes place, during the processing itself and upon request of the data subject. Besides the principle of transparent processing that applies throughout all types of processing, articles 12 until 14 of the GDPR impose obligations on the side of data controllers as well as rights upon data subjects to request information. In the context of profiling especially article 13 is relevant where, in the context of providing information, it states: "(…)the existence of automated decision-making, including profiling, referred to in Article 22 [START_REF] Van Der Sloot | International and Comparative Study on Big Data[END_REF] and ( 4) and, at least in those cases, meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject." Recital 39 of the GDPR also pays specific attention to transparency, it states: "The principle of transparency requires that any information and communication relating to the processing of those personal data be easily accessible and easy to understand, and that clear and plain language be used". Thus this principle already implies; first, that information about the processing should be available to the data subject; second, that this information should be easy to access; third, the information itself should be easily understandable. The same aspects of transparency are highlighted in recital 58. Recital 60 underlines the importance of awareness as a component of transparency, by stating that transparency requires the data subject being informed of the existence of the processing. In contrast, in the LED the principle of transparent processing is not present. The only relevant reference to transparency is in recital 26. Recital 26 merely mentions that processing should be done in a transparent manner with regard to the persons concerned, while at the same time acknowledging the necessity of some covert operations and surveillance measures. The critical component is "provided by law and constitute a necessary and proportionate measure in a democratic society with due regard for the legitimate interests of the natural person concerned". So the requirements for transparency and limits of opaqueness are determined on a case to case basis where practices differ much from country to country. Again, national criminal procedural law also has a role to play here, as regulation is left to the Member States in this area. Some countries might have more provisions on transparency than others.

Transparency in the context of data processing has also become a much debated topic in literature. On the one hand arguments are presented in favor of more transparency of algorithms and algorithmic decision making [START_REF] Zarsky | Transparent Predictions[END_REF], on the other hand there is a continuously increasing awareness that 'transparency' as such is not an allencompassing answer for issues with algorithms. [START_REF] Mittelstadt | The ethics of algorithms: Mapping the debate[END_REF][32] Transparency more often than not, requires balancing of transparency as a value and other values such as protecting trade secrets, national security, and privacy of others. [19][20] Especially in literature on the law enforcement sector, transparency is discussed in the context of a trade-off or balance between security and transparency -sometimes as an aspect of the right to privacy-. [START_REF] Brkan | Do algorithms rule the world? Algorithmic decision-making in the framework of the GDPR and beyond[END_REF] However, that there are arguments in favor of law enforcement agencies operating under a certain level of secrecy, does not mean that there is no room for transparency at all. Requiring law enforcement to explain why someone is profiled in a certain way puts up a safeguard in general against illegal discrimination and errors, as requiring an explanation stimulates checking the analysis to see whether the proper data was used and how the data was weighted to come to this result, as well as the level of probability. Providing explanations also serves a more specific purpose. While the process of profiling becomes more automated and technically complicated, it is important that law enforcement actors can still understand how a profile or decision came about, putting up a safeguard against algorithms that become so opaque and complex that humans cannot understand or justify the outcomes anymore. Law enforcement agencies need to be able to explain their decisions to a judge that checks the legality of, for example, searching a phone or computer; public prosecution needs to be able to explain during a trial why the prosecution authorities started the investigation, meaning why the person in case was suspect according to the risk profiling system. This requirement is inherent in criminal justice systems [START_REF]Commissie modernisering opsporingsonderzoek in het digitale tijdperk, Regulering van opsporingsbevoegdheden in een digitale omgeving[END_REF], if law enforcement cannot explain a decision, the judge will probably not accept it. However, giving these sort of explanations might be more challenging in automated processes, or processes with minimal human intervention. Therefore, it would be good to lay down an explicit requirement in national law, whether in data protection legislation or criminal law, for explaining profiling and automated decision making. [START_REF]Commissie modernisering opsporingsonderzoek in het digitale tijdperk, Regulering van opsporingsbevoegdheden in een digitale omgeving[END_REF] While the actors using a risk profiling system need to maintain a certain understanding of how it works so that they can be accountable for their decisions, it is equally important that the human actors involved do not over rely on the technology. In literature this has been discussed as 'automation bias', meaning that humans have a tendency to over rely on the accuracy of automated analysis and decisions, the result is assumed to be correct and no counterfactual evidence is sought out. [START_REF] Cummings | Automation Bias in Intelligent Time Critical Decision Support Systems[END_REF] With this risk in mind, explaining the decision also ensures that human actors do not take the outcome for granted but investigate how it came about.

As stated in the introduction, this paper is not the place to develop what explanations of risk profiling in the law enforcement sector should or could look like exactly. It does offer food for thought though, especially with all the new transparency provisions under the GDPR.

Conclusion

Preventive and risk based policing is increasingly becoming the new form of policing. However, safeguards might be attuned to more traditional, less data-driven, policing and criminal procedures. This means that now and in the future there will be challenges on this front, such as dealing with probabilities, discrimination, effects on groups and shifting to the pre-investigation phase. This paper proposed to look at transparency for dealing with these challenges. Making the broad notion of transparency more feasible to grasp using Heald's varieties of transparency, it becomes clear that there are a lot of different options regarding to whom transparency could be offered and what the object of this transparency would be. Basing decisions on these risk profiles can have very serious consequences from the perspective of those affected by the risk profiles, for example when it comes to their rights not to be discriminated against or the right to fair trial and equality of arms in being subjected to complex data analysis that might even concern future behavior. While transparency is prominent in the GDPR and in literature concerning the GDPR, the debate about transparency is not really taking place yet in the literature about the LED and literature about profiling in the law enforcement sector. The increasing use of Big Data and algorithms in policing and in prosecution, such as in the form of profiling and automated decision making will, however, only make transparency more important to discuss. This paper made a first step in discussing why transparency is important, by examining the challenges of risk profiling and the different options of transparency, and why we especially need explanations in the law enforcement domain as a means of transparency. The time has now come to also talk about explanations of profiling in the law enforcement sector to assess what role they could play and what they could look like.

This definition of risk profiling is the author's own and is a working definition.

The Convention 108 has recently been modernized. The amending Protocol (CETS No. 223) to Convention 108 was adopted by the Committee of Ministers of the Council of Europe on 18 May 2018.

Under article

of the GDPR.