Matthias Marx

Erik Sy

Christian Burkert

Hannes Federrath

Anonymity Online -Current Solutions and Challenges

Keywords: privacy, anonymity, tracking, fingerprinting

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction to Anonymity

Anonymity online is the effort to communicate over the Internet while disclosing no or as little as possible of directly or indirectly identifying information unless explicitly wanted. Identification happens directly if someone discloses his or her own name accidentally or deliberately, e.g., when registering an account or signing a message. Indirect ways of identification emerge when disclosed information can be combined with additional knowledge containing sufficiently identifying information. When communicating over the Internet, being literally anonymous could mean that neither a communication partner, any intermediary, nor an outside observer is able to identify a communicating person.

This paper provides an overview of past and contemporary efforts to provide anonymity throughout different technology layers involved in online communication. Sect. 1 provides the legal background on online anonymity, the definition of important terms and a motivation for doing research in the field of online privacy. Sect. 2 discusses the problem of and countermeasures against application-level tracking, followed by a discussion of network-level anonymisation techniques in Sect. 3. Sect. 4 concludes with an overview of the presented challenges regarding anonymity online.

Anonymity from a Legal Perspective

From a legal perspective, the term anonymity is known in association with privacy and data protection regulations. In this section, we focus on EU regulations and court decisions, i.e. the General Data Protection Regulation (GDPR), the ePrivacy Regulation, and a fundamental decision by the European Court of Justice regarding the identifiability of personal data.

EU General Data Protection Regulation (GDPR)

The GDPR mentions anonymity only in its recitals. Recital 26 defines data as anonymous when it is either not related to an identified or identifiable natural person or it has been rendered anonymous such that the data subject is not or no longer identifiable. Therefore, anonymous data is not considered as personal data. Its processing does not fall within the material scope of the GDPR as defined in Article 2.

Whether given data is anonymous or not comes down to the question of how much effort is needed by an attacker to identify a person and how likely it is that the attacker undergoes such efforts, e.g., by linking several attributes of an anonymized dataset with a priori information held by the attacker. Recital 26 lists aspects that should be taken into consideration if one assess which means are likely to be used by an attacker. Aspects to be considered are the costs and the amount of time required for identification as well as the available technology at the time of the processing and technological developments.

EU ePrivacy Regulation

As part of the amendment of EU data protection regulations, the ePrivacy Regulation (ePR) is developed to repeal the current ePrivacy Directive and to complement the GDPR as a lex specialis for electronic communication services.

Article 10 of the Commission's draft [START_REF]European Commission: Proposal for a regulation on privacy and electronic communications[END_REF] proposes the obligation for software manufacturers to provide privacy settings that allow end-users to prevent third parties from storing or processing information on their devices. Third parties are components and services that are used by the first party services to fulfill a specific task like customer research or advertisement. Recital 22 of the Commission's proposal describes the proposed privacy settings as a more user-friendly approach to express consent to website cookies compared to the overwhelming numbers of consent requests currently prompted by individual websites. Recital 23 sees such privacy settings as an implementation of the principles of data protection by design and by default as defined in Article 25 of the GDPR and suggests differentiated cookie settings that should be prominently and intelligibly presented to end-users.

Regardless of the explicit reference to cookies in the recitals, Article 10's opt-out of processing information already stored on end-user devices could also be interpreted as a prohibition of techniques that utilise stored device information, e.g. device fingerprinting. Such a wider interpretation of Article 10 would require software manufacturers to further confine the capabilities of third parties to query device or user-specific information. As a consequence, web browser manufacturers could be obliged to put code from third parties into a sandbox environment and deny unredacted access to senstive APIs.

Identification through IP Addresses

In October 2016, the European Court of Justice (ECJ) ruled in case C-582/14 [START_REF]Judgement on Case C-582/14[END_REF] with regard to the nature of dynamically assigned IP addresses, that such addresses, when registered by an online media service provider, constitute personal data, provided that the online media service provider has the legal means to identify a user behind the IP address by consulting additional data stored e.g. by the Internet Service Provider (ISP) of that person. This decision clarifies the fundamental dispute between the relative and the absolute approach to the identification of data subjects [START_REF] Reid | The European Court of Justice case of Breyer[END_REF]. While the relative theory states that the question of identification depends on the individual processing context and the knowledge of the processor, the absolute theory considers data as identifiable as long as anyone has the means to attribute that data to a natural person [START_REF] Reid | The European Court of Justice case of Breyer[END_REF]. ECJ followed the relative approach, but additionally used a wide interpretation of means of identification such that also information and capabilities of third parties should be taken into consideration if the processor has the legal means to utilise them.

Formalising Anonymity

Legal definitions of anonymity and identifiability are designed to be adaptable to future technological developments. Additionally, there is also a demand for more formalised and objective notions to facilitate compliance or to objectify scientific efforts. This section presents various formal definitions of anonymity and of algorithmic properties, which limit the disclosure of potentially personal data.

Anonymity Set Pfitzmann and Hansen [START_REF] Pfitzmann | A terminology for talking about privacy by data minimization: Anonymity, unlinkability, undetectability, unobservability[END_REF] define anonymity as not being identifiable within a set of subjects, the anonymity set. Not being identifiable means that the subject is not distinguishable from other subjects within the anonymity set. Pfitzmann and Hansen describe the ability to distinguish subjects as a function of the attacker, its knowledge and capabilities. Thus, they state anonymity as a property relative to an attacker. A subject can be considered as being identified, if an attacker can attribute a given action to that subject with a probability exceeding a certain threshold. Pfitzmann and Hansen differentiate between anonymity of individual users of a system and global anonymity provided by the system. The level of global anonymity increases by a more even distribution of attribution probabilities. Consequently, a high global anonymity does not guarantee a high anonymity of each individual subject. Ensuring individual anonymity would require enforcing a uniform behaviour within a set of subjects, which Pfitzmann and Hansen consider both very difficult to enforce and not desirable from a human rights perspective.

If a subject is known to be part of multiple anonymity sets, e.g. due to repeated interactions over time, the effective anonymity set is reduced to the intersection of all known anonymity sets. This is known as an intersection attack.

k-Anonymity Sweeney [START_REF] Sweeney | k-anonymity: A model for protecting privacy[END_REF] defines a set of data records as k-anonymous, if every record is indistinguishable from at least k -1 other records in terms of identifiability. Such an indistinguishable group of records is called a bucket and corresponds to Pfitzmann and Hansen's anonymity set. Sweeney uses the term quasi-identifiers to denote a subset of record fields or attributes that are sensitive to linking attacks, i.e. attributes that are likely to appear in other data collections. Therefore, such quasi-identifiers can be used to correlate data collections and possibly disclose the identity of subjects. Given a data collection, Sweeney assumes, it is possible to find attributes that qualify as quasi-identifiers. Based on that assumption a data holder wishing to release an anonymised version of its data collection could redact values of quasi-identifiers until an acceptably large bucket size is reached. Under Sweeney's quasi-identifier assumption, variations within non-quasi-identifier attributes of the same bucket are not considered problematic.

-Diversity Machanavajjhala et al. [START_REF] Machanavajjhala | l-diversity: Privacy beyond k-anonymity[END_REF] proposed -diversity as an enhancement of k-anonymity. Two attacks against k-anonymity are presented which utilise potentially low diversity within non-quasi-identifier attributes, denoted as sensitive attributes. The homogeneity attack shows that the size of a bucket is insignificant if a sufficiently high proportion of records within a bucket share the same sensitive attribute. A subject known to be part of that bucket can be assumed to share that value with reasonable likelihood, too. The background knowledge attack demonstrates that background knowledge about a subject can be used to single out the corresponding bucket and to reduce the anonymity set of that bucket by eliminating records that are incompatible with the attacker's background knowledge. As a countermeasure, -diversity requires different values for each sensitive attribute within a bucket. t-Closeness Li et al. [START_REF] Li | t-closeness: Privacy beyond k-anonymity and l-diversity[END_REF] demonstrate that the sensitive values within each bucket need to be distributed closely to the distribution of the overall data collection to avoid two kinds of attacks that are still possible with -diverse data. The skewness attack utilises a potential mismatch between the relative frequency of a stigmatising sensitive value within a bucket and that within the overall data collection. Based on that mismatch, an attacker can infer that subjects within that bucket are more likely to share the sensitive value than subjects in other buckets. The similarity attack shows that -diversity is not sufficient to protect against the homogeneity attack, if the diverse values are semantically similar and thus fall within the same category rendering the bucket homogeneous. Li et al. introduce the notion of t-closeness which requires the value distributions of each bucket to differ no more than t from the distribution of the overall data collection.

Differential Privacy Dwork [START_REF] Dwork | Differential privacy[END_REF] introduces Differential Privacy not as a metric for the degree of anonymity of sanitised data collections, but as a property of data processing algorithms. An algorithm is considered -differentially private if it processes similar inputs into outputs which are only distinguishable with a certainty that is bound by . As a consequence, the probability is limited that such an algorithm exposes discernible information after adding or removing a subject from the input data. Differential Privacy does neither limit the amount nor the sensitivity of the information which is exposed within the tolerance of the boundary.

Summary While the definition of Pfitzmann and Hansen mostly conforms with legal notions of anonymity and is less formal, the other metrics aim at providing a provable property that allows an unambiguous reasoning about the sensitivity of data or algorithms. Such formalisation necessarily comes at the cost of simplification. Focusing on quasi-identifiers neglected that practically any information can be used in background knowledge attack to single out individuals or at least reduce the anonymity set. Regardless of the size of an anonymity set, information about subjects is disclosed if the anonymity set as a whole is abnormal and this abnormality reflects on all subjects within this anonymity set. For an analysis of the aforementioned notions and anonymisation techniques in the context of EU regulations we refer to the opinion paper of the Article 29 data protection working party [9].

Profiling and Unlinkability

Striving for anonymity and for less disclosure of personal data is not sufficient to protect individuals against a non-transparent and potentially malicious data processing. Even if a subject might not be identifiable by a processor, his or her data can be linkable and thus aggregated over time to build a profile of that subject. Consider a system in which each user is only represented and reidentified by a unique token, that is only meaningful in the context of that processor and not linkable to any external data. Profiles of such users would be considered legally anonymous if the processor had no likely means to identify the natural person behind that profile. For example, consider a news aggregation service, which knows neither its users' names nor e-mail addresses, IP addresses, or any other identifying information that has any meaning to third parties. The service recognises its users by a randomly chosen unique token and records their news preferences to provide a targeted selection of news to each user. This service could plausibly argue, that due to the lack of identifiability, no personal data is handled and therefore data protection regulations do not apply. Consequently, this service could legally sell those profiles or use them for different purposes like targeted advertising without any restrictions.

Unlinkability is defined by Pfitzmann and Hansen [START_REF] Pfitzmann | A terminology for talking about privacy by data minimization: Anonymity, unlinkability, undetectability, unobservability[END_REF] as the property of two or more items to be indistinguishable regarding their relation from the perspective of an attacker. Applied to the example of a news aggregation service, unlinkability of users' news consumption demands that two news requests by the same user appear indistinguishable from requests which were made by two different users. With unlinkability, profiling of users is impossible, since each profile comprises one item only. Unlinkability is a strong privacy guarantee which in turn comes at the cost of losing the ability to personalise services.

Lightweight Anonymity and Privacy

In 2012, Hsiao et al. proposed a setting for anonymous communication networks (ACNs) called Lightweight Anonymity and Privacy [START_REF] Hsiao | LAP: Lightweight anonymity and privacy[END_REF]. In this setting, the attacker model is relaxed, packets travel near-optimal routes, and only an intermediate level of privacy can be achieved. Lightweight anonymisation techniques can achieve higher efficiency compared to other anonymisation techniques. They can be a tool for so-called zero-effort privacy [START_REF] Herrmann | Anonymity Online for Everyone: What is missing for zero-effort privacy on the Internet?[END_REF].

Anonymity on the Application Level

This section provides an introduction to tracking mechanisms on the application level and discusses their threat to anonymity online.

Tracking

Useful and legitimate applications for online tracking include the provision of personalised services, the distribution of personalised advertisements, and the measurement of website or application utilisation in order to derive patterns from the collected data. However, the collection and aggregation of this data can provide deep insights into online activities of a single user [START_REF] Bujlow | A Survey on Web Tracking: Mechanisms, Implications, and Defenses[END_REF].

Tracking does not only influence the privacy of a user, it additionally introduces a high risk of discrimination based on the collected data. Findings by Hannak et al. [START_REF] Hannak | Measuring price discrimination and steering on e-commerce web sites[END_REF] show that some e-commerce sites offer their products with different prices based on individual user profiles, which is also known as price discrimination.

In the following, we consider common tracking mechanisms (such as storagebased tracking and fingerprinting) which are capable to uniquely identify a computer system. Furthermore, we present behaviour-based tracking as an approach to identify a specific user instead of a computer system. Tracking mechanisms are a field of active research and development. Thus, countermeasures are required to continuously adapt protection mechanisms to the technical progress.

We consider the attacker to be a remote entity such as an online service with the privilege to store data or execute code on the victims machine. However, we 1994 HTTP Cookies [START_REF] Schwartz | Giving Web a Memory Cost Its Users Privacy[END_REF] 2011 TLS Session Resumption [START_REF] Perry | Disable TLS Session resumption and Session IDs[END_REF] 2005 Clock-skew Fingerprinting [START_REF] Kohno | Remote physical device fingerprinting[END_REF] 2012 IDFA [START_REF] Edwards | Apple Has Quietly Started Tracking iPhone Users Again, And It's Tricky To Opt Out[END_REF]; Canvas Fingerprinting [START_REF] Mowery | Pixel perfect: Fingerprinting canvas in HTML5[END_REF] 1999 Mouse Movements [START_REF] Goecks | Automatically labeling web pages based on normal user actions[END_REF] 1997 Keystroke Dynamics [START_REF] Monrose | Authentication via keystroke dynamics[END_REF] 2007 Browsing History [START_REF] Padmanabhan | Clickprints on the web: Are there signatures in web browsing data?[END_REF] Fig. 1: Timeline of tracking mechanisms based on their first documented occurance. assume the attacker is not capable of compromising the operating system of the host device. This assumption applies to website operators who run their code in the user's browser or to application providers, whose software is installed on the user's machine.

Storage-based Tracking

With storage-based tracking mechanisms, users can be uniquely identified through information stored on their device. Among others, these mechanisms include HTTP cookies [START_REF] Schwartz | Giving Web a Memory Cost Its Users Privacy[END_REF], session identifiers stored in hidden fields, Apple's advertising identifier (IDFA) [START_REF] Edwards | Apple Has Quietly Started Tracking iPhone Users Again, And It's Tricky To Opt Out[END_REF], and the TLS session resumption cache [START_REF] Perry | Disable TLS Session resumption and Session IDs[END_REF]. For trackers, it might be attractive to access stored information which can persist on the user's device and is available within different sessions or even different applications such as the IDFA. For this group of tracking mechanisms, the user has at least theoretically the option to delete the stored tracking information and thus, thwart the creation of profiles.

Fingerprinting Fingerprinting mechanisms collect information about the user's computer system with the aim to discriminate the system among an anonymity set. Preferably, the collected information is difficult or unlikely to be changed by the user, thus, fingerprints can be used to recognise the user's computer system for long periods of time. However, if the fingerprint can extensively aggregate information about the user's computer system, the user might be recognised even after a partial modification of his or her system. Examples for fingerprinting are clock-skew measurements [START_REF] Kohno | Remote physical device fingerprinting[END_REF] and canvas fingerprinting [START_REF] Mowery | Pixel perfect: Fingerprinting canvas in HTML5[END_REF].

Behaviour-based Tracking Behaviour-based tracking mechanisms aim to identify a user by characteristic traits such as mouse movements [START_REF] Goecks | Automatically labeling web pages based on normal user actions[END_REF], keystroke dynamics [START_REF] Monrose | Authentication via keystroke dynamics[END_REF], and browsing history [START_REF] Padmanabhan | Clickprints on the web: Are there signatures in web browsing data?[END_REF]. While storage-based tracking and fingerprinting is capable to identify a specific device or application, the mechanism of behaviour-based tracking targets to identify a specific user. Research indicates, that the precision of identification for behaviour-based tracking decreases for large sample sizes [START_REF] Banerjee | Biometric authentication and identification using keystroke dynamics: A survey[END_REF]. In comparison, storage-based tracking can be used to store unique information with a high entropy on a user's device and therefore has a negligible error rate when used for tracking purposes.

Current Challenges for Anonymity on the Application Level

Tracking does not only occur in the context of web browsing. Empirical studies show that many popular mobile applications connect to services on the Internet [START_REF] Sy | AppPETs: A Framework for Privacy-Preserving Apps[END_REF]. Half of the Top 100 mobile applications under iOS, Android and Windows Phone transfer personal information to online services [START_REF] Ren | Recon: Revealing and controlling pii leaks in mobile network traffic[END_REF]. Therefore, every application on a computer system needs to be regarded as a potential leak of personally identifying information. In the following we describe challenges and solutions for the prevention of tracking on the application level.

Access Control for Runtime Environments

To restrict applications in their ability to collect, process, store and transfer personally identifying information access controls for runtime environments can be used. Therefore, this approach can enforce restrictions onto applications that reduce their functionality and thus limit their capabilities to track the user. For example, an application without access to network interfaces has no means to directly transfer collected data to a tracking service.

Unification of System Configurations

The unification of system configurations aims to make tracking by fingerprinting more difficult by reducing the number of possible system configurations which are visible to an application, or by reducing the probability that a system deviates from the standard configuration. A naive utilisation of this approach is to reset the system configuration to the standard configuration after a period of time. However, a malicious application might still deduce differences in systems hardware or user specifics characteristics usage. It remains a challenge to address this issue and to reduce the capabilities of trackers.

Disguise User Behaviour This approach addresses the problem of behaviourbased tracking and aims to reduce the exposure of characteristic user behaviour towards malicious applications. For example, a system might use keystroke dynamics or mouse movements by randomly modifying the latency between two mouse or keyboard events. Characteristic user behaviour can also be deduced from contextual information such as browsing histories. Research by Herrmann et al. [START_REF] Herrmann | Behavior-based tracking: Exploiting characteristic patterns in DNS traffic[END_REF] indicates that datasets containing browsing sessions of 24 hours have 85.4% accuracy of finding and matching the sessions of the same users, while shorter sessions of 5 minutes yield only an accuracy of 30.4%. Besides such temporal schemes to disguise characteristic user behaviour, also contextual schemes could be applied, where sessions are separated when a new website is visited.

Current Solutions for Anonymity on the Application Level

The presented challenges point out, that current solutions, and popular operating systems in particular, do not sufficiently protect against tracking. We now investigate operating systems such as Tails [START_REF]The Tails Project: Tails[END_REF], Qubes OS [START_REF]The Qubes OS Project: Qubes OS[END_REF] with Whonix [START_REF]Whonix developers[END_REF], and Subgraph OS [START_REF] Ahmad | Subgraph OS[END_REF] in terms of their ability to prevent tracking. These operating systems assume a stronger attacker model compared to popular operating systems and aim to solve the current challenges for anonymity online. A brief comparison of these operating systems is given in Table 1 and afterwards.

Tails Tails is designed as a live operating system, which is directly bootable from an external medium and aims to leave no traces of its usage on the computer used. As a security feature, the Tails OS provides high barriers for subsequently installed potentially malicious applications to persist after a system reboot. The drawback of this design decision is the limited usability of the system for users who want to install additional applications or to personalise their operating systems with an individual configuration. Tails uses the AppArmor [START_REF]AppArmor developers: AppArmor[END_REF] Linux kernel security module for access control policies of installed applications.

Qubes OS Qubes OS with a Whonix virtual machine aims at providing anonymous Internet access as well as strict security by confinements. In this context confinements describe security mechanisms to separate running programs. All applications are installed in virtual machines, and the host utilises the Xen hypervisor [START_REF]Xen developers: Xen Project[END_REF]. This architecture reduces the trusted code base of the host in comparison to typical monolithic operating systems such as the Linux kernel and therefore the attack surface is diminished. Single virtual machines can be configured with specific security configurations such as a restricted filesystem access, usage of ACNs, or firewall rules. However, the design of Qubes OS comes along with high hardware requirements for the execution of multiple parallel virtual machines.

Subgraph OS Subgraph OS provides anonymous Internet access and a design for strict application confinement. This operating system includes a hardened Linux kernel which is patched with Grsecurity/Pax [32] to provide additional memory protection and enhanced local access control. Subgraph allows a finegrained confinement of individual applications by, among other mechanisms, application specific firewall rules, control of filesystem access, seccomp filter to restrict permitted system calls, isolation of some drivers such as audio, control of desktop access or process visibility.

Usability and Security

We tested whether a Tor Browser is able to collect hardware information on Qubes OS with Whonix, Subgraph OS, or Tails. We noticed that the default privileges of the Tor Browser on these systems are sufficient to collect detailed hardware information. These information could be used for device fingerprinting. Tails, Subgraph OS, and Qubes OS with Whonix provide the Tor Browser within their standard configuration, which provides the features of tab isolation and stream isolation [START_REF] Appelbaum | Description of Tor Stream Isolation[END_REF] in order to protect against identification based on contextual information such as browsing habits of a user.

Furthermore, Qubes OS and Subgraph OS implement GUI isolation towards the restriction of applications in observing user behaviour in the context of other applications. However, the investigated operating systems do not provide functionalities to protect user tracking based on mouse movements or keystroke dynamics.

As a live operating system, Tails implements the unification of system configurations by resetting the system after each reboot. Consequently, storage-based tracking methods do not persist a restart of the system. Hence, this approach makes it more difficult for fingerprinting mechanisms to collect personally identifying information. However, malicious applications can still retrieve information about the hardware of the system.

Within Qubes OS the feature of disposable VMs supports the approach of a unification of system configurations. In this way, the user installs a potentially malicious application in a separate VM, which can be disposed after its usage. Thus, storage-based tracking over multiple sessions of application usage becomes more difficult, since the VM can be easily disposed in the meantime. Disposable VMs also improve the defence against fingerprinting mechanisms, since modifications of the VM by the user are removed with every disposal of the VM.

Tails and Subgraph OS are based on the Debian Linux distribution and provide a similar usability as Debian. However, as a live operating system, which returns to its initial state after each reboot, Tails has limited use cases. In Qubes OS, it is the responsibility of the user to isolate applications from each other by installing them in different virtual machines. Thus, Qubes OS requires a higher security awareness of the user which limits its usability. As a negative side effect on usability, Qubes OS has higher hardware requirements in comparison to the other operating systems.

Anonymity on the Network Level

This section provides a chronology of techniques which have been used to prevent tracking on the network level. Furthermore, we will present selected applications that are in use today. Finally, we discuss current challenges.

Chronology

In 1978, Rivest, Shamir and Adleman presented a method for obtaining digital signatures and public key cryptosystems that became known as RSA cryptosystem [START_REF] Rivest | A method for obtaining digital signatures and public-key cryptosystems[END_REF]. The cryptosystem had the novel property that a public encryption key does not reveal the corresponding decryption key. A sender encrypts the message to be sent with the receiver's public key and transmits it via a potentially insecure channel. Only the receiver could decrypt the message with his or her secret private key. Similarly, messages can be signed [START_REF] Rivest | A method for obtaining digital signatures and public-key cryptosystems[END_REF]. The RSA cryptosystem serves as building block for various privacy enhancing techniques until today.

A technique based on public key cryptography that allows unlinkability of sender and recipient was presented by Chaum in 1981. The basic idea makes use of so-called mixes, which sample messages of same length in a batch, change their appearance and forward all of them at the same point of time but in a different order [START_REF] Chaum | Untraceable electronic mail, return addresses, and digital pseudonyms[END_REF]. Unlinkability can be achieved if more than one mix is used, if the mixes are operated by different operators and if at least one mix operates trustworthy and honestly. In the same paper, Chaum introduced digital pseudonyms. A digital pseudonym is a public key with which digital signatures of an anonymous holder of the corresponding private key can be verified. A combination of mixes and digital pseudonyms enables electronic elections in which any party can verify that the votes have been properly counted [START_REF] Chaum | Untraceable electronic mail, return addresses, and digital pseudonyms[END_REF]. In 1991, Pfitzmann et al. presented ISDN-MIXes, a combination of Chaum's mixes, dummy traffic and broadcasts. ISDN-MIXes allow untraceable communication with low communication overhead [START_REF] Pfitzmann | ISDN-mixes: Untraceable communication with very small bandwidth overhead[END_REF]. Federrath et al. presented mixes in mobile communication systems in 1991. Their mix-based system utilises untraceable return-addresses to hide routing information and achieves location privacy [START_REF] Federrath | MIXes in mobile communication systems: Location management with privacy[END_REF]. In 1998, Kesdogan et al. introduced Stop-and-Go-MIXes that provide probabilistic anonymity. Unlike other mixes, Stop-and-Go-MIXes do not collect a fixed number of messages in a batch [START_REF] Kesdogan | Stop-and-Go-MIXes Providing Probabilistic Anonymity in an Open System[END_REF].

Blind signatures were introduced by Chaum in 1983. As traditional digital signatures, blind signatures can guarantee authenticity of a message. However, blind signatures allow signing of a message without revealing the message itself to the signer [START_REF] Chaum | Blind signatures for untraceable payments[END_REF]. Blind signature schemes are utilised in electronic cash and electronic voting systems.

Tracking organisations use personally identifiable information, such as name, date and place of birth, or address, to match or link records with those provided by other organisations. With Chaum's credential system, presented in 1985, an individual could use unlinkable pseudonyms to interact with different organisations. For instance, a one-time-use pseudonym may be used to purchase goods from a shop and a persistent pseudonym may be used to open an account with a bank. The credential system ensures that individuals are held accountable for abuses created under their pseudonyms. Also, organisations could limit the number of pseudonyms per individual and individuals are able to authenticate ownership of their pseudonyms [START_REF] Chaum | Security without identification: Transaction systems to make big brother obsolete[END_REF].

A communication protocol that achieves unconditional sender and recipient untraceability was published by Chaum in 1988. In contrast to mix networks, the Dining Cryptographers Network (DC-Net) relies on secure multi-party computation [START_REF] Chaum | The dining cryptographers problem: Unconditional sender and recipient untraceability[END_REF].

In a privacy preserving value exchange (e.g. unobservable and anonymous exchange of digital money) over a network, the main problem is the lack of simultaneity. It gives a temporary advantage to one party who can stop the communication midway through the value exchange process. In 1990, Bürk and Pfitzmann compared two approaches that utilise third parties to overcome this problem [START_REF] Bürk | Value exchange systems enabling security and unobservability[END_REF].

In 1995, Cooper and Birman introduced a service that allows reading from a shared memory without revealing which piece of information is being read. Those so-called blind message services can be used as an alternative to mixes to build a message service that achieves location privacy [START_REF] Cooper | Preserving privacy in a network of mobile computers[END_REF].

Goldschlag, Reed and Syversen introduced onion routing in 1996 [START_REF] Goldschlag | Hiding routing information[END_REF], a lightweight approach to the dissemination of mixes.

Fig. 2 shows a timeline of development of the aforementioned privacy enhancing technologies.

1978 Public Key Cryptography [START_REF] Rivest | A method for obtaining digital signatures and public-key cryptosystems[END_REF] 1981 Mix [START_REF] Chaum | Untraceable electronic mail, return addresses, and digital pseudonyms[END_REF]; Pseudonyms [START_REF] Chaum | Untraceable electronic mail, return addresses, and digital pseudonyms[END_REF] 1983 Blind Signatures [START_REF] Chaum | Blind signatures for untraceable payments[END_REF] 1985 Credentials [START_REF] Chaum | Security without identification: Transaction systems to make big brother obsolete[END_REF] 1988 DC-Net [START_REF] Chaum | The dining cryptographers problem: Unconditional sender and recipient untraceability[END_REF] 1990 Privacy Preserving Value Exchange [START_REF] Bürk | Value exchange systems enabling security and unobservability[END_REF] 1991 ISDN-MIXes [START_REF] Pfitzmann | ISDN-mixes: Untraceable communication with very small bandwidth overhead[END_REF] 1995 Blind Message Service [START_REF] Cooper | Preserving privacy in a network of mobile computers[END_REF] 1996 MIXes in Mobile Communications [START_REF] Federrath | MIXes in mobile communication systems: Location management with privacy[END_REF];

Onion Routing [START_REF] Goldschlag | Hiding routing information[END_REF] 1998 Stop-and-Go MIXes [START_REF] Kesdogan | Stop-and-Go-MIXes Providing Probabilistic Anonymity in an Open System[END_REF] Fig. 2: Timeline of development of privacy enhancing technologies.

Current Solutions

JAP, formerly known as Java Anon Proxy, is a mix-based solution for anonymous Internet access. A first test version was launched in October 2000 and was developed by the research project AN.ON -Anonymity.Online [START_REF] On | Anonymität[END_REF]. The full service is running since February 2001 and has been outsourced to Jon-Dos GmbH [START_REF] Gmbh | JonDonym[END_REF]. JAP is currently used by 5000 paying and several thousand non-paying users [START_REF]Status der Mixkaskaden[END_REF]. The research project named AN.ON-Next -Anonymity Online for the next Generation [START_REF] On | Next: Anonymität Online der nächsten Generation[END_REF] aims to further develop JAP's mix-based anonymisation techniques.

Tor is the most widely used anonymisation network. It was presented by Dingledine et al. in 2004 and is based on onion routing [START_REF] Dingledine | Tor: The second-generation onion router[END_REF]. Today, Tor has millions of users [START_REF]Tor Metrics[END_REF]. The servers that are used to relay traffic are mainly run by volunteers. One noteworthy product of the Tor project is the Tor Browser. It combines Tor's protection on the network level with a modified Mozilla Firefox Browser in order to thwart tracking based on fingerprinting on the application level [START_REF] Perry | The Design and Implementation of the Tor Browser[END_REF]. JAP and Tor protect against relatively strong adversaries. Other solutions focus on weaker adversaries.

IPv6 Pseudonymisation Long-lived IP addresses are one of the easiest ways for tracking. Daily changing IP addresses (on reconnect or through IPv6 Privacy Extensions) are not sufficient [START_REF] Herrmann | Behavior-based tracking: Exploiting characteristic patterns in DNS traffic[END_REF]. As long as unlinkability of actions against ad networks and websites based on an IP address is intended and sufficient, ISPs can offer anonymity with a new approach to IP address assignment. Multiple users could share the same IP address (Address Sharing) or a single user could frequently change his or her IP address (Address Hopping). The anonymisation functionality can be implemented on the router or in the datacenter of the ISP. An advantage of this solution is that users are not required to make any changes to their operating system or client software and hardware. As a positive side effect, also devices, whose network configuration can not be changed, can be protected.

In our attacker model, we assume the user's ISP to be trustworthy. The attacker can be a web service operator who controls one or multiple web services, or a third party tracker that links activities across multiple web services. Furthermore, the attacker could be a man-in-the-middle between the ISP's border routers and the connection's endpoint.

Fig. 3a shows two users with different IP addresses a and b who communicate over the Internet with three different services 1, 2 and 3. Without IPv6 pseudonymisation, a third party tracker that is embedded in all three services can conclude that the user with IP address a is using the services 1 and 3 and that the user with IP address b is using the service 2.

With Address Sharing, one IP address is shared among multiple users at a given point in time. All users are using the same IP address and thus form an anonymity group, implying that trackers cannot distinguish users based on their IP address anymore. Address Sharing can be implemented with state-of-the-art techniques such as Network Address Translation (NAT) has been deployed on (a) Without IPv6 pseudonymisation, a third party tracker can conclude that the user with IP address a is using the services 1 and 3 and that the user with IP address b is using the service 2. Internet gateways for decades. Fig. 3b shows three users who communicate via a router through the Internet with different services. Each user has his or her own IP address a, b or c. The router replaces the IP addresses on its public network interface, with the result that all users share the same public IP address d. The third party tracker cannot discriminate between the users based on their IP addresses. From the viewpoint of the tracker, it appears to be a single user with IP address d who is using services 1, 2 and 3.

Address Hopping means that each user distributes his or her traffic over multiple IP addresses within a short period of time. Web services or third party trackers can link activities for which the same IP address is used. However trackers cannot link activities based on IP addresses, when the IP address is changed frequently. Fig. 3c shows a single user with IP address a who communicates via a router through the Internet with different services. The router replaces the IP address, with the result that the different services 1, 2 and 3 see different IP addresses b, c and d. A third party tracker cannot link the user's IP addresses easily. It seems (for the attacker) that the different services are being used by different users. Given the large number of IP addresses that are assigned to each user with IPv6 [START_REF] Narten | IPv6 Address Assignment to End Sites[END_REF], Address Hopping becomes possible.

Conclusion

We introduced multiple perspectives on online anonymity, including recent developments in EU data protection regulations regarding anonymity. We argued that certain forms of aggregated user profiles might be considered legally anonymous. To extend the scope of data protection regulations to such profiling, the concept of linkability needs to be incorporated into legal interpretations of identifiability and personal data.

We discussed fingerprinting as well as storage-based and behaviour-based tracking and the challenges to achieve online anonymity on the application level. Security hardened operating systems Tails, Subgraph OS, and Qubes OS with Whonix may support the users efforts to online anonymity. However, they do not completely protect against tracking by installed applications.

Building anonymous communication networks on the network level is a challenging effort. As examples, we introduced Tor and JAP, both of which protect against strong adversaries. Lightweight solutions such as IPv6 pseudonymisation which protects against a weaker adversary but aims for a broader public have the potential to gain broad acceptance as the protection comes usually without any significant performance limitations.

Acknowledgement

Part of this work has been developed in the projects AN.ON-next (reference number: 16KIS0368) and AppPETs (reference number: 16KIS0381K). Both projects are partly funded by the German ministry of education and research (BMBF).

We thank Ephraim Zimmer and Tobias Mueller for their helpful comments and discussion.

  (b) With Address Sharing, one IP address is shared among multiple users. The third party tracker cannot discriminate between the users based on their IP addresses. (c) With Address Hopping, each user distributes his or her traffic over multiple IP addresses. It seems that the different services are being used by different users.

Fig. 3 :

 3 Fig. 3: IPv6 pseudonymisation.

Table 1 :

 1 Comparison of operating systems with support for online anonymity.

	System Properties	Tails	Qubes OS + Whonix Subgraph OS
	Live USB OS	Yes	No	No
	Firewall Rules	Entire OS	Per VM	Per App
	Filesystem Isolation	Per App	Per VM	Per App
	Hardware Compatibility	High	Limited	High
	ACN Usage	Default	Per VM	Per App
	Host Architecture	Debian-based Xen Hypervisor	Debian-based
	Seccomp Filter	No	No	Yes, per App
	Isolate Devices and Drivers No	Yes	Yes
	GUI Isolation	Yes	Limited, per VM	Limited, per App
	Process Visibility	Yes	Limited, per VM	Limited, per App


Kjetil Rommetveit
email: kjetil.rommetveit@uib.no

Alessia Tanas
email: alessia.tanas@vub.ac.be

Niels Van Dijk
email: niels.van.dijk@vub.be

Data Protection by Design: Promises and Perils in Crossing the Rubicon Between Law and Engineering

Keywords: privacy and data protection by design; privacy engineering, techno-epistemic network, organisations, law, engineering, socio-technical infrastructures

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The explosion of digital developments such as the internet of things, big data, and radically enhanced interconnectedness and sensoring capacities, have placed privacy and other fundamental rights and freedoms under strong pressure. The recently adopted EU General Data Protection Regulation (GDPR, that will take effect in May 2018) recognises these developments, and introduces a number of new tools for protecting and upholding fundamental rights and freedoms, such as data protection by design and by default (DPbD). DPbD consists in designing and building privacy and data protection into the emerging systems, technologies and infrastructures themselves, a move that is seen as necessary in order to handle the ubiquity, complexity and general unpredictability of digital innovations and technologies. This design-based approach is not new. It has developed over time, (ie. since the mid 1990s), in various sites and by various actors, initially focusing on Privacy Enhancing Technologies (PETs) and Privacy by Design. It has now become mandatory in EU legislation under the DPbD designation, for entities controlling and processing personal data ("data controllers"). This development has catalysed the evolution of the technological field of practice devoted to its realisation, frequently referred to as 'privacy engineering' [START_REF] Cavoukian | Privacy by design: The 7 foundational principles[END_REF][START_REF] Danezis | Privacy and Data Protection by Design -From Policy to Engineering[END_REF][START_REF] Gürses | Privacy Engineering: Shaping an Emerging Field of Research and Practice[END_REF][START_REF] Notario | PRIPARE: Integrating Privacy Best Practices into a Privacy Engineering Methodology IEEE CS Security and Privacy Workshops[END_REF].

The introduction of DPbD does not come in isolation. It comes along with other related developments, such as a risk-based approach towards fundamental rights and freedoms, where significant future risks to the rights of individuals, ("data subjects"), are to be mapped, and turned into organisational measures as well as technological and engineering ones (Art. 35 GDPR). Another related trend is a gradual reinforcement of self-regulation, or (in Europe) co-regulation, where greater responsibility for the safeguarding of privacy and data protection is placed on the data controllers (Art. [START_REF] Star | Steps Toward an Ecology of Infrastructure: Design and Access for Large Information Spaces[END_REF]). These developments must therefore be seen as parts of a concerted package [START_REF] De Hert | The new General Data Protection Regulation: Still a sound system for the protection of individuals? Computer[END_REF] encapsulated by the GDPR, and representative of broad developments at the intersections of technology, markets and society.

That such transformations take place should come as no surprise to observers of the fields of privacy and personal data protection. Some 20 years ago, the privacy activist Simon Davis noted how privacy had metamorphosed from an issue of societal power relationships to one of strictly defined legal rights [7, p. 143]. Implied in Davis diagnosis was the claim that the notion of privacy was changing: an issue that had started as a social and political project, driven and shaped by civic activism [see 1] was gradually transformed into a consumer and rights issue subject to regulatory and bureaucratic requirements and means, and moving closer to the (German) notion of data protection (Datenschutz). Now, with the turn towards a design-based approach to privacy and personal data we observe a next stage in the evolution of these concepts, one strictly dependent on engineering, and with outcomes still uncertain. On the one hand, digital technologies have strong impacts on fundamental rights and freedoms such as privacy and data protection, and regulation is extending into these areas in order for technology not to become too invasive. On the other hand, in order to deal with these issues, regulation is increasingly relying upon the contributions of engineers, technologists and other practitioners. These new entanglements raise serious questions about the ways in which rights' protections become conceptualised and implemented: the case is not simply one of law formulating the principles, and engineers adapting them to new practices; as we describe, quite fundamental changes to rights and principles take place through these new exchanges and collaborations. The notion of the Rubicon in the title refers to an existing divide between legal and engineering methods and to questions on whether, when and how such divides should be crossed.

In this brief article, we recount some results from recent empirical investigations into the turn towards privacy-by-design, including the introduction of risk-based approaches [START_REF] Van Dijk | A Risk to a Right? Beyond Data Protection Impact Assessments[END_REF]. Using social science methods such as written consultations, interviews and focus groups, we have conducted an 'extended peer consultation' and mapped out different existing logics, but also perceptions, imaginations and expectations related to privacy engineering. We extended invitations to representatives from DPAs, universities, the standardisation field and the business sector, but also technology developers and software engineers. We also included other peers with experience in articulating privacy like legal practitioners and judges in European high courts, civil rights organisations, technology prosumers, ethical hackers, social science and humanities scholars, and practitioners of value sensitive design. Peers were presented with issues concerning data protection by design and by default and data protection impact assessment, to which they provided written responses. In a next round, we also carried out face-to-face interviews, focusing on more in-depth issues discovered during the first stage of consultation. The findings from the consultations were validated through a workshop that included some representatives from the prior consultation. Throughout this process, we wanted to understand what constitutes privacy and personal data protection rights in design, and how design-based techniques relate to notions such as fundamental rights, freedoms and legal protections.

A main outcome from our investigation is that, within the overall network, different ways of imagining, understanding and articulating these rights occur. We mapped various modes of articulation invoked by the different peers, and related these to their occupational, organisational or civic backgrounds. Important here is how, what we term a 'techno-epistemic network' of professionals dedicated to the engineering of rights and for this purpose work to exchange knowledge and create collaborations across boundaries that were previously kept largely separate. Involved in this work are practitioners from engineering, regulation and managerial practices and more, favoring the emergence of the technical and regulatory field of privacy engineering for (D)PbD . Based on this, we have elaborated upon different networked modes [START_REF] Van Dijk | forthcoming: Right engineering? The redesign of privacy and personal data protection[END_REF] in which rights become (re-)articulated and implemented by actors situated within the techno-epistemic network, or claimed by actors situated outside the network, like legal practitioners, civil rights organizations, ICT prosumers, ethical hackers, etc. We observe and describe how these different modes converge or not, when seen in relation to the shared objective of designing rights and legal principles in technological infrastructures and artefacts. It is here, in the comparison between the various approaches and positions taken, that we point to tensions or contradictions. As argued elsewhere [START_REF] Rommetveit | Working responsibly across boundaries? Some practical and theoretical lessons[END_REF], we think that such tensions are not to be overlooked, but clarified in their practical and theoretical implications. The argument is a shortened version that complements a larger paper, which traces the formation of the techno-epistemic network and the way it has so far aligned and unaligned different articulations of privacy [START_REF] Van Dijk | A Risk to a Right? Beyond Data Protection Impact Assessments[END_REF].

Requirements and expectations on data protection and privacy Legal

Privacy and data protection change due to complex reasons, simultaneously technological, legal, political and cultural. Within the recent paradigmatic shift [cf. 2] towards privacy engineering, the major driving forces may be seen to be technological, or 'data-driven' [START_REF] Hildebrandt | Smart Technologies and the End(s) of Law: Novel Entanglements of Law and Technology[END_REF]. Yet, data protection and privacy are not shaped by technology only, and belong within dynamically evolving clusters of principles, practices, institutions, means and technologies. As to the EU regulatory framework, this can be clearly seen in preparatory documents for the GDPR, where DPbD was described as beneficial for a variety of reasons: it enhances the protection of individual rights and the efficiency and security of processing; it was also argued to increase oversight and accountability, significantly through its firm focus on data processors and (large) organisations. And, DPbD was invoked as a fundamental tool in the building of the European digital market, since European industries could become world leaders in privacy enhancing technology or privacy by design solutions [23]. Hence, the drivers and rationales that enter into data protection by design are composite and incorporate differing policy and digital market goals, interests and logics.

In our consultations with actors involved (in different ways) with the making of data protection by design, we observe how differing logics [cf., 3] are at work in the project to implement data protection (and privacy) by design. There is dedication within the GDPR to the legal logic of fundamental rights and freedoms of individuals, but this logic cannot be fully detached from the economic goals of creating and enhancing the internal digital market. In the claims for enhanced accountability and efficiency, we detect a bureaucratic or regulatory logic at work. All of these approaches are now to be integrated with engineering ways of doing things, following engineering logics. Finally, hovering above (or underneath) all of this, there is the original civic goal of protecting public values of autonomy, dignity, freedom of thought and expression. We can observe the continued reality of this civic approach in public actions against privacy-invasive projects and technologies in Europe, such as the privacy class-action against Facebook by Europe vs. Facebook, protests in the Netherlands against mandatory introduction of smart metering devices, or initiatives in the UK to take the Government Communications Headquarter's (GCHQ) surveillance initiatives to court.

Designing data protection: articulations and frictions

During our research, we observed that these logics re-occur in new modes within networked practices dedicated to operationalise DPbD, although we point to different visions and practices still in the making. As part of this we indicate tensions, gaps, limits and perhaps even contradictions at work. In the next section, we point to three overarching points of tension, where different modes and visions of rights and engineering are at work: 1) individual versus organisational autonomy; 2) law versus engineering, and 3) global versus local in the making of infrastructures. Articulating and describing some of these require contributions from social scientists, social actors, philosophers and legal scholars, since the challenges involved with DPbD are not merely technical, but importantly also social, practice-based and disciplinary. They crucially depend on the possibilities of establishing cross-institutional, disciplinary and experiential collaborations between the various actors.

To reiterate, insofar as real tensions or contradictions exist among logics or modes of articulating rights to privacy, we believe that constructive approaches can only come from a proper formulation of these tensions, since we agree with the constructive proposition (from philosophical pragmatism) that "a problem well put is half-solved" [START_REF] Dewey | Logic: The Theory of Inquiry[END_REF]. DPbD requires careful consideration of limits (technical, legal, civic, etc.), and due appreciation of the various values, interests, regimes and logics at work. Spelling out some of these can help improve actors' mutual understandings, and possibly also overcome some misunderstandings. As for the limitations involved, becoming clear about what can and what cannot be done, can direct practitioners towards searching for other solutions where necessary. In what follows, we briefly introduce some tensions in practice, as discovered in our consultations.

Organisations versus autonomous data subjects?

Data protection law relies decisively upon large organisations for the attainment of its goals. As such the rights and principles themselves take on characteristics and logics typical of work inside organisations. Here, we have learnt about several challenges.

First, managerial and cultural issues pertain to the accustomed workings of the organisations that implement DPbD: today's large corporations, public institutions, or small and medium enterprises are not really trained or geared towards considering people's privacy concerns, or towards thoroughly understanding their own data flows in terms of the threats they could pose to the rights of natural persons. One problem here has to do with the very nature of the alleged contemporary 'information economy' or 'surveillance economy'. There is a proliferation and over-production of ('big') data, and many actors are getting involved in the hope of extracting value from the data. Yet, there are still great uncertainties about how to do this [START_REF] Kitchin | The Data Revolution[END_REF], or whether indeed data turns out to be the 'new oil'. Therefore, the chosen strategy is often to generate as much data as possible, then work out the necessary business strategies afterwards. From the point of view of data protection, however, this place the activities of the organisations in a difficult position with regard to data protection principles, such as data minimisation, purpose limitation and specification. As expressed by one of our informants, a data protection consultant to the private sector, businesses and corporations are not promoting privacy by design, because data are of high value and if you apply privacy by design techniques the amount of data you would collect would diminish and therefore you have impact on your business model (data protection consultant). A second, and related, problem, has to do with a lack of understanding of the data processing operations taking place within the organisations, since some of these may indeed have become 'too big to understand': My experience is that in order to understand this, organisations have to analyse in depth their data flows and most organisations haven't done that. Most of them actually do not know what kind of processing is taking place in their organisations (ibid.). This is problematic, since major presuppositions of data protection and privacy by design rest upon the assumption that data flows are properly mapped and understood in the first place [cf. [START_REF] Cavoukian | Privacy by design: The 7 foundational principles[END_REF][START_REF] Wright | Privacy Impact Assessment, Media[END_REF].

Secondly, even if these challenges would be tackled, new challenges arise, since the implementation of personal data protection becomes dependent on the operational logics of organisations. As stated, organisations work according to their own goals, means, and strategies when also having to take into account the needs and concerns of single individuals, users, and data subjects. Our informant systematically refers to privacy breaches as possible risks to 'an asset'. But an 'asset' is something typically belonging to the world of business as a resource that can be owned or controlled by a company to produce economic value. This is at odds with the spaces and processes in which notions of privacy normally arise (the home, family life, correspondence, browsing habits, etc.) and that have become legally acknowledged. To an actual person concerned with privacy, such spaces and processes are not economic 'assets' but often pose definite limits that cannot be so easily traded away. Yet this talk about privacy rights as assets is no mere slip of the tongue, but rather representative of a steady development in which privacy is increasingly being conceived as a risk to the reputation of organisations [START_REF] Van Dijk | A Risk to a Right? Beyond Data Protection Impact Assessments[END_REF].

Similarly, a person working as a DPA described how IT people are good at thinking about risks, but it is usually the risks to the organisation. Whereas some have argued that this makes for a win-win situation [START_REF] Cavoukian | Privacy by design: The 7 foundational principles[END_REF] for both organisations and data subjects, the interview points to several possible conflicting interests: organisations may want to produce and retain maximum amounts of information on individuals, often without their knowledge; they may combine data in new ways, thus producing sensitive data from nonsensitive sources; organisations may create representations of data subjects that do not correspond with the self-image of the subject, they may hold data secret and without the knowledge of the subject, and so on.

Therefore, whereas the transformation of data protection into 'technical and organisational measures' (GDPR, Art. 24, 1) seems like a necessary step for effectively protecting privacy (as it may contribute to bring organisations on board), this mode of operationalisation may come at the expense of certain trade-offs with the autonomy of individuals (natural, data subjects, etc.). Furthermore, this happens in a situation where the entitlements through which data subjects could oppose such developments and influence decisions (ex ante, prior to processing) are limited. Article 35 of the GDPR on data protection impact assessments, provides in its point 9 that the controller 'shall seek the views of data subjects or their representatives on the intended processing'. However, this should happen only where 'appropriate' and 'without prejudice to the protection of commercial or public interests or the security of processing operations', all of which are aspects that the controller is given the full mandate to decide upon. The entitlement of data subjects to influence decisions over protection of their rights during assessment procedures is thus limited and it does not correspond to a duty on the side of the controller to take these views on board. One of our informants, a member of a prominent civil rights privacy organisation, expressed dissatisfaction with this general state of affairs: Privacy by Design and Privacy Impact Assessments are used as an excuse for innovation. Once it is written they have been done, no one opposes (...) them and no one checks the quality of the process. Politicians have no notice of the contents (civil rights activist). When introduced together with other measures of the GDPR, such as privacy seals and data protection certification, impact assessments and data protection by design could be used to deflect the expectations of individual right-holders, activists and publics, de facto excluding them. Expectations are that such early interventions will enable controllers, data subjects and society at large, to avoid right infringements before they materialise [cf. 26]. However, the informant from the privacy NGO argued that these practices can also be used pre-emptively in order to avoid public opposition to privacy-infringing projects and technologies.

Law versus engineering

At the heart of design-based approaches to personal data protection, privacy and other fundamental rights and freedoms, we find expectations about new and innovative interactions between the practices of law and engineering. This could be described as the main instantiation of the imperative to cross the 'Rubicon' of data protection by design, since legal principles related to personal data protection and other fundamental rights and freedoms as spelled out by lawyers and judges, should be implemented by engineers and designers. Yet, there are huge differences between lawyers and engineers: in terms of their basic assumptions and methods; in terms of the medium through which they work; in terms of the procedural checks and balancing exercises they are subject to, and in terms of the scope of their interventions. Again, we single out a few major issues as encountered in our empirical data.

Firstly, we find decisive differences and limitations in terms of the practices of law and engineering, where legal principles cannot so easily be translated into something that can be rendered operable by engineers. Whereas a paradigmatic statement holds that 'law is code' [START_REF] Lessig | Code: Version 2.0[END_REF], people trying to turn this into practice easily end up perplexed. We already know this problem from the privacy design literature, where bridging efforts have been made through articulations of 'privacy goals' and 'design strategies' [START_REF] Hoepman | Privacy design strategies[END_REF][START_REF] Notario | PRIPARE: Integrating Privacy Best Practices into a Privacy Engineering Methodology IEEE CS Security and Privacy Workshops[END_REF]. A fundamental problem here is that legal principles and texts are by definition and nature polysemic, ie. they have multiple possible meanings and interpretations, without which they lose much of their meaning and function as legal principles [START_REF] Dworkin | Taking Rights Seriously[END_REF]. The GDPR only provides few general instructions, and these are not sufficient to perform the necessary translations. One of our informants, working in interaction design, expressed this as follows: There is a difference between the moral reasoning linked to human rights and the attempt of solving an engineering problem, which is technically and mathematically specified (human-computer interaction practitioner). In contradistinction to law, engineering goals and means are usually dependent on unilateral, non-ambiguous meanings, on reducing the design space, in order for coding operations to be able to proceed. Even small changes to the original parameters may be highly demanding, in terms of work and resources: Data Protection by Design and by Default can be costly in terms of computations, speed, and accuracy of models. In many cases this can be alleviated, but it usually requires very substantial research and work to achieve a good outcome. It can also be less flexible since the approach is often tailored to a particular goal and algorithm, and a small change can require a lot of work (privacy engineer).

Secondly, this difference of law and engineering may put a spanner in the wheels of technically oriented efforts towards 'prospective adjudication', whereby legal principles are invoked by designers and risk assessors before the fact of the infringement of a right. As explained to us by a judge, engineers do not think about human rights when they work. This is why law must play a role which is of course posterior to that of technical design. It should not be the role of legislation to foresee all possible breaches of rights: situations are so different (…) even if you provide for detailed rules in law, in certain cases they will not be applicable or their application would create a bad result (…) this is the task of law, of doctrine, of case law to find in a concrete case a justified solution" (judge, European Court of Justice). The Rubicon of law and engineering is, for such reasons, not to be crossed, according to the judge. The judge's statement is about the proper role and domain of law. Nevertheless, in a situation of broad discretion currently afforded by organisations as to self -restriction on how to ensure rights' protections, some bridging towards engineering and technology is needed. This applies especially when data protection by design becomes itself a legal obligation.

First, this might require attuning design processes where engineers have to come up with specific privacy solutions by applying generic legal principles to concrete technological contexts, to the ways in which law practitioners apply such principles within specific legal cases. Second, procedural checks could be introduced, to enhance oversight of decisions to be made within design processes and inspired by the long-standing procedural guarantees enjoyed by fundamental rights within institutional settings and courts.

Finally, even if such aspects (hermeneutics, procedural) could be worked out, we encounter differing ideas about what design is and what specific role it could play in the process of translation. According to a classical image, design is a uni-directional process in which the designer oversees and integrates an impressive amount of knowledge, building it into the material artefact. This image conveys the process as linear, and based upon neat separations between designers, producers and consumers, whereas in actual software development, these roles are much more blurred [START_REF] Stewart | The Wrong Trousers? Beyond the Design Fallacy: Social Learning and the User[END_REF]. Within the incipient field of privacy engineering, we hear talk about an 'agile turn' [START_REF] Gürses | Privacy After the Agile Turn[END_REF] as replacing previous modes ('waterfall'), in which the main emphasis is placed upon shorter development cycles, user centricity, constant updates and developments. Indeed, such changes in design processes seem necessary, since they follow and replicate what is going on anyway in software developments more generally [START_REF] Gürses | Privacy After the Agile Turn[END_REF].

We think these novel approaches, along with other related developments such as 'values in design' are highly interesting and relevant to the challenges at hand. Here, privacy engineering ceases to be a 'science', and turns towards 'artfulness' and creativity in the process possibly becoming more of a craft [START_REF] Gürses | Privacy Engineering: Shaping an Emerging Field of Research and Practice[END_REF]. In so doing they possibly open up towards the broader meanings, interpretations and methods required by law (as just described), and concerned social actors, since the process goes beyond classical applications of scientific or engineering principles deemed as objective and beyond discussion. Yet, we can also see how this may run up against other main principles of data protection, such as the basic requirement that data subjects give their informed consent to a processing operation. This becomes difficult in processes of agile design, where a software product may be seen as in a permanent state of flux: 'permanently Beta', under constant development. The old linear modes of design would have offered some assurance here, since there would be a decisive body of stabilised knowledge on the basis of which data subjects could make up their minds and provide their consents (or not). Yet, in the current design modes, other options must be sought out, since there is little consensus or technical guidance as to how this could happen. Therefore, we maintain that 'privacy engineering' remains an interesting field where valuable experimental efforts have already taken place, but this field should not be institutionally and politically overcharged.

Global standards versus local requirements

As implied in the above section, solutions for how to carry out DPbD are simultaneously being sought on various levels, from single technologies (PETs) to practices (law, engineering, design), at single organisational level and beyond. Yet, the overarching reference is at the level of standards and infrastructures, since this is where interventions must be made in order to render the internal digital market a reality, and to technically connect the various systems involved, for instance for the making of the Internet of Things. Here, we encounter another mode, which we have termed 'privacy by network' [START_REF] Van Dijk | forthcoming: Right engineering? The redesign of privacy and personal data protection[END_REF]. If market actors are to place their trust in the digital value chains, data protection should be safeguarded across all levels of infrastructures, and also include basic information security. Yet, here the challenges increase, since now the focus is on whole value chains, and incorporating all actors involved in the making of IoT products: devices, applications, IoT semantics and other services, or in processing of data. Several of our informants mention issues of 'systemic risk', according to which weaknesses in one part of the chain transmit to other parts, rendering the whole chain vulnerable. Yet, the GDPR places the main responsibility for the protection of personal data on 'data controllers'. This responsibility does not seem to symmetrically extend to other actors in the chain, such as the designers and producers of the hard-and software used by the data controllers. Hence, a privacy and security advisor involved in EU activities aimed at the implementation of smart grids, told us how existing approaches are insufficient, and how the discussion should have been taken from the chain point of view. In this way the transparency of the smart meter would have been discussed in an early stage with all the stakeholders that are related in the chain (privacy and security officer -energy utility). From an infrastructural point of view this makes sense, and triggers the question of how personal data protection and privacy could be implemented across all actors involved. Our informants also referred to the notion of privacy and data protection as 'transversal concerns', meaning 'cross-cutting' matters of concern to be implemented across the entire infrastructural chain by all actors:

When we want to take into account privacy and other concerns, we have to take them into account as transversal concerns...: security, privacy, safety, energy consumption or taking into account ethical aspects and things like that.

… we need to be able to engineer transversal concerns and "capabilities" in things (privacy and security consultant).

However, in order to be able to build privacy, security and data protection as 'capabilities', there is a need to first establish interoperability. In the case of the Internet of Things, to which the quote refers, this is a long-standing effort of digital-physical engineering that has turned out to be more complicated than previously expected. Indeed, there are too many formats and standards in play, and global efforts to reduce the huge plurality have so far not succeeded in establishing interoperability across regions or sectors, or between different producers [START_REF] Vermesan | Visions and challenges for realizing the Internet of Things[END_REF]. This means that there is no stable technical base from which to start, in relation to which privacy concerns could be assessed and communicated. Therefore, the project to implement rights and values as transversal 'concerns' of engineering in large-scale infrastructures and systems, exists more as a promise than as real intervention according to known principles and standards. Yet, similarly to what we have seen in relation to organisations and individuals, this promise may end up having real effects since it becomes central to the organisation and coordination of largescale engineering and regulatory efforts. Here, the dangers are even greater that the scale and complexity of these infrastructural efforts have the sideeffect of disregarding crucial inputs from users and from societal actors on what they expect from IoT applications. This problem is also replicated in the case of privacy and personal data protection, in the tension between how legal principles should be invoked, and how the semantic spaces and design spaces should be compressed to enable standardisation. Here, we can draw an analogy to the arguments of the judge, and the limits to how legal principles and reasoning can be translated into engineering principles. In order to enable standardisation efforts to go ahead, an overall problem pertains to the level of generality and scale of implementation. If some IoT application or system is to be rendered functional, and to include legal principles and privacy concerns, they also need to make intuitive sense to the people using and operating them, including ordinary users and lawyers/judges. As stated in a major work on the social dimensions of information infrastructure, 'an infrastructure occurs when local practices are afforded by a larger-scale technology, which can then be used in a natural, ready-to-hand fashion' [24, p. 381]. Here, there are huge challenges pertaining to the kind of language that could be used to communicate privacy concerns and how to translate these into design, where such concerns are frequently of a local, personal and singular character, and not global and standardisable. Indeed, we could say that many privacy concerns of people arise in the face of efforts to build increasingly globalised and centralised systems, and that what they implicitly or explicitly seek is to recapture and bring powers back to local levels. Furthermore, the technical challenges are immense, since many devices, applications, interoperability services and platforms are built by different companies, using different standards: Many efforts currently go into putting technical complexity at work ...99% focus of technical people is about solving that (DPbD and standardisation consultant). As in the case of the organisations, this points to the dual requirement of making something that works within highly complex, heterogeneous networks. Here, privacy becomes infrastructurally articulated and coarticulated with other transversal concerns, especially security, safety, trust and interoperability. Adequacy of the protections is associated to degrees of users' trust. The connotation of privacy and personal data protection as fundamental rights does not seem to play a determining role in the legitimation of the system. As the peer explains, 'trust is about psychology'. The challenges here are rather huge, and chances are that only very thin, and minimal conceptions of rights can be integrated within this narrowed-down semiotic and infrastructural space.

Concluding remarks

In this article, we have pointed to some tensions and frictions as discovered during consultations with actors within and around the emerging field of privacy and data protection engineering. Some of these, such as lack of public and organisational awareness, may be temporary, and subject to change. A few of the tensions, such as working out the proper relations and design spaces for engineers and lawyers to communicate in better ways, may be eased, given time and spaces for learning. However, we also think that some of the lessons learnt point to decisive limits that should not be transgressed: here we include the impossibility of 'law becoming code' in the strict sense of the word. This would effectively turn law into a mere instrument in a mix aimed at technocratic regulation (which was actually Lessig's prescription) [START_REF] Lessig | Code: Version 2.0[END_REF]. Hence, law needs to retain its own autonomy in articulating privacy and data protection rights, including judgments about (un)successful privacy design, as was also implied by the above quotes from the ECJ judge. In spite of this, legal practices (both legislative, adjudicative and procedural) crucially also need to understand these new design practices and interact with them, and their fast-changing technological and social realities. Here, we argued that the new and emerging field of privacy engineering is interesting. We also argued that design practices will remain bound to intrinsic constraints, to what can realistically be made subject to engineering approaches, to the interests of those involved in their making and should not be overcharged with political promises as to what can realistically be achieved or guaranteed. This points at a need for a firmer embedding of design-based approaches to rights within 'extended' ecologies of practice, in which mutual checks can be exercised: between different epistemic and normative commitments and as provided for by robust public and legal guarantees. Furthermore, our empirical materials demonstrated that there are real tensions involved in the project to turn personal data protection into organisational principles, and into standards for global engineering of infrastructures. Whereas infrastructures, and the markets enabled by them, are increasingly global, people's privacy concerns and legal data protection implications, remain stubbornly attached to the local and singular. The meanings of data protection and privacy, therefore, cannot be detached from questions about where, by whom and through what methods, they are enacted.